Skip to main content

Memory System

Estuary characters have persistent memory. Every conversation is remembered, and important details are automatically extracted and consolidated over time. This page describes how memory works and how to access it.

Memory Types​

Each extracted memory is classified into one of eight types:

TypeDescriptionExample
factObjective information about the user"Works as a software engineer at a startup"
preferenceUser likes, dislikes, and preferences"Prefers dark roast coffee"
relationshipConnections between people in the user's life"Wife is named Sarah"
eventSomething that happened at a specific time"Started a new job last week"
emotional_stateEmotional tone or sentiment observed"Was excited when discussing travel plans"
correctionUser explicitly correcting previous information"Corrected: moved to Maui, not mainland Hawaii"
character_selfThe character's own traits and reflections"I enjoy discussing philosophy with this person"
spatial_changeChanges in the user's physical environment (vision/AR)"Moved from the kitchen to the living room"
Character Self Memory

character_self memories are shared across all players of the same character, so the character maintains a consistent identity. All other types are private to the specific player-character pair.

How Memory Works​

Extraction​

After each conversation turn, the system automatically extracts memories, core facts, and entity relationships in the background. Each memory receives a confidence score (0.0-1.0) reflecting how certain the system is about the memory's accuracy. Duplicate memories are automatically prevented.

Retrieval​

When a user sends a message, the system retrieves the most relevant memories and includes them in the character's context. Memories are ranked by a combination of semantic relevance, recency, and confidence. Core facts are always included.

Decay​

Memory relevance decreases over time. Less recently accessed memories are deprioritized during retrieval, but they are never deleted. When a memory is retrieved, its relevance is reinforced -- frequently recalled memories stay strong while unused memories fade.

Core Facts​

Core facts are concise, structured key-value pairs about a user -- things like their name, location, and occupation. They are derived from conversations and updated as new information emerges.

Examples:

  • Name: "Alex"
  • Location: "San Francisco"
  • Occupation: "Software engineer at a startup"
  • Interest: "Hiking, cooking, sci-fi movies"

Core facts are always included in the character's context, ensuring fundamental details are never forgotten.

Entities​

Entities represent people, places, and things mentioned across conversations. Each entity has:

  • Name -- The entity's name or label
  • Type -- person, location, organization, or thing

Entity Relationships​

Entities can be connected by relationships:

  • "Alex" -> works at -> "TechCorp"
  • "Max" -> is pet of -> "Alex"
  • "Alex" -> lives in -> "San Francisco"

Knowledge Graph​

All memory components connect into a knowledge graph accessible via the REST API:

  • User nodes represent each player
  • Memory nodes are individual memories
  • Cluster nodes group related memories by topic
  • Entity nodes represent extracted entities (people, places, things)

See the Memories API for the graph endpoint and response format.

Real-Time Memory Events​

SDKs can receive notifications when new memories are extracted by enabling realtime_memory (or realtimeMemory in TypeScript) in the client configuration. See the Conversation Protocol for event details.

REST API​

The REST API provides read access to a player's memory data:

  • List memories -- Filter by type, status, sort by confidence or recency
  • Core facts -- Structured key-value facts always in context
  • Search -- Semantic vector search across memories
  • Timeline -- Memories grouped by date
  • Stats -- Aggregate memory statistics
  • Knowledge graph -- Clustered memories, entities, and relationships
  • Delete -- Remove all memories for a player (GDPR)

All endpoints use agent_id (character ID) and player_id in the path. See Memories API for the full reference.

Next Steps​

Last updated