Skip to main content

Memory System

Estuary characters have persistent, multi-layered memory. Every conversation is remembered, and important details are extracted and consolidated over time. This page describes how memory works and how to access it.

Memory Types​

Characters maintain four types of memory:

Episodic Memory​

Episodic memories record what happened in conversations. Each memory captures a specific event, exchange, or moment with context about when and where it occurred.

Examples:

  • "The user told me they just started a new job at a tech company"
  • "We discussed their favorite hiking trails in Colorado"
  • "The user seemed frustrated about a bug they couldn't fix"

Semantic Memory​

Semantic memories capture general knowledge and facts extracted from conversations. Unlike episodic memory, these are decontextualized -- they represent what the character knows rather than what happened.

Examples:

  • "The user works in software engineering"
  • "The user prefers dark roast coffee"
  • "The user has a dog named Max"

Emotional Memory​

Emotional memories track the relationship dynamics between the character and the user. They record emotional tone, sentiment shifts, and the quality of interactions.

Examples:

  • "The user was excited when discussing their travel plans"
  • "Our conversation about their family was warm and open"
  • "The user became withdrawn when I asked about work"

Character Self Memory​

Self memories are reflections the character forms about its own behavior and role in the relationship. These help the character maintain consistency and adapt its personality.

Examples:

  • "I should be more encouraging when the user talks about their projects"
  • "The user responds well to humor -- I should use it more"

Memory Lifecycle​

Creation​

Memories are created during and after each conversation. The LLM identifies important moments, facts, and emotional signals as they occur.

Consolidation​

After a conversation ends, memories are consolidated:

  1. Extraction -- Key facts, entities, and emotional signals are extracted
  2. Deduplication -- New memories are compared against existing ones to avoid redundancy
  3. Summarization -- Related episodic memories may be merged into higher-level semantic memories
  4. Embedding -- Memories are embedded as vectors for semantic retrieval

Decay​

Memories have a relevance score that can decrease over time. Less frequently accessed or less important memories may be deprioritized during retrieval, though they are not deleted.

Core Facts​

Core facts are concise, structured statements about a user that the character considers most important. They are derived from conversations and updated as new information emerges.

Examples:

  • Name: "Alex"
  • Location: "San Francisco"
  • Occupation: "Software engineer at a startup"
  • Interest: "Hiking, cooking, sci-fi movies"

Core facts are always included in the LLM context, ensuring the character never forgets fundamental details about a user.

Entities​

Entities represent people, places, and things mentioned across conversations. Each entity has:

  • Name -- The entity's name or label
  • Type -- Person, place, organization, concept, etc.
  • Description -- Accumulated knowledge about the entity
  • First mentioned -- When the entity was first discussed

Entity Relationships​

Entities can be connected by relationships:

  • "Alex" → works at → "TechCorp"
  • "Max" → is pet of → "Alex"
  • "Alex" → lives in → "San Francisco"

Knowledge Graph​

All memory components connect into a knowledge graph with four node types:

┌─────────┐       ┌──────────┐
│  User   │──────>│  Memory  │
│ (player)│       │(episodic,│
└────┬────┘       │ semantic)│
     │            └──────────┘
     │
     ▼
┌─────────┐       ┌──────────┐
│ Cluster │──────>│  Entity  │
│(topic   │       │(person,  │
│ group)  │       │ place)   │
└─────────┘       └──────────┘
  • User nodes represent each player
  • Memory nodes are individual memories (all types)
  • Cluster nodes group related memories by topic
  • Entity nodes represent extracted entities

Edges encode relationships: mentions, relates_to, part_of, associated_with, etc.

During a conversation, the retrieval system queries this graph to find the most relevant memories for the current context, using a combination of semantic similarity (vector search) and graph traversal.

REST API for Memory​

The REST API provides read access to a player's memory data. All endpoints use agent_id (the character's ID) and player_id in the path.

List Memories​

GET /api/agents/{agent_id}/players/{player_id}/memories
X-API-Key: est_...

Returns memories for a player, optionally filtered by type (?memory_type=semantic).

Get Core Facts​

GET /api/agents/{agent_id}/players/{player_id}/memories/core-facts
X-API-Key: est_...

Returns the core facts the character knows about a specific player.

Search Memories​

GET /api/agents/{agent_id}/players/{player_id}/memories/search?q=hiking
X-API-Key: est_...

Semantic vector search across a player's memories.

Delete Memories​

DELETE /api/agents/{agent_id}/players/{player_id}/memories?confirm=true
X-API-Key: est_...

Permanently deletes all memories for a player (GDPR compliance).

For the full API reference including timeline, stats, and knowledge graph endpoints, see Memories API.

Next Steps​

Last updated