Core Concepts

Understand the architecture and design patterns of the Estuary Unity SDK.

Architecture

The SDK is organized into four layers:

Components Layer

MonoBehaviour components that you attach to GameObjects in the Unity Editor. These provide the high-level API:

• EstuaryManager -- Singleton that manages the connection
• EstuaryCharacter -- Per-character conversation interface
• EstuaryMicrophone -- Microphone capture for voice input
• EstuaryAudioSource -- TTS audio playback
• EstuaryActionManager -- Routes parsed actions to handlers

Core Layer

Plain C# classes that handle the protocol and transport. The core has no dependency on the LiveKit Unity SDK -- it talks to LiveKit through an interface so that text-only setups compile cleanly:

• EstuaryClient -- Socket.IO v4 client (manual WebSocket implementation, no third-party library)
• EstuaryConfig -- ScriptableObject for global settings
• EstuaryEvents -- C# delegate definitions for all events
• ILiveKitVoiceManager -- Interface used by core components for voice
• LiveKitBridge -- Service locator that returns a concrete ILiveKitVoiceManager if the optional Estuary.LiveKit assembly is loaded

Optional LiveKit Bridge

When io.livekit.livekit-sdk is installed, Unity defines ESTUARY_LIVEKIT automatically and compiles the Estuary.LiveKit assembly, which contains:

• LiveKitVoiceManager -- Concrete ILiveKitVoiceManager backed by the LiveKit Unity SDK
• LiveKitRegistrar -- Registers the factory with LiveKitBridge on app start
• DirectMicrophoneSource / VPIOAudioSource / AndroidMicrophoneSource -- Native mic capture sources

Without LiveKit installed this assembly is skipped, LiveKitBridge.IsAvailable returns false, and the SDK runs in WebSocket-only mode.

Models & Utilities

Data classes matching the SDK Contract and helper utilities for audio conversion, action parsing, and base64 encoding.

---

EstuaryManager (Singleton)

EstuaryManager is the central coordinator. It:

1. Creates and owns the EstuaryClient (Socket.IO connection)
2. Creates and owns the LiveKitVoiceManager (WebRTC voice)
3. Routes server events to the active EstuaryCharacter
4. Manages character registration and switching

Access it from anywhere:

// Check if connected
if (EstuaryManager.Instance.IsConnected)
{
    await EstuaryManager.Instance.SendTextAsync("Hello!");
}

// Check LiveKit state
if (EstuaryManager.Instance.IsLiveKitReady)
{
    Debug.Log("Voice is ready");
}

The manager is created automatically if accessed via Instance and no instance exists in the scene. It marks itself DontDestroyOnLoad.

---

EstuaryConfig (ScriptableObject)

EstuaryConfig stores global settings as a Unity asset. Create one via Assets > Create > Estuary > Config.

Field	Default	Description
ServerUrl	https://api.estuary-ai.com	Server URL
ApiKey	(empty)	Your est_... API key
VoiceMode	LiveKit	LiveKit (WebRTC) or WebSocket
DebugLogging	false	Enable verbose logs

:::note Sample rates Sample rates are not configured on EstuaryConfig. The SDK uses Unity's AudioSettings.outputSampleRate for TTS playback and forwards it to the server during the handshake, so the backend matches Unity's audio output rate. WebSocket-mode microphone capture is fixed at 16 kHz for STT; the contract default for TTS playback is 24 kHz. :::

You can set the API key at runtime to avoid embedding it in the asset:

EstuaryManager.Instance.Config.SetApiKeyRuntime("est_...");

---

EstuaryCharacter (Per-Character)

Each AI character in your scene gets an EstuaryCharacter component. It holds character-specific settings and exposes conversation methods.

// Send a message
character.SendText("What's the weather like?");

// Start voice
character.StartVoiceSession();

// Listen for responses
character.OnBotResponse += (response) =>
{
    if (response.IsFinal)
        chatUI.AddMessage(response.Text);
};

Key properties:

Property	Type	Description
CharacterId	string	Character UUID
PlayerId	string	End user identifier
IsConnected	bool	Connection status
IsVoiceSessionActive	bool	Whether voice is active
CurrentPartialResponse	string	Accumulated streaming text
CurrentSession	SessionInfo	Session details

---

Event System

The SDK uses C# delegates and Unity Events for all callbacks.

C# Delegates

Subscribe in code for programmatic control:

character.OnBotResponse += (BotResponse response) => { ... };
character.OnTranscript += (SttResponse stt) => { ... };
character.OnInterrupt += (InterruptData data) => { ... };
character.OnActionReceived += (AgentAction action) => { ... };
character.OnConnected += (SessionInfo session) => { ... };
character.OnDisconnected += () => { ... };
character.OnError += (string error) => { ... };

UnityEvents (Inspector)

Drag-and-drop event binding in the Inspector. The EstuaryCharacter component exposes these as serialized UnityEvents:

• onConnected (SessionInfo)
• onDisconnected
• onBotResponse (BotResponse)
• onVoiceReceived (BotVoice)
• onTranscript (SttResponse)
• onInterrupt
• onError (string)
• onActionReceived (AgentAction)

---

Connection Lifecycle

Connection states are defined in the ConnectionState enum:

• Disconnected -- Not connected
• Connecting -- Attempting to connect
• Connected -- Ready to communicate
• Reconnecting -- Lost connection, retrying (up to 5 attempts with linear backoff)
• Error -- Connection failed

---

Main Thread Dispatching

The Socket.IO connection runs on a background thread. All events are dispatched to Unity's main thread via a ConcurrentQueue that EstuaryManager.Update() drains every frame. You can safely update UI, trigger animations, and call Unity APIs from any event callback.

---

Next Steps

• Text Chat -- Build a text chat interface
• Voice Connection -- Set up voice input and output
• API Reference -- Full component documentation