Core Classes
Connection, the placeable character actor, project settings, static helpers, and the latent async nodes.
UEstuarySubsystem
UGameInstanceSubsystem — one per Game Instance, created automatically. Owns the connection and routes all server events.
Access:
UEstuarySubsystem* Estuary = GetGameInstance()->GetSubsystem<UEstuarySubsystem>();
Blueprint: Get Estuary Subsystem (auto-wires world context) or Get Game Instance → Get Subsystem with the class pin set to EstuarySubsystem.
Connection
| Function | Description |
|---|---|
void Connect(const FString& ServerUrl, const FEstuaryAuthPayload& Auth) | Open and authenticate a session. No-op if already connecting/connected. |
void Disconnect() | Close the session and cancel any pending reconnect. Safe in any state. |
EEstuarySessionState GetSessionState() const | Current state. |
FEstuarySessionInfo GetSessionInfo() const | Session metadata (empty until connected). |
Conversation (forwarded from components)
These are normally called via UEstuaryCharacterComponent, but are available directly:
| Function | Description |
|---|---|
void SendUserText(const FEstuaryUserText& Payload) | Emit a user text message. |
void SendSayLine(const FString& Text, bool bTextOnly) | Speak a scripted line (TTS only). |
void SendInterrupt(const FEstuaryInterruptPayload& Payload) | Abort the in-flight response. |
void SendUpdatePreferences(const FEstuaryPreferences& Payload) | Update session preferences. |
Voice
| Function | Description |
|---|---|
void SendStartVoice() / void SendStopVoice() | Begin / end a voice session (Blueprint convenience). |
void SendStreamAudio(const FEstuaryStreamAudio& Payload) | Send one base64 PCM16 audio chunk. |
EEstuaryVoiceMode GetVoiceMode() const | WebSocket or LiveKit. |
FString GetActiveVoiceCharacterId() const | Character ID owning the active voice session. |
LiveKit
| Function | Description |
|---|---|
void RequestLiveKitToken() | Request a WebRTC room token (auto-called when voice starts and LiveKit is available). |
void LeaveLiveKitRoom() | Leave the room and revert to WebSocket voice. |
Camera / Vision
| Function | Description |
|---|---|
void EmitCameraImage(const FEstuaryCameraImage& Payload) | Send a captured frame (called by the camera component). |
EEstuaryCaptureSource ToggleCaptureSource() | Flip all registered cameras between scene and webcam. |
void SetCaptureSourceAll(EEstuaryCaptureSource NewSource) | Set capture source on all cameras. |
bool IsWebcamCaptureMode() const | True if the current source is webcam. |
FText GetCaptureSourceLabel() const | UI label, e.g. "Camera: Webcam". |
Events (BlueprintAssignable)
| Delegate | Fires when |
|---|---|
OnSessionConnected(FEstuarySessionInfo) | Session is live (session_info received). |
OnAuthError(FEstuaryAuthError) | Credentials rejected. |
OnSessionRejected(FEstuarySessionRejected) | Server policy cap hit (e.g., concurrent limit). |
OnError(FEstuaryError) | Non-terminal server error (session stays open). |
OnQuotaExceeded(FEstuaryQuotaExceeded) | Quota / share-limit hit (server disconnects right after). |
OnMemoryUpdated(FEstuaryMemoryUpdated) | Background memory extraction completed. |
OnStateChanged(EEstuarySessionState) | Any state transition. |
AEstuaryCharacter
Blueprintable AActor — drop into a level to host a conversation. A thin container that spawns its sub-components.
Properties
| Property | Type | Description |
|---|---|---|
CharacterId | FString | Character UUID. Must match the character_id in the connect auth payload. Auto-generates a GUID (and warns) if empty at BeginPlay. |
CharacterComponent | UEstuaryCharacterComponent* | Conversation API. |
MicrophoneComponent | UEstuaryMicrophoneComponent* | Voice input. |
AnimationReceiver | UEstuaryAnimationReceiver* | ARKit-52 lipsync. |
UserPositionComponent | UEstuaryUserPositionComponent* | Player position streaming (body animation). |
Add UEstuaryAudioPlayerComponent (TTS) and UEstuaryCameraComponent (vision) yourself.
UEstuaryDeveloperSettings
UDeveloperSettings — Project Settings → Plugins → Estuary. Config file: Config/DefaultEstuaryPlugin.ini.
Connection
| Setting | Default | Description |
|---|---|---|
| Server URL | https://api.estuary-ai.com | Estuary gateway URL. |
| Reconnect Max Backoff (seconds) | 30 | Exponential backoff ceiling. |
| Disable LiveKit (WebSocket-only voice) | false | Force WebSocket voice; skip all LiveKit joins. |
Audio
| Setting | Default | Description |
|---|---|---|
| Audio Sample Rate | 24000 | TTS playback rate in Hz (8000–48000). Mic capture is always 16 kHz. |
| Enable Animation | false | Request the ARKit-52 lipsync stream. |
| Enable Body Animation | false | Request the body-pose stream (experimental; forces 16 kHz). |
| LiveKit Echo Cancellation (AEC) | true | Cancel the character's TTS from the mic path (LiveKit mode). |
| AEC Stream Delay (ms) | 50 | Latency hint for echo cancellation. |
The amplitude-jaw fallback and body finger-policy defaults are also configurable here — see Animation Components.
Camera
| Setting | Default | Description |
|---|---|---|
| JPEG Quality | 75 | Compression quality (1–100) for vision frames. |
API key
The API key is not on this class. It lives on a separate Estuary Secret settings panel (Project Settings → Plugins → Estuary Secret → Auth Token), persisted to a local, version-control-ignored file (Config/DefaultEstuarySecret.ini).
// Resolves the key from the secret settings, then the ESTUARY_API_KEY env var
FString Key = UEstuaryDeveloperSettings::GetEffectiveAuthToken();
:::caution Shipping builds
Config is cooked into packaged builds, so don't ship a build with a real key set. For production, pass the key at runtime via Connect(Auth).
:::
UEstuaryBlueprintLibrary
Static helpers in the Blueprint palette under Estuary → Helpers.
| Function | Description |
|---|---|
FString NewMessageId() | Generate a 32-char hex correlation ID. |
bool DecodeBase64ToPcm16(Base64, out Samples) | Decode base64 → PCM16 sample array. |
bool EncodePcm16ToBase64(Samples, out Base64) | Encode PCM16 → base64. |
FEstuaryAuthPayload MakeDefaultAuthPayload(ApiKey, CharacterId, PlayerId, bEnableAnimation, bEnableBodyAnimation, SampleRate) | Build an auth payload from parameters. Forces 16 kHz when body animation is enabled. |
FEstuaryAuthPayload MakeAuthPayloadFromSettings(CharacterId, PlayerId, out ServerUrl) | Build an auth payload from Project Settings (reads API key, sample rate, server URL). Animation flags forced off. |
UEstuarySubsystem* GetEstuarySubsystem(WorldContextObject) | Convenience subsystem accessor. |
UEstuaryConnectAction — the latent Connect node
The Connect (Latent) node (Blueprint palette → Estuary → Transport) wraps a single Connect call and fires exactly one of four output pins:
| Pin | Payload | Fires when |
|---|---|---|
| On Connected | FEstuarySessionInfo | Session is live. |
| On Auth Failed | FEstuaryAuthError | Credentials rejected. |
| On Rejected | FEstuarySessionRejected | Server policy cap hit. |
| On Network Error | FString | Transport-level failure. |
Connect (Latent) [ServerUrl, Auth]
├─ On Connected
├─ On Auth Failed
├─ On Rejected
└─ On Network Error
Intended for Blueprint. In C++, call UEstuarySubsystem::Connect directly and bind the subsystem delegates.
UEstuaryPermissionAction — mobile permission nodes
Latent nodes (Blueprint palette → Estuary → Permissions) for iOS/Android runtime permissions. Each fires Granted or Denied:
- Acquire Microphone Permission
- Acquire Camera Permission
Call these before starting microphone capture or camera capture on mobile.