Voice (LiveKit)

Use LiveKit WebRTC for lower-latency, higher-quality voice conversations with built-in acoustic echo cancellation (AEC).

Why LiveKit

The default WebSocket voice transport works well, but LiveKit offers several advantages:

Feature	WebSocket	LiveKit
Latency	Good	Lower (WebRTC)
Echo Cancellation	None (use headphones)	Built-in AEC
Audio Quality	16 kHz PCM	Opus codec, adaptive bitrate
NAT Traversal	N/A	ICE/TURN

LiveKit is recommended for production applications, especially on devices without headphones where echo cancellation is important.

Prerequisites

Install the LiveKit client library alongside the Estuary SDK:

npm install @estuary-ai/sdk livekit-client

No additional configuration is needed -- the SDK detects livekit-client at runtime.

warning

LiveKit voice requires that your Estuary server has LiveKit enabled and configured. If the server does not support LiveKit, startVoice() will fail with a connection error. Use voiceTransport: 'websocket' to bypass LiveKit, or voiceTransport: 'auto' to fall back automatically.

Starting Voice with LiveKit

Set voiceTransport to 'livekit' in your client configuration:

import { EstuaryClient } from '@estuary-ai/sdk';

const client = new EstuaryClient({
  serverUrl: 'https://api.estuary-ai.com',
  apiKey: 'est_your_api_key',
  characterId: 'your-character-uuid',
  playerId: 'user-123',
  voiceTransport: 'livekit',
});

await client.connect();
await client.startVoice();

How It Works

When you call startVoice() with the LiveKit transport:

┌──────────┐  1. livekit_token  ┌────────────────┐
│  Client  │───────────────────►│ Estuary Server │
└────┬─────┘                    └───────┬────────┘
     │                                  │
     │  2. token + url + room           │
     │◄─────────────────────────────────┘
     │
     │  3. Connect to LiveKit room
     │─────────────────────────────────►┌──────────────┐
     │                                  │ LiveKit SFU  │
     │  4. Publish mic track            └──────────────┘
     │──────────────────────────────────►
     │
     │  5. Bot joins room + publishes audio
     │◄──────────────────────────────────
     │
     │  6. livekit_ready                ┌────────────────┐
     │◄─────────────────────────────────│ Estuary Server │
     └──────────────────────────────────└────────────────┘

1. The SDK requests a LiveKit token from the Estuary server.
2. The server returns a token, LiveKit server URL, and room name.
3. The SDK connects to the LiveKit room using livekit-client.
4. The client publishes its microphone track to the room.
5. The bot participant joins the room and publishes its audio response track.
6. The server emits livekit_ready to confirm the room is active.

All audio routing happens through LiveKit's WebRTC infrastructure. Speech-to-text and bot responses still arrive via the Socket.IO connection.

LiveKit Events

client.on('livekitConnected', (room) => {
  console.log('LiveKit room connected:', room);
});

client.on('livekitDisconnected', () => {
  console.log('LiveKit room disconnected');
});

Text responses and STT events work the same way as with WebSocket voice:

client.on('sttResponse', (response) => {
  if (response.isFinal) {
    console.log('You:', response.text);
  }
});

client.on('botResponse', (response) => {
  if (response.isFinal) {
    console.log('Bot:', response.text);
  }
});

Auto Mode

The default voiceTransport: 'auto' prefers LiveKit when livekit-client is installed:

const client = new EstuaryClient({
  serverUrl: 'https://api.estuary-ai.com',
  apiKey: 'est_your_api_key',
  characterId: 'your-character-uuid',
  playerId: 'user-123',
  // voiceTransport defaults to 'auto'
});

await client.startVoice();
// Uses LiveKit if livekit-client is installed, otherwise WebSocket

If livekit-client is not installed and voiceTransport is set to 'livekit', the SDK falls back to WebSocket voice and logs a warning.

caution

With voiceTransport: 'auto', the SDK will attempt LiveKit whenever livekit-client is installed -- even if the server doesn't support it. If you know your server doesn't have LiveKit enabled, set voiceTransport: 'websocket' explicitly to avoid connection errors.

Muting and Stopping

Mute and stop work identically to WebSocket voice:

// Toggle mute
client.toggleMute();
console.log('Muted:', client.isMuted);

// Stop voice session
client.stopVoice();

Error Handling

LiveKit voice can fail at several stages. The SDK includes the underlying error message to help diagnose issues:

try {
  await client.startVoice();
} catch (err) {
  if (err instanceof EstuaryError) {
    switch (err.code) {
      case ErrorCode.LIVEKIT_UNAVAILABLE:
        // livekit-client package not installed
        break;
      case ErrorCode.CONNECTION_TIMEOUT:
        // Server didn't return a LiveKit token in time
        break;
      case ErrorCode.CONNECTION_FAILED:
        // Failed to connect to the LiveKit room (server may not have LiveKit enabled)
        break;
      case ErrorCode.MICROPHONE_DENIED:
        // Browser denied microphone access
        break;
    }
    console.error(err.message); // Includes the underlying reason
  }
}

Common causes of CONNECTION_FAILED:

• The Estuary server does not have LiveKit enabled
• The LiveKit server URL returned by the token is unreachable
• Network firewall blocking WebRTC connections

Example: LiveKit Voice Chat

import { EstuaryClient } from '@estuary-ai/sdk';

const client = new EstuaryClient({
  serverUrl: 'https://api.estuary-ai.com',
  apiKey: 'est_your_api_key',
  characterId: 'your-character-uuid',
  playerId: 'user-123',
  voiceTransport: 'livekit',
});

client.on('livekitConnected', (room) => {
  console.log('LiveKit room ready:', room);
});

client.on('sttResponse', (response) => {
  if (response.isFinal) {
    console.log('You:', response.text);
  }
});

client.on('botResponse', (response) => {
  if (response.isFinal) {
    console.log('Bot:', response.text);
  }
});

client.on('error', (err) => {
  console.error('Error:', err.message);
});

async function main() {
  await client.connect();
  await client.startVoice();
  console.log('Voice active -- speak into your microphone.');
}

main().catch(console.error);

Next Steps

• Voice (WebSocket) -- Fallback voice transport
• Memory & Knowledge Graph -- Query character memory
• API Reference: EstuaryClient -- Full method reference