Latency Optimization

Minimize delay between user speech and agent response.

Understanding Latency

Total latency = STT + LLM + TTS + Network

Optimization Strategies

1. Use Native Audio Models

Bypass STT/TTS entirely:

{
  "llmProvider": "gemini-live-2.5"
}

2. Use Vertex AI (Gemini Live)

Vertex AI reduces Gemini Live latency significantly:

{
  "llmConfig": {
    "gemini-live-2.5": {
      "vertexai": {
        "enabled": true,
        "region": "us-central1"
      }
    }
  }
}

3. Choose Fast Providers

Fastest STT:

• Deepgram Nova-2

Fastest TTS:

• Cartesia
• Deepgram
• Sarvam

Fastest LLM:

• Gemini 2.5 Flash-Lite

4. Optimize LLM Response

{
  "llmConfig": {
    "maxTokens": 150,
    "temperature": 0.5
  }
}

Shorter responses = faster TTS.

5. Reduce Network Latency

• Use nearest region
• Check for network issues
• Consider dedicated connection

Measuring Latency

Check latency in Call History:

1. Open a call
2. View Latency Breakdown
3. See per-component timing

Latency Targets

Quick Wins

1. Switch to Gemini Live
2. Enable Vertex AI
3. Use Deepgram STT
4. Keep responses short