STT Providers Overview

Speech-to-Text (STT) is a critical component of your voice agent. The right choice depends on your language requirements, latency needs, and budget.

Supported Providers

Choosing a Provider

For Lowest Latency

Recommended: Deepgram Nova-3

• Industry-leading streaming latency (~150ms)
• Excellent English accuracy
• Interim results for faster response

For Indian Languages

Recommended: ElevenLabs Scribe or Google Chirp

• Hindi, Tamil, Telugu, Assamese support
• Reasonable accuracy (10-25% WER)
• Good cost-performance ratio

For Enterprise

Recommended: Azure Speech

• SOC 2, HIPAA compliance
• Excellent Indic language support
• Custom model training available

Configuration

Agent-Level Configuration

Each agent can have its own STT provider:

{
  "name": "Hindi Support Agent",
  "language": "hi",
  "sttProvider": "elevenlabs",
  "sttModel": "scribe",
  "sttConfig": {
    "endpointing": 300,
    "utterance_end_ms": 1500
  }
}

Provider-Specific Settings

Deepgram

{
  "sttProvider": "deepgram",
  "sttConfig": {
    "model": "nova-3",
    "endpointing": 200,
    "utterance_end_ms": 1000,
    "interim_results": true,
    "smart_format": true
  }
}

Google Chirp

{
  "sttProvider": "google",
  "sttConfig": {
    "model": "chirp",
    "enable_automatic_punctuation": true,
    "enable_spoken_punctuation": false
  }
}

Latency Comparison

User finishes speaking
        │
        ▼
┌───────────────────────────────────────────────┐
│ Deepgram Nova-3   ████████░░░░░░░░ 150ms      │
│ Azure Speech      █████████░░░░░░░ 180ms      │
│ Google Chirp      ██████████░░░░░░ 200ms      │
│ AssemblyAI        ██████████░░░░░░ 200ms      │
│ ElevenLabs Scribe ████████████░░░░ 250ms      │
│ OpenAI Whisper    ███████████████░ 300ms      │
└───────────────────────────────────────────────┘
                                        → Time

Streaming vs Batch

Streaming STT (Recommended)

• Real-time transcription as user speaks
• Interim results enable faster LLM response
• Lower perceived latency

User: "What is my order—"
STT:  [interim] "What is my"
STT:  [interim] "What is my order"
STT:  [final] "What is my order status?"

Batch STT

• Full audio processed at once
• Higher accuracy potential
• Higher latency

Endpointing Configuration

Endpointing determines when the user has finished speaking:

Aggressive (faster response):

{
  "endpointing": 200,
  "utterance_end_ms": 800
}

Conservative (more complete sentences):

{
  "endpointing": 400,
  "utterance_end_ms": 1500
}

Cost Optimization

1. Use appropriate model: Nova-2 is cheaper than Nova-3
2. Optimize audio: Compress silence, use VAD
3. Cache common phrases: Skip STT for known patterns
4. Batch non-real-time: Use batch API for recordings

Next Steps

• Deepgram Configuration - Detailed Deepgram setup
• TTS Providers - Text-to-Speech options
• Latency Optimization - Further reduce response time