Native Audio-to-Audio Processing

Native audio-to-audio means the AI model directly processes audio input and produces audio output, without converting speech to text (STT) or text to speech (TTS) as intermediate steps. Models like Gemini Live and OpenAI Realtime are trained on audio directly, understanding and generating speech natively.

Traditional pipeline (STT→LLM→TTS) typically takes 700-1000ms per turn. Native audio-to-audio achieves 377ms with Vertex AI backend - that's 76% faster. This difference is immediately noticeable in conversation flow.

Currently, we support two native audio models: Google Gemini Live (2.0 and 2.5 HD) and OpenAI Realtime (GPT-4o and GPT-4o-mini). Both process audio directly without STT/TTS conversion.

Per-minute costs are slightly higher (Rs 8-12/min vs Rs 6/min), but you save on infrastructure since there's only one API call instead of three. For high-volume use cases, the total cost can actually be lower while delivering better quality.

Yes! Native audio is optional. You can use our traditional pipeline with any combination of 6 STT providers and 9 TTS providers. Native audio is best for latency-critical, natural conversation scenarios.

Native audio models are trained on massive audio datasets and often match or exceed traditional STT accuracy. For complex audio (accents, background noise, code-switching), native audio can actually be more accurate because it processes the full audio context.

Gemini Live 2.5 HD supports 24 languages natively including Hindi and several Indian languages. For specialized Indian language support (Tamil, Telugu, Bengali, etc.), we recommend our traditional pipeline with Google Chirp STT + Sarvam TTS for best results.