# Voice Activity Detection (VAD) for AI Voice Agents

> Configure Silero VAD for real-time speech detection in voice agents. Reduce latency, filter background noise, and enable natural turn-taking with optimized VAD settings.

Source: https://edesy.in/docs/voice-agent/pipeline/vad

VAD is the first stage of audio processing - it determines when the user is speaking and when they've finished. Getting VAD right is crucial for natural conversations.

## Why VAD Matters

```
Without proper VAD:
─────────────────────────────────────────────────────────────
User: "I want to check my order status" [background noise...]
Bot:  [Waits for noise to stop... 3 seconds pass]
      "I'd be happy to help! What's your order number?"
User: [Frustrated by delay]

With optimized VAD:
─────────────────────────────────────────────────────────────
User: "I want to check my order status"
Bot:  [Detects end of speech in 200ms]
      "I'd be happy to help! What's your order number?"
User: [Smooth conversation]
```

## Silero VAD

We use Silero VAD, a state-of-the-art neural network-based VAD that runs locally:

### Advantages

- **High accuracy**: 99%+ on clean speech
- **Low latency**: ~10ms per frame
- **No cloud dependency**: Runs entirely on-device
- **Small footprint**: ~2MB model file

### How It Works

```
Audio Input (8kHz, 16-bit PCM)
        │
        ▼
┌───────────────────┐
│   Silero ONNX     │
│   Neural Network  │
│                   │
│ Input: 512 samples│
│ Output: 0.0-1.0   │
│ (speech prob)     │
└───────────────────┘
        │
        ▼
Speech Probability
        │
    ┌───┴───┐
    │       │
   <0.8    ≥0.8
    │       │
 Silence  Speech
```

## Configuration

### Basic Configuration

```go
cfg := silero.DetectorConfig{
    ModelPath:            "./silero_vad.onnx",
    SampleRate:           8000,          // Match telephony audio
    Threshold:            0.8,           // Speech probability threshold
    MinSilenceDurationMs: 200,           // Silence before end-of-speech
    LogLevel:             silero.LogLevelError,
}

detector, err := silero.NewDetector(cfg)
```

### Configuration Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `ModelPath` | string | required | Path to silero_vad.onnx |
| `SampleRate` | int | 8000 | Audio sample rate |
| `Threshold` | float32 | 0.8 | Speech probability threshold (0.0-1.0) |
| `MinSilenceDurationMs` | int | 200 | Silence duration before end-of-speech |
| `VolumeThreshold` | float32 | 0.0 | Minimum audio level for processing |

### Threshold Tuning

```
Threshold = 0.5 (too sensitive)
─────────────────────────────────────────────────────────────
Detects: speech, breathing, background noise, air conditioning
Result: False positives, bot interrupts itself

Threshold = 0.9 (too strict)
─────────────────────────────────────────────────────────────
Detects: Only loud, clear speech
Result: Misses quiet speakers, soft endings

Threshold = 0.8 (balanced)
─────────────────────────────────────────────────────────────
Detects: Normal speech, ignores most noise
Result: Good for most telephony use cases
```

### Per-Agent Configuration

Configure VAD per agent based on use case:

```json
{
  "agent": {
    "name": "Call Center Agent",
    "vadConfig": {
      "threshold": 0.75,
      "minSilenceDurationMs": 250,
      "volumeThreshold": 0.02
    }
  }
}
```

## Speech Detection Events

The VAD emits events that drive the pipeline:

```go
type VADEvent struct {
    Type      VADEventType
    Timestamp time.Time
    Duration  time.Duration
}

type VADEventType int

const (
    SpeechStart  VADEventType = iota  // User started speaking
    SpeechEnd                          // User stopped speaking
    Interruption                       // User spoke during bot output
)
```

### Event Flow Example

```
Timeline (ms):  0    100   200   300   400   500   600   700   800
User Audio:     [─────████████████████████████─────────────────────]
                      │                      │
Events:           SpeechStart            SpeechEnd
                    (100ms)               (500ms)
                                         + 200ms silence
                                         = 700ms event fired
```

## Volume Threshold

Filter out low-level background noise before VAD processing:

```go
// Calculate RMS volume
func calculateVolume(samples []int16) float32 {
    var sum float64
    for _, sample := range samples {
        sum += float64(sample) * float64(sample)
    }
    rms := math.Sqrt(sum / float64(len(samples)))
    return float32(rms / 32768.0) // Normalize to 0.0-1.0
}

// In VAD processing
func (d *Detector) processAudio(audio []byte) {
    volume := calculateVolume(audio)

    if volume < d.VolumeThreshold {
        // Skip VAD processing for very quiet audio
        return
    }

    // Proceed with neural network inference
    probability := d.model.Infer(audio)
    // ...
}
```

### Volume Threshold Guidelines

| Environment | Recommended Threshold |
|-------------|----------------------|
| Quiet office | 0.01 |
| Normal telephony | 0.02 |
| Noisy call center | 0.03-0.05 |
| Very noisy (street) | 0.05-0.1 |

## Interruption Detection

VAD also enables interruption handling when the user speaks during bot output:

```go
type InterruptionHandler struct {
    callback func()
    enabled  bool
}

func (h *InterruptionHandler) OnVADEvent(event VADEvent) {
    if event.Type == SpeechStart && h.enabled {
        // User is speaking while bot is outputting
        h.callback() // Trigger interruption
    }
}

// In call provider
provider.SetInterruptionHandler(&InterruptionHandler{
    callback: func() {
        provider.ClearPlayback()     // Stop current audio
        llm.CancelGeneration()       // Cancel LLM
        // Process new user input
    },
    enabled: agent.AllowInterruptions,
})
```

## STT Mute Filter

Control when audio is sent to STT based on VAD and bot state:

```go
type STTMuteFilter struct {
    muteStrategies []MuteStrategy
}

type MuteStrategy interface {
    ShouldMute() bool
}

// Mute during bot speech (echo cancellation)
type MuteDuringBotSpeech struct {
    user *session.User
}

func (m *MuteDuringBotSpeech) ShouldMute() bool {
    return m.user.IsBotSpeaking()
}

// Mute during function execution
type MuteDuringFunctionCall struct {
    user *session.User
}

func (m *MuteDuringFunctionCall) ShouldMute() bool {
    return m.user.IsExecutingFunction()
}
```

## Best Practices

### 1. Match Sample Rate

Ensure audio sample rate matches VAD configuration:

```go
// ❌ Wrong - sample rate mismatch
cfg := silero.DetectorConfig{
    SampleRate: 16000, // VAD expects 16kHz
}
// But telephony sends 8kHz audio - will cause detection issues

// ✅ Correct
cfg := silero.DetectorConfig{
    SampleRate: 8000, // Matches telephony audio
}
```

### 2. Handle Edge Cases

```go
// Very short utterances (< 200ms)
if speechDuration < 200*time.Millisecond {
    // Might be noise, cough, or "uh-huh"
    // Consider waiting for more speech
}

// Very long silence after speech
if silenceDuration > 3*time.Second {
    // User might be thinking or distracted
    // Consider sending a prompt
}
```

### 3. Adaptive Thresholds

Adjust thresholds based on call quality:

```go
func adaptThreshold(initialNoiseLevel float32) float32 {
    if initialNoiseLevel > 0.1 {
        return 0.85 // Noisy environment - be more strict
    }
    return 0.8 // Normal threshold
}
```

## Debugging VAD Issues

### Common Problems

| Issue | Symptom | Solution |
|-------|---------|----------|
| False positives | Bot interrupts itself | Increase threshold, add volume filter |
| Missed speech | Bot doesn't respond | Decrease threshold |
| Late detection | Delayed responses | Reduce MinSilenceDurationMs |
| Cut-off speech | Responses before user finishes | Increase MinSilenceDurationMs |

### Logging VAD Events

```go
detector.SetLogLevel(silero.LogLevelDebug)

// Output:
// [VAD] Frame 1: prob=0.12, speaking=false
// [VAD] Frame 2: prob=0.45, speaking=false
// [VAD] Frame 3: prob=0.89, speaking=true, event=SpeechStart
// [VAD] Frame 4: prob=0.92, speaking=true
// ...
// [VAD] Frame N: prob=0.15, speaking=true
// [VAD] Frame N+1: prob=0.08, speaking=false
// [VAD] Silence duration: 200ms, event=SpeechEnd
```

## Next Steps

- [Interruption Handling](https://edesy.in/docs/voice-agent/pipeline/interruptions) - Handle user barge-in
- [Turn Detection](https://edesy.in/docs/voice-agent/pipeline/turn-detection) - Advanced turn-taking
- [Latency Optimization](https://edesy.in/docs/voice-agent/advanced/latency) - Reduce response time