# Testing Voice Agents with Evals: Suites, Cases and Assertions (2026)

> How to test an AI voice agent with evals: build a suite of cases, write assertions, run simulated conversations with an LLM judge, and re-run after every change.

Source: https://edesy.in/blog/testing-voice-agents-with-evals-2026

**Evals are saved tests for your voice agent.** Each case is a simulated conversation plus assertions
about what the agent must or must not do. You group cases into a suite and re-run it after every prompt,
tool or model change. Edesy evals run in text mode with an optional LLM judge, catching regressions
before customers hear them.

> **Build your first eval suite:** [Try Edesy free](https://voice-agent.edesy.in/?utm_source=edesy.in&utm_medium=blog&utm_campaign=testing-voice-agents-with-evals-2026) — ₹50 free credit, no card needed.

## Why manual testing stops working

The first version of an agent is usually tested by hand: you call it a few times, it sounds fine, you
launch. The trouble starts with the third or fourth change. You fix how the agent handles "I already
paid", and without noticing, it stops asking for the payment date. Nobody re-tests the old cases by hand
every time.

Evals fix that. Every important behaviour becomes a case you can run in a few minutes, as often as you
like. See the [testing and evals feature](https://edesy.in/ai-voice-agent/features/testing-and-evals) for the full list
of what the Test Center includes.

## The building blocks

| Part | What it is | Example |
|---|---|---|
| Suite | A group of cases for one agent | "EMI reminder: pre-launch" |
| Case | One scenario to simulate | Customer says they already paid |
| Simulated conversation | The caller's side, played by a model or a script | "Maine kal hi pay kar diya" |
| Assertion | A check on the agent's reply or action | Agent asks for the transaction reference |
| LLM judge | A model that grades rules you can't match exactly | "Agent stayed polite and did not argue" |

## What evals check, and what they don't

Edesy evals are **text mode**. They are good at checking:

- **Content:** did the agent give the right answer, mention the right amount, ask the required question?
- **Behaviour:** did it end the call when asked, refuse an off-topic request, avoid a forbidden promise?
- **Tool use:** did it call *schedule callback*, *transfer call* or *collect data* when it should have?

They do **not** check how the agent sounds: voice quality, accent, pauses, or how it handles being
interrupted mid-sentence. For those, use browser voice calls in the Test Center and listen. Treat the two
as layers: evals for logic, voice calls for feel.

## How to build a useful suite

### 1. Start from the call's goal

Write down what a successful call looks like, then one case for that happy path. For an appointment
reminder: confirm the slot, check the address, end politely.

### 2. Add the common detours

Look at your script and list the things callers say that pull the call off track:

- "Who is this?" or "How did you get my number?"
- "Call me later."
- "I want to talk to a person."
- "Wrong number."
- A different language from the one you expected.

Each one becomes a case with an assertion about the right response.

### 3. Add the expensive mistakes

Some mistakes cost more than others. Give each of these its own case:

- Reading out the wrong amount or date from variables.
- Promising something the business can't deliver (a waiver, a discount, a delivery time).
- Not transferring when the caller asks for a human.
- Not marking "do not call" when the caller asks to stop.

### 4. Write assertions you can trust

Use exact checks where the answer is fixed, such as "the reply contains `{{due_date}}`" or "the *set call
outcome* tool was called with *do not call*". Use the LLM judge for judgement calls, such as "the agent
did not pressure the caller". Keep each judge rule to one idea; a rule with three conditions is hard to
debug when it fails.

## A sample suite for a collections reminder

| Case | Caller says | Assertion |
|---|---|---|
| Happy path | "Haan, Friday tak kar dunga" | Outcome set with promised date |
| Already paid | "Maine pay kar diya" | Asks for reference; does not repeat demand |
| Dispute | "Amount galat hai" | Transfers to the collections desk |
| Not the borrower | "Woh ghar pe nahi hain" | Does not disclose the amount |
| Stop calling | "Mujhe call mat karo" | Outcome set to do not call; ends call |
| Asks for link | "Link bhej do" | *Send payment link* tool called |

For the post-call side of the same flow, see [promise-to-pay tracking](https://edesy.in/blog/promise-to-pay-tracking-reminder-calls-2026).

## When to run it

- **Before the first campaign.** Every case should pass.
- **After every change** to the prompt, tools, knowledge base or variables.
- **After switching the LLM** or the voice engine preset. A different model can change behaviour even
  when the prompt is identical.
- **After a real call fails.** Turn that call into a new case so the mistake can't come back unnoticed.

Prompt version history and diffs help here: when a case starts failing, compare the current prompt with
the last version that passed.

## Evals plus post-call review

Evals only test what you thought of. Real callers will always find something new. Close the loop:

1. Review real calls after they end, looking at outcomes, transcripts and extracted fields. Our post on
   [AI call QA after hang-up](https://edesy.in/blog/evaluate-ai-calls-after-hang-up-qa-2026) covers what to track.
2. When you find a bad call, ask [Copilot](https://edesy.in/ai-voice-agent/features/copilot) why it failed and approve
   the fix.
3. Add the call as an eval case.
4. Re-run the suite before the next campaign.

Over a few weeks, the suite grows into a record of every mistake your agent has made, and a guarantee it
won't make them again.

## Practical tips

- **Keep cases short.** Three to five turns is enough to test one behaviour.
- **Name cases by behaviour,** not by number: "asks for human, gets transfer" is easier to read in a
  failed-run report than "case 14".
- **Test variables with odd values.** Long names, amounts with paise, dates on month boundaries.
- **Use scenarios for multi-agent setups.** If you use [squads](https://edesy.in/ai-voice-agent/features/multi-agent-squads),
  simulate the hand-off between members too.

## What it costs

Evals and the Test Center are part of the product. Calls are billed from **₹4/min on plans and credit
packs (₹6/min pay as you go)**, plus telephony; see the [pricing page](https://edesy.in/ai-voice-agent/pricing). Sign-up
includes ₹50 free credit with no card needed. For setup steps, start with the
[quick start docs](https://edesy.in/docs/voice-agent/getting-started/quick-start), and if you want a ready-made agent to
test, pick one from the [templates](https://edesy.in/ai-voice-agent/templates).

## Frequently Asked Questions

### What is an eval for a voice agent?

An eval is a saved test: a simulated conversation plus the things that must be true about the agent's replies. You group evals into a suite and re-run the suite whenever you change the prompt, tools or settings.

### Do Edesy evals test the voice or the text?

Edesy evals run in text mode. They check what the agent says and does (answers, questions asked, tools called), not how it sounds. Test voice, accents and interruptions with browser voice calls in the Test Center.

### What is an LLM judge?

A second model that reads the simulated conversation and decides whether a rule was followed, for example 'the agent never promised a discount'. It is useful for rules that can't be checked with exact text matching.

### How many eval cases do I need?

Start with 10 to 20: the main happy path, the most common objections, and every case where a mistake would be costly, such as wrong amounts or a missed transfer request. Add a case every time a real call goes wrong.

### When should I run the eval suite?

Before the first campaign, after every prompt or tool change, and after switching the LLM. If a case that passed before now fails, fix it before going live.

### Are evals the same as checking calls after they end?

No. Evals run before calls, on simulated conversations. Post-call review looks at real calls afterwards. You need both: evals stop known mistakes coming back, post-call review finds new ones.