Evals are saved tests for your voice agent. Each case is a simulated conversation plus assertions about what the agent must or must not do. You group cases into a suite and re-run it after every prompt, tool or model change. Edesy evals run in text mode with an optional LLM judge, catching regressions before customers hear them.
Build your first eval suite: Try Edesy free — ₹50 free credit, no card needed.
Why manual testing stops working
The first version of an agent is usually tested by hand: you call it a few times, it sounds fine, you launch. The trouble starts with the third or fourth change. You fix how the agent handles "I already paid", and without noticing, it stops asking for the payment date. Nobody re-tests the old cases by hand every time.
Evals fix that. Every important behaviour becomes a case you can run in a few minutes, as often as you like. See the testing and evals feature for the full list of what the Test Center includes.
The building blocks
| Part | What it is | Example |
|---|---|---|
| Suite | A group of cases for one agent | "EMI reminder: pre-launch" |
| Case | One scenario to simulate | Customer says they already paid |
| Simulated conversation | The caller's side, played by a model or a script | "Maine kal hi pay kar diya" |
| Assertion | A check on the agent's reply or action | Agent asks for the transaction reference |
| LLM judge | A model that grades rules you can't match exactly | "Agent stayed polite and did not argue" |
What evals check, and what they don't
Edesy evals are text mode. They are good at checking:
- Content: did the agent give the right answer, mention the right amount, ask the required question?
- Behaviour: did it end the call when asked, refuse an off-topic request, avoid a forbidden promise?
- Tool use: did it call schedule callback, transfer call or collect data when it should have?
They do not check how the agent sounds: voice quality, accent, pauses, or how it handles being interrupted mid-sentence. For those, use browser voice calls in the Test Center and listen. Treat the two as layers: evals for logic, voice calls for feel.
How to build a useful suite
1. Start from the call's goal
Write down what a successful call looks like, then one case for that happy path. For an appointment reminder: confirm the slot, check the address, end politely.
2. Add the common detours
Look at your script and list the things callers say that pull the call off track:
- "Who is this?" or "How did you get my number?"
- "Call me later."
- "I want to talk to a person."
- "Wrong number."
- A different language from the one you expected.
Each one becomes a case with an assertion about the right response.
3. Add the expensive mistakes
Some mistakes cost more than others. Give each of these its own case:
- Reading out the wrong amount or date from variables.
- Promising something the business can't deliver (a waiver, a discount, a delivery time).
- Not transferring when the caller asks for a human.
- Not marking "do not call" when the caller asks to stop.
4. Write assertions you can trust
Use exact checks where the answer is fixed, such as "the reply contains {{due_date}}" or "the set call
outcome tool was called with do not call". Use the LLM judge for judgement calls, such as "the agent
did not pressure the caller". Keep each judge rule to one idea; a rule with three conditions is hard to
debug when it fails.
A sample suite for a collections reminder
| Case | Caller says | Assertion |
|---|---|---|
| Happy path | "Haan, Friday tak kar dunga" | Outcome set with promised date |
| Already paid | "Maine pay kar diya" | Asks for reference; does not repeat demand |
| Dispute | "Amount galat hai" | Transfers to the collections desk |
| Not the borrower | "Woh ghar pe nahi hain" | Does not disclose the amount |
| Stop calling | "Mujhe call mat karo" | Outcome set to do not call; ends call |
| Asks for link | "Link bhej do" | Send payment link tool called |
For the post-call side of the same flow, see promise-to-pay tracking.
When to run it
- Before the first campaign. Every case should pass.
- After every change to the prompt, tools, knowledge base or variables.
- After switching the LLM or the voice engine preset. A different model can change behaviour even when the prompt is identical.
- After a real call fails. Turn that call into a new case so the mistake can't come back unnoticed.
Prompt version history and diffs help here: when a case starts failing, compare the current prompt with the last version that passed.
Evals plus post-call review
Evals only test what you thought of. Real callers will always find something new. Close the loop:
- Review real calls after they end, looking at outcomes, transcripts and extracted fields. Our post on AI call QA after hang-up covers what to track.
- When you find a bad call, ask Copilot why it failed and approve the fix.
- Add the call as an eval case.
- Re-run the suite before the next campaign.
Over a few weeks, the suite grows into a record of every mistake your agent has made, and a guarantee it won't make them again.
Practical tips
- Keep cases short. Three to five turns is enough to test one behaviour.
- Name cases by behaviour, not by number: "asks for human, gets transfer" is easier to read in a failed-run report than "case 14".
- Test variables with odd values. Long names, amounts with paise, dates on month boundaries.
- Use scenarios for multi-agent setups. If you use squads, simulate the hand-off between members too.
What it costs
Evals and the Test Center are part of the product. Calls are billed from ₹4/min on plans and credit packs (₹6/min pay as you go), plus telephony; see the pricing page. Sign-up includes ₹50 free credit with no card needed. For setup steps, start with the quick start docs, and if you want a ready-made agent to test, pick one from the templates.