Scoring AI calls is only useful if every call is scored the same way, by something that isn't also trying to hold the conversation. In one deployment, a CBSE exam-prep book publisher runs about a dozen evaluation fields on its teacher calls. Getting there took two fixes: moving evaluation out of the live call, and making sure every agent that ran calls had it attached. This post covers both, and the fields worth tracking.
Set up call QA from day one: Start free on Edesy — Rs 50 free credit, no demo needed.
Lesson 1: evaluate after hang-up, not during the call
The first version put the evaluation fields into the live call's data, which meant the agent was asked to judge its own conversation while having it. Three problems followed:
- A second job. The agent's instructions now included "decide if the caller suspects AI" and "note script violations". Some of that leaked into how it spoke.
- Partial information. Mid-call, the agent can't know how the call ends.
- Slower turns. More instructions and more to track on every reply.
The fix was simple: keep the live agent focused on the conversation, and run evaluation after the call ends, on the full transcript. Post-call extraction sees the whole call, runs once, and never affects what the caller hears.
Lesson 2: every agent that runs calls needs the template
Teams often add a second agent: a follow-up call, a different language, a new campaign. In the same deployment, the follow-up agent ran its first calls with no evaluation at all, because the template was scoped to the original agent. Nobody noticed until they looked for the numbers.
Checklist whenever you add or clone an agent:
- attach an evaluation template before the first real call;
- run five test calls and confirm every field fills;
- check that unanswered dials still produce a row.
The fields worth tracking
| Field | Values | What it tells you |
|---|---|---|
| Call outcome | e.g. approved, not interested, wrong person, callback, dropped early, hostile | Your funnel, one word per call |
| Drop reason | Why a failed call failed | Which objection or moment loses people |
| AI suspected | Yes / no | Whether the caller realised it was an AI |
| AI suspicion stage | Opening, qualification, offer, close | Which line gives it away |
| Script violation | Yes / no + short note | Prompt bugs, before customers complain |
| Qualification verdict | Qualified / not qualified / unclear | Whether the person met your criteria |
| Call score | 1 to 5 | A quick sort for listening to the worst calls first |
Two of these deserve a closer look.
AI suspicion and its stage. "Did they suspect AI?" alone isn't actionable. "They suspected at the opening line" is: rewrite the greeting. Being honest when asked is the right policy; the goal is a natural conversation, not hiding what it is.
Script violations. One real example: the agent occasionally answered its own question in the caller's voice ("Aap kaunsi class padhate hain? ... Class 8, okay"). A violation field catches this within a day, instead of after a week of bad data.
Reviewing the results
A weekly routine that works:
- Sort by call score, lowest first, and listen to ten calls.
- Filter AI suspected = yes and group by stage. Fix the most common stage.
- Read every script violation.
- Compare outcomes week on week after each prompt change.
For a deeper look at failures, see why AI calls fail: drop reasons by script stage. For writing the field descriptions, see score every AI call automatically.
Try it
Post-call extraction is included on Edesy. Results appear in call history and dashboards, and can be exported to CSV or Google Sheets or sent to your CRM by webhook. Create a free account, or read how a publisher verifies teachers with AI calls.