By Greg Nowak. Updated 28 August 2026.
A voice agent can sound convincing, respond quickly and perform beautifully in a controlled demo. That does not mean it is ready for customers.
The real test comes when an account lookup fails, the caller changes direction, or the request requires judgement the agent should not exercise. Current realtime voice models can converse and use tools, but greater capability creates a larger operational surface. The service must know when to stop, where to send the caller and what information should travel with them.
For the business approving the launch, the decisive question is simple: when automation reaches its limit, does the customer still get useful help?
Design handoff as a normal outcome
Human escalation should be part of the main journey, not an exception added after the automated flow is finished. A handoff can be the correct resolution when the request falls outside the agent’s authority or when continuing would waste the caller’s time.
Start by defining triggers that the project team can test. Common examples include:
- the caller explicitly asks for a person;
- identity or account verification cannot be completed safely;
- a tool times out, fails or returns contradictory information;
- the agent repeatedly misunderstands the same request;
- the next action involves a refund, exception or other sensitive decision;
- the caller shows strong frustration, distress or urgency.
“Transfer when appropriate” is not a complete requirement. For every trigger, specify what the caller hears, which queue receives the call, what happens outside opening hours and how the service behaves if no employee is available. The operations lead should own these decisions; they are customer-service policy expressed through software.
Transfer the conversation, not just the call
A technically successful transfer can still be a poor handoff. If the employee receives only an incoming call, the customer must repeat their identity, request and troubleshooting history. The phone connection survived, but the conversation did not.
A useful handoff package should contain a verified customer identifier where appropriate, the stated reason for contact, relevant answers already collected, tools attempted and their results, unresolved questions, the escalation trigger and routing attributes such as language or department.
Keep the summary structured and short. Preserve the full transcript only when there is a justified operational or compliance need. Most employees need an accurate briefing, not several pages of dialogue to read while the caller waits.
The wording must also preserve uncertainty. “Caller says the invoice was paid” is not the same as “payment confirmed.” Separate customer claims, model interpretations and facts verified by business systems.
| Release gate | Question to answer | Evidence before launch |
|---|---|---|
| Escalation | What causes the agent to stop? | Testable trigger catalogue |
| Routing | Can each request reach the right available team? | Queue, closed-hours and no-answer tests |
| Context | Can the employee continue without restarting discovery? | Reviewed handoff payload and agent-screen test |
| Authority | Can the agent perform only permitted actions? | Tool allowlist, scopes and approval rules |
| Privacy | Where do audio, transcripts, summaries and logs go? | Redaction, access, retention and deletion decisions |
| Operations | Can the team see whether callers were helped? | Resolution, repeat-contact and abandonment measures |
Keep authority outside the model
Once the agent can read customer records, update a CRM or book appointments, tool access becomes a business-risk decision. The application around the model must enforce the boundaries.
Apply least privilege: expose only the tools required for the approved journey, use read-only access where possible and constrain operations to the authenticated customer and current task. Require independent approval for irreversible, financial, administrative or externally visible actions.
Do not treat a confident tool request as authorisation. Before executing it, the application should validate identity, permissions, parameters, current state and any approval requirement. A prompt can guide behaviour, but it is not an access-control system.
Handoff data deserves the same discipline. Audio, transcripts, model summaries, tool results, operational logs and CRM notes may all have different retention needs. Decide which records are necessary, who can see them and when they are deleted. Avoid passing sensitive information simply because it was mentioned during the call.
Test the moments a polished demo avoids
Happy-path tests prove that components connect. Production tests must prove that the service fails safely.
Test callers interrupting, speaking over background noise, changing intent, remaining silent and giving ambiguous answers. Simulate expired authentication, slow APIs, malformed tool output and unavailable queues. Ask for a person at inconvenient points. Attempt actions outside the agent’s role and verify that application controls reject them.
Run the whole handoff, including the employee experience. Confirm that automation stops responding, the caller hears an honest transition message, the correct queue receives the call and the context appears in a usable form. If a provider supports a handed-off state or equivalent control, use it to prevent the AI and employee from responding simultaneously.
Repeat these tests after material changes to prompts, tools, memory, routing, models or providers. Keep the tested configuration, expected outcome, observed result and accepted residual risks with the release record.
Measure resolution, not containment
A low transfer rate is not automatically good. It may mean the agent resolves simple requests, or it may hide abandoned and unresolved calls. Likewise, a temporary rise in handoffs can indicate that a cautious new workflow is protecting customers.
Review several measures together: automated resolution, completed transfers, routing accuracy, waiting time, abandonment, repeat contact and whether employees must repeat verification or discovery. Sample transcripts and handoff summaries as well as looking at dashboards; aggregate numbers rarely explain why a journey failed.
Assign an owner and a review rhythm before launch. Someone must be able to change escalation rules when policies, staffing or customer language change.
Readiness belongs to the whole service
A production voice agent is more than a model, prompt and telephone connection. It is a service spanning conversation design, identity, tools, routing, privacy, employee workflows, testing and ongoing ownership.
If you are planning or rescuing such a project, Greg can help turn those moving parts into a practical service blueprint: clear boundaries, testable handoffs, named owners and an acceptance plan the business can actually use.
The final release test remains straightforward: when the agent cannot finish the job, does the customer still feel that the business understands what they need and is ready to continue? Until that works consistently, the voice agent is not finished.
Related on GrN.dk
- OpenAI Computer Use: Browser Agents Need Credentials, Not Demos
- OpenAI Presence Is Here. Is Your Workflow Ready for an Agent?
- Your AI Agent Has Shell Access. What Can It Reach?
Need help with this kind of work?
Plan a production-ready voice workflow Get in touch with Greg.
