Voice Agents for Inbound Customer Service: UK SME Setup

A 35-person UK logistics firm recorded three months of inbound customer service calls in January. When we mapped the transcript data, 78% of calls asked one of 11 questions: delivery status, collection window times, damage claim process, invoice queries, and six others. Three customer service staff were handling those calls at an average of 4.1 minutes each. The voice agent now handles the 78% in under 90 seconds and transfers the rest with the caller's account number, query type, and relevant history already on the rep's screen. The remaining 22% — disputes, customs delays, damaged-goods escalations — get a human who starts with context rather than starting cold.

That outcome is repeatable. Three weeks of structured work, not a lengthy programme.

Mapping your call mix: the transcript audit that shows which queries a voice agent can own

Before you write a single prompt, pull your last 90 days of call recordings and transcribe them. We use Deepgram Nova-3 for batch transcription — low cost, structured JSON output piped directly to a Postgres table. If you already have a VoIP provider (RingCentral, 8x8, Twilio), call recordings are almost certainly being stored somewhere; you just need to extract them.

From the transcript table, run a frequency analysis on caller intent. The first signal you are looking for is concentration: do your top 10 query types cover 60% or more of volume? If yes, a voice agent is viable. If your call mix is flat — 30 distinct query types each at 3% — the economics are harder to justify.

Tag each transcript with three fields: - Intent — what the caller wanted - Resolution — resolved, transferred, or callback - Duration — average handle time per intent

Intents that resolve with a database lookup plus a policy statement are your automation targets. Intents that require human judgement, emotional labour, or authority delegation are not. The logistics firm's 11 automatable intents averaged 3.8 minutes with a human agent. At 140 calls per day, moving 78% to a 90-second automated path returns roughly six staff-hours daily.

Knowledge base design: structured Q&A vs RAG over policy documents for customer service

The knowledge retrieval decision shapes everything downstream. Two options:

For a first-line customer service agent, start with structured Q&A for your top 15 intents. The answers are deterministic; no retrieval needed. For queries requiring account-specific data — order status, invoice balance — wire a function call to your CRM or order management system instead.

Introduce RAG only for policy lookups that have no clean answer-pair format: "what are your terms for international shipping damage?" where the answer spans three paragraphs of a PDF. The Document RAG case study covers the retrieval architecture we use on those, including chunk sizing and metadata filtering.

If you do go RAG for customer service, chunk at the section level rather than paragraph level. A retrieval failure that returns a half-answer is worse than a clean "I'll connect you to the team for that one." Graceful uncertainty beats confident wrongness. The same chunking and retrieval principles apply if you are running a parallel RAG knowledge agent over internal docs for staff queries.

Escalation trigger design: the call conditions that require a human and how to detect them mid-call

Define your escalation triggers before you write any prompts. The conditions that require a human transfer:

• Emotional state — anger keywords, raised vocal affect, repeated requests for a manager
• Complexity overflow — three or more intent switches in one call without resolution
• Policy edge case — the caller's situation matches no defined answer path
• Explicit request — "speak to someone", "transfer me", "I need a person"
• Regulatory trigger — any mention of a formal complaint, solicitor, or legal action
• Authentication failure — caller cannot verify identity after two attempts

In Retell.ai, implement these as server-side event listeners plus a function tool the LLM can invoke:

{
  "type": "function",
  "name": "transfer_to_human",
  "description": "Transfer to a human agent. Use when: explicit request, 3+ failed resolution attempts, complaint keyword detected, or auth failed twice.",
  "parameters": {
    "reason": {
      "type": "string",
      "enum": [
        "explicit_request",
        "resolution_failure",
        "complaint_keyword",
        "auth_failure",
        "policy_edge_case"
      ]
    },
    "context_summary": {
      "type": "string",
      "description": "One sentence summary of the call so far, for the receiving agent"
    },
    "caller_id": {
      "type": "string",
      "description": "Account number confirmed during authentication"
    }
  }
}

The context_summary field is non-negotiable. When the transfer fires, the receiving rep sees why they're getting the call, who it is, and what was already established. Warm transfers without context are where customer frustration spikes. See the voice agent transfer-to-human implementation guide for the full handoff pattern including screen-pop delivery.

Caller authentication without friction: verifying identity in a first-line voice agent

Authentication is where customer service voice agents fail more often than on any technical limitation. The failure mode is asking for too much. A caller who must recite a full date of birth, a postcode, and an account number before getting their delivery status will abandon or escalate.

For most UK SMEs, a two-point check is sufficient for information retrieval: account number plus one of {postcode, order reference, last four digits of invoice amount}. This is not adequate for financial or regulated transactions, but it covers delivery status and invoice queries comfortably.

Implement authentication as the first function call in the conversation, before any data is retrieved. If the caller fails both attempts, transfer immediately — do not let the agent attempt to help without verified identity. Providing order data to an unverified caller carries real exposure under UK GDPR.

Keep the authentication prompt short and factual: "To pull up your account, I need your customer number and the postcode on the account. Go ahead." Under 20 words. Not: "For security purposes, before we can proceed with your enquiry today, I will need to verify your identity."

Tone and persona design for customer service: different constraints from outbound sales agents

Outbound sales agents can afford to be brisk and goal-directed. Customer service agents cannot. The caller has a problem. Patience must be the persona baseline, not enthusiasm.

The specific differences from outbound:

• No urgency language. "Act now", "don't miss out" — these read as dismissive when someone is chasing a lost parcel.
• Longer silence tolerance before re-prompting. Default to 2.5 seconds, not 1.5. Callers with a complaint often pause while thinking; cutting them off reads as indifference.
• Acknowledgement before resolution. The agent should verbally confirm what it heard before answering: "That's a delivery status query on order 4471 — let me check that now" rather than immediately reading the status.
• De-escalation path. When anger keywords trigger, the agent should acknowledge before deciding on transfer. Not every frustrated caller wants a human; some want two sentences of acknowledgement and then the answer.

Do not give the agent a name that implies it is human. "You're through to Quantum's service line" rather than "Hi, I'm Sarah from Quantum." The ICO's guidance on AI and data protection requires disclosure that the system is automated; making it clear from the first sentence avoids compliance questions and complaint risk later.

Multi-turn conversation design: handling compound queries that don't fit a single flow state

Most customer service calls contain more than one query. "What's the status of my delivery, and also can you tell me your Saturday collection hours?" is two intents in one breath. Flow-state machines that expect a single intent per turn break on these — and they are common.

The right architecture is intent collection before resolution:

1. Let the caller speak their full query without interrupting
2. Extract all identified intents from the utterance
3. Confirm back: "I can check your delivery status and give you Saturday collection hours — I'll start with the delivery"
4. Resolve in sequence, confirming after each: "Delivery is due Thursday. Shall I give you the Saturday hours now?"

In the LLM system prompt, maintain a running intent queue as a structured field. After each resolution step, the model updates the queue and decides whether to continue or confirm completion. The failure mode here is the agent resolving intent one and terminating the call before intent two — turning a 90-second automated call into a callback. For the state-machine patterns that handle this properly, see the call flow design guide. For sub-second turn latency across those state transitions, prompt engineering for voice agent latency covers the system prompt patterns that keep response time under 800ms.

Metrics for customer service voice agents: FCR, AHT, transfer rate, and CSAT proxies

The four metrics that matter, and how to capture them without specialist analytics tooling:

First-Call Resolution (FCR): Did the caller's query get resolved without a transfer or callback? At call level: if transfer_fired = false and the call ended on a natural close utterance, count it as resolved. Target FCR above 70% on automatable queries in the first month of live operation.

Average Handle Time (AHT): Total call duration. Measure separately for contained calls and transferred calls — a long transferred call is a human problem, not an agent failure. The logistics firm's human AHT was 4.1 minutes; the voice agent AHT is 82 seconds on contained calls.

Transfer Rate: Percentage of calls that hit a transfer trigger. Target below 30% in the first deployment; optimise toward below 20% by month three. A transfer rate that stays above 35% after two weeks almost always means a high-volume intent is missing from the knowledge base. If your inbound line mixes service queries with new-lead calls, see AI-assisted inbound lead routing for separating those streams before they reach the agent.

CSAT Proxy: Post-call surveys require an add-on. Use intent-resolution rate as the proxy instead: if the caller stated a query and the agent resolved it without confusion loops or repair sequences, mark it resolved. Compare monthly against callback requests logged in your CRM to validate the proxy is tracking real satisfaction.

Log all four at call level to Postgres. Query weekly — daily variance is too noisy for meaningful trend reading in the first 60 days.

What changed in 2025–2026: cross-session memory and persistent caller context in Retell and VAPI

The most significant change for customer service voice agents in the past 12 months is persistent caller context. Until mid-2025, each call was stateless — the agent had no memory of a previous call from the same number, and every repeat caller started from scratch.

Both Retell.ai and VAPI now support cross-session caller memory, storing structured data against a caller identifier — phone number or account ID — that persists across sessions. In practice:

• A caller who reported a damage claim last week does not need to re-explain it
• Authentication is faster for repeat callers: one-point check once a prior verification is on record
• The agent can reference history: "I can see you called on Monday about the delivery delay — shall I check the updated status?"

This is not a conversation history dump. The stored context is a structured object: {last_intent, last_resolution, open_tickets: [], account_verified: true, verified_at: "2026-07-28"}. The LLM reads it at session start to personalise the opening and skip re-authentication for callers verified in the last 48 hours.

The privacy implication matters: this data must appear in your Record of Processing Activities and your customer-facing privacy notice. Cross-session memory adds a new retention category — indefinite storage of call intent data — that most UK SMEs have not yet documented.

Good / Bad / Ugly: three customer service voice agent deployments and what each one missed

Good — The logistics firm (this post). A pre-deployment transcript audit identified the 11 automatable intents. Knowledge base built as structured Q&A for those 11, plus a function call for live order lookups. Hard transfer after two resolution failures. Month-three metrics: 78% containment, 82-second AHT, 18% transfer rate. What they got right: the audit came before the build. What they missed: the DPIA was not completed until week four after go-live. The ICO's guidance on AI processing should have been reviewed before the agent went live, not after.

Bad — A UK SaaS firm, 60 seats. Deployed without auditing call mix. Built 40 intent handlers based on what the team assumed callers asked. Real data showed 55% of calls were billing dispute queries the agent had no handler for. Month-one transfer rate: 61%. The agent recognised "billing query" but had no resolution path, so it looped twice before transferring — exactly the behaviour that frustrates callers most. Fix: back to the audit, rebuilt to 18 intents. Month two: 52% containment. Still below target, but the remaining queries are genuinely complex and human-appropriate.

Ugly — A facilities management company. Deployed without a disclosure statement. Callers were not told they were speaking to an automated system. A customer complained to the ICO after the agent denied liability for a missed service visit — a decision the customer believed a human had made. Ofcom's research on automated service complaints identifies non-disclosure as the fastest-growing category of grievance in AI-handled calls. The remediation required a complete persona rebuild plus mandatory disclosure in the first three seconds of every call. Six-week delay and reputational fallout; entirely avoidable.