Voice Agent Monitoring: Catching Failures Before Clients Do

A mortgage broker's outbound campaign ran for three days in March before anyone noticed the answer rate had collapsed. It dropped from 38% to 14%, and when we dug in, the error logs were clean. The agent configuration had not changed. Call volume was normal. The failure was upstream: a carrier routing change had silently added 2.4 seconds of ringback before the agent's first word. No alert fired. Three days of campaign budget had gone to calls where prospects hung up before a single syllable of the script landed.

That is a monitoring failure, not a voice agent failure.

The five voice agent failure modes that never appear in error logs

Error logs capture your application stack. They do not capture the carrier's routing layer, the network between your stack and the caller's handset, or a TTS provider's render queue under load. These are the five failure modes we have seen produce clean logs while campaigns silently degraded:

1. Carrier ringback delay. The SIP handshake succeeds, but the carrier inserts 2–4 seconds of dead ringback before bridging audio. Your logs show a completed call; the caller has already hung up.

2. Number flagging. Your number pool has accumulated enough rapid hang-ups to trigger a "Potential Spam" label from a carrier analytics provider such as TNS or First Orion. Answer rates drop 40–60% overnight. No error. The SIP layer is working fine.

3. AMD false positives at scale. Under high concurrency — 50 or more simultaneous calls — the answering machine detection model classifies 12–18% of live human answers as voicemail. Completed calls in your logs; missed conversations in the CRM.

4. TTS render queue saturation. Your TTS provider hits a capacity constraint and P99 render latency climbs from 280ms to 1,800ms. The caller hears two seconds of silence, then abandons. No error is thrown anywhere in your application.

5. SIP signalling lag on geographic routing. Campaigns targeting mobile numbers in specific UK regions route through an unexpected carrier path. The call connects 3–5 seconds later than expected. Callers don't wait.

All five produce 200 OK responses at your application layer. The only way to catch them before they compound is to measure outcomes, not HTTP status codes.

Answer rate monitoring: the leading indicator most teams start tracking after the first disaster

Answer rate — calls where a human actually spoke, divided by total dials — captures all five failure modes above. Most teams start tracking it the week after something goes wrong.

Segment it. Three outcome categories for every call:

• Human answered: caller spoke and remained past 8 seconds of two-way audio
• Voicemail or AMD: call connected but terminated by machine detection logic
• Early drop: call connected at the SIP layer but audio exchange lasted under 5 seconds

Track per 5-minute window, per number pool, and per carrier route. A single campaign-level aggregate hides the pattern — a flagged pool alongside a clean pool blends to "low" rather than "alarming" until the flagged pool handles most of the volume.

Alert threshold: amber at >15% below the 7-day rolling average for that pool; red — consider pausing — at >30%. If you just ramped volume by 3×, recalibrate the baseline first.

Our voice agent QA scorecards guide covers scoring individual call quality alongside answer rate — you need both to distinguish a volume problem from a conversation quality problem.

Latency dashboards: instrumenting STT, LLM, and TTS latency separately in production

Total turn latency is the time from the end of the caller's speech to the first byte of TTS audio. Target: under 800ms at P95. Above 1,200ms, hang-up rates climb. Above 1,800ms, callers assume the line has dropped.

Log total latency only and you know that something is wrong — not what. Instrument these three segments separately, tagged by campaign and number pool:

• STT latency: end-of-utterance (VAD firing) to transcript delivery from Deepgram or Whisper
• LLM latency: transcript received to first token generated
• TTS latency: first text chunk received to first audio byte from ElevenLabs or Cartesia

Here is the structure we use in our Node.js broker layer, publishing to Datadog via dogstatsd:

// broker/monitoring.js
const { StatsD } = require('hot-shots');
const dd = new StatsD({ host: 'localhost', port: 8125, prefix: 'voice_agent.' });

function recordTurnLatency(callId, phase, startMs, endMs, tags = {}) {
  const durationMs = endMs - startMs;
  const metricTags = [
    `call_id:${callId}`,
    `phase:${phase}`,            // 'stt' | 'llm' | 'tts'
    `campaign:${tags.campaign}`,
    `number_pool:${tags.numberPool}`,
  ];
  dd.histogram('turn_latency_ms', durationMs, metricTags);
  if (durationMs > 1200) {
    dd.increment('turn_latency_breach', metricTags);
  }
}

With this in place, a P95 spike is diagnosable in under two minutes: phase:tts spiking points to your TTS provider; phase:llm spiking points to your model or streaming config; phase:stt high means Deepgram is queueing or your VAD sensitivity is firing late. For more on reducing latency at the prompt level, see our guide on prompt engineering for voice agent latency.

Call drop detection and the SLA thresholds that trigger a response before the client notices

A drop happens after the audio bridge was established — call connected, audio was exchanged — but terminated before a meaningful conversation occurred. Our threshold: 8 seconds of two-way audio. Calls that end before that without a voicemail classification are drops.

Track three signals:

• Early drop rate: calls terminated before 8s of two-way audio, as a percentage of connected calls
• Agent non-response events: caller spoke but the agent produced no audio within 3 seconds
• Carrier-initiated disconnects: terminated by the network rather than by either party (SIP reason codes 503, 408, or 480)

SLA thresholds for a UK outbound campaign:

Financial services callers are less tolerant of dead air than e-commerce callers. If your vertical is mortgage broking or insurance, tighten the amber thresholds by roughly 20%.

Alerting stack: Datadog vs Grafana vs n8n webhook flows at UK SME-scale deployments

No single alerting tool fits every team. At UK SME scale, the honest trade-off:

For a 5-person ops team: n8n for operational alerts, Grafana Cloud for latency dashboards. Twilio Event Streams pushes call events to an n8n webhook; Retell webhooks deliver per-call analytics. Two n8n flows — one on answer rate, one on early drop rate — posting to a Slack channel will catch the majority of the failures described above.

Datadog earns its cost when you want distributed traces correlating a high-latency call through your broker, LLM, and TTS in a single view. For a single deployment, Grafana Cloud plus n8n is sufficient. For a comparison of the telephony platforms that feed these tools, see our Twilio vs Retell vs VAPI breakdown.

Carrier-side failures: distinguishing network issues from agent configuration problems

The fastest diagnostic is the parallel test dial. Trigger 10 test calls from a fresh number pool to numbers you control while the affected campaign runs. If the test pool also shows degraded rates, the problem is carrier-side. If it performs normally, the issue is campaign-specific.

Carrier-side signals: degradation hits all campaigns on the same routing path simultaneously; SIP reason codes 480, 503, or 408 appear; Twilio Voice Insights call quality alerts shows carrier-reported events; degradation is geographically concentrated in specific UK mobile prefixes.

Number-health or config signals: degradation is campaign-specific; CNAM lookup returns "Scam Likely"; onset was within 48 hours of a configuration deployment; outbound dials degrade but inbound test calls on the same numbers connect normally.

If number flagging is the root cause, remediation differs by flag source — our guide on phone number warmup and spam flag recovery covers the TNS, CNAM, and carrier-scoring paths separately.

Runbook design: the response procedure for each alert type, written before the alert fires

A runbook written after an alert fires is a post-mortem. Written before, it is the difference between a 20-minute resolution and a 3-hour investigation at 9pm.

For each alert type, define three things before go-live: the immediate response (what one person does within 5 minutes), the escalation trigger (when it is not resolving, who gets the message and what they need to act), and the campaign action (pause, reduce concurrency, or continue with elevated monitoring).

Here is an example for the answer rate amber alert:

alert: answer_rate_amber
threshold: ">15% below 7-day baseline for any number pool"
immediate_response:
  - Check Twilio carrier health dashboard for the affected route
  - Trigger 10 test dials from reserve pool to internal test numbers
  - Run CNAM lookup on affected number pool (First Orion portal or TNS lookup)
  - Check deployment log for configuration changes in the prior 48h
resolution_criteria: "Answer rate returns within 10% of baseline for 15 consecutive minutes"
escalation_trigger: "No improvement after 30 minutes, or rate continues to drop"
campaign_action: "Reduce concurrency by 50% while investigating"
client_notification: "Notify account owner if campaign paused for more than 2 hours"

Write one entry per alert type. Store runbooks in the same repository as your n8n flows, not in a separate wiki. The person covering an absent team member should reach resolution without prior context.

What changed in 2025–2026: provider-native observability dashboards in Retell and VAPI

Twelve months ago, monitoring a Retell or VAPI deployment meant building your own metrics pipeline from scratch. In 2025 and into 2026, both platforms shipped native analytics.

Retell's call analytics dashboard, released in Q3 2025, exposes per-call latency breakdowns for STT, LLM, and TTS — filterable by agent, date range, and call outcome. VAPI added webhook-based monitoring events in early 2026 that fire on configurable latency thresholds, so you can route events directly into an n8n flow.

Neither platform's dashboard sends alerts to Slack or PagerDuty without custom plumbing. Provider dashboards are excellent investigation tooling; the detection layer is still yours to build.

One counterpoint worth heeding: the Google SRE book on monitoring distributed systems argues against alert proliferation — every alert without a clear response procedure becomes noise that trains operators to ignore the channel. Start with answer rate and P95 turn latency; add others only once you have runbooks for them.

Good / Bad / Ugly: three monitoring setups and the failure each one missed

Good: Grafana plus n8n plus per-phase latency, built before go-live. A fintech client running eight voice agents had Grafana dashboards for STT, LLM, and TTS latency separately, n8n alert flows for answer rate and early drop rate, and a runbook for each alert type. When a TTS provider hit a capacity issue in January 2026, the latency alert fired at P95 = 1,340ms within six minutes. The on-call person confirmed the issue was TTS-specific, switched to the ElevenLabs fallback in the broker config, and restored normal latency within 18 minutes. No campaign was paused; no budget was wasted.

Bad: Twilio call logs only, no per-phase instrumentation. A property management company had call volume reporting but no answer rate baseline, no latency instrumentation, and no drop rate tracking. An AMD configuration change in Retell caused a 22% increase in false-positive voicemail classifications over 4 days. Call volume looked normal. One in five live human answers was being classified as voicemail and disconnected before the agent spoke. The issue surfaced when a prospect called back to ask why they had been hung up on — retrospective analysis estimated 340 missed conversations.

Ugly: No monitoring, discovered by the client. The mortgage broker from the opening of this post. No baselines, no latency tracking, no carrier health checks. The 2.4-second ringback addition was visible in Twilio Voice Insights after the fact — nobody was looking. The campaign spent approximately £4,200 over three days before the account manager noticed the CRM was not showing expected lead volume. By then, two numbers required a formal dispute process with the carrier analytics provider, and recovery needed a full number pool rotation plus two weeks of warmup before the campaign could resume.

Answer rate baselines, per-phase latency histograms, carrier health checks, and pre-written runbooks would have caught all three of these failures before they compounded. For the full instrumentation context, see our voice AI architecture overview and the voice AI and document analysis case study, where monitoring was built into the deployment from day one.