Reviewing 2% of conversations at a 3% hallucination rate means only 0.06% of your traffic is a hallucination you actually see, so 98% of hallucinations go unreviewed.
Short answer: you build a scoring pipeline that runs on every conversation, checks every factual claim against a traceable source, and logs a hallucination rate you can graph against deployments. Not a prompt tweak. An actual system.
TL;DR
A hallucination = any claim not traceable to context, a tool result, or an earlier turn - whether it’s false or just unsupported.
Detection stack = grounding/NLI check → LLM-as-judge → deterministic rules, in that order, for cost reasons.
Track two metrics: faithfulness and citation coverage. Formulas below.
Sampling fails the math: reviewing 2% of calls at a 3% hallucination rate catches ~0.06% of total hallucinations.
Every flagged case becomes a regression test - not just an alert that gets dismissed.
Step 1: Define What Counts as a Hallucination
Split flags into two categories, because you’ll want different thresholds for each (per Datadog’s framework):
Contradiction - the output directly conflicts with the provided context (context is assumed correct).
Unsupported claim - the output isn’t contradicted by context, but also isn’t backed by it - it’s outside the retrieved scope or drawn from parametric knowledge.
For voice agents, add a third source entirely: ASR-layer hallucination - the speech-to-text transcript invents words the caller never said. One cited study found this in ~1% of transcriptions, concentrated during silence/pauses, with ~40% of those flagged as harmful. This is a transcription bug, not an LLM bug, and it needs its own detector - checking the LLM’s reasoning against a hallucinated transcript will always look “grounded” even though the input was wrong.
Decide upfront: do you flag contradictions only, or both? Regulated use cases (healthcare, finance, legal) should flag both. Lower-stakes use cases can start with contradictions only to reduce noise.
Step 2: Build the Grounding Check (Your First-Pass Filter)
For every response, extract atomic claims (one fact per unit), then check each claim against the source set available at that turn: retrieved documents, tool call results, system policy, and prior conversation turns.
Two metrics to compute per response:
faithfulness = (claims supported by source) / (total claims)
citation_coverage = (claims with a traceable source attached) / (total claims)
A response can have high citation coverage but low faithfulness - it cites something, but the citation doesn’t actually support the claim (this is a known failure mode: models attach plausible-looking citations to fabricated claims). Track both separately; don’t collapse them into one score.
Implementation-wise, this is usually a Natural Language Inference (NLI) model or an embedding-similarity check between each claim and the closest source chunk, used as a cheap first pass before anything more expensive runs.
Step 3: Add LLM-as-Judge for Nuance
NLI/embedding checks miss claims that require reasoning (multi-hop inference, numeric comparison, policy interpretation). For those, run a judge model with a structured prompt, for example:
You are checking factual grounding. Given:
CONTEXT: {retrieved docs + tool results + prior turns}
RESPONSE: {agent's response}
For each factual claim in RESPONSE, output:
- claim text
- verdict: SUPPORTED | CONTRADICTED | UNSUPPORTED
- which part of CONTEXT (if any) supports or contradicts it
Run this on claims the grounding check couldn’t confidently resolve, not on 100% of claims - the judge call is the expensive step in the pipeline, so gate it behind the cheap NLI filter.
Step 4: Layer Deterministic Rules on Top
These catch patterns that don’t need a model at all, and they’re nearly free to run on every single conversation:
Numbers in the response that don’t match any number returned by a tool call
Policy language (“we offer,” “our guarantee”) not present in the system prompt or knowledge base
Claimed actions (“I’ve sent the email,” “I’ve filed the ticket”) with no corresponding tool call in the trace
False affirmations - the agent confirms something the user said that contradicts an earlier tool result
For voice: transcript tokens with no phonetic plausibility match to the audio (an ASR-specific check, separate from the LLM check)
Order matters for cost: deterministic rules first (cheapest), then NLI/grounding (medium), then LLM-as-judge only on what’s still ambiguous (most expensive).
Step 5: Run It on 100% of Traffic, Not a Sample
The math here is unforgiving. Reviewing 2% of conversations against a 3% hallucination rate catches roughly 0.06% of total hallucinations - because you’re sampling a rare event out of a rare event. Worse, hallucinations are systematic, not random: a bad tool schema or a stale knowledge-base entry will trigger the same failure repeatedly, and a small sample can miss that entire cluster while everything still “looks fine” in your dashboard.
The pipeline above (deterministic → NLI → judge-on-ambiguous-cases) is specifically designed to make full-traffic scoring affordable - you’re not running an expensive LLM judge on every message, just on the subset the cheap layers couldn’t resolve.
Step 6: Track Rate Over Time, Segmented
Log every flagged hallucination with: model version, prompt version, tool(s) called, intent/conversation type, and environment. Then graph the rate (flags / total conversations) segmented by each of those dimensions, and overlay it against your deploy timeline. A rate that jumps right after a deploy is your fastest diagnostic signal - Datadog’s approach breaks this down by model, tool, and span specifically so you can isolate where it’s climbing, not just that it is.
Step 7: Close the Loop - Turn Flags Into Regression Tests
A flagged hallucination that doesn’t become a permanent test case will just recur.
Cluster flagged cases by root cause (bad tool response, stale KB doc, ambiguous prompt instruction, ASR mishear pattern).
Replay the real conversation as a held-out test case in your eval suite - exact input, exact expected grounding behavior.
Simulate the fix against your own past traffic before shipping, including the cases that already failed.
Ship and verify - confirm the rate for that specific cluster actually dropped post-merge, not just that the one example now passes.
Where Does VoxMith Fit?
VoxMith runs this full pipeline - grounding checks, clustering, and regression tracking - across 100% of production conversations, not a sample. It clusters recurring hallucination patterns into a ranked list by frequency and business impact, simulates proposed fixes against your own past traffic before you ship, and verifies the failure rate actually dropped after merge. A human approves every change.
FAQs
What’s the difference between faithfulness and citation coverage?
Faithfulness measures whether claims are actually supported by context. Citation coverage measures whether claims have a source attached at all. A response can score high on one and low on the other.
Do I need an LLM judge for every message?
Does RAG eliminate hallucinations?
No. Retrieval reduces them but models can still fabricate claims and attach a plausible-looking citation that doesn’t actually support what’s claimed.
Is ASR hallucination really separate from LLM hallucination?
Yes - it happens at the transcription layer, before the LLM ever sees the (wrong) input. It needs its own detector; checking LLM reasoning against a bad transcript will look “grounded” even though the input itself was hallucinated.
How much traffic before full-pipeline scoring is worth building?