AI agent optimization backed by
real conversations

AI agent optimization backed by real conversations

Every fix tested on the conversations it was meant to fix, then proven on live traffic.

Mountain landscape at dusk

What is AI agent optimization?

What is AI agent optimization?

AI agent optimization is improving a deployed agent's accuracy, speed, outcomes
and cost using evidence from real conversations rather than guesswork.

AI agent optimization is improving a deployed agent's accuracy, speed, outcomes and cost using evidence from real conversations rather than guesswork.

VoxMith is product analytics for AI agents.

It reads every conversation, finds what failed, tests the fix on real traffic, ships the fix and proves it worked.

It is continuous rather than a project. The agent changes, the models underneath it change, and the people using it find new ways to ask for things. An agent that was tuned six months ago is not tuned now.

Reads every conversation

Finds what failed

Failed

Tests the fix on real traffic

Ships the fix

Merged

Proves it worked

Stopped

Why agents plateau after launch

Why agents plateau after launch

The first few weeks are easy. The obvious failures are loud,
someone fixes them, and the numbers move. Then it stops.

The first few weeks are easy. The obvious failures are loud, someone fixes them, and the numbers move. Then it stops.

Prompts get edited on instinct

A complaint reaches Slack, somebody rewrites a paragraph, and nobody can say afterwards whether it helped or hurt.

Model swaps quietly make things worse

The new model is better on benchmarks and worse at your particular job, and you find out from churn rather than from a test.

Fixing the loud problem is not fixing the big one

The failure someone complained about is rarely the one costing the most. Without a ranked list you work on whatever was mentioned most recently.

There is no way to prove a change worked

So nobody is confident enough to make the next one, and the agent sits where it is.

Prompts get edited on instinct

A complaint reaches Slack, somebody rewrites a paragraph, and nobody can say afterwards whether it helped or hurt.

Model swaps quietly make things worse

The new model is better on benchmarks and worse at your particular job, and you find out from churn rather than from a test.

Fixing the loud problem is not fixing the big one

The failure someone complained about is rarely the one costing the most. Without a ranked list you work on whatever was mentioned most recently.

There is no way to prove a change worked

So nobody is confident enough to make the next one, and the agent sits where it is.

What VoxMith optimizes

What VoxMith optimizes

Conversation quality

Wrong answers, missing knowledge and flows that confuse people. Each one traced to the prompt, the tool, the retrieval or the knowledge base that produced it, rather than treated as a general model problem.

Prompt patch

Traced to

Prompt

Tool

Retrieval

Knowledge base

Speed

Which stage is adding the delay on real sessions, and which lever shortens it. Latency is not one number. It is a budget split across transcription, model, tools and output, and only one line of it is usually the problem.

Where a smaller or faster model would hold quality

Transcription

Model

Tools

Output

Outcomes

Resolution and containment up, avoidable escalations and drop-offs down. These are the numbers your buyer is measured on, so they are the ones worth moving first.

Resolution

Containment

Avoidable escalations

Drop-offs

Cost per session

Fewer wasted turns and fewer failed sessions lower what every resolution costs. Usage data also shows where a smaller or faster model would hold quality.

Useful session

Useful session

Where a smaller or faster model would hold quality

Where a smaller or faster model would hold quality

How optimization works in VoxMith

How optimization works in VoxMith

1

Find the pattern

Failures cluster by cause and rank by what they cost. You see the biggest one first, not the loudest one.

Biggest by cost

Next by cost

Loudest complaint

2

Get candidate fixes

A prompt patch, a tool change, a flow change or a knowledge gap to close. Each comes with the sessions that prove it is needed.

Prompt patch

Tool change

Flow change

Knowledge gap

1

Find the pattern

Failures cluster by cause and rank by what they cost. You see the biggest one first, not the loudest one.

Biggest by cost

Next by cost

Loudest complaint

2

Get candidate fixes

A prompt patch, a tool change, a flow change or a knowledge gap to close. Each comes with the sessions that prove it is needed.

Prompt patch

Tool change

Flow change

Knowledge gap

3

Platform

Every candidate runs against the real conversations it was meant to fix. You see which one resolves the most and what it breaks elsewhere.

Step three is the one that matters. A recommendation without it is a model suggesting a prompt tweak, which anyone can get for free. Replayed against the sessions that actually failed, it becomes a comparison between options with a result attached.

Without replay

A suggested prompt tweak

Failures cluster by cause and rank by what they cost.

Failures cluster by cause and rank by what they cost. You see the biggest one first, not the loudest one.

Replay them against the sessions that failed

Replayed on failed sessions

Candidate A

Resolves the most

Candidate B

Candidate C

Breaks elsewhere

4

Ship the winner your way

As a pull request in GitHub, a ticket in Jira or Linear, a downloadable skill, or PR-ready code you can drop in when Git is not connected. Each carries the replay results and the evidence. You approve it.

Pull request in GitHub

Ticket in Jira or Linear

Downloadable skill

PR-ready code

5

Track it on live traffic

We watch that failure class after the change ships and tell you whether it stopped.

Failing

Before

Stopped

After it shipped

4

Ship the winner your way

As a pull request in GitHub, a ticket in Jira or Linear, a downloadable skill, or PR-ready code you can drop in when Git is not connected. Each carries the replay results and the evidence. You approve it.

Pull request in GitHub

Ticket in Jira or Linear

Downloadable skill

PR-ready code

5

Track it on live traffic

We watch that failure class after the change ships and tell you whether it stopped.

Failing

Before

Stopped

After it shipped

These are real conversations your users already had.

Simulated callers

Synthetic personas

Invented test cases

No simulated callers, no synthetic personas, no invented test cases.

You approve every change

Your agent only changes when you merge the change.

There is no mode where VoxMith edits your production behaviour on its own.

Your agent only changes when you merge the change. There is no mode where VoxMith edits your production behaviour on its own.

This is a deliberate choice rather than a missing feature.

Automatic self-optimization sounds efficient until...

A regulated disclosure quietly gets rephrased.

A model decides a shorter answer is a better one.

Proposed change

Waiting for review

The evidence

The difference

The replay result

Resovle the most

Waits for a person

Decline

Approve & Merge

Proposed change

Waiting for review

The evidence

The difference

The replay result

Resovle the most

Waits for a person

Decline

Approve & Merge

Measuring the impact of every fix

Measuring the impact of every fix

Each change gets a before and after on the numbers that matter:
the rate of the failure class it targeted, resolution, containment, latency and cost per resolved session.

Each change gets a before and after on the numbers that matter:
the rate of the failure class it targeted, resolution, containment, latency and cost per resolved session.

Changes appear as markers on your performance charts, so when a number moves you can see which change it followed. That is the part teams usually cannot answer, and it is the reason agents plateau.

Once you can prove a change worked, you make the next one.

Performance Chart

A change

The number moves

Before and after, for every change

Before and after, for every change

Failure class rate

Before

After

Resolution

Before

After

Containment

Before

After

Latency

Before

After

Cost per resolved session

Before

After

Failure class rate

Before

After

Resolution

Before

After

Containment

Before

After

Latency

Before

After

Cost per resolved session

Before

After

Work out what yours cost:

AI agent optimization FAQs

AI agent optimization FAQs

What is AI agent optimization?

AI agent optimization is improving a deployed agent's accuracy, speed, outcomes and cost using evidence from real conversations rather than guesswork. It is continuous, because the agent, the models underneath it and the people using it all keep changing.

How do you improve an AI agent's performance?

Find the failures that cost the most, work out the cause, change the prompt, tool, flow or knowledge that caused it, and verify the change against the conversations that failed before shipping. Then watch that failure class on live traffic to confirm it stopped.

Does VoxMith change my agent automatically?

No. Every change arrives as a pull request, a Jira or Linear ticket, a downloadable skill or PR-ready code, with the evidence and replay results attached. You review it and apply it. Your agent only changes when you decide it does.

How do you know an optimization actually worked?

Two checks. Before it ships, each candidate fix is replayed against the real sessions that failed, so you see how many it resolves and what it breaks elsewhere. After it ships, VoxMith watches that failure class on live traffic and reports whether the rate fell.

How do you increase resolution rate for an AI agent?

Work from the failure causes, not the intents. Group every unresolved session by why it failed, rank the groups by what they cost, and fix the top one. Most of the gap usually sits in three or four causes, often a missing capability, an empty retrieval or context dropped between turns.

How do you reduce AI agent latency?

Find the stage carrying the delay before changing anything. Latency is a budget split across transcription, model time to first token, tool calls and output. Teams usually blame the model when a slow tool call is the problem. Stream everything, cache what repeats, and run tool calls in parallel.

What is a good response latency for an AI agent?

Under roughly 800 milliseconds for a conversational turn, based on published industry guidance. Past about a second in voice, callers start talking over the agent or assume the line has dropped. In chat the tolerance is higher but abandonment still rises.

Can VoxMith help reduce cost per session?

Yes, in two ways. Fewer wasted turns and fewer failed sessions lower the cost of every resolution. Separately, usage data shows where a smaller or faster model holds quality, so you are not paying frontier prices for work that does not need it.

Make your agent better every week

Find what is holding it back, ship the fix, and prove it worked. Start small, grow into it.

Make your agent better every week

Find what is holding it back, ship the fix, and prove it worked. Start small, grow into it.

Make your agent better every week

Find what is holding it back, ship the fix, and prove it worked. Start small, grow into it.