Using the confusion matrix to improve advanced AI agent performance (Legacy)

The confusion matrix provides a weekly visualization of the performance of an AI model, measured by its intent recognition. It shows you whether your AI model is able to distinctly recognize similar expressions under different intents or whether it is confused by them. This information can help you understand your AI agent performance.


What's my plan?







Add-on

AI agents - Advanced










The functionality described in this article is being rolled
out to all Suite and Support plans from May 11 through June 12, 2026. You may not
see this functionality in your account yet. See this announcement for details.



This article applies only to legacy expression-based AI agents.

The confusion matrix provides a weekly visualization of the performance of an AI model,
measured by its intent recognition. It shows you whether your AI model is able to distinctly
recognize similar expressions under different intents or whether it is confused by them. This
information can help you understand your AI agent performance.
In assessing AI agent performance, you might find that a message you expect to be recognized
as one intent is picked up as another intent because two intents overlap in their expressions.
You can use the confusion matrix to understand where the overlap occurs. You can then use that
information to adapt the manually trained expressions to eliminate inconsistencies and improve
performance.
There is also a list of issues that you can sort by priority to help you determine what to
work on.
This article covers the following topics:


Viewing the confusion matrix
Viewing issues raised in the confusion
matrix
Retraining the confusion
matrix




Viewing the confusion matrix

The confusion matrix is a color-coded grid that indicates how well the model is
performing in terms of understanding intents. A good model should have a dark line running
diagonally across the table.
You should review the confusion matrix when you notice the AI agent answered rate is
decreasing. A baseline of 80% is recommended. You might also review the confusion matrix if
you notice that your AI agent is regularly categorizing messages under the wrong intent.
To view the confusion matrix


In AI agents - Advanced, click Training Center in the
sidebar, then select Confusion Matrix.
Click the Confusion matrix tab at the top. The two dimensions, the X
(horizontal) and Y (vertical) axes, represent the actual and predicted intents
respectively. In this example, column 1 and row 1 are the actual and predicted intent
Affirmative, row 2 and column 2 are the intent Negative, and so on.


Review the colored cells in the matrix. The darker the cell, the more a predicted
intent overlaps with an actual intent. A good model should have a dark line running
diagonally across the table.
Colored cells outside of the diagonal represent how
often messages in an intent are predicted to another. Cells with darker colors outside
of the diagonal indicate a problem in the accuracy of the AI model caused by potential
overlaps of expressions between two intents.


Next, you can view the list of issues
where overlapping issues are automatically listed according to priority.



Viewing issues raised in the confusion matrix

You can use the list of issues to drill into issues surfaced in the confusion matrix.
Overlapping issues are automatically listed according to high, medium, or low priority to
help you determine what to focus on.
To access the list of issues


In AI agents - Advanced, click Training Center in the
sidebar, then select Confusion Matrix.
Click the List of issues tab at the top.
Click the Priority heading to sort by priority.

Use the Advanced filters to define the confusion level, as needed, or use
Search to find a specific intent you think might be causing confusion with other
intents.
Click Solve issue, then click Manage expressions to open the expressions
management view.
To manage highlighted items:
Click the left and right arrows to move an expression to a different intent.
Click the x to delete the expression.



Take any of the following actions as you work through the highlighted items:
Untrain the expressions or move them to the other intent.

Create a new intent with those expressions.

Merge the intents if they should be the same.
Help your AI model learn by training more expressions to those two confused intents.
Only do this if you are absolutely certain the two intents are very different and
should be separated.


Click Mark as solved to keep track of your progress.




Retraining the confusion matrix

The confusion matrix is automatically generated every Tuesday night (Pacific time).
If your AI agent has recently undergone extensive training, or if major alterations have been
made to it (changes to expressions or intents), you can perform manual retraining outside of the
schedule.
To retrain a confusion matrix


In AI agents - Advanced, click Training Center in the
sidebar, then select Confusion Matrix.
Click Re-train in the top-right corner.



{
"@context": "https://schema.org",
"@type": "TechArticle",
"@id": "https://support.zendesk.com/hc/en-us/articles/8357756496922",
"url": "https://support.zendesk.com/hc/en-us/articles/8357756496922",
"headline": "Using the confusion matrix to improve advanced AI agent performance (Legacy)",
"description": "You can use the confusion matrix to identify and resolve overlapping intent recognition issues in legacy expression-based AI agents by reviewing performance data and managing expressions to improve accuracy.",
"articleSection": "Using expression-based AI agents (Legacy)",
"datePublished": "2024-11-11T22:24:37+00:00",
"dateModified": "2026-05-12T13:53:57.542232+00:00",
"inLanguage": "en-US",
"mainEntityOfPage": {
"@type": "WebPage",
"@id": "https://support.zendesk.com/hc/en-us/articles/8357756496922"