Managing and strengthening the AI model for an advanced AI agent (Legacy)

This article applies only to expression-based AI agents, which is functionality that moved into legacy status in December 2024. For more information, see About AI agents. As additional intents and expressions are added and the AI model becomes more advanced, it is essential to monitor how the AI model is performing and identify where improvements can be made. We offer a number of tools you can use to strengthen the AI model. Optimized training Here’s a step-by-step guide to optimizing your training by using our AI-based tools. This is an iterative process that can be done both pre- and post-launch. Step 1: Generate a new evaluated model
Go to AI > Models > + New Model > Tick ‘Run Cross-validation’ > Train AI agent.
This can take several hours. The status will move from Queued > Started > Finished, so you’ll know when it’s done.
Tip: run the night before if you want to train the following morning.
Step 2: Consult Confusion Matrix Go to Training Center > Confusion Matrix. Use ‘Confusion Matrix’ and ‘List of Issues’ to evaluate which intents are "confused" or overlapping with other intents. For the Medium & High intents, go to Manage Expressions to deep dive into which expressions are causing confusion. These will be highlighted in yellow. Ask yourself:
Are both intents well defined? Is the scope of each intent clear? Tip: use the Intent Description to clearly outline scope of an intent
Does the topic of this intent or its expression overlap with another intent? If there is overlap, does it make sense to merge the intents?
Does the intent have enough expressions to live on its own? Note: the more expressions an intent has the more weight it has in the model, so if an intent isn’t being recognized when it should be, it can be due to a lack of sufficient training data.
Step 3: Train from Impact Report

Generate a new Impact Report:



Go to AI > Impact Report > + New Report


Select “From Existing Messages” if you wish to use data already imported, or “From CRM” if you wish to import additional data


Specify a timeframe: depending on the timeframe selected and the volume of messages, this can take anywhere from 10 mins to an hour


Set ‘Benchmark Model’ as ‘Current Model’ > Create.


Use the new impact report to train. Use the filters to target your training. After Step 3, you could re-run an evaluated model and check whether the Confusion Matrix has improved.
{
"@context": "https://schema.org",
"@type": "TechArticle",
"@id": "https://support.zendesk.com/hc/en-us/articles/8357756708378",
"url": "https://support.zendesk.com/hc/en-us/articles/8357756708378-Managing-and-strengthening-the-AI-model-for-an-advanced-AI-agent",
"headline": "Managing and strengthening the AI model for an advanced AI agent",
"description": "You can manage and improve an advanced AI agent's model by generating evaluated models, analyzing confusion matrices, and training from impact reports to identify and address intent overlaps and data gaps.",
"articleSection": "Using expression-based AI agents (Legacy)",
"datePublished": "2024-11-11T22:25:01+00:00",
"dateModified": "2025-09-05T23:18:59+00:00",
"inLanguage": "en-US",
"mainEntityOfPage": {
"@type": "WebPage",
"@id": "https://support.zendesk.com/hc/en-us/articles/8357756708378"