Improving AutoQA accuracy

<body>
<p id="docs-hc-snippet" style="display: none;">Autoscoring supports your manual review efforts by automatically evaluating and scoring customer interactions for 100% of your ticket volume based on predefined categories. This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time, allowing you to focus on categories that need more attention.</p>
<div class="article-banners">
<div id="docs-wmp-link">
<a href="https://support.zendesk.com/hc/en-us/articles/5411234991258-plan" rel="noopener" target="_blank">What's my plan?</a>
</div>

<div class="article-banner" id="addon_zqa">
<table>
<tbody>
<tr>
<td>
<strong>Add-on</strong>
</td>
<td>Quality Assurance (QA) or Workforce Engagement Management (WEM)</td>
</tr>
</tbody>
</table>
</div>

</div>
<div id="docs-ai-summary">
<p id="docs-ai-summary-toggle">Summary: <span id="up-triangle">◀</span><span id="down-triangle">▼</span></p>
<p id="docs-ai-summary-content">AutoQA helps you score conversations against predefined categories, compare AI ratings with human reviews, and spot where quality checks need more attention. You can submit a manual review to improve accuracy tracking, see how well categories align with your team’s ratings, and use accuracy scores as a confidence check for agent performance and quality assurance.</p>
</div>

<div class="body">

<div class="p">
<div class="note fastpath">
<span class="fastpathtitle">Location: </span> Zendesk QA &gt; Conversations</div>
</div>
<p class="p" dir="ltr">Autoscoring supports your manual review efforts by automatically evaluating and
scoring customer interactions for 100% of your ticket volume based on predefined categories.
This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time,
allowing you to focus on categories that need more attention.</p>
<p class="p">The accuracy score measures the consistency and level of agreement between ratings generated
by AutoQA and those provided by human reviewers for the same conversation. </p>
<div class="p">This article contains the following topics: <ul class="ul" id="topic-1__ul_ahw_jpm_5hc">
<li class="li"><a class="xref" href="#topic-1__ul_ahw_jpm_5hc">Understanding AutoQA
accuracy</a></li>
<li class="li"><a class="xref" href="#topic_mnh_kb2_fkc">Manually submitting a review on an AutoQA
category to improve accuracy</a></li>
</ul>
</div>
<div class="p">Related articles: <ul class="ul" id="topic-1__ul_mjc_lpm_5hc">
<li class="li"><a class="xref" href="https://support.zendesk.com/hc/en-us/articles/7043747123354" target="_blank">Understanding autoscoring categories</a></li>
<li class="li"><a class="xref" href="https://support.zendesk.com/hc/en-us/articles/9019507481242" target="_blank">Understanding the AutoQA dashboard in Zendesk QA</a></li>
</ul>
</div>
</div>
<div aria-labelledby="ariaid-title2" class="topic nested1" id="topic_prh_hb2_fkc">
<h2 class="title topictitle2" id="ariaid-title2">Understanding AutoQA accuracy</h2>
<div class="body">
<p class="p" dir="ltr">Autoscores can vary for different agents. You can’t change an autoscore however,
as a reviewer, if you disagree with the autoscore, you can <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/7043669307418#topic_ows_lv2_p2c" target="_blank">submit a different rating for the conversation</a>.</p>
<p class="p">Although AutoQA doesn’t learn directly from your feedback, you can monitor system
performance by checking the <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/9019507481242#autoqa_dashboard_examples__accuracy_score" target="_blank">accuracy score</a> on the AutoQA dashboard.</p>
<p class="p">The accuracy score measures the consistency and level of agreement between ratings
generated by AutoQA and those provided by human reviewers for the same conversation. </p>
<p class="p">Ratings vary based on the scale used. For example:</p>
<div class="p">
<ul class="ul" id="topic_prh_hb2_fkc__ul_n5x_3b2_fkc">
<li class="li">
<strong class="ph b">Binary scale</strong>:<ul class="ul" id="topic_prh_hb2_fkc__ul_o5x_3b2_fkc">
<li class="li">If you agree, the accuracy is 100%.</li>
<li class="li">If you disagree, the accuracy is 0%.</li>
</ul>
</li>
<li class="li">
<strong class="ph b">Five-point scale</strong>:<ul class="ul" id="topic_prh_hb2_fkc__ul_p5x_3b2_fkc">
<li class="li">A rating of 5 out of 5 has 100% accuracy.</li>
<li class="li">A rating of 4 out of 5 has 75% accuracy.</li>
<li class="li">A rating of 3 out of 5 has 50% accuracy.</li>
<li class="li">A rating of 2 out of 5 has 25% accuracy.</li>
<li class="li">A rating of 1 out of 5 has 0% accuracy.</li>
</ul>
</li>
</ul>
</div>
<p class="p">AutoQA accuracy helps you:</p>
<ul class="ul" id="topic_prh_hb2_fkc__ul_q5x_3b2_fkc">
<li class="li">Understand which AutoQA categories align best with human reviews and how reliable AutoQA
is. <p class="p">For example, if <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/7043747123354#understanding_autoscoring_categories__comprehension" target="_blank">Comprehension</a> has an accuracy score of 90%, that
means human reviewers agreed with AutoQA's rating on 9 out of 10 conversations they both
reviewed. </p>
</li>
<li class="li">Use the AI’s scoring as a confidence indicator when evaluating agent performance.</li>
<li class="li">Reduce time spent manually checking every conversation to identify categories that may
need more review attention.</li>
</ul>
<p class="p" dir="ltr">Automated reviews are tracked using the <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/9019507481242#autoqa_dashboard_examples__AQS" target="_blank">Auto Quality Score (AQS)</a>, while manually submitted
grades contribute to the agents’ <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/7043701093786#topic_sws_4ww_42c" target="_blank">Internal Quality Score (IQS)</a>. </p>
<p class="p" dir="ltr">AQS counts toward the IQS of a conversation only after a reviewer manually
submits the review.</p>
</div>
</div>
<div aria-labelledby="ariaid-title3" class="topic nested1" id="topic_mnh_kb2_fkc">
<h2 class="title topictitle2" id="ariaid-title3">Manually submitting a review on an AutoQA category to improve accuracy</h2>
<div class="body">
<p class="p">To improve AutoQA accuracy, ensure that your scorecard includes the AutoQA categories you
want to evaluate before you manually submit a review. This lets the system compare your
rating with AutoQA’s rating for the same category.</p>
<p class="p"><strong class="ph b">To manually submit a review on an AutoQA category to improve accuracy</strong></p>
<ol class="ol" id="topic_mnh_kb2_fkc__ol_jbw_lb2_fkc">
<li class="li" dir="ltr"><span class="ph">In <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/4408838272410" target="_blank">Quality assurance</a>, click <img class="image" id="topic_mnh_kb2_fkc__d8e80" src="https://zen-marketing-documentation.s3.amazonaws.com/docs/en/qa_conversations_icon.png" width="22">
<strong class="ph b">Conversations</strong> in the sidebar.</span></li>
<li class="li" dir="ltr">Select the conversation you want to review.<div class="p">
<div class="note attention">
<span class="attentiontitle"></span> <strong class="ph b">Tip:</strong> You can <a class="xref" href="https://support.zendesk.com/hc/en-us/articles/7043759449114" target="_blank">use custom filters to find conversations</a> using AutoQA
categories to review. </div>
</div>
</li>
<li class="li" dir="ltr">Click <strong class="ph b">Review</strong>.<p class="p"><img class="image" id="topic_mnh_kb2_fkc__image_kbw_lb2_fkc" src="https://zen-marketing-documentation.s3.amazonaws.com/docs/en/zqa_review_autoscore.png" width="300"></p>
</li>
<li class="li">Select a scorecard. </li>
<li class="li">Select your new rating using the AutoQA category ratings that are already filled in for
reference.</li>
<li class="li">Click <strong class="ph b">Submit</strong>.</li>
</ol>
</div>
</div>
<script id="docs-schema-markup" type="application/ld+json">{
"@context": "https://schema.org",
"@type": "TechArticle",
"@id": "https://support.zendesk.com/hc/en-us/articles/9046364914842",
"url": "https://support.zendesk.com/hc/en-us/articles/9046364914842",
"headline": "Improving AutoQA accuracy",
"description": "You learn how to assess AutoQA accuracy and submit a manual review for an AutoQA category so you can compare ratings and improve agreement with human reviews.",
"articleSection": "Working with conversations in Zendesk QA",
"datePublished": "2025-03-20T17:17:48+00:00",
"dateModified": "2026-08-11T14:19:29.647060+00:00",
"inLanguage": "en",
"mainEntityOfPage": {
"@type": "WebPage",
"@id": "https://support.zendesk.com/hc/en-us/articles/9046364914842"
},
"author": {
"@type": "Organization",
"name": "Zendesk",
"@id": "https://www.zendesk.com"
},
"publisher": {
"@type": "Organization",
"name": "Zendesk",
"@id": "https://www.zendesk.com"
},
"isPartOf": {
"@type": "WebSite",
"name": "Zendesk Help",
"@id": "https://support.zendesk.com/hc/"
},
"about": {
"@type": "SoftwareApplication",
"name": "Zendesk",
"@id": "https://www.zendesk.com",
"softwareAddOn": {
"@type": "SoftwareApplication",
"name": "Quality Assurance",
"@id": "https://support.zendesk.com/hc/en-us/sections/6999625340058"
}
}
}</script><script id="fast-path-script">
//<![CDATA[
const fast_path_style = document.createElement('style');
fast_path_style.innerHTML = `#article-body .fastpath{font-size:.9em;background-color:#f3f3f3;border:1px solid #d3d3d3;padding:10px 10px 8px;margin-bottom:16px}#article-body .fastpath::before{content:none!important}`;
document.head.appendChild(fast_path_style);
//]]>
</script><script id="ai-summary-script">
//<![CDATA[
const ai_summary_style = document.createElement('style');
ai_summary_style.innerHTML = `#styled-article-body{padding:0 0 0}#article-body #docs-ai-summary{background-color:#f3f3f3;border:1px solid #d3d3d3;padding:10px 10px 6px;margin-bottom:16px}#article-body #docs-ai-summary #docs-ai-summary-toggle{font-size:.9em;font-weight:700;margin-bottom:4px;cursor:pointer;display:flex;justify-content:space-between;align-items:center}#article-body #docs-ai-summary #docs-ai-summary-toggle #up-triangle{margin-left:auto;padding-right:10px;display:none}#article-body #docs-ai-summary #docs-ai-summary-toggle #down-triangle{margin-left:auto;padding-right:10px}#article-body #docs-ai-summary #docs-ai-summary-content{font-size:.9em;padding-left:25px;padding-right:25px;margin-bottom:12px}`;
document.head.appendChild(ai_summary_style);

const toggleAISummary = () => {
const summary = document.getElementById("docs-ai-summary-content");
const upTriangle = document.getElementById("up-triangle");
const downTriangle = document.getElementById("down-triangle");
if (summary.style.display === "none") {
summary.style.display = "block";
upTriangle.style.display = "none";
downTriangle.style.display = "inline";
} else {
summary.style.display = "none";
upTriangle.style.display = "inline";
downTriangle.style.display = "none";
}
};

document.getElementById("docs-ai-summary-toggle").addEventListener("click", toggleAISummary);
//]]>
</script>
</body>