Evaluating the performance of reviewers

Evaluate reviewer performance by creating a unique scorecard to conduct spot checks and assign an Internal Quality Score (IQS). Use this scorecard to review and monitor reviewers' work, identifying discrepancies without needing immediate calibration sessions. Leverage dashboards to analyze performance, focusing on areas like comment quality and review accuracy, ensuring consistent and accurate quality assurance processes.






Location: Zendesk QA > Settings > Scorecards

Evaluating the reviews completed by your reviewers in Zendesk QA is an
important part of a consistent and accurate QA process. By performing spot checks, you
can monitor and identify any discrepancies in your reviewers' work. You can think of
this as “grading the grader.”
This article describes how to create and use a unique scorecard to review your
reviewers. This method gives your reviewers their own Internal Quality Score (IQS) that is traceable
in the dashboard, without the need for immediate
calibration sessions.
This article contains the following topics:


Creating a grade
the grader scorecard
Reviewing your
reviewers
Monitoring your
reviewers performance


Related articles

Creating scorecards



Creating a grade the grader scorecard

To create a scorecard to evaluate reviewers



Create a separate workspace in
your account. For example, you can name it “Grade the grader”. This
is where the reviews for the reviewers will go.

Create a unique scorecard for
reviewing your reviewers. For example, you can name it
“Spot-check”.
Under Workspaces, select the workspace for reviewing
reviewers that you created earlier.
Under Workspaces with AutoQA enabled make sure No workspaces is selected.
Click Add a group section.
Add three binary categories to the
Spot-check group. For example:
Comment quality
Review accuracy
Review time



Select Add root causes to explain rating and add Root causes for Negative
ratings ( ) for each category in
the scorecard used by the reviewer to perform reviews:
[Category name]: Score too high
[Category name]: Score too low

Where [Category name] is replaced with the name of the
respective categories in the scorecard used by the reviewer to perform
reviews. For example: Tone Score too high; Tone Score too
low.

Configure any other settings for each category within the scorecard as
needed.
Click Apply.
Click Publish.





Reviewing your reviewers

To review your reviewers

In Quality assurance, click Conversations
in the sidebar.

Filter your conversation list for
conversations that have been reviewed in the past week.
Click Review.
Select the scorecard you have created for reviewing the reviewers. For
example, “Spot-check.”
Leave a review for the reviewer using the scorecard.





Monitoring your reviewers performance

Use dashboards to understand your reviewer's
performance.
To monitor your reviewers using dashboards



Filter your Reviews dashboard
specifically for this scorecard to visualize how your reviewers are
performing.

Use the Categories dashboard to dive
deeper into where the reviewer is misaligned (for example, scoring too high
or low for a specific category).