How does read-behind scoring work

Related articles

• Read behind scoring

• How to grade an assessment

• Global and Group roles

• Creating a new group

• Creating a session

Applies to

Personas

• Administrators

• Scorers

• Score reviewers

Prerequisites

To follow along with this playbook, ensure that:

• Manual scoring has been configured for the assessment.

• Read-behind scoring has been enabled as a beta feature in System Settings.

• A session has been created with Read-behind Scoring enabled.

• Users have been assigned the appropriate Scorer and Score Reviewer group roles.

Purpose

Read-behind scoring adds a second layer of quality assurance to manually scored assessments. After a scorer completes grading, a score reviewer independently reviews a configurable percentage of completed responses before the scoring process is finalized.

This playbook explains how read-behind scoring is configured, how reviewers participate in the workflow, and how reviewed scores are managed within TAO.

Recommended workflow

When using read-behind scoring, follow these phases:

1. Enable the feature in System Settings.

2. Configure read-behind scoring for a session.

3. Complete manual scoring.

4. Review selected responses.

5. Finalize reviewed scores.

Workflow

1. Understanding the Read-behind Scoring workflow

Read-behind scoring is a two-step manual grading process:

1. A Scorer grades the assessment.

2. A Score Reviewer reviews a selected percentage of completed responses.

Unlike standard manual scoring, reviewers do not begin their work until scoring has been completed for a test-taker's session.

The percentage of responses reviewed is determined during session configuration.

2. Enable Read-behind Scoring

Read-behind scoring is available as a beta feature.

Before it can be used:

1. Open System Settings.

2. Navigate to the TAO Grader configuration.

3. Enable Read-Behind Scoring as a beta feature.

4. Save the configuration.

Once enabled globally, the feature becomes available when creating or editing individual sessions.

3. Configure Read-behind scoring for a session

When creating or editing a session:

1. Navigate to the Scoring configuration options > Scoring section.

2. Enable Read-behind scoring.

3. (Optional) Define the review open and close dates.

4. Enter the Percentage of read-behind Review. For example:

   • 25 reviews approximately 25% of completed responses.

   • 50 reviews approximately half of completed responses.

   • 100 reviews every manually scored response.

5. Optionally, enable Score test-takers anonymously to hide the names and usernames of scorers and reviewers during the scoring and review process.

4. Assign the appropriate roles

Read-behind scoring requires two group roles:

Both roles must be assigned to users before the scoring workflow begins.

Group roles can be added while:

• Creating or updating a group (Users tab)

• Editing an individual user’s group role (Groups tab)

5. Complete the first round of manual scoring

Lead (initial) scorers complete the assessment using the standard manual scoring workflow.

Once the first round of grading is complete for a test-taker, the session becomes available for review.

6. Reviewing scores as a score reviewer

Score Reviewers begin the review process from the session overview.

To review scores:

1. Open the session.

2. Select Enter Session.

3. Locate the test-taker whose status is Score review needed.

4. Select Review scores.

To review multiple completed assessments, select Review All.

The grading interface opens with the review panel.

6a. Evaluating the submitted scores

During the review process, reviewers can examine the work completed by the original scorer.

The review interface displays:

• The Outcome Declaration(s) configured for the item.

• The available score range for each scoring outcome.

• The scorer's assigned score.

• Any notes entered by the scorer.

To complete the review, the reviewer:

1. Assigns their own scores.

2. Adds reviewer notes.

3. Annotates responses using the marking tools.

4. Highlights portions of the response and provide inline feedback using Write Feedback.

These tools allow reviewers to verify or adjust scoring while documenting their decisions.

6b. Submitting the review

Once the required responses have been reviewed:

1. Confirm review progress using the completion indicator.

2. Select Send Scores.

The session status updates to Scoring reviewed.

7. Viewing completed scoring

Score Reviewers can access completed scoring in two different ways.

View scores

View Scores opens a read-only version of the completed scoring.

Reviewers can view:

• Original scorer scores.

• Reviewer scores.

• Scorer notes.

• Reviewer notes.

• Inline feedback.

This option is available:

• While the review process is in progress.

• After the review process has been completed.

Review test

Review Test displays the assessment as the test-taker experienced it.

Three tabs are available:

This view becomes available as soon as the assessment has been submitted, regardless of the scoring status.