Configuring automatic scoring

Item scores are determined by the student’s performance in the various Interactions which make up the Items of which the Test is composed. Interactions generate individual scores which count towards the overall score of the Item. These individual scores can be tallied in different ways - one of these ways is items being scored automatically in/by TAO. This topic shows how to configure the automatic scoring of items.

Manual and Test scoring

TAO provides the possibility for human scorers to score open-ended questions; this is available to Ignite and Enterprise customers. For more information, see the following topics:

• Scoring

• Manual scoring (TAO Core)

• TAO Grader (TAO Core)

For information on test scoring rules, see Scoring Tests.

Automatic scoring

The scoring method for an item needs to be defined during the item authoring process. There are two options for automatic scoring:

• The response of a test-taker is marked as wholly correct or wholly incorrect. In this case, no partial credit is awarded, so if part of the response is incorrect, the whole answer is marked as incorrect;

• Partial credit is awarded for the correct parts of a response, even if the response is not wholly correct.

Once you have configured automatic scoring for an item, it is a good practice to validate the configuration by previewing the item. By interacting with the item as a test taker would, and submitting the response in the item’s preview, you can ensure the scoring configuration behaves as expected.

To set up your chosen scoring method for an item after you have created your interaction, follow the steps below.

1. Go to the Question tab in the pop-up window within the item/interaction. In the Interaction Properties Panel, select the number of choices for the interaction (if applicable).

Partial credit is often awarded when multiple choices are allowed. For more information on configuring the interaction properties, see the topics on how to create an interaction.

If there is only one choice/correct answer in the interaction, skip this step and proceed to step 2.

If there are multiple choices allowed, select Multiple choices. Under Other constraints, set the minimum and maximum number of choices allowed.

2. Go to the Response tab in the pop-up window within the item/interaction. In the Response Properties panel on the right, locate the Response Processing pull-down choice box.

Not every Response Processing option is available for every interaction – the topics describing each interaction under QTI Interaction Types and PCI Interaction Types will tell you what is possible for that particular interaction.

The following list describes each possible Response Processing option:

• Match Correct: With this processing option, either the question is answered correctly or incorrectly. One or more (depending on how many correct) of the listed answers is the correct answer, and will be defined. The test taker must get the exact defined response(s) to get a full score on the item.

• Map Response: With this processing option, partial credit is awarded for correct parts of the response. This is useful for questions where the answer is given in multiple parts. With the map response option, more marks can be awarded for some parts of the answer than other parts. You can enter positive values to award points for correct answers, or negative values to deduct points for incorrect ones.

• None: There is no correct answer.

• Custom: This will open an xml editor where you can write your own scoring rules.

3. Select the Response Processing type which is appropriate for the question.

This is going to depend to a large extent on preference, particularly with respect to partial credit.

• If you select Match Correct, select the correct answer within the interaction’s available choices. The scoring configuration for this type of interaction is then complete.

• If you select Map Response, review the responses in your interaction, then determine and assign the corresponding marks to each potential response.

  Partial credit can be awarded on the Interaction canvas by entering in the score box the number of marks you wish to award for that response.

For most interaction types, score boxes are located to the right of the potential responses.

4. Optional: set the values of the Score Range fields, located in the Response Properties panel.

The optional Score Range field lets you set the minimum and maximum marks for the interaction. The minimum score defines the lowest possible score, which is useful when using negative marking for incorrect answers—for example, setting it to 0 ensures the score can’t be negative overall. For the maximum score, if multiple responses can be selected, it should reflect the total marks for all correct answers. If only one response is expected, it should match the value of the highest-marked response.

So if, for example, there are two correct responses in the question, and you have assigned 1 mark to one of them, and 2 marks to the other, the Test-taker would get a score of 3 if they are both correct (providing the maximum is set to 3 or above).

Other values in the Score Range field include Mapping Default. For more information, see the section below.

Mapping Default / Define Correct Response (optional)

The Mapping Default field, if used, sets the default value for any of the options where the author did not set an explicit score value for a response.

In this example, there are three possible choices with the correct response having a value of 2. Since the mapping default is set to -1, the incorrect responses will each have a score of -1.

Assuming multiple choices are allowed, choosing the correct response alone would give the test-taker a score of 2. If they chose the correct and an incorrect response they would receive a score of 1. And if they chose two incorrect responses they would receive a score of -2.

If you want to define a correct response, select the box in the interaction properties. When using the Map Response processing option, the Define Correct Response box allows you to tag the correct answer so it registers in the system and contributes to backend result processing.