Configuring automatic scoring

Item scores are determined by the student’s performance in the various Interactions which make up the Items of which the Test is composed. Interactions generate individual scores which count towards the overall score of the Item. These individual scores can be tallied in different ways - one of these ways is items being scored automatically in/by TAO. This topic shows how to configure the automatic scoring of items.

Manual and Test scoring

TAO provides the possibility for human scorers to score open-ended questions; this is available to Ignite and Enterprise customers. For more information, see the following topics:

• Scoring

• Manual scoring (TAO Core)

• TAO Grader (TAO Core)

For information on test scoring rules, see Scoring Tests.

Automatic scoring

The scoring method for an item needs to be defined during the item authoring process. There are two options for automatic scoring:

• The response of a test-taker is marked as wholly correct or wholly incorrect. In this case, no partial credit is awarded, so if part of the response is incorrect, the whole answer is marked as incorrect;

• Partial credit is awarded for the correct parts of a response, even if the response is not wholly correct.

To set up your chosen scoring method for the item in question, go to the Response window after you have created your interaction and follow the steps below.

1. Select the number of choices for the interaction (if applicable).

Partial credit is often awarded when multiple choices are allowed. For more information on configuring the interaction properties, see the topics on how to create an interaction.

If there is only one choice/correct answer in the interaction, skip this step and proceed to step 2.

Go to the Question tab in the pop-up window within the item/interaction.

Select the maximum number of choices in the Interaction Properties Panel.

2. Select the Response tab in the pop-up window within the item/interaction. In the Response Properties panel on the right, locate the Response Processing pull-down choice box.

Not every Response Processing option is available for every interaction – the topics describing each interaction under QTI Interaction Types and PCI Interaction Types will tell you what is possible for that particular interaction.

The following list describes each possible Response Processing option:

• Match Correct: With this processing option, either the question is answered correctly or incorrectly. One or more (depending on how many correct) of the listed answers is the correct answer, and will be defined. The test taker must get the exact defined response(s) to get full score on the item.

• Map Response: With this processing option, partial credit is awarded for correct parts of the response. This is useful for questions where the answer is given in multiple parts. With the map response option, more marks can be awarded for some parts of the answer than other parts, if you consider them to be of higher importance.

• None: There is no correct answer.

• Custom: This will open an xml editor where you can write your own scoring rules.

3. Select the Response Processing type which is appropriate for the question.

This is going to depend to a large extent on preference, particularly with respect to partial credit.

• If you select Match Correct, select the correct answer within the interaction’s available choices. The scoring configuration for this type of interaction is then complete.

• If you select Map Response, review the responses in your interaction, then determine and assign the corresponding marks to each potential response.

  Partial credit can be awarded on the Interaction canvas by entering in the score box the number of marks you wish to award for that response.

For most interaction types, score boxes are located to the right of the potential responses.

4. Optional: set the values of the Score Range fields, located in the Response Properties panel.

This is where you can specify the minimum and maximum number of marks awarded for this interaction. Its use is optional.

The minimum score indicates the minimum number of responses the Test-taker is required to select for it to be a valid answer. If the interaction involves selecting more than one response, the maximum score should reflect the total marks for all correct responses, i.e. the maximum score possible for your Interaction. If only one response is expected, the maximum should equal that of the marks in the score box with the highest number. Adjust these values as needed.

So if, for example, there are two correct responses in the question, and you have assigned 1 mark to one of them, and 2 marks to the other, the Test-taker would get a score of 3 if they are both correct (providing the maximum is set to 3 or above).

Other values in the Score Range field include Mapping Default. For more information, see the section below.

Map Response (optional)

The Mapping Default field, if used, contains the default value given if no specific scores are assigned to a response, and the check box used to Define Correct Response. The latter should be checked if there are specific correct responses, and left unchecked if the correctness of the answer is dependent on the sum of marks accumulated by the Test-taker in answering the interaction.

In this example, there are three possible choices with the correct response having a value of 2 and the incorrect responses each having a score of -1 from the default mapping setting.

Choosing the correct response alone would give the test-taker a score of 2. If they chose the correct and an incorrect response they would receive a score of 1. And if they chose two incorrect responses they would receive a score of -2.