IMS Logo

IMS Question and Test Interoperability Results Reporting

Version 2.1 Public Draft (revision 2) Specification

Copyright © 2006 IMS Global Learning Consortium, Inc. All Rights Reserved.
The IMS Logo is a registered trademark of IMS GLC.
Document Name: IMS Question and Test Interoperability Results Reporting


Date Issued:
8 June 2006


Caution: this specification is incomplete in its current state. The IMS QTI project group is in the process of evolving this specification based on input from market participants. Suppliers of products and services are encouraged to participate by contacting Mark McKell at [email protected]. This specification will be superseded by an updated release based on the input of the project group participants.

Please note that supplier's claims as to implementation of QTI v2.1 and conformance to it HAVE NOT BEEN VALIDATED by IMS GLC. While such suppliers are likely well-intentioned, IMS GLC member organizations have not yet put in place the testing process to validate these claims. IMS GLC currently grants a conformance mark to the Common Cartridge profile of QTI v1.2.1. The authoritative source of products and services that meet this conformance is contained in the IMS online product directory http://www.imsglobal.org/ProductDirectory/directory.cfm

Thank you for your interest in and support of IMS QTI.

IPR and Distribution Notices

Recipients of this document are requested to submit, with their comments, notification of any relevant patent claims or other intellectual property rights of which they may be aware that might be infringed by any implementation of the specification set forth in this document, and to provide supporting documentation.

IMS takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on IMS's procedures with respect to rights in IMS specifications can be found at the IMS Intellectual Property Rights web page: http://www.imsglobal.org/ipr/imsipr_policyFinal.pdf.

Copyright © 2006 IMS Global Learning Consortium. All Rights Reserved.

If you wish to copy or distribute this document, you must complete a valid Registered User license registration with IMS and receive an email from IMS granting the license to distribute the specification. To register, follow the instructions on the IMS website: http://www.imsglobal.org/specificationdownload.cfm.

This document may be copied and furnished to others by Registered Users who have registered on the IMS website provided that the above copyright notice and this paragraph are included on all such copies. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to IMS, except as needed for the purpose of developing IMS specifications, under the auspices of a chartered IMS project group.

Use of this specification to develop products or services is governed by the license with IMS found on the IMS website: http://www.imsglobal.org/license.html.

The limited permissions granted above are perpetual and will not be revoked by IMS or its successors or assigns.

THIS SPECIFICATION IS BEING OFFERED WITHOUT ANY WARRANTY WHATSOEVER, AND IN PARTICULAR, ANY WARRANTY OF NONINFRINGEMENT IS EXPRESSLY DISCLAIMED. ANY USE OF THIS SPECIFICATION SHALL BE MADE ENTIRELY AT THE IMPLEMENTER'S OWN RISK, AND NEITHER THE CONSORTIUM, NOR ANY OF ITS MEMBERS OR SUBMITTERS, SHALL HAVE ANY LIABILITY WHATSOEVER TO ANY IMPLEMENTER OR THIRD PARTY FOR ANY DAMAGES OF ANY NATURE WHATSOEVER, DIRECTLY OR INDIRECTLY, ARISING FROM THE USE OF THIS SPECIFICATION.

Table of Contents

1. Introduction
2. References
3. Assessment Result

1. Introduction

2. References

IMS_LIP
IMS Learner Information Package Specification, Version 1.0
http://www.imsglobal.org/profiles/index.html

3. Assessment Result

Class : assessmentResult

An Assessment Result is used to report the results of a candidate's interaction with a test and/or one or more items attempted. Information about the test is optional, in some systems it may be possible to interact with items that are not organized into a test at all. For example, items that are organized with learning resources and presented individually in a formative context.

Contains : context [1]

Contains : testResult [0..1]
When a test result is given the following item results must relate only to items that were selected for presentation as part of the corresponding test session. Furthermore, all items selected for presentation should be reported with a corresponding itemResult.

Contains : itemResult [*]
A summary report for a test is represented by an assessment result containing a testResult but no itemResults.

Class : context

Associated classes:
assessmentResult

Contains : sessionIdentifier [*]
The system that creates the result (for example, the test delivery system) should assign a session identifier that it can use to identify the session. Subsequent systems that process the result might assign their own identifier to the session which should be added to the context if the result is modified and exported for transport again.

Contains : identification [0..1]

Class : sessionIdentifier

Associated classes:
context

Attribute : sourceID [1]: uri
A unique identifier of the system which added this identifier to the result.

Attribute : identifier [1]: string
The system that creates the report should add a session identifier. Subsequent systems that process the results might use their own identifier for the session and should add this too if the result is exported again for further transport.

Class : identification

Associated classes:
context

The format of the information used to identify the candidate is defined by the [IMS_LIP].

Class : testResult

Associated classes:
assessmentResult

Attribute : identifier [1]: string
The identifier of the test for which this is a result.

Attribute : datestamp [1]: datetime
The date stamp of when this result was recorded.

Contains : itemVariable [*]
The values of the test outcomes and any durations that were tracked during the test. Note that durations are reported as built-in test-level response variables with name duration. The duration of individual test parts or sections being distinguished by prefixing them with the associated identifier as described in Assessment Test, Section and Item Information Model.

Class : itemResult

Associated classes:
assessmentResult

The result of an item session is reported with an itemResult. A report may contain multiple results for the same instance of an item representing multiple attempts, progression through an adaptive item, or even more detailed tracking. In these cases, each item result must have a different datestamp.

Attribute : identifier [1]: string
The identifier of the item for which this is a result. For item results that are reported as part of a test result this is the identifier used to refer to the item in the test (see assessmentItemRef). For item results that are reported on their own, this can be any suitable identifier for the item. Where possible, the value should match the identifier attribute on the associated assessmentItem.

Attribute : sequenceIndex [0..1]: integer
For item results that are reported as part of a test, this attribute must be used to indicate the position of the item within the specific instance of the test. The first item of the first part of the test is defined to have sequence index 1.

Attribute : datestamp [1]: datetime
The date stamp of when this result was recorded.

Attribute : sessionStatus [1]: sessionStatus
The session status is used to interpret the values of the item variables. See sessionStatus below.

Contains : itemVariable [*]
During the item session the delivery engine keeps track of the current values assigned to all itemVariables. The values of including the values of the built-in variables numAttempts, duration, and completionStatus. Each value is represented in the report by an instance of itemVariable.

Contains : candidateComment [0..1]
An optional comment supplied by the candidate (see allowComment).

Enumeration: sessionStatus

The session status is used to keep track of the status of the item variables in an item session.

initial
The value to use for sessions in the initial state, as described above. This value can only be used to describe sessions for which the response variable numAttempts is 0. The values of the variables are set according to the rules defined in the appropriate declarations (see
responseDeclaration, outcomeDeclaration and templateDeclaration).

pendingSubmission
The value to use when the item variables represent a snapshot of the current values during an attempt (in other words, while interacting or suspended). The values of the response variables represent work in progress that has not yet been submitted for response processing by the candidate. The values of the outcome variables represent the values assigned during response processing at the end of the previous attempt or, in the case of the first attempt, the default values given in the variable declarations.

pendingResponseProcessing
The value to use when the item variables represent the values of the response variables after submission but before response processing has taken place. Again, the outcomes are those assigned at the end of the previous attempt as they are awaiting response processing.

final
The value to use when the item variables represent the values at the end of an attempt after response processing has taken place. In other words, after the outcome values have been updated to reflect the values of the response variables.

Abstract class : itemVariable

Derived classes:
outcomeVariable, responseVariable, templateVariable
Associated classes:
itemResult, testResult

Attribute : identifier [1]: identifier
The purpose of an itemVariable is to report the value of the item variable with the given identifier.

Attribute : cardinality [1]: cardinality
The cardinality of the variable, taken from the corresponding declaration or definition.

Attribute : baseType [0..1]: baseType
The base type of the variable, taken from the corresponding declaration of definition. This value is omitted only for variables with record cardinality.

Class : responseVariable (itemVariable)

Attribute : choiceSequence [*]: identifier
When a response variable is bound to an interaction that supports the shuffling of choices, the sequence of choices experienced by the candidate will vary between test instances. When shuffling is in effect, the sequence of choices should be reported as a sequence of choice identifiers using this attribute.

Contains : correctResponse [0..1]
The correct response may be output as part of the report if desired.

Contains : candidateResponse [1]
The response given by the candidate.

Class : candidateResponse

Associated classes:
responseVariable

Contains : value [*]
The value(s) of the response variable. A NULL value, resulting from no response, is indicated by the absence of any value.

Class : outcomeVariable (itemVariable)

Attribute : view [*]: view
The views (if any) declared for the outcome must be copied to the report to enable systems that render the report to hide information not relevant in a specific situation. If no values are given, the outcome's value should be considered relevant in all views.

Attribute : interpretation [0..1]: string
See interpretation.

Attribute : longInterpretation [0..1]: uri
See longInterpretation.

Attribute : normalMaximum [0..1]: float
Taken from the corresponding outcomeDeclaration.

Attribute : normalMinimum [0..1]: float
Taken from the corresponding outcomeDeclaration.

Attribute : masteryValue [0..1]: float
If a mastery value is specified in the corresponding outcomeDeclaration it may be reported alongside the value of the outcomeVariable. In some cases, the mastery value may not be an attribute of the item itself, but be determined by the context in which the item is delivered, for example, by examining the candidates in a specific cohort. The mastery value may be reported with the outcome value even when there is no corresponding value in the declaration.

Contains : value [*]
The value(s) of the outcome variable.

Class : templateVariable (itemVariable)

Contains : value [*]
The value(s) of the template variable.

Class : candidateComment

Associated classes:
itemResult

The class used for comments from the candidate. A simple run of text.

About This Document

Title
IMS Question and Test Interoperability Results Reporting
Editors
Steve Lay (Cambridge Assessment), Pierre Gorissen (SURF)
Version
Public Draft v2.1 (revision 2)
Version Date
8 June 2006
Status
Public Draft (revision 2) Specification
Summary
This document describes the QTI Results Reporting specification.
Revision Information
8 June 2006
Purpose
This document has been approved by the IMS Technical Board and is made available for public review and comment.
Document Location
http://www.imsglobal.org/question/qtiv2p1pd2/imsqti_resultv2p1pd2.html

To register any comments or questions about this specification please visit: http://www.imsglobal.org/developers/ims/imsforum/categories.cfm?catid=23

List of Contributors

The following individuals contributed to the development of this document:

Name Organization
Dick Bacon
University of Surrey
Niall Barr
Question Mark
Lance Blackstone
Pearson VUE
Jeanne Ferrante
ETS
Pierre Gorissen
SURF
Regina Hoag
ETS
Gopal Krishnan
Pearson VUE
Steve Lay
Cambridge Assessment
Rowin Young
CETIS

Revision History

Version No. Release Date Comments
Base Document 2.1
14 October 2005
The first version of the QTI v2.1 specification.
Public Draft 2.1
9 January 2006
The Public Draft v2.1 of the QTI specification.
Public Draft 2.1 (revision 2)
8 June 2006
The Public Draft v2.1 (revision 2) of the QTI specification.

 

 

 

IMS Global Learning Consortium, Inc. ("IMS/GLC") is publishing the information contained in this IMS Question and Test Interoperability Results Reporting ("Specification") for purposes of scientific, experimental, and scholarly collaboration only.

IMS/GLC makes no warranty or representation regarding the accuracy or completeness of the Specification.
This material is provided on an "As Is" and "As Available" basis.

The Specification is at all times subject to change and revision without notice.

It is your sole responsibility to evaluate the usefulness, accuracy, and completeness of the Specification as it relates to you.

IMS/GLC would appreciate receiving your comments and suggestions.

Please contact IMS/GLC through our website at http://www.imsglobal.org

Please refer to Document Name:
IMS Question and Test Interoperability Results Reporting Revision: 8 June 2006