Citolab.QTI.ScoringEngine
1.6.0
dotnet add package Citolab.QTI.ScoringEngine --version 1.6.0
NuGet\Install-Package Citolab.QTI.ScoringEngine -Version 1.6.0
<PackageReference Include="Citolab.QTI.ScoringEngine" Version="1.6.0" />
<PackageVersion Include="Citolab.QTI.ScoringEngine" Version="1.6.0" />
<PackageReference Include="Citolab.QTI.ScoringEngine" />
paket add Citolab.QTI.ScoringEngine --version 1.6.0
#r "nuget: Citolab.QTI.ScoringEngine, 1.6.0"
#:package Citolab.QTI.ScoringEngine@1.6.0
#addin nuget:?package=Citolab.QTI.ScoringEngine&version=1.6.0
#tool nuget:?package=Citolab.QTI.ScoringEngine&version=1.6.0
Citolab QTI Scoring Engine
Scores QTI assessment results: it runs the response processing of your items and the outcome processing of your test, and writes the outcome variables into the assessment results it returns.
- Reads QTI 2.x and 3.0 items and tests, and QTI Results Reporting 2.x assessment results.
- Works on
XDocuments, so nothing is read from or written to disk. - Not every expression of the specification is implemented, but everything in 3.2 Simple Items is scored correctly. What is supported lists the rest.
Contents
- Installation
- Quick start
- Usage
- What is supported
- Records and QTI_CONTEXT
- Custom operators
- Errors and logging
- Processing a package
Installation
dotnet add package Citolab.QTI.ScoringEngine
The package targets net9.0 and netstandard2.0.
Quick start
using Citolab.QTI.ScoringEngine;
var scoringEngine = new ScoringEngine();
var scoredAssessmentResults = scoringEngine.ProcessResponsesAndOutcomes(new ScoringContext
{
AssessmentItems = assessmentItemXDocs, // List<XDocument>
AssessmentTest = assessmentTestXDoc, // XDocument
AssessmentmentResults = assessmentResultXDocs, // List<XDocument>
Logger = logger // optional
});
scoredAssessmentResults holds the assessment results with the calculated outcome variables.
Usage
IScoringEngine has three methods:
public interface IScoringEngine
{
List<XDocument> ProcessResponses(IResponseProcessingContext ctx, ResponseProcessingScoringsOptions options = null);
List<XDocument> ProcessOutcomes(IOutcomeProcessingContext ctx);
List<XDocument> ProcessResponsesAndOutcomes(IScoringContext ctx, ResponseProcessingScoringsOptions options = null);
}
| method | runs | needs |
|---|---|---|
ProcessResponses |
the response processing of every item | AssessmentItems |
ProcessOutcomes |
the outcome processing of the test | AssessmentTest |
ProcessResponsesAndOutcomes |
response processing first, then outcome processing | both |
The context
Pass one of ScoringContext, ResponseProcessingContext or OutcomeProcessingContext -
or your own implementation of the matching interface.
| property | used by | description |
|---|---|---|
List<XDocument> AssessmentmentResults |
both | The assessment results to score. For response processing an itemResult should hold the candidateResponse of the item. |
List<XDocument> AssessmentItems |
responses | The assessment items, in QTI 2.x or 3.0. |
XDocument AssessmentTest |
outcomes | The assessment test, in QTI 2.x or 3.0. |
ILogger Logger |
both | Optional. Logs every processing step, and the errors that do not throw. Nothing is logged when it is left out. |
Dictionary<string, ICustomOperator> CustomOperators |
responses | Optional. See Custom operators. |
bool? ProcessParallel |
both | Optional. Scores the results in parallel, which can make a real difference when scoring a lot of results at once. |
Options
Options are passed as the argument of the method, not through the context:
scoringEngine.ProcessResponses(ctx, new ResponseProcessingScoringsOptions
{
StripAlphanumericsFromNumericResponses = true,
QtiContextFields = new Dictionary<string, string> { { "environmentIdentifier", "ENV_1" } }
});
| option | description |
|---|---|
StripAlphanumericsFromNumericResponses |
A best effort to compare a response like "12 euro" as the number 12: everything that is not a digit or the first dot is dropped before an integer or float is compared. |
QtiContextFields |
Fields for the QTI_CONTEXT record. See Records and QTI_CONTEXT. |
What the engine changes
The engine scores copies: the XDocuments you pass in are not modified. Use the list that
is returned - the AssessmentmentResults of your context is replaced by that same list.
Per item result:
- Outcomes that the response processing writes (
qti-set-outcome-value,qti-lookup-outcome-value) are reset to the default of their declaration -0for a number without one - and recalculated. Not for an adaptive item: see Adaptive items. - Outcomes that the response processing does not write are left exactly as they are, so an
outcome like
numAttemptsor a human scored outcome survives a rescore. An outcome variable markedexternal-scored="human"is never overwritten. - An item that has no
itemResultis skipped; an item result that has no candidate response still gets the outcome variables of the response processing, with their default value.
Per test result:
- A
testResultis added when the result does not have one yet. - The outcomes that the outcome processing writes are reset to 0 and recalculated.
Adaptive items
An item with adaptive="true" is answered in several attempts, and its outcomes keep their values from one
attempt to the next: the response processing of an attempt starts from the outcomes the previous attempt left. That
is how an adaptive item shows a follow-up question, gives a hint or keeps the score of a step it already scored. The
item is done when its response processing sets the built-in outcome completionStatus to completed.
Score such an item attempt by attempt, and pass the state along: the itemResult of an attempt holds the
outcomeVariables that the previous attempt returned, next to the candidateResponses of the new attempt.
// attempt 1: only responses; every outcome starts at its default
var scored = scoringEngine.ProcessResponses(new ResponseProcessingContext
{
AssessmentItems = [item],
AssessmentmentResults = [resultWithAttempt1Responses]
}).Single();
// attempt 2: the itemResult of `scored`, with its responseVariables replaced by those of attempt 2
var scoredAgain = scoringEngine.ProcessResponses(new ResponseProcessingContext
{
AssessmentItems = [item],
AssessmentmentResults = [scoredWithAttempt2Responses]
}).Single();
For an adaptive item the engine:
- does not reset the outcomes that the response processing writes. An outcome the
itemResultdoes not hold yet - on the first attempt, none - starts at its default; - knows
completionStatusas an identifier outcome (defaultunknown) without a declaration in the item; - does not count attempts: when the response processing reads
numAttempts, pass it as a response variable.
Scoring all responses once at the end is not the same: the response processing of an adaptive item depends on the order of its attempts. A delivery system that scores on the server scores each attempt when it ends.
What is supported
The lists below use the QTI 3.0 element names. QTI 2.x items and tests are converted to the 3.0
spelling while they are read, so a 2.x <responseCondition> is the qti-response-condition
below, and a 2.x attribute baseType is base-type. Assessment results keep their 2.x
Results Reporting spelling.
Cardinality
single, multiple, ordered and record (see Records and QTI_CONTEXT).
Base types
| base type | how it is handled |
|---|---|
identifier, string |
compared as text, case-sensitive unless the expression says otherwise |
integer, float |
compared as numbers |
boolean |
compared as booleans |
pair, directedPair |
compared as two identifiers; the order within a pair does not matter |
point |
scored with qti-map-response-point and qti-inside; two points are not compared directly |
duration |
a number of seconds, compared as a number (qti-duration-lt, qti-duration-gte) |
uri, intOrIdentifier |
compared as plain text, with no further interpretation |
file |
not supported |
Response processing templates
An item that refers to one of the standard templates is scored by that template, so it does not need response processing of its own:
match_correctmap_responsemap_response_point
Rules
Both kinds of processing:
qti-set-outcome-valueqti-lookup-outcome-value
Response processing only:
qti-response-condition,qti-response-if,qti-response-else-if,qti-response-elseqti-exit-response- ends the response processing of the attempt; the outcomes that were already set are keptqti-response-processing-fragment- processed in place. Fragments in a separate file are not pulled in, becauseqti-includeis not resolved.
Outcome processing only:
qti-outcome-condition,qti-outcome-if,qti-outcome-else-if,qti-outcome-else
Not supported: qti-include.
Expressions
Supported:
qti-and |
qti-index |
qti-not |
qti-any-n |
qti-inside |
qti-null |
qti-base-value |
qti-integer-divide |
qti-or |
qti-container-size |
qti-integer-modulus |
qti-ordered |
qti-contains |
qti-integer-to-float |
qti-pattern-match |
qti-correct |
qti-is-null |
qti-power |
qti-custom-operator |
qti-lcm |
qti-product |
qti-default |
qti-lt |
qti-random |
qti-delete |
qti-lte |
qti-repeat |
qti-divide |
qti-map-response |
qti-round |
qti-duration-gte |
qti-map-response-point |
qti-round-to |
qti-duration-lt |
qti-match |
qti-stats-operator |
qti-equal * |
qti-math-constant |
qti-string-match |
qti-equal-rounded |
qti-math-operator |
qti-substring |
qti-field-value |
qti-max |
qti-subtract |
qti-gcd |
qti-member |
qti-sum |
qti-gt |
qti-min |
qti-truncate |
qti-gte |
qti-multiple |
qti-variable |
* qti-equal only supports tolerance-mode="exact".
Outcome processing only: qti-test-variables, qti-number-selected.
Not supported:
qti-number-correct- an item is left untouched rather than scored 0: what the delivery engine calculated stays in the result.qti-number-incorrect,qti-number-presented,qti-number-respondedqti-outcome-maximum,qti-outcome-minimumqti-random-float,qti-random-integer
Lookup tables
qti-lookup-outcome-value reads the table of the outcome declaration it writes to:
qti-interpolation-tablewithqti-interpolation-table-entry- matched on an exactsource-value, not on a range.qti-match-tablewithqti-match-table-entry.
How the operators behave
- The operators that take numbers (
qti-product,qti-divide,qti-power,qti-math-operator, ...) are NULL when a child is NULL, is not a number, or when there is no finite result - dividing by zero, the square root of a negative number.qti-sumandqti-subtractpredate this and count a child that is not a number as 0. qti-integer-dividerounds down:integerDivide(7, 3) = 2, and the remainder ofqti-integer-modulusgoes with it:integerModulus(7, 3) = 1.qti-round-totakesrounding-mode="decimalPlaces"or"significantFigures";significantFiguresis the default, as it is forqti-equal-rounded.qti-math-operatorsupportssin,cos,tan,secant,cosecant,cotangent,asin,acos,atan,atan2,sinh,cosh,tanh,exp,log(base 10),ln,sqrt,abs,floor,ceil,signum,toDegreesandtoRadians. Angles are in radians.qti-stats-operatorsupportsmean,median,popVariance,popSD,sampleVarianceandsampleSD.qti-pattern-matchmatches the pattern against the whole value, the way XML Schema patterns work, and gives up on a pattern that runs for more than a second.qti-repeatevaluates its children at most 1000 times, so anumber-repeatsthat is read from a variable cannot build an endless container.
Records and QTI_CONTEXT
A record is a set of named fields, each with its own base type. qti-field-value reads one
field of a record:
<qti-field-value field-identifier="environmentIdentifier">
<qti-variable identifier="QTI_CONTEXT"/>
</qti-field-value>
A record can come from:
QTI_CONTEXT, the built-in record below.- A response or outcome variable of the assessment result declared
cardinality="record". Its values carry a field identifier and their own base type:<value fieldIdentifier="floatValue" baseType="float">1.5</value>. - A custom operator that returns a
BaseValuewithCardinality.Recordand itsFieldsset. - The
qti-default-valueof a record declaration, read withqti-default.
qti-set-outcome-value can write a record to an outcome of record cardinality: it is added to
the item result as one value element per field.
qti-field-value is NULL when the record has no such field, so qti-is-null is how to check
whether a field is there.
QTI_CONTEXT
QTI_CONTEXT can be read with qti-variable in response and outcome processing. The engine
fills what the assessment result knows:
| field | read from |
|---|---|
candidateIdentifier |
the sourcedId attribute of the context element |
testIdentifier |
the testResult of the result, or the test being processed |
environmentIdentifier |
the environmentIdentifier attribute of the context element |
Everything else is only known to the delivery engine, so fields can be passed in - and the ones above overridden - through the options:
scoringEngine.ProcessResponses(ctx, new ResponseProcessingScoringsOptions
{
QtiContextFields = new Dictionary<string, string>
{
{ "environmentIdentifier", "ENV_1" }
}
});
A field without a value is left out of the record instead of being set to an empty string.
Custom operators
Custom operators are usually specific to a delivery engine, so they are passed to the scoring
engine. The key of the dictionary is the definition attribute of the element:
<qti-custom-operator definition="depcp:Trim">
<qti-variable identifier="RESPONSE"/>
</qti-custom-operator>
public class Trim : ICustomOperator
{
public BaseValue Apply(List<BaseValue> values)
{
var value = values.FirstOrDefault();
if (value?.Value != null)
{
value.Value = value.Value.Trim();
}
return value;
}
}
var ctx = new ResponseProcessingContext
{
// ...
CustomOperators = new Dictionary<string, ICustomOperator>
{
{ "depcp:Trim", new Trim() }
}
};
An operator is handed the values of its child expressions and returns the value of the expression - including, if that is what it computes, a record.
Three operators are built in, each under the prefixes depcp:, questify: and qade::
| operator | does |
|---|---|
Trim |
trims the value |
ToAscii |
replaces diacritics |
ParseCommaDecimal |
replaces a decimal comma with a dot |
An operator you pass in overrides a built-in one with the same key.
Errors and logging
ScoringEngineException
Thrown when the engine is called with nothing to work with:
- a context that is null,
- response processing without
AssessmentItems, - outcome processing without an
AssessmentTest.
Other exceptions
Items, tests and results are indexed by identifier for fast lookups, so a duplicate identifier throws:
- the same item reference twice in a test,
- more than one
itemResultfor the same item in an assessment result, - more than one outcome or response declaration with the same identifier in an item.
Two assessment items with the same identifier are an exception to this: that is logged as a warning and the first one is used.
Logging
Every step is logged as information, which makes it possible to follow how a score came about. Problems that do not stop the scoring are logged as a warning or an error, and usually end in a score of 0 or in a NULL value. For example:
- a value that does not fit its base type, such as a float of
'hello', - a lookup that finds nothing in an interpolation table or a mapping,
- an expression or a rule the engine does not know.
Processing a package
Console.ScoringEngine is a worked example that scores the assessment results of a QTI
package: it reads the imsmanifest.xml of the package, hands the items, the test and the
results to the engine, and writes the scored results to a processed folder next to them.
dotnet run Console.ScoringEngine c:/mypackage.zip c:/assessmentResults
The first argument is the package, the second the folder holding the assessment results. Both
can also be set as AppSettings:PackageLocation and AppSettings:AssessmentResultFolder in
appsettings.json.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 is compatible. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net10.0 was computed. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
| .NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
| .NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
| .NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
| MonoAndroid | monoandroid was computed. |
| MonoMac | monomac was computed. |
| MonoTouch | monotouch was computed. |
| Tizen | tizen40 was computed. tizen60 was computed. |
| Xamarin.iOS | xamarinios was computed. |
| Xamarin.Mac | xamarinmac was computed. |
| Xamarin.TVOS | xamarintvos was computed. |
| Xamarin.WatchOS | xamarinwatchos was computed. |
-
.NETStandard 2.0
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.7)
-
net9.0
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.7)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 1.6.0 | 476 | 10/15/2026 |
| 1.5.0 | 112 | 10/14/2026 |
| 1.4.1 | 224 | 10/9/2026 |
| 1.4.0 | 346 | 9/17/2026 |
| 1.3.7 | 128 | 9/17/2026 |
| 1.3.6 | 385 | 6/16/2026 |
| 1.3.5 | 656 | 5/11/2026 |
| 1.3.4 | 168 | 5/7/2026 |
| 1.3.3 | 800 | 10/29/2025 |
| 1.3.2 | 297 | 7/29/2025 |
| 1.3.1 | 1,271 | 4/18/2024 |
| 1.3.0 | 885 | 3/5/2024 |
| 1.3.0-beta2 | 439 | 2/9/2024 |
| 1.3.0-beta1 | 213 | 2/9/2024 |
| 1.2.5 | 580 | 1/24/2024 |
| 1.2.4 | 289 | 1/23/2024 |
| 1.2.3 | 300 | 1/18/2024 |
| 1.2.2 | 747 | 9/14/2023 |
| 1.2.1 | 278 | 9/8/2023 |
| 1.2.0 | 334 | 8/3/2023 |