Scaling? (1 Viewer)

[ND]

Does anyone know whether individual texts are scaled according to the performance of the cadiature in the rest of the paper(s)?

 

[Lazarus]

If a comparison of the distributions of marks between option questions reveals discrepancies, then yes, the marks will be statistically moderated to avoid them.

Further, the distribution of marks for the advanced modules paper as a whole is shaped to the distribution of marks gained by those same students on the common paper.

 

[ND]

Ah ok, so does that mean that they are not generally scaled (the individual questions on paper 2 i mean), they only do it when there is a (significant?) discrepancy?

 

[Lazarus]

That's right.

Here's an excerpt from the Masters' review (p29):

The issue here is not so much whether optional questions are equally difficult, as whether students choosing different options are treated fairly when those options are not of the same difficulty. In general, it is desirable that an attempt is made to write optional questions so that they are of approximately equal difficulty. However, this is not always possible, and when options differ in difficulty, these differences should be identified and statistical adjustments made to remove the effects of unintended differences.

The Board currently has a routine 'question scaling' process that looks for unintended differences in the difficulties of optional questions and makes appropriate adjustments. However, this process does not seem to be widely known - even by markers - and it may be useful for the Board to include the intention to adjust for differences in the difficulties of optional questions in its examination development principles.