Hecs lotto: Does marker variability make examinations a lottery?

Year: 1999

Author: Barrett, Steven

Type of paper: Refereed paper

Focus groups that have been conduced with undergraduate students of the Division of Business and Management at the University of South Australia revealed general concerns about marker variability and the possible impact on examination results and student performance. This study has two aims. First, to analyse the relationships between student performance on an essay style examination, the questions answered and the markers. Second, to identify and determine the nature and the extent of the marking errors on the examination.

These relationships were analysed using two commercially available software packages, RUMM and Conquest to develop Rasch Test Models. The analyses revealed minor differences in item difficulty, but considerable inter-rater variability. Furthermore, intra-rater variability was even more pronounced. Four of the five common marking errors were also identified.