Using Rasch analysis to investigate item functioning in a national mathematics assessment in South Africa

Year: 2019

Author: Bansilal, Sarah

Type of paper: Abstract refereed

Abstract:
The Annual National Assessments (ANA) is an intervention of the South African education authorities that was aimed at improving the quality of education by administering a nationally designed assessment to all school learners in selected grades. The study reported in this paper, focusing on the Grade 9 mathematics ANA was carried out with seven schools. Data were generated from the written responses of 1187 learners to the grade 9 mathematics ANA. The purpose of this study was to investigate the functioning the items by means of a Rasch analysis.

This paper illustrates how Rasch analysis can be used to improve the functioning of items which make up an assessment. Rasch analysis is the process of examining the extent to which the responses from a scale approach the pattern required to construct measurement. In Rasch measurement theory, the student proficiency and item difficulty are located on a common interval scale. When a test adheres to the requirements for measurement- like interpretations, then they allow for inferences related to comparisons of item and person proficiency locations. Rasch analysis is the process of examining the extent to which the responses from a scale approach the pattern required to satisfy axioms of measurement in order to construct measurement. When some of the measurement criteria are not satisfied, the identification of the anomalies can contribute to a deeper understanding of the construct under scrutiny.

In this paper I consider aspects such as overall fit and item misfit. The ordering of the items is used to identify interesting trends such as the fact that the geometry items were over-represented at the higher difficulty levels, and under-represented at the lower difficulty levels. The most misfitting items are discussed. Possible reasons that can account for the misfit could be guessing, item dependency, differential item functioning as well as scoring rubrics that did not work as intended. It is hoped that some of the insights that are gained can be used to contribute to an improved design of future assessments.

Back