Are there mode effects in NAPLAN 2018?

Year: 2019

Author: Thompson, Greg, Rutkowski, Leslie, Rutkowski, David

Type of paper: Abstract refereed

Abstract:
Australian school students in Years 3, 5, 7 and 9 have been sitting NAPLAN since 2008. In 2015 ACARA began trials of an online version of the test which would utilise an adaptive, rather than a linear, design. National and international large-scale assessments (LSAs) are increasingly adopting a computer platform for test administration. Prime examples include NAPLAN, the Trends in International Mathematics and Science Study (TIMSS), and the Programme for International Student Assessment (PISA). Administering LSAs in an electronic environment opens up numerous possibilitieshoweverone challenge regards the equivalence of a computer platform with more traditional pen-and-paper administrations. This online version of the NAPLAN tests was trialled in 2016 and 2017 with ACARA satisfied that the online version performed similarly to the paper version (ACARA, 2019). However, when the online trial was extended in 2018 to 20% of schools and results were included in the larger dataset, concerns were raised regarding the mode equivalence (ie whether the mode in which the test was administered impacted results). The concern regarding mode equivalence has led the Victorian Education Minister to suggest that the results were not comparable and to advise parents, educators and the media that caution should be exercised when interpreting student and school results in NAPLAN 2018 (McGowan, 2019). In this presentationwe investigate the assumption of measurement equivalence in the computer-based version of NAPLAN.



References

ACARA. (2019). 2018 NAPLAN Results. NAP: National Assessment Program. 29thAugust, 2019. https://www.theguardian.com/australia-news/2018/aug/09/act-concerns-about-online-naplan-tests-rejected-by-minister-months-ago [https://www.theguardian.com/australia-news/2018/aug/09/act-concerns-about-online-naplan-tests-rejected-by-minister-months-ago]

McGowan, M. (2019). “Education chiefs have botched NAPLAN online test, says Victorian minister”. The Guardian.9thAugust, 2019. https://www.theguardian.com/australia-news/2018/aug/09/act-concerns-about-online-naplan-tests-rejected-by-minister-months-ago [https://www.theguardian.com/australia-news/2018/aug/09/act-concerns-about-online-naplan-tests-rejected-by-minister-months-ago]

Back