Messages from large-scale assessment studies

Year: 2013

Author: Ainley, John, Gebhardt, Eveline

Type of paper: Abstract refereed

Over the past two decades there has been a number of large-scale assessment surveys conducted in Australia. Some of these have been part of international studies of achievement in reading mathematics and science: the IEA Progress in International Reading Literacy Study (PIRLS) of Year 4 students in 2010, the Programme for International Student Assessment (PISA) of 15-year-olds every three years since 2000, and the IEA Trends in International Mathematics and Science Study (TIMSS) every four years among students in Year 4 and Year 8 since 1994. Other large-scale assessment surveys have been conducted as part of the National Assessment Program (NAP) in the form of the annual assessment of literacy and numeracy (NAPLAN) for Years 3, 5, 7, and 9 since 2008 and a series of triennial sample studies in science (since 2003), civics and citizenship (since 2004), and information and communication technology literacy (since 2005). All of these surveys use item response theory as a basis for the psychometric analyses of assessment data, all except NAPLAN use rotated block designs, and all equate measures over time so that change can be estimated in an absolute sense. Reports from these studies have provided a plethora of information about the achievements of students in Australian schools and how those achievements differ among jurisdictions and among groups of students. This report is based on data that have been published in reports of the studies. Standard analytical procedures have been applied to those published data following the principles of meta-analysis. These procedures provide the basis for comparing results from different studies and for determining whether differences are statistically significant. It examines differences in achievement among groups of students in Australia, comparisons of the achievements of students in Australia with their peers in other countries and changes in achievement over time for Australian students overall and for groups of students. It is predicated on the assumption that perspectives on the impact of policies and practices on student outcomes can be informed by evidence about the ways in which achievement co-varies with differences in policy and practice and about the extent to which achievement changes over time. It notes that there has been a small decline in reading and mathematics achievement among students in the middle years of secondary school since 2000, stability in science and mathematics achievement among Year 8 students since 1994, a small improvement in mathematics achievement among students in Year 4 since 1994 and a small improvement in reading achievement among students in Year 3 since 2008.