Establishing the validity of a performance assessment in numeracy

Year: 2003

Author: Callingham, Rosemary

Type of paper: Abstract refereed

Validity is the extent to which the inferences drawn from scores on a test or assessment can be justified empirically and theoretically. Establishing the validity of an assessment can be seen as an ongoing process of judgment, using different forms of evidence for substantiation. Traditionally validity has been defined within an evidential framework of three interrelated aspects: criterion, content and construct validity. Messick's (1989) integrated view of validity led to inferences drawn from the information gathered from the assessment being pivotal to establishing validity and extended the understanding of validity to the consequences of test or assessment use, and led to a consideration of construct validity as the over-arching evidential basis. Performance assessment, in which students provide some form of product or performance, typically takes place under less standardised conditions than traditional test forms. It has been suggested that validity standards should be different for this form of assessment, relating more directly to the specific performance. Students' observed performances on a complex performance task in a numeracy context, as well as on tests of mathematics and mathematical problem solving, were analysed using Rasch modelling techniques. Six criteria suggested by Messick for construct validity evidence, were applied to the data. Findings indicated that the performance assessment information validity was high, and provided data useful for multiple purposes. The implications of these findings for performance assessment are discussed.