Using a computerised adaptive assessment tool for formative purposes: The determining factors

In Australia, like other parts of the world, the debate on how to enhance teacher use of assessment data to inform learning and teaching of English as a second or additional language (ESL/EAL) has been the subject of much academic scholarship. Teachers are expected to design and implement a range of assessments to drive learning and teaching. However, the central problem in EAL teaching is the variability of classroom assessment practices, resulting from diverse teacher assessment knowledge, skill and beliefs as well as external constraints. To address this concern, scholars in second language teaching have proposed the complementary use of a large-scale computer-based (online) formative assessment to meet the needs of teachers and providing more standardised on-demand assessment tools for areas of learning which are more difficult to assess, for example, reading and vocabulary development. Such a complementary form of assessment can be constructed in such a way as to promote assessment for learning (AfL) in the contemporary school system.

This paper reports the preliminary findings of the evaluation of an externally provided computer-adaptive assessment tool, based on AfL principles, a key component of a large-scale commissioned project called “Tools to Enhance Assessment Literacy for Teachers of English as an additional language (TEAL)”, for all Victorian schools. Specifically, the TEAL project includes an on-demand reading and vocabulary assessment tool for English as an Additional Language, (called RVEAL), which is intended to enhance the teacher-based assessment of the reading and vocabulary development of EAL students.

The paper explores teachers’ beliefs about assessment and teachers’ knowledge and skills as influencing factors in the teacher evaluation of RVEAL tool. The participants in the study comprised K-12 EAL specialist teachers in government schools, selected through a purposeful sampling technique. Data generated through semi-structured interviews (via zoom) was subjected to content analysis. The findings from the resulting data analysis provide useful insights into teachers’ beliefs about AfL, and factors contributing to the practicality, usefulness and trustworthiness of the assessment tool. Drawing on theoretical explanations of classroom assessment practices, the findings have been used to develop a Teacher Assessment Tool Evaluation Questionnaire (TATE-Q) for further empirical investigation of the impact of the instrument on teacher assessment practices and language teaching and learning. This paper concludes that engaging teachers is essential to evaluate assessment instruments and presents implications for assessment design, teacher assessment practices and language education.

Keywords: Adaptive testing, AfL, evaluation, formative assessment, validation