Year: 2017
Author: Jonathan, Christin, Tan, Jennifer, Koh, Elizabeth, Calen, Imelda, Tay, Sui
Type of paper: Abstract refereed
Abstract:
Learning analytics (LA) dashboards and visualizations are being increasingly used to foster 21st Century (21C) learning and literacies such as creativity, collaboration, and critical thinking - skills required in contemporary knowledge-based societies (Voogt & Roblin, 2012). However, findings on the effectiveness of LA dashboards have been inconclusive, with some suggesting that LA dashboards have not improved the quality of feedback students receive (Pardo et al., 2016) and can even be harmful for learning (Corrin & de Barba, 2014). Therefore, the purposeful design of theory-driven and empirically-informed LA dashboards is imperative for enhancing 21C learning outcomes. Drawing from Fredrickson's (2004) broaden-and-build theory of positive emotions and Martin and Elliot's (2016) recent findings emphasizing personal best goals as strong predictors of academic motivation and engagement, a LA dashboard visualizing students' latest learning data against their own data from previous weeks (i.e., self-referenced data) was developed and evaluated as part of a larger ongoing study - WiREAD.
In brief, WiREAD is a computer-supported collaborative reading and LA environment aimed at fostering students' 21C learning outcomes in English language (EL), specifically engagement, self-concept, and critical reading skills. During a 10-week trial in a Singapore Secondary school, Secondary 3 students (n=101) critically read, commented on, and replied to classmates' posts on multimodal texts. Through the self-referenced LA dashboard's dynamic and real-time visualizations, they could view their current and previous week's EL discourse and social learning network data, together with periodically updated dispositional data and school-based EL assessment data. A mixed methods research design was employed to gather quantitative and qualitative data from students pre- and post-trial using critical reading tests and self-report questionnaires, as well as open-ended feedback forms at the end of the trial.
Preliminary analysis involving t-tests revealed statistically significant improvements in (i) critical reading test performance, and self-reported (ii) cognitive engagement, (iii) academic self-concept and (iv) critical reading ability. Students' qualitative feedback highlighted the following key benefits of the self-referenced dashboard: creating greater self-awareness, encouraging learning motivation and engagement, and cultivating connective literacy. The findings also pointed to several perceived shortcomings, including students' desire for more system/expert-generated quality indicators, preference for 'live' data indicators, and an emergent understanding of the significance of dispositional factors and social learning connections. Implications of these findings for the design and use of LA dashboards and visualizations to improve student learning in the secondary schooling sector will be discussed.
In brief, WiREAD is a computer-supported collaborative reading and LA environment aimed at fostering students' 21C learning outcomes in English language (EL), specifically engagement, self-concept, and critical reading skills. During a 10-week trial in a Singapore Secondary school, Secondary 3 students (n=101) critically read, commented on, and replied to classmates' posts on multimodal texts. Through the self-referenced LA dashboard's dynamic and real-time visualizations, they could view their current and previous week's EL discourse and social learning network data, together with periodically updated dispositional data and school-based EL assessment data. A mixed methods research design was employed to gather quantitative and qualitative data from students pre- and post-trial using critical reading tests and self-report questionnaires, as well as open-ended feedback forms at the end of the trial.
Preliminary analysis involving t-tests revealed statistically significant improvements in (i) critical reading test performance, and self-reported (ii) cognitive engagement, (iii) academic self-concept and (iv) critical reading ability. Students' qualitative feedback highlighted the following key benefits of the self-referenced dashboard: creating greater self-awareness, encouraging learning motivation and engagement, and cultivating connective literacy. The findings also pointed to several perceived shortcomings, including students' desire for more system/expert-generated quality indicators, preference for 'live' data indicators, and an emergent understanding of the significance of dispositional factors and social learning connections. Implications of these findings for the design and use of LA dashboards and visualizations to improve student learning in the secondary schooling sector will be discussed.