Development and Validation of the Construct of Problem Solving

Year: 2016

Author: Rahman, Shafqat, Griffin, Patrick, Zhang, Zhonghuua

Type of paper: Refereed paper

This paper explores the concept of problem solving based on game like problems not requiring prior knowledge. A set of indicators of student performance generated by Zoanetti (2009) are calibrated using a psychometric model (IRT). The relationships between the indicators are then explored to determine whether a conceptual model of problem solving reported by Griffin (2001) can be used to define a construct represented by a developmental progression derived from an item response analysis of indicators in log stream data (Griffin, Mak & Wu, 2006). Latent trait analysis approach is used to calibrate the indicators and to determine properties of the construct in terms of reliability and validity of derived quality measures. The more detailed Callingham (2004) developmental framework is used to examine whether the hierarchy in the cognitive processes defined by Zoanetti’s indicators describe a domain of general problem solving using Zoanetti’s (2009) dynamic and interactive game based tasks to capture process data similar to Funke’s (2010) MicroDYN approach of assessment design.The literature does not generally document an endeavour to identify problem solving as a set of cognitive processes which move from inductive to deductive reasoning. In the Assessment and teaching of 21st century skills (ATC21’s project), Hesse, et al. (2015) and Griffin (2001, 2014) described the link to hypothetico deductive reasoning. This research attempts to fill a gap by defining this construct in the domain of individual, interactive problem solving. It defines a learning continuum of cognitive processes which can be used to assist teachers to determine the students’ zone of proximal development (ZPD) to measure progression, and opportunities for effective intervention. The development of the progression uses the Assessment Research Centre (ARC) (Griffin, Mak, & Wu, 2006) data and undertakes a secondary data analysis. The analysis confirmed the hypothesis of the study. This study establishes a methodology to develop and validate interactive problem solving progression. It further validates that this hierarchical model best fits the indicators extracted from the log stream file with in psychometric model. Two scales creative problem solving (CPS) and interactive problem solving (IPS) were found moderately correlated (r =0.51), establishes an external validity criteria. This scale could be used as a valuable formative assessment instrument and therefore, has practical implication for teaching and learning. Furthermore, this approach has the capacity to automate the assessment process to be able to make detailed inferences about student problem solving behavior.