The purpose of the presentation is to provide an overview of the assessment of one 21st Century Skill (i.e., adaptive problem solving) using technology. The technology focus will be on computer simulations and games. There is evidence that computer simulations and games teach people effectively. Twenty-first Century Skills are the mental preparation an individual needs to establish and sustain competent performance in a complex and unpredictable environment.
We view 21st Century Skills through a knowledge, skills, and attributes (KSA) lens, i.e., knowledge is domain-specific. In our model, the five skill constructs are either domain-specific or domain-independent. The skills are adaptability, adaptive problem-solving, communication, decision-making, and situation awareness. There are four competencies or attributes, which are relatively domain-independent. These attributes are adaptive expertise, creative thinking, self-regulation, and teamwork.
The presentation will discuss various assessment techniques based on a synthesis of Mayer's definition of adaptive problem solving (“Adaptive problem solving involves the ability to invent solutions to problems that the problem solver has not encountered before. In adaptive problem solving, problem solvers must adapt their existing knowledge to fit the requirements of a novel problem.”) and O'Neil's (“O'Neil views problem solving as being composed of content understanding, problem solving strategies, and self-regulation.”).
Examples of assessments for conceptual and strategic problem solving knowledge via knowledge maps will be discussed. Problem solving domain specific strategies will be assessed via essay responses to a retention prompt, e.g., given a decision making simulation scenario list the domain specific strategy you used to solve the problem. An example of transfer prompt would be to list how many change strategies are possible if the given's in a decision making problem change. An example of an assessment strategy for self-regulation would be to use a questionnaire.
Our base approach to construct a validity argument is to design computer scenarios that would require adaptive problem solving, e.g., these characteristics would be: high cognitive complexity; multiple competing priorities; unpredictable environments; no “best” solution; many potentially “disastrous” solutions, trained procedures/principles may not work, creative solutions may be needed. Finally, an example of such an adaptive problem solving assessment using a computer scenario based on these characteristics will be provided.