ARNOL97.049 Problem-solving and mathematical processing
Evaluating contemporary instructional technologies :
Effects on problem-solving and mathematical processing
by
Lynette Arnold
based on research undertaken as part of
Master of Arts (Education) Research Thesis
Flinders University of South Australia
presented at the
Brisbane AARE Annual Conference
'Researching Education in New Times'
Brisbane, Australia
December 1st, 1997
Email: larnold@decspop.nexus.edu.au
INTRODUCTION
A new genre of resource and reference material is appearing on the
commercial market aimed at supporting and complementing educational
outcomes for students in the guise of interactive multimedia software.
Over the past 3-5 years, schools in Australia have been investing large
amounts of money and resources into the purchase of computers - whether
they be stand-alones, networked : Macintosh or PC based. Subsequent to
purchasing the hardware, educators are faced with the selection and
purchase of relevant, useful software which supports the educational
outcomes of the school and its community.
Interactive Multimedia, for the purposes of this paper is defined as
educational technologies controlled via a computer, and constitutes
one of the options being promoted as technologies which can -
* promote or support teaching and learning; and
* provide computer assisted instruction (CAI) or computer assisted
learning (CAL)
Various claims have been made, particularly by developers and
designers, about the effects Interactive Multimedia (IMM) will have on
learning. A common factor between each claim is that they each imply a
direct link between multimedia, or interactive multimedia, and an
automatic guarantee of changes to the way humans learn and activate
their thought processes. It was this type of claim that I was
interested in investigating.
Whilst I do not deny the fact that IMM may have an effect on student
disposition and performance in learning situations - the claims I refer
to are often unsubstantiated, at this stage, by any quantitative or
qualitative research and more often than not are designed as part of a
full-scale marketing strategy by developers and designers.
For example, Preece, J and Davies G (1992) claim that IMM 'promises to
enhance learning'. IMM may enhance a student's disposition toward a
task however will IMM also enhance their performance? If the design of
the IMM software does not support student-centred and
student-controlled learning it may not, in fact, enhance active,
constructive or cumulative learning, all aspects of learning which are
highlighted in contemporary pedagogy.
As for 'reducing the learning curve and accelerating
learning'(Reynolds, L & Ehrlich D 1992) there is no evidence to support
this statement nor can the proposal that 'higher levels of analytical
skills will be developed at earlier stages of education' (Broadband
Servcies Expert Group 1995) be supported by current research findings.
Regarding the claim that IMM will 'change the way we think and solve
problems' (Ambron, S & Hooper, K 1988) I believe that it may change the
types and nature of the problems a learner must solve but not,
necessarily, the way he/she solves the problem.
Questions spring to mind - such as;
Why will computer controlled educational technologies bring about such
dramatic effects when previous educational technologies - ie.
television, radio, film, video, overhead projectors and teaching
machines - have not already had this effect within learning
environments?
How will IMM have a significant effect on the way students solve
problems, perceive educational concepts and construct knowledge?
What can we do as educators to ensure that these technologies are used
effectively in teaching and learning environments?
There has been a long standing debate amongst researchers and educators
as to whether or not media does, in fact, affect learning - let alone
to what degree! Clarke (1994) states that Robert Kozma agrees with him
in 'that evidence does not yet support the claim that media or media
attributes influence learning' however Robert Kozma (1994) in his
article goes on to claim 'if there is no relationship between media and
learning it may be because we have not yet made one' and Salomon et al
(1979) believe that it is 'not the medium which influenced learning but
instead certain attibutes of media that can be modelled by learners and
can shape the development of unique "cognitive processes"...'
This led me to the next question I addressed - What are we trying to effect?
Basically it is - what the student learns. By abstracting a range of
literature based on a general constructivist view I define knowledge
construction as a dynamic, problem-solving decision-making process
which is cumulative, goal-oriented, knowledge-dependent and
context-based in nature and involves the learner actively and
constructively. (Anderson, J.R. 1982, 1983; Rumelhart & Norman 1981;
Glaser, R 1984; Resnick 1989; Chi, Glaser & Rees 1982; Shuell, T.J.
1988) Further knowledge construction does not occur in isolation from
environmental or internal factors.
INTERACTIVITY
When beginning my research I found I needed to clarify the term
"interactivity or interaction" and what this meant in relation to
multimedia and education or learning and look for the similarities and
differences.
When computer programmers and technologists refer to interactivity in
relation to multimedia they generally are discussing it in terms of the
dynamic which occurs between the hardware, software and the potential
user - although the interface is often the beginning and end of
interactivity in these terms.
When discussing interactivity in educational contexts there is a vast
body of research which investigates parent, teacher, student and peer
group interactions and their effect on teaching and learning in a range
of contexts.
A lot of time, money and resources are invested by multimedia
developers in creating the interaction between the computer and the
software but what educational programs need to do, to a high degree, is
interface effectively with a wide range of learners and address their
individual needs.
Each person has a different cognitive model and what the software
package to do is respond equally successfully with each and every user
based on their needs and prior knowledge using appropriate language.
Assuming that interaction occurs in a range of learning contexts I
investigated exactly what interaction looks like and means in
successful teaching and learning environments.
Based on my literature review I conclude that interaction which
promotes quality learning is reciprocal or bi-directional,
personalised, responsive to learner needs, uses appropriate language,
is contextually sound, states explicit goals, and provides a balance
between guidance and learner exploration.
There is research being undertaken in various parts of the world which
explore the ways in which children and young adults interact and
respond to IMM. This research is based on a wide range of hypotheses
and methodologies. For example, researchers at the Queensland
University of Technology are currently investigating the educational
significance of the cognitive processes teenagers engage in while
playing video games and the possiblities for the transfer of knowledge
between a games platform and an educational multimedia system. Such
research findings should prove informative and helpful in understanding
the role of interaction in educational multimedia environments.
What I was particularly interested in was the cognitive interaction
which a student experienced in solving a problem when working within an
interactive multimedia environment and the effect this had on knowledge
construction and how it differed from the cognitive interaction
experienced and knowledge construction of students working with a
traditional medium.
Further I was interested in investigating the claims that multimedia
environments encourage more interaction between students when working
collaboratively and whether or not this interaction was of a quality
nature and related to the problem task or problems created by the
media.
PROGRAM
As I was also interested in investigating some of the claims made about
IMM and specifically whether an IMM product would effect a student's
cognitive processing and performance I selected a program called
"Working Mathematically : Space" which
* is complex, both in design and the content it deals with
* goes beyond drill and practice
* has a stand-alone, student directed, instructional intention
The package has been designed for middle primary through to junior
secondary students (i.e. Years 5-9) and is recommended for use with
other mathematics resources to enrich current practice.
It provides 134 investigations presented in three levels of difficulty
in addition to the opportunity to experiment in free-form mode. The
work environment provides a 3D Constructor and a 2D Builder.
"They (students) will be able to do things that are impossible to do on
paper or with physical materials.....The designers of this truly
interactive package have created work environments in which students
can explore and record spatially-based mathematical ideas. Students
will be able to:
* investigate, discuss and describe mathematical situations
* visualise and be creative
* conjecture and test
* contemplate, reconsider and extend their understanding of mathematical ideas
* generalise
* become, over time, more sophisticated in their use of mathematical
language and their ability to apply mathematics in situations of
interest or importance." Olssen, K and Walsh S. Working Mathematically
CD-ROM : Space Support Book (1996) p. 5
METHOD
All groups undertook exactly the same problem-solving task and
post-construction rotational and symmetry exercises using an identical,
colour coded tower.
The problem-solving task involved constructing a tower which was
symmetrical, four storeys high, had only one cube in the top storey,
and consisted of exactly 44 cubes using either the 3D Constructor of
the 'Working Mathematically CD-ROM : Space' program or 1cm construction
blocks.
Even though the software package offered alternative environments the
students used only the 3D Constructor for the task and needed to use
only the viewing positions, add and delete cubes, and cube counter
options to successfully complete the problem-solving task. The
orthographic view, magnification and rotational tools were additional
options which students could access if they chose.
Hence I utilised only a particular aspect of the package and
subsequently my results are limited to this aspect.
The tower building task came from the middle level of the
transformations section of the program, however I presented the task in
print form to all groups and did not utilise the theatrette
presentation of the task to ensure consistency for all groups. I also
provided various support materials not provided by the IMM package to
assist students in the completion of the task. These materials, which
were provided to all students regardless of the medium, included an
outline of the task specification, an isometric diagram showing how to
build the first two storeys and an orthographic view of the tower from
the front. I also had a sample of the tower built in blocks however
this was only presented to the students if they were unable to complete
the task within the 20 minute time allocation and specifically
requested it.
I set out to investigate the following four research questions.
Do students who experience different forms of instructional interaction -
1. develop different knowledge of the relevant mathematical concepts
and procedures in an area of spatial relations?
2. differ in levels of success in solving spatial mathematical problems
requiring near and far transfer?
3. differ in the ways they access and use problem-relevant knowledge?
4. differ in their dispositions toward the content and the medium of
instruction?
The group I worked with consisted of 54 girls in year 7 (i.e. 11-12
years of age). During the previous eighteen months all students had
experienced the same middle schooling mathematics programs with the
same teacher.
The students were randomly placed into four groups based on pre-test
scores and the mathematics and class teachers' assessment of ability,
performance and progress.
All 54 students were presented with the same problem-solving task,
rotational and symmetry exercises and experienced equivalent pre-task
training. The pre- and post- tests were specifically designed to test
visualisation, symmetry, 2D and 3D, and rotation and were adapted from
various ACER mathematical tests. All students completed a Likert-style
questionnaire and twenty students were randomly selected for interviews
at the end of the four week test period.
Students were assigned to pairs within the following four groups -
Group 1 - interactive CD-ROM program with no teacher interaction
Group 2 - traditional concrete media with no teacher interaction
Group 3 - traditional concrete media with teacher interaction
Group 4 - interactive CD-ROM program with teacher interaction
Students were video taped and audio tape recorded throughout the task
using a think aloud method.
RESULTS
The first set of results I will discuss are based on quantitative
analysis of pre-test and post-test scores, and the second set relates
to preliminary qualitative analyses of a combination of transcripts
gained from the 'talk aloud' method and the observable behaviours of
students recorded on video tape.
Firstly, analysis of the post-test means with the pre-test scores as a
covariate showed no significant difference and further analysis
comparing post-test results using the pre-test score as covariate for
total score; computer vs. concrete groups; interaction vs. non
interaction groups and specific aspects of spatial mathematics ie.
rotation, 2D, symmetry, hidden cubes and visualisaton; all showed no
significant difference.
Qualitative analysis of the video and audio tape recordings examines
planning strategies (metacognition), the types and quality of
interaction, and the processes and strategies used to complete the
task. (refer to Appendix 1 for details of coding structure)
The initial quantitative results lead me to question what these results
actually indicated and whether I could expect a significant change
within the learner on such a task? My response is 'yes' if the claims
I outlined earlier have any credence. Further in considering the task
the students undertook it could be argued that the IMM software
program may have required more mental, as opposed to physical, activity
to complete the task and that the IMM task environment was therefore
more effortful and so more encouraging of knowledge construction. For
example, the computer pairs had to consciously consider the rotation of
the tower in space throughout the construction phase whereas the
concrete groups tended to do this automatically without comment or
discussion. Similarly the computer pairs needed to consistently
recreate a side of the tower they could no longer see when it was
rotated whereas the concrete pairs could view parts or the whole tower
at any time with just a glance.
Therefore I conclude that based on this set of quantitative statistical
analysis the use of either this particular interactive multimedia or
the concrete construction environment appear to be equally effective
for problem-solving task described.
However, trends in the qualitative analysis reveal possible differences
in processes between the media environments. For example, interaction
has been coded 'high' or 'low'. 'Low' interaction between students
working in pairs is identified as simple statements and yes/no answers
whereas 'High' interaction involves elaboration and explanation of
concepts or instructions related to the task. In the interactive
multimedia environment students did verbally interact more, in fact
twice as often as the concrete groups, but 80% of the interaction has
been coded as 'low' whereas the concrete groups recorded 57% of the
interaction episodes as 'low'.
One explanation for the higher rate of interaction in the IMM groups
could be that it took them, on average, longer to complete the task.
Approximately 22 minutes whereas the concrete groups took around 12
minutes. Thus the IMM groups had more time to generate interaction,
however it does not adequately account for the difference in the amount
of low and high interaction taking place.
On examining the transcripts it can be seen that much of the
interaction related to process tasks, such as 'should I click on this
one' and 'how do I do that' whereas the concrete groups show little of
this type of interaction.
These trends in the level of interaction could be linked to an aspect
of the processes and strategies, namely 'identifying a problem' and
'solving a problem'. Interactive multimedia groups spent on average
47% of their process and strategy time identifying or solving problems
whereas concrete groups spent only 25% of time on the same task. I am
further endeavouring to establish the ratio of problems related
directly to the medium so I can hypothesise the degree to which the
interface of the interactive multimedia may have interfered in the
problem solving task.
Within the processes and strategies category, the interactive
multimedia groups tended to spend approximately 33% of their time on
monitoring and evaluating compared to the concrete groups' 20% average,
highlighting once again the extra effort the IMM groups were expending
to complete the same task.
One area which I have designated separately from the processes and
strategies category is that of rotation. Whilst it could be seen as a
process or strategy, I have found the results quite interesting and
felt that they needed to be considered in their own right. For
example, a snapshot of two pairs shows that the pair using the IMM
program undertook 102 rotations throughout the task whereas the pair
using the concrete media executed only 31 rotations (nb. this result is
an average coding result) . It would appear that in respect to this
particular problem-solving task the interactive medium requires the
users to consciously rotate the structure far more than the
concrete medium required, however this extra effort did not show any
significant effect in student's transfer on rotation questions in the
post-test results.
The trends based on preliminary analysis indicate a difference in
process between the two media. However, as to whether or not these
differences advantage or disadvantage the learner needs further
investigation.
CONCLUSIONS
Whilst the IMM and concrete construction environments appear to be
equally effective for this task based on the pre- and post-test
analysis there are indications that possible differences in process
occurred. Questions I am still to investigate are - to what extent do
these differences advantage or disadvantage the learner, and if there
are any disadvantages, are these disadvantages caused by the type or
quality of interaction the Interactive Multimedia environment, ie.
Working Mathematically : Space program, provides.
It is feasible to argue that other aspects of the IMM program may have
an advantage over traditional concrete media in developing student's
knowledge construction and transfer. For example the reflectional tool
may prove beneficial for visualistion concepts, however that was within
the scope of this particular body of research.
Of particular note, is the observable student reliance on the print
support materials I provided. These materials were consulted and
referred to repeatedly throughout the problem-solving task and as this
type of support is not available within the IMM software environment it
highlights the need to teachers to provide similar materials to enable
students to achieve a level of self-efficacy in problem-solving thereby
avoiding the need to repeatedly refer to the teacher for assistance and
guidance.
I also designed the activity in such a way that protected the students
from many of the problems and pitfalls they may have encountered when
using the IMM work environment. For example, I ensured the 'scale' was
pre-set to the optimal level so that the tower would fit within the
workspace without hitting boundaries or barriers thereby sparing
students another potential problem to solve which did not exist in the
concrete environment. By deliberately neutralising many features of
the program and interface I was not only able to ensure consistency
between groups but also avoided expanding the realm of potential
problems and difficulties which students normally will encounter within
the Working Mathematically : Space IMM environment.
Based on the observable actions, comments and post-task interviews with
students it is clear that many students would not have been able to
complete the problem-solving task successfully without this level of
support.
The findings, to date, tend to indicate that teachers need to consider
carefully how, when and why they use particulary the Working
Mathematically CD-ROM, as well as other interactive multimedia programs
within classroom environments. Teachers, I believe, need to examine
the type and quality of interaction that students will experience
throughout the process. Further I believe there is a need to question
the claims made by the developers and carefully examine any IMM package
prior to implementation within a classroom program to ensure that it
will, in fact, achieve the educational outcomes promised and desired
and if it can't and a teacher still wishes to implement the program
then they may need to plan for and provide appropriate support,
guidance and direction which will enable students to experience success
in their interactions within the chosen interactive multimedia environment.
BIBLIOGRAPHY
Ambron, Sueanne & Hooper, Kristine, Eds. (1988) Interactive
Multimedia : Visions of Multimedia for Developers, Educators and
Information Providers. Redmond, Washington : Microsoft Press
Anderson, John R. (1982) 'Acquisition of cognitive skills' in
Psychological Review 89, pp. 361-406
Anderson, John R (1983) The Architecture of Cognition. Cambridge, MA
: Harvard University Press
Broadband Services Expert Group (1995) Networking Australia's Future
: The Final Report of the Broadband Services Expert Group, December
1994. Canberra, ACT : Australian Government Publishing Service.
Chi, M.T.H. Glaser, R & Rees, E 91982) 'Expertise in Problem Solving'
in J.R. Anderson (Ed) Cognitive Skills and their Acquisition.
Hillsdale, NJ : Lawrence Erlbaum Associates, pp. 335-359
Clarke, Richard E. (1994) 'Media Will Never Influence Learning' in
Educational Technology Research and Development Vol. 4. No.2. pp.
21-29
Glaser, R. (1984) 'Education and Thinking : The Role of Knowledge' in
American Psychologist, Vol. 39, No.2 February 1984, pp.93-104
Kozma, Robert (1994) 'Will Media Influence Learning/ Reframing the
Debate' in Educational Technology Research and Development Vol. 4. No.
2 p.7
Olssen, K and Walsh, S (1996) Working Mathematically CD-ROM : Space
Support Book Carlton, Vic. Curriculum Corporation
Preece, Jenny & Davies, Gordon (1992) 'Multimedia : Some Promises,
Some Problems and Some Issues in Human-Systems Interaction' in
Proceedings of the Interanational Interactive Multimedia Symposium
Perth, Western Australia. pp. 259-268
Reynolds, Lynne & Ehrlich, Diane (1992) 'Multimedia in Industry and
Education: A Decision Model for Design in Proceedings of the
Interanational Interactive Multimedia Symposium Perth, Western
Australia. pp. 117-126
Rumelhart, D.E. & Norman, D.A. (1981) 'Analogical Processes in
Learning' in J.R. Anderson (Ed) Cognitive Skills and their Acquisition
Hillsdale, NJ : Lawrence Erlbaum Associates, pp. 335-359
Salomon, G et al (1979) Interaction of Media, Cognition and Learning.
San Francisco : Jossey Boss
Shuell, Thomas J (1988) 'The Role of the Student in Learning from
Instruction' in Contemporary Educational Psychology 13 pp.276-295
Shuell, Thomas J (1989) 'Towarding Evaluating Software According to
Principles of Learning and Teaching' in Journal of Educational
Computing Research Vol.5 (2) pp. 35-149
L.Arnold - Evaluating Contemporary Instructional Technologies : Effects
on Problem-Solving and Mathematical Processing.
2 12/4/97