The effects of metacognitive strategy and attributional interventions on students' ability to solve mathematical word problems Rebecca L Gillies, Richard A Walker & Michael Bailey Department of Education The University of Sydney Paper presented at the 1995 AARE Conference, Hobart, Tasmania, 26th November to 30th November, 1995. ABSTRACT This paper reports on the effects of strategy instruction and attributional training on students' ability to solve mathematical word problems, student attributional beliefs and self-efficacy. A total of thirty-nine fifth grades from two classes within two New South Wales primary schools participated in the intervention. All students received mathematical strategy training which aimed to improve students' ability to solve mathematical word problems. The students received the training in four groups: two groups of students received strategy only training and two groups received combined strategy and attribution training. The attribution training aimed to alter students' maladaptive attributional beliefs which hinder successful learning. All groups received twelve half hour sessions of strategy training, which was administered over four weeks. Additionally, students from each treatment type acted as a control, prior to receiving the strategy only or the combined training. Repeated measures analysis was used to show the effects on mathematical achievement, strategy knowledge and usage, attribution and self-efficacy beliefs on three testing occasions. The effects of metacognitive strategy and attributional interventions on students' ability to solve mathematical word problems Metacognition has been shown to be a determinant of success in a variety of academic settings, with metacognitive strategy instruction producing very successful results in the domain of reading comprehension (Palincsar & Brown, 1984). Student achievement in metacognitive interventions has been further increased by the addition of an attributional component to training studies (Carr & Borkowski, 1989; Chan, 1993). While empirical investigation has supported the teaching of metacognitive skills in relation to reading comprehension, evidence of replicated findings in the domain of mathematics has not been so definite. In the area of mathematics, strategy instruction has resulted in a reasonable degree of success (Montague & Bos, 1986), however, the results are encouraging. The interventions carried out in the mathematics domain have generally shown that while metacognitive training in mathematics does improve achievement, it does not occur to the extent of improvement in reading comprehension interventions (Charles & Lester, 1984; Resnick, 1989; Brown & Campione, 1990). The addition of an attributional component to strategy instruction in mathematics may lead to more greatly improved results, and indeed, an attributional component in reading interventions have produced much improved students achievement as has been demonstrated in intervention studies by Borkowski and his colleagues (Borkowski, Weyhing & Carr, 1988; Carr & Borkowski, 1989). Borkowski's research strongly suggests that positive attribution is an essential part of metacognition, and the encouraging results of combined metacognition and attributional training in the domain of reading strongly reflect this. The use of attributional training in combination with metacognitive strategy instruction has produced extremely promising results, but this combination has not yet been used in mathematical strategy interventions, only in reading. The results from combined interventions in the reading domain suggest that these same results should be replicated in the area of mathematics. The present study set out to see if these results could be replicated. The aim of the present study was to ascertain the effectiveness of strategy training in improving mathematical achievement, to determine the effectiveness of the addition of an attributional component to further enhance the strategy training, and the role of self-efficacy and attributional beliefs in mediating the effects on achievement resulting from the training. METHOD Design The present study is experimental in design and consists of pretest, intervention and post-test phases, and also includes a wait control group. The first phase of the design involved pretests, after which the students were then randomly assigned to two treatment conditions: strategy plus attribution training (N=19) and strategy-only training (N=20). There were four groups in total, with one of each different type of treatment in each of the schools. The wait control group acted as a control group for four weeks and then received either the strategy only treatment or the combined strategy plus attribution treatment. During the sessions, the first author met with the students in small groups, which ranged in size from eight to eleven students. In the first school, one group of students received the strategy plus attribution training, while the other group acted as control. Inversely in the second school, one group of students received strategy only training while the other group acted as a control. After the our weeks, the control groups in each school received the strategy only training and strategy plus attribution training, respectively. Therefore, in each school each group of students received a different treatment type. This design allowed all students to benefit from the treatment as well as ensuring that comparisons could be made between the treatment and control group. All students, including those in the control group, received posttests at the end of the strategy training. Subjects Thirty-nine fifth grade students from two public schools in New South Wales participated in the study. From one school, twenty-two students participated, while in the second school seventeen students took part. The participants were drawn from two classes, which spanned the whole range of ability levels. The twenty-three boys (59%) and sixteen girls (41%) in the study were predominantly from lower-middle class socio-economic backgrounds and represented a variety of different ethnic backgrounds. Procedure The training was implemented over a period of two months, from mid-July to mid-September, during term three of the school year. Pretest measures of knowledge and usage of mathematical problem solving strategies, causal attributions, problem solving ability and self efficacy in relation to problem solving, were collected in the first week of Term 3. The training was implemented in small groups during the first eight weeks of Term 3. The posttests were administered two days after the completion of the training. Materials i) Instruments Students' maths problem solving ability, attributional beliefs, self efficacy beliefs and knowledge and use of metacognitive strategies in mathematics were measured prior to and after treatment. Mathematics Achievement Mathematics achievement was measured by a standardised M5 PEP maths problems test (1977), developed by the N.S.W. Primary Evaluation Programme, and was used as an assessment of the students' ability to solve maths word problems. The twenty-five questions were marked on the basis of right and wrong answers. The test consists of a mainly product problems, which can be solved by simply translating to one or more number sentences. As the test was multiple choice, students were required to choose the correct answer out of four possible answers. Mathematics achievement was also assessed by a non-standardised maths problem test, which was devised by the first author. This test, named the Mathematics Problems Test 2 (MPT2), measures students' success at solving problems which require a higher degree of reflective thinking and metacognitive skill. The level of difficulty of this test is higher than that of the M5 PEP test, not in regards to numeric concepts, but because students are required to be more reflective and "metacognitive" in order to find the solution successfully. Causal Attributions Causal attributions were assessed through the use of mathematics items only of the Sydney Attribution Scale (Marsh, 1986). This measure consists of forty items which allow for the assessment of the extent to which students make internal attributions to ability, effort, strategy and external attributions, in relation to success and failure. Students had to consider ten scenarios (six situations of failure and four situations of success) which are typical to fifth grade students, and indicate on a true to false scale to what degree the reason for the occurrence of the event described is true or false for them. Self-Regulated Learning The Self -Regulated Learning Scale developed by Fairbarin, Moore and Chan (1994) was used to assess students awareness and regulation of learning strategies. The questionnaire has sixteen items to which students were required to respond on two subscales: "How helpful is this?" (strategy knowledge) and "How often do you do this?" (reported strategy usage). The items were designed to fit into the seven categories of the strategy training model developed by Montague and Bos (1986b). The metacognitive strategies in the measure are those which were emphasised by Montague and Bos in their intervention studies. Three items considered to be important were added to the measure by the author, and seven items which were inappropriate in relation to the strategy training employed in this study were deleted. Self-Efficacy Beliefs Students' self-efficacy beliefs were measured by a Self-efficacy Scale adapted from a procedure employed by Schunk (1982) to measure how confident students feel about solving maths problems. Students were presented with fifteen maths problems in their treatment groups, one question at a time on an overhead projector, for approximately five seconds allowing for enough time for the problem to be read, but not to be solved. Students were required to indicate on a scale of 0-100 the extent to which they felt that they had the ability to correctly solve the problem. ii) Training materials Approximately forty mathematical word problems were extracted from fourth and fifth grade level text books for use in the training sessions. Students were given a range of product and process problems which were arranged into levels of difficulty. All students were given an seventeen page activity book which was developed by the author, and was used in the initial six sessions. This booklet contained information about the strategy steps and included a copy of the seven step maths strategy. Attributional cartoons, were used with the groups receiving attributional training in guided discussions about the causes of success and failure. A cartoon which displays the phrase "I tried hard and used the strategy" was permanently displayed next to the wall chart of the strategy, in order to reinforce the need for effort and strategy use. The group motto "The strategy + effort = success" was also displayed as a poster near the strategy. Training activities The strategy training was provided by the first author to small groups of eight to eleven students over twelve half hour sessions extending over a period of four weeks. The small numbers in each group allowed for the provision of maximal participation of all students, for maximal assistance with individual students and ongoing positive and corrective feedback. Each instructional group was organised so that it comprised of a range of abilities. In the first six sessions, students were taught the meaning of each strategy step and were given a demonstration by the instructor as to how the strategy could be applied to a maths problem. In the final six sessions students were given the opportunity to practice using the strategy in relation to maths word problems. Students in the strategy plus attribution treatment received attributional training from the third session onwards in the form of planned attributional dialogues emphasising the importance of effort and strategy use. Control Students from both schools acted as a control group (N= 20) before receiving either the strategy only treatment or the strategy plus attribution treatment. During this time, these students were withdrawn from class for the same time and frequency as the treatment groups. However, instead of receiving training in maths problem solving, the control group received lessons in Health, which were developed by the author. Health was chosen as a focus subject area for the control group because it is removed from mathematics and mathematical thinking. Treatment conditions Students from each class were assigned, through a stratified procedure, to one of two treatment conditions so that each group was balanced in terms of ability and gender. Two groups received metacognitive strategy only training and two groups received combined strategy plus attributional training. The attributional training aimed at emphasising the importance of effort and strategy use in order to reach success in solving word problems. In addition, two groups served as a wait control group prior to the treatment of either strategy training only or the combined treatment. Thus, the students in the wait control group participated for eight weeks in total; four weeks for Health lessons and four weeks for the strategy training. i) Strategy training. All treatment groups were exposed to the mathematical strategy instruction which included teaching students a seven step problem solving strategy derived from the work of Montague and Bos (1986). The first six sessions consisted of strategy acquisition, where students learnt the meaning of each strategy step and how it can be used, while the last six sessions consisted of strategy practice, where students were given the opportunity to practice the strategy as a whole. The seven steps of the strategy are described below: 1. Read (for understanding) Read the problem. If I don't understand, read it again. Have I read and understood the problem? Do I need to ask about the meaning of any words? Check for understanding as I solve the problem. 2. Paraphrase (put the problem into your own words) Underline the important information. Put the problem in my own words. Have I underlined the important information? What is the question? What am I looking for? Check that the information goes with the question. 3. Visualise (draw a picture or a diagram) Make a drawing or diagram using the information given in the problem. Does the picture fit the problem? Have I used only the necessary information in the picture? Check the picture against the problem information. 4. .Plan (how you are going to solve the problem) Decide how many steps and operations are needed. Write the operation symbols. If I do this, what will I get? If I do this, what will I need to do next? How many steps are needed? Check that the plan makes sense. If not, ask for help. 5. Estimate (make a prediction of the answer) Round the numbers, do the problem in my head, and write the estimate. Did I round up or down? Did I write the estimate? Check that I used the important information. 6. Compute (find the answer) Do the operations in the right order. How does my answer compare with my estimate? Does my answer make sense? Are the decimals or money signs in the right order? Check that all the operations were done in the right order. 7. Check (make sure you have done everything right) Check the computation and that the answer is correct. Have I checked every step? Have I checked the computation? Is my answer right? If unsure, return to earlier strategy steps and if still unsure ask for help. ii) Attribution plus strategy training. Two treatment groups received combined attributional plus strategy training. In addition to learning about the maths strategy, students allocated to the combined treatment group received attributional training aimed at emphasising the importance of effort and strategy use in order to achieve success. The attributional training occurred in the form of planned, guided discussions about the importance of attributing successes and failures to strategy use and effort, instead of external factors such as luck, teacher bias or ability. In addition to the use of the attributional discussions and cartoons, students were encouraged to believe that the effortful use of the maths strategy would help them to solve the problem more successfully. The group motto "The strategy + effort = success" was introduced in the first session and reinforced throughout the training period. RESULTS AND DISCUSSION Three research issues were explored: the effectiveness of the metacognition only training, the effectiveness of the metacognition plus attribution training and the mediating role of self-efficacy and causal attributions in relation to mathematical achievement. The results were analysed in terms of the four treatment groups: the wait control combined training, wait control strategy-only training, combined training and strategy-only training, in relation to the three testing occasions. Means scores and standard deviations on three testing occasions for mathematics achievement, strategy knowledge, reported strategy usage, attributions and self-efficacy beliefs were collected. The comparison of differences on the tests between the groups used a repeated measures analysis of variance. Research issue one: The effectiveness of the metacognitive training. The mean and standard deviation scores showed that all treatment groups improved after the training. The greatest improvements, however, occurred in relation to students' ability to solve process problems. The treatment groups did not improve to the same extent in regards to product problems. This may have occurred as a result of the test being standardised, as previous researchers have noted that standardised tests often do not show improvements. In addition to improving achievement scores, the results show that the strategy knowledge and usage of all groups increased after the training. An ANOVA test showed that the treatment effects on strategy knowledge between groups proved to be significant at the .0001 level. Reported strategy usage between treatment groups was also significant at the 0.001 level. Research issue two: The effectiveness of the combined strategy and attribution training. The results from the standard deviation and mean scores show that the combined groups outperformed the strategy groups only to a small degree. Therefore, it does not appear that the combined attribution plus strategy treatment produced significant improvements in achievement in comparison to the strategy only group. The results, however, show that one group receiving the immediate combined treatment continued to improve four weeks after the commencement of the training. This suggests that these students generalised the use of the strategy to the classroom. The results clearly show that the two combined metacognition and attribution groups were markedly different in terms of mathematics achievement at the pretest and posttest levels. The immediate strategy plus attribution group performed higher on both mathematics tests in comparison to the delayed strategy plus attribution group. This suggests that mathematical ability may affect the extent to which students benefit from strategy instruction. Research issue three: The mediating role of self-efficacy and attributional beliefs. Significant treatment effects occurred in relation to students' positive self-efficacy beliefs. An ANOVA test showed that the treatment effects were significant at the .002 level. This occurred in relation to the combined treatment groups, and suggests that the attribution training may have produced these effects on self-efficacy beliefs. In relation to causal attributions, changes occurred but not as a result of the attribution training. The changes were experienced by all treatment groups, and were not unique to the combined strategy plus attribution groups. The changes in regards to strategy attributions were the most significant, at the 0.02 level, with all groups displaying improvements after the instruction. Key findings, Limitations and Implications The most significant finding from the present study was the effectiveness of the metacognitive strategy instruction in terms of producing improved mathematical achievement. As a result of the strategy instruction, the treatment groups showed gains in relation to mathematical achievement, strategy knowledge and strategy usage. While the combined strategy and attribution intervention did not produce a more greatly improved achievement in comparison to the strategy-only intervention as was expected, however, a longer intervention may have eventually led to more successful results. The time period of the attribution training may not have been sufficient in order to produce changes in students' maladaptive attributional beliefs. Through contact with the students, the instructor noticed that some students were benefiting from the intervention more than others. For example, students with a solid grounding in mathematics were showing more success than the students with a limited understanding and knowledge of mathematical concepts. As the small sample size of the study was not sufficient to deal with this issue, the author suggests that this important issue could be more appropriately addressed by future researchers. The present study provides support for the value of the seven step- strategy developed by Montague, and for the value of strategy instruction. The findings from this study in relation to the combined strategy attributional training in mathematics, while do not replicate those from the reading domain, serve to encourage future research of this kind in the domain of mathematics. References Australian Council for Educational Research (1977). PEP Mathematics tests. Teachers handbook. Hawthorne: Australian Council for Educational Research. Borkowski, J.G., Weyhing, R.S., & Carr, M. (1988). Effects of attributional retraining on strategy-based reading comprehension in learning disabled students. Journal of Educational Psychology, 80, 46-53. Brown, A.L., & Campione, J.C. (1990). Interactive learning environments and the teaching of science and mathematics. In M.H. Gardner, J.G. Greeno, F. Reif, A.H. Schoenfeld, A. Di Sessa, & E.Stage (Eds.), Towards a scientific practice of science education (pp. 122-39). Hillsdale, NJ: Erlbaum. Carr, M., & Borkowski, J.G. (1989). Attributional training and the generalisation of reading strategies with underachieving children. Learning and Individual Differences, 1, 327-341. Chan, L. K.S. (1993). Combined strategy and attributional training for seventh grade average and poor readers. Paper presented at the Annual Conference of the Australian Association for Research in Education, Fremantle . November 22-25. Charles, R.I., & Lester, F.K. (1984). An evaluation of a process-oriented instructional program in mathematical problem solving in grades 5 and 7. Journal for Research in Mathematics Education, 15, 15-34. Fairbarin, H.M., Moore, P.J., & Chan, L.K.S. (1994). Assessing approaches to learning, attributional beliefs and strategic learning in specific subjects domains: Scale development. Paper presented at the 1994 AARE Annual Conference, Newcastle, New South Wales, 27th November- 1st December, 1994. Marsh, H.W. (1986). Self-serving effect (bias?) in academic attributions: Its relation to academic achievement and self-concept. Journal of Educational Psychology, 78, 190-200. Montague, M., & Bos, C.S. (1986). The effect of cognitive strategy training on verbal math problem solving performance of learning disabled adolescents. Journal of Learning Disabilities, 19, 26-33. Palincsar, A.S., & Brown, A.L. (1984). Reciprocal teaching and comprehension-monitoring activities. Cognition and Instruction, 1, 117-175. Resnick, L. (1989). Treating mathematics as an ill-structured discipline. In R.I. Charles, & E.A. Silver (Eds.), Research agenda for mathematics education: Teaching and assessment of mathematical problem solving (pp. 32-60). Hillsdale, NJ: Erlbaum. Schunk, D.H. (1982). Effects of effort attributional feedback on children's perceived self-efficacy and achievement. Journal of Educational Psychology, 74, 548-556. 13