Mathematics education: Perspectives on research techniques Helen J. Forgasz La Trobe University Abstract Within the educational research methodology literature, heated debate continues on the relative merits of quantitative and qualitative research methods and whetherthetwoapproachescanbecombined successfully. In the mathematics education fields of affect, gender, and studies of their interactions, calls have been made to engage in a wider range of research methods appropriate to the research question, in order to add breadth and depth to our understanding. The development of new measurement tools and the use of multiple research perspectives were advocated. In this paper, the rationale behind the combination of research methodologies used in each of two mathematics education research studies is discussed. The strengths and limitations are considered. In light of technological advances and growing recognition that mathematics education research issues are complex, the acceptance of flexibility in research design is advocated. Background In a recent radio interview (AM, Monday October 9, ABC radio), the views of HughMcKay, a social researcher, were sought in response to comments by the Australian Prime Minister, Paul Keating, that quantitative pre-election polling was useless: I think the weakness is not so much in the polling as in the interpretation.... What the prime minister is probably complaining about is people... who think that when you see a poll which is based on a question about how people are going to vote, that that information tells you how people are going to vote. And, of course, that's simply nonsense.... [From] constant polling about who is the preferred prime minister, there's no evidence that that is a predictor of how people are going to vote in an election. Whereas, more detailed diagnostic questioning... about the characteristics of the prime ministerial candidates and the sort of characteristics you look for in a prime minister, are often more useful. When asked if it was a matter of "balancing the quantitative with the qualitative to get the more accurate result", Mackay replied: Absolutely. As usual, it's neither one nor the other. Qualitative research is absolutely valuable for getting inside the mind of the voter, for getting some perhaps more sensitive impression of the intensity of people's commitment one way or another, and for getting some impression of the sort of issues that really are worrying people. And quantitative research is just a useful long term indicator of how people's thinking about an election or even how they're thinking about the mood of the electorate might be going. But anyone who looks at a set of figures and says "Ah, that's how people would vote today" or "that's how they are going to vote in an election" is being breathtakingly naive. Mackay's perspective on the limitations of quantitative data, the value of qualitative data, and the more informative picture presented by a balanced combination of the two, echoes that of many who write about educational research methodology. In this paper two mathematics education studies using a combination of quantitative and qualitative research methods are described. The research questions of interest and the reasons for adopting the various research methods are discussed. The first study is complete; the second is in progress. To set the context for the discussions, a short overview of current thinking on combining research methods in the broad field of educational research is presented. Parallel concerns within mathematics education research circles arealso considered. The paradigms debate in educational research Within the literature on educational research methodology there has been continuing debate about the relative merits of quantitative and qualitative research methods. Whether the two methodological approaches can be combined successfully has also been examined. The arguments are enmeshed in the debate between those clinging to and those breaking away from an historic emphasis on positivism (empirical, scientific methods) in educational research. Supporters of positivist paradigms have vigorously defended their positions (e.g. Schrag, 1992) while other writers have responded critically (eg. Eisner, 1992; Erickson, 1992). Eisner (1992), for example, did not totally dismiss positivist methods but rejected their exclusivity and emphasis on measurement. A pluralistic conception of research methods was proposed. The paradigms debate is also entwined with arguments on the compatibility of combining quantitative with qualitative research methods. Firestone (1987) described the views of the two major protagonists: the purists' for whom "the two methods types are incompatible because they are based on paradigms that make different assumptions about the world and what constitutes valid research" (p.16) and the pragmatists' for whom "methods are more collections of techniques" (p.16). Others have acknowledged a blurring of the boundaries (Linn, 1992), made comparisons between the paradigms, or put arguments for and against compatibility and co-operation among them (e.g., Gage, 1989; Howe, 1988; Smith & Heshusius, 1986; Wiersma, 1986). Alternative paradigms were supported by Salomon (1991) who maintained that mutual acceptance of complementary paradigms had been recognised on the practical levelby researchers. Howe (1988) maintained that: The compatibility thesis supports the view, beginning to dominate practice, that combining quantitative and qualitative methods is a good thing and denies that such a wedding of methods is epistemologically incoherent. (p.10) Two studies, one qualitative and the other quantitative, that investigated the same topic were discussed by Firestone (1987). The findings, it was argued, provided complementary information, and the differences reflected the unique strengths of the different research methods used. How quantitative and qualitative data can be combined both disjunctively (to investigate separate issues) and conjunctively (as multiple indicators) was illustrated by Howe (1985). The results of disjunctive combinations were consideredrelatively straightforward to interpret by drawing "distinct conclusions based on distinct evidence" (p.16). The more interesting situation involved the conjunctive analysis where "two kinds of data checked one another, reducing the confidence that could be placed in either alone" (p.16). Wiersma (1986) argued that taking an either/or position on the relative merits of quantitative and qualitative research was not fruitful, adding that: Whatever research methods are used, reliability and validity of the research must be considered, and the methods should be used correctly. (p.259) In preparing and planning data collection, Wiersma argued that issues of internal and external reliability and validity can be addressed successfully for all research designs. Validity and reliability in quantitative and qualitative methods Quantitative methods frequently require the collection of data from many individuals. The focus is on group data and the determination of overall trends, or on the provision of a framework for credible prediction from the trends. Individuals are of less importance than are groups of individuals. Reliability and validity issues tend to focus on sampling procedures, the instruments used to determine the quantitative measures required, and the procedures adopted to gather the data and to analyse them. Research questions involving qualitative designs are less concerned with findingnumerical patterns or interpretations associated with groups of individuals. Accurate descriptions of the phenomena are usually of more interest (Wiersma, 1986).To provide sufficient data upon which to base conclusions, observation periods are often extended. Both the context in which individuals are found and their behaviour within that context are relevant. Interviews and quantitative data are sometimes gathered to supplementobservations. According to Dey (1993), the low status previously accorded qualitative research and its marginalisation lead to a heavy emphasis on rigorous analysis. A common concern of all the analytical approaches is "to categorize data and make connections between categories" (Dey, 1993, p.6). Techniques vary in degrees of detail. At one extreme is the bit by bit' method associated with Strauss and Corbin's grounded theory, which aims to generate theory that is fully grounded in the data (Dey, 1993). More holistic strategies are feasible if the researcher has a sense of what is being sought in the data. A middle-order approach provides a flexible compromise. Qualitative research can be susceptible to reliability problems. Wiersma (1986) argued that the context of the research, the research problem, and the status of the observer must be clearly defined. "The use of multiple data collection procedures, along with triangulation, tends to enhance internal reliability" (p.254). While replicability may not be of great concern to some qualitative researchers, reliability is dependent on the clearidentification of issues and full discussion of data analyses (Firestone, 1987; Wiersma, 1986). To establish internal validity, possible causes for the data need to be systematically reasoned. The process involves deduction and induction. External validity can be strengthened by multi-site studies (Firestone, 1987; Wiersma, 1986). Current views on research methods in mathematics education Within mathematics education, Romberg (1992) noted increased diversity in the research methods adopted which ran parallel to the growth of research in the field. In the past, a problem-led approach had not been possible "because of the limited range of research procedures assumed to be legitimate" (Bishop, 1992, p.716). Yet, according to Salomon (1991), some settings, including classrooms, have increasingly been viewed as complex, often nested, conglomerates of interdependent variables, events, perceptions, attitudes, expectations, and behaviors, and thus their study cannot be approached in the same way that the study of single events and single variables can. (p.11) With recognition of the value of alternative research methods, studies in mathematics education employing a mixture of methodologies have taken place (e.g., Cobb et al., 1991;Leder, 1993). Affective variables and gender issues were common concerns of the two research studies described in this paper. With increased recognition of the role of affect and its relationship to student's learning of mathematics, researchers have called for new approaches to research. The development of new measurement tools and the use of multiple research perspectives have been advocated. Macleod (1992) claimed that: The debate over qualitative versus quantitative research methods appears to be almost over, and the time for intelligent use of multiple research methods that fit the research problems is here. (p.591) In the field of gender and mathematics learning, Leder and Fennema (1993) maintained that more information was needed about characteristics of classrooms to determine how they influence students differentially. Leder (1992) suggested that: Supplementing the more common large-scale studies with in-depth small- sample research should provide further insights into the factors that contributetodifferences in mathematics learning within as well as between groups and should lead to more constructive ways of counteracting them. (p.617) While much has been learnt about gender issues in mathematics learning from positivist research perspectives, there has been a relative paucity of qualitative studies. Fennema (1993) argued against abandoning large scale positivist studies but acknowledged theirlimitations: Perhaps it is evidence of a narrow vision, but I do not believe that we shall understand gender and mathematics until scholarly efforts conductedin a positivist framework are complemented with scholarly efforts that utilize other perspectives. (p.15) In the two mathematics education research studies discussed next, a combination of quantitative and qualitative research methods is used. The first study is complete and was conducted with grade 7 students. The second is in progress and involves tertiary mathematics students. An important difference between the two studies is the order in which the gathering of the quantitative and the qualitative data take place. In both studies the research questions guided the choice of research methods adopted. Research Study 1 To gain a better understanding of gender differences in mathematics learning outcomes, this study aimed to examine the relationship between classroom factors and grade 7 students' beliefs about themselves as learners of mathematics. Explanatory models for gender differences in mathematics learning outcomes implicate a range of contributing cognitive (e.g., spatial abilities, confidence, causal attributions for success and failure etc.) and environmental (e.g., society, parents, peer group, teacher etc.) factors (see Leder, 1992). Many of these models include several common affective (belief) variables. The theoretical framework for the study was one of these models, the Autonomous Learning Behavior [ALB] model (Fennema & Peterson, 1985). The ALB-model is considered to be classroom-situated. Factors that might influence the set of beliefs incorporated in the modelwere investigated. The research literature indicated that more was known about the relationships between affective variables and mathematics achievement and between achievement and classroom factors than between affective variables and classroom factors. Findings revealed that the traditional' mathematics classroom, the site of much previous work on gender issues in mathematics learning, has not brought about gender equity in learning outcomes. It appeared that classrooms factors consistent with contemporary notions of how children learn mathematics had the potential to influence students' beliefs positively. But other classroom factors might also be involved. How then might the relationships between a set of affective variables and classroom factors be explored broadly to provide a better understanding of gender differences in mathematics learning outcomes? The research questions were as follows: 1. Does a relationship exist between students' beliefs and aspects of the mathematics classroom learning environment? 2. Is this relationship the same for males and females? 3. Can classroom factors be identified which contribute to a better understanding of gender differences in mathematics learning outcomes? The set of questions lead to the following train of thought about which researchmethods to use: * Previously published instruments were available to measure the range of affective variables. * An instrument was found that would ascertain students' perceptions of aspects of the classroom learning environment. The five subscales were consistent with classroom implementation of contemporary notions of how children learn mathematics effectively. * A large scale survey would enable a relationship between the two sets of variables, affective and classroom factors, to be explored. Male and female cohort's responses could be examined separately. * The range of factors included in the classroom environment instrument was limited, however, and other classroom factors might also be related to students' beliefs. * In-depth qualitative studies of two classrooms would enable an exploration of additional classroom factors and would overcome the limitations of the classroom environment instrument, complement findings from the large scale survey, and, in being multi-sited, would strengthen reliability considerations. Hence a combination of quantitative and qualitative methods, with the large scale survey conducted first, was considered appropriate to address the research questions. For the qualitative component of the study, a major stumbling block was how to infer students' beliefs from their classroom behaviours. Videotaping a sequence of mathematics lessons was considered the optimum means through which to gather the required data. One of the strengths of videotape as a research tool is that it can be reviewed as often as required. The internal reliability of the study is strengthened as a consequence but the time required for the analyses is great (see Forgasz, Landvogt & Leder, 1993). During a pilot study, a set of operational definitions for the behaviours from which beliefs would be inferred was developed and refined. The videotaped records had contributed to the validity of the constructs under consideration. For the main study, four students (two males and two females) in each class were targeted for closer study. Coupled with an extended observation period, this would maximise the potential for multiple observations of the various behaviours of interest. Data analysis involved transcribing the videotaped lessons, keeping records of the behaviours of interest and the circumstances surrounding their manifestation, and then searching for patterns linking the students' beliefs (inferred from behaviours) to classroom factors. The findings from the in-depth studies were not inconsistent with the general conclusions drawn from the large scale survey. That is, the relationships between the affective and the classroom variables identified for large groups were generally supported in individual classrooms, and, to a lesser extent, for individual students. The qualitative results sharpened some of the conclusions drawn from the survey and blurred others. They provided a broader perspective on the relationships between the affective variables of interest and classroom factors that might influence them. The qualitative analyses yielded partial explanations for gender differences on some of the affective variables frequently noted in the literature. In addition, a number of other classroom factors, beyond those investigated in the survey, were identified as likely contributors to the development of students' beliefs. Many new research questions and issues worthy of further investigation arose as a consequence of using the combination of research methods. Full details of theproject are found in Forgasz (1995). Research Study 2 This study is in progress. The aim is to investigate the factors influencing students' decisions to study tertiary level mathematics. Finding reasons for gender differences in participation rates, particularly at higher tertiary levels, is a subsidiary aim. There is growing concern in Australia that the number of students in mathematics and related fields, and female participation rates in particular, has not kept pace with the increase in the availability of tertiary places. While there is some knowledge about the factors influencing decisions about pursuing mathematics at school, much less is known about the tertiarylevel. For the study described earlier, there was an appropriate theoretical framework, the ALB model, around which a research design could be developed. The literature did not yield any models specifically postulating explanations for students' decisions to pursue tertiary level mathematics studies. Two models, one general and one specific to the school context, informed the study, however. The first was the Model of Academic Choice, proposed by Eccles et al. (1985) to explain the factors involved in students' academic choices. It is a general model and may explain gender differences in decisions about taking mathematics courses at the tertiary level. Leder (1992) summarised the set of interacting factors the model incorporates: the cultural milieu, the behaviors, attitudes and expectations of socializers, the child's perceptions of these attitudes and expectations, the individual's goals and general self-schemata or self-image, the perceptions of the value of the task, achievement behaviors, expectancies, task-specific beliefs, past events as well as their interpretations, and the differential aptitudes of the child. (p.609) The second model was developed by Ainley and Sheret (1992) to analyse the relative influence of a range of variables on students' intentions to remain at school beyond grade 10. The following variables were included: student background (e.g., family socio-economic status, parental expectations, gender, non-English speaking background), school membership, achievement level (mathematics and reading comprehension), satisfaction with school (score on quality of school life), and educational plans (intention to stay beyond grade 10). With no assumption of causality, the model assumes that the variablesinteract and that: school achievement may be influenced by background variables and that educational plans may be influenced by both school achievement and background variables. (Ainley & Sheret, 1992, p.78). Relevant variables from the two models were included in the research design. The Eccles et al. (1985) and Ainley and Sheret (1992) models may be pertinent to the tertiary level. The study, however, does not assume this to be the case. That is, the applicability of the models to the tertiary sector is not being tested. Rather, the study aims to explore the underlying factors contributing to students' decision, using the variables incorporated in the models as a starting point. The research question is very broad: "What are the critical factors implicated in students' decisions to study mathematics at the tertiary level?" The literature provided very little guidance. The research design is shown below. It should be noted that Phase 1 of the study is now complete. * Phase 1: Exploring the issues Semi-structured interviews have been conducted with undergraduate mathematicsstudents from one tertiary institution. The basic set of questions was derived from the variables identified in the Eccles et al. (1985) and the Ainley and Sheret (1992) models. Students were encouraged to tell their own stories, relate their personal experiences, and reflect on the perspectives of other students. A background information sheet (e.g., year that grade 12 was completed, type of school attended, and receiving Austudy) was prepared and completed by all interviewees to reduce the number of interview questions. * The interview transcripts and background information sheets are being analysed to tease out the factors that seemed to have influenced the students' decision making. Based on the findings, a large scale survey instrument will be developed. * Phase 2: Large scale survey Following a small pilot study to iron out any difficulties, thesurvey instrument will be administered widely (to undergraduate mathematics students enrolled at three Victorian tertiary institutions). To complement the survey data, a small number of students at each participating institution will be interviewed. The survey data will be subjectedto numerous statistical analyses. The interview data will undergo analyses similar to those in Phase 1. * Phase 3: Longitudinal perspective Phase 2 will be repeated twelve months later to strengthen the reliability of the findings. Re-surveying the same cohorts of students one year further into their studies will provide a longitudinal dimension to the project. Data gathering for the first phase of the study is complete. With no specific theoretical model to work from, the research problem had to be addressed differently from that used in the first study. Early indications from the analyses support the decision to commence the investigation with a qualitative dimension, and for having a semi-structured, open-ended interview protocol to explore for factors potentially related to students' decision-making. Although the data analysis is incomplete, some preliminary findings appear tochallenge assumptions that the two models informing the study are directly transferable to the tertiary level. Particular variables incorporated in the models may not be relevant and others, not found in the models, appear to be related. The interviewer's experiences as students completed the background information sheets, further reinforced during the interviews, suggested that initial reliance on a pen-and-paper approach, with items based on the two models, may have lead to inappropriate interpretations of responses. Examples of the findings and their potential impact on the design of Phase 2 of the study are discussedbriefly. Preliminary findings from Phase 1 All efforts were made to obtain a representative sample of the undergraduate student cohort enrolled in mathematics at one university in Melbourne to participate in the interviews. The resultant sample of 23 undergraduates was not fully representative in terms of year level of study, gender, ethnicity, and commencement age. About half of the interviewees were over the age of 21 (mature-aged students) when they began their tertiary studies of mathematics. For many of them, their experiences during the period between finishing school and commencing their current tertiary courses seemed to have significantly motivated them towards the study of mathematics. The contrast in interest and motivation between school leavers and the `mature-aged' students was stark. The reasons given for studying mathematics by Bill (a mature-aged student who left school after grade 10, and was 25 when he began tertiary studies) and by Anne and Kip (school leavers), exemplify thesedifferences: Bill: Well, I had a big break.... I worked for a while and I didn't like that, just in factories and so on.. I travelled from City A to City B... and I think I was on the dole and I liked reading science fiction books and popular science books and through those sorts of things I got an idea that I thought I would really like to study science and mathematics... so I enrolled in a correspondence course that taught me basic maths and science and then returned to school. I just thought the sort of things I was reading I wanted to know in greater detail. Anne was studying mathematics: honestly, because I don't like any of the other subjects that were offered... and I thought maths would be a logical subject and all I have to know is how to use the formulas and I would be able to get along with it. Kip: Iguess that [mathematics] is the last option that I had because my [VCE] mark was not so flash and that was my last option... with which I could get into uni so I guess I had to take that option. Maths is not a big overly difficult subject for me. I don't find it too hard. So I thought it would not be too bad to do that. The age at which students commence their tertiary studies seems to be related to motivational factors associated with the study of mathematics at the tertiary level. This variable will be incorporated into the large-scale survey instrument for Phase 2 of thestudy. Several home background factors are included in both the Eccles et al. (1985) and the Ainley and Sheret (1992) models. The variables "geographical location of home" and "term residence location" become somewhat ambiguous when mature-aged students are considered. For many of them, home' is no longer the parental home, the assumption underlying the term in the school context of the two models. Wrongly assuming that most respondents were school leavers and that home' was the parental home, the background sheet completed by the interviewees included the following items: Home: city / country town / rural Term residence: home / university college / other.................... These items caused much confusion for the mature-aged students. Had the items been part of a large-scale survey, the interpretation of any results involving them would have been questionable. For Phase 2 of the study, the terminology used will change. So, too, will any other item referring to other home background factors. For this study, the qualitative component of Phase 1 will be multi- functional. The interview data appear to have provided a rich perspective on a group of twenty three undergraduate mathematics students. In many cases, the individual stories of the mature-aged students are inspirational. The process of gathering the data, and the data themselves, will guide the development of a large scale survey instrument. It is anticipated that the findings will also complement those from the quantitative and qualitative dimensions of thelater stages of the project. Conclusions The two research projects discussed in this paper combined qualitative and quantitative research methods. It has been argued that the choice of research methods should evolve from the research problem. In the first study, the research questions and study design were structured around an existing theoretical framework. The qualitative component of the study overcame the constraints imposed by the quantitative survey instrument, complementing and supplementing the findings. Theoretical models informed rather than framed the direction of inquiry in the second study. The research question was very general. The qualitative first stage of the research plan was designed to serve as a testing ground for the development of a survey instrument to be used in later stages. Not only did it serve this purpose but the interview data provided a rich profile of a group of undergraduate mathematics students, yielding several unexpected findings. A recent review of articles published in the Journal for Research in Mathematics Education [JRME] noted the paucity of qualitative research studies on gender and mathematics in the journal's 25 year history (Fennema & Hart, 1994). An obstacle to the adoption of a range of research techniques may well be associated with difficulties in publishing research outcomes. Some researchers are now prepared to utilise a range of research methods relevant to the problem under investigation. This is a logical extension of the increased recognition that issues of interest to educational researchers are complex. For knowledge to advance, researchers and journal editors must be flexible and respond to new challenges. High technology offers a range of new means by which data can be gathered and analysed: sophisticated computers, the Internet and e-mail, for example. Whatever research methods, or combination of methods, are adopted, they should be used correctlyand ethically. References Ainley, J., & Sheret, M. (1992). Progress through high school: A study of senior secondary schooling in New South Wales. Hawthorn, Victoria: The Australian Council for Educational Research. Bishop, A. J. (1992). International perspectives on research in mathematics education. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp.710-723). New York: Macmillan. Cobb, P., Wood, T., Yackel, E., Nicholls, J., Wheatley, G., Trigatti, B., & Perlwitz, M. (1991). Assessment of a problem-centred second-grade mathematics project. Journal for Research in Mathematics Education, 22(1), 3-29. Dey, I. (1993). Qualitative data analysis: A user-friendly guide for social scientists. London: Routledge. Eccles, J., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, D. (1985). Self-perceptions, task perceptions, socializing influences, and the decision to enroll in mathematics. In S. F. Chipman, L. R. Brush, & D. M. Wilson (Eds.), Women and mathematics: Balancing the equation (pp.95-121). Hillsdale, NJ: Lawrence Erlbaum. Eisner, E. W. (1992). Are all causal claims positivist? A reply to Francis Schrag. Educational Researcher, 21(5), 8-9. Erickson, F. (1992). Why the clinical trial doesn't work as a metaphor for educational research: A response to Schrag. Educational Researcher, 21(5), 9-11. Fennema, E. (1993). Mathematics, gender and research. Paper presented at the International Commission on Mathematics Instruction conference: Gender and mathematics education. H""r, Sweden. Fennema, E., & Hart, L. (1994). Gender and the JRME. Journal for Research in Mathematics Education, 25(6), 648-659. Fennema, E., & Peterson, P. (1985). Autonomous learning behavior: A possible explanation of gender-related differences in mathematics. In L. C. Wilkinson & C.B. Marrett (Eds.), Gender related differences in classroom interaction (pp.17-35). New York: Academic Press. Firestone, W. A. (1987). Meaning in method: The rhetoric of quantitative and qualitative research. Educational Researcher, 16(7), 16-21. Forgasz, H. J. (1995). Learning mathematics: Affect, gender, and classroom factors. Unpublished PhD thesis (Monash University). Forgasz, H. J., Landvogt, J., & Leder, G. C. (1993). Videotapes and mathematics learning: Methodological and technical issues. In B. Atweh, C. Kanes, M. Carss, & G. Booker (Eds.). Contexts in mathematics education. Proceedings of the sixteenth annual conferences of the Mathematics Education Research Group of Australasia (MERGA) (pp.253-258). Queensland: Mathematics Education Research Group of Australasia. Gage, N. L. (1989). The paradigm wars and their aftermath: A "historical" sketch of research on teaching since 1989. Educational Researcher, 18(7), 4-10. Howe, K. R. (1985). Two dogmas of educational research. Educational Researcher, 14(8), 10-18. Howe, K. R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17(8), 10-16. Leder, G. C. (1992). Mathematics and gender: Changing perspectives. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp.597-622). New York: Macmillan. Leder, G. C. (1993). Teacher/student interactions in the mathematics classroom: A different perspective. In E. Fennema & G. C. Leder (Eds.), Mathematics and gender (pp.149-168). St Lucia, Queensland: Queensland University Press. Leder, G. C., & Fennema, E. (1993). Gender differences in mathematics: A synthesis. In E. Fennema & G. C. Leder (Eds.), Mathematics and gender (pp.188-199). St Lucia, Queensland: Queensland University Press. Linn, R. L. (1992). Quantitative methods in research on teaching. In M. C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.) (pp.92-118). New York: Macmillan. McLeod, D. B. (1992). Research on affect in mathematics education: A reconceptualization. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp.575-596). New York: Macmillan. Romberg, T. A. (1992). Perspectives on scholarship and research methods. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp.49-64). New York: Macmillan. Salomon, G. (1991). Transcending the qualitative-quantitative debate: The analytic and systemic approaches to educational research. Educational Researcher, 20(6), 10-18. Schrag, F. (1992). In defense of positivist research paradigms. Educational Researcher, 21(5), 5-8. Smith, J. K., & Heshusius, L. (1986). Closing down the conversation: The end of the quantitative-qualitative debate among educational inquirers. Educational Researcher, 15(1), 4-12. Wiersma, W. (1986). Research methods in education: An introduction (4th ed.). Boston: Allyn and Bacon.