Paper presented at the Conference on Educational Research (Singapore, November 1996) Title: Complementary, not contradictory: the spurious conflict between qualitative and quantitative research methodologies Azam MASHHADI Affiliation: Department of Educational Studies, University of Oxford, 15 Norham Gardens, Oxford OX2 6PY, United Kingdom Address for correspondence: Block 16 #07-125, Hougang Avenue 3, Republic of Singapore 530016 e-mail: chrizam@pacific.net.sg Abstract Zubir and Pope (1984), and Howe (1985, 1988) have both argued against the `tyranny of methodological dogma' and that the division between quantitative psychometric and qualitative phenomenological and anthropological traditions is unnecessary. The post-modern self-consciousness of educational research has resulted in the realisation that there is an unavoidable interaction between the researcher and the researched. Likewise modern physics acknowledges that it is not a mirror of nature but a 'myth' about it (Rorty, 1989). The history of science is arguably not a history of discovery but a history of metaphoric construction (Sutton, 1992). Niels Bohr's framework of complementarity provides a powerful metaphoric conceptual viewpoint for resolving the paradigm war between quantitative and qualitative methodologies. The reductionist-mechanistic and holistic-anthropomorphic methodologies or paradigms are not contradictory but complementary. Both quantitative and qualitative methodologies provide insight into differing aspects of a constructed reality that is too complex to be comprehended from only one view-point. Strand: Curriculum development and reform Key words: research methodology, quantitative and qualitative paradigms, complementarity Bio-data: Azam MASHHADI's doctoral thesis, at the University of Oxford, addressed the question of What is the nature of the understanding of the concept of 'wave-particle duality' among Advanced level Physics students? Following degrees in Physics and Astrophysics (University of London) and Astronomy (University of Sussex) he taught for several years at a college in London (UK) before completing a MSc in Educational Research Methodology (Oxford). His research interests include student learning, teacher education, the use of IT in education, research methodology, and philosophy of science. 1 Introduction Research into students' acquisition of, say, scientific concepts is usually allocated to one of two different types of paradigm: one is traditional, scientific, experimental, reductionist, prescriptive, quantitative and nomotechnical, and the other is non-traditional, artistic, naturalistic, holistic, descriptive, qualitative and idiographic. Zubir and Pope (1984), and Howe (1985, 1988) have both argued against the `tyranny of methodological dogma' and that the division between quantitative psychometric and qualitative phenomenological and anthropological traditions is unnecessary. 2 Complementary Underlying much of the contention that quantitative and qualitative methods are incompatible is the outmoded identification of positivism with science, and the positivistic notion that scientific inference consists of building quantitative laws in a mechanistic fashion. As Howe (1985: 16) points out: ...the contention that quantitative and qualitative methods are incompatible is an upshot of the positivistic notion that scientific inference consists in building quantitative laws in a mechanistic fashion. Research into the history of science, and the actual practice of scientists argues against this identification (Barnes, 1974; Feyerabend, 1978; Nader, 1996). In addition King (1987) has argued that there is 'no best method' and that the methodology adopted should be suited to the topic being explored. Researchers should proceed on the pragmatic basis of `what works' (see Broadfoot, 1988; Bryman, 1984; Hammersley, 1992; Osborne, 1995). The post-modern self-consciousness of educational research has resulted in the realisation that there is an unavoidable interaction between the researcher and the researched. Likewise modern physics acknowledges that it is not a mirror of nature but a 'myth' about it (Rorty, 1989: 16). The history of science is arguably not a history of discovery but a history of metaphoric construction (Sutton, 1992). The physicist Niels Bohr's framework of complementarity provides a powerful metaphoric conceptual viewpoint for resolving the 'paradigm war' between quantitative and qualitative methodologies. Lewis Elton (1977) has drawn on Bohr's complementarity principle and has argued that the reductionist-mechanistic and holistic-anthropomorphic methodologies are not contradictory but complementary. Each methodology provides insight into differing aspects of a reality that is too complex to be comprehended from only one view-point. As Elton (1977: 38) points out: The choice between opposing methodologies is not therefore between right and wrong, but between appropriate and inappropriate. The crucial judgement that a researcher must make at the very beginning of his research is which methodology is appropriate for the research which he wishes to pursue. If he chooses an inappropriate one, he will still get results - research is like that - but they will be meaningless. 3 'Complementarity' of research paradigms The `tyranny of methodological dogma' and the division between quantitative psychometric and qualitative phenomenological and anthropological traditions is unnecessary. Both quantitative and qualitative methodologies provide insight into differing aspects of a constructed reality that is too complex to be comprehended from only one view-point. For a given study, the researcher's conception of the 'nature' of what is being investigated influences the choice of method (Hashweh, 1988). The choice will depend upon a number of factors: the nature of the study, the type of information required (e.g. factual data, `understanding', thought processes), the purpose of the study, the outcomes for the data (e.g. predictions, generalisations), research skills of the researcher, time and resource constraints, and the sample population (e.g. availability and access). The key factor for the choice of methods for data collection and analysis is the nature of the research questions. As Elton (1977: 38) points out: The choice between opposing methodologies is not therefore between right and wrong, but between appropriate and inappropriate. The crucial judgement that a researcher must make at the very beginning of his research is which methodology is appropriate for the research which he wishes to pursue. If he chooses an inappropriate one, he will still get results - research is like that - but they will be meaningless. 4 Qualitative understanding through quantitative methodology Elementary particles seem to be waves on Mondays, Wednesdays and Fridays, and particles on Tuesdays, Thursdays and Saturdays. Sir William Bragg A study recently completed on students' understanding of the concept of 'wave-particle duality' will be used to illustrate the complementary of quantitative and qualitative (see Mashhadi, 1995, 1996). Students experience considerable conceptual difficulties in trying to incorporate the ideas of quantum physics into their overall conceptual framework (Faucher, 1987; Gil and Solbes, 1993). In this study the powerful metaphor of the map is used to construct graphic representations of A-level students' `understanding' of quantum physics. The aim of the study is to try and go behind students' overt performance and describe the organisation of knowledge that underpins overt performance, and define understanding in terms of elements of memory and the pattern of association of these elements (White, 1988). The study has adopted an operational definition or limited `measure' of understanding at the level of the population group in which understanding is represented by the relationships or groupings of ideas (conceptions). The nature of students' understanding being represented by their construction of groupings of ideas in a personal psychological space, with underlying dimensions providing a coordinate system for their conceptions. Kelly (1955) suggested the idea of a 'psychological space' as a term for a region in which an individual can place and classify elements of his/her experience. 5 Cognitive structure or the metaphor of the map This research study utilises the powerful heuristic 'metaphor of the map' to reflect the psychological structure of knowledge in the area of quantum physics as perceived by the sample population of A-level physics students. The act of mapping involves the combination of a reduction of reality and the construction of an analogical space (Robinson, 1982). It also enables structures to be constructed or discovered that would remain unknown if not `mapped'. Maps constructed by different population samples can be used to research differences in 'understanding' between, say, first and second year A-level students. All maps are approximations and involve distortions of perceived reality, as they inherently involve the use of a projection, and a systematic reference frame. However the `metaphor of the map' is a powerful means for the holistic representation of knowledge (Wandersee, 1990). 6 Data gathering methodology The definition of understanding adopted in terms of the structural relations between conceptions immediately raises the methodological question of how to access and represent such a conceptual structure if it is present. As students progress through a physics course they do get better at 'playing the game of physics'. Students in maintaining a separation between 'physics' and 'the real world' avoid the need for making basic conceptual changes. Students may however give the impression that by 'learning' the material they have made such changes. To use the type and style of questions to which they are used to would be to 'trigger' reproductive memory techniques. In physics courses students typically solve problems by `rote equation cranking'. Problems test more than understanding. The elicitation of conceptions in a problem solving situation brings in problems related to meta-cognitive skills, and a mastery of knowledge that is not strictly within the domain of the subject. It also has the disadvantage of tending to cue school science knowledge. The research instrument, therefore, should not utilise definitions and mathematical manipulations that feature so heavily in an A-level physics syllabus and course. The students' written work is also ruled out as class notes may well be tightly structured by the teacher or be heavily influenced by text in books. This would sharply limit their usefulness, and any insight obtained on students' understanding. The general strategy was, therefore, to develop specific statements which represent a range of conceptions, and then ask students to respond to them. Since it is the students' perceptions that are sought for in response to particular statements then the use of an attitude scale is appropriate. The simplest method of response scaling is the five-point scale, where the subjects are, essentially, being asked to 'agree' to 'strongly disagree'. Responses on five-point scales are ordinal-level, as the psychological distance between each of the points may not be equal. The scale is discontinuous, and an individual's response will be subjective. The use of a response scale enables students to indicate the degree of uncertainty in their answers. The unit of analysis in this study is taken as the group, and not the individual. The results, therefore, reflect the tendencies of the group's responses to the statements, and not necessarily the perceptions of individuals. The quantification of a categorical measurement does not fundamentally change its nature. When a student is asked to rate his/her feelings or respond to statements by using a response scale the data obtained has been quantified. However, despite quantification, the data still reflects the student's subjective viewpoint. A questionnaire, therefore, appeared to be the most suitable research instrument to address my research question, and it enabled me to access a fairly large sample population from two different year groups, and across a variety of schools. Previously most work on students' conceptions has been carried out using homogenous population samples. The investigation of differences between groups is then only possible by comparing the outcomes of several studies. 7 Data analysis methodology The research question is based on the hypothesis that there might be regularities and similarities in forms of reasoning or understanding shared by groups of individuals. The forms of reasoning may emerge as non-random patterns in the relationships between responses to specific statements, rather than individual responses to individual questions. The aim of data reduction is to reduce as much as possible the large amount of information obtained to a small number of factors or dimensions, while preserving its essential characteristics and provide an accurate summary. Also to abstract from the data any hidden structure that results from some basic typology (using cluster analysis), or any latent dimensions (using multidimensional scaling and principal components analysis). It should be pointed out that although quantitative methods are employed the aim is not to arrive at or build quantitative laws. The essential aim of this study with respect to students' conceptions of quantum physics is the same as the aim of any map which is to construct a bounded graphic representation that corresponds to a perceived reality. Just as a map cannot be reduced to strings of text, student understanding of science concepts is usually non-linear, hierarchical and web like. The methodology enables a graphic representation of the students' scientific knowledge to be constructed. A pictorial representation of underlying dimensions and clusters of statements or propositions is `visually efficient' and easy to understand. For instance, as in the diagram below:  The responses by students were entered into an EXCEL spreadsheet, and the essentials of the structure of the underlying reasoning, in terms of fundamental factors or dimensions, were obtained using multivariate statistical techniques (multidimensional scaling, principal components analysis and cluster analysis) on SPSS (Statistical Package for the Social Sciences). Multidimensional scaling (MDS) can be used to determine if there are any underlying dimensions. Principal components analysis is very similar to multidimensional scaling, and since the principal factors are equivalent to the dimensions from MDS confidence in their interpretation is enhanced. Cluster analysis can be used to further define and help interpret any groupings. The validity, reliability and plausibility of any structure and its interpretation is established and enhanced using 'triangulation' between three different analytical techniques. Multidimensional scaling Multidimensional scaling is designed to analyse data that indicates the degree of dissimilarity (or similarity) of two things in such a way as to display the structure of the distance-like data as a geometrical picture. Each object (or variable) is represented by a point in a multidimensional space. Two similar objects are represented by two points that are close together, and two dissimilar objects by two points that are far apart. The space is usually a two- or three-dimensional Euclidean space but may have more dimensions. The five point response scale provides ordinal data, which means that the space is generated with very few assumptions about the distribution of the data. Since the data is ordinal level non-metric multidimensional scaling using the ALSCAL program in SPSS is used. The `goodness-of-fit' of the data to the model increases each time another dimension is added as the additional dimension allows more freedom to arrange the points. The number of dimensions is determined through considerations of `goodness-of-fit', parsimony and interpretability of the dimensions generated. The model generated by the multidimensional scaling software (ALSCAL) provides coordinate axis that can be interpreted as perceptual dimensions. The label given to the dimensions or axis of the map are the result of an interpretation depending on the nature and location of specific statements. The dimensions are orthogonal, and their interpretation can be considered independently of each other. Principal components analysis Suppose there are no particular groupings of conceptions or statements in the questionnaire. If this supposition were valid then, say, factor analysis of student responses to the statements should reveal that they do not fall significantly into particular groupings. The factors generated are rotated, to simplify the structure as much as possible and thereby aid the process of interpretation, using Varimax rotation. Varimax rotation was used based on its general usefulness with orthogonally rotated factors, and it gave the most easily interpretable factors. The rotated factor matrix highlights significant loadings of variables (statements) which contribute to a factor. The process of interpreting and naming the factors has to be done in such a way that it represents the essence of the variables loading on it. Each variable has ideally a large loading on just one factor and low loadings on the other factors. Since a variable has loadings on the other factors an interpretation of a factor solution should not be regarded as definite or `correct', but the most likely possible interpretation from a set of possibilities. Cluster analysis Cluster analysis was used to investigate the grouping of conceptions. The clustering technique used is an Agglomerative Hierarchical method: the Complete Linkage method. In this method the individual statements or variables (and not the respondents or cases) are classified into groups, and the process repeated at different levels to form a tree (dendogram graph) through a series of successive fusions of the variables into groups. The groups which initially consist of single variables are fused according to the distance between their nearest members. The distance between groups being defined as the distance between their most remote pair of statements. Each fusion decreases by one the number of groups, and proceeds until all the statements are clustered. The Complete Linkage method is used as the values attributed to the statements are ordinal, and it avoids having too many small clusters. A `conventional' viewpoint might regard the use of multivariate statistical techniques as inadequate for eliciting any deep or significant aspects of thought, especially as the statements utilised a fixed (five-point) response scale. Answers to statements might seem to need more subtlety of response. However, as Da Silva (1994: 262) expresses it: People can be directly asked about things they are conscious of thinking or feeling, but deeper and less conscious patterns of thought are less directly accessible, and are to be identified not in individual responses to particular questions, but in patterns of responses to many related situations or questions. 8 Conclusion In this study quantitative techniques are being used to arrive at a qualitative appreciation of underlying dimensions of reasoning employed by students when faced with questions on quantum phenomena. The method of analysis provides a pictorial or graphical representation of the structure or relationships between ideas. However this representation inherently involves the researcher's interpretation of data. All studies, whether employing qualitative or quantitative methodologies, are inherently interpretive. The principal `problem' of all studies is still that of the epistemological question of the hermeneutic circle - the researcher's knowledge of students' conceptions is dependent on the researcher's constructions, which are based on the researcher's conceptions. References Barnes, B. (1974): Scientific Knowledge and Sociological Theory (Routledge and Kegan Paul: Boston). Broadfoot, P. (1988): `Educational research: two cultures and three estates', British Educational Research Journal, Vol. 14, pp. 3-15. Bryman, A. (1984): `The debate about quantitative and qualitative research: A question of method or epistemology?', British Journal of Sociology, Vol. 35, No. 1, pp. 75-92. Da Silva, L.E.F. (1994): Commonsense reasoning about processes: a study of ideas about reversibility, unpublished Ph D thesis (University of London Institute of Education). Elton, L.R.B. (1977): `Methodological themata in educational research', Research Intelligence, Vol. 3, No. 2, pp. 36-39. Faucher, G. (1987): 'Pragmatical Conceptions in the Atomic Domain', in J. Novak (Ed.), Proceedings of 2nd International Seminar on Misconceptions and Educational Strategies in Science and Mathematics (July 26-29 1987, Cornell University). Feyerabend, P. (1978): Science in a Free Society (NLB Press: London). Gil, D. and Solbes, J. (1993): 'The introduction of modern physics: overcoming a deformed vision of science', International Journal of Science Education, Vol. 15, No.3, pp.255-260. Hammersley, M. (1992): What's wrong with Ethnography? (Routledge: London). Hashweh, M. (1988): 'Descriptive studies of students' conceptions in science', Journal of Research in Science Teaching, Vol. 25, No. 2, pp. 121-134. Howe, K. (1985): `Two dogmas of educational research', Educational Researcher, Vol. 14, No. 8, pp. 10-18. Howe, K. (1988) `Against the quantitative-qualitative incompatibility thesis (or, dogmas die hard)', Educational Researcher, Vol. 17, No. 8, pp. 10-16. Kelly, G.A. (1955): The Psychology of Personal Constructs (Norton: New York). King, R. A. (1987): 'No best method: qualitative and quantitative research in the sociology of education', in G. Walford (Ed.), Doing Sociology of Education (Falmer Press: Lewes). Mashhadi, A. (1995): `Students' conceptions of Quantum Physics', in C. Bernardini, C. Tarsitani and M. Vicentini (Eds.) Thinking Physics for Teaching (Plenum Press: New York), pp. 313-328. Mashhadi, A. (1996): What is the nature of the understanding of the concept of 'wave-particle duality' among Advanced level Physics students?, unpublished D Phil thesis (St Cross College/ University of Oxford). Nader, L. (Ed.) (1996): Naked Science: Anthropological Inquiry into Boundaries, Power, and Knowledge (Routledge: London) Osborne, J. (1995): Young children's understanding of science in four domains, unpublished Ph D thesis (King's College, University of London). Robinson, A. H. (1982): Early Thematic Mapping in the History of Cartography (University of Chicago Press: Chicago). Rorty, R. (1989): Contingency, Irony, and Solidarity (Cambridge University Press: Cambridge). Sutton, C. (1992): Words, Science and Learning (Open University Press: Buckingham). Wandersee, J. H. (1990): `Concept mapping and the cartography of cognition', Journal of Research in Science Teaching, Vol. 27, No. 10, pp. 923-936. White, R. (1988): Learning Science (Blackwell: Oxford). Zubir, R. and Pope, M. (1984): `A meeting of paradigms: The use of triangulation method in an investigation into student learning', Research Intelligence, Vol. 17, pp. 7-10.