IDENTIFYING TEACHER AND STUDENT THINKING FROM QUALITATIVE ANALYSES OF OPEN-ENDED WRITTEN RESPONSES Jude Butcher, Australian Catholic University Michael Prosser, La Trobe University INTRODUCTION In recent years there has been a growing amount of research into teacher and student thinking. Much of this research has been conducted from a phenomenographic perspective which attempts to identify the different understandings people have of the phenomena around them - an experiential focus. This perspective has been termed a second-order perspective as against a first order perspective which attempts to describe the phenomena themselves (Marton, 1981, 1986). Much of this research has been based upon detailed analyses of open-ended interviews with relatively small numbers of participants. The issue of concern for this paper is whether this perspective can be maintained in more extensive studies involving substantially larger samples. Research from this phenomenographic perspective attempts to describe peoples' experiences of phenomena in the world around them. The experiences are conceived of as person-world relations in the sense that the experience is always an experience of something. The experience, as described by the experiencer, tells as much about that which is being experienced as it does about the person who is experiencing. The experiences are described in terms of a set of qualitatively different categories of description. The focus of phenomenography in its early days was on the experiences of students engaged in studying real academic tasks (Marton and Saljo, 1976; Marton and Saljo, 1984; Svensson, 1976). It described "how" they went about the task - reading an article on university reform - and "what" they learnt from the task. The how was described in terms of qualitatively approaches to reading the article, and the what was described in terms of students' qualitatively different conceptions of what the article was about. These studies on students experiences of learning, based upon students reading academic texts, identified two qualitatively different approaches - a surface approach and a deep approach. A deep approach is one in which students have an intention to understand the material being studied, and in so doing attempt, for example, to relate the various parts of the material being studied to one another and to construct an overarching concept of the material. On the other hand, a surface approach is one in which the intention is to reproduce the material. Rote learning of parts of the material is an example of a strategy used by students adopting this approach. The text itself was about reforms needed to bring university pass rates in line with polytechnic institutions. The author argued that selective measures - aimed at groups of students with low pass rates - were required. But the students understood the article in four qualitatively different ways: A. Selective measures should be taken to improve pass rates of particular groups of students; B. Differential measures should be taken to improve pass rates of each of the different groups of students; C. General measures should be taken to improve pass rates of all students; D. There were no differences between different groups of students. These categories represent four qualitatively different ways of understanding the article. In discussing these conceptions, Marton and Saljo argue that they are hierarchically related, with Categories A and B both involving the use of evidence presented in the text in support of the conclusion and Categories C and D representing descriptions without the use of supporting evidence. The categories are relational in that they were not conceived to be static entities in cognitive structure, but dynamic relations between students, the text and the general learning context. Given a different text and the same context, the students may approach the reading in a different way. Given a different context, but the same text, they may also have approached the reading differently, with different reading outcomes. The group of four categories was identified from rigorous qualitative analyses of transcripts of in-depth interviews with students engaged in particular learning tasks. The transcripts were treated as a whole, and a set of logically and empirically related categories of description were constructed to describe the qualitatively different ways in which the students described their approaches and outcomes. The transcripts were then analysed in terms of these categories. Before describing our adaptations to the methodology for more extensive studies, we will first outline a typical example of the methodology for intensive studies. Example of the Methodology used in an Intensive Phenomenographic Study A typical example of this approach is a project aimed at identifying first year university physical science teachers' approaches to teaching and conceptions of teaching and learning (Trigwell, Prosser and Taylor, in press; Prosser, Trigwell and Taylor, submitted for publication). In that project, twenty four teachers of first year university science courses were intensively interviewed. Trial interviews were conducted, transcribed and reviewed in terms of the quality of the interviews. The focus at this stage was on developing probes to be used in different situations. After the interview schedule had been developed the main interviews were conducted, transcribed and analysed. That analysis began by the three researchers firstly identifying a set of categories of description for the approaches to and conceptions of teaching. They then met, and after comparing their sets of categories and further discussion and debate, identified an agreed set of categories. The categories were again compared to the transcripts and revised by the individual researchers. After several iterations between the researchers working individually and in a group, an agreed set of related categories was identified. The final task was to return to the transcripts and do a final categorisation of the transcripts in terms of the agreed categories. This included two independent categorisations for each of the approaches and conceptions, comparisons of these categorisations, and some minor adjustment to them. Such a process of collecting and analysing the data is time consuming and inappropriate for more extensive studies. The problem is, how do we take the benefits of this case study approach to the analysis of more extensive data. We now describe two examples of how we have adapted this research approach to such studies. CASE STUDIES Case Study 1: Study of Teachers' Conceptions of classroom management Background to the study The research into teacher thinking reported in this paper is concerned with teacher development from novice through beginner to expert in the classroom management domain. The purpose of this research was to gain an understanding of the development of teachersŐ management schemata and the implications of this development for teacher education. The focus of this paper is upon measuring expressions of schema meaning which is the general framework or approach identified through the expressed management strategy and/or intention. There were two concurrent phases in the research, an intensive and an extensive phase. Intensive phase data were collected from a small group of subjects selected from those also involved in the extensive phase. The extensive phase provided data from a large group of subjects who completed a set of ordered tree and questionnaire instruments. The data analysed here are from teachers' responses to an item asking them for a statement of their beliefs about classroom management. Cross-sectional data were obtained from 102 participants across Years 1 to 3 of pre- service teacher education and expert classroom managers. Longitudinal data were obtained from 147 participants who were studied across more than one occasion. Category construction The categories for the analysis of teachers' management approaches needed to describe teachers' experiences of classroom management as expressed in their statements of beliefs. The grounded theory approach to the construction of a theory of teacher development in the management domain required that the categories emerge from close coding and analysis of the data. These coding and analyses were to proceed concurrently with and be allowed to influence further data collection (Glaser & Strauss 1967). The measurement and analysis were grounded in the data collection to ensure the construction of a comprehensive and dense theory derived from rich descriptions of the phenomenon being studied rather than from logical deductions based upon prior assumptions about development or its goals. This procedure saw theory construction as the second level of data analysis which was dependent upon the establishment of valid categories as the first level of data analysis. To minimise possible individual researcher bias and to benefit from examining the data from different perspectives, the researcher was assisted in category development and data analysis by frequent meetings with other researchers, who devoted time to familiarising themselves with at least some of the extensive phase data, and two research assistants, one of whom grounded herself in the questionnaire and the other in the ordered tree data. These meetings were part of a "flip-flop" procedure (Strauss & Corbin, 1990) used to ensure that the data were examined from a number of alternative perspectives to see which perspective(s) best suited the data and the purposes of the study. The process of category development was time consuming, with the meaning categories not being finalised until the fourth year of the study. This process also required the researcher to be conscious of the problems of adopting preconceived perspectives, including those used by previous research teams, and an openness to forsake early coding systems for the construction of other category systems more sensitive to differences in the data and to the construction of a theory of teacher development. The gradual emergence of increasingly valid category sets arose from an iterative process beginning with a study of expert managers' responses, then examining differences between their responses and those from subsets of Year 1 and year 3 novices. The initial category sets were then used in the analysis of the cross-sectional data from the second year of the study, and finally in analysing the data for patterns which may indicate possible schema changes and transitions. Category definition for schema meaning involved tracing underlying dimensions in the data and searching for more holistic expressions of the relationships between underlying dimensions through the use of verbal or graphic representations. Results and Discussion Five management approaches for classifying belief responses were identified. These five categories (See Table 1) were developed from an analysis of subjects' responses to the ordered tree task and the belief, advice and self-description items in the management questionnaire and from an analysis of interview transcripts. Within the management categories the strategies range from one in which teachers' commands are to ensure orderliness to one in which the teacher structures the situation emphasising pupil responsibility within a positive learning environment. The intentions reflected teachersŐ concern with the ensuring of an orderly classroom, the developing of relationships with the class, the achieving of an effective learning environment and gaining pupil active cooperation, or the fostering of a social environment conductive to learning. Table 1 Management approach categories and representative responses for belief statements Category and its descriptionRepresentative responseUncodable approach Response is uncodable because strategy and intention are not clear.Present beliefs about classroom management are of trust, attitude and planning. The teacher needs these beliefs to perform in a manageable classroomNull Approach (N) Teacher presents no management approach Response indicates that subject has no formulated management approach.I have no set beliefs about classroom management.Approach A Teacher directive strategy with the intention of ensuring teacher control and class orderliness. This approach is one in which class orderliness is of prime importance and is a major criterion of teaching success. The teacher adopts the role of an authority who closely monitors the orderliness of commands, expectations and procedures. I think that hard discipline but also being fair with the students makes managing a classroom much easier.Approach B Teacher directive strategy with the intention of establishing good relationships with the class The teacher is, again, the monitor of commands and expectations. There is, however, substantial evidence of a concern for the fostering of good relationships with the class, but these are contingent upon the maintenance of a high degree of orderliness. The desired relationships focus upon the needs of the teacher who wants to be liked or to get on well with the class rather than focussing upon pupil learning. The teacher's role here is one of a likeable authority figure. The teacher seeks two-way respect between teacher and pupils.That to manage the class you must firstly gain the students' respect not only as a teacher but as a friend. You must be fair at all times or this respect is not going to be gained.Approach C Teacher directive strategy with the intention being to achieve learning goals The goal of the teacher in this approach is to achieve learning so that pupils are expected to be predominantly on task. There is a common understanding of expectations. Also, commands and expectations become more incorporated within established routines designed to ensure pupil behaviour appropriate to learning. However, the routines remain fairly explicit and are firmly and consistently monitored. Relationships with the class are important, though the major focus is on pupil learning behaviour, rather than on the needs of the teacher. Providing behaviour, such as pupil talk, is work related, it is tolerated by the teacher.Some methods used by teachers are effective as most students behave in the classroom and most of the work that is set is completed. Category and its descriptionRepresentative responseApproach D Teacher facilitative strategy with the intention of achieving an effective learning environment and active pupil cooperation with the teacher. Teachers' role is to be a catalyst and facilitator of learning. As classroom managers, they see themselves as guides, mentors and helpers in achieving effective learning. These teachers focus upon enhancing learning and preventing management problems. Routines are still used but they require simple and less frequent cues. Behaviour limits are set but they are administered flexibly with regard for individual and situational differences. This teacher aims for pupil attention and cooperation which are freely given rather than being enforced by the teacher. In relating to pupils, teachers emphasise the personal dimension more than the pupil role dimension.Classroom management is a two way siutation which varies from class to class and year to year. It is two way in that students and teachers create a level which is set as normal. The environment plays a factor eg tidiness, heating facilities etc. but initial implementation and control is to be made by the teacher. Individualism, rules and goals are to be prerequisites easily changed but always enforced.Approach E Teacher nurturant strategy with the intention of fostering social environment conducive to learning This approach is one where good teaching and effective learning are very important but substantial attention is given to the group maintenance of the class. Teachers intend to provide a group climate which is comfortable, and usually enjoyable while still being productive. Teachers' strategies are marked by humaneness and often include the use of humour. They foresee and prevent difficulties. In their perceptions of pupils these teachers use a broader frame of reference so that they aim to develop pupil independence, self-management and self-motivation with less reliance on teacher intervention. These areas are sufficiently integrated to allow greater attention to be given to individual and group welfare.It should be founded on a humane teacher approach and an acknowledgement that students need to be managed effectively. The teacher should have an understanding of the needs of the young and so satisfy these needs. thus aiding classroom management. It should be hoped that the goal is inner management by each student, rather than an external imposition of good order. Classroom management is a means to an end- the provision of an effective, caring learning environment. Such management should avoid the expremes of authoritarianism and "free for all". The categories provide a set of logically ordered categories in which the higher level approaches include dimensions expressed in the lower order approaches. The structure within the categories allowed them to be used as an ordinal scale for measuring teachers' management approaches. The logical ordering within the categories is evident in Figure 1 where the direction is from left to right showing a change in goals within the directive strategy before a shift to a facilitative strategy with a learning goal. The last shift is to a nurturant strategy with the goal of a social environment suited to learning. While the logic of the categories is reflected in these tables the order of development does not necessarily follow in this sequence. A high level of intercoder reliability (.8 or above) in the use of the categories was maintained by having each response double coded and regularly calculating intercoder reliabilities. The sets of categories were appropriate for showing differences between cohorts of student teachers and expert classroom managers, with the majority of Year 1 novice teachers expressing a directive strategy for learning while most Year 2 students expressed a directive strategy for orderliness. The Year 3 cohort was characterised by a greater emphasis on directive strategy for learning while many of the expert managers expressed a nurturant strategy for a social environment conducive to learning. Case Study 2: Study of Students' Conceptions of Mathematics Background to the Study This study is being conducted in a first year university mathematics course for mathematics majors. The aim is to study students' experiences of learning in the course in order to work towards improving the quality of their mathematical learning outcomes. The research is focussing on the variation in students' conceptions of mathematics, their approaches to studying it, and their understanding of some key mathematical ideas. In this paper, the focus is on that part of the project looking at students' conceptions of mathematics. The data were collected through an open-ended response questionnaire, distributed to students in the first week of their studies in mathematics. One of the questions included in that questionnaire was: Think about the maths you've done so far. What do you think mathematics is about? The analysis is based upon 242 responses to approximately 300 questionnaires distributed. Category Construction The first stage in the analysis of the data was to identify a set of ordered categories of description of the open-ended question which would best describe the variation is students conceptions of what is mathematics. The five researchers involved in the analysis independently identified a set of categories which each thought described the variation in the responses to the question of a set of 21 responses. They met, compared and contrasted their categories, and agreed on a revised set. The researchers then independently classified the twenty one responses, met again, compared their classifications and found a high level of agreement. They then made some minor adjustments to categories and their descriptions to further improve the reliability of the classifications. One of the researchers then classified the remaining responses, leaving aside any which she had difficulty in classifying. These were then brought to the whole group for classification. Results and Discussion This analysis resulted in the identification of five categories which were judged to adequately describe the variation in response. The categories and representative responses are shown in Table. Table 2: Categories of Description of Conceptions of Mathematics Category and its DescriptionRepresentative ResponseA1. Mathematics is numbers, rules and formulae.(This represents a fragmented view of mathematics)Maths is the study of numbers, and the application of various methods of changing numbersA2. Mathematics is numbers, rules and formulae which can be applied to solve problems. (This represents a fragmented view of mathematics)Mathematics is the study of numbers and their applications in other subjects and the physical worldB1. Mathematics is a complex, logical system; a way of thinking.(This represents a more cohesive view of mathematics)Mathematics is the study of logic. Numbers and symbols are used to study life in a systematic perspective and requires the mind to think a logical and often precise mannerB2. Mathematics is a complex logical system which can be used to solve complex problems.(This represents a more cohesive view of mathematics)Maths is an abstract reasoning process which can be utilised to explore and solve problemsB3. Mathematics is a complex logical system which can be used to solve complex problems and provides new insights which develops our understanding of the world.(This represents a more cohesive view of mathematics)Tech niques for thinking about observable, physical phenomena in a quantitative way and also thinking more abstractly with little or no relation to the directly observable universe. These categories form a logically ordered set, as shown in Table 3. The Table orders the categories in terms of what is focussed on in the students descriptions (referential aspect) and how that focus is achieved (structural aspect) Table 3: Logical Relationship between categories of Description for Conceptions of Mathematics Referential aspect Structural aspect Fragmented Cohesive 1. Numbers/ rules/ formulae A1 2. 1 & solve problems A2 3. Complex logical way of thinking B1 4. 3 & solve complex problems B2 5. 4 & understand the world B3 Table 3 shows that in referential terms, Conception A2 is an extension of Conception A1, while both share a fragmented views of mathematics in structural terms. While each of the conceptions are qualitatively different, there seems to be a more profound shift qualitative shift between Conceptions A2 and Conception B1. In referential terms, Conception B1 is not an extension of Conception A2, but is logically distinct. Conception B1 also represents a cohesive, rather that a fragmented view of mathematics. Conceptions B1, B2 and B3 are similar in structural terms. Referentially, Conception B2 is an extension of Conception B1, and Conception B3 is an extension of Conception B2. This study illustrates how a logically related set of categories of description of student experiences of study can be identified in an extensive study, using student written responses to an open-ended question. The methodology of treating the trial set of data as a whole, without focussing on the variation between individuals, but focussing on the variation in the set of data as a whole, allows the researchers to see more in the data than does a content analysis of individual responses. Having identified a set of categories of description, it is then possible to return to individual statements and classify them in terms of the identified set of categories. The researchers involved in that study have since developed a Conceptions of Mathematics questionnaire, with sub-scales measuring the two extreme conceptions. The questionnaire is presently being used in an extensive study of changes in students' conceptions as a result of studying a first year university mathematics course, how the conceptions relate to approaches to study and perceptions of the learning context, and how all of these relate to students' learning outcomes. DISCUSSION Sets of logically related categories were identified in each of the two case studies for classifying teachers' experiences of classroom management as expressed in their belief statements and students' experiences of studying mathematics. The category systems showed qualitative differences between each category with more profound shifts also being noted between different subsets of the categories. In the management approaches there were fundamental shifts in strategies from directive to facilitative then nurturant while with the goals there were shifts from order and relationships to learning focussed goals. For the conceptions of mathematics there were two major shifts one from a focus upon numbers, rules and formulae to a complex, logical system and the other from a fragmented view to a more cohesive view of Mathematics. Both category systems indicate how there are significant differences in the ways people experience and perceive the phenomena being studied. These categories are interpreted from a phenomenonographic perspective as describing relationships between the individual and the phenomena. The use of a phenomenographic approach in each of these studies provided sets of categories which reflect differences in people's experiences and conceptions of certain phenomena. These categories were used for both identifying differences amongst participants in the studies in their conceptions of management or mathematics. They were and are being used both to compare conceptions expressed by different groups of participants (eg. differences across year 1 to 3 students and expert managers in their conceptions of management) and to trace changes in people's conceptions of management or mathematics over time. The development of measures sensitive to differences between and within groups of participants has been an important contribution of phenomenonographic research. The use of such category systems with larger groups of participants is a new development in this area of research. This development has allowed this research approach to incorporate both intensive (case study) and extensive (large group) phases in their research designs. The use of open-ended questionnaire items in the two studies discussed in the paper show the suitability of phenomenographic methodologies to extensive studies of people's conceptions of reality. The application of the category system developed in the second case study to the Conceptions of Mathematics questionnaire is a further indication of the value of this methodology for establishing a valid set of categories describing people's experiences and conceptions. The content validity of the categories is first seen in the ways the different conceptions were derived from a close study of the data and the continuing iteration between category construction and data analysis. This involved the use of two or more researchers checking on the set of categories and each other's use of them. The conceptual validation of these category systems can be seen in the way they meet the criteria of "educational criticism", description, interpretation, evaluation and thematics (Eisner, 1991). Both studies have provided an understanding of different conceptions of classroom management or mathematics, are able to show changes in conceptions over time and can be used in making judgements about the value of different teaching or learning experiences. The management approaches have also been used to help identify themes emerging from the data in that study to serve as guides in further research into teacher development and teacher education. Ensuring the reliability of the categories and their application required coding of the data by more than one researcher until a high level or reliability was obtained and then the cross- checking of the coding of data found difficult to categorise by the main coder (mathematics study) or regular cross-checking of coding by both researchers (management study). While the construction of valid and reliable categories was very time consuming in both studies the subsequent classification was less demanding and time consuming than was expected. CONCLUSIONS These two case studies of research using the phenomenonographic methodology have shown how this method can be applied with large groups of subjects using questionnaire data. The category construction was time consuming but provided a valid set of categories to describe people's conceptions of phenomena. The reliability of the categories and their applications was ensured through a consensus approach amongst the researchers and the auditing of data coded by other members of the research teams. The categories provided measures for identifying changes in conceptions over time and differences in conceptions between and within groups of participants. The application of the mathematics categories in a Conceptions of Mathematics Questionnaire showed how the conceptualisations which emerged from that study were a basis for the construction of a questionnaire for use in research with large populations of participants. The categories derived from each study were the outcome of the first level of data analysis. Qualitative differences in the conceptions of management and mathematics were identified and formed the basis for the logical, ordered and hierarchical set of categories. These properties of the category systems made them suitable in data analyses using inferential statistics. REFERENCES Glaser, B. G., & Strauss, A.L. (1967). The discovery of grounded theory. New York: Aldine. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage. Crawford, K., Gordon, S., Nicholas, J. and Prosser, M. (1993). Learning Mathematics at the University Level: initial conceptions of mathematics. In W. Atweh, C Kanes, M. Carss and G Booker (Eds). Contexts in mathematics education: Proceedings of the sixteenth annual conference of the Mathematics Education Research Group of Australasia, Brisbane, Australia. Prosser, M., Trigwell, K. and Taylor, P. A phenomenographic study of academics' conceptions of science teaching and learning. Submitted for publication. Marton, F. (1981). Phenomenography - Describing Conceptions of the World Around Us. Instructional Science, 10, 177-200. Marton, F. (1986). Phenomenography - A research approach to investigating different understandings of reality. Journal of Thought, 21, 28-49. Marton, F. and SŠljš, R. (1976). On qualitative differences in learning: 1. Outcome and process. British Journal of Educational Psychology, 46, 4-11. Marton, F. and SŠljš, R. (1984). Approaches to learning. In Marton, F., Hounsell, D.J. & Entwistle, N.J. (Eds.) (1984) The Experience of Learning. Edinburgh: Scottish Academic Press. Svensson, L. (1976). Study skill and learning. Gothenburg: Acta Universitatis Gothonburgensis. Trigwell, K. Prosser, M and Taylor, P. (In press). Qualitative differences in approaches to teaching first year university science courses. Higher Education.