John Church, PhD

Education Department

University of Canterbury

New Zealand

Because education is a field which includes a number of different disciplines, educational researchers make use of a wide variety of different kinds of research procedures. This paper addresses a number of questions regarding the types of research methods training which are most appropriate for undergraduate and graduate students of education - both those in liberal arts programmes and those in teacher education programmes.

As I have argued elsewhere (Church, 1997, 1998), three distinct approaches to the study of learning and teaching can be identified in the research literature: the ethnographic approach (which is most commonly referred to as qualitative inquiry), the social science approach (the dominant mode of inquiry), and the natural science approach (of which the most common is behaviour analysis). This paper begins with an overview of the major differences between these three research methodologies and the assumptions which underlie them.

Paper presented to the AARE-NZARE Conference on Research in Education, Melbourne, November, 1999. The research upon which this paper is based was undertaken while I was on study leave from the University of Canterbury in 1996. This study leave is gratefully acknowledged. The author's addresses are: Education Department, University of Canterbury, Private Bag 4800, Christchurch. E-mail j.church@educ.canterbury.ac.nz.

Observational procedures. Qualitative researchers employ a range of data gathering procedures including participant surveys, participant and key informant interviews, life history interviews, participant observation, non-participant observation, stream of behaviour records, the examination of written documents, and artefact collection (Patton, 1980). The way in which these procedures are to be used is not governed by any fixed procedural rules. "There is no codified body of procedures that will tell someone how to produce a perceptive, insightful, or illuminating study of the educational world" (Eisner, 1991, p. 169).

Because the qualitative researcher typically spends many months observing in the setting of interest, he or she is well placed to study changes in the behaviour and understandings of participants. The early social anthropologists, for example, studied their communities over several years so that changes over time and changes resulting from the passage of the seasons could be recorded.

Research strategies used. Qualitative research is always descriptive research (LeCompte & Preissle, 1993) and the research is usually broadly focused on all of the events which are occurring in the setting of interest. Although ethnographers use a descriptive methodology, they tend to operate flexibly within each particular investigation. They consider it quite appropriate to introduce new research questions, to change their focus, and to explore new avenues of inquiry as an investigation proceeds. "Modifications in an ethnographic design are common, even after data collection has begun" (LeCompte & Preissle, 1993, p. 48).

Data classification and interpretation. Ethnographic data is usually analysed inductively - often using the constant comparative method described by Glaser and Strauss (1967). However, the data contained in field notes and interview transcripts is not coded or grouped into any kind of standard categories. The researcher is free to generate whatever categories he or she considers to be appropriate to the investigation (Bogdan & Biklen, 1992). Because there are no rules regarding the categories to be used, the categories which are used in practice tend mostly to be natural language categories and the results of such research are normally reported in ordinary language rather than in any kind of technical language.

Theories of knowledge. There has been considerable discussion amongst qualitative researchers regarding the assumptions underlying qualitative research and several different schools of qualitative research may be distinguished (Jacob, 1987). Some qualitative researchers adopt a critical realist position with respect to knowledge (e.g. LeCompte & Preissle, 1993), some adopt a pragmatic position (e.g. Eisner, 1991), and some adopt an idealist or constructivist position (e.g. Guba, 1990). The constructivist position (that the same event is always interpreted by different people in different ways) is perhaps the most common. A thoughtful and perceptive critique of the constructivist position (and many of the claims which have been made for qualitative inquiry) will be found in Phillips (1992).

Aims. Some qualitative researchers argue that the aim of ethnographic research is to identify commonalties in the social life of different groups of people (e.g. Dobbert, 1982; Erickson, 1986) while others argue that searching for generalisations about human conduct is futile (e.g. Smith, 1983; Lincoln & Guba, 1985). "The interpretive-idealist approach to research rejects the possibility that laws will ever be found" (Smith, 1983, p. 12). One of the reasons for this is "the overwhelming complexity of the social world" (Smith, 1983, p. 12).

Types of explanation sought. Unlike social scientists and behaviour analysts, qualitative researchers assume that an explanation of why people behave as they do can be arrived at using a purely descriptive (rather than an experimental) methodology. When qualitative researchers search for explanations (which they often refer to as "understandings") they most commonly look for these explanations in the meanings, reasons, or purposes which individuals give for their actions. Qualitative researchers refer to these explanations as "interpretations" and their mode of explanation as "interpretivist" or "hermeneutical". (These two words mean the same thing.) "From the interpretive-idealist perspective... the investigator seeks to understand the nature of the activity and the meaning that the actor assigns to his or her own actions - the "why" of the activity" (Smith, 1983, p. 12). In this paper, this kind of explanation is referred to as intentional explanation because it attempts to explain particular actions in terms of the intention of the actor (e.g. "Katie studied hard for the test because she wanted to get a good grade"). In practice, the intentional explanations of the qualitative researcher differ little from the intentional explanations of everyday language.

A qualitative study can serve several useful purposes. It can provide information about what people do, how they interact with each other, and the consequences of different courses of action in particular settings. This information can be particularly valuable during the initial phases of investigation in a new field of inquiry because it can serve as a data base for speculation regarding the particular events which will need to be examined in greater detail in future investigations. Such studies can also provide useful information regarding the social rules, conventions, and contingencies which operate in particular settings. This information is of particular interest to behaviour analysts. Thirdly, ethnographic studies can provide useful information regarding the perspectives of individuals in a given context - whether, for example, they like or dislike what they are expected to do, or what is happening to them, and why this might be the case.

However, qualitative research is always descriptive research so it cannot be used to answer questions which require an experimental analysis: questions about the way in which human actions should be grouped for analysis, questions about how learning occurs, questions about the relative effectiveness of different kinds of teaching materials or teaching procedures, or questions about the relationships between different kinds of experience and different kinds of learning. Nor can it be used to answer questions about the distribution of resources, the prevalence of particular behaviours, practices or contingencies, or the outcomes which are being achieved within groups or populations of learners, classrooms or schools.

Social science research methods became popular during the 1930s and 1940s following the development of a range of statistical procedures for analysing sets of scores. They quickly became the most popular and commonly used educational research procedures and remain so to this day.

Observational procedures. Social scientists group behaviours into fairly broad or inclusive categories which they refer to as constructs or variables (constructs such as intelligence, reading achievement, self-esteem, and so on). These constructs are then operationalised by devising tests, rating scales, self-report scales, questionnaires, structured interviews, observation schedules, and so on which can be administered to a reasonably large number of individuals.

The measurement procedures of the social scientist have two important characteristics. First, the great majority of the measurement scales used by social scientists are ordinal scales which have no zero, no named units, and consist of units which vary in size from one investigation to the next (depending upon sample variability). They simply provide a ranking for each of the people to whom the instrument is applied. "Intelligence, aptitude, and personality test scores are ... ordinal. They indicate with more or less accuracy not the amounts of intelligence, aptitude, and personality traits of individuals, but rather the rank-order positions of the individuals. ... ordinal scales do not possess the desirable characteristics of equal intervals or absolute zeros" (Kerlinger, 1964, p. 425). Secondly, the social scientist rarely reports the scores of each of the subjects taking part in an investigation. Rather the scores, indexes, or ranks are averaged to produce a mean level of performance or a mean index for each of the sub-samples in the investigation and it is only these means which are reported. Social scientists are limited to the study of the average response of groups of people because the data interpretation procedures which they use are procedures which can be applied only to sets of scores.

Repeated measures of performance are occasionally collected by social scientists but the most common practice is that of observing or testing the experimental subjects just once (or twice) during the course of a particular investigation. Social scientists have largely ignored the observation and analysis of change over time (Drew, 1976).

Research strategies used. Social scientists employ a variety of different kinds of correlational designs, regression designs, quasi-experimental and experimental designs and these are described in most social science methods texts (e.g. Borg and Gall, 1989). The experimental designs are between-groups designs, that is designs in which the subjects in the experimental group(s) are exposed to the experimental treatment while the subjects in the control group(s) are not. The design of most social science investigations precludes any kind of modification to the investigation while it is in progress. This is because conclusions cannot be drawn from the data of a between-groups experiment until they have been subjected to the appropriate statistical analysis and this step can only be undertaken once data collection has been completed.

Data classification and interpretation. In a social science investigation, the phenomena to be observed (the variables to be studied) are selected prior to the investigation. Social science constructs are often very similar to the mental and behavioural concepts of everyday language. The cognitive scientist's use of the term "memory", for example, is closely similar to the everyday concept of memory as a location where "memories" are stored for future recall. Just as natural language categories tend to distinguish between different classes of behaviour on the basis of their form (rather than their function or purpose) so too do the constructs of the social scientist. There are few conventions governing the creation of new constructs, that is, there are no agreed criteria or procedural rules for determining whether or not a new construct should be added to the scientific literature. This has tended to result in the proliferation of a very large number of different constructs. (See, for example, Reber, 1995.)

The way in which social science data should be interpreted was codified during the first half of the 20th century and involves a set of statistical procedures and decision making rules (Null Hypothesis Significance Testing) which are subscribed to by almost all social scientists. The publication rules operated by most of the journals which publish social science research require the investigator first to demonstrate that the null hypothesis (that there is no relationship between the variables of interest in the investigation sample) can be rejected before concluding that any relationship has, in fact, been observed. "If the empirical data fit the chance model, then it is said they are "not significant". If they do not fit the chance model, if they depart sufficiently from the chance model, it is said that they are "significant" (Kerlinger, 1964, p. 148). The convention operated by most journals is that the null hypotheses can be rejected if there is a less than 5% probability of the observed correlation or mean difference fitting a random differences model. The NHST procedure has been the subject of much critical scrutiny for more than 40 years. Carver (1978) referred to it as the "corrupt scientific method."

Theories of knowledge. Most social science methods texts adopt the position that the behaviour of others exists independently of our perception of it (the realist assumption) and that constructs such as achievement, self-efficacy, conduct disorder, and so on can be objectively observed and recorded. Cook and Campbell, the authors of a much cited text on experimentation, describe their position as "realist because it assumes that causal relationships exist outside of the human mind, and ... critical realist because it assumes that these valid causal relationships cannot be perceived with total accuracy by our imperfect sensory and intellective capacities" (Cook and Campbell, 1979, p. 29). Evidence for the belief in an objectively observable social world is provided by the social scientist's preoccupation with accurate measurement. "The precise measurement of behaviors and their causes is of utmost importance because scientific research determines causality by focusing on change. ... Unless these factors and behaviors have been measured very accurately, it may be difficult to determine whether any changes have, in fact, taken place" (Vasta, Haith & Miller, 1992, p. 61). One of the contradictions of the social science methodology is that, although it rests on a realist epistemology many of its constructs refer to non-physical entities (mentalisms such as mind, memory, cognition, metacognition, and so on) which cannot be observed.

Aims. The stated aim of social science research is to construct a general theory of human behaviour. Social science researchers believe that relationships between constructs can be discovered using the hypothetico-deductive method and that it is possible to discover generalisations about learning and teaching which apply across individuals (Kerlinger, 1965; Borg and Gall, 1989).

Types of explanation sought. The type of explanation most commonly used by social scientists is immediate cause explanation. The majority of cognitive science theories, for example, are simple S->O->R theories in which an intervening variable (usually some kind of mental process) is introduced to bridge the gap between experience and behaviour. Researchers who prefer hermeneutical explanations tend to criticise this kind of explanation as "positivist". Behaviour analysts, on the other hand, reject this kind of explanation on quite different grounds. "Proposing an extraphysical universe as a source of both primary subject matter and explanation for physical events is unique to the social sciences. Because this mental universe is not bound by any physical laws, it is especially troublesome. It provides an endless source of theories about and explanations for behaviour that cannot ultimately be falsified" (Johnston & Pennypacker, 1993b, p. 5).

Because social scientists study the behaviour, attitudes and beliefs of groups of people it is a particularly appropriate method for finding the answers to epidemiological and prevalence questions, for measuring the degree of correlation between different measures of performance (and measures of the same performance taken at different times), and for undertaking larger scale programme evaluations and system evaluations.

However, because social science methods can only be used to study or compare the responses of groups of people, they are ill-suited to the study of those kinds of changes over time which we refer to as learning, they cannot be used to study the learning of individuals, and they cannot be used to study the culture of a particular setting and its impact upon individuals.

The natural science approach is the oldest of the three research traditions described in this paper - having been used by biologists since the middle of the 19th century. The most widely used natural science methodology currently being used to study learning is that which has come to be known as behaviour analysis. Behaviour analysis research procedures were initially developed in the 1950s to study animal learning and were first applied to the study of human learning in the late 1960s. Behaviour analysis research procedures are not normally described in educational research methods texts which means that students who wish to study these methods must consult a suitable behaviour analysis methods text such as Barlow, Hayes & Nelson, (1983), Church (1996), Cooper, Heron & Heward (1987), Johnston & Pennypacker (1993a), Neuman & McCormick (1995), or Poling, Methot & LeSage (1995).

Observational procedures. Behaviour analysts draw a clear distinction between direct and indirect measures of human conduct and, wherever possible, study behaviour change by observing it directly, rather than by probing and recording the recollections of participants. Where direct observation is not possible because the behaviour is a private behaviour (e.g. thinking), the behaviour analyst usually substitutes a behaviour which is observable (e.g. thinking out loud) or a behavioural product (e.g. the number of problems solved). Behaviour analysts also draw a clear distinction between the ordinal measures of the social scientist (which they refer to as vaganotic measures) and measures which have a true zero, a named measurement unit, a unit of standard size and an equal interval measurement scale (which they refer to as idemnotic measures). Idemnotic measures are preferred because the results of measurements involving standard units are directly comparable from one occasion to another within an investigation and from one investigation to the next (Johnston & Pennypacker, 1993a). The idemnotic measures employed by behaviour analysts include counting (event recording), timing (duration recording and latency recording), interval recording, momentary time-sampling, accuracy testing, fluency testing, the counting of behavioural products, measures of response magnitude, the measurement of maintenance (recall), and so on (Cooper, Heron & Heward, 1987).

Behaviour analysts are particularly interested in the analysis of behaviour change, especially those kinds of change which we commonly refer to as "learning". To study change, behaviour analysts make repeated observations of the same behaviour (e.g. working a particular type of problem) by a given learner on a number of occasions over an extended period of time. Repeated observations are made because these yield a more detailed picture of behaviour change than that provided when a particular behaviour or skill is observed or tested on only one or two occasions (West, Young & Spooner, 1990).

Research strategies used. Most behaviour analysis reports are reports of within-subject experiments in which the effects of a particular environmental intervention (such as a change in teaching procedure) are measured by observing and recording the performance of individual learners, first, on a number of occasions prior to the intervention and, then, on a number of occasions following the intervention. The use of a within-subject, time series method allows the behaviour analyst to make modifications to his or her experiment as the experiment proceeds and hence to explore new questions in addition to the question which initially motivated the investigation.

Data classification and analysis. Behaviour analysts divide and group human interactions with the environment into response classes on the basis of their common purpose, rather than on the basis of their form, structure, developmental progression, or value as evidence of some theoretical construct. (Behaviour analysts refer to these response classes as "operants" because they "operate" on the environment.) A complete description of the interactions within a particular response class involves naming the context in which the action typically occurs (its antecedents), the action, and the effects of the action (its consequences). "Typically, we assign a particular act to a functional class on the basis of both its effect and its context. ... We may consider "compliance with a threat" an operant because its members occur in a certain context (a "threat") and have historically had a certain effect (removal of the threat). Handing my wallet to my wife for her to remove money is a different operant from handing my wallet to a mugger" (Baum, 1994, p. 77).

Behaviour analysts present the results of their repeated observations of the responses in a given response class graphically. The graphing procedures which are used are fairly standardised and this greatly facilitates the comparison of results obtained from different experiments. Graphs of performance are examined to identify phase-to-phase changes in (a) level, (b) trend, and (c) within-phase (e.g. day-to-day) variability (Cooper, Heron & Heward, 1987). The graphical presentation of data allows the reader of an experimental report to draw their own conclusions from the data which have been presented (rather than having to rely on the statistical computations of the author).

Theory of knowledge. The behaviour analytic philosophy of science has been almost completely disregarded in the "qualitative vs. quantitative" debate and its inclusion greatly complicates the debate because behaviour analysts reject the epistemological and methodological assumptions of both qualitative inquiry and social science research. Unlike qualitative researchers, behaviour analysts share a common philosophy of science and have written a number of books describing that philosophy of science (e.g. Baum, 1994; Chiesa, 1994; Lee, 1988; Skinner, 1974). This philosophy of science is a pragmatic one which makes no distinction between the subjective and the objective worlds and takes no position on the idealist-realist debate regarding the nature of truth (Baum, 1994). It asks "What difference would it practically make to any one if this notion rather than that notion were true?" The origins of this philosophy of science can be traced from the American philosophers Charles Pierce and William James, the physicist Ernst Mach (who was a friend of James), the research psychologist B.F. Skinner (who was greatly influenced by Mach) down to modern behaviour analysts and modern philosophers of science such as Thomas Kuhn.

Most behaviour analysts are acutely aware that the way in which the researcher approaches the study of learning and teaching is profoundly influenced by the learning history of the researcher, the language of the researcher, and the conceptual structure of that language (Chiesa, 1994). Because the concepts and grammatical structure of everyday language influence the way in which we conceptualise changes in behaviour, behaviour analysts argue that a natural science of human learning must develop its own, carefully defined terms - that it cannot simply use everyday terms (Skinner, 1989).

Aims. For the behaviour analyst, the aim of research is to detect and describe the relationships between behaviour and its context which exist in nature. In contrast to social scientists (who define classes of behaviour in structural terms), behaviour analysts define classes of behaviour in functional terms - with behaviours which serve the same function or purpose being grouped together for the purposes of study. This means that behaviours which are evoked by different antecedents, or which have different consequences (different purposes) must be grouped into separate response classes. They cannot be grouped together in the way that they are grouped by the concepts of everyday language or by the constructs of the social scientist. Behaviour analysts further argue that this aim is best pursued using an experimental analysis (that it cannot be pursued using a descriptive methodology or a correlational methodology). Thirdly, behaviour analysts argue that it is very easy to create a category of behaviour but that the only categories of behaviour which should be allowed in a natural science are those which can be shown consistently to be related to some category of environmental events (Johnston & Pennypacker, 1993a).

Types of explanation sought. For the behaviour analyst, the development of particular ways of behaving, thinking and feeling occurs, in part, as a result of biology (the genetic template transmitted at conception) and, in part, as a result of experience during the life of the individual (Baum, 1994; Catania, 1998). The historical explanations of the behaviour analyst are quite different from the intentional explanations of the interpretivist researcher, and the immediate cause explanations of the cognitive scientist. Behaviour analysts argue that learning is possible because (a) we engage in particular actions on many occasions with the passage of time, (b) each time we engage in a particular action there is some variation in its performance, and (c) different versions of the same action are more or less successful. In order to study learning, therefore, the investigator must monitor repeated instances of a given action over time (to see whether it is changing) and (b) track the outcome (the success) of that behaviour each time it is performed (in order to identify the events which are always present when behaviour is changing in a particular way) (Baum, 1994). Elements of an individual's learning history can only be studied by direct observation. They cannot be studied by interviewing the individual because individuals are unable to recall (describe) each of the hundreds of interactions (and their outcomes) which occurred during the acquisition of a particular behaviour, understanding, belief, or feeling.

Behaviour analysts argue that generalisations about how we learn are discoverable but argue that such generalisations are likely to emerge only slowly. This is because a single experiment can never provide any information regarding the generality of the effect which has been observed - regardless of the number of subjects taking part in that experiment. Many dozens of investigations must be completed before even an outline emerges of the conditions under which a given effect will and will not occur (Blampied, 1997). In other words, generalisations and theory emerge inductively - as a way of summarising the results of large numbers of controlled observations of the same phenomena - and not as a result of a single study of a single site or as a result of one or two critical experiments.

Behaviour analysis shares with qualitative research an interactionist perspective (that what people do is a function of the conditions under which they live and work). And behaviour analysis shares with social science a belief that a science of learning is possible. However, behaviour analysts also argue that learning should be studied at the level of the individual, that behaviours should be grouped according to their common purpose, that environmental variables should be grouped according to their effects on behaviour, and that cognitive processes (thinking), memories (remembering), and emotions (emotional reactions) are not hidden causes of behaviour but are just more behaviour to be explained by examining the history of experiences which give rise to them. In these respects, behaviour analysis is a research tradition which is quite distinct from both ethnography and social science.

Behaviour analysis research methods were specifically designed to study changes in the motivation and competency of individual learners. They have, therefore, much to offer the investigator who is interested in questions about the way in which human actions should be grouped for the purposes of study, questions about the learning processes of individual learners, questions about the effects of particular kinds of experiences on particular kinds of learning, questions about the effects of particular aspects of teaching practice on the learning of individual children, and questions about how best to remedy particular kinds of motivational deficits and learning deficits in individual children.

As with all research methods in education there are, of course, many questions which cannot be studied using behaviour analysis procedures. These procedures cannot be used to find the answer to epidemiological or prevalence questions, they cannot easily be used for large scale programme evaluations, and they have not been designed to generate a holistic picture of the social life or culture of particular sites such as classrooms or schools.

The utility of a research methodology may be assessed with respect to a variety of criteria. This paper examines the relative utility of the three research methodologies under discussion against two criteria: (a) the degree of fit between the subject matter of learning and teaching and the subject matter which each of the three methodologies was designed to investigate and (b) the relative productivity of each of the three methodologies in advancing our understanding of learning and teaching processes.

Research into learning and teaching is research into a subject matter which has certain important characteristics. Learning involves change. It is the individual who profits from experience. Most human action is purposeful. Actions have effects. And the primary purpose of teaching is to facilitate learning. One of the criteria against which we can evaluate the utility of each of the three research methodologies under examination is in terms of the "degree of fit" between the subject matter of learning and teaching and the subject matter which qualitative, social science and behaviour analysis research procedures were designed to investigate.

Learning involves change

Learning involves change - changes over time in understanding, performance, skill, ability, attitude, motivation, level of automaticity, and so on. A child who is learning to print is a child whose ability to print is changing (improving) with the passage of time. So time is an integral component of learning as a subject matter. Any procedure which we use to study learning must, therefore, be a procedure which is capable of tracking change over time.

All three of the research methodologies under examination can be used to track changes in performance over time. It is accepted practice in qualitative research that the investigation of a particular site should continue for a number of months. This means that the qualitative researcher is well positioned to study change. On the other hand, the qualitative researcher often enters the site to be studied with no predetermined conceptions of which behaviours and beliefs should be observed across the duration of the study. This means that some of the changes which might be studied during the investigation are neither observed nor recorded - at least not initially. In addition, the qualitative researcher usually does not make use of any kind of testing procedure to assess learning and this further limits the kinds of changes which can be observed and recorded during this kind of investigation.

Social scientists have also developed a set of procedures for measuring change. These are the within-groups, repeated measures designs. However, a recent analysis of the research designs actually used by social scientists to study learning and teaching (Church, 1998) indicated that well over 80 per cent of the experimental studies of learning undertaken by social scientists employed between-groups rather than within-groups designs. So although procedures for studying change have been developed, they are not often used by social scientists. This may be because the effort which is involved in collecting daily or weekly measures of performance on a large sample of individuals is often prohibitive.

The behaviour analysis methodology was developed specifically to track changes in performance over time and it is this methodology which provides the best picture of those kinds of changes which we refer to as learning.

Learning occurs in individuals

It is the individual who profits from experience. In other words, the changes which we refer to as learning are changes which are occurring at the level of the individual. Any procedure which we use to study learning must, therefore, be a procedure which is capable of tracking changes in the motivation, competency and attitudes of individual learners.

The qualitative researcher is well placed to study the learning of individuals. Because the qualitative researcher is present on site for most of the investigation there is a high probability that they will be present and in a position to observe changes in learner performance at the time when these changes are occurring. To date, however, qualitative researchers have shown little interest in the study of learning. "The interpretive model neglects questions about the origins, causes and results of actors adopting certain interpretations of their actions and social life" (Carr and Kemmis, 1986, p. 95).

Social science research has very little to say about the conditions which result in changes in the behaviour and abilities of individual learners. This is because the subject matter of social science research is the mean performance of groups of people. This is not to say that social science procedures preclude the reporting of, say, the test scores of each of the individuals in a particular study but simply to say that these individual scores are almost never reported.

The subject matter of behaviour analysis, on the other hand, is behaviour change at the level of the individual learner and behaviour analysts draw a careful distinction between data which describe the behaviour of individuals and data derived from the statistical aggregation of the performances of groups of individuals. They argue that the data from single subject analyses and the data from statistical analyses are two completely different subject matters and that, because the two types of data cannot be compared, the researcher must make an explicit choice to study either one subject matter or the other (e.g. Sidman, 1960; Johnston & Pennypacker, 1993b).

Most actions are engaged in for some purpose. When a teacher gives an instruction the teacher usually expects or intends that the instruction will be followed. The same action may be engaged in on different occasions for different purposes. On some occasions, a child may put her hand up to indicate that she wants to answer the question, on other occasions to indicate that she wants some help, and on other occasions to indicate that she wants to go to the toilet. Furthermore, different actions may be engaged in for the same purpose. There are many different actions which a child may use to attract the attention of the teacher, for example. This observation suggests that, for the study of learning, the most appropriate unit of analysis is not a particular type of action (defined structurally or topographically) but a unit which consists of both action-and-purpose (Lee, 1988) and that any procedure which we use to study learning or teaching must be one which is capable of tracking these action-and-purpose units over time.

Qualitative researchers are particularly interested in the purposes or reasons why individuals behave in particular ways in particular settings. For the qualitative researcher, the description of a particular action involves a description not only of the behaviour engaged in but also the "meaning" of that action. For some qualitative researchers, identifying the "meaning" of an action is simply part of the task of describing the action (e.g. Carr & Kemmis, 1986). For other qualitative researchers "meanings" operate in a causal fashion to explain why that action was engaged in (e.g. Erickson, 1986).

The constructs of the social scientist are usually defined without reference to purpose. The behaviours tested by an intelligence test, for example, are included not because they all serve a common purpose, but because they are thought to be good indicators of mental ability. The behaviours listed under the heading Attention Deficit Disorder in a diagnostic manual such as DSM IV are not grouped together because they serve a common function but because they are considered to be good indicators of the ADHD trait.

Behaviour analysts, like qualitative researchers, are intimately concerned with purpose. "We are concerned, first and foremost, with the functions of behaviour, or in lay terms, with purpose" (Carr, 1993, p. 47). "Knowing that a young boy diagnosed as autistic exhibits self-injury is, by itself, not very interesting. What is interesting is why the self-injury occurs (i.e. of what variables is it a function)" (Carr, 1993, p. 48). Behaviour analysts also recognise that quite different behaviours can serve the same purpose. "Further, many different topographies (e.g. self-injury, aggression, verbal abuse, sarcasm, singing obscene songs) may all serve the same purpose (e.g. attention seeking)" (Carr, 1993, p. 48).

The actions of learners and teachers also have effects. These effects may be the same as the purpose for which the action was engaged in or they may be different. In other words, an action may be successful in achieving the goal intended, or it may be unsuccessful. A teacher may give an instruction with the intention that it be followed, but the instruction may not be followed by certain children. A child may raise her hand with the intention of attracting the teacher's attention, but this action may be unsuccessful. It follows, therefore, that any procedure which we use to study learning or teaching must be one which is capable of tracking not only particular actions-and-their-purposes but also their effects (that is, whether or not they are successful) when they occur.

Qualitative researchers are advised to record the effects of the actions which they observe. "Educational ethnographers examine the ... intended and unintended consequences of observed interaction patterns" (LeCompte & Preissle, 1993, p. 28). However, there has been little if any qualitative research into the effects of success (or lack of success) on learner motivation or the development of new skills or understandings.

The social science methodology has been used to study the effects of different kinds of feedback following correct and incorrect responses during learning in a variety of different kinds of learning tasks. It has also been used to study the effects of different levels of task difficulty on measures of learner attitudes, self-efficacy, motivation and so on. So there has been some recognition by social science researchers that the consequences which follow practice responses can have an effect on learning.

Analysis of the consequences of engaging in particular actions is central to the behaviour analysis research enterprise. Many of the generalisations which have emerged from behaviour analysis research are generalisations regarding the effects of different kinds of response-consequence relationships on future motivation, skill development, rate of learning, and memory in individual learners (Catania, 1998).

Learning can occur without any teaching. A child's ball handling skills may improve simply as a result of solitary practice - without anyone providing any teaching or tuition in this skill. It is possible, in other words, to conduct an investigation into learning without necessarily conducting an investigation into teaching. However, we would not normally claim that teaching had occurred (effective teaching, that is) if the learner (the person being taught) had learned nothing at all from that teaching. In other words, a useful investigation into teaching will normally be an investigation into both a sequence of teaching activities and the learning which is resulting from those teaching activities.

All three of the research methodologies under examination can be adapted to study aspects of teaching and the learning which is resulting from that teaching. Since the behaviour analysis methodology provides the most accurate and the most detailed picture of those changes which we refer to as learning, it is also the methodology which appears to have the greatest potential to identify the kinds of teaching and practice conditions which are necessary in order for different kinds of learning to occur.

A second way of evaluating the relative utility of the three research methodologies under discussion is to examine the degree to which the research within each tradition has contributed to our understanding of learning and teaching. Before attempting this, however, it needs to be noted that there are very considerable differences with respect to the amount of research which is being undertaken by qualitative researchers, social scientists and behaviour analysts.

Last year I examined all of the research reports in the 1995 volumes of a representative sample of 17 educational research journals and classified each according to the research methodology which had been used (Church, 1998). This journal sample included a total of 633 articles of which 509 (80%) were empirical reports. A social science methodology was used in 60 per cent of these 509 investigations. Reports of qualitative studies and reports of natural science investigations appeared in approximately equal numbers (17% and 18% respectively). All but two of the natural science studies involved a behaviour analysis methodology. Five per cent of the reports either used a dual methodology, or an unclassifiable methodology.

One particularly surprising finding was the finding that only 256 (50 per cent of the 509 empirical reports) involved studies of change. The journals contained in the sample were all chosen as likely publication outlets for research on learning. Six educational psychology journals were included because they have traditionally been the main publication outlet for research into learning in educational settings. Three teaching research journals and three special education journals were included on the grounds that they would probably include some studies of learning and some studies of teaching (that is, studies of efforts to facilitate learning). Analysis of the reports however, showed these assumptions to be incorrect. The great majority of the descriptive studies, causal comparative studies and correlational studies contained in the 17 journals were not studies of change. They did not attempt to observe change and did not contain any data on change. While many of the correlational studies purported to be studies of learning, in fact, they included no data on learning, just data on the correlations between various scores which had all been collected at the same time.

As far as the studies of learning and teaching were concerned, social science approaches still accounted for some 54 per cent of the research, qualitative studies of any of the kinds of change which might be construed as "learning" dropped to some 7 per cent of the total, and the remaining 33 per cent of studies of change were studies using a behaviour analysis methodology. As far as research methods were concerned, just on 40 per cent of the studies of change made use of a between-groups method and just on 30 per cent made use of a within-subjects, repeated measures method.

Qualitative studies of educational sites such as classrooms have been undertaken for more than 30 years and some 17 per cent of the sample of research reports described above were reports of qualitative studies. Of the 30 research reports contained in the 1995 issues of Teaching and Teacher Education, 24 were reports of qualitative studies - suggesting that qualitative inquiry is now the dominant methodology used by teaching researchers.

The qualitative research effort has been singularly unproductive. The student of learning and teaching will find no textbooks which summarise the findings of qualitative research into learning and teaching, or which describe the relationships between learning and teaching which have been discovered by qualitative researchers. As far as can be ascertained, there are not even any review articles summarising the results of qualitative research into particular aspects of learning or particular aspects of teaching. Qualitative researchers have developed no agreed categories for describing different kinds of learning and no agreed categories for describing different kinds of teaching events. The teacher who looks to the results of qualitative research into learning and teaching will find nothing other than the craft knowledge of other teachers in this corpus of research. They will find no generalisations which can be used to guide decision making with respect to the central concerns of teachers: motivating students, teaching different kinds of skills and knowledge to students, or designing effective teaching materials and teaching activities.

Studies of learning and teaching using a social science methodology have been appearing in research journals for just on 100 years. Notwithstanding recent developments in both qualitative and behaviour analysis methods, social science procedures remain the dominant research procedures which are being used to study learning and teaching.

First impressions of the social science research into learning and teaching suggest that this kind of research has been relatively productive. Several dozen meta-analyses of the research into various teaching variables have been published and there now exist several reviews of these meta-analyses (e.g. Lipsey & Wilson, 1993; Walberg, 1986; Walberg & Wang, 1987). The social science research in a number of fields (child development, special education, teaching, teacher education, and so on) have been summarised in encyclopaedia length treatments (e.g. Wang, Reynolds & Walberg, 1987; Wittrock, 1986). And the most popular educational psychology textbooks (which rely primarily on the results of the social science research into learning and teaching) are substantial text books which have been through many editions (e.g. Gage & Berliner, 1992; Slavin, 1991; Travers, Elliott & Kratochwill, 1993).

Closer reading of these encyclopaedias and textbooks, however, reveals a number of features which are not immediately apparent. To a large extent, the encyclopaedias of research are not so much summaries of research as discussions of the various competing theories which have been advanced to account for perception, cognitive processing, memory, learning, and the development of different kinds of competencies. In the educational psychology textbooks, the material on teaching consists primarily of common sense notions of how to teach buttressed from time to time by reference to a research study which supports that common sense notion. As in the encyclopaedias, material on learning is almost always presented from a variety of theoretical orientations: developmental views of learning, behavioural views of learning, information processing views of learning, and so on. Thirdly, an analysis of the research citations in both the encyclopaedias and the textbooks indicates that where research based generalisations are being advanced, they are often based upon the results of very small number of empirical investigations - often just a single investigation.

This is not to say that social science research has discovered nothing about learning and teaching. The best indication of what has been found out as a result of social science research (as distinct from the application of common sense) is provided by the meta-analyses of research reviews listed above. These reviews of reviews suggest that there are a number of variables which have been found to have measurable and reproducible effects on remembering - variables such as the age of the learner, recency, list length, amount of practice, meaningfulness of the material to be remembered, the quality of the explanations provided by the teacher, the provision of learning goals and study questions, procedures which sustain active engagement by the learner, conditions which motivate rehearsal of the material to be learned, spaced rehearsal, corrective feedback following errors, explicit or direct teaching, teaching to mastery, the setting and evaluation of homework, certain types of co-operative learning procedures, the teaching of certain types of study skills, the teaching of grapheme and word recognition skills in reading, and so on (Lipsey & Wilson, 1993).

It needs to be noted, however, that all of these conclusions are conclusions about the effects of certain variables on the average performance of groups of learners. The extent to which these findings apply to individual learners, the kinds of individuals to whom they apply, and the conditions under which they operate to determine the learning of individual students cannot be ascertained from the social science research literature because these questions have not been studied by social scientists. This greatly reduces the utility of these research findings for teachers because teachers are responsible for the learning of each of the individuals in their class, not just the average level of improvement of the class as a group.

It also needs to be noted that there are huge wastelands of research into learning and teaching (using the social science methodology). Consider, for example, the following areas of research: the pursuit rotor research, the paired associate research, the teacher personality research, the classroom interaction research, research into the effectiveness of different educational media, the programmed instruction research, the discovery learning research, the research into open plan classrooms, the adjunct questions research, the self-concept research, and the self-efficacy research. Each of these areas of research (and many others) generated hundreds of investigations, failed to yield reproducible results, never generated anything in the way of new knowledge, and no longer receive even a brief mention in educational psychology textbooks.

Behaviour analytic studies of learning and teaching first began to appear in the educational literature in 1968. In the 1995 journal sample described above, just on one-third of the studies of change were studies which had been undertaken using a behaviour analysis methodology. It is difficult to compare the productivity of behaviour analysis research with the productivity of social science research into learning and teaching because the number of behaviour analysis reports is such a small fraction relative to the number of social science reports. However, the following observations can be made.

First, behaviour analysts have made good progress in developing a useful procedure for classifying both types of learning and types of variables which affect learning. Both of these classifications are functional classifications. Particular classes of events, experiences, and teaching variables are admitted as classes of events which are worth distinguishing only when they have been shown (as a result of experimental analysis) to have predictable effects on the motivation, the performance, or the learning of individuals (Catania, 1998; Johnston & Pennypacker, 1993a).

Secondly, behaviour analysts have made good progress in identifying a number of elements of an inductively derived "theory of learning". In behaviour analysis, the integrating statements which have been developed to date are usually referred to as principles of behaviour. These principles of behaviour, together with examples of the research from which they are derived, constitute the central content of most behaviour analysis textbooks (e.g. Catania, 1998; Cooper, Heron & Heward, 1987; Malott, Whaley & Malott, 1993). The reader who compares the content of behaviour analyses textbooks will find that the references in behaviour analysis text books are mostly references to primary sources (that is, to actual experiments) rather than references to secondary sources and theoretical works as is the case in most educational psychology texts. There is also a very close degree of agreement across all recently written texts with respect to the technical terms employed, the way in which these terms are defined, and the way in which they are used.

Thirdly, there appears to be a cumulative development occurring within behaviour analysis research. There is an evolutionary progression in behaviour analysis text books with later textbooks covering a wider set of behaviour principles than earlier textbooks. In not much more than 30 years, behaviour analysts have succeeded in identifying a number of the variables which affect motivation, the acquisition of new behaviours and skills, and the retention of new skills over time (Church, 1994). As a result, behaviour analysts are starting to find that they can predict the conditions under which particular kinds of learning (behaviour change) will and will not occur (e.g. Albers & Greer, 1991), and they have been able to develop, for a range of different kinds of skills, teaching procedures which are considerably more effective than anything previously reported in the educational research literature (e.g. Heward, 1994; Johnson & Layng, 1994; Kinder & Carnine, 1991; Lam & Greer, 1991; Lindsley, 1991).

Having identified the main differences between the methods, underlying assumptions, and relative utilities of ethnographic, social science and behaviour analysis research, it now becomes possible to consider questions regarding the possible content of research methods courses for students of education. Of course, education students are not a homogenous group with respect to their probable careers. Students in teacher preparation courses have different career goals from students in liberal arts courses. Students who are pursuing a masters level qualification are more likely to become involved in research than are students who are pursuing a bachelors level qualification. The final section of this paper attempts to identify the most appropriate content for research methods courses which are being provided for these rather different groups of students.

Some Education students are not enrolled in teacher education courses, they are enrolled for a B.A. or some other kind of first degree and are taking some Education courses as part of these degrees. Few of these students are likely to embark upon a career in education (although some may) and few are likely to end up in positions where they will be expected to engage in educational research. While we would probably want such students to have some exposure to educational research during their study of Education, there is probably little point in attempting to teach such students how to do research.

What would be more valuable, intellectually, and more consistent with the aims of a liberal arts degree would be to enable such students to develop some understanding of what educational research is, the assumptions on which it rests, and what it has and has not yet been able to accomplish. This is probably best achieved within an issues based course (rather than a methods course). Some of the issues which might be included in such a course include a study of the way in which new understandings (about, say, learning) arise, what counts as research and how research differs from other modes of inquiry, why it is that educational researchers have been unable to reach agreement regarding the most appropriate research procedures for studying learning and teaching, how it is that the social science methodology came to dominate educational research for almost a century, why it is that educational practice has been so little affected by research in a century which has witnessed quite startling achievements in other areas of inquiry, whether there is any procedure yet devised which could be used to identify, say, some of the conditions upon which learning depends, and so on.

One way of studying these issues would be by reading and analysing actual qualitative, social science and behaviour analysis research reports - perhaps reports on a common theme (such as the development of motivation, the development of understanding, or the development of automaticity, for example.) Students in their third year of undergraduate study might extend the analysis of selected research reports to include a consideration of some of the technical issues which are involved in interpreting research results. Examples of such issues include the relationship between data quality and the believability of a research result, the chain of reasoning which is involved in interpreting the result of an experiment (especially a between-groups experiment), the question of whether thinking and other cognitive processes count as explanations for behaving in a particular way or whether they are simply further behaviours to be accounted for by studying the conditions under which they arise, and the question of how many times the observation of a relationship between two variables needs to be replicated before it becomes prudent to generalise the results of those observations to other learning tasks, or learners, or situations, or sites.

A high proportion of preservice teacher education programmes now include courses in research methods. If the aim of training in research is to bring the novice teacher to the point where they can read and interpret published research reports, then this raises the question which kinds of research reports?

It is probably not necessary to spend time teaching preservice trainees how to read and interpret qualitative research reports since there is almost no qualitative research into how and when learning occurs, or how best to teach particular kinds of skills and understandings to different kinds of learners.

The social science research which is most directly relevant to teaching is that which has attempted to measure the average achievement gain which results from different types of teaching procedures. The main task to be accomplished in teaching novice teachers how to interpret this kind of research is to ensure that students learn to how to distinguish between statistically significant improvements and educationally relevant improvements. Future teachers also need to learn that it is not possible to infer from a between-groups experiment what the effect of a given treatment will be on individual learners (unless the report contains the test results obtained by each individual child).

Given that some 33 per cent of the research into learning is behaviour analysis research, some training in reading behaviour analysis reports obviously needs to be included. Since behaviour analysis data can usually be interpreted without too much formal training, the focus in this part of the course should be on making sense of the technical language in which behaviour analysis reports are written.

If the aim of training in research is to bring teacher trainees to the point where they can undertake research then this raises important questions regarding the particular research procedures which should be taught. What would a teacher need to know, and be able to do, in order to engage in research which is likely to result in improvements in their teaching practice?

There would seem to be little point trying to teach ethnographic methods given, first, that ethnographic research is yielding few insights into how best to facilitate the learning of individuals and, secondly, that few practising teachers will have the time to engage in ethnographic research. Nor need much time be spent on teaching social science research procedures given that practising teachers will rarely be involved in randomised between-groups evaluations and that a high level of statistical sophistication is required in order to interpret the results of a social science investigation.

If the aim of training in research methods is to produce teachers who are able to undertake research, then a mastery of the basic behaviour analysis measurement procedures is likely to be of most direct benefit to the practising teacher. This is because the within-subject methodology is the only procedure which can be used to measure the effects of teaching on the learning of individual children. In addition, behaviour analysis research can be situated in teaching. (This is, perhaps its greatest strength.) It is, in fact, the only research method of the three which can be used to evaluate the effects of particular teaching procedures on learning during the course of that teaching. It is, therefore, the most powerful of the research methods available to teachers, and the method which has the greatest potential to stimulate improvements in teaching practice. "This methodology provides practice-oriented ways for field-based personnel to learn more about what is effective - and what is not - in what they do" (Neuman & McCormick, 1995, p. 29).

The routine use of behaviour analysis procedures to track the learning of individual pupils in the classroom offers practitioners the chance to engage in reflective practice which is based not simply on intuition, hunch, or casual observation, but on the systematic observation and accurate appreciation of the learning of individual pupils. Experience suggests that good progress in teaching behaviour analysis research procedures can be made in a 24 hour course (Church, 1992, 1996).

It is widely assumed that the graduate study of education should include not only training in research methods but also some kind of supervised research project as a practical research training exercise. This raises the question of what kind of research methods training should be provided at this level. It is not possible to provide training in three different research methodologies in a single 50 hour methods course. However, it is possible to provide a research methods training which will be relevant to the career goals of individual students.

Students who intend to return to teaching might most appropriately take advanced courses in research methods which can be used to collect data for reflective practice, to answer questions about learning, and to aid decision making with respect to the educational programming which is most appropriate to individual students. The research methodology which has most to offer in this regard is behaviour analysis.

Students who are planning a career in a government or state department or ministry of education, or a private research unit, or who are moving up to a management role in the education sector, might most appropriately take advanced courses in research methods which can be used to collect prevalence and other kinds of population data, to conduct large scale evaluations of different kinds of educational provisions, and to undertake research into the relative effectiveness or efficiency of different funding regimes, management regimes, teacher education programmes, and so on. The research methodology which is most appropriate to this kind of research is the social science methodology. Such students might also be given the opportunity to learn the rudiments of qualitative research procedures in case they ever become involved in projects which are sufficiently well funded to allow not only the collection of evaluation data but also the collection of some case study data.

Regardless of their career goals, the graduate study of research methods needs to deal with the issues raised in this paper. It needs to include some study of the theories of knowledge which underpin qualitative, social science, and behaviour analysis research, the types of explanation sought by interpretive, social science and behaviour analysis researchers, the types of questions which can and cannot be answered by the three different types of research, and the conditions which must be met before valid generalisations about learning and teaching phenomena can be made.

Finally, I would argue that research methods courses, regardless of their purpose, need to begin to recognise that the often talked about distinction between "qualitative" and "quantitative" methodologies is a distinction of very little heuristic value. While the distinction served a useful purpose during the paradigm debates of the 1980s, it fails to summarise in any meaningful way the diversity of epistemological and methodological approaches currently employed by educational researchers and fails to make clear that there are three, not two, fundamentally different approaches to educational research. Qualitative researchers commonly argue that intentional explanations produce a better level of understanding of human conduct than the immediate cause (and often mechanistic) explanations of the social scientist. However, both intentional explanations and immediate cause explanations are relatively primitive forms of explanation (relying as they do on the cause-and-effect notions which are implicit in the concepts and grammar of everyday language) and other forms of explanation (such as historical explanations) are possible and, in fact, are proving to be even more useful when it comes to understanding how it is that individuals come to behave in the way that they do.