Debra M.B. Herbert

Jennifer S. Burt

School of Psychology, The University of Queensland

Paper presented at the 1998 Conference for the Australian Association for Research in Education; November 30 - December 4, University of Adelaide, Adelaide, Australia.

Abstract

Research into learning and the long-term retention of knowledge acquired through formal education has tended to centre on the issue of schematisation. However, surprisingly little is known about how knowledge gained in a specific episode (i.e. a lecture) can, over the course of an extended period learning, become a conceptual framework of knowledge relatively free of reference to the details of the specific learning episode. Recent research (Conway, Gardiner, Perfect, Anderson & Cohen, 1997) has linked the concepts of episodic/semantic memory and states of memory awareness originally from cognitive psychology, to the study of the schematisation process in a 'real world' learning environment with interesting results. It proposes that when a new knowledge base is to be learned, memory is initially represented in a way that supports recollection of the specific learning episodes. As learning proceeds, the underlying memory representations may change so that they no longer primarily lead to recollective experiences but instead become so highly familiar that they are simply 'known'. Therefore, as learning progresses, memory awareness should vary systematically with the degree of schematisation of knowledge. The purpose of the present research is to investigate the process of learning and schematisation in the university context in order to further examine the role of episodic and semantic memory and the changes in memory awareness.

Memory Awareness and Schematisation: Learning in the University Context

In recent years, there has been a renewal of interest in research regarding the consciousness, or awareness, which has been suggested to accompany different aspects of memory (e.g., Brewer, 1996; Conway, Gardiner, Perfect, Anderson & Cohen, 1997; Gardiner & Java, 1993; Rajaram, & Roediger, 1997; Tulving, 1985). One approach introduced by Tulving (1985) suggests that there are separate memory systems, including the episodic and the semantic systems. Semantic memory relates to the conceptual knowledge that individuals possess about the world. It is characterised by memory awareness involving feelings of familiarity or 'knowing', and allows the individual to use this knowledge in their everyday experience. The episodic memory system is embedded in the semantic memory system and involves representations of personally experienced events accompanied by the conscious recollection or remembering of these past experiences. For example, 'knowing' that a dog produces a sound called a bark would be linked to the semantic memory system, whereas 'remembering' that the neighbour's dog barked at you last week would be linked to the episodic memory system. Tulving (1985) describes memory performance in terms of the above concepts through a model of recall and recognition. which shows how memory performance can be a result of different combinations of episodic and semantic memory and memory awareness. The recollective experience accompanying memory performance is characterised by a greater degree of remembering as the amount of episodic memory utilised increases.

Remembering and knowing have also been related to processing models of memory, such as the dual-processing model of recognition memory (e.g., Jacoby, 1991; Jacoby & Dallas, 1981) which suggests that there are occasions when memory involves reflecting on the specific context of the study episode (controlled recollection process) or when something just seems familiar (automatic familiarity process). In opposition to such theories, remembering and knowing have also been interpreted by signal detection models to simply represent differences in confidence or trace strength (e.g., Donaldson, 1996; Hirshman & Master, 1997). In such models, remember and know responses are considered the outcome of two different response criteria. The first criterion divides positive recognition responses from negative recognition responses, and the second criterion divides positive recognition responses into stronger responses (remember) and weaker responses (know). Thus, both remember and know responses involve only the episodic memory system.

Experiments investigating memory awareness have generally involved participants studying word lists and then completing a recognition memory test. Participants are instructed that remembering involves re-experiencing at retrieval some thought or event that took place during study, whereas knowing refers to knowing for a fact that an item was present in the study list but not having any specific recollection of studying the item. Tulving's (1985) initial theory inspired the application of memory awareness to research with older adults (Parkin & Walter, 1992), forgetting in recognition memory (Gardiner & Java, 1991), autobiographical memory (Conway, Collins, Gathercole & Anderson, 1995), and schizophrenia (Huron, Danion, Giacomoni, Grange, Robert & Rizzo, 1995). Findings have confirmed the validity of remember/know judgements, providing evidence that subjects can readily distinguish between the two states of awareness, and that these judgements do not simply reflect conventional measures of accuracy or confidence (see Gardiner & Java, 1993, for a review).

However, much of the research that has taken up the procedures used by Tulving (1985) has been related more to dual-process accounts of recognition memory such as that of Mandler (1980). Mandler has described two components of recognition memory; elaboration or encoding of relations between different mental representations which is associated with conscious recollection, and activation of internal features within mental representations which give rise to feelings of familiarity without conscious recollection. This has resulted in a deviation from Tulving's original description of a 'know' response as being related to permanent semantic memory, to that of transient feelings of familiarity relating to some recent encounter that cannot be recollected. Such a definition is more in line with the signal detection models that have been developed to explain the remember/know phenomenon.

Because of the ambiguity regarding how a 'know' response is defined, Conway, Gardiner, Perfect, Anderson and Cohen (1997) have made a further distinction between the different states of memory awareness which may help clarify this issue. When conscious recollection of a particular learning episode is involved in retrieval, the subject is instructed to make a 'remember' response. If, however, the information is simply known and is a part of the person's knowledge, the subject makes a 'know' response. When feelings of familiarity are associated with retrieval without conscious recollection or simply 'knowing', the person makes a 'familiar' response. And when the person cannot retrieve the required information and simply guesses, he or she makes a 'guess' response. Previous research has suggested that participants may have included as a know response, responses that are simply guesses and would have distinguished between the two if that option had been allowed (Strack & Forster, 1995). Therefore, Conway et al's new distinctions should help to elucidate more accurately how the different aspects of memory awareness interact in different situations.

Conway et al (1997) applied these memory awareness concepts which originated in cognitive psychology to an applied learning situation. They were interested in how the different memory awareness states which accompany knowledge access, change over the course of learning a new body of knowledge. They addressed the question of how material processed in a specific episode, such as a university lecture, can over a period of learning become conceptual knowledge virtually free of reference to the details of that specific episode, and discussed this in terms of the development of a schema.

Over several decades, many theories regarding schemas have been discussed. While there are differences among these theories there is some accordance about what the general properties of a schema are considered to be. First, a schema is not a record of all individual experiences of a particular concept, but is something that develops when learning of generalised features occurs across multiple experiences. Second, the schema can represent underlying objects, situations, events, actions, and sequences of event or actions that relate to the particular concept. Finally, schemas do not simply contain individual elements of knowledge but contain a network of interrelations. Ideas about the nature and development of schemas have been applied to many different areas of learning, such as story comprehension (Anderson, 1984, 1994; Rand, 1984), analogical problem solving (Hinsley, Hayes & Simon, 1977) and transfer (Reed, 1993), and instruction in mathematics (Skemp, 1987) and science (Roth, 1990). However, little is known about the particular conditions under which schemas are formed. For example, Gick and Holyoak (1983) presented subjects with a story in conjunction with either summarisation instructions, a verbal statement of the underlying principle, or a diagrammatic representation of the underlying principle. All three manipulations were designed to promote abstraction of a general problem schema or strategy from the single story. It was expected that the subjects would then utilise this schema in a subsequent problem-solving exercise. Irrespective of the type of manipulation, subjects did not abstract a general schema from the story and hence did not transfer the problem-solving strategy to the subsequent problem-solving exercise. So although there appears to be some convergence of ideas regarding the schema there is still uncertainty about the exact nature of the schema and the way in which it develops.

Memory Awareness and Schematisation in Applied Research

Conway et al (1997) applied the memory awareness constructs to a 'real world' learning situation with university students enrolled in first year psychology courses. Students took multiple-choice exams following each lecture or research methods course in the first year program. One of the lecture courses was also later retested. In conjunction with each answer students indicated which state of memory awareness accompanied their answer; recollective experience of a particular episode (remember), just know as part of their knowledge base (know), the answer seems familiar somehow (familiar) or simply guess. For the lecture courses, students who performed best indicated a high proportion of 'remember' responses, while for the research methods courses and the later retest, these students showed a higher proportion of 'know' responses. In addition, there were differences in the accuracy of remember and know responses. When remember was the dominant memory awareness category these responses were the most accurate. However, when remember responses had given way to know responses as the dominant category, the accuracy of these responses declined and know responses were the most accurate. In contrast to this shift, the accuracy of know responses was consistently high,even when know was not the dominant response category.

Conway et al suggest that during the process of learning, knowledge becomes 'schematised'. In the initial stages of learning, knowledge is more likely to be retained in episodic form which students are able to 'remember' specifically. As learning progresses, these memory representations change from being primarily episodic in nature to more conceptual, highly familiar, generalised knowledge which students tend to simply 'know'. However, Conway et al propose that this shift is not an all or nothing phenomenon, and both types of memory and memory awareness can be involved in the students knowledge at the point in time. Similar to Tulving's model of recall and recognition, memory awareness should vary systematically with the degree of 'shift' or schematisation, such that as schematisation develops, remember responses and the role of episodic memory should decrease as know responses and the role of semantic memory increase.

In terms of their research findings, Conway et al propose that the process of schematisation, or the shift from episodic to semantic knowledge (the remember-to-know shift), is dependent on the level of learning obtained, the type of course or instruction, and the retention interval. They suggest that schematisation occurred at a faster rate in the research methods courses because of the practical nature of courses and the fact that they covered fewer topic areas or sub-domains than the lecture courses which tended to involve many different areas that were less interrelated. The remember-to-know shift was not evident in the initial lecture course exams, but apparent later in the re-test of one lecture course with students giving a higher proportion of responses than remember responses. In terms of accuracy, it is suggested that know memory awareness is more accurate even when it is not the dominant response category because it is linked to semantic memory that tends to be more permanent than episodic memory as it contains our conceptual knowledge about the world. In contrast, remember memory awareness is most accurate when it is the dominant response category and accuracy declines as remember responses decline along with the accessibility of episodic memories of specific learning experiences.

Two processes are suggested to be responsible for the remember-to-know shift; a loss of access to episodic details, and the development of some sort of 'conceptual organisation'. Both processes are presumed to result from repeated experiences with the same information but in rather different contexts. Conway et al (1997) explain this as students encountering the same concepts and facts multiple times across lectures and tutorials or laboratory classes where there is a strengthening of the knowledge that is common to the varying episodic representations in memory. It is further postulated that a first step in the schematisation process may be the loss of access to episodic representations. Access to episodic memory declines quite rapidly which may promote or encourage a change to easier access of semantic knowledge. While this conjures up an idea of passive learning, Conway and his colleagues go on to suggest that repetition of experiences (and loss of episodic details) will not in itself result in schematisation, but rather the students need to actively take the opportunity to schematise their knowledge on a conscious level. This notion is compatible with most schema theories, which suggest that the individual actively constructs or organises the information that he or she processes. It is also consistent with more recent research involved in the long-term retention of knowledge taught in school (e.g., Conway, Cohen & Stanhope, 1991, 1992; Semb & Ellis, 1994).

From their review of several studies examining long-term retention (eg. Bahrick, 1984; Bahrick & Hall, 1991; Semb, Ellis & Araujo, 1993), Semb and Ellis (1994) suggested that it is not some simple strengthening of the instructional content which improves long-term retention, but rather qualitative changes in students' memory structures, or schemas, through active learning. For example, from examining student performance in a final year cognitive psychology course, Conway, Cohen and Stanhope (1992) reported that when students were actively involved in learning, completing practical experiments in which they designed their own studies, collected and analysed data, they tended to have more stable long-term memory structures and a higher level of memory performance. Memory for research methodology remained consistently high over retention intervals as long as 125 months, while memory for more specific general facts and names showed a steady decline. Conway, et al (1992) suggest that in the context of school or university, when schemas are formed and retained the organisation of knowledge in memory is generally stable and effective access to the knowledge is maintained.

In contrast to this idea, Bahrick (1984) concluded from his study on the long-term retention of a foreign language (Spanish), that students learn a large number of responses and those which are stored in memory are resistant to forgetting over long periods of time. Remembering is then a result of simple recall of individual responses that had been rehearsed during learning, rather than being mediated or supported by a schema. Neisser (1984) proposed an alternative conclusion to Bahrick's study, which suggests that what students may actually acquire during learning is a schema for Spanish, which contains abstract knowledge relating to the grammar, syntax, phonology and semantics of the language. The process of remembering then involves a reconstruction of knowledge, rather than simply a reproduction or recall of specific responses. Neisser also suggested that while individuals are able to remember very specific instances or experiences, these episodes may actually represent and be supported by the larger, deeper web of knowledge contained in the relevant schema.

Memory Awareness and Schematisation in Cognitive Psychology

Marshall (1995) suggests that when investigating schema development it is important to consider the nature of the pieces of information relevant to the schema that an individual acquires. In her own research on problem-solving schemas, Marshall reports findings that are somewhat consistent with Conway et al's ideas. While not adopting the terms, episodic and semantic, to distinguish between different memory types, Marshall's terms of 'specific' and 'abstract' reflect similar concepts. For example, specific formation involves particular details of examples encountered during instruction, and episodic memory would generally contain details of the time and place of learning. Abstract information involves general features or definitions of different concepts, and semantic memory involves a persons understanding of meaning, or their conceptual and symbolic knowledge.

Using a computer-based instructional system, college students from introductory psychology classes completed a short program involving learning about different types of problem situations, and completing exercises on identifying problems according to whether they 'fit' with one of the problem situations. Students were also interviewed after each session and asked to talk about the situations, recalling the different types and then describing each one as fully as possible. This data was used to construct a cognitive map for each student. A map consists of a set of points, representing the distinct pieces of information given by the student, and links between the points, presenting associations given among the pieces of information.

A main point of the study was to investigate whether students remember more specific information (e.g., particular details of examples given in instruction), or abstract information (e.g., general features or definition of a situation) or whether both types are important in learning. Marshall found that in predicting performance in the identifying problems task, generally those students with higher ratios of abstract to specific information performed more successfully than students with smaller ratios. Marshall also noted that if students had abstract knowledge, they always seemed to use it in preference to giving specific details. However, when prompted for more information they were usually able to provide details to support their abstract knowledge. In contrast, some students first recalled specific details and could not then provide relevant abstract information when prompted. In regard to schema development, Marshall suggests that students may first encode the specific information and then build an abstract network around it. Once the abstract network is formed it becomes dominant, as with subsequent experiences the example is related to the abstract, or generalised information rather than specific details of previous experiences. This process then further strengthens the abstract network. If abstract information is not yet encoded by the student, the specific details remain the most salient elements until students are able to relate new experiences to abstract information they have encoded. After conducting a subsequent study that manipulated the type of information with which students were initially provided, Marshall further suggested that perhaps for either type of information to be encoded fully, the other type also needs to be available. In the absence of any specific examples, it may be difficult for students to process or understand the abstract information, and vice versa.

In a similar vein, Ross and his colleagues (Ross, 1984; Ross, Perkins & Tenpenny, 1990) have developed a reminding-based theory of category learning, where a 'reminding' refers to the recall and use of an earlier episode. They propose that remindings that occur during the learning of a category lead to generalisations. The main assumption behind this theory is that the derivation of general information about a category is not an automatic process; it depends upon the individual experiencing multiple instances and noticing common aspects between instances when remindings occur. Ross et al suggest that while they have demonstrated that general information can be derived from remindings in early learning, the occurrence and use of remindings may continue throughout the learning process. Some studies have shown that the effects of comparing instances do not end with greater learning or expertise. Even in formal domains, experts may have multiple schemas for solving some types of problems, with some of these schemas containing surface feature information (e.g., Hinsley, Hayes & Simon, 1977).

In summary, there has been a recent resurgence of interest regarding memory and states of memory awareness in various areas of research, however the relevance of this topic for student learning has not been fully explored. Conway et al (1997) appear to be the first to investigate memory awareness in the applied student learning environment. In particular, they have identified a possible link between changes in memory awareness during learning and the relative importance of episodic memory in the development of schemas. From a review of both cognitive and educational psychology literature there is some support for the findings reported by Conway and his colleagues. It is evident that in some way, episodic memory plays an important role in the process of acquiring a new body of conceptual knowledge. How and when students experience particular 'episodes' or events, or process specific details seems to affect the schematisation process and the long-term retention of conceptual knowledge. In the university context, this learning process generally begins at the lecture and progresses to the end-of-year assessment and into the long-term. The purpose of the present research is directed towards understanding how schematisation develops across this time course.

In terms of examining memory awareness and schematisation, there are two lines of inquiry. First, drawing from Conway et al's work, it is important to replicate and then further examine the remember-to-know shift in memory awareness over time. The main assumption that underlies Conway et al's conclusions regarding this shift, is that as episodic memories give way to semantic memory, a schema is developing. Hence, the student must have some degree of understanding regarding the particular topic. Conway et al noted that students who achieved higher grades on essay-based examinations showed conceptual organisation of knowledge, whereas students who performed poorly tended to simply 'list' facts and concepts. Therefore, the second line of inquiry involves a more direct measure of the student's understanding. This will be in the form of qualitative data from open-ended type questioning. Student performance will be analysed using the Structure of Observed Learning Outcomes (SOLO) Taxonomy (Biggs & Collis, 1982; 1989) which was developed to measure the quality of student learning from tests requiring understanding of meaning. The lower to middle levels of the taxonomy represent little or no knowledge to knowledge of specific items. The upper levels represent knowledge of several items and the ways in which they are related in an overall framework. It is hypothesised that the dominant form of memory awareness accompanying students recall will depend both on their level of achievement and the type of course being studied. For lecture courses, higher achieving students will perform better because they 'remember' more, whereas higher achieving students in the methods course will initially show more 'know' responses. Over time, higher achievers in the lecture course will show fewer 'remember' and more 'know' responses, whereas for the methods course, the domination of 'know' responses in comparison to the other memory awareness categories will become even more apparent. Student's performance as measured qualitatively should follow this pattern and reflect the dominant response category and its accompanying memory system, either episodic or semantic. If remember responses are dominant, students should score between the low and middle levels of the taxonomy, and if know responses are dominant students should score between the middle and upper levels of the taxonomy. In terms of the accuracy of memory awareness responses, hypotheses follow the findings of Conway et al (1997). When remember is the dominant response category these responses will be the most accurate. As remember responses decline in quantity, so will the level of accuracy. In contrast, know responses should remain consistently high even when know is not the dominant response category. While it also expected that lower achievers will show a similar pattern, the difference between the remember and know response categories are not expected to be as distinct.

Method

The two first year lecture courses in psychology (PY120 and PY130) are semester long (13 lecture weeks) subjects designed to introduce students from a wide range of disciplines to the study of psychology. Either subject may be taken in a semester, one of them covering basic psychological processes from cognitive and biological perspectives and the other subject covering developmental and social processes. Each subject involves two 1 hour lectures per week and a 1 hour tutorial session per week. Assessment includes a mid-semester exam, end-semester exam, one essay assignment and a small lab report.

The third year research methodology course for psychology is a year-long subject designed for students who will complete a 4^th year of study, particularly an honours degree in psychology. It comprises one 2 hour lecture per week in both semester one (14 weeks) and semester two (13 weeks). In addition, students attend a 1 hour tutorial session each week. Assessment in semester one involves an end of term exam including both multiple choice and short answer questions, and several small assignments which involve the reading and analysis of journal articles. In semester two, students are involved in collecting data for two major assignments. These assignments involve writing an experimental report and addressing several issues relevant to the theory, methodology and data collection and the statistical analyses of the data.

For the first testing session, a total of 90 students enrolled in either first year lecture course participated for course credit as part of the research participation component of the first year psychology program (54 were enrolled in PY120 and 36 were enrolled in PY130). These students were selected because of their intent to complete one of the first year courses in semester one and the other in semester two. This would allow the students to complete the second test in the following semester, again as part of the research participation program. Due to natural attrition and changing of enrolment, only 50 of these students participated in the second testing session (30 from the original PY120 group and 20 from the original PY130 group). Students' data were anonymous.

A total of 119 students enrolled in the third year research methods course participated in this study as part of the data collection activities for a class assignment. Each student was identified by a code to preserve anonymity. Complete data with matches in identification codes for all sources of data was obtained for 88 students.

A class test comprising multiple-choice questions (MCQs) was developed for each course in conjunction with the course coordinator (30 questions for first year and 40 questions for third year). There were four alternatives for each question and approximately 30% of questions referred to specific illustrations and examples presented in lectures or as part of required readings. For each question, students were required to circle one of the four alternative answers, along with one of four memory awareness categories. The first year lecture students were also required to made confidence judgements on the first MCQ test. For each question, students were asked to rate from 1 (least confident) to 5 (most confident) how confident they were that they had chosen the correct answer. This task was included so as to determine if, according to alternative views, remember and know judgements simply reflect different levels of confidence in recall.

A short answer question was also included in the class test and required students to write a passage about a particular concept/s from the course. This question was designed as a method of assessing the degree of schematisation of knowledge for each student. The following is an example from the research methods test;

The MCQ and short answer exam was administered twice during the year. The time of testing in this study are comparable to those used by Conway et al (1997). Their lecture courses ran in 6-week blocks and their methods courses were of 12 weeks duration. Students were tested approximately 2-3 weeks after the end of each course. One of the lecture courses was retested 25 weeks after the first class test. In the present study the students in the lecture course were tested after 6 weeks of learning and retested after an interval of 24 weeks. The first test was completed one week after students had sat for a mid-semester exam, they would have recently studied material relevant to the test in this study. The second test was completed approximately 2 months after students had sat for the final course exam. Students in the methods course were tested at the end of the half-year, after 13 weeks of learning and retested after a 9 week interval. Students were made aware prior to the first test that it would be a practice for the end of semester exam and they were advised to have reviewed the course material beforehand. The second test was completed approximately 6 weeks after students had sat for the formal exam.

The MCQ exam and short answer question were presented in a printed booklet. Standard instructions were provided at the beginning of the testing session on an overhead projector at the front of the room. The lecturer/experimenter read out the instructions to the class and answered any questions regarding the procedure. The test took approximately 35 - 40 minutes to complete. First year students were advised that the test would be useful as a practice test for the exam. Third year students were advised that this test was designed for two purposes. First, it provided an opportunity for students to practice for the end of semester exam, and second, it would provide the class with some data on which they would be using for an assignment next semester.

The memory awareness response instructions which were presented to the students followed those used by Conway et al (1997). Students were advised that they may have selected an answer because they had remembered a specific episode from one of the lectures or a textbook. For example, they may virtually 'hear' again or 'see' again the lecturer presenting some item of information. If so, students were to circle 'R' for remember. However, if they 'just knew' the correct and the answer chosen 'stood out' from the other alternatives, students were asked to circle 'K' for know. Alternatively, if they chose an answer because it seemed more familiar than any of the other choices, students were advised to circle 'F' for familiar. Finally, if the answer they chose was simply a guess students were asked to circle 'G' for guess. The following is an example from the third year test.

a. to that condition by chance.

b. to the condition in which participants are representative of people in general.

c. in a fashion that ensures that the independent variable will have a strong impact.

1. to the condition in which participants are all very similar in personality

Results

Results are presented in three main sections; the first year lecture course, the third year research methods course and a comparison of the two courses. In each section, separate areas of student performance will be addressed in turn. First, the results from the MCQ test and memory awareness categories will be examined. Then the short answer question will be discussed in relation to these findings. Analyses of the two courses, individually, involved three main variables which are explained below; memory awareness category (4 levels), test occasion (2 levels) and achievement level (4 levels). These variables were combined in a three-way interaction with test occasion and memory awareness as within-subjects factors and achievement level as a between-subjects factor. This was performed separately for the memory awareness quantity and accuracy data (as outlined below). In comparing the two courses, the variables of course (2 levels), memory awareness category (4 levels) and achievement level (4 levels) were analysed in a three-way design.

The memory awareness data was scored in two ways following those used by Conway et al (1998). First, for the correct answers only, the number of correct answers falling in each of the four memory awareness categories was calculated for each student. This measure is referred to as the 'quantity' of memory awareness and scores sum to 100% for each participant in each test. Second, for all memory awareness judgements the proportion of correct answers from the total of answers given in each memory awareness category was calculated individually for each student. This measure is termed the 'accuracy' of memory awareness.

Students in the lecture course were tested after 6 weeks of learning and retested after a 24 week interval. Students in the methods course were tested at the end of the half-year, after 13 weeks of learning and retested after a 9 week interval. These testing schedules are comparable to those used by Conway et al (1997).

The MCQ test, memory awareness and short answer test data was also examined in terms of achievement level. On the basis of the mean score for the MCQs across both testing sessions, students were divided into four achievement levels; high achievers who performed at +1 or more standard deviations from the overall mean, midhigh achievers who scored between the mean and +1 standard deviation, midlow achievers who scored between -1 standard deviation and the mean and low achievers who performed at -1 or more standard deviations from the overall mean.

Tables 1 and 2 below present the overall mean quantity of correct answers and overall mean accuracy of answers for the MCQ test for both courses and test occasions across achievement levels. As there was no difference between the two first year courses in any of the analyses, the data was combined. Before turning to the first section of the results, the confidence ratings taken from test one of the first year students are examined.

Mean confidence ratings were calculated for each memory awareness category separating confidence ratings for each student into those accompanying a 'remember' judgement, those accompanying a 'know' judgement, and so on. The mean ratings (with standard deviations in parentheses) are as follows; remember = 3.22 (.6246), know = 3.13 (.6055), familiar = 2.55 (.5437) and guess = 2.48 (.9718). Remember and know confidence ratings were not significantly different, nor were familiar and guess confidence ratings. However, remember and know judgments were significantly more confident than familiar and guess judgments, as indicated by the significant difference between the know and familiar confidence means, t(79) = 8.228, p<.001. These results suggest that remember and know judgments are not simply a reflection of differing levels of confidence, with a remember response being more confident than a know response, but are valid distinctions between different types of memory awareness (cf. Gardiner & Java, 1993).

Table 1.

Table 2.

Note. For all questions chance is .25, and numbers in bold are significantly above chance, p<.05.

The overall mean number of correct answers (with standard deviations in parentheses) for the first test was 22.3 (2.89) and for the second test was 22.08 (2.71). There was no significant difference between the two tests. Test occasion, achievement level and memory awareness were combined in a three-way analysis of variance (ANOVA) for both the memory awareness quantity and accuracy data, with test occasion and memory awareness as within-subjects factors and achievement level as a between-subject factor. The following analyses are computed on the data for students who participated in both testing occasions (50 students). Results are presented for memory awareness quantity first, followed by those for memory awareness accuracy.

The memory awareness categories are dependent upon one another and as mentioned previously, the quantity data for each category when added together equals 100%. Therefore, main or interactive effects that collapse over memory awareness category for the quantity measure cannot be assessed and will not be reported. In the analyses conducted for test occasion, memory awareness quantity and achievement level, there was no significant three-way interaction. However, there was a significant main effect of memory awareness category, F(3,135)=35.189, p<.001. Overall, there were more remember responses than know, familiar and guess responses, respectively. Students made more remember responses than know responses, t(99)=5.622, p<.001, and more familiar than guess responses, t(99)=3.412, p<.001. There was no difference between the amount of know and familiar responses. There was also a significant two-way interaction between test occasion and memory awareness category, F(3, 138) = 9.284, p<.001. In order to investigate this interaction, one-way ANOVA's were conducted separately for each of the memory awareness categories. There was a reliable main effect of test occasion on remember quantity, F(1,98) = 10.429, p<.05, with students showing more remember responses on the first test occasion than on the second test occasion. In contrast, students made more familiar responses on test two than on test one, F(1,98) = 5.606, p<.05. There were no significant differences between testing occasions for know quantity and guess quantity. These results provide only partial support for the predicted shift from remember responses on test one to know responses on test two. As can be seen in Table 1, there is a large decrease in remember responses and a small, unreliable, increase in know responses.

Figure 1 shows the mean accuracy for each memory awareness category across achievement level and test occasion. All mean accuracy responses were reliably above chance, except for the guess responses for low achievers in the first test. Analysis of variance revealed a marginally significant three-way interaction between test occasion, achievement level and memory awareness category, F(9,138)=1.710, p<.09. As can be seen in Figure 1, know and remember responses increased in accuracy from test one to test two but really only for middle and low achieving students. On test one, high achievers' remember and know responses were more accurate than the other achievement levels, while all levels of achievement were performing at more similar levels on test two.

Figure 1. Accuracy of memory awareness categories by achievement level and test occasion

for the lecture course.

There was also a significant main effect of memory awareness category on accuracy, F(3, 138)=50.816, p<.001. Irrespective of test occasion, remember responses were more accurate than know responses, t(99)=2.603, p<.01, know responses were more accurate than familiar responses, t(99)=5.798, p<.001, and familiar responses were more accurate than guess responses, t(99)=4.009, p<.001. The interaction between test occasion and memory awareness category was also significant, F(3,138)= 2.935, p<.05. To examine this effect one-way ANOVAs were conducted separately for each of the memory awareness categories. There was a reliable main effect of test occasion on remember accuracy, F(1,98) = 5.903, p<.05, and know accuracy, F(1,98)= 4.898, p<.05. Both of these memory awareness categories increased in accuracy from test one to test two. These results are actually inconsistent with those reported by Conway et al (1997), who found that memory awareness accuracy for the first year course which was retested, reflected an overall decrease in remember accuracy but no systematic differences in know accuracy.

The short answer question was scored using the Structure of Observed Learning Outcomes (SOLO) Taxonomy (Biggs & Collis, 1982;1989) which was developed to measure the quality of student learning from tests requiring understanding of meaning. Table 3. illustrates each level of the taxonomy as originally defined by Biggs and Collis and as defined for the present study.

Students' answers were scored according to above system. Where an answer was very close to the next level, the student was awarded half marks. For example, a student may have demonstrated knowledge of several relevant aspects of the topic and attempted to integrate most of them together. However, on the whole, the student's answer could not be classified as 'relational'. In such a case, the student would be awarded 3.5.

Table 3.

The mean SOLO score (with standard deviations in parentheses) for the short answer question in test one was 2.93 (0.55), and for the short answer question in test two was, 3.12 (0.58). A one-way ANOVA was performed on this data with test occasion as the grouping variable, however no significant effect was found. The distribution of scores is presented in the following table.

Table 4.

As can be seen from the distribution of scores, the majority of students on both testing occasions scored between 3 and 3.5. However, while no one scored above 3.5 on the first test, a small number of students scored a 4 or 4.5 on the second test. This trend is consistent with expectations according to the memory awareness quantity analyses, where there was a significant decrease in remember responses with a small (not significant) increase in know responses, and hence a small increase in the level of semantic memory used.

The achievement levels calculated on MCQ performance were also used in analyses of SOLO scores. One-way ANOVAs were performed on the SOLO scores for test one and test two with achievement level as the grouping factor. The mean SOLO scores across each achievement level for test one were; high (3.13), midhigh (3.12), midlow (2.79) and low (2.65). For test two, the mean SOLO scores for each level of achievement were; high (3.42), midhigh (3.18), midlow (3.05) and low (2.75). There was no significant effect of achievement level for test one, however this effect was significant for test two, F(3,46)=2.861, p<.05. Post hoc comparisons using Tukey's HSD showed that high achievers scored better than all other levels, p<.05, and midhigh achievers and midlow achievers scored better than low achievers, p<.05. These results are consistent with the memory awareness quantity data (figure 3) in which it appears that higher achievers show a remember-to-know shift, but low achievers do not alter substantially from test one to test two.

The overall mean number of correct answers (with standard deviations in parentheses) for the first test was 28.65 (3.48) and for the second test was 28.31 (3.86). There was no significant difference between the two tests. However, overall performance on the MCQ test was correlated with students' performance on the formal end of semester exam (r = .60, p<.05). Identical analyses to those presented above for the first year students were performed on complete data with matches in identification codes for 88 students.

Analyses of variance revealed a marginally significant three-way interaction between test occasion, achievement level and memory awareness quantity, F(9,249)=1.781, p<.07. As can be seen in Figure 2, in general the amount of know responses increase from test one to test two, but this is more apparent for the high and middle achievers than the low achievers. Overall, remember, familiar and guess responses all decrease in quantity from test one to test two, however low achievers made the most familiar responses on both tests and these amounts did not really alter from test one to test two. A reliable main effect of memory awareness category was also revealed, F(3,249)=81.226, p<.001. Overall, there were more know responses than remember responses, familiar and guess responses, respectively. Students made significantly more know responses than remember responses, t(175)=08.354, p<.001, more familiar than guess responses, t(175)=10.766, p<.001, but the difference between remember and familiar responses was not significant.

Figure 2. Quantity of memory awareness categories by achievement level and test occasion

There was a significant two-way interaction between memory awareness quantity and achievement level, F(9,249)= 3.533, p<.001. Post hoc comparisons using Tukey's HSD showed that irrespective of test occasion, high achievers made more know responses than midhigh achievers and low achievers, p<.05. However, low achievers made more familiar responses than high achievers, p<.01. Analyses also revealed a significant two-way interaction between memory awareness and test occasion, F(3, 249) = 41.695, p<.001. Separate analyses conducted for each of the memory awareness categories revealed a reliable main effect of test occasion on remember quantity, F(1,174) = 4.597, p<.05, with students showing more remember responses on the first test occasion than on the second test occasion. This trend was also present for the familiar quantity, F(1,174) = 11.221, p<.001 and guess quantity, F(1,174) = 31.596, p<.001. However, for the know quantity, students showed fewer know responses on the first test occasion than on the second test occasion, F(1,174) = 57.585, p<.001. These findings are consistent with Conway et al's (1997) results for their research methods courses. Unlike their lecture courses which initially showed a predominance of remember responses, the research methods courses showed more know responses on the first (and only) testing occasion. Following from their hypotheses, it can be presumed that although the research methods courses already show a predominance of know responses, this trend would become more pronounced at subsequent testing occasions, reflecting further learning and schematisation of knowledge.

All mean accuracy responses were reliably above chance, except for the guess responses in the second test (see Table 2). Analyses of variance between test occasion, achievement level and memory awareness accuracy revealed no significant three-way interaction, and no significant two-way interactions. There was a significant main effect of memory awareness, F(3,261) = 4.223, p<.05. As can be seen from Table 2, accuracy remained fairly consistent across the two tests, with know responses being more accurate than remember responses, t(175)=-4.764, p<.001, remember responses being more accurate than familiar responses, t(175)=5.992, p<.001, and familiar responses being more accurate than guess responses, t(175)=6.974, p<.001. These accuracy means for the memory awareness categories follow a similar trend to the mean quantity data and are also consistent with the accuracy results of Conway et al (1998) for their research methods courses.

Overall performance on the short answer question was correlated with both the formal exam results (r=.43, p<.05) and overall performance on the MCQ test (r=.32, p<.05). The mean SOLO score (with standard deviations in parentheses) for the short answer question in test one was 2.81 (0.57), and for test two was, 3.20 (0.69). A one-way ANOVA was performed on this data, and a significant main effect of test occasion was found, F(1,174)=14.941, p<.001. Overall, students scored higher on the second test than they scored on the first test. The distribution of scores is presented in the following table.

Table 5.

As can be seen from this distribution, in test one students tended to score between 2 and 3, while in test two students were more likely to score between 3 and 4. No student scored above 4 on the first test, whereas a small number of students did score 4.5 on the second test. This reflects the results in the above section for the test occasion by memory awareness quantity analyses. While students made more correct know answers in test one as well as in test two, the difference between their know responses and the other categories was more marked on the second test and hence an increase in use of semantic knowledge.

One-way ANOVAs were also performed on the SOLO scores for test one and test two with achievement level as the grouping factor. The mean SOLO scores across each achievement level for test one were; high (3.13), midhigh (2.63), midlow (2.70) and low (2.50). For test two, the mean SOLO scores for each level of achievement were; high (3.68), midhigh (2.93), midlow (2.77) and low (2.69). There was a significant effect of achievement level for test one, F(3,84)=3.525, p<.05 and test two, F(3,84)=5.728, p<.001. Post hoc comparisons (Tukey's HSD) revealed that for test one, high achievers scored better than midhigh p<.06, and low achievers, p<.05. For test two, high achievers scored better than all other achievement levels, p<.05.

As suggested by Conway et al (1997), the development of schematisation is most likely dependent upon the students' ability and level of achievement but also on the type of course. According to their findings, research methods courses are more likely to show the remember-to-know shift earlier on in learning in comparison to lecture type courses. To compare performance between the lecture and methods course, a three-way ANOVA was performed separately on the test one and test two data with course and achievement level as between-subjects factors and memory awareness (quantity and accuracy) as a within-subjects factor. Analyses was split into test one and test two for two main reasons. First, Conway et al did not retest any of the research methods courses and only one of the lecture courses. Therefore, in order to compare our findings with those reported by Conway et al, it is appropriate to examine the first test separately. Second, it would not be possible to include all data collected at both test occasions in a combined analyses as not all students completed both tests. Complete data for 114 third year students(methods course) and 90 first year students (lecture course) was used for test one analyses and complete data for 88 third year students and 50 first year students was used for test two analyses. Results are presented first for test one, followed by test two.

There was no significant three-way interaction between course, achievement level and memory awareness category. However, analyses did reveal several other significant effects. There was a significant main effect of memory awareness category, F(3,387)=40.679, p<.001. Overall, students made more remember responses than know responses, t(203)=2.283, p<.05, more know than familiar responses, t(203)=2.857, p<.01, and more familiar than guess responses, t(203)=8.393, p<.001. There was also significant two-way interaction between course and memory awareness category, F (3,561)=34.004, p<.001. Separate one-way analyses of the memory awareness categories showed a significant main effect of course on remember quantity, F(1,199)=57.763, p<.001, know quantity, F(1,199)=17.465, p<.001 and familiar quantity, F(1,199)=19.942, p<.001. The third year/methods students made fewer remember responses than the first year/lecture students but made more know and familiar responses. Analyses also revealed a significant interaction between achievement level and memory awareness category, F(9,387) = 30.193, p<.05. Separate analyses revealed a significant main effect of achievement level on quantity for know responses, F(3,134)=2.679, p<.05 and familiar responses, F(3,134)=4.256, p<.01. Post hoc comparisons (Tukey's HSD) showed that high achievers made more know responses than low achievers, p<.05, but they made fewer familiar responses than low achievers, p<.01.

Analyses showed no significant three-way interaction between course, achievement level and memory awareness category. However, there was a significant main effect of memory awareness category, F(3,561)=132.127, p<.001. Overall, there was no difference between accuracy of know and remember responses, however, remember responses were more accurate than familiar responses, t(137)=6.346, p,.001, and familiar responses were more accurate than guess responses, t(137)=6.116, p<.001. There was also a significant interaction between course and memory awareness category, F(3,561)=3.665, p<.05. In the separate analyses of memory awareness categories, there was a reliable main effect of course on remember accuracy, F(1,199)=9.474, p<.01, and guess accuracy, F(1,199)=4.311, p<.05, but was only approaching significance for know accuracy, F(1,199)=3.082, p<.08 and familiar accuracy, F(1,199)=2.959, p<.08. Third year/methods students were less accurate in their remember, familiar and guess responses than the first year/lecture students, but their know responses were more accurate.

Analyses revealed a significant three-way interaction between course, achievement level and memory awareness category, F(9,387)=1.812, p<.06. As illustrated in Figure 3, it can be seen that the type of course determines which memory awareness category is more likely to have be involved in students MCQ performance. For the lecture course (first years), remember is the dominant response, whereas know is the dominant response category for the research methods course(third years). This effect of course is also a function of the level of achievement attained. For the lecture course, the amount of remember responses declines with lower levels of achievement, whereas for the methods course, it is the amount of know responses which decrease along with a decrease in achievement. In conjunction with these systematic difference are similar effects upon familiar responses, which increase with decreasing levels of achievement for both the lecture and methods courses.

Figure 3. Mean quantity correct of memory awareness categories by

achievement level and course for test two.

There was a significant main effect of memory awareness category, F(3,387)=54.646, p<.001. Overall, students made more know responses than remember, familiar and guess responses, respectively. There was a significant difference between the quantity of know and remember responses, t(137)=-5.454, p<.01, and between the quantity of familiar and guess responses, t(137)=7.413, p<.001. A significant interaction between memory awareness category and course was also found, F(3,387)=41.381, p<.001. Separate analyses of memory awareness categories found a reliable main effect of course on remember quantity, F(1,136)=25.616, p,.001, know quantity, F(1,136)=78.960,P<.001 and guess quantity, F(1,136)=28.768, p<.001. The third year/methods students made fewer remember and guess responses than the first year/lecture students, but made more know responses. There was also a significant interaction between achievement level and memory awareness category, F(9,387)= 2.557, p<.01. Separate analyses showed a significant main effect of achievement level on know responses, F(3,134)=3.008, p<.05 and familiar responses, F(3,134)=7.658, p<.001. Post hoc comparisons (Tukey's HSD) indicate that high achievers made more know responses than low achievers, p<.05, and both high and midhigh achievers made less familiar responses than low achievers, p<.01. While the three-way interaction was not significant for test one, as Conway et al (1997) reported in their study, the above results for test two are consistent with Conway et al's findings. 

The three-way interaction between course, achievement level and memory awareness category was not significant, however analyses did reveal a significant main effect of memory awareness category, F(3,402)=97.733, p<.001. Overall, there was a significant difference between all memory awareness categories. Know responses were more accurate than remember responses, t(137)=-2.594, p<.01. Remember responses were more accurate than familiar responses, t(137)=7.444, p,.001, and familiar responses were more accurate than guess responses, t(137)=5.001,p<.001. There was also a significant interaction effect of memory awareness category by course, F(3,405)=5.970, p<.001. Third year/methods students were less accurate than the first year/lecture students on remember responses, F(1,136)=20.872, p,.001, familiar responses, f(1,136)=3.484, p<.06, and guess responses, F(1,136)=12.906, p<.001. Conway et al (1997) also reported a three-way interaction effect for their accuracy data. Even though this interaction is not significant for the present study, results are similar although the accuracy of remember and know responses were slightly more accurate for both course and across achievement levels. Again, this may be due to a low level of statistical power because of the small number of students in each achievement level for the lecture course.

General Discussion

Overall, the results of the present study follow a similar pattern to those hypothesised according to the findings reported by Conway et al (1997). First, it was expected that the dominant form of memory awareness accompanying students recall will depend both on their level of achievement and the type of course being studied. As hypothesised for lecture courses, students showed more remember memory awareness on the first test with a declining quantity of remember responses as achievement level decreased. However, on the second test students still made more remember responses overall rather than showing a shift to know as the dominant response category. Only high achievers showed the expected remember-to-know shift. For the methods course, results were as hypothesised. On the first test, students showed more know memory awareness, with a decline in quantity of know responses as achievement level decreased. This pattern was replicated on the second test, but with a more distinct difference between the quantity of know responses and the other memory awareness categories.

Student's performance on the short answer question followed the above patterns of memory awareness quantity (refer to Tables 3 and 4). On both test one and test two of the lecture course, students were most likely to score between the lower and middle SOLO levels. These results indicate that students were able to recall specific items of information but this knowledge was not contained in an overall conceptual framework of interrelations among these items. Such information is more representative of episodic type memories rather than the conceptual knowledge contained in semantic memory. Performance across achievement levels was also consistent with the memory awareness quantity results, with decreasing SOLO levels as achievement level deceased. Moreover, where high achievers showed more know responses on test two they also scored between the middle and first upper SOLO level on test two, indicating that these students were beginning to show a conceptual framework of knowledge instead of just specific items of information. For the methods course, students were most likely to score in the middle SOLO level on test one, showing knowledge of specific items of information and some attempt to link these items together. On the second test, performance is more consistent with the dominance of know memory awareness. Approximately 50% of students scored above the middle SOLO level, which is indicative of having knowledge of specific items within a conceptual framework of interrelations.

In terms of memory awareness accuracy, results were generally as hypothesised. For the lecture course, where remember was the dominant memory awareness category, remember responses were the most accurate responses. However, for both test one and test two high achievers showed similar high levels of accuracy for remember and know responses. On test two, high achievers made more know responses than remember responses, but remember responses were still more accurate than know responses. For the methods course where know was the dominant response category for both test one and test two, know responses were also the most accurate responses. Similar to the lecture course, high achievers showed high levels of accuracy for both know and remember responses on test one but this pattern changed for test two, where know responses were the most accurate. Finally, know responses were consistently high in accuracy even when know was not the dominant response category. This was evident for the methods course on test two where know and familiar memory awareness were similar in levels of quantity but know responses were more accurate. A more detailed examination of the above results in relation to previous research and their implications for teaching and learning are discussed in the following sections.

For the first test, these results were consistent with the findings of Conway et al (1997). Students in the lecture course performed well because they were able to 'remember' more from specific learning episodes. However, students in the methods course performed well because they 'just knew' more information. As Conway et al suggested, one possible explanation for this difference is due to the differences between these two courses. The lecture course is made up of two first year psychology subjects that aim to introduce students from a wide range of disciplines to the different areas of psychology. There are two lecture streams which present material from two major areas (e.g. cognitive and physiological) and several different topics are covered over the semester. Often a different topic is covered each week. In contrast, the methods course is a year-long research methodology subject for third year students who are aiming to complete a fourth year (i.e. honours) of study in psychology. While different concepts are covered from week to week, these concepts are linked together and often students encounter each concept across multiple occasions and in different contexts. As suggested by Conway et al, the two processes involved in schematisation, loss of episodic details and emergence of conceptual organisation, may both result from repeated encounters with the same information but in rather different contexts. Therefore, it may be that courses such as research methodology are more conducive to schematisation of knowledge, and students are more likely to develop schemas earlier in the learning process. Another possible factor that may have produced the differences in memory awareness across the two courses is the time of testing. Students in the lecture course were tested after 6 weeks of learning, while students in the methods course were tested after a full semester, 13 weeks of learning, in order to be comparable to the lecture courses in Conway et al's study. Therefore, it is likely that students in the methods course had more opportunities to learn and/or encounter the information they were tested on. As suggested by our initial review of the psychological and education literature, learning and schematisation does not just occur naturally, but involves the student taking advantage of different learning opportunities and experiences.

It is also possible in our study that the difference in year level may have also contributed to the differences in memory awareness. Third year students in the methods course were completing a major in psychology and most intended to go on to complete a fourth year of study in psychology. This particular course also operates on a quota entry system which limits the number of students enrolled (between 100 and 130) and also the level of achievement. Students in this course would have received an overall above average level of achievement across their first and second years of study in psychology. In contrast, the first year course caters for over one thousand students from a wide range of disciplines and years of study. Many students complete the course as an elective in the first, second or third year of university study. Therefore, there is a wider range of achievement level than in the methods course, and students are not only focussed on studying psychology. It is possible that third year students showed earlier schematisation of knowledge because of a higher level of ability and also a greater level of motivation and interest in studying psychology. This is also a possible factor in Conway et al's study where only students majoring in psychology took the methods courses whereas these students made up less than half of the number of students in the lecture courses.

The results for test two were less consistent with those reported by Conway et al for students in the lecture course that was retested. In their study, these students showed a remember-to-know shift from test one to the retest, whereas in the present study only the high achievers showed this shift. Overall, remember was still the dominant memory awareness category for test two. There are several possible reasons for this inconsistency. First, the students retested by Conway et al were invited to retake any of the class tests as part of an optional revision day. Only 52 of the original 201 students made use of this opportunity, the group consisting entirely of single and joint honours psychology students with middle to high levels of achievement. For single honours students, psychology subjects make up two thirds of their degree, and joint honours students complete a third of their subjects in psychology. In contrast, students enrolled in the lecture course for the present study were from a variety of disciples and were not yet committed to majoring in psychology. Moreover, students' level of achievement was not restricted to middle and high levels, but included low, middle and high achievers. With these differences in achievement and commitment to the discipline, it is not surprising that most students in the lecture course did not show the remember-to-know shift as reported by Conway et al. It is also possible that students retested by Conway et al were more motivated to learn and study around the time of retest as they would be tested on similar material in their forthcoming summer examinations. Students from the university in the present study do not sit for similar end-of-year examinations, but are only examined either mid-way through the course or at the end of the course. Therefore, as students in the present study had already sat for their final course exams, they had most probably 'moved on' and were concentrating on the subjects they were currently studying. They were only taking the second test as part of the research participation program and received course credit for their time.

Conway et al (1997) did not retest their research methods courses and therefore we have no comparison group for the present methods course on test two. However, our results are consistent with the ideas developed by Conway et al regarding the changes in memory awareness over time and the accompanying development of schematisation. Although students had already sat for their final exam, the knowledge they had gained over the first semester was continually being encountered in new situations and was often a necessity in order to understand new concepts introduced in the second semester. So, although students were unlikely to have 'studied' for the second test, the material which the test covered was still be utilized quite frequently during lectures and tutorials. Given this, the increase in know responses from the first test to the second test is what would be expected as schematisation of knowledge would still be developing in order to 'fit' in the new situations and related concepts that students were currently encountering. Interestingly, on test one high achievers showed a larger number of know responses in comparison to the other achievement levels who showed similar levels of know responses. However, on test two the middle and low achievers showed a greater increase in know responses in comparison to the high achievers. The process of schematisation appears to have been quicker for the high achievers than the middle and lower achievers, who were beginning to show more similar levels of know responses to those of the high achievers on test two.

On the first test for the lecture course, results were consistent with those of Conway et al (1997), and followed the same general pattern as the quantity of memory awareness categories. Remember was the most dominant memory awareness category and these responses were also the most accurate. Know responses were not as accurate as remember responses but were quite high in comparison to the familiar and guess responses (These results are not surprising considering the mean confidence ratings given for responses in test one. Know and remember responses were both associated with more confidence than familiar and guess responses. There was also no difference in confidence rating between remember and know responses). The pattern for high achievers was somewhat different from this overall finding, in that high achievers showed similar levels of accuracy for remember and know responses, both being over 90% accurate. However, on the second test the middle and low achievers 'caught up', showing similarly high levels of accuracy for both remember and know responses, with the difference between the achievement levels being less distinct. Of interest is the discrepancy between the results found by Conway et al between test one and retest, and those found in the present study. Conway et al found that remember responses decreased in accuracy from test one to retest while know responses remained consistent across test occasions. The present study however, found that both remember and know responses increased in accuracy from test one to test two. One major reason for this pattern may be the fact that, unlike Conway et al's study, there was no remember-to-know shift in the quantity of memory awareness. Therefore, remember was still the dominant response category and so according to Conway et al's suggestions, remember responses should also be the most accurate. This is because when memory for specific learning experiences is good, students are more confident about recalling these experiences and so they are more likely to make a remember response. However, know responses also increased in accuracy and this may be because although there was no overall remember-to-know shift, know responses had increased in quantity from test one to test two. The difference between the amount of remember and know responses had become less distinct which may indicate that the remember-to-know shift was beginning to take place.

For the methods course, results were as expected for both tests and followed the pattern of quantities for the memory awareness categories. For both test one and test two, know was the dominant response category and know responses were the most accurate responses. While there were no significant differences between the two testing occasions, remember responses did decrease in accuracy to a small degree while know responses remained at the same level of accuracy. This trend is consistent with the pattern of results for the quantity of remember and know responses. The quantity of remember responses decreased from test one to test two, but know responses increased in quantity from test one to test two. Such results are supportive of Conway et al's suggestion that a high level of accuracy occurs for know responses regardless of the quantity because this information is mediated by schematic knowledge contained in semantic memory. In contrast, remember responses only attain high levels of accuracy when they are more prevalent. Conway et al suggest that part of the schematisation process involves a loss of access to episodic details, and it is the falling accuracy and quantity of remember responses that reflects this loss of accessibility to memories of specific learning experiences.

By including this qualitative measure, the present study was able to extend and attempt to further support the ideas proposed by Conway et al (1997) regarding changes in memory awareness and the process of schematisation. Overall, the results for the short answer question followed the expected pattern according to the findings for the memory awareness categories. For the lecture course the majority of students on both testing occasions scored around the middle SOLO level, with no students scoring above the middle level on test one and very few scoring above this level for test two. Overall, there was also no remember-to-know shift in memory awareness from test one to test two. If the remember-to-know shift does reflect the development of learning and schematisation, then it would be expected that there would be an accompanying shift from the low and middle SOLO levels to the upper SOLO levels; a move from being able to recall at least two concepts but without knowledge of the interrelationships among these concepts, to being able to show knowledge of several concepts and their interrelationships together in a meaningful way. The results for the high achievers support this proposal. In conjunction with making more know responses than remember responses on test two, high achievers also increased in performance from the middle SOLO level to above the middle SOLO level across the two testing occasions.

There was also support for the link between changes in memory awareness and schematisation in the results for the methods course. While students made more know responses than remember responses on both test one and test two, this difference was more marked on the second test. Following this pattern, while the majority of students scored above the middle SOLO level on test one, this number of students increased on test two. Again, high achievers showed this accordance between changes in memory awareness quantity and SOLO scores to a greater degree than the middle and lower achieving students.

The remember-to-know shift and accompanying schematisation of knowledge is what educators surely hope to occur in their students. It seems that for students to perform better, they need to engage or use episodic memory as much as possible in early learning. The ability to do so may reflect other individual differences such as motivation, study and learning strategies, general memory and listening comprehension ability. Further research is being undertaken by the authors of this study in order to investigate the role that such factors play in the process of schematisation and associated changes in memory awareness.

Whether the ability to preserve and use episodic memories early in learning is something that is fairly stable over time and within the individual, it is possible to change the way a course is structured and delivered. The speed at which all students made the remember-to-know shift in the methods course may be largely due to the type of course and the learning experiences encountered during the course. Students were actively involved on several occasions in collecting data for class assignments and assessment also included several small assignments that involve the reading and analysis of journal articles. These assignments were also 'recycled' where the student was required to rewrite any answers that did not show an adequate level of knowledge and understanding. Studies have shown that when students are more actively engaged in learning and exploring the course material, performance is better and knowledge is more likely to be retained over a long time (e.g., Conway et al, 1991;1992, MacKenzie & White, 1982). For example, MacKenzie & White examined the effects of three different programs for learning high school geography. One group of students completed a program that included pictures, worked examples, indications of relevance of information, and transfer of verbal proposition to maps and diagrams. Another group completed this program plus a tradition fieldwork excursion in which students were given an explanatory field guide designed to reinforce content in the first learning program. The third group also completed the first learning program in conjunction with a 'processing' fieldwork excursion, involving a worksheet with tasks to complete; observing, sketching, recording and answering questions. These students also participated in activities such as walking through the mud of a mangrove shore, testing foliage for salinity and wading in the sea. The mean relative loss of knowledge over a 12 week interval for the processing excursion group was 10%, whereas the loss for the tradition excursion group and the learning program only group was 41% and 48%, respectively. Mackenzie & White suggested that the processing excursion provided easily recalled episodes that interlinked with the general content being taught and that the active processing experience aided students in generating meaning for this content.

While it is not always feasible to organise such exciting and adventurous tutorial activities as those above, attempting to make learning experiences as salient as possible should perhaps be what we aim to provide in each lecture and tutorial. For example, simply by choosing a 'real life' scenario to explain a concept will generally make the example more interesting, enlightening and easier to remember. In a preliminary study conducted by the authors, several small groups of students were interviewed and asked about what things they are likely to remember from a lecture. The overwhelming majority of responses were to do with funny or anecdotal stories, real life scenarios relating to the type of occupational area they intended to work in, or simply a scenario that had some personal relevance for them. Most students reported that these types of learning episodes tended to 'stick' with them throughout the course and often helped when studying for an exam.

Another important consideration is the timing of assessment during the course. If students only sat for examinations half way through a course, it is most likely that at this point schematisation of knowledge would not have occurred or only be in the very early stages of development. If there is no further examination at the end of the course, learning may cease at the last lecture and schematisation of knowledge may never occur. This is particularly relevant for students who are not majoring in the subject area or do not intend to pursue further study in the area. When assessment is completed there is no formal requirement to continue learning and studying and it is likely that knowledge will not continue to be schematised and therefore represented in a more durable form. Furthermore, if the state of memory awareness is still predominantly 'remember' then these episodic memories will be prone to rapid loss if the student does not have further learning experiences with this information. Such a scenario is probable for the majority of students in the lecture course tested for this study. On the second testing occasion, after the completion of the subject, students generally showed a decrease in remember responses and no significant increase in know responses. It is likely that any further schematisation of knowledge will not occur for most of these students, especially those who only completed the course as an elective. In the preliminary study conducted by the authors where groups of students were interviewed, many students indicated that they preferred having assessment staggered throughout a course. They felt that it was important for them to get feedback on their level of understanding as early as possible so they could then address any problems while they were still learning. The students also felt that having several different modes and times of assessment helped them to learn the material more quickly and also form more understanding and conceptual knowledge around the concepts presented in class. Perhaps such insights can help to improve the way in which courses are structured and delivered and hence, our students' learning.

References

Anderson, R.C. (1994). Role of the reader's schema in comprehension, learning and memory. In R.B. Ruddell, M. Rapp Ruddell and H. Singer (Eds.), Theoretical models and processes of reading. Newark, DE: International Reading Association.

Bahrick, H.P. (1984). Semantic memory content in permastore: 50 years of memory for Spanish learned in school. Journal of Experimental Psychology: General, 113, 1-29.

Bahrick, H.P., & Hall, L.K. (1991). Lifetime maintenance of high-school mathematics content. Journal of Experimental Psychology: General, 120, 20-33.

Bartlett, F.C. (1932). Remembering: A study in experimental and social psychology. Cambridge: Cambridge University Press.

Biggs, J.B. & Collis, K.F. (1982). Evaluating the quality of learning: The SOLO taxonomy. New York: Academic Press.

Biggs, J.B. & Collis, K.F. (1989). Towards a model of school-based curriculum development and assessment: using the SOLO taxonomy. Australian Journal of Education, 33, 149-161

Brewer, W.F. (1992). Phenomenal experience in laboratory and autobiographical memory. In M.A. Conway, D.C. Rubin, H. Spinnler, & W.A. Wagenaar (Eds.), Theoretical perspectives on autobiographical memory (pp. 31-51). Dordecht, NL: Kluwer.

Conway, M.A., Cohen, G.M., & Stanhope, N. (1991). On the very long-term retention of knowledge acquired through formal education: Twelve years of cognitive psychology. Journal of Experimental Psychology: General, 120, 1-22.

Conway, M.A., Cohen, G.M., & Stanhope, N. (1992). Very long-term memory for knowledge acquired at school and university. Applied Cognitive Psychology, 6, 169-189.

Conway, M.A., Collins, A.F., Gathercole. S.E., & Anderson, S.J. (1996). Recollections of true and false autobiographical memories. Journal of Experimental Psychology: General, 125, 69-95.

Conway, M.A., Gardiner, J.M., Perfect, T.J., Anderson, S.J., & Cohen, G.M. (1997). Changes in memory awareness during learning: The acquisition of knowledge by psychology undergraduates. Journal of Experimental Psychology: General, 393 - 413.

Donaldson, W. (1996). The role of decision processes in remembering and knowing. Memory and Cognition, 24, 523-533.

Duker, S. (1968). Listening Bibliography. New York: Academic Press.

Gardiner, J.M., & Java, R.I. (1991). Forgetting in recognition memory with and without recollective experience. Memory and Cognition, 19, 617-623.

Gardiner, J.M., & Java, R.I. (1993). Recognising and remembering. In A.F. Collins, S.E. Gathercole, M.A. Conway, & P.E. Morris (Eds.), Theories of memory (pp. 163-187). Hillsdale, NJ: Lawrence Erlbaum Associates.

Gick, M. & Holyoak, K.J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15, 1-38.

Hinsley, D., Hayes, J., & Simon, H. (1977). From words to equations: Meaning and representation in algebra work problems. In M. Just & P. Carpenter (Eds.), Cognitive processes in comprehension (pp.89-106). Hillsdale, NJ: Lawrence Erlbaum Associates.

Hirshman, E., & Master, S. (1997). Modelling the conscious correlates of recognition memory: Reflections on the remember-know paradigm. Memory and Cognition, 25, 345-351.

Huron, C., Danion, J.M., Giacomini, F., Grange, D., Robert, P., & Rizzo, L. (1995). Schizophrenia impairs recognition memory with, but not without, conscious recollection. American Journal of Psychiatry, 152, 1737-1742.

Jacoby, L.L., (1991). A process-dissociation framework: Separating automatic from intentio memory. Journal of Memory and Language, 30, 513-541.

Jacoby, L.L., & Dallas, M. (1981). On the relationship between autobiographical memory and learning. Journal of Experimental Psychology: General, 110, 306-340.

MacKenzie, A.A., & White, R.T. (1982). Fieldwork in geography and long-term memory. American Educational Research Journal, 19, 623-632.

Mandler, G. (1980). Recognising: the judgement of previous occurrence. Psychological Review, 87, 252-270.

Marshall, S.P. (1995). Schemas in problem solving. New York: Cambridge University Press.

Neisser, U. (1984). Interpreting Harry Bahrick's discovery: What confers immunity against forgetting? Journal of Experimental Psychology: General, 113, 32-35.

Parkin, A.J., & Walters, B.M. (1992). Recollective experience, normal aging, and frontal dysfunction. Psychology and Aging, 7, 290-298.

Rajaram, S., & Roediger, H.L., III (1997). Remembering and knowing as states of consciousness during recollection. In J.D. Cohen & J.W. Schooler (Eds.), Scientific approaches to the question of consciousness (pp. 213-240). Hillsdale, NJ: Erlbaum.

Rand, M.K. (1984). Story schema: Theory, research and practice. Reading Teacher, 37, 377-382.

Reed, S.K. (1993). A schema-based theory of transfer. In D.K. Detterman & R.J. Sternberg (Eds.), Transfer on trial: Intelligence, cognition and instruction. Norwood, NJ: Ablex Publishing Corp.

Ross, B.H. (1984). Remindings are their effects in learning a cognitive skill. Cognitive Psychology, 16, 371-416.

Ross, B.H., Perkins, S.J., & Tenpenny, P.L. (1990). Reminding-based category learning. Cognitive Psychology, 22, 460-492.

Roth, K.J. (1990). Developing meaningful, conceptual understanding in science. In B. Fly-Jones & L. Idol (Eds.), Dimensions of thinking and cognitive instruction (pp.139-175). Hillsdale, NJ: Lawrence Erlbaum Associates.

Rumelhart, D.E. & Ortony, A. (1977). The representation of knowledge in memory. In R.C. Anderson, R.J. Spiro & W.E. Montague (Eds.), Schooling and the acquisition of knowledge (pp.99-135). Hillsdale, NJ: Lawrence Erlbaum Associates.

Semb, G.B., & Ellis, J.A (1994). Knowledge taught in school: What is remembered. Review of Educational Research, 64, 253-286.

Semb, G.B., Ellis, J.A, & Araujo, J. (1993). Long-term retention of knowledge learned in school. Journal of Educational Psychology, 85, 305-316.

Skemp, R.R. (1987). The psychology of learning mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Strack, F., & Forster, J. (1995). Reporting recollective experiences: Direct access to memory systems? Psychological Science, 6, 352-358.

Tulving, E. (1985). Memory and Consciousness. Canadian Psychology, 26, 1-12.

Appendix A. Sample Short Answer Question and SOLO Scoring

"Please write a page or so on what you understand to be the important principles involved in the generality and sensitivity of research" (Methods Course)

1 " Very important but I don't understand much" (no knowledge)

2 "Generality is the extent to which results from an experiment can be assumed to effect the whole population in the same way as in the representative sample. Sensitivity is the stuff that men don't have and what makes women the weaker sex" (only has correct knowledge of one concept)

2.5 "Generality is important because it is a construct upon which the basis of psychological and scientific research is built. Basically, it means that any results/findings from an experiment can be extrapolated to apply to a wider population. Eg. data from drug treatment studies of 200 patients for depression used in prescribing pharmacological treatment to the general populace. To have generality, the participants included in the sample must be representative of the population that study will be generalised to, and the design and results of the study must be capable of replication. If these cannot be established, generality does not exist. Sensitivity: is the ability of experimental study to actually measure the construct in question"

3 "Generality is how well the results of the study can be linked to the population and real-life settings. Generality can be increased by using a random sample of the population to which you wish to generalise. Restricting variables (eg. age) and holding variables constant (eg. sex, occupation) decreases the generality of a study. Sensitivity of a study is how valid and reliable the methods and operations of the study are. An internally valid study or a study with high fact validity will be more sensitive to picking up a relationship. A study which is sensitive can be related with a study of similar measures/theories"

4 "Generality reflects the extent to which the results from a particular study can be applied to the population. Need to ensure random assignment, representative samples; the larger the better. Lack of random assignment can be devastating to results, since volunteers or categorical/SAV assignment introduces confounds. Sensitivity is the likelihood that the effect will be picked up, when it does exist. Need to limit variance unrelated to the focal IV and using more homogeneous groups. However, increased sensitivity can also result in decreased generality. Experimenter thus need to be creative in eliminating extraneous variance without causing the results to the seem to be limited to the experiment.

Sensitivity Generality

Experiment Ü Results Ü Reality "