EVALUATION OF A CD-ROM TUTORIAL ON WRITING SKILLS FOR FIRST-YEAR STUDENTS Sandra Bochner School of Education Macquarie University ABSTRACT Over recent years, many academic staff have become concerned about the difficulties experienced by first-year students in adjusting to the oral and written components of undergraduate study. These problems are encountered both by students who performed well in university entrance examinations, as well as those who performed poorly, and can be traced back to the very different essay-writing and communication skills needed in secondary school, compared with the skills required for university study. Most students are able to write descriptive prose, but are not accustomed to assembling information from a variety of sources, analysing it to find material relevant to a set essay topic, synthesising ideas, developing arguments and writing in a scholarly style. It was in recognition of these problems that an application was made for a 1993 CAUT National teaching and Development grant to design a CD-ROM Tutorial on essay-writing skills for use by first-year students. The proposed Tutorial would be located in the University Library and would provide an optional source of help for students needing help with essay writing. A pilot study of essay writing experiences among a group of first-year students suggested that the students’ main problems were in understanding the set question, identifying relevant references and integrating this material for the purpose of discussion. These issues are targeted in the CD-ROM Tutorial, together with other related topics identified through a review of the relevant literature. An evaluation study to compare the effectiveness of the CD-ROM Tutorial with more traditional forms of instruction in writing skills is planned for Semester 1, 1995, when Hypercard programming is completed. In 1992, concern about what was perceived to be the poorly developed information-gathering, essay-writing and communication skills of many new undergraduates at Macquarie University led to the Senate setting up a Working Party to identify ways of assisting students to overcome these problems. Difficulties in adjusting to the oral and written components of university study were encountered by students who had performed well in university entrance examinations, as well as those who performed poorly, These problems appeared to result from the very different essay-writing and communication skills needed in secondary school, compared with the skills required for university study. Most students were able to plan essays and write descriptive prose, but were not accustomed to assembling information from a variety of sources, analysing it to find material relevant to a set essay topic, synthesising ideas, developing arguments and writing in a scholarly style. The fact that the various disciplines within the University use different styles for written work only added to the problems faced by newly-arrived students. Within the School of Education, one outcome of the University-wide concern about the poor communication skills of many new undergraduates was for a group of staff to apply for a 1993 CAUT National teaching and Development Grant. The aim of the proposed project was to improve undergraduate writing skills through the development of an essay-writing tutorial using CD-ROM technology. The written competencies needed by students would be identified, and instructional modules developed that could be used independently in the University Library by students who were concerned about their writing skills. A report on the progress of this project is presented below. Background to the project Students have probably always had some problems in adjusting to the demands of academic study during their first year at university. Indeed, it was in recognition of these difficulties that undergraduate teaching programs at Macquarie University have included both lectures and small-group teaching methods involving face-to-face contact between students and experienced staff. Over recent years, increasing pressure to gain a high university entrance score has resulted in secondary schools focussing very narrowly on the preparation of senior students for the final secondary examinations. Little attention is paid to developing the more general skills needed for tertiary study. This lack of preparation is compounded by increasing pressures on University staff to achieve efficiencies in their teaching, resulting in the gradual re-introduction of higher staff-student ratios, larger tutorial groups, reduced face-to-face contact, greater reliance on lectures as the primary method of teaching, and on final examinations as a primary method of assessment. These strategies are characteristic of the more traditional undergraduate teaching practices that foundation members of the academic staff at Macquarie had attempted to avoid. Essay writing among university students has been the focus of considerable research and there is an extensive literature on the topic. Much of this material stems from studies conducted in tertiary institutions in the USA and UK (for example, Green & Klug, 1990; Hunter-Carsch, 1990; Moss & Holder, 1988; Norton, 1990; Sublett, 1989; Williamson, 1993). Relatively little research has been published on the essay-writing problems experienced by Australasian undergraduates, though Mahalski (1992) reported on a study involving geography students at Otago University and John Biggs has written extensively on levels of text comprehension and essay writing, using his “deep” and “surface” classification system (e.g. Biggs, 1988a; 1988b). Since there was so little information available on the problems experienced by Australian undergraduates in essay writing, a pilot study was planned to explore the writing experiences of a sample of undergraduates in the School of Education. The main focus of the writing-skills pilot study (McLean, 1993) was on the experiences of undergraduates in writing analytically. The aim was to find out if their difficulties were related to lack of experience with this genre of writing, in writing sustained arguments in general, or in understanding the question and using the literature effectively. Students enrolled in a first year-Education unit were invited to take part in the study and a small group (n=12), all from English-speaking backgrounds, agreed to participate. The students were asked to keep an log while writing an essay for the unit, noting the amount of time spent on different activities, commenting on the strategies they used and any problems encountered. Interviews were also conducted with the students (prior to writing the essay, after it was completed and when the marked essay had been returned). The results of this study suggested that students had no real difficulties with planning an essay, or with the mechanics of citing references and so on. Their major problems were in understanding the question, identifying relevant references, and integrating this material for the purposes of discussion. They had problems with being asked to express their own opinions, and in writing a conclusion to draw the essay together. In short, the study suggested that students’ problems in essay writing were associated with understanding the essay topic, recognising the “main issue”, finding and using reference materials and recognising what was required in a university assignment in comparison with essay writing at school. One of the first tasks of the project team was to review the resources available at the University to support undergraduate students worried about their essay-writing skills. A number of self-instructional essay-writing guides designed for general community and undergraduate use were located. Some of these were concerned with generic essay-writing skills (e.g. Bate & Sharpe, 1990: Doenau, 1989; Fletcher, 1990; Peters, 1985; Samson & Radloff, 1994) and others were written for students in specific discipline areas, including the social sciences (e.g. O’Shea, 1993; Samson & Radloff, 1992). Reference was found to one CD-ROM on undergraduate essay writing which had been written during the 1980’s. It was reported that the program had been popular with students. Courses on essay writing at university were offered by the School of English and Linguistics, and a service was available for individual students to consult a writing adviser after an essay has been submitted and marked. Specialised courses on English for Academic Purposes were also available for students whose first language was not English. Students were informed about these sources of help during Orientation Week and information on courses offered at different times during each semester were advertised around the campus. However, in spite of the availability of these resources, students continued to have difficulties with essay writing. The CD-ROM MODULES The aim of the writing-skills project was to develop a further source of assistance for students in the form of a computer-based tutorial that could readily accessed at any time. CD-ROM modules, located in the University Library, seemed to provide an appropriate means for achieving this aim. Students could be directed to the program by a lecturer, as an initial step prior to essay writing, or it could be used as one option for overcoming identified problems. The modules would not be designed for the specific needs of students whose first language is not English, though these students would have access to the package, along with other students. The modules are organised around six questions derived from the steps involved in writing an essay. These questions were identified in the initial review of the literature, as follows: What does the essay question mean? How do I find and record information for the essay? How do I organise the information? How do I plan the essay? What format does it take? How do I write the first draft? What should be done to finish the essay? Each module takes the form of a tutorial on one of the questions listed above. Within each tutorial, there are three components: a Reference Book (or straightforward statement of information about the topic); a sample essay (with associated working papers) written by Kerri-Lee, a “typical” first year student; and a set of practice activities designed to help the student begin to apply the concepts that are the focus of the Tutorial. For example, the topic “What does the essay question mean?” is introduced with a general statement about the function of instructional and topic words in essay questions. Frequently used action words are listed, together with a dictionary and “everyday” definition of each word, and examples of their use in essay questions. Students who wish to know more about this topic can look at what a “typical” student, Kerri Lee, did in preparing to write an essay on “rape”, “Piaget” or “anxiety and learning”. This strand continues throughout each of the tutorial modules, so that an interested student can trace through the steps followed as the essay was written, from the collection of reference material (useful and not useful), notes taken from these sources, identification of possible topics to include in the essay, the development of an essay plan, the first draft, and the completed essay. They can then chose to work through a series of activities; for example, identifying the “key (topic) terms” and “action (instructional) words” in a set of essay questions, selecting appropriate action words for a set of essay topics, or typing different action words to create a personal list with associated definitions. Students can use the CD-ROM modules in any way. They may begin with any of the six tutorials, and can simply read (and copy) the Reference Book material, look at the steps followed by Kerri Lee in writing her “sample” essays, or work through the various practice ideas. Guidelines on the way in which the Writing Tutorial could be used have been suggested by Library staff in light of the high pattern of undergraduate usage of the Library, particularly from the School of Education. For example, it was suggested that students should be limited to spending no more than 30 minutes working on the modules at any one time. Provision can be made for students to print selected material from the modules, but the program will not be available for use as word processor. Staff at the Library already provide considerable support to students in relation to essay-writing activities and they see the Tutorial as a very useful addition to the Library’s resources. Developing the modules A number of issues needed to be resolved during the development of the modules. For example, at an early stage, it became evident that the scope of the initial proposal was too broad. It was agreed, therefore, that we would concentrate on the skills of students within the School of Education, though it was assumed that once an appropriate model had been developed, further CD-ROM modules could be designed for students in other discipline areas. Another aspect of the project that is worth commenting on concerns the composition of the “project team”. From the start, it was agreed that the project would be conducted by the group as a whole, rather than one “course developer”. Preparation of the tutorial modules may have progressed more quickly if it had been given to one person to complete. However, different people became involved and, as a result, rather than one staff member acquiring expertise in undergraduate essay-writing skills and the design of CD-ROM, the project has had an impact on a number of members of the School of Education. Evaluation of the writing-skills project An evaluation study of the CD-ROM Tutorial was planned for Semester 2, 1994, but delays in preparation of the modules meant that the study was postponed to 1995. It is proposed that students in Educational Studies1: EDUC 105 (enrolment approximately 450) will be offered a course in essay-writing skills in the month prior to the date of completion of their first written assignment. From a group of volunteers, 60 will be selected to participate in the evaluation. These students will be allocated to one of three groups: a waiting list, with no intervention until after the first essay in EDUC 105 has been written; a computer-assisted learning group; and a conventional tutorial on essay-writing such as is currently available to undergraduates at the university. Essay writing can be measured both quantitatively and qualitatively. Data will be collected on the grades received by students before and after intervention (i.e., essay grades received in EDUC 105 and EDUC 106) to determine if there have been differential effects from the interventions. In addition, essays written in EDUC 105 and EDUC 106 will be analysed to determine if there have been qualitative changes in planning, style, referencing procedures and so on. Since one of the major aims of the project was to change the manner in which students approach the writing of essays, a small number of students from each of the three intervention groups will be invited to keep an “essay log” or diary of the process of writing an essay. The “essay log” technique will be used again to find out if there are differences in the manner in which students approach essay-writing, in the absence of better grades. Interviews will also be held with a small number of students to assess their responses to both the CD-ROM Tutorial and the more traditional tutorial options. Impact of the CAUT project Within the school of education: The CAUT project has already had an indirect impact on undergraduate essay writing, as a result of staff involvement in the project. For example, the process of reviewing the literature and developing a framework for the CD-ROM, based on the steps that first-year students would expect to follow when writing an essay in Education, has increased staff awareness of the difficulties that newly-arrived students experience in essay writing, and ways of minimising these problems. At the first-year level, this has resulted in more careful wording of essay topics, more detailed instructions on essay-writing in unit outlines, better advice during tutorials and a more structured approach to the marking of essays A second area in which the CAUT project has had a major impact on staff is on their knowledge of the general area of computer-assisted learning, and of CD-ROM technology in particular. Most people working in the discipline area of Education have some knowledge of programmed learning, teaching machines and so on. However, many are much less knowledgeable about more recent developments in this area. As pressures to achieve efficiencies in teaching continue, in association with greater variability in the level of competency of newly-arrived undergraduates, academic staff need to keep up-to-date with new techniques that may be useful in some aspects of the teaching process. This is of particular importance in a School of Education where future teachers are being trained to work in classrooms that have the potential to link in with the “information super-highway”. The CAUT project certainly exposed a number of staff to some of the possibilities associated with new technologies, and the need to ensure that all students, including those who had, thus far in their education, managed to avoid computers, had an opportunity to complete a computer-based task in a friendly and supportive atmosphere. Another outcome is that there are now a small group of staff who are experienced in programming in Hypercard. Within the University: During the initial phases of the CAUT project, a five-day workshop was held at the University for a group of staff who might contribute to the design and development of the CD-ROM. Participants included both academic and non-academic staff who could contribute to the task in terms of teaching methods, content, art, video, the Library and computer design. Apart from members of the project team who were most likely to be directly involved in the project, there were representatives of the Library (where the modules would be located), the Office of Computing Services, Audiovisual Services, the National Centre for English Language and Research, and the Centre for Higher Education and Professional Development. Several members of the University staff who had also received CAUT Teaching and Development Grants attended. The workshop was very successful, with some draft modules being produced. In addition, an informal network was established within the university for the sharing of ideas, information and skills related to the project. At the University-wide level, the Vice Chancellor has allocated funds to the Library and the Office of Computing Services to assist in the establishment of multimedia work stations as a means of supporting development of computer-assisted teaching modules within the Schools. The Writing Skills Tutorial will be placed at these work stations for the use of students in the School of Education. CONCLUSION As pressures on university staff increase, the CAUT grant provided a very useful stimulus for academic staff to review their teaching procedures and to explore new approaches to their teaching. From this point of view, the grant that is the subject of this report was most valuable. Everyone involved in the project was forced to think about issues surrounding student writing skills, to identify areas where students are experiencing problems and to find ways of helping them more effectively, while, at the same time, not increasing their own teaching load. This latter issue is of particular importance as pressures increase on academic staff to contribution to the profile of their school or department. From this point of view, the Teaching and Development Grant was a very useful aid to the development of new teaching ideas and resources. References Bate, D. & Sharpe, P. (1990). Student writer’s handbook. Sydney: Harcourt, Brace,Jovanovich Group. Biggs, J. (1988). Student approaches to essay-writing and the quality of the written product. Hong Kong: University of Hong Kong. (ERIC Document Reproduction Service No. ED 293 145). Biggs, J. (1988)Approaches to learning and to essay writing. In R.R. Schmeck (Ed.), Learning strategies and learning styles. New York: Plenum Press. Doenau, S. (1989). A comma for your thoughts. Sydney: Edvance Publications. Fletcher, C. (1990). Essay clinic. Melbourne: Macmillan. Green, C. & Klug, H. (1990). Teaching critical thinking and writing through debates; An experimental evaluation. Teaching Sociology, 18(4), 462-471. Hunter-Carsch, C.M. (1990). Improving students” essay writing. Reading, 24(2), 76-91. Mahalski, P.A. (1992). Essay writing: Do study manuals give relevant advice? Higher Education. 24(1), 113-132. McLean, B. (1993) Essay writing among first year students. Unpublished manuscript. Moss, A. & Holder, C. (1988). Improving student writing: A guidebook for faculty in all disciples. Dubuque, Ia: Kendall/Hunt Publishing. Norton, L. (1990). Essay-writing: What really counts? Higher Education, 20(4), 411-442. Peters, P. (1985). Strategies for student writers: A guide to writing essays, tutorial papers, exam papers and reports. Brisbane: John Wiley & Sons. O’Shea, R.P. (1993). Writing for Psychology: An introductory guide for students. Sydney: Harcourt, Brace, Jovanovich. Samson, J. & Radloff, A. (1992). The essay: A guide to writing essays and reports in the Faculty of Education. Perth: Paradigm Press. Samson, J. & Radloff, A. (1994). In writing: A guide to writing effectively at the tertiary level. Perth: Paradigm Books. Sublett, M.D. (1989). Using a model essay as a teaching tool in Geography courses. Normal, Ill.: Illinios State University. (ERIC Document Reproduction Service No ED 322 077).