EVALUATING CLINICAL LEARNING EXERIENCES* 

* Merv Batchler and Kris Malko 





Abstract 

In this study the aim was to conduct a formative evaluation of the 
implementation of clinical education in the Diploma of Applied Science 
(Nursing) program at Armidale College of Advanced Education.  This program was 
one of several to commence in New South Wales in 1985 as part of the general 
transfer of nurse education to the tertiary sector throughout Australia.  

The intention was to identify those factors, both intended and unintended, 
which influence the implementation of clinical education in an innovative 
program, according to the perceptions of students, clinical educators, 
registered nurses, a participant observer and other stake holders in areas 
where clinical education is undertaken.  

As the intention of this evaluative study was to portray all facets of the 
clinical education component of the program to enable judgements and decisions 
to be make, and because of the importance of the environment where clinical 
education takes place an illuminative, naturalistic approach to evaluation was 
adopted.  The findings demonstrated the importance of thorough preparation for 
clinical education and the establishment of trust and good channels of 
communication between all participants prior to, during and following clinical 
education practicums, in order to maximise learning opportunities.  






*Dr. Merv Batchler and Ms. Kris Malko are Senior Lecturers, Faculty of 
Education, Nursing and Professional Studies, The University of New England.  

*Data for this paper were obtained from Malko, K. 1989 The Evaluation of 
Clinical Education in Nursing, unpublished M.Ed thesis.  

Introduction 

The purpose of the study reported in this paper was to conduct a formative 
evaluation of the implementation of clinical education in the Diploma of 
Applied Science (Nursing) at Armidale College of Advanced Education.  This was 
to be achieved through an identification of those factors which influence the 
implementation of this component of the program, according to the perceptions 
of students, clinical educators, registered nurses, a participant observer and 
other people in areas where clinical experience practicums are conducted.  
Findings from this study were presented to interested parties or stake holders, 
and decisions were subsequently made regarding the future of this component of 
this program.  The stake holders to whom the results of the evaluation were 
presented included lecturers and clinical educators of the program, members of 
the curriculum development committee for the program, students and nursing 
representatives from the agencies being used for clinical placements.  

The aim of the curriculum was identified as being to produce graduates who are 
academically and experientially prepared to assume responsibility for 
competent, individualised nursing care at beginning practitioner level.  In 
order to achieve this aim the students were provided with clinical learning 
opportunities in patient care settings which would facilitate the application 
of theoretical concepts and skills from all areas of the program.  

The reason for the need to demonstrate the development of professional 
competencies in students is related to a demand for the implementers of the 
curriculum to be accountable to produce competent practitioners.  Over a number 
of years considerable disquiet had been expressed in a questioning of the 
ability of these programs to produce competent practitioners because students 
spend less time in clinical areas than they did in traditional hospital based 
programs.  

Furthermore, because of this limited time in clinical settings it is essential 
to ensure that clinical education experiences which facilitate the maximisation 
of learning and development are undertaken, in order to meet the goal of the 
program.  Therefore, because the perceived success or lack of success of the 
implementation of this program would depend largely on the extent to which 
graduates would develop and subsequently exhibit what are considered to be 
appropriate and necessary values and competencies in clinical settings, the 
development of these attributes and the assessment of student performance 
needed to be evaluated.  Additionally, because the learning milieu plays a role 
in the development of these values and competencies, its evaluation was 
considered to be necessary.  

A number of studies (Kratz, 1979; Mackay, 1981 Lonsdale et al, 1982 and 
McArthur et al,  1983) indicate that the clinical education milieu is a 
significant determinant of nursing performance.  Features of this clinical 
education milieu which warrant analysis in an evaluative study were identified 
as including the selection of appropriate learning environments, the on-campus 
clinical laboratory, pre and post clinical conferences, the role of the 
clinical educator and the assessment of student performance.  

Problems Associated with Clinical Assessment 

A review of the literature indicates that the process of assessing and 
recording students' clinical performances, or the attainment of competencies, 
is a widely discussed and vexing problem.  According to del Bueno (1977, 21), 
successfully implementing it is similar to achieving peace in the world.  Wood 
(1972, 1982) and Woolley (1977) have provided informative historical overviews 
focused on assessing student performances within clinical settings and have 
discussed various proposals put forward for the assessment of nursing students.  
However, basic problems which they identify are perennial and therefore need 
addressing.  Central to these is the fact that student assessment is undertaken 
in an environment that is everchanging.  'These changes of milieu, staff and 
clients present both positive and negative learning elements and they 
complicate..(assessment)... significantly' (Wood, 1982, 12), resulting in a 
situation where a format developed for one program and area may not be usable 
in another program or area.  

Wood notes that because of the number of students that a clinical educator must 
accommodate simultaneously... assessment of an individual student's performance 
is inevitably based on a sample of the student's total experience' (1982, 12).  
She then draws attention to the fact that students undertaking clinical 
practicums are being assessed while that are still learning and stresses the 
need for this to be taken into account when assessments are being conducted.  

The presence of an educator observing students perform changes the dynamics of 
a situation, causing students to act differently to normal, even influencing 
the interactions between students and patients (Jackson, 1983, 30).  Having an 
educator present may cause students to suffer anxiety which can inhibit 
performance.  Student anxiety is likely to be further increased if an 
instructor's procedures make students unsure about what they must do in order 
to pass or obtain a good result (Fowler and Heater, 1983, 403).  

As previously stated, methods used to assess students in the clinical field 
predominantly involve direct human observation which 'exhibits inherent bias 
and subjectivity, and is a subjective process' (Wood, 1982, 11).  According to 
Fowler and Heater (1983, 402) and Reilly and Oermann (1985, 306) educator 
variables such as past experiences, values, attitudes and pre-existing 
knowledge of students influence the assessment of students in relation to the 
behaviours noted and the way in which they are interpreted, contributing to low 
inter-rater reliability.  Woolley noted that grades awarded may vary 
considerably from one assessor to another when observing the same procedure 
being performed (1977, 308).  

Of further concern are the particular difficulties associated with assessing in 
the affective domain (Jackson, 1983, 30).  Inadequate clarification of 
assessment criteria and failure to adequately prepare educators regarding the 
use of the tool are likely to increase subjectivity still further (Wysocki, 
1980; Wood, 1982).  

Several significant biases are common amongst observers.  They include personal 
response tendency, halo effect and logical error (Craig, 1978, 27; Rezler and 
Stevens, 1978, 136-137).  Personal response tendency relates to individual 
variations between assessors in which some are consistently too lenient, giving 
students the benefit of the doubt, or too severe, demanding compliance in every 
detail, while others see most behaviours as average.  The tendency to take the 
middle of the road course is common (Spencer, 1985, 43), in some cases even to 
the point where students are never failed.  Halo effect concerns the influence 
of good or bad general impressions on observations.  When this occurs, a good 
overall impression of student performance makes an observer more likely to pass 
off an observation of an isolated ineffective behaviour as being unimportant or 
'just one of those things' (Craig, 1978, 27).  Logical error arises when 
observers are influenced by preconceived assumptions that certain things go 
together.  Rezler and Stevens (1978, 136-137) see that these problems are most 
likely to occur when the behaviour is not easily observable, is not frequently 
discussed, is not clearly defined, involves the reactions of other people and 
contains some socially desired characteristics.  Awareness of these potential 
biases can assist in making observers more impartial.  

In an attempt to overcome problems related to the subjective nature of 
performance assessment, Brozenec et al (1987 43-44) suggest a number of 
strategies which are summarised in the following points: 


ù	Using criterion referenced instead of norm referenced tools; ù 	Using a variety 
of methods of assessment that are appropriate to learning objectives enabling 
better student assessment and identification of problem areas; ù	Developing 
tools that measure behavioural objectives which reflect critical concepts of 
the curriculum; ù	Avoiding the use of excessively lengthy evaluation tools; 
ù	Decreasing the number of points on rating scales; ù	Discussing and clarifying 
items on the assessment tools; ù	Keeping anecdotal notes to facilitate recall at 
evaluation time; ù	Involving other educators and ward staff in clinical 
observation and assessment; ù	Incorporating self assessment by students; 
ù	Counselling students and providing frequent feedback; ù	Ensuring students are 
provided with appropriate situations to enable them to demonstrate clinical 
competence in key areas.  

A number of authors have noted that while it is essential that the assessment 
process is as fair as possible, attaining total objectivity may not be 
possible.  According to Wysocki (1980, 43) 

the quest to eliminate subjectivity from the process is commendable but 
unrealistic.  Better to accept that it exists, and make allowances, rather than 
delude ourselves that total objectivity has been achieved.  

Blomquist (1985, 8-11) agrees with this view, and believes that the trend 
towards increasing subjectivity and validity in assessment is reductionistic 
and has resulted in a devaluation of the intuition and judgement which is 
necessary in a clinically competent nurse.  She sees that while clear clinical 
objectives may assist in providing guidance for both student learning and 
assessment, it is important to not forget those aspects of clinical learning 
and competence which are not so easily identified.  Her views are based upon 
Benner's (1982) model of skill acquisition which suggests that an individual 
passes through five levels of proficiency in the acquisition and development of 
skills.  Benner (1982, 402) sees that these levels reflect changes in two 
aspects of performance.  

One is a movement from reliance on abstract principles to the use of past, 
concrete experience as paradigms.  The other is a change in the perception and 
understanding of a demand situation so that the situation is seen less as a 
compilation of equally relevant bits and more as a complete whole in which only 
certain parts are relevant.  

These distinctions between levels of performance are similar to the two major 
dimensions of performance identified by del Bueno (1977, 21).  They are 'doing 
the thing right', which indicates technical competence, and 'doing the right 
thing', or the application of prior knowledge and experiences to make 
judgements, set priorities and evaluate consequences of actions.  

*Methodology 

Because of the nature of the research question, which was to investigate the 
factors, both intended and unintended, which influence the implementation of 
clinical education in this program, Parlett and Hamilton's (1972) Illuminative 
Evaluation using concepts of Guba and Lincoln's (1987) Naturalistic evaluation 
enquiry methods was seen to provide the most appropriate general approach to 
data gathering.  Batchler's (1983) modified version of Stake's (1967) model of 
curriculum evaluation (figure 1) was used to provide a framework for the 
general categorisation of data collected and for the presentation of findings 
and decisions.  

WHAT WAS INTENDED 

WHAT WE NEEDED 

PEOPLE, RESOURCES 

TO BE USED IN 

PROJECT, AS WELL 

AS A DESCRIPTION 

OF STUDENTS _______________________ 

WHAT WE INTENDED 

TO DO: PROCEDURES 

TO BE USED TO 

PRODUCE INTENDED 

RESULTS _______________________ 

WHAT WE INTENDED 

TO HAPPEN: 

WHAT WAS INTENDED 

TO HAPPEN AS A 

RESULT OF THE 

PROGRAM 







RESOURCES 


 





PROCESSES 


 





RESULTS 


 


WHAT HAPPENED 

WHAT WE HAD: 

ACTUAL 

CHARACTERISTICS 

OF STUDENTS, STAFF 

AND RESOURCES 


_______________________ 

WHAT WE ACTUALLY 

DID DURING THE 

IMPLEMENTATION OF 

THE PROGRAM 



_______________________ 

WHAT ACTUALLY 

HAPPENED: 

IMMEDIATE AND LONG 

TERM RESULTS OF 

THE PROGRAM 





Figure 1:  Batchler's Modified Version of Stake's (1967) Countenance Model.  


Quantitative and qualitative methodologies were both used to define phenomena 
and collect data.  Quantitative data were collected in a survey using close-
ended items in questionnaires.  Qualitative data were obtained using a 
combination of open-ended items in the questionnaire, anecdotal recordings, 
open ended interviews and participant-as-observer participant observation 
during each practicum.  

There were a number of potential problem areas regarding data collection and 
processing.  The unidirectional nature of the scale in the student 
questionnaire could have led to the "Halo effect" occurring with some students.  
This appeared to be the case with a small number of students who responded with 
the category "mostly" for all questions.  This could have been a genuine 
response to individual questions, but also could have been due to a general 
feeling of satisfaction, and hence agreement across the board.  This problem 
was anticipated.  However, it was decided that the maintenance of a single 
directional scale would be more satisfactory, because randomisation had led to 
some confusion among students in one of the pilot studies, despite clear 
instructions.  

The second area of concern related to the interviews.  Although the evaluator 
guarded against leading respondents to answer in particular ways, it was felt 
that a number of comments were made primarily because the interviewer was also 
the co-ordinator of clinical education experiences.  Respondents may have had a 
vested interest in getting a certain message across to the co-ordinator, who 
was seen to be in a position to make suggested changes.  Similar concerns were 
felt by the evaluator in the participant observer role.  It was felt, however, 
that having an evaluator who was closely involved with the program and who was 
known and trusted would contribute to the collection of a significant amount of 
data which would possibly not have been elicited by an outsider.  

Findings 

In relation to the category of Resources there were a range of responses which 
indicated that inadequate preparatory work had been undertaken.  Although most 
students felt that they had been adequately prepared for the first practicum on 
the whole, a number indicated that there were specific areas such as report 
writing, documentation, lifting techniques, drugs and equipment usage in which 
they felt they had received inadequate tuition.  Students also felt that they 
had received inadequate and insufficient orientations to ward areas and to 
agencies in general.  

Although the majority of students indicated that they could relate well to ward 
staff, there was a concern about the lack of knowledge of the ward staff 
regarding the program, the educational needs of students and their abilities.  
This was evident despite the fact that two liaison sessions were conducted at 
each venue prior to the first  practicum.  It became evident that many of the 
nurses who were working with the students had not attended due to the fact that 
they were on days-off or were too busy to be released from the ward to attend.  
This lack of knowledge appeared to contribute to hostility.  When information 
was provided across a broad scale over the three week period, much of the 
questioning disappeared as did the hostility.  

Considerable stress was felt by the clinical educators because of their 
inability to attend preparatory sessions or because of a breakdown in the 
system of rostering for supervision of students.  This was also a major 
criticism noted by students and ward staff.  

Despite the fact that all of the preparatory activities had been undertaken, 
such as the Clinical Experience Handbook and the orientation and liaison 
sessions, and a number of respondents commented on the positive nature of 
these, they were insufficient for the needs generated in the practicum.  

In relation to the category of Processes similar themes arose as those in the 
resources category in that problems arose as a result of the inadequate 
preparation of individuals.  The behaviour and attitudes of clinical educators 
were seen as a major influence on the practicum, as were the behaviour and 
attitudes of ward staff.  Students seemed to be more willing to take 
responsibility and use their initiative when they felt secure in the knowledge 
that clinical educators and ward staff were supportive and readily available to 
assist them if required.  The hostility on the part of ward staff particularly 
in the early stages, contributed to feelings of inadequacy in the students.  
Again, as a result of inadequate preparation and inappropriate student coverage 
the clinical educators were not able to carry out their activities 
sufficiently, creating considerable stress, particularly if they did not have 
previous experience in this role.  

In relation to the category of Results the major findings related to the 
assessment of students.  There seemed to be a general feeling of satisfaction 
among the students about their performance and this was echoed in the comments 
of educators, registered nurses and the participant observer.  However, there 
was also general agreement that the assessment was excessive and in some cases 
unfair, with minimal inter-rater reliability. There was concern that there was 
too much emphasis on performance appraisal with too little attention being paid 
to the learning process.  Of equal concern to the curriculum development team 
was the fact that all students were given a passing grade for each practicum, 
despite the fact that some students were not successful in initial assessments.  
The result of these findings was the formation of a sub committee to develop 
new assessment instruments and associated guidelines for assessment techniques 
for educators.  

Another finding was that good relationships developed during clinical practice 
among students, clinical educators, ward staff and patients.  Although these 
could be attributed to better pre clinical preparation of students and clinical 
educators, increased two-way communications and the increased inclusion of ward 
staff in the education and assessment of students, there seemed also to be a 
greater general acceptance of the innovative program and its participants, 
simply because people were becoming more familiar with the program and thus, 
less threatened by it.  

Conclusion 

The findings of this study indicate that there is a need to conduct ongoing 
evaluations of the clinical education component of this program, due to the 
fact that the curriculum is undergoing constant refinements and also because of 
the everchanging clinical experience milieu.  Although evaluations have been 
conducted of individual courses in this program a general evaluation of the 
whole program has not been undertaken.  This could provide valuable information 
as it is designed to be an integrated curriculum, with the clinical education 
component building upon all other courses in the program.  

Further studies could focus on the perceptions of graduates regarding the 
adequacy of the program in general and the clinical education component in 
particular for meeting their professional needs.  Choice of areas of work by 
graduates could be investigated as could enrolment in post graduate programs.  
Finally, a research study could address the perceptions of nurse academics who 
have been involved in the transition of nurse education to the tertiary sector 
regarding changes which have taken place in the past, and those changes which 
they are experiencing currently because of the general alterations in the 
tertiary education sector throughout Australia.  

References 

Batchler, M. 1983.   Some Approaches to Program Evaluation, Unpublished Paper, 
The University of New England.  Benner, P. 1982.  From Novice to Expert, 
American Journal of Nursing, March, 402-407.  Blomquist, K.B. 1985.  Evaluation 
of Students Intuition is Important, Nurse Educator, 10, (6), November/December, 
8-11.  Brozenec, S., Marshall, J.R., Thomas, C. and Walsh, M. 1987.  Evaluating 
Borderline Students, Journal of Nursing Education,  26, (1), January, 42-44.  
Craig, J. 1978.  Anecdotal Records, The Canadian Nurse, 74, (5), May, 25-27.  
del Bueno, D.J., 1977. Performance evaluation:  When all is said and done, more 
is said than done, Journal of Nursing Administration,  December, 21-23.  
Fowler, G.A. and Heater, B. 1983. Guidelines for Clinical Evaluation, Journal 
of Nursing Education, 22, (9), November, 402-404.  Guba, E.G. and Lincoln, Y.S. 
1987.  The Countenances of Fourth-Generation Evaluation:  Description, 
Judgement, and Negotiation, in Evaluation Studies Review Annual,  Vol. 11, 
1987, D.S. Cordray and M.W. Lipsey, (eds), Newbury Park, California: Sage 
Publications, 70-88.  Jackson, E. 1983.  Student assessment: Does it work?  
Nursing Mirror, June 22, 28-31.  Kratz, C. 1979. Nursing education evaluation 
project:  report on nurse graduates qualified in 1977 and 1978, Perth:  
Department of Nursing and Educational Development Unit, W.A.I.T.  Lonsdale, 
A.J. Dennis, N.C., Hooke, S. and Norvilas, C. 1982.  Evaluating the nursing 
performance of college and hospital graduates:  procedures, outcomes and 
issues.  A report of the Nursing Evaluation Project, Education Development 
Unit, Western Australia Institute of Technology, August.  Mackay, L.D., Brooke, 
A. and Bruni, N. 1981.  An evaluation of a college-based course in nursing, 
Commonwealth Tertiary Education Commission, Canberra: AGPS.  McArthur, J., 
Walsh, A. and Bruni, N.V. 1983.  Further comparative evaluation of students and 
graduates from college and hospital-based basic nursing programs, Commonwealth 
Tertiary Education Commission, Canberra: AGPS.  Parlett, M. and Hamilton, D. 
1972.  Evaluation as illumination:  A new approach to the study of innovatory 
programs, Centre for Research in the Educational Sciences, Occasional Paper 9, 
Edinburgh: University of Edinburgh, Scotland, 1-19.  Reilly, D.E. and Oermann, 
M.H. 1985.  The Clinical Field:  Its Use in Nursing Education, Norwalk, 
Connecticut:  Appleton-Century-Crofts.  Rezler, A.G. and Stevens, B.J. 1978.  
The Nurse Evaluator in Education and Service,  New York: McGraw-Hill.  Spencer, 
S. 1985.  A move in the right direction...evaluating nurse learners' clinical 
competence, Nursing Times, April 10, 43-48.  Stake, R. 1967.  The Countenance 
of Educational Evaluation, Teachers College Record, 68 (7), April, 523-540.  
Wood, V. 1972.  Evaluation of student nurse clinical performance:  A problem 
that won't go away, International Nursing Review, 19, (4), 336-343.  Wood, V. 
1982.  Evaluation of Student Nurse Clinical Performance - a Continuing Problem, 
International Nursing Review, 29, (1), 11-18.  Wood, V. 1986.  Clinical 
evaluation of student nurses: syllabus needs for nursing instructors, Nurse 
Education Today, 6 (5), October, 208-214.  Wood, V. and Campbell, D.B. 1985.  
The instructor, the student and appeals, Nurse Education Today,, 5 (6), 
December, 241-246.  Woolley, A.S. 1977.  The Long and Tortured History of 
Clinical Evaluation, Nursing Outlook, 25, 5, May, 308-315.  Wysocki, R. 1980.  
Evaluation of student clinical performance, The Australian Nurses Journal, 10, 
(5), November, 42-43.  


* The methodology is reported here in general terms.  Further details are 
available from the authors.