1C:\WORD5\NORMAL.STYEPSONLQ3“@JśŠźµ°Æ±µASSESSMENT OF PROFESSIONAL COMPETENCE IN NURSING The primary purpose of assessment for licensure in any profession is to protect the public by ensuring that those so licensed possess the requisite knowledge, values and skills to perform professional functions safely and effectively. The professional may be assessed using a variety of methods - tests which assess, for instance, accumulated cognitive knowledge, technical skills, problem solving abilities, attitudes, or decision making. Only assessment in the practice arena, however, assesses the integration of learning necessary to cope with the complexities and ambiguities of professional practice. Stark et al (1) proposed that professional education should produce graduates who are knowledgeable in their field, able to perform the practical tasks required, understand the social and cultural context or environment in which they practise, relate well with others and engage in activities which enhance their own learning and contribute to professional knowledge and prestige. That is, they should have conceptual, technical, contextual and interpersonal communication competence, be motivated to continue learning and have a scholarly concern for the improvement of the profession. Professionals also need integrative and adaptive competence. Adaptive competence permits the graduate to adjust to uncertainty and change and to function in a rapidly changing environment. Integrative competence allows the graduate to harmonise all of the other competences to perform as a professional. While all competences are necessary, assessment in professional courses tends to concentrate on the conceptual, technical, contextual and interpersonal communication competences rather than on the adaptive and integrative. In nursing, there seems to be general agreement that competent, knowledgeable, reflective, ethical, caring, enabling, professional nurses who are accountable for sound clinical judgements, effective communication and safe interventions are desirable. The Registered Nurse is expected to be able to perform competently as a clinician, care co-ordinator, counsellor, client advocate, change agent, clinical and health teacher, and supervisor of nursing team members (2). These roles and functions are expected to be learned and incorporated into an integrated professional performance. Curriculum developers in nursing have recognised the importance of integrated clinical performance by ensuring that a significant proportion of nursing programs is concerned with developing and practising clinical skills in real and simulated settings. Kermode (3) refers to the richness of the clinical setting and the contextual subtleties that impinge on the process of clinical judgement in calling for clinical learning. However, although learning opportunities are provided, assessment of competence in this area remains a vexing problem. In applied professional fields such as nursing, the reality component of assessment is essential for public and professional accountability.(4) When the real world is used for assessment, the constantly changing environment, ambiguity and multiple stimuli of the clinical setting pose difficulties for the assessor. The complexity of the interrelationships among candidate, patient/client, environment and assessor makes the task of separating the intrinsic performance of the candidate from these other factors difficult. On the other hand, alternate means of assessing professional performance, by attempting to reduce the complexities and thus simplify the assessment task, also reduce the validity of the assessment. An assessment method which does not seek to eliminate the complexities, ambiguities and uncertainties of the clinical setting must be incorporated into any assessment package designed to assess professional fitness to practice. Assessment Options The act of assessment is the making of inferences about performance. A legal paradigm involving weighing collected evidence is more appropriate for assessing professional competence than a scientific one entailing measurement. There is widespread dissatisfaction with many current assessment procedures. Some of the reasons for this dissatisfaction are that they assess too narrow a range of practice; are biased toward the assessment of knowledge, probably because of the convenience of existing technology; and virtually never assess personal qualities or attitudes. It would be rare for a professional to answer a Multiple Choice Question in everyday practice and even more unusual for them to practice a specific skill in isolation from other aspects of their overall performance. Yet, assessments which are used to determine fitness to practice professionally, often rely on these techniques. The need to ensure a relationship between practice and assessment methods is well documented.(5) With the move of nursing to tertiary education it was necessary to dissociate from two traditions - the hospital training system with its reliance on Multiple Choice Questions and Clinical Skills Assessment Tests (6) and the university tradition of reliance on examination rituals (7). Professional readiness assessment - the interpretation of data derived from assessment of performance resulting in a judgement that the candidate is safe and effective to practice autonomously (8) - is one alternative which achieves the aim of relating assessment to practice. Performance is not seen in terms of the components such as knowledge, skills and values, but as an integrated whole. All of the above is not to denigrate other forms of assessment. Each form is appropriate for assessing particular components, or combinations of components. Professional readiness assessment is not a replacement technology. Rather, it builds on what is already available. Menges suggests that the accuracy of decisions about fitness to practice is improved by using multiple sources of evidence.(9) Sutnick et al (10) found that experts rated assessment methods in the following order of priority for assessing competencies: - real patients and direct observation by professional raters - standardised patients and professional raters - standardised patients and patient raters - electronic simulations - Patient Management Problems - Multiple Choice Questions. The Nursing Competence Assessment Project recommended a combination of well constructed written tests, meticulous observation of clinical practice, structured interviews of the candidate, client and peers, analysis of documents used in practice and self assessment to fulfil the needs of professional readiness assessment. (11) Problems in Assessment All assessments of knowledge, skill, values and performance have drawbacks. No assessment can examine the full range of professional knowledge or performance. The range of subject matter selected depends on the perspective of the examiner in the case of written or simulated tests, and on the exigencies of the clinical setting in the case of observation of performance. With written or oral testing it is possible to assess what the candidate says they will do but not what they will actually do. The information obtained from such tests is also dependent on the skills of the writers of the examination items and of criteria for marking. Observation of skills, either in simulations or the real world, is expensive as it involves one-to-one assessment. It is also dependent on the skill of the observer. The success of interviewing in eliciting valid information depends on the self knowledge and honesty of the interviewee and the skill of interviewer. Moreover, the lack of shared standards with clients and peers makes evidence collected from these people difficult to interpret. Simulations have been suggested as a compromise between assessments which concentrate on knowledge and reasoning processes and the assessments in the clinical area where uncontrolled variables may disadvantage candidates. McDowell et al (12) found that performance in simulation assessments with real patients correlated well with assessment ratings in patient care settings. The correlation was lower with electronic patients. However, simulations, even with real patients, refine and reduce the focus of assessment in order to ensure that the criteria are specific and thereby remove much of the ambiguity and uncertainty not to mention the other stimuli which impinge on performance in the clinical setting. It is important to know how a potential professional will cope with exactly those circumstances which are removed in such assessments. Objectivity vs subjectivity In any of the forms of assessment discussed above, the information gathered must be interpreted in terms of, for instance, accuracy, comprehensiveness, or quality and a judgement made about the standard for competence. This raises the question of subjectivity. Qualitative judgements are inevitable in assessment whether the assessor is judging the quality of wine or of the removal of sutures. In the latter case, it would be unusual to judge just the technical withdrawing of suture material from skin. The nurse-patient interaction during the procedure would also be of importance. It is the holistic episode which is important not the component parts, though they may be expressed as separate competencies. Objectivity tends to be equated with unbiased and fair judgement. However, it is absurd to believe that subjectivity can be eliminated from the assessment process. It is also undesirable to do so. Judgement about the quality of a performance depends on the tacit knowledge of standards of performance which the expert assessor holds in their head. This does not mean that subjectivity is equivalent to bias. It is the learned, relative standards of the assessor being used as the basis for assessment judgements. Moreover, if standards can be learned they can also be shared and brought into some relationship of uniformity with the subjective standards of others with equivalent knowledge and experience. Objectivity is the intelligent learned use of subjectivity, not a denial of it. It is the assessor who delivers objectivity, not the data. NCAP Research The Nursing Competencies Assessment Project (NCAP) was established to validate and develop an assessment technology for lists of competencies for registration and enrolment of nurses produced by the Australasian Nurse Registering Authorities Conference (ANRAC). It was intended that the lists and the accompanying assessment technology would be used to communicate the goals of professional nursing education to students, public, and assessors and become a curriculum design guide. ANRAC competencies specify a range of competence but not the specific standards of practice for each competency. (13) They have been validated for use with beginning practitioners. The competencies are of medium to high inference. That is, an assessor using them would need to infer from available evidence whether the candidate met the competency in question. None of the competencies is written in terms which allow direct observation without the application of a process of judgement. It is doubtful whether such a statement could be written, and if it could, whether it would be worth knowing. As part of the process of developing the assessment technology for use with the ANRAC competencies, NCAP undertook observation studies of newly registered nurses and analysed assessment instruments used throughout Australia for performance appraisal. The observations of actual practice provided data on the observability / inferability of the competencies in the clinical setting. The analysis of 74 assessment instruments by 555 nurses in 6 one-day workshops provided data on how competencies are currently assessed. Data on the differences of interpretation for each competency was also obtained. The assessment instruments analysed in this way were generally of the kind which required a supervisor as assessor with a significant proportion also requiring some form of self assessment. Most were directed at broad concepts of competence, rather than a narrow, specific skill. The assessor was required to make judgements about the performance level of the assessee. Little supporting documentation accompanied the assessment instruments. Where it did, it relied heavily on the work of Kathleen Bondy.(14) NCAP recommendations In any assessment technology based on competencies, each competency needs a standard. There must also be a means of obtaining shared understanding of that standard. Tacit knowledge of standards held by expert assessors needs to be brought into consensus with the standards of others of similar experience. Slater Stewart (15) claimed that all nurses held a common standard of the level of performance expected of a competent nurse. However, she did not try to verbalise this standard. Benner (16) suggests that this type of knowledge may not be able to be verbalised. NCAP therefore provides verbal descriptions of the expected level of performance for each competency and cues to support the assessors' judgement. The verbal descriptions detail the expected level of performance in terms such as safety, accuracy, comprehensiveness, appropriateness and effectiveness. Cues are selected concrete examples of activities illustrative of the competency. These cues were derived from the observation and instrument analysis data. They are not criteria and should not be assessed in their own right. As well as the verbal descriptions and cues, the NCAP assessment technology provides sources of evidence which guide assessors to appropriate modes of assessment and to likely contexts where evidence may be obtained to assess the competency. For instance, with the competency "Establishes and maintains constructive relationships with colleagues in the nursing team" the verbal description of the expected standard is Interdependent relationships are maintained and appropriate interpersonal techniques used. The sources of evidence for this competency are Observation supported by interview of colleagues. Cues include Demonstrates an ability to work as a team member; Assists other staff; Requests and receives assistance from other staff. It is clear from this that sharing of standards is necessary. This sharing will allow assessors to reach consensus on what constitutes maintained and appropriate. This shared understanding of standards depends on assessors who are clinical experts, who have been trained in observational assessment, using verbal description of the expected level of performance which are as precise as possible, and developing cues and exemplars to provide support for their judgements. There must also obviously be an opportunity to share standards, discuss actual exemplars, and reach consensus. Inter-rater reliability is high if these processes are followed. (17) In order to achieve valid and reliable assessments, the assessors need training needed in: - self understanding, including the effects of their perspective of nursing to ensure that past experience does not adversely affect judgement; - where to gather evidence; - the effects of context on assessment; - observational skills; - interviewing skills including the development of questionnaires; - item construction skills; - skills in interpreting evidence; - the scope and level of practice to be assessed in relation to their own; - sharing standards and developing exemplars. The implementation of the competency-based professional readiness assessment process in nursing will depend on the willingness of assessors to undertake training in the purposes, possibilities, procedures and pitfalls of the assessment techniques. Conclusion Competency-based professional readiness assessment is not foolproof. As well as the difficulties noted above, there are the additional risks (not confined solely to this mode of assessment) of narrow interpretation of competence, and of the competencies and associated assessments becoming unrepresentative of current practice. These limitations can, however, be avoided by providing for regular review of the competency set and of the assessment procedures by open democratic processes. Such should also help to ensure that unequal impact on gender, racial, or other groups is avoided. References 1. Stark, J., Lowther, M.A. & Hagerty, B.M.K. "Responsive Professional Education: Balancing Outcomes and Opportunities", ASHE-ERIC Higher Education Report No 3, 1986. 2. ANRAC role statement for the registered nurse. 3. Kermode, S. "Pre clinical preparation of undergraduate nursing students", Australian Journal of Advanced Nursing, 5:1, 1987, pp. 5-10. 4. Fahy, K. and Lumby, J. "Clinical Assessment in a College Program", Australian Journal of Advanced Nursing, 5:4, 1988, pp. 5-9. 5. Hyman, R.T. "Testing Teacher Competence : the logic, the law and the implications", Journal of Teacher Education, 35:2, 1984, pp. 14-18; McGaghie, W.C. "Assessing Readiness for Professional Practice" American Educational Research Association, San Francisco, March 1989; Salman, R.D. "Putting Testers to the Test: Relating Professional Licensing Examinations to Practice",Conference on the Professions: Professional Education, Professional Examinations and Practice - Defining the Right Relationships, New York, November 1981. 6. Fahy, K. and Lumby, J. op. cit. 7. Morgan, A. "Interactive Assessment in a Nursing Program", Australian Journal of Advanced Nursing, 6:4, 1989, pp. 29-32. 8. McGaghie, W.C. op. cit. 9. Menges, R.J. "Assessing Readiness for Professional Practice", Review of Educational Research, 45:2, 1975, pp. 173-207. 10. Sutnick, A.I., Barrows, H.S., Hart, I.R. Klass, D.J., Norcini, J.J. & Stillman, P.L. Educational Commission for Foreign Medical Graduates, Clinical Skills Test Development Committee Unpublished Report, USA, 1989. 11. Nursing Competencies Assessment Project final report to the Australasian Nurse Registering Authorities Conference, September 1990. 12. McDowell, B.J., Nardini, D.L., Negley, S.A. and White, J.E. "Evaluating Clinical Performance using Simulated Patients", Journal of Nursing Education, 23:1, 1984, pp. 37-39. 13. ANRAC/NCAP define competence as the ability to fulfil a role effectively. The concept can be broken down into component competencies to make it useful. Competencies are defined as those attributes of a person which result in effective performance. Competencies are usually not tangible nor visible but must be inferred from performance in the practice domain. Competencies holistically combine knowledge, skills and values and therefore cannot be assessed by any scheme which focuses on only one of these. 14. Bondy, K.N. "Criterion-referenced Definitions for Rating Scales in Clinical Evaluation", Journal of Nursing Education, 22, 1983, pp. 376-382. 15. Wandelt, M. & Slater Stewart, D. Slater Nursing Competencies Rating Scale, Appleton-Century-Crofts, New York, 1975. 16. Benner, P. From Novice to Expert, Addison Wesley, California, 1984. 17. Maatsch, J.L., Huang, R.R., Downing, S.M. & Munger, B.S. "Examiner Assessments of Clinical Performance: What do they tell us about clinical competence?", Evaluation and Program Planning, 10, 1987, pp. 13-17. ÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜÜ܀“v’’q ’’“n£’’»kN"’’k"h(’’œ(e91’’O1bD7’’7_ÜÜÜÜܝ7Ļ7’’’7x8’’†8u9’’9r9’’)9o+9’’- 9ln>’’z>iĻ@’’Ū@fVA’’|Aa|A  B’’0Bv’B’’øBq*C’’FClWD’’ŒDgD’’ÄDbĒD’’ŚD]E’’EXVAE\E’’‚EvÆE’’¶Eq’E’’ Fl•F’’GgH’’:Hb·J’’ÉJ]ŹJ’’ÓJXVAÓJK’’"Kv#K’’