The Context Post secondary, school of education

Research on the accreditation of courses in tertiary and continuing professional education: Why few are listening.

KOO98280

Anthony John Koop

School of Education

Macquarie University NSW 2109

Australia

Phone: 02 9850 8615

E-mail: Tony.Koop@mq.edu.au

Paper presented at the Australian Association for Educational Research

1998 Annual Conference

Adelaide, South Australia ~ 29 November - 3 December

Abstract

The process of accreditation of courses comes in many guises. For over fifteen years I have been associated with accreditation in higher education, vocational education, continuing education in the professions and accreditation of private provider courses. In this paper I explore similarities and differences among accreditation processes; their common strengths and limitations; and, possible reasons why many of these limitations remain unaddressed. I then propose an alternative approach designed to improve the accreditation process and, as a consequence, the quality of courses.

Accreditation is about official recognition. This recognition is usually associated with meeting official requirements. A certificate is normally issued that attests to the fact that accreditation has been given and associated standards have been met. People can gain accreditation, for example as a tax agent, an investment consultant, an occupational health and safety workplace committee trainer, or collector for a charity. Businesses can gain accreditation to belong and to operate as an official member of an industry group, with all associated rights, privileges and responsibilities. Private schools and health care facilities are examples. In this paper the focus is on accreditation of courses or programs associated with education and training. The paper draws on experiences associated with post secondary education and training courses provided by colleges, universities, membership and professional associations, in house staff development units within government agencies and private enterprise and private providers.

Why have accreditation

Accreditation existed long before the quality movement. Indeed in large measure accreditation can be viewed as synonymous with quality. In education and training it provided and continues to provide credibility and recognition for programs and courses and has been designed to ensure that the community can undertake an accredited program in the knowledge that it has been evaluated as meeting official standards and is likely to deliver what it promises. An accreditation process is thus a form of policing that protects the community and acts as a mechanism for the maintenance of standards within a particular field, occupation or profession. Accreditation is not a new concept.

Accreditation in context

A formal accreditation process was in place for courses offered by Colleges of Advanced Education (CAEs) prior to the restructuring of tertiary education under Federal Minister Dawkins in the late 1980s. Through restructuring these colleges were absorbed into a new national unified system of universities. As universities were responsible for the accreditation of their own courses, in the state of New South Wales for example, there appeared to be no longer a need for the Higher Education Board that oversaw the accreditation process. The accreditation process had been a key factor in universities granting credit for studies undertaken in CAEs. Graduates of teachers colleges, the forerunner of CAEs, gained no credit from universities for their college studies as the courses had no formal accreditation.

The demise of the Higher Education Board in NSW left a void. Some organisations classified as CAEs did not become part of the university system. An example is the New South Wales College of Nursing. This professional membership body at the invitation of the nursing profession in NSW set up an accreditation system designed to ensure the credibility and quality of nursing courses offered by that college and by other providers such as the Health Department through staff development units within health care facilities.

Courses offered within the technical and further education (TAFE) sector have always undergone some form of accreditation. Prior to the Dawkins era industry representatives were members of advisory committees that provided input at appropriate stages of trade course development or revision. The industry advisory committee played an important role in the overall TAFE accreditation process.

Until the national restructuring of industry and associated training that emerged as the central thrust of the Dawkins quality focussed national agenda TAFE had a virtual monopoly in trade and technical training. Restructuring opened the way for competition in this area by offering accreditation to private providers of courses. To assist in this expanded accreditation process the Vocational Education and Training Accreditation Board (VETAB) was set up. This board accredits vocational education and training courses that meet requirements of the National Framework for the Recognition of Training (NFROT). This accreditation strategy was linked to new Industry Training Advisory Boards (ITABs). VETAB accreditation is competency based. In the last ten years many accreditation services have developed accreditation criteria based on competency standards. This approach has emerged along with the development of nationally recognised competency standards within all industries in Australia. Competency standards developed through Industry Training Advisory Boards have become the basis for much training course development. Such standards apply to all work levels from unskilled through to professional. VETAB requires each course to be identified in terms of its place within the National Awards Framework and for competencies to be attained through the course to be stated in behavioural terms. Through this accreditation process VETAB provides a system for national recognition and portability of courses. Its recognition of privately provided courses has resulted in greater competition among course providers and in greater community awareness of and acceptance of alternative course providers.

Under VETAB arrangements a Project Steering Committee which guides the development of a course may then act as an Accreditation Review Panel for the purposes of accreditation at the completion of the project.

At the higher education level each university has its own internal strategy for course approval. However the professions and major employers play a significant role in the accreditation of many university courses. Indeed the future employability of graduates of professional development courses in universities is dependent upon the accreditation of that course by the relevant profession or industry. An accreditation system also exists for courses offered by colleges and other providers at the higher education level.

Across the various levels and sectors of postsecondary education and training the various accrediting bodies have very similar approaches to the accreditation process.

The accreditation process

Documentation reviewed by an expert panel with clarifying input from course developers, is the essential and common core of accreditation processes. Guidelines for the preparation of documentation to be submitted provide the framework for the process. These guidelines shape the formal submission and the thinking that goes into the structuring of the substance of the submission. Some accreditation services provide support to the developers in the interpretation of guidelines and the associated criteria for accreditation. Others make available a list of consultants who might assist in the process for a fee. Yet other services do not see such support as a legitimate part of their role. The guidelines are employed by the expert review or accreditation panel in determining the way the actual face to face review might progress and in flagging aspects of the submission most likely to attract attention.

Documentation required includes details about the course provider organisation related to its ability to provide facilities and ongoing support for the proposed course, the objects of the organisation, the nature of its teaching/learning resources and the qualifications and experience of teaching staff, together with details of the course, its purposes, level of award, duration, selection criteria, content, teaching, learning and assessment strategies, and associated literature and other support materials. Some accreditation processes include an inspection of facilities to validate claims made in written submissions.

The review panel is usually chaired by a person experienced in the accreditation process and knowledgeable about the rules and requirements that govern accreditation. Membership of the panel is probably as much associated with availability and being known as it is about appropriate expertise. Usually panels include industry representatives and experts in fields associated with the course content. Past students of the provider who have participated in an earlier version of the course or a related course are included on some panels. Some accreditation systems pay committee members a sitting fee, often members are representing and are employed by professional associations or institutions of tertiary education and no further payment is made, other systems work on goodwill and members provide their services on a pro bono basis.

The course developers and a representative of their organisation are usually invited to review panel meetings to assist members to better understand the proposed course. Developers may be asked to address seeming omissions and inconsistencies or clarify other aspects of the written submission. In some cases these people may be invited to be present throughout the panel's deliberations. It is more usual for them to be present for only part of the meeting.

Many accreditation systems have a single formal meeting of the panel. Recommendations of the panel would be to accredit for a certain number of years.. The number of years may reflect the perceived quality of the submission or the need for courses to be updated regularly to keep pace with advances in the field. The recommendation to accredit may be subject to specified changes being made. Examples of changes include: varying the student entry criteria; clarifying the assessment process ad its link to intended course outcomes; and, linking teaching strategies to the stated course philosophy. The final option is to recommend that the course not be accredited.

There is generally a fee associated with a submission for accreditation. Part of this fee is to cover expenses incurred by the accrediting body. Some accrediting agencies run the service at a loss and see it as part of a service to members. Others run it on a cost recovery basis, yet others seek to make a profit from their accreditation service.

Where a course or program gains accreditation, a certificate attesting to the fact is issued. This certification enables the course provider to employ its accreditation within its recruitment program. In all instances there is a payoff for gaining accreditation. For example, accreditation may be a prerequisite to gaining access to a particular market or to funding or support from a government agency.

The accreditation strategy described above is widely practised. The approach has a number of strengths and limitations. A range of these are listed below.

Strengths

The process appears transparent.

Formal review can be completed within a relatively short time span.

It is purported to involve key stakeholders.

Once accreditation is achieved providers can forget about the course and move on to other things.

The process provides an indication of strengths and weaknesses of the proposed course.

The process provides opportunities for developers to refine the course prior to accreditation and implementation.

Guidelines enable people with limited experience to work through requirements and succeed.

The review involves a range of experts to provide input.

The review meeting has the potential to be a positive learning experience for all involved.

The process can assist committed providers to learn how to best document courses for accreditation.

The process is less expensive to conduct than an ongoing one.

Limitations

Accreditation is given before the course is actually implemented.

Implementers of the course need not have any input into the accreditation process and may have reduced commitment to a course developed by others.

Required documentation may be developed by a consultant using the prescribed formula with minimal input from those providing the course.

Accreditation may be viewed as merely a means to an end. It has to be endured. Some may not be committed to quality, yet know what is required and how to jump through the hoops to gain the accreditation they need to enable them to, for example, attract overseas students to high cost courses.

There is no guarantee that the process will assist in developing a commitment to quality course development or implementation.

There is no guarantee staff listed in a submission will be the same when the course is implemented.

Qualifications and experience are no guarantee of actual quality of staff as educators or trainers.

The process can be superficial as the review meeting is usually limited to two to three hours in which a detailed and complex document needs to be addressed and the accreditation process pushed through. This requires a business like approach where important issues can easily be discounted or trivialised to ensure time constraints are met.

Guarantees given through accreditation are at best guarantees of intention and potential to deliver.

Freedom of course design is limited by accreditation documentation requirements.

The process is often very directed and based on formulae and language protocols that devalue and demean the well developed and supported ideas or approaches. Statements from consultants such as, "This won't get through unless you . . ..", are commonly heard. Rarely is a consultant heard saying, "This is different from what is really accepted but we will be able to make a case that will be accepted."

The good work that has been done by course developers can be trivialised by the process.

Where competency approaches are the focus of accreditation, language required tends to reflect the lower order of Bloom's taxonomy, such as identify, list, demonstrate. Higher order terms such as discuss, analyse and synthesise are discounted along with their potential to foster indepth learning.

Inexperienced people can come to believe that following the guidelines is the way all curriculum work is done. They may never come to appreciate that there are other approaches to course design.

The listing of all likely content is sometimes a panacea. This, together with the assumption that once a competency has been tested and passed then the person has it for life, pays no heed to research to the contrary.

There is sometimes no sense that content may need to be revisited.

Curriculum is seen as a product rather than a process.

The process does not include ongoing formal monitoring of the course.

Experts in course content areas may have very limited expertise in course design and teaching learning theory.

Experts on accreditation review panels may stifle creative and imaginative approaches to course design.

There may be only one panellist in a particular area of expertise, such as English as a second language. Time is thus taken briefing other members of the panel who largely have to accept the person's advice on face value.

Content tends to drive the decision making rather than the adequacy of the proposed process.

While adult learning principles are claimed as the basis for most course designs, submissions often reflect a limited appreciation of their application.

The course that is finally accredited may be a very different product from that envisioned by the developers. This may be a strength or a distinct limitation.

Course developers are at the mercy of the review panel and have limited power to take full responsibility for the development and ongoing refinement of the course.

Review panel meetings are not always harmonious.

Review panel meetings are always stressful for the provider and developers.

When sensitivity is lacking meetings can be a devastating experience for developers.

Why few are listening

The list of limitations is far longer than that of strengths. Yet the existing process does not appear to be under threat. Evidence suggests that many who have been involved in the accreditation process are dissatisfied with the process and sometimes with the outcome. Why then is little changing? Why are so few listening? An inspection of the strengths points to a possible answer. The existing process is relatively short term, inexpensive and fits within an easily managed and administered framework of guidelines, criteria and policies. It is far easier to have everybody using the same basic approach than to cater for the rich diversity that is possible when creativity and energy are devoted to the course development activity. It is less costly and time consuming to have an up front process with a defined beginning and end. The result is that courses are provided, these courses are accredited and the short term goals are achieved.

What is missing?

Missing is a process that is ongoing and that enables the developers to take increasing control of the process as their expertise is developed or demonstrated. Curriculum has been defined as a progressively modifiable plan. What is needed is an approach which provides accountability and support for ongoing development, improvement and modification. This might be best achieved where the approach to course design is consistent with and an integral part of the whole organisation's approach to its business. The starting point for such a process would be an attempt to identify just where the organisation is at, its strengths and limitations in each area of its operation, and, the setting of priorities for beginning the ongoing task of moving towards the ideal learning organisation. This approach is consistent with the characteristics of what Guba and Lincoln (1989) refer to as fourth generation evaluation.

Fourth generation evaluation is an ongoing cycle of monitoring, assessing, action and evaluation designed to provide information to be used by all within the organisation to improve the quality of their endeavours. The process is supportive and educational. In the case of course accreditation the aim is to gradually empower all associated with course design, implementation and evaluation to together work, to come to understand their own function and the ways their contributions can better link with the course efforts of others. It is designed to empower people to take increasing responsibility for assessing and refining their own efforts. The expected outcomes are quality courses developed and offered to clients, improved job satisfaction and increased demand in the market place.

Fourth generation evaluation leads to an accreditation process that is based on both collaboration and empowerment. Factors critical to its success include: the existence of an enlightened and committed leadership; and, a strong education and support program. The purpose of the support program is to induct experts engaged in the accreditation process and staff of course providers into the ideas the new approach, to coach them in its ongoing working and to assist them when inevitable blocks to progress occur. It may well be that any attempt to implement such a process will be blocked as too costly, too time consuming, not rigorous and too subjective. However such fourth generation evaluation systems do exist and have been developed as accreditation tools.

A way forward

In Australia we are fortunate to have available an example of a fourth generation evaluation program. This program is the Australian Council on Healthcare Standards' Evaluation and Quality Improvement Program (EQuIP). EQuIP is an ongoing accreditation process designed to accredit health care facilities. Its design is such that is can be equally applied to education and training courses and programs. Indeed, as staff development is an integral pat of the operation of health care facilities, EQuIP is currently employed in the evaluation of education and training programs.

The principles underpinning the EQuIP process are widely recognised and employed in a range of settings worldwide. However, as with current course accreditation strategies, many health care executives see EQuIP merely as a way of gaining accreditation and thus a way of maintaining credibility and funding.

Used effectively EQuIP leads to the involvement of all staff, including executives. It requires active engagement in and commitment to a process of growth for individuals and for the organisation. It is designed to empower staff to take real control of the assessment and improvement of their own contribution. EQuIP can be viewed as an educational program, which, when employed effectively, will foster learning in all daily activities of both staff and management. In essence, the process is largely workplace oriented, based on experiential learning stemming from work related experiences (Kolb, 1984; Evans, 1994) and supported by a process of reflection on practice (Schon, 1989). It is thus an ideal tool that fosters collaborative course development and ongoing modification. It still involves experts, documentation and formal accreditation visits or meetings. However the feedback from experts is related to achievements, challenges and difficulties that the education and training teams have identified themselves. The feedback is supportive and designed to assist growth and understanding.

While there is not time to explore EQuIP or fourth generation evaluation strategies more fully here, I commend them to you. I suggest they are worthy of careful examination and evaluation as they provide the essence of a viable alternative and a more credible approach to accreditation than the strategies commonly employed.

References

Dawkins, J. (1990) Quality teaching: An issue for all. Canberra: Australian Government Publishing Service.

Evans, N. (1994) Experiential learning for all. London: Cassell.

Guba, E.G. & Lincoln, Y.S. (1989) Fourth generation evaluation. Newbury Park, CA : Sage Publications.

Kolb, D.A. (1984) Experiential learning: experience as the source of learning and development. Englewood Cliffs: Prentice Hall.

Schon, D.A. (1987) Educating the reflective practitioner. San Francisco: Jossey-Bass.

Senge, P.M. (1990) The fifth discipline: The art and practice of the learning organisation. New York: Doubleday.

The Australian Council on Healthcare Standards (1996) The EQuIP Guide. Sydney: ACHS.

A J Koop AARE 1998 Paper KOO98280