Tom Lowrie

Paper presented at the joint AARE/NZARE Conference "Global issues and effects: The challenge of educational research," Melbourne, December 1999.

Competency Based Training (CBT) has had a major impact on the evolution of Vocational Education and Training (VET) (Waymark, 1997). Decisions concerning methods of delivery, teaching and learning, assessment, and transferability of qualifications have been strongly influenced by a competency-based training environment (Lowrie, Smith & Hill, 1999). In Australia, CBT has been legislated to a greater extent than most other countries. Policy directives at the national/federal level in the early 1990’s have ensured that competency-based training would become the preferred method of delivery of VET in Australia, with substantial implementation occurring by 1993. Today, CBT is synonymous with training in Australia.

The VET sector accommodates a diverse range of individuals in many fields of study across thousands of Technical and Further Education (TAFE) and non-TAFE providers. As a result, CBT means different things to different people. In general terms, however, CBT can be explained as having a focus on the outcome of training (ANTA, 1997). These outcomes are measured against specific standards and not against other students and the standards are directly related to industry. It is reasonable to assume that competency-based approaches have affected individuals in different ways considering the diverse nature of the sector.

The implementation of the CBT model of teaching and training in the VET sector has been a difficult one. It has involved changes in the relationship between VET and industry, particularly in the introduction of industry competency standards as the basis for VET curriculum, in the way in which curriculum is developed, and in the way in which curriculum is delivered and assessed. Although the definition of CBT is contested and its practice varies from provider to provider, and from teacher to teacher (Smith, Lowrie, Hill, Bush, & Lobegeier, 1997), there are enough common elements to enable CBT to be studied as a single phenomenon.

Smith et al, (1997, p. 3), for example, developed a set of key points that were common in most definitions of CBT. These points included the:

• focus of training is on of the outcome of the training;
• outcome is measured against specific standards, not against other students; and
• the standards relate to industry.

The changes brought about during the 1990s by the adoption of CBT have impacted upon everyone working in the VET sector. It can be argued, however, that the greatest effects have been upon VET teachers and trainers, since they have had to change their everyday practice to accommodate CBT. Moreover, they hold the ultimate responsibility for ensuring that CBT makes a difference to VET outcomes.

Teachers and trainers have had to change their everyday practice to accommodate CBT (Cleminson & Bradford, 1996; Smith & Lowrie 1998). Moreover, they hold the ultimate responsibility for ensuring that CBT makes a difference to VET outcomes. The change has been described generally as moving from an ‘up-front teaching’ model to a ‘facilitator’ model (e.g. Harris et al, 1995). The assumption behind this description is that, under CBT, students are more likely to be using self-paced learning materials, which they will work through on their own with assistance from the teacher; and also that, under CBT, students have clear knowledge of the required learning outcomes and hence power is shared more equally between student and teacher. Although these assumptions are highly debatable, research has shown (eg Smith et al, 1997) that some such shifts have taken place. It would be expected that those teachers accustomed to the ‘up-front teaching’ model would experience some discomfort, and indeed it was pointed out even before the 1990s ‘version’ of CBT (Hobart & Harris, 1980) that teachers might not welcome the prospect of becoming, in effect, a ‘resource person’.

It is comparatively easy, and common, to dismiss teachers’ discomfort as representing merely an unwillingness to adapt to change. Such responses have been analysed with reference to various models of organisational and individual change (e.g. Klein & Sorra, 1996; Stenhouse, 1975; Hord & Huling-Austin, 1986, in Smith et al, 1997). Teachers’ discomfort has often been associated with other radical changes in teachers’ and trainers’ working conditions, such as other features of training reform like the opening of the training market, and re-organisations of State and Territory TAFE systems.

However, instead of viewing teachers’ problems with adopting CBT as resulting merely from inflexibility and fear of change, it is possible to discern more concrete reasons why the change to CBT proved difficult for teachers. In some cases the practices they were asked to carry out were in fact educationally unsound. This was often a result of early interpretations of CBT that had not yet utilised holistic assessment practices. For example, Robinson (1993) documents the case of a TAFE cookery teacher who was required to assess students who were making a béchamel sauce by means of a checklist, instead of assessing the final product and the whole process. When faced with such demands, teachers’ confidence was undermined and some teachers withdrew from certain aspects of CBT (Smith et al 1997).

Lack of preparation and staff development was another common cause of teachers’ difficulty with CBT. Smith and Nangle (1995) and Choy (1996), amongst others, have documented evidence of teachers having received inadequate training in how to use CBT. CBT staff development has seemed to concentrate upon ‘big picture’ information on training reform rather than upon teaching strategies. There have been some successful staff development strategies such as the CBT in Action scheme (Kelleher & Murray, 1996), based on action learning principles, but these initiatives have reached only a small proportion of VET practitioners.

In particular it found that selection of CBT features for a course depended upon industry area, AQF level and type of provider. In addition, within TAFE systems, different States and Territories had different policies with relation to some matters such as the use of non-graded assessment. Beyond these differences, however, the way teachers used CBT varied even where the ‘mix’ of CBT features was the same. Teachers’ attitudes towards CBT, their sensitivity to the needs of their students and the resources available all affected the way in which CBT was used.

Finally, it is important when documenting changes to teachers’ roles to understand that our understanding (and teachers’ understanding) of the changes are affected by beliefs about what it is ‘normal’ for teachers to do, and also by what individual teacher’s practices were before they began to change. The changes for some teachers to CBT practice may not have been as great as for others. Several teachers in the Lowrie et al (1999) study maintained that their teaching had not changed much because their previous practice contained many elements of CBT. This may, of course, have represented ‘true’ or ‘false’ beliefs about both their previous practice and about what CBT consisted of.

The investigation examined the impact CBT has had on the role and responsibilities of teachers and trainers across the VET sector. The following five research questions were investigated in the study:

• What is the quality and nature of CBT instruction in a range of providers across AQF levels, industry areas and geographical locations?
• To what extent are levels of understanding of CBT by instructors in the VET sector common across a range of locations?
• How have instructors adapted their practice to accommodate competency-based training, and how have new practices evolved?
• What are some of the staff development issues present in CBT?
• What type of staff development model can be applied to instructors in the VET sector when introducing an innovation like CBT?

A range of techniques were used to evaluate the extent to which competency-based approaches had influenced, or changed, the role of instructors’ across the sector. These techniques included: (a) a nation-wide survey of instructors; and (b) a detailed investigation of six VET providers who utilise CBT. Survey data were analysed using descriptive, bivariate and univariate techniques through SPSS (1990) software. Criteria for analysing the VET sites were developed through case study methods (Yinn, 1994).

The survey was designed to assess instructors’ level of understanding of CBT and provide information, which can be used to interpret individuals’ knowledge, beliefs, attitudes and practices related to CBT. The survey can be seen as a way of assessing the extent to which CBT has been accepted by instructors in the VET sector and the extent to which they understand the principles unpinning CBT and how satisfied they are with their progress in establishing CBT practice. Furthermore, the survey permitted the identification of a range of factors that influenced the way instructors have responded to CBT. These factors included: the field of study in which the instructor taught; whether the site was a TAFE or non-TAFE provider; the AQF level of most of the students an instructor taught; and d) the nature of the students. It is important to note that most of the instructors who responded to the survey were from the TAFE sector.

Case study sites were predominantly from the non-TAFE sector and included a range of different providers in different states and territories. In most instances, data from case studies were collected over a two-day period; with the researcher interviewing several instructors at each site in both individual and group sessions. The views and perceptions of senior management and educational staff were also sought, and were taken into consideration when analysing data from each site.

The survey was designed to assess the effects of competency-based approaches on the role of instructors across a range of providers in the VET sector. Data from the survey were analysed across a number of categories—including type of provider, location of provider, AQF level of students, and nature of student—in order to access information about the instructors view of teaching, their organisation, their opinions of CBT, their experience in CBT, and ways in which they learnt about CBT.

Generally, instructors had a positive opinion of CBT. Over two-thirds (67%) of those surveyed indicated that they were on the "strongly for" side of the spectrum (ie. selected either one or two on a five point Lickert scale). Similarly, 70% of the instructors selected these categories for the question related to the appropriateness of a CBT format to their field of study. When asked to describe their understanding of CBT, over 80% of the instructors considered that they had a thorough, or quite strong, understanding of CBT. This suggests that CBT has now been in place for long enough for most practitioners to have developed a satisfactory understanding of its process and structure.

Despite the fact that most instructors had a fairly positive opinion of CBT, instructors were finding some features more difficult to implement than others. Although this is understandable, two of the features appeared to be creating many more difficulties than the others identified in the survey. These features—flexible entry and exit and assessment of demand—were creating moderate or major difficulties for approximately one-third of the survey respondents. Consequently, instructors had argued that the need to implement these two features into courses had a negative effect on the way they would like to develop teaching/training experiences. In an earlier study (see Smith et al, 1997) the research team found that a lack of resources at sites was a major cause of such difficulties. For TAFE teachers assessment at least partly in the workplace while working was also causing considerable difficulties. Again, a lack of available resources could account for the difficulties associated with the implementation of this feature.

It was evident that instructors’ understanding or perceptions of what distinguished CBT from other forms of practice were not always transferred to the courses these people taught. There were discrepancies between the type of features highlighted by instructors as necessary components of CBT when compared to features that would most likely be taught in courses at a given site. Instructors, for example, indicated that industry was involved in course monitoring (3rd highest response) in their course but did not consider such a feature to distinguish CBT from other forms of training (2nd lowest response). Similarly, the feature assessment criteria are made public to students was frequently present in courses taught by the instructors (4th highest response) but was not considered to be a distinguishing feature of CBT (4th lowest). In several other instances features that were most frequently present in courses were not the features the instructors felt distinguished competency-based approaches from other forms of training.

A closer analysis of the type of features identified by instructors as being most recognisable as features of CBT revealed other dichotomies. The following table, Table 1, presents some of the highest and lowest ranked competency-based features identified by instructors. The table has been divided into two columns—one with CBT features that were related to curriculum and accreditation, the other with features that were associated with teaching/learning interactions. Although it could be argued that there is a degree of overlap between all the CBT features presented in the survey, this table does reveal quite different patterns in the way instructors ranked the features.

This ranking allocation tends to suggest that the instructors’ view of CBT is linked to nationally accredited competencies and not issues that impact on themselves, and their students, more directly. In other words, features usually associated with the interaction between instructors and their students are not yet given the same attention as standards-based requirements that they have no control over. CBT is seen as something that is imposed from outside and not necessarily related to what they do in the classroom.

It also needs to be recognised that the competency-based environment is still relatively new for many instructors in the VET sector. Most instructors have probably focused on the administrative components of CBT when adapting to new practices before focusing on the type of features more directly associated with students. As instructors become more competent in dealing with the CBT environment themselves some of the features presently given a low priority may be valued more highly by instructors across the sector.

Comparisons between the case study sites were analysed in two main ways. First, two members of the research team examined the reports in detail to identify common themes. These themes were then circulated to those who conducted the case studies for comment and elaboration. As a result of this process the following generalisations across the case studies were drawn.

1. There is a variety of understandings about the nature and practice of CBT among individuals and groups and these have changed over time.
1. The understanding of CBT is influenced by a range of factors including whether one is in a TAFE or non-TAFE setting, industry area, the way in which it was introduced, the level and kind of staff development support, initial teacher/trainer preparation and key players.
2. The way in which instructors learnt about CBT was not always in accord with the way in which they claimed to prefer to learn.
3. The main contribution to the development of staff in terms of their understanding and practice of CBT involved learning on the job. Other forms of learning probably became more important later when instructors began to feel more competent. There is strong evidence that the introduction of CBT has stimulated new learning.
4. CBT is in reality practised in a variety of forms that reflect the industry and organisational context and the staff and students involved.
5. CBT is seen as problematic in some situations and institutions and is uncontroversial in others.
6. The introduction of CBT was seen to be marked by a limited series of events rather than a coherently organised process of change and development at most sites. Many teachers are critical of the way in which CBT was introduced and the level of support they received in terms of staff development.
7. Staff development that met the immediate needs and concerns was seen as valuable in the early stages of the implementation of CBT.
8. There is often limited communication between individuals and groups about CBT practice within larger institutions.
9. There is some evidence that CBT has had a positive response from industry.
10. Teachers and trainers appreciate having a variety of avenues for professional development.

Second, those who conducted the six case studies were asked to complete the matrix set out below (see Table 2). The matrix represented an attempt to identify the relative importance of factors identified across the case studies in the first stage of the analysis, in influencing:

• understanding of CBT;
• the nature and quality of CBT instruction; and
• adaptation of practice.

The researchers considered this procedure not only preserved the rich detail of the case studies at the various sites but enabled legitimate generalisations to be made which can inform the development of a model of professional development which takes into account significant environmental variables. Members of the research team were asked to indicate the extent to which forms of staff development had impacted on individual’s understanding of CBT at the respective sites.

The four factors—including initial staff development in CBT, initial teacher preparation, on-the-job learning, and collegial support—differed in the extent to which they were perceived to influence the three key aspects of the study. Initial staff development in CBT refers to courses that were used to inform instructors about the competency-based philosophy. This form of staff development may include a one-day inservice course or a training video. Initial teacher preparation would include, for example, an individual undertaking a Certificate IV in workplace training or university subject. On-the-job learning would include informal experiences that an individual is engaged in as part of his/her "usual" practice. Collegial support would include informal interactions with colleagues that shape individuals personal practice. The first two factors could be classified as external influences whereas the latter two would be more personal developments.

Each researcher ranked the four factors on a three point scale (from a major influence [3] to a minor influence [1]). The totals in each column were then categorised for each factor across the three "aspects of CBT" (see Table 2). It needs to be recognised that these rankings are a personal viewpoint of the researcher after a two-day visit at the site. The main purpose of this analysis was to monitor the extent to which the four staff development factors affected important factors of CBT.

Although each of the six sites was compared on an individual basis, it was not our intention to report these results in isolation. It was more beneficial, and educational sound, to monitor trends across the six sites. By analysing the data in this way we were able to monitor the extent to which a range of staff development initiatives impacted on competency-based approaches across a diverse range of providers. Although data from the survey was analysed in a similar manner, these data were generated from the observations and perceptions of the respective researchers, as opposed to the personal views of instructors.

With respect to understanding of CBT, informal on-the-job experiences were considered to be most influential in shaping instructors understanding of CBT (a score of three indicates that this form of learning was "high" at each of the six sites). At most sites, instructors maintained that their knowledge of CBT increased as they engaged in teaching/learning situations that were directly applicable to their students and industry area. Similarly, collegial support was very strong at most sites. This is not surprising, when you consider that several of the sites were relatively small providers. Importantly, these two "work related" factors were more influential in shaping instructors understanding of CBT than the more structured, or generalised, staff development options.

In contrast, the nature and quality of CBT instruction was not weighted toward the more informal, personalised, factors. Generally, each of the four factors made a strong contribution to the way in which CBT was implemented at the case study sites. The four staff development factors highlighted in this analysis made an important contribution to the quality and nature of CBT instruction at the respective sites. Initial staff development and initial teacher preparation, were at there most influential within this aspect of CBT.

Not surprisingly, the relationship between theory and practice was most influentially shaped through on-the-job learning experiences. In an increasingly competitive working environment, providers were required to adapt CBT practices to successful business principles. Thus, decisions about the way CBT was delivered were framed around current trends in industry. Moreover, decisions about the way resources would be used were influenced by specific circumstances at the site and not always based on CBT principles. Perhaps, as a result, the influence of initial teacher preparation was at its lowest rank in this aspect of CBT.

Findings from the surveys and case studies showed that the level of understanding of CBT is consistent across the VET sector. However, CBT is practised in a variety of forms that reflect the industry and organisational context of the staff and students involved. In general terms, instructors from non-TAFE providers have a more positive view of competency-based approaches than that of instructors in the TAFE sector. It could be argued that many non-TAFE providers have been able to shape CBT practices to a teaching/learning environment that suit their ‘competitive’ needs more easily than that of TAFE providers. TAFE teachers, for example, appear to be experiencing more difficulty introducing competency standards into their courses than instructors in the non-TAFE sector. On the other hand, many non-TAFE providers have indicated that a CBT framework is conducive to the training approaches they use.

Instructors who indicated that a CBT framework suited their particular field of study were more likely to have a positive attitude toward CBT in general. An implication of this is that any new innovations in the sector need to address educational and philosophical ideas associated with specific fields of study in order to gain acceptance in the future.

Modifications and adaptations to practice were more likely to occur across provider type (in this case, TAFE versus non-TAFE sectors) and course level (differences across AQF levels). Instructors in the TAFE sector were more likely to modify competency standards and assessment criteria in courses that they taught than non-TAFE instructors. In other words, TAFE providers, who found it more difficult to have their courses based on competency standards and linked to assessment standards, modified their practice more frequently.

It was apparent that instructors appreciated having a variety of avenues for staff development. Moreover, there was a diverse range of preferred staff development options among instructors in the sector. With respect to developing an understanding of CBT, informal on-the-job experiences and collegial support were considered to be most influential in shaping many instructors attitudes and understandings. Generally, other factors, including initial staff development and initial teacher preparation, made a strong contribution to the way in which teacher/trainers attempted to implement CBT. The way in which new teachers first learnt about CBT tended to shape their attitude toward it.

Staff development that met the immediate needs and concerns of instructors was seen as valuable in the early stages of the implementation of CBT. Furthermore, staff development in a teacher’s industry area may be just as important as staff development in teaching. It also appeared that action learning methods for staff development are as yet unproven in their efficacy.

The study proposes two models depicting staff development relating to externally-driven innovations in the VET sector. One model relates to the different levels of responsibility for implementation of the innovation and different phases of implementation and interaction with external stakeholders. The second model describes factors that affect individual instructors’ engagement with staff development activities. More detailed descriptions of the models can be found in Lowrie et al (1999).