Paper delivered at the Conference of the Australian Association for Research in Education , Newcastle, November 1994 ISSUES IN THE EVALUATION OF THE DELIVERY OF CAREER EDUCATION PROGRAMS IN SELECTED VICTORIAN SECONDARY SCHOOLS Margaret Bride (Deakin University) Quantitative methods of evaluation were used to test the hypothesis that student outcomes, in this case vocational maturity, were not affected by whether or not the career education program was delivered as an integrated part of the general curriculum. An initial survey, sent to all post-primary schools in two educational regions of Victoria, was used to identify ten schools with an integrated approach to delivery of career education programs and ten schools where the program was not integrated. The Career Development Inventory- Australia (CDI-A) was administered to a selected group of year 11 student sin each of the 20 schools, totalling 400 students. The results showed a differential in student outcomes in the cognitive aspects but not in term s of the attitudinal aspects of career development between the two types of delivery. Research issues to be addressed in the paper are: * the challenges of conducting research across a school system which has decentralised much of its decision making to the school level; * the difficulties of obtaining a suitable sample of students in the later years of schooling when pressure on student time is great; * the use of a well researched and standardised test in program evaluation. Issues relating to career education are: * which students benefit by an integrated approach to the delivery of career education programs and in what aspects of career development they benefit; * the interaction between gender type and type of program delivery. 1. CONTEXT 1.1 Career Development Career education in Australia has been strongly influenced by career development theory from the United States. A key question addressed by career development theorists is how people's careers develop: what are the factors which impact on the individual and her or his career and how significant is the impact of each of these. The process is identi fied by which an individual develops vocational identity through a dynamic interaction between the individual and the individual's environment. That is between the abilities, values, personal style and interests of the individual and the external environmental factors of family, social background, schooling, employment, and society including the economy. One of the most influential career development theorists is Donald Super whose work spans the years 1950 to 1988. He adapted the concepts of Havinghurst (1953) on life stages and developmental tasks associated with them, to the field of career development. His work has, both directly and indirectly, had a major impact on career education in Australia. Super developed the concept of vocational or career maturity as a corollary to the developmental stages concept. It is significant in the application of career development theory to the practical area of career education and is defined as (Super 1985 p4): readiness to cope with the developmental tasks of one's life stage, to make socially required career decisions, and to cope appropriately with the tasks with which society confronts the developing youth and adult. Analysing the components of career maturity, Super defined two broad areas, the cognitive and attitudinal. Between 1960 and 1972 Super's theories were tested in a number of research projects, particularly in the longitudinal Career Pattern Study. From the team work on this derived the Career Development Inventory, an instrument for assessing career maturity, su itable for use with adolescents and young adults, and for assessing career education programs.. 1.2 Career Education In the early 1970's in the United States the increased concern to improve the employment rate of young people when they left school, especially those from minority groups and areas of social disadvantage, gave impetus to the career education movement. A prime motive wa s to assist young people to make choices which would enable them to gain greater satisfaction from working life. Effective choice does not require a complete knowledge of either the world of work or of the individual. For most people, however, the likeli hood of an effective, satisfying choice is increased in direct proportion to the extent to which their knowledge of themselves and the world of work is increased. (Isaacson 1971) One of the problems about career education has been that it has often been seen as a means by which society would induct young people into the existing values of a work oriented society. For example: We see careers education as the total effort of ;public education and the community aimed as helping individuals to become familiar with the values of a work oriented society, to integrate these values into their personal value systems, and to implement these values into their personal lives in such a way that work become possible, meaningful and satisfying to each individual.(Evans,Hoyt and Mangum, 1973 p1). An early Australian definition broadened the scope somewhat to include the learning of the skills required for participation in working life. Career education aims at helping students toward a deeper understanding of their own abilities, interests and desires, at developing their ability to make decisions about their future employment and life chances, at increasing their knowledge of various occupations and of the working world, and at understanding the role work plays in society and in an individual's life.(Career Education Association of Victoria, quoted in VISE News Bulletin 15, 1979) Definitions, such as these, can be useful in defining the broad goals of career education but they are only a starting point in any process of evaluation of career education programs. 1.3 Career Education in Australian Schools Interest in career education has been inclined to wax and wane with the rise and fall of youth unemployment. Greater interest was shown in Australia in 1981-2, for example when the School to Work Transition policies were put in place, than in the mid 1980's when there was less youth unemployment, and the follow up Participation and Equity program was wound down. In 1986-7 the Curriculum Development Centre in Canberra sponsored a Curriculum and the World of Work Project , which included the publication of a series of papers on the theme of schooling and work, including Career education and counselling in secondary schools and colleges. (McCowan, 1987). The CDC project and its publications aroused some interest in career education circles and among employer and union organisations with an interest in improving links between schools curriculum and the world of work, for example at the conference 'Education Forum' held in Sydney 1988, but it was not followed up by the general curriculum policy units in the Australian states to any great extent. By 1990, however, there was a much stronger recognition that the relationship between schooling, particularly in the post-compulsory years, and the world of work was dysfunctional. In 1991 this recognition led to the Australian Education Council appointin g a review committee into the participation of young people in post-compulsory education and training, better known as the Finn review. They made recommendations (Australian Education Council Review Committee 1991) for changes in the nature, structure and delivery of schooling to young people over the compulsory school leaving age. This was followed shortly afterward by proposals for post- school training by the Carmichael committee.(National Board of Employment Education and Training 1992). Both these documents give the practice and delivery of career education in post- primary schools a high priority. At the same time the Schools Commission published a report Strengthening career education in Australian schools , National Board of Employment, Eduation and Training, ( 1991). This, together with a research paper published in 1990 by the department of Employment Education and Training called Career education in Australian secondary schools, (Davis and Braithwaite, 1990), was used as a basis for a paper published by the Australian Education Council in 1992 Career education in Australian school: a statement of national goals, student and system outcomes and evaluative arrangements. Finally the National Board of Employment, Education and Training (NBEET) produced A national training framework for careers coordinators: a proposal.(1992). The key issues on career education in schools which are addressed in these recent studies and government policy documents are the : * availability of career education to all students in post-primary schools; * training and deployment of trained career practitioners in the school and the community; * development of a common set of goals for career education across all states; * provision of career information; * place of career education in the school curriculum and how it can most effectively be delivered. 1.4 Delivery of Career Education in Schools All the reports agree that career education should not be an isolated area of learning separated from the general education program of the school. For example in the National Board report, Strengthening career education in schools (1991 p;7-8) it is stated that the goals and content of career education should be the development of self awareness, the development of the skills and knowledge to be able to make career decisions including opportunity awareness, transition learning and understanding of the world of work, and that much of this is integral to learning throughout the general curriculum...the role of career education is to make this process more explicit. (p8) Moreover the report goes on to indicate that career education will be, in part, be provided by teachers integrating elements of learning about the world of work into their subject teaching. (p9) Davis and Braithwaite document the congruence of policy recommendations on this matter between the various states and say: Virtually all systems agree with the principle of integration of career education from K-12 and across the curriculum (1990 p39) An examination of the literature on just what is this ntegration reveals that the term needs to be carefully defined in the context of current school practice. There were hopes, in the early 1970's in the United States, that career education would be a force able to transform the curriculum, making it more relevant to the working world and more meaningful for many students. (Evans et al 1973). Some of the protagonists of this point of view also advocated a totally infused approach to the delivery of career education, arguing that all school learning should be infused with learning about the world of work, learning the skills and competencies required for working life, and aimed at assisting students to that level of self understanding which would assist career decision making. Such an infused delivery would tend to mean there was a loss of program identity, It assumed also that schooling should be focussed on the employment outcomes and was subject to severe criticism for too great an emphasis on this. As career education has come to be more accepted as a part only of general education (Australian Education Council Review Committee, 1991, p153) and its goals have been defined more narrowly and at the same time more realistically, the importance of a career education program being an identifiable part of the school program has also been accepted. (Gysbers,1994, p19) It has frequently been stated that the program should be an integral part of the school program and not an isolated component, unconnected with the general subject learning in the school. A recent Australian statement on the desirability of this is made in the Carmichael report on training which argues for all teachers in the post-compulsory years of secondary schooling to teach in a contextual way, by which the report means in the context of the lives of young people, especially the context to their working lives, acknowledging that the content of what is learned will be more interesting to students and more readily learned if they are aware of the relevance the learning has for life outside school and after schooling has been completed. (National Skills Formation Council, 1992, p71) This integration of the learning about the world of work and the teaching of attitudes to learning and work which will encourage young people to participate in post school training as well as post school work could be supplemented by specific teaching in the area of career education at those stages of schooling when students are faces with major decisions and when specific content, especially about post-school options and how to access them, is required. ( Ministry of Education, 1989,p9). (Terminology: In Victoria the official term used in the Ministry of Education for career education from 1987 to 1993 was 'work education'. In this paper the term 'career education' has been used throughout except where direct reference is made to Victorian policy in that period or when quoting from a Ministry source.) 1.5 Requirements for an Integrated Delivery of Career Education Programs However Davis and Braithwaite also note that there are still schools where the career education program is delivered by a careers teacher, operating through individual counselling and class teaching, with very little contribution from the remainder of the teaching staff. In order to have the high level of staff participation in the career eduction program which is essential for an integrated approach to delivery, the NBEET report (NBEET, 1991, p 10-11) acknowledges that a considerable effort must be put into co-ordination. Elsewhere it is also acknowledged that an integrated approach, with the participation of a high percentage of the school staff, requires that professional development be provided for them to perform this role. (NBEET, 1992, p 18-19; 30). The integration of career education into the school program therefore requires considerable effort and the participation of teachers from all areas of the school, an effort which has not been made in many schools, in spite of the policy statements. An investigation was appropriate to discover whether such a process of program delivery actually resulted in better outcomes for the students in the school. The literature search revealed little evidence on this matter. The key factor which indicates an integrated program is the extent to which teachers, other than the careers teacher participate in the delivery of the program. If there is an integrated program then teachers participate by teaching about work. as part of the teaching of their own subjects by assisting students to understand themselves in relation to work, by linking their subject teaching to the applications of those leanings in the wider community of the working world through industry links, by teaching appropriate topics of career education and by participating in the informing and advising of students about subject choice and post-school options. Such participation is the indicator of an integrated program, and the absence of such participation an indicator of a program which is delivered as a separate entity. 2. PROCESS 2.1 Survey of Schools 2.1.1 Selection of Schools for the Survey It was decided to conduct the research in selected government secondary schools in Victoria. The non-Catholic non-government schools in Victoria are not systemic, in that there is no coordination of the curriculum between them. In the career education field there are no guidelines issued on the content or delivery of career education. In the Catholic secondary schools some professional development has been offered to schools by the Catholic Office of Education in the Dioceses of Melbourne, but there are no guidelines for Catholic schools on career education. The data presented by Davis and Braithwaite (1990, p28-29) indicates that the programs in both Catholic and Independent schools in Victoria is delivered in different ways from the careers programs delivery in government schools. In Victorian government schools, in 1993, the policy document Work education (Ministry of Education 1989) provided policy directions on the aims of career education and on the processes by which it should be delivered in schools. For these reasons there would be more variables affecting the student outcomes in these two types of schools and they were not included in the study. It was decided to survey selected government secondary schools The purpose of this survey was to identify ten schools which delivered their careers programs in an integrated way with a high level of staff participation and ten schools where the career education program was delivered by the careers teacher alone, or with a minimum of participation by other teachers. In order to reduce the impact on the students' competence in career planning of factors other than the mode of delivery of the careers program, it was decided that the schools initially surveyed should come from a metropolitan region, and a country region with both town and rural schools. In initial discussions with the Department of School Education they indicated that the schools in the Western (metropolitan) region were already involved in several research studies. It was decided to use the Eastern (metropolitan) region and the Barwon South-West region, since both regions had a spread of socio economic groups and would include secondary schools of all sizes, including those already on multi-campus sights. 2.1.2 Design of the Survey Instrument Initially it was thought that the survey would ask schools to indicate how much time was spent in carer education activities by each member of staff at each of the school levels from year 7 to year 12. the survey would indicate to the school the elements of the careers program which could be offered at each of these levels and state some typical aims of such a component using as a basis Career education in Australian schools (AEC 1992). A trial version of this type of survey was used with a group of teachers. It was found that the answers to the questions would involve the school administration in several hours of work using records of teacher allotments and the timetable, as well as the detailed course outlines for each faculty at each level in the school. If this type of survey was used then it could be anticipated that the return rate would be low. A second survey was devised which was based on the known career education activities usually delivered in years 7-12 in government post-primary schools as components of the careers education program. In this survey 6 areas were identified and for each component four or five alternative ways of delivering it were described, ranging from the delivery mode which involved the highest number of teachers participating to the one which involved only the careers teacher delivering the component. For each of the components the goals were identified based again on the statement of national goals for career education published earlier in 1993. (AEC, 1992) This was trialed with a group of teachers who were students in the post-graduate diploma in career education. They were asked first to complete the survey for their own school, then comment in a discussion group on the time required to complete it, the adequacy of the 6 components to cover all that was happening in their school in career education, and finally whether the descriptors of the delivery process were realistically what actually happened. It was hoped that the survey could be completed in 30-40 minutes as it was judged that this was about the time most schools would consider spending on an activity which did not result in any direct benefit to them. The teachers found it easy to complete and the descriptors adequately described the alternative ways by which each of these six components were in fact delivered. (Previous experience in surveying careers programs in schools had indicated that if the descriptors are not good then teachers will write in their own description of how it is delivered, thus negating their return). However they suggested that there should be two components to cover the delivery of career education across the curriculum within the main subject areas, one for the delivery of general information about the world of work and one for the areas of self in relation to the world of work. They pointed out that the subject areas of Personal Development, English and Social Education were often separately involved in the latter, through value clarification and exercises in self understanding through essay writing, and th is might not be recognised under the heading of general 'work education'. They also stated that some schools could integrate the learning about one of these but not the other. 2.1.3 Developing an Index of Teacher Participation The descriptors of how each component in the survey was delivered were each given a numerical value ranging from 5 for a process involving a large number of teachers to 1 where the career teacher delivered it alone and 0 if the component was not part of the school program. These scores were used to develop an 'index of teacher participation in the career education program' for each school. From the results ten schools with the highest level of teacher participation and the ten schools with the lowest level of teacher participation would be selected. Since this was the purpose of the survey, it was important that the answers to each section of the survey should be scored in such a way as to give a strong separation between the two alternative modes of delivery. At the same time the total results from each school, when plotted on a graph should represent as near as possible to a normal distribution, with the majority of the school around the mean. The survey was posted to all the schools in the two selected regions, 63 to schools in the Eastern region and 37 to schools in the Barwon South West Region, a total of 90 schools. By the due date there was a 62% response rate from the Eastern region schools and a 53% response rate from the Barwon-South West schools. 2.2 Selection of School for Stage 2 From the Index In order to have a sample of schools which reflected the Melbourne/ non Melbourne population split in Victoria the schools were chosen by the index of staff participation as follows: Group 1 (high participation) : 7 Eastern region; 3 Barwon S-W region Group 2(low participation) : 7 Eastern region; 3 Barwon S-W region An analysis of the result of the survey showed that the minimum score on the index of teacher participation was 11 and the maximum was 35, with a mean of 20.396 and a standard deviation of 5.927. There was a skewness of .313 indicating that the results were close to a normal distribution but that more schools were above the mean than below. There was a negative kurtosis of -.619, indicating that the tails of the dis tribution were flatter than in a normal distribution with the same standard deviation. This analysis of the index indicated that the purpose of the survey, to select a sample of the 10 highest scoring schools and the 10 lowest scoring schools, could proceed. The selection of the schools for stage two was done as follows: The top eleven high scoring schools there were 8 from Eastern region and 3 from Barwon South West region. Their scores were as follows: school region score 1.1 ER 35 1.2 BSW 31 1.3 ER 30 1.4 ER 30 1.5 ER 30 1.6 ER 29 1.7 BSW 27 1.8 ER 26 1.9 ER 26 1.10 ER 26 1.11 BSW 25 In order to keep the balance between Melbourne metropolitan and country regional schools school 10 was omitted in favour of school 11. The 11 lowest scoring school were seven from Eastern region and 4 from Barwon South West. school region score 2.11 ER 14 2.10 ER 14 2.9 BSW 13 2.8 BSW 13 2.7 ER 13 2.6 ER 13 2.5 BSW 13 2.4 ER 12 2.3 ER 12 2.2 ER 11 2.1 BSW 11 Again, in order to have the balance between the regions school 2.5 was omitted. When the schools were contacted to confirm that they were able to participate in stage 2, school 1.11 indicated that the careers teacher had resigned , making it difficult for the school to participate. School 1.2, with a score of 12, was substituted. 2.3 Selection of Students from the Sample Schools The most desirable group of students would have been those who had completed the full secondary school program, that is the students in year 12 in 1993. However it was recognised that schools would be unwilling to release these students for the 45-50 minut es required in Stage 2. Therefore it was decided to use year 11 students who, at the time of data collection, would be in the last semester of their year 11 program. It was also recognised that if a random sample of year 11 students was selected in each school, all the year 11 classes could be disrupted for the period of the test. This was also likely;y to result in some of the school selected by the index of teacher participation refusing to be involved in stage 2. Since it was important, for the purposes of the research, that the group 1 schools should be as clearly separated on the criterion of staff participation as possible, a compromise on the choice of students for the testing was made. Each school was requested to make a year 11 English class available. Since this was a compulsory study, any English class would be more representative of the total year 11 population in the school than any other class, although clearlynot as good a sample as one randomly selected. 2.4 Selection of an Instrument to Test the Career Maturity of the Students. A revised form of the Career Development Inventory-Australia (Lokan 1987) was used as the instrument to test the career maturity of the selected year 11 students. This instrument is based on the Career Development Inventory (Thompson and Lindeman 1984) which was developed from the research by Super and others in the Career Pattern Study. The revision of the CDI-A (ACER, 1987) was made specifically for this research, in consultation with Dr Lokan who adapted the CDI-A from the CDI. the revised CDI-A was printed in trial form at ACER. The revisions consisted of new 'correct' answers to 5 questions affected by changes in Australian education and training since 1987 and ten new World of Work questions, inserted as the first step in what might be an updated CDI-A. The CDI-A tests both the cognitive and affective areas of career maturity using a series of multiple choice questions which can be completed in approximately 40 minutes. Bride and Lokan (1994) presented a detailed discussion of the use of the CDI-A in this study was presented at the International Association of Applied Psychology Conference in Madrid, July 1994 in a paper 'Evaluation of Career Education Programs: A study involving the use of the Career Development Inventory in Australia'. A comparative analysis of the results from the 1993 sample and those of the year 11 sample of 988 students administered the CDI-A in 1987 showed that the instrument has remained a valid and reliable one for testing the career maturity of secondary school students at the year 11 level. 3 RESULTS 3.1 Comparison between the Student Scores on the revised CDI-A of Students in Group 1 Schools with Students in Group 2 Schools The total number of student answers which were processed was 381, 201 from the group 1 schools which had an integrated program of career education delivered by a range of teachers in the schools and 180 from the group 2 schools where the career education program was delivered by a career teacher, alone or a minimum of participation by other teachers in the school. Table 1 Comparing the Results on CDI-A for the Two Groups of Schools The scores of the students in the schools where there was an integrated program of career education were higher on all four scales and the composite scales, CDA and CDK. The group 1 school group also had less of a spread of scores than the group 2 school students.. The students' results on the cognitive composite, Career Development Knowledge (CDK), and on the two subscales, World of Work- Australia(WW-A) and, Decision Making (DM), which make up the cognitive scale are statistically significant at the .05 level. (The addition of the new items in the World of Work (WW) scale also produce a statistically significant difference at the .05 level but this information indicates that the results are slightly less significant when this scale is used compared with the original WW-A items). This suggests that students have a better knowledge of the world of work and a greater capacity to apply that knowledge in the decision making process when the career education program in their school is delivered in an integrated way across the curriculum with a high degree of participation in the delivery of the program by the teachers in the school, than when it is a separate component of the curriculum delivered largely by one teacher. However there is no significant difference between the students from the two groups of schools on the attitudinal composite, Career Development Attitude (CDA), or on either of the scales, Career Planning (CP) and Career Exploration (CE), which go to make up the total attitudinal scale. The students in both groups of schools state that they plan and explore their career options at a similar level of application and the results can be interpreted to show that students in both groups of schools have much the same degree of maturity in Career Development Attitude. Their attitude or application to these activities is not significantly affected by the way in which the career education program in their schools is delivered. 3.2 Analysis of Data by Gender and School Type The data also had information of the gender of the respondents. Since one of the issues raised in the literature is the differences between male and female career development the data was analysed by gender Table 2 Comparing the results of female and male respondents on each CDI-A Scale There were differences between male and female students on all the scales. These were significant differences at the .05 level. Since the maturity of girls at year 11 is likely to be greater than that of boys at the same year level it is possible that the differences between the male and female students shown in this data was simply a reflection of the congruence between general maturity and career maturity. An analysis of the variance in fact showed that there was a more significant difference in the results between males and females than there was between the two school types. It was interesting to explore the interrelationship between gender and schools type since the gender differences were statistically significant Table 3 ANOVA Results of Gender by School Type These results indicate that there is a difference in male and female response to the two different modes of delivery of the program which were under investigation in this study but the differences re not significant. Although both male and female students in group 1 schools scored higher on the on the WW scale there was a greater difference in the scores of the male students, suggesting that the boys possibly responded better to the delivery by a range of teachers as far as learning information. The difference in the male and female students scores on the WW sub-scale show that the difference between the two groups of schools on this sub-scale was in fact more a difference in the boys in each of the two groups of schools and that the girls scores contributed to the difference to a lesser degree. It is possible that the differences in response to the type of program delivery between male and female students is in fact a reflection of the different stages of maturity of the two groups at the year 11 level. The gender of the teacher who delivers the program alone could also be a factor. 4 DISCUSSION 4.1 Responses of Female and Male Students to the Delivery Mode of Career Education Programs The evidence is that the male students in the group 1 schools with an integrated career education program had higher scores on the cognitive scale CDK than the male students in the group 2 schools. The significant difference between the outcomes for the students in the group 1 schools on the cognitive aspect of career development was really a difference for the male students rather than an overall difference. There is no data in this study to suggest any explanation for this. It would certainly be a topic which could be explored in a furtherresearch. Some suggested explanations which could be explored are: * an interaction between the gender of the students and the gender of the teacher when the program is delivered by a career teacher alone; for example male students may respond less positively to a female careers teacher; * whether male adolescent students do not respond as positively as female students to career counselling, assuming that the programs delivered by a careers teacher alone were delivered on a one to one basis more often than those with a higher level of part icipation by other teachers; * whether the general level of maturity in adolescents affects their response to an individual approach to the delivery of career education compared to a classroom based approach. 4.2 The Impact on Career Decision Making of Different Rates of Maturation of Male and Female Students In 1984 Thompson and Lindeman acknowledged that there was a difference based on gender in the level of career maturity of adolescents shown by the CDI, but stated that it was not an important issue. A very different view of the importance of this difference is taken by Betz and Hackett who suggests that the earlier maturity of female students could have a very significant affect on the career pathways of female students.(Betz & Hackett, 1981 p399). Their concern was that female students were more mature at the end of their compulsory years of schooling and therefore more likely to make an earlier closure decision than boys at that stage of their career development. Applying Betz and Hackett' s ideas to girls at year 10, in the Victorian system, suggests that they would be more likely to decide for certain careers and against certain others than would boys at year 10. Because there are such rapid changes occurring in the labour market as well as in education and training, it is more important than ever that young people are flexible in their approach to career planning and that they keep open as many options as possible during the post-compulsory years of schooling. If greater career maturity results in a less flexible approach to career planning, as maintained by Betz and Hackett, then the approach to career education for female students needs to be examined. There is no information on these matters in this data, but the difference between males and females suggests that much more information should be sought on what this difference in maturity means for the career decision making of girls. In Victorian schools there are a number of affirmative action career programs operating at state and school level, mostly directed at girls in years 10-12. One possible impact on these programs could be that the programs need to be directed at much younger girls, a change which has been suggested by many of the people involved in delivering them. 4.3 The Difference in Effect of Delivery Between Attitudinal and Cognitive Areas of Career Maturity There is no indication in the data collected from the students which could explain why the cognitive aspects of career maturity are significantly affected by the mode of delivery but the attitudinal aspects are not significantly affected. It could be that the lack of difference between the students outcomes in the attitudinal aspects of the scale is an indication that there has been less attention to attitude in all programs, however they are delivered. However in the school survey school responses to item 5 (work education) and 6 (self and work) would not confirm this view since only 2 schools indicated that they did not offer programs in self and work while 3 did not offer world of work programs. Another explanation could be that the compulsion on all these students to make VCE course selections in the year previous to when they completed the CDI-A had stimulated their application to career planning and exploration and thus affected their responses to that section of the inventory, regardless of the nature of the process by which the career eduction program was delivered. If this is the case some concern could be felt that the students are stimulated to explore and plan when they experience a program delivered by a single teacher, but do not get the level of information and decision making training which would make that exploration and planning worth while. 4.4 The Challenge of Research in a Decentralised School System In 1993 there had been effective school level of decision making in Victorian government schools for at least 5 years.(Education Department of Victoria, 1985). The appointment of a careers teacher, or the decision not to appoint one, and the time allocated for them to fill this role, the precise content and nature of delivery of the program and the way, if any, that other teachers input was organised, were all decisions devolved to the school level. This decentralisation of decision making to the school level may appear to make it more difficult to compare effectiveness of the programs delivered in one school with those delivered in another. It also raises questions about the possible difficulties of getting co-operation from schools individually in a research project. In connection with the second point, it has already been shown that the co-operation of the schools in responding to the initial survey and in participating in stage 2, was achieved quite readily. Although there were no longer centrally controlled programs in career education, or any other area of the curriculum, the devolution of responsibility to the school level in 1985 had been accompanied by strong encouragement to schools to design programs which provided equal access to a sound education for all students. State guidelines were published for career education (Ministry of Education 1989) setting out goals and suggested processes. At the same time professional development and publications by the Department of School Education urged schools to conform, especially in the matters of integration into the curriculum, provision of courses at year 10 and advice on post-school options and school industry links. In the years 1989-1992 there was also an intensive professional development program for teachers associated with the introduction of 2 compulsory units of Australian Studies in the VCE, this would have greatly increased the number of teachers in each school with knowledge about the world of work and expertise in how to deliver a program of work education to year 11 students. The evidence of this study suggests that in the area of career education, although the school decides on the actual content and process of delivery, the student outcomes can be assessed and used in evaluation if the goals are clearly set out and the schools have been given assistance to assist teachers to have the expertise to implement them. 4.5 Use of Established Instruments in Evaluation of the Outcomes of an Educational Program There has been a period of distrust of such instruments and their use, but to some extent this has now turned around and there is more acceptance of the use of such tests. If that is the case there will be an increased interest in developing new instruments to test student outcomes and in exploring the use of established instruments. If the latter are to be used it needs to be demonstrated that the instrument is still reliable. In the careers field the evidence of the comparison between the results of this study with those of Lokan (ACER 1987) shows that the CDI-A has stood the test of time very well. This raises the question as to whether it could not be more widely used (possibly in a revised form) in the area of evaluation of career programs rather than effort being expended on the development of new measures. 4.6 Issues of Sampling Students in the Senior Secondary School In designing this study it was decided to make every effort to intrude as little as possible on the time and patience of the school and the students, since it was assumed that schools would be reluctant to allow staff and students to be involved in the research. It is therefore interesting that, with very little effort a satisfactory return rate on the survey was achieved. In fact it had been expected that follow up phone calls and possibly a letter might have been required to obtain this return rate. On the school returns to the survey they were asked if the school would be willing to participate in stage 2 if they were selected. A brief explanation of what would be required was included. In fact all those which returned the survey indicated that they would be willing to have their students involved in stage 2. Once the 20 schools were selected, all of them agreed to provide access to a sample of the year 11 students and moreover all were very helpful in setting times for the researcher to visit the school and have access to the class requested. The one selected school which was replaced at this stage was apologetic and their replacement was as much suggested by the possible difficulty for the researcher as for the convenience of the school. This suggests that schools are more willing to cooperate in research if it bears on a matter of educational practice, even when it involves senior school students, than some researchers anticipate. Bibliography Australian Education Council (1992) Career educationin Australian schools: a statement of national goals, school and system outcomes and evaluative arrangements, Australian Education Council, Melbourne Australian Education Council Review Committee, (1991) Young people's participation in post- compulsory education and training, Australian Government Publishing - Service, Canberra Betz, N.E., and Hackett, G., (1981) 'The relationship of career related self-efficacy - expectations to perceived career options in college men and women' Journal of consulting - psychology, Vol 28, p399-419 Davis, D., and Braithwaite, R.J., (1990) Career education in Australian - secondary schools, Crew Consultants, Ryde, NSW Education Department of Victoria, (1985) Ministerial Papers 1-6 , Victorian Government - Printer, Melbourne Employment and Skills Formation Council (1992) The Australian vocational certificate training - system, National Board of Employment, Education and Training, Canberra Evans, R.P., Hoyt, K.M., Mangum, G.L. (1973) Career education in the - middle / junior high school , Olympus Publishing Company, Utah, USA Gysbers,N.C., (1994) 'From position to program', Australian journal of career development, - Vol 3, No 1, p18 Havinghurst, R.J., (1953) Human development and education, Longmans - Greer, London Isaacson, L.E., (1971) Career information in counselling and teaching, Allun and Bacon, - Boston, USA Lokan, J.,(1984) Career development inventory: Australia, Australian Council for Educational - Research, Victoria McCowan, C., (1987) Career education and counselling in secondary - schools and colleges, - Curriculum Development Centre, Canberra Ministry of Education, Victoria, (1989) Ministerial policy: work education , Victorian - Government Printer, Melbourne National Board of Employment, Education and Training (1991) Strengthening career education - in Australian schools, Australian Government Publishing Service, Canberra National Board of Employment, Education and Training (1992) A national training framework - for career co-ordinators: a proposal, Australian Government Publishing Service, Canberra Super, D.E., (1950) 'Vocational adjustment: implementing a self-concept' - Occupations, Vol 30, No 3, reprinted Career development quarterly, - (1988) Vol 36, p 351 - (1973) 'The career development inventory', British journal of guidance and - counselling, Vol 1, No 2 p 37 - (1985) New dimensions in adult vocational and career counseling, National Centre for Research in Vocational Education, Ohio, USA Thompson, A.S., Lindeman, R.H., (1984) Career development inventory - Volume 2: technical manual, Consulting Psychologists Press, Inc., Pao- Alto, California Victorian Institute of Secondary Education, (1979) News bulletin, Victorian Institute of - Secondary Education, Melbourne