SPACE FOR SYMPOSIUM NOTES

Some points to consider from the Introductory Paper, Paper #1 and the critical friend/discussant comments

(Use these points as a framework for the remaining papers)

DETAILS FOR ONGOING CONVERSATIONS

If you would like to continue the conversation that we have begun in this symposium, please provide your contact details below and leave this sheet with us

Name: ________________________________

Institution: ___________________________

Contact Details:

Postal address:

Phone:

Fax:

E Mail:

THE SYMPOSIUM PAPERS

INTRODUCTORY PAPER

Professional practice research derives from Stenhouse's (1975) view of the teacher as researcher and from the Action Research movement which his ideas spawned in the UK in the mid to late 1970's and in Australia in the late 1970's and into the 1980's (See Kemmis and McTaggart, 1988). In more recent years the Teacher Research and Practitioner Research initiatives have sharpened the focus on professional practitioners as they have interrogated and investigated their professional practice as a basis for critically-informed advocacy and activism as well as transformative/reconstructive action. Somekh's (1995) view of Action Research in social endeavours is worth reading in this regard. A much more recent example is Groundwater-Smith's (2000) characterisation of evidence-based practice with in knowledge-building/creating schools.

Professional practice research, then is an interrogation and investigation of professional practice by the professional practitioners themselves. It is research that is critically-informed, politically-activist, and action-oriented. It aims for a deeper understanding of professional practice, an enriched capacity to engage in professional practice and a commitment to an ongoing quest for quality improvement in professional practice.

This characterisation of professional practice research derives from a view of Action Research which is critical and collaborative (Aspland, Macpherson, Proudford & Whitmore, 1996); from an approach to collaborative activities (Macpherson, Aspland, Elliott, Proudford, Shaw & Thurlow, 1998); from a stance about professional learning (Macpherson, Brooker, Aspland & Elliott, 1998); from a view of curriculum leadership in which a range of stakeholders collaborate in leading learning (Macpherson, Aspland, Brooker & Elliott, 1999; Macpherson, Aspland & Brooker, 2001); and from a position about collaborative research (Macpherson, Aspland & Brooker, 2000).

Our ontological position is one of democratic participation and inclusion; our epistemological stance is associated with socially-critical constructions of knowledge; and our methodological approach is a "working with" rather than a "working on" people. For us, people are research participants and not research subjects. Our overall view of research sits within the view that the purposes of educational research are to extend theory, illuminate practice and inform policy. Ethical matters are, of course, of utmost importance in a characterization of professional practice research within what is a heavily value-laden position. We grapple with these matters by seeking to maintain a balance between what we term passion and responsibility in research efforts (Macpherson, Brooker & Aspland, 2001).

So, what does it mean to cross borders and to explore new frontiers in professional practice research? Take the following statements, for example, and consider the implications embedded in the questions which follow each statement.

But,

But,

These ideas and questions, then, are the conceptual framing that we are providing for this symposium. This Introductory Paper is an invitation for us all to explore the territory of professional practice research. Can we argue for its recognition or place and what are the opportunities or space for us to engage in it?

Aspland, T., Macpherson, I., Proudford, C. & Whitmore, L. (1996). "Critical collaborative action research as a means of curriculum inquiry and empowerment." Educational Action Research, 4 (1):93-104.

Groundwater-Smith, S. (2000). "Evidence-based practice - Towards whole school Improvement", Paper presented at the Annual Conference of the Australian Association for Research in Education, Sydney, December.

Macpherson, I., Aspland, T., Elliott, R., Proudford, C., Shaw, L. & Thurlow, G. (1998). "A journey into a learning partnership: a university and a state system working together for curriculum change", Chapter 8 In Atweh, B,, Kemmis, S. & Weeks, P. (1998). Action Research in Practice: Partnerships for Social Justice in Education. London and New York: Routledge.

Macpherson, I., Brooker, R., Aspland, T. & Elliott, B. (1998). "Putting professional learning up front: A perspective of professional development within a context of collaborative research about curriculum leadership." Journal of Inservice Education, 24 (1):73-86.

Macpherson, I., Aspland, T., Brooker, R. & Elliott. B. (1999). Places and Spaces for Teachers in Curriculum Leadership. Canberra: Goanna Print and Aust. Curriculum Studies Association.

Macpherson, I., Aspland, T. & Brooker, R. (2000). "Is there a future for collaborative research?" Paper presented in a symposium, " Interrogating collaborative research - Who is inside and who is out?" at the Annual Conference of the Australian Association for Research in Education, Sydney, November.

Macpherson, I., Aspland, T. & Brooker, R. (2001) " Passion and Responsibility in Qualitative Research - 'Both/And" or 'Either/Or'? A dispassionate reflection and a deliberative reconstruction". Roundtable Paper Division D - Qualitative Research) presented at the Annual Meeting of the American Educational Research Association, Seattle, 10-14 April.

Macpherson, I., Aspland, T. & Brooker, R. (2001). "Traversing a decade of curriculum thinking and practice: A conversational case study of curriculum leadership from down under". Roundtable Paper (Division B -Curriculum) presented at the Annual Meeting of the American Educational Research Association, Seattle, 10-14 April.

Somekh, B. (1995). "The contribution of Action Research to development in social endeavours: a position paper on action research methodology", British Educational Research Journal, 21(3):339-355.

A proactive stance by school-based personnel is certainly a border crossing when it comes to research partnerships between universities and schools!

The new frontiers are associated with both the establishment of the research partnership and its sustainability. Within the professional practice research partnership, the parity of esteem among the various researchers in terms of their unique and complementary contributions; the ownership of the research focus, approach and outcomes; and the significance, usefulness and applicability of the outcomes are all an ongoing challenge.

The title of the research project in this partnership is: "Sustaining a research base for futures-oriented curriculum planning and implementation at the school level: A local school district (and a small number of schools within the district) working in partnership with a research team from the Faculty of Education to theorise school-level curriculum work within contemporary and future contexts".

The research project focuses on a futures-oriented perspective on curriculum planning & implementation. District personnel in schools identified the need to develop a research base for their school-level curriculum initiatives within the complex contexts which demand, for example, outcomes-based approaches to curriculum & assessment, the incorporation of key learning areas into the curriculum, and the consideration of new basics & productive pedagogies as a means of contributing to the "Smart State" (Mooloolaba School District, 2001a, 2001b Education Queensland, 1999, 2000a, 2000b, 2000c, 2001; Queensland School Curriculum Council, 1998, Ministerial Council on Employment, Education, Training & Youth Affairs, 1999, Watt, 2000.). How can schools at the local level respond to this multiplicity of demands in developing curriculum programs that meet the diverse needs of learners, & take them on pathways leading to worthwhile learning outcomes? Case studies in four schools are exploring this question.

The aims of the research project are to:

1. investigate and document via a case study approach the ways in which school sites interpret national policies and system priorities as a basis for local curriculum planning and implementation

2. critique the ways in which these school sites operate using theoretical frameworks associated with change and accountability in futures-oriented curriculum planning and implementation at the local level

3. generate a range of emerging propositions about sustaining a research

4. validate these emerging propositions though dissemination to local and wider networks of schools and other relevant educational personnel

5. develop a theoretically-informed and practically-oriented basis upon which further research-based activity can be built

Research, with aims such as these, is of local, national & international significance as it helps to extend the theory about how local schools develop a research-based curriculum culture within contexts of a futures-orientation, change & accountability; it helps to illuminate policy about how best a system & District can support schools in developing such a culture; and it helps to improve practice in supporting & sustaining a theoretically/critically-informed research-based culture both for the system & the District as well as for local "sites". The research also is significant in that it provides a platform for wider-scale research-based inquiry and practice at system, District and school levels. Arguably, the research is most significant in that it involves teachers & other stakeholders at the local level in interrogations, investigations, implications & (re)inventions of their own practice relative to the research focus (See research title, question, aims & strategies) of this project (in partnership with District & University personnel).

Besides the substantive focus of the project, the procedural aspects are very significant in terms of what can be learned about research partnerships between Universities & industry partners. In this project, partnership is being defined & acted upon at a number of levels. The Principals' Group (chaired by the Principal of one of the participating schools & liaising with the District Director & the District Manager of Education Services) is providing the first level of partnership with the Chief Investigators from QUT. These district & QUT persons also form a Management Team to oversee the project. (It meets at least monthly). A second level of partnership is occurring between school-based personnel & the QUT personnel at the school-level at which the case studies of futures-oriented curriculum planning & implementation are taking place. A third level of partnership is occurring when QUT personnel interact with school-level personnel in critiquing the data emerging from the case studies, as a basis for an ongoing documentation & dissemination of the case studies (across the entire School District with the support of the Staff College within the District). At this level, QUT personnel are providing the broad (as well as the more specific) theoretical/critical frameworks for critiquing the sorts of curriculum cultures & frameworks which are emerging at each case study site. A fourth level of partnership involves a Reference Group (It meets at least quarterly) which comments on the overall progress of the project; provides feedback about ongoing directions for the project itself & for wider-scale dissemination of research findings; & links with other District and School initiatives involving a partnership with QUT (e.g. Technology Project). A fifth level of partnership for this project involves all personnel in developing action plans for wider-scale dissemination (led by District personnel) & in developing an ARC Linkages Grant Application in early 2002 - for 2003 to 2005 (led by QUT personnel).

The research approach is a case study approach where the purpose is to describe in as thick detail as possible what is happening relative to the research topic at the four local "sites". The research approach sits within Brown & Dowling's (1998) view of educational research as "the continuous application of a particularly coherent and systemic and reflexive way of questioning, a mode of interrogation (Page 1). The approach heeds Wolcott's (1994) assertion that it is better to err on the side of description than analysis; but it also takes Simons' (1996) point that case study is a way to "see things anew". Using a case study approach (See Yin, 1989, Stake, 1995 & 1998, Bassey, 1999, Macpherson, Brooker & Ainsworth, 2000), this project focuses on the authentic participation of school-based personnel in researching their own practice relative to futures-oriented curriculum planning and implementation at the local level. Such participation is set within the ethos of Action Research (See Aspland, Macpherson, Proudford & Whitmore, 1996, Atweh, Kemmis & Weeks, 1998, and of Practitioner Research (a form of Action Research where the emphasis is on practitioners researching their own practice and thereby developing a research-based culture as a milieu for their work - See Anderson & Herr, 1999, Cochran-Smith & Lytle, 1999, Brooker & Macpherson, 1999, Groundwater-Smith, 2000). The research, therefore, will be guided by the application of principles that aim to maintain productive and sustainable relationships among the various partners (See Macpherson, Brooker & Ainsworth, 2000). A range of research strategies are being used be used to complete the case studies at the local sites. Primarily, these strategies will focus on:

• surveying existing documentation of curriculum activity at each site (in terms of the research focus and overarching research question) (See Aim 1);
• interacting with personnel at each site (through narratives, interviews and ongoing conversations) regarding their perspectives on curriculum activity (that has occurred, is occurring and is likely to/should occur) (See Aim 1);
• intersecting descriptive data relative to Aim 1 with relevant theoretical/critical frameworks (See, for example, Macpherson, Brooker, Aspland & Elliott, 1998, Macpherson, Aspland, Brooker & Elliott, 1999, Macpherson, 2000, Fullan, 1998, Jennings, 2000) (see Aims 2 & 3);
• working collaboratively (though ongoing conversations with personnel at each site, & using emerging frameworks from the above strategies) to interpret the descriptive & first-level analytical data and to develop a set of emerging propositions for each site about the research question (See Aim 3)
• Validating these propositions across sites & within the District through an ongoing collaboration in interpreting & identifying implications out of the case study data for further consideration at each site, across sites, & within the District. (See Aims 4 & 5) (See Denzin & Lincoln, (Editors), 1998, Brooker, Macpherson & Aspland, 2000, Peshkin, 2000);
• Disseminating research processes & outcomes throughout the project via a variety of communication forms within & across sites, across the District, & in the relevant professional communities (conference and journal papers) (See Aim 5); and
• Developing a theoretically- & critically-informed basis for further research activity at each site (& other sites) within & beyond the District: & for the preparation of an ARC Linkages Grant Application for 2003-5 (See Aim 5).

The emerging themes about professional practice research from this research partnership are as follows:

A consideration of these themes within the context of the statements and questions at the end of the Introductory Paper would suggest that matters of balance between theory and practice and between university and school personnel; of the impact of this sort of research on systemic and national policy; of ethics in relation to participation, commitment and ownership; and of effective dissemination forms for both theory-oriented and practice-oriented audiences are of importance for us as we go on charting the territory of professional practice research.

Anderson, G.L. & Herr, K. (1999). "The new paradigm wars: Is there room for rigorous practitioner knowledge in schools and universities?", Educational Researcher, 28(5):12-21.

Aspland, T., Macpherson, I., Proudford, C. & Whitmore, L. (1996). "Critical collaborative action research as a means of curriculum inquiry and empowerment." Educational Action Research, 4 (1):93-104.

Atweh, B,, Kemmis, S. & Weeks, P. (1998). Action Research in Practice: Partnerships for Social Justice in Education. London and new York: Routledge.

Bassey, M. (1999). Case Study Research in Educational Settings. Buckingham, Philadeplhia: Open University Press.

Brooker, R., Macpherson, I. & Aspland, T. (2000). "Taking Action Research from the local to the global via an hermeneutic spiral". Paper (Division D - Qualitative Research) presented D at the Annual Meeting of the American Educational Research Association, New Orleans, April.

Denzin, N.K. & Lincoln, Y.S. (Editors). (1998). Collecting and Interpreting Qualitative Materials. Thousand Oaks: Sage.

Education Queensland. (1999). Queensland State Education - 2010. Brisbane: Author.

Education Queensland. ( 2000 a, May). Policy and Guidelines for Core Curriculum for Years 1-10 in Education Queensland Schools (Draft). Brisbane: Author.

Education Queensland. (2000b). New Basics - Curriculum Organisers. Brisbane: Author.

Education Queensland. (2000c). New Basics Technical Paper. Brisbane: Author.

Education Queensland. (2001). Key Learning Areas and New Basics (Special Lift-out). In Education Views 10(1), January 19, 2001.

Fullan, M. (1998). "The meaning if educational change: A quarter of a century of learning" (pages 214-228) in Hargreaves, A., Lieberman, A., & Fullan, M. (Editors), International Handbook of Educational Change, Volume 1. London: Kluwer Academic Publishers.

Groundwater-Smith, S. (2000). "Evidence-based practice - Towards whole school Improvement", Paper presented at the Annual Conference of the Australian Association for Research in Education, Sydney, December.

Macpherson, I. (2000). "Investigating space for significant stakeholders in curriculum leadership: the beginnings of an ongoing conversation". Curriculum Perspectives, 20(1):33-44.

Macpherson, I., Brooker, R., Aspland, T. & Elliott, B. (1998). "Putting professional learning up front: A perspective of professional development within a context of collaborative research about curriculum leadership." Journal of Inservice Education, 24 (1):73-86.

Macpherson, I., Aspland, T., Brooker, R. & Elliott. B. (1999). Places and Spaces for Teachers in Curriculum Leadership. Canberra: Goanna Print and Australian Curriculum Studies Association.

Macpherson, I., Brooker, R. & Ainsworth. P. (2000). "Case study in the contemporary world of research: using notions of purpose, place, process and product to develop some principles for practice". The International Journal of Social Research Methodology Theory and Practice, 3(1):49-62.

Ministerial Council on Education, Training and Youth Affairs. (1999). The Adelaide Declaration. Canberra:Author.

Mooloolaba School District. (2001a). Mooloolaba District/QUT Research and Development Initiative. Mooloolaba:Author.

Mooloolaba School District. (2001b) Provision of Curriculum and learning Technology Support to Schools in the Mooloolaba District. Mooloolaba: Author.

Queensland School Curriculum Council. (1998). Interlink, Number 7, August, 1998. Brisbane: Queensland Government Press.

Peshkin, A. (2000). "The nature of interpretation in qualitative research", Educational Researcher, 29(9):5-10.

Simons, H. (1996). "The paradox of case study", Manuscript of paper submitted to Cambridge Journal of Education for publication in April, 1996.

Stake, R. (1995). The Art of Case Study Research. Thousand Oaks: Sage.

Stake, R. (1998). "Case Studies", in Denzin, N.K. & Lincoln, Y.S. (1998). Strategies of Qualitative Inquiry. Thousand Oaks: Sage.

Watt, M.G. (2000). "The National Agenda, 1996-2000: Its impact on curriculum reform in the States and Territories". Curriculum Perspectives, 20(3):37-48.

Wolcott, H. (1994). Transforming Qualitative Data: Description, Analysis and Interpretation. Thousand Oaks; Sage.

Yin, R. (1989). Case Study Research: Design and Methods (Revised Edition). Thousand Oaks:Sage.

In 1992, the Queensland Senior Home Economics syllabus underwent a minor revision. In that revision, the main changes were to assessment criteria and some aspects of learning experiences. In 1996, the Home Economics Subject Advisory Committee submitted proposals for a major revision of the syllabus, to focus on course organisation, subject matter and assessment. Particular attention was paid to course structure, to allow a degree of specialisation within the areas of study, and to ensure that any course of study developed from the syllabus would address an adequate range of the features unique to Home Economics as an area of study.

The major aspects of the proposed revision were:

1. the recognition that the unique focus of Home Economics as an area of study is the development of the knowledge, processes and skills to enhance the well-being of individuals and families;
2. the development of an underpinning theme of the individual and the family, to provide a focus for all studies within the syllabus;
3. three areas of study: food studies, living environments, and textile studies;
4. a structure within each area of study which incorporates a core unit plus a range of elective units;
5. the development of a socially-critical approach, to be incorporated both within learning experiences and within objectives and criteria; and
6. the incorporation, where appropriate, of a strong practical focus.

In addition to the formal purposes for the evaluation, the researchers were interested in the changes to teachers' levels of competence and confidence throughout the implementation of the Home Economics syllabus. In this paper we have outlined the research process and provided a summary of the data.

Data collection in 1999 was completed in five phases (see Table 1). In order to progressively monitor the Trial-Pilot implementation, school visits in 1999 were undertaken in two stages: school terms 2 and 3 (Phases 2 & 3 in Table 1).

In addition to the data collected from teachers and students, school curriculum coordinators (mostly a member of the administration team) were interviewed to gain some understanding of the curriculum context for Home Economics in schools.

Data collection in 2000 was completed in five phases (see Table 2). In order to progressively monitor the trial-pilot implementation, school visits in 2000 were undertaken in two stages: school terms 2 and 3 (Phases 2 and 3 in Table 2).

At both the October 1998 (first year of implementation) and 1999 (second year of implementation) Teacher Conferences, participants were asked to indicate their level of competence to implement the new syllabus (see Table 3) and to identify factors that affected that competence (see Table 4).

* In October 1999, 15 teachers (58%) commented on the fact that they felt more competent than they had previously.

** Five teachers (19%) added that they were still improving/had some way to go.

* In 1999, 8 teachers (32%) commented on the fact that they felt more confident than they had previously

At the October 2000 Teachers' Conference, teachers were asked to identify the factors that have contributed to the growth of their competence and confidence in relation to working with the Home Economics syllabus (see Table 7).

In terms of confidence, experience and support from colleagues were again referred to as factors having a positive influence on their confidence. In addition, enthusiasm was seen as an important factor. After a year of curriculum making, enthusiasm was positively related to the opportunity to interact with colleagues outside of the school and to support from the school administration (see Table 6).

In terms of both competence and confidence, assessment was identified early as a factor of concern and that level of concern had increased after a year of implementation (see Tables 5 and 6).

After two years of the curriculum-making project, teachers identified a range of factors that had contributed to the growth of their competence and confidence throughout the curriculum-making project (see Table 7). The factors can be grouped into three broad areas:

• Networking with colleagues (Items 1, 4, 5, 9)
• Experience gained from working with the syllabus (Items 2, 6, 7, 12, 13, 15)
• Positive feedback from curriculum authority's processes and from students (Items 3, 8, 14, 16)

The study has identified that issues of teacher competence and confidence are important contributors to the implementation of a new curriculum. It has also shown that the factors which influence teachers' competence and confidence, are grounded in the curriculum-making process. These findings have implications for curriculum making projects. In circumstances where new curricula are being introduced, it is not sufficient for only a limited number of teachers (as is the case in a curriculum trial) to have access to the full range of professional learning experiences that are necessary to implement successful curriculum change. Opportunities such as networking with colleagues, first hand engagement with the curriculum-making process and positive feedback must be inherent in the curriculum implementation process.

A project like this one, then, has implications for those who would engage in curriculum-making as an example of professional practice research. Notions of confidence and competence point to the need for empowering practitioners as researchers.

The Evaluation of the Pilot Senior Syllabus in Aboriginal and Torres Strait Islander Studies was funded to assess the syllabus in respect to a range of criteria required of new syllabi by a curriculum authority in Queensland. Following the trial pilot of the syllabus and the adoption of recommendations made by the evaluation team, approval was given for general implementation of the syllabus in Queensland Secondary schools in 2003.

The roles of the evaluators was determined by the curriculum authority (Dudley 1992) and was framed by the purposes of piloting a subject, which are:

1. to discover what the needs of teachers are in their implementation of the syllabus
2. to continue the development and dissemination of the standards of assessment with the state review panel and district review panels.

Thus, it was determined by the authority that the evaluation should engage in the following tasks:

• an examination of the work programs from all pilot schools, with particular note of the teacher and material resources available and used in the schools;
• an examination of how schools go about developing work programs;
• an evaluation of the actual implementation of the syllabus via the work programs to determine needs for source material, support programs and in-service programs required to ensure effective teaching, learning and assessment;

• clarification of appropriate standards of assessment;
• any other information that can lead to the improvement of the syllabus.

The Evaluation Team (Aspland, Synott & BinDol, 2001) however, challenged the traditional model of curriculum evaluation at the tendering stage and pushed the boundaries of the project to be more responsive to contextual features and guidelines pertinent to conducting evaluations in this study area.

In particular this evaluation was guided by a range of principles identified by the National Aboriginal and Torres Strait Islander Education Policy, to which the Queensland Government has subscribed. Just as the development of the syllabus was undertaken by a Subject Advisory Committee, with significant Aboriginal and Torres Strait Islander consultation, this evaluation attempted to establish a more inclusive and culturally-responsive approach in its undertaking. Thus, in designing the research project key evaluation principles from the relevant literature in Aboriginal and Torres Strait Islander Education, were juxtaposed with the traditional criteria set by the curriculum authority. These principles and criteria provided the framework for the evaluation of the syllabus.

Extrapolating core approaches from a range of literature prior to 1980, Catchpole (1981) presented the following evaluation questions for Aboriginal and Torres Strait Islander Studies Courses:

Aims: Does this syllabus direct its aims at:

1. Attacking racial prejudice?
2. Developing tolerance and understanding of other cultures?
3. Promoting the development of pride in Indigenous identity?

People:

1. Were Indigenous people involved in the development and teaching of this syllabus?
2. Are Indigenous people involved in this evaluation?
3. What attitudes do the teachers of this course have towards Indigenous people?
4. Do teachers have the expertise, the knowledge, the sensitivity to teach an Indigenous Studies syllabus?

Selection of Content:

1. Does the syllabus cover a selection of Indigenous cultures/lifestyles? i.e. does content include examination of (a) traditional cultures; (b) culture-contact period;

(c) contemporary Indigenous peoples lifestyles?

2. Does content examine race relations and associated concepts in relation to 1 (b) and (c) above?
3. In relation to traditional society, are the concepts and information being taught correct in relation to understanding traditional society?
4. Does content include sacred/secret information? If it does, who has revealed this information? Has it been sanctioned by Indigenous people?

Methodology:

1. Are teaching strategies being used which involve the affective and behavioural aspects of attitudes as well as the cognitive?
2. Are such strategies as role-playing, socio-drama, puppet theatre, simulation games, small group discussions, debating, films, literature and excursions being used?

Resources

1. Are the textbooks and teaching materials used by teachers appropriate?
2. Are teachers aware of the dubious merits of many textbooks and teaching materials about Indigenous Australians?
3. Is the material available for use by students in the school library appropriate?

The Recipients

1. Have syllabus developers and teachers considered the school clientele and its community setting in the selection of content and methodology?
2. Is content and methodology appropriate for the school clientele and its community setting?

Similarly, the report of the Commonwealth Aboriginal Studies working group (Budby, 1982) emphasised the following essential features in the design of Aboriginal and Torres Strait Islander Studies:

• Teachers should have adequate time, to allow for the appropriate treatment of materials
• A planned and sequential approach to content needs to be adopted.
• A recognition of the need to balance the study of traditional and contemporary Indigenous cultures as imperative
• A recognition of the need for teacher expertise in assessing resources as essential.
• The involvement of Aboriginal and Torres Strait Islander people through community consultation.
• A consideration of the learning needs of Indigenous students.

Therefore, in considering the implementation of the syllabus and the designing of teacher work programs, the evaluation team adopted this framework through the data collection.

In a published book on the Effective Teaching of Aboriginal and Torres Strait Islander Studies, Groome (1994) advised that the basic criteria in evaluating the effectiveness of teaching Indigenous Studies should include the following:

Students should gain:

• understandings of the range of Indigenous lifestyles, past and present;
• understandings of the complexities and subtleties of the lives of Indigenous people over times and between groups;
• understandings of how racism has operated in Australia, especially how it has affected the relationships between Indigenous/Non-Indigenous Australians, and the life chances of Indigenous Australians.

Groome emphasised the following:

• The primacy of consultation /partnership with Aboriginal communities and individuals;
• The need to incorporate aims which relate to the unique needs and characteristics of school contexts and the particular groups of students being taught;
• Recognition of local community background, and the attitudes of the local school contexts;
• The adequacy of resources in respect to people, places, and media.

These principles informed the evaluation of the pilot syllabus and provided a useful framework for undertaking aspects of the study.

Finally, we were also responsive to relevant information from the Final Report of the National Review of Education for Aboriginal and Torres Strait Islander Peoples (Commonwealth of Australia 1995) that identified Aboriginal and Torres Strait Islander Studies as:

According to the Report, the key requirement for the effective teaching of Aboriginal and Torres Strait Islander Studies is the professional development of teachers on a wide scale (Commonwealth of Australia 1995, 97).

Key principles and Recommendation 27 of the Final Report of the National Review of Education for Aboriginal and Torres Strait Islander Peoples (Commonwealth of Australia, 1995), contained the following clauses:

• Making maximum use of Aboriginal and Torres Strait Islander peoples in all aspects of course development and implementation and having an explicit preference for employment of appropriately qualified Aboriginal and Torres Strait Islander peoples to teach the courses;
• Provision for the inclusion of local content at the school level in the development of Aboriginal and Torres Strait Islander curriculum frameworks, on the advice of local communities;
• Ensuring that Aboriginal and Torres Strait Islander Studies curricula encompass both traditional and contemporary cultures and lifestyles;
• Accreditation and assessment bodies should formally recognise the prior knowledge and understanding that Aboriginal and Torres Strait Islander students hold in relation to these courses.

This literature portrayed a consistent set of approaches and criteria in the evaluation of Aboriginal and Torres Strait Islander Studies courses. It acknowledges the special nature of Aboriginal and Torres Strait Islander Studies courses in the Australian educational context and the need to evaluate such courses within a framework that is informed by the perspectives and procedures of a diverse community of Aboriginal and Torres Strait Islander peoples. Thus our professional practice research need to move beyond a traditional essentialist model of curriculum evaluation and take a more inclusive positioning that valued and incorporated the values and cultural knowledges of its stakeholders.

As such, this evaluation sought to examine the situational influences on the syllabus, the opinions of the key stakeholders involved as to the strengths and weaknesses of the syllabus, and how student performances are most affected. The evaluation was designed to discern and discuss significant features of the syllabus and the critical processes involved in implementing it. It was essential to also identify the desirable components of the syllabus from the stakeholders point of view as well as the components or processes that are perceived to be undesirable. The syllabus evaluation was composed of four distinct components: observation, inquiry (1999) and explanation and reporting (2000). The task of the evaluators, as we designed the project, was to "tell it like it is" revealing the stakeholders reactions to the syllabus to the Board. This was done through developing a 'portrayal' of the way in which the syllabus was adopted and implemented in unique sites, the issues surrounding this process, the reactions of the people involved in the process (principals, teachers, students, parents and community members) and the complexities and patterns of experiences that were documented across seventeen school sites.

This study adopted a multi-methods approach within the curriculum evaluation project in order to elicit a quantitative and a qualitative data base. The early part of 1999 focused on the first stage of the project that of observation and interview. This allowed the evaluators to develop a rapport with the significant stakeholders in each site as well as gain an initial understanding of how each teacher was using the syllabus in a particular context. This was essential in research of this type. The process was enriched by a series of focus interviews (Minichiello et al., 1995) and teacher journal writing that provided a more extensive qualitative data base to enrich and check the observations within the study. Teacher, student, parent and community member narratives, both written and spoken, were designed to be a key construct within this methodology. The qualitative data elicited in this manner:

Later in 1999 the quantitative data was gathered through the administration of a survey to all teachers, students, parents and administrators engaging in the trial project.The questionnaire consisted of a number of sections that measured teachers' perceptions of

This set of data was of great interest to the curriculum authority but the team argued that it was necessary to enrich these findings with a series of case studies.

In both 1999 and 2000 all of the significant stakeholders across the trial sites were invited to participate in two focus group interviews based on personal journal writing of their experiences with the syllabus. The framing of the dialogue within this research community was significantly shaped by the "narrative" literature, considered both a mode of inquiry as well as a research method. It has been argued that narrative (Clandinin and Connelly,1990) is the most appropriate form of research set in educational settings of this type where the research participants are given the opportunity to access their own thinking about their experiences with the syllabus in their unique context. Each community recounted their experiences through "stories" (or narratives) that brought to the fore "the images, rituals, habits, cycles, routines and rhythms" that constituted their experiences of the syllabus in a specific socio-cultural context. This form of inquiry is most suitable for a curriculum evaluation project that aims to access the experiences of teachers, students and members of the local communities that vary greatly in their history and their political and socio-cultural constructs. Further the use of conversation and storying enabled participants to express their experiences in ways that did not cause discomfort or alienation.

The research team transcribed the interviews and collated stories as the basis of the analysis. The initial analysis was reported back to the participants through the use of site-based narratives to capture the essence of teachers, parents students and community members experiences. The participants were invited to read the analysis with a view to altering, reconstructing, adding, deleting the data as a form of member checking and triangulation of the data.

The secondary analysis generated a number of case studies for reporting. The key foci of the case studies was phenomenological in essence in order to:

• portray the experiences (dilemmas, struggles and successes) of stakeholders in trailing the syllabus in specific sites;
• highlight idiosyncratic contextual variables that impact on teachers' planning and implementation of the syllabus;
• identify the viability and appropriateness of the syllabus constructs and key concepts in specific contexts;
• capture student response to the implementation of the syllabus in specific contexts;
• highlights the appropriateness of syllabus resources in specific contexts;
• record the assessment strategies teachers adopted in specific contexts; and
• document teachers' personal/professional reactions to the syllabus in specific contexts.

The case studies were of significance to both the curriculum authorities and the participants.