ASSESSMENT & REPORTING IN NSW SCHOOLS


98 Abstracts

LAURIE BRADY

A Paper Presented at The Australian Association For Research

In Education Annual Conference Nov 29 - Dec 3, 1998, Adelaide

Assoc. Prof. Laurie Brady

University of Technology Sydney

PO Box 222

Lindfield 2070

Introduction

The national curriculum was introduced for schools in Australia in the early nineties. Eight key learning areas (English, Maths, Science, Technology, Human Society and its Environment, Personal Development Health and Physical Education, the Creative Arts, and Languages other than English) were specified, and each was to have a 'statement' or agreed national position, and a profile or description of progress in learning outcomes. The states were to develop their own statements and profiles consistent with the national position.

Adoption by the states was slow and inconsistent. A survey of the state departments of education by the Curriculum Corporation in July 1997 indicated that almost all states were using the statements and profiles as a basis for curriculum development though with local variation, and that they perceived the nationally developed documents as providing a common language, a national set of standards, and a framework for planning, assessing and reporting.

As a result of a review, Focusing on Learning in 1995, NSW abandoned the nationally developed statements and profiles for implementation through syllabuses, yet remains committed to 'a manageable number of outcomes'. While the most common modification made to the national profiles by other Australian states has been a reduction in the number of outcomes, outcomes have been adopted nation-wide as a means of system accountability because they are overt, observable and therefore assessable indicators of student achievement.

While student assessment has been a perennial and contentious area of debate among educators in this country, the relatively recently created need to assess within an outcomes framework, has caused a resurgence of interest.

After the initial impact of the national curriculum in the early to mid nineties, there was a hiatus before the implications for assessment were fully addressed. In NSW, the Department of School Education (DSE), restructured as The Department of Education and Training (DET) in 1996, produced publications on assessment in 1996 and 1997, both of which focused on how to assess within an outcomes framework.

This article reports on survey research in a sample of primary or elementary schools in NSW which investigated teachers' assessment and reporting practices in relation to outcomes. As the research is the only study known to the author which has examined assessment and reporting practices within an outcomes framework, it is considered to be of value to curriculum developers as they develop curriculum and support materials for teachers; teacher educators as they skill prospective teachers in current and desirable assessment practice; and policy makers as they determine future directions and staff development.

The literature

The assessment literature in Australia in the nineties focuses on three generic areas. First there is the literature that seeks to define assessment and identify its purpose. Brady and Kennedy (1999), Brady (1995), Conner (1991), Griffin and Nix (1991) and Duncan and Dunn (1988) seek to clarify the concept, and Groundwater-Smith and White (1995) report on the five functions of assessment listed by Eisner (1993, 24-5) in examining the issue of assessment and accountability.

Second, there is the literature which indicates principles for effective assessment (Department of Education and Training 1997, Board of Studies 1996, NSW Department of School Education 1996, Brady 1995, Broadfoot 1991, Griffin and Nix 1991). Such principles involve assessment as varied, valid, engaging the learner, valuing teacher judgments, being 'situated', being sensitive to 'wholes' rather than discrete elements, and being a continuous and integral part of teaching and learning.

Third, there is the literature which presents assessment strategies. These are most fully itemised in Brady and Kennedy (1999) as anecdotal records, checklists, conferences, contracts, diaries, direct questions, extended writing, extrapolation, interviews, journals, matching, multiple choice questions, peer and self assessment, performance assessment, portfolios, rating scales, running records, sentence completion or cloze and sociometry.

The reporting literature largely parallels the assessment literature in focusing on purpose, principles and strategies. In fact, the two are often addressed together in much the same way as teaching and learning. Again, Brady and Kennedy (1999) provide a comprehensive list of reporting strategies subsumed under reports; parent-teacher interviews; portfolios; homework; written material; curriculum and policy statements; open days, speech nights, assemblies; and public displays.

However, reporting also receives additional coverage. Groundwater-Smith and White (1995) provide a fine analysis of the information needs of parents, suggesting that these needs are cyclical (varying through the year), contextual (involving the community) and social (relating to the processes conducive to sharing information). Recording as the basis for effective reporting, also figures prominently in the work of Broadfoot (1991) and Conner (1991).

Brady (1999, 1997) found that while the prevailing constructivist view in Australia by which learners and teachers are regarded as independent makers of their own knowledge, operated against a strong behaviourist approach to outcome based education, outcomes, as observable benchmarks of student performance, have nonetheless been adopted by teachers in their planning for teaching.

The study

The study aimed to investigate teachers' assessment and reporting practice in relation to outcomes. It involved a survey of teaching staff in 'a stratified proportional systematic selection' (Fox 1969) of primary schools in NSW.

Given the manageable sample of 72 schools, nine school districts from the 40 state districts were selected, predominantly from the Sydney area, with the intention of selecting eight schools from each district.

To ensure representativeness as regards the distribution of different school types, stratified proportional sampling was used. In NSW, primary schools are classified from P1 to P6 depending upon the total enrolment of students (a P1 is the largest in terms of student numbers, and a P6 is the smallest). The Directory of Government Schools in NSW (1996) was consulted to determine the proportion of school types in each school district. As there were six different school types, and only eight schools were required from each district, it was decided to combine the six types into three broader types (P1 and P2; P3 and P4; and P5 and P6) to ensure a more meaningful distribution. Table 1 displays the proportion of school types for each district. The first column displays the percentage of the type in the district sample; the second column displays the numerical equivalent for a sample of eight; and the third column displays the actual number of schools selected.

Take in Table 1

The schools in each district were alphabetically ordered and numbered for school type. Every fourth school was then systematically selected for each school type, and when the end of the list had been reached, the procedure resumed until the sample size was obtained. Schools were then contacted to secure a commitment.

The survey contained 44 questions with a likert scale providing five response options. It focused on the strategies used for assessment and reporting; the purposes and principles of assessment; and the extent to which outcomes are the focus in both general areas (cognitive, affective, inquiry, creativity) and specific key learning areas. Data were also sought on the personal/professional attributes of respondents. Item validity was checked by numerous personnel within the DET, and a small pilot was conducted in June 1998.

The survey was administered in September 1998 to 1290 teachers, and the response rate from participating schools was 72 percent.

The results

Data were analysed using SPSS (1988) to obtain frequency distributions, T tests for significance and multiple analysis of variance to determine significant differences according to school type, status, teaching experience, age and gender. The major findings follow.

1. Teachers assess the outcomes of learning

Many items determining the extent to which teachers assess according to outcomes recorded the highest means on the survey. That item focussing generically on the extent to which teachers assess using outcomes, obtained a mean of 4.0 (maximum 5.0), and items recording assessment by outcomes in maths and english obtained means of 4.34 and 4.29 respectively. Confirmation of the primacy of outcomes was reflected in those items determining the degree to which teachers teach with outcomes in mind (4.13) and select teaching methods to achieve particular outcomes (4.06).

However, teachers indicated that not all areas are amenable to assessment by outcomes. The relatively low means for assessing outcomes for inquiry (2.83) and creativity (2.81) either reflect the difficulty of demonstrating certain learning outcomes, or the need to codify learning in those areas.

2. Teachers assess outcomes differentially according to key learning area

Analysis of data indicated that teachers assess by outcomes to different degrees according to key learning area. Furthermore these differences formed a sequence with statistically significant differences between each area. The sequence from most to least assessment by outcomes comprised maths, english, science and technology, human society and its environment, personal development/health/ physical education, and the creative arts.

Several explanations are possible. Arguably the most persuasive is the nature of current curriculum documents in the respective areas. Since the abandonment of the national statements and profiles in 1995, NSW has been gradually introducing new curricula. Maths and english are two such curricula and are heavily based on the achievement of outcomes.

A second explanation involves a perceived hierarchy of worth in learning areas. English, maths and science have traditionally been dubbed the 'basic skills' and form the basis of state wide testing in literacy and numeracy. Perhaps the use of outcomes in assessing these areas reflects the need for clear benchmarks of student achievement in basic skills areas.

An equally plausible explanation is that such a sequence also comprises a sequence relating to ease of assessment. It may well be more difficult to formulate precise statements of educational intent in affective areas (aspects of society and personal development), or in areas involving greater subjectivity of assessment (personal development and the creative arts). Conversely, outcomes may be more easily demonstrated in subjects like maths and science.

3. Teachers assess specific outcomes of learning

The national curriculum profiles developed in the early 90s often formulated outcomes that were to be achieved by students at the end of a stage or 20 month period. As such outcomes were so broad and general, the national curriculum added indicators or 'signals of the achievement of an outcome'. While the survey items did not specify definitions of specific and broad outcomes, data revealed that teachers were significantly more likely to assess the specific outcomes of learning than the broad outcomes. Such specific outcomes almost certainly relate to a topic area or unit of work.

4. Teachers assess most in the cognitive domain

Teachers were significantly more likely to assess outcomes in the cognitive domain than the affective domain; and significantly more likely to assess outcomes in the affective domain than the psychomotor domain. As the great majority of the stipulated outcomes of schooling are subsumed by the cognitive domain, the greater likelihood of assessing outcomes in this domain is not surprising. While there are affective components in english, maths and science for instance, the greater proportion of outcomes relate to knowledge and skills.

The explanation for the sequence of cognitive, affective and psychomotor may be as simple as the amount of time devoted to each. While the affective component of all learning areas may be difficult to determine, the psychomotor area, which lends itself to demonstrated behaviour, comprises relatively little of the primary school curriculum.

5. Teachers use a variety of assessment strategies including portfolios

Teachers claimed to use a variety of assessment tasks (m = 4.00) and rated portfolios as the most frequently used strategy (m = 4.12). Portfolios are work sample folders retained by students that provide samples of their best work. Some portfolio samples specify from the relevant curriculum the outcomes that the work sample exemplifies. The portfolios are also used as the basis of reporting, and were used significantly more than other reporting mechanisms including parent/teacher interviews, and a variety of report card formats (ticking specific outcomes, marks, and combinations of these).

Table 2 displays the means for the use of the various assessment strategies. While not necessarily 'discrete' as strategies, they are derived from the literature reported in the review, and yield useful information.

Take in Table 2

Findings that may surprise in an emerging constructivist ethos are the relatively low ratings for the use of self assessment (2.90) and peer assessment (2.21).

6. Teachers control the assessment process

Such a claim may seem both necessary and desirable. It may also seem an extravagant claim to make on the basis of a few items. It is substantially based on the relatively low mean for teachers negotiating assessment tasks with students (2.53). A similarly low mean for the item relating to the use of assessment tasks that students will experience outside school (2.66) suggests a perception of schooling involving different means of evaluation than those required in the world of work. Such a perception of schooling as separate preparation may well be valid, though some futurists may see the function of the curriculum, both in content and process terms, as preparation in a more dynamic sense.

7. Younger and less experienced teachers are significantly more likely to assess by outcomes

Teachers under the age of 25 were significantly more likely to assess by outcomes in those areas in which assessment by outcomes was most common, viz english (0.00), maths (0.01), science and technology (0.03), and studies of society (0.03). Conversely this age group was significantly less likely (0.00) to use subjective assessment.

The date of emergence of the national curriculum may well explain this finding. As the national curriculum with its emphasis on outcomes, was introduced in the early to mid 90s, this youngest group of teachers was the first to be appointed under the banner of the new national system. Many may well have been trained in the planning for, and assessment of outcomes, in their teacher education programs.

Assuming an approximate correlation between age and teaching experience, the teachers with least teaching experience were significantly more likely to assess using outcomes in science and technology (0.02), studies of society (0.01), and personal development/health/physical education (0.02). While these findings may be explained in the same way as those for age, the additional fact that the direction for each of these findings was unilinear across six cells of teaching experience requires further explanation that the author cannot provide.

Another unilinear finding involved the fact that the more experienced teachers were significantly more likely to use subjective assessment (0.00). A possible explanation is that intuitive and subjective judgment is the legacy of experience.

8. Teacher gender accounts for differences in some assessment and reporting practice

Female teachers were significantly more likely to assess using portfolios (0.01) and written sentences (0.04); to assess outcomes in the creative arts (0.01); to assess in the psychomotor area (0.04); and to select teaching methods with outcomes in mind (0.03). Male teachers were significantly more likely to use diagnostic assessment (0.00); subjective assessment (0.04); and to use parent-teacher interviews to report. No explanations are offered of these findings.

9. Teacher status accounts for differences in some assessment and reporting practice

Most significant differences highlighted the distinctiveness of the teaching principals as a group. While several findings related to the responsibilities and accountabilities of particular status groups (teaching principals, assistant principals, executive teachers, teachers), there were three significant and unidirectional findings. Teaching principals were significantly more likely to negotiate assessment tasks (0.01); assess with outcomes in mind (0.01); and assess as an ongoing part of teaching/learning (0.04). Perhaps a common denominator is the 'political correctness' of these practices. If so, principals, and executive staff in turn, may have adopted them as a model of desirable practice, or as a reflection of their greater involvement in research and policy directions.

10. Larger schools are significantly more likely to assess by outcomes

When the six types of school were categorised for ease of analysis as large (PP1 and PP2) and small (PP3 - PP6), it was found that the larger schools were significantly more likely to assess the specific outcomes of learning (0.01), and significantly more likely to assess using outcomes in english (0.01) and maths (0.01). This may be a reflection of the need to implement tighter and more uniform policy to manage the assessment practices of a large staff, or it may be explained by higher status principals (in NSW, appointment to the larger schools constitutes a promotion) implementing department policies more stringently.

Conclusion

The trend towards planning by outcomes across the curricula, a legacy of the national curriculum, has been further consolidated by the emphasis on assessment by outcomes. While there may be differences in the extent of assessing by outcomes according to learning area, the practice is now firmly established.

There is however some evidence to suggest that the primacy of outcomes relates to cognitive or knowledge and skill dimensions of the school curricula, and that teachers still struggle to assess by outcomes in the less easily demonstrated affective dimensions of schooling; the more subjective and problematic areas like creativity; and the process areas like inquiry. The continuing and concerted outcomes 'push' requires innovative and creative ways of determining demonstrable and desirable student behaviours or products in these areas.

Another assessment revolution of the 90s has been the emergence and consolidation of portfolios or work sample folders as the principal means of assessing student achievement and reporting it to parents. While conventional report card methods incorporating marks and comments still retain a place as a reporting mechanism, the portfolio which clearly demonstrates student work and the extent to which it meets specified outcomes, is both a satisfying assessment and reporting tool, and an infallible demonstration of teacher accountability.

References

Board of Studies (1996), Assessing and Reporting Using Stage Outcomes, Board of Studies, McMahons Point.

Brady, L. and Kennedy, K. (1999), Curriculum Construction, Prentice Hall Australia, Sydney

Brady, L. (1999), 'Outcomes for accountability in curriculum planning in Australia', International Journal of Education Management (in press)

Brady, L. (1997), 'Outcomes without profiles: planning by outcomes in NSW schools', Educational Practice and Theory, 19, 1, 83-9

Brady, L. (1995), Curriculum Development. Fifth Edition, Prentice Hall Australia, Sydney

Broadfoot, P. (1991), Assessment. A Celebration of Learning, Workshop Report, ACSA, Canberra

Conner, C. (1991), Assessment and Testing in the Primary School, The Falmer Press, London

Curriculum Corporation (1997), Take Up of Statements and Profiles, Curriculum Corporation, Melbourne

Duncan, A. and Dunn, W. (1988), What Primary Teachers Should Know About Assessment, Hodder and Stoughton, London

Eisner, E.W. (1993), 'Reshaping assessment in education: some criteria in search of practice', Journal of Curriculum Studies, 25, 3, 219-33

Focusing on Learning. Report of The Review of Outcomes and Profiles in New South Wales Schooling (1995), Sydney

Fox, D.J. (1969), The Research Process in Education, Holt Rinehart and Winston, U.S.A.

Griffin, P. and Nix, P. (1991), Educational Assessment and Reporting, Harcourt Brace Jovanovich, Sydney

Groundwater-Smith, S. and White, V. (1995), Improving Our Primary Schools. Education and Assessment Through Participation, Harcourt Brace, Sydney

New South Wales Department of School Education (1997), Strategies for Assessment and Reporting. Primary Schools, D.S.E., Sydney

New South Wales Department of School Education (1996), Principles for Assessment and Reporting in NSW Government Schools, D.S.E., Sydney

New South Wales Department of School Education (1996), Directory of Government Schools in NSW, D.S.E., Sydney

SPSS (1988), SPSS-X. User's Guide 3rd Edition, SPSS Inc., Illinois.

 

Table 1. Sample by district and school type

 

Type 1 and 2

Type 3 and 4

Type 5 and 6

District 1

23% (1.84) 2

46% (3.68) 4

31% (2.48) 2

2

46% (3.68) 4

46% (3.68) 4

8% (0.64) 0

3

59% (4.72) 5

32% (2.56) 2

9% (0.72) 1

4

45% (3.60) 4

48% (3.84) 4

7% (0.56) 0

5

38% (3.04) 3

45% (3.60) 4

17% (1.36) 1

6

38% (3.04) 3

58% (4.64) 5

3% (0.24) 0

7

67% (5.36) 5

33% (2.64) 3

0% (0.00) 0

8

34% (2.72) 3

57% (4.56) 4

9% (0.72) 1

9

28% (2.24) 2

64% (5.12) 5

8% (0.64) 1

 

 

Table 2. Assessment strategies

 

Strategy

Mean

Portfolios

3.85

Teacher made tests

3.61

Anecdotal/running/narrative records

3.57

Performance assessment

3.50

Written sentences/paragraphs

3.48

Direct/short answer questions

3.43

Checklists

3.34

Conferencing

3.18

Self assessment

2.90

Interviews

2.63

Contracts

2.39

Rating scales

2.38

Peer assessment

2.21