Associate Professor, University of Technology, Sydney

Paper delivered at the Australian Association for Research in Education

Annual Conference on 4 - 7 December, 2000, Sydney

Two trends influencing school assessment practice in NSW in the 90's are the authentic assessment movement involving performance and situated assessment, and outcomes based education which was introduced with the national curriculum. Portfolios emerged relatively recently as an assessment and reporting strategy that exemplifies both trends.

This article reports a study involving a survey of 64 randomly selected primary schools in NSW, and a case study of a single school. The purpose of the study was to ascertain teacher interpretations of the purpose of portfolios; to identify what teachers include as contents; to determine how portfolios are used to assess and report; and to examine the impact of portfolio use on one school's assessment and reporting practice.

The results indicate that teachers regard portfolios as strategic collections of student work demonstrating the achievement of outcomes, particularly in English and Maths, and that student self assessment, and teacher-student collaboration in content selection is not yet well developed. These results are explained both in terms of the context and the recency of the innovation.

In Australia from the early to mid 90's, there was a strong concern that the mandate of outcomes based education, a legacy of 'the national curriculum', might inevitably result in a formal testing regime. Some interpretations of outcome based education, particularly a generally dreaded mastery learning interpretation, was perceived as consistent with the more formal assessment of specific outcomes.

However, this trend towards outcome based education in Australia ran parallel with a trend towards authentic assessment, previously dubbed 'alternative assessment'. Cumming and Maxwell (1999) identify four major interpretations of authentic assessment:

• performance assessment or tasks assessed by actual demonstration;
• situated assessment or tasks assessed in context;
• problem-based assessment or assessment of tasks that involve more than mere technical facility;
• competence-based assessment or tasks assessed for competence in the workplace.

Despite these variations in interpretation, there appears to be consensus on several elements (Wiener and Cohen 1997; Darling-Hammond 1995; Hiebert, Valencia and Afflerbach 1994; and Valencia, Hiebert and Afflerbach 1994). First, it involves a variety of assessment strategies that capture the quality of a student's work. Second, these assessments explore a student's normal daily performance rather than focus on tests; and third, such assessments reflect the actual learning and teaching of the classroom and beyond.

The portfolio as a strategy for assessment and reporting reflects each of these emphases. It is also identified by Brady (1998) as the most common assessment and reporting strategy emerging in NSW primary schools in the late 90's as they implement an outcomes approach to assessment and reporting.

The international literature on portfolios focuses on their purpose and characteristics, including that which extols the merits of the strategy (National Schools Network 1979; Wiener and Cohen 1997; Richter 1997; Karoly 1996; Snider, Skawinski Lima and Devito 1994); their contents (inevitably linked to purpose); and their implementation.

• the showcase portfolio which includes the student's best work;
• the evaluation portfolio which includes specified and marked work;
• the documentation portfolio which contains student work systematically kept by the teacher but not marked;
• the process portfolio which contains ongoing work and student self reflection.

• the off track portfolio which is simply a selection of student work or assessment with no coherent principle for organising the material;
• the emerging portfolio in which there is a sense of greater organisation or purpose. While some relationship between certain entries is apparent, the organisation is not sufficiently developed to comprise a satisfying learning narrative;
• the on track portfolio in which there is a relationship between the parts, and an emerging story of the student as learner. There is evidence of student self assessment and the learner is engaged in the process of selecting and explaining entries;
• the outstanding portfolio which is a coherent story of the student as a reflective learner and in which there is a clear purpose and relationship between the parts.

From the mid 90's in NSW, there has been a strong Department of Education and Training (DET) (previously the Department of School Education (DSE)) emphasis on the assessment implications of outcomes based education. The authentic assessment movement of the 90's gave further momentum to the push to identify appropriate assessment strategies.

The emergence of the portfolio as such a strategy may be traced from the two paragraphs in DSE (1996) Principles of Assessment and Reporting in NSW Government Schools, to the prominence given to portfolios as a discrete strategy in DSE (1997) Strategies For Assessment and Reporting. Primary Schools, and the DET (1999) Assessment and Reporting Issues 7 - 12: Bulletin 5 dealing exclusively with portfolios. Defining portfolios as 'a deliberate, strategic and specific collection of student work or evidence of student work... that demonstrates that learning has occurred', the DET (1999) identifies the purpose of portfolios for teachers, students and parents, before addressing the issues of what should be included and how evidence should be collected. The bulletin is not prescriptive. For instance, beyond suggesting that work should be gathered from a wide variety of tasks, and that a balance be achieved between teacher and student control of the portfolio, it identifies issues (for example, all evidence or selected best pieces) rather than provides solutions. It does, however, advocate student self assessment as an essential part of the process.

Given the recency of portfolio implementation in NSW schools, the empirical research is scant. One recent multi-case study is that of Cullen (1999) who investigated the impact of portfolio assessment on student learning; changes in classroom practices as a result of portfolios; and changes in teacher and parent beliefs. However, this was a case study employing qualitative methodology and the fact that there is negligible data from a large sample of practising primary teachers, provided the rationale for the reported study.

The study was conducted in two phases: a quantitative study involving the survey of a stratified proportional systematic selection of primary school teachers in NSW; and the second was a qualitative study involving a case study of one school. The aims of the study were:

• to ascertain teacher interpretations of the purpose of portfolios;
• to identify what teachers include as the contents of portfolios;
• to determine how portfolios are used to assess and report;
• to examine the impact of portfolio use on one school's assessment and teaching practices.

A 'stratified proportional systematic selection' (Fox, 1969) of 64 primary schools in eight non-contiguous school districts was selected with the intention of selecting eight schools from each district. To ensure representativeness as regards the distribution of different school types, stratified proportional sampling was used. In NSW, primary schools are classified from P1 to P6 depending upon the total enrolment of students (a P1 is the largest in terms of student numbers, and a P6 is the smallest). The Directory of the NSW Department of Education and Training (1999) was consulted to determine the proportion of school types in each school district. As there were six different school types and only eight schools were required from each district, it was decided to combine the six types into three broader types (P1 and P2; P3 and P4; P5 and P6) to ensure a more meaningful distribution. Table I displays the proportion of school types for each district. The first column displays the percentage of the type in the district sample; the second column displays the numerical equivalent for a sample of eight; and the third column displays the actual number of schools selected.

The schools in each district were alphabetically ordered and numbered for school type. Every fifth school was then systematically selected for each school type and when the end of the list had been reached, the procedure resumed until the sample size was obtained. Schools were then contacted to secure a commitment.

The survey contained 29 questions with a likert scale providing five response options. It focused on teacher perceptions of the purpose of portfolios; the desired contents of portfolios; the contexts in which portfolios are used; and the relative responsibilities of teachers and students in the development of portfolios. Item validity was checked and a small pilot conducted.

The survey was administered in term two, 2000 to 950 teachers and the response rate was 74 percent. Data were analysed to obtain frequency distributions and meausres of significance, and multiple analysis of variance was used to determine differences according to school type, status, teaching experience, age and gender. The major findings follow.

• Teachers are significantly more likely to report with portfolios than to assess by them (0.00).

• Portfolios are perceived as a strategic collection of student work that demonstrates outcomes

The purpose of portfolios involves:

• showcasing students best work 1.64
• assembling a collection of all student work that demonstrates learning over time 1.64
• assembling a strategic collection of all student work that demonstrates learning

• indicating that syllabus outcomes have been achieved 4.14
• assembling marked/graded work as a basis for evaluation/accountability 3.56
• enabling students to engage in ongoing assessment of their own 3.31

• Portfolio contents include work samples, particularly in English and Maths, that demonstrate the achievement of outcomes

• Portfolio development is the responsibility of year level teachers

• Teachers observe a school-wide policy on portfolio implementation

• Younger and less experienced teachers include portfolio work samples in English and Maths significantly more than other teachers

• Smaller schools were significantly more likely to engage students in portfolio use

As no single method of data gathering could produce a comprehensive picture of portfolio implementation, it was decided to supplement the survey data with a case study on one school. Yin (1993, 31) claims that the case study method is especially appropriate to this type of research question as "the contextual variables are so numerous and rich that no experimental design can be applied". The methodology is further based on the belief that complex processes like assessing and reporting can best be understood by investigation within a bounded system (Stake 1988). The 'bounded' system is portfolio implementation in a particular school.

The case school, although on the upper north shore of Sydney, is socioeconomically diverse. Single houses, unit dwelling and community housing support a range of ethnic communities. While 52 languages are represented in the school, there is no one dominant group. There are 18 regular classes, 515 students, and a range of ages and teaching experience among staff.

Having completed the survey, nine teachers were interviewed, each for approximately 45 minutes. A range of portfolios from all grades was examined, as were assessment and reporting policy documents. Evidence from the three sources (interview, survey, documents) was clustered in themes, in such a way that the themes emerge from the data rather than being imposed on it (Miles and Huberman 1994). The final reduced set of data was displayed on matrices with text in cells. Each cell in the teacher interview matrix contained a teacher response to a particular interview question. So results are reported by noting clusters/patterns. Following is a brief reporting of these results:

The portfolio, or 'student progress folder' is spiral bound, contained in a hard cover complete with school crest, and work is preserved in plastic envelopes. Advertised as 'an integral part of our assessment and reporting procedures', to 'assist communication between home and school', it contains outcomes, indicators (of outcomes) and relevant work samples in number, space, measurement, reading and writing. Each outcome has the response options of 'working towards', 'achieving' and 'achieving above', and boxes relating to the more precise indicators have to be ticked. For example, for the outcome 'counts, compares and orders whole numbers up to seven digits and represents them in symbols and words', the indicators include 'states the place value of any digit in a number', 'places a group of numbers in ascending or descending order', and 'counts on and back by 10, 100, 1000 from a given number'.

The samples, with related outcomes sheets in Maths and English, are the basis of a parent-teacher interview in April. The equivalent information will be sent home to parents in the progress folder in September. The progress folder with a written report covering all key learning areas and social development with selected work samples will be sent home both in July and at the end of the school year when an interview may be arranged on request.

The work samples are typically printed sheets that have to be completed and are often marked, sometimes numerically. Examples include determining sides and angles in drawn 3D shapes; writing a narrative on 'A Strange Thing Happened on the Way to School'; using appropriate words in a sentence (cloze); rewriting centimetres as metres and determining both the area of shaded parts and the volume of containers; and artwork using circles, triangles and ovals.

Many of the interviewed teachers affirmed the primacy of outcomes in assessing and reporting, referring to outcomes based education as a 'mindset'. The link between the outcomes based assessment regime and portfolios was often made explicit. The following statement about the impact of portfolios on assessment and reporting practice within the outcomes context, was typical:

In the following extract, in which the teacher expresses concern abut the time-consuming nature of portfolio implementation, the notions of portfolios and outcomes based assessment are even more fused:

The Deputy Principal, given the responsibility of introducing portfolios as part of an assessment and reporting policy, viewed portfolios as 'the expression of outcomes based education'; 'a tool to educate people in outcomes based education'. For her, the greatest problem in implementing portfolios was in 'identifying what outcomes looked like', a perception which justifies her gradual implementation of portfolios in selected key learning areas. What at first may appear to be a narrow or limited approach to portfolio development was certainly not the product of limited vision. It was part of a coherent program of professional development.

While the link between portfolios and outcomes was generally regarded as both automatic and desirable, there was the odd exception:

A second change in assessing and reporting practice noted by teachers was the reduced dominance of marks, though this observation was confined to those teachers with relatively more teaching experience (those whose teaching experience pre-dated the outcomes movement). Reference was made to the transition from marks, 'to which the community was wedded', to outcomes based reporting. One teacher, referring to the impact of portfolios on her assessment and reporting practice, claims that 'it's turned it upside down'. She continues:

A third change, though not a common response, relates to the benefits espoused by the authentic assessment movement, viz performance assessment that is contextualised:

Teachers were more willing to concede magnitude of change in reporting than assessing. Common reference was made to 'talking to the folder' in parent-teacher interviews, and how the portfolio was helpful in exemplifying the nature of student achievement. Even so, there were exceptions. One teacher, conceding the benefits of the portfolio in reporting, argued that NESB parents simply want to know 'is my child good in class', and 'where are they coming in the class'. Another teacher, while acknowledging the value of the portfolio as 'a presentation of what students can do', claims:

Portfolios, particularly in the context of outcomes based assessment, have given teachers direction and focus in their planning. The impact of portfolios is indicated explicitly in the following comments:

The impact of portfolios on teacher planning specifically within the context of outcomes based education is expressed by the claim that:

Apart from reference to focusing on curriculum content, the greater direction provided to resource provision was also mentioned.

A number of teachers attributed the increase in focus and awareness to the reduction in summative assessment and the consequent increase in formative assessment associated with the introduction of portfolios.

However, while teachers argued that portfolios have impacted on their planning, there was little expression of an impact on teaching practice. Of course, some planning decisions do have implications for pedagogy:

A common theme in describing the purpose of portfolios was that of accountability to parents. The following are typical:

While the latter extract suggests a greater degree of student responsibility, the role of the portfolio was generally not seen as a tool for student learning. Even the survey response had relatively low means for items relating to student self appraisal; student determination of portfolio contents; and teacher-student collaboration in selecting work.

Only two teachers nominated portfolio strengths as the provision of opportunity for students to appraise their own work. One, a teacher of a senior 'opportunity' class, claimed to explain to students why they were assessed in particular ways, and how the portfolio sample demonstrated the outcomes.

This case represents only one instance of primary school practice in relation to portfolios in NSW. While some schools known to the author have been engaged in portfolio implementation for a few years, and others not at all, this school only began the process in a concerted way in 2000. The staff regard the portfolio as a highly select strategic collection of student work (an accountability/showcase portfolio), though the school intends to broaden the application to include greater content coverage and student involvement in selection and reflection. The major reason for 'hastening slowly' is the perceived need for staff to fully understand the application of portfolios within an outcomes framework. Currently the portfolio is an instance of Paulson and Paulson's (1994) 'emerging portfolio', in that there is a sense of organisation and purpose, but it hasn't yet achieved the status of a comprehensive learning narrative.

One response to the mandate of outcomes based education in NSW may well have been a concerted return to formal testing. However, the authentic assessment movement of the 90's with its emphasis on performance and situated assessment has produced a leavening in traditional assessment to the extent that with the advent of the new millennium, performance assessment and traditional assessment comfortably supplement each other. In this context, portfolios enjoy recent and widespread prominence.

The international literature proposes a great variety of models and purposes for portfolios. In NSW primary schools, they are regarded as strategic collections of student work that demonstrate the achievement of syllabus outcomes. They are not just a showcase of student's best work, but nor are they 'process' documents involving students in continuous self reflection. In fact, the degree both of student self assessment, and student negotiation of submissions, is arguably secondary to the priority of teacher accountability.

Thus, there is an emphasis on including work samples in English and Maths with statements of outcomes attached, with such inclusions determined by executive or year level teachers. Other 'artefacts' (Wolf, 1991) like learning contracts, test papers, accounts of out-of-school experiences, merit certificates, and student journal entries relating to learning, do not enjoy the same importance.

Portfolios have only recently evolved in NSW, though their development has been dramatic. The case study school is arguably typical both in the degree of implementation, and in the deliberate attempt to introduce portfolios gradually as part of a coherent professional development program.