Abstract:
Introduction
Debates over best practice for assessing students stem, in part, from divergent ideas as to the purposes of assessment. Researchers have suggested that stakeholders hold four major conceptions about the purposes for assessment:
• It improves teaching and learning.
• It makes students accountable for learning, partly through issuing certificates and credentials
• It demonstrates the quality and accountability of schools and teachers
• It should be rejected because it is invalid, irrelevant, and negatively affects teachers, students, curriculum, and teaching (Brown, 2004; Heaton, 1975; Shohamy, 2001; Torrance & Pryor, 1998; Warren & Nisbet, 1999; Webb, 1992).
During the last decade, there has been an increased focus on formative assessment practices, under the rubric 'assessment for learning' (Black, Harrison, Lee, Marshall, & Wiliam, 2003; Black & Wiliam, 1998; Clarke, 2005). These recommendations align with the conception that the purpose of assessment is improvement. Formative practices are often contrasted with more formal and summative practices such as standardized testing that teachers commonly associate with both the accountability or irrelevance conceptions. Over-emphasis on accountability has been shown to have negative consequences for teachers, teaching, curriculum, and learning (Firestone, Schorr, & Monfils, 2004; Hamilton, 2003; Jones, 2001).
The research problem
While literature has identified four major ways of thinking about the purposes of assessment, more work is needed to examine how closely these relate to or align with practicing teachers' conceptions. Teacher conceptions are important since teacher thinking affects their practices (Clark & Peterson, 1986; Pajares, 1992; Thompson, 1992). In turn, teacher practices can greatly affect student outcomes (Muijs, 2006; Muijs & Reynolds, 2005).
Brown's (2002) work on teacher's conceptions of assessment has made a significant contribution to our understanding of how assessment purposes inter-relate in the minds of teachers. His survey instrument on teacher's conceptions of assessment has been used with both primary and secondary teachers in New Zealand and Queensland, Australia. The validity of the model has been demonstrated through confirmatory factor analyses which demonstrated acceptable fit to data among these four populations (Brown, 2004, 2006, 2007 ; Brown & Lake, 2006).
However, this instrument in incapable of capturing data relating to teacher thinking about assessment that falls outside these four categories. It also does not allow us to understand why teachers may be aligned with particular conception. The research study reported in this paper adopts a qualitative approach to investigate teacher conceptions of assessment and seeks to answer the question: What qualitatively different conceptions of assessment are held by New Zealand teachers of years 5-10? As this question deals with variation in conceptions, a phenomenographic approach was selected to investigate this question.
Phenomenographic research investigates the human understandings developed through interactions with the world. Central to this approach is the ontological assumption that a non-dualist world exists (Marton, 1981; Marton & Booth, 1997; Sjostrom & Dahlgren, 2002; Svensson, 1997). Because of this theoretical stance, phenomenographers utilise a second order perspective when analysing data, trying to understand the phenomenon from the perspective of the participant instead of evaluating the response from their own viewpoint (Marton, 1981, 1996). Differentiation, abstraction, reduction, and comparison of meaning are fundamental processes in phenomenographic analysis (Svensson, 1997).
Data collection and analysis
The study reported in this paper is part of the Measuring Teachers' Assessment Practices (MTAP) project at The University of Auckland. An invitation to participate was sent to all primary, intermediate, and secondary schools in the greater Auckland region. Over thirty schools agreed to give access to teachers of mathematics and/or English working with students in Years 5 to 10. All such teachers were asked to complete Brown's COA III abridged questionnaire (Brown, 2006). The second author analysed these questionnaire data and selected 20 teachers who exhibited noticeably different profiles in their conceptions of assessment.
The first author then conducted semi-structured interviews with the teachers, without being aware of each participant's conceptions profile. All data were transcribed verbatim and each utterance was labeled as per the method described in Lankshear and Knobel (2004).
After the data were transcribed and label, phenomenographic analysis was conducted as described by Marton (1981, 1986). Analytical decisions were made based on Sjostrom and Dahlgren's (2002) three indicators:
1. Frequency - how often an idea is articulated
2. Position - where the statement is positioned; often the most significant elements are found in the introductory parts of an answer
3. Pregnancy - when participants explicitly emphasise that certain aspects re more important than others (pp. 341-342).
First, similar conceptions were grouped together into pools of meaning and then abstracted to create categories of description. Each category of description represented a qualitatively different way of experiencing the phenomenon. These categories of description were hierarchically organized into the outcome space by complexity. To judge complexity, similarities and differences between the categories were examined and examples of pregnancy (Sjostrom & Dahlgren, 2002) were often used to make decisions.
Results
As the interviews are scheduled for June, 2008, results will be available in time for the conference. It is expected that the data will find evidence that Brown's four categories accurately represent teacher thinking about the purpose of assessment. However, it is considered likely that additional ways of thinking about assessment will also be present. These data will also help better explain the reasons why teachers are aligned with specific conceptions.
Debates over best practice for assessing students stem, in part, from divergent ideas as to the purposes of assessment. Researchers have suggested that stakeholders hold four major conceptions about the purposes for assessment:
• It improves teaching and learning.
• It makes students accountable for learning, partly through issuing certificates and credentials
• It demonstrates the quality and accountability of schools and teachers
• It should be rejected because it is invalid, irrelevant, and negatively affects teachers, students, curriculum, and teaching (Brown, 2004; Heaton, 1975; Shohamy, 2001; Torrance & Pryor, 1998; Warren & Nisbet, 1999; Webb, 1992).
During the last decade, there has been an increased focus on formative assessment practices, under the rubric 'assessment for learning' (Black, Harrison, Lee, Marshall, & Wiliam, 2003; Black & Wiliam, 1998; Clarke, 2005). These recommendations align with the conception that the purpose of assessment is improvement. Formative practices are often contrasted with more formal and summative practices such as standardized testing that teachers commonly associate with both the accountability or irrelevance conceptions. Over-emphasis on accountability has been shown to have negative consequences for teachers, teaching, curriculum, and learning (Firestone, Schorr, & Monfils, 2004; Hamilton, 2003; Jones, 2001).
The research problem
While literature has identified four major ways of thinking about the purposes of assessment, more work is needed to examine how closely these relate to or align with practicing teachers' conceptions. Teacher conceptions are important since teacher thinking affects their practices (Clark & Peterson, 1986; Pajares, 1992; Thompson, 1992). In turn, teacher practices can greatly affect student outcomes (Muijs, 2006; Muijs & Reynolds, 2005).
Brown's (2002) work on teacher's conceptions of assessment has made a significant contribution to our understanding of how assessment purposes inter-relate in the minds of teachers. His survey instrument on teacher's conceptions of assessment has been used with both primary and secondary teachers in New Zealand and Queensland, Australia. The validity of the model has been demonstrated through confirmatory factor analyses which demonstrated acceptable fit to data among these four populations (Brown, 2004, 2006, 2007 ; Brown & Lake, 2006).
However, this instrument in incapable of capturing data relating to teacher thinking about assessment that falls outside these four categories. It also does not allow us to understand why teachers may be aligned with particular conception. The research study reported in this paper adopts a qualitative approach to investigate teacher conceptions of assessment and seeks to answer the question: What qualitatively different conceptions of assessment are held by New Zealand teachers of years 5-10? As this question deals with variation in conceptions, a phenomenographic approach was selected to investigate this question.
Phenomenographic research investigates the human understandings developed through interactions with the world. Central to this approach is the ontological assumption that a non-dualist world exists (Marton, 1981; Marton & Booth, 1997; Sjostrom & Dahlgren, 2002; Svensson, 1997). Because of this theoretical stance, phenomenographers utilise a second order perspective when analysing data, trying to understand the phenomenon from the perspective of the participant instead of evaluating the response from their own viewpoint (Marton, 1981, 1996). Differentiation, abstraction, reduction, and comparison of meaning are fundamental processes in phenomenographic analysis (Svensson, 1997).
Data collection and analysis
The study reported in this paper is part of the Measuring Teachers' Assessment Practices (MTAP) project at The University of Auckland. An invitation to participate was sent to all primary, intermediate, and secondary schools in the greater Auckland region. Over thirty schools agreed to give access to teachers of mathematics and/or English working with students in Years 5 to 10. All such teachers were asked to complete Brown's COA III abridged questionnaire (Brown, 2006). The second author analysed these questionnaire data and selected 20 teachers who exhibited noticeably different profiles in their conceptions of assessment.
The first author then conducted semi-structured interviews with the teachers, without being aware of each participant's conceptions profile. All data were transcribed verbatim and each utterance was labeled as per the method described in Lankshear and Knobel (2004).
After the data were transcribed and label, phenomenographic analysis was conducted as described by Marton (1981, 1986). Analytical decisions were made based on Sjostrom and Dahlgren's (2002) three indicators:
1. Frequency - how often an idea is articulated
2. Position - where the statement is positioned; often the most significant elements are found in the introductory parts of an answer
3. Pregnancy - when participants explicitly emphasise that certain aspects re more important than others (pp. 341-342).
First, similar conceptions were grouped together into pools of meaning and then abstracted to create categories of description. Each category of description represented a qualitatively different way of experiencing the phenomenon. These categories of description were hierarchically organized into the outcome space by complexity. To judge complexity, similarities and differences between the categories were examined and examples of pregnancy (Sjostrom & Dahlgren, 2002) were often used to make decisions.
Results
As the interviews are scheduled for June, 2008, results will be available in time for the conference. It is expected that the data will find evidence that Brown's four categories accurately represent teacher thinking about the purpose of assessment. However, it is considered likely that additional ways of thinking about assessment will also be present. These data will also help better explain the reasons why teachers are aligned with specific conceptions.