Nicole Mockler

Confusion on PIRLS reporting – some outlets make major mistakes

PIRLS, Progress in International Reading Literacy Study (PIRLS), standardised testing, , , , ,

The Progress in International Reading Literacy Study (PIRLS) results were released last Tuesday, generating the usual flurry of media reports. PIRLS selects a random sample of schools and students around Australia, and assesses the reading comprehension of Year 4 students. The sampling strategy ensures that the results are as representative of the Australian population of Year 4 students as they can be. 

These latest results were those from the round of testing that took place in 2021 amid the considerable disruptions to schooling that came with the COVID-19 pandemic. Indeed, the official report released by the team at ACER acknowledged the impacts on schools, teachers and students, especially given that PIRLS was undertaken in the second half of the 2021 school year – a period with the longest interruptions to face-to-face schooling in the two largest states (NSW and Victoria). 

Notwithstanding these disruptions, the PIRLS results showed no decline in the average score for Australian students since the previous testing round (2016), maintaining the average increase from the first PIRLS round Australia participated in (2011). The chart below shows the figure from the ACER report (Hillman et al., 2023, p.22).

The y-axis on this chart is centred at the historical mean (a score of 500) and spans one standard deviation above and below the mean on the PIRLS scale (1SD = 100). The dashed line between 2016 and 2021 is explained in the report: 

“Due to differences in the timing of the PIRLS 2021 assessment and the potential impact of COVID-19 and school closures on the results for PIRLS 2021, the lines between the 2016 and 2021 cycles are dashed.” (Hillman et al., 2023, p.22).

Despite these results, and the balanced reporting of the ACER official report, reiterated in their media release and piece in The Conversation, the major newspapers around Australia still found something negative to write about. Indeed, initial reporting collectively reiterated a common theme of large-scale educational decline. 

The Sydney Morning Herald ran with the headline: ‘Falling through the cracks’: NSW boys fail to keep up with girls in reading. While it’s true to say the average difference between girls and boys has increased since 2011 (from 14 scale scores to 25 in 2021), boys in NSW are by no means the worst performing group. Girls’ and boys’ average reading scores mirror a general trend in PIRLS: that is, improvement from 2011 and pretty consistent thereafter (see Figure 2.11 from the PIRLS report below). Observed gender gaps in standardised tests are a persistent, and as yet unresolved, problem – one that researchers and teachers the world over have been considering for decades. The words ‘falling through the cracks’ implies that no one is looking out for boys’ reading achievement, an idea that couldn’t be further from the truth. 

Similarly, and under the dramatic headline, Nine-year-olds’ literacy at standstill, The Australian Financial Review also ran with the gender-difference story, but at least indicated that there was no marked change since 2016. The Age took a slightly different tack, proclaiming, Victorian results slip as other states hold steady, notwithstanding that a) the Victorian average was the second highest nationally after the ACT, and b) Victorian students had by far the longest time in lockdown and remote learning during 2021. 

Perhaps the most egregious reporting came from The Australian. The story claimed that the PIRLS results showed “twice as many children floundered at the lowest level of reading, compared with the previous test in 2016 … with 14 per cent ranked as ‘below low’ and 6 per cent as ‘low’”. These alarming results were accompanied by a graph showing the ‘below low’ proportion in a dangerous red. The problem here is that whoever has created the graph has got the numbers wrong. The article has reversed the proportions of students in the two lowest categories. 

A quick check of the official ACER report shows how they’ve got it wrong. The figure below shows percentages of Australian students at each of the five benchmarks in the 2021 round of tests (top panel) and the 2016 round (bottom panel), taken directly from the respective year’s reports. The proportions in the bottom two categories – and indeed all the categories – have remained stable over the five-year span. This is pretty remarkable considering the disruption to face-to-face schooling that many Year 4 children would have experienced during 2021.

But, apart from the unforgivable lack of attention to detail, why is this poor reporting a problem? Surely everyone knows that news articles must have an angle, and that disaster stories sell? 

The key problem, I think, is the reach of these stories relative to that of the official reporting released by ACER, and by implication, the impact they have on public perceptions of schools and teachers. If politicians and policymakers are amongst the audiences of the media reports, but never access the full story presented in the ACER reports, what conclusions are they drawing about the efficacy of Australian schools and teachers? How does this information feed into the current round of reviews being undertaken by the federal government – including the Quality Initial Teacher Education Review and the Review to Inform a Better and Fairer Education System? If the information is blatantly incorrect, as in The Australian’s story, is this record ever corrected?

The thematic treatment of the PIRLS results in the media echoes Nicole Mockler’s work on media portrayals of teachers. Mockler found portrayals of teachers in news media over the last 25 years were predominantly negative,continually calling into question the quality of the teaching profession as a whole. A similar theme is evident even for a casual observer of media reporting of standardised assessment results. 

Another problem is the proliferation of poor causal inferences about standardised assessment results on social media platforms – often from people who should know better. Newspapers use words like ‘failed’, ‘floundered’, ‘slipped’, and suddenly everyone wants to attribute causes to these phenomena without apparently questioning the accuracy of the reporting in the first place. The causes of increases or declines in population average scores on standardised assessments are complex and multifaceted. It’s unlikely that one specific intervention or alteration (even if it’s your favourite one) will cause substantial change at a population level, and gathering evidence to show that any educational intervention works is enormously difficult.

Notwithstanding the many good stories – the successes and the improvements that are evident in the data – my prediction is that next time there’s a standardised assessment to report on, news media will find the negative angle and run with it. Stay tuned for NAPLAN results 2023.

Sally Larsen is a Lecturer in Learning, Teaching and Inclusive Education at the University of New England. Her research is in the area of reading and maths development across the primary and early secondary school years in Australia, including investigating patterns of growth in NAPLAN assessment data. She is interested in educational measurement and quantitative methods in social and educational research. You can find her on Twitter @SallyLars_27

Happy new year reading: our most popular posts of all time

Aboriginal and Torres Strait Islander education, ATAR, Class size, Direct Instruction, First Nations education, Indigenous education, initial teacher education, multimodal texts, Music education, NAPLAN, online learning, positive personal attributes for teachers, Teacher education, Teacher quality, Teachers as researchers, teaching reading, , , , , , , , , , , , , , , ,

EduResearch Matters began back in 2014 under the stewardship of the amazing Maralyn Parker. At the end of 2020, Maralyn retired and I tried to fill very big shoes. The unusual thing about EduResearch Matters is that even posts published in the first couple of years of the blog’s existence continue to get readers – good research continues to inform and inspire. Some posts are shared many times on social media, some get barely a handful of shares yet continue to be widely read. Here are our top 15 posts of all time. We all need something to read over the break and I thought it might be lovely to see what our best read posts are. To all the authors, from PhD students to professors, thank you for your contribution. To prospective authors, please email ideas to jenna@aare.edu.au. Enjoy. Happy new year!

Jenna Price, editor, EduResearch Matters

  1. If we truly care about all Australian children and young people becoming literate I believe it is vital we understand and define the complexity of literacy, writes Robyn Ewing (2016).
Teaching literacy is more than teaching simple reading skills: it can’t be done in five easy steps

2. What does effective teaching of Aboriginal and Torres Strait Islander students look like? Thousands of research studies have been dedicated to finding answers to this question. But much of what we think we know, or hear, about Indigenous education remains mired in myths and legends, writes Cathie Burgess (2019).

Effective teaching methods that work for Indigenous students: latest research

3. As I see it, music education has now been in the ‘too hard basket’ for at least a generation of Australian students. We continue to suffer a malaise in long-term governmental policy direction, writes Leon R de Bruin (2019).music

Music ed isn’t a luxury. All of our children should be learning music

4. I did not become a teacher the day I walked out of university. I was trained as a teacher but it took many years for me to feel like a teacher. I’m still not sure I’m there yet, writes Naomi Barnes (2016).

It is not easy being a teacher: my story


5. Christopher Pyne [former Coalition minister for education] is embarking on his own education revolution. He wants our nation’s teachers to use a teaching method called Direct Instruction.  For forty years, the specific US-developed approach has been the object of education debates, controversies and substantial research. It has not been adopted for system-wide implementation in any US state or Canadian province, writes Alan Luke (2014)

Direct Instruction is not a solution for Australian schools

6. Positive personal attributes such as fairness, humour and kindness, I believe, should be considered necessary attributes for a teacher, writes Nan Bahr (2016).

Positive personal attributes: why teachers need them and how teacher education can help (despite negative media)

7. There is a lot of misinformation out there, as well as ill informed commentary, about how we prepare teachers to teach reading and writing in Australian schools today, writes Eileen Honan (2015).

This is how Australian teachers are taught how to teach children to read: not just phonics

8. Online learning has become a well-recognised part of the broader landscape of higher education. It is also proving to have a critical place in widening access and equity within this landscape. Increasing numbers of students from backgrounds historically under-represented at university are taking the opportunity to study online, particularly through open-entry and alternative pathways, with many of these learners being the first in their family or community to undertake university studies, writes Cathy Stone (2017).

Ten ways to improve online learning for students

9. For decades there has been an overrepresentation of Indigenous students across Australia in disciplinary school records. Suspensions, exclusions and a range of other negative reports fill the school records. As a result low attendance, low retention and under achievement have been the more commonly reported trajectories for Indigenous Australians, writes Helen Boon (2016).

Why and how to use different teaching methods with Indigenous students

10. When a text uses two or more modes we call it a multimodal text. I have been researching how teachers use and teach multimodal texts and I believe Australia needs to update the way we understand multimodality in our schools and how we assess our students across the curriculum, writes Georgina Barton (2018).

Multimodal texts surround us. What are they? How can we use them in our teaching?

11. Money spent on reducing class sizes has not been wasted as Education Minister Christopher Pyne believes. The advice he has been given is wrong. Reducing class size does make a difference, and the biggest difference it makes is to the schooling outcomes of our most vulnerable children, writes David Zyngier (2014).

Class size DOES make a difference: latest research shows smaller classes have lasting effect

. 12. Schools all around Australia are currently hosting research projects involving classroom teachers. But it can be difficult for teachers to engage in research because it takes a lot of time and energy, not just in the classroom but also due to the paperwork and meetings involved. However, I believe if we don’t work with each other, teachers risk reinventing wheels or becoming trapped within an echo chamber, and researchers risk irrelevance, writes Charlotte Pezaro (2015).

Teachers as researchers: what they do, where to find them and how academic researchers can engage with them

13. What is the obsession with Band 6s? Band 6s sound elite, the very best. But the facts are that a Band 4 or 5 in a difficult subject such as Physics or Chemistry may make as big – or even bigger – contribution to ATAR (Australian Tertiary Admission Rank) (more on that later)  than a Band 6 in say, Music. Also, Band 6s are the only metric made publicly available and shared with the media, writes Simon Crook (2021).

The amazing secrets of band six (and what you should know)

14. You know there is something going wrong with Australia’s national testing program when the education minister of the largest state calls for it to be axed. The testing program, which started today across the nation, should be urgently dumped according to NSW Education Minister, Rob Stokes, because it is being “used dishonestly as a school rating system” and that it has “sprouted an industry that extorts money from desperate families”. I think it should be dumped too, in its current form, but for an even more compelling reason than Stokes has aired. I believe we are not being honest with parents about how misleading the results can be, writes Nicole Mockler (2018).

It’s time to be honest with parents about NAPLAN: your child’s report is misleading, here’s how

15. Australian teachers are doing well. They are not under-qualified and they are certainly not under-educated, as some media stories would have you believe. They are doing an admirable job managing exhausting workloads and constantly changing government policies and processes. They are more able than past generations to identify and help students with wide ranging needs. They are, indeed, far better qualified and prepared than those in our nation’s glorious past that so many commentators reminisce wistfully about, write Nan Bahr, Donna Pendergast and Jo-Anne Ferreira.

Teachers are NOT under-qualified and NOT under-educated: here’s what is really happening

Here’s what a brave new minister for education could do right away to fix the horrific teacher shortage

initial teacher education, Minister for Education, teacher attrition, teacher recruitment, , , , , ,

The new Federal Minister for Education Jason Clare announced last Friday he would convene a Teacher Workforce Roundtable focussed on tackling the nationwide teacher shortage, to be held on August 12. The roundtable will include principals, teachers and education experts.

The critical shortage of teachers is a crisis of our own making. 

We knew the teaching workforce was ageing a long time ago and we knew we would reach a point where we would have so many teachers retiring that we would need to increase the number entering to make up the gap. 

Too many pundits are blaming the stress of the pandemic – but this is not the consequence of COVID. It is a failure of workforce planning by successive governments and now we all have to take some responsibility for – and find new ways – of working together to address the problem.

First, let me say that the ministers, both Federal and State, must urgently address the disparity between state schools and private schools in the recruitment process.

As head of the University of Sydney School of Education and Social Work, I observe closely what happens to students in their final year who are able to apply for Conditional accreditation, and to teach up to a recommended 0.6 of a full time teacher’s load. Some of my colleagues have wisely pointed out the risks of mixing work and study, and the potential to hasten burnout.  

The final year, with its long placement (internship), and opportunities for well-managed Conditional accreditation, is also a great opportunity for schools to assess a student teacher’s capacity and whether they are likely to fit within the school culture. 

Independent schools use this as a kind of probation period, and some provide scholarships to students in the final year of their teaching degrees AND, if they work out well,  guarantee them permanent, secure jobs. That’s not something public school principals are able to do. I hear from so many public school principals who say they would love to be able to offer similar incentives.

Instead, the majority of young students eager to stay in the public system have to work years as casuals before they can get a permanent secure job. We have outstanding student teachers who are committed to the public system but the public system is not committed to them.

Why is the public system so hamstrung? Our students, not just at Sydney University but across the nation, should be snapped up and looked after, instead of being abandoned to such a casual approach.

We need to value the contributions of those who have committed to a career in education. Instead, there is a chorus of critics. The immediate previous federal minister for education Stuart Robert attacked public school teachers as duds without a shred of evidence. While it is pleasing to note that the new minister has a vastly different approach, the general attitude of politicans and pundits is poor. As my colleague Nicole Mockler has written elsewhere, there is a lot of focus on “teacher quality” but none on system quality. Poor performance is blamed on “teachers themselves, rather than to the system in which they practise”.

As Mockler says: “It has been used to justify tighter controls on who enters teaching, denigrate teachers and evade difficult questions of equity and funding.”

For a moment, at the height of lockdown, I thought that changed. The work of teachers was valued, particularly by parents trying to teach their children at home. Suddenly everyone understood how hard it was to teach just one or two children. Imagine 30 in a classroom at once.

But that momentary shift in attitude appears to have disappeared and there has been a return to denigrating the profession, those who enter teaching, and those who teach teachers. This has an influence on recruitment and an impact on young people’s choices. Why would you join a profession that is so lacking in value and respect. 

We also need to do a much better job looking after and retaining the teachers we have. Almost two-thirds (59%) of teachers surveyed from New South Wales, the Northern Territory and South Australia indicated that they either intended to leave the profession before they retire, or were unsure if they would stay until retirement. If we think the problem is bad now, imagine what it will be like if those teachers do leave.

Public education must be made more attractive to our graduates but also to the teachers who are already in our public schools. It is difficult to maintain a steady flow into the public school system for the reasons I’ve outlined above – but even teachers early in their careers are deterred by the lack of security and flexibility. The incentives to work in regional, rural or remote areas are not enough to attract and keep teachers. Many of our graduates want to make a difference but they also need to be able to look after themselves and their families. Without any prospect of a permanent position, other systems and occupations become too attractive, especially when they offer higher starting and award rates, and more opportunities for earlier progression to higher rates of pay. This is especially so in the current tight labour market.

This is what poor workforce planning looks like.

It is not the time to shake up initial teacher education because it is not the problem causing a shortage in teachers, and we must not risk undermining the quality of these programs or of the graduates they produce.

We can have confidence in initial teacher education in this country. It is in very good shape. There is too much focus on the intake into teacher education. The fact is that, in NSW for example, school leavers wanting to enter an education degree must have three band fives in their HSC (including English) or equivalent – and those who don’t must do well in the first year of university studies before they are admitted. 

We can also have confidence in the quality of our graduates because of the standards they have to meet to become accredited. But what we must do is mentor our new graduates. Give them additional time for preparation, to continue to learn the craft of teaching. Don’t just throw them into the deep end and expect them to swim, because the job of teaching is complex and difficult and without proper support they are likely to not thrive, and may not survive.

It is great that new federal education minister, Jason Clare, has called a meeting of his state counterparts and other key stakeholders because to solve the teacher shortage, we must all  work together and be solution-focussed. The Labor Party has committed to new ‘universities accord’. What better challenge to meet first through this collaborative approach than bringing all stakeholders together to fix the teaching workforce crisis?  

This can’t just be another opportunity to continue unhelpful criticism of teachers or of young people who choose to be teachers or of initial teacher education. We must stop criticising people who are committed to teaching.

But there is something which can be done immediately in schools to help address the crisis. We can employ many more paraprofessionals, who can undertake the tasks that teachers currently do that don’t require a teaching qualification. Relieving teachers of these time-consuming administrative tasks is likely to assist in retaining our existing teachers.

There is also something that can be done in funding arrangements. State health systems receive large amounts of Commonwealth funding for teaching, training and research activities which occur in public hospital services. NSW alone gets $750m this year and the Commonwealth hands over $2billion nationally for it. This funding is provided to state health systems in recognition of the critical importance of education, training and research to the ongoing quality and sustainability of our health system nationally. This kind of funding is not available to school systems. 

Pay our teachers better. Improve their conditions. Invest in training and research in our public, catholic and independent school systems to improve quality and a pipeline of skilled graduates to renew our ageing teaching workforce. Teachers are striking because their pay and conditions are not adequate for the work they do. Entice back the teachers who have left by easing the burden of accreditation. Drop so many of the barriers we have.

The deans of education across Australian universities are wanting to work in cooperation with systems and with ministers. We are keen to do our bit by continuing to produce high quality graduates who will help to fill the teaching shortage.

The views expressed here in this blog are those of the author alone and are not made on behalf of or are intended to represent the views of the University of Sydney.

Debra Hayes is professor of education and equity, and head of the University of Sydney School of Education and Social Work.  Her most recent book (with Ruth Lupton) is Great Mistakes in Education Policy: How to avoid them in the Future (Policy Press, 2021). She tweets at @DrDebHayes.

What we really mean when we talk about teacher quality

Australian Teacher Corpus, media representation of teachers, PISA, Teacher quality, , , , ,

Anyone who’s being paying attention of late can tell you that we’re in the midst of a critical teacher shortage, and that attracting people into the profession is a problem, as well as retaining them into and beyond mid-career. Some people, like education workforce researcher Barbara Preston, have been predicting the current situation for years now, even while Governments of all persuasions have simultaneously castigated universities for preparing too many teachers, but that’s another story for another day.

Teaching has an image problem, and while this isn’t entirely the fault of the media, my research suggests that the print media both creates and amplifies discourses about teachers that aren’t helpful to the profession or to society more broadly. 

For research about to be published in an upcoming book, I created and analysed a corpus of over 65,000 articles published in the twelve national and capital city daily newspapers from 1996 to 2020. The Australian Teacher Corpus (ATC) comprises every article from these sources including three or more references to ‘teacher/s’. 65,604 articles – or about 63 every week for 25 years – felt like a lot to me, and one of the first things I did after creating the ATC was to look into how many articles would be included in a similar corpus about other occupations. As Figure 1 highlights, there were more articles published about teachers in the Australian print media over this timeframe than about any of the other occupational groups I investigated, and over twice as many than for nurses, the occupation often thought to be commensurate with teaching in terms of professional education, working conditions and status. 

There’s a density of media coverage about teachers that exceeds that of other professions, possibly because of the inherent ‘human interest’ factor in stories about schools and teachers: we pretty much all went to school, have children and young people dear to us who go to school, and/or are involved in school as parents. School is something the vast majority of us understand, for better or worse, and that’s reflected in the amount of media coverage of teachers and their work. 

In my analysis of the ATC, the issue of quality, and specifically teacher quality emerged as significant. Quality is in the top 1.5% of words in the ATC by frequency – there are over 200,000 different ‘word types’ in the corpus, and quality comes in at around rank 300. About 200 of those top 300 words are grammatical words like the, at, in, of, etc, so that means quality really is quite prominent in the ATC. In one part of the analysis I identified discourses shaped around the quality of teachers, teaching and education as three key concerns within the corpus and set about tracing these over the 25 year period, looking at how prominent each was over time. 

Figure 2 shows the growth of these discourses of quality particularly over the years from 2007 to 2013, from the Rudd-Gillard Education Revolution of the 2007 electionto the Australian Education Act of 2013. At almost every point from the mid-2000s to 2020, teacher quality was the most prominent of these three discourses. 

There’s a problem with the problem of teacher quality. Over this same period of time, it’s been used to justify tighter controls on who comes into the teaching profession (almost like it’s too hard to criticise the quality of current teachers, but prospective teachers are fair game); to pivot discussions about education from difficult questions of equity and funding to easier questions of performance and quality (Mockler, 2014); and to justify ever-increasing mandates and performative accountability measures for the teaching profession and initial teacher education (Barnes & Cross, 2020). 

None of these are great, but the biggest problem of all with teacher quality is that it links poor performance (on international tests such as PISA, literacy and numeracy outcomes, or whatever the flavour of the day is) to teachers themselves rather than to their practices. When it happens so consistently over such a long period of time, the discursive effect is to make teachers look like a bad bunch, a club we could forgive the ‘best and brightest’ for not wanting to become a member of. 

When we talk persistently in the public space about needing to improve teacher quality there is an implied, consistently negative judgement about the intentions and actions of teachers themselves at work. A negative judgement about teachers’ hearts and minds, rendered even more problematic than it might otherwise be because teachers are largely in it for the love of the job rather than for the enormous salaries they don’t earn or the 55+ working hours per week they do put in (Stacey, et al., 2020). 

Discussions of improving teaching quality, on the other hand, assume that teaching is practised rather than embodied (Gore, Ladwig & King, 2004), and that good teachers can and will work over the course of their careers to  continue to develop and shape their practice to the benefit of their students. It’s the difference between denigrating the profession as a pack of ‘dud teachers’ and recognising that teaching is a complex, difficult endeavour, a craft that takes time and intellectual effort and commitment to master. 

The teacher shortage will not be solved by attempting to shore up teacher quality, and any media outlet or political party that thinks it will is barking up the wrong tree. 

In just the last week, we’ve once again had bipartisan agreement that teacher quality is an election issue, with solutions proffered on both sides of politics and widely reported in the media as evidence of the ongoing crisis of teacher quality. If, to quote the Shadow Minister for Education Tanya Plibersek last week, “having an acting education minister who calls public teachers ‘duds’ doesn’t help keep highly experienced, highly competent people in the classroom”, neither does banging on about how teacher quality is an enormous problem in need of a fix. 

What might get us out of this current squeeze is a real commitment to addressing teacher burnout and demoralisation (Santoro, 2018), to improving teachers’ working conditions and to extending the kind of respect to them that understands that teaching is hard, that teaching is complex, and that the quest for teaching quality is one that extends over the course of a career. Now there are election promises I could get behind. 

Dr Nicole Mockler is an associate professor of Education at the University of Sydney. Her research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas. Her new book Constructing Teacher Identities will be published by Bloomsbury Publishing (UK) in June this year.

Image by Adobe Stock

What’s good ‘evidence-based’ practice for classrooms? We asked the teachers, here’s what they said

evidence-based practices, , , , ,

Calls for Australian schools and teachers to engage in ‘evidence-based practice’ have become increasingly loud over the past decade. Like ‘quality’, it’s hard to argue against evidence or the use of evidence in education, but also like ‘quality’, the devil’s in the detail: much depends on what we mean by ‘evidence’, what counts as ‘evidence’, and who gets to say what constitutes good ‘evidence’ of practice.

In this post we want to tell you about the conversations around what ‘evidence’ means when people talk about evidence-based practice in Australian schools, and importantly we want to tell you about our research into what teachers think good evidence is.

Often when people talk about ‘evidence’ in education they are talking about two different types of evidence. The first is the evidence of teacher professional judgment collected and used at classroom level involving things like student feedback and teacher self-assessment. The second is ‘objective’ or clinical evidence collected by tools like system-wide standardised tests.

Evidence of teacher professional judgment

This type of evidence is represented in the Australian Teacher Performance and Development Framework. For example, the framework suggests that good evidence of teachers’ practice is rich and complex, requiring that teachers possess and use sharp and well-honed professional judgement. It says: “an important part of effective professional practice is collecting evidence that provides the basis for ongoing feedback, reflection and further development. The complex work of teaching generates a rich and varied range of evidence that can inform meaningful evaluations of practice for both formative and summative purposes” (p.6). It goes on to suggest that sources of this kind of evidence might include observation, student feedback, parent feedback and teacher self-assessment and reflection, among others.

‘Objective’ evidence

The second discussion around evidence promotes good evidence of practice as something that should be ‘objective’ or clinical, something that should be independent of the ‘subjectivity’ of teacher judgement. We see this reflected in, for example, the much lauded “formative assessment tool” announced in the wake of Gonski 2.0 and to be developed by KPMG. The tool will track every child and ‘sound alarms’ if a child is slipping behind. It aims to remedy the purportedly unreliable nature of assessment of student learning that hasn’t been validated by standardising formative assessment practices. Indeed, the Gonski 2.0 report is very strongly imbued with the idea that evidence of learning that relies on teacher professional judgement is in need of being overridden by more objective measures.  

But what do teachers themselves think good evidence is?

We’ve been talking to teachers about their understanding and use of evidence, as part of our Teachers, Educational Data and Evidence-informed Practice project. We began with 21 interviews with teachers and school leaders in mid-2018, and have recently run an online questionnaire that gained over 500 responses from primary and secondary teachers around Australia.

Our research shows that teachers clearly think deeply about what constitutes good evidence of their practice. For many of them, the fact that students are engaged in their learning provides the best evidence of good teaching. Teachers were very expansive and articulate about what the indicators of such engagement are:

I know I’m teaching well based on how well my students synthesise their knowledge and readily apply it in different contexts. Also by the quality of their questions they ask me and each other in class. They come prepared to debate. Also when they help each other and are not afraid to take risks. When they send me essays and ideas they might be thinking about. Essentially I know I’m teaching well because the relationship is positive and students can articulate what they’re doing, why they’re doing it and can also show they understand, by teaching their peers. (Secondary teacher, NSW)

Furthermore, teachers know that ‘assessment’ is not something that stands independent of them – that the very act of using evidence to inform practice involves judgement. Their role in knowing their students, knowing about learning, and assessing and supporting their students to increase their knowledge and understanding is crucial. Balanced and thoughtful assessment of student learning relies on knowledge of how to assess, and of what constitutes good evidence.

Good evidence is gathering a range of pieces of student work to use to arrive at a balanced assessment. I believe I am teaching well when the student data shows learning and good outcomes. (Primary teacher, SA)

Gathering good evidence of teaching and learning is an iterative process, that is it is a process of evaluating and adjusting that teachers constantly repeat and build on. It is part of the very fabric of teaching, and something that good teachers do every day in order to make decisions about what needs to happen next.

I use strategies like exit cards sometimes to find out about content knowledge and also to hear questions from students about what they still need to know/understand. I use questioning strategies in class and make judgements based on the answers or further questions of my students. (Secondary teacher, Vic)

I get immediate feedback each class from my students.  I know them well and can see when they are engaged and learning and when I’m having very little effect. (Secondary teacher, Qld)

Where does NAPLAN sit as ‘evidence’ for teachers?

Teachers are not afraid to reflect on and gather evidence of their practice, but too often, calls for ‘evidence-based practice’ in education ignore the evidence that really counts. Narrow definitions of evidence where it is linked to external testing are highly problematic. While external testing is part of the puzzle, it can be harmful to use that evidence for purposes beyond what it can really tell us – as one of us has argued before. And the teachers in our study well understood this. For them, NAPLAN data, for instance, was bottom of the list when it comes to evidence of their practice, as seen in the chart below.

This doesn’t mean they discount the potentially, perhaps partially, informative value in such testing (after all, about 72% think it’s at least a ‘somewhat’ valid and reliable form of evidence), but it does mean that, in their view, the best evidence is that which is tied to the day to day work that goes on in their classrooms.

Evidence rated from not useful to extremely useful by teachers in our survey

Teachers value a range of sources of evidence of their practice, placing particular emphasis on that which has a front row seat to their work, their own reflections and observations, and those of the students they teach. Perhaps this is because they need this constant stream of information to enable them to make the thousands of decisions they make about their practice in the course of a day – or an hour, or a minute. The ‘complex work of teaching’ does not need a formalised, ‘objective’ tool to help it along. Instead, we need to properly recognise the complexity of teaching, and the inherent, interwoven necessity of teacher judgement that makes it what it is.

What do teachers want?

Teachers were very clear about what they didn’t want.

Teachers are time poor. We are tired. It sounds good to do all this extra stuff but unless we are given more time it will just be another layer of pressure. (Secondary teacher, NSW)

Teachers believe in and want to rely on useful data but they don’t have the time to do it well. (Primary teacher, NSW)

It must be practical, helpful and not EXTRA. (Primary teacher, Vic)

They don’t want “extra stuff” to do.

They want relevant, high quality and localised professional learning. They want to better understand and work with a range of forms of useful data and research. They particularly find in-school teacher research with support useful, along with access to curated readings with classroom value. Social media also features as a useful tool for teachers.

Our research is ongoing. Our next task is to work further with teachers to develop and refine resources to support them in these endeavours.

We believe teachers should be heard more clearly in the conversations about evidence; policy makers and other decision-makers need to listen to teachers. The type of evidence that teachers want and can use should be basic to any plan around ‘evidence-based’ or ‘evidence-informed’ teaching in Australian schools.

Dr Nicole Mockler is Associate Professor of Education, at the Sydney School of  Education and Social Work at the University of Sydney. She is a former teacher and  school leader, and her research and writing primarily focuses on education policy and  politics and teacher professional identity and learning. Her recent scholarly books  include Questioning the Language of improvement and reform in education: Reclaiming  meaning (Routledge, 2018) and Engaging with student voice in research, education and  community: Beyond legitimation and guardianship (Springer 2015), both co-authored  with Susan Groundwater-Smith. Nicole is currently Editor in Chief of The Australian  Educational Researcher.Nicole is on Twitter @nicolemockler

Dr Meghan Stacey is a lecturer in the sociology of education and education policy in the School of Education at the University of New South Wales. Taking a particular interest in teachers, her research considers how teachers’ work is framed by policy, as well as the effects of such policy for those who work with, within and against it. Meghan completed her PhD with the University of Sydney in 2018. Meghan is on Twitter@meghanrstacey

It’s time to be honest with parents about NAPLAN: your child’s report is misleading, here’s how

NAPLAN, , , ,

You know there is something going wrong with Australia’s national testing program when the education minister of the largest state calls for it to be axed. The testing program, which started today across the nation, should be urgently dumped according to NSW Education Minister, Rob Stokes, because it is being “used dishonestly as a school rating system” and that it has “sprouted an industry that extorts money from desperate families”.

I think it should be dumped too, in its current form, but for an even more compelling reason than Stokes has aired. I believe we are not being honest with parents about how misleading the results can be.

The Federal Minister for Education, Simon Birmingham, has rejected the NSW minister’s call, of course, arguing that “parents like NAPLAN”. Birmingham is probably right. Many parents see NAPLAN scores as one of the few clear indications they get of how their child’s performance at school compares to other children.

Parents receive a NAPLAN student report showing their child’s score as dots on a band, clearly positioned in relation to their school average and national average. It all looks very precise.

But how precise are the NAPLAN results? Should parents, or any of us, be so trusting of the results?

There is considerable fallout from the reporting of NAPLAN results, so I believe it is important to talk about what is going on. The comparisons we make, the decisions we make, and the assumptions we make about those NAPLAN results can all be based on very imprecise data.

How NAPLAN results can be widely inaccurate

While communication of results to parents suggests a very high level of precision, the technical report issued by ACARA each year suggests something quite different. Professor Margaret Wu, a world leading expert in educational measurement and statistics, has done excellent and sustained work over a number of years on what national testing data can (and can’t) tell us.

Her work says that because of the relatively small number of questions asked in each section of NAPLAN tests, that are then used to estimate a child’s performance for each (very large) assessment area, there is a lot of what statisticians call ‘measurement error’ involved. This means that while parents are provided with an indication of their child’s performance that looks very precise, the real story is quite different.

Here’s an example of how wrong the results can be

Figure A is based on performance on the 2016 Year 7 Grammar and Punctuation test: in this case, the student has achieved a score of 615, placing them in the middle of Band 8. We can see that on the basis of this, we might conclude that they are performing above their school average of about 590 and well above the national average of 540. Furthermore, the student is at the cut-off of the 60% shaded area, which means their performance appears to be just in the top 20% of students nationally.

Figure A

However, Figure B tells a different story. Here we have the same result, with the ‘error bars’ added (using the figures provided in the 2016 NAPLAN Technical Report, and a 90% Confidence Interval, consistent with the MySchool website). The solid error bars on Figure B indicate that while the student has received a score of 615 on this particular test, we can be 90% confident on the basis of this that their true ability in grammar and punctuation lies somewhere between 558 and 672, about two bands worth. If we were to use a 95% confidence interval, which is the standard in educational statistics, the span would be even wider, from 547 to 683 – this is shown by the dotted error bars.

In other words, the student’s ‘true ability’ might be very close to the national average, toward the bottom of Band 7, or quite close to the top of Band 9.

That’s a very wide ‘window’ indeed.

Figure B

Wu goes on to note that NAPLAN is also not very good at representing student ability at the class or school level because of what statisticians call ‘sampling error’, the error caused by variation in mean scores of students due to the characteristics of different cohorts. (Sampling error is affected by the number of students in a year group – the smaller the cohort size, the larger the sampling error. Wu points out that the margin of error on school means can easily be close to, or indeed larger than, one year of expected annual growth. So for schools with cohorts of 50 or less, a very significant change in mean performance from one year to another would be possible just on the basis of sampling error.)

The problem is school NAPLAN results are published on the MySchool website. Major decisions are made based on them and on the information parents get in their child’s individual report; parents can spend a lot of money (tutoring, changing school, choosing a school) based on them. As Minister Stokes said a big industry has developed around selling NAPLAN text books, programs and tutoring services. But the results we get are not precise. The precision argument just doesn’t hold. Don’t fall for it.

Any teacher worth their salt, especially one who hadn’t felt the pressure to engage in weeks of NAPLAN preparation with their students, would be far more precise than any dot on a band, in assessing their students’ ability. Teachers continually assess their students and continually collect substantial evidence as to how their students are performing.

Research also suggests that publishing the NAPLAN results on the MySchool website has played a driving role in Australian teachers and students experiencing NAPLAN as ‘high stakes’.

So is NAPLAN good for anything?

At the national level, however, the story is different. What NAPLAN is good for, and indeed what it was originally designed for, is to provide a national snapshot of student ability, and conducting comparisons between different groups (for example, students with a language background other than English and students from English-speaking backgrounds) on a national level.

This is important data to have. It tells us where support and resources are needed in particular. But we could collect the data we need by using a rigorous sampling method, where a smaller number of children are tested (a sample) rather than having every student in every school sit tests every two years. This a move that would be a lot more cost effective, both financially and in terms of other costs to our education system.

So, does NAPLAN need to be urgently dumped?

Our current use of NAPLAN data definitely does need to be urgently dumped. We need to start using NAPLAN results for, and only for, the purpose for which they are fit. I believe we need to get the individual school results off the MySchool website for starters. That would quite quickly cut out much of the hype and anxiety. I think it is time, at the very least, to be honest with parents about what NAPLAN does and doesn’t tell them about their children’s learning and about their schools.

In the process we might free up some of that precious classroom time for more productive things than test preparation.

*With thanks to A/Prof James Ladwig for his helpful comments on the draft of this post.

 

Dr Nicole Mockler is an Associate Professor in Education at the University of Sydney. She has a background in secondary school teaching and teacher professional learning. In the past she has held senior leadership roles in secondary schools, and after completing her PhD in Education at the University of Sydney in 2008, she joined the University of Newcastle in 2009, where she was a Senior Lecturer in the School of Education until early 2015. Nicole’s research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas.

Nicole is currently the Editor in Chief of The Australian Educational Researcher, and a member of the International Advisory Board of the British Educational Research Journal and Educational Action Research. She was the Communications Co-ordinator for the Australian Association for Research in Education from 2011 until 2016, and until December 2016 was Associate Editor for both Critical Studies in Education and Teaching and Teacher Education.

(Note to readers from the Ed. We are having tech problems with our share counters at the moment across some search engines and devices. For those interested – as of 5:25pm 18/5/18 this post had been shared over 1000 times (1K+) on FB and 102 on Twitter.)

TeachING quality is not teachER quality. How we talk about ‘quality’ matters

quality teachers, quality teaching, talking about teachers, talking about teaching, Teacher quality, ,

The language we use to discuss the work of teachers in the public domain matters. It matters to our shared understanding of education as a society and it impacts on teachers’ work both directly and indirectly. My research at the moment focuses in part on the notion of quality in education, specifically how issues of teacher quality are represented in the mainstream print media. As part of this work I decided to look more closely at the use of the terms teacher quality and teaching quality in mainstream print media.

It might seem at first glance that these terms are interchangeable or that if there is a difference, it is negligible. However there is a difference, and I became intrigued at how they are used differently in the media and how that difference can impact readers.

For my study I collected every newspaper article that used either ‘teacher quality’ or ‘teaching quality’ in the 12 national and capital city daily newspapers over a four-year period, from January 2014 to December 2017. There were 432 articles in all, spread fairly evenly over each of the four years, comprising over a quarter of a million words.

I used a range of different techniques to analyse the language used across the group of articles. To start I explored the use of the terms themselves (teacher quality and teaching quality) in context: which teachers were the journalists referring to and what exactly were they saying about the quality of said teachers and/or their teaching?

Use of the terms

While all the articles mentioned either teacher quality or teaching quality, some of them used the terms more than once: there were 369 mentions of teacher quality and 179 mentions of teaching quality in the 432 articles. So teacher quality was used in these articles over twice as many times as teaching quality. All 548 mentions referenced either school teachers or teachers in higher or vocational education institutions (universities and colleges).

Significantly, of the 369 instances of the use of teacher quality in the texts, 360 (that’s 98%) referenced school teachers, while only 9 (the remaining 2%) referenced teachers in higher or vocational education contexts. Somewhat conversely, of the 179 instances of teaching quality in the texts, only 57 (32%) related to school teachers, while the majority of references (122, or 68%) related to teachers in the higher and vocational education sectors.

Different uses

When ‘teacher quality’ is mentioned, with very few exceptions, the talk is about schools. So if the media report or comment is about quality issues in schools the term teacher quality is much more likely to be used than teaching quality.

The opposite is true of university lecturers and teachers in vocational and technical colleges, where issues are generally rendered in terms of teaching quality.

The different use matters

 The overwhelming use of term teacher quality when referring to schools implies that if there is a problem with quality in schools it is a problem with the teachers themselves. In this case quality is linked directly to who is doing the teaching.

On the other hand, the use of teaching quality while referring to universities and further education contexts implies the quality has to do with how a teacher is teaching. So if there is a problem with quality it is not a problem with the teachers themselves but with the teaching methods or curriculum they are using.

This issue might seem subtle, almost theoretical, but it has some very practical implications. In the paper Professional learning, pedagogical improvement, and the circulation of power researchers from the University of Newcastle drew attention to the difference between understanding good teaching as practised (as implied in ‘teaching quality’) and embodied (as implied in ‘teacher quality’), arguing that “where good teaching is understood as being about practices rather than bodies, there is likely to be a stronger receptivity to the idea of change in pedagogy”.

The ongoing use of teacher quality in relation to school teachers reinforces the idea that there is something implicitly wrong with the teachers themselves, rather than with their practices. I know from my considerable work alongside practising teachers in schools the relief that often comes from shifting the conversation to recognise that teaching is practised rather than embodied. In other words, the conversation shifts from one about who teachers are, to what teachers do, and in the process all kinds of doors open to honest critique and the hard work of collaboratively improving classroom practice.

When quality is viewed as a problem, and it inevitably is by the media, politicians and education bureaucrats, it would a much easier problem to solve if it is a problem with pedagogy and curriculum rather than a problem with the actual people who are employed to do the teaching. But it was interesting to delve further into my slice of media and look at who, ultimately, is seen to be responsible for solving any perceived problem with teacher quality.

Who is seen to have solutions to teacher quality problems?

My analysis of these articles found discussion of teacher quality most often occurred in the context of a broader discussion about improvement of teacher quality. Sometimes the discussion directly referenced improvement, while at other times it focused on falling quality (with the associated need for this to be remedied). Almost exclusively, however, the proposed agent of improvement lay beyond both the school and the teaching profession. Governments, accrediting bodies, departments of education and in some cases, universities were seen to have the power and responsibility for improving teacher quality, generally through enforcing stricter controls on who comes into the profession and what they learn on their way into teaching. In only the rarest of cases are teachers themselves positioned as active players in the improvement of teacher quality.

The influence of government use of the terms

 Of course, newspapers aren’t entirely responsible for the prevailing focus on teacher rather than teaching quality. Teacher quality is well and truly out there in the public space. It was one of the four pillars of the current Federal Government’s Students First strategy, and was a darling of the Rudd-Gillard Governments during the Education Revolution. It is embedded in numerous current national and state and territory education policy documents. These things have a knock-on effect to the way the education is reported in the press, and to the way teachers’ work gets discussed within the community and sometimes even by teachers and other educators themselves.

But here’s the thing: when it comes to education, if we’re really interested in quality, we need to shift the conversation. We need to make it more about helping teachers to improve the quality of what goes on in their classrooms, and less about casting them as personally or professionally inadequate in the public space. We need to make it more about teachers’ practices and less about teachers as people. We need to make it more about real, collegial professional learning for improvement and less about trying to regulate our way to quality.

 

Nicole Mockler is an Associate Professor in Education at the University of Sydney. She has a background in secondary school teaching and teacher professional learning. In the past she has held senior leadership roles in secondary schools, and after completing her PhD in Education at the University of Sydney in 2008, she joined the University of Newcastle in 2009, where she was a Senior Lecturer in the School of Education until early 2015. Nicole’s research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas.

Nicole is currently the Editor in Chief of The Australian Educational Researcher, and a member of the International Advisory Board of the British Educational Research Journal and Educational Action Research. She was the Communications Co-ordinator for the Australian Association for Research in Education from 2011 until 2016, and until December 2016 was Associate Editor for both Critical Studies in Education and Teaching and Teacher Education.

Susan Groundwater-Smith and Nicole write about other issues related to the language of improvement and reform in their forthcoming book Questioning the Language of Improvement and Reform in Education: Reclaiming Meaning.

How we talk about teachers is changing. Does it matter?

talking about teachers, , , ,

The way teachers are talked about in the public space is important. It affects teacher morale and how people might interact with them both professionally and socially. It even affects the way new teachers perceive their career pathway unfolding, or not. As an educator working in teacher education I am especially interested in the way early career teachers are talked about, as this immediately affects our students when they graduate.

Early career teachers seem to be a current obsession of both politicians and media commentators. To me the message in the public space was going something like this: if new teachers in Australia were brighter/of a higher ‘quality’/more suitable/better trained/more dedicated/harder working/perfectly-chosen-in-every-way our standards would improve. It was a hunch that this is a change of direction in how early career teachers were being talked about in the public space so I decided to embark on an analysis of policy and media texts to explore how early career teachers are talked about and what, if anything has changed.

What I did

I chose to compare documents from 1998/99 to those from 2014/15. With over 100 reviews of teaching and teacher education having been held on a state and national scale since the late 1970s, there were plenty of sources to choose from.

Specifically, I examined the Commonwealth Government’s response, provided in 1999, to the report from a 1998 Senate Inquiry into the status of the teaching profession known as A Class Act, and the 2015 Commonwealth Government’s response to the Teacher Education Ministerial Advisory Group (TEMAG) report, Action Now: Classroom Ready Teachers.

I chose the 2015 response because it represented the most recent ‘policy settlement’ in relation to early career teachers at the time of the research. I chose the response to the 1998 report because it was the most recent general review that had been conducted on a federal level, focused at least in part on early career teachers. I also considered that the 15 or so years between the two responses, and the fact that the responses both came from Coalition governments, made them a good comparison.

I supplemented these with 228 newspaper articles from the twelve national and capital city daily newspapers published in 1998/99 and 2014/15 identified using the search terms ‘graduate teachers’ and ‘teaching graduates’.

In analysing the texts, I was interested in whether and how far early career teachers were represented by government and news media sources as a ‘problem’, and whether this had changed over time. Some of this research has recently been published in the Journal of Education Policy.

What I found

1998: A Class Act

In a nutshell, the problem with early career teachers in 1998 was that there were not enough of them. There was a lot of talk about a current or impending shortage of new teachers. This was linked to an identified problem with the status of the teaching profession. (Status was low therefore people, supposedly, did not want a teaching career.) The report argues teacher status could and should be improved by the introduction of things like a “code of high professional standards” for teachers.

A very interesting aspect of all this was that the Government saw the implementation and ongoing assessment of professional practice against such standards as “the responsibility of the profession itself”. Oh how times have changed!

Other ideas to attract and retain teachers back in 1998 were to give beginning teachers better support through induction programs and improved employment conditions (including a move away from short-term contracts to secure employment).

In terms of teacher education, it was understood that while some national consistency was desirable, it was very important to ensure the differing needs of different states and territories were able to be met. Indeed, these observations about the tension between national consistency and local requirements infuse the whole Government response to A Class Act.

Early career teachers were understood to be novices, rather than ‘fully formed’, working toward becoming expert practitioners in this first part of their careers:

It is generally acknowledged by all those involved – university educators, practising teachers, education departments and beginning teachers themselves – that no pre-service training can fully prepare new teachers to perform at their full capacity from their first day at work. This is not a reflection on the quality of new teachers nor on the standard of pre-service training. It is a recognition of the complexity of teaching and of the large number of variables…affecting a teacher’s performance. (Commonwealth of Australia 1998, 204)

2015: Classroom Ready Teachers

Jump forward to 2015 and early career teachers are a problem on a number of fronts. They are said to be lacking in basic literacy and numeracy skills, lacking in the ‘right’ motivations for entering the profession, lacking the skills they need to make a positive impact on student learning and, of course, lacking in ‘classroom readiness’.

Absent from the 2015 response is the recognition that good teaching practice is something that begins development during initial teacher education and continues well into and beyond the early years of teaching. While the response does argue for “a nationally consistent approach to the induction and support of beginning teachers to make sure they reach their full potential once they enter the profession”, it also provides a strong vision of beginning teachers who can claim an impact on student learning and be ‘classroom ready’ from the outset.

Solutions proffered to the ‘problem’ of early career teachers in 2015 were many and varied. Alternative entry pathways for teacher education courses to catch those ‘unsuitable’ would-be teachers, is one. Others include the introduction of literacy and numeracy testing for initial teacher education students (designed to catch those with poor skills prior to graduation) and a ‘tightening up’ of requirements and processes for registration of initial teacher education courses.

Significantly, the delegation of greater powers to the Australian Institute for Teaching and School Leadership (AITSL) is seen as a key mechanism for remedying the problems of early career teachers.

What has changed?

So there’s been a shift in how early career teachers are talked about in the public space. They have gone from having a problem (not enough of them, lacking in status, not getting enough help) to being the problem.

As I see it, there is political expediency in laying blame for ‘falling standards’ or ‘stagnating standards’ (or anything else that might be going wrong in schooling) onto new teachers. They’re an easy target. Also the focus on early career teachers has easily segued into further action to federalise control of teacher spaces.

I believe the way we talk about teachers in the public space does matter and how it plays (deliberately or not) into power shifts is important. Early career doctors are not blamed for all that is wrong with our health system, new politicians are not blamed for stagnation in government policy, new lawyers aren’t blamed for expensive out-dated practices in law.

Early career teachers are embarking on a career that can help change the world. They deserve as much support as we can give them, not an unfounded suspicion of their motives and skills, especially at a time when teacher retention and attrition are ongoing concerns.

 

Nicole Mockler is a Senior Lecturer in Education at the University of Sydney. She has a background in secondary school teaching and teacher professional learning. In the past she has held senior leadership roles in secondary schools, and after completing her PhD in Education at the University of Sydney in 2008, she joined the University of Newcastle in 2009, where she was a Senior Lecturer in the School of Education until early 2015. Nicole’s research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas.

Roll back curriculum constraints and give teachers the freedom to make professional judgements

quality teaching, Teachers' work,

The role of the teacher in an Australian classroom is changing, and not in a good way. As I see it, the relentless pressure for schools to perform well in NAPLAN, the demands of various mandated curriculum and the ubiquitous concerns about ‘quality teaching’ are making teachers lose confidence in their own professional abilities. There is little space left for them to make their own decisions and act on their own ideas and knowledge as educators.

I believe it is time in Australia to start reclaiming the notion of teacher as curriculum worker, that is someone who can translate and transform their professional knowledge into appropriate conditions for learning for their particular students in their particular schools. There needs to be a pushback to the current constraints.

We know from myriad research (just look at Finland) that teachers flourish and children learn when teachers are given such freedoms.

I am not saying we should embark on a mission to get rid of MySchool or NAPLAN, or try to dismantle the national curriculum, that would probably not be a fruitful mission for our energy. There are, however, I believe, some ways in which we, as an education community, each with our different roles, might walk this tension between the enabling and constraining factors to help teachers make this space for themselves.

How the problem grew

In their 2007 book Schooling by Design, Wiggins and McTighe expressed their frustration with what they saw as an uncomfortable relationship between teachers and curriculum:

Over the years, we have observed countless examples of teachers who, though industrious and well meaning, act in ways that suggest that they misunderstand their jobs. It may seem odd or even outrageous to say that many teachers misconceive their obligations. But we believe this is the case. Nor do we think this is surprising or an aspersion on the character or insight of teachers. We believe that teachers, in good faith, act on an inaccurate understanding of the role of “teacher” because they imitate what they experienced, and their supervisors rarely make clear that the job is to cause understanding, not merely to march through the curriculum and hope that some content will stick. (2007, p. 128)

This observation probably made them seriously unpopular with teachers, but I think the issue is at least as much a systemic one as it is an individual one. To be honest, I think we’ve been deprofessionalised in terms of our capacity as a profession to undertake curriculum work over the past 20 years.

As the amount of curriculum content has gone up, we’ve been encouraged to see the tick box list of dot points (as we like to call them in NSW) as the curriculum itself for the purposes of accountability, and like the frog in the pot of gradually boiling water, we perhaps haven’t noticed how stark the difference really is. Personally, I don’t think that initial teacher education programs have, as a rule, been good at supporting the development of ‘curriculum worker’ as a principal dimension of beginning teacher identity either, preoccupied largely with the ‘what’ and less than we should be with the ‘how’.

The original Shape of the Australian Curriculum paper, published in 2009, had the following to say about teachers as curriculum workers:

The curriculum should allow jurisdictions, systems and schools to implement it in a way that values teachers’ professional knowledge and that reflects the needs and interests evident in local contexts, as it will be teachers who decide how best to organise learning for students. Organisation of learning should take account of individual family, cultural and community backgrounds; acknowledge and build on prior learning experiences; and fill gaps in those experiences. (ACARA, 2009, p. 8)

The national curriculum will describe a learning entitlement for each Australian student, clearly explaining what is to be taught and learned in each area. Implementing the national curriculum, as in the case of state and territory curriculums, will rely on teachers’ professional judgments about how best to organise learning for students, how to reflect local and regional circumstances, and how best to take advantage of their own specialised professional knowledge and their students’ interests. (ACARA, 2009, p. 11)

By the 2012 version of the paper, these passages had morphed into:

Jurisdictions, systems and schools will be able to implement the Australian Curriculum in ways that value teachers’ professional knowledge, reflect local contexts and take into account individual students’ family, cultural and community backgrounds. Schools and teachers determine pedagogical and other delivery considerations. 
(ACARA, 2012, p. 11)

The Australian Curriculum makes clear to teachers what is to be taught. It also makes clear to students what they should learn and the quality of learning expected of them. Schools are able to decide how best to deliver the curriculum, drawing on integrated approaches where appropriate and using pedagogical approaches that account for students’ needs, interests and the school and community context. (ACARA, 2012, p. 25)

The differences are subtle but the shift from teachers deciding how best to organise learning for students to schools being able to decide how best to deliver the curriculum is not just a semantic one.

Teachers as curriculum workers

 The role I am thinking of is where teachers understand curriculum work as a complex process involving prioritisation, translation, and transformation of knowledge into appropriate conditions for learning. It is about understanding curriculum work as a deeply creative and productive process that relies on confidence with and command of content; deep pedagogical expertise; and a good understanding of the learners in question. It is understanding teaching as scholarly work, as intellectual work, as knowledge work.

As I see it, it is around embracing and consciously growing teacher professional judgement as a matter of professional development priority. Teacher professional judgement has been regarded with increasing suspicion over the past 20 years, but so much of teachers’ curriculum work, not to mention other work, relies on finely honed professional judgement. We’ve come to think of it as unreliable and ‘subjective’, when in actual fact we should be fighting this take on it and working collaboratively to sharpen it.

We might do this by sustaining real conversations about curricular and pedagogical practice, pushing each other to draw evidence from a broad range of sources and use it in both employing our judgement and opening that judgement up to the scrutiny of others. I know of no teacher in touch with their students and their learning who can’t tell you vastly more about those students’ performance than a supposedly objective test score.

I won’t pretend that professional judgement is the ‘silver bullet’ that professional standards were posed to be in the early 2000s, but so much of engaging in critical curriculum work relies on confident and well developed professional judgement that I believe we must focus on this as a matter of priority, lest it disappear entirely down the rabbit hole in our fixation on ‘objective data’.

We’re hearing a lot of late about the possibilities for curriculum integration in STEM (Science, Technology, Engineering and Maths). Without teachers being supported to embrace their identity as curriculum workers more overtly, more stridently and more expansively, visions of integration, whether oriented toward STEM, STEAM (STEM + Arts), or anything else, are, to my mind, unlikely to come about.

Dr Nicole Mockler is a Senior Lecturer in Education at the University of Sydney. She has a background in secondary school teaching and teacher professional learning. In the past she has held senior leadership roles in secondary schools, and after completing her PhD in Education at the University of Sydney in 2008, she joined the University of Newcastle in 2009, where she was a Senior Lecturer in the School of Education until early 2015. Nicole’s research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas.

What’s a Good Education System Worth?

Action Now: Classroom Ready Teachers, Education policy

The Turnbull Government’s education election manifesto, Quality Schools, Quality Outcomes claims (several times over) that when it comes to the funding of schooling, it matters less how much is spent and more how the resources are spent.

This is a convenient argument for a government chasing a budget surplus, and the Turnbull Government makes no secret of its plans for future funding arrangements, “the funding must be affordable, based on a realistic appraisal of the current budget situation”.

Budgets are all about political priorities
A first point to be made here is that budgets are about priorities, and economic decisions are just that, decisions by politicians running their political agenda, rather than ‘givens’.  When it comes to education, surely our first concern should be about the long term health of our society rather than a quick, politically narrow focus on the immediate health of our economy.

Where’s the evidence that how funding is spent is more important than how much?Secondly, where did the claim come from that ‘how’ is more important than ‘how much’ in relation to spending on education? The primary evidence presented for this argument in Quality Schools, Quality Outcomes comes from an OECD report published in 2012, entitled Does money buy strong performance in PISA?.

The first thing important to note is the OECD is reporting on performance on international standardised testing, not on the general health and performance of a national system of schooling.

Unfortunately, in Australia we tend to have bipartisan support for the conflation of these two things. The last Labor Government gave us the Australian Education Act (2013), in which “top 5 by 2025” is enshrined. But just because we haven’t had a national conversation about how far performance on international standardised testing counts as a proxy for the quality of our education system doesn’t mean we shouldn’t. And in fact, we absolutely should.

Nevertheless, let’s for a moment accept the premise that the two are interrelated. What exactly does the OECD say about the relative importance of how much is spent on education funding and how the funding is spent? Drawing on the 2009 PISA results, the OECD report indicates that the amount that high-income countries (defined as those countries with a per capita GDP of above $20,000USD) spend on education is not related to performance on PISA.

For example, countries that spend more than USD 100 000 per student from the age of 6 to 15, such as Luxembourg, Norway, Switzerland and the United States, show similar levels of performance as countries that spend less than half that amount per student, such as Estonia, Hungary and Poland. Meanwhile, New Zealand, a top performer in PISA, spends a lower-than-average amount per student from the age of 6 to 15. (p.2)

So what does make a difference, in the eyes of the OECD? We might come away from a reading of Quality Schools, Quality Outcomes thinking that the OECD is silent on this, but in fact the report goes on to point to the higher salaries and status of teachers in high-performing systems on PISA.

Higher teacher salaries and higher status of teacher makes the difference
This is what the OECD has to say about what has ‘greater impact’:

A school system’s attitudes towards teachers and students have a greater impact on student performance. The strongest performers among high-income countries and economies tend to invest more in teachers…In general, the countries that perform well in PISA attract the best students into the teaching profession by offering them higher salaries and greater professional status. (p.3)

So while we see the dictum that it matters less how much is spent and more how the resources are spent oft-repeated, what kind of attitudes toward, and how much real investment in, teachers can we see mooted in in Quality Schools, Quality Outcomes?

Is investment in quality teaching or yet another low cost political fix?

There’s a proud emphasis on Quality Teaching, specifically through the implementation of recommendations from the Teacher Education Ministerial Advisory Group report Action Now: Classroom Ready Teachers.  As it notes, the recommendations are grouped into five themes: stronger quality assurance of teacher education courses; rigorous selection for entry to teacher education; improved and structured practical experience for teaching students; robust assessment of graduates to ensure classroom readiness; and improved national research and workforce planning.

Further reforms around standards and accountability are also part of the strategy, despite both research and anecdotal evidence suggesting that these reforms may cost the teaching profession, and the quest for “quality teaching” more than they reap.

It’s no secret that members of the current government declared initial teacher education rather than funding “the problem” even before they were in government. On the 23rd of February 2013, seven months prior to the election, the then Shadow Minister for Education, Christopher Pyne, told ABC Radio National:

… the first thing we would do is address issues of teacher quality in our universities.  The first thing we could do is to make sure that the training of our teachers at university is of world standard. … We would immediately instigate a very short term Ministerial advisory group to advise me on the best model for teaching in the world.  How to bring out more practical teaching methods, based on more didactic teaching methods or more traditional methods rather than the child centred learning that has dominated the system for the last 20, 30 or 40 years, so teaching quality would be at my highest priority, followed by a robust curriculum, principal autonomy and more traditional pedagogy.  So I want to make the education debate, move it on from this almost asinine debate about more money and make it about values because while money is important Fran, what we are teaching our children and how we are teaching them and who is teaching them is all much more important. (Kelly, 2013, February 23)

The TEMAG report, however, did not suggest an end to child centred learning or a focus on “traditional” pedagogy. While it’s hard to argue against “quality assurance” or “rigorous selection”, it could be argued that once again in Quality Schools, Quality Outcomes, we see an attendance to the ‘low hanging fruit’ of keeping ‘undesirables’ out of the teaching profession and tidying up the edges of initial teacher education, both of which come at a relatively low cost of any kind to the Government. They do, however, come with the bonus of having tangible outcomes within the confines of an election cycle.

Teacher education is the cheaper, easier political target
Higher salaries and greater professional status for teachers, those things that the OECD claims are associated with higher performance, are far more difficult to attain. Furthermore, we won’t get there by creating a smokescreen wherein initial teacher education is constituted as the primary problem that we can assiduously ‘solve’. Nor will we get there by crafting an argument that says that “effective teachers” will only be produced through new, rigorous selection processes for teaching candidates, an argument at whose very core lies a deep disrespect for the teaching profession.

Performance pay for teachers is an old, failed idea
We also won’t get there via the Government’s mooted plan for performance-based pay for teachers. On this, Quality Schools, Quality Outcomes claims that:

Research has shown that teacher effectiveness can be increased by recognising high performing teachers and rewarding them with increased pay by linking their performance to higher bands of pay in industrial agreements. (p.10)

The evidence provided for this claim is a report published by the Grattan Institute in 2010, which in fact does not mention performance pay for teachers at all.  Research has actually shown repeatedly that there is no link between performance pay and improved student learning, with the OECD itself arguing that “a look at the overall picture reveals no relationship between average student performance in a country and the use of performance-based pay schemes…but the picture changes when taking into account how well teachers are paid overall in comparison with national income”.

What does matter
If the Government wants to take the advice of the OECD on how the amount of funding for education doesn’t really matter, then it needs to also pay attention to the other half of the equation, which is about what does.

Higher salaries for teachers and greater professional status would be a good start (although only a start), but higher salaries won’t come without prioritising social factors over economic ones; and greater professional status won’t come via undermining teacher professionalism while repeating the mantra of “Quality Teaching”.

The bottom line is there is no easy or cheap fix that can be conjured up via teacher education or by imposing any other simple, politically focussed “education reform”.

Perhaps real progress will come only when governments of all persuasions recognise that, to borrow the words of Ryan Fuller, who has worked as both a rocket scientist and a teacher, “teaching isn’t rocket science – it’s harder”.

MocklerDr Nicole Mockler is a Lecturer in Education at the University of Sydney. She has a background in secondary school teaching and teacher professional learning. In the past she has held senior leadership roles in secondary schools, and after completing her PhD in Education at the University of Sydney in 2008, she joined the University of Newcastle in 2009, where she was a Senior Lecturer in the School of Education until early 2015. Nicole’s research interests are in education policy and politics, professional learning and curriculum and pedagogy, and she also continues to work with teachers and schools in these areas.