Debra Hayes

The White Paper: old, tired and lacking evidence

In the months before the pandemic gripped the world, the NSW Productivity Commission released a presciently titled discussion paper, Kickstarting the Productivity Conversation. Its recently released followup White Paper sets out its plan for rebooting the economy

Lifting school results is part of the plan. The Commission acknowledges  the ‘pandemic has shown how quickly schools, teachers and students can innovate and adapt to new ways of teaching and learning’. These responses were rapid, mostly impressive and sustained. The impact of inequality on learning was also laid bare. 

However, beyond this brief quote, these and other insights generated during these epoch-making times hardly rate a mention in the White Paper. The flickers of optimism in the earlier discussion paper due to improved performance on some measures have been replaced by a very gloomy picture of decline and stagnation. 

To be fair, the task of assessing whether education has become better or worse is extremely difficult at the best of times because education systems are complex, with multiple attributes, institutions and actors. They can be better in some respects and worse in others. For example, they can be better resourced but less efficient and better in terms of overall outcomes but worse in terms of inequalities between students. It also depends on your view of what education should be for and about.

The Productivity Commission’s view is squarely aligned to the economic purposes of education but even within this economic framing, tensions can exist between those who argue for systems that, variously: promote basic skills in English and mathematics and those who promote skills and attributes for networked, flexible and changing labour markets and ‘portfolio’ careers, and so on.

These disputes are not the only reason it is difficult to work out what is going on.

The volume of research in education has expanded rapidly and policy makers are faced with a cacophony of policy-relevant information. While there is widespread agreement about the importance of research and evidence in education, battles have been fought about what kind of evidence matters, and who is considered an expert. It’s sometimes difficult to get past the problem of ‘not being able to see the wood for the trees’ but I believe that it is possible to present a clear and concise view of what’s gone wrong in the past, and what needs to change.

Educational research is incredibly diverse. What’s needed is the assembly of a wide range of research evidence, qualitative and quantitative, and a balanced review and synopsis. What does not help is the misuse and over-claiming that often follows the phrase, ‘research has shown’. Although there are still many unknowns, there are issues where the evidence is clear and which can be agreed upon.

So, what research evidence does the Productivity Commission draw upon? How does it respond to its own claim: ‘if we want to turn things around, we need to keep following the best available evidence? On what basis does it assert: ‘the evidence is that school results will be most affected by teaching quality’?

The chapter in the White Paper dedicated to schooling, Best practice teaching to lift school results, contains close to 80 references including reports by policy and research institutes (about 24); government reports (about 15); commissioned research (about 9), and; reports by organisations such as the OECD (about 6). While some of these references might be considered scholarly, the most trustworthy source of research evidence is derived from peer reviewed journals and some academic books. 

Less than 27% of all references in the White Paper’s chapter on best practice teaching are peer reviewed, and about 38% of these are more than 10 years old. In addition, 19% of the total number of peer reviewed papers draw upon Australian data. To be clear, the Productivity Commission’s chapter on lifting school results in NSW includes a total of four peer reviewed papers that contain data specifically related to the Australian context. 

This collection of references sets a very low benchmark for evidence-based policy advice, and falls well short of the Productivity Commission’s stated aspiration of ‘following the best available evidence’.

Educational researchers are keen to work with policy makers who, like us, want to improve the educational outcomes of young people, respect the commitment and professionalism of teachers, and ensure that public money is well spent. We also share a commitment to evidence-based practice because the quality of decision making is most likely to improve when we make use of trustworthy evidence. 

There are some clear and practical things we could do now to support these shared aspirations.

  • Establish mechanisms for policy makers to draw on a wider range of educational expertise and knowledge. Rather than relying on ‘what works’ research and poorly curated reviews of the academic literature, governments could co-fund broad-based education research institutes, with a mandate and resources to conduct, synthesise and disseminate education research. The national evidence institute is a step in the right direction.
  • Promote teacher and expert research further up the evidence hierarchy. Initial teacher education and professional development could be recast as research-informed processes aimed at developing teachers who have the capacity to conduct research into their own practice, and to utilise educational research to inform their practice. Teachers are not just end-users of research. Research production and consumption should be part of the professional work of teachers, with sufficient time allocated to such tasks.

As we edge towards the post-pandemic epoch, we must leverage this period of rapid and unprecedented change and disruption. Let’s not waste the opportunity to build a new consensus for change and improvement in education that recognises the capacity of teachers and students to innovate, adapt and learn. We are obligated to draw upon the best available evidence to support and resource them. The NSW Government would be well advised to set aside the Productivity Commission’s White Paper, at least as it relates to schools, and start afresh to build a new, broad, and evidence-based consensus for change in education that draws upon the best available evidence.

Debra Hayes is professor of education and equity, and head of the Sydney School of Education and Social Work.  Formerly a secondary science teacher, she researches, writes and teaches about inequality in education. Material in this blog is drawn from her new book with Ruth Lupton titled Great Mistakes in Education Policy: How to avoid them in the Future (Policy Press, 2021) 

New research shows what makes a difference in teaching literacy and why ‘evidence-based’ is not enough

Public discourse about schooling generally assumes that it’s in crisis. The script goes something like this: There’s a problem and it’s big – really big! Test results show us Australia is going downhill and teachers need to be accountable. There are ‘evidence-based’ solutions but teachers are not using them. If they did, literacy standards would improve, test results would improve, and Australia would be among the best in the world again.

Well we have some good news and bad news for you.

The good news is our research, a long-term study of schools in communities in Australia experiencing high-levels of disadvantage, funded by the Australian Research Council, shows that teachers are indeed now implementing the ‘evidence-based’ local literacy agreements that they have been asked to implement and that their work includes a diverse range of research-informed approaches to literacy learning.

The bad news is despite the highly professional and caring use of evidence based methods by the teachers in the four schools in our study, the literacy levels for children from the most disadvantaged families remain persistently low.

The majority of teachers in our study were implementing their schools’ well-developed literacy agreements. They were not loyal to methods of the past or inadequately trained. They adopted recommended methods of evaluation and regularly assessed their students’ progress.

We used ‘fly on the wall’ type of research, which included regular and constant visits to the classrooms of the teachers involved, to study what was happening and try to work out why it was happening. I want to share with you some of our findings.

Use of common pedagogies

Teachers used what we call common pedagogical approaches. These often took the form of tightly scripted lessons, in which teachers, operating in good faith, implemented a range of literacy programs.
The cost of such cooperation is well documented in the research literature: teachers’ pedagogical choices (choice of teaching methods) are narrowed, and their professional autonomy is weakened. This phenomenon is sometimes referred to as curriculum narrowing, and in the case of literacy, ‘fickle literacies’.

We observed numerous missed opportunities for learning due to:
• too much emphasis on only the kinds of skills that can be easily tested;
• narrow views of literacy constraining the purpose of literacy teaching and learning;
• a prevalence of models of teaching that assume students need to have mastered particular basic skills, or sets of skills, before they can move on to other more demanding tasks (which is not the case for many children). These models, called normative developmental models, can be rigid, repetitive and disengaging, however highly qualified, experienced teachers were expected to adopt them because they were part of the school literacy plan or agreement.
• deficit views about the capabilities of young people from disadvantaged backgrounds. These are views that lead teachers to expect less of some children and make it less likely those children will be offered more intellectually challenging literacy learning activities.

These missed opportunities were an unintended effect of teachers trying to do what they believed was expected of them.

Use of uncommon pedagogies

We also encountered a smaller number of teachers who adopted uncommon pedagogical approaches. Their practices stood out from the more common practices of their peers because they were palpably different. These teachers incorporated the requirements of their school’s literacy agreements into an already rich repertoire of teaching practice.

We observed numerous ways in which these teachers supported literacy learning by:
• recognising the knowledge and experiences that students have, and connecting these to school learning;
• actively and positively connecting classroom practice to families and communities;
• designing learning tasks that were open ended and that demanded complex thinking and language;
• providing opportunities for students to think about significant personal and social issues, such as loneliness, hope and relationships, by engaging with relevant texts.

These uncommon pedagogical approaches led to much higher levels of engagement and success by students. They built on relationships, especially with families and helped to develop trust that in turn can contribute to learning at home and at school.

The importance of a teacher’s body of work

Barbara Comber, one of Australia’s foremost literacy researchers, has argued that we do not usually think of teachers’ practices as demonstrating a body of work, perhaps because it is so ephemeral and of the moment. Instead, teachers are assumed to translate theory into practice or implement policy. However, the uncommon pedagogies of the teachers we observed illustrated complex designs, that demonstrated their intention to keep learning about how to improve their work. These rich banks of knowledge and experience could well be considered their ‘oeuvre’ or body of work, in the sense of what they create across a career.

How might we support teachers to develop their oeuvre? What might the public discourse of schooling look like if it were to be based upon a deep respect for teachers, their knowledge and their understanding of the local conditions of teaching and learning?

Peter Freebody and Allan Luke, two other highly respected Australian literacy researchers, reminded us some time ago that:

‘it is not that some literacy teaching methods work and others do not. They all work to shape and construct different literate repertoires in classrooms…What do particular combinations and blends of families of practices work to produce? In which combinations and emphases do they work with specific communities of students? For what practices, places, times, and occasions do they prepare students? And for what political and ideological configurations?’

Understanding teachers’ work is vital to improving literacy in Australia

We need to change the script that blames teachers for low literacy levels by telling them how to do their job. Our observation of uncommon pedagogies is an indication of how doing a teacher’s job can’t be simplified into a set of ‘evidence-based’ methods.

Well-intentioned efforts to improve literacy in Australia should be built upon the understanding that the work of teachers is complex, situated in particular classroom with particular children, and dependent upon a range of factors including a teacher’s own body of work, relationships with students and their families, the local context, and the availability of opportunities for sustained professional development and dialogue.

The prevalence of common pedagogies is a sign that educational policy is working, it’s just not working in ways that address the problem it is intended to solve.

 

Deb Hayes is Professor of Equity and Education at the Sydney School of Education and Social Work, University of Sydney. She is co-author of the book, Literacy, Leading and Learning (Routledge, 2017) with Robert Hattam, Barbara Comber, Lyn Kerkham, Ruth Lupton and Pat Thomson. Follow Deb on Twitter @DrDebHayes

How educators might work in the fake news world

I want to share here the deep concern I have for the role of educational researchers and teachers in this burgeoning post-truth and fake news world.

Educators know that policy and practice should be informed by more than one kind of evidence. Educational research is not like medical research where if a drug is found to work it will usually keep working in that same way wherever it is used. When we find something that works in education we need to do detailed case studies, preferably conducted over time, to see its effects elsewhere with other teachers and in other classrooms. As I see it the era of post-truth that we are experiencing today risks undermining the important gains that have been made in education, in recognising and valuing knowledge from a number of different sources.

There is no disputing that fact checked journalism, admissible legal evidence and peer-reviewed scholarship must now compete for legitimacy amid multiple other forms of ‘evidence’. Knowledge is indeed powerful, even when it is based on weak evidence, or lies. This new contest over knowledge is perhaps better described by the term, post-fact politics, and the proliferation of lies as a deliberate political tool.

It is an era that holds profound consequences for all educators. We have the responsibility of educating a generation of students who are likely to be active members of online communities where post-fact politics and fake news abounds.

These new communities function like echo chambers. Most of us, teachers and parents included, are members of at least one. The views, people and news we like proliferates, while those we don’t want to hear are filtered out by our community on Facebook, Twitter, Tumblr, Instagram, Snapchat and other social media.

The issues are multiple and constantly changing as new ways of connecting online develop. How do educators engage in productive dialogue with members of these communities who are informed by post-fact ‘evidence’?

Educators are already addressing the issues in everyday-type classroom exchanges, such as a child who brings to school information found on the Internet that is untrue or misleading, or having a discussion with senior students about their concerns around the behaviour of the current United States President.

One way I see as obvious and attainable as an educator is a stronger public discussion about the values we teach, and the values we value, which are not necessarily the same thing. For example, compassion, like other important human values, although seemingly increasingly rare, has the potential to unite us in our common humanity. A drowned child washed up on a beach, another shell shocked and alone in the back of an ambulance, these seem to have been moments that cut through. They tapped a vein in a world fatigued by war, famine and poverty. Schools must be, and many already are, places where young people experience and practice such values.

Teachers have always had the opportunity to influence the lives and chances of young people. But I believe the values we teach in the post-fact world are more important than ever.

Those concerned with education, and a fair and equitable schooling system need to lead the way. Our diversity as scholars, policy makers and practitioners is our strength, not a weakness. We should be helping each other confront the issues by guarding against lies without forfeiting our ability to contest claims to truth.

Politicians should trust teachers to work together at the local level with parents to understand and address the needs of young people and provide resources to support local decision and collaboration.

Instead of admonishing each other for weak practice and evidence, educators need to recognise that the complex educational problems we face can’t be solved by a simplistic view of knowledge or science, or by political quick fixes.

 

Debra Hayes PhD is an Associate Professor in the Sydney School of Education and Social Work at the University of Sydney. Her research investigates the unintended detrimental effects of schooling in contexts where there are high levels of poverty and difference. Her forthcoming co-authored book is titled: Literacy, Leading and Learning: Beyond Pedagogies of Poverty (Routledge)

 

If you want to read more:

Hayes, D. & Doherty, C. (2017) Valuing epistemic diversity in educational research: an agenda for improving research impact and initial teacher education, Australian Educational Researcher 44(2):123-139. doi:10.1007/s13384-016-0224-5

Latour, B. (2004) Why has Critique Run out of Steam? From Matters of Fact to Matters of Concern, Critical Inquiry 30: 225-248 (Winter)