June.15.2021

The White Paper: old, tired and lacking evidence

By Debra Hayes

In the months before the pandemic gripped the world, the NSW Productivity Commission released a presciently titled discussion paper, Kickstarting the Productivity Conversation. Its recently released followup White Paper sets out its plan for rebooting the economy

Lifting school results is part of the plan. The Commission acknowledges  the ‘pandemic has shown how quickly schools, teachers and students can innovate and adapt to new ways of teaching and learning’. These responses were rapid, mostly impressive and sustained. The impact of inequality on learning was also laid bare. 

However, beyond this brief quote, these and other insights generated during these epoch-making times hardly rate a mention in the White Paper. The flickers of optimism in the earlier discussion paper due to improved performance on some measures have been replaced by a very gloomy picture of decline and stagnation. 

To be fair, the task of assessing whether education has become better or worse is extremely difficult at the best of times because education systems are complex, with multiple attributes, institutions and actors. They can be better in some respects and worse in others. For example, they can be better resourced but less efficient and better in terms of overall outcomes but worse in terms of inequalities between students. It also depends on your view of what education should be for and about.

The Productivity Commission’s view is squarely aligned to the economic purposes of education but even within this economic framing, tensions can exist between those who argue for systems that, variously: promote basic skills in English and mathematics and those who promote skills and attributes for networked, flexible and changing labour markets and ‘portfolio’ careers, and so on.

These disputes are not the only reason it is difficult to work out what is going on.

The volume of research in education has expanded rapidly and policy makers are faced with a cacophony of policy-relevant information. While there is widespread agreement about the importance of research and evidence in education, battles have been fought about what kind of evidence matters, and who is considered an expert. It’s sometimes difficult to get past the problem of ‘not being able to see the wood for the trees’ but I believe that it is possible to present a clear and concise view of what’s gone wrong in the past, and what needs to change.

Educational research is incredibly diverse. What’s needed is the assembly of a wide range of research evidence, qualitative and quantitative, and a balanced review and synopsis. What does not help is the misuse and over-claiming that often follows the phrase, ‘research has shown’. Although there are still many unknowns, there are issues where the evidence is clear and which can be agreed upon.

So, what research evidence does the Productivity Commission draw upon? How does it respond to its own claim: ‘if we want to turn things around, we need to keep following the best available evidence? On what basis does it assert: ‘the evidence is that school results will be most affected by teaching quality’?

The chapter in the White Paper dedicated to schooling, Best practice teaching to lift school results, contains close to 80 references including reports by policy and research institutes (about 24); government reports (about 15); commissioned research (about 9), and; reports by organisations such as the OECD (about 6). While some of these references might be considered scholarly, the most trustworthy source of research evidence is derived from peer reviewed journals and some academic books. 

Less than 27% of all references in the White Paper’s chapter on best practice teaching are peer reviewed, and about 38% of these are more than 10 years old. In addition, 19% of the total number of peer reviewed papers draw upon Australian data. To be clear, the Productivity Commission’s chapter on lifting school results in NSW includes a total of four peer reviewed papers that contain data specifically related to the Australian context. 

This collection of references sets a very low benchmark for evidence-based policy advice, and falls well short of the Productivity Commission’s stated aspiration of ‘following the best available evidence’.

Educational researchers are keen to work with policy makers who, like us, want to improve the educational outcomes of young people, respect the commitment and professionalism of teachers, and ensure that public money is well spent. We also share a commitment to evidence-based practice because the quality of decision making is most likely to improve when we make use of trustworthy evidence. 

There are some clear and practical things we could do now to support these shared aspirations.

  • Establish mechanisms for policy makers to draw on a wider range of educational expertise and knowledge. Rather than relying on ‘what works’ research and poorly curated reviews of the academic literature, governments could co-fund broad-based education research institutes, with a mandate and resources to conduct, synthesise and disseminate education research. The national evidence institute is a step in the right direction.
  • Promote teacher and expert research further up the evidence hierarchy. Initial teacher education and professional development could be recast as research-informed processes aimed at developing teachers who have the capacity to conduct research into their own practice, and to utilise educational research to inform their practice. Teachers are not just end-users of research. Research production and consumption should be part of the professional work of teachers, with sufficient time allocated to such tasks.

As we edge towards the post-pandemic epoch, we must leverage this period of rapid and unprecedented change and disruption. Let’s not waste the opportunity to build a new consensus for change and improvement in education that recognises the capacity of teachers and students to innovate, adapt and learn. We are obligated to draw upon the best available evidence to support and resource them. The NSW Government would be well advised to set aside the Productivity Commission’s White Paper, at least as it relates to schools, and start afresh to build a new, broad, and evidence-based consensus for change in education that draws upon the best available evidence.

Debra Hayes is professor of education and equity, and head of the Sydney School of Education and Social Work.  Formerly a secondary science teacher, she researches, writes and teaches about inequality in education. Material in this blog is drawn from her new book with Ruth Lupton titled Great Mistakes in Education Policy: How to avoid them in the Future (Policy Press, 2021) 

Republish this article for free, online or in print, under Creative Commons licence.

4 thoughts on “The White Paper: old, tired and lacking evidence

  1. If researchers want their research to be used by policy makers, then it has to be timely, relevant and readable. Are there more credible sources, which you think the White Paper should have cited which they did not? Perhaps the reason they used what they did was that was the best they could find.

    Governments are not going to fund an education research institute, unless researchers can show they can produce results of use in a timely manner. One way, I suggest, is to produce short summaries of new research, pointing out their relevance to current issues. Ideally do this so that policy makers can cut and paste into policy. I write my blog posts this way. Minister’s staff tell me they do read them and occasionally I see some of my wording pasted into official statements.

    Proposing to recast initial teacher education as “research-informed processes” I suggest is not a good idea. For those skeptical of the value of research, this will suggest even less relevant teachers. Everyone else will ask: “Why aren’t you doing this already?”.

    For NSW Government to set aside the Productivity Commission’s White Paper, is not feasible (and not necessary as governments hardly ever act on such reports). If education researchers don’t like what is in the report, then come up with an alternative.

  2. Deb Hayes says:

    Hi Tom,
    Thank you for your comment.
    Educational researchers, like other academics, publish their work in peer-reviewed journals, and scholarly books. These are not hard to find, and the process of peer review ensures quality, which does take a little time.
    The pandemic has made us all more research literate, and just as governments rely upon medical researchers to provide high-quality evidence-based knowledge, they can and should rely more on educational researchers to do the same.
    The federal government has already committed $50million dollars to a national evidence institute but, to date, only a director seems to have been appointed. This is an important initiative that needs to be fully delivered.
    Many academic researchers do take seriously making their research accessible. It was a challenge that Ruth Lupton and I took on in writing our new book. I hope you’ll find it an accessible read that synthesises research. This EduResearchblog is also a way that many researchers reach a non-academic audience – see also The Conversation. Twitter is another medium by which academics are making their research more accessible
    A challenge for policy-makers is to appreciate that teachers and teacher-educators are continually engaged in research related to their own practice. This is the stuff of teaching and learning that involves collaborative sustained reflection on practice. We just need others to recognise the value of this kind of research, and to support it through resources, particularly time.
    Thanks again Tom for engaging with my post, I do hope you read my book with Ruth Lupton as we have attempted to do just what you say – come up with an alternative.

  3. Li says:

    Deb is right. Her recently published book is addressing the education policy issue based on evidences. It is about the education direction rather than an issue which can be fixed by patching here and there like mending a piece of clothes.

  4. George Lilley says:

    Thanks Deb, more detailed discussions like this about evidence are needed. Scott Eacott, in The Cult of the Guru, made an important observation that politicians and senior bureaucrats choose evidence that suits their administrative purpose. This new National Institute of Evidence wont solve that problem as it is subservient to the Federal Govt. Also, it has appointed Kevan Collins to their board, who founded the dominant English institution – Education Endowment Foundation (EEF). The EEF has come under a lot of critique about their Meta-meta-analysis method in the peer review (as has John Hattie) but this is ignored by educational authorities in England.& Australia.

    Your question, On what basis does it assert: ‘the evidence is that school results will be most affected by teaching quality’? Is an important one. The Ex federal Ed Minister, Chris Pyne cited Hattie for this highly questionable claim – I’ve looked into this claim in detail here – https://visablelearning.blogspot.com/p/teacher-agency.html

    Another important issue you raise is , “battles have been fought about what kind of evidence matters, and who is considered an expert” but it appears in Australia, Hattie is cited by most state govt as the expert. In Victoria, NSW, ACT and Qld, educational policies are heavily based on Hattie’s claims. Yet, there is significant peer review critique of his claims. But, most teachers are not aware of his critique and Educational authorities do not promote them.

    So Eacott’s notion of “admistrative purpose” combined with the hierarchical nature of Education, is to me is the most important issue to focus on, regarding what is good, relevant evidence.

Comments are closed.