September.16.2019

National Evidence Institute: let’s push for quality of USE, not just quality of evidence

By Mark Rickinson

The use of evidence in education has never had a higher profile. In Australia much interest in school evidence use has been sparked by the promise to develop an ‘independent national evidence institute’ as part of the recent National School Reform Agreement. It is how this institute will work that interests me.

The aim of the institute, as the agreement puts it, will be to make sense of what works in improving school outcomes and – most importantly – then translate this research into practical resources that can be used by classroom teachers and school leaders

As we are still in the planning stages, based on research on evidence use elsewhere in the world, I believe there are qualities and characteristics that educators could be pushing to help ensure the institute will work well for them.

1. We need a national evidence institute that focuses on use as well as evidence

Any evidence institute faces the risk of being drawn into expectations of just providing evidence rather than supporting how teachers and educational leaders will use that evidence. Arguably this latter process is far more important and difficult to establish.

The experiences of the What Works Centres in the UK are illustrative. A recent analysis of their work against their three main aims (generating evidence, translating evidence, and supporting evidence adoption), found that there was far less activity around the evidence adoption relative to evidence generation and translation.

 It seems, then, that it is all too easy for evidence centres to slip into ‘a research production (push) approach to the use of research, rather than problem-solving, demand-led (pull) approach’.

I believe in Australia there is an exciting opportunity for the Australian evidence institute to take a different approach, to learn from experiences in the UK and elsewhere, and articulate the explicit aspiration to be a national evidence use institute. 

2. We need a national evidence institute that supports quality of use as well as quality of evidence

A Monash University project designed specifically to improve the use of research evidence in Australian schools, the Q Project, has argued that discussions about quality in relation to evidence use have focused almost exclusively on the quality of the evidence, but not the quality of its use. While there has been long-standing debate about what counts as quality evidence, deliberation about what counts as quality use has been much more limited.

This situation is changing as awareness and understanding of ‘quality use’ starts to grow. Work within and beyond education is beginning to suggest that quality use involves not only appropriate, rigorous evidence but also thoughtful, critical use of that evidence within decision-making processes that are transparent and accountable.

In addition, it requires not only relevant skills and understandings but also inquiry mindsets and relationships of respect and challenge. Fostering the development of these kinds of characteristics therefore offers a distinctive opportunity for the Australian national evidence institute to be part of supporting not only high-quality evidence but also high-quality use.

As head of one of the What Works Centres in the UK, Sir Kevan Collins, reasoned recently: ‘Used intelligently, evidence is the teacher’s friend’.

Helping to work out what it means to use evidence intelligently in Australian schools and school systems needs to be a key priority for any new national evidence institute.

3. We need a national evidence institute that frames everything around improvement

Another key mission for a new national evidence institute should be establishing clarity about the relationship between using evidence and improving education. As Harvard professor, Carol Weiss, argued thirty years ago, educators would do well to stop thinking about ‘How can we increase the use of research in decision making?’ and focus instead on ‘How can we make wiser decisions, and to what extent, in what ways, and under what conditions, can social research help?’. 

These two questions are subtly (but significantly) different because they shift the focus from increasing the impact of research to supporting the improvement of practice, reminding us that evidence use is a means to an end, not an end in itself.

To me this suggests that the underlying purpose of a national evidence institute should be to support educational improvement and to make clear the distinctive contribution that using evidence can make to realising this aspiration.  Work in the US on improving the use of research evidence talks about ‘advancing the use of research evidence in ways that benefit youth’. The final five words of this statement are the most important ones, as they make clear the ‘So what?’ of evidence use. 

4. We need a national evidence institute that follows an ethos of ‘less is more’

In a world of information abundance it is increasingly being argued that it is most effective to focus ‘on a small number of carefully selected and optimised activities that strongly support things you value, and then happily miss out on everything else’ (as professor of computer science at Georgetown University in the US, Cal Newport, reminds us). This is potentially good advice when it comes to educational evidence use.

Indeed, there are a number of ways in which a national evidence institute would benefit from being intentionally minimalist or ‘less is more’ in its approach.

First, is the benefit of being very clear about the limits as well as the potential of evidence –as emeritus professor of educational assessment at University College London, Dylan Wiliam, argues, ‘Evidence is important, but what is more important is […] teacher expertise and professionalism [to] make better judgments about when, and how, to use research’.

Second, is the benefit of being clear which educational challenges to focus on – such as by developing processes to identify practice in need of evidence or by developing evidence-based guidance on high-priority issues with strong evidence but inconsistent practice.

Finally, is the benefit of using evidence to help people and organisations to know what to stop doing. In this respect, a national evidence institute needs to show leadership not only in focusing on ‘a small number of carefully selected and optimised activities’ but also in ‘happily missing out on everything else’.

Against the backdrop of calls for a research-rich teaching profession in Australia, the establishment of an independent national evidence institute represents a unique opportunity to better understand and support evidence-informed improvement in our schools. Let’s work together to realise this potential.

Mark Rickinson is an Associate Professor in the Faculty of Education at Monash University in Melbourne. Mark’s work is focused on improving the use and usefulness of educational research in policy and practice. He is currently leading a new 5-year initiative (The Q Project) to improve the use of research evidence in Australian schools. Mark is on Twitter @mark_rickinson

The above blog post is based on a presentation Mark gave as part of a panel discussion event about the National Evidence Institute on 26 July 2019 organised by the ‘Schools and Education Systems’ Special Interest Group (SIG) of the Australian Association of Research in Education (AARE).  

Republish this article for free, online or in print, under Creative Commons licence.

2 thoughts on “National Evidence Institute: let’s push for quality of USE, not just quality of evidence

  1. Calling for research-rich teaching and an evidence institute implies teachers and are uninformed amateurs., I suggest instead emphasizing teachers are professionals and lobbying for more funding for education research.

  2. I agree Mark, but there are other very important issues to address too regarding the “top down” approach from Education Depts like ours in Victoria.

    Scott Eacott addressed this in his paper – School leadership and the cult of the guru: the neo-Taylorism of Hattie.

    The Dept has defined “10 high impact teaching strategies” based on Hattie. School performance, Professional Devel, Interviews, Principal and Teacher appraisal is based on these. I can’t question or dispute those strategies even though there are over 50 peer reviews that show his work is at best “low quality” and at worse completely misleading.

    The Dept is also implementing a Literacy/Numeracy strategy for all govt school which will cost around $240 million. This is also based on Hattie’s work. Once again we can’t scrutinise or critique the evidence and there is never a mention of the significant peer review critiques.

    So we have a system problem that needs to be addressed first.

    The recent edition of “Educational Research and Evaluation” tries to address this issue. We need academics in Australia to do the same and challenge the dominance of Hattie and also his commercial interests (much like academics in New Zealand and also the NZ teacher unions).

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.