Lucinda McKnight

Does the new AI Framework serve schools or edtech?

artificial intelligence, Artificial Intelligence in schools, , , , ,

On 30 November, 2023, the Australian federal government released its Australian Framework for Generative AI in Schools. This is an important step forward. It provides much-needed advice for schools following the November 2022 release of ChatGPT, a technological product capable of creating human-like text and other content. This Framework has undergone several rounds of consultation across the education sector. The Framework does important work in acknowledging opportunities while also foregrounding the importance of human wellbeing, privacy, security and safety.

Out of date already? 

However, in this fast-moving space, the policy may already be out of date. Following early enthusiasm (despite a ban in many schools), the hype around generative AI in education is shifting. As experts in generative AI in education,researching it for some years now, we have moved to a much more cautious stance. A recent UNESCO article stated that “AI must be kept in check in schools”. The challenges in using generative AI safely and ethically, for human flourishing, are increasingly becoming apparent.

Some questions and suggestions

In this article, we suggest some of the ways that the policy already needs to be updated and improved to better reflect emerging understandings of generative AI’s threats and limitations. With a 12-month review cycle, teachers may find the Framework provides less policy support than hoped. We also wonder to what extent the educational technology industry’s influence has affected the tone of this policy work.

What is the Framework?

The Framework addresses six “core principles” of generative AI in education: Teaching and Learning; Human and Social Wellbeing; Transparency; Fairness, Accountability; and Privacy, Security and Safety. It provides guiding statements under each concept. However, some of these concepts are much less straightforward than the Framework suggests.

Problems with generative AI

Over time, users have become increasingly aware that generative AI does not provide reliable information. It is inherently biased, through the biased material it has “read” in its training. It is prone to data leaks and malfunctions. Its workings cannot be readily perceived or understood by its own makers and vendors; it is therefore not transparent. It is the subject of global claims of copyright infringement in its development and use. It is vulnerable to power and broadband outages, suggesting the dangers of developing reliance on it for composing content.

Impossible expectations

The Framework may therefore have expectations of schools and teachers that are impossible to fulfil. It suggests schools and teachers can use tools that are inherently flawed, biased, mysterious and insecure, in ways that are sound, un-biased, transparent and ethical. If teachers feel their heads are spinning on reading the Framework, it is not surprising! Creators of the Framework need to interrogate their own assumptions, for example that “safe” and “high quality” generative AI exists, and who these assumptions serve.

As a policy document, the Framework also puts an extraordinary onus on schools and teachers to do high-stakes work for which they may not be qualified (such as conducting risk assessments of algorithms), or that they do not have time or funding to complete. The latter include designing appropriate learning experiences, revising assessments, consulting with communities, learning about and applying intellectual property rights and copyright law and becoming expert in the use of generative AI. It is not clear how this can possibly be achieved within existing workloads, and when the nature and ethics of generative AI are complex and contested.

What needs to change in the next iteration?

  1. A better definition: At the outset, the definition of generative AI needs to acknowledge that it is, in most cases, a proprietary tool that may involve the extraction of school and student data. 
  2. A more honest stance on generative AI: As a tool, generative AI is deeply flawed. As computer scientist Deborah Raji says, experts need to stop talking about it “as if it works”. The Framework misunderstands that generative AI is always biased, in that it is trained on limited datasets and with motivated “guardrails” created largely by white, male and United States-based developers. For example, a current version of ChatGPT does not speak in or use Australian First Nations words, for valid reasons related to the integrity of cultural knowledges. However, this indicates the whiteness of its “voice” and the problems inherent in requiring students to use or rely on this “voice”. The “potential” bias mentioned in the Framework would be better framed as “inevitable”. Policy also needs to acknowledge that generative AI is already creating profound harms, for example to children, to students, and to climate through its unsustainable environmental impacts.
  3. A more honest stance on edtech and the digital divide: A recent UNESCO report has confirmed there is little evidence of any improvement to learning from the use of digital technology in classrooms over decades. The use of technology does not automatically improve teaching and learning. This honest stance also needs to acknowledge that there is an existing digital divide related to basic technological access (to hardware, software and connectivity) that means that students will not have equitable experiences of generative AI from the outset.
  4. Evidence: Education is meant to be evidence-informed. Given there is little research that demonstrates the benefits of generative AI use in education, but research does show the harms of algorithms, policymakers and educators should proceed with caution. Schools need support to develop processes and procedures to monitor and evaluate the use of generative AI by both staff and students. This should not be a form of surveillance, but rather take the form of teacher-led action research, to provide future high-quality and deeply contextual evidence. 
  5. Locating policy in existing research: This policy has missed an opportunity to connect to extensive policy, theory, research and practice around digital literacies since the 1990s, especially in English and literacy education, so that all disciplines could benefit from this. The policy has similarly missed an opportunity to foreground how digital AI-literacies need to be embedded across the curriculum, supported by relevant existing Frameworks, such as the Literacy in 3D model (developed for cross curricular work), with its focus on operational, cultural and critical dimensions of any technological literacy. Another key concept from digital literacies is the need to learn “with” and “about” generative AI. Education policy needs to reference educational concepts, principles and issues, also including automated essay scoring, learning styles, personalised learning, machine instruction and so on, with a glossary of terms.
  6. Acknowledging the known dangers of bots: It would also be useful for policy to be framed by long-standing research that demonstrates the dangers of chatbots, and their compelling capacity to shut down human creativity and criticality and suggest ways to mitigate these effects from the outset. This is particularly important given the threats to democracy posed by misinformation and disinformation generated at scale by humans using generative AI. 
  7. Teacher transparency: All use of generative AI in schools needs to be disclosed. The use of generative AI by staff in the preparation of teaching materials and the planning of lessons needs to be disclosed to management, peers, students and families. The Framework seems to focus on students and their activities, whereas “academic integrity” needs to be modelled first by teachers and school leaders. Trust and investment in meaningful communication depend on readers knowing the sources of content, or cynicism may result. This disclosure is also necessary to monitor and manage the threat to teacher professionalism through the replacement of teacher intellectual labour by generative AI.
  8. Stronger acknowledgement of teacher expertise: Teachers are experts in more than just subject matter. They are expert in the pedagogical content knowledge of their disciplines, or how to teach those disciplines. They are also expert in their contexts, and in their students’ needs. Policy needs to support education in countering the rhetoric of edtech that teachers need to be removed or replaced by generative AI and remain only in support roles. The complex profession of teaching, based in relationality and community, needs to be elevated, not relegated to “knowing stuff about content”. 
  9. Leadership around ethical assessment: OpenAI made a clear statement in 2023 that generative AI should not be used for summative assessment, and that this should be done by humans. It is unfortunate the Australian government did not reinforce this advice at a national policy level, to uphold the rights of students and protect the intellectual labour of teachers.
  10. More detail: While acknowledging this is a high-level policy document and Framework, we call for more detail to assist the implementation of policy in schools. Given the aim of “defining what safe, ethical and responsible use of generative AI should look like” the document would benefit from more detail; a related  education document from the US runs to 67 pages.

A radical policy imagination

At the 2023 Australian Association for Research in Education (AARE) conference, Jane Kenway encouraged participants to develop radical research imaginations. The extraordinary impacts of generative AI require a radical policy imagination, rather than timid or bland statements balancing opportunities and threats. It is increasingly clear that the threats cannot readily be dealt with by schools. The recent thoughts of UNESCO’s Assistant Director-General for Education on generative AI are sobering.

A significant part of this policy imagination needs to find the financial and other resources to support slow and safe implementation. It also needs to acknowledge, at the highest possible level, that if you identify as female, if you are a First Nations Australian, indeed, if you are anything other than white, male, affluent, able-bodied, heterosexual and compliant with multiple other norms of “mainstream” society, it is highly likely that generative AI does not speak for you. Policy must define a role for schools in developing students who can shape a more just future generative AI, not just use existing tools effectively.

Who is in charge . . . and who benefits?

Policy needs to enable and elevate the work of teachers and education researchers around generative AI, and the work of the education discipline overall, to contribute to raising the status of teachers. We look forward to some of the above suggestions being taken up in future iterations of the Framework. We also hope that all future work in this area will be led by teachers, not merely involve consultation with them. This includes the forthcoming work by Education Services Australia on evaluating generative AI tools. We trust that no staff or consultants on that project will have any links whatsoever to the edtech, or broader technology industries. This is the kind of detail that may help the general public decide exactly who educational policy serves.

Generative AI was not used at any stage in the writing of this article.

The header image was definitely produced using Generative AI.

An edited and shorter version of this piece appeared in The Conversation.

Lucinda McKnight is an Australian Research Council Senior Research Fellow in the Research for Educational Impact Centre at Deakin University, undertaking a national study into the teaching of writing with generative AI. Leon Furze is a PhD Candidate at Deakin University studying the implications of Generative Artificial Intelligence in education, particularly for teachers of writing. Leon blogs about Generative AI, reading and writing.

Pausing NAPLAN did not destroy society – but new changes might not fix the future

NAPLAN, , ,

NAPLAN is again in the news. Last week, it was the Ministers tinkering with NAPLAN reporting and timing. This week it is media league tables ranking schools and sectors, according to NAPLAN results, coinciding with the upload of latest school-level data to the ‘My School’ website. We are now about one month out from the new March test window so expect to hear a lot more in the coming weeks. Many schools will be already deeply into NAPLAN test preparation. 

NAPLAN and My School website were initially introduced by PM Julia Gillard as levers for parental choice. Last week’s ACARA media release reiterates that their primary purpose is so parents can make ‘informed choices’ about their children’s schooling. Media analysis of NAPLAN results correctly identifies what researchers know only too well: that affluence skews educational outcomes to further advantage the already advantaged. 

The Sydney Morning Herald notes that “Public schools with and without opportunity classes, high-fee private institutions and Catholic schools in affluent areas have dominated the top 100 schools…” The reporters are careful to draw attention to a couple of atypical public schools, achieving better results than might be expected from their demographics. A closer look at the SMH table of Top Performing primary schools shows that most low ICSEA public schools ‘punching above their weight’ are very small regional schools. 

No doubt there is a lot to learn from highly effective and usually overlooked small rural schools, but few families can move to them from the city. Parental choice is constrained by income, residential address, work opportunities and a myriad of other factors. In any case, as Stewart Riddle reminds us, what makes a ‘good school’ is far more subtle and complex than anything that a NAPLAN can tell us. 

NAPLAN has gradually morphed into a diagnostic tool for individual students, though there are other tools more fit for this purpose. Notably, the pandemic-induced NAPLAN pause did not lead to the collapse of Australian education but was seen by many teachers as a relief when they were dealing with so many more important aspects of young people’s learning and well-being. 

Education Ministers’ adjustments to NAPLAN indicate that they are at last responding to some of the more trenchant critiques of NAPLAN. The creation of a teacher panel by ACARA as part of the process of setting standards hints that the professional expertise and voices of teachers are valued. Bringing NAPLAN testing forward will hopefully make it more useful where it really matters – in schools and classrooms.

The move to four levels of reporting will be more accessible to parents. Pleasingly, the new descriptor for the lowest quartile – ‘Needs additional support’ – puts the onus on the school and school systems to respond to student needs.

Yet one of the keenest critiques of NAPLAN has not been addressed. There have been widespread calls from educators and academics for the NAPLAN writing test to be withdrawn. It has been found to have a narrowing effect on both the teaching of writing and students’ capacity to write. There is also a whole “how to do NAPLAN” industry of tutors and books pushing formulaic approaches to writing and playing on families’ anxieties.

The failure of the current round of changes to address the NAPLAN writing test leaves students writing like robots. Meanwhile, the release of ChatGPT means that students doing NAPLAN writing for no real purpose or audience of their own are wasting their time. Robots can do it better! These changes needed to map writing better to the National Curriculum, and endorse more meaningful, creative, multimodal and life-relevant writing practices.

 As a single point in time test, NAPLAN has always been just one source of data that teachers and schools can draw upon to design targeted interventions to support student learning. Nevertheless, earlier results will mean that schools will have robust evidence about their need for additional resources. Professional expertise in literacy, numeracy and inclusive education support must be prioritised. 

Parents might be able to resist the inclination to shuffle their children from school to school as a reaction to media headlines, school rankings, and promotional campaigns from the independent sector. Alliances might form between parents and schools to support greater action by state and federal Ministers to address the deeply entrenched divisions that have become baked into Australian schooling.

Attention to NAPLAN continues to mask serious ongoing questions about why Australian governments have created conditions where educational inequities, segregation and stratification are now defining characteristics of our education system. Numerous reports and inquiries have identified flaws and perverse effects from NAPLAN as high stakes testing, especially in relation to the writing test. There is a lot of work yet to be done if NAPLAN is to really be useful and relevant for Australian schools, teachers, parents and learners.

Professor Susanne Gannon is an expert in educational research across a range of domains and methodologies. Much of her research focuses on equity issues in educational policy and practices. Recent research projects include investigations of the impact of NAPLAN on the teaching of writing in secondary school English, young people’s experiences of school closures due to COVID-19 in 2020, and vocational education for students from refugee backgrounds in NSW schools.

Dr Lucinda McKnight is an Australian Research Council Fellow in Deakin University’s Research for Education Impact (REDI) centre. She is undertaking a three year national project examining how the conceptualisation of writing is changing in digital contexts. Follow her Teaching Digital Writing project blog or her twitter account @lucindamcknight8

Header image courtesy of Rory Boon.

We refuse to value care – why sexism is at the core of our early childhood crisis

Early Childhood, Early childhood education and care, , , ,

Introduction: The old, old problem

The introduction of an extra year of education for three and four-year olds in New South Wales (by 2030) and Victoria (by 2025) is an ambitious initiative. Articles in response argue that promises to boost provision may be difficult to deliver. Australia already has a problem filling existing positions in childcare.

Yet the commentary frequently glosses over the fundamental cause of these workforce problems. It’s sexism. Discrimination based on stereotypical understandings about gender. That old, old problem that is ever present, even in these post #metoo days, in which society has awakened to #everydaysexism.

Of course, pay, conditions and turnover affect recruitment and retention in the sector. But without naming and addressing the gender inequalities underpinning these issues, they will not be adequately addressed.

Working in a feminised profession

So what’s going on? Education in Australia (as in the UK and Canada) is a feminised profession.

This means both that women do most of the work, especially at lower pay levels, and that it is perceived as “women’s work”. The majority of teachers, at all levels of education from early years to tertiary, are women. In Australian early childhood education, women make up 96% of the workforce.

Feminist theorist Professor Madeleine Grumet has pointed out the relationship between nurturing at home and at school. Devaluing of women’s work at home (and indeed that of all those who act as carers in our society) is echoed in the devaluing of teachers’ work. In early childhood education these are closely interwoven, and it is therefore undermined as ‘child care’ or ‘glorified baby sitting’. Children themselves are also devalued and dismissed as not worthy until they become “fully formed”, idealised adults.

Caring for children, therefore, is “abjectified”. It is pushed beyond the boundaries of what is recognised and rewarded by society. Caring for children involves snot and poo, dribble, phlegm, sweat, tears, glue, mud, paint, food, vomit, crouching on the floor, carrying heavy bodies, tirelessly comforting, calming, encouraging and supporting. It is also rewarding, tender, exhilarating, creative, loving, funny and inspirational.

The gender binary’s impacts

Many feminist researchers believe that society is organised by a gender binary that privileges what is perceived as masculine over what is perceived as feminine. So the “masculine”, or what is serious, scientific, rigorous, rational, measurable, finite, cleanly defined, standardised, programmable, instrumental and technical is valued above the messy, woolly, grubby world of the so-called “feminine”.

This binary operates at countless levels, in countless ways, to keep the hierarchical status quo in place. Misogyny, the hatred of women, and mysopedy, the hatred of children, are at the heart of a larger system that refuses to value caring. This binary’s stereotypes and assumptions also discriminate against male educators working in the sector.

Early childhood teachers are discriminated against, paid low wages and employed under poor conditions because of the gendered nature of their work. This sexism feeds into the discrimination faced by all primary carers, because undermining the quality and extent of “childcare” affects participation in the workforce more generally. We argue that to change early childhood education, the sexism at its heart needs to be openly named, critiqued and challenged.

What can be done?

So… how to go about this, and ultimately, to make the profession more attractive to all, including those of all genders, colours, abilities, class backgrounds and ages? At the most basic level, researchers, the media, policymakers and politicians need to start naming sexism as the basis for the challenges faced by the early childhood sector.

The labour of workers in this sector is never gender-neutral, but always caught up in societal judgements based on its alliance with child-bearing and rearing in the home. We need to stop pretending these challenges are about early childhood education as a career being rejected simply as “boring”, low-paid or hard work.

Nothing less than a paradigm shift is necessary. If the early years are the most vital years of education, in which children develop at an astonishing rate, then we need investment that places these years at the top of any hierarchy.

Instead of valuing a medical paradigm in education, where being efficient, scientific and  “clinical” are revered, we need to value what is culturally considered to be more feminine. We might ask, “Are there kindergarten surgeons, who model their practice on the patience, kindness and empathy of early childhood teachers?” Flipping a scenario is often a handy way to expose the gender stereotypes and power asymmetries that underpin it.

Conclusion: Reinvention needs both imagination… and funding

Raising the status of early childhood teachers, paying them more, restructuring their work to acknowledge its intensity and toll, creating and rewarding career progression, making early years programs permeable with local communities, creating vibrant, accessible and well-resourced professional learning environments, enhancing early childhood teacher preparation, incentivising transfer for those in other careers, funding vitally needed research… there is no shortage of ideas for where money can be spent.

Whether there is the courage and honesty to address the real problem at the heart of early childhood education is another matter.

Dr Lucinda McKnight is an Australian Research Council (ARC) Discovery Early Career Research Award (DECRA) Fellow in the REDI (Research for Educational Impact) Centre at Deakin University. She uses a range of feminist theories in her work on teacher autonomy and professionalism. She is also a mother of two children and has spent many hours caring for children at home, and providing community support in early childhood education as a parent helper. Follow her Teaching Digital Writing project blog or her twitter account @lucindamcknight8

Dr Natalie Robertson is a Senior Lecturer in Early Childhood Education at Deakin University. During her time working as an early childhood teacher, she developed a strong interest in workforce issues and play-based learning. These interests have followed Natalie into her later research and work in initial teacher education.  Natalie’s focus on workforce issues has framed her professional and research interests towards the attraction and retention of teachers in early childhood education. She is currently working with the Victorian Department of Education and Training to deliver the Early Childhood Professional Practice Partnerships (ECPPP) project) and the Innovate ITE program: Accelerated Bachelor of Early Childhood Education.

Do elite private boys’ school alumni have justice politics?

Education of boys, Gender, sexism in schools, single sex schools, teaching gender respect, , , , , , , , , ,

Featured Symposium at AARE 2021: Elite private boys’ schooling, feminism and gender justice: reimagining research in a post #me too world

On November 30 2021, while many of us were in paper sessions at the annual AARE conference, the findings of a review of workplace culture in parliament house were released. The review, led by Sex Discrimination Commissioner Kate Jenkins, was sparked by rape allegations made earlier this year by Brittany Higgins. The findings indicated that one in three people working in federal parliament has experienced some kind of sexual harassment there (Australian Human Rights Commission, 2021). What is also true is that a large number of MPs in the current parliament attended boys’ only schools, and recent revelations about the conduct of some boys in high fee-paying private boys’ schools have shone a negative light on them.

In September 2020, a year 12 muck-up day challenge at Sydney’s Shore school was made public which included such challenges as “spit on a homeless man”, “deck a stranger”, “sack whack a complete random walking past”, “get with someone below (age) 15”, and “get with an Asian chick”. In February 2021 Ms Chanel Contos, a former student at Kambala – an elite private girls’ school in Sydney – commenced a petition on social media for consent education to be taught earlier. This also attracted many testimonies from young women across the country regarding sexual assault from young men, many of whom attended elite private boys’ schools. 

A spotlight has therefore been focused on private boys’ schools and the male leaders they produce. All but two of Australia’s post war Prime Ministers (Bob Hawke and Julia Gillard) attended boys’ only schools, as did many men in the current parliamentary cabinet. Many of the men who attend boys’ only schools will come to occupy positions of significant privilege and power. There are crucial questions to be asked about the gender, class and race lessons being learnt by the young men attending such schools, and the way these travel with them as they come to occupy positions of influence in post-school life. Emeritus Professor Jane Kenway calls this the ‘misogyny pipeline’.

Published research shows us that such schools can be environments that are toxic for women teachers (Higham, 2018; Variyan, 2021) and indicates the sense of entitlement that can be fostered in such schools (Gaztambide-Fernández, Cairns & Desai, 2013). However it also indicates they are institutions that frequently engage in practices that are ostensibly about improving society and ameliorating justice (Kenway & Fahey, 2015). Indeed, how might these schools and their current and former students contribute to social justice rather than reproduce virulent forms of misogyny, classism and racism?

In response to such questions, AARE featured the research symposium Elite private boys’ schooling, feminism and gender justice: reimagining research in a post #me too world, at its annual conference. The symposium involved Drs Claire Charles and Lucinda McKnight, and Professor Amanda Keddie (Deakin University); Dr George Variyan (Monash University); Emeritus Professor Jane Kenway (Melbourne University); Professor Adam Howard (Colby College, USA), and Leanne Higham (LaTrobe University).

The symposium identified a range of challenges and opportunities for understanding questions of gender, class and race in elite private boys’ education both in Australia and the USA. A particular challenge identified was the ‘rules of entitlement’ that such schools implicitly teach their boys (Kenway). One such rule is that boys must know how to stay on top of all the hierarchies that matter. Given how strongly invested such schools, and their clients, are in hierarchies it was asked is it even possible to challenge this rule?

A key theme, in line with the conference title, was how we might re-imagine research in politically charged spaces, and in particular in/with elite private school boys and such schools’ alumni. Access to elite schooling for the purposes of research can be difficult. The symposium explored some different approaches to gaining insight into a culture where ‘what is part of the family stays with the family’. The schools were likened to a ‘secret brotherhood’ (Howard) where unsavoury are kept under a code of silence, although can sometimes be revealed to ‘insider’ researchers such as men who also attended elite boys’ schools, or by alumni who actively take up a more progressive justice politics. As part of re-imagining research in this space, the symposium also explored how researchers need to acknowledge their own positioning and investments (Charles, McKnight & Variyan).

A second theme was around how the schools themselves typically respond to revelations about their misogynistic cultures when they hit the media. Their crisis management techniques were identified. For example, they often respond by suggesting that such events are the result of a few ‘bad apples’ and are not representative of the broader culture or values of the school. A further strategy was their ‘dignified determination’ to address the issues. These defensive responses were described as a form of ‘misogyny masking’ (Kenway).

A key question, therefore, is how research, and the schools themselves, might address these problems. In particular, how research and teaching in elite private boys’ schools might seek to involve boys and men in working toward social justice. It is well established in research that involving men and boys in feminist projects can be a challenge yet one that is necessary if we are to change the status quo (Messner, Greenberg & Peretz, 2015). The symposium explored the discomfort and emotional intensities that boys and men often experience when they are invited to reflect on their complicity in perpetuating gender injustice (Keddie). It found that while such discomfort can be difficult, it is a necessary part of gender transformative work because you are dealing with personal violation. Such discomfort and emotions can be channelled in productive was for gender justice (Keddie). The role of researchers’ own relationships and emotions with regard to these schools was also explored (Charles, McKnight & Variyan).

In summary, recommendations arising from the symposium include the following:

·       That researchers continue to work with alumni from these schools to identify and further understand the factors that might assist some men to develop progressive justice politics both at school and later in life;

·       That further research is conducted into what may make elite private boys’ schools different from other elite schools that are co-educational or girls’ only schools, when it comes to addressing the problems outlined above;

·       That research and pedagogy continue to engage boys in working toward gender justice – including boys attending elite private boys’ schools.

Dr Claire Charles is a senior lecturer in the School of Education at Deakin University. Her research advances understanding of the justice politics of privileged young people in an unfair world.

Dr Charles pulled together this overview of research, including her own, presented at AARE201. The other authors are: Dr Lucinda McKnight is a senior lecturer in pedagogy and curriculum at Deakin University. She conducts award-winning research into curriculum design’s role in teacher identity, autonomy and professionalism, especially in English.  Dr George Variyan is a lecturer in Master of Educational Leadership in the Faculty of Education at Monash University. George’s engagement in research is based on a critically orientated sociology, which explores human agency in the relationship between education and society. Amanda Keddie is a Professor of Education at Deakin University. She leads the program: Children, Young People and their Communities within the REDI (Research for Educational Impact) Centre. Her research interests and publications are in the broad field of social justice and schooling. Professor Jane Kenway is an elected Fellow of the Academy of Social Sciences; Australia, Emeritus Professor at Monash University and Professorial Fellow at the University of Melbourne. Her research expertise is in educational sociology.  Adam Howard, Ed.D., is the Charles A. Dana Professor of Education and Chair of Education Program at Colby College, USA. Professor Howard’s research explores social class issues in education with a particular focus on privilege and elite education. Leanne Higham is a Lecturer in the School of Education at La Trobe University. A former secondary teacher, she is interested in the everyday practices of schooling and how these increase and enhance the capacities of those within schools, and/or limit and constrain them.

Image by Adobe Stock

The problem with using scientific evidence in education (why teachers should stop trying to be more like doctors)

evidence-based practices, National Evidence Base, randomised control trials, standardized practice, Teachers, Educational Data and Evidence-informed Practice, , , , ,

For teachers to be like doctors, and base practice on more “scientific” research, might seem like a good idea. But medical doctors are already questioning the narrow reliance in medicine on randomised controlled trials that Australia seems intent on implementing in education.

In randomised controlled trials of new drugs, researchers get two groups of comparable people with a specific problem and give one group the new drug and the other group the old drug or a placebo.  No one knows who gets what. Not the doctor, not the patient and not the person assessing the outcomes. Then statistical analysis of the results informs guidelines for clinical practice. 

In education, though, students are very different from each other. Unlike those administering placebos and real drugs in a medical trial, teachers know if they are delivering an intervention. Students know they are getting one thing or another. The person assessing the situation knows an intervention has taken place. Constructing a reliable educational randomised controlled trial is highly problematic and open to bias.

As a doctor and teacher thinking, writing and researching together we believe that a more honest understanding of the ambivalences and failures of evidence-based medicine is essential for education.

Before Australia decides teachers need to be like doctors, we want to tell you what is happening and give you some reasons why evidence based medicine itself is said to be in crisis. 

1. Randomised controlled trials are just one kind of evidence

Medicine now recognises a much broader evidence base than just randomised controlled trials. Other kinds of medical evidence include: practical “on-the-job” expertise; professional knowledge; insights provided by other research such as case studies; intuition; wisdom gained from listening to patient histories and discussions with patients that allow for shared decision-making or negotiation.

Privileging randomised controlled trials allows them to become sticks that beat practitioners into uniformity of practice, no matter what their patients want or need. Such practitioners become “cookbook” doctors or, in education, potentially, “cookbook” teachers. The best and most recent forms of evidence based medicine value a broad range of evidence and do not create hierarchies of evidence. Education policy needs to consider this carefully and treat all forms of evidence equally.

2. Medicine can be used as a bully

Teaching is a feminised profession, with a much lower status than medicine. It is easy for science to exert a masculinist authority over teachers, who are required to be ever more scientific to seem professional.  They are called on to be phallic teachers, using data, tools, tests, rubrics, standards, benchmarks, probes and scientific trials, rather than “soft” skills of listening, empathising, reflecting and sharing.

A Western scientific evidence-base for practice similarly does not value Indigenous knowledges or philosophies of learning. Externally mandated guidelines also negate the concepts of student voice and negotiated curriculum. While confident doctors know the randomised controlled trial-based statistics and effect sizes need to be read with scepticism, this is not so easy for many teachers. If randomised controlled trial-based guidelines are to rule teaching, teachers will also potentially be monitored for compliance with guidelines they may not fully understand or accept, and which may potentially harm their students.

3. Evidence based medicine is about populations, not people

While medical randomised controlled trials save lives by demonstrating the broad effects of interventions, they make individuals and their needs harder to perceive and respect.  Randomised controlled trial-based guidelines can mean that diverse people are forced to conform to simplistic ideals. Rather than starting with the patient, the doctor starts with the rule. Is this what we want for teaching? When medical guidelines are applied in rigid ways, patients can be harmed.

Trials cannot be done on every single kind of person and so inevitably, many individuals are forced to have treatments that will not benefit them at all, or that are at odds with their wishes and beliefs. Educators need to ensure that teachers, not bureaucrats or researchers, remain the authority in their classrooms.

5. Scientific evidence gives rise to gurus

Evidence-based practice can give rise to the cult of the guru. Researchers such as John Hattie, and their trademarked programs like “Visible Learning” based on apparently infallible science, can rapidly colonise and dominate education. Yet their medicalised glamour disguises the reality that there is no universal and enduring formula for “what works”.

In 2009, in his book Visible learning: A synthesis of over 800 meta-analyses relating to achievement Hattie advised that, based on evidence, all healthy people should take aspirin to prevent heart attacks. Yet also in 2009, new medical evidence “proved” that the harms in healthy people taking aspirin outweigh the benefits.

In 2009 Hattie said class size does not matter. In 2014, further research found that reducing class size has an important and lasting impact, especially for students from disadvantaged backgrounds.

While medical-style guidelines may seem to have come from God, such guidelines, even in medicine are often multiple and contradictory. The “cookbook” teacher will always be chasing the latest guideline, disempowered by top-down interference in the classroom.

In medicine, over five years, fifty percent of guideline recommendations are overturned by new evidence. A comparable situation in education would create unimaginable turmoil for teachers.

6. Evidence-based practice risks conflicts of interest

Educational publishers and platforms are very interested in “scientific” evidence.  If a researcher can “prove” an intervention works and should be applied to all, this means big dollars. Randomised controlled trials in medicine routinely produce outcomes that are to the benefit of industry. Only certain trials get funded. Much unfavourable research is never published. Drug and medical companies set agendas rather than responding to patient needs, in what has been described as a guideline “factory”.

Imagine how this will play out in education. Do we want what happens in classrooms to be dictated by profit driven companies, or student-centred teachers?

What needs to happen?

We call for an urgent halt to the imposition of ‘evidence-based’ education on Australian teachers, until there a fuller understanding of the benefits and costs of narrow, statistical evidence-based practice. In particular, education needs protection from the likely exploitation of evidence-based guidelines by industries with vested interests.

Rather than removing teacher agency and enforcing subordination to gurus and data-based cults, education needs to embrace a wide range of evidence and reinstate the teacher as the expert who decides whether or not a guideline applies to each student.

Pretending teachers are doctors, without acknowledging the risks and costs of this, leaves students consigned to boring, standardised and ineffective cookbook teaching. Do we want teachers to start with a recipe, or the person in front of them?

Here is our paper for those who want more: A broken paradigm? What education needs to learn from evidence-based medicine by Lucinda McKnight and Andy Morgan

Dr Lucinda McKnight is a pre-service teacher educator and senior lecturer in pedagogy and curriculum at Deakin University, Melbourne. She is also a qualified health and fitness professional. She is interested in the use of scientific and medical metaphor in education. Lucinda can be found on Twitter@LucindaMcKnigh8

Dr Andy Morgan is a British Australian medical doctor and senior lecturer in general practice at Monash University, Melbourne. He has an MA in Clinical Education from the Institute of Education, UCL, London. His research interests are in consultation skills and patient-centred care. He is a former fellow of the Royal College of General Practitioners, and current fellow of the Australian Royal College of General Practitioners.