See this presentation in real time today (December 2, 2021) in the Schools and Education Systems SIG at 10am
Large research trials are complex and difficult to manage at the best of times. At AARE 2021 this week, around 900 papers have been presented, many reporting clean and tidy findings from research studies. Twenty minutes doesn’t provide enough time to tell the full story.
And it’s not one that researchers are encouraged to tell.
I want to use my experience as the project manager of the largest randomised controlled trial in Australian education research history to expose the messy, unpredictable, challenging, and at times down-right insane rollercoaster of conducting school-based research.
Our massive, four-arm randomised controlled trial began in 2019 and is in the throes of final data collection right now. Over the past three years we have had to contend with several catastrophes of epic proportions including the Black Summer Bush Fires, state-wide flooding, a global pandemic followed by the local Delta outbreak.
Now throw in a touch more flooding, and a teachers’ strike to boot.
And yet, despite these challenges, we have (just about) successfully completed this research, gathered incredible amounts of data and published ground-breaking findings. We’ve also learned lessons about the realities of school-based research that I believe would be valuable to share.
We set out in 2018 to recruit 200 NSW government primary schools, with four teachers from each to participate in this research trial. Our first major challenge was recruiting schools. When baseline data collection began in Terms 1 and 2 2019, we had just 125 schools. This necessitated a split-cohort design, with a second cohort of 80 schools planned for 2020.
To manage the huge scale of baseline and follow-up data collection we built up our team of research assistants to more than 50. We almost made it through the follow-up data collection in Term 4 2019 when catastrophic bushfires broke out throughout NSW. 20 of our research schools were closed, which meant constant reshuffling of school visits and monitoring bushfire locations to ensure the safety of our research assistants. Remarkably, we were able to collect data from 124 of 125 schools.
The bushfires continued to hamper our efforts into the start of 2020 as we finalised cohort 2 recruitment and prepared for baseline data collection. Adding to the emergency situation, the fires were followed by significant flooding across many parts of regional NSW, again affecting a number of our research schools (one school was literally wiped off the map).
We’d almost completed baseline data collection for cohort 2 when, in March 2020, COVID-19 forced state-wide school closures. The decision was made to postpone the trial to 2021. However, with the baseline data already collected and comparable control group data from the previous year, we were uniquely positioned to repurpose the data to complete one of the world’s earliest empirical studies on the effects of COVID-19 on student learning.
We maintained strong relationships with our research schools throughout this incredibly challenging year and, with support from the NSW Department of Education, we were able to get follow-up data to see what, if any, impact COVID had on student achievement.
As a strong sign of support for our work, most of the 2020 schools signed up again to participate in 2021. Everything started smoothly, baseline data were collected, teachers participated in QTR, then Delta hit on the eve of the Term 2 holidays.
Despite an entire term of remote learning, we are back in the 80 schools right now collecting follow up data. Changing government health orders over the last few weeks meant asking teachers to collect student data on our behalf, then being able to send research assistants to visit schools after all. It’s meant rapid scaling up and scaling down of our team, organising training and support for teachers, as well as organising logistics for research assistants to visit schools.
It has required incredible flexibility, adaptability and coordination in a very short time period, while COVID continues to impact schools. Next week we’re heading to the last of the schools, though right now we are juggling schedules around the planned industrial action.
Since 2019, 205 schools, 757 teachers and more than 10,000 students have participated in this study. To date we’ve visited schools to collect; 1,102 full lesson observations, more than 45,000 PATs, 15,000 student surveys and 1,700 teacher surveys. We’ve published significant findings and world-leading research.
Conducting research of this scale has required constant evaluation and refinement and has led to several important learnings.
Research with schools is hard and complex. It’s costly and it’s taxing. Both on workloads and on wellbeing. I think it’s important to recognise that.
Contingency planning is critical. Things will go wrong. We could not have anticipated a global pandemic, but having plans for quickly responding to school closures or emergency situations helps when the unexpected happens.
Effectively navigating institutional constraints and regularly refining processes are essential for work of this scale. Our processes look a lot different now to when we began in 2019.
Stakeholder relationship management is crucial. Ensuring buy in from department executives and funding body representatives, school leaders and teachers – and even research support staff – will help when things invariably go pear-shaped.
Schools do want to engage in meaningful research. It’s important that the research has explicit links to school priorities, has reasonable expectations of participants, provides access to useful data that schools can engage with and, finally, includes a capacity building dimension for teachers or leaders.
We are blessed to have a supportive funding partner and a significant and rare amount of funding which has enabled us to postpone, restart, repurpose data, and persevere. Research is difficult and it is messy. Learning from experience is important.
Wendy Taggart is the senior project manager in the School of Education, College of Human and Social Futures, University of Newcastle. This work is from a paper co-authored with Jenny Gore, Andrew Miller, Jess Harris and Leanne Fray.
At every university around the country, academics in schools and faculties of Education have been hit hard. Hundreds, maybe thousands, have lost their jobs. Many of them are people we know. Yet it is not easy to identify the particular staff who have ‘disappeared’ from classes, courses and schools of Education among the seventeen and a half thousand other university staff who lost jobs around Australia during the initial COVID response alone. These losses continue: we read about them daily. And higher education job losses affect far more than individuals and their personal aspirations. They also affect their families, their health, their mortgages, and the families and welfare of the communities in which they live, work and shop. The fall-out is being felt everywhere, although it is most obvious in those regional cities highly dependent on the local university for their economic prosperity.
But what we are failing to notice is that these effects are particularly important in our Education faculties, at a time when states are facing a looming teacher shortage and the Federal Minister for Education and Youth is reviewing the capacity of our universities to attract high-quality candidates into teaching and to supply highly effective teachers. If education is crucial to nation-building, there could not be a greater need for high-quality graduates to staff schools around the country.
But the academics who have survived in our schools of Education, either scraping a career together as short-term casuals, or scrabbling to remain as full-timers, are doing it tough.
The climate of anxiety and insecurity in which these people (our neighbours, relatives, friends, clients and colleagues) are living is reminiscent of accounts of totalitarian regimes. In every capital city and university town around Australia, people are living in fear – afraid to say no when they are asked to do things that do not sit well on their consciences; afraid not to agree with the rationalisation to course content and assessment review needed to cope with increased workload; afraid to admit that they haven’t had time to properly read and consider the implications of the policy changes they are being asked to approve in governance committees. Heads down, they are keeping under the radar as much as they can in order to survive. They are not proud of what they are doing at work, and they know the quality of what students are being offered is suffering too. Headlines this week such as Murdoch Uni gags staff as students disillusioned over education quality are beginning to reflect one reason why departing staff are often silenced by the non-disclosure clauses in their redundancy agreements.
She fears for her career if she names this place. In another university, a key professional staff member, whose knowledge and expertise in supporting the faculty’s upcoming course accreditation renewal are literally irreplaceable in the short term, has chosen to move on because he can no longer live with the moral disappointments of his daily work. He needs to keep referees on side. And in a third institution another casual staff member, studying for the PhD that may now, ironically, lessen her chances of future employment, has been given three new subjects to teach with less than a fortnight’s notice. Only one of these subjects is in her area of expertise. She knows she hasn’t got time to read the material she will be teaching, but she needs the work. She will do her best, based on years of classroom teaching experience. While she knows it isn’t, her generalist knowledge is deemed adequate to teach the specialist knowledge that the Course Team, the Academic Board, AITSL, the profession, and Education Minister Tudge all see as necessary for her students to meet Australia’s Graduate Teacher standards. A staffer at QUT, ‘safe’ for the moment, describes effects that are also experienced by peers in other places: “Everything that gets done is being pushed back to academic staff – everything. Academics who are not experts in professional tasks are doing professional tasks, which takes incredible amounts of time. There is a training video or a pdf training note for everything – and you get sent hyperlinks for these if you ask for help”. The loss of professional staff, or their relocation to central service areas, also affects the quality of what can be done across the board. For Education, this is not good enough.
Schools and faculties of Education have been particularly hard-hit by longer-term structural change and stringency in universities, beginning before COVID. More recent reports of stress, overwork, anxiety are not limited to Education staff of course, highlighting the bleak picture across institutions. Staff who are still employed must pick up the work of lost colleagues, and they are increasingly worried about what they are offering their students. This is a sector in crisis. A WHS survey conducted earlier this year at the University of Wollongong indicated that, there, 90% of respondents believe there are not enough staff to get the work done, and 66% have considered leaving because of workplace stress (NTEU 2021p. 3). And alongside the serious problem of human and workforce costs, there is a pressing long-term issue for the nation in terms of the quality of what faculties of Education can offer their students ‘on the cheap’.
It is obvious that the people who are being made ‘redundant’, or who are ‘separating’ from the institutions where they work are workers – the people who get things done. They are not the managers, the highly paid senior executive staff, outside of Faculties, who direct and should govern what goes on. Mostly it is more senior academics – the more experienced workers – who are targeted for redundancy, because they are by definition not at the lowest pay rate.
In some institutions, such as the University of New South Wales, Canberra, QUT, and UniSA, the impact of staff losses is not visible in current numbers. At UNSW, four senior Education staff took Voluntary Redundancies at the end of 2020, but as a staff member there says, these are being replaced by three new appointments this term. Staff at UniSA say the situation is similar there. At UNSW, there are still hidden impacts – the increase in workload due to online and dual mode delivery, an increase in class sizes and what colleagues see as the exploitation of casual academics who are pressured by students to spend more time working with them online – and are afraid to refuse. At Macquarie, while education staff are hopeful that after losing six staff in 2020, they should avoid further redundancies in 2021 because they have made “sufficient internal savings”, yet staff cuts within the faculty will again be considered at the beginning of 2022.
Other places are already in real trouble. At the University of Newcastle, which has earned a strong reputation for its educational research in NSW, staff say their numbers in 2020 were already down more than 10 in recent years, and they will have lost at least another 10 FTE staff members by the end of this year. Unlike other areas of that university, it seems, these education positions do not seem to merit replacement.
Similarly, staff at the University of Melbourne report the loss of at least 13 FTE academic staff who have left Education, either taking redundancy packages or losing fixed-term contracts – half of these positions were at senior levels. While this has also been effective as a cost-saving strategy for the University, staff who are left report that they now find it hard to contract sessional staff, who are getting much more secure and rewarding work as casual teachers in schools (and who are being targeted by some state departments as potential ‘career-changers’ for more permanent roles). Staff at Griffith University say they had around 55 full time academics in 2020, but this is down to around 44. And at UTS, over recent years, education has been steadily decreasing in size. What was a Faculty of Education was reduced to a School of Education, and then most recently to a merged School of International Studies and Education. As one staff member reports, “We started to feel more and more invisible, despite being told by the University leadership that commitment to Education was a part of the University’s social justice mission”.
But they don’t have relationships with experienced professional staff, and they often don’t know the reasons why they need to adhere to policies. They don’t know who to talk to when they need to understand something to give good advice to a student; and they can’t see why they should not ‘improve’ the assessment task that has been carefully designed by a course team and approved by an Academic Board.
In some cases, the disappearing staff are also taking the higher-level disciplinary expertise that the faculty relies on to meet TEQSA’s HE standards for staff qualifications. As a staff member at one institution says regretfully: “It is now even more possible that a student undertaking their Master of Teaching course at this university may get through their whole degree and have only been taught by sessional teaching staff. This is in a faculty that is supposed to be ranked 1 or 2 in Australia for Education!”
Worse, universities are disguising this information in their reports to government. James Guthrie and Brendan O’Connell’s analysis of data from the Department of Education, Skills and Employment shows that changes have been made by universities in how they are accounting for their employees in 2020. This means that official government reports can not be reconciled with the numbers for staffing presented in the same universities’ 2020 annual reports. In his account of the obfuscation of numbers currently reported at the University of Wollongong, Guthrie has also pointed to the unacceptable variation of reported figures to the public and government of staff losses – estimating that this accounts for up to 500 positions in this university alone.
And even worse still, at a time when the Morrison government has indicated that Australia’s 39 comprehensive universities may not be offering Australia an “optimal model for the quality of teaching or research”, the scene is being set for a possible return to a binary system in higher education. After the 2019 Coaldrake Report’s insistence that (real) universities should be involved in both research and teaching, these impacts on the quality of teaching in Education schools are indeed alarming. Regulations that institutions are required to meet in the HES are simply not being met. While some universities clearly consider that this can be disguised for a short time, the academic risks are enormous, and some universities are clearly making no effort to sustain the quality of their Education faculties.
This is particularly noticeable in relation to the Coaldrake requirements for universities to be producing high-quality, world standard research in the disciplines they wish to teach – and it is now a matter of urgency for Education faculties around the country. A recent report into the critical need for addressing research with education faculties cites Coaldrake to argue that these events are not just bad luck or bad timing for Education. The university ideal of retaining both teaching and research in one academic position is fundamental to the teaching-research nexus in academic work. It seems “more than a minor oversight that the move to increase teaching-only positions in many universities also prevents them from doing research” (pp.52-53).
In many institutions, even those that have not yet had academic job losses, staff report that this is happening. At UTS, at Flinders, at QUT, “People are stretched, and time for research for most is limited. Some struggle to find time even for service or HDR supervision because they are doing so much teaching and have so many students to mark for.”
As the AARE national survey of staff in Education schools and faculties has found, education research is becoming a luxury. Their data shows that “education research is now not only being subsidized by significant amounts of unpaid labour but also the direct financial contributions of individual academics trying to keep their main form of research development available” (Brennan et al., 2020, p. 36). One academic in one of the few Faculties of Education left in Australia speaks of how her research profile is only being sustained by “the generosity of colleagues in other institutions”, as her teaching workload allows no time to contribute to the writing up of their research.
The example of the decline of Education at UTS shows the inevitable result of these circumstances. As staffing cuts led to the structural changes noted above, staff tell how in the new Faculty they were presented with data about the ‘viability’ of Education as a discipline. It is not surprising that fewer staff produce less research income and fewer high impact publications. As one staff member says: “We’d lost so many of our Level D and E academics over time, and Level Cs were expected to demonstrate research leadership beyond their experience.
It is clear that both education research and teaching are under threat in our universities as well as in our university system. Education is the key to any sort of future for Australia, and while every state appears to be facing imminent teacher shortages, the complicity of universities in allowing the quality of education research and teaching, at the present time, is a serious concern that should be worrying TEQSA as well as our politicians.
Jo-Anne Reid is Professor Emeritus in the Faculty of Arts and Education at Charles Sturt University. She has collaborated on a range of national competitive grants over her career, focussing on primary/secondary literacy and English teaching, teacher education, minority-group and Indigenous teachers, literacy and the environment, and rural teacher education.