Leanne Fray

How our messy research journey survived floods, fires and COVID19

See this presentation in real time today (December 2, 2021) in the Schools and Education Systems SIG at 10am

Large research trials are complex and difficult to manage at the best of times. At AARE 2021 this week, around 900 papers have been presented, many reporting clean and tidy findings from research studies. Twenty minutes doesn’t provide enough time to tell the full story.

And it’s not one that researchers are encouraged to tell.

I want to use my experience as the project manager of the largest randomised controlled trial in Australian education research history to expose the messy, unpredictable, challenging, and at times down-right insane rollercoaster of conducting school-based research.

In 2018, the Teachers and Teaching Research Centre was awarded $17.1M in funding from the Paul Ramsay Foundation to undertake a comprehensive and rigorous program of research examining the impact of Quality Teaching Rounds (QTR) on teacher and student outcomes.

Our massive, four-arm randomised controlled trial began in 2019 and is in the throes of final data collection right now. Over the past three years we have had to contend with several catastrophes of epic proportions including the Black Summer Bush Fires, state-wide flooding, a global pandemic followed by the local Delta outbreak. 

Now throw in a touch more flooding, and a teachers’ strike to boot.

And yet, despite these challenges, we have (just about) successfully completed this research, gathered incredible amounts of data and published ground-breaking findings. We’ve also learned lessons about the realities of school-based research that I believe would be valuable to share.

We set out in 2018 to recruit 200 NSW government primary schools, with four teachers from each to participate in this research trial. Our first major challenge was recruiting schools. When baseline data collection began in Terms 1 and 2 2019, we had just 125 schools. This necessitated a split-cohort design, with a second cohort of 80 schools planned for 2020.

To manage the huge scale of baseline and follow-up data collection we built up our team of research assistants to more than 50. We almost made it through the follow-up data collection in Term 4 2019 when catastrophic bushfires broke out throughout NSW. 20 of our research schools were closed, which meant constant reshuffling of school visits and monitoring bushfire locations to ensure the safety of our research assistants. Remarkably, we were able to collect data from 124 of 125 schools.

The bushfires continued to hamper our efforts into the start of 2020 as we finalised cohort 2 recruitment and prepared for baseline data collection. Adding to the emergency situation, the fires were followed by significant flooding across many parts of regional NSW, again affecting a number of our research schools (one school was literally wiped off the map).

We’d almost completed baseline data collection for cohort 2 when, in March 2020, COVID-19 forced state-wide school closures. The decision was made to postpone the trial to 2021. However, with the baseline data already collected and comparable control group data from the previous year, we were uniquely positioned to repurpose the data to complete one of the world’s earliest empirical studies on the effects of COVID-19 on student learning

We maintained strong relationships with our research schools throughout this incredibly challenging year and, with support from the NSW Department of Education, we were able to get follow-up data to see what, if any, impact COVID had on student achievement.

As a strong sign of support for our work, most of the 2020 schools signed up again to participate in 2021. Everything started smoothly, baseline data were collected, teachers participated in QTR, then Delta hit on the eve of the Term 2 holidays.

Despite an entire term of remote learning, we are back in the 80 schools right now collecting follow up data. Changing government health orders over the last few weeks meant asking teachers to collect student data on our behalf, then being able to send research assistants to visit schools after all. It’s meant rapid scaling up and scaling down of our team, organising training and support for teachers, as well as organising logistics for research assistants to visit schools.

It has required incredible flexibility, adaptability and coordination in a very short time period, while COVID continues to impact schools. Next week we’re heading to the last of the schools, though right now we are juggling schedules around the planned industrial action.

Since 2019, 205 schools, 757 teachers and more than 10,000 students have participated in this study. To date we’ve visited schools to collect; 1,102 full lesson observations, more than 45,000 PATs, 15,000 student surveys and 1,700 teacher surveys. We’ve published significant findings and world-leading research.

Conducting research of this scale has required constant evaluation and refinement and has led to several important learnings. 

Research with schools is hard and complex. It’s costly and it’s taxing. Both on workloads and on wellbeing. I think it’s important to recognise that.

Contingency planning is critical. Things will go wrong. We could not have anticipated a global pandemic, but having plans for quickly responding to school closures or emergency situations helps when the unexpected happens.

Effectively navigating institutional constraints and regularly refining processes are essential for work of this scale. Our processes look a lot different now to when we began in 2019.

Stakeholder relationship management is crucial. Ensuring buy in from department executives and funding body representatives, school leaders and teachers – and even research support staff – will help when things invariably go pear-shaped. 

Schools do want to engage in meaningful research. It’s important that the research has explicit links to school priorities, has reasonable expectations of participants, provides access to useful data that schools can engage with and, finally, includes a capacity building dimension for teachers or leaders.

We are blessed to have a supportive funding partner and a significant and rare amount of funding which has enabled us to postpone, restart, repurpose data, and persevere. Research is difficult and it is messy. Learning from experience is important.

Wendy Taggart is the senior project manager in the School of Education, College of Human and Social Futures, University of Newcastle. This work is from a paper co-authored with Jenny Gore, Andrew Miller, Jess Harris and Leanne Fray.

How children’s aspirations change as they grow up: latest research

Two research findings from our University of Newcastle large-scale longitudinal study of the career aspirations of Australian children attracted a lot of media coverage recently. The first was that Australian children begin to form career aspirations from a very young age and the second that children have similar aspirations whether they are from low or high economic status families.

These findings are inspiring some rethinking around career education in Australian schools and how things might change to help children realise their aspirations.

But there is another aspect of our findings that has not yet been given the media spotlight, and it may be just as significant. It is the way children’s aspirations change over time.

Of course some change is to be expected, but as we unpack what is happening we can see patterns emerging and believe schools, teachers, parents and university recruiters should be paying much closer attention to what is happening.

What we found

In Year 3, children aspired most to having a career as Arts professionals (musician, artist, writer and so on), followed by School Teachers, Veterinarians, Architects, and Science professionals. These were the top five occupations where a university education was involved.

The next most popular careers were Engineering Professionals, Medical Professionals, Social and Welfare Professionals, Legal Professionals and Registered Nurses/Midwives.

However we found that interest in some occupations – arts, architecture and veterinary science – declines in the later years of schooling, while interest in others – engineering, nursing, and social and welfare work – grows.

Interest in teaching, medicine, legal and science careers is more stable across the school years.

In some occupational categories, interest appears to rise or fall towards the very end of high school. For example, students are less likely to aspire to be a vet or artist as they mature, but more likely to aspire to architecture, engineering, medicine, social work or law. Furthermore, significant interest in these careers is often expressed as early as Year 7, sometimes Year 5. In other careers, such as teaching and science, student interest is more consistent across year levels.

Why do children change their aspirations?

The data we have collected gives us a clearer view of how and when aspirations change. This evidence provides fertile ground for any policy maker or program developer involved in career education.

The variations we found across year levels might relate to ongoing assessment by students of their abilities and achievement levels as they age or, indeed, to a more realistic understanding of what is involved in certain careers.

However it is possible these patterns indicate a range of quite specific influences, such as: how a teacher communicates expectations of a child (whether they will continue on to university); or a family’s understanding of how paying for university education works (believing it costs too much and they can’t afford it); or even an understanding that university study is involved in the pathway to a certain career or belief that the pathway is possible.

The process of forming aspirations can have a profound influence on the life prospects of a child. Our ongoing research is looking closely at what is happening here, with the aim of informing teachers, higher education providers, and policy makers.

 

Jenny Gore is Professor of Education at the University of Newcastle. Her research interests have consistently focussed on the quality of teaching and learning, teacher socialisation, alternative pedagogy, power relations in teaching, reform in teacher education and pedagogical reform. She has been involved in and/or managed several large research grants, with research income over $5.9 million. Jenny was a member of the research team that generated the concept of Productive Pedagogy and, with Associate Professor James Ladwig, was co-author of the NSW model of pedagogy known as Quality Teaching. Professor Gore was Dean of Education and Head of the School of Education at the University of Newcastle (2008-2013) and and has held positions as President of the NSW Teacher Education Council, Executive member of the Australian Council of Deans of Education, Research Training Coordinator for the Australian Association for Research in Education, and Associate Editor of Teaching and Teacher Education. Jenny’s major books include The Struggle for Pedagogies: Critical and feminist discourses as regimes of truth and Feminisms and Critical Pedagogy (edited with Carmen Luke). Her current research programs focus on understanding student aspirations for greater equity and investigating teacher professional development through Quality Teaching Rounds.

Research evidence of issues facing disadvantaged students in higher education

The issues facing disadvantaged students wanting a tertiary education are multi-faceted. Just getting into a course at university can be difficult, then there are many hurdles students will face before they actually complete their degree.

This is why funding of over $1 million was made available by The National Centre for Student Equity in Higher Education (NCSEHE) during 2014 and 2015 for research projects at Australian universities and other research organisations to investigate aspects of student equity in higher education.

The competitive research grants program was designed to further investigate the impact higher education policy has on marginalised and disadvantaged students and how we could improve participation and success. The NCSEHE publication ‘Informing Policy and Practice’ highlights the outcomes of the first 12 research reports.

Each report addresses different, but related, aspects of higher education student equity. They all bring evidence-based investigation to the consideration of policy and practice. This research highlights the complexity of the issues the researchers are attempting to unravel, and that simple statements arising from analysis need to be carefully considered.

The results confirm that more needs to be done to ensure that capable people are not prevented from accessing and completing higher education.

Higher education confers significant individual benefits in terms of personal development, career opportunities and lifetime learning. Higher education is also the key to the social well-being and economic prosperity of Australia. Providing access to higher levels of education to people from all backgrounds enhances social inclusion and reduces social and economic disadvantage.

In the interests of individuals and for the nation, higher education equity for all capable people must be seen as an objective of the system.

We know, from our research, that the policy framework needed to achieve the required change for disadvantaged people will not result from a single policy decision or funding program. It is complex and challenging and needs a wide-ranging response.

There are 12 research reports available. They include research across the various equity groups:-

Resilience/Thriving in Post-Secondary Students with Disabilities: An Exploratory Study
by Dr Rahul Ganguly, Dr Charlotte Bronwlow, Dr Jan Du Preez and Dr Coralie Graham (University of Southern Queensland)

Educational outcomes of young Indigenous Australians
by Stephane Mahuteau, Tom Karmel, Kostas Mavromaras and Rong Zhu (National Institute of Labour Studies at Flinders University)

Are low SES students disadvantaged in the university application process?
by Dr Buly Cardak (La Trobe University), Dr Mark Bowden and Mr John Bahtsevanoglou (Swinburne University of Technology)

Choosing university: The impact of schools and schooling
by Jenny Gore, Kathryn Holmes, Max Smith, Andrew Lyell, Hywel Ellis and Leanne Fray (University of Newcastle)

Do individual background characteristics influence tertiary completion rates?
by Patrick Lim, National Centre for Vocational Education Research (NCVER)

Completing university in a growing sector: is equity an issue?
by Dr Daniel Edwards and Dr Julie McMillan (ACER)

Exploring the experience of being first in family at university
by Associate Professor Sharron King (University of South Australia), Dr Ann Luzeckyj (Flinders University), Associate Professor Ben McCann (University of Adelaide) and Ms Charmaine Graham (University of South Australia)

Secondary School Graduate Preferences for Bachelor Degrees and Institutions
by Trevor Gale (Deakin University), Stephen Parker, Tebeje Molla, Kim Findlay, with Tim Sealey

Best practice bridging: facilitating Indigenous participation through regional dual-sector universities
by Bronwyn Fredericks ( CQUniversity) et al

University access and achievement of people from out-of-home care backgrounds
by Andrew Harvey, Patricia McNamara, Lisa Andrewartha, Michael Luckman (La Trobe University)

Understanding Evaluation for Equity Programs: A guide to effective program evaluation
by Dr Ryan Naylor, Melbourne Centre for the Study of Higher Education (University of Melbourne)

Equity groups and predictors of academic success in higher education
by Jill Scevak, Erica Southgate, Mark Rubin, Suzanne Macqueen, Heather Douglas, Paul Williams (University of Newcastle)

 

Sue_Trinidad_Retouched_2 copy

 

Professor Sue Trinidad – Prior to becoming the Centre for Student Equity in Higher Education’s Director, Sue was Deputy Pro-Vice Chancellor and Dean of Teaching and Learning in the Faculty of Humanities at Curtin during 2007-2012. In these roles she provided academic leadership for the five schools and led the Higher Education Equity Participation Program for a large faculty which had many LSES, Indigenous and regional students. Sue is an established scholar in the areas of higher education pedagogy and change management, the use of technology and student learning. Her research covers higher education and leadership, including the use of technology for regional, rural and remote areas to provide equity access to all students regardless of their geographical location. Sue has also been involved in consultancies, research projects and grants both in Australia and internationally, including Australian Research Council and Office for Learning and Teaching funded research. She currently sits as an advisor to the Western Australian Minister of Education on the Regional and Remote Advisory Council (RREAC).  Her teaching, learning and research have been acknowledged by a number of awards including the 2001 Life Membership Award for the Educational Computing Association of Western Australia for her work with teachers, two best research paper awards in 2004 and 2006,  the Vice-Chancellor’s Award for Excellence and Innovation in Higher Education in 2010; a Citation for Outstanding Contributions to Student Learning 2014; and the PTCWA Outstanding Professional Service Award 2014.