Any minute, your university students will get an email with a link. That link leads to one of the most dire tools of university performance, the evaluations of course content and teaching quality.
These evaluations are meant to provide feedback to enhance course design and teaching methods. However, for several decades research has shown that despite the questions being asked, the factors influencing students’ responses have a minimal amount to do with either the course or teaching quality.
They are instead shaped by student demographics, prejudice towards the teaching academic, and biases shaped by the classroom and university setting.
Despite the clear flaws underpinning the data student evaluations collect, universities continue to use this data as a measure of an academic’s teaching performance. Evaluation results influence an academic’s likelihood of being hired on a continuing basis for contract and sessional staff, receiving promotions for existing staff, and being fired or managed out during staff restructures.
This is a flawed method of evaluating people and it raises questions of why the sector continues to use student evaluations. But the negative impact is complicated further by the fact that we know evaluations impact on different groups of academics to different degrees. The groups impacted the most are the groups the academy declares to value, hopes to protect, and claims to have an interest in fostering their careers.
I recently completed a study where I reviewed the findings of existing research about student evaluations of courses and teaching. The paper, Sexism, racism, prejudice, and bias: a literature review and synthesis of research surrounding student evaluations of courses and teaching, found that across studies covering more than 1,000,000 student evaluations, it is clear that women are at a disadvantage compared to men.
Different studies suggest the disadvantage can vary in size, and is highly dependent on disciplinary area, student demographics and other factors, but across the board, women are judged more harshly than men. At the extreme, this means women are more likely to fail evaluations than men, and researchers have routinely cited examples of more capable and higher performing women receiving lower scores than their less capable male counterparts. These results predictably mean women fare worse in job applications and promotions, and has been cited as a reason why women are represented less in the professoriate, and fill fewer leadership positions.
The same is true of factors such as race, gender, sexual identity, disability, language and other marginalising characteristics. Studies in different locations across more than two decades of solid research continually find that if an academic is not a white, English speaking, male in the approximately 35-50 year old age group and who students perceive to be able-bodied and heterosexual, this will result in some form of lower evaluation result. The negative repercussions of these results are also cumulative; a woman will receive lower results, and a person with a visible disability will receive lower, so a woman with a visible disability is likely to be treated extra harshly in the evaluations of her course and teaching.
What also cannot be ignored is that as a majority of the existing data originates from large-scale quantitative surveys, repeatedly researchers have noted that the rates of people within the sector who are disabled, identify as LGBTIQA+, people of colour, are refugees or immigrants, or a part of other marginalised groups are so underrepresented in the higher education sector that they do not count as a valid sample size.
At the broadest level, multiple studies showed that evaluation results can be impacted by disciplinary area and assessment type. Several studies have shown that academics in the sciences and associated fields receive lower evaluation results than their counterparts in the humanities and social sciences. Similarly, it has been noted that academics whose courses use essays and presentations for assessment fare better than those who rely on exams.
Institutional factors that have nothing to do with the class, or the academic teaching the class, have also been cited as reasons an academic will receive a lower evaluation score. Lower results can be given because of the class scheduling, class location, classroom design, class cleanliness, library facilities, and even the food options available on campus; all factors beyond the control of the academic teaching the class.
Official university responses to why they continue to carry out student evaluations when evaluations are so flawed and prejudiced towards the sector’s most vulnerable groups are rare. Existing studies suggest universities need data about course content, teaching quality, and student satisfaction, and student evaluations are the most cost and time effective method of gaining this information. In the past, perhaps the lack of data around evaluations was enough to convince institutions that a method of data collection that was seemingly not perfect was still acceptable due to the data that could be obtained rather quickly and easily.
Considering what we know in 2021, time and cost effectiveness are not good enough reasons to continue a flawed practice that so blatantly discriminates against the sector’s women and those from marginalised groups.
Dr Troy Heffernan is Lecturer in Leadership at La Trobe University. His research examines higher education administration and policy with a particular focus on investigating the inequities that persist in the sector.
