Abstract:
Systematic reviews have become a popular means of gathering evidence of effectiveness in equity programs. With their measures to remove researcher bias and efforts to provide replicability, systematic reviews have been seen, particularly by policy-makers, as producing a clear and sound basis for the allocation of scarce equity funds. Systematic reviews help researchers to leave the ‘bubbles’ of their own expertise; theoretical perspective, national context, and allow them to synthesise international evidence about an area of research. Such reviews claim to provide impartial accounts of best practice through a neutral process that reports on the results of other published research. While there are real strengths to casting a wide net to capture what we may not know, there are a series of implicit methodological assumptions behind systematic reviewing which, we argue, bend the results in very distinct directions. In this paper, we offer a critique of systematic reviewing and question its suitability for understanding problems of equity and inequality. We argue that the ingredients of a systematic review in fact bias the kinds of research that is evaluated towards quantitative studies that may tell us very little about the nature of inequality in higher education and programs that impact on this. Especially when studying a large literature, the reliance systematic reviewing places on database algorithms to provide the ‘best’ evidence selects papers from the geopolitical centre, at the expense of research from less well-funded institutions and more marginalised regions. Indeed, we argue that systematic review techniques tend to reinforce an epistemic violence that centres ‘legitimate knowledge’ in the global north. Every link of the systematic review chain is embedded in the prevailing political economy of academic publishing houses, corporate databases, and public and private funding sources. No matter how thoroughly we test our search terms, the system will continue to produce results that favour research produced in richer nations, quantitative over qualitative research methods, better funded Higher Education institutions, research published in English etc. We explore why systematic reviewing techniques tend to collect studies with clear empirical evidence, but that lack theoretical and methodological coherence. We argue that the solution to these limitations is for researchers to reassert their own critical and methodological expertise when selecting and analysing studies. It is the expertise of the researcher that is minimised in systematic reviewing, and this expertise, we believe, is essential to evaluating the literature rigorously.