Using examiner reports to identify quality in PhD theses

Year: 2005

Author: Bourke, Sid, Holbrook, Allyson, Lovat, Terence

Type of paper: Refereed paper

Judgements about quality in doctoral research have two, sometimes competing, criteria to consider: the quality of the research training received and the quality of the research output, that is the thesis. Research reported in PhD theses represents a major component of all research undertaken in Australia. In Education, as many candidates are also recently or currently practising teachers and administrators, PhD research often has an immediate impact on schools and schooling. Thus the quality of theses is important as a contribution to our national research profile and professional practice.

A national study in progress has collected candidature information and examiner reports for 804 PhD candidates at eight universities across all discipline areas who submitted a thesis for examination – of these, 84 theses were in the field of Education. Currently the texts of examiner reports have been analysed for 401 candidates at four of the eight universities.

Evaluative comment (summative and other judgement) on each of the assessable areas covered that could be clearly recognised as positive (on average 28% of each examiner report) or negative (4%), have been analysed. The focus in this paper is on theses identified by examiners at both ends of the quality scale – high quality theses and marginal theses, first across all discipline areas and subsequently for theses in Education. The evaluative comments are used to illustrate what examiners write in their reports when assessing high quality and marginal theses. Suggestions for using examiner comment categories to distinguish thesis quality are made and discussed.