Reflection has been advocated both as a process for academics to examine thoroughly and gain insights into teaching that guide their actions. External sources such as student feedback and peer lens can mediate teachers’ reflection by encouraging the questioning of assumptions, thought processes, values, prejudices and habitual actions, and attitudes for understanding of practice and finding necessary strategies for improvement. This paper aims to discuss the challenges academics experienced in the process of peer observation of teaching (POT), part of a study examining the use of teacher-tailored student evaluation of teaching (TT-SET) augmented with POT in tertiary quality teaching for reflecting on and making changes to teaching. This study used a qualitative case study within a constructivist perspective with complementary data collection methods. Participants were eleven academics teaching business English at a university in Vietnam. Data were analysed inductively and thematically. Challenges were articulated during semi-structured interviews and the researcher’s journals, including limited learning from their junior peer, disagreement with feedback, lack of sensitivity in giving and receiving feedback, and limited time for POT and changes. These findings suggest that the nature of the peer relationship, academics skills for POT, and a supportive institutional environment appear to be key for the success of POT implementation.