Determining the influence of video-based benchmarking (VBB) on examiner variability in objective structured clinical exams (OSCE): The Align study

Yeates, Peter, Edwards, Rebecca Jane, Narain, Aditya, McInley, Robert, Lefroy, Janet, McCray, Gareth, Roberts, Giles, Hammond, Ellie, McBain, Stu, Blythe, Andrew, Cullen, Kathy, Napier, Craig, Sims, Laura, Thampy, Harish, Vince, Tushar, Ensaff, Sue, Goodfellow, Rhian, Harrison, Christopher, Chung, Ching-Wa, Capey, Steven, Roberts, Chris and Vallender, Rebecca 2026. Determining the influence of video-based benchmarking (VBB) on examiner variability in objective structured clinical exams (OSCE): The Align study. Medical Teacher 10.1080/0142159x.2026.2631743

PDF - Published Version Available under License Creative Commons Attribution. Download (2MB)

Abstract

Introduction Reducing examiner variability in Objective Structured Clinical Exams (OSCEs) is a priority within clinical performance assessment. In contrast to typical OSCE examiner training, video-based benchmarking (VBB) involves examiners scoring videos a/from their specific station b/shortly before the OSCE and then reflecting on and discussing scores/justifications agreed by an expert panel. Whilst realist evaluation has described mechanisms and contexts by which VBB may operate, VBB’s overall efficacy is unknown. Methods We performed a multi-centre (12 UK medical schools) stratified randomised controlled trial of VBB versus control to determine the influence of VBB on examiners’ score variability and other score characteristics. Secondarily, we compared the average scores allocated by examiners from different schools. Results 171 medically qualified, trained OSCE examiners participated in the study. VBB showed no significant effect on overall examiner variability. In pre-specified analyses, VBB reduced variability from group mean of initially ‘outlying’ examiners on the borderline performance (VBB mean variability 3.02 out of 27 (IQR1.98-4.98), control 4.70 (3.91–5.70), p < 0.016) and made examiners more likely to correctly fail a minimally failing performance (p < 0.03, OR = 2.133[95% CI 1.081–4.208]). VBB caused a small increase in confidence. There were no significant differences in average scores by school. Conclusions VBB may enhance trust in OSCEs through more accurate classification of borderline performances and aligning outlying examiners scoring.

Item Type:	Article
Date Type:	Published Online
Status:	Published
Schools:	Schools > Medicine
Publisher:	Informa UK Limited
ISSN:	0142-159X
Date of First Compliant Deposit:	9 March 2026
Date of Acceptance:	9 February 2026
Last Modified:	09 Mar 2026 14:00
URI:	https://orca.cardiff.ac.uk/id/eprint/185614

Actions (repository staff only)

Edit Item