Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

An investigation into the optimal number of distractors in single-best answer exams

Kilgour, James M. ORCID: https://orcid.org/0000-0001-8067-2306 and Tayyaba, Saadia 2016. An investigation into the optimal number of distractors in single-best answer exams. Advances in Health Sciences Education 21 (3) , pp. 571-585. 10.1007/s10459-015-9652-7

[thumbnail of AHSE Published Edition.pdf]
Preview
PDF - Published Version
Available under License Creative Commons Attribution.

Download (811kB) | Preview

Abstract

In UK medical schools, five-option single-best answer (SBA) questions are the most widely accepted format of summative knowledge assessment. However, writing SBA questions with four effective incorrect options is difficult and time consuming, and consequently, many SBAs contain a high frequency of implausible distractors. Previous research has suggested that fewer than five-options could hence be used for assessment, without deterioration in quality. Despite an existing body of empirical research in this area however, evidence from undergraduate medical education is sparse. The study investigated the frequency of non-functioning distractors in a sample of 480 summative SBA questions at Cardiff University. Distractor functionality was analysed, and then various question models were tested to investigate the impact of reducing the number of distractors per question on examination difficulty, reliability, discrimination and pass rates. A survey questionnaire was additionally administered to 108 students (33 % response rate) to gain insight into their perceptions of these models. The simulation of various exam models revealed that, for four and three-option SBA models, pass rates, reliability, and mean item discrimination remained relatively constant. The average percentage mark however consistently increased by 1–3 % with the four and three-option models, respectively. The questionnaire survey revealed that the student body had mixed views towards the proposed format change. This study is one of the first to comprehensively investigate distractor performance in SBA examinations in undergraduate medical education. It provides evidence to suggest that using three-option SBA questions would maximise efficiency whilst maintaining, or possibly improving, psychometric quality, through allowing a greater number of questions per exam paper.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Medicine
Subjects: L Education > L Education (General)
L Education > LB Theory and practice of education > LB2300 Higher Education
R Medicine > R Medicine (General)
Uncontrolled Keywords: Assessment; Testing; Written examination; Examination reliability; Examination quality; Single-best answer exams; Undergraduate medical education
Publisher: Springer Verlag
ISSN: 1382-4996
Date of First Compliant Deposit: 30 March 2016
Date of Acceptance: 12 November 2015
Last Modified: 06 May 2023 04:51
URI: https://orca.cardiff.ac.uk/id/eprint/82022

Citation Data

Cited 16 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics