The objective structured public health examination: A study of reliability using multi-level analysis

Davison, Ian, Cooper, Rob and Bullock, Alison Deborah ORCID: https://orcid.org/0000-0003-3800-2186 2010. The objective structured public health examination: A study of reliability using multi-level analysis. Medical Teacher 32 (7) , pp. 582-585. 10.3109/01421590903514622

Full text not available from this repository.

Abstract

Background: Introduced in 2006, the objective structured public health examination (OSPHE) is a part of the examination for the UK Membership of the Faculty of Public Health. Designed to simulate real work in public health, the same five generic competencies were assessed simultaneously in each of six stations in 1 h 48 min. This study estimates the examination reliability of the OSPHE. Methods: Data from all 198 candidates, 54 questions, 39 examiners over 9 test days in 2006 were analysed for reliability and sources of error using multilevel cross-classified analysis and generalisability theory. Results: A reliability co-efficient of 0.85 was estimated (95% confidence interval 0.82–0.88). Little variance was accounted for by questions or examiners. Twelve questions would increase reliability to 0.92. More than one examiner per question has negligible impact on reliability. Conclusion: Multilevel modelling estimates examination reliability which is more rigorous than Cronbach's alpha. Traditional generalisability methods based on the analysis of variance (ANOVA) are unable to do this. The reliability of this examination compares favourably with the literature despite having just six test stations and under 2 h per candidate. Other specialties could consider the use of parallel testing of the same key competencies at all stations within simulated workplace events.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Cardiff Unit for Research and Evaluation in Medical and Dental Education (CUREMeDE) Social Sciences (Includes Criminology and Education)
Subjects:	H Social Sciences > H Social Sciences (General) R Medicine > R Medicine (General)
Publisher:	Informa Healthcare
ISSN:	0142-159X
Last Modified:	20 Oct 2022 08:22
URI:	https://orca.cardiff.ac.uk/id/eprint/28115

Citation Data

Cited 2 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item