Can a reflective rubric be applied consistently with raters globally? A study across three countries

Lucas, Cherie, Smith, Lorraine, Lonie, John M., Hough, Michael, Rogers, Kris and Mantzourani, Efi ORCID: https://orcid.org/0000-0002-6313-1409 2019. Can a reflective rubric be applied consistently with raters globally? A study across three countries. Currents in Pharmacy Teaching and Learning 11 (10) , pp. 987-994. 10.1016/j.cptl.2019.06.004

Preview

PDF - Accepted Post-Print Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (493kB) | Preview

Abstract

Introduction Reflection is a powerful tool for assisting students to develop the skills to make better informed decisions. As a pharmacy competency standard, reliable and fair assessment strategies are required to measure reflective skills and support students in developing their reflective capacity. The aim of this research was to explore whether we can extend the applicability of a previously tested rubric to a range of educational settings, to account for diversity of pharmacy educators and curricula internationally. Methods Four raters from three countries applied a reflective rubric to assess a sample (n = 43) of reflective accounts, representing 41% of a cohort of 105 second-year undergraduate pharmacy students. The interrater reliability (IRR) was measured utilizing the intra-class correlation coefficient (ICC), using a two-way random effects model with absolute agreement, to determine the level of agreement between the raters' absolute scores. Generalizability Theory analysis was used to estimate generalizability of raters and stages. Results Results indicated agreement of raters for (i) each of the seven stages of reflection and (ii) overall score for the reflective account, with moderate to substantial agreement (ICC = 0.55–0.69, p < 0.001); and high agreement for all raters for the overall score (ICC = 0.96, p < 0.001), respectively. The G-Study estimated a relative error coefficient of 0.78. Conclusion This additional analysis further confirms the reliability and applicability of the rubric to a range of rater academic backgrounds.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Pharmacy
Publisher:	Elsevier
ISSN:	1877-1297
Date of First Compliant Deposit:	12 August 2019
Date of Acceptance:	4 June 2019
Last Modified:	29 Nov 2024 07:45
URI:	https://orca.cardiff.ac.uk/id/eprint/124892

Citation Data

Cited 6 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item