Inter-rater reliability of functional MRI data quality control assessments: A standardised protocol and practical guide using pyfMRIqc

Williams, Brendan, Hedger, Nicholas, McNabb, Carolyn B., Rossetti, Gabriella M. K. and Christakou, Anastasia 2023. Inter-rater reliability of functional MRI data quality control assessments: A standardised protocol and practical guide using pyfMRIqc. Frontiers in Neuroscience 17 , 1070413. 10.3389/fnins.2023.1070413

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (6MB) | Preview

Abstract

Quality control is a critical step in the processing and analysis of functional magnetic resonance imaging data. Its purpose is to remove problematic data that could otherwise lead to downstream errors in the analysis and reporting of results. The manual inspection of data can be a laborious and error-prone process that is susceptible to human error. The development of automated tools aims to mitigate these issues. One such tool is pyfMRIqc, which we previously developed as a user-friendly method for assessing data quality. Yet, these methods still generate output that requires subjective interpretations about whether the quality of a given dataset meets an acceptable standard for further analysis. Here we present a quality control protocol using pyfMRIqc and assess the inter-rater reliability of four independent raters using this protocol for data from the fMRI Open QC project (https://osf.io/qaesm/). Data were classified by raters as either “include,” “uncertain,” or “exclude.” There was moderate to substantial agreement between raters for “include” and “exclude,” but little to no agreement for “uncertain.” In most cases only a single rater used the “uncertain” classification for a given participant’s data, with the remaining raters showing agreement for “include”/“exclude” decisions in all but one case. We suggest several approaches to increase rater agreement and reduce disagreement for “uncertain” cases, aiding classification consistency.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Psychology Cardiff University Brain Research Imaging Centre (CUBRIC)
Publisher:	Frontiers Media
ISSN:	1662-453X
Date of First Compliant Deposit:	31 January 2023
Date of Acceptance:	11 January 2023
Last Modified:	03 May 2023 08:07
URI:	https://orca.cardiff.ac.uk/id/eprint/156395

Actions (repository staff only)

Edit Item