Comparative analysis of Registered Reports and the standard research literature

O'Mahony, Aoife 2023. Comparative analysis of Registered Reports and the standard research literature. PhD Thesis, Cardiff University. Item availability restricted.

Preview	PDF - Accepted Post-Print Version Download (2MB) | Preview
	PDF (Cardiff University Electronic Theses and Dissertations Form - Memorandum of Understanding) - Supplemental Material Restricted to Repository staff only Download (162kB)

Abstract

The replication crisis revealed high levels of bias and questionable research practices (QRPs) in psychology research. Registered Reports (RRs) have been increasingly adopted as a possible solution, but there has been relatively little evidence of whether this novel publishing format appears to be working as intended to reduce bias and QRPs. This project sought to build a detailed database of RRs and closely matched standard reports (SRs), to investigate whether RRs perform better than SRs on indicators of quality, rigour, and transparency. 170 RRs were gathered, representing psychology, health, and related disciplines. Each RR was matched to 2 SRs and their characteristics were coded and compared between the two article types. Some brief descriptive analyses were also undertaken on a larger total sample of RRs (n = 359) that did not have a comparison sample, and a smaller sample of 12 RRs and 12 SRs was examined for signs of HARKing. Six key findings were observed. First, RRs exhibit lower rates of supported hypotheses and higher rates of unsupported hypotheses compared with SRs. Second, rates of open practices are higher among RRs than SRs. Third, RRs also appear to be more strongly associated with some methodological practices indicative of greater rigour and transparency. Fourth, author demographics and article citation rates revealed few differences between the article types. Fifth, higher citation rates were associated with more positive findings and fewer negative findings within both the SRs and RRs, but there was no statistically significant relationship between the journal impact factor and whether hypotheses were supported, for either article type. Finally, HARKing appeared to be non-existent in RRs, while some evidence of HARKing was observed in SRs, however, replication is needed. Overall, the evidence presented in this work demonstrates that, while there are still areas for improvement, RRs do appear to be working as intended in being associated with improved research practices, although further research is needed to determine causal impacts.

Item Type:	Thesis (PhD)
Date Type:	Completion
Status:	Unpublished
Schools:	Schools > Psychology
Date of First Compliant Deposit:	3 April 2024
Last Modified:	03 Apr 2024 11:31
URI:	https://orca.cardiff.ac.uk/id/eprint/167686

Actions (repository staff only)

Edit Item