A Bayesian bird's eye view of 'replications of important results in social psychology'

Marsman, Maarten, Shonbrodt, Felix D., Morey, Richard D. ORCID: https://orcid.org/0000-0001-9220-3179, Yao, Yuling, Gelman, Andrew and Wagenmakers, Eric-Jan 2017. A Bayesian bird's eye view of 'replications of important results in social psychology'. Royal Society Open Science 4 (1) , 160426. 10.1098/rsos.160426

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (500kB) | Preview

Abstract

We applied three Bayesian methods to reanalyze the preregistered contributions to the Social Psychology special issue “Replications of Important Results in Social Psychology' [1]. First, individual-experiment Bayesian parameter estimation revealed that for directed effect size measures, only 3 out of 44 central 95% credible intervals did not overlap with zero and fell in the expected direction. For undirected effect size measures, only 4 out of 59 credible intervals contained values greater than :10 (10% of variance explained), and only 19 intervals contained values larger than :05. Second, a Bayesian random-effects meta-analysis for all 38 t-tests showed that only one out of the 38 hierarchically estimated credible intervals did not overlap with zero and fell in the expected direction. Third, a Bayes factor hypothesis test was used to quantify the evidence for the null hypothesis against a default one-sided alternative. Only 7 out of 60 Bayes factors indicated non-anecdotal support in favor of the alternative hypothesis (BF10 > 3), whereas 51 Bayes factors indicated at least some support for the null hypothesis. We hope that future analyses of replication success will embrace a more inclusive statistical approach by adopting a wider range of complementary techniques.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Psychology
Uncontrolled Keywords:	preregistration; evidence; reproducibility; credible interval; Bayes factor
Additional Information:	Published by the Royal Society under the terms of the Creative Commons Attribution License.
Publisher:	Royal Society, The
ISSN:	2054-5703
Date of First Compliant Deposit:	12 December 2016
Date of Acceptance:	12 December 2016
Last Modified:	05 May 2023 08:15
URI:	https://orca.cardiff.ac.uk/id/eprint/96854

Citation Data

Cited 23 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item