Turn of Events: Academic events as a platform for preregistration

Prereg posters are conference posters that present planned scientific projects. We provide empirical evidence for their value, and establish prereg posters as means to receive valuable feedback, promote Open Science, and support Early Career Researchers.


Introduction
In recent years, it has been repeatedly shown that the results of many scientific studies fail to replicate (Baker, 2015;Munafo, 2017a;Munafo et al., 2017b). The growing awareness of this problem has prompted several attempts to tackle it. Foremost among these is study preregistration (Nosek et al., 2017), in which researchers commit to specific data collection and analysis procedures prior to knowledge of the results. Preregistration helps to prevent the statistical biases that can arise from post hoc inferences, while also providing room for transparent exploratory analyses.
Researchers can preregister their hypotheses, study design and analyses on various online platforms such as the Open Science Framework (https://osf.io/prereg/), though these are not normally peerreviewed. An alternative is to submit a registered report to a journal, which is peer-reviewed and has the opportunity to be revised, before any data are collected (Chambers, 2013;Chambers et al., 2015;Munafo, 2017;COS Team, 2013). Not only does the peer-review potentially improve the study design and analyses pipeline, but if accepted, the journal is committed to publishing the results, whatever the outcome, which reduces the "file-drawer" problem in which many null results are never published (Head et al., 2015). Registered reports, first offered by the journal Cortex, are now supported by over 200 peer-reviewed journals, including Nature Human Behaviour.
Recently, we proposed a third possible avenue for preregistration, namely posters at academic conferences (Tibon et al., 2018) -so-called "prereg posters". We suggested that conferences allow prereg posters that present planned work as means to promote preregistration and stimulate discussion throughout the entire lifecycle of the scientific work. We argued that this would allow researchers to receive feedback on early stages of their projects, when such feedback is arguably most valuable (that is, before the protocol is finalised and data are collected). Moreover, colleagues with shared scientific interests would become aware of the study early on, which can open the door to 2 collaborations. We further argued that authors who submit prereg posters would be more likely to use these as the basis of a registered report (amended following conference feedback).
Prereg posters were recently adopted by the BNA2019 Festival of Neuroscience covering a diverse range of neuroscience topics and disciplines, and the overall impression was that they were enthusiastically welcomed. The survey data were collected at two time points (before and after the event), via two different surveys. The full set of questions and response data are available here: https://bit.ly/36CkfLG. Below, we analyse data from the questionnaire items that address three main themes: prereg posters as means to (1) receive valuable feedback, (2) promote Open Science, and (3) support Early Career Researchers (ECRs).

Methods
The first survey (pre-conference survey) was administered three months prior to the event, and offered to all 445 participants whose poster was accepted. Of the 200 who responded, 151 were presenters of traditional posters and 49 of prereg posters. The second survey (post-conference survey) was administered one month after the event, and was completed by 97 participants, 66 of whom were presenters of traditional posters, and 29 of prereg posters. For analysis of ordinal data, we used a two sided two-sample Wilcoxon Rank Sum test (where W refers to the sum of ranks for a sample) and a significance (alpha) level of .05.

Results and Discussion
Theme 1: Prereg posters as means for getting valuable feedback. As a presenter, the motivation to present a prereg poster is clear: it allows one to get feedback on work at early stages. Indeed, 33% of the 123 responses to the pre-conference question "why did you submit a prereg abstract instead of a traditional abstract" indicated that this choice was made in order to get feedback (either in general or on specific analyses) before completing the research. As shown in Figure 1, the groups differed in the type of feedback that they subsequently received at the conference. Not surprisingly, presenters of traditional posters mostly received feedback regarding future projects, whereas presenters of prereg posters mostly received feedback regarding methods and experimental design. [scale: 1 -"highly unlikely" to 7 -"highly likely"], IQR = 2, W = 944; traditional: Median = 6, IQR = 2, W = 970; p = .91). This is even though projects presented as prereg posters were at their early stages, and in most cases were still at (or prior to) the data collection phase. This might reflect the presenters' trust in the preregistration system, where publication does not depend on results. But it might also suggest that presenters of prereg posters felt that they have received valuable feedback that would promote their ability to publish their work later on.
One potential objection to prereg posters is that conference attendees may not bother to discuss "half-baked" work and therefore there is no point in "wasting" conference space on posters that would not attract visitors. To test this, we analysed responses to two questions in the post-conference survey: (1) did you receive feedback on your work during the poster session, and (2)

Conclusions
We provide empirical evidence that prereg posters can be a useful tool in promoting academic discussion of planned and on-going research, supporting Open Science and benefiting Early Career Researchers. We hope this encourages their adoption by an increasing number of future scientific conferences. As discussed in Tibon et al (2018), and as with any new initiative, we believe it is vital to instruct all the people involved in conferences (organisers, presenters, poster reviewers and attendees) about the aim and value of these posters, otherwise the initiative might flounder (e.g, if reviewers score prereg posters lower owing to their lack of results, because the reviewers were not properly informed of the aims). Additionally, at least until they become more customary, we recommend highlighting prereg posters in the conference program and at the stand (e.g., by Open Science badges, as done at BNA2019). Further practical advice and useful tips can be found on the BNA website: https://www.bna.org.uk/mediacentre/news/pre-reg-posters/. Importantly, our results demonstrate that conferences and academic events should not only support specific scientific topics, but also act as venues that increase the quality of scientific research.