Evidence for prereg posters as a platform for preregistration

Prereg posters are conference posters that present planned scientific projects. We provide preliminary evidence for their value in receiving constructive feedback, promoting open science and supporting early-career researchers.

I n recent years, it has been repeatedly shown that the results of many scientific studies fail to replicate [1][2][3] . The growing awareness of this problem has prompted several attempts to tackle it. One way is to submit a Registered Report to a journal, which is peer-reviewed and has the opportunity to be revised before any data are collected [3][4][5][6] . Not only does the peer-review potentially improve the study design and analyses, but once accepted, the design and analyses are 'locked' , preventing researchers from adjusting their hypotheses or analyses post hoc, i.e., after having seen the data (reporting such additional analyses is allowed, but only when transparently labelled as 'exploratory'). Moreover, the journal is committed to publishing the results, whatever the outcome, which reduces the 'file-drawer' problem of many null results not being published 7 . Registered Reports, first offered by the journal Cortex, are now supported by over 200 peer-reviewed journals, including Nature Human Behaviour.
Another approach is to preregister studies on a publically accessible platform such as the Open Science Framework 6,8 . Though they can be locked, such preregistered studies are not normally peer-reviewed before data collection, and they are not binding for any subsequent publications. Indeed, substantial deviations from preregistered plans have been observed 9 . Two likely reasons for the large number of deviations are immature preregistration (for example, of a protocol that was not yet finalised) and acceptance of post-preregistration feedback (for example, following presentation of data at a conference).
Recently, we proposed another possible avenue to help preregistration, namely presenting posters at academic conferences about planned research, before data are collected 10 ; called 'prereg posters' . This allows presenters to receive feedback on their hypotheses, design and analyses from their colleagues (conference attendees), which is likely to improve the study. In turn, this can improve more formal preregistration, reducing the chances of subsequent deviation, and/or facilitate submission of the work as a Registered Report. Moreover, colleagues with shared scientific interests become aware of the study early on, which can open the door to collaborations.
Nearly a fifth (100 of 491) of all submitted posters at the BNA2019 Festival conformed to the new prereg format, covering a diverse range of neuroscience topics and disciplines, and the overall impression was that they were enthusiastically welcomed. The informal survey data were collected at two time points (before and after the event), via two different online surveys. The full set of questions and response data are available here: https:// osf.io/3h6w8/. The first informal survey (pre-conference survey) was administered 3 months before the event and offered to all 445 participants whose poster was accepted. Of the 200 who responded, 151 were presenters of traditional posters and 49 of prereg posters. The second informal survey (post-conference survey) was administered 1 month after the event and was completed by 95 participants, 66 of whom were presenters of traditional posters and 29 of prereg posters. Below, we present data from the questionnaire items that address three main themes: prereg posters as means to (i) receive valuable feedback, (ii) promote open science and (iii) support early-career researchers (ECRs).

Getting valuable feedback
As a presenter, the motivation to present a prereg poster is clear: it allows one to get feedback on work at early stages. Indeed, 33% of the 123 responses to the preconference question, "why did you submit a prereg abstract instead of a traditional abstract?" indicated that this choice was made to get feedback (either in general or comment on specific analyses) before completing the research. As shown in Fig. 1, the groups differed in the type of feedback that they subsequently received at the conference. Interestingly, presenters of traditional posters mostly received feedback regarding future projects, whereas presenters of prereg posters mostly received feedback regarding methods and experimental design. Furthermore, the groups gave similar estimations of the likelihood of the presented research being published in a peer-reviewed journal upon completion (prereg: median = 6 (scale: 1 'highly unlikely' to 7 'highly likely'), interquartile range (IQR) = 2; traditional: median = 6, IQR = 2). This is even though projects presented as prereg posters were at their early stages and in most cases were still before data collection. While it could be the case that researchers' estimation of publication likelihood remains stable throughout the course of a project, we speculate that it could also reflect the presenters' trust in the preregistration system, where publication does not depend on results, and/or the presenters' impression that they received valuable feedback that would promote their ability to publish the work later.
One potential objection to prereg posters is that conference attendees may not bother to discuss 'half-baked' work and therefore there is no point in 'wasting' conference space on posters that would not attract visitors. However, responses to two questions in the post-conference survey-"did you receive feedback on your work during the poster session?" and "how many people talked to you about your poster during the poster session?"-suggest that this is not the case: prereg posters did not receive less feedback than traditional posters (prereg = 75%; traditional = 68%), nor fewer visitors (prereg: median category = 2 (5-8 people), IQR = 1; traditional: median category = 2 (5-8 people), IQR = 1), suggesting that conference attendees do not avoid prereg posters, despite the fact that these posters present planned or preliminary work.

Promoting open science
Our second theme concerns attitudes among the participants towards open science in general, and preregistration in particular. Two questions from the postconference survey were indicative of such attitudes: "to what degree would you say that preregistration of work is necessary in today's neuroscience community?" and "to what degree did BNA2019 increase your awareness of preregistration?" In general, presenters of both prereg and traditional posters agreed that preregistration is necessary (prereg: median = 4 (scale: 1 'not at all' to 5 'incredibly necessary'), IQR = 1; traditional: median = 4, IQR = 2). Moreover, participants in both groups indicated that the event increased their awareness of preregistration, and the increment was somewhat greater for prereg poster presenters (prereg: median = 6 (scale: 1 'not at all' to 7 'completely'), IQR = 2; traditional: median = 5, IQR = 2.75). Furthermore, in the pre-conference survey, 61% of prereg posters submitters indicated that one reason they chose this form of presentation was to gain experience and confidence with preregistration and with Registered Reports. These results suggest that the attendees of BNA2019 acknowledged the importance of preregistration and that the event was helpful in promoting this understanding further, particularly for those presenting prereg posters.

Supporting early-career researchers
Although all scientists can benefit from their colleagues' comments, these can be particularly beneficial to ECRs who are still developing their networks and expertise in the field. Because on most occasions (especially for ECRs), travel funding is only available when work is presented, and given that ECRs often have fewer existing datasets to present compared to more senior colleagues, prereg posters can provide ECRs additional opportunities to attend academic events. Therefore, the opportunity to present a prereg poster may be especially compelling for ECRs. Data from the present informal surveys that are relevant to this question are shown in Fig. 2. While the data do not support strong claims regarding the relations between career stage and the choice to present a prereg poster, they do indicate that presenters of BNA2019 prereg posters had more limited research experience and had presented fewer posters throughout their career compared to traditional poster presenters.

Conclusions and outlook
The qualitative evidence reviewed above supports our prior claim that prereg posters can be a useful tool in promoting academic discussion of planned and on-going research, encouraging open science, and benefiting early career researchers. We hope this encourages the adoption of prereg posters by an increasing number of future scientific conferences. Notably, unlike some forms of study registration, whose main goal is to commit researchers to a registered plan, the main goal of prereg posters is to provide an opportunity for adjusting and improving studies before they are formally registered, following feedback from colleagues. Prereg posters can be routinely archived in a public repository, and then cited in any subsequent formal preregistrations that arise from the nascent protocol. This would capture the history and thus provenance of the idea generation. Indeed, it may be helpful to distinguish between 'unlocked preregistration' , which refers to research plans that are presented as posters or uploaded to a public website, but are still subject to change, and 'locked (pre)registration' , such as a registered report, which refers to a finalised research plan.
As we previously discussed 10 , and as with any new initiative, we believe it is vital to instruct all the people involved in conferences (organisers, presenters, poster reviewers and attendees) about the aim and value of these posters, otherwise the initiative might flounder (e.g., if reviewers score prereg posters lower owing to their lack of results because the reviewers were not properly informed of the aims). Additionally, at least until they become more customary, we recommend highlighting