The Ideal of Preregistration and What Warrants it

The Journal Club: “ReproducibiliTea” at LMU Munich continues with its 13th meeting. The organization team is happy to  welcome Aaron Peikert (Max-Planck-Institute Berlin) as our guest who will give the introductory presentation and participate in the open discussion afterwards.

The meeting is dedicated to the topic “The Ideal of Preregistration and what warrants it”. Aaron Peikert has recently uploaded a draft for a paper on “Why does preregistration increase the persuasiveness of evidence? A Bayesian rationalization”. During our meeting, he will present some of his ideas from this draft and take up further debates around the issue of preregistration. Below you will find a short abstract that provides more information on the topic of the presentation.

We prepared the following key questions to guide you through the discussion and also provide some guidance while reading the paper. The presentation will take up some of the ideas from the paper, but present them in a more “discussion-friendly” way and go beyond the topic. For each question, you can think about the suggestions from the paper, but also bring in your own ideas:

• What are objections to preregistration?
• To what extent can incomplete preregistrations (e.g. only regarding study variables, but without an exact analysis plan) be useful?
• What is the actual purpose of preregistration?

You can find a short abstract for the introductory talk by Aaron Peikert below:

The Ideal of Preregistration and what warrants it

The replication crisis spurned many researchers to preregister their analyses before acquiring data. However, there is no agreement on what preregistration should accomplish and why it is uniquely suited to these goals. A widespread view is that preregistration should limit how much the data may influence the hypotheses tested on the same data (i.e., restrict researchers’ degrees of freedom)). If no such influence occurs, an analysis is generally considered confirmatory. Consequently, many researchers believe that only confirmatory studies benefit from preregistration.

In this presentation, Aaron Peikert argues that limiting preregistration to confirmatory research is unnecessarily restrictive. To that end, he will suggest how to formalize the objective of preregistration and aim to demonstrate that exploratory studies also benefit from the practice. Lastly, implications for the practice of preregistration will be discussed.

About the speaker:

Aaron Peikert is a PhD student at the Max-Planck-Institute for Human Development in Berlin and has been an Open Science enthusiast since his bachelor’s degree. His work focuses on preregistration, reproducible workflows, structural equation models and machine learning approaches for nested/ hierarchical data.

Concept of the Journal Club

The interdisciplinary journal club “ReproducibiliTea” is launched every 3-4 weeks and addresses main questions and current topics concerning Open Science and the credibility of research findings. The idea - originated from Great Britain - was already developed in many other countries using the same name. During on-site meetings tea, juice and small snacks are provided to live up to the slogan. However, some meetings are held online due to organizational reasons. No matter where the sessions take place, everybody is welcome to bring their own food and drinks and thus connect the journal club with their lunch break.

The Journal Club consists of open talk sessions for discussing Open Science, reproducibility, and other related issues and is explicitly open to all disciplines and institutions to create the most interesting interchange. Each session starts with a short talk about the paper and then continues with an open discussion in which you can contribute your own thoughts and questions.

During a longer break between 2020 and 2021, the concept was slightly revised, and the organization team decided to invite the speaker(s) of the respective paper that is discussed in each session. The author herself/himself starts with the introductory talk including a short summary of the paper. Afterwards, there is enough time for direct interaction with the author(s) and, as before, an open discussion with all of the participants.

You are warmly welcome to join the next meetings. You can simply send an e-mail to reproducibilitea.munich@gmail.com to join the mailing list to get information about the next meetings and chosen papers. For each session it would be helpful to have read the announced paper, but there is no obligation to do so.

The organization team is looking forward to seeing you at one of the next Journal Club sessions.

organization team:

• Laura Goetz, Medical Student at LMU Munich
• Stephan Nuding, Psychology Student at LMU Munich
• Leonhard Schramm, Psychology Student at LMU Munich
• Franka Etzel, Psychology Student at LMU Munich
• Viktoria Wiegelmann, Psychology Student at LMU Munich
• Gracia Pruem, Psychology Student at LMU Munich
• Alp Kaan Aksu, Psychology Student at LMU Munich
• Finia Loeb, Psychology Student at LMU Munich

Presenters

• Aaron Peikert