Johann Gagnon-Bartsch
Assistant Professor – Statistics, University of Michigan
View Recording
Everyday Reproducibility: A multi-pronged approach to ensure analyses are fully reproducible, easy to access, and easy to use
In creating a reproducible analysis, one often hopes to achieve several related but distinct goals — to ensure that the entirety of the analysis is fully reproducible; to make the code easy to access and easy to understand; and to make the code portable and useful to other researchers. Each of these goals may be best achieved through different tools. I will discuss a multi-pronged approach that my research group and I have taken to achieve these goals, in the context of three separate research projects. Although our approach provides users with multiple modes of access to our analyses, it does not require substantial additional effort to implement, and we hope that our analyses can serve as a template for others.
—————————————————————
The Reproducibility Showcase features a series of online presentations and tutorials from May to August, 2020. Presenters are selected from the MIDAS Reproducibility Challenge 2020.
A significant challenge across scientific fields is the reproducibility of research results, and third-party assessment of such reproducibility. The goal of the MIDAS Reproducibility Challenge is to highlight high-quality, reproducible work at the University of Michigan by collecting examples of best practices across diverse fields. We received a large number of entries that illustrate wonderful work in the following areas:
- Theory – A definition of reproducibility and what aspects of reproducibility are critical in a particular domain or in general.
- Reproducing a Particular Study – Comprehensive record of parameters and code that allows for others to reproduce the results in a particular project.
- Generalizable Tools – A general platform for coding or running analyses that standardizes the methods for reproducible results across studies.
- Robustness – Metadata, tools and processes to improve the robustness of results to variations in data, computational hardware and software, and human decisions.
- Assessments of Reproducibility – Methods to test the consistency of results from multiple projects, such as meta-analysis or the provision of parameters that can be compared across studies.
- Reproducibility under Constraints – Sharing code and/or data to reproduce results without violating privacy or other restrictions.
On Sept. 14, 2020, MIDAS will also host a Reproducibility Day, which is a workshop on concepts and best practices of research reproducibility. Please save the date on your calendar.