The Reproducibility Showcase features a series of online presentations and tutorials from May to August, 2020. Presenters are selected from the MIDAS Reproducibility Challenge 2020.
A significant challenge across scientific fields is the reproducibility of research results, and third-party assessment of such reproducibility. The goal of the MIDAS Reproducibility Challenge is to highlight high-quality, reproducible work at the University of Michigan by collecting examples of best practices across diverse fields. We received a large number of entries that illustrate wonderful work in the following areas:
- Theory – A definition of reproducibility and what aspects of reproducibility are critical in a particular domain or in general.
- Reproducing a Particular Study – Comprehensive record of parameters and code that allows for others to reproduce the results in a particular project.
- Generalizable Tools – A general platform for coding or running analyses that standardizes the methods for reproducible results across studies.
- Robustness – Metadata, tools and processes to improve the robustness of results to variations in data, computational hardware and software, and human decisions.
- Assessments of Reproducibility – Methods to test the consistency of results from multiple projects, such as meta-analysis or the provision of parameters that can be compared across studies.
- Reproducibility under Constraints – Sharing code and/or data to reproduce results without violating privacy or other restrictions.
This showcase will lead up to our Reproducibility Day on Sept. 14, 2020.
MIDAS Reproducibility Challenge Showcase: Thomas Valley
Thomas Valley, MD, MSc Assistant Professor – Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, University of Michigan Establishing systematic practices to share statistical code View Recording Additional Speakers: Neil Kamdar (Lead Statistician, Institute for Healthcare Policy & Innovation, U-M) & Wyndy Wiitala (Research Health Scientist, US Dept. of Veteran’s Affairs) Abstract: Better communication of analytical approaches … Read more
MIDAS Reproducibility Challenge Showcase: Chris Brooks, Josh Gardner, Ryan Baker
Chris Brooks Assistant Professor of Information – School of Information, University of Michigan in Collaboration with Josh Gardner (University of Washington) and Ryan Baker (University of Pennsylvania) replicate.education: lessons learned building a platform for educational data science replications View Recording Abstract: This presentation will outline a program of reproducibility research in the domain of learning analytics with a specific … Read more
MIDAS Reproducibility Challenge Showcase: Sharon Glotzer – An Open Software Approach for Reproducible Research for Materials Design
Sharon Glotzer Professor – Chemical Engineering and Materials Science and Engineering, University of Michigan Dr. Joshua Anderson (Lead Research Specialist)*, Allen LaCour (PhD. Candidate)*, Dr. Tim Moore (Postdoctoral Researcher)*, Kelly Wang (PhD. Candidate)** *Chemical Engineering, Biointerfaces Institute, University of Michigan **Macromolecular Science and Engineering, BiointerfacesInstitute, University of Michigan An Open Software Approach for Reproducible Research … Read more
MIDAS Reproducibility Challenge Showcase: John Silberholz & Nigel Michki
View Full Recording What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO Iain Dunning (Hudson River Trading), Swati Gupta (Assistant Professor, Industrial and Systems Engineering, Georgia Tech), John Silberholz (Assistant Professor, Ross School of Business, University of Michigan) View John Silberholz Recording A Multi-informatic Cellular Visualization tool for interactively interrogating high-dimensional datasets Nigel … Read more
MIDAS Reproducibility Challenge Showcase: Xiaoquan (William) Wen & Yi Zhao – Quantify and Control Reproducibility in High-throughput Experiments
Xiaoquan (William) Wen* & Yi Zhao *Associate Professor – Biostatistics, University of Michigan & ** University of Michigan Alum – Biostatistics Quantify and Control Reproducibility in High-throughput Experiments View Recording Abstract: Despite the critical importance of reproducible research, the computational tools that aid quantifying and controlling reproducibility in high-throughput biological experiments are still lacking. In … Read more
MIDAS Reproducibility Challenge Showcase: Oliver He & Jie Song
Dr. Yongqun “Oliver” He & Jie Song Associate Professor – Microbiology and Immunology, University of Michigan Graduate Student, Computer Science and Engineering, University of Michigan View Recording (Full) View Recording (Oliver He) View Recording (Jie Song) Oliver He: XOD: The eXtensible ontology development (XOD) principles and tool implementation to support ontology interoperability and data reproducibility The … Read more
MIDAS Reproducibility Challenge Showcase: Jared Lyle & Jacob Fisher
Jared Lyle | Jacob Fisher Archivist, ICPSR, University of Michigan | Research Investigator, Survey Research Center, University of Michigan View Full Recording View Jared Lyle Recording View Jacob Fisher Recording Jared Lyle Co-Presenters: Lars Vilhuber (Executive Director, Labor Dynamics Institue, Cornell University), Maggie Levenstein (Director, ICPSR, University of Michigan), Chelsea Goforth (Data Project Manager, ICPSR, Univerisity of Michigan) Title: American Economic … Read more
MIDAS Reproducibility Challenge Showcase: Johann Gagnon-Bartsch – Everyday Reproducibility: A multi-pronged approach to ensure analyses are fully reproducible, easy to access, and easy to use
Johann Gagnon-Bartsch Assistant Professor – Statistics, University of Michigan View Recording Everyday Reproducibility: A multi-pronged approach to ensure analyses are fully reproducible, easy to access, and easy to use In creating a reproducible analysis, one often hopes to achieve several related but distinct goals — to ensure that the entirety of the analysis is fully … Read more