Interdisciplinary Open Science in Social and Behavioral Research

MIDAS and ICPSR Mini-Symposium

Public Presentations

February 22, 2024

Faculty and Researcher Workshops

February 23, 2024

Program-at-a-Glance

Day One: Thursday, February 22, 2024

Dr. Jagadish

9:00 am: Opening remarks

Dr. H. V. Jagadish, Edgar F Codd Distinguished University Professor of Electrical Engineering and Computer Science, Bernard A Galler Collegiate Professor of Electrical Engineering and Computer Science, Professor of Electrical Engineering and Computer Science, College of Engineering; Director of the Michigan Institute for Data Science, University of Michigan

A.J. Millon, Director, National Archive of Criminal Justice Data; Research Investigator, Institute for Social Research, University of Michigan

9:10 am: Dr. Wade Bishop, Professor, School of Information Sciences, University of Tennessee – Knoxville

“Data Management Plan Evaluations, Data Discovery, and Accessibility of Collections”

Open data undergirds open science. Data reuse improves and advances science permitting others to verify results; enabling the repetition of experiments; and to conduct new research. Before data may be discovered and potentially reused in science, open data passes through several stages of a data life cycle outlined in a formal document—a Data Management Plan (DMP). With data management and sharing plans required by many funding agencies, their importance has grown as a roadmap to where and how data may be discovered and accessed. This presentation will share recent research findings on DMP evaluation, scientists’ data discovery behaviors, and access issues to physical collections. The audience will gain an understanding of the options and considerations for the implementation open data with end reusers in-mind through case studies across a variety of domains.

Research Overview

Wade Bishop is a Professor in the School of Information Sciences at the University of Tennessee-Knoxville. He is the Director of Graduate Studies as well as the Research Data Management Certificate Coordinator. His research focus is on Research Data Management, Data Discovery, Geographic Information Science, as well as the study of data occupations, education, and training. He has other research expertise that includes physical access for users to U.S. public libraries (using Geographic information Systems (GIS)) and the evaluation of many other services and resources in academic and public libraries. Bishop has an M.L.I.S. from the University of South Florida School of Information and a Ph.D. from Florida State University’s School of Information. He serves as Associate Editor for Telematics and Informatics and on several other editorial boards.

10:00 am: Dr. Jessica Kay Flake, Assistant Professor, Department of Psychology, McGill University

“The role of measurement and modeling in the reproducibility and replication of social science research”

Psychology’s ‘replication crisis’ has prompted a methodological reform movement. How we do our research is changing: open science, big team science, and concerns over questionable research practices are here to stay. First, I’ll discuss how measurement plays a fundamental role in the quality and replicability of social sciences. I will share my research on measurement practices in original and replication studies and discuss prevalent challenges. Then I’ll present my efforts to conduct large-scale and reproducible measurement research as a part of big team science initiatives and what I’ve learned about which open science practices need further development. I will close with next steps for methodological development, particularly the need to develop practices for creating preregistered and reproducible analysis pipelines.  

Research Overview

JK Flake received a BS in Psychology from Northern Kentucky University in 2010, an MA in Quantitative Psychology from James Madison University in 2012, and a PhD in Measurement, Evaluation, and Assessment from the University of Connecticut in 2015. From 2015 to 2018 she worked as a postdoctoral researcher in quantitative psychology at York University and educational psychology at the University of Virginia. In 2018 she started her lab in the Quantitative Psychology area of the Department of Psychology at McGill University. JK Flake is also the Assistant Director for Methods at the Psychological Science Accelerator, and a member of the Technical Advisory Panel of the Enrollment Management Association. Her research interests include the development and application of latent variable models for use in educational and social psychological research and the improvement of measurement practices in psychology more broadly.

10:50 - 11:00 am
*Break*

11:00 am: Dr. Ivan Oransky, Executive Director, Center for Scientific Integrity, Co-founder, Retraction Watch; Editor-in-chief, Spectrum; and Distinguished Journalist-in-Residence, New York University’s Arthur Carter Journalism Institute

“Retractions: On The Rise, But Not Enough”

In 2000, there were about 40 retractions from the scholarly literature. In 2023, there were more than 10,000. That is a dramatic increase, even accounting for the growing number of papers published per year. In this talk, I will explore the reasons for the increase, why it is good news, and why the real number should be even higher — including in the social and behavioral sciences. I will tell the stories of the sleuths who are finding problems in the literature, drawing on more than a decade of experience at Retraction Watch.

Research Overview

Ivan Oransky, MD, the executive director of the Center For Scientific Integrity, is co-founder of Retraction Watch, editor in chief of Spectrum, and distinguished writer in residence at New York University’s Arthur Carter Journalism Institute. Ivan previously was president of the Association of Health Care Journalists and vice president of editorial at Medscape. He has also held editorial leadership positions at MedPage Today, Reuters Health, Scientific American and The Scientist. He is the recipient of the 2015 John P. McGovern Medal for excellence in biomedical communication from the American Medical Writers Association, and in 2017 was awarded an honorary doctorate in civil laws from The University of the South (Sewanee). In 2019, the judges for the John Maddox Prize, which promotes those who stand up for science in the face of hostility, gave him a commendation for his work at Retraction Watch.

1:00 pm: Dr. David Mellor, Director of Policy, Center for Open Science

“Improving Research Culture with Open Science”

Open Science is a movement to bring more transparency to the process of science and its outcomes in order to increase focus on rigorous methods over surprising findings. The Center for Open Science exists to support this change through a holistic strategy to build infrastructure to enable new practices, programs to train and normalize new practices, and policies to incentivize or require them. In this talk, David Mellor, Director of Policy at COS will cover this strategy by pointing to the evidence that there is a need for reform, ways to implement reform, and specific practices that researchers can adopt. These practices will include data sharing, preprinting, and preregistration.

Research Overview

David Mellor leads the Policy and Incentives programs at the Center for Open Science (COS), which involves both grassroots and institutional efforts to include open science in research. These require implementing the Transparency and Openness Promotion (TOP) Guidelines, preregistration, Registered Reports, and signaling adherence to best practices with badges.

Mellor received his PhD in Ecology and Evolution from Rutgers University. As Director of Advising in the Division of Life Sciences at Rutgers University, he supported undergraduate research and education. At Virginia Tech, Mellor investigated the relationships between land managers and citizen scientists contributing to authentic scientific knowledge and land use decisions. His research interests cover the behavioral ecology of cichlid fish, citizen science, and reproducibility.

1:50 pm: Dr. Christian Zimmerman, Assistant Vice President of Research Information Services, Federal Reserve Bank of St. Louis

“The pre-print culture in economics: How RePEc helps making science more open”

Since 1997, the volunteer-driven RePEc initiative facilitates the dissemination of economic research, and in particular pre-prints. The mission is to democratize the access to frontier research for the authors and the readers. This talk will discuss the pre-print culture in economics, how popular they are, and what RePEc does to sustain their centrality. We conclude with some of the challenges that we face.

Research Overview

Christian Zimmermann is Assistant Vice President at the Federal Reserve Bank of St. Louis. He holds an undergraduate degree from the Université de Lausanne and a PhD from Carnegie Mellon University. He has previously worked at the Université du Québec à Montréal, the Bank of Canada and the University of Connecticut. He has published articles in the International Economic Review, Journal of Monetary Economics, European Economic Review, Journal of Population Economics, Journal of Comparative Economics, Journal of Macroeconomics, and others. He is associate editor of the Review of Economic Dynamics and a core member of the RePEc team. His current research relates to optimal unemployment insurance, fluctuations and child labor, among others.

2:40 – 3:30 pm: panel discussion 

Day Two: Friday, February 23, 2024

Faculty and Researcher Workshops

9:00 am – 2:00 pm: we invite U-M researchers to join us for day two and participate in a workshop, which will include research discussions on specific themes and 1:1’s with the speakers.

Organizer

A.J. Millon, Director, National Archive of Criminal Justice Data; Research Investigator, Institute for Social Research, University of Michigan