Overview
At the “Implementing AI in Health” colloquium, a collaborative event presented by the Michigan Institute for Data Science and LHS Collaboratory, speakers shared their knowledge and expertise about the implementation of health AI and their prior, ongoing, and planned work in this area. This event featured four esteemed speakers: Dr. Barbara A. Barry, Dr. Michael J. Kim, Dr. Lisa S. Lehmann, and Dr. Michael Pencina.
Schedule
Welcome Remarks
9:00 AM
| (View Recording) |
| H.V. Jagadish – Edgar F Codd Distinguished University Professor of Electrical Engineering and Computer Science, Bernard A Galler Collegiate Professor of Electrical Engineering and Computer Science, Professor of Electrical Engineering and Computer Science, College of Engineering and Director of the Michigan Institute for Data Science |
| Karandeep Singh – Assistant Professor of Learning Health Sciences, Assistant Professor of Internal Medicine, Assistant Professor of Urology, Associate Chief Medical Information Officer of Artificial Intelligence, Medical School and Assistant Professor of Information, School of Information |
Keynote Talk #1: Lisa Lehmann
9:10 AM
| A Roadmap for Implementing Trustworthy AI in Healthcare (View Slides) (View Recording) |
| AI holds the promise of transforming healthcare by improving health outcomes for all people through improved diagnosis, predictions, and therapeutics. Identifying and addressing ethical concerns such as the safety and efficacy of AI systems, the quality of the data used to train and validate models, the risk of bias against marginalized groups, and how clinicians and patients interact with AI will be essential to realizing the benefits of AI in health care. I will identify ethical risks and potential mitigations for the use of AI in health care and provide a roadmap for the implementation of trustworthy AI. |
Break
10:55 AM
Keynote Talk #2: Michael Pencina
10:05 AM
| Evaluation and Governance of AI-based Clinical Decision Support tools in Health Systems (View Recording) |
| With the increased interest in applications of AI-based Clinical Decision Support tools in health systems, there is an urgent need to create an appropriate framework for their management across the lifecycle. Despite numerous best practices being put forth for evaluation of these tools, many developers rush through the process, resulting in creation of many algorithms of questionable quality. Furthermore, health systems struggle with establishing the right governance structures for identifying, implementing and managing AI-based Clinical Decisions Support tools. In this presentation we outline best practices for development, evaluation and monitoring of clinical algorithms and share our experience with establishing a governance structure for their implementation in Duke Health. |
Break
10:50 AM
Moderated Panel Discussion with Drs. Lehmann, Pencina, and Platt
10:55 AM
Networking Lunch
11:20 AM
Welcome Back
12:10 PM
| (View Recording) |
| Gretchen Piatt – Associate Chair, Department of Learning Health Sciences, Associate Professor of Learning Health Sciences, Medical School and Associate Professor of Health Behavior and Health Education, School of Public Health |
| Karandeep Singh – Assistant Professor of Learning Health Sciences, Assistant Professor of Internal Medicine, Assistant Professor of Urology, Associate Chief Medical Information Officer of Artificial Intelligence, Medical School and Assistant Professor of Information, School of Information |
Keynote Talk #3: Barbara Barry
12:15 PM
| Implementing AI in Health Care with a Human-in-the-Loop (View Slides) (View Recording) |
| As Artificial Intelligence (AI) and Machine Learning (ML) are introduced into heath care delivery, the phrase “human-in-the-loop” echos throughout implementation planning, regulatory guidance, and bioethics discussions. In this talk, I will explore what we mean by human-in-the-loop, where it succeeds and fails, and how human-AI interaction can inform adoption of AI/ML in health care. |
Break
1:00 PM
Keynote Talk #4: Michael Kim
1:10 PM
| Translating Trustworthy AI Frameworks into Practice at the US Department of Veterans Affairs (View Slides) (View Recording) |
| Leaders around the world are demanding transparent regulation and oversight of AI systems to protect end users. The United States government has already taken action through Executive Order 13960, which carries the force of law within the federal government to ensure that AI systems are trustworthy. Additional guidelines such as the Blueprint for an AI Bill of Rights and Executive Order 14091 further strengthen the U.S. government’s commitment to ensuring that AI systems are trustworthy and equitable. Please join Michael J. Kim, MD, Chief of Staff at the National Artificial Intelligence Institute (NAII), as he outlines Trustworthy AI policy, frameworks, and pilots from the U.S. Department of Veterans Affairs that demonstrate how the federal government’s regulation of AI is protecting our Veterans. |
Break
1:55 PM
Moderated Panel Discussion with Drs. Barry, Kim, and Waljee
2:00 PM
Closing Remarks
2:25 PM
| Gretchen Piatt – Associate Chair, Department of Learning Health Sciences, Associate Professor of Learning Health Sciences, Medical School and Associate Professor of Health Behavior and Health Education, School of Public Health |
Post-Event Networking and Refreshments Available
2:30 – 3:00 PM
About the Speakers
Barbara A. Barry
Assistant Professor of Medicine | Collaborative Scientist, Robert D. & Patricia E. Kern Center for the Science of Healthcare Delivery, & Mayo Clinic Alix School of Medicine
Barbara A. Barry, Ph.D., is a human-computer interaction (HCI) researcher who studies how interaction with artificial intelligence (AI) impacts human intelligence, communication and behavior. Dr. Barry has made fundamental contributions to AI, developed and evaluated intelligent interactive software agents, and collaborated on large-scale implementations of new technologies to improve health and education. Dr. Barry’s work focuses on how technology enables (or disables) human agency in health care and how to create productive, ethical and evidence-based use of AI within a learning health care system.
Michael J. Kim
Chief of Staff, National Artificial Intelligence Institute, U.S. Department of Veterans Affairs
Dr. Kim began his career with VA Long Beach Healthcare System in 2012 as the Section Chief of Neuroradiology and then served as Chief of Imaging Healthcare Group from 2017 to 2019.
Dr. Kim received his B.S. in Biochemistry and Molecular Biology from the University of California – Santa Cruz, his Medical Doctorate from the Feinberg School of Medicine at Northwestern University, Diagnostic Radiology Residency at the University of California – Irvine, and Neuroradiology Fellowship at the Keck School of Medicine at the University of Southern California. Dr. Kim holds an appointment with the School of Medicine at University of California – Irvine as an Associate Clinical Professor in the Department of Radiological Sciences.
Dr. Kim has held a variety of leadership positions ranging from Chief Resident, Chief Fellow, Section Chief, and Service Chief while also receiving a variety of awards including, Intern of the Year, Teacher of the Year, and Best Healthcare Group Chief. His research interests include the use of artificial intelligence and has a research grant with the Center of Artificial Intelligence in Diagnostic Medicine (CAIDM) at the School of Medicine at the University of California- Irvine. Dr. Kim is passionate in process improvement and has a Lean Six Sigma Black Belt in Healthcare Management.
Lisa S. Lehmann
Associate Professor of Medicine, Center for Bioethics, Harvard Medical School
Lisa Soleymani Lehmann is a primary care physician and bioethicist. She is Associate Professor of Medicine (part-time) at Harvard Medical School (HMS) and Associate Professor of Health Policy and Management (part-time) at the Harvard T. H. Chan School of Public Health. From 2020-2023 Lisa was Director of Bioethics at Google. She has also previously held positions as Chief Medical Officer (2017-2020) for the VA New England Healthcare System which is a multi-state integrated healthcare system comprised of 8 medical centers and 41 community-based outpatient clinics that serves over 260,000 Veterans. Prior to serving VA New England, Lisa was Executive Director of the VA National Center for Ethics in Health Care, Director of Bioethics at Brigham and Women’s Hospital, and a practicing primary care physician and health services researcher at Brigham and Women’s Hospital.
Michael Pencina
Vice Dean for Data Science and Director of Duke AI Health, Duke University School of Medicine
As Vice Dean of Data Science, Dr. Pencina is responsible for developing and implementing quantitative science strategies as they pertain to the education and training, and laboratory, clinical science, and data science missions of the School of Medicine. He leads the School’s IT strategic direction and investments, working in collaboration with the vice presidents and chief information officers of Duke Health and Duke University’s Office of Information Technology.
Dr. Pencina is a professor of Biostatistics and Bioinformatics at Duke University and served as director of Biostatistics at the Duke Clinical Research Institute (DRCI). He is an internationally recognized expert in risk prediction model development and evaluation. Expert panels and guideline groups frequently recommend methods for quantifying improvement in model performance proposed in his research.
Dr. Pencina is actively involved in the design, conduct and analysis of clinical studies with particular focus on novel and efficient designs and applications of machine learning for medical decision support. He interacts regularly with investigators from academic and industry institutions as well as with the Food and Drug Administration. Thomson Reuters/Clarivate Analytics recognized Dr. Pencina as a Highly Cited Researcher in social sciences and clinical medicine for the years 2014 – 2020. He is co-author of over 370 manuscripts published in peer-reviewed journals and has been cited nearly 65,000 times in professional publications. He serves as Deputy Editor for Statistics at JAMA-Cardiology and Associate Editor for Statistics in Medicine.
In 2003, Dr. Pencina received his PhD in Mathematics and Statistics from Boston University. He holds master’s degrees from the University of Warsaw in actuarial mathematics and business culture. Dr. Pencina joined the Duke University faculty in 2013. He served as an associate professor in the Department of Biostatistics at Boston University and the Framingham Heart Study and as director of Statistical Consulting at the Harvard Clinical Research Institute.
Co-organizer
Questions? Contact Us.
Message the MIDAS team: midas-contact@umich.edu