Overview
At the “Implementing AI in Health” colloquium, a collaborative event presented by the Michigan Institute for Data Science and LHS Collaboratory, speakers shared their knowledge and expertise about the implementation of health AI and their prior, ongoing, and planned work in this area. This event featured four esteemed speakers: Dr. Barbara A. Barry, Dr. Michael J. Kim, Dr. Lisa S. Lehmann, and Dr. Michael Pencina.
Schedule
9:00 AM – 9:10 AM
Opening Remarks
H.V. Jagadish, Edgar F Codd Distinguished University Professor of Electrical Engineering and Computer Science, Bernard A Galler Collegiate Professor of Electrical Engineering and Computer Science, Professor of Electrical Engineering and Computer Science, College of Engineering and Director of the Michigan Institute for Data Science
Karandeep Singh, Assistant Professor of Learning Health Sciences, Assistant Professor of Internal Medicine, Assistant Professor of Urology, Associate Chief Medical Information Officer of Artificial Intelligence, Medical School and Assistant Professor of Information, School of Information
9:10 AM – 9:55 AM
Keynote Speaker – Lisa Lehmann, Associate Professor of Medicine, Center for Bioethics, Harvard Medical School
A Roadmap for Implementing Trustworthy AI in Healthcare
View Recording View Slides9:55 AM – 10:05 AM
Break
10:05 AM – 10:50 AM
Keynote Speaker – Michael Pencina, Vice Dean for Data Science and Director of Duke AI Health, Duke University School of Medicine
Evaluation and Governance of AI-based Clinical Decision Support tools in Health Systems
View Recording10:50 AM – 10:55 AM
Break
10:55 AM – 11:20 AM
Panel Discussion
Moderator: Gretchen Piatt, Associate Chair, Department of Learning Health Sciences, Associate Professor of Learning Health Sciences, Medical School and Associate Professor of Health Behavior and Health Education, School of Public Health
Panelists:
Lisa Lehmann, Associate Professor of Medicine, Center for Bioethics, Harvard Medical School
Michael Pencina, Vice Dean for Data Science and Director of Duke AI Health, Duke University School of Medicine
11:20 AM – 12:10 PM
Networking Lunch
12:10 PM – 12:15 PM
Welcome Back
Gretchen Piatt, Associate Chair, Department of Learning Health Sciences, Associate Professor of Learning Health Sciences, Medical School and Associate Professor of Health Behavior and Health Education, School of Public Health
Karandeep Singh, Assistant Professor of Learning Health Sciences, Assistant Professor of Internal Medicine, Assistant Professor of Urology, Associate Chief Medical Information Officer of Artificial Intelligence, Medical School and Assistant Professor of Information, School of Information
12:15 PM – 1:00 PM
Keynote Speaker – Barbara A. Barry, Assistant Professor of Medicine | Collaborative Scientist, Robert D. & Patricia E. Kern Center for the Science of Healthcare Delivery, & Mayo Clinic Alix School of Medicine
Implementing AI in Health Care with a Human-in-the-Loop
View Recording View Slides1:00 PM – 1:10 PM
Break
1:10 PM – 1:55 PM
Keynote Speaker – Michael Kim, Chief of Staff, National Artificial Intelligence Institute, U.S. Department of Veterans Affairs
Translating Trustworthy AI Frameworks into Practice at the US Department of Veterans Affairs
View Recording View Slides1:55 PM – 2:00 PM
Break
2:00 PM – 2:25 PM
Panel Discussion
Moderator: Akbar Waljee, Lyle C Roll Professor of Medicine, Michigan Medical School
Panelists:
Barbara A. Barry, Assistant Professor of Medicine | Collaborative Scientist, Robert D. & Patricia E. Kern Center for the Science of Healthcare Delivery, & Mayo Clinic Alix School of Medicine
Michael Kim, Chief of Staff, National Artificial Intelligence Institute, U.S. Department of Veterans Affairs
2:25 PM – 2:30 PM
Closing Remarks
Gretchen Piatt, Associate Chair, Department of Learning Health Sciences, Associate Professor of Learning Health Sciences, Medical School and Associate Professor of Health Behavior and Health Education, School of Public Health
2:30 PM – 3:00 PM
Post-Event Networking and Refreshments Available
About the Speakers
Barbara A. Barry
Assistant Professor of Medicine | Collaborative Scientist, Robert D. & Patricia E. Kern Center for the Science of Healthcare Delivery, & Mayo Clinic Alix School of Medicine
Bio
Barbara A. Barry, Ph.D., is a human-computer interaction (HCI) researcher who studies how interaction with artificial intelligence (AI) impacts human intelligence, communication and behavior. Dr. Barry has made fundamental contributions to AI, developed and evaluated intelligent interactive software agents, and collaborated on large-scale implementations of new technologies to improve health and education. Dr. Barry’s work focuses on how technology enables (or disables) human agency in health care and how to create productive, ethical and evidence-based use of AI within a learning health care system.
Implementing AI in Health Care with a Human-in-the-Loop
Abstract
As Artificial Intelligence (AI) and Machine Learning (ML) are introduced into heath care delivery, the phrase “human-in-the-loop” echos throughout implementation planning, regulatory guidance, and bioethics discussions. In this talk, I will explore what we mean by human-in-the-loop, where it succeeds and fails, and how human-AI interaction can inform adoption of AI/ML in health care.
Michael J. Kim
Chief of Staff, National Artificial Intelligence Institute, U.S. Department of Veterans Affairs
Bio
Dr. Kim began his career with VA Long Beach Healthcare System in 2012 as the Section Chief of Neuroradiology and then served as Chief of Imaging Healthcare Group from 2017 to 2019. Dr. Kim received his B.S. in Biochemistry and Molecular Biology from the University of California – Santa Cruz, his Medical Doctorate from the Feinberg School of Medicine at Northwestern University, Diagnostic Radiology Residency at the University of California – Irvine, and Neuroradiology Fellowship at the Keck School of Medicine at the University of Southern California. Dr. Kim holds an appointment with the School of Medicine at University of California – Irvine as an Associate Clinical Professor in the Department of Radiological Sciences.
Dr. Kim has held a variety of leadership positions ranging from Chief Resident, Chief Fellow, Section Chief, and Service Chief while also receiving a variety of awards including, Intern of the Year, Teacher of the Year, and Best Healthcare Group Chief. His research interests include the use of artificial intelligence and has a research grant with the Center of Artificial Intelligence in Diagnostic Medicine (CAIDM) at the School of Medicine at the University of California- Irvine. Dr. Kim is passionate in process improvement and has a Lean Six Sigma Black Belt in Healthcare Management.
Translating Trustworthy AI Frameworks into Practice at the US Department of Veterans Affairs
Abstract
Leaders around the world are demanding transparent regulation and oversight of AI systems to protect end users. The United States government has already taken action through Executive Order 13960, which carries the force of law within the federal government to ensure that AI systems are trustworthy. Additional guidelines such as the Blueprint for an AI Bill of Rights and Executive Order 14091 further strengthen the U.S. government’s commitment to ensuring that AI systems are trustworthy and equitable. Please join Michael J. Kim, MD, Chief of Staff at the National Artificial Intelligence Institute (NAII), as he outlines Trustworthy AI policy, frameworks, and pilots from the U.S. Department of Veterans Affairs that demonstrate how the federal government’s regulation of AI is protecting our Veterans.
Lisa S. Lehmann
Associate Professor of Medicine, Center for Bioethics, Harvard Medical School
Bio
Lisa Soleymani Lehmann is a primary care physician and bioethicist. She is Associate Professor of Medicine (part-time) at Harvard Medical School (HMS) and Associate Professor of Health Policy and Management (part-time) at the Harvard T. H. Chan School of Public Health. From 2020-2023 Lisa was Director of Bioethics at Google. She has also previously held positions as Chief Medical Officer (2017-2020) for the VA New England Healthcare System which is a multi-state integrated healthcare system comprised of 8 medical centers and 41 community-based outpatient clinics that serves over 260,000 Veterans. Prior to serving VA New England, Lisa was Executive Director of the VA National Center for Ethics in Health Care, Director of Bioethics at Brigham and Women’s Hospital, and a practicing primary care physician and health services researcher at Brigham and Women’s Hospital.
A Roadmap for Implementing Trustworthy AI in Healthcare
Abstract
AI holds the promise of transforming healthcare by improving health outcomes for all people through improved diagnosis, predictions, and therapeutics. Identifying and addressing ethical concerns such as the safety and efficacy of AI systems, the quality of the data used to train and validate models, the risk of bias against marginalized groups, and how clinicians and patients interact with AI will be essential to realizing the benefits of AI in health care. I will identify ethical risks and potential mitigations for the use of AI in health care and provide a roadmap for the implementation of trustworthy AI.
Michael Pencina
Vice Dean for Data Science and Director of Duke AI Health, Duke University School of Medicine
Bio
As Vice Dean of Data Science, Dr. Pencina is responsible for developing and implementing quantitative science strategies as they pertain to the education and training, and laboratory, clinical science, and data science missions of the School of Medicine. He leads the School’s IT strategic direction and investments, working in collaboration with the vice presidents and chief information officers of Duke Health and Duke University’s Office of Information Technology.
Dr. Pencina is a professor of Biostatistics and Bioinformatics at Duke University and served as director of Biostatistics at the Duke Clinical Research Institute (DRCI). He is an internationally recognized expert in risk prediction model development and evaluation. Expert panels and guideline groups frequently recommend methods for quantifying improvement in model performance proposed in his research.
Dr. Pencina is actively involved in the design, conduct and analysis of clinical studies with particular focus on novel and efficient designs and applications of machine learning for medical decision support. He interacts regularly with investigators from academic and industry institutions as well as with the Food and Drug Administration. Thomson Reuters/Clarivate Analytics recognized Dr. Pencina as a Highly Cited Researcher in social sciences and clinical medicine for the years 2014 – 2020. He is co-author of over 370 manuscripts published in peer-reviewed journals and has been cited nearly 65,000 times in professional publications. He serves as Deputy Editor for Statistics at JAMA-Cardiology and Associate Editor for Statistics in Medicine.
In 2003, Dr. Pencina received his PhD in Mathematics and Statistics from Boston University. He holds master’s degrees from the University of Warsaw in actuarial mathematics and business culture. Dr. Pencina joined the Duke University faculty in 2013. He served as an associate professor in the Department of Biostatistics at Boston University and the Framingham Heart Study and as director of Statistical Consulting at the Harvard Clinical Research Institute.
Evaluation and Governance of AI-based Clinical Decision Support tools in Health Systems
Abstract
With the increased interest in applications of AI-based Clinical Decision Support tools in health systems, there is an urgent need to create an appropriate framework for their management across the lifecycle. Despite numerous best practices being put forth for evaluation of these tools, many developers rush through the process, resulting in creation of many algorithms of questionable quality. Furthermore, health systems struggle with establishing the right governance structures for identifying, implementing and managing AI-based Clinical Decisions Support tools. In this presentation we outline best practices for development, evaluation and monitoring of clinical algorithms and share our experience with establishing a governance structure for their implementation in Duke Health.
Co-organizer
Questions? Contact Us.
Message the MIDAS team: [email protected]