William Hersh, MD, Professor and Chair, OHSU
Blog: Informatics Professor
LinkedIn: William Hersh, MD
X: @williamhersh
How much of the hype for artificial intelligence (AI) that will truly impact health, healthcare, and research is an unknown. The potential benefits are unequivocal, from assistant patients pursue actions to improve their health to giving guidance to clinicians in diagnosis and treatment to helping researchers find information and devise new ideas to advance their research.
I have published an invited post in the National Library of Medicine (NLM) Musings from the Mezzanine Blog, the blog of the Director of the NLM. I chose to update some of my past writings posted in this blog with a new discussion of what I call translational AI.
The tl;dr is:
- The actual day-to-day use of clinical AI in healthcare is still modest, according to surveys.
- While there are thousands of machine learning model papers that are published, and many systematic reviews of those model papers, there are a much small number, probably on the order of 100, randomized controlled trials (RCTs) of AI interventions in healthcare.
- Of those RCTs, not all have resulted in positive outcomes and a number of them have risk of bias concerns.
Clearly, as in all of healthcare, we cannot do RCTs on every permutation of model, implementation, setting, etc. of AI. However, we must treat AI the same way as any other tool we use in healthcare: Show us the evidence. Granted, evaluating the use of AI has plenty of differences from evaluating other interventions used in patient care, such as drugs and devices. It is difficult to conure a “placebo” for AI, and hard to perform controlled studies when AI, such as ChatGPT, is all around us.
Nonetheless, we can apply evidence-based medicine (EBM) to help inform its clinical use. The ideal way to do that is through randomized controlled trials (RCTs), or ideally systematic reviews of RCTs. As I note in the post, this is imperative not only for those of us who promote the use of AI and other biomedical and health informatics interventions, but also for students and trainees looking for projects to develop impactful research programs in their careers.
This article post first appeared on The Informatics Professor. Dr. Hersh is a frequent contributing expert to HealthIT Answers.