The Problem With Machine Bias

By Dr. Nick van Terheyden aka Dr. Nick
Twitter: @drnic1
Host of Dr. Nick: The Incrementalist – #TheIncrementalist

On this episode I am talking to Brian Powers, MD, (@brianwpowers) resident physician at the Brigham and Women’s Hospital in Boston. As you will hear he has already achieved a great deal as an emerging voice in American Medicine but we focused on the recently published paper “Dissecting racial bias in an algorithm used to manage the health of populations” where he is a co-author, that studied predictive algorithm’s that are widely used for population health management but exhibit racial bias.

Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias:

Episode NOW on Demand

I discussed this in my recent interview with Patrice Harris, MD, the current President of the AMA at the HLTH conference where she describes the importance of addressing the perils of technology that if unchecked would simply replicate the bias that already exists in both the data and the systems.

We talk about the path to this paper which includes some keen insights and incremental steps for others to learn from. Based on his earlier work (Practice and Protest: Black Physicians and the Evolution of Race-Conscious Professionalism) he already was sensitive to discrepancies in the data that reflected care but as he points out it required that instead of ignoring these data rather digging deeper and resisting the urge or push that exists today to push on and not deviate from the original target.

This is a recurring theme in many of my interviews where many note the need to be open to alternative explanations and be prepared to follow the data, bearing in mind that many of these sojourns remain un-reported but exist in the classical picture I’ve shared before that the path to success is not a straight line bu rather filled with many dips and bumps.

We talk about the end for precision in the use of data and the labeling of that data and the outcomes we are seeking and the proxies we use to assess this. Listen in to hear his thoughts on how we continue to use and maximize the benefit of Augmented intelligence whilst combatting the challenge of using data with existing bias. The good news is this is the canary in the calming on AI use and will help highlight the importance of board team-based approaches to the development of these tools that assess the value in the context of desired and expected outcomes.

About the Show
For years Dr. Nick van Terheyden aka Dr. Nick, has served as a voice on the impact of new technologies on healthcare, earning a reputation as a leading authority on where the future of medicine is going. Combining powers of observation and real world experience, Dr. Nick has seen many predictions come true and makes the case that innovations in healthcare can be accomplished incrementally, not just by moonshot events. Tune in to hear Dr. Nick: The Incrementalist and his guests discuss what the future of healthcare looks like, how we will get there, and what it will take to improve healthcare for all.

This article was originally published on the Dr. Nick – The Incrementalist blog and is republished here with permission.