Conflicting Studies on Using EHR Technology
William Hersh, MD
Professor and Chair
Department of Medical Informatics & Clinical Epidemiology
Oregon Health & Science University
Blog: Informatics Professor
Last winter, the informatics world was abuzz with a study published in the journal Archives of Internal Medicine that found a lack of improvement in healthcare quality measures for patients whose physicians had adopted electronic health records (EHRs) [1]. However, as noted by a colleague and I in a follow-up letter to the editor, this study had a number of flaws [2]. Our main complaint was that the quality measures assessed were independent of the EHR implementation, hence any association, whether it would have been positive or negative, was indirect at best. The study was also based on data from a half-decade ago, when our use of EHRs and the systems themselves were much more basic.
The furor about the paper died down, and most people got back to working on implementing meaningful use. No one disagreed that we need more research on whether EHR systems do improve healthcare quality, including studies with better methodology.
In August, another study came along. Published in the New England Journal of Medicine (NEJM), this study used a somewhat similar methodology to assess 46 practices in the Cleveland area, 33 of which had adopted EHRs [3]. The study assessed the outcomes of 27,207 patients with diabetes mellitus who were followed by a total of 569 providers. The study looked at four process measures and five outcome measures in those diabetic patients, comparing them for providers who had and had not adopted EHRs. Overall composite quality measures were developed for the process and outcome measures, and found to be 35.1% higher in the former and 15.2% higher in the latter. The difference was found to persist across all insurance types and, even more gratifying, for “safety net” clinics that historically see more complicated patients of lower socioeconomic status.
This study did use a roughly similar methodology to the Archives study, and as such must be viewed as having a weaker form of evidence than a direct randomized controlled trial (RCT). Of course, in reality, such an RCT would be near impossible to do, i.e., randomizing patients to receive their care from a provider having an EHR or not. We also know that there can be confounders between practices utilizing and not utilizing EHRs.
Nonetheless, the NEJM study did have advantages over similar studies done before it, including the Archives study. One clear advantage was that the study had complete data on all patients, unlike the Archives study, which only relied on a data set from the CDC National Center for Health Statistics (NCHS). The researchers also had precise data on the providers, the EHR implementation, and how the quality measures were integrated into the provision of care.
While this new study received a great deal of press, another study that received less press, which was published shortly after the publication of the Archives study, should have received more [4]. Although still not an RCT design, this study did use a before-and-after methodology to examine change in compliance with 16 quality measures before and after implementation of a commercial EHR in a large academic internal medicine practice. The results showed improvement after the EHR was implemented.
In an editorial accompanying the NEJM study, Classen and Bates noted that the study showed the “meaning in meaningful use” [5]. They correctly pointed out that implementing EHRs was not what HITECH should be about, but rather showing that the technology was able to be used to make meaningful improvement in the health of patients whose providers use it. As in most areas of medicine, we cannot wait for the perfect study or studies to answer all questions unequivocally, but the evidence base is growing for the value of informatics, especially when systems are implemented properly.
References
1. Romano, M. and Stafford, R. (2011). Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Archives of Internal Medicine, 171: 897-903.
2. Mohan, V. and Hersh, W. (2011). EHRs and health care quality: correlation with out-of-date, differently purposed data does not equate with causality. Archives of Internal Medicine, 171: 952-953.
3. Cebul, R., Love, T., et al. (2011). Electronic health records and quality of diabetes care. New England Journal of Medicine, 365: 825-833.
4. Persell, S., Kaiser, D., et al. (2011). Changes in performance after implementation of a multifaceted electronic-health-record-based quality improvement system. Medical Care, 49: 117-125.
5. Classen, D. and Bates, D. (2011). Finding the meaning in meaningful use. New England Journal of Medicine, 365: 855-858.