By Catherine Strawley and Jordan Everson, ASTP/ONC
LinkedIn: Catherine Strawley, MPH
LinkedIn: Jordan Everson
LinkedIn: ASTP/ONC
What’s in a number? A lot, it turns out, especially when it comes to measuring interoperability progress. For well over a decade, we have collaborated with federal and private sector partners on interoperability-related surveys to answer a big picture question: “how are we doing?” Unlike other aspects of our lives where we can measure progress based on a single metric (like weight loss), measuring interoperability progress has become more challenging over time.
Changes in technology, policy, and standards have enabled different exchange methods to become more widespread (e.g., national and regional networks, FHIR®-based APIs) and support new interoperability use cases, which can add important context toward how we perceive progress. To meet this measurement challenge, ASTP/ONC, in collaboration with our colleagues at the University of California San Francisco and a panel of technical experts, developed a new set of indices to track interoperability among U.S. non-federal acute care hospitals. These indices are comparable to other economic indices that can be followed over time, for example, the Consumer Confidence Index or the Economic Confidence Index.
We developed three indices to track interoperability:
- “Core Index” measuring levels of adoption of foundational interoperability capabilities,
- “Pathfinder Index” representing the extent to which hospitals have adopted advanced technologies for interoperability and engage with auxiliary exchange, and
- “Friction Index” quantifying the extent to which hospitals face barriers to interoperability.
We believe these indices will provide an important means to capture and trend progress over time as well as identify when we have achieved widespread interoperability for certain use cases across US hospitals.
Building Holistic Interoperability Indices
As described in a new study in JAMIA, the three indices build upon existing measures of interoperability while accounting for advancements in technology, reporting, and exchange practices that have become more common following the 21st Century Cures Act and the COVID-19 pandemic, as well as challenges that accompany information exchange.
These indices were derived using items from the 2022 and 2023 American Health Association Health IT Supplement, a nationally representative survey that is fielded to hospitals annually. To design these indices, we relied on guidance from a technical expert panel (TEP), which included representatives from health information organizations, health systems, trade groups and technology companies. The TEP reviewed questions on the survey and worked with the study team to design and finalize the structure of the index while identifying areas for additional measurement. The final structure of each index accounts for the following interoperability concepts.
Each component was scored separately, assigned equal weights, and aggregated to achieve final index scores on a continuous scale ranging from 0-100. Because of the way the survey is fielded, we will re-calculate the Pathfinder Index on even years (next, for 2024) and the Core and Friction Indices on odd survey years (next, for 2025).
Why These Measures Matter
We find that hospitals perform better with foundational interoperability capabilities, scoring higher on the Core Index compared to the Pathfinder Index. In general, according to our Friction Index, hospitals are experiencing moderate levels of friction, particularly with respect to barriers to exchange and needing to use numerous methods to exchange information.
Ongoing tracking of index performance, specifically scores on individual component items, can identify needs for targeted policy to address a specific issue and allow us to assess whether a targeted policy has achieved its goal. The current indices offer baseline interoperability scores but as we receive new data, we can implement longitudinal performance tracking.
Our work also highlights major differences in hospital performance, with several hospital types (those that are smaller in size, are critical access, are not located in metropolitan areas, and do not use a market leading EHR) performing significantly worse on the Core and Pathfinder Indices compared to their counterparts. Longitudinal tracking of index scores can also allow us to determine how these differences persist or shift over time.
Similar to existing economic indices, policymakers can use the indices we developed to monitor the state of interoperability nationwide, inform policy adjustments, and communicate progress.
This article was originally published on the Health IT Buzz and is syndicated here with permission.