By Nitish Surana, Lead Engineer, January AI
LinkedIn: Nitish S.
LinkedIn: January AI
An electronic health record can show that a medication was prescribed and is active, even after the patient quietly stopped taking it weeks ago. If an AI tool treats that field as current fact, it can produce a confident, wrong answer. A clinician can resolve the uncertainty by asking a follow-up question. An AI system has no equivalent move unless its designers build one in.
That gap between accessing information and understanding it is where much of the risk in healthcare AI lives. Health leaders evaluating tools that generate insights from patient records often ask which data sources can be accessed. A more revealing question is how each system interprets that data before generating an output.
How does the AI system determine what is current?
Health data changes over time, but records do not always keep up. Medication lists become outdated, symptoms resolve, and new information can alter the meaning of an earlier result.
AHRQ’s Patient Safety Network has highlighted how failures to communicate discontinued medications electronically can leave outdated prescriptions appearing active. The dangerous version of an AI system is not the one that acknowledges uncertainty. It is the one that presents an inference as a verified fact.
Ask vendors how their AI systems distinguish historical information from a patient’s current state. Does the system consider timestamps, later encounters, and patient-reported updates? Does it surface uncertainty when those signals are incomplete?
How is patient data prepared before it reaches the model?
Healthcare data arrives in different formats, terminologies, and units. Two systems may use different names for the same medication or laboratory test. One may provide structured codes while another relies on free text.
Standards help address this fragmentation. HL7 FHIR standardizes how systems exchange records, LOINC resolves two labs’ different names for the same test to one concept, RxNorm does the same for medications, and UCUM encodes units precisely enough to make values genuinely comparable. But standards do not guarantee consistent implementation. One organization may map a field precisely, another only partially, and a third not at all.
Health leaders should ask who normalizes the data, how incomplete mappings are handled, and whether the original source is preserved. An AI model cannot reliably interpret information that has lost its meaning before reaching it.
How does the AI system handle contradictory records?
Even after data is normalized, patient records can conflict. An EHR record might note a cough the patient says is gone two weeks later. Elsewhere, a record shows high blood pressure that, after lifestyle changes, the patient self-declares is resolved, with no visit to confirm it. A resolved symptom is straightforward. A self-reported reversal of a diagnosis with no clinical confirmation is a different kind of claim, and treating the two the same way is a mistake.
The most responsible system is not necessarily the one that always resolves the conflict. In some cases, the appropriate response is to flag the discrepancy, explain the competing signals, and request human review.
During a vendor evaluation, test the AI system with synthetic patient data designed to contradict itself. Does the system surface the conflict, silently select one record, or ignore the discrepancy? Product demonstrations often rely on clean data. Healthcare rarely does.
Can the AI system trace the basis for its output?
A trustworthy AI system should allow users to trace an output to the information that shaped it. For a straightforward fact, that may mean identifying the record, source, and date.
For a derived conclusion, source traceability is only the first layer. Health leaders should also be able to identify the clinical rule or evidence source used to interpret the patient-specific data. These are separate questions: Where did the patient information come from? What supported the interpretation?
HL7 FHIR includes a Provenance resource that can record who or what created or updated a resource and when. That supports data lineage, but provenance alone does not establish that an AI-generated conclusion is clinically sound. Organizations must still examine the evidence behind the interpretation and monitor the system over time.
Evaluate the System, Not Just the Output
Taken together, these four questions help reveal whether an AI tool is simply connected to patient data or whether the broader system can interpret that information responsibly. Health leaders do not need to become data scientists, but they should ask vendors to demonstrate how their systems handle outdated, incomplete, and contradictory records, explain the basis for their outputs, and monitor performance after deployment.
Healthcare organizations will not gain the most value by connecting AI to the greatest volume of patient data. They will gain it by understanding how their systems turn that data into context, where uncertainty remains and when human judgment must take over.