Designing Healthcare AI for Incomplete or Conflicting Data - Delphyr
In this blog series, Delphyr Engineering, we share practical insights from building AI systems for real clinical use. In this blog, our engineer Tim dives into an important question: what happens when the record itself doesn't tell the whole story?
The EHR is a representation of the patient
It's easy to treat an electronic health record as ground truth. In practice, it's a representation: assembled over time, by different people, under different pressures. Diagnoses that are no longer relevant can still be there. Medications that were stopped months ago can still be listed. And, less obviously, fields that should have been loaded are sometimes empty.
None of this is a flaw specific to any one EHR. It's a structural feature of documentation in healthcare: the chart reflects what was entered, not necessarily what is currently true. Any AI system built on top of an EHR inherits that gap, whether it acknowledges it or not.
When a clinician disagrees with the system
Sometimes a clinician will flag that Delphyr's read of the record doesn't match what they know to be true. That kind of disagreement raises an immediate design question: when a user pushes back on a factual claim with high confidence, should the system defer, correct itself, or hold its ground?
The instinctive answer is "defer to the clinician": they know the patient, the system doesn't. But that's not automatically the right answer either. Sometimes the clinician is right and the system is missing data. Sometimes the system is right and the clinician is misremembering, or thinking of a different visit. Simply flipping to agree with whoever pushes back hardest isn't a safety property, it's just a different kind of confident wrongness.
Designing for the disagreement, not around it
This is where we think the more durable answer lies: not in trying to arbitrate who is right, but in being explicit about where a piece of information comes from, and keeping that origin visible rather than flattening everything into a single fact.
Records will keep containing gaps, and clinicians will keep knowing things the record doesn't say. A system that assumes the EHR is complete will eventually be confidently wrong. A system that assumes the clinician is always right will eventually be wrong in a different, harder-to-catch way, because it stops checking.
This is part of why Delphyr is designed for summarization of data in the record, not for interpretation of the underlying data. The distance between the two is bigger than it might seem. Summarization carries what we'd call summary-risk: the risk of leaving something out, or representing it in a way that shifts its weight.
Interpretation, or advice-giving, carries that same risk, plus another one on top: a representation risk, where the system's read of the record can start to stand in for the record itself. That matters especially because of automation bias, the tendency to follow a system's output without applying the same scrutiny you would to a colleague. The more confidently a system appears to interpret, the more that risk grows.
That's why the more durable answer, for now, is to keep the EHR as one source among several, keep track of what's confirmed versus what's stated, and let healthcare professionals see that distinction rather than resolving it invisibly on their behalf. It's less satisfying than a system that simply "knows the answer," but it keeps the clinician doing the weighing, not just receiving conclusions.
The bottom line
An EHR is a partial, sometimes-stale representation of a patient, not the patient. Building clinical AI on top of it means designing for that gap, not around it: being honest about what's confirmed versus asserted, and giving clinicians visibility into which is which. The goal isn't a system that's never contradicted. It's a system that's clear about why.