Imaging AI is everywhere. Only 1 in 5 rate it a hit.

In a 2024 survey of 43 large US health systems, imaging AI was the most widely deployed AI: 38 of the 42 that answered ran it at least in part. AI scribes ran at 26 of 43. Yet only about 1 in 5 systems rated diagnostic AI, the category that holds imaging, a high success (5 of the 27 that rated it), against about half for documentation AI, the category that holds scribes (21 of 40).

Why the gap? Last week's Prelude found that 1,576 of the 1,614 FDA-authorized AI devices are sold to help a clinician, who still has to check the AI's work. Phase 1 called the cost of that check the verification tax. This article asks who carries the check, and what they get back.

"High success" is a proxy. One person per system rated it, most often the chief medical information officer. Not every system rated every kind: 27 of the 40 systems active in imaging AI rated diagnostic AI, against 40 of 43 for documentation. And it was early: at least 14 of the 21 systems that rated documentation AI a high success had not fully deployed any of it. Still, the gap is too large to be chance (Fisher's exact test, p = 0.006). What a buyer needs is renewal: is the tool still paid for after the pilot? The next two articles follow that.

Time saved doesn't explain the gap

Put on one scale, as the change in a clinician's time on the task, scribes and imaging AI that assists each read save about the same. Across five scribe results, note time fell by up to 21% (heavy users), a median of 14.5%; one arm saw no significant change. Across six results for imaging AI that assists each read, reading time changed by anywhere from 38% less to 10% more, a median of about 12% less. Only AI that takes over a read saves much more: 44% to 62% fewer readings. In a Swedish trial of 105,934 women, AI sent low-risk mammograms to one radiologist instead of two and marked the riskiest, and about 29% more cancers were found. England's evaluation of chest-imaging AI at 10 NHS trusts found "little evidence of staff time saving," though staff wanted to keep it. If time saved decided success, imaging AI would be rated at least as highly.

What a check costs

1. No check: the output is the result, as with an autonomous eye exam.
2. Exceptions only: a person reviews flagged cases, such as insurance denials.
3. Edit a draft: a person checks every output, but the edit replaces work they used to do, as with scribes.
4. Full review on top: a person still does the whole task and checks the AI's marks too, as with read-assist imaging AI.

The check is often policy: UCHealth lets AI prepare orders, but a clinician must accept them. And a check that eats the saving can end a pilot. Nebraska Medicine built AI to fill in its cancer registry. Staff wanted 95% accuracy before it ran unchecked; it reached a little over 80%, nurses re-checked even its summaries, and after two months an executive committee stopped it, its digital chief told Becker's Hospital Review.

Removes a task, or adds a step

A scribe takes a task away from the doctor. The doctor still checks the draft but no longer writes the note, and gets the patient back. "I feel like I can be more face-to-face with them and [have] more eye contact," a Stanford physician said. "It has greatly reduced the amount of work I bring home with me," said a clinician in Philadelphia, where after-hours record time fell 30%. In a 30-day survey at six health systems, burnout fell from 51.9% to 38.8%, and in a 24-week randomized trial the relief lasted. The gain isn't universal: in a study of 8,581 clinicians, after-hours time did not change, and in Dutch general practice, where doctors already typed little, documentation time fell by just 42.7 seconds a visit, with every draft needing review. The health system also bills more, about $3,044 a year per doctor at UCSF, though the Peterson Health Technology Institute's 2025 review found no clear financial return yet. Drafted replies to patient messages, which also hand the doctor a draft, were rated a high success at only 3 of 33 systems; in two studies they saved no time, so the draft didn't take the writing away.

Imaging AI that assists each read adds a step, even where it trims reading time. The radiologist still reads the whole image, then checks every mark. "It slows you down because you have to verify each little dot," a consultant radiologist in Brisbane said. A flag the radiologist disagrees with still has to be ruled out: "AI has generated more work because some cases I thought were normal, but the AI flags something, I feel obliged to investigate more," said a radiologist in Singapore. At a US teleradiology practice, only 7 of 65 radiologists agreed that its bleed-detection AI cut their reading time per head CT, and a third said the time spent on false alarms outweighed its benefits. The gain, an extra finding or a faster call, goes mostly to the patient and the emergency department. Apart from temporary Medicare add-ons for a few triage tools, no one pays the hospital extra for read-assist AI.

Buyers see the same thing. In England, a cancer network's skin group was offered a skin-cancer AI, "effectively cost free" for three years. Some members worried that, after the free years, it "would involve paying twice - for the software and then for a clinician to look at the images."

Agents move the check to the vendor

Agents promise to shrink the check to exceptions. Medicare's WISeR model pays six technology firms 25% of the spending their prior-authorization reviews avert. A licensed clinician must review every denial, the firm pays its processing costs, and it loses the payment if a denial is overturned on appeal. The check stays; the vendor pays for it.

What we found

Our reading: success follows what the person who checks gets back. When the person who checks the AI, or the budget that pays for it, keeps the gain, we call that the same pocket; success here tracks the person. When the AI takes a task away from them, as scribes do, they rate it a success even though the time saved is modest. When it adds a step to work they still own, as read-assist imaging AI does, the gain goes to someone else. AI that takes over a whole read, as in the Swedish trial, removes a task too, and saves time. The budget alone doesn't earn the label: revenue-cycle AI, meant to help the hospital get paid, was rated a high success at only 7 of 31 systems. Whether the paying budget decides renewal is what the next two articles test.

So what?

• Product managers: count the steps your AI removes and the steps it adds; if it adds a check to a full read, sell the reads it removes (where double reading exists, as in Europe) or faster treatment.
• Health-system buyers: make renewal depend on measured checking time against what the tool returns, and count the staff time to validate and monitor each model.
• Payers: before paying for AI a clinician checks, ask what the check catches and whether the AI shifts what gets coded.
• Investors: ask who pays for the minutes of human review; if the seller does the check, as WISeR's vendors must, it belongs in the gross margin.

When your team rolled out its last AI tool, who checked its output, and was that checking time in the business case?

Join the discussion on LinkedIn: the post