An AI tool earned only the money the hospital would not have collected without it, and only a comparison fixed before the tool picks its claims can measure that. Take one claim for $900. The insurer denies it, someone corrects it, and the insurer pays. An AI tool touched the claim along the way, so its dashboard adds $900 to “recovered revenue.” On that one claim the tool may have earned $900, about a dollar, or nothing.
It earned $900 if it found an error that the billing staff would never have found. It earned a week’s interest on $900, roughly a dollar, if the staff would have made the same correction seven days later. It earned nothing if it flagged a claim that someone was already fixing. The insurer’s payment looks the same in all three cases, because a payment records what happened and is silent on what would have happened otherwise.
So the arrival of the money is an accounting event, and the tool’s share of it is a question of cause. Answering it takes two things. The first is a reliable history of each claim. The second is a comparison with similar claims that chance, and not the tool, set aside for the usual work. A hospital that pays on the dashboard’s total pays the tool for every claim it touched, including the ones that would have been paid anyway.
As a physician I was trained not to credit a treatment because a patient improved after it. Some patients improve without it, so a treatment earns credit only for the difference between patients assigned to it and patients who were not. A claim paid after the tool touched it is that patient.
Why the question is urgent now
By one seller’s count across more than 2,300 hospitals, insurers denied about one claim in nine on first submission in the first half of 2026.1 In another seller’s survey of 204 finance and billing leaders, fielded in August 2026, 64% said they definitely or probably plan to invest in AI for claims and denials.2 The evidence has not kept pace with the plans to buy, and the Check reports a search that found no controlled test of such a tool.
What the payment record can say
The first need is a history of each claim, and the insurer’s payment file supplies only part of it. When an insurer finishes with a batch of claims, it sends the hospital an electronic file that lists each payment and each amount it held back. The file is the electronic remittance advice, called the 835 after the number of its transaction standard. An amount held back is an adjustment, and Medicare’s version of the file can adjust one line of a claim, a whole claim, or the provider’s account.3
Each adjustment carries codes. A group code says who bears the unpaid amount, the provider or the patient, and a reason code, called a CARC, gives the insurer’s reason.3 The 835 is therefore good evidence of what the insurer decided. It does not say which person or program caused a correction, whether the correction was needed, or whether ordinary work would have brought the same payment.
A payment in the file is also not yet cash that stays. An adjustment to the provider’s account, such as the recovery of an earlier overpayment, is usually tied to no claim in that file.3 So when an insurer takes back a payment, the file may record the loss against the account, and a dashboard that adds up paid claims may never see the money leave. The seller quoted above counted fewer first denials than a year earlier, 10.63% of claims against 12.00%, yet cash came in no faster. Its researchers pointed to what insurers do after they pay, including taking payments back.1
One set of claims gives two denial rates
The history must also say which denial rate it reports, because a pilot can show a falling rate by changing the definition. Suppose a hospital sends 1,000 claims and follows every one to its end. The insurer denies 150 on first submission and later pays 60 of those after correction. These numbers are an example. The share denied at first is 150 in 1,000, or 15%. The share still denied at the end is 90 in 1,000, or 9%.
A task force of the Healthcare Financial Management Association defines an initial denial rate that counts only the first denial of a claim and leaves resubmitted bills out.4 KFF’s analysis of plans sold on HealthCare.gov does the reverse. A claim that was denied, resubmitted and paid is not counted as denied, and on that basis insurers denied 19% of in-network claims in 2024.5
The first rate asks how often the first attempt fails, and the second asks how much is lost in the end. The danger lies in moving from one to the other. A report that showed 15% last year and 9% this year shows a fall of 40% in which no claim fared any better.
A fair count also waits. Claims denied last month have had little time to be appealed, so their final rate will look worse than that of claims a year old. Before a pilot starts, the hospital should therefore write down which claims are eligible, what the denominator is, how long each claim is followed, and how it counts partial payments and reversals.
A correction can split one claim in two
A correction can erase the history altogether. Return to the $900 claim, and suppose the correction changed its procedure code. The insurer usually gives a corrected claim a control number of its own, so the claim’s number can change when it is sent again.6 An analyst whose data lack the link between the two numbers may fall back on a key built from four fields. They are the patient, the provider, the date of service and the procedure code. The corrected claim now has a different key, so the analyst’s table holds two claims where the hospital sent one.
No dollar moved, but the history that would show a recovery is gone. KFF describes this effect in the federal data. When a provider files a new claim instead of resubmitting the original, “the original claim would be counted as denied, even if the new claim was ultimately paid”.5
Suppose that in the 1,000-claim example, half of the 60 recoveries involved a change to a field in the key. The analyst then sees 1,030 claims and 30 recoveries among 150 denials, so the recovery rate reads 20% when the truth is 40%. The error can also differ between the two groups an evaluation compares. If the tool’s corrections change codes more often than the staff’s do, the key hides more of the tool’s recoveries, and the bias runs against the vendor.
The repair belongs in the data, and the hospital claim already has a place for it. A replacement bill is marked as one by a digit in its bill type. A separate field carries the control number of the claim it replaces, and that number is the link.7 A claim sent as new carries neither.
The comparison must come before the tool chooses
The second need is a comparison, and it must be fixed before the tool chooses. A list of the claims a tool touched describes its workload. It cannot show the tool’s effect, because the claims on the list were selected. The tool may pick the denials most likely to be overturned, or the staff may send it easy claims and keep the hard ones. An insurer may fix an error of its own across hundreds of claims, and each will sit on the list as a recovery.
Many recoveries happen with no new tool. In claims from 2019 that covered 30% of the Medicare Advantage market, insurers denied 17% of first submissions, and 57% of those denials were later overturned.8 One health system told an HFMA reporter this year that it had been overturning 68% to 70% of the denials it appealed. With AI drafting part of some appeal letters, it put the rate at “probably 71% to 72% now”.9 Taken at face value, about 70 of every 100 appeals were already being won. The tool could then claim one to four appeals in each 100, far fewer than 71 or 72, and a before-and-after count cannot show even that.
The cleanest test borrows the clinical trial’s method. Before anyone works a batch of eligible denied claims, chance decides which go to AI-assisted work and which to usual work. Chance made the split, so the two groups hold the same mix of easy and hard claims, give or take luck. A later difference in collections can then be credited to the tool. This is random assignment.
Three rules keep the test honest. The first is to count every claim in the group it was assigned to, including claims the tool never acted on. A tool that staff ignore has failed in a way the hospital pays for, and dropping its claims hides the failure. Trials call this rule intention to treat. The second is to assign at the right moment. A tool meant to prevent denials must be tested among claims not yet sent, because a study limited to claims still denied has selected on the outcome the tool should change.
The third rule concerns the staff. If the same people work both groups, the time the tool saves on its claims can flow to the others. The usual-work group then does better than it would have, and the tool looks smaller than it is. The opposite error is also possible, because staff who know which claims are the tool’s may favor them. Where either is likely, chance should assign whole teams or sites, so that no one works both groups.
Here is such a test with example numbers. A hospital assigns 1,000 denied claims to each group at random and follows both for the same time. The AI-assisted group recovers 360 claims and the usual-work group recovers 300. To keep the arithmetic plain, every recovery pays $900 and none is reversed. The dashboard credits the tool with 360 × $900, or $324,000. The usual-work group collected $270,000, so the estimate of what the tool added is the difference, $54,000. The dashboard credits the tool with six times what it added. Suppose the tool’s fees and setup cost $20,000 and the staff time to review its output $10,000, so that $24,000 remains.
Even $54,000 is an estimate. The tool’s lead is 6 points, 36% of claims recovered against 30%. With 1,000 claims in each group, the luck of the split leaves a 95% range of about 4 points either way, or $17,000 to $91,000. The range excludes zero, so at the usual 95% confidence the tool helps. Its low end does not cover the $30,000 of costs, so the pilot has not yet shown that the tool pays for itself.
Real claims do not all pay $900. A few large ones may decide a pilot, so the range in dollars is wider than the range in claims. The chance split should therefore be made within bands of claim size, and the dollar difference reported with a range of its own.
Three kinds of value, each counted once
A tool can add value in three ways. The hospital may collect more in the end, collect the same money sooner, or spend less work collecting it.
Suppose a tool brings forward $1,000,000 that would otherwise have arrived 30 days later. The hospital has not earned another $1,000,000. It has gained 30 days’ use of the money. If the hospital pays 8% a year to borrow, an example rate, that use is worth $1,000,000 × 0.08 × 30/365, or about $6,575. Counting the same $1,000,000 as both new revenue and faster cash counts it twice. A pilot can make the same mistake. At 60 days the AI-assisted group may lead only because its claims were paid sooner, so the difference should be reported at 60 days and again after every claim has settled.
Speed is worth more than interest in some cases. Medicare gives a provider 120 days from receiving a claim decision to request the first appeal, and a late request can be dismissed.10 Past a deadline of that kind, later can mean never. A tool that gets an appeal filed in time, where the staff would have missed the date, earns the whole $900.
Work saved needs the same care. An hour that the tool frees becomes a cash saving only if the hospital pays for one hour less. If the hour goes to other claims and their revenue is counted, its wages cannot be counted as saved too.
Predicting a denial is not preventing one
A model that predicts which claims an insurer will deny has made the first of four links. Someone must then make a change that the record supports. The insurer must respond differently because of it. The gain must exceed the cost of the review. A high prediction score establishes the first link only.
The authors of one recent model say so in their discussion. They trained it on 678,881 claim lines sent to a single insurer, of which 31.9% were denied. Whether showing those predictions to billing staff lowers the denial rate, they write, “cannot be answered retrospectively”.11
A denial is also the insurer’s decision, and no finding that the bill was wrong. Federal inspectors sampled payment denials issued by 15 of the largest Medicare Advantage insurers in one week of June 2019. They estimated that 18% of the denied requests met Medicare’s coverage rules and the insurer’s own billing rules.12 A model trained on denials can therefore learn the insurer’s mistakes, and then steer staff to change bills that were right.
Pay for the difference
A contract that pays the vendor a percentage of every dollar received on a touched claim rewards contact. The tool then earns most by touching the claims most likely to be paid anyway. A fee tied to the estimated increment pays for what the hospital wants. Such a fee needs the estimate kept current after the pilot. A small fixed share of eligible claims, chosen by chance and always worked the usual way, does that. The contract can then pay on each year’s estimate, or on the low end of its range.
The contract should also name the record. Before signing, both sides should write down which claims enter the test, what the usual work is, how reversals and refunds are handled, and whether review time counts as a cost. Every event needs two dates, when it happened and when the hospital learned of it. Without the second, a March reversal of a January payment silently rewrites January, and a rerun report no longer matches the one the vendor was paid on. The hospital should be able to export the record and recompute the result, so that the evidence outlasts the contract.
The test has a price, and it should be named. If the tool works, every claim held in the usual-work group gives up the gain for as long as the test runs. In the example that is about $54,000 for each 1,000 claims held back. The step is reversible, since the hospital can send all claims to the tool on the day the test ends.
Money is not the only outcome to watch. A payment is no audit of the coding behind it, so a tool that raises collections by coding what the chart does not support has added risk. Fewer denials are also no gain if unsupported balances move to patients, and the group code shows when they do. An amount moved from the provider’s column to the patient’s has been billed to the patient, and nothing has been recovered.
The case for trusting the dashboard
A hospital losing money to denials cannot wait a year for a trial, and a vendor’s before-and-after numbers can be right. They are close to right when the tool works claims that no one worked before. If staff used to abandon every small denial, then nearly every dollar the tool recovers on small denials is new. An analysis of 234 million claims from 2019 notes that “many denied claims are never appealed”.13
The wait is shorter than it sounds. The tool starts on half of the eligible claims on the first day, so the hospital delays only half of whatever gain there is, and only until the claims settle. Size is the harder objection. The example needs 2,000 eligible denials, and skewed dollars need more, so a small hospital must pool many months or rely on the baseline check below.
A chance split of single claims is not the only sound design. A hospital can stagger the start across teams, sites or insurers, and compare each group’s change with the change in groups still waiting. That comparison of changes is called difference in differences. It rests on the assumption that the groups would have moved alike without the tool, which should be written down and tested against the months before the start. A plain before-and-after count remains the weakest design, because the insurer’s rules, the staff and the mix of claims all change while the tool is installed.
Check
The strongest point against this essay is that it has no evidence of the kind it asks for. No study found for this essay shows that dashboards overstate, and the sixfold gap comes from the example’s numbers. The published rates cut both ways. If 57% of denials were overturned in 2019, then 43% were not.8 A tool that recovers claims from that 43% earns close to what its dashboard says.
A search of PubMed and the open web on November 11 and 12, 2026, found no randomized or otherwise controlled test of AI in denial management with dollars collected as the outcome. The nearest studies concern other products. In a randomized crossover trial with 15 participants, an AI coding assistant sped the coding of long notes, with no significant change in accuracy and no payment data.14 At one health system, a difference in differences compared 698 physicians who chose an AI scribe with 867 who did not. Adopters’ weekly relative value units, the study’s measure of billed work, rose 1.81 more per physician than non-adopters’ did, and the share of claims with a denial did not differ.15 By the authors’ estimate, the gain was worth $3,044 a year per physician at 2025 Medicare rates. Such a search is no systematic review, and vendors may hold trials they have not published.
The essay ignores three things. Insurers can change how they decide, so a tool’s effect this year may not hold next year. The usual work is treated as fixed, though staff improve. The analyst’s time is not priced.
A hospital can run a check that could prove the essay wrong for its own case. Before looking at results, list the kinds of denial the tool now works, each defined by insurer and reason code. For those kinds, compute the share of denied claims that were paid within a fixed window, say 180 days, in the year before the tool and in its first year. If the earlier share is under about a tenth of the present one, the dashboard’s total is within a tenth of the tool’s contribution, and the essay overstates the problem there. If the earlier share is half the present one, the dashboard credits the tool with twice what it added. The check is itself a before-and-after count, so its answer is a first estimate, not a verdict.
I am confident of the logic, because a payment after a touch cannot show cause and each kind of value can be counted only once. I am much less sure of the size. I do not know how far dashboards overstate at a typical hospital, and it may be little where the tool works claims that were being abandoned. A published trial, or a hospital’s own baseline, showing touched-claim totals within a tenth of the measured increment would change what I advise about contracts. It would not change the question.
Sources
- TechTarget RevCycle Management, September 23, 2026. Insurers deny less just to claw back more later, analysis finds. Reports Kodiak Solutions’ revenue-cycle data: initial denial rate 10.63% in the first half of 2026 against 12.00% a year earlier, and the researchers’ account of post-payment activity. TechTarget gives 2,400 hospitals; Kodiak’s own release of September 15, 2026, says more than 2,300. Kodiak sells revenue-cycle software and services. techtarget.com ↑
- Experian Health, October 28, 2026. State of Claims 2026: a survey of 204 healthcare finance and billing leaders, fielded August 5–23, 2026. Experian Health sells claims software. PDF ↑
- Centers for Medicare & Medicaid Services. Health Care Payment and Remittance Advice: the 835, adjustments at line, claim and provider level, group codes, claim adjustment reason codes and provider-level balance codes. Retrieved November 12, 2026. cms.gov ↑
- HFMA Claim Integrity Task Force, 2022. Standardizing denial metrics for revenue cycle benchmarking and process improvement: the initial denial rate counts “only the first denial that was made for a given claim” and excludes rebills by bill type. hfma.org ↑
- Long, Lo & Pestaina, KFF, March 24, 2026. Claims denials and appeals in ACA Marketplace plans in 2024: an analysis of insurers’ reports to CMS, with its treatment of claims denied and later paid. kff.org ↑
- Centers for Medicare & Medicaid Services. PERM SC fast facts for claim adjustments: “ICNs of adjusted claims usually end in a number other than one and are tied to the original claim.” Describes state Medicaid claims data. Retrieved November 12, 2026. cms.gov ↑
- Mississippi Division of Medicaid, 2022. UB-04 Claim Form Version CMS-1450, section 5.8, UB-04 claim form instructions for institutional claims (Gainwell Technologies, version 1.0): bill-type frequency codes for interim, replacement and void claims, and Form Locator 64, “Enter the original claim ICN that is being requested to be replaced/adjusted or voided.” One state program’s instructions for the national hospital claim form. PDF ↑
- Vabson, Hicks & Chernew, Health Affairs 44(6):702–706, June 2025. Medicare Advantage denies 17 percent of initial claims; most denials are reversed, but provider payouts dip 7 percent. Claims from plans covering 30% of the Medicare Advantage market in 2019. doi.org ↑
- Williams, Healthcare Financial Management Association, January 30, 2026, updated February 6, 2026. Battle of the bots intensifies over denials: a health system’s own account of its appeal overturn rate, with no comparison group described. hfma.org ↑
- Centers for Medicare & Medicaid Services. First level of appeal: redetermination by a Medicare contractor. A party “has 120 days from the date of receipt of the initial claim determination to file a redetermination request.” Retrieved November 12, 2026. cms.gov ↑
- Jadoon et al., Frontiers in Health Services 6:1927175, October 14, 2026. A machine learning framework for predicting healthcare insurance claim denials: retrospective, 678,881 claim lines, one insurer. PMC ↑
- U.S. Department of Health and Human Services, Office of Inspector General, April 2022. Some Medicare Advantage organization denials of prior authorization requests raise concerns about beneficiary access to medically necessary care, OEI-09-18-00260: a stratified random sample of 250 prior authorization denials and 250 payment denials, June 1–7, 2019. PDF ↑
- Weinreb & Landon, Health Affairs Scholar 4(8):qxag206, August 2026. Variation in medical claim denials: safety-net providers are hardest hit. 234.2 million claims from 2019 across Medicare Advantage, Medicaid and commercial plans. PMC ↑
- Chomutare et al., Journal of Medical Internet Research 27:e71904, July 3, 2025. Artificial intelligence to improve clinical coding practice in Scandinavia: crossover randomized controlled trial. 15 participants and 300 coded notes analyzed; the tool’s developers are among the authors. PMC ↑
- Holmgren et al., JAMA Network Open 9(1):e2553233, 2026. Ambient artificial intelligence scribes and physician financial productivity: a cohort study at one health system, 1,565 physicians, January 2023 to April 2025, difference in differences. PMC ↑