
I recently finished a manual audit for a client: 300 claim files. To get through them, we put five reviewers on the project, each reviewing 7 to 10 files a day for a full week. They did nothing else. Even then, we needed more review and QC passes at the end to make sure the scoring was consistent from one reviewer to the next.
Start to finish, it took about two weeks to get the client their information. And that was a perfect world.
Most companies don’t have five reviewers who can drop everything. The reviewer usually has a desk full of other work, so an audit that size gets spread out over weeks, sometimes months. By the time the findings land, the adjusters have handled another few hundred files the same way.
AI could have done that same review, against the same standard, and produced a report in under an hour. The reason I’m confident in that isn’t the one most people assume.
What an auditor actually does
When I audit a file, I’m not discovering anything new about claims handling. I bring what I already know: key words, themes, years of experience, and a clear picture of what best practice looks like for that line of business and jurisdiction. Then I read the notes and ask one question over and over: did this handler do what should have been done, when it should have been done?
Strip that down and it’s a comparison. My standard on one side, the file on the other. The hard part, and the part that takes real expertise, is building the standard. Once that base is set correctly, the work is checking it against what’s in the notes. That’s where all the hours go.
Machines are good at that kind of comparison. The AI is not the source of the information. The auditor is. The AI adds speed to what a qualified human would do anyway. Models are most reliable when they work from reference material you hand them, not from whatever they picked up in training. In an audit, the auditor’s standard is that reference material.
You can see the same pattern in legal work. In a 2024 study comparing AI to lawyers on contract review, the leading models matched or slightly beat junior lawyers at identifying the issues. They took under five minutes per contract. The junior lawyers averaged 56. Contract review against a playbook isn’t claims audit, but the structure is the same.
Three things this changes
I’ll take each of these up in its own post over the next few weeks. For now, here’s the short version.
It’s the right place to start with AI
A lot of carriers and TPAs are nervous about putting AI anywhere near claims, and that’s fair. Regulators are paying attention: about half the states have now adopted the NAIC’s model bulletin on insurers’ use of AI. But audit is a natural first step. The AI isn’t deciding claims or paying anyone. It reads files against your standard and flags what it finds, and a human reviews the findings before anything goes in a report. If you want to test AI in claims, start here.
Audit stops being a sample
We sample files because reading them has always been expensive. When reading takes minutes instead of weeks, you can look at the whole book, review open files while there’s still time to fix them, and spot patterns by adjuster, office, or TPA that a small sample would never show.
The missing document tells the story

People often ask whether the AI needs the claim documents, not just the notes. Documents are valuable for confirming what the adjuster wrote, or for independently agreeing or disagreeing with their conclusions. In my experience, though, if the documents are in the file and the notes show the handler never reacted to them, the documents don’t help. Missing or poor documentation is one of the most consistent markers of bad handling I see. The NAIC’s model regulation requires claim files to contain enough documentation “to permit reconstruction of the insurer’s activities relative to each claim.” The documents matter, but what counts is whether the handler acted on them, and only the notes show that. Documents tell you what the adjuster knew. Notes tell you what the adjuster did, and audit is about what they did.
Documents tell you what the adjuster knew. Notes tell you what the adjuster did.
Back to the 300 files
The expertise in that audit didn’t come from the five reviewers reading faster or slower. It came from the standard we built and the judgment applied to what we found. Of the two weeks, most went to reading, the part a machine does best.
That’s the idea behind The Audit Portal. When I built it, I made one decision early: it doesn’t run on my standards and beliefs. It runs on the client’s. Every carrier and TPA has its own best practices, guidelines and reporting requirements, and those are what its adjusters are held to. So the client’s standard goes in first, and every file is measured against it. The judgment stays with the people who own the book.
Put the client’s standard in an AI review, keep a qualified person reviewing the findings, and you get the same expertise in a fraction of the time, applied the same way to file 1 and file 300.
The AI never knew more than I did. It just read faster than I ever could.
Next up: why audit is the safest and smartest place to start with AI in claims.