The number landed quietly in an ACCA report and most finance leaders scrolled past it. Global businesses lost an estimated US$67 billion in 2024 to AI hallucinations – outputs that looked credible, read confidently, and were simply wrong. That figure is not a projection. It reflects decisions already made on bad data, filings already submitted, contracts already signed.
And it is not a technology problem that IT can own. It is a finance problem.
If your team is using AI-assisted tools to draft reports, summarise data, flag anomalies, or prepare management accounts – and in 2026, most are, in some form – the question is not whether hallucination risk exists in your workflows. The question is whether your review controls are actually designed to catch it.
What a hallucination looks like in practice
The term sounds abstract until you see it. An AI tool summarises a dataset and invents a variance that does not exist. It cites a regulation that was amended eighteen months ago. It produces a cash flow narrative that is internally coherent but built on a figure pulled from the wrong period. The output looks clean. The formatting is professional. Nothing flags as obviously wrong – because hallucinations do not flag themselves.
This year, EY and KPMG were among the firms publicly caught out by unverified AI outputs in professional work. These are organisations with significant compliance infrastructure and technically sophisticated teams. The issue was not that people were careless. The issue was that the review process was not rebuilt around what AI actually does wrong – which is different from what humans do wrong.
Humans make errors through distraction, fatigue, or gaps in knowledge. AI makes errors with complete confidence, at scale, in polished prose. That distinction matters when you are designing a sign-off process.
Why finance teams are particularly exposed
Finance outputs carry weight in ways that other business functions often do not. A hallucinated figure in a board report does not stay in the report – it informs a decision. A misquoted covenant in a lender summary gets tested the moment you go back to the bank. An error in a statutory filing has consequences that do not disappear when you correct it.
Finance teams also tend to move fast under real pressure. Month-end closes, audit prep, investor reporting – these are not environments where people have long stretches of quiet time to interrogate every AI-generated paragraph. They are environments where a plausible-looking output gets signed off because the deadline is tomorrow and the numbers feel about right.
That combination – high-stakes outputs, compressed timelines, and a tool that produces errors confidently – is where the risk concentrates.
The controls gap is a people and process problem, not just a tech problem
Most businesses that have adopted AI tools have not meaningfully updated the human review layer around them. They have added capability without adding corresponding scrutiny. The assumption, often unstated, is that AI catches errors rather than introduces them.
Good review controls for AI-assisted finance work look different from traditional review. They require someone who understands what the AI tool is likely to get wrong, not just someone who can check arithmetic. They require source verification as a step in the process, not an afterthought. They require a culture where questioning an AI output carries no more friction than questioning a junior team member’s work – which means it has to be genuinely acceptable to slow down and push back.
That last point is harder than it sounds. In teams where AI adoption has been positioned as an efficiency gain, there can be subtle pressure not to undermine that narrative by raising concerns too often.
What this means for how you build your finance team
If your current team does not have the bandwidth or the specific capability to run meaningful review on AI-generated outputs, that is a resourcing question as much as a process question. Technical rigour – the kind that comes through years of working in a structured, supervised environment like practice – is exactly what catches the errors that polish conceals.
A qualified accountant who has spent three or four years in a mid-tier or Big Four firm has been trained, often without realising it, to be suspicious of outputs that look too clean. They have been taught to trace a figure back to its source before signing off. That instinct is not automatic. It is learned. And it is increasingly valuable in teams where AI is doing more of the drafting.
The firms that will navigate this well are the ones that treat AI as a capable first-draft tool operating inside a robust human review framework – not a replacement for the judgement that comes from genuine technical training.
Before the next reporting cycle, it is worth asking one honest question: if your AI tool produced a confident, well-formatted hallucination in last month’s management accounts, would your current process have caught it?
If the answer is uncertain, that is where to start.
