§1The thing that actually happened
Courts across multiple jurisdictions have stopped treating AI-fabricated citations as an embarrassing one-off and started treating them as sanctionable conduct. Lawyers have filed briefs containing cases that do not exist, statutes that were never enacted, and quotes attributed to judges who never wrote them. The model produced something that read like law. Nobody checked whether it was law. A judge did, and the result was financial penalties, referrals to bar associations, and public naming.
The lazy read is that the AI lied. It did not. A language model has no concept of a real case versus an invented one. It produces text that is statistically shaped like a citation. Smith v Jones [2019] EWCA Civ 1422 is generated the same way as a genuine reference, because to the model they are the same kind of object: a plausible string. The fabrication is not a bug in the model. It is the model working exactly as designed, deployed into a context that demands something the model cannot provide on its own.
§2What the context actually demands
A legal brief is not prose. It is a load-bearing structure where every citation is a claim about reality: this authority exists, it says what I claim it says, and it is binding in this jurisdiction. The reader is a judge whose job is to test those claims. The cost of a false claim is not a bad sentence. It is a struck pleading, a costs order, and a regulator asking why you signed it.
So the question is never whether the model hallucinates. Of course it does. The question is what sits between the model's output and the judge's desk. In the sanctioned cases the answer was nothing. The lawyer read the draft, it looked right, and they filed it. That gap, the absence of anything that resolves each citation to a real retrievable source before it leaves the building, is the entire failure. The model is downstream of the problem. The missing verifier is the problem.
§3Tool-grounded claims, not confident text
The fix is structural and it is boring, which is why people skip it. In any regulated context where an agent composes a submission from retrieved authorities, every factual claim must be a tool-grounded claim. Not a claim the model is confident about. A claim backed by a tool-call receipt that proves the authority exists in the jurisdiction's database.
Concretely: the agent does not write Smith v Jones [2019] EWCA Civ 1422 and move on. It calls a retrieval tool against a real case law database. If the case resolves, the citation is allowed into the draft carrying a receipt. If it does not resolve, the citation cannot enter the draft at all. The model is not trusted to know whether a case exists. It is only trusted to ask, and the tool is trusted to answer.
This turns the agent from a confabulating intern into an auditable research assistant. The difference is not intelligence. It is that the second one cannot make a claim it has not checked, because the architecture forbids it. The receipt is the deliverable, not the prose.
§4The pre-send verifier is the boundary that was missing
The single highest-leverage component here is a pre-send verifier. Before any submission is allowed to leave the system, a separate pass walks every legal reference in the document and confirms each one resolves to a real, retrievable source. No receipt, no send. The verifier does not care how confident the drafting agent was. It cares whether the citation survives a fresh lookup.
This is the same discipline I run across an eight-agent fleet at Workloft. One agent drafts, a different mechanism checks, and the check has authority to block. You do not let the thing that produced the claim also be the thing that approves the claim. That is marking your own homework, and the sanctioned lawyers were the marking-their-own-homework layer. The model drafted, the human glanced, and the glance was not a verification. It was a vibe check.
The reason this matters beyond law is that the same shape recurs everywhere a UK regulated buyer operates. A medical summary citing a guideline that does not exist. A financial report referencing a regulation that was repealed. A compliance filing quoting a standard that says the opposite. In every case the model will happily produce the false claim, and in every case the only defence is a verifier that resolves the claim against ground truth before it ships.
§5What builders get wrong
The common mistake is treating hallucination as a model-quality problem to be solved by a better model or a sterner prompt. You write only cite real cases in the system prompt and feel safer. You are not safer. The model still cannot distinguish a real case from an invented one, and the instruction changes nothing about its capability. You have added a comforting sentence to a system that has no idea whether it is obeying it.
The second mistake is putting a human in the loop and calling that verification. A human reviewer skimming a forty-page brief is not a verifier. They are a slower, more expensive, equally unreliable version of the same vibe check that got lawyers sanctioned. Verification means resolving each claim against a source, mechanically, every time, with the power to block. If it can be skipped under deadline pressure, it will be, and that is exactly when the fabricated citation slips through.
The third mistake is building the verifier as advisory. A warning that says this citation could not be confirmed is ignored under pressure. The verifier must hold the send. The boundary is the point. An agent that can confabulate is fine. An agent that can confabulate and then file is a liability, and the courts have now put a number on it.