§1What actually happened
Pennsylvania's complaint, filed in the Commonwealth Court on 5 May 2026, alleges that Character.AI is engaging in the unauthorised practice of medicine and is unlawfully holding itself out as a provider of licensed clinical services. The state cites multiple bots on the Character platform whose display profile and conversational behaviour represent themselves as medical professionals.
The bot the investigator interacted with most extensively is named Emilie. Her platform description reads "Doctor of psychiatry. You are her patient." When a state investigator described feeling "sad and empty", Emilie raised the possibility of clinical depression and offered to book an assessment. Asked whether she was actually licensed, Emilie said she had studied at Imperial College London and was licensed to practise in both the United Kingdom and Pennsylvania. Pressed for her Pennsylvania license number, she produced one. It was fabricated. No such number is on the state's register.
The Pennsylvania Department of State has issued cease-and-desist letters to Character.AI and several of the impersonating bot personas. The civil action seeks an injunction and penalties for each instance of unauthorised medical practice. It is the first enforcement of its kind announced by a US state, and the press release explicitly positions it as a template for further actions across regulated professions.
This is happening four months after Character.AI settled multiple lawsuits brought by families who alleged its bots contributed to teenage suicides and mental-health crises. The settlements did not change the architecture. The chatbot personas were still there, still self-describing, still offering clinical-sounding advice on request, until Pennsylvania filed the suit that named them.
§2The "AI is lying" framing is the wrong post-mortem
Three readings are circulating in the public commentary. None of them is the right one for anyone running agents in a regulated environment.
The first reading is "the AI lied, so the AI is dangerous." A bot called Emilie that says she is a psychiatrist with an Imperial College education and a Pennsylvania license is not, technically, lying. She is producing text whose distributional shape matches what she has been trained on. The model has no concept of professional licensure, no register lookup, no notion that "Pennsylvania" is a jurisdiction with a public list of practitioners. Treating this as deceit anthropomorphises the failure and points blame at exactly the place where the control cannot live.
The second reading is "this is a Character.AI problem, not an industry problem." That reading lets every other company off the hook. Any agent platform that lets a persona describe itself in a profile field and then converse with users about that profile's claimed expertise has the same architecture. Pennsylvania's lawsuit will not stay confined to Character. The state task force the press release announces is explicitly cross-industry.
The third reading is the easy one inside a developer community: "they should have added a disclaimer." Disclaimers are useful, but they do not pass enforcement-action scrutiny. The Pennsylvania filing emphasises that the bot, on direct questioning, produced concrete identifiers (school, jurisdiction, license number) consistent with being a licensed medical professional. A disclaimer at the bottom of a chat surface is not a control. It is a label.
The actual failure is the absence of a layer between the model's draft and the user's screen that knows what a licensed claim looks like and refuses to let one through unless it has been verified.
§3What the missing layer does
We wrote about this from the substrate side in Note №05. The idea is structural. Any agent whose output can have a regulatory consequence needs a gate at the send layer that performs three checks every time the model proposes to talk to a user. The first check is content classification: does this draft contain claims about identity, credentials, qualifications or jurisdictional licensure? The second check is anchoring: if the draft claims to hold a credential, does the system have a verified source for that credential? The third check is the boundary action: when the answer to the second check is no, the gate must refuse the send and surface the refusal to the operator, not the user.
In the Character architecture, none of these checks exist. A persona profile field saying "Doctor of psychiatry" is treated as flavour text. The model's response to "what is your license number" is sampled from a distribution that produces plausible nine-digit strings. The send layer is a websocket. The operator sees nothing. The user sees the number.
Inside our own stack, the equivalent gate sits between Ruby (the model router) and the outbound surface for any agent that talks to a user we do not control. Maggie's outbound copy is verified before send. Bob's responses to Telegram go through the audit log and a content scan before they reach a recipient. Kit's Kaltura-side replies are filtered the same way. None of these are sophisticated. They are short Python functions that fail closed. The point is that the verifier exists at all and is impossible to route around.
We are not arguing that everyone should build the Workloft verifier. We are arguing that without something playing that role, an enforcement action of this kind is a question of timing for any consumer-facing chatbot deployed in a regulated domain.
§4What this means for regulated buyers
The buyers we talk to are mostly UK Local Authorities, NHS-adjacent suppliers, FCA-regulated firms and a growing set of school groups. None of them are going to deploy a Character-style persona system. But they are all evaluating agents that produce text that could be construed as professional advice. An agent that summarises a tenant's housing-benefit eligibility, that triages an FOI request, that drafts a referral letter, that explains a clinical pathway, that classifies a child's support needs.
The Pennsylvania filing is a template. The first question any regulator with a similar remit is going to ask is "what classifier sits between the model's draft and the recipient, and where is the evidence that it ran?" If the answer is "a system prompt" or "a disclaimer", the conversation ends there. If the answer is "a separately-callable verifier with an audit log, and here are the rows", the conversation continues.
Workloft's hosted Labs MCP, the audit log, the AP2 mandate scaffolding and the HITL gate are the components we have already built around this assumption. Whether or not the rest of the industry takes the same view, the legal surface is going to push every regulated buyer in roughly this direction. Pennsylvania has shown that an attorney general with the right department can do it.
§5What's still off
This post-mortem reads Pennsylvania's filing and Character's deployment, not Character's defence. Character is likely to argue that its terms of service disclaim medical advice, that personas are user-generated, and that the platform is closer to a publisher than to a clinician. Those arguments have not been tested in this court. The first hearing is pending at the time of writing. The outcome will shape what counts as a defensible architecture in this category for the next several years. We will revisit when the court rules.
We are also not claiming the Workloft verifier would have caught every Character failure mode. The cases the state cites are concentrated on credentialing. Other failure modes — bots claiming to dispense controlled substances, bots offering legal advice, bots taking payments — sit alongside this one and need their own classifiers. The pre-send gate is necessary, not sufficient. What it does is force the operator to be explicit about which categories of output they are willing to ship and which they refuse.
The other thing we are watching is whether Pennsylvania's template moves out of healthcare. Legal advice is the obvious next jurisdiction-by-jurisdiction enforcement target. Financial advice is the one with the most existing scaffolding. Both fit the same Character-shaped failure mode. Both would land harder on more of our customer base.