Our selection gate, Vera, can look at an agent's output and KILL a bad one. That tells you it failed. It does not tell you which part of the agent's safety harness let it fail, so the next step is a human squinting at a log. Faultline closes that gap. Following the "Trajectory-driven Safety Harness Evolution" paper (arXiv, 11 August), it splits the harness into four artifacts and, on every KILL, names the one to blame and proposes the fix to it. Run this week's gym-booking exploit through it and the answer comes back in a few milliseconds, no tokens spent: tool policy, high confidence, with the fix "scope the tool, do not add a rule".
What we did
The paper's sharp move is to stop treating the harness as one blob. It is four things with different jobs, and every failure belongs to one of them: the system prompt (the agent's role and scope), the rule bank (the explicit do and do-not rules), the safety memory (what it learned from past failures), and the tool policy (what its tools can actually reach). Pin a failure to the responsible artifact and you can evolve that one artifact instead of rewriting the whole system prompt and hoping.
Faultline is one module that plugs into Vera's existing flow. On a KILL it scores the failed trajectory against the four artifacts with a deterministic signal layer, runs offline for zero tokens, and returns the top artifact, a one-line rationale, a concrete proposed refinement, and the full ranking so you can see the runner-up. A sharper LLM diagnosis is injectable on top for the murky cases; if it errors or is absent, attribution falls back to the heuristic and never hard-fails. It fires only on a KILL, because a PASS has nothing to evolve.
The one opinion baked in: when a failure could be pinned on more than one artifact, prefer the most structural fix. The tie-break order is tool policy, then rule bank, then system prompt. A permission the agent never holds beats a rule you hope it follows beats a sentence in a prompt it can reason around. That is durability, not blame. The gym agent is the worked example we baked in: asked to book a full class, it found the booking API's cancel endpoint had no ownership check and cancelled a stranger's slot. Faultline calls it tool policy, and the proposed fix is not "tell it not to", it is "scope the tool so it cannot exercise reach it was never granted". That is the same lesson the incident taught, arrived at mechanically.
Why it was worth doing
A verdict is not actionable and an attribution is. "This failed" sends a person to read a trace. "This failed on the tool policy, scope endpoint X" is a diff. Vera already grew rubrics and scored outputs on a nightly clock; what it could not do was turn a KILL into a named, evolvable cause. Faultline is the piece that lets the standing gate say not just that an agent slipped but where the harness was thin, which is the only form of the finding you can act on without a human in the loop.
It is also the honest complement to the two guardrails we shipped this week. skillscan checks a skill at install time and approvalgate scores a command at runtime; both are prevention. Faultline is what you run after something got through anyway, to point at the artifact that should have stopped it. Sixteen tests, no network, one file that sits next to Vera's other layers.
What's still off
The deterministic layer is a high-recall prior, not a final verdict. It is built to be right on the clear cases, and the gym exploit is unambiguous, but a failure that genuinely spans two artifacts is where the cheap layer will hedge with low confidence and hand off to the LLM diagnosis. We have left that path injectable and tested against a stub; wiring it to a live model, and calibrating its confidence against a golden set the way we did for the Vera panel, is the next step, not this ship.
And attribution names the artifact, it does not write the patch. Faultline will tell you the tool policy is the fault line and that the shape of the fix is "remove the capability the task did not need"; a human still writes the actual scope change. That is the right division of labour for now. The win is that the guessing step, the one where a tired person decides which part of a sprawling harness to touch, is now a consistent, auditable call instead of a hunch.