How do you stop a self-improving agent from breaking things it already got right?

Add a regression guard. After the loop proposes a rewrite to fix the failing cases, re-run that rewrite against every case that previously passed, using the same executor that produced the log. If any previously-passing case now fails, refuse to bless the rewrite and list the regressions for a human. A fix is only safe if nothing that already worked breaks. In our demo the guard caught three regressions that a plausible but naive rewrite would otherwise have shipped.

Why does a rewrite that fixes the failures still cause regressions?

Because a rewrite drafted only from the failing cases can overfit to their shape. In our demo the failures were all postcodes with a four-character outward code, so a naive rule, put the space after the fourth character, fixed every failure it was shown and quietly broke the short codes it never saw fail. The failing cases tell you what to fix; they do not tell you what you might break. Only the passing cases can do that, which is why the guard re-runs them.

The fix passed the failures and broke three that worked.

Earlier today we shipped a loop that reads an agent's own failures and proposes a rewrite of the instruction that caused them. We also named its one real weakness out loud: it never checked that the rewrite left the already-passing cases alone. This is the fix. It re-runs every case that already worked, and it just caught three that a plausible rewrite would have quietly broken.

The trap

The loop only ever sees the failures, because those are what a failure log contains. So the rewrite it drafts is shaped entirely by what went wrong, which is exactly how you overfit. In the demo, every failing postcode happened to have a four-character outward code. A perfectly reasonable rewrite, "put the space after the fourth character", fixes all of those. It also breaks every postcode with a shorter code, none of which were in the failure log, because they were passing fine. The fix is correct about the cases it saw and wrong about the cases it did not. The failing cases tell you what to fix. They cannot tell you what you are about to break.

What we built

A regression guard, wired in after the proposal and before anything is blessed. It takes the cases that already passed, runs the proposed instruction against every one of them with the real executor, the same thing that produced the log in the first place, and collects any that no longer match. If the list is empty, the proposal is stamped SAFE. If it is not, the proposal is stamped NEEDS REVISION and the exact regressions are written out: which input, what it should still return, what the rewrite now gives instead. Nothing is applied either way, because the whole tool still only writes proposals for a human to accept. The guard just makes the human's yes or no an informed one.

The demo runs both sides so you can see it work. A vague starting instruction half-fails, seven of eleven. The loop proposes a proper fix from the four failures, and the guard confirms it breaks none of the seven that worked: SAFE. Then, for contrast, we hand the guard the naive "fourth character" rewrite, and it refuses it, naming the three short codes it would have broken. Same loop, same passing set, two rewrites, and the guard is the thing that tells them apart.

Why it matters

This is the difference between a self-improving system you can leave running and one you cannot. Without the guard, every automated fix is a coin flip: it might clear the failures and hold the line, or it might clear the failures and silently regress something nobody was watching, and you would not find out until that case came round again in production. Regression is the quiet failure mode of every optimiser that only looks at what is currently broken. The guard turns "this fixes the failures" into "this fixes the failures and breaks nothing that worked", which is the only version of a fix worth shipping.

What's still off

The honest limits. The guard is only as good as your record of what passed: if a case was passing by luck rather than because the instruction was right, a rewrite that breaks it will be flagged as a regression when arguably it should not be, and that is a judgement only a human reviewing the proposal can make. It also verifies against the cases you have, so a class of input that is in neither your passing nor your failing set is still invisible to it, same as it is to you. And it costs a re-run of the passing set on every proposal, which is cheap here and would not be for a slow or expensive executor, so at scale you would sample rather than run all of them. It is a guard, not a proof. It stops the regressions you can see, which is most of them, and it is honest about the ones you cannot.

What's now in the stack

A regression guard on the self-improving loop: after a rewrite is proposed, re-run it against every previously-passing case with the real executor and refuse to bless it if any regress.
A proposal that comes out stamped SAFE or NEEDS REVISION, with each regression named: the input, the answer that should still hold, and what the rewrite gives instead. Still a proposal, still gated on a human.
A live demo showing the guard passing the loop's real fix and refusing a plausible over-correction that broke three passing cases, plus tests, dependency-free, on GitHub.