We gave a deliberately weak agent fifteen lines of text and asked it to pull the postcode out of each one. It scored zero. Then, with the same model and the same fifteen inputs, it scored fifteen. The only thing that changed was one rewritten instruction, and a loop wrote the rewrite by reading its own failures. The point of the build is not the fifteen. It is that the model never moved.
The thing three papers said in one week
In the space of a few days, a run of separate research landed on the same claim from different directions: the wrapper around a model matters more than the model. The wrapper is the boring part, the instruction and the retrieval and the output contract, and the finding is that the same model in a better wrapper can swing from useless to reliable. If that is true, the obvious next move is to close the loop and let an agent read its own failures and rewrite the instruction that caused them. The interesting question is not whether you can. It is where you stop.
What we built
One small tool, one stdlib file. It reads an agent's failure log, one run per line: the instruction that ran, the input, what was expected, what came back, and whether it passed. It ranks the instructions by how much of the failure is concentrated on each one, and it will not blame an instruction that has only run a couple of times, so one unlucky call cannot top the chart. It picks the culprit, gathers the actual failing traces, and asks the strong model to rewrite the instruction, showing it the traces and not just a score. That last part is the whole idea from the research made concrete: you fix a wrapper by looking at what it produced, not at a number.
The demo runs it live. The weak actor is Claude Haiku. The starting instruction is the kind of thing you would actually write in a hurry: "Find the postcode in the text." Haiku, being a capable model, found every postcode correctly. It just handed them back wrapped in a sentence, "The postcode in the text is SW1A 2AA", or without the canonical spacing, so an exact-match check scored all fifteen as wrong. The loop read those fifteen failures, noticed the pattern, and proposed a new instruction that asked for the bare postcode in canonical form and nothing else. Same model, run again: fifteen out of fifteen. The failures were never a knowledge problem. The model knew. The wrapper was throwing the answer away.
The design decision that matters
The loop can read your instructions. It can only write to one place, a proposals directory, and it refuses any path outside it. It never touches the live instruction. A human reads the proposal and accepts it, or does not.
This is the deliberate line, and it is worth saying plainly. An instruction the agent can silently rewrite is not an instruction, it is a suggestion. If a self-improving loop could edit its own live guardrails, the first bad rewrite is permanent and the loop has no floor. It is also a security hole, not a hypothetical one: a persistent store an agent writes to is a live attack surface, and context poisoning is exactly the trick of planting a bad instruction now for the agent to execute later. So the third step of this self-improving loop is a person, on purpose. The agent proposes; a human applies. This is the same rule we shipped a fortnight ago in a different shape: a control that lives somewhere the agent can write to is not a control.
Why it matters
Most teams reach for a bigger model when an agent underperforms, because the model is the visible, expensive thing and swapping it feels like progress. Very often the model was never the problem. The instruction was underspecified, the output contract was loose, the retrieval fed it the wrong slice. Those are all cheap to fix and invisible until you look at the failing traces, which almost nobody does, because failures get counted and then discarded. A loop that reads the traces back and drafts the fix turns a discarded failure log into the one artefact that actually improves the system. And gating it means you get the compounding benefit of self-improvement without handing the agent a pen and leaving the room.
What's still off
This is the honest part. The tool localises one culprit and proposes one rewrite; it does not yet loop until dry across many instructions, and it does not check that a fix for these failures leaves the already-passing cases alone, which is the obvious next feature and the one that stops a rewrite trading one failure mode for another. The proposer is a live model call, so a proposal is only as good as the traces you feed it: a thin or unrepresentative failure log produces a confident, narrow rewrite. And a postcode with an exact-match check is a clean demo precisely because the right answer is unambiguous. Real tasks have fuzzier scoring, and a loop is only ever as trustworthy as the signal that says a run failed. The gate is doing real work here: it means a bad proposal costs you a rejected file, not a poisoned instruction.
What's now in the stack
- A self-improving harness loop, built and tested, that reads an agent's failure log, localises the instruction most tied to the failures, and proposes a rewrite grounded in the failing traces, not just the scores.
- A hard gate: the loop writes proposals to one directory and refuses any path outside it, so it can never edit a live instruction. The agent proposes, a human applies.
- A live end-to-end demo (0 of 15 to 15 of 15, same model) and a stdlib test suite, dependency-free, on GitHub.