We Rebuilt a Research Paper's Self-Tuning Loop in 110 Lines

§1The same idea, twice, in a heavy coat

Two research groups shipped the same good idea this year. In June, Cisco's Foundation AI team put out FAPO, a system that optimises an LLM pipeline by attributing its failures and rewriting the prompts that caused them. This week a second paper, AutoSaddler, does the same job from a slightly different angle: diagnose the failure traces, generate structured patches to the agent harness, keep the ones that generalise. Both report real gains. FAPO beats the prior best method (GEPA) on fifteen of eighteen benchmark comparisons. AutoSaddler adds nine to ten points on GAIA2, SWE-Bench Pro and Terminal-Bench. The idea is sound and it is arriving from more than one direction at once, which is usually the sign that it is real.

Both also arrive wrapped in a framework. FAPO ships as a full suite: LangGraph pipelines, an agent (Claude Code, or Codex) driving the optimisation, tenant directories, an evaluation CLI, and, in the deploy folder, optional Kubernetes scripts and a Google Cloud Storage sync for datasets. That is a lot of apparatus. The temptation, when you see the apparatus, is to conclude you need it.

§2What the loop actually is

Strip the coat off and the idea is small. Every one of these systems, underneath, is the same five-step loop:

• Run the current prompt over a dataset and score it.
• Look at the cases that failed and name the single dominant reason they failed.
• Propose a targeted rewrite that addresses that reason.
• Score the rewrite. Keep it only if it beats what you had.
• Repeat, up to a budget, and stop when you hit the target.

That is the whole engine. None of those five steps needs a cluster. Kubernetes and cloud storage are how you would run this at scale across many tenants on someone else's infrastructure. They are the delivery rig, not the idea. To be fair to FAPO, and this is the correction worth making out loud: it does not actually require Kubernetes to run. It runs locally on Python. The lazy version of this note would be "the paper needs a cluster, we didn't". That is not true. The truer, more useful point is quieter. The framework around the loop is large, the loop itself is tiny, and you can take the second without adopting the first.

§3So we built just the loop

We wrote the loop as a single Python module, about 110 lines. No LangGraph, no external agent orchestrating it, no deploy folder. Every model call it makes goes through our own router, so it runs on one box, it is cheap, and it can point at a local open-weight model instead of a paid API if we want it to. There is no cloud dependency and nothing to lock into. It optimises at the prompt level, which is the first and highest-return tier of what FAPO does. The parameter and pipeline-structure tiers are deliberately left for later, because the prompt tier alone earns its keep.

The proof is a demo you can run in a few seconds. We hand it a deliberately bad system prompt, one that invites the model to explain itself, against a set of yes-or-no questions scored on exact match. A chatty answer fails the check. Here is the actual run:

One round. The loop looked at the failures, correctly named the cause in a sentence, wrote a prompt that fixed it, checked that the fix actually scored better before keeping it, and stopped. That last part is the bit that matters and the bit people skip: it keeps a change only if the change is measured to be an improvement. No vibes, no "that reads better to me". A number went up, or the change was thrown away.

§4The honest limits

This is not the paper, and it should not be sold as the paper. Our version does the prompt tier only. It does not yet touch model parameters or restructure the pipeline, which is exactly where FAPO reports its largest jumps, on the harder benchmarks. The demo runs on a toy dataset with an exact-match scorer, chosen because it makes the mechanism legible in one screen, not because it is a hard task. And the loop is only ever as good as the two things you feed it: the dataset it optimises against and the scorer that decides what "better" means. Give it a weak scorer and it will happily optimise towards the wrong thing, quickly and confidently. None of that is a flaw in the method. It is the method telling you where the real work is, which is in the evaluation, not the optimiser.

What you get in exchange for those limits is worth naming too. The loop is ours. It runs where our other agents run, it bills through the same router, it can fall back to a local model, and there is no version of it that stops working because a framework moved or a cloud bill came due. For a small shop that is not a compromise, it is the point.

§5What to take

The lesson outlives these two papers. When a piece of research lands wearing a framework, separate the idea from the delivery before you decide what to adopt. Ask the plain question: which part of this is the insight, and which part is the machinery for running the insight at someone else's scale? Most of the time the insight is small enough to hold in your head and rebuild in an afternoon, and the machinery is there to solve a problem you do not have yet. Copy the loop. Leave the cluster on the shelf until the day you actually need it.

We did not do anything clever here. We read what the loop was, wrote the loop, and ran it until a number moved. The discipline was in refusing to import the rest. That is usually where the leverage is: not in the framework you adopt, but in the ninety per cent of it you decline.