Re-sending everything cost our agent loop 7.4x the tokens.

A new paper, SoL-Pi, claims four harness tricks cut a coding agent's token traffic by about half. We reimplemented all four, ran them over a real trajectory with a real tokenizer, and measured our own number. Against a loop that re-sends everything, the mechanisms cut input tokens by 86.5%, a 7.4x drop. One of the four did most of the work, and none of them lost a byte of evidence.

The bill is the transcript, re-sent

A chat model is stateless between calls, so an agent re-sends its whole transcript on every turn. That means a large tool output is not paid for once. It is paid for again on every request after it appears. Read a 24 KiB file on turn three of a twelve-turn task and you have not spent it once, you have spent it nine more times. The real bill is cumulative input tokens, the sum over every request of the context size, and it grows fast if nothing trims it.

SoL-Pi (arXiv:2609.20519) names four mechanisms that attack exactly this. We wanted our own number, on our own trajectory, not a headline to repeat.

What we built

A small, dependency-light reimplementation of the four mechanisms, plus a deterministic loop simulator that counts the exact tokens a coding agent would re-send. No model is called. The measured quantity is input tokens, counted with tiktoken's o200k tokenizer, over one fixed trajectory (reproduce a failing test, search the codebase, read a large module, re-check the log, fix and verify, answer) run over real files: a 40 KiB failing test log, a 24 KiB source module, a 12 KiB search result.

• ObservationPack: outputs over 10 KiB go full for two requests, then collapse to a stable handle plus a head and tail excerpt.
• Evidence-Preserving Reducer: build and test logs over 4 KiB collapse to a verified receipt (exit status, the error lines, head and tail), and fall back to the full log if the receipt fails to shrink it.
• Online Context Compact: at a subtask boundary, consumed observations collapse to one line, but only when the projected saving beats the cost of rewriting the prompt cache.
• Action Fusion: fuse an edit and its verify command into one observation, removing a whole model round trip.

The rule that makes it safe: every archived output is kept byte-exact behind its handle, so any later step can pull the exact bytes back. Nothing is destroyed, only deferred.

What we measured

Naive, re-sending everything: 182,490 input tokens over 12 requests. With all four mechanisms: 24,614 tokens over 11 requests. That is an 86.5% cut, and the naive loop sent 7.4x the tokens it needed to. Switching the mechanisms on one at a time, in cost order, shows where the saving actually comes from:

• naive, no management: 100% (182,490 tokens)
• add ObservationPack: down to 21.5%
• add the Evidence Reducer: down to 17.0%
• add Online Context Compact: down to 13.7%
• add Action Fusion: down to 13.5%

One mechanism did almost all of it. Bounding large tool outputs took the run from 100% to 21.5%, nearly 79 of the 86.5 points. The other three split the rest. If you only ever do one thing to an agent loop, cap the size of what large tool calls put back into the context.

And the evidence survived. The failing test name, the exact assertion (-900 != 90), the buggy function and a symbol buried in the large module were all still retrievable byte-exact from the archive. The demo asserts this and fails loudly if any of it goes missing.

The honest part: baseline and cache

86.5% is not the paper's number, and it should not be. The paper measures against Pi, an already-tuned harness, and gets 44.7 to 49.0%. Our baseline re-sends everything with zero management, so our figure is the ceiling against the worst case. Plenty of home-grown loops are that worst case, which is the point, but do not read our 86% as beating their 47%. Different starting line.

The bigger caveat is money. Raw tokens fall 86.5%, but with prompt caching the dollar saving is smaller, because a re-sent prefix is cheap when it is cached. Worse, compaction rewrites that prefix, so it can break the cache and cost more than it saves. That is exactly why the real mechanism, and our reimplementation, gates compaction on the rewrite cost rather than always compacting. The two mechanisms that did the most here, ObservationPack and the reducer, only change new observations as they arrive or age, so they shrink tokens without fighting the cache. The cheap wins are also the cache-safe ones.

What actually transfers

• Cap large tool outputs first. It is the biggest lever and it does not touch the cache. Full for a turn or two, then a handle and an excerpt, with exact retrieval on demand.
• Give build and test logs a verified receipt, not the raw wall of text, and fall back to raw if the receipt does not shrink it.
• Reach for compaction last, and gate it on whether the saving beats the cache rewrite. It is the one that can lose you money.
• Keep every archived output byte-exact behind a handle, so trimming is deferral, never loss.

What is still off

It is one short trajectory, not a benchmark suite, so treat the number as a shape not a guarantee. No live model runs, so we measured input tokens, not task accuracy. The retrievability check proves the evidence is still there, it does not prove a live model would reach the same answer with the shorter context. And our reducer is deterministic where the paper uses a cheap model to write the receipt, which is a stricter, cheaper variant, not a like-for-like copy.

What is now in the stack

• solpi: the four mechanisms, a deterministic loop simulator, a runnable demo and eleven tests, all dependency-light Python. MIT licensed and on GitHub. Clone it, run python3 demo.py, and you get the same number. If your agent loop re-sends full tool outputs every turn, the shape transfers: the cheapest, cache-safe win is to bound what large tool calls hand back.