§1The claim, and why we didn't take it
A skill called Ponytail is doing the rounds this week. It is MIT-licensed, installs into a coding agent in two lines, and promises to make the agent write about half as much code and spend roughly a fifth less. The mechanism is not a tool or a clever hook. It is a paragraph appended to the system prompt telling the model to behave like a lazy senior developer: before writing anything, stop at the first rung of a ladder that holds.
Headlines about token savings are cheap, and every one of them is measured on someone else's tasks. The only number that decides whether a thing belongs in our stack is ours, on the work our fleet actually does. So instead of reading the benchmark in the readme, we ran our own.
§2What it actually is
The whole product, stripped of packaging, is this ladder. You are meant to climb it only after you understand the task, and stop the moment a rung holds:
- 1. Does this need to be built at all? (YAGNI)
- 2. Does it already exist in this codebase? Reuse it.
- 3. Does the standard library already do this? Use it.
- 4. Does a native platform feature cover it? Use it.
- 5. Does an already-installed dependency solve it? Use it.
- 6. Can this be one line? Make it one line.
- 7. Only then: write the minimum code that works.
That is it. No abstractions nobody asked for, no new dependency if one can be avoided, deletion over addition. It is a good instinct written down and pointed at a model that does not have it by default. The best skills are often exactly this: a well-aimed paragraph, not a codebase. Which is also why the honest question is not "is the prompt sensible" (it plainly is) but "how much does it move on the tasks we run".
§3The test, and the numbers
We built a controlled run: three tasks, two arms each (with the ladder and without, injected identically as an appended system prompt so the instruction was the only variable), three repeats per cell, all on Haiku 4.5, each in a fresh empty repository. We measured cost, output tokens, and lines of code added by git diff.
Two of the tasks were chosen to tempt over-building: a UK postcode validator and a debounce utility, the sort of thing an agent gilds with a regex zoo, four exported variants, and a test suite nobody asked for. The third was a control that is already minimal by nature: reverse a string.
| Task | Lines of code | Output tokens | Cost |
|---|---|---|---|
| postcode (over-build trap) | −79% | −66% | −56% |
| debounce (over-build trap) | −72% | −53% | −32% |
| reverse a string (control) | −48%* | −6% | −3% |
Change with the ladder vs without, means of three reps. *The control's line drop is off a tiny base (25 lines to 13): the baseline added a test file and docstrings, the lazy arm did not. Cost and tokens are flat, within noise.
On the two traps the saving is real and large. On the postcode task the baseline agent wrote 296 lines across as many as seven files; the lazy agent wrote 61 across three, and both validated a postcode. That beats the vendor's own headline, they claim about 20% cheaper, we saw 32 to 56% on these, precisely because we picked tasks that reward laziness. Which is the whole point of what comes next.
§4The catch the headline skips
The control is the finding, not the footnote. Where there was no fat, there was no saving. The ladder cannot cut code that was never going to be written. So the win is not a property of the skill at all. It is a property of the task: how much your agent was about to over-build. Point it at work that is already lean and it does nothing, politely.
For us that reframes the whole thing. Our fleet's bill is not dominated by writing greenfield features. It is transcript-mining, document generation, research fan-out, grading, reading. That work emits very little code to trim, and its cost is dominated by input tokens, the context read in, which a lazy-coding instruction never touches. A fleet-wide "this makes everything cheaper" reading of Ponytail is simply wrong for a fleet shaped like ours.
A token-saver that shrinks generated code is worth exactly what your generated-code line is worth. If most of your spend is the model reading, not the model writing, the cure is aimed at the wrong half of the bill.
Note what we did to be fair to the skill and still landed here: we handpicked over-build traps, ran the arms identically, and it won every one of them. Cherry-picked in the vendor's favour, and the honest conclusion is still "only sometimes". That is not a knock on the skill. It is a knock on reading any token-saving headline as if it applied to your workload.
§5What to actually do with it
Two things are worth taking from this, and they are not the same size.
The small one: adopt the ladder where it fits. It is a genuinely good default for any agent writing new code, and it costs a paragraph to add and a paragraph to remove. We are wiring it into our ReferRoute feature work, where an agent building a form or an endpoint from scratch is exactly the over-build case the benchmark rewards. We are not adding it to the research and grading agents, where the numbers say it would earn nothing.
The large one: measure the shape of your own burn before you buy a cure for it. This is the lesson the benchmark taught, not the skill. A "token-saver" is really a "generated-code reducer", and those are only the same thing if generated code is where your tokens go. Ours mostly is not. If you are a team shipping features all day, this may be the best paragraph you add this month. If you are mostly reading, it will quietly do nothing, and you will probably credit it anyway, because the readme told you to and you never ran the control. We ran the control. That is the entire difference between a recommendation and a number.
--append-system-prompt so the instruction was the only variable between arms. Cost and token counts come from the CLI's JSON output; lines from git diff --numstat. Ponytail is DietrichGebert/ponytail (MIT); the ladder is quoted from its universal rule file. The two over-build figures are means of three reps; the control is a single already-minimal task and behaves like one. We deliberately chose tasks that favour the skill and still report the case where it does nothing, because that case is the point.