A new paper, SKILLER, claims you can get a small free model to do work that normally needs an expensive one, not by training it, but by writing it a good enough skill. We reproduced the core loop with the pieces we already run, pointed it at a real extraction task, and put a kill gate on it. A free local model went from 0.938 to a perfect 1.000 on records it had never seen, matching the cloud model, after one round. The honest part: the gap it closed was small. So this is proof the machine works, and works cheaply, not proof it can haul a weak model up a tall hill.
What we did
The word "framework" in the paper hides how simple the loop is, and the simplicity is the point. A strong model writes a skill, a short block of instructions. The small model runs the task with that skill glued to the front of its prompt. A reward scores how it did. The strong model reads the mistakes and rewrites the skill. Repeat. Nothing is trained, no weights move, no GPU is booked. It is prompt engineering with a scoreboard and a critic, run in a loop, which is why it took an afternoon rather than a cluster.
The task was deliberately the kind a small model fumbles in a fixable way: messy one-line invoice records, each salted with distractors. A buyer as well as a seller. Three dates: ordered, invoiced, due. A subtotal, a VAT line and a total. An invoice number as well as a reference code. The right answer is always the seller, the invoice date, the grand total and the ref. A capable model reads the labels and picks correctly. A 7B skims and grabs the nearest number, which is exactly the patterned error a skill can teach it out of. We used the free local Qwen-2.5-7B as the executor, a frontier model as the skill writer, eight records to learn from and eight, kept back, to be judged on.
The local model started at 0.938 against the cloud model's perfect score, a gap of six points, mostly it picking the buyer as the vendor or the VAT line as the amount. One pass of the loop produced a skill that spelled out the disambiguation in plain words, and the local model then scored 1.000 on the held-out eight. Full marks, on records the skill writer never saw, from a model that costs nothing to run.
Why it was worth doing
Because it is the one cost lever whose runtime goes to zero. Better routing makes a cheap call cheaper. A guardrail stops a runaway. This moves the work off the cloud entirely, onto a model already sitting idle on our own box. Where a skill can close the gap, that task runs free from then on, at any volume, and the data never leaves the machine. One honest qualifier on that "free": writing the skill takes a frontier model, one round of it here. The bigger model is the author, not the worker, so what drops to zero is the ongoing runtime, not the one-off authoring. You pay a strong model once to teach the cheap one, then never again. And SKILLER fits our stack unusually well because the expensive part, a reward signal, is something we already have: the reward here was a deterministic field check, but for open-ended work our own judge could score it instead.
It is also a useful corrective to the reflex of reaching for a bigger model the moment a small one stumbles. The 7B was not too stupid for this task. It just did not know the house rules. Written down once, in language, those rules were worth more than a model ten times the size, and they cost a single loop to discover.
What's still off
The gap was small, and I will not dress that up. The 7B was already at 0.938, so the skill fixed a patterned confusion rather than lifting a model that was genuinely out of its depth. It also converged in one iteration, which is a good sign but means the multi-round refinement the paper leans on never got a workout here. And the reward was deterministic only because the task had checkable answers; the moment the work is open-ended, you are back to a model judging a model, with the cost and noise that brings. This is a spike: one task, one run, eight held-out records.
So the finding is narrow and honest: the loop works, it is cheap, and it fits our pieces. What it has not yet earned is the big claim, that it can take a task where the local model is genuinely far behind and drag it up to parity. That is the next test, and now that the machine is built and proven, it is a cheap one to run. If a skill can close a wide gap as neatly as it closed this narrow one, that category moves to the free local tier and stays there.