Is a frontier model worth its higher price for everyday agent work?

On our cost-per-solved-task benchmark, no. GPT-6 Astra and Claude Opus 4.8 both passed every easy task, but Astra cost about 31% more per solved task and ran slower, despite using 37% fewer output tokens, because its output price is twice as high. On everyday work the cheaper workhorse wins on cost and speed at the same quality.

When does an expensive frontier model become worth it?

As task difficulty rises. On our harder set (competition maths, dynamic-programming coding, large-context multi-hop) the cost gap between Astra and Opus 4.8 collapsed from about 31% to about 6%, and Astra became faster and used 56% fewer output tokens. The frontier model's token efficiency scales with difficulty, so it earns its price only in its spike band, not on routine work.

How should you add an expensive model to an agent fleet without runaway cost?

Make it an escalation lane, not a default. Route to it only on explicit triggers (task in its proven spike band, repeated tool-call failure at a cheaper tier, or low self-reported confidence), cap the number of escalations per conversation, put a hard monthly spend cap on its key, and add a confirmation gate before any write action given its elevated capability. Everyday work stays on cheaper tiers.

Astra 6 is here | Workloft Ships

GPT-6 Astra, a frontier model priced at twice our workhorse on output, costs about 31% more per solved task on everyday work and only draws level when the task gets genuinely hard. So we did not make it the default. We wired it into the fleet as an escalation lane that fires only when the work is in its spike band, a cheaper tier has failed repeatedly, or the model says it is unsure.

What we tested

Astra landed with the usual framing: the most capable model, built for the hardest end-to-end work. The practical question is narrower: is it worth twice the output price for the work our fleet actually does? So we built a cost-per-solved-task bench (run_eval.py), graded the honest way with no judge model in the loop and every answer key computed by code, not hand-written. We ran two sets: an everyday set (coding, maths, extraction, long-context, instruction-following) and a harder frontier set (competition number theory, dynamic-programming coding, large-context multi-hop). The same harness ran GPT-6 Astra and Claude Opus 4.8, our current workhorse. Cost is real token usage times list price.

What we found

EVERYDAY set        pass    cost / solved   out tokens   latency
Claude Opus 4.8     10/10   $0.00258        810          1.89s
GPT-6 Astra         10/10   $0.00337        511          2.40s

HARD set            pass    cost / solved   out tokens   latency
Claude Opus 4.8     12/12*  $0.0184         5634         6.17s
GPT-6 Astra         12/12   $0.0195         2472         5.05s

Quality did not separate them: both passed everything (Opus's single miss on the hard set was a token-budget truncation of a verbose method, not a wrong answer, and it passes with more budget). What moved was cost, and it moved with difficulty. On everyday work Astra cost about 31% more per solved task and ran slower. On the hard set that gap collapsed to about 6%, and Astra flipped to faster. The reason is token efficiency: Astra used 37% fewer output tokens on the easy set and 56% fewer on the hard set. Its efficiency grows with difficulty, and that is exactly where it starts to cancel out its double output price.

How we wired it in

The crossover is the whole argument for a cascade rather than picking one model. So Astra sits as a fourth tier above our flash, workhorse and complex tiers, and it is invoked only on an escalation trigger: the task is in its proven spike band (browser and computer use, cyber and vulnerability work, frontier maths, very large context), or a cheaper tier has failed tool calls twice, or the lower tier self-reported low confidence. Escalations are capped per conversation, the key carries a hard monthly spend cap, and any Astra task that can write (shell, files, browser, deploy, payment) hits a confirmation gate before it runs, because a more capable model with tool access is exactly the one you do not let act unchecked.

Why it matters

The headline for a frontier model is one number, usually a benchmark or the "fewer tokens" claim. Your bill is a different number: your task difficulty times the full price sheet. The "fewer tokens" part was true here, but on easy work it did not cover a 2x output price, so the cheaper model won. Only as the work got hard did the efficiency catch up. The lesson is the same one our router and our cheap-model bench keep teaching: match the tier to the difficulty of the task, and make the expensive model prove it is needed before it runs, rather than paying its premium on work a cheaper tier does just as well.

What's still off

The load-bearing caveat is that we have not tested Astra where its edge should be biggest. Both sets are text-in, text-out and deterministically graded. Genuinely agentic work (long-horizon browser and computer use, multi-step tool loops) and true million-token context are where a frontier model is meant to pull away, and we cannot grade those without a live tool and browser harness we have not built yet. So read this as "on gradable reasoning and coding, the premium only pays off as difficulty rises," not as a verdict on agentic use. Second, N is small (10 and 12 tasks, single pass). Third, the escalation triggers are sensible defaults, not yet tuned against production traffic. All of that is the next run.

What's now in the stack

run_eval.py and generate_v2.py: a reproducible cost-per-solved-task bench with deterministic grading and code-computed answer keys, across a frontier model and the workhorse.
escalation.py: the cascade policy (spike-band, repeated-failure and low-confidence triggers) plus a confirmation gate for write-access tasks on the frontier tier.
cache.py: a semantic response cache (exact plus near-duplicate match) that can skip the call entirely for repeat traffic, with a pluggable embedder.
The kit on GitHub. Point it at your own tasks and your own contract prices.