RouteWeaver is a paper about a single policy that picks, per task, how hard to work (one shot, multiple rounds, or full agentic) and which model to use within that choice. We rebuilt the mechanism on a laptop CPU. The central idea reproduces cleanly: an adaptive router beats every fixed strategy and matches the oracle. The two training tricks the paper is named for do not earn their keep on a small router. Useful to know before you build one.
What we did
RouteWeaver (arXiv:2610.19705) proposes a unified routing policy over three execution modes, single-round, multi-round and agentic, plus a model choice inside each mode. Its novelty is two training ideas: COMET-GRPO, which splits the reward signal into a mode-selection part and a within-mode part so each decision gets targeted credit, and progressive exploration, which forces the modes early so the router learns to run each one before it learns when to pick it.
We cannot re-run their live-model experiments on a night build with no GPU, so we did the honest version: a controlled, deterministic reproduction of the mechanism in a synthetic environment with a known reward structure, built so the paper's premise holds (no single mode is best everywhere, and the richer modes need a capable model to pay off). Everything is numpy, a few seconds on CPU, and every number is measured in that environment.
Why it was worth doing
The central claim reproduces. A unified two-level policy trained with plain group-baseline REINFORCE learns to route per task type and lands at 86.2% accuracy, against 83.3% for the best fixed mode and 84.8% for the reward-optimal oracle. It does this by using single-round on the task types where that is enough and spending the richer modes only where they pay. The shape of the paper's result is real: adapting the mode per task beats any one fixed pipeline, and you can get there with a very simple learner.
That matters because mode selection is exactly the lever we keep pulling across the fleet: cheap model versus flagship, one shot versus a loop. The reproduction says the big win is the framing, not the training rig.
What's still off
The two refinements the paper is built around did not separate from plain GRPO in our setting. Across three experiment designs, budget sweeps from 100 to 3000 steps and 20 to 40 seeds each, COMET-GRPO's decoupled credit assignment gave no measurable, seed-robust advantage (mean gap under 0.7 points, inside the noise). Removing progressive exploration did not collapse the policy to single-round as the paper reports; single-round usage held at its correct 40%, because here it is genuinely the right call for two of the five task types, not a degenerate trap.
This is a boundary condition, not a contradiction. The paper's machinery is built for hard, high-dimensional, sparse-reward routing where credit assignment is the bottleneck. On a small router (three modes, a few models, a handful of task classes) a group-baseline bandit already recovers the oracle, so the extra RL scaffolding adds complexity with no payoff.
The verdict for our stack
Steal the idea, skip the rig. A single policy that picks execution mode and model per task is cheap to build and recovers most of the gain. The COMET-GRPO and progressive-exploration training is only worth paying for at real scale. For a fleet router over a handful of task classes, a plain group-baseline bandit is enough. Code and the full findings are in the public mirror.