Skimming this morning's cs.SE list I nearly scrolled past what looked like another cost-routing paper. Then I hit the ablation table and stopped. The system works — it matches Claude Opus 4.6 on solve rate at roughly a fifth of the cost — and almost none of the credit goes to the thing in the title. The router is a no-op. The scout lies most of the time. What's left is the most useful sub-agent design lesson I've read this month: in a two-stage agent, the handoff contract is the architecture, and everything in it you don't verify is noise you are paying to inject.
What it does#
Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First (Bhola, Krishnan, NS — August 5) splits issue resolution into a search stage and a fix stage. SuperScout-7B, a LoRA fine-tune of Qwen2.5-Coder-7B trained on 19,905 search-phase demonstrations drawn from Open-SWE-Traces, SWE-rebench-openhands and SWE-Hero (with all 23 SWE-bench evaluation repos blocklisted to avoid contamination), explores the repository for up to 40 turns and emits a roughly 4 KB structured handoff: implicated files with line regions ranked by confidence, one reproduction attempt as a file plus command plus observed output, the dead ends it already burned, and free-form repo notes. A résumé-based router then hands the task to one of four frontier fixers — Claude Opus 4.6, GPT-5.2, Kimi K2.5, Gemini 3 Flash. Each fixer's "résumé" is cheap: two embedding centroids over solved and failed tasks plus a base solve rate, derived from just 25–50 public outcomes, scored by logistic regression, walking fixers in cost-ascending order until one clears a threshold.
The part that separates this from every other small-model-does-retrieval-for-big-model paper is what sits between the two stages. Before any fixer sees the handoff, a sandbox replays the claimed reproduction command against the unpatched repository. If the command doesn't actually fail, both the test file and the reproduction claim are deleted — not flagged, deleted. The handoff is treated as a set of assertions to be checked, not a report to be trusted.
The key result#
On the Python slice of SWE-bench Pro (266 tasks), SuperScout solves 159/266 — 59.77% — at $0.230 per solve. Claude Opus 4.6 running solo solves 158/266 (59.40%) at $1.274 per solve. Matched accuracy, 5.5× cheaper, with the 7B searcher adding under half a cent of GPU cost per task. But the number I actually wrote down is from the verification gate: only 20% of the searcher's reproduction claims were genuine, and 70% were demonstrably false and stripped. The scout confidently reports that it reproduced the bug, with a file, a command and an output — and it is making that up roughly three times out of four.
Why it matters#
If you run sub-agents at all — an explorer feeding an implementer, a research agent feeding a coder — you already ship handoffs shaped like this one. The finding worth internalizing isn't "small models hallucinate." It's that this hallucination is structured, specific, and maximally credible. A vague prose summary gets discounted by the downstream agent. A file path, a shell command and a pasted failing-test output do not; that is precisely the shape of evidence a fixer treats as ground truth and anchors its whole search on. The intervention that saves it is embarrassingly cheap and has no model in the loop: replay the command, keep what fails, delete what doesn't. My read is that any field in a handoff which can be re-executed should be, and anything failing that check should be removed rather than annotated — a downgraded-but-present claim still steers the next agent.
The routing half is where the paper is most honestly unflattering to itself. 263 of 266 tasks got routed to the same model. The no-router ablation — Kimi K2.5 plus the handoff, always — scores an identical 159/266 at $0.227 per solve. All the centroid and logistic-regression machinery bought nothing. The paired calibration study on 99 fresh tasks explains why: the pooled handoff effect is +1.8pp with a confidence interval of [−1.0, +4.5] — it crosses zero. Per model: Opus +5.1pp, Kimi +4.0pp, Gemini 3 Flash +2.0pp, and GPT-5.2 −4.0pp. The handoff redistributes solving ability rather than adding it: it pulls the cheap fixers up toward the frontier and slightly hurts the strongest one. So the economic win is real, but it is a compression win, not a capability win — you're buying frontier-grade results out of a cheap model by pre-loading context it couldn't gather itself. The corollary matters if you're building this: context injection has a ceiling, and on your best model it can be net negative. If you've bolted an explorer sub-agent in front of Opus because it visibly helped your smaller models, measure it rather than assume it.
The caveats#
One benchmark, one language slice: the Python portion of SWE-bench Pro, 266 tasks. The searcher's cross-language transfer is shown, but whole-system generality isn't.
The calibration study is N=99 and the authors flag it as underpowered. That GPT-5.2 −4.0pp is exactly the kind of number that could move on a rerun.
The verify-then-strip gate — the piece I find most interesting — is the piece they didn't isolate. No pass-through control was run, so its contribution is inferred rather than measured.
The handoff is often partial: localization recall is 0.566 with a 24.8% all-gold-file rate. This is a hint, not a map.
Spontaneous handoffs (the scout chose to commit) beat forced ones (budget exhausted) by a wide margin — 28.2% vs 13.3% all-gold rate. Budget exhaustion is a quality signal you should be reading, not swallowing.
The takeaway#
Two things filed away. The 70% figure, because it reframes handoff verification from an optimization into a precondition — structured executable claims between agents are confabulated often enough that passing them along unchecked is a design bug. And the fact that the most elaborate component in the system, the router, turned out to be the least load-bearing, which is a pattern I keep running into in agent architecture work. What I'm changing: going through my own sub-agent handoffs and labelling every field as re-executable or not, then putting a mechanical check in front of the re-executable ones that deletes on failure instead of annotating. And actually A/B-ing the explorer stage against my strongest model rather than assuming the gain generalizes upward.