A reviewer subagent that checks every artefact sounds like diligence. Then you read where the tokens went: in one set of spec-driven runs with Claude Code, re-reviews consumed 75.3% of the reviewer subagent's tokens, and about two thirds of that (49.8% of the total) followed a report that had found nothing blocking. I think that is the most useful number in A Gated Spec-Driven Workflow: An Audit-Trail Study of Coding-Agent Work by Abdo and Brom, and it is buried under a more cautious headline.
I will say up front that I am skeptical of the paper's evidence and fairly convinced by its design lessons. Those are different things, so I'll keep them apart.
What was actually built, and how thin the evidence is#
The workflow runs in Claude Code and installs from a single Markdown file: six skills, five templates, project memory and two subagents. Work moves through PDR, SPEC, PLAN, TASKS, EXECUTE and ARCHIVE, and the human approves each artefact before the next stage starts. A read-only reviewer subagent checks every artefact against a fixed defect list (8 items for PDR up to 22 for TASKS). One executor subagent per task may touch only the files that task declares. After each block of tasks the orchestrator diffs the output folder against a round-start inventory and stops on any undeclared change.
The data is nine features across seven projects, all run by the authors: a Fortran solver, Python programs, books, and the paper itself. 581 tasks were planned; 288 were checked as done. Five features finished. There is no control condition and no repeated runs, and the authors say plainly that no result supports a causal claim about what the controls prevented. They are also the subjects, and an AI agent did the log analysis.
So this is an audit trail of one team's own process, not an experiment. Read it as a field report.
Reviewers always find something, which is the problem#
The numbers on the reviewer side are striking:
111 reports over 20 artefacts, 508 findings: 75 blocking, 163 serious, 270 observations
15 of 20 artefacts had at least one blocking finding in the first review
Only 5 reports were clean
51 of the reviewer calls were re-reviews; 30 of those followed a report with no blocking findings, and 6 of those 30 still reported blocking findings in the revised artefact
In project P2, 47 of 63 reviewer calls came with no author message since the previous call
The authors note that the findings were counted, not validated as real defects. That matters. A fixed defect list plus a model asked to apply it is a machine that produces findings. The paper's own aside, that "no defect found" is a legitimate answer, reads to me like a prompt-design note: the reviewer needs explicit permission to come back empty, and the loop needs a rule for when to stop asking.
The six re-reviews that found blocking issues after a clean report cut the other way, and I would not wave them off. Either the first pass missed something or the second pass invents. The paper cannot tell us which, because nobody adjudicated the findings.
The orchestrator grades its own homework#
The weak point was the orchestrator, which runs and judges its own checks.
This is the design lesson I would take. The executor behaved: it left one undeclared change in the deliverable folder, and the inventory check caught it. The orchestrator is the agent that runs the block check, interprets the result and decides whether to halt. It flagged undeclared changes in 11 of 247 block checks and stopped the round in only 4. The authors attribute most of the 11 to their own in-round requests and to the environment, not to executor misbehaviour.
Their framing is that almost every control is "an instruction carried out by the same model that it constrains". I agree, and it is the same trap I fall into with CLAUDE.md rules. A file-inventory diff is deterministic, so run it from a hook or a script outside the model and let a non-zero exit stop the round. Leaving the model to decide whether the diff matters is how 7 of 11 flagged changes ended up not stopping anything.
Why the gates still look worth keeping#
None of this says the gating is wasteful. Fifteen of twenty artefacts drew a blocking finding on first review, and if even half of those are real, catching them at the PDR, SPEC or PLAN stage is far cheaper than discovering them after 288 tasks of execution. The human approval at each stage is also what makes the audit trail readable at all: every deviation and every reviewer episode can be tied to a point where someone said yes.
The paper logged 154 deviation rows and 430 valve rows, which tells me the authors treated departures from protocol as data, not embarrassment. That habit is worth copying whatever you think of the workflow. If your agent harness cannot tell you how often the agent deviated from its own instructions, you are running on trust.
What I would change in my own setup#
Three things, none of which the paper tested; the authors list tooling-run checks and an in-round request channel as proposals only.
Cap re-reviews. If a report has no blocking findings, the artefact moves on unless a human edited it. That alone targets the 49.8%.
Move the stop rule out of the model into a script the orchestrator cannot reinterpret.
Give mid-round requests a place to go, so a change you ask for in conversation does not show up later as an undeclared change.
The token figure is also a lower bound, covering only the reviewer subagent, with 21 of 132 known reviewer calls having no recoverable report. One agent, one model family, seven projects, one stretch of weeks. I would not generalise any rate from it.
What I would watch for next is someone running this with a no-gate baseline and validated defects. If reviewer findings turn out to be mostly real, 75% re-review spend is a bargain. If they are mostly noise, it is the largest line item in the workflow.