Evals.

Recovered requirements and architecture left held-out pass rate flat (33.7% to 35.8%) but lifted module-structure F1 from 0.54 to 0.63. Tests miss design; add a structural metric to your evals.

CRBugBench: the best review agent catches 71.9% of bugs in its top five comments, and humans plus agent reach 82.1% versus about 50% for humans alone. Recall drops up to 25.5% on industry monorepos.

Six models wrote test suites from specs for 165 CLI tools. Validity hit 96%, but mutation scores ranged only 19–56%. Giving agents the executable fixed validity, not coverage.

Four popular efficiency methods were tested on 200 vulnerability-discovery agent traces; only 24.4% of pairs cut cost without losing success. Count helper-model costs before you celebrate savings.

Across 16,416 generated tests on SciPy, Qiskit and pandas, 8.4–16.9% enforced faulty behavior and 83–91% of those survived later commits. Treat generated tests as characterization, not verification.

A similar skill took 1 in 5 runs from the installed one in 6,368 Claude Code runs, with no drop in task completion. A pre-tool hook at the first skill read fixed most of it.

ParanoiaEval measures unnecessary defensive work by coding agents: 11.2%–58.7% of runs despite evidence against it, hidden behind 88–98% task success. Bounds written into instructions help most.

ASAD sizes its agent team to bug difficulty and beat both fixed and maximum teams on medium bugs while using far fewer tokens. Agent count should be a decision, not a default.

Review quality tracked whether evidence was grounded in execution, not reviewer size, but the deployable cascade still rejected 66% of good patches.