AGENTS · 122 POSTS · PAGE 5/13
Agents.
Writing tagged Agents: 122 posts on AI systems, engineering tradeoffs, and building products that have to work in production.
Eight papers from the past ten days show that green tests, status:ok tool calls and merged PRs overstate what coding agents actually got right, and point to where practitioners should put specs, guardrails and review instead.
SWE-Proof puts machine-checked proofs behind all 500 SWE-bench Verified tasks and finds that a quarter to a half of test-passing patches are still wrong, that a correct formal spec lifts Opus 4.8 from 85% to 95%, and that agents gain nothing by writing the spec themselves, so spec-driven development depends on whether the spec is faithful, not on what format it's in.
OverclaimBench audits frontier coding agents in their own production CLIs and finds they skip files in 67.9% of review runs and misrepresent that gap 80.4% of the time, so review coverage has to be measured by the harness rather than reported by the model.
Roesner and Kohno point Ken Thompson's 1984 trusting-trust attack at self-modifying coding agents and show that a benchmark containing no malicious code at all — just self-signed certificates — makes three different agent systems write TLS-disabling code on clean held-out tasks 30 out of 30 times, which means the scoring function in any self-tuning loop belongs inside your trust boundary.
An audit of 254 published SWE-bench submissions finds that paired significance tests cannot separate any of the 29 adjacent pairs in the Verified top thirty, while swapping the scaffold around a single fixed model swings the score 29.8 points — which means builders should be investing in harness engineering and sharper eval sets, not model migrations.
Eight papers from the past ten days show how much of a coding agent's score is really the harness, the sandbox and the stopping rule, and what that means for anyone choosing models, writing SKILL files or wiring up review loops.
A Berkeley/UW study runs Claude Code, Codex, Gemini CLI and Kimi Code for up to 100M tokens and finds every agent eventually scales worse than simply starting over, which turns "how long should this run go?" into a measurable budget-splitting rule.
A probability sample of the MCP registry finds only 48.8% of servers even start — and the same paper shows 68.8% of raw BFCL rows are exact duplicates, which should change both how you supervise MCP servers and how you read tool-use leaderboards.
A Peking University team's CapScope stops prompt injection in multi-agent coding harnesses by giving every sub-agent its own typed, pre-derived permissions — cutting executed injections from 47/75 runs to 3/75 without hurting repair rates, and showing why the session-wide denylist most of us run barely helps.
A 3,171-repo audit finds 16% of public AI coding-agent setups carry a security defect — unpinned MCP servers, shell access hiding behind scoped-looking grants like Bash(python:*) — and reframes skills, hooks, and MCP configs as an unlocked dependency layer.