AI · 135 POSTS
AI.
Writing tagged AI: 135 posts on AI systems, engineering tradeoffs, and building products that have to work in production.
A placebo-controlled study of 5,835 coding-agent runs found telling the agent its token budget never beat an unrelated sentence. The cap itself is where the leverage is.
A meta-analysis of 516 multi-agent setups finds they tie or trail compute-matched single agents 74% of the time. Coding agents can escape that, but only through tests and split context.
HerAgent's ablation shows that environment-setup fixes made in a live container but never written back into the setup script mostly fail a clean rebuild: 25.9% vs 90.7% final success.
Writing property invariants from the spec before reading code found 56 bugs a code-first agent missed, at the same cost. But spec-only did worse; the ordering is what matters.
Asking agents to review their work barely helped on DeepSWE. Framing the same check as advice to the spec's maintainers lifted pass@1 from 44.25% to 50.22% at under half Self-Refine's cost.
Across 6,541 agent commits, 37% of checkable AGENTS.md rules were never followed and 54% always were. Rewording rarely helps; give each rule a check or a hook.
Compiling one recorded Claude Code skill run into programs cut repeat-run tokens 74–81% on 87 SkillsBench tasks; compiling from the skill text alone made most tasks costlier.
FlowPlan makes Claude Code state its fix as a control-flow plan and check the edit against it, cutting tokens to as little as 0.32x with no significant change in resolution rate.
BoxBox let a coding agent edit its own harness during a run and reached 1.702× speedup at lower cost, while offline harness evolution scored below the frozen baseline.
Stale AGENTS.md lines barely changed task success across 732 runs, but a dead test command produced silent no-op checks in 64 of 80 runs and no agent ever fixed the file.