AI · 135 POSTS · PAGE 2/14
AI.
Writing tagged AI: 135 posts on AI systems, engineering tradeoffs, and building products that have to work in production.
Five code-map views cut Claude Code's token use 17–39% on localization and repair, but resolution stayed flat and test generation got costlier. Maps solve finding, not fixing.
Trajectory-based failure predictors for coding agents added nothing over issue difficulty plus model identity, and model switching gains were within noise. Test your alarms against run-blind baselines.
In Multi-step SWE, only 12% of failed steps used their full time, and 52% ended with the agent claiming completion. Treat "done" as a trigger for independent checks, not a verdict.
An audit of nine Claude Code spec-driven features found 75.3% of reviewer tokens went to re-reviews, and the orchestrator that checked its own work stopped 4 of 11 flagged rounds.
An audit of HAL and τ2-bench found only ~51% of one scaffold's episodes were graded; scoring the rest erased a 13-point gap. Check your graded fraction before trusting any comparison.
On 181 failed bug-fix attempts, a model-written lesson matched the raw transcript (−1.7 points), and gains shrank once the full test suite was checked. Skip the distillation call until you measure it.
Malicious memory files moved secure-code rates only −8 to +6 points across three agents, but only 4 of 18 sessions that called the file an attack said so in the final message.
Across 5,625 trials, coding agents followed local repo precedent over conflicting instructions; one written rule in CLAUDE.md fixed it 59 of 60 times where a code example managed 38.
Giving agents program structure as code-like stubs cut relative test error by 48.4% versus the best behavioral specs, and let a model 15–26× cheaper land within 5–8 points of the large one.
A harness cut 3,827 of 11,189 tool results into broken JSON while using 5.75% of the context window. If your agent seems oddly blind, audit truncation before blaming the model.