AI · 135 POSTS · PAGE 7/14
AI.
Writing tagged AI: 135 posts on AI systems, engineering tradeoffs, and building products that have to work in production.
A probability sample of the MCP registry finds only 48.8% of servers even start — and the same paper shows 68.8% of raw BFCL rows are exact duplicates, which should change both how you supervise MCP servers and how you read tool-use leaderboards.
A Peking University team's CapScope stops prompt injection in multi-agent coding harnesses by giving every sub-agent its own typed, pre-derived permissions — cutting executed injections from 47/75 runs to 3/75 without hurting repair rates, and showing why the session-wide denylist most of us run barely helps.
A 3,171-repo audit finds 16% of public AI coding-agent setups carry a security defect — unpinned MCP servers, shell access hiding behind scoped-looking grants like Bash(python:*) — and reframes skills, hooks, and MCP configs as an unlocked dependency layer.
A carefully controlled RL study finds the evaluation harness swings SWE-bench solve rate by 4.3x while the training recipe moves it 1.16x — and that pooling rewards across harnesses buys configuration adaptation, not portable capability.
Seven papers from the past ten days, and almost none are about making models write better code — they are about the gap between a patch that passes and a patch that is acceptable, and what it costs to tell the difference.
A mining study of 921 requirement arrivals in real coding-agent sessions puts a 2x rework tax on late requirements — and a controlled experiment finds that warning the agent one is coming does nothing at all.
A placebo-controlled study finds that spectrum-based fault localization loses decisively to blind resampling at matched budget — and that the failing-test signal it depends on exists only 9% of the time.
HarnessDev asks whether LLMs can build and evolve their own agent harness — they can build a decent one, but their self-improvement feedback predicts real held-out gains only 53% of the time, and the harnesses they tune don't transfer across models.
Seven papers from the past ten days converge on one uncomfortable point: the harness, the config file and the context window explain more of your agent's results than the model does.
A new benchmark shows coding-agent vulnerability to poisoned repos depends less on the attacker's payload than on how you invoke the agent — “run the tests” hits a 45.5% attack success rate while “fix this bug” hits 8.6%, and security rules in your skills files raise alerts without lowering risk.