Most of this morning's coding-agent papers were benchmarks again. One wasn't. A group out of KIT went through thousands of real coding-agent sessions to ask a requirements-engineering question that benchmarks structurally cannot ask: what happens when the user changes their mind after the agent has already written the code? They put a number on it, then ran an experiment on the fix everyone assumes works. The thesis: late requirements are a measurable rework tax, and warning your agent that one is coming buys you exactly nothing.
What it does#
Requirements After the First Edit mines SWE-chat, a corpus of real coding-agent sessions linked to git repositories and commits with line-level provenance — meaning you can tell which lines the agent wrote from which lines were already there. The snapshot holds 5,851 sessions. The authors filter to 3,553 eligible, reconstruct roughly 2,060 with a clean start, and land on 921 requirement-emergence events across 402 sessions and 74 repositories. Each event is coded on three axes: entailment (does it follow from what was already said, or contradict it), operation (add, constrain, relax, remove, replace), and trigger (observed failure, implementation feedback, preference, external).
What makes this different from prior work is the linking step. Curated benchmarks fix requirements before implementation by construction — that is the whole point of a benchmark, and it means they cannot observe this phenomenon at all. Observational studies have counted how often users push back, but stopped there. This paper connects each requirement arrival to a proxy for the damage it causes: deletion or replacement of lines the agent had already authored. Then it backs the observational half with a controlled experiment that separates two things people routinely conflate — when a requirement is revealed, and whether the agent was warned it was coming.
The key result#
A requirement arrival is followed by 57.5 invalidated lines on average, against 29.4 for matched non-requirement edits — a ratio of 1.96 (95% CI [1.31, 2.82]), or roughly 28 extra lines destroyed per arrival. Call it double. It survives a stricter net-deletion measure (ratio 2.28) and holds across model backends. Two details sharpen it. First, 54% of these arrivals land after the session midpoint, so this is not a first-turn clarification problem you can solve with a better opening prompt. Second, the burden shows no detectable decline as a session progresses — long sessions do not settle down.
Why it matters#
The 2x is the number people will quote, but the experiment is where the useful finding lives. The authors ran two arms. In the first, they varied when the requirement was revealed: upfront got both parts in round one, delayed held the addition back to round two. Round-two churn was 0.1 lines upfront against 9.3 delayed (+9.2, [+6.4, +12.8]) — delayed disclosure relocates implementation, exactly as you would expect. In the second arm, both groups did identical round-two work; the only difference was whether round one included advance notice that a requirement was coming, without saying what it was. The effect on prior-code overwriting: +0.16 lines, 95% CI [−0.56, +1.06]. Nothing. Telling the agent that more requirements are coming is theater. Content does the work, not the heads-up.
That kills a specific habit. If your CLAUDE.md or system prompt carries hedging boilerplate — “anticipate future requirements”, “expect the spec to evolve” — this paper says the warning half is not buying you resilience. Those tokens are better spent on elicitation before the first edit than on priming the agent to brace. Two further findings push the same direction. 89.3% of late arrivals are neutral with respect to what was already said rather than contradictions, so most of this cost comes from ordinary additions, not from users reversing themselves — being decisive does not exempt you. And destructive operations (remove, replace) showed no significant rework premium over additive ones: +3.3 lines, [−19.3, +26.7]. You cannot triage late requirements by how destructive they sound. They are all expensive.
The no-decline finding is the one that should change how you structure a session. If the rework burden stayed flat as a session ran on, then a long session is not converging on a shared understanding — the tenth late requirement costs about what the second one did. That argues against the very long, single-thread session as a default, and for cutting work into shorter units with an explicit requirements pass at the front of each. It also reframes what a sub-agent boundary is for: not just context isolation, but a natural place to force elicitation before any code gets written. The corollary for spec-driven workflows is uncomfortable but fair — the spec earns its keep by being complete before the first edit, not by existing as a document you amend as you go.
The caveats#
The invalidation measure is syntactic. On a deliberately adversarial test set, line-level precision was 0.40 and recall 0.67. It counts lines that got deleted, not lines that became semantically wrong — movement, reformatting and regeneration all confound it.
Nothing here is causal, and the authors say so repeatedly. Matched controls cannot rule out task-phase confounding, or establish that a deletion actually implements the arriving requirement.
Only about 58% of sessions pass the clean-start reconstruction screen. The authors call the result a characterized convenience sample and explicitly decline to claim it represents all sessions.
Annotation agreement on the fine-grained categories is weak — AC1 of 0.58 for operation and 0.38 for trigger — so the operation-type null reads as underpowered rather than clean.
The specification-checkpoint intervention the paper points toward is future work. They supply power calculations for it; they did not run it. Nobody has shown the fix works yet.
The takeaway#
What I am filing away is not the 2x — I would have guessed late requirements were expensive. It is the null result on advance warning, because that is the one I would have gotten wrong. I have written prompt boilerplate telling agents to expect evolving specs, on the theory that a braced agent writes more provisional, easier-to-revise code. The one controlled test of that idea found no effect at all. So: less priming, more elicitation before the first edit. If I want an agent to write code that survives a spec change, that has to come from the content I hand it up front, not from a warning that a change is on its way.