Most of this week's agentic-coding arxiv has been meta: benchmarks measuring benchmarks, studies of why human reviewers rubber-stamp agent PRs. So a paper that just builds a sharper repair agent and reports honest, slightly disappointing numbers stood out while I was skimming this morning's list. It comes from the KTH group — Reyes, Baudry, and Monperrus — who have done some of the most rigorous agent work I have read. The hook is a single design decision: stop having your repair agent emit a patch, and have it emit a codemod. My thesis for this post is that the output format of a repair agent is an architectural lever, and most of us are leaving it on the table.
What it does#
The target problem is the most boring, most universal tax in software: a dependency bumps a version, the API changes, and your build breaks. The usual LLM approach is to generate a patch — a project-specific diff that fixes your call sites and is then thrown away. Agentic Generation of AST Transformation Rules for Fixing Breaking Updates does something different. Its framework, BigBag, generates a fixing transformation: a standalone, executable Java program that walks the client's abstract syntax tree, finds the constructs the breaking change touched, and rewrites them to the new API. The program is the artifact — and because it operates at the API level rather than against one repo's specific lines, the same transformation can be re-run against any other project broken by the same update.
The loop has four steps. The agent gets the broken project, documentation for an AST engine (either Spoon or JavaParser), a transformation template with Maven config, and the new dependency's API plus Javadoc. It runs mvn compile, reads the failures, writes the transformation program, compiles that program standalone, applies it, and re-enters the loop on compiler feedback if the build still fails. Then — and this is the part I like — a separate verification step re-applies each transformation in isolation to confirm that it alone resolves the errors, cleanly separating what the agent contributed from everything else. The contrast with prior repair work is explicit: earlier approaches generate project-specific patches that cannot be reused. Here the unit of repair is itself a reusable program.
The key result#
The headline single-project number is strong: a 78.6% fix rate (best config: Gemini-3.1-Pro with JavaParser) against roughly 27% for the LLM-patch baseline from the same group's earlier work, with 94.3% of generated transformations compiling at all. But the number I would actually pin to the wall is the gap: cross-project transfer lands at 33.3% overall, but climbs to 80% or above when every client invokes the broken API the same way. That single sentence is the whole paper. Reusability is real, but conditional on usage uniformity — the agent encodes only the patterns it could see in its one seed project, so the more idiosyncratically downstream code uses an API, the worse the transfer. The evaluation runs on 157 compilation-breaking updates from the BUMP benchmark across 69 client projects and 70 libraries, swept over eight configurations (four LLMs × two AST engines).
Why it matters#
Breaking dependency updates are a tax every team pays, usually by hand. Dependabot or Renovate opens the PR; a human still goes and fixes the call sites. The interesting move here is not that an agent fixes my build — we already have that. It is that the agent produces something that could fix the build for everyone downstream of the same library bump. That is the OpenRewrite / jscodeshift / codemod model, except the recipe is synthesized on demand instead of hand-authored by a library maintainer. If you are building anything in the dependency-maintenance space, the unit of work shifts from one patch per repo to one recipe per breaking change, and the economics flip: you amortize a single agent run across an entire ecosystem instead of paying for it once per affected project.
The broader lesson for anyone wiring up agents is about what your agent emits, not just whether it solves the task. A patch is a dead artifact — it fixes one repo and disappears. A transformation is a program: inspectable, testable, re-runnable, reviewed once and applied many times. That maps directly onto sub-agent architectures. A fixer sub-agent that returns a codemod is more composable than one that returns a diff, because the orchestrator can validate it in isolation — which is exactly what BigBag's verification step does — and then reuse it elsewhere. There is also a smaller, very practitioner-y detail worth filing away: JavaParser, the lighter engine with fewer semantic guarantees, beat Spoon, the one with full type resolution. The heavier framework's type-inference demands simply gave the agent more ways to crash. Less scaffolding, more reliable agent — a pattern I keep seeing and keep underweighting.
The caveats#
Java and Maven only. The whole approach leans on mature AST transformation engines that do not have clean equivalents in every ecosystem.
The 33.3% cross-project number is the honest average; the 80% is a conditional best case. Transfer fails when the generated matching pattern is wrong (72 cases, finds no sites to touch) or too loose (14 cases, over-applies the rewrite).
Agents sometimes bypass the AST framework entirely and fall back to string replacement or direct file edits — quietly regressing to the brittle patch the method was meant to replace.
The hardest cases — massive migrations with 271+ compilation errors, or heavy generics — still defeat it. This is a small-to-medium API migration tool, not a rewrite-my-framework tool.
The takeaway#
What I am filing away: for any repair or migration agent, make the output a reusable program rather than a one-shot patch, and bolt on a verification step that re-applies it in isolation before you trust it. Even when transfer is only partial, an inspectable codemod beats an opaque diff for both review and reuse — and that is a property of the output format, which you are free to choose. What I am doing differently after reading this: when I design an agent's return value, I will ask whether it could be a program instead of a result. And I will reach for the lighter AST tooling first.