Scrolling through this morning's arxiv listings, one phrase made me stop: constraint decay. A team from EURECOM has put a name to something I keep running into when I push coding agents past throwaway scripts into real backend work — they get more brittle the more you tell them to obey. The paper measures it on 100 backend tasks across eight web frameworks and the curve is brutal. Today's thesis: the SWE-bench era of "can it pass the test" is hiding a much harder problem about whether agents can build software that fits.
What it does#
The paper, Constraint Decay: The Fragility of LLM Agents in Backend Code Generation (Dente, Satriani, Papotti, May 7), asks a question that benchmarks like SWE-bench mostly dodge: how well do coding agents perform when they have to satisfy structural constraints, not just functional ones? Production backend code lives or dies on architectural patterns, ORM conventions, database schemas, and framework idioms. A solution that passes end-to-end tests but routes through hand-rolled SQL when the codebase uses an ORM is a regression, not a win.
To isolate the effect of structural complexity, the authors fix a unified API contract across 80 greenfield generation tasks and 20 feature-implementation tasks, spanning eight web frameworks (Flask, FastAPI, Django, and five others). They evaluate two ways at once: end-to-end behavioral tests (does the API do what it should?) and static verifiers (does the implementation actually follow the structural constraints?). This dual evaluation is the whole point. Behavioral tests alone reward agents that find any path to the right output. Static checks force them to use the right path.
The key result#
As structural requirements pile on top of functional ones, agent performance falls off a cliff. From the baseline (minimal constraints) to the fully specified setting, capable agent configurations lose 30 points on average in assertion pass rates, and weaker ones approach zero. The single most striking line in the abstract is the framework breakdown: agents do reasonably well on minimal, explicit frameworks like Flask, but degrade substantially on convention-heavy environments like FastAPI and Django. Error analysis traces the dominant failure mode to the data layer — incorrect query composition and ORM runtime violations are the leading root causes.
That last detail is the one I'll be chewing on. The failures aren't "the model can't write Python." The failures are at the seam between what the model knows about a library's surface API and what it knows about the conventions of how that library expects to be used inside a real app.
Why it matters#
If you're building anything with agents that touches a backend — and at this point that's most of us — this is the failure mode that sneaks past your eval. SWE-bench-style fix-this-bug tasks measure behavior on a constrained surface. They don't measure whether the agent's edits are idiomatic, whether the new endpoint respects your ORM patterns, whether the migration matches how the rest of the codebase shapes migrations. Constraint decay says: the more your codebase has opinions, the worse agents do. That's the opposite of the cleanroom benchmark world we mostly evaluate in.
Concretely, two things I'm taking into how I build agent harnesses. First, structural verifiers belong in the inner loop alongside tests. Static checks for ORM usage, route registration, dependency injection patterns — these are cheap, deterministic, and they catch exactly the failures this paper surfaces. Second, framework choice is now a part of the agent-readability story. "Convention-heavy" frameworks compress code at the cost of implicit context, and that implicit context is where agents fail. If your stack is Django or FastAPI, your CLAUDE.md and architecture docs are doing more work than you think; if your stack is Flask, you've accidentally been making your codebase agent-friendly all along.
The caveats#
The study is Python web backends. Constraint decay is plausible in TypeScript, Go, or Rust ecosystems too, but the magnitude could differ — Spring or Rails would be a much harsher test.
100 tasks is a real eval, but it's not huge. The 30-point drop is an average; the variance across tasks and configurations is what would actually shape decisions in production.
The abstract names "capable configurations" without telling us which models or harnesses. The framework-by-framework breakdown for Sonnet vs GPT vs Gemini under different agentic scaffolds is the thing I really want to see in the full paper.
Static verifiers are only as good as the rules you encode. There's a meta-question lurking here about who writes the structural constraints and whether agents can learn to satisfy them from examples rather than explicit rules.
The takeaway#
Filing this one under "benchmarks that change my mental model." SWE-bench told us agents can fix bugs. FixedBench (yesterday's paper) told us agents fix things that aren't broken. Constraint Decay tells us agents struggle to fit in. The throughline is the same: behavioral correctness has stopped being the binding constraint on agentic coding. The next round of progress is about structure, restraint, and conformance — making agents pass the code review, not just the test suite. What I'm doing differently after reading this: adding a structural-conformance step to the verification loop on every agent harness I build, and treating framework conventions as a first-class part of the spec.