Skimming this morning's arxiv drop, most of the software-engineering papers were the usual leaderboard shuffle — a new eval, a new benchmark row. Then one title stopped me: “Obey, Diverge, Collapse.” Its premise is a quiet heresy against how we all test coding agents: every benchmark assumes the instruction the model receives is correct. This paper asks what happens when it isn't — and the answer is worse than “the model gets it wrong.” My thesis: the scariest failure inside a repair loop isn't ignorance, it's obedience.
What it does#
Obey, Diverge, Collapse (Jaiswal et al., IIIT Delhi / Microsoft Research India / IIT Kanpur) runs five code LLMs — GPT-5.3 Codex, Claude Sonnet 4.6, Qwen3-Coder, GLM-5, and Kimi K2.5 — through a chain of four experiments on 538 deterministic Python bug-fix problems from RunBugRun. The design move that makes it different: instead of feeding the model a correct instruction or a vague one, it feeds a plausible but wrong one — a human-style misdiagnosis that confidently fingers the wrong root cause, while the executable test cases quietly contradict it. Because RunBugRun has deterministic oracles, the conflict is unambiguous: the tests say the model is wrong, and we get to measure whether the model notices, and whether noticing changes anything. The four questions form a progressive chain — does the model obey a wrong instruction, how far can clean self-repair recover a buggy patch, do the errors compound across iterative wrong instructions, and once they've compounded, can correct self-guided repair ever undo them.
The key result#
The headline is a decoupling that should unsettle anyone shipping an agent. In the evaluator role — asked plainly “is this instruction correct?” — the models nail it: GPT-5.3 Codex flags the wrong instruction as wrong in 529 of 538 cases (98%), Qwen3-Coder in 515, Claude Sonnet 4.6 in 464. Then, handed the same instruction in the generator role on the same context window, they follow it anyway. Among the cases where the model explicitly knew the instruction was wrong, obedience ran to 42% for Qwen3-Coder (218/515), 42% for GLM-5, and 37% for Claude Sonnet 4.6 — GPT-5.3 Codex was the most resistant at roughly 15%. McNemar's test confirms the asymmetry is systematic across all five models (p < 0.001). Detection is not resistance. The model's judgment about the instruction never reaches the generation process — the instruction arrives as input and is acted on as input, regardless of what the model knows about it.
Why it matters#
Two things here rewire how I think about repair loops.
First, the compounding. This isn't a one-shot wrong answer — it's a corrupted starting state for every pass that follows. The paper names the artifacts “Ghost Errors”: structural faults the model introduces by patching the wrong location, which then displace the original bug's intent a little further with each iteration. And once that damage is in, correct self-guided repair can't dig you out. Problems that entered the damaged state basically never escaped across five clean recovery passes; Qwen3-Coder, the most obedient model, arrived at the recovery stage with about 80 irrecoverable problems, versus a handful for GPT-5.3 Codex. Recovery is front-loaded — if it doesn't happen in the first two passes, more passes don't help. If you run a fixed repair budget of 5 or 10 iterations on the theory that persistence pays off, this says persistence past a wrong turn just chases ghosts.
Second, the counterintuitive one: a wrong human diagnosis does more damage than no diagnosis at all. For the open-source models, pass rates under a confidently-wrong instruction fell below the self-thinking condition where the model got no external guidance at all. That inverts a core assumption of how we prompt agents — we hand them the bug report, the Slack thread, the reviewer's guess, on the theory that context helps. When that context is wrong, it doesn't just fail to help; it's actively worse than silence, because the model treats a terminal directive as ground truth it has no arbitration layer to overrule. And more thinking doesn't save you: raising the reasoning budget made most models worse, burning the whole token budget generating chains before any code came out. The barrier is structural, not a compute problem.
For a builder, the move is to stop treating the upstream instruction as trusted input. If your sub-agent architecture pipes a “diagnosis” or “plan” from one agent into an editor agent, that hand-off needs an arbitration step where execution evidence can override the instruction — not just an evaluator agent that says “this looks wrong” and gets ignored downstream, which is exactly the failure mode this paper reproduces inside a single model. Checkpoint the pre-repair state and diff against the tests each pass; if the failing-test count rises instead of falls, that's a wrong-turn signal to roll back, not to iterate harder.
The caveats#
Narrow domain. RunBugRun is single-function algorithmic Python with clean deterministic oracles. Real repos have fuzzy tests, multi-file blast radius, and no golden signal — the “irrecoverable” claim may look different where the model has richer structure to reason over.
Synthetic wrong instructions. The incorrect instructions in the iterative stage are generated by a GPT-5.1 Codex proxy playing a misdiagnosing reviewer, not mined from real bug reports. Plausible, but a modeling choice.
Bounded reasoning ablation. The “more reasoning doesn't help” result comes from a 100-problem subset at zero/low/medium levels — no frontier xhigh thinking budgets, so it's bounded to that range.
Fuzzy line. “Blind obedience” versus reasonable deference isn't crisp. Sometimes following a confident instruction over your own read is correct; the paper assumes the tests are the sole arbiter, which they are here but aren't everywhere.
The takeaway#
What I'm filing away: an evaluator agent that can correctly flag a bad instruction buys you nothing if that judgment doesn't gate the editor. The whole “reviewer sub-agent” pattern I've been enthusiastic about is only as good as the wire between detection and action — and this paper shows that wire is missing by default, even inside a single model that already knows the answer. What I'm doing differently: in any repair loop, I'll treat a rising failing-test count across passes as a hard stop-and-rollback, not a reason to spend more iterations — because past a wrong turn, the model isn't fixing the bug anymore, it's chasing the ghosts it just created.