Your instruction file is probably the most trusted thing in your repo, and it is also the thing most likely to be out of date. The question nobody tests is what the agent does when CLAUDE.md says one thing and the code says another. Xiaonan Xu and Wenjing Wu tested it in Stale Instruction Files Override the Code in Coding Agents, and the answer is uncomfortable: the file wins, about nine times out of ten.
The setup is small and the conflict is deliberate#
They built 40 small Python projects, each with a passing test suite, a CLAUDE.md or AGENTS.md, and a feature request. In every project the instruction file asks for one convention while the existing code uses the other. Sixteen kinds of convention, both directions. Claude Code 2.1.302 on Sonnet 5.5 and Codex CLI 0.163.0 on GPT-6 Sol each got 2 runs per project per condition, 320 runs in total.
Without any mitigation, the agents followed the stale file in 90.1% of choices (649 of 720). They followed the existing code in 4.2%. Claude Code sat at 86.4% following the file, Codex CLI at 93.9%.
Noticing the conflict did not change the behavior#
This is the part I keep coming back to. Claude Code named the conflict in its final reply in 79 of 80 runs. It saw the problem, told the user about it, and still followed the file in 86.2% of its choices in those runs. Awareness and action are decoupled. If you only audit the final summary, you will see a diligent agent and ship the wrong convention anyway.
Codex CLI was worse in a different way. It mentioned the conflict in just 2 of 80 replies, and in 20 of 80 runs it rewrote existing code to match the file. The original tests failed in 17 of those 20. The median rewrite removed 198 lines across 9 pre-existing files. Claude Code never rewrote existing code in either condition, so the failure modes are not interchangeable: one agent is quietly inconsistent, the other quietly destructive.
It also changes how I read agent transcripts. A reply that says "I noticed the docs and code disagree" sounds reassuring, and until now I treated it as a sign the agent had handled things. The 79 of 80 figure says otherwise: the flag and the decision come from different places in the model's priorities, and only one of them showed up in the summary.
There is a cost angle too. Codex CLI's rewrites meant removing hundreds of lines of working code to satisfy a sentence in a markdown file, then failing the existing tests in 17 of 20 cases. That is a full repair loop spent undoing damage the instruction file caused, which is the kind of waste nobody sees in a pass-rate table because the final number only counts the end state.
The fix is one sentence at user level#
The authors add a single instruction to the user-level file (~/.claude/CLAUDE.md or ~/.codex/AGENTS.md): when documentation and code disagree, follow the conventions in the existing code and report each inconsistency in the final reply. The results:
Code-following went from 4.2% to 91.4% of choices, a gain of 87.2 points (95% interval 83.6 to 90.9).
Claude Code moved from 7.5% to 94.4%; Codex CLI from 0.8% to 88.3%.
Both agents reported the conflict in all 160 runs with the sentence.
No Codex run rewrote existing code, and the test pass rate went from 89.4% to 100% (160 of 160).
The authors' explanation is that the instruction file arrives as an instruction while the code arrives as tool output, which models are trained to rank lower. The sentence works because it is itself an instruction that says to prefer the code. They argue this from the results; they did not test the mechanism directly. I find it plausible, and it matches how I think about priority in prompts, but treat it as a hypothesis.
What I would change in my own setup#
First, add the sentence. It costs nothing and the effect size is huge. Second, notice the placement: user level, not the project file. A stale project file is exactly the thing that cannot be trusted to contain its own correction, and a rule that lives somewhere else survives the rot.
Third, stop treating the instruction file as ground truth in review. I would rather have a check that diffs claimed conventions against what the code actually does, because the agent evidently will not do that arbitration for me. Even a crude script that greps for the style rules in CLAUDE.md and counts violations in the repo would catch the drift before the agent amplifies it.
Fourth, for anyone building agent products: this reads like a priority bug. Putting repository-derived evidence below user-authored text is a defensible default, but the default should not mean silently executing a contradiction the model has already flagged.
Where I would not stretch the result#
Forty synthetic Python projects, single unattended runs, two agents, and the instruction file is always the outdated side. Real repos have conflicts where the file is right and the code is the legacy mess you are trying to migrate away from. A blanket "follow the code" rule would have the opposite problem there, and the paper does not measure it. The conflict judgments also come from an LLM judge (Claude Opus 5.5) with a criterion fixed before the runs, and the 20-line threshold for counting a rewrite is the authors' operational choice.
So my read: the 90% finding is solid enough to act on, the 91% fix is a strong first draft, and the unanswered question is a version of the sentence that distinguishes stale docs from intentional migrations. I would bet that the best wording ends up asking the agent to flag and ask, not to pick a side.