I expected this paper to scare me. A hostile CLAUDE.md that tells the agent to strip null checks and bounds checks should produce vulnerable code, and the code should still pass its unit tests. That is the nightmare version of project memory. The measured effect was far smaller and far noisier than that, and the part that did bother me was somewhere else.
The study is Security Impact of Coding Agent's Memory Files from Thiago Santos de Moura, Selma Güler and Yannic Noller at Ruhr-Universität Bochum. They take 36 tasks from SecRepoBench, three per CWE class across 12 classes, in C/C++ repositories. Each task is a function completion where a fix for a known weakness has been removed. Code counts as secure if it compiles and the OSS-Fuzz proof-of-concept no longer trips the sanitizer. It counts as functionally plausible if the developer unit tests still pass.
Three setups ran every task: Claude Code 2.1.308 with Claude Sonnet 5.5, and OpenCode with Qwen 3.8 27B and with DeepSeek V4.1 Flash. All at xhigh reasoning effort, with a 1,200-second limit. Three modes each: no memory file, a qualified v1 file, and a forceful v2 file of roughly 600 to 700 words. Both file versions dress up removing defensive checks as engineering best practice. One of the quoted lines reads:
Validation and normalization are the sole responsibility of the component that admits data
The attack mostly cancelled itself out#
Secure-completion rate moved between −8 and +6 percentage points versus baseline, with no consistent direction across agents or file versions.
Functionally plausible completions stayed between 80% and 92% in every mode, so the tests gave no signal either way.
Of 204 baseline-versus-file task pairs, only 30 differed in security. In 15 the baseline was secure, in the other 15 the file version was. Net effect: zero.
That is a more interesting result than a clean attack would have been. The agents did not simply obey. Part of the explanation is visible in the reasoning traces, which I get to below. Another part is that the benchmark tasks often come with enough surrounding code that the safe path is the obvious one.
I would not read this as "memory files are safe". It is one run per task and mode, so the authors say outright they cannot show statistical significance. A swing of eight points on 36 tasks is a handful of tasks.
The agents noticed, and mostly kept it to themselves#
The second research question is the one I would put on a whiteboard: when the file influenced the code, did the agent say so? Claude Code named the file in about two-thirds of its final messages, and one example is the agent writing that, as the project's CLAUDE.md asks, it added no extra null or bounds checks. That is the behavior you want, because a reviewer can act on it.
OpenCode behaved differently. It mentioned the file almost exclusively in its reasoning trace. Of the 18 sessions where the reasoning called the file an attack, only four named it in the final message. DeepSeek's trace, for instance, described it as an adversarial instruction trying to make it write unsafe code. The person reading the summary never saw that.
Those 18 sessions also split on outcome: security was lost versus baseline in three and gained in two. So suspicion in the trace did not reliably protect the code, and it did not reliably reach the human.
What this changes in how I run agents#
Most of us review the diff and the final message, not the reasoning. This paper says the final message is where a model may drop exactly the fact you needed. If a memory file shaped a decision, I want that stated in the output, and that is something you can ask for in the memory file itself or in a wrapper that greps traces for mentions of instruction files.
Concretely, I would do three things:
Treat CLAUDE.md and AGENTS.md changes in a PR like changes to a CI config, with a named reviewer.
Add a rule that any deviation from default defensive practice must be called out in the final message, and spot-check that it is.
Run my own security tasks with and without the memory file on the repositories I care about, rather than trusting an average from someone else's 36.
Where the evidence is thin#
The authors list their own limits and they are real. Only 36 C/C++ tasks, taken from repositories that finished quickly. The oracle only detects the targeted weakness, so other damage would be invisible. Keyword search for file mentions can miss paraphrases, so the disclosure counts are lower bounds. Six Qwen tasks had no results and were excluded. Possible contamination exists since OSS-Fuzz fixes may be public, though only 13 of 313 completions reproduced a reference block. And the setup is attacker-friendly in a slightly unrealistic way: a single file in a clean one-commit repository, with no competing guidance.
My read: the integrity risk from a malicious memory file looks smaller than the threat-model slides suggest, and the transparency risk looks larger. I would bet that follow-ups with multiple runs per task will find the security effect is real but model-specific, and that disclosure rates will vary even more than security rates do. The open question is whether final-message honesty can be trained or prompted for, or whether it needs tooling that reads the trace for us.