Here is a failure that no dashboard of yours will ever show: the task passes, the diff looks fine, and the skill you installed to enforce a rule never ran. In One Skill Too Many, Yan et al. measured how often that happens when two skills claim the same job. Their number: a similar skill takes one in five runs away from the installed one, with no drop in task completion.
One of their examples is the kind I would have shrugged at. A project skill called checkpoint forbids any change to git state. Add a similar skill, and the model used that one instead, committed the work, and never mentioned which skill it had used. Both runs completed the task.
Skills compete on a name and a description, nothing else#
A skill is a directory with a SKILL.md that tells the model when and how to do something. The model picks between them using only name and description. That is a routing decision made by a language model, and it gets no review.
The authors mined 822,109 candidate similar-skill pairs from 20,947 repositories, had an LLM judge review a stratified sample of 3,754, and ran 312 confirmed pairs on Sonnet 4.6, Haiku 4.5 and Opus 5 inside Claude Code. That produced 6,368 runs, 169,294 tool calls and 542 agent-hours. Nearly one in four installed skills sits next to one that does the same job, and 37% of judged skills live inside copied collections. If you have ever pulled in a community skill pack on top of your own, you are in that population.
Most of the conflicting installed skills are normative: rules about how to behave, like the git one above. Capability skills come second. That matters because normative skills are the ones whose absence is invisible. A missing capability produces a visibly worse result. A missing rule produces a result that looks fine.
The headline numbers#
Co-installing a similar skill cut use of the installed skill by 19.9 points (95% CI 16.7 to 23.1), against 5.9 points for an unrelated skill.
With both installed, 12.1% of runs used only the similar skill and 38.0% used neither.
Runs that opened the similar skill first lost over a third of the exclusive core functions that only the installed skill provides.
When the similar skill replaced the installed one, the final reply named it in just 0.9% of runs.
The last one is the number I keep coming back to. Reading the transcript summary, which is what most of us do, would almost never tell you the wrong skill ran. This is the same lesson as treating an agent's final summary as a claim rather than evidence, with a new place for it to bite.
Where the skill lives beats where it is listed#
I expected listing order to matter. It barely does: +0.6 points. Install scope matters a lot. A personal-scope skill overrides the project one outright. The project skill vanished from the listings, and its use fell 35.0 points, although the model still read its files in 29.2% of runs. Plugin-scope skills sit alongside instead: the installed skill was used in 58.0% of runs and the plugin skill in 2.9%, a drop of 6.2 points.
So the practical advice is boring and specific. Whatever is in your home directory can silently shadow what your repo ships. If a rule matters to the project, check what a teammate's personal skills do to it.
Everything is decided at the first read#
The part I would steal is the intervention. Conflicts are settled at the first skill read, almost always before any file changes. So the authors put a pre-tool hook on that read. When the hook denied the look-alike, the model switched to the installed skill in 96% of cases (Sonnet 4.6 94%, Haiku 4.5 96%, Opus 5 99%).
Fidelity on exclusive core functions rose 9.1 points overall, and by 12.8 to 20.7 points among runs that had reached for the look-alike first. Task completion moved by +4.1 points, with a confidence interval of −1.3 to +9.6, so nothing was traded away that they could detect.
Benchmarks should score exclusive core functions, and platforms should guard the first skill read and show which skill was used.
That is my paraphrase of their conclusion, not a direct quote, but it is the right instinct. A hook is cheap, and you already have the mechanism if you run Claude Code.
How far I would trust it#
Each combination of model, configuration and pair ran once, and the authors say runs are nondeterministic. The full design ran only on Claude Code, at about $5,550, with a Codex check on 193 pairs. Only 48% of installed skills in confirmed pairs were runnable offline, and scripts decided 327 of 921 primary core functions; an LLM judge scored the rest. I would treat the 19.9 points as an order of magnitude, not a constant. The mechanism, though, is easy to reproduce on your own setup in an afternoon.
What I am doing about it#
Three changes. I will audit my installed skills for overlapping descriptions, and make the ones carrying hard rules distinctive enough that a look-alike cannot win a description match. I will log which skill was actually read per session rather than trusting the reply. And any rule I cannot afford to lose moves out of a skill into a hook, because a hook does not compete on a description.
My prediction: skill registries will need namespacing and conflict warnings within a year, the way package managers eventually grew them for name collisions.