← all writing
07 · 12 Nov 2026 · 7 MIN READ

Stop paying your agent to re-orchestrate the same skill on every run

Most of what my agents do when they run a skill is clerical. Read the SKILL.md, compose a command, copy a path out of one tool result into the next call, check the output, repeat. The model is expensive, and I am paying it to be a very slow shell script.

A team at NVIDIA put a number on how much of that can be stripped out. In How Much Skill Orchestration Can LLM Agents Compile Away?, they record one completed skill invocation in Claude Code, compile it into small programs plus a few natural-language steps, and let a plain executor drive those programs on the next request. On all 87 SkillsBench tasks, the repeat run used 81% fewer tokens with Sonnet 4.6 and 74% fewer with Opus 4.8, with mean reward inside run-to-run noise (0.538 to 0.546 for Sonnet, 0.692 to 0.690 for Opus).

That headline is nice, but it is not the part that changed my thinking. The interesting part is what did not work.

Writing the script up front made 58 of 84 tasks more expensive

The obvious move, and one I have tried myself, is to hand a model the skill and the request and say: turn this into a script so we never orchestrate it by hand again. The authors test exactly that, calling it zero-shot specialization, on 84 tasks with Sonnet 4.6.

Once you count the compile call, 58 of 84 tasks used more tokens than just running the skill directly. Total tokens dropped 5%, but a single task held 18% of the direct-execution tokens; take it out and the total rose 11%. Reward went up on 10 tasks and down on 12.

The failure modes are the ones anyone who has generated glue code from docs will recognise. On 32 tasks a compiled program failed loudly and the model had to recover through the skill anyway, paying twice. On 10 others a program returned a wrong answer without any error, and 9 of those scored zero. The skill text describes the procedure, but it does not tell you the actual file paths, argument shapes and branch the task takes at runtime. The compiler guesses, and guesses are where it breaks.

Even when they bolted their own executor onto the zero-shot capsules, those only beat direct execution on 30 of 84 tasks once compile cost was included, against 67 of 87 for capsules built from a real run.

One recorded run is what turns a skill into code

SkillCache borrows its design from tracing JIT compilers: interpret once, record the path, compile that path, and fall back when an assumption breaks. In practice:

  1. Observe. The cold run executes normally with the skill. Its trace (tool calls, arguments, results, errors, agent messages) is saved.

  2. Compile. An isolated compiler agent, which never sees the benchmark's tests, prunes failed exploration, fuses adjacent deterministic tool calls into programs, keeps judgment calls as natural-language steps, and lifts request-dependent values into named bindings.

  3. Qualify. Three gates with no model involved: dataflow checks, a build or syntax check per program, then an end-to-end replay in a fresh sandbox. Failures go back to the compiler for up to three repair attempts, and if nothing qualifies it falls back to a plain written procedure.

  4. Execute. On the warm run, a CLI tool called skillcache-run runs programs back to back and only hands control to the model when it needs a binding, a judgment step, or a program fails.

The fallback is per step. If a program exits nonzero twice, times out, or omits a declared output, only that step goes back to the model, along with the program's source so it can see how the step was meant to work. In the main runs that happened in 11 of 481 warm executions of hybrid capsules.

The line from the paper that sums up the philosophy:

Only judgment needs a model, yet every recurring request has the model redo the mechanical work.

Pasting the old transcript back in only gets you halfway

If you already do some form of trajectory reuse, the baseline comparison is the most useful table. Each warm context ran in the same harness with the same skills:

None of the three moved mean reward by more than 0.01. So the cheap trick, stuffing the last successful run into the prompt, already saves about half. That's worth knowing on its own.

The ablation splits the remaining gap in two. Running the same capsules agent-driven, where the model calls each program itself, costs 26% more tokens with Sonnet and 21% more with Opus than letting the executor drive. And even agent-driven, the capsule beat the raw trace and the written procedure, which the authors attribute to the model seeing programs through their interface instead of reading the code. Savings track model calls almost exactly: when every step is a program, the model makes about three calls per task, and tokens drop 89% (Sonnet) and 87% (Opus). Textual-only capsules saved just 56% and 48%.

Every model call rereads the context. Remove the calls and you remove the cost.

Replay with the original inputs won't catch a hard-coded path

I would not ship this as-is, and the paper is fairly upfront about why.

The main result replays the same request that was compiled. To test generalisation they built 66 probes on 24 tasks that changed an input value, renamed files, or reworded the request. Token savings held at 70 to 87% and reward stayed level or rose. But the probes were built only from tasks whose cold run succeeded, and three Sonnet capsules hard-coded request-specific values and produced wrong answers on the probes where the old answer fails. Replay qualification ran with the original values, so it could not catch that. That is the silent-wrong-answer failure from the zero-shot study, now in a smaller form.

Then the scope: one benchmark, one harness, and each task is handed its own capsule. There is no retrieval, no cache invalidation when a skill or environment changes, and no test of what happens when a stale capsule matches a request it shouldn't. For a production skill library, those are the hard parts.

The economics are more convincing than I expected. Compiling all 87 tasks cost $67 with Sonnet and $116 with Opus, including repairs and fallbacks. One reuse saved $64 and $79, so it breaks even after 1.1 and 1.5 reuses in aggregate. Anything you run weekly pays for itself.

Compile with the expensive model, replay with the cheap one

The result I keep coming back to: capsules compiled by Opus raised the mean reward of DeepSeek-V4-Flash, DeepSeek-V4-Pro and Kimi-K3 by 0.16 to 0.25 while cutting their tokens 72 to 81%, bringing them close to Opus's own 0.690. DeepSeek-V4-Flash at low effort running Sonnet's capsules went from 0.437 to 0.553, slightly above Sonnet's 0.546.

That suggests a split I'd like to try in a sub-agent setup: let the frontier model do the first run of a recurring workflow (release notes, dependency bumps, a data export), compile it, and hand the capsule to a cheap worker for every repeat. My guess is that within a year, skills in harnesses like Claude Code will ship two artifacts: the SKILL.md for the first run, and a recorded, replay-tested program path for the rest. What I'm doing this week is smaller: for each skill I run more than twice, I'm taking one good trace and pulling the deterministic steps into a script with explicit inputs, so the model only touches the steps that actually need judgment.


Working on something similar?

Say hello — I read every email.