Skill Files Are the New Supply Chain Attack Surface

Last updated: August 12, 2026

Here is the strangest number we have read in months of watching agent security: an enterprise coding agent, operating on its own skill files, recognized that it was being attacked 1.99 percent of the time. The other 98 percent of the time it just executed. Not "considered and decided to comply". Executed, because nothing in the file it was reading looked like an attack to it. Across 5,629 completed runs, Gemini CLI was exploited in 95.5 to 96.1 percent of them by skill files that an automated pipeline had laundered from real shell commands into benign-appearing natural language (arXiv:2608.05223).

The attack surface is not a prompt. It is a folder. And the folder is the thing every agent product now ships.

The argument we are extending#

In late July we argued that agent memory is moving into the model, and that the external store - files, caches, skill folders, anything durable an agent reads - survives as verification and hygiene rather than retrieval cleverness (agent-memory-moving-into-the-model). The security half of that argument was blunt: durable state replays into every future run, so poisoning state beats injecting prompts, one planted document flips a deep-research agent to a false conclusion 54.7 percent of the time even with cross-model verification.

What has changed since is that the attack surface became measurable, and the measurements are worse than the theory predicted. The theory said state is an attack surface. The measurements say the skill interface - the exact format the whole ecosystem standardized on in the last twelve months - is currently the easiest attack surface in the stack, with exploit rates in a band nobody sane accepts for anything else. We think this is the sharpest instance yet of the position we have been building, and we want to lay out the evidence, name what would prove us wrong, and tell you what to do about it before the incident happens.

Act one: the file itself#

The paper behind that 95.5 percent number is a skill-synthesis attack. The authors took 471 real shell commands, transformed them into 2,826 adversarial "skills" - natural-language instruction files that describe a benign-sounding capability, mapped across 11 MITRE ATT&CK tactics - and ran them against two enterprise-grade coding agents. The results are nearly invariant to which model generated the skill, which matters: it means the vulnerability is in the interface, not the model. Qwen Code is exploited in 71.6 to 74.0 percent of runs. The safety layer recognizes what is happening 1.99 percent of the time, and that judge pipeline was itself validated against blind human annotation at kappa 0.85, so the 1.99 percent is not a measurement artifact.

The mechanism is the format. A prose skill file carries the command in natural language, which is exactly the representation that looks like documentation to a human reviewer and like documentation to the agent's own safety layer. The shell command is there, transformed: "git commit" becomes "finalize the working session state", and the agent dutifully finalizes the working session state. This is the same failure we documented in June when we argued config files are executable supply chain (agent-config-files-are-executable-supply-chain): the trust boundary sits on a file the agent executes without a review event. The difference is scale. Config files are static context. Skill files are load-bearing executable capabilities that every major harness now pulls in by default, and they are being shared, downloaded, and marketplaced like packages.

Act two: the promotion path#

The file is act one. Act two is worse, because it means the problem is not just the files you download - it is the files your agent writes for itself. Self-evolving agents distill reusable skills from their own trajectories, and that promotion path is now shown to be attackable at trivially low attacker support. With 10 percent of the evidence pool attacker-influenced, target behaviors embed in 91.0 percent of trials across six LLM evolvers, and three consistent attacker-authored records suffice in a 30-record batch (arXiv:2608.05563). Three records. An attacker does not need to own your agent's skill folder. They need to own a sliver of the evidence stream your agent learns from.

The mechanism behind that is structural: in self-evolving skill pools, a defective skill becomes reference material for later distillations, forming cross-round contamination chains that survive the removal of the source skill. Post-hoc rollback recovers only a fraction of the lost performance (arXiv:2608.05810). This is the point where the standard security instinct fails. The instinct says: audit the writes, and if something bad gets in, delete it. The measured result says deletion is not a repair, because the bad thing already trained the next thing. Admission is the only control that works, and it has to happen before the skill enters the runtime context, not after.

And there is a second trap inside act two: a verifier at the gate does not help if the evidence used to promote skills is attacker-influenced, because the gate reads the attacker's input. Content-level semantic checks capped at 7.4 percent detection under cloaking in our earlier coverage of this thread, and the promotion-path result is the same lesson one level up: what must be provenance-checked is not the skill content but the records that justify it. Whoever feeds the loop's selection signal owns the loop.

Act three: the carriers and the composition#

Acts one and two are about skills specifically. Act three is about why the whole class generalizes. A benchmark of persistent-state safety in agent harnesses - 328 executable cases across seven persistent-carrier families, from memory to skills to tools to shared artifacts - shows attacker influence persisting across system boundaries and firing later against benign triggers, with containment that is carrier-specific and harness-model dependent (arXiv:2608.06984). The practical reading: "we passed the security benchmark" is uninformative without a carrier breakdown, and hardening must happen per carrier, not per surface.

The same batch shows the attack class composing across steps. Multi-step indirect injection decomposes one adversarial goal into innocuous-looking sub-steps distributed across a chain of pages an agent navigates, and raising the chain length from one to three steps lifts attack success from 41.7 to 72.9 percent on one frontier model (arXiv:2608.06477). Each sub-step passes the per-page safety check. The composition is the attack.

Put the three acts together and the shape is clear: the file is a vector, the promotion path is a persistence layer, the carriers give it shelf life, and composition hides it in time. This is stored XSS with a long incubation period, and the "stored" half is exactly the durable state we flagged as the coming attack surface in the memory piece. It is arriving on schedule, just through the skill folder first.

The lifecycle half#

There is a counterintuitive bright spot, and it is the piece of the defense that is genuinely new: revocation as a first-class memory operation. A study of persistent memory under regime drift shows that append-only and last-write-wins stores do not merely fail to help when the world changes - they score below having no memory at all (0.210 versus 0.309), because superseded facts pollute the prompt forever. Making validity an explicit, revocable state - keyed precedents that get invalidated when fresh evidence contradicts them, with the revoked history preserved for audit - holds 0.950 through full drift reversal (arXiv:2608.07429).

The security translation is direct: the same mechanism that retires stale facts retires poisoned ones, and audit-preserving revocation (invalidate, don't delete) keeps the write path accountable. We have spent this year watching defense after defense fail on durable state. Revocation is the first primitive that works on the lifecycle itself rather than on the content, which is why content checks keep failing.

The counter-case, with the steel it deserves#

Before we get to the bet, the honest boundary. Four objections deserve real weight, and we want them on the record.

First, these are benchmarks, not incidents. The record of real-world state-poisoning incidents is thin; the documented agent escapes this summer were sandbox-boundary failures, not skill-store poisonings. Attackers target what ships, and skill files ship in every product now, but measured exploit rates on synthetic pipelines are not a breach report. We are predicting the incident class, not reporting one.

Second, hardened deployments are a different population. The 95.5 percent figure is measured with no active defense in the loop. Organizations running network-allowlisted, sandboxed, human-gated agents face a lower number, and we have not quantified how much lower. Neither has anyone else, which is exactly the problem: nobody can quote you the defended number because nobody has run the defense against the attack class.

Third, the vendor response is real and it is happening. Marketplace governance is shipping: allow and block lists at org scope, permission-class hardening, worktree isolation. The ecosystem is not standing still. The question is whether these controls are admission gates or boundary fences - and the phase-transition result says fences cannot fix what admission could have stopped, because the poisoned state self-replicates before the fence ever sees it.

Fourth, the counter-move may be cheap. If skill stores standardize on typed, compiled formats with deterministic provenance tracking - the direction our skills-thesis has tracked all year, where reuse becomes a deterministic audit surface (AUROC 0.938 on transformed-reuse detection, arXiv:2608.05204) - the natural-language laundering trick dies, because the executable content is no longer hidden inside prose. The attack surface may partially retire on format economics before the incident happens. We would be happy to be wrong in that direction, and we will grade ourselves on it.

What we believe now#

Here is the claim, stated so it can be graded: by late 2027, skill admission will be treated as a code-review event with runtime canaries in any agent product that distributes or self-evolves skills, promotion records will carry provenance checks, and revocation will be a standard memory operation. The falsifiable edge is the incident, not the practice: within that window, at least one widely reported compromise will trace its root cause to a skill or promotion-path poisoning, and the postmortem will name admission as the control that would have stopped it.

What would prove us wrong: a year without an incident while skill distribution grows, plus defended-deployment measurements showing the exploit band collapses below single digits with admission gates in place. We will report either outcome with the same care. And for the record, we run a skills library ourselves, which is precisely why we are writing this before it becomes a postmortem. Our own admission path just became a security review path, and we think yours should too.

What developers should do#

1. Treat skill admission as a code-review event. A skill file is executable code wearing a documentation costume. Review it like a PR: who authored it, what does it actually run, what can it reach. If your skill folders have no review event between download and execution, you have a supply chain with no gate.

2. Gate the promotion path, not just the content. If your agent writes its own skills, the evidence records that justify a promotion are the security boundary. Validate their provenance before the gate reads them - a verifier reading attacker-influenced evidence is just an expensive yes-man.

3. Ship revocation, not deletion. Poisoned and superseded state need the same primitive: invalidate the key, keep the history, let the read path know the difference. If your memory system's only tool is delete, it has no defense against the replay half of the attack.

4. Ask for the carrier breakdown. Next time a vendor says their agent passed a security evaluation, ask which persistent carriers it covered. If the answer is "the benchmark", you have learned something too.

5. Re-run your own exploit number. The 95.5 percent figure is one pipeline, one lab, no defenses. Take the benchmark's skill library, run it against your actual stack with your actual controls, and get your number. The gap between the unhardened and the hardened exploit rate is the only number in this post that actually matters for your deployment, and right now nobody can tell you what it is.

FAQ#

How often do adversarial skill files exploit coding agents?#

In the skill-synthesis study (arXiv:2608.05223), 2,826 adversarial skills built from 471 real shell commands exploited Gemini CLI in 95.5 to 96.1 percent of 5,629 completed runs, and Qwen Code in 71.6 to 74.0 percent. The agent's safety layer recognized the attack 1.99 percent of the time, and those runs had no active defense in the loop.

Why do safety checks miss malicious skill files?#

A prose skill file carries the shell command as natural language, so it reads like documentation to a human reviewer and to the agent's own safety layer. In the post's example, "git commit" becomes "finalize the working session state". Because results barely change with the model that generated the skill, the weakness sits in the interface rather than the model.

Can deleting a poisoned skill undo the damage?#

Not reliably. In self-evolving skill pools, a defective skill becomes reference material for later distillations, forming contamination chains that survive removal of the source skill, and post-hoc rollback recovers only a fraction of the lost performance (arXiv:2608.05810). That is why the post argues admission, before a skill enters the runtime context, is the control that works.

How should developers defend against skill file attacks?#

The post recommends treating skill admission as a code-review event, validating the provenance of the evidence records that justify a skill promotion, shipping revocation (invalidate the key, keep the history) instead of deletion, asking vendors which persistent carriers a security evaluation covered, and re-running the exploit number against your own stack and controls.

Continue Reading#

• Agent Memory Is Moving Into the Model - the original position this piece extends: durable state is the attack surface, and the external store survives as verification and hygiene
• Agent Config Files Are Executable Supply Chain - the trust boundary argument for agent-loaded files, now with measured exploit rates
• Agent Skills Package Manager Governance - how skill distribution platforms are thinking about governance today
• Kill Your Agent Runs Early - the lifecycle discipline that pairs with admission gates: gate before the run, kill early, restart with state
• The Judge Leaves the Loop - why content-level checks by LLM judges fail, and what replaces them
• Approve Effects, Not Invocations - the composition problem: a harmful objective distributed across several clean skill files evades per-file scanners, so admission has to reason over interactions
• Anthropic Self-Replicating AI Research: What Small Models Can Actually Do - Anthropic's replication research and the agent hardening it points to

Sources#

• Adversarial skill synthesis: arXiv:2608.05223 (2026-08-07)
• Skill-evolution phase transition and pre-commit admission: arXiv:2608.05810 (2026-08-07)
• PoisonedEvolution, promotion-path poisoning: arXiv:2608.05563 (2026-08-08)
• SkillTrace, deterministic reuse provenance: arXiv:2608.05204 (2026-08-08)
• StepJack, multi-step indirect injection: arXiv:2608.06477 (2026-08-10)
• HarnessSafe, persistent-carrier containment: arXiv:2608.06984 (2026-08-10)
• TEPA, revocable memory state: arXiv:2608.07429 (2026-08-10)