AgentSkill Security

Official implementation for the paper: Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry (Paper Link).

This project studies SKILL.md-only semantic supply-chain attacks across the three registry-facing phases described in the paper:

1. Discovery: manipulate how adversarial skills are retrieved/ranked.
2. Selection: bias agent skill choice among discovered candidates.
3. Governance: evade registry moderation/scanning checks.

The core idea is that SKILL.md is operational text, not passive documentation: small natural-language changes can affect which skills are surfaced, selected, and accepted.

Attack Overview

Repository Overview

• benign_skills/

  • Clean baseline skill corpus used across experiments.

• discover/

  • Phase 1 implementation: discovery manipulation and score analysis.
  • See: discover/README.md

• selection/

  • Phase 2 implementation: paired skill-selection experiments with Hermes-based harnesses and scenario generation.
  • See: selection/README.md

• governance/

  • Phase 3 implementation: moderation pipeline replication and semantic-evasion attack variants.
  • See: governance/README.md

• Each phase implementation includes an environment.yaml for creating a phase-specific Conda environment for that experiment.

Phase-by-Phase Mapping to the Paper

1) Discovery Phase (discover/)

Implements black-box and white-box SKILL.md trigger optimization workflows, then evaluates resulting discoverability shifts with ClawHub scoring/vector-score comparisons.

• Black-box beam-search based trigger/delta optimization and merge
• White-box gradient-based payload generation and injection
• ClawHub scoring analysis (single-skill and category-wide)

Entry point:

• discover/README.md

2) Selection Phase (selection/)

Implements scenario construction and execution for measuring whether modified skills are selected over functionally similar benign variants.

• Prompt and scenario generation utilities
• Adversarial variant skill sets per strategy
• Agent based execution harness and run artifacts

Entry point:

• selection/README.md

3) Governance Phase (governance/)

Implements moderation-side evaluation and semantic evasion testing.

• ClawHub-style moderation pipeline (submit/finalize)
• Multiple attack-variant generators (malicious, paraphrase, DoD, append-after-canary, jailbreaking-evaluator)
• Batch moderation scripts over pre-generated variant corpora

Entry point:

• governance/README.md

Citation

To cite our work:

@misc{saha2026hoodskillmdsemanticsupplychain,
      title={Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry}, 
      author={Shoumik Saha and Kazem Faghih and Soheil Feizi},
      year={2026},
      eprint={2605.11418},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2605.11418}, 
}