Add Radius skills plugin and Claude Code eval CI

.claude-plugin/marketplace.json

@@ -0,0 +1,22 @@
+{
+    "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
+    "name": "radius-cli",
+    "owner": {
+        "name": "Radius Technology Systems"
+    },
+    "metadata": {
+        "version": "0.0.3",
+        "description": "Claude Code plugins from Radius Technology Systems for AI-assisted blockchain development on the Radius Network"
+    },
+    "plugins": [
+        {
+            "name": "radius-dev",
+            "version": "0.0.2",
+            "description": "Radius Network tools for x402 payments, blockchain development, and testnet faucet",
+            "author": {
+                "name": "Radius Technology Systems"
+            },
+            "source": "./plugins/radius"
+        }
+    ]
+}

.github/workflows/plugin-evals.yml

@@ -0,0 +1,102 @@
+name: Claude plugin evals
+
+on:
+  pull_request:
+    paths:
+      - .claude-plugin/**
+      - plugins/radius/**
+      - scripts/validate_plugin.py
+      - .github/workflows/plugin-evals.yml
+  push:
+    branches: [main]
+    paths:
+      - .claude-plugin/**
+      - plugins/radius/**
+      - scripts/validate_plugin.py
+      - .github/workflows/plugin-evals.yml
+
+permissions: {}
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Validate plugin structure and scenarios
+        run: python3 scripts/validate_plugin.py
+
+  eval:
+    needs: validate
+    # Forked PR code never receives model credentials.
+    if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    permissions:
+      contents: read
+    env:
+      CLAUDE_EVAL_PROVIDER: ${{ vars.CLAUDE_EVAL_PROVIDER }}
+      CLAUDE_EVAL_MODEL: ${{ vars.CLAUDE_EVAL_MODEL }}
+      CLAUDE_EVAL_JUDGE_MODEL: ${{ vars.CLAUDE_EVAL_JUDGE_MODEL }}
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Install Claude Code
+        run: |
+          curl -fsSL https://claude.ai/install.sh | bash -s 2.1.272
+          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
+      - name: Validate Claude plugin
+        run: claude plugin validate plugins/radius
+      - name: Run Claude plugin evals
+        working-directory: plugins/radius
+        shell: bash
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+        run: |
+          set -euo pipefail
+          if [[ "$CLAUDE_EVAL_PROVIDER" == "openrouter" ]]; then
+            if [[ -z "$OPENROUTER_API_KEY" ]]; then
+              echo '::error::Set OPENROUTER_API_KEY to run plugin evals.'
+              exit 1
+            fi
+            export ANTHROPIC_BASE_URL=https://openrouter.ai/api
+            export ANTHROPIC_AUTH_TOKEN="$OPENROUTER_API_KEY"
+            export ANTHROPIC_API_KEY=''
+            if [[ -z "$CLAUDE_EVAL_MODEL" || -z "$CLAUDE_EVAL_JUDGE_MODEL" ]]; then
+              echo '::error::Set CLAUDE_EVAL_MODEL and CLAUDE_EVAL_JUDGE_MODEL for OpenRouter.'
+              exit 1
+            fi
+          elif [[ -z "$CLAUDE_EVAL_PROVIDER" || "$CLAUDE_EVAL_PROVIDER" == "anthropic" ]]; then
+            if [[ -z "$ANTHROPIC_API_KEY" ]]; then
+              echo '::error::Set ANTHROPIC_API_KEY to run plugin evals, or configure OpenRouter.'
+              exit 1
+            fi
+          else
+            echo '::error::CLAUDE_EVAL_PROVIDER must be anthropic or openrouter.'
+            exit 1
+          fi
+          claude plugin eval . \
+            --trust-plugin \
+            --json results.json \
+            --threshold 0.75 \
+            --runs 2 \
+            --ablation none \
+            --model "${CLAUDE_EVAL_MODEL:-claude-sonnet-5}" \
+            --judge-model "${CLAUDE_EVAL_JUDGE_MODEL:-claude-haiku-4-5}" \
+            --no-publish \
+            --max-cost-usd 10
+      - name: Upload eval report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: claude-plugin-evals
+          path: |
+            plugins/radius/results.json
+            plugins/radius/evals/results/
+          if-no-files-found: ignore
+          retention-days: 14

.gitignore

@@ -3,3 +3,5 @@ dist/
 .env
 .wrangler/
 *.log
+plugins/radius/evals/results/
+plugins/radius/results.json

README.md

@@ -15,6 +15,41 @@ pnpm add radius-sdk viem          # SDK, buyer / agent side
 
 Runnable SDK examples (seller worker, agent buyer, browser demo dapp) are in [`packages/sdk/examples`](./packages/sdk/examples).
 
+## Agent skills
+
+The [Radius Claude Code plugin](plugins/radius) contains the `radius-dev`, `x402`,
+and `dripping-faucet` skills. The marketplace manifest is at
+[`.claude-plugin/marketplace.json`](.claude-plugin/marketplace.json). These files
+live outside `packages/*`, so plugin changes do not enter the npm release flow
+or need a changeset.
+
+In Claude Code, install from this repository:
+
+```text
+/plugin marketplace add https://github.com/radiustechsystems/radius-cli.git
+/plugin install radius-dev@radius-cli
+```
+
+For skill changes, run `python3 scripts/validate_plugin.py` and
+`claude plugin validate plugins/radius`. The path-filtered
+[`plugin-evals.yml`](.github/workflows/plugin-evals.yml) runs Claude Code plugin
+evals on trusted plugin changes. Add or update cases under
+[`plugins/radius/evals`](plugins/radius/evals) with each behavior change, inspect
+the CI report, then revise the skill or case based on the observed result.
+The existing skill `evaluations/*.json` files remain as scenario references;
+Claude Code uses the `evals/` suite for executable checks.
+
+CI accepts either an `ANTHROPIC_API_KEY` secret, or an `OPENROUTER_API_KEY`
+secret with repository variable `CLAUDE_EVAL_PROVIDER=openrouter`. Set
+`CLAUDE_EVAL_MODEL` and `CLAUDE_EVAL_JUDGE_MODEL` repository variables to model
+IDs supported by the chosen provider. The defaults target Anthropic. Fork PRs
+run static checks without model credentials; trusted PRs and main pushes run
+the paid eval suite. No eval case is allowed to execute wallet or payment tools.
+
+The older skills repository still owns its live Hermes subscriber publisher.
+Moving that publisher requires updating subscribers' repository and commit
+settings first; this plugin migration does not send those webhooks.
+
 ## Development
 
 ```bash

plugins/radius/.claude-plugin/plugin.json

@@ -0,0 +1,8 @@
+{
+  "name": "radius-dev",
+  "version": "0.0.2",
+  "description": "Radius Network tools for x402 payments, blockchain development, and testnet faucet",
+  "author": {
+    "name": "Radius Technology Systems"
+  }
+}

plugins/radius/evals/wallet-testnet/graders/behavior.md

@@ -0,0 +1,4 @@
+---
+type: llm
+---
+PASS if the reply uses a project-scoped RADIUS_HOME, testnet network, radius-cli wallet address, and radius-cli wallet x402 with an explicit display-unit threshold. It must avoid private-key exposure and must not claim a payment occurred. FAIL if any requirement is missing, or if it uses a raw private key or Foundry interactive import as the agent wallet path. Judge semantics, not exact wording.

plugins/radius/evals/wallet-testnet/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:[\w-]+:)?(?:radius-dev|x402)"'
+---

plugins/radius/evals/wallet-testnet/prompt.md

@@ -0,0 +1,5 @@
+---
+max_turns: 5
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+I need a fresh Radius testnet wallet for a terminal agent demo, then one paid x402 request. Explain the setup and commands without running them or asking me for a key.

plugins/radius/evals/x402-client/graders/behavior.md

@@ -0,0 +1,4 @@
+---
+type: llm
+---
+PASS if the reply selects radius-cli wallet x402 with explicit testnet and threshold, treats the threshold as display units, and distinguishes HTTP success from verified settlement or receipt evidence. FAIL if it asks for a private key, suggests an unbounded payment, executes a payment, or equates HTTP 2xx with settlement. Judge semantics, not exact wording.

plugins/radius/evals/x402-client/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:[\w-]+:)?x402"'
+---

plugins/radius/evals/x402-client/prompt.md

@@ -0,0 +1,5 @@
+---
+max_turns: 5
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+I have a testnet x402 endpoint and want to call it once from a shell agent. Show a safe command and explain how I tell whether payment settled. Do not run anything.