diff --git a/CLAUDE.md b/CLAUDE.md index 774fe55..b8657c8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,8 +37,13 @@ scripts/ clone-and-claude.ps1 # Clone repo & launch Claude (Windows) csa-plugins.txt # Public default plugin list (fetched from HEAD at runtime) csa-plugins-internal.txt # CSA-internal default plugin list (fetched from HEAD at runtime) +tools/ + check-all.sh # Everything CI runs, locally, in one command + sweep-csa-sources.sh # Weekly drift sweep (network + gh, NOT in check-all.sh) archives/ # Previous script versions for reference -docs/ # Design documents (e.g., Windows AI tools design/process) +docs/ + periodic-sweep.md # Weekly sweep runbook — what drifts and where to fix it + mcp-servers.md # Historical MCP config reference (third-party servers) TODO.md # Audit findings, priority-grouped with file:line citations .github/ ISSUE_TEMPLATE/ # Issue templates for contributions @@ -186,6 +191,16 @@ The plugin-install contract is shared across all five scripts — `macos-ai-tool 4. **Output is silent unless registration actually happened.** On success, print a `Registered Claude Code MCP server: csa-mcp` line followed by `Run /mcp inside Claude Code to authenticate with the CSA MCP server.` (the OAuth flow is browser-driven and must be initiated by the user). On `add` failure, print a warn line with the captured stderr indented underneath, matching the marketplace-add error format. 5. Currently Claude Code only. Codex and Gemini support OAuth-HTTP MCP transports too but their config formats differ; adding them is future work. +### Local CSA MCP servers (`setup_csa_internal_tools`) +Separate mechanism from the hosted `csa-mcp` above, and a **third** place the lists drift. The same five scripts run `setup_csa_internal_tools`, which `gh`-probes the gate repo and then fetches one setup script per server from `CloudSecurityAlliance-Internal/CSA-Plugins/internal-setup/`, executing each with `CSA_NESTED=1`. The servers live in their own public repos (`csa-google-workspace`, `csa-skilljar`, and `csa-zendesk` when it is ready); the setup scripts live in the private gate repo because they carry CSA's OAuth client. A server is wired up by appending its `-setup.sh` to the `setups=()` array — in all five scripts, with a `SCRIPT_VERSION` bump each. The loop uses `continue`, not `return`, so a setup script that is absent (unmerged, renamed) cannot silently disable the servers listed after it. + +### Periodic source sweep (weekly) +Nothing in CSA notifies this repo when new tooling appears, so **run `./tools/sweep-csa-sources.sh` weekly**. It probes the CSA orgs and reports three kinds of drift against three different extension points: unregistered plugin **marketplaces** (`CSA_MARKETPLACES`, 5 scripts), published **plugins** nobody installs (`scripts/csa-plugins*.txt`, list-only change), and **MCP servers** that are ready to wire (`setups=()`, 5 scripts). Exit `0` no drift, `1` drift, `2` could not complete — `2` means "I learned nothing", never "no drift". + +Deliberately **not** in `check-all.sh`: it needs the network and a `gh` token with CSA-Internal access, and a check that cannot pass in CI is a check that gets deleted. + +**Do not make its probing parallel.** An early version used `xargs -P 12` and reported three repos as having no `marketplace.json` when all three do — a probe that fails under load is indistinguishable from a repo that genuinely lacks the file, so the sweep under-reports and the failure looks exactly like success. It probes sequentially and separates 404 from other errors for that reason. Full rationale, the MCP-description heuristic's known blind spot, and what to do with each finding: [`docs/periodic-sweep.md`](docs/periodic-sweep.md). + ### Script execution flow All macOS scripts follow the same pattern: `main` → preconditions → preflight (show plan) → confirm → action steps → summary. `macos-ai-tools.sh` adds a migration layer: `detect_migrations()` runs during preflight, then `migrate_*()` runs before each tool's install to remove wrong-method installs. `macos-update.sh` takes a pre-update snapshot (to `~/Library/Logs/CSA-DesktopSetup/`) before showing the plan, enabling version rollback if updates break something. diff --git a/docs/periodic-sweep.md b/docs/periodic-sweep.md new file mode 100644 index 0000000..16dc075 --- /dev/null +++ b/docs/periodic-sweep.md @@ -0,0 +1,134 @@ +# Periodic source sweep + +**Cadence: weekly.** Run `./tools/sweep-csa-sources.sh`. + +## Why this exists + +DesktopSetup wires CSA tooling into new machines, but nothing in CSA tells DesktopSetup +when there is new tooling to wire. A plugin marketplace gets created, a plugin ships into +an existing marketplace, an MCP server becomes ready — and this repo carries on installing +the set it knew about the day someone last edited it. The gap is invisible from inside the +repo: every script parses, every check passes, and new hires quietly get a smaller toolset +than the people who set their own machines up by hand. + +The sweep exists to make that gap visible on a schedule instead of on a coincidence. + +## What drifts, and where it has to be fixed + +Three different extension points, and knowing which one a finding belongs to is most of +the work: + +| # | What appeared | Where it gets wired | Cost of the change | +|---|---|---|---| +| 1 | A new plugin **marketplace** repo | `CSA_MARKETPLACES` + `plugin_marketplace_repo` — **5 scripts** | script edit + `SCRIPT_VERSION` bump in each | +| 2 | A new **plugin** in a registered marketplace | `scripts/csa-plugins.txt` or `scripts/csa-plugins-internal.txt` | one commit to `main`, **no** version bump | +| 3 | A new **MCP server** | `setups=()` in `setup_csa_internal_tools` — **5 scripts** | script edit + `SCRIPT_VERSION` bump in each | + +The five scripts for #1 and #3 are `macos-ai-tools.sh`, `macos-update.sh`, +`macos-plugins.sh`, `windows-ai-tools.ps1`, `windows-plugins.ps1`. `check-duplication.py` +will catch you if you update four of them. + +Category 2 is the cheap one and the one that matters most in practice — a list-only change +reaches every existing user on their next `macos-update.sh` run without them reinstalling +anything. + +## Reading the output + +``` +$ ./tools/sweep-csa-sources.sh +``` + +Exit codes: `0` no drift, `1` drift found, `2` **could not complete** — missing `gh`, not +authenticated, or a probe failed. Treat `2` as "I learned nothing", never as "no drift". + +Four kinds of line: + +- **`UNREGISTERED marketplace`** — a real marketplace with plugins in it that no script + knows about. Category 1. +- **`not in the install lists`** — published plugins nobody installs. Category 2. This + will usually be non-empty and that is fine: not every plugin is meant for every hire. + The question to ask each week is whether any of them have *become* ready, not whether + the list is empty. +- **`READY TO WIRE`** — an MCP server whose `internal-setup/-setup.sh` exists in the + gate repo but is not in `setups=()`. Category 3. This is the highest-signal line the + sweep produces: someone finished a server and it is sitting there unused. +- **`not ready`** — an MCP server repo with no setup script yet. Informational. `csa-zendesk` + sits here today ("Research and design phase — no implementation yet"). Nothing to do. + +## What the sweep deliberately does not flag + +- **Forks of registered marketplaces.** `CloudSecurityAlliance/Research-Plugins` is a private + fork of the internal one and is *behind* it. Registering a fork shadows the real + marketplace with stale content, so the sweep names it and moves on. +- **Empty marketplaces.** `accounting-plugins` declares `"plugins": []`. It is registered by + all five scripts and installs nothing. Correct, and not worth a weekly reminder. +- **`csa-*` repos with no "MCP" in the description.** The org has a dozen `csa-ai-exam-*` + and `csa-research-*` data repos that are not servers. See the caveat below. + +## The two ways this sweep can lie to you + +Both are under-reporting — it will tell you there is no drift when there is. Neither +produces a visible error, which is why they are written down here. + +**1. Parallel probing produces false negatives.** An early version used `xargs -P 12` and +reported three repos as having no `marketplace.json` when all three demonstrably do. A +probe that fails under load is indistinguishable from a repo that genuinely lacks the +file. The script now probes sequentially and separates 404 from every other error, +exiting `2` if any probe fails. **Do not "optimise" it back into parallelism.** A minute +a week is not a problem worth solving. + +**2. The MCP heuristic depends on repo descriptions.** A candidate must be named `csa-*` +*and* mention "MCP" in its GitHub description. A new server whose description omits the +word is skipped silently. The script prints how many repos it skipped for this reason, so +if a server you expect is missing from the list, check its description first. + +Also worth knowing: **GitHub code search does not work for this.** Querying +`org:CloudSecurityAlliance path:.claude-plugin filename:marketplace.json` returns +`total_count: 0` despite seven manifests existing — code search does not reliably index +dotfile directories or private repos. The contents API is the only trustworthy probe. + +## Running this as a scheduled routine + +A cloud routine runs the sweep weekly — **Mondays 15:04 UTC** (9am MDT / 8am MST; the cron +is fixed UTC, so it shifts an hour across DST). Routine +`trig_01TQh4GMWKRnt4L4QpM5mhJc`, managed at . + +**This section is the routine's spec.** Its prompt is deliberately short and defers here, so +changing the job means editing this file rather than the routine. Steps: + +1. `ls -l tools/sweep-csa-sources.sh`. If it is missing, the checkout predates the sweep — + say so and stop. Do not improvise a replacement. +2. Verify access: `gh api repos/CloudSecurityAlliance-Internal/CSA-Plugins --jq .full_name`. + If it fails, the cloud token lacks CSA-Internal read access. **Stop and report exactly + that.** Never report "no drift" from a run that could not see the private orgs. +3. Run `./tools/sweep-csa-sources.sh`, capturing output and exit code. Expect one to two + minutes; it probes ~200 repos sequentially on purpose. +4. Act on the exit code: + - **2** — could not complete. Report the failure and what caused it. + - **0** — no drift. Say so and stop. Do not open anything. + - **1** — drift found. Go to step 5. +5. Report drift as a GitHub issue in `CloudSecurityAlliance/DesktopSetup`, but **do not + create a duplicate**: `gh issue list --repo CloudSecurityAlliance/DesktopSetup --state open + --search "Weekly source sweep" --json number,title` first. If an open issue exists, add a + comment with this week's findings. Otherwise create one titled + `Weekly source sweep: drift found` containing the sweep output verbatim, the date, and + which of the three categories each finding belongs to. + +Silent when clean, by design — the same contract the installers follow. A weekly issue that +says "nothing to do" is a weekly issue nobody reads. + +**Known constraint:** the cloud environment's GitHub token may not carry CSA-Internal read +access. If it does not, step 2 stops the run every week and the routine is useless until the +token is fixed. That is the intended failure — a loud, honest "could not check" beats a +false all-clear. Verify this on the routine's first real run. + +## Acting on findings + +For category 2 (the common case), the whole change is an edit to +`scripts/csa-plugins-internal.txt` and a PR. For 1 and 3, edit all five scripts, bump each +`SCRIPT_VERSION` to the current `YYYY.MMDDHHSS`, and run `./tools/check-all.sh` before +pushing. + +Judgement required, and the sweep cannot make it for you: **a plugin existing is not a +plugin being ready for every new hire.** The sweep reports what is published; deciding +what belongs in a default install is a human call. diff --git a/tools/sweep-csa-sources.sh b/tools/sweep-csa-sources.sh new file mode 100755 index 0000000..2c7278a --- /dev/null +++ b/tools/sweep-csa-sources.sh @@ -0,0 +1,240 @@ +#!/usr/bin/env bash +# Weekly sweep for CSA sources this repo should be wiring up but isn't yet. +# +# ./tools/sweep-csa-sources.sh # the two CSA orgs +# ./tools/sweep-csa-sources.sh --all-orgs # every org your gh token can see +# ./tools/sweep-csa-sources.sh --quiet # only print drift +# +# Three things drift, in three different places, and nothing else notices: +# +# 1. A new plugin marketplace repo appears -> CSA_MARKETPLACES (5 scripts) +# 2. A new plugin lands in a registered marketplace -> scripts/csa-plugins*.txt +# 3. A new csa-* MCP server repo appears -> setups=() in setup_csa_internal_tools +# (5 scripts) +# +# NOT part of check-all.sh, deliberately: this needs the network and a gh token with +# CSA-Internal access. CI has neither, and a check that cannot pass in CI is a check +# that gets deleted. Run it by hand, or from the scheduled routine — see +# docs/periodic-sweep.md. +# +# Exit codes: 0 = no drift 1 = drift found 2 = could not run (no gh / not authed) +# +# ── The one thing to know before you change the probing ─────────────────────────── +# Probe repos SEQUENTIALLY. An earlier version of this sweep ran `xargs -P 12` and +# reported three repos as having no marketplace.json when all three demonstrably do. +# The failure mode here is silent and asymmetric: a probe that errors under +# parallelism looks exactly like a repo that legitimately has no manifest, so the +# sweep under-reports and you conclude "no drift" when there is drift. This script +# therefore probes one at a time and distinguishes 404 (a real absence) from any +# other error (a probe that failed), reporting the latter loudly rather than +# folding it into the "no" pile. ~200 repos takes about a minute. That is fine for +# something that runs weekly. +set -uo pipefail +cd "$(dirname "$0")/.." || exit 2 + +# ── output ────────────────────────────────────────────────────────────────────── +if [[ -t 1 ]]; then + B=$'\033[1m'; R=$'\033[31m'; G=$'\033[32m'; Y=$'\033[33m'; D=$'\033[2m'; N=$'\033[0m' +else + B=''; R=''; G=''; Y=''; D=''; N='' +fi +QUIET=0 +step() { [[ $QUIET -eq 1 ]] || printf '\n%s==> %s%s\n' "$B" "$1" "$N"; } +ok() { [[ $QUIET -eq 1 ]] || printf ' %s%s%s\n' "$G" "$1" "$N"; } +drift() { printf ' %s%s%s\n' "$Y" "$1" "$N"; } +err() { printf ' %s%s%s\n' "$R" "$1" "$N"; } +note() { [[ $QUIET -eq 1 ]] || printf ' %s%s%s\n' "$D" "$1" "$N"; } + +ORGS="CloudSecurityAlliance CloudSecurityAlliance-Internal" +for arg in "$@"; do + case "$arg" in + --all-orgs) ORGS="" ;; + --quiet) QUIET=1 ;; + -h|--help) sed -n '2,30p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) err "unknown argument: $arg"; exit 2 ;; + esac +done + +command -v gh >/dev/null 2>&1 || { err "gh not installed"; exit 2; } +gh auth status >/dev/null 2>&1 || { err "gh not authenticated — run: gh auth login"; exit 2; } + +if [[ -z "$ORGS" ]]; then + ORGS="$(gh api user/orgs --jq '.[].login' 2>/dev/null | tr '\n' ' ')" +fi + +AI_TOOLS="scripts/macos-ai-tools.sh" +GATE_REPO="CloudSecurityAlliance-Internal/CSA-Plugins" +found_drift=0 +probe_errors=0 + +# ── source of truth ───────────────────────────────────────────────────────────── +# Parsed out of the scripts rather than restated here. This file is a sixth place +# the lists could drift, and the whole point is to not have one. +registered_marketplaces() { + sed -n '/^CSA_MARKETPLACES=(/,/^)/p' "$AI_TOOLS" \ + | grep -oE '"[^"]+/[^"]+"' | tr -d '"' +} + +wired_mcp_servers() { + sed -n '/^ local setups=(/,/^ )/p' "$AI_TOOLS" \ + | grep -oE '[a-z0-9-]+-setup\.sh' | sed 's/-setup\.sh$//' +} + +listed_plugins() { + cat scripts/csa-plugins.txt scripts/csa-plugins-internal.txt 2>/dev/null \ + | grep -v -E '^[[:space:]]*(#|$)' | sed 's/@.*//' +} + +# ── repo enumeration ──────────────────────────────────────────────────────────── +step "Enumerating repos" +ALL_REPOS="$(mktemp)"; trap 'rm -f "$ALL_REPOS"' EXIT +for org in $ORGS; do + gh repo list "$org" --limit 1000 --json nameWithOwner --jq '.[].nameWithOwner' 2>/dev/null +done | sort -u > "$ALL_REPOS" +repo_count=$(wc -l < "$ALL_REPOS" | tr -d ' ') +note "$repo_count repos across: $ORGS" + +# ── 1. marketplaces ───────────────────────────────────────────────────────────── +# One sequential probe per repo for .claude-plugin/marketplace.json. See the note at +# the top about why this is not parallel. +step "1. Plugin marketplaces" +REGISTERED="$(registered_marketplaces)" +unregistered="" +while read -r repo; do + [[ -n "$repo" ]] || continue + probe_out=$(gh api "repos/$repo/contents/.claude-plugin/marketplace.json" 2>&1 >/dev/null) + case $? in + 0) ;; + *) if echo "$probe_out" | grep -q "Not Found\|repository is empty"; then + continue + else + err "probe failed: $repo :: $(echo "$probe_out" | head -1)" + probe_errors=$((probe_errors + 1)); continue + fi ;; + esac + if echo "$REGISTERED" | grep -qxF "$repo"; then + note "registered: $repo" + else + unregistered="$unregistered$repo"$'\n' + fi +done < "$ALL_REPOS" + +if [[ -n "$unregistered" ]]; then + while read -r repo; do + [[ -n "$repo" ]] || continue + # A fork of an already-registered marketplace is not a new marketplace — it is a + # stale copy, and registering it would shadow the real one. Report it as noise, + # not as drift. (CloudSecurityAlliance/Research-Plugins is exactly this.) + parent=$(gh api "repos/$repo" --jq '.parent.full_name // empty' 2>/dev/null) + if [[ -n "$parent" ]] && echo "$REGISTERED" | grep -qxF "$parent"; then + note "fork of registered $parent — ignoring: $repo" + continue + fi + count=$(gh api "repos/$repo/contents/.claude-plugin/marketplace.json" --jq '.content' 2>/dev/null \ + | base64 --decode 2>/dev/null \ + | python3 -c 'import json,sys; print(len(json.load(sys.stdin).get("plugins",[])))' 2>/dev/null || echo "?") + if [[ "$count" == "0" ]]; then + note "empty marketplace (0 plugins) — nothing to install yet: $repo" + continue + fi + drift "UNREGISTERED marketplace: $repo ($count plugins)" + drift " -> add to CSA_MARKETPLACES + plugin_marketplace_repo in all 5 scripts, bump SCRIPT_VERSION" + found_drift=1 + done <<< "$unregistered" +fi +[[ $found_drift -eq 1 ]] || ok "all marketplaces registered" + +# ── 2. plugins inside registered marketplaces ─────────────────────────────────── +step "2. Plugins in registered marketplaces" +LISTED="$(listed_plugins)" +plugin_drift=0 +for repo in $REGISTERED; do + manifest=$(gh api "repos/$repo/contents/.claude-plugin/marketplace.json" --jq '.content' 2>/dev/null \ + | base64 --decode 2>/dev/null) + [[ -n "$manifest" ]] || { err "could not read manifest: $repo"; probe_errors=$((probe_errors + 1)); continue; } + mkt=$(echo "$manifest" | python3 -c 'import json,sys; print(json.load(sys.stdin).get("name",""))' 2>/dev/null) + missing=$(echo "$manifest" \ + | python3 -c 'import json,sys +d=json.load(sys.stdin) +for p in d.get("plugins",[]): + print(p.get("name") if isinstance(p,dict) else p)' 2>/dev/null \ + | while read -r p; do + [[ -n "$p" ]] || continue + echo "$LISTED" | grep -qxF "$p" || echo "$p" + done | tr '\n' ' ') + if [[ -n "${missing// /}" ]]; then + drift "$mkt: not in the install lists -> ${missing% }" + plugin_drift=1 + fi +done +if [[ $plugin_drift -eq 1 ]]; then + drift " -> add the ones that are ready to scripts/csa-plugins-internal.txt" + drift " (list-only change: one commit to main, no SCRIPT_VERSION bump)" + found_drift=1 +else + ok "every published plugin is referenced by an install list" +fi + +# ── 3. MCP servers ────────────────────────────────────────────────────────────── +# Local per-user MCP servers follow a naming convention: lowercase csa- in the +# public org (csa-google-workspace, csa-skilljar, csa-zendesk). Each is wired up by a +# csa--setup.sh in the gate repo's internal-setup/ directory, which is what +# setup_csa_internal_tools fetches. A server is "ready to wire" exactly when that +# setup script exists — that is the signal this section reports on, because a repo +# existing tells you nothing about whether it is finished. +# +# Two filters, because `csa-*` alone is far too broad — the org has a dozen csa-ai-exam-* +# and csa-research-* data repos that are not servers and never will be. A repo has to +# both carry the prefix AND say "MCP" in its description to be treated as a candidate. +# That is a heuristic and it can miss: a new server whose description omits "MCP" will +# be skipped silently, so the skipped count is printed rather than hidden. If a server +# ever goes missing from this list, the description is the first thing to check. +step "3. MCP servers" +WIRED="$(wired_mcp_servers)" +mcp_drift=0 +skipped_noise=0 +while read -r repo; do + name="${repo##*/}" + case "$name" in + csa-[a-z0-9]*) ;; + *) continue ;; + esac + # skip the hosted server's own infrastructure repos + case "$name" in csa-mcp|csa-plugins*) continue ;; esac + if echo "$WIRED" | grep -qxF "$name"; then + note "wired: $name" + continue + fi + desc=$(gh api "repos/$repo" --jq '.description // ""' 2>/dev/null) + if ! echo "$desc" | grep -qi 'mcp'; then + skipped_noise=$((skipped_noise + 1)) + continue + fi + if gh api "repos/$GATE_REPO/contents/internal-setup/${name}-setup.sh" >/dev/null 2>&1; then + drift "READY TO WIRE: $name — internal-setup/${name}-setup.sh exists but is not in setups=()" + drift " -> append ${name}-setup.sh to setups=() in all 5 scripts, bump SCRIPT_VERSION" + mcp_drift=1 + else + note "not ready: $name — no internal-setup/${name}-setup.sh yet" + note " $(echo "$desc" | cut -c1-72)" + fi +done < "$ALL_REPOS" +[[ $skipped_noise -eq 0 ]] || note "($skipped_noise other csa-* repos have no 'MCP' in their description — not treated as servers)" +if [[ $mcp_drift -eq 1 ]]; then + found_drift=1 +else + ok "no MCP server is waiting to be wired up" +fi + +# ── summary ───────────────────────────────────────────────────────────────────── +printf '\n' +if [[ $probe_errors -gt 0 ]]; then + err "$probe_errors probe(s) failed — results are INCOMPLETE, do not read this as 'no drift'" + exit 2 +fi +if [[ $found_drift -eq 1 ]]; then + printf '%sdrift found%s — see docs/periodic-sweep.md for what to do with each kind\n' "$Y" "$N" + exit 1 +fi +printf '%sno drift%s — every CSA source is wired up\n' "$G" "$N" +exit 0