feat: weekly sweep for CSA sources this repo isn't wiring up yet - #49
Merged
Conversation
Nothing in CSA notifies DesktopSetup when new tooling appears, so the installed set silently lags what exists. Three extension points drift independently and none of them is checked by anything: 1. CSA_MARKETPLACES (5 scripts) — new marketplace repos 2. scripts/csa-plugins*.txt — new plugins in known marketplaces 3. setups=() (5 scripts) — new local MCP servers Adds tools/sweep-csa-sources.sh, which probes the CSA orgs and diffs against all three. It parses the lists out of macos-ai-tools.sh rather than restating them, so the sweep cannot itself become a sixth place the lists drift. Current findings on main: no unregistered marketplaces, 15 published plugins referenced by no install list (9 of them in csa-cino-plugins), and csa-zendesk present but not ready (no internal-setup script yet). Not wired into check-all.sh: it needs network and a gh token with CSA-Internal access, and a check that cannot pass in CI gets deleted. Probing is sequential on purpose. An earlier version used xargs -P 12 and reported three repos as having no marketplace.json when all three do — under load a failed probe is indistinguishable from a genuine absence, so the sweep under-reports and reads as "no drift". It now separates 404 from other errors and exits 2 if any probe fails, so an incomplete run can never be mistaken for a clean one. Also documents setup_csa_internal_tools in CLAUDE.md, which was the one registration mechanism of the three with no write-up.
The weekly routine (trig_01TQh4GMWKRnt4L4QpM5mhJc, Mondays 15:04 UTC) carries a deliberately short prompt that says 'read docs/periodic-sweep.md, section Running this as a scheduled routine, and follow it exactly'. That section did not exist yet. Adding it, so the job is defined in the repo where it can be reviewed and changed, rather than inside a prompt nobody can see from here. Includes the constraint to verify on first run: the cloud token may not have CSA-Internal read access, in which case the routine stops and says so rather than reporting a false all-clear.
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Nothing in CSA tells DesktopSetup when there is new tooling to wire up. A marketplace gets created, a plugin ships, an MCP server becomes ready — and this repo carries on installing the set it knew about the day someone last edited it. The gap is invisible from inside the repo: every script parses, every check passes, and new hires get a smaller toolset than the people who set their machines up by hand.
Three extension points drift independently, and none of them was checked by anything:
CSA_MARKETPLACES+plugin_marketplace_repo— 5 scriptsSCRIPT_VERSIONbump ×5scripts/csa-plugins*.txtsetups=()insetup_csa_internal_tools— 5 scriptsSCRIPT_VERSIONbump ×5What's here
tools/sweep-csa-sources.sh— probes the CSA orgs and diffs against all three. Flags--all-orgs,--quiet. Exit0no drift /1drift /2could not complete.docs/periodic-sweep.md— runbook: what each finding means, which of the three places to fix it, and the two ways the sweep can under-report.CLAUDE.md— a weekly-cadence pointer, plus a write-up ofsetup_csa_internal_tools, which was the one registration mechanism of the three with no documentation.The sweep parses the lists out of
macos-ai-tools.shrather than restating them, so it can't itself become a sixth place the lists drift.Findings on
maintodayThis PR only adds the sweep; it does not act on the findings. Whether any of those 15 plugins belong in a default install is a human call.
Two deliberate choices worth reviewing
Not in
check-all.sh. It needs the network and aghtoken with CSA-Internal access. CI has neither, and a check that cannot pass in CI is a check that gets deleted.Probing is sequential. An earlier version used
xargs -P 12and reported three repos as having nomarketplace.jsonwhen all three demonstrably do. Under load, a failed probe is indistinguishable from a genuine absence — so the sweep under-reports and the failure reads exactly like success. It now probes one at a time, separates 404 from other errors, and exits2if any probe fails, so an incomplete run can never be mistaken for a clean one. This is called out in the script header, the runbook, and CLAUDE.md, because it will look like an obvious thing to optimise. ~1 min/week.Known blind spot
Section 3 requires a candidate to be named
csa-*and mention "MCP" in its GitHub description —csa-*alone matched 13csa-ai-exam-*/csa-research-*data repos that are not servers. A new server whose description omits "MCP" will be skipped; the script prints the skipped count rather than hiding it. Documented in both the script and the runbook.Verification
./tools/check-all.sh— all checks pass (bash -n, shellcheck, duplication, native-call guards, paste safety, debug mode, PS parse, PSScriptAnalyzer, 26 Pester tests).shellcheck --severity=warningclean on the new script. Sweep run end-to-end against the live orgs (193 repos), output above.