Skip to content

fix: PR #3 review follow-ups and outside-author CI blocker - #4

Merged
tristanroth1 merged 5 commits into
mainfrom
fix/review-followups
Oct 19, 2026
Merged

tristanroth1 merged 5 commits into
mainfrom
fix/review-followups

Conversation

@tristanroth1

Copy link
Copy Markdown
Contributor

Fixes the three follow-ups from the independent review of #3, plus one CI blocker found tonight. No release, tag or deploy in this PR.

Owner: founder
Next: founder approves release v2026.10.19.2

1. llms.txt run steps: no unpinned npx (medium)

The "If you were asked to run grc-setup-review" steps offered npx skills add ... next to the clone. That fetches and runs an npm package, while the user's request only covers the one pinned clone. Step 1 is now the pinned clone only; npx stays under ## Install. site/tests/built.test.mjs and site/tests/llms.test.mjs assert the run-steps section contains no npx, and that the pinned npx line is still under Install.

2. CHANGELOG contradiction (low)

v2026.10.19.1 said "The site prompt now pins the release, as the README did." Now: "The prompt carries no release tag; llms.txt pins the release."

3. "Where you decide" narrowed (low)

decisionRules (site/scripts/content-lib.mjs) keeps only Rules lines that leave a decision or sign-off with a person: acting on someone's behalf, accepting, marking effective/approved, approving, signing, deciding, sending/submitting/contacting, changing live systems or records, or routing a call to a person (owner, reader, HR, legal, counsel, privacy lead, auditor). Evidence-quality rules ("never rate X on Y alone", "without the contract", "without checking") and data-handling rules drop out, even when they mention approval or effectiveness. The intro reads "From this skill's rules: where you decide". The draft-only boundary moved out of the list (it is not from the skill's rules); a skill with no qualifying rule shows the boundary alone.

Spot-check of all 60 skills: 22 lines left, 25 came in (mostly routing lines that had been missed, plus sending and system-change lines), 3 skills now show only the boundary, because every rule they have is about evidence quality or data handling: first-gap-assessment, multi-framework-overlap-map, privacy-and-ai-control-applicability. Sample:

risk-register-first-pass
  Never accept a risk on the owner's behalf or record a risk as accepted without a named human decision.
  Never change a system, account, ticket or document to treat a risk; propose the treatment and let its owner act.
  Always agree the scoring method with the reader before scoring any risk.
grc-setup-review
  Never change a compliance platform, ticket or document; recommend, and let the reader act.
  Never mark a control as effective or a requirement as met; the reader's people and auditors decide that.
dpa-security-annex-review
  Never sign, accept or send redlines to the other party; counsel or the data protection officer decides.
change-management-evidence-sampling
  Never change repository settings, branch rules, approvals or deploy permissions.
  Never approve, merge, revert or re-review a historical change to fix an exception.
  Never grade an exception as acceptable or effective; record the facts for the reader.
ropa-first-draft
  Never rely on the small-organisation exemption without counsel's documented view.
  Always mark every proposed legal basis and retention period as proposed until the privacy lead confirms it.
acceptable-use-policy-drafting
  Never approve or publish the policy on the company's behalf.
  Always get the reader's answer on devices, AI tools and monitoring before drafting those sections.
  Always route monitoring and disciplinary wording to HR or legal for review.

Dropped, for example: "Never ban a tool ... without naming the approved alternative", "Never mark a tool approved on the strength of vendor marketing alone", "Never create, backdate, re-sign or alter evidence", "Never present the review as legal advice" (no one named who decides).

Tests: site/tests/content.test.mjs has a negative list (evidence-quality and data-handling lines must yield nothing) and a positive line per kind of call; built.test.mjs checks one <li> per rule, the boundary outside the list, and the intro only above a non-empty list.

4. First-time outside author can pass CI (blocker)

tools/validate.mjs required every skills/<author>/ folder in .claude-plugin/plugin.json, and outside PRs may not touch .claude-plugin/, so a new outside author could never go green.

Checked whether the manifest can point at the whole skills/ tree: Claude Code 2.1.303, isolated CLAUDE_CONFIG_DIR, local marketplace install, claude plugin details: "./skills/", "./skills", "./skills/*/", "./skills/**/" and no skills key all expose 0 skills (discovery is one level below each listed folder). Listing ./skills/tristan-roth/ gives 60; adding a fixture ./skills/new-author/ gives 61 including the fixture skill. So per-author entries stay.

Fix: on a PR, an unlisted author folder is a note: on stdout, not an error. node tools/validate.mjs --release requires every author folder to be listed, and release.yml now runs it on the tag, so nothing ships with a new author missing from the plugin. Listed entries that do not exist are still errors on PRs. AGENTS.md says a maintainer adds the entry before the next release tag.

Tests (tools/validate.test.mjs): a fork-mode PR adding skills/new-author/author.md plus one skill passes both CI steps (--changed guard and full validate on the candidate) with no manifest change, and --release on the same tree fails with plugin-skills; plus the PR-vs-release split on a unit fixture and a no-skills/ CLI regression.

Checks

  • node tools/validate.mjs and --release: ok, 60 skills
  • node --test tools/*.test.mjs: 147 pass
  • site npm run build, tsc --noEmit, npm test: 43 pass
  • claude plugin validate .claude-plugin/plugin.json: passes (the CLAUDE.md-at-root warning predates this PR)

CODEX_REVIEW: Codex (read-only, sentinel verified each round), 4 rounds. r1 FIX-FIRST (missed routing phrasings), r2 FIX-FIRST (mixed send lines, counsel and authority lines, bare legal-advice lines), r3 FIX-FIRST (validator crashed on a checkout with no skills/ folder; three phrasings), r4 SHIP. All findings folded.

🤖 Generated with Claude Code

…cide)

- llms.txt run steps: drop the unpinned npx alternative; the user's
  request covers only the pinned clone. npx stays under ## Install.
- CHANGELOG v2026.10.19.1: correct the prompt-pinning sentence.
- Where you decide: only rules that keep a decision or sign-off with a
  person; quality-of-evidence rules drop out. New intro, boundary below
  the list, tests for both kinds of rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex review r1: routing to HR, legal or a reviewer, agreeing a method
with the reader, raising to an owner, and a NEVER line that routes to an
acceptance decision now qualify. Tests cover each phrasing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mixed contact-and-send lines qualify; only pure keep-on-machine lines
drop. Authority to accept and counsel's documented view qualify. A bare
'not legal advice' line drops unless it names who decides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude Code reads plugin skills one level below each listed folder, so
"./skills/" cannot replace the per-author entries (checked with claude
plugin details in an isolated config: ./skills/, ./skills, globs and no
key all expose 0 skills). The validator now notes an unlisted author on a
PR and requires the listing only with --release, which release.yml runs
on the tag. Test: a fork-mode PR adding skills/new-author/ passes both CI
steps with no manifest change, and --release still asks for the entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
validate() returns notes on the no-skills/ early path (the CLI crashed);
Where you decide also keeps rules that leave a call to the auditor, an
owner, or the reader's answer before drafting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristanroth1
tristanroth1 merged commit b064cfd into main Oct 19, 2026
2 checks passed
@tristanroth1
tristanroth1 deleted the fix/review-followups branch October 19, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants