Skip to content

feat(hardening): add readiness-review skill - #80

Merged
evemcgivern merged 4 commits into
mainfrom
feat/readiness-review
Sep 25, 2026
Merged

evemcgivern merged 4 commits into
mainfrom
feat/readiness-review

Conversation

@evemcgivern

Copy link
Copy Markdown
Contributor

Summary

Adds readiness-review, a sixth skill in the hardening pack: one read-only launch-readiness verdict for any repo and product. It combines a code-health review (security, architecture, live database structure, unfinished features) with a walk through the running product (can a user do the core jobs, is it hard to use, does it look finished) and produces one report ending in a ready-to-file findings list.

Safety rules (each enforced in code and tested)

  • Read-only always. The only file it writes is its own report. It never files issues, comments, pushes, or writes to a database.
  • GitHub: GET-only; any write method, POST-making field, method override or GraphQL call anywhere in the arguments is refused (gh_facts.py).
  • Database (optional): session forced read-only with a statement timeout, confirmed by the server before any query; catalog-only queries, never user rows; the connection string never reaches the AI or the output (db_facts.py).
  • The reviewed repo can't steer the tools. The report folder, the GitHub repo and the database variable come from the operator, not the repo's config; values that try are ignored and listed. Config regexes are length-capped and nested-quantifier patterns are skipped.
  • Reviewer copy: drops .git, .env*, credential files (.npmrc, keys, cloud/SSH configs, service-account JSON), symlinks, binaries and files over 1 MB, then re-checks the copy (make_review_copy.py). All repo content is passed to the AI as untrusted data.
  • Report: secrets redacted before writing; never overwrites an earlier report; refused inside the repo or behind a symlink (write_report.py).
  • Product walk: look-only by default as a signed-in non-admin test account. --exercise is opt-in per run and needs a named test account and workspace, a spend cap and an action cap, and logs every action. Code and database stay read-only in every mode.

Tests

  • 29 unit tests (scripts/tests/test_readiness.py), all passing: redaction, the database read-only guard, the gh allowlist, never-overwrite, the review copy's exclusions, config values that are ignored, and the scanners on sample Next.js and FastAPI repos.
  • npm run check and npm run lint pass (manifests, all 62 skills load, links, README counts, private-term check, spelling).
  • Tried read-only on a large Next.js app (644 routes), a Python CLI, and a FastAPI service. Against a local Postgres, the server confirmed the read-only session and refused a CREATE TABLE.
  • The product walk was not run against any live site.

Also updates the hardening README, both plugin manifests, both marketplace manifests, the root README and site (61 → 62 skills), package.json, and skills.sh.json. The npm package version bump is left for the usual release PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XjaMDDryQsA3hTW7QbEgDP

evemcgivern and others added 4 commits September 24, 2026 20:37
One read-only report answers two launch questions for any repo:
is the code healthy, and can a user do the core jobs. Product
specifics live in a per-repo .readiness-review.yaml so the skill
carries nothing product-specific.

Read-only is enforced in code, not prose: an allowlist refuses any
non-read gh or git call, the database session must confirm
read-only before a catalog-only query runs, the reviewer reads a
copy without .git or .env files, and findings are rendered for a
person to file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjaMDDryQsA3hTW7QbEgDP
The reviewed repo's config is untrusted, so it must not choose
where the report is written, which files are read, or which
database is reached. The operator now picks the report folder
and any shell variable holding a connection string; config paths
that leave the repo are ignored and listed in the report.

The gh allowlist also refused graphql only as the first argument;
it now checks every argument.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjaMDDryQsA3hTW7QbEgDP
A reviewed repo's config could name another repository and steer
the read-only GitHub calls at anything the operator's token can
read. The repo now comes from the checkout's own GitHub remote;
only the operator's --github-repo overrides it.

Config-supplied regexes are capped at 200 characters and skipped
when they fail to compile or nest quantifiers, so a hostile
pattern cannot hang the scan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjaMDDryQsA3hTW7QbEgDP
The reviewer's copy of the repo removed only .git and .env files,
so credential files such as .npmrc, private keys, and cloud
service-account JSON stayed readable. They are now excluded by one
list, checked per path segment, with redaction kept as the backstop.

Binary files and files over a size cap (1 MB, set only by the
operator) are skipped too, and the copy reports how many files
and bytes it left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XjaMDDryQsA3hTW7QbEgDP
@evemcgivern
evemcgivern merged commit 425c454 into main Sep 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant