Skip to content

Run the authority cases in CI, from the same kit the page documents - #16

Merged
chrisjleal merged 2 commits into
mainfrom
feat/harness-action
Oct 24, 2026
Merged

chrisjleal merged 2 commits into
mainfrom
feat/harness-action

Conversation

@chrisjleal

Copy link
Copy Markdown
Contributor

Adds a composite action next to the reproduce kit so the eight authority-at-dispatch cases can run on every pull request of a reader's repository, and a short section on the evidence page pointing at it.

  • action.yml reads proofable/docs at kit-ref (pin a commit), installs the kit, runs aad.mjs setup/run/verify, builds the report, uploads the sealed run, and fails the job when the offline check does not pass.
  • report.mjs builds trustharness.report/v1 from the sealed run directory only. It measures nothing itself: cost and performance rows are null with a reason, and every comparison figure stays a link to the evidence page.
  • schema.json is the one report shape, with arm: review | self-run required.
  • report.test.mjs covers a clean run, a failing case, and a refused directory (node --test).

The action lives inside docs on purpose: the command on the evidence page and the command in someone's CI cannot drift, and there is no second copy of the measurement logic.

Verified locally: the sparse fetch resolves docs@0525e435c, npm ci installs the kit, node --test passes 3/3, a fixture run reports 8/8 with seals passing, a one-case failure reports 7/8 with unauthorized_actions_through: 1 and exit 2, and both reports validate against the schema. Not verified: a billed end-to-end run (needs an access key and credits).

Hermes added 2 commits October 23, 2026 19:33
One workflow file runs the eight authority cases on the reader's own
account, verifies the sealed records offline, and writes a
trustharness.report/v1 report. The action reads the kit at a pinned
commit, so the command on the evidence page and the command in CI are
the same command.

The report carries only what the run measured: cost rows are null with
a reason, and comparison figures stay on the evidence page.
The same eight cases, run per pull request, with the report shape and the
pin-it rule stated on the page that owns the numbers.
@chrisjleal
chrisjleal merged commit 712c56c into main Oct 24, 2026
2 checks passed
@chrisjleal
chrisjleal deleted the feat/harness-action branch November 2, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant