From de53d5e4a8e610c097d14949663740d1068e3fb5 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 21:27:26 +0000 Subject: [PATCH 1/7] feat(k9): reusable k9-conformance lane with the conformance verdict grid Consumers had no way to call the canonical K9 validator: deed-conformance had a workflow_call entry point and no caller, and K9 had no reusable at all, so every consumer ran a second validator (the k9-ecosystem action at @main, a grep for a `contract` line, or k9iser's generated output). Of the 53 consumer .k9.ncl files reachable on 2026-10-19, 44 fail contract v1.0.0, every rsr-template-repo contractile among them (standards#1058). This adds the lane at the source: - .github/workflows/k9-conformance.yml: push/pull_request self-run here, workflow_call for consumers. It checks out validator, contract and renderer at job.workflow_sha, installs the estate-pinned Nickel, runs the validator self-test, typechecks the contract, validates every committed K9 file at L0-L2 --strict --json, honours the caller's shrink-only .machine_readable/k9-contract-debt.txt, renders the grid, uploads the k9-grid artifact and exposes the `verdict` output. It asks for contents: read only, so the caller posts the comment (the same split as hypatia-scan-reusable). Inputs: require-files, fail-on-indeterminate. - scripts/k9-grid.sh + scripts/lib/k9-grid.jq: the conformance-family grid of docs/VERDICT-GRID-SPEC.adoc in bash and jq (no runtime to install; no JS growth). The item and state rules are those of 1-formats/k9/tools/conformance-report.mjs. report.json and badge.json are I-JSON serialised as JCS through ijson-jcs canon when installed, else jq -S -c, byte-identical for these documents (D306). - scripts/tests/k9-grid-test.sh: 43 assertions; missing, skipped or unreadable evidence is never a clean row, the ledger rules, routed and empty corpora are grey, revisions compare only under one profile, the artifacts are JCS. - docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc: the determinations of 2026-10-19 (the interface panel is the verdict grid; one validator per family, pinned by the caller; L3 not certified; k9iser retired) and the install checklist. - WO-9 (template contractiles to v1.0.0) and WO-10 (install wave across echidna, echidnabot, proof-burrower, hypatia, panic-attack; retire k9iser) in WORK-ORDERS-2026-10-19.adoc (v1.5.0). - Cross-references in VERDICT-GRID-SPEC "Where it runs", CONFORMANCE-REPORTING "Reproduce" and the k9 tools README. - actions.lock: the k9-conformance entry written by the repair chain (relock-sha-keys, complete-job-refs, close-lock, prune-stale; check-lock-sync in sync). Every action it uses was already locked. Local gates: k9-grid tests 43/43, verdict-grid tests 29/29, Mustfile 14 passed, check-standards-map GATE D, duplicate-key check clean, link-rot on docs and 1-formats/k9/tools identical to main. The Debtfile deno-residue breach and the canon-lockstep hypatia score (no Hex in this container) reproduce on main unchanged. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- .github/workflows/actions.lock | 4 + .github/workflows/k9-conformance.yml | 245 ++++++++++++++++++ 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc | 18 ++ 1-formats/k9/tools/README.adoc | 4 + docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc | 149 +++++++++++ docs/VERDICT-GRID-SPEC.adoc | 1 + .../WORK-ORDERS-2026-10-19.adoc | 111 +++++++- scripts/k9-grid.sh | 177 +++++++++++++ scripts/lib/k9-grid.jq | 184 +++++++++++++ scripts/tests/k9-grid-test.sh | 175 +++++++++++++ 10 files changed, 1066 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/k9-conformance.yml create mode 100644 docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc create mode 100755 scripts/k9-grid.sh create mode 100644 scripts/lib/k9-grid.jq create mode 100644 scripts/tests/k9-grid-test.sh diff --git a/.github/workflows/actions.lock b/.github/workflows/actions.lock index 958d9599..a997efd9 100644 --- a/.github/workflows/actions.lock +++ b/.github/workflows/actions.lock @@ -171,6 +171,10 @@ workflows: - 'tailscale/github-action@d1b6cd204f8dceda5b3eaad7f1f767be390056cd' '.github/workflows/uuid-v8.yml': - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' + '.github/workflows/k9-conformance.yml': + - 'actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9' + - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' + - 'actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9' dependencies: 'actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9': ref: 'v6.1.0' diff --git a/.github/workflows/k9-conformance.yml b/.github/workflows/k9-conformance.yml new file mode 100644 index 00000000..68aa1d1c --- /dev/null +++ b/.github/workflows/k9-conformance.yml @@ -0,0 +1,245 @@ +# This workflow is managed by gh actions-lock. +# SPDX-License-Identifier: MPL-2.0 +# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) +name: k9-conformance + +# Conformance lane for K9 contracts (standards#1058, ruling D173). The +# canonical validator is 1-formats/k9/tools/k9-validate.sh; the grid it +# reports through is scripts/k9-grid.sh (docs/VERDICT-GRID-SPEC.adoc, +# conformance family). docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc says +# where this lane sits beside the DEED lane and the Hypatia lane. +# +# In this repo (push / pull_request) it checks the validator's self-test, +# the contract's own typecheck, and every committed K9 file. +# +# As a reusable (workflow_call) it validates the CALLER's committed K9 files +# with the validator, contract and renderer at THIS workflow's own commit +# (job.workflow_sha), so a caller pinned to a standards SHA gets exactly +# that validator. Layers L0–L2 run; L3 (signature) is outside this lane and +# the grid says so. The caller's shrink-only debt ledger +# (.machine_readable/k9-contract-debt.txt) is honoured as +# 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc states it. +# +# Outputs: the `k9-grid` artifact (report.json and badge.json as I-JSON/JCS, +# report.md, the validator logs) and the `verdict` output. The caller posts +# report.md to its pull request and publishes badge.json itself: a reusable +# cannot ask for more permissions than it is given, and this one asks for +# `contents: read` only. +# +# Caller: +# jobs: +# k9: +# uses: hyperpolymath/standards/.github/workflows/k9-conformance.yml@ +# with: +# require-files: true # fail when the caller has no K9 file at all +# fail-on-indeterminate: true # orange fails the job too (default: only red fails) + +on: + push: + paths: + - '**.k9.ncl' + - '**.k9' + - '**.k9.lock' + - '1-formats/k9/**' + - 'scripts/k9-grid.sh' + - 'scripts/lib/k9-grid.jq' + - 'scripts/tests/k9-grid-test.sh' + - '.machine_readable/k9-contract-debt.txt' + - '.github/workflows/k9-conformance.yml' + pull_request: + paths: + - '**.k9.ncl' + - '**.k9' + - '**.k9.lock' + - '1-formats/k9/**' + - 'scripts/k9-grid.sh' + - 'scripts/lib/k9-grid.jq' + - 'scripts/tests/k9-grid-test.sh' + - '.machine_readable/k9-contract-debt.txt' + - '.github/workflows/k9-conformance.yml' + workflow_call: + inputs: + require-files: + description: "Fail when the caller has no committed K9 file (a zero-file run is otherwise grey, not red)." + type: boolean + required: false + default: false + fail-on-indeterminate: + description: "Fail the job on an orange grid as well as a red one." + type: boolean + required: false + default: false + outputs: + verdict: + description: "Grid state: red, orange, yellow, green or grey." + value: ${{ jobs.validate.outputs.verdict }} + +permissions: {} + +jobs: + validate: + runs-on: ubuntu-latest + timeout-minutes: 15 + permissions: + contents: read + outputs: + verdict: ${{ steps.grid.outputs.state }} + env: + # In this repo the PR's own validator is in the first checkout; a caller + # gets validator, contract and renderer from the canon checkout at + # job.workflow_sha. + CANON: ${{ github.repository == 'hyperpolymath/standards' && '.' || '.k9-canon' }} + CANON_REF: ${{ github.repository == 'hyperpolymath/standards' && (github.event.pull_request.head.sha || github.sha) || job.workflow_sha }} + NICKEL_VERSION: '1.18.0' + NICKEL_SHA256: '9cba4dd65ae9915ec61f73033aafcff307a377665a83fd8f530df086763318cb' + IJSON_JCS_REV: '0fdeea716fd558e9807362acf6f8202d56d8d88f' + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + persist-credentials: false + + - name: checkout the K9 validator, contract and renderer at this workflow's commit + if: github.repository != 'hyperpolymath/standards' + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: hyperpolymath/standards + ref: ${{ job.workflow_sha }} + path: .k9-canon + persist-credentials: false + sparse-checkout: "1-formats/k9\nscripts/k9-grid.sh\nscripts/lib/k9-grid.jq\n" + sparse-checkout-cone-mode: false + + - name: Install Nickel (pinned, sha256-verified; one version estate-wide) + run: | + set -euo pipefail + URL="https://github.com/tweag/nickel/releases/download/${NICKEL_VERSION}/nickel-x86_64-linux" + curl -sSfL --proto '=https' --proto-redir '=https' --tlsv1.2 --retry 3 -o "$RUNNER_TEMP/nickel" "$URL" + echo "${NICKEL_SHA256} ${RUNNER_TEMP}/nickel" | sha256sum -c - + chmod +x "$RUNNER_TEMP/nickel" + echo "$RUNNER_TEMP" >> "$GITHUB_PATH" + + - name: Cache the ijson-jcs canonicaliser + id: jcs-cache + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: ~/.cargo/bin/ijson-jcs + key: ijson-jcs-${{ env.IJSON_JCS_REV }}-${{ runner.os }} + + - name: Install ijson-jcs (3-practice/JSON-POLICY.adoc, D306) + if: steps.jcs-cache.outputs.cache-hit != 'true' + run: | + set -euo pipefail + # Best effort: the renderer falls back to jq's sorted compact form, + # byte-identical for its ASCII integer-only documents, and the grid + # records which serialiser ran. + cargo install --quiet --git https://github.com/hyperpolymath/ijson-jcs --rev "$IJSON_JCS_REV" --locked ijson-jcs \ + || echo "::warning::ijson-jcs did not install; report.json falls back to jq -S -c" + + - name: Renderer regression tests + if: github.repository == 'hyperpolymath/standards' + run: bash scripts/tests/k9-grid-test.sh + + - name: Validator self-test + run: | + set +e + bash "$CANON/1-formats/k9/tools/k9-validate.sh" --self-test > "$RUNNER_TEMP/selftest.log" 2>&1 + echo "SELFTEST_CODE=$?" >> "$GITHUB_ENV" + tail -n 5 "$RUNNER_TEMP/selftest.log" + + - name: Contract typecheck + run: | + set +e + nickel typecheck "$CANON/1-formats/k9/spec/contract/k9_contract.ncl" > "$RUNNER_TEMP/typecheck.log" 2>&1 + echo "TYPECHECK_CODE=$?" >> "$GITHUB_ENV" + cat "$RUNNER_TEMP/typecheck.log" + + - name: Enumerate the committed K9 files and the PR's changed files + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + run: | + set -euo pipefail + # Tracked files only: an untracked file is not a committed contract. + git ls-files -z -- '*.k9.ncl' '*.k9' '*.k9.lock' \ + | tr '\0' '\n' \ + | grep -v -E '^(\.k9-canon/|9-archive/|1-formats/k9/tools/fixtures/)' \ + | sort > "$RUNNER_TEMP/expected" + echo "$(wc -l < "$RUNNER_TEMP/expected") committed K9 file(s)" + : > "$RUNNER_TEMP/changed" + echo "BASE_UNRESOLVED=" >> "$GITHUB_ENV" + if [ -n "$BASE_SHA" ]; then + if git fetch --quiet --depth=1 origin "$BASE_SHA" 2>/dev/null && git diff --name-only "$BASE_SHA" "$HEAD_SHA" > "$RUNNER_TEMP/changed" 2>/dev/null; then + echo "$(wc -l < "$RUNNER_TEMP/changed") file(s) changed against the PR base" + else + echo "::warning::could not resolve the PR base/head range; debt exemptions are reported as indeterminate" + echo "BASE_UNRESOLVED=--base-unresolved" >> "$GITHUB_ENV" + fi + fi + + - name: Validate every committed K9 file (L0–L2, strict) + run: | + set +e + : > "$RUNNER_TEMP/results.jsonl" + worst=0 + while IFS= read -r f; do + [ -n "$f" ] || continue + bash "$CANON/1-formats/k9/tools/k9-validate.sh" --strict --layer L2 --json --quiet "$f" >> "$RUNNER_TEMP/results.jsonl" 2>> "$RUNNER_TEMP/corpus.log" + rc=$? + [ "$rc" -gt "$worst" ] && worst=$rc + done < "$RUNNER_TEMP/expected" + echo "CORPUS_CODE=$worst" >> "$GITHUB_ENV" + echo "validator worst exit status: $worst" + + - name: Render the verdict grid + id: grid + env: + REQUIRE_FILES: ${{ inputs.require-files || false }} + PR_NUMBER: ${{ github.event.pull_request.number || 'local' }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + run: | + set -euo pipefail + ledger=.machine_readable/k9-contract-debt.txt + args=(--out "$RUNNER_TEMP/k9-grid" --results "$RUNNER_TEMP/results.jsonl" --expected "$RUNNER_TEMP/expected" + --changed "$RUNNER_TEMP/changed" --selftest "$RUNNER_TEMP/selftest.log" --selftest-code "$SELFTEST_CODE" + --typecheck "$RUNNER_TEMP/typecheck.log" --typecheck-code "$TYPECHECK_CODE" --corpus-code "$CORPUS_CODE" + --sha "$HEAD_SHA" --run "$GITHUB_RUN_ID" --url "$RUN_URL" --pr "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" + --canon-ref "$CANON_REF") + [ -f "$ledger" ] && args+=(--ledger "$ledger") + [ -n "${BASE_UNRESOLVED:-}" ] && args+=("$BASE_UNRESOLVED") + bash "$CANON/scripts/k9-grid.sh" render "${args[@]}" + cp "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/typecheck.log" "$RUNNER_TEMP/results.jsonl" "$RUNNER_TEMP/k9-grid/" + [ -f "$RUNNER_TEMP/corpus.log" ] && cp "$RUNNER_TEMP/corpus.log" "$RUNNER_TEMP/k9-grid/" + if [ "$REQUIRE_FILES" = "true" ] && [ ! -s "$RUNNER_TEMP/expected" ]; then + echo "::error::require-files is set and no committed K9 file was found" + echo "state=red" >> "$GITHUB_OUTPUT" + fi + + - name: Archive the grid and its evidence + if: ${{ always() && steps.grid.outcome != 'skipped' }} + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: k9-grid + path: ${{ runner.temp }}/k9-grid + if-no-files-found: error + retention-days: 90 + + - name: Gate + env: + STATE: ${{ steps.grid.outputs.state }} + FAIL_ON_INDETERMINATE: ${{ inputs.fail-on-indeterminate || false }} + run: | + set -euo pipefail + echo "K9 verdict grid: $STATE" + case "$STATE" in + red) echo "::error::K9 conformance failed; see the k9-grid artifact and the step summary"; exit 1 ;; + orange) + if [ "$FAIL_ON_INDETERMINATE" = "true" ]; then + echo "::error::K9 conformance is indeterminate and fail-on-indeterminate is set"; exit 1 + fi + echo "::warning::K9 conformance is indeterminate: evidence is missing, not clean" ;; + yellow) echo "::notice::K9 conformance passes with ledgered debt or warnings" ;; + green|grey) ;; + *) echo "::error::no verdict was produced"; exit 1 ;; + esac diff --git a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc index 1fab0478..136d4246 100644 --- a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc +++ b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc @@ -62,6 +62,24 @@ bun test 1-formats/k9/tools/conformance-report.test.mjs bun 1-formats/k9/tools/conformance-report.mjs ---- +The reusable lane (`.github/workflows/k9-conformance.yml`, which a consumer +calls at a pinned standards SHA) renders the same five-state grid with +`scripts/k9-grid.sh` and `scripts/lib/k9-grid.jq` (bash and jq, no runtime to +install) and writes `report.json` and `badge.json` as I-JSON/JCS. Its rules are +those above; `scripts/tests/k9-grid-test.sh` checks them. To reproduce what the +lane posts for a consumer: + +[source,bash] +---- +bash 1-formats/k9/tools/k9-validate.sh --self-test > selftest.log; echo $? > selftest.code +nickel typecheck 1-formats/k9/spec/contract/k9_contract.ncl > typecheck.log; echo $? > typecheck.code +git ls-files -- "*.k9.ncl" "*.k9" "*.k9.lock" | sort > expected +while read -r f; do bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json --quiet "$f"; done < expected > results.jsonl +bash scripts/k9-grid.sh render --out grid --results results.jsonl --expected expected \ + --ledger .machine_readable/k9-contract-debt.txt --selftest selftest.log --selftest-code "$(cat selftest.code)" \ + --typecheck typecheck.log --typecheck-code "$(cat typecheck.code)" +---- + Exit 0 means green, yellow or explicit N/A; exit 1 means a blocking failure; exit 2 means evidence could not establish the verdict. CI retains its required job name. Existing structural/Mustfile checks still run and can fail the job diff --git a/1-formats/k9/tools/README.adoc b/1-formats/k9/tools/README.adoc index cd5de85d..e85c4560 100644 --- a/1-formats/k9/tools/README.adoc +++ b/1-formats/k9/tools/README.adoc @@ -29,6 +29,10 @@ link:CONFORMANCE-REPORTING.adoc[Conformance reporting] defines the evidence schema, five-state verdicts, comparison rules and dated artifact layout. `conformance-report.mjs` renders one updatable PR table with expandable findings; `conformance-report.test.mjs` checks failure and missing-evidence behavior. +Consumers do not run that script: they call the reusable +`.github/workflows/k9-conformance.yml` at a pinned standards SHA, which runs +`k9-validate.sh` from that commit and renders the grid with +`scripts/k9-grid.sh` (see `docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc`). == Running it diff --git a/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc new file mode 100644 index 00000000..81c827c4 --- /dev/null +++ b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) += Validating the contractile–descriptile system +Jonathan D.A. Jewell (hyperpolymath) +v1.0.0, 2026-10-19 +:toc: left +:toclevels: 2 +:sectnums: +:icons: font + +[.lead] +Every estate repository carries two kinds of machine-readable metadata: +the *descriptiles* (what the repository is, in its `_chora.deed`) +and the *contractiles* (what it promises, as K9 contracts under +`.machine_readable/contractiles//`). This document fixes which +machinery validates each, where that machinery runs, how its verdict is +shown, and what a consuming repository has to install. It records the +determinations made on 2026-10-19 (standards#1058) after the second +edition of both grammars landed. + +== The two families, and their validators + +[cols="1,2,2,2",options="header"] +|=== +|Family |Files |Grammar (normative) |Validator (canonical) +|Descriptile (descriptive: what is) +|`_chora.deed`, one per repository (D312); `.machine_readable/descriptiles/*.deed` for anchors not yet folded in +|`1-formats/deed/spec/abnf/deed.abnf` (D308) +|`1-formats/deed/tools/deed_lint.js`, run by the reusable `deed-conformance.yml` +|Contractile (normative: what must hold) +|`.machine_readable/contractiles//.k9.ncl` for the verbs `intend`, `must`, `trust`, `adjust`, `dust`, `bust`; any other `*.k9.ncl`; `coordination.k9` (routed, not validated as a contract) +|`1-formats/k9/spec/K9-CONTRACT-SPEC.adoc` v1.0.0 (D173) and its contract `1-formats/k9/spec/contract/k9_contract.ncl` +|`1-formats/k9/tools/k9-validate.sh`, run by the reusable `k9-conformance.yml` +|=== + +The deed attests to the contractiles (`:contractiles_required` and a +`(contractiles …)` clause). The deed linter checks that attestation is +well-formed; the K9 validator checks that each attested contract is a +valid K9 component. Neither validator reads the other's files, and a +green from one says nothing about the other. That separation is +deliberate: a deed that names six contractiles is true even while one of +them fails to typecheck, and the grid shows both facts side by side. + +== Determinations (2026-10-19) + +. *The "interface panel" is the verdict grid.* The compact comment the + owner piloted on MetaManifold-WebUI pull requests, and the one + `k9-contractile.yml` posts in this repository, are the two families of + the grid in `docs/VERDICT-GRID-SPEC.adoc`: the conformance family + (red / orange / yellow / green / grey, for DEED and K9) and the findings + family (for Hypatia). There is no third interface. +. *No consumer was calling the canonical validators.* Before this + change, `deed-conformance.yml` had the `workflow_call` entry point and + no caller, and K9 had no reusable at all: consumers ran the + `k9-ecosystem` action from `@main` or grepped for a `contract` line. + Measured on 2026-10-19, 44 of the 53 consumer `.k9.ncl` files reachable + from this session fail contract v1.0.0, including every contractile in + `rsr-template-repo`, which every new repository is instantiated from. + The fix is at the source: the template first, then each repository. +. *k9iser is retired.* It generated K9 files from TOML manifests in a + shape the v1.0.0 contract rejects (no `schema_version`, no + `component_type`, no `semantics`), and nothing consumed its output + except the `generated/k9iser/` directories it wrote. Contracts are + authored, not generated; `K9-CONTRACT-SPEC.adoc` is the authoring + guide. Repositories that still carry `k9iser.toml`, `generated/k9iser/` + or a `check-k9iser-paths` job remove them in their install PR. +. *One validator per family, pinned by the caller.* A consumer calls the + reusable at a standards commit SHA and gets the validator, the contract + and the renderer at exactly that commit (`job.workflow_sha`). Nothing in + the consumer's own tree validates these files; a second validator is + the defect this document closes. +. *Layers.* CI certifies K9 layers L0–L2 (envelope, structure, Nickel + semantics against the contract). L3, the Ed25519 signature, needs a + verifier the validator deliberately does not embed; the grid's scope + line says L3 is not certified. A `Hunt`-leash contract therefore gets + no execution licence from a green grid. +. *Debt is a shrink-only ledger, honoured in the grid.* A file named in + the consumer's `.machine_readable/k9-contract-debt.txt` and untouched + by the pull request fails as _advisory_ (yellow). Touch it and it must + conform (red). Fix it and leave it in the ledger and the grid is red + with "Stale debt exemption" until the line is removed. Growing the + ledger needs a `Ratchet-exception:` line in a commit or the PR body + (owner norm, 2026-10-14). +. *Artifacts are I-JSON serialised as JCS.* `report.json` and + `badge.json` go through `ijson-jcs canon` (D306) when it is installed, + else jq's sorted compact form, which is byte-identical for these + documents; one final LF either way. The workflow files stay YAML until + the standards#1025 step-6 gate admits KYAML workflows (D280). + +== Where each lane runs + +[cols="1,2,2",options="header"] +|=== +|Lane |Reusable |What the caller does +|DEED |`.github/workflows/deed-conformance.yml@` with `require-files: true` |Nothing more: the job fails on a lint error. (A grid for this lane is a follow-up; the linter's own output is the evidence today.) +|K9 |`.github/workflows/k9-conformance.yml@` with `require-files: true` |Downloads the `k9-grid` artifact in a second job and posts `report.md` to the pull request (marker ``), and publishes `badge.json` where its README badge reads it. The reusable asks for `contents: read` only, so it cannot post. +|Hypatia |`.github/workflows/hypatia-scan-reusable.yml@` with the explicit `HYPATIA_SCAN_PAT` secret |Same posting job, artifact `verdict-grid`, marker ``. +|=== + +The posting pattern is in `docs/VERDICT-GRID-SPEC.adoc` (_Posting from a +caller_). MetaManifold-WebUI's `governance.yml` is the worked example: +one job per lane, one verdict job that posts all the grids, one badges +job that commits `badge.json` files to a `governance-badges` branch. + +== Install checklist for a repository + +. Pin: every `uses: hyperpolymath/standards/…@` in the repository + moves to one SHA, and `actions.lock` is repaired with the chain + (`gh actions-lock`, `relock-sha-keys`, `complete-job-refs`, + `close-lock`, `prune-stale`; never by hand). +. Replace any in-tree deed or K9 validation (`k9-ecosystem` action, + `grep contract`, `nickel typecheck` of `*.k9.ncl` outside the reusable, + k9iser jobs) with the two reusable calls above. +. Add the verdict posting job and the badges job from the worked example. +. Migrate the six contractiles to contract v1.0.0: add `schema_version`, + `component_type` (`contractile:`), `semantics`, + `security.capabilities` and `side_effects`; a `Hunt` leash needs a + signature block. Route `coordination.k9` by its dialect line instead of + validating it as a component. Delete placeholder `template-*` and + `examples/` K9 files that will never be real contracts. +. Run the validator locally before pushing: + `bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json FILE` + from a standards checkout, and `bash scripts/k9-grid.sh render …` to + see the grid the lane will post (`scripts/tests/k9-grid-test.sh` shows + every flag). +. Put any contract that cannot be fixed in the same PR into + `.machine_readable/k9-contract-debt.txt` with a `Ratchet-exception:` + line, and open the issue that will shrink it. + +The repositories in the first wave, in order: `rsr-template-repo` (the +source every other repository is instantiated from), then `echidna`, +`echidnabot`, `proof-burrower`, `hypatia`, `panic-attack`, and this +repository's own self-run. The work orders are WO-9 and WO-10 in +`docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc`. + +== What a green grid means, and does not + +A green K9 grid says: the validator's own self-test passed, the contract +typechecked, and every committed K9 file is a valid component at L0–L2 +under contract v1.0.0 at the named standards commit, with no skipped +check. It does not say the contract's semantics are true of the +repository (that is Hypatia's lane and the Mustfile's), that L3 holds, or +that a routed `coordination.k9` conforms to its own grammar (no validator +exists for it yet; `2-protocols/k9-coordination/` says so). + +Orange is missing evidence (a skipped layer, a malformed validator record, +an unresolved PR range), never a pass. Grey is an explicit scope +exclusion (routed files only, or no K9 files at all when `require-files` +is off). The rules are tested in `scripts/tests/k9-grid-test.sh`. diff --git a/docs/VERDICT-GRID-SPEC.adoc b/docs/VERDICT-GRID-SPEC.adoc index 1aefcc14..c601a727 100644 --- a/docs/VERDICT-GRID-SPEC.adoc +++ b/docs/VERDICT-GRID-SPEC.adoc @@ -194,6 +194,7 @@ certification; the scope footer says so in every rendering. |=== |Lane |Renderer |Workflow |DEED–K9 (conformance) |`1-formats/k9/tools/conformance-report.mjs` (tests: `conformance-report.test.mjs`) |`k9-contractile.yml`, which also posts the comment. +|K9 (conformance, reusable) |`scripts/k9-grid.sh` + `scripts/lib/k9-grid.jq` (tests: `scripts/tests/k9-grid-test.sh`) |`k9-conformance.yml` renders the grid into the job summary and the `k9-grid` artifact (`report.json` and `badge.json` as I-JSON/JCS, `report.md`, validator logs) and exposes the `verdict` output. Like the Hypatia lane it declares `contents: read` only, so the caller posts the comment. Where it sits beside the DEED lane: `docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc`. |Hypatia (findings) |`scripts/verdict-grid.sh` + `scripts/lib/verdict-grid.jq` (tests: `scripts/tests/verdict-grid-test.sh`) |`hypatia-scan-reusable.yml` renders the grid into the job summary and the `verdict-grid` artifact and exposes the `verdict` output. It does not post the comment: it declares only the permissions every existing caller already grants, and a reusable may not request more than its caller. The caller posts (see below). |=== diff --git a/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc b/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc index c292082a..cb9ff8ee 100644 --- a/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc +++ b/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc @@ -2,14 +2,14 @@ // SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) = Work orders, 2026-10-19: the first execution wave Jonathan D.A. Jewell (hyperpolymath) -v1.4.3, 2026-10-19 +v1.5.0, 2026-10-19 :toc: left :toclevels: 2 :sectnums: :icons: font [.lead] -Eight work orders for a sonnet-tier executor, planned at the top tier so +Ten work orders for a sonnet-tier executor, planned at the top tier so the executor never has to rediscover a fact. Every fact below was measured against `main` at `75bd5994` on 2026-10-19, and re-checked against every review posted on this file's pull request. Each order names its goal, its authority, the @@ -725,6 +725,113 @@ than a self-reference (move it to bucket B), or if any bucket B tool is red on `main` because of the rename (then the owner's answer is urgent, and the PR body says which gate is red). +== WO-9: the template's contractiles conform to K9 v1.0.0 (standards#1058) + +*Goal.* `hyperpolymath/rsr-template-repo` validates its deed and its +contractiles with the two canonical reusables, pinned to one standards +commit, and every contractile it ships passes contract v1.0.0. Every +repository instantiated from the template then inherits a working lane. + +*Authority.* D173 (K9 contract v1.0.0), D312 (contractiles as k9 +contracts the deed attests to), the owner's instruction of 2026-10-19 +("fix the template first, then propagate"; k9iser retired estate-wide), +and `docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc`. + +*Facts measured (2026-10-19).* The template's six contractiles +(`.machine_readable/contractiles/{intend,must,trust,adjust,dust,bust}/*.k9.ncl`) +share `_base.ncl` and all fail v1.0.0 at L1 (K9-S002/S003/S007/S009/S010: +no `schema_version`, `component_type`, `semantics`, +`security.capabilities`, `side_effects`) and L2 (K9-N001). Its +`dogfood-gate.yml` validates K9 with the `k9-ecosystem` validate action +at `@main`, not the canonical validator; `k9-validate.yml` greps. Its +`hypatia-scan.yml` calls the reusable but posts no grid. Placeholder +files (`template-*.k9.ncl`, `examples/*.k9.ncl`, `coapt`, `claude-md`, +`deploy`, `methodology-guard`, `session/custom-checks`) are not contracts +anyone evaluates. + +*Steps.* +. Migrate `_base.ncl` and the six verb files: add `schema_version`, + `component_type = "contractile:"`, `semantics`, + `security.capabilities`, `side_effects`; keep the leash at `Yard` unless + a verb genuinely executes, in which case it needs a signature block for + `Hunt`. Validate each with + `bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json FILE` + from a standards checkout at the pinned SHA. +. Delete the placeholder K9 files, or make each a real contract. A file + that stays must pass. Route `coordination.k9.ncl` (rename to + `coordination.k9`, first line `K9!`) so the validator routes it instead + of failing it. +. Replace the deed and K9 steps in `dogfood-gate.yml` with + `deed-conformance.yml@` and `k9-conformance.yml@` (both with + `require-files: true`); delete `k9-validate.yml`. Add the verdict + posting job (K9 and Hypatia grids) and the badges job from + `docs/VERDICT-GRID-SPEC.adoc` _Posting from a caller_. +. Repair `actions.lock` with the chain, in CI if the local `gh` cannot + resolve (see the echidna and panic-attack lock PRs of 2026-10-19 for + the one-off job pattern). Never hand-edit the lock. +. Update the template's `CLAUDE.md` and `rsr-template-repo_chora.deed` + `(contractiles …)` clause if a file was renamed or removed. + +*Acceptance.* +---- +for f in .machine_readable/contractiles/*/*.k9.ncl; do bash /1-formats/k9/tools/k9-validate.sh --strict --layer L2 "$f"; done -> every file: exit 0 +bun /1-formats/deed/tools/deed_lint.js rsr-template-repo_chora.deed -> exit 0 +grep -rn 'k9-ecosystem\|k9iser' .github/ Justfile justfile 2>/dev/null -> no hits +bash /scripts/check-lock-sync.sh .github/workflows -> in sync +On the PR: deed-conformance green, k9-conformance green with a 🟢 K9 grid comment, hypatia grid comment present. +---- + +*Escalate if* a verb's contract cannot be expressed in v1.0.0 without a +new capability name, or if a contractile needs `Hunt`: the signing key +and verifier are owner-held. + +== WO-10: install the lanes across the first wave, and retire k9iser + +*Goal.* `echidna`, `echidnabot`, `proof-burrower`, `hypatia` and +`panic-attack` each call `deed-conformance.yml`, `k9-conformance.yml` +and `hypatia-scan-reusable.yml` at one standards SHA, post the grids, +carry contractiles that pass v1.0.0 (or a shrink-only ledger with an +issue per line), and carry no k9iser residue. + +*Authority.* As WO-9. One PR per repository, each independently +reviewable (owner norm: stacked, small PRs). Run after WO-9 merges so the +fixed contractiles can be copied from the template rather than re-derived. + +*Facts measured (2026-10-19).* `hypatia` has `k9iser.toml`, a `ci.yml` +job and a `check-k9iser-paths` check; `echidna` has `k9iser.toml`; +`panic-attack` removed `generated/k9iser/` and `k9iser.toml` in +hyperpolymath/panic-attack#223 and its `dogfood-gate.yml` messages now +point at the K9 spec. Of the 53 consumer `.k9.ncl` files reachable, 44 +fail v1.0.0; the failures are the same five L1 rules as the template's. + +*Steps, per repository.* +. Copy the migrated contractiles from the template; keep each + repository's own `semantics` and `metadata`. +. Install the three reusable calls, the verdict posting job and the + badges job; delete the in-tree validators and every k9iser file, job + and check. Where a required status check names a job that disappears, + rename the new job to the required name in the same PR or tell the + owner which branch-protection entry to change. +. Repair `actions.lock` with the chain. +. For any contract that cannot be fixed in the PR: one line in + `.machine_readable/k9-contract-debt.txt`, one `Ratchet-exception:` + line in the PR body naming the ledger, one issue per line. +. `echidnabot` is also checked into `echidna/echidnabot/`; land the + `.machine_readable/` change in both (dual-truth note in its CLAUDE.md). + +*Acceptance (each repository).* +---- +grep -rln 'k9iser' . --exclude-dir=.git -> no hits +grep -n 'hyperpolymath/standards/' .github/workflows/*.yml | grep -o '@[0-9a-f]*' | sort -u | wc -l -> 1 +k9-conformance job green or yellow; the K9 grid comment names every ledgered file as advisory and no file as indeterminate +deed-conformance green; hypatia grid comment present +---- + +*Escalate if* a repository's required checks are owner-managed branch +protection the PR cannot satisfy, or if a repository's contractiles +encode behaviour the template's `semantics` do not cover (then write +them, do not copy). + == Owner items this wave depends on . Grant this session permission to push `tidy/*` branches, one line. diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh new file mode 100755 index 00000000..98a04acd --- /dev/null +++ b/scripts/k9-grid.sh @@ -0,0 +1,177 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# k9-grid.sh — render the compact verdict grid for the K9 conformance lane. +# Policy and anatomy: docs/VERDICT-GRID-SPEC.adoc. Logic: scripts/lib/k9-grid.jq. +# The lane that calls it: .github/workflows/k9-conformance.yml. +# +# Usage: +# k9-grid.sh render --out DIR --results FILE [evidence] [identity] +# evidence +# --results FILE k9-validate.sh --json output, one record per line +# --expected FILE the files the lane set out to check, one per line +# --ledger FILE the caller's .machine_readable/k9-contract-debt.txt +# --changed FILE files the PR touches, one per line +# --base-unresolved the PR base/head range could not be diffed +# --selftest LOG --selftest-code N validator --self-test evidence +# --typecheck LOG --typecheck-code N nickel typecheck of the contract +# --corpus-code N the validator's exit status over the corpus +# --previous FILE body of the previous grid comment (delta column) +# identity +# --lane NAME grid lane (default: k9) +# --sha SHA --run ID --url URL --pr N --repo OWNER/NAME +# --canon-ref SHA the hyperpolymath/standards commit that supplied the validator +# k9-grid.sh baseline FILE +# Print the baseline embedded in a previous comment body, or null. +# +# Outputs in DIR: report.json (structured), report.md (the grid), badge.json +# (shields.io endpoint schema). The two JSON files are I-JSON serialised as +# JCS (3-practice/JSON-POLICY.adoc): through `ijson-jcs canon` when it is on +# PATH, else jq's sorted compact form, which is byte-identical for the ASCII, +# integer-only documents this renderer emits. Exit 0 when a report was +# rendered, whatever its verdict; exit 2 on a usage error. Gating is the +# caller's decision, made from report.json, never from this exit code. +# +# Dependencies: bash, jq, sha256sum, base64. Optional: ijson-jcs. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +JQ_LIB="$SCRIPT_DIR/lib/k9-grid.jq" + +usage() { sed -n '5,34p' "$0" >&2; exit 2; } + +# A path list file to a JSON array; a missing file is an empty list. +list_json() { + if [ -n "${1:-}" ] && [ -f "$1" ]; then + grep -vE '^[[:space:]]*(#|$)' "$1" | jq -R . | jq -s . + else + echo '[]' + fi +} + +# A log file plus exit status to {code, output}; a missing log is null code. +evidence_json() { + local log="${1:-}" code="${2:-}" + if [ -n "$log" ] && [ -f "$log" ]; then + jq -Rs --arg code "$code" '{output: ., code: (if $code == "" then null else ($code | tonumber) end)}' "$log" + else + jq -n '{output: "", code: null}' + fi +} + +# results.jsonl to a JSON array; a line that is not JSON is kept as evidence +# of malformed validator output rather than dropped. +results_json() { + local file="${1:-}" + [ -n "$file" ] && [ -f "$file" ] || { echo '[]'; return; } + while IFS= read -r line || [ -n "$line" ]; do + [ -n "$line" ] || continue + if printf '%s' "$line" | jq -e 'type == "object"' >/dev/null 2>&1; then + printf '%s\n' "$line" + else + jq -n --arg l "$line" '{malformed: $l}' + fi + done < "$file" | jq -s . +} + +canon_json() { + if command -v ijson-jcs >/dev/null 2>&1; then + ijson-jcs canon + else + jq -S -c . + fi +} + +baseline_of() { + local file="$1" b64 + [ -s "$file" ] || { echo null; return; } + b64=$( { grep -o '' "$file" || true; } | tail -n1 \ + | sed 's/^$//') + if [ -z "$b64" ] || [ "${#b64}" -gt 32000 ]; then echo null; return; fi + printf '%s' "$b64" | base64 -d 2>/dev/null | jq -c ' + if type == "object" and (.categories | type) == "array" + and all(.categories[]; . as $c | ($c.id | type) == "string" and ($c.counts | type) == "object" + and all(["passed", "failed", "indeterminate", "not_applicable", "advisory"][]; . as $k + | ($c.counts | has($k)) and ($c.counts[$k] | type) == "number" and $c.counts[$k] >= 0)) + then . else null end' 2>/dev/null || echo null +} + +cmd="${1:-}"; shift || true +case "$cmd" in + baseline) + [ -n "${1:-}" ] || usage + baseline_of "$1" + ;; + render) + out="" results="" expected="" ledger="" changed="" prev="" lane="k9" + selftest="" selftest_code="" typecheck="" typecheck_code="" corpus_code="" + base_resolved=true sha="" run="local" url="" pr="local" repo="" canon="" + while [ $# -gt 0 ]; do + case "$1" in + --out) out="$2"; shift 2 ;; + --results) results="$2"; shift 2 ;; + --expected) expected="$2"; shift 2 ;; + --ledger) ledger="$2"; shift 2 ;; + --changed) changed="$2"; shift 2 ;; + --base-unresolved) base_resolved=false; shift ;; + --selftest) selftest="$2"; shift 2 ;; + --selftest-code) selftest_code="$2"; shift 2 ;; + --typecheck) typecheck="$2"; shift 2 ;; + --typecheck-code) typecheck_code="$2"; shift 2 ;; + --corpus-code) corpus_code="$2"; shift 2 ;; + --previous) prev="$2"; shift 2 ;; + --lane) lane="$2"; shift 2 ;; + --sha) sha="$2"; shift 2 ;; + --run) run="$2"; shift 2 ;; + --url) url="$2"; shift 2 ;; + --pr) pr="$2"; shift 2 ;; + --repo) repo="$2"; shift 2 ;; + --canon-ref) canon="$2"; shift 2 ;; + *) echo "unknown option: $1" >&2; usage ;; + esac + done + [ -n "$out" ] && [ -n "$results" ] || usage + [ -f "$JQ_LIB" ] || { echo "missing $JQ_LIB" >&2; exit 2; } + mkdir -p "$out" + + previous=null + [ -n "$prev" ] && previous=$(baseline_of "$prev") + + # The profile names what produced the grid: this renderer, the standards + # commit that supplied validator and contract, the ledger and the lane. + # Two revisions compare only under the same profile. + profile=$( { cat "$0" "$JQ_LIB"; [ -n "$ledger" ] && [ -f "$ledger" ] && cat "$ledger"; printf '%s\n' "$canon" "$lane"; } | sha256sum | cut -d' ' -f1) + + dirty=false + if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + [ -z "$sha" ] && sha=$(git rev-parse HEAD) + git diff --quiet HEAD 2>/dev/null || dirty=true + fi + + corpus_json=null + [ -n "$corpus_code" ] && corpus_json="$corpus_code" + + jq -n --argjson results "$(results_json "$results")" --argjson expected "$(list_json "$expected")" \ + --argjson ledger "$(list_json "$ledger")" --argjson changed "$(list_json "$changed")" \ + --argjson base_resolved "$base_resolved" --argjson selftest "$(evidence_json "$selftest" "$selftest_code")" \ + --argjson typecheck "$(evidence_json "$typecheck" "$typecheck_code")" --argjson corpus_code "$corpus_json" \ + --arg profile "$profile" --arg lane "$lane" --arg sha "$sha" --arg run "$run" --arg url "$url" \ + --arg pr "$pr" --arg repo "$repo" --argjson dirty "$dirty" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + --arg canon "$canon" \ + '{results: $results, expected: $expected, ledger: $ledger, changed: $changed, base_resolved: $base_resolved, + selftest: $selftest, typecheck: $typecheck, corpus_code: $corpus_code, profile: $profile, + identity: {lane: $lane, sha: $sha, run: $run, url: $url, pr: $pr, repo: $repo, dirty: $dirty, + generated: $generated, canon_ref: $canon}}' \ + | jq --arg mode build --argjson previous null -f "$JQ_LIB" | canon_json > "$out/report.json" + + jq -r --arg mode render --argjson previous "$previous" -f "$JQ_LIB" "$out/report.json" > "$out/report.md" + jq --arg mode badge --argjson previous null -f "$JQ_LIB" "$out/report.json" | canon_json > "$out/badge.json" + + if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then cat "$out/report.md" >> "$GITHUB_STEP_SUMMARY"; fi + if [ -n "${GITHUB_OUTPUT:-}" ]; then echo "state=$(jq -r .state "$out/report.json")" >> "$GITHUB_OUTPUT"; fi + cat "$out/report.md" + ;; + *) usage ;; +esac diff --git a/scripts/lib/k9-grid.jq b/scripts/lib/k9-grid.jq new file mode 100644 index 00000000..b37abb25 --- /dev/null +++ b/scripts/lib/k9-grid.jq @@ -0,0 +1,184 @@ +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# k9-grid.jq — the conformance-family verdict grid for the K9 lane a consumer +# repository calls (docs/VERDICT-GRID-SPEC.adoc §Conformance family). +# +# This is a port of the item, state and delta rules in +# 1-formats/k9/tools/conformance-report.mjs (the DEED–K9 lane that runs in +# this repository), so the two K9 lanes grade one result the same way. Where +# the two disagree, the .mjs is the reference rendering and this file is a bug. +# It is jq rather than JavaScript so the estate's authored-JavaScript surface +# does not grow (docs/NO-JAVASCRIPT-SOURCE-POLICY.adoc). +# +# Input (one object, assembled by scripts/k9-grid.sh): +# { identity: {lane, sha, run, url, pr, repo, dirty, generated, canon_ref}, +# profile: "sha256 hex", +# expected: [ "path" ... ], # every file the lane set out to check +# results: [ one k9-validate.sh --json record per file, or +# {malformed: ""} for a line that was not JSON ], +# ledger: [ "path" ... ], # .machine_readable/k9-contract-debt.txt +# changed: [ "path" ... ], # files this PR touches +# base_resolved: bool, # false when the PR range could not be diffed +# selftest: {code: int|null, output: "..."}, +# typecheck: {code: int|null, output: "..."}, +# corpus_code: int|null } # the validator's own exit status +# Output: the structured report; `render` turns it into Markdown. + +def item(id; status; reason; advisory): + { id: (id | tostring), status: status, reason: (reason // "" | tostring), advisory: advisory }; +def item(id; status; reason): item(id; status; reason; false); + +def counts: + { passed: ([.[] | select(.status == "pass")] | length), + failed: ([.[] | select(.status == "fail")] | length), + indeterminate: ([.[] | select(.status == "indeterminate")] | length), + not_applicable: ([.[] | select(.status == "not_applicable")] | length), + advisory: ([.[] | select(.status == "fail" and .advisory)] | length) }; + +# Precedence: red > orange > yellow > green; grey only when every item is grey. +def state: + if length == 0 then "orange" + elif any(.[]; .status == "fail" and (.advisory | not)) then "red" + elif any(.[]; .status == "indeterminate") then "orange" + elif any(.[]; .advisory) then "yellow" + elif all(.[]; .status == "not_applicable") then "grey" + else "green" end; + +# `ok NAME` / `FAIL NAME` / `SKIP NAME` lines from the validator's self-test. +def assertions(output): + [ (output // "" | gsub("\u001b\\[[0-9;]*m"; "") | split("\n")[]) + | capture("^(?ok|FAIL|SKIP)\\s+(?.+)$")? + | item(.rest; (if .k == "ok" then "pass" elif .k == "FAIL" then "fail" else "indeterminate" end); + (if .k == "ok" then "" else .rest end)) ]; + +def checked_assertions(output; code): + assertions(output) as $rows + | $rows + + (if ($rows | length) == 0 then [item("check execution"; "indeterminate"; "No assertion evidence was produced.")] else [] end) + + (if code != 0 and ($rows | all(.[]; .status == "pass")) and ($rows | length) > 0 + then [item("check execution"; "indeterminate"; "Process exited \(code // "without a status") despite passing assertion lines.")] else [] end); + +def selftest_items(st): + checked_assertions(st.output; st.code) + + (if st.code == 0 and ((st.output // "") | contains("self-test: all assertions passed") | not) + then [item("completion"; "indeterminate"; "Missing self-test completion evidence.")] else [] end); + +def typecheck_items(tc; contract): + [ item(contract; + (if tc.code == 0 then "pass" elif tc.code == null then "indeterminate" else "fail" end); + (if tc.code == 0 then "" else (tc.output // "" | sub("\\s+$"; "")) end)) ]; + +# One live-corpus item per validator record (conformance-report.mjs corpusItems). +def corpus_items(results; expected; ledger; changed): + (ledger | map({key: ., value: true}) | from_entries) as $ledger + | (changed | map({key: ., value: true}) | from_entries) as $changed + | (expected | map({key: ., value: true}) | from_entries) as $expected + | reduce results[] as $r ({seen: {}, rows: []}; + if ($r | has("malformed")) then + .rows += [item("validator output"; "indeterminate"; "Malformed JSON result.")] + elif ($expected[$r.file] | not) or (.seen[$r.file] // false) + or (($r.verdict // "") | IN("pass", "fail", "error", "routed") | not) + or (($r.findings | type) != "array") then + .rows += [item(($r.file // "validator output"); "indeterminate"; "Unexpected, duplicate or invalid result.")] + else + .seen[$r.file] = true + | ([$r.findings[] | select(.severity == "skipped")]) as $unavailable + | ([$r.findings[] | select(.severity == "error")]) as $errors + | ([$r.findings[] | select(.severity == "warning")]) as $warnings + | ([$r.findings[] | "\(.rule): \(.message)"] | join("; ")) as $reason + | .rows += [ + if $r.verdict == "routed" then + item($r.file; "not_applicable"; "Routed to another dialect; not validated as a K9 contract.") + elif ($errors | any(.rule == "K9-N002")) and ($errors | any(.rule != "K9-N002") | not) then + item($r.file; "indeterminate"; $reason) + elif ($errors | length) > 0 or $r.verdict == "fail" or $r.verdict == "error" then + item($r.file; "fail"; $reason; (($ledger[$r.file] // false) and (($changed[$r.file] // false) | not))) + elif ($unavailable | length) > 0 then + item($r.file; "indeterminate"; $reason) + elif ($ledger[$r.file] // false) then + item($r.file; "fail"; "Stale debt exemption: this file now conforms.") + else + item($r.file; "pass"; (if ($warnings | length) > 0 then $reason else "" end); (($warnings | length) > 0)) + end ] + end) + | . as $acc + | $acc.rows + + [ expected[] | select(($acc.seen[.] // false) | not) | item(.; "indeterminate"; "Validator did not return a result for this file.") ] + + (if (expected | length) == 0 then [item("live corpus"; "not_applicable"; "No tracked K9 contracts or routing files.")] else [] end); + +def titles: { typecheck: "K9 contract typecheck", selftest: "K9 validator self-test", corpus: "K9 live corpus (L0–L2)" }; + +def category(id; items): { id: id, title: titles[id], items: items, counts: (items | counts), state: (items | state) }; + +def build: + . as $in + | ("1-formats/k9/spec/contract/k9_contract.ncl") as $contract + | (corpus_items($in.results // []; $in.expected // []; $in.ledger // []; $in.changed // []) + + (if $in.base_resolved == false then [item("change scope"; "indeterminate"; "Cannot resolve the PR base/head range; debt exemptions cannot be determined.")] else [] end)) as $corpus0 + | ($corpus0 + + (if ($in.corpus_code // 0) != 0 and ($corpus0 | any(.[]; .status == "indeterminate" or (.status == "fail" and (.advisory | not))) | not) + then [item("corpus gate"; "indeterminate"; "Validator exited \($in.corpus_code); inspect corpus.log.")] else [] end)) as $corpus + | [ category("typecheck"; typecheck_items($in.typecheck // {code: null, output: ""}; $contract)), + category("selftest"; selftest_items($in.selftest // {code: null, output: ""})), + category("corpus"; $corpus) ] as $categories + | ([$categories[].items[]]) as $all + | { schema: 1, lane: ($in.identity.lane // "k9"), profile: ($in.profile // ""), + sha: ($in.identity.sha // ""), run: ($in.identity.run // "local"), url: ($in.identity.url // ""), + pr: ($in.identity.pr // "local"), repo: ($in.identity.repo // ""), dirty: ($in.identity.dirty // false), + generated: ($in.identity.generated // ""), canon_ref: ($in.identity.canon_ref // ""), + categories: $categories, state: ($all | state), counts: ($all | counts) }; + +# One category's movement since the previous revision of the same PR. +def delta(prev; report): + . as $c + | if prev == null or prev.schema != 1 or prev.profile != report.profile or prev.sha == report.sha or prev.pr != report.pr + then "— no comparable baseline" + else (prev.categories | map(select(.id == $c.id)) | first) as $b + | if $b == null or ($b.counts.indeterminate > 0) or ($c.counts.indeterminate > 0) then "— indeterminate" + elif $b.counts.not_applicable != $c.counts.not_applicable then "— scope changed" + elif $b.counts.failed != $c.counts.failed then + (if $c.counts.failed < $b.counts.failed then "↑ improved (\($b.counts.failed - $c.counts.failed) fewer failures)" + else "↓ regressed (\($c.counts.failed - $b.counts.failed) more failures)" end) + elif $c.counts.advisory < $b.counts.advisory then "↑ fewer advisories" + elif $c.counts.advisory > $b.counts.advisory then "↓ more advisories" + else "↔ unchanged" end end; + +def state_label: {red: "🔴 failing", orange: "🟠 indeterminate", yellow: "🟡 advisory", green: "🟢 conformant", grey: "⚪ not applicable"}[.]; +def colour: {red: "red", orange: "orange", yellow: "yellow", green: "brightgreen", grey: "lightgrey"}[.]; +def word: {red: "failing", orange: "indeterminate", yellow: "advisory", green: "conformant", grey: "not_applicable"}[.]; +def esc: tostring | gsub("&"; "&") | gsub("<"; "<") | gsub(">"; ">") | gsub("\\|"; "|") + | gsub("`"; "`") | gsub("\r?\n"; " "); +def grid_label: "K9 L0–L2"; +def badge_url(st): "https://img.shields.io/badge/\(grid_label | @uri)-\(st | word | @uri)-\(st | colour)"; + +def render(prev): + . as $r + | "\n" + + "![\(grid_label) \($r.state)](\(badge_url($r.state)))\n\n" + + "\($r.state | state_label) · commit `\($r.sha[0:12] | esc)` · " + + (if $r.url != "" then "[run \($r.run | esc)](\($r.url))" else "local run" end) + + (if $r.canon_ref != "" then " · validator hyperpolymath/standards@`\($r.canon_ref[0:12] | esc)`" else "" end) + + (if $r.dirty then " · working tree modified" else "" end) + "\n\n" + + "| Category | Status | Pass | Fail | Indeterminate | N/A | Since previous revision |\n|---|---|---:|---:|---:|---:|---|\n" + + ([ $r.categories[] + | "| \(.title) | \(.state | state_label) | \(.counts.passed) | \(.counts.failed)\(if .counts.advisory > 0 then " (\(.counts.advisory) advisory)" else "" end) | \(.counts.indeterminate) | \(.counts.not_applicable) | \(delta(prev; $r)) |\n" ] | add // "") + + ([ $r.categories[] + | ([.items[] | select(.status != "pass" or .advisory)]) as $f + | select(($f | length) > 0) + | "\n
\n\(.title): \($f | length) finding(s) / scope notes\n\n" + + ([ $f[0:20][] | "- **\(.id | esc | .[0:240])** — \(.status | esc)\(if .advisory then " (advisory)" else "" end): \(.reason | esc | .[0:280])\n" ] | add // "") + + (if ($f | length) > 20 then "- \(($f | length) - 20) more; see the structured report and logs in the artifact.\n" else "" end) + + "\n
\n" ] | add // "") + + "\nScope: K9 components L0–L2 against contract v1.0.0 at the named standards commit. L3 signature verification and routed coordination-dialect files are not certified. Green is limited to this scope. Orange is missing evidence; yellow is grandfathered debt or warnings; grey is an explicit scope exclusion. report.json, badge.json and the validator logs are in the k9-grid artifact on the linked run.\n" + + "\n\n"; + +def badge: {schemaVersion: 1, label: grid_label, message: (.state | word | gsub("_"; " ")), color: (.state | colour)}; + +# Entry point: $mode selects build (input: the assembled object), render +# (input: report.json; $previous: the prior baseline or null) or badge. +if $mode == "build" then build +elif $mode == "render" then render($previous) +elif $mode == "badge" then badge +else error("unknown mode \($mode)") end diff --git a/scripts/tests/k9-grid-test.sh b/scripts/tests/k9-grid-test.sh new file mode 100644 index 00000000..b0aee6f8 --- /dev/null +++ b/scripts/tests/k9-grid-test.sh @@ -0,0 +1,175 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) +# +# k9-grid-test.sh — the conformance-family grid for the K9 lane never turns +# missing, skipped or unreadable evidence into a clean row, honours the +# shrink-only debt ledger exactly as CONFORMANCE-REPORTING.adoc states it, +# keeps routed files and an empty corpus grey, compares revisions only under +# one profile, and writes I-JSON/JCS artifacts. docs/VERDICT-GRID-SPEC.adoc +# and 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc are the contracts under test. +# +# Run: bash scripts/tests/k9-grid-test.sh + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +GRID="$SCRIPT_DIR/../k9-grid.sh" +WORK="$(mktemp -d)" +trap 'rm -rf "$WORK"' EXIT + +pass=0 +fail=0 +check() { + local label="$1" expected="$2" got="$3" + if [ "$got" = "$expected" ]; then + echo "PASS: $label ($got)"; pass=$((pass + 1)) + else + echo "FAIL: $label expected=$expected got=$got"; fail=$((fail + 1)) + fi +} +render() { bash "$GRID" render --out "$1" "${@:2}" >/dev/null; } +state() { jq -r .state "$1/report.json"; } +cat_state() { jq -r --arg id "$2" '.categories[] | select(.id == $id) | .state' "$1/report.json"; } +reason() { jq -r --arg id "$2" '[.categories[].items[] | select(.id == $id)][0].reason' "$1/report.json"; } + +# One validator record per line, in k9-validate.sh --json shape. +rec() { # rec FILE VERDICT FINDINGS_JSON + jq -nc --arg f "$1" --arg v "$2" --argjson fd "$3" \ + '{file: $f, dialect: "component", contract_version: "1.0.0", verdict: $v, + errors: ([$fd[] | select(.severity == "error")] | length), + warnings: ([$fd[] | select(.severity == "warning")] | length), + skipped: ([$fd[] | select(.severity == "skipped")] | length), findings: $fd}' +} +E_S002='[{"severity":"error","rule":"K9-S002","layer":"L1","message":"component_type missing"}]' +E_N002='[{"severity":"skipped","rule":"K9-N002","layer":"L2","message":"contract not typechecked"}]' +E_N002_ERR='[{"severity":"error","rule":"K9-N002","layer":"L2","message":"the normative contract itself failed to typecheck"}]' +S_C001='[{"severity":"skipped","rule":"K9-C001","layer":"L3","message":"no verifier ran"}]' +W_ONE='[{"severity":"warning","rule":"K9-W001","layer":"L1","message":"description is short"}]' + +printf '%s\n' a.k9.ncl b.k9.ncl > "$WORK/expected" +printf '%s\n' a.k9.ncl > "$WORK/expected-a" +: > "$WORK/expected-none" +printf '%s\n' b.k9.ncl > "$WORK/ledger-b" +printf '# comment\n\nb.k9.ncl\n' > "$WORK/ledger-b-commented" +printf '%s\n' b.k9.ncl > "$WORK/changed-b" + +ok_selftest="$WORK/selftest-ok.log" +printf 'ok one\nok two\n\nself-test: all assertions passed\n' > "$ok_selftest" +bad_selftest="$WORK/selftest-bad.log" +printf 'ok one\nFAIL two\n' > "$bad_selftest" +: > "$WORK/empty.log" +printf 'error: type mismatch\n' > "$WORK/typecheck-bad.log" + +good_evidence=(--selftest "$ok_selftest" --selftest-code 0 --typecheck "$WORK/empty.log" --typecheck-code 0 --corpus-code 0) + +# 1. Every row clean -> green, and the artifacts are JCS. +{ rec a.k9.ncl pass '[]'; rec b.k9.ncl pass '[]'; } > "$WORK/r-clean" +render "$WORK/o1" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 --run 1 +check "clean corpus is green" green "$(state "$WORK/o1")" +check "report.json is JCS (sorted, compact, one LF)" yes "$( [ "$(jq -S -c . "$WORK/o1/report.json"; echo)" = "$(cat "$WORK/o1/report.json")" ] && [ "$(tail -c1 "$WORK/o1/report.json" | od -An -c | tr -d ' ')" = '\n' ] && echo yes || echo no)" +check "badge.json uses the shields schema" "K9 L0–L2/conformant/brightgreen" "$(jq -r '"\(.label)/\(.message)/\(.color)"' "$WORK/o1/badge.json")" +check "report.md carries the lane marker" yes "$(grep -q '^' "$WORK/o1/report.md" && echo yes || echo no)" +check "report.md carries a baseline" yes "$(grep -q '\n' > "$WORK/corrupt"; bash "$GRID" baseline "$WORK/corrupt")" +check "baseline of a real report round-trips" 7 "$(bash "$GRID" baseline "$WORK/o11a/report.md" | jq -r .pr)" + +# 12. Usage errors exit 2. +check "missing --results exits 2" 2 "$(bash "$GRID" render --out "$WORK/o12" >/dev/null 2>&1; echo $?)" +check "unknown subcommand exits 2" 2 "$(bash "$GRID" frobnicate >/dev/null 2>&1; echo $?)" + +echo +echo "k9-grid-test: $pass passed, $fail failed" +[ "$fail" -eq 0 ] From 69ad4445ef2dd50423f454ac6f0d49b49d23fc0f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 21:32:00 +0000 Subject: [PATCH 2/7] fix(k9-conformance): record the canon ref in a step; job context is not available to job-level env The first run of the lane (39327642240) failed before any job started: job-level env cannot read the job context, and CANON_REF used job.workflow_sha there. The ref is now written to GITHUB_ENV by a step, where job.workflow_sha is permitted, exactly as deed-conformance uses it in a step's with: block. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- .github/workflows/k9-conformance.yml | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/.github/workflows/k9-conformance.yml b/.github/workflows/k9-conformance.yml index 68aa1d1c..a699dab5 100644 --- a/.github/workflows/k9-conformance.yml +++ b/.github/workflows/k9-conformance.yml @@ -89,7 +89,6 @@ jobs: # gets validator, contract and renderer from the canon checkout at # job.workflow_sha. CANON: ${{ github.repository == 'hyperpolymath/standards' && '.' || '.k9-canon' }} - CANON_REF: ${{ github.repository == 'hyperpolymath/standards' && (github.event.pull_request.head.sha || github.sha) || job.workflow_sha }} NICKEL_VERSION: '1.18.0' NICKEL_SHA256: '9cba4dd65ae9915ec61f73033aafcff307a377665a83fd8f530df086763318cb' IJSON_JCS_REV: '0fdeea716fd558e9807362acf6f8202d56d8d88f' @@ -110,6 +109,19 @@ jobs: sparse-checkout: "1-formats/k9\nscripts/k9-grid.sh\nscripts/lib/k9-grid.jq\n" sparse-checkout-cone-mode: false + - name: Record which standards commit supplies validator, contract and renderer + env: + # The job context is not available to job-level env, so the canon + # ref is recorded here. A caller gets this workflow's own commit. + WORKFLOW_SHA: ${{ job.workflow_sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + run: | + if [ "$GITHUB_REPOSITORY" = "hyperpolymath/standards" ]; then + echo "CANON_REF=$HEAD_SHA" >> "$GITHUB_ENV" + else + echo "CANON_REF=$WORKFLOW_SHA" >> "$GITHUB_ENV" + fi + - name: Install Nickel (pinned, sha256-verified; one version estate-wide) run: | set -euo pipefail From f99aa013d0e91a3eebcadb7185e9e15e90385fa0 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 21:35:01 +0000 Subject: [PATCH 3/7] fix(k9-grid): end report.json and badge.json with exactly one LF after ijson-jcs canon ijson-jcs canon writes the JCS bytes with no final newline, so on a runner with the canonicaliser installed report.json lacked the one LF 3-practice/JSON-POLICY.adoc requires, and the 'report.json is JCS' assertion failed in both the lane's self-run and the repo self-tests (42/43). The jq fallback already ended with LF. Both paths now pass through sed '$a\', which appends a newline only when it is missing. Verified locally with the pinned ijson-jcs on PATH and without it: 43/43 either way. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- scripts/k9-grid.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh index 98a04acd..d0871579 100755 --- a/scripts/k9-grid.sh +++ b/scripts/k9-grid.sh @@ -77,10 +77,12 @@ results_json() { } canon_json() { + # ijson-jcs canon emits the JCS bytes with no final newline; the policy + # wants exactly one. sed adds one only when it is missing. if command -v ijson-jcs >/dev/null 2>&1; then - ijson-jcs canon + ijson-jcs canon | sed -e '$a\' else - jq -S -c . + jq -S -c . | sed -e '$a\' fi } From 18acd1120f406ed3e20987aff75f3f3a3554cc63 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 22:02:26 +0000 Subject: [PATCH 4/7] fix(k9-conformance): refuse undeclared ledger growth and keep advisory exits out of orange Codex review on #1308, each at its source: - An entry the PR adds to .machine_readable/k9-contract-debt.txt exempts nothing unless a commit message or the PR body carries a Ratchet-exception naming the ledger (the exemption ratchet's form). The lane diffs the ledger against the merge base, writes the undeclared additions to --ledger-growth, and the grid gives each a failing row of its own while grading the file as unledgered. The governance exemption ratchet now watches this ledger too. - A ledgered (advisory) failure explains the validator's nonzero exit: the exit here is the raw validator's, not the ledger-aware hook's, so the "corpus gate" row no longer turns a yellow grid orange. - A ledger path that names no tracked K9 file is a stale exemption (red). The twelve session-management PROTOCOL.k9 paths in this repo's ledger were never tracked files; removed (shrink). - A validator record with no string file key is invalid evidence, not a jq abort. - The profile covers the corpus, as conformance-report.mjs does, so a corpus that gained or lost a file is "no comparable baseline", never "fewer failures". - Passing-with-warnings rows are counted (counts.warnings) and the delta column reports them. - On a pull request the checkout is full, the changed-file list is the merge-base diff, and the lane reads the last posted grid comment (pull-requests: read) so --previous is actually supplied. - An enumeration with no K9 file no longer fails under pipefail. scripts/tests/k9-grid-test.sh: 43 -> 54 assertions; the enumerate step was exercised in a scratch repository for undeclared growth, growth declared in a commit, growth declared in the PR body, a base branch that moved, a push event and an empty corpus. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- .github/workflows/k9-conformance.yml | 105 ++++++++++++++++-- .machine_readable/k9-contract-debt.txt | 17 +-- 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc | 14 ++- docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc | 11 +- docs/VERDICT-GRID-SPEC.adoc | 2 +- scripts/check-exemption-ratchet.sh | 3 + scripts/k9-grid.sh | 25 +++-- scripts/lib/k9-grid.jq | 48 ++++++-- scripts/tests/k9-grid-test.sh | 36 +++++- 9 files changed, 208 insertions(+), 53 deletions(-) diff --git a/.github/workflows/k9-conformance.yml b/.github/workflows/k9-conformance.yml index a699dab5..30f3fc5c 100644 --- a/.github/workflows/k9-conformance.yml +++ b/.github/workflows/k9-conformance.yml @@ -18,18 +18,27 @@ name: k9-conformance # that validator. Layers L0–L2 run; L3 (signature) is outside this lane and # the grid says so. The caller's shrink-only debt ledger # (.machine_readable/k9-contract-debt.txt) is honoured as -# 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc states it. +# 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc states it, and compared with +# the PR base: an entry the PR adds exempts nothing unless a commit message or +# the PR body carries `Ratchet-exception: .machine_readable/k9-contract-debt.txt +# — ` (the governance exemption ratchet's form; it watches this ledger +# too). On a pull request the checkout is full so the changed-file list is the +# merge-base diff and the PR's commit messages are readable. # # Outputs: the `k9-grid` artifact (report.json and badge.json as I-JSON/JCS, # report.md, the validator logs) and the `verdict` output. The caller posts # report.md to its pull request and publishes badge.json itself: a reusable # cannot ask for more permissions than it is given, and this one asks for -# `contents: read` only. +# `contents: read` plus `pull-requests: read` (to read the grid comment it +# posted last time, for the "since previous revision" column). # # Caller: # jobs: # k9: # uses: hyperpolymath/standards/.github/workflows/k9-conformance.yml@ +# permissions: +# contents: read +# pull-requests: read # with: # require-files: true # fail when the caller has no K9 file at all # fail-on-indeterminate: true # orange fails the job too (default: only red fails) @@ -82,6 +91,7 @@ jobs: timeout-minutes: 15 permissions: contents: read + pull-requests: read outputs: verdict: ${{ steps.grid.outputs.state }} env: @@ -96,6 +106,10 @@ jobs: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: ref: ${{ github.event.pull_request.head.sha || github.sha }} + # Full history on a pull request: the changed-file list is the + # merge-base diff, and the ledger ratchet reads the PR's commit + # messages. Strings, not numbers: a bare 0 is falsy in an expression. + fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} persist-credentials: false - name: checkout the K9 validator, contract and renderer at this workflow's commit @@ -166,27 +180,66 @@ jobs: echo "TYPECHECK_CODE=$?" >> "$GITHUB_ENV" cat "$RUNNER_TEMP/typecheck.log" - - name: Enumerate the committed K9 files and the PR's changed files + - name: Enumerate the committed K9 files, the PR's changed files and ledger growth env: BASE_SHA: ${{ github.event.pull_request.base.sha }} HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + LEDGER: .machine_readable/k9-contract-debt.txt run: | set -euo pipefail # Tracked files only: an untracked file is not a committed contract. + # A caller with no K9 file at all is a legitimate (grey) corpus, so + # grep's no-match exit is not an error here. git ls-files -z -- '*.k9.ncl' '*.k9' '*.k9.lock' \ | tr '\0' '\n' \ - | grep -v -E '^(\.k9-canon/|9-archive/|1-formats/k9/tools/fixtures/)' \ + | { grep -v -E '^(\.k9-canon/|9-archive/|1-formats/k9/tools/fixtures/)' || [ $? -eq 1 ]; } \ | sort > "$RUNNER_TEMP/expected" echo "$(wc -l < "$RUNNER_TEMP/expected") committed K9 file(s)" : > "$RUNNER_TEMP/changed" + : > "$RUNNER_TEMP/ledger-growth" echo "BASE_UNRESOLVED=" >> "$GITHUB_ENV" - if [ -n "$BASE_SHA" ]; then - if git fetch --quiet --depth=1 origin "$BASE_SHA" 2>/dev/null && git diff --name-only "$BASE_SHA" "$HEAD_SHA" > "$RUNNER_TEMP/changed" 2>/dev/null; then - echo "$(wc -l < "$RUNNER_TEMP/changed") file(s) changed against the PR base" - else - echo "::warning::could not resolve the PR base/head range; debt exemptions are reported as indeterminate" - echo "BASE_UNRESOLVED=--base-unresolved" >> "$GITHUB_ENV" - fi + [ -n "$BASE_SHA" ] || exit 0 + + # The PR's own changes are those since its merge base, not since the + # base branch tip: a file the base branch changed after the PR + # branched is not this PR's edit. + if git fetch --quiet origin "$BASE_SHA" 2>/dev/null \ + && merge_base=$(git merge-base "$BASE_SHA" "$HEAD_SHA" 2>/dev/null) \ + && git diff --name-only "$merge_base" "$HEAD_SHA" > "$RUNNER_TEMP/changed" 2>/dev/null; then + echo "$(wc -l < "$RUNNER_TEMP/changed") file(s) changed since the merge base $merge_base" + else + echo "::warning::could not resolve the PR base/head range; debt exemptions are reported as indeterminate" + echo "BASE_UNRESOLVED=--base-unresolved" >> "$GITHUB_ENV" + exit 0 + fi + + # The ledger is shrink-only. Entries present at HEAD and absent at + # the merge base are growth; growth exempts nothing unless a commit + # message in the PR or the PR body declares it in the exemption + # ratchet's form, naming this ledger: + # Ratchet-exception: .machine_readable/k9-contract-debt.txt — + # Ratchet-exception(.machine_readable/k9-contract-debt.txt): + entries() { { grep -vE '^[[:space:]]*(#|$)' || [ $? -eq 1 ]; } | sort -u; } + head_entries=$RUNNER_TEMP/ledger-head + base_entries=$RUNNER_TEMP/ledger-base + if [ -f "$LEDGER" ]; then entries < "$LEDGER" > "$head_entries"; else : > "$head_entries"; fi + { git show "$merge_base:$LEDGER" 2>/dev/null || true; } | entries > "$base_entries" + comm -13 "$base_entries" "$head_entries" > "$RUNNER_TEMP/ledger-growth" + [ -s "$RUNNER_TEMP/ledger-growth" ] || exit 0 + declared=no + if git log --format=%B "$merge_base..$HEAD_SHA" | grep -iE '^Ratchet-exception' | grep -qF "$LEDGER"; then + declared=yes + elif [ -f "${GITHUB_EVENT_PATH:-}" ] \ + && jq -r '.pull_request.body // ""' "$GITHUB_EVENT_PATH" | grep -iE '^Ratchet-exception' | grep -qF "$LEDGER"; then + declared=yes + fi + n=$(wc -l < "$RUNNER_TEMP/ledger-growth") + if [ "$declared" = yes ]; then + echo "::notice::$LEDGER grew by $n entr(y/ies) under a declared Ratchet-exception" + : > "$RUNNER_TEMP/ledger-growth" + else + echo "::error::$LEDGER grew by $n entr(y/ies) with no Ratchet-exception naming it; the new entries exempt nothing" + sed 's/^/ + /' "$RUNNER_TEMP/ledger-growth" fi - name: Validate every committed K9 file (L0–L2, strict) @@ -203,6 +256,34 @@ jobs: echo "CORPUS_CODE=$worst" >> "$GITHUB_ENV" echo "validator worst exit status: $worst" + - name: Read the grid comment from the previous revision (delta column) + if: github.event_name == 'pull_request' + env: + GH_TOKEN: ${{ github.token }} + COMMENTS_URL: ${{ github.event.pull_request.comments_url }} + MARKER: '' + run: | + set -euo pipefail + # Best effort: the caller posts the grid under this marker and + # regenerates it in place, so the last comment carrying it is the + # previous revision's grid and its baseline. Any failure here only + # costs the "since previous revision" column. + : > "$RUNNER_TEMP/previous.md" + page=1 + while [ "$page" -le 10 ]; do + if ! curl -sSf --retry 2 -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" \ + "$COMMENTS_URL?per_page=100&page=$page" > "$RUNNER_TEMP/comments.json"; then + echo "::notice::could not read the pull request comments; the delta column reports no comparable baseline" + break + fi + jq -r --arg m "$MARKER" '[.[] | select((.body // "") | contains($m))] | last | .body // empty' \ + "$RUNNER_TEMP/comments.json" > "$RUNNER_TEMP/previous-page.md" + [ -s "$RUNNER_TEMP/previous-page.md" ] && mv "$RUNNER_TEMP/previous-page.md" "$RUNNER_TEMP/previous.md" + [ "$(jq length "$RUNNER_TEMP/comments.json")" -eq 100 ] || break + page=$((page + 1)) + done + [ -s "$RUNNER_TEMP/previous.md" ] && echo "previous grid found" || echo "no previous grid" + - name: Render the verdict grid id: grid env: @@ -219,6 +300,8 @@ jobs: --sha "$HEAD_SHA" --run "$GITHUB_RUN_ID" --url "$RUN_URL" --pr "$PR_NUMBER" --repo "$GITHUB_REPOSITORY" --canon-ref "$CANON_REF") [ -f "$ledger" ] && args+=(--ledger "$ledger") + [ -s "$RUNNER_TEMP/ledger-growth" ] && args+=(--ledger-growth "$RUNNER_TEMP/ledger-growth") + [ -s "$RUNNER_TEMP/previous.md" ] && args+=(--previous "$RUNNER_TEMP/previous.md") [ -n "${BASE_UNRESOLVED:-}" ] && args+=("$BASE_UNRESOLVED") bash "$CANON/scripts/k9-grid.sh" render "${args[@]}" cp "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/typecheck.log" "$RUNNER_TEMP/results.jsonl" "$RUNNER_TEMP/k9-grid/" diff --git a/.machine_readable/k9-contract-debt.txt b/.machine_readable/k9-contract-debt.txt index 392f4258..23077f54 100644 --- a/.machine_readable/k9-contract-debt.txt +++ b/.machine_readable/k9-contract-debt.txt @@ -22,15 +22,8 @@ # added; 3 templates renamed off the reserved suffix as *.k9.ncl.in — a template # is not a component and is never loaded; setup-repo.k9.ncl granted, described # and given a signature block. MIGRATION-1058 M3 / standards#1058 #C) -3-practice/session-management-standards/continuity/checkpoint-before-major-change/PROTOCOL.k9 -3-practice/session-management-standards/continuity/emergency-termination/PROTOCOL.k9 -3-practice/session-management-standards/continuity/planned-session-close/PROTOCOL.k9 -3-practice/session-management-standards/continuity/recovery-operation/PROTOCOL.k9 -3-practice/session-management-standards/continuity/repo-intake/PROTOCOL.k9 -3-practice/session-management-standards/handover/collaborative-transfer/PROTOCOL.k9 -3-practice/session-management-standards/handover/full-transfer/PROTOCOL.k9 -3-practice/session-management-standards/handover/human-transfer/PROTOCOL.k9 -3-practice/session-management-standards/handover/model-transfer/PROTOCOL.k9 -3-practice/session-management-standards/verify/maintenance-sweep/PROTOCOL.k9 -3-practice/session-management-standards/verify/release-audit/PROTOCOL.k9 -3-practice/session-management-standards/verify/substantial-completion/PROTOCOL.k9 +# 2026-10-19: the remaining 12 session-management PROTOCOL.k9 paths are not +# tracked files (never were, under that path); an entry that names no tracked +# K9 file exempts nothing and the k9-conformance grid reports it as a stale +# exemption, so they are removed. Count: 0. The ledger stays, empty, as the +# place a future exemption goes (with its Ratchet-exception line). diff --git a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc index 136d4246..659cf5bc 100644 --- a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc +++ b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc @@ -66,8 +66,15 @@ The reusable lane (`.github/workflows/k9-conformance.yml`, which a consumer calls at a pinned standards SHA) renders the same five-state grid with `scripts/k9-grid.sh` and `scripts/lib/k9-grid.jq` (bash and jq, no runtime to install) and writes `report.json` and `badge.json` as I-JSON/JCS. Its rules are -those above; `scripts/tests/k9-grid-test.sh` checks them. To reproduce what the -lane posts for a consumer: +those above, plus two of its own because its evidence differs: the validator's +raw exit status is explained by a ledgered (advisory) failure as well as a +blocking one, and a ledger entry the pull request adds exempts nothing unless +a commit message or the PR body declares it in the exemption ratchet's form +(`Ratchet-exception: .machine_readable/k9-contract-debt.txt — `); a +ledger path that names no tracked K9 file is a stale exemption. The +"since previous revision" column compares only under one profile (renderer, +validator commit, ledger, lane and the corpus itself). `scripts/tests/k9-grid-test.sh` +checks them. To reproduce what the lane posts for a consumer: [source,bash] ---- @@ -78,6 +85,9 @@ while read -r f; do bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 - bash scripts/k9-grid.sh render --out grid --results results.jsonl --expected expected \ --ledger .machine_readable/k9-contract-debt.txt --selftest selftest.log --selftest-code "$(cat selftest.code)" \ --typecheck typecheck.log --typecheck-code "$(cat typecheck.code)" +# On a pull request the lane also passes --changed (the merge-base diff), +# --ledger-growth (undeclared ledger additions) and --previous (the last +# posted grid, for the delta column); see the workflow's enumerate step. ---- Exit 0 means green, yellow or explicit N/A; exit 1 means a blocking failure; diff --git a/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc index 81c827c4..f68d8671 100644 --- a/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc +++ b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc @@ -78,9 +78,12 @@ them fails to typecheck, and the grid shows both facts side by side. the consumer's `.machine_readable/k9-contract-debt.txt` and untouched by the pull request fails as _advisory_ (yellow). Touch it and it must conform (red). Fix it and leave it in the ledger and the grid is red - with "Stale debt exemption" until the line is removed. Growing the - ledger needs a `Ratchet-exception:` line in a commit or the PR body - (owner norm, 2026-10-14). + with "Stale debt exemption" until the line is removed, as is a ledger + line that names no tracked K9 file. Growing the ledger needs a + `Ratchet-exception:` line naming it in a commit or the PR body (owner + norm, 2026-10-14): the lane compares the ledger with the PR's merge base + and an undeclared addition exempts nothing (red), and the governance + exemption ratchet rejects the growth as well. . *Artifacts are I-JSON serialised as JCS.* `report.json` and `badge.json` go through `ijson-jcs canon` (D306) when it is installed, else jq's sorted compact form, which is byte-identical for these @@ -93,7 +96,7 @@ them fails to typecheck, and the grid shows both facts side by side. |=== |Lane |Reusable |What the caller does |DEED |`.github/workflows/deed-conformance.yml@` with `require-files: true` |Nothing more: the job fails on a lint error. (A grid for this lane is a follow-up; the linter's own output is the evidence today.) -|K9 |`.github/workflows/k9-conformance.yml@` with `require-files: true` |Downloads the `k9-grid` artifact in a second job and posts `report.md` to the pull request (marker ``), and publishes `badge.json` where its README badge reads it. The reusable asks for `contents: read` only, so it cannot post. +|K9 |`.github/workflows/k9-conformance.yml@` with `require-files: true`, granting `contents: read` and `pull-requests: read` |Downloads the `k9-grid` artifact in a second job and posts `report.md` to the pull request (marker ``), and publishes `badge.json` where its README badge reads it. The reusable reads the previous grid comment for its delta column but asks for no write permission, so it cannot post. |Hypatia |`.github/workflows/hypatia-scan-reusable.yml@` with the explicit `HYPATIA_SCAN_PAT` secret |Same posting job, artifact `verdict-grid`, marker ``. |=== diff --git a/docs/VERDICT-GRID-SPEC.adoc b/docs/VERDICT-GRID-SPEC.adoc index c601a727..d94a0757 100644 --- a/docs/VERDICT-GRID-SPEC.adoc +++ b/docs/VERDICT-GRID-SPEC.adoc @@ -194,7 +194,7 @@ certification; the scope footer says so in every rendering. |=== |Lane |Renderer |Workflow |DEED–K9 (conformance) |`1-formats/k9/tools/conformance-report.mjs` (tests: `conformance-report.test.mjs`) |`k9-contractile.yml`, which also posts the comment. -|K9 (conformance, reusable) |`scripts/k9-grid.sh` + `scripts/lib/k9-grid.jq` (tests: `scripts/tests/k9-grid-test.sh`) |`k9-conformance.yml` renders the grid into the job summary and the `k9-grid` artifact (`report.json` and `badge.json` as I-JSON/JCS, `report.md`, validator logs) and exposes the `verdict` output. Like the Hypatia lane it declares `contents: read` only, so the caller posts the comment. Where it sits beside the DEED lane: `docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc`. +|K9 (conformance, reusable) |`scripts/k9-grid.sh` + `scripts/lib/k9-grid.jq` (tests: `scripts/tests/k9-grid-test.sh`) |`k9-conformance.yml` renders the grid into the job summary and the `k9-grid` artifact (`report.json` and `badge.json` as I-JSON/JCS, `report.md`, validator logs) and exposes the `verdict` output. It declares `contents: read` and `pull-requests: read` (to read the grid it posted last time, for the delta column), never write, so the caller posts the comment. Where it sits beside the DEED lane: `docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc`. |Hypatia (findings) |`scripts/verdict-grid.sh` + `scripts/lib/verdict-grid.jq` (tests: `scripts/tests/verdict-grid-test.sh`) |`hypatia-scan-reusable.yml` renders the grid into the job summary and the `verdict-grid` artifact and exposes the `verdict` output. It does not post the comment: it declares only the permissions every existing caller already grants, and a reusable may not request more than its caller. The caller posts (see below). |=== diff --git a/scripts/check-exemption-ratchet.sh b/scripts/check-exemption-ratchet.sh index cf05244d..4c218016 100755 --- a/scripts/check-exemption-ratchet.sh +++ b/scripts/check-exemption-ratchet.sh @@ -149,6 +149,9 @@ LEDGERS=( ".machine_readable/root-allow.txt" ".machine_readable/lock-allow.txt" ".machine_readable/pipeline-allow.txt" + # The K9 debt ledger (1-formats/k9/tools/CONFORMANCE-REPORTING.adoc): the + # k9-conformance lane also refuses an undeclared addition on its own. + ".machine_readable/k9-contract-debt.txt" ) echo "Exemption ratchet — comparing against ${BASE_REF}" diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh index d0871579..341ba17b 100755 --- a/scripts/k9-grid.sh +++ b/scripts/k9-grid.sh @@ -12,6 +12,8 @@ # --results FILE k9-validate.sh --json output, one record per line # --expected FILE the files the lane set out to check, one per line # --ledger FILE the caller's .machine_readable/k9-contract-debt.txt +# --ledger-growth FILE ledger entries the PR adds with no Ratchet-exception +# naming the ledger, one per line: they exempt nothing # --changed FILE files the PR touches, one per line # --base-unresolved the PR base/head range could not be diffed # --selftest LOG --selftest-code N validator --self-test evidence @@ -42,10 +44,11 @@ JQ_LIB="$SCRIPT_DIR/lib/k9-grid.jq" usage() { sed -n '5,34p' "$0" >&2; exit 2; } -# A path list file to a JSON array; a missing file is an empty list. +# A path list file to a JSON array; a missing file is an empty list, and so +# is a file of comments and blanks (grep's no-match exit is not an error). list_json() { if [ -n "${1:-}" ] && [ -f "$1" ]; then - grep -vE '^[[:space:]]*(#|$)' "$1" | jq -R . | jq -s . + { grep -vE '^[[:space:]]*(#|$)' "$1" || true; } | jq -R . | jq -s . else echo '[]' fi @@ -95,7 +98,7 @@ baseline_of() { printf '%s' "$b64" | base64 -d 2>/dev/null | jq -c ' if type == "object" and (.categories | type) == "array" and all(.categories[]; . as $c | ($c.id | type) == "string" and ($c.counts | type) == "object" - and all(["passed", "failed", "indeterminate", "not_applicable", "advisory"][]; . as $k + and all(["passed", "failed", "indeterminate", "not_applicable", "advisory", "warnings"][]; . as $k | ($c.counts | has($k)) and ($c.counts[$k] | type) == "number" and $c.counts[$k] >= 0)) then . else null end' 2>/dev/null || echo null } @@ -107,7 +110,7 @@ case "$cmd" in baseline_of "$1" ;; render) - out="" results="" expected="" ledger="" changed="" prev="" lane="k9" + out="" results="" expected="" ledger="" growth="" changed="" prev="" lane="k9" selftest="" selftest_code="" typecheck="" typecheck_code="" corpus_code="" base_resolved=true sha="" run="local" url="" pr="local" repo="" canon="" while [ $# -gt 0 ]; do @@ -116,6 +119,7 @@ case "$cmd" in --results) results="$2"; shift 2 ;; --expected) expected="$2"; shift 2 ;; --ledger) ledger="$2"; shift 2 ;; + --ledger-growth) growth="$2"; shift 2 ;; --changed) changed="$2"; shift 2 ;; --base-unresolved) base_resolved=false; shift ;; --selftest) selftest="$2"; shift 2 ;; @@ -142,9 +146,12 @@ case "$cmd" in [ -n "$prev" ] && previous=$(baseline_of "$prev") # The profile names what produced the grid: this renderer, the standards - # commit that supplied validator and contract, the ledger and the lane. - # Two revisions compare only under the same profile. - profile=$( { cat "$0" "$JQ_LIB"; [ -n "$ledger" ] && [ -f "$ledger" ] && cat "$ledger"; printf '%s\n' "$canon" "$lane"; } | sha256sum | cut -d' ' -f1) + # commit that supplied validator and contract, the ledger, the lane and + # the corpus the lane set out to check (as conformance-report.mjs does). + # Two revisions compare only under the same profile, so a corpus that + # gained or lost a file is never read as fewer or more failures. + profile=$( { cat "$0" "$JQ_LIB"; [ -n "$ledger" ] && [ -f "$ledger" ] && cat "$ledger"; + printf '%s\n' "$canon" "$lane"; list_json "$expected" | jq -c 'sort'; } | sha256sum | cut -d' ' -f1) dirty=false if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then @@ -156,13 +163,13 @@ case "$cmd" in [ -n "$corpus_code" ] && corpus_json="$corpus_code" jq -n --argjson results "$(results_json "$results")" --argjson expected "$(list_json "$expected")" \ - --argjson ledger "$(list_json "$ledger")" --argjson changed "$(list_json "$changed")" \ + --argjson ledger "$(list_json "$ledger")" --argjson growth "$(list_json "$growth")" --argjson changed "$(list_json "$changed")" \ --argjson base_resolved "$base_resolved" --argjson selftest "$(evidence_json "$selftest" "$selftest_code")" \ --argjson typecheck "$(evidence_json "$typecheck" "$typecheck_code")" --argjson corpus_code "$corpus_json" \ --arg profile "$profile" --arg lane "$lane" --arg sha "$sha" --arg run "$run" --arg url "$url" \ --arg pr "$pr" --arg repo "$repo" --argjson dirty "$dirty" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ --arg canon "$canon" \ - '{results: $results, expected: $expected, ledger: $ledger, changed: $changed, base_resolved: $base_resolved, + '{results: $results, expected: $expected, ledger: $ledger, ledger_growth: $growth, changed: $changed, base_resolved: $base_resolved, selftest: $selftest, typecheck: $typecheck, corpus_code: $corpus_code, profile: $profile, identity: {lane: $lane, sha: $sha, run: $run, url: $url, pr: $pr, repo: $repo, dirty: $dirty, generated: $generated, canon_ref: $canon}}' \ diff --git a/scripts/lib/k9-grid.jq b/scripts/lib/k9-grid.jq index b37abb25..ab6fdd92 100644 --- a/scripts/lib/k9-grid.jq +++ b/scripts/lib/k9-grid.jq @@ -7,9 +7,13 @@ # This is a port of the item, state and delta rules in # 1-formats/k9/tools/conformance-report.mjs (the DEED–K9 lane that runs in # this repository), so the two K9 lanes grade one result the same way. Where -# the two disagree, the .mjs is the reference rendering and this file is a bug. -# It is jq rather than JavaScript so the estate's authored-JavaScript surface -# does not grow (docs/NO-JAVASCRIPT-SOURCE-POLICY.adoc). +# the two disagree on a rule they share, the .mjs is the reference rendering +# and this file is a bug. Two rules are this lane's own, because its evidence +# differs, and are marked "(this lane)" below: the corpus exit status here is +# the raw validator's, not the ledger-aware hook's; and the caller's ledger is +# compared with the PR base, so an entry added without a Ratchet-exception +# buys no exemption. It is jq rather than JavaScript so the estate's +# authored-JavaScript surface does not grow (docs/NO-JAVASCRIPT-SOURCE-POLICY.adoc). # # Input (one object, assembled by scripts/k9-grid.sh): # { identity: {lane, sha, run, url, pr, repo, dirty, generated, canon_ref}, @@ -18,6 +22,7 @@ # results: [ one k9-validate.sh --json record per file, or # {malformed: ""} for a line that was not JSON ], # ledger: [ "path" ... ], # .machine_readable/k9-contract-debt.txt +# ledger_growth: [ "path" ... ], # ledger entries this PR adds with no Ratchet-exception # changed: [ "path" ... ], # files this PR touches # base_resolved: bool, # false when the PR range could not be diffed # selftest: {code: int|null, output: "..."}, @@ -34,7 +39,8 @@ def counts: failed: ([.[] | select(.status == "fail")] | length), indeterminate: ([.[] | select(.status == "indeterminate")] | length), not_applicable: ([.[] | select(.status == "not_applicable")] | length), - advisory: ([.[] | select(.status == "fail" and .advisory)] | length) }; + advisory: ([.[] | select(.status == "fail" and .advisory)] | length), + warnings: ([.[] | select(.status == "pass" and .advisory)] | length) }; # Precedence: red > orange > yellow > green; grey only when every item is grey. def state: @@ -70,33 +76,39 @@ def typecheck_items(tc; contract): (if tc.code == 0 then "" else (tc.output // "" | sub("\\s+$"; "")) end)) ]; # One live-corpus item per validator record (conformance-report.mjs corpusItems). -def corpus_items(results; expected; ledger; changed): +# A ledger entry exempts a file only when the entry was already in the ledger +# at the PR base, or the PR declares the growth (this lane): an undeclared +# addition is listed in `growth`, gets a failing row of its own, and the file +# it names is graded as if it were not in the ledger. +def corpus_items(results; expected; ledger; growth; changed): (ledger | map({key: ., value: true}) | from_entries) as $ledger + | (growth | map({key: ., value: true}) | from_entries) as $growth | (changed | map({key: ., value: true}) | from_entries) as $changed | (expected | map({key: ., value: true}) | from_entries) as $expected | reduce results[] as $r ({seen: {}, rows: []}; if ($r | has("malformed")) then .rows += [item("validator output"; "indeterminate"; "Malformed JSON result.")] - elif ($expected[$r.file] | not) or (.seen[$r.file] // false) + elif (($r.file | type) != "string") or ($expected[$r.file] | not) or (.seen[$r.file] // false) or (($r.verdict // "") | IN("pass", "fail", "error", "routed") | not) or (($r.findings | type) != "array") then - .rows += [item(($r.file // "validator output"); "indeterminate"; "Unexpected, duplicate or invalid result.")] + .rows += [item((if ($r.file | type) == "string" then $r.file else "validator output" end); "indeterminate"; "Unexpected, duplicate or invalid result.")] else .seen[$r.file] = true | ([$r.findings[] | select(.severity == "skipped")]) as $unavailable | ([$r.findings[] | select(.severity == "error")]) as $errors | ([$r.findings[] | select(.severity == "warning")]) as $warnings | ([$r.findings[] | "\(.rule): \(.message)"] | join("; ")) as $reason + | (($ledger[$r.file] // false) and (($growth[$r.file] // false) | not)) as $listed | .rows += [ if $r.verdict == "routed" then item($r.file; "not_applicable"; "Routed to another dialect; not validated as a K9 contract.") elif ($errors | any(.rule == "K9-N002")) and ($errors | any(.rule != "K9-N002") | not) then item($r.file; "indeterminate"; $reason) elif ($errors | length) > 0 or $r.verdict == "fail" or $r.verdict == "error" then - item($r.file; "fail"; $reason; (($ledger[$r.file] // false) and (($changed[$r.file] // false) | not))) + item($r.file; "fail"; $reason; ($listed and (($changed[$r.file] // false) | not))) elif ($unavailable | length) > 0 then item($r.file; "indeterminate"; $reason) - elif ($ledger[$r.file] // false) then + elif $listed then item($r.file; "fail"; "Stale debt exemption: this file now conforms.") else item($r.file; "pass"; (if ($warnings | length) > 0 then $reason else "" end); (($warnings | length) > 0)) @@ -105,6 +117,11 @@ def corpus_items(results; expected; ledger; changed): | . as $acc | $acc.rows + [ expected[] | select(($acc.seen[.] // false) | not) | item(.; "indeterminate"; "Validator did not return a result for this file.") ] + # A ledger entry that names no tracked K9 file exempts nothing and is debt + # the ledger must shed (this lane): the shrink-only rule has no other way + # to see it. + + [ ledger[] | select(($expected[.] // false) | not) | item(.; "fail"; "Stale debt exemption: this path is not a tracked K9 file.") ] + + [ growth[] | item("ledger + \(.)"; "fail"; "Added to the debt ledger in this PR with no Ratchet-exception naming the ledger; the entry exempts nothing.") ] + (if (expected | length) == 0 then [item("live corpus"; "not_applicable"; "No tracked K9 contracts or routing files.")] else [] end); def titles: { typecheck: "K9 contract typecheck", selftest: "K9 validator self-test", corpus: "K9 live corpus (L0–L2)" }; @@ -114,10 +131,15 @@ def category(id; items): { id: id, title: titles[id], items: items, counts: (ite def build: . as $in | ("1-formats/k9/spec/contract/k9_contract.ncl") as $contract - | (corpus_items($in.results // []; $in.expected // []; $in.ledger // []; $in.changed // []) + | (corpus_items($in.results // []; $in.expected // []; $in.ledger // []; $in.ledger_growth // []; $in.changed // []) + (if $in.base_resolved == false then [item("change scope"; "indeterminate"; "Cannot resolve the PR base/head range; debt exemptions cannot be determined.")] else [] end)) as $corpus0 + # The exit status is the raw validator's worst over the corpus, so a + # ledgered (advisory) failure explains it as well as a blocking one does + # (this lane; the .mjs reads the ledger-aware hook, where only a blocking + # row can). Only an exit that no fail or indeterminate row accounts for is + # missing evidence. | ($corpus0 - + (if ($in.corpus_code // 0) != 0 and ($corpus0 | any(.[]; .status == "indeterminate" or (.status == "fail" and (.advisory | not))) | not) + + (if ($in.corpus_code // 0) != 0 and ($corpus0 | any(.[]; .status == "indeterminate" or .status == "fail") | not) then [item("corpus gate"; "indeterminate"; "Validator exited \($in.corpus_code); inspect corpus.log.")] else [] end)) as $corpus | [ category("typecheck"; typecheck_items($in.typecheck // {code: null, output: ""}; $contract)), category("selftest"; selftest_items($in.selftest // {code: null, output: ""})), @@ -142,6 +164,8 @@ def delta(prev; report): else "↓ regressed (\($c.counts.failed - $b.counts.failed) more failures)" end) elif $c.counts.advisory < $b.counts.advisory then "↑ fewer advisories" elif $c.counts.advisory > $b.counts.advisory then "↓ more advisories" + elif ($c.counts.warnings // 0) < ($b.counts.warnings // 0) then "↑ fewer warnings" + elif ($c.counts.warnings // 0) > ($b.counts.warnings // 0) then "↓ more warnings" else "↔ unchanged" end end; def state_label: {red: "🔴 failing", orange: "🟠 indeterminate", yellow: "🟡 advisory", green: "🟢 conformant", grey: "⚪ not applicable"}[.]; @@ -162,7 +186,7 @@ def render(prev): + (if $r.dirty then " · working tree modified" else "" end) + "\n\n" + "| Category | Status | Pass | Fail | Indeterminate | N/A | Since previous revision |\n|---|---|---:|---:|---:|---:|---|\n" + ([ $r.categories[] - | "| \(.title) | \(.state | state_label) | \(.counts.passed) | \(.counts.failed)\(if .counts.advisory > 0 then " (\(.counts.advisory) advisory)" else "" end) | \(.counts.indeterminate) | \(.counts.not_applicable) | \(delta(prev; $r)) |\n" ] | add // "") + | "| \(.title) | \(.state | state_label) | \(.counts.passed)\(if .counts.warnings > 0 then " (\(.counts.warnings) with warnings)" else "" end) | \(.counts.failed)\(if .counts.advisory > 0 then " (\(.counts.advisory) advisory)" else "" end) | \(.counts.indeterminate) | \(.counts.not_applicable) | \(delta(prev; $r)) |\n" ] | add // "") + ([ $r.categories[] | ([.items[] | select(.status != "pass" or .advisory)]) as $f | select(($f | length) > 0) diff --git a/scripts/tests/k9-grid-test.sh b/scripts/tests/k9-grid-test.sh index b0aee6f8..5cf3a19a 100644 --- a/scripts/tests/k9-grid-test.sh +++ b/scripts/tests/k9-grid-test.sh @@ -5,8 +5,10 @@ # k9-grid-test.sh — the conformance-family grid for the K9 lane never turns # missing, skipped or unreadable evidence into a clean row, honours the # shrink-only debt ledger exactly as CONFORMANCE-REPORTING.adoc states it, -# keeps routed files and an empty corpus grey, compares revisions only under -# one profile, and writes I-JSON/JCS artifacts. docs/VERDICT-GRID-SPEC.adoc +# keeps routed files and an empty corpus grey, refuses an exemption the PR +# adds without a Ratchet-exception, compares revisions only under one profile +# (renderer, validator commit, ledger, lane, corpus), and writes I-JSON/JCS +# artifacts. docs/VERDICT-GRID-SPEC.adoc # and 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc are the contracts under test. # # Run: bash scripts/tests/k9-grid-test.sh @@ -123,11 +125,34 @@ render "$WORK/o7f" --results "$WORK/r-clean" --expected "$WORK/expected" "${good check "a validator exit status with no finding to explain it is orange" orange "$(state "$WORK/o7f")" check "a validator exit status explained by a red row stays red" red "$(render "$WORK/o7g" --results "$WORK/r-fail" --expected "$WORK/expected" "${good_evidence[@]}" --corpus-code 1; state "$WORK/o7g")" +# 7g. A validator exit status explained by a ledgered (advisory) failure is +# yellow, not orange: the exit is the raw validator's, which fails +# ledgered files too. +render "$WORK/o7h" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-b" "${good_evidence[@]}" --corpus-code 1 +check "a validator exit status explained by an advisory row stays yellow" yellow "$(state "$WORK/o7h")" +# 7i. A result without a file key is invalid evidence, never an abort. +{ rec a.k9.ncl pass '[]'; jq -nc '{verdict: "pass", findings: []}'; } > "$WORK/r-nofile" +render "$WORK/o7i" --results "$WORK/r-nofile" --expected "$WORK/expected" "${good_evidence[@]}" +check "a result with no file key is orange" orange "$(state "$WORK/o7i")" +check "a result with no file key is reported as validator output" "Unexpected, duplicate or invalid result." "$(reason "$WORK/o7i" "validator output")" + +# 7j. Ledger growth without a Ratchet-exception buys no exemption: red. +render "$WORK/o7j" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-b" --ledger-growth "$WORK/ledger-b" "${good_evidence[@]}" +check "an undeclared ledger addition is red" red "$(state "$WORK/o7j")" +check "the undeclared addition is a finding of its own" yes "$(reason "$WORK/o7j" "ledger + b.k9.ncl" | grep -q '^Added to the debt ledger in this PR with no Ratchet-exception' && echo yes || echo no)" +check "the file the addition names is graded as unledgered" false "$(jq -r '[.categories[].items[] | select(.id == "b.k9.ncl")][0].advisory' "$WORK/o7j/report.json")" +# 7k. A ledger path that is not a tracked K9 file is a stale exemption: red. +printf '%s\n' b.k9.ncl gone.k9.ncl > "$WORK/ledger-gone" +render "$WORK/o7k" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-gone" "${good_evidence[@]}" +check "a ledger path outside the corpus is red" red "$(state "$WORK/o7k")" +check "the untracked ledger path is named" "Stale debt exemption: this path is not a tracked K9 file." "$(reason "$WORK/o7k" gone.k9.ncl)" + # 8. Warnings are advisory: yellow, counted but not failed. { rec a.k9.ncl pass "$W_ONE"; rec b.k9.ncl pass '[]'; } > "$WORK/r-warn" render "$WORK/o8" --results "$WORK/r-warn" --expected "$WORK/expected" "${good_evidence[@]}" check "a warning is yellow" yellow "$(state "$WORK/o8")" check "a warning does not count as failed" 0 "$(jq -r '.counts.failed' "$WORK/o8/report.json")" +check "a warning is counted" 1 "$(jq -r '.counts.warnings' "$WORK/o8/report.json")" # 9. Routed files and an empty corpus are explicit scope exclusions: grey rows. { rec a.k9.ncl routed '[]'; } > "$WORK/r-routed" @@ -162,6 +187,13 @@ render "$WORK/o11d" --results "$WORK/r-clean" --expected "$WORK/expected" "${goo check "another PR is not compared" yes "$(grep -q 'no comparable baseline' "$WORK/o11d/report.md" && echo yes || echo no)" render "$WORK/o11e" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --lane other --previous "$WORK/o11a/report.md" check "another profile is not compared" yes "$(grep -q 'no comparable baseline' "$WORK/o11e/report.md" && echo yes || echo no)" +render "$WORK/o11f" --results "$WORK/r-warn" --expected "$WORK/expected" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 +render "$WORK/o11g" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11f/report.md" +check "delta reports a cleared warning" yes "$(grep -q 'K9 live corpus.*| ↑ fewer warnings' "$WORK/o11g/report.md" && echo yes || echo no)" +printf '%s\n' a.k9.ncl b.k9.ncl c.k9.ncl > "$WORK/expected-abc" +{ rec a.k9.ncl pass '[]'; rec b.k9.ncl pass '[]'; rec c.k9.ncl pass '[]'; } > "$WORK/r-clean-abc" +render "$WORK/o11h" --results "$WORK/r-clean-abc" --expected "$WORK/expected-abc" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11a/report.md" +check "a corpus that gained a file is not compared" yes "$(grep -q 'K9 live corpus.*| — no comparable baseline' "$WORK/o11h/report.md" && echo yes || echo no)" check "baseline of garbage is null" null "$(printf 'no marker here\n' > "$WORK/garbage"; bash "$GRID" baseline "$WORK/garbage")" check "baseline of a corrupt marker is null" null "$(printf '\n' > "$WORK/corrupt"; bash "$GRID" baseline "$WORK/corrupt")" check "baseline of a real report round-trips" 7 "$(bash "$GRID" baseline "$WORK/o11a/report.md" | jq -r .pr)" From 12cbeb0b67aeec248cd7651db6facb8ffe003ac9 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 22:12:45 +0000 Subject: [PATCH 5/7] fix(k9-conformance): carry require-files into the grid, authenticate the baseline, and order the delta by blocking failures Codex review on 18acd112: - require-files with an empty corpus is now a failing "live corpus" row rendered by the grid (--require-files), so report.md and badge.json carry the red the job reports instead of a green artifact under a failed job; the post-render output override is gone. - The base commit is used from the full checkout when present; the credential-less fetch is a fallback only, so private callers no longer land on the unresolved-range path. - Only the posting account's comment (input grid-author, default github-actions[bot]) is read as the previous revision's baseline, so a contributor cannot plant a marker with a forged baseline. - The reproduction recipe applies the lane's exclusions (fixtures, 9-archive). - delta compares blocking failures (failed minus advisory) before the advisory count, so a ledgered failure the PR starts touching reads as a regression and the reverse as an improvement; the .mjs ordering is noted as the one to update. scripts/tests/k9-grid-test.sh: 54 -> 58 assertions. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- .github/workflows/k9-conformance.yml | 34 +++++++++++++------ 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc | 5 ++- docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc | 2 +- scripts/k9-grid.sh | 8 +++-- scripts/lib/k9-grid.jq | 32 ++++++++++++----- scripts/tests/k9-grid-test.sh | 12 ++++++- 6 files changed, 67 insertions(+), 26 deletions(-) diff --git a/.github/workflows/k9-conformance.yml b/.github/workflows/k9-conformance.yml index 30f3fc5c..ccaa8e9b 100644 --- a/.github/workflows/k9-conformance.yml +++ b/.github/workflows/k9-conformance.yml @@ -42,6 +42,7 @@ name: k9-conformance # with: # require-files: true # fail when the caller has no K9 file at all # fail-on-indeterminate: true # orange fails the job too (default: only red fails) +# grid-author: github-actions[bot] # the account whose grid comment is the delta baseline on: push: @@ -78,6 +79,11 @@ on: type: boolean required: false default: false + grid-author: + description: "The account that posts the grid comment in the caller; only its comment is read as the previous revision's baseline, so a contributor's comment cannot pose as one." + type: string + required: false + default: github-actions[bot] outputs: verdict: description: "Grid state: red, orange, yellow, green or grey." @@ -202,8 +208,11 @@ jobs: # The PR's own changes are those since its merge base, not since the # base branch tip: a file the base branch changed after the PR - # branched is not this PR's edit. - if git fetch --quiet origin "$BASE_SHA" 2>/dev/null \ + # branched is not this PR's edit. The full checkout already holds + # the base commit; the fetch is a fallback only, and runs without + # credentials (persist-credentials is off), so it serves public + # repositories. + if { git cat-file -e "$BASE_SHA^{commit}" 2>/dev/null || git fetch --quiet origin "$BASE_SHA" 2>/dev/null; } \ && merge_base=$(git merge-base "$BASE_SHA" "$HEAD_SHA" 2>/dev/null) \ && git diff --name-only "$merge_base" "$HEAD_SHA" > "$RUNNER_TEMP/changed" 2>/dev/null; then echo "$(wc -l < "$RUNNER_TEMP/changed") file(s) changed since the merge base $merge_base" @@ -262,12 +271,15 @@ jobs: GH_TOKEN: ${{ github.token }} COMMENTS_URL: ${{ github.event.pull_request.comments_url }} MARKER: '' + GRID_AUTHOR: ${{ inputs.grid-author || 'github-actions[bot]' }} run: | set -euo pipefail - # Best effort: the caller posts the grid under this marker and - # regenerates it in place, so the last comment carrying it is the - # previous revision's grid and its baseline. Any failure here only - # costs the "since previous revision" column. + # Best effort: the caller's posting account writes the grid under + # this marker and regenerates it in place, so its last comment + # carrying the marker is the previous revision's grid and baseline. + # Only that account's comments count: anyone who can comment could + # otherwise plant a marker and a forged baseline. Any failure here + # only costs the "since previous revision" column. : > "$RUNNER_TEMP/previous.md" page=1 while [ "$page" -le 10 ]; do @@ -276,7 +288,8 @@ jobs: echo "::notice::could not read the pull request comments; the delta column reports no comparable baseline" break fi - jq -r --arg m "$MARKER" '[.[] | select((.body // "") | contains($m))] | last | .body // empty' \ + jq -r --arg m "$MARKER" --arg a "$GRID_AUTHOR" \ + '[.[] | select((.user.login // "") == $a and ((.body // "") | contains($m)))] | last | .body // empty' \ "$RUNNER_TEMP/comments.json" > "$RUNNER_TEMP/previous-page.md" [ -s "$RUNNER_TEMP/previous-page.md" ] && mv "$RUNNER_TEMP/previous-page.md" "$RUNNER_TEMP/previous.md" [ "$(jq length "$RUNNER_TEMP/comments.json")" -eq 100 ] || break @@ -303,13 +316,12 @@ jobs: [ -s "$RUNNER_TEMP/ledger-growth" ] && args+=(--ledger-growth "$RUNNER_TEMP/ledger-growth") [ -s "$RUNNER_TEMP/previous.md" ] && args+=(--previous "$RUNNER_TEMP/previous.md") [ -n "${BASE_UNRESOLVED:-}" ] && args+=("$BASE_UNRESOLVED") + # With require-files the empty corpus is a failing row in the grid + # itself, so report.md and badge.json carry the red the job reports. + [ "$REQUIRE_FILES" = "true" ] && args+=(--require-files) bash "$CANON/scripts/k9-grid.sh" render "${args[@]}" cp "$RUNNER_TEMP/selftest.log" "$RUNNER_TEMP/typecheck.log" "$RUNNER_TEMP/results.jsonl" "$RUNNER_TEMP/k9-grid/" [ -f "$RUNNER_TEMP/corpus.log" ] && cp "$RUNNER_TEMP/corpus.log" "$RUNNER_TEMP/k9-grid/" - if [ "$REQUIRE_FILES" = "true" ] && [ ! -s "$RUNNER_TEMP/expected" ]; then - echo "::error::require-files is set and no committed K9 file was found" - echo "state=red" >> "$GITHUB_OUTPUT" - fi - name: Archive the grid and its evidence if: ${{ always() && steps.grid.outcome != 'skipped' }} diff --git a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc index 659cf5bc..6006d6c9 100644 --- a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc +++ b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc @@ -80,7 +80,10 @@ checks them. To reproduce what the lane posts for a consumer: ---- bash 1-formats/k9/tools/k9-validate.sh --self-test > selftest.log; echo $? > selftest.code nickel typecheck 1-formats/k9/spec/contract/k9_contract.ncl > typecheck.log; echo $? > typecheck.code -git ls-files -- "*.k9.ncl" "*.k9" "*.k9.lock" | sort > expected +# The same exclusions as the lane: the fixture corpus is deliberately invalid +# and the archive is out of scope. +git ls-files -- "*.k9.ncl" "*.k9" "*.k9.lock" \ + | { grep -v -E "^(9-archive/|1-formats/k9/tools/fixtures/)" || [ $? -eq 1 ]; } | sort > expected while read -r f; do bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json --quiet "$f"; done < expected > results.jsonl bash scripts/k9-grid.sh render --out grid --results results.jsonl --expected expected \ --ledger .machine_readable/k9-contract-debt.txt --selftest selftest.log --selftest-code "$(cat selftest.code)" \ diff --git a/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc index f68d8671..8faf39de 100644 --- a/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc +++ b/docs/CONTRACTILE-DESCRIPTILE-VALIDATION.adoc @@ -96,7 +96,7 @@ them fails to typecheck, and the grid shows both facts side by side. |=== |Lane |Reusable |What the caller does |DEED |`.github/workflows/deed-conformance.yml@` with `require-files: true` |Nothing more: the job fails on a lint error. (A grid for this lane is a follow-up; the linter's own output is the evidence today.) -|K9 |`.github/workflows/k9-conformance.yml@` with `require-files: true`, granting `contents: read` and `pull-requests: read` |Downloads the `k9-grid` artifact in a second job and posts `report.md` to the pull request (marker ``), and publishes `badge.json` where its README badge reads it. The reusable reads the previous grid comment for its delta column but asks for no write permission, so it cannot post. +|K9 |`.github/workflows/k9-conformance.yml@` with `require-files: true`, granting `contents: read` and `pull-requests: read`; `grid-author` names the posting account when it is not `github-actions[bot]` |Downloads the `k9-grid` artifact in a second job and posts `report.md` to the pull request (marker ``), and publishes `badge.json` where its README badge reads it. The reusable reads the previous grid comment for its delta column but asks for no write permission, so it cannot post. |Hypatia |`.github/workflows/hypatia-scan-reusable.yml@` with the explicit `HYPATIA_SCAN_PAT` secret |Same posting job, artifact `verdict-grid`, marker ``. |=== diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh index 341ba17b..5154edfc 100755 --- a/scripts/k9-grid.sh +++ b/scripts/k9-grid.sh @@ -16,6 +16,7 @@ # naming the ledger, one per line: they exempt nothing # --changed FILE files the PR touches, one per line # --base-unresolved the PR base/head range could not be diffed +# --require-files an empty corpus is a failing row, not a grey one # --selftest LOG --selftest-code N validator --self-test evidence # --typecheck LOG --typecheck-code N nickel typecheck of the contract # --corpus-code N the validator's exit status over the corpus @@ -112,7 +113,7 @@ case "$cmd" in render) out="" results="" expected="" ledger="" growth="" changed="" prev="" lane="k9" selftest="" selftest_code="" typecheck="" typecheck_code="" corpus_code="" - base_resolved=true sha="" run="local" url="" pr="local" repo="" canon="" + base_resolved=true require_files=false sha="" run="local" url="" pr="local" repo="" canon="" while [ $# -gt 0 ]; do case "$1" in --out) out="$2"; shift 2 ;; @@ -122,6 +123,7 @@ case "$cmd" in --ledger-growth) growth="$2"; shift 2 ;; --changed) changed="$2"; shift 2 ;; --base-unresolved) base_resolved=false; shift ;; + --require-files) require_files=true; shift ;; --selftest) selftest="$2"; shift 2 ;; --selftest-code) selftest_code="$2"; shift 2 ;; --typecheck) typecheck="$2"; shift 2 ;; @@ -164,12 +166,12 @@ case "$cmd" in jq -n --argjson results "$(results_json "$results")" --argjson expected "$(list_json "$expected")" \ --argjson ledger "$(list_json "$ledger")" --argjson growth "$(list_json "$growth")" --argjson changed "$(list_json "$changed")" \ - --argjson base_resolved "$base_resolved" --argjson selftest "$(evidence_json "$selftest" "$selftest_code")" \ + --argjson base_resolved "$base_resolved" --argjson require_files "$require_files" --argjson selftest "$(evidence_json "$selftest" "$selftest_code")" \ --argjson typecheck "$(evidence_json "$typecheck" "$typecheck_code")" --argjson corpus_code "$corpus_json" \ --arg profile "$profile" --arg lane "$lane" --arg sha "$sha" --arg run "$run" --arg url "$url" \ --arg pr "$pr" --arg repo "$repo" --argjson dirty "$dirty" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ --arg canon "$canon" \ - '{results: $results, expected: $expected, ledger: $ledger, ledger_growth: $growth, changed: $changed, base_resolved: $base_resolved, + '{results: $results, expected: $expected, ledger: $ledger, ledger_growth: $growth, changed: $changed, base_resolved: $base_resolved, require_files: $require_files, selftest: $selftest, typecheck: $typecheck, corpus_code: $corpus_code, profile: $profile, identity: {lane: $lane, sha: $sha, run: $run, url: $url, pr: $pr, repo: $repo, dirty: $dirty, generated: $generated, canon_ref: $canon}}' \ diff --git a/scripts/lib/k9-grid.jq b/scripts/lib/k9-grid.jq index ab6fdd92..e96a85b3 100644 --- a/scripts/lib/k9-grid.jq +++ b/scripts/lib/k9-grid.jq @@ -8,12 +8,16 @@ # 1-formats/k9/tools/conformance-report.mjs (the DEED–K9 lane that runs in # this repository), so the two K9 lanes grade one result the same way. Where # the two disagree on a rule they share, the .mjs is the reference rendering -# and this file is a bug. Two rules are this lane's own, because its evidence -# differs, and are marked "(this lane)" below: the corpus exit status here is -# the raw validator's, not the ledger-aware hook's; and the caller's ledger is -# compared with the PR base, so an entry added without a Ratchet-exception -# buys no exemption. It is jq rather than JavaScript so the estate's -# authored-JavaScript surface does not grow (docs/NO-JAVASCRIPT-SOURCE-POLICY.adoc). +# and this file is a bug, with one exception: `delta` here compares blocking +# failures before advisory counts, so a ledgered failure that the PR starts +# touching (yellow to red) reads as a regression; the .mjs ordering reads it +# as "fewer advisories" and should adopt this. Two further rules are this +# lane's own, because its evidence differs, and are marked "(this lane)" +# below: the corpus exit status here is the raw validator's, not the +# ledger-aware hook's; and the caller's ledger is compared with the PR base, +# so an entry added without a Ratchet-exception buys no exemption. It is jq +# rather than JavaScript so the estate's authored-JavaScript surface does not +# grow (docs/NO-JAVASCRIPT-SOURCE-POLICY.adoc). # # Input (one object, assembled by scripts/k9-grid.sh): # { identity: {lane, sha, run, url, pr, repo, dirty, generated, canon_ref}, @@ -25,6 +29,7 @@ # ledger_growth: [ "path" ... ], # ledger entries this PR adds with no Ratchet-exception # changed: [ "path" ... ], # files this PR touches # base_resolved: bool, # false when the PR range could not be diffed +# require_files: bool, # an empty corpus is a failure, not a scope exclusion # selftest: {code: int|null, output: "..."}, # typecheck: {code: int|null, output: "..."}, # corpus_code: int|null } # the validator's own exit status @@ -80,7 +85,7 @@ def typecheck_items(tc; contract): # at the PR base, or the PR declares the growth (this lane): an undeclared # addition is listed in `growth`, gets a failing row of its own, and the file # it names is graded as if it were not in the ledger. -def corpus_items(results; expected; ledger; growth; changed): +def corpus_items(results; expected; ledger; growth; changed; require_files): (ledger | map({key: ., value: true}) | from_entries) as $ledger | (growth | map({key: ., value: true}) | from_entries) as $growth | (changed | map({key: ., value: true}) | from_entries) as $changed @@ -122,7 +127,10 @@ def corpus_items(results; expected; ledger; growth; changed): # to see it. + [ ledger[] | select(($expected[.] // false) | not) | item(.; "fail"; "Stale debt exemption: this path is not a tracked K9 file.") ] + [ growth[] | item("ledger + \(.)"; "fail"; "Added to the debt ledger in this PR with no Ratchet-exception naming the ledger; the entry exempts nothing.") ] - + (if (expected | length) == 0 then [item("live corpus"; "not_applicable"; "No tracked K9 contracts or routing files.")] else [] end); + + (if (expected | length) == 0 then + [ if require_files then item("live corpus"; "fail"; "require-files is set and the caller has no committed K9 file.") + else item("live corpus"; "not_applicable"; "No tracked K9 contracts or routing files.") end ] + else [] end); def titles: { typecheck: "K9 contract typecheck", selftest: "K9 validator self-test", corpus: "K9 live corpus (L0–L2)" }; @@ -131,7 +139,7 @@ def category(id; items): { id: id, title: titles[id], items: items, counts: (ite def build: . as $in | ("1-formats/k9/spec/contract/k9_contract.ncl") as $contract - | (corpus_items($in.results // []; $in.expected // []; $in.ledger // []; $in.ledger_growth // []; $in.changed // []) + | (corpus_items($in.results // []; $in.expected // []; $in.ledger // []; $in.ledger_growth // []; $in.changed // []; $in.require_files // false) + (if $in.base_resolved == false then [item("change scope"; "indeterminate"; "Cannot resolve the PR base/head range; debt exemptions cannot be determined.")] else [] end)) as $corpus0 # The exit status is the raw validator's worst over the corpus, so a # ledgered (advisory) failure explains it as well as a blocking one does @@ -159,6 +167,12 @@ def delta(prev; report): else (prev.categories | map(select(.id == $c.id)) | first) as $b | if $b == null or ($b.counts.indeterminate > 0) or ($c.counts.indeterminate > 0) then "— indeterminate" elif $b.counts.not_applicable != $c.counts.not_applicable then "— scope changed" + # Blocking failures first: a ledgered failure the PR now touches moves + # from advisory to blocking with the failed count unchanged. + elif ($b.counts.failed - $b.counts.advisory) != ($c.counts.failed - $c.counts.advisory) then + (($b.counts.failed - $b.counts.advisory) as $bb | ($c.counts.failed - $c.counts.advisory) as $cb + | if $cb < $bb then "↑ improved (\($bb - $cb) fewer blocking failures)" + else "↓ regressed (\($cb - $bb) more blocking failures)" end) elif $b.counts.failed != $c.counts.failed then (if $c.counts.failed < $b.counts.failed then "↑ improved (\($b.counts.failed - $c.counts.failed) fewer failures)" else "↓ regressed (\($c.counts.failed - $b.counts.failed) more failures)" end) diff --git a/scripts/tests/k9-grid-test.sh b/scripts/tests/k9-grid-test.sh index 5cf3a19a..b1c9ce16 100644 --- a/scripts/tests/k9-grid-test.sh +++ b/scripts/tests/k9-grid-test.sh @@ -161,6 +161,9 @@ check "a routed-only corpus is grey" grey "$(cat_state "$WORK/o9" corpus)" check "grey corpus with green checks is green overall" green "$(state "$WORK/o9")" render "$WORK/o9b" --results "$WORK/expected-none" --expected "$WORK/expected-none" "${good_evidence[@]}" check "an empty corpus is grey" grey "$(cat_state "$WORK/o9b" corpus)" +render "$WORK/o9c" --results "$WORK/expected-none" --expected "$WORK/expected-none" "${good_evidence[@]}" --require-files +check "an empty corpus under --require-files is red in the grid itself" red "$(state "$WORK/o9c")" +check "and the badge says so" failing "$(jq -r .message "$WORK/o9c/badge.json")" # 10. Self-test and typecheck evidence. render "$WORK/o10" --results "$WORK/r-clean" --expected "$WORK/expected" --selftest "$bad_selftest" --selftest-code 1 --typecheck "$WORK/empty.log" --typecheck-code 0 --corpus-code 0 @@ -180,7 +183,7 @@ check "a self-test that did not run is orange" orange "$(state "$WORK/o10f")" # 11. Revision comparison: only under one profile, same PR, different commit. render "$WORK/o11a" --results "$WORK/r-fail" --expected "$WORK/expected" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 render "$WORK/o11b" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11a/report.md" -check "delta reports the fixed row" yes "$(grep -q 'K9 live corpus.*| ↑ improved (1 fewer failures)' "$WORK/o11b/report.md" && echo yes || echo no)" +check "delta reports the fixed row" yes "$(grep -q 'K9 live corpus.*| ↑ improved (1 fewer blocking failures)' "$WORK/o11b/report.md" && echo yes || echo no)" render "$WORK/o11c" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 --previous "$WORK/o11a/report.md" check "same commit is not compared" yes "$(grep -q 'no comparable baseline' "$WORK/o11c/report.md" && echo yes || echo no)" render "$WORK/o11d" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 8 --previous "$WORK/o11a/report.md" @@ -190,6 +193,13 @@ check "another profile is not compared" yes "$(grep -q 'no comparable baseline' render "$WORK/o11f" --results "$WORK/r-warn" --expected "$WORK/expected" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 render "$WORK/o11g" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11f/report.md" check "delta reports a cleared warning" yes "$(grep -q 'K9 live corpus.*| ↑ fewer warnings' "$WORK/o11g/report.md" && echo yes || echo no)" +# A ledgered failure the PR starts touching goes from advisory to blocking with +# the failed count unchanged: that is a regression, not "fewer advisories". +render "$WORK/o11i" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-b" "${good_evidence[@]}" --sha 1111111111111111111111111111111111111111 --pr 7 +render "$WORK/o11j" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-b" --changed "$WORK/changed-b" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11i/report.md" +check "advisory becoming blocking is a regression" yes "$(grep -q 'K9 live corpus.*| ↓ regressed (1 more blocking failures)' "$WORK/o11j/report.md" && echo yes || echo no)" +render "$WORK/o11k" --results "$WORK/r-fail" --expected "$WORK/expected" --ledger "$WORK/ledger-b" "${good_evidence[@]}" --sha 3333333333333333333333333333333333333333 --pr 7 --previous "$WORK/o11j/report.md" +check "blocking becoming advisory is an improvement" yes "$(grep -q 'K9 live corpus.*| ↑ improved (1 fewer blocking failures)' "$WORK/o11k/report.md" && echo yes || echo no)" printf '%s\n' a.k9.ncl b.k9.ncl c.k9.ncl > "$WORK/expected-abc" { rec a.k9.ncl pass '[]'; rec b.k9.ncl pass '[]'; rec c.k9.ncl pass '[]'; } > "$WORK/r-clean-abc" render "$WORK/o11h" --results "$WORK/r-clean-abc" --expected "$WORK/expected-abc" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/o11a/report.md" From bb8c4bf0eeb908d55a7691980ffaf52ca717b610 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 19 Oct 2026 22:17:00 +0000 Subject: [PATCH 6/7] fix(k9-grid): one JSON value from any baseline marker; complete the reproduction recipe CodeRabbit review on 18acd112: - baseline_of slurps the decoded marker and accepts only a single object, and a decode or validation failure is exactly one null. A marker that decoded to two objects, or to an object with trailing junk, used to emit two values, which --argjson rejects, so a planted comment could stop the grid rendering instead of only costing the delta column. Three regression tests. - The reproduction recipe tracks the validator's worst exit status and passes it as --corpus-code, as the lane does. - The WO-9 acceptance commands use a quoted $STANDARDS checkout variable instead of a placeholder the shell would read as redirections. The base-commit check and the recipe exclusions it also raised were fixed in 12cbeb0b. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_019a95kk88TZ1iegsBK3kox3 --- 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc | 10 ++++++++-- .../2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc | 7 ++++--- scripts/k9-grid.sh | 13 ++++++++++--- scripts/tests/k9-grid-test.sh | 10 ++++++++++ 4 files changed, 32 insertions(+), 8 deletions(-) diff --git a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc index 6006d6c9..8bd7593a 100644 --- a/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc +++ b/1-formats/k9/tools/CONFORMANCE-REPORTING.adoc @@ -84,10 +84,16 @@ nickel typecheck 1-formats/k9/spec/contract/k9_contract.ncl > typecheck.log; ech # and the archive is out of scope. git ls-files -- "*.k9.ncl" "*.k9" "*.k9.lock" \ | { grep -v -E "^(9-archive/|1-formats/k9/tools/fixtures/)" || [ $? -eq 1 ]; } | sort > expected -while read -r f; do bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json --quiet "$f"; done < expected > results.jsonl +# The lane also records the validator's worst exit status over the corpus. +corpus_code=0 +while read -r f; do + rc=0 + bash 1-formats/k9/tools/k9-validate.sh --strict --layer L2 --json --quiet "$f" || rc=$? + [ "$rc" -le "$corpus_code" ] || corpus_code=$rc +done < expected > results.jsonl bash scripts/k9-grid.sh render --out grid --results results.jsonl --expected expected \ --ledger .machine_readable/k9-contract-debt.txt --selftest selftest.log --selftest-code "$(cat selftest.code)" \ - --typecheck typecheck.log --typecheck-code "$(cat typecheck.code)" + --typecheck typecheck.log --typecheck-code "$(cat typecheck.code)" --corpus-code "$corpus_code" # On a pull request the lane also passes --changed (the merge-base diff), # --ledger-growth (undeclared ledger additions) and --previous (the last # posted grid, for the delta column); see the workflow's enumerate step. diff --git a/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc b/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc index cb9ff8ee..cb94b1e5 100644 --- a/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc +++ b/docs/reorg/2026-10-14-tidy/WORK-ORDERS-2026-10-19.adoc @@ -774,10 +774,11 @@ anyone evaluates. *Acceptance.* ---- -for f in .machine_readable/contractiles/*/*.k9.ncl; do bash /1-formats/k9/tools/k9-validate.sh --strict --layer L2 "$f"; done -> every file: exit 0 -bun /1-formats/deed/tools/deed_lint.js rsr-template-repo_chora.deed -> exit 0 +STANDARDS=/path/to/a/standards/checkout +for f in .machine_readable/contractiles/*/*.k9.ncl; do bash "$STANDARDS/1-formats/k9/tools/k9-validate.sh" --strict --layer L2 "$f"; done -> every file: exit 0 +bun "$STANDARDS/1-formats/deed/tools/deed_lint.js" rsr-template-repo_chora.deed -> exit 0 grep -rn 'k9-ecosystem\|k9iser' .github/ Justfile justfile 2>/dev/null -> no hits -bash /scripts/check-lock-sync.sh .github/workflows -> in sync +bash "$STANDARDS/scripts/check-lock-sync.sh" .github/workflows -> in sync On the PR: deed-conformance green, k9-conformance green with a 🟢 K9 grid comment, hypatia grid comment present. ---- diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh index 5154edfc..6906a694 100755 --- a/scripts/k9-grid.sh +++ b/scripts/k9-grid.sh @@ -96,12 +96,19 @@ baseline_of() { b64=$( { grep -o '' "$file" || true; } | tail -n1 \ | sed 's/^$//') if [ -z "$b64" ] || [ "${#b64}" -gt 32000 ]; then echo null; return; fi - printf '%s' "$b64" | base64 -d 2>/dev/null | jq -c ' - if type == "object" and (.categories | type) == "array" + # Exactly one JSON value comes out, whatever the marker holds: the decoded + # text is slurped and accepted only as a single object, and any decode or + # validation failure is one null (not a value followed by a null). The + # caller feeds this to --argjson, which rejects two values. + local out + out=$(printf '%s' "$b64" | base64 -d 2>/dev/null | jq -c -s ' + (if length == 1 then .[0] else null end) + | if type == "object" and (.categories | type) == "array" and all(.categories[]; . as $c | ($c.id | type) == "string" and ($c.counts | type) == "object" and all(["passed", "failed", "indeterminate", "not_applicable", "advisory", "warnings"][]; . as $k | ($c.counts | has($k)) and ($c.counts[$k] | type) == "number" and $c.counts[$k] >= 0)) - then . else null end' 2>/dev/null || echo null + then . else null end' 2>/dev/null) || out=null + printf '%s\n' "${out:-null}" } cmd="${1:-}"; shift || true diff --git a/scripts/tests/k9-grid-test.sh b/scripts/tests/k9-grid-test.sh index b1c9ce16..393112ef 100644 --- a/scripts/tests/k9-grid-test.sh +++ b/scripts/tests/k9-grid-test.sh @@ -207,6 +207,16 @@ check "a corpus that gained a file is not compared" yes "$(grep -q 'K9 live corp check "baseline of garbage is null" null "$(printf 'no marker here\n' > "$WORK/garbage"; bash "$GRID" baseline "$WORK/garbage")" check "baseline of a corrupt marker is null" null "$(printf '\n' > "$WORK/corrupt"; bash "$GRID" baseline "$WORK/corrupt")" check "baseline of a real report round-trips" 7 "$(bash "$GRID" baseline "$WORK/o11a/report.md" | jq -r .pr)" +# A marker is whatever the last comment held: two objects, or an object with +# trailing junk, must still yield exactly one value (null), never two. +two=$(printf '{"categories":[]}{"categories":[]}' | base64 -w0) +printf '\n' "$two" > "$WORK/two-objects" +check "baseline of two objects is one null" "null" "$(bash "$GRID" baseline "$WORK/two-objects" | tr '\n' '|' | sed 's/|$//')" +junk="$(printf '{"categories":[]}' | base64 -w0)=" +printf '\n' "$junk" > "$WORK/trailing-junk" +check "baseline with trailing junk is one value" 1 "$(bash "$GRID" baseline "$WORK/trailing-junk" | wc -l)" +render "$WORK/o11l" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/two-objects" +check "a forged two-object baseline still renders" yes "$(grep -q 'no comparable baseline' "$WORK/o11l/report.md" && echo yes || echo no)" # 12. Usage errors exit 2. check "missing --results exits 2" 2 "$(bash "$GRID" render --out "$WORK/o12" >/dev/null 2>&1; echo $?)" From 6a37a3dfbd12d14eb823b2de2b9c6ed61358450e Mon Sep 17 00:00:00 2001 From: "coderabbitai[bot]" <136622811+coderabbitai[bot]@users.noreply.github.com> Date: Mon, 19 Oct 2026 22:23:54 +0000 Subject: [PATCH 7/7] docs(k9): document verdict grid helpers and ledger parsing --- .github/workflows/k9-conformance.yml | 2 ++ scripts/k9-grid.sh | 9 +++++++ scripts/lib/k9-grid.jq | 35 +++++++++++++++++++++++++++- 3 files changed, 45 insertions(+), 1 deletion(-) diff --git a/.github/workflows/k9-conformance.yml b/.github/workflows/k9-conformance.yml index ccaa8e9b..5cad4041 100644 --- a/.github/workflows/k9-conformance.yml +++ b/.github/workflows/k9-conformance.yml @@ -228,6 +228,8 @@ jobs: # ratchet's form, naming this ledger: # Ratchet-exception: .machine_readable/k9-contract-debt.txt — # Ratchet-exception(.machine_readable/k9-contract-debt.txt): + # Read ledger paths from stdin and emit sorted unique entries, + # excluding comments and blank lines; an empty ledger succeeds. entries() { { grep -vE '^[[:space:]]*(#|$)' || [ $? -eq 1 ]; } | sort -u; } head_entries=$RUNNER_TEMP/ledger-head base_entries=$RUNNER_TEMP/ledger-base diff --git a/scripts/k9-grid.sh b/scripts/k9-grid.sh index 6906a694..5bbb8c2d 100755 --- a/scripts/k9-grid.sh +++ b/scripts/k9-grid.sh @@ -43,6 +43,7 @@ set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" JQ_LIB="$SCRIPT_DIR/lib/k9-grid.jq" +# Print the invocation summary to stderr and terminate with status 2. usage() { sed -n '5,34p' "$0" >&2; exit 2; } # A path list file to a JSON array; a missing file is an empty list, and so @@ -56,6 +57,8 @@ list_json() { } # A log file plus exit status to {code, output}; a missing log is null code. +# An omitted status is also null; a status in a present log that jq cannot +# parse as a number causes an error to reach the caller. evidence_json() { local log="${1:-}" code="${2:-}" if [ -n "$log" ] && [ -f "$log" ]; then @@ -80,6 +83,8 @@ results_json() { done < "$file" | jq -s . } +# Serialise JSON from stdin to stdout with one final LF, preferring ijson-jcs +# and falling back to jq's sorted compact form. Serialiser errors propagate. canon_json() { # ijson-jcs canon emits the JCS bytes with no final newline; the policy # wants exactly one. sed adds one only when it is missing. @@ -90,6 +95,10 @@ canon_json() { fi } +# Print one baseline object from the last matching marker in the supplied +# comment file. Missing or empty files, absent markers, payloads over 32000 +# encoded characters, decoding errors and invalid category counts yield null. +# Schema, profile and revision comparability are checked later by delta. baseline_of() { local file="$1" b64 [ -s "$file" ] || { echo null; return; } diff --git a/scripts/lib/k9-grid.jq b/scripts/lib/k9-grid.jq index e96a85b3..3017d625 100644 --- a/scripts/lib/k9-grid.jq +++ b/scripts/lib/k9-grid.jq @@ -35,10 +35,14 @@ # corpus_code: int|null } # the validator's own exit status # Output: the structured report; `render` turns it into Markdown. +# Normalise a row's id and reason to text; null or false reasons become empty. +# The three-argument form creates a row without advisory status. def item(id; status; reason; advisory): { id: (id | tostring), status: status, reason: (reason // "" | tostring), advisory: advisory }; def item(id; status; reason): item(id; status; reason; false); +# Summarise item rows. Advisory failures also count as failed; warnings count +# passing advisory rows, rather than individual validator findings. def counts: { passed: ([.[] | select(.status == "pass")] | length), failed: ([.[] | select(.status == "fail")] | length), @@ -48,6 +52,7 @@ def counts: warnings: ([.[] | select(.status == "pass" and .advisory)] | length) }; # Precedence: red > orange > yellow > green; grey only when every item is grey. +# An empty item array is orange because there is no evidence to grade. def state: if length == 0 then "orange" elif any(.[]; .status == "fail" and (.advisory | not)) then "red" @@ -63,6 +68,8 @@ def assertions(output): | item(.rest; (if .k == "ok" then "pass" elif .k == "FAIL" then "fail" else "indeterminate" end); (if .k == "ok" then "" else .rest end)) ]; +# Return assertion rows, adding indeterminate evidence when none were found +# or when all passed but the process status is nonzero or missing. def checked_assertions(output; code): assertions(output) as $rows | $rows @@ -70,17 +77,27 @@ def checked_assertions(output; code): + (if code != 0 and ($rows | all(.[]; .status == "pass")) and ($rows | length) > 0 then [item("check execution"; "indeterminate"; "Process exited \(code // "without a status") despite passing assertion lines.")] else [] end); +# Grade self-test evidence, requiring the completion text as well as assertion +# rows when the recorded process status is zero. def selftest_items(st): checked_assertions(st.output; st.code) + (if st.code == 0 and ((st.output // "") | contains("self-test: all assertions passed") | not) then [item("completion"; "indeterminate"; "Missing self-test completion evidence.")] else [] end); +# Grade the named contract from its recorded typecheck status: zero passes, +# null is indeterminate and any other value fails, retaining diagnostic text. def typecheck_items(tc; contract): [ item(contract; (if tc.code == 0 then "pass" elif tc.code == null then "indeterminate" else "fail" end); (if tc.code == 0 then "" else (tc.output // "" | sub("\\s+$"; "")) end)) ]; -# One live-corpus item per validator record (conformance-report.mjs corpusItems). +# Grade validator records against the expected corpus (conformance-report.mjs +# corpusItems). Invalid, duplicate and missing results are indeterminate. +# Routed files are not applicable; K9-N002-only errors are indeterminate. +# Other errors fail; skipped findings make otherwise passing results +# indeterminate. Failures for ledgered files are advisory only when the PR +# leaves the file untouched; stale ledger entries fail. +# An empty expected corpus is not applicable unless require_files is true. # A ledger entry exempts a file only when the entry was already in the ledger # at the PR base, or the PR declares the growth (this lane): an undeclared # addition is listed in `growth`, gets a failing row of its own, and the file @@ -134,8 +151,13 @@ def corpus_items(results; expected; ledger; growth; changed; require_files): def titles: { typecheck: "K9 contract typecheck", selftest: "K9 validator self-test", corpus: "K9 live corpus (L0–L2)" }; +# Group item rows with their display title, counts and aggregate verdict. def category(id; items): { id: id, title: titles[id], items: items, counts: (items | counts), state: (items | state) }; +# Build the structured report from the assembled evidence object, preserving +# run identity and aggregating typecheck, self-test and corpus verdicts. +# An unresolved PR range or unexplained nonzero corpus exit adds +# indeterminate evidence. def build: . as $in | ("1-formats/k9/spec/contract/k9_contract.ncl") as $contract @@ -160,6 +182,11 @@ def build: categories: $categories, state: ($all | state), counts: ($all | counts) }; # One category's movement since the previous revision of the same PR. +# Input is the current category; report supplies its profile, PR and revision. +# Compare only schema 1 baselines with the same profile and PR but a different +# commit. Missing categories or indeterminate counts prevent grading movement; +# changed not-applicable counts signal a scope change. Blocking failures take +# precedence over total failures, advisories and warnings. def delta(prev; report): . as $c | if prev == null or prev.schema != 1 or prev.profile != report.profile or prev.sha == report.sha or prev.pr != report.pr @@ -185,11 +212,16 @@ def delta(prev; report): def state_label: {red: "🔴 failing", orange: "🟠 indeterminate", yellow: "🟡 advisory", green: "🟢 conformant", grey: "⚪ not applicable"}[.]; def colour: {red: "red", orange: "orange", yellow: "yellow", green: "brightgreen", grey: "lightgrey"}[.]; def word: {red: "failing", orange: "indeterminate", yellow: "advisory", green: "conformant", grey: "not_applicable"}[.]; +# Escape text for grid cells and finding summaries, flattening line breaks. def esc: tostring | gsub("&"; "&") | gsub("<"; "<") | gsub(">"; ">") | gsub("\\|"; "|") | gsub("`"; "`") | gsub("\r?\n"; " "); def grid_label: "K9 L0–L2"; def badge_url(st): "https://img.shields.io/badge/\(grid_label | @uri)-\(st | word | @uri)-\(st | colour)"; +# Render the input report as Markdown, comparing categories with prev or null. +# Include at most 20 findings or scope notes per category, truncating escaped +# ids to 240 characters and reasons to 280, and embed counts for the next +# baseline. def render(prev): . as $r | "\n" @@ -212,6 +244,7 @@ def render(prev): + "\n\n"; +# Produce a shields.io endpoint document for the input report's overall verdict. def badge: {schemaVersion: 1, label: grid_label, message: (.state | word | gsub("_"; " ")), color: (.state | colour)}; # Entry point: $mode selects build (input: the assembled object), render