fix(scorecard): reconcile action-pin findings per workflow file by hyperpolymath · Pull Request #1168 · hyperpolymath/standards

hyperpolymath · 2026-10-06T16:41:48Z

Summary

scripts/reconcile-scorecard-actions-lock.rb aborted the whole SARIF document on the first workflow that failed native action-lock verification. It raised inside the per-result filter, the CLI exited 2 without writing results.reconciled.sarif, and scorecard-reusable.yml then fell back to uploading the raw SARIF. Every lock-pinned action in every workflow came back as a code-scanning alert.

Observed: metadatastician/ZenodoDeposits.jl#15 (a deliberate positive control that desyncs quality.yml), run 37472579223, job 112299614531:

HEAD is now at 591cea1 fix(ci): reconcile native action pins and retain Hypatia warning severity
Scorecard reconciliation failed: Native action-lock verification failed for .github/workflows/quality.yml
##[warning]Uploading raw results.sarif. Native action pins will appear UNRECONCILED in this upload.

That run produced 51 PinnedDependenciesID alerts. 48 of them were in workflows the PR never touched (the PR changes only quality.yml).

Fix. Verification is now confined to each workflow file:

New verify_workflow(root, relative) wraps four things: the job-level reusable-ref lock check, the gh actions-lock --verify call, JSON parsing and acceptance. Any failure raises for that file only.
reconcile catches that per file and records failures[relative] = reason. It keeps all of that file's findings and goes on reconciling every other workflow. An unparseable workflow YAML is handled the same way; before this PR it also aborted everything. Failures are cached, so gh runs at most once per workflow.
reconcile now returns [document, audit, failures]. Callers that destructure two values (result, audit = ...) are unaffected.
CLI exit contract (documented in the script header):
- 0: everything verified.
- 3: some workflows failed. OUTPUT and AUDIT_JSON are written first. Each failed workflow is logged to stderr, and only its findings remain.
- 2: usage or SARIF-structure error. OUTPUT is not written, as before.
With scorecard-reusable.yml as written, this keeps the existing semantics:
- select-sarif keys on -s results.reconciled.sarif, so on exit 3 it uploads the reconciled SARIF, which contains only the failing workflow's alerts.
- Fail if reconciliation did not succeed keys on any non-success outcome, so the run still fails by design.

Not weakened: a workflow that fails verification still surfaces every one of its alerts and still fails the run. Only the collateral alerts from verified workflows are no longer resurrected.

Refs #1036. That issue is the adjacent defect: a false stale verdict from gh actions-lock on job-level reusable refs. It is the most common trigger of exactly this whole-document blast radius. By the same mechanism, #1036's index case (burble's release.yml failing on a false stale) would have un-reconciled every other burble workflow's pin alerts too. That is inferred from the code path; burble's alert count was not measured here. This PR does not change the #1036 acceptance logic, which #1064 already landed in verification_accepted?. It confines the damage when any single file fails, for a #1036 reason or a real one. See the note on AC3 below for why #1036 is Refs, not Closes.

Type of change

🐛 Bug fix (non-breaking change that fixes an issue). One workflow failing verification no longer un-reconciles all the others.
✨ New feature: not applicable.
💥 Breaking change: not breaking. The CLI's args are unchanged, exit 2 still means "nothing written", and the new exit 3 is still non-zero, so the reusable's fail step still fires. The extra return value is ignored by two-value destructuring.
🕳️ Soundness fix: not ticked. This removes false positives (48 spurious alerts). The 3 real alerts were already present in the raw upload, so no false negative is fixed.
📖 Documentation: only in-file docstrings and the header comment were changed.
🧹 Refactor / tech debt: not applicable. Behaviour changes on purpose.
⚡ Performance: not applicable.
🔧 Build / CI / tooling. This is a CI helper that scorecard-reusable.yml calls.

📌 New pins

Head SHA: e19b8023d4dc792b794f554c73e1892eca1aa865
No pins are added or changed. This PR touches only scripts/reconcile-scorecard-actions-lock.rb and its test, with no uses:, actions.lock, lockfile or container digest changes. In particular, the helper ref: in scorecard-reusable.yml is deliberately not bumped here, because it cannot point at this PR's own merge commit. See Follow-up.

How has this been verified?

All commands below were run locally in the worktree at the head SHA above. Ruby is 4.0.6 and gh actions-lock is v0.1.6, the same pin the reusable installs.

1. Test suite for this script. Baseline on origin/main was 6 runs, 22 assertions, all green.

$ bash scripts/tests/reconcile-scorecard-actions-lock-test.sh
10 runs, 65 assertions, 0 failures, 0 errors, 0 skips

There were no pre-existing failures in this suite. In CI this file runs in Self Test (.github/workflows/self-test.yml → scripts/run-shell-test-suite.sh, which globs scripts/tests/*.sh and installs ruby-minitest), so the regression test is enforced on this PR. Separately, Self Test on main was already red at 2f8c3eeb (2026-10-04); that predates this PR and was not investigated here.

New or changed tests:

test_one_failing_workflow_does_not_unreconcile_the_others is the regression test. Its fixture SARIF covers two workflows. The failing one (quality.yml, per-file stub response valid:false, ref-changed, exit 1) is placed first. The verified one (ci.yml) comes after it. The test asserts:
- both quality.yml findings are kept;
- both ci.yml findings are dropped and appear in the audit;
- failures names quality.yml;
- gh was invoked exactly twice.
test_unparseable_workflow_is_confined_to_itself: malformed YAML in one file does not stop the other file being reconciled.
test_cli_partial_failure_writes_output_and_exits_nonzero runs the script as a subprocess, exactly as the reusable invokes it. It asserts exit 3, that the reconciled SARIF is written and holds only the failing workflow's finding, that the audit has 1 entry, and that stderr names the failing file.
test_cli_success_and_structural_error_exit_codes asserts exit 0 with an empty result on success, and exit 2 with no output file on a bad SARIF.
Two existing tests changed their assertions: test_invalid_verification_never_becomes_a_clean_result and test_arm_d_… (renamed …_is_kept_and_reported). They used assert_raises, which encoded the whole-document abort itself. They now assert the per-file contract instead: the document is unchanged, the audit is empty, and the failure is recorded for that file. Every input those tests fed in (malformed JSON, valid:false with no findings, non-zero exit, and the Arm D job-level ref absent from the lock) still produces no suppression. This is the same guarantee, stated at file granularity, not a weakening.
The gh stub gained per-workflow responses (responses/<basename>.json and .exit), so a single run can mix a verifying workflow with a failing one.

2. Mutants. Each mutant was syntax-checked with ruby -c (Syntax OK) and the fix was restored after each one.

mutant	what it reintroduces	result
M0: `git show origin/main:scripts/reconcile-scorecard-actions-lock.rb` dropped in place	the real pre-fix whole-document `raise`	killed: `10 runs, 23 assertions, 1 failures, 4 errors`. `test_one_failing_workflow…`, `test_unparseable…`, `test_cli_partial_failure…`, `test_invalid_verification…` and `test_arm_d…` fail.
M1: `failures[relative] = error.message` → `raise error`	whole-document abort on a verification failure	killed: `1 failures, 3 errors`. `test_one_failing_workflow…`, `test_cli_partial_failure…`, `test_invalid_verification…` and `test_arm_d…` fail.
M2: `next false if failures.key?(relative)` → `next false unless failures.empty?`	quieter variant that stops filtering after the first failure	killed: `3 failures`. `test_one_failing_workflow…`, `test_cli_partial_failure…` and `test_unparseable…` fail.

After restoring: 10 runs, 65 assertions, 0 failures, 0 errors, 0 skips.

3. Real data: ZenodoDeposits.jl#15 replayed locally. This used that run's own scorecard-pr-results artifact (gh run download 37472579223 -n scorecard-pr-results) and a git archive of PR head f82394a5 (the run's headSha), verified with the real gh actions-lock v0.1.6.

helper	exit	reconciled SARIF written	PinnedDependencies alerts uploaded
`591cea1b` (what the reusable pins today)	2	no → raw uploaded	51
`origin/main` (`44a0de48`)	2	no → raw uploaded	51
this PR	3	yes	3, all in `.github/workflows/quality.yml`

The fixed run logged Scorecard reconciliation kept findings for .github/workflows/quality.yml: Native action-lock verification failed for .github/workflows/quality.yml and Reconciled 48 verified native action-pin false positives. Total results went from 62 to 14, and the audit has 48 entries.

quality.yml fails for a real reason, not the #1036 false stale: gh actions-lock .github/workflows/quality.yml --verify reports ref-changed (error, "workflow uses ref v7.0.0 but lockfile pins v7.0.1") plus the consequent stale. Those 3 alerts are the correct output of that positive control, and they are kept.

4. Hooks. The repo's pre-commit checks passed: actions-lock coverage, permissions, CodeQL config, bot directives and canonical names. .githooks/docstring-scan.sh --worktree --check reports both files as skipped because the scanner does not cover Ruby. That gives no verdict, so I checked by hand: every new function (verify_workflow and the test helpers assert_verification_failed, add_failing_workflow, run_cli, plus each new test) has a doc comment, and so do the touched-but-undocumented verification_accepted?, setup and the renamed Arm D test.

Checklist

My commits are signed (git commit -S). The SSH signature is verified locally: git log -1 --format=%G? → G.
I ran the project's own checks/tests locally and they pass: scripts/tests/reconcile-scorecard-actions-lock-test.sh (10/10) and the pre-commit hooks.
New files carry the correct SPDX-License-Identifier. There are no new files; both edited files keep their existing MPL-2.0 header, and nothing was relicensed.
Docs are updated, and no public claim now overstates what the code does. The script header now documents the exit codes, and the reconcile docstring no longer claims it "raises when verification fails". One caveat: the reusable's fail-step message says "Raw SARIF was uploaded". That becomes inaccurate for the exit-3 path only once the helper ref: is bumped, so it is left for that follow-up PR (below).
I have not introduced a soundness hole. A failing workflow's findings are never dropped (asserted by the regression test and the CLI test), and Arm D is still not suppressed. The one judgement call: a lock-file or gh problem that affects all workflows now yields per-file failures with every finding kept, plus exit 3. That has the same alert set as the old raw upload and still fails the run.

Notes for reviewers

This fix is not live until three steps happen, in order. scorecard-reusable.yml checks out the helper at a pinned ref: (591cea1b, in both the scorecard and pull-request jobs), not at main:

Merge this PR.
Follow-up PR in standards: bump the helper ref: in both jobs of .github/workflows/scorecard-reusable.yml to the merge SHA. At the same time, update the Fail if reconciliation did not succeed message so it distinguishes exit 3 ("reconciled SARIF uploaded; N workflow(s) failed verification") from the raw-upload fallback.
Follow-up in each caller: callers pin scorecard-reusable.yml by SHA, so each caller's uses: and actions.lock entry need a bump to the SHA from step 2. That work is not done here.

#1036 is Refs, not Closes. The 591cea1b pin also means the #1036 stale-tolerance from #1064 (verification_accepted?, lock_covers_reusable_deps?) has never run in CI. The ZenodoDeposits log above shows HEAD is now at 591cea1. So #1036 AC3 (burble and ephapax green without touching their YAML or lock) stays blocked on the same step-2 bump. AC4 (the upstream gh-actions-lock report) and AC2's arm-D direction are not addressed by this PR beyond keeping the existing Arm D test, now asserting kept-and-reported instead of raise.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm

reconcile-scorecard-actions-lock.rb raised on the first workflow that failed native action-lock verification. The CLI then exited 2 without writing results.reconciled.sarif, so scorecard-reusable.yml uploaded the RAW SARIF and every lock-pinned action in every workflow reappeared as a code-scanning alert. Measured on metadatastician/ZenodoDeposits.jl#15 (run 37472579223): 51 PinnedDependencies alerts, 48 of them in workflows the PR never touched, because quality.yml (deliberately desynced as a positive control) failed verification. Verification is now confined to the workflow that fails: - verify_workflow() wraps the job-level-ref lock check, the gh call and JSON parsing; any failure there (or unparseable workflow YAML) is recorded for that workflow only, and its findings are kept. - reconcile() returns [document, audit, failures]; existing two-value destructuring callers are unaffected. - CLI: on a partial failure it writes OUTPUT and AUDIT_JSON, logs each failed workflow and exits 3, so the workflow uploads the reconciled SARIF (only the failing workflow's alerts) and the run still fails by design. Usage/SARIF-structure errors still exit 2 with no OUTPUT. Nothing is weakened: a workflow that fails verification still surfaces all of its alerts and still fails the run. Two tests that asserted a raise now assert the stronger per-file contract (finding kept, no audit record, failure reported). New tests cover a failing workflow ahead of a verified one, an unparseable workflow, and the CLI exit/file contract. Mutants (re-raise; stop filtering after first failure; the pre-fix origin/main script) are each killed. Refs #1036 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm

coderabbitai · 2026-10-06T16:42:07Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 3605a6f0-001c-4db5-bb61-73cd68b32b21

📥 Commits

Reviewing files that changed from the base of the PR and between b935e63 and e19b802.

📒 Files selected for processing (2)

scripts/reconcile-scorecard-actions-lock.rb
scripts/tests/reconcile-scorecard-actions-lock-test.rb

 __________________________________________
< Dollars to donuts, you didn't test this. >
 ------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

🔴 Error committing to branch - (🔄 Check to retry)
Create a new PR

Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-06T16:42:13Z

K9 contract conformance

run https://github.com/hyperpolymath/standards/actions/runs/37497739263

K9 normative contract typecheck

k9_contract.ncl typechecks

K9 contract self-test

== the bash mirrors cannot drift from the normative contract ==
ok   leash_levels mirrors k9_contract.ncl
ok   core_capabilities mirrors k9_contract.ncl
ok   contract_version mirrors k9_contract.ncl
ok   schema_major mirrors k9_contract.ncl
== capability arithmetic (§8) ==
ok   capability_ok fs.read accepted
ok   capability_ok rollback.apply accepted
ok   capability_ok x-acme.gpu.alloc accepted
ok   capability_ok x-acme rejected
ok   capability_ok x-.gpu rejected
ok   capability_ok fs.delete rejected
ok   capability_ok  rejected
== the extractor ==
ok   extracts pedigree.security.leash
ok   extracts pedigree.component_type
ok   extracts pedigree.metadata.name
ok   pedigree leash is not reported as top-level leash
ok   required_capabilities for a quiet component
ok   required_capabilities follows allow_network
== the envelope strip keeps line numbers (§3.6) ==
ok   line 1 becomes a comment
ok   line count is preserved
ok   schema_version stays on line 5
== L3: signature presence is not verification (§10) ==
ok   no verifier -> K9-C001 is SKIPPED, never a pass
ok   the skip states presence does not authorise 'Hunt
ok   verifier accepts -> verdict 'Verified, no K9-C001 finding
ok   verifier refuses -> K9-C001 error, verdict 'Rejected
== the fixture runner's attribution cannot be fooled by a filename ==
ok   every extracted finding is well-formed rule+layer
ok   the rule that really fired is attributed
ok   a rule named only in the filename is NOT attributed
ok   K9-C001 is present as a skipped finding
ok   and that same finding is NOT extractable as a rejection
== no Nickel reserved word is used as an identifier ==
ok   the contract and all 27 fixtures avoid Nickel's reserved words

self-test: all assertions passed

K9 conformance fixtures

== positive controls (must pass) ==
ok   extension-capability.k9.ncl
ERROR   K9-N001 [L2] 1-formats/k9/tools/fixtures/valid/extension-fields.k9.ncl: component violates the K9.Component contract: error: contract broken by a value        extra fields `failure_mode_defenses`, `execution`    ┌─ /home/runner/work/standards/standards/1-formats/k9/tools/fixtures/valid/.k9-validate.9055.9336.driver.ncl:3:1    │  3 │ k9_doc | K9.Component 
FAIL extension-fields.k9.ncl should conform (exit 1)
ok   hunt-fully-granted.k9.ncl
ok   kennel-data.k9.ncl
ok   library-base.ncl
ok   yard-typed-config.k9.ncl

== negative controls (must fail, by the named rule) ==
ok   L0-K9-E001-bad-magic.k9.ncl (rejected by K9-E001 at L0)
ok   L0-K9-E002-nul-byte.k9.ncl (rejected by K9-E002 at L0)
ok   L0-K9-E003-crlf.k9.ncl (rejected by K9-E003 at L0)
ok   L0-K9-E004-no-spdx.k9.ncl (rejected by K9-E004 at L0)
ok   L0-K9-E005-unclaimed-body.k9.ncl (rejected by K9-E005 at L0)
ok   L0-K9-S012-library-with-pedigree.ncl (rejected by K9-S012 at L0)
ok   L0-K9-S014-stray-leash.ncl (rejected by K9-S014 at L0)
ok   L1-K9-S001-no-pedigree.k9.ncl (rejected by K9-S001 at L1)
ok   L1-K9-S002-wrong-major.k9.ncl (rejected by K9-S002 at L1)
ok   L1-K9-S003-todo-component-type.k9.ncl (rejected by K9-S003 at L1)
ok   L1-K9-S004-unknown-leash.k9.ncl (rejected by K9-S004 at L1)
ok   L1-K9-S005-missing-name.k9.ncl (rejected by K9-S005 at L1)
ok   L1-K9-S006-unknown-capability.k9.ncl (rejected by K9-S006 at L1)
ok   L1-K9-S007-ungranted-flag.k9.ncl (rejected by K9-S007 at L1)
ok   L1-K9-S008-hunt-signature-not-required.k9.ncl (rejected by K9-S008 at L1)
ok   L1-K9-S009-hunt-no-signature-block.k9.ncl (rejected by K9-S009 at L1)
ok   L1-K9-S010-hunt-empty-side-effects.k9.ncl (rejected by K9-S010 at L1)
ok   L1-K9-S011-recipes-at-yard.k9.ncl (rejected by K9-S011 at L1)
ok   L1-K9-S013-dangling-import.k9.ncl (rejected by K9-S013 at L1)
ok   L2-K9-N001-two-segment-version.k9.ncl (rejected by K9-N001 at L2)
ok   L2-K9-N001-wrong-field-type.k9.ncl (rejected by K9-N001 at L2)

fixtures: 6 positive, 21 negative (0 needing nickel), 1 failure(s)

github-actions · 2026-10-06T16:46:54Z

K9 contract conformance

run https://github.com/hyperpolymath/standards/actions/runs/37498356760

K9 normative contract typecheck

k9_contract.ncl typechecks

K9 contract self-test

== the bash mirrors cannot drift from the normative contract ==
ok   leash_levels mirrors k9_contract.ncl
ok   core_capabilities mirrors k9_contract.ncl
ok   contract_version mirrors k9_contract.ncl
ok   schema_major mirrors k9_contract.ncl
== capability arithmetic (§8) ==
ok   capability_ok fs.read accepted
ok   capability_ok rollback.apply accepted
ok   capability_ok x-acme.gpu.alloc accepted
ok   capability_ok x-acme rejected
ok   capability_ok x-.gpu rejected
ok   capability_ok fs.delete rejected
ok   capability_ok  rejected
== the extractor ==
ok   extracts pedigree.security.leash
ok   extracts pedigree.component_type
ok   extracts pedigree.metadata.name
ok   pedigree leash is not reported as top-level leash
ok   required_capabilities for a quiet component
ok   required_capabilities follows allow_network
== the envelope strip keeps line numbers (§3.6) ==
ok   line 1 becomes a comment
ok   line count is preserved
ok   schema_version stays on line 5
== L3: signature presence is not verification (§10) ==
ok   no verifier -> K9-C001 is SKIPPED, never a pass
ok   the skip states presence does not authorise 'Hunt
ok   verifier accepts -> verdict 'Verified, no K9-C001 finding
ok   verifier refuses -> K9-C001 error, verdict 'Rejected
== the fixture runner's attribution cannot be fooled by a filename ==
ok   every extracted finding is well-formed rule+layer
ok   the rule that really fired is attributed
ok   a rule named only in the filename is NOT attributed
ok   K9-C001 is present as a skipped finding
ok   and that same finding is NOT extractable as a rejection
== no Nickel reserved word is used as an identifier ==
ok   the contract and all 27 fixtures avoid Nickel's reserved words

self-test: all assertions passed

K9 conformance fixtures

== positive controls (must pass) ==
ok   extension-capability.k9.ncl
ERROR   K9-N001 [L2] 1-formats/k9/tools/fixtures/valid/extension-fields.k9.ncl: component violates the K9.Component contract: error: contract broken by a value        extra fields `failure_mode_defenses`, `execution`    ┌─ /home/runner/work/standards/standards/1-formats/k9/tools/fixtures/valid/.k9-validate.8796.26747.driver.ncl:3:1    │  3 │ k9_doc | K9.Component 
FAIL extension-fields.k9.ncl should conform (exit 1)
ok   hunt-fully-granted.k9.ncl
ok   kennel-data.k9.ncl
ok   library-base.ncl
ok   yard-typed-config.k9.ncl

== negative controls (must fail, by the named rule) ==
ok   L0-K9-E001-bad-magic.k9.ncl (rejected by K9-E001 at L0)
ok   L0-K9-E002-nul-byte.k9.ncl (rejected by K9-E002 at L0)
ok   L0-K9-E003-crlf.k9.ncl (rejected by K9-E003 at L0)
ok   L0-K9-E004-no-spdx.k9.ncl (rejected by K9-E004 at L0)
ok   L0-K9-E005-unclaimed-body.k9.ncl (rejected by K9-E005 at L0)
ok   L0-K9-S012-library-with-pedigree.ncl (rejected by K9-S012 at L0)
ok   L0-K9-S014-stray-leash.ncl (rejected by K9-S014 at L0)
ok   L1-K9-S001-no-pedigree.k9.ncl (rejected by K9-S001 at L1)
ok   L1-K9-S002-wrong-major.k9.ncl (rejected by K9-S002 at L1)
ok   L1-K9-S003-todo-component-type.k9.ncl (rejected by K9-S003 at L1)
ok   L1-K9-S004-unknown-leash.k9.ncl (rejected by K9-S004 at L1)
ok   L1-K9-S005-missing-name.k9.ncl (rejected by K9-S005 at L1)
ok   L1-K9-S006-unknown-capability.k9.ncl (rejected by K9-S006 at L1)
ok   L1-K9-S007-ungranted-flag.k9.ncl (rejected by K9-S007 at L1)
ok   L1-K9-S008-hunt-signature-not-required.k9.ncl (rejected by K9-S008 at L1)
ok   L1-K9-S009-hunt-no-signature-block.k9.ncl (rejected by K9-S009 at L1)
ok   L1-K9-S010-hunt-empty-side-effects.k9.ncl (rejected by K9-S010 at L1)
ok   L1-K9-S011-recipes-at-yard.k9.ncl (rejected by K9-S011 at L1)
ok   L1-K9-S013-dangling-import.k9.ncl (rejected by K9-S013 at L1)
ok   L2-K9-N001-two-segment-version.k9.ncl (rejected by K9-N001 at L2)
ok   L2-K9-N001-wrong-field-type.k9.ncl (rejected by K9-N001 at L2)

fixtures: 6 positive, 21 negative (0 needing nickel), 1 failure(s)

github-actions · 2026-10-06T16:47:25Z

K9 contract conformance

run https://github.com/hyperpolymath/standards/actions/runs/37498380657

K9 normative contract typecheck

k9_contract.ncl typechecks

K9 contract self-test

== the bash mirrors cannot drift from the normative contract ==
ok   leash_levels mirrors k9_contract.ncl
ok   core_capabilities mirrors k9_contract.ncl
ok   contract_version mirrors k9_contract.ncl
ok   schema_major mirrors k9_contract.ncl
== capability arithmetic (§8) ==
ok   capability_ok fs.read accepted
ok   capability_ok rollback.apply accepted
ok   capability_ok x-acme.gpu.alloc accepted
ok   capability_ok x-acme rejected
ok   capability_ok x-.gpu rejected
ok   capability_ok fs.delete rejected
ok   capability_ok  rejected
== the extractor ==
ok   extracts pedigree.security.leash
ok   extracts pedigree.component_type
ok   extracts pedigree.metadata.name
ok   pedigree leash is not reported as top-level leash
ok   required_capabilities for a quiet component
ok   required_capabilities follows allow_network
== the envelope strip keeps line numbers (§3.6) ==
ok   line 1 becomes a comment
ok   line count is preserved
ok   schema_version stays on line 5
== L3: signature presence is not verification (§10) ==
ok   no verifier -> K9-C001 is SKIPPED, never a pass
ok   the skip states presence does not authorise 'Hunt
ok   verifier accepts -> verdict 'Verified, no K9-C001 finding
ok   verifier refuses -> K9-C001 error, verdict 'Rejected
== the fixture runner's attribution cannot be fooled by a filename ==
ok   every extracted finding is well-formed rule+layer
ok   the rule that really fired is attributed
ok   a rule named only in the filename is NOT attributed
ok   K9-C001 is present as a skipped finding
ok   and that same finding is NOT extractable as a rejection
== no Nickel reserved word is used as an identifier ==
ok   the contract and all 27 fixtures avoid Nickel's reserved words

self-test: all assertions passed

K9 conformance fixtures

== positive controls (must pass) ==
ok   extension-capability.k9.ncl
ERROR   K9-N001 [L2] 1-formats/k9/tools/fixtures/valid/extension-fields.k9.ncl: component violates the K9.Component contract: error: contract broken by a value        extra fields `failure_mode_defenses`, `execution`    ┌─ /home/runner/work/standards/standards/1-formats/k9/tools/fixtures/valid/.k9-validate.8982.23745.driver.ncl:3:1    │  3 │ k9_doc | K9.Component 
FAIL extension-fields.k9.ncl should conform (exit 1)
ok   hunt-fully-granted.k9.ncl
ok   kennel-data.k9.ncl
ok   library-base.ncl
ok   yard-typed-config.k9.ncl

== negative controls (must fail, by the named rule) ==
ok   L0-K9-E001-bad-magic.k9.ncl (rejected by K9-E001 at L0)
ok   L0-K9-E002-nul-byte.k9.ncl (rejected by K9-E002 at L0)
ok   L0-K9-E003-crlf.k9.ncl (rejected by K9-E003 at L0)
ok   L0-K9-E004-no-spdx.k9.ncl (rejected by K9-E004 at L0)
ok   L0-K9-E005-unclaimed-body.k9.ncl (rejected by K9-E005 at L0)
ok   L0-K9-S012-library-with-pedigree.ncl (rejected by K9-S012 at L0)
ok   L0-K9-S014-stray-leash.ncl (rejected by K9-S014 at L0)
ok   L1-K9-S001-no-pedigree.k9.ncl (rejected by K9-S001 at L1)
ok   L1-K9-S002-wrong-major.k9.ncl (rejected by K9-S002 at L1)
ok   L1-K9-S003-todo-component-type.k9.ncl (rejected by K9-S003 at L1)
ok   L1-K9-S004-unknown-leash.k9.ncl (rejected by K9-S004 at L1)
ok   L1-K9-S005-missing-name.k9.ncl (rejected by K9-S005 at L1)
ok   L1-K9-S006-unknown-capability.k9.ncl (rejected by K9-S006 at L1)
ok   L1-K9-S007-ungranted-flag.k9.ncl (rejected by K9-S007 at L1)
ok   L1-K9-S008-hunt-signature-not-required.k9.ncl (rejected by K9-S008 at L1)
ok   L1-K9-S009-hunt-no-signature-block.k9.ncl (rejected by K9-S009 at L1)
ok   L1-K9-S010-hunt-empty-side-effects.k9.ncl (rejected by K9-S010 at L1)
ok   L1-K9-S011-recipes-at-yard.k9.ncl (rejected by K9-S011 at L1)
ok   L1-K9-S013-dangling-import.k9.ncl (rejected by K9-S013 at L1)
ok   L2-K9-N001-two-segment-version.k9.ncl (rejected by K9-N001 at L2)
ok   L2-K9-N001-wrong-field-type.k9.ncl (rejected by K9-N001 at L2)

fixtures: 6 positive, 21 negative (0 needing nickel), 1 failure(s)

sonarqubecloud · 2026-10-06T16:47:25Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

coderabbitai · 2026-10-06T16:52:33Z

Add Carrot credits or activate Agent usage billing to use Autopilot

coderabbitai · 2026-10-06T16:52:38Z

❌ Failed to create Coding Agent finishing-touch task. Please try again.

…1183) ## Summary `scorecard-reusable.yml` checks the reconciliation helper out at a pinned standards commit, `591cea1b`. That commit predates #1168 (per-file verification) and #1064 (stale-tolerance), so neither fix has ever run in CI. Every caller still gets the old, whole-repo reconciliation. This PR: - **Bumps the helper ref** in both jobs (`scheduled` and `pull-request`) from `591cea1b` to `5a0891e9`, the #1168 merge commit on `main`. - **Passes the reconcile exit code on** as a step output, `rc`. The helper exits 3 when it wrote and uploaded the reconciled SARIF but some workflows failed native lock verification and keep their findings. It exits 2 on a usage or SARIF-structure error, when nothing is written. - **Corrects the final error message.** On exit 3 it used to say "Raw SARIF was uploaded", which was false: the reconciled file was uploaded. It now says "reconciliation incomplete" and points at the step log that names each workflow. The old message stays for the real failure case. **Both cases still fail the run.** No gate is weakened. Refs #1036. The callers need no change: they pick this up through their existing pin to this reusable once they bump it, as tracked in #1036. ## Type of change - [x] 🐛 Bug fix (non-breaking change that fixes an issue): the reusable ran a reconciler without the fixes merged in #1168 and #1064, and misreported exit 3. - [ ] ✨ New feature: no new behaviour beyond running the already-merged helper. - [ ] 💥 Breaking change: the job fails in exactly the cases it failed before. - [ ] 🕳️ Soundness fix: no checker false negative is closed here. - [ ] 📖 Documentation: no docs change. - [ ] 🧹 Refactor / tech debt: behaviour changes (newer helper), so not behaviour-preserving. - [ ] ⚡ Performance: not affected. - [x] 🔧 Build / CI / tooling: reusable workflow change. ## 📌 New pins > **Head: `d13c091c23ffe3df071f5831bcdbd37111aaa69d`**, the commit CI runs on. > > - **`hyperpolymath/standards` checkout `ref:` `591cea1b074b56367f25ec0084a0c4ab66b42a7e` → `5a0891e9e58b785181a92e141505ca96ca987b42`** (the #1168 merge), in both jobs. This is an `actions/checkout` `with: ref:`, not a `uses:` ref, so it needs no `actions.lock` entry. > - No `uses:` refs, `actions.lock` entries, lockfiles or container digests are added or changed. ## How has this been verified? - `gh api repos/hyperpolymath/standards/git/commits/5a0891e9e58b785181a92e141505ca96ca987b42` returned the commit "fix(scorecard): reconcile action-pin findings per workflow file (#1168)". So the ref is a commit, not a blob or tag object (the #987 trap). - `git merge-base --is-ancestor 5a0891e… origin/main`: it is on `main`. - `git show 5a0891e:scripts/reconcile-scorecard-actions-lock.rb`: the helper's `exit 3` (line 222) and `exit 2` (line 226) are the two codes the new message branches on. - `actionlint .github/workflows/scorecard-reusable.yml`: rc=0, no findings. - `git merge-tree --write-tree origin/main HEAD`: merges cleanly. The branch is 8 commits behind `main`, but none of them touches this file. - **Not verified: a CI run of this reusable at this head.** The exit-3 path is exercised only when a caller's Scorecard run hits a workflow that fails native lock verification, and that has not happened yet. ## Checklist - [x] My commits are **signed** (`git commit -S`): `d13c091c` shows `G`. - [x] I ran the project's own checks/tests locally and they pass: actionlint is clean. No standards test covers this workflow's step logic. - [x] New files carry the correct `SPDX-License-Identifier`: there are no new files, and the existing file's header is unchanged. - [x] Docs are updated, and no public claim now overstates what the code does: the error text now states what was actually uploaded. - [x] I have not introduced a soundness hole: both failure paths still `exit 1`. ## Notes for reviewers - The `rc=0 … || rc=$?` form keeps `set -e` from killing the step before the code is recorded. The step then re-raises the same code with `exit "$rc"`, so `steps.reconcile.outcome` is unchanged from before. - Follow-up (#1036): bump the callers' pin to this reusable once this merges. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01X3hgXxWm6umMgZkjYyHnnm Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

hyperpolymath added 2 commits October 6, 2026 17:46

Merge branch 'main' into fix/scorecard-reconcile-per-file

88692ca

Merge branch 'main' into fix/scorecard-reconcile-per-file

6fa1b7a

hyperpolymath enabled auto-merge (squash) October 6, 2026 16:46

hyperpolymath merged commit 5a0891e into main Oct 6, 2026
54 of 61 checks passed

hyperpolymath deleted the fix/scorecard-reconcile-per-file branch October 6, 2026 16:47

hyperpolymath mentioned this pull request Oct 6, 2026

fix(scorecard): run the per-file reconciler and name exit 3 honestly #1183

Merged

7 of 13 tasks

Navigation Menu

Repository navigation

fix(scorecard): reconcile action-pin findings per workflow file - #1168

fix(scorecard): reconcile action-pin findings per workflow file#1168
hyperpolymath merged 3 commits into
mainfrom
fix/scorecard-reconcile-per-file

hyperpolymath commented Oct 6, 2026

Uh oh!

coderabbitai Bot commented Oct 6, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

sonarqubecloud Bot commented Oct 6, 2026

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 6, 2026

Uh oh!

coderabbitai Bot commented Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

hyperpolymath commented Oct 6, 2026

Summary

Type of change

📌 New pins

How has this been verified?

Checklist

Notes for reviewers

Uh oh!

coderabbitai Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026

K9 contract conformance

K9 normative contract typecheck

K9 contract self-test

K9 conformance fixtures

Uh oh!

github-actions Bot commented Oct 6, 2026

K9 contract conformance

K9 normative contract typecheck

K9 contract self-test

K9 conformance fixtures

Uh oh!

github-actions Bot commented Oct 6, 2026

K9 contract conformance

K9 normative contract typecheck

K9 contract self-test

K9 conformance fixtures

Uh oh!

sonarqubecloud Bot commented Oct 6, 2026

Quality Gate passed

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 6, 2026

Uh oh!

coderabbitai Bot commented Oct 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Oct 6, 2026 •

edited

Loading