Skip to content

fix(ci): four red checks on main, one root cause — workflows drifted off actions.lock - #266

Merged
hyperpolymath merged 4 commits into
mainfrom
arena/c04d58de-typed-wasm
Oct 10, 2026
Merged

hyperpolymath merged 4 commits into
mainfrom
arena/c04d58de-typed-wasm

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

What is red on main right now

Check State for 9+ days Cause
E2E Validation startup_failure workflow asks actions/setup-node@v7.1.0, actions.lock declares v7.0.0
Static Analysis Gate startup_failure upload-artifact@v7.0.2 / download-artifact@v8.0.2 vs locked v7.0.1 / v8.0.1
CodeQL Security Analysis startup_failure codeql-action@v4.38.2 vs locked v4.38.0
Governance / Workflow security linter failure (≥13 days) pin 81dbf2dd tests SPDX with head -1; gh actions-lock prepends its banner at line 1 of all 20 workflows
🤖 Rhodibot — RSR Compliance Canary failure not investigated here — listed below, deliberately out of scope

startup_failure with zero jobs and zero check runs is not a failing step: it is the
workflow being rejected at validation time. .github/workflows/actions.lock is the file
standards/scripts/set-allowed-actions.sh publishes into the organisation's
specific actions allowed list, so a ref absent from the lock is a ref the org does not
allow — and #265 (a Dependabot branch) bumped 13 workflow refs without a lock refresh.

Nothing in this repo checked lock↔workflow consistency, which is why it rotted silently.

Changes

Commit Change
87d5bc8 Snap the 13 drifted refs back to what the lock declares (5 files) — incl. c5-regenerate.yml and release.yml, which carry the same drift and would fail the moment they trigger
e3fbd1b Watched pins → the template's canonical da2c748a; actions: read where a reusable uploads
8bed496 secret-scanner-reusable to the same pin (was 1 commit behind; five reusables on four refs is how "which gate ran?" becomes unanswerable)
d506ccd Dependabot: hold CodeQL out of the group bump, point npm at the only manifest that exists, drop three dead ecosystems

Rollback targets are not guesses: every tag was checked against the API and resolves to
exactly the commit the lock records — setup-node 820762786026, upload-artifact 043fb46d1a93, download-artifact 3e5f45b2cfb9, setup-ocaml e89b2ded52a6,
codeql-action b96794f015df, setup-beam 54075bcc5e24.

CodeQL has a second, independent reason to stay at v4.38.0: v4.38.1+ fails workflow-startup
validation estate-wide (phantom-metal-taste#71).

Why the pin bump and not "fix the headers"

Both gates were extracted from the reusable and run against the same tree:

81dbf2dd  Check SPDX headers + permissions (head -1)   -> exit 1, 20 ERROR lines
da2c748a  Check SPDX headers + permissions (block scan)-> exit 0, "All workflows have SPDX headers + permissions"

Moving SPDX to line 1 would satisfy a gate the estate has since deleted, re-break on the
next lock refresh (standards#727: the tool re-prepends the banner), and still leave
Check SHA-pinned actions — which greps all of .github/workflows/ and, replayed here,
finds 43 violations, one of them inside actions.lock itself ( uses: at line
87). That gate pre-dates the lockfile model, so it flags precisely the symbolic refs the
lock exists to authorise; it is masked today only because SPDX fails first in the same job.
da2c748a removes it. Also replayed from the new pin, check-action-pins-resolve.sh:
All 2 verifiable action pin(s) resolve upstream — exit 0.

actions: read is granted where the callee uploads (hypatia SARIF+artifact at workflow
level, scorecard artifact at job level, because a job-level block replaces the
workflow map — standards#1035), and withheld from governance.yml, whose reusable has no
upload step. Least privilege beats copying the sibling diff.

Verification

  • lock↔workflow drift: 17 → 2, and both survivors are explained below
  • 21 YAML files: parse clean + duplicate-key clean under js-yaml semantics (PyYAML
    forgives duplicates, GitHub's parser does not — that mismatch is a known startup_failure source)
  • every run: edit asserted against an expected match count before writing (13/13, no blind sed)
  • this PR is the test: e2e, static-analysis-gate, codeql and governance all
    trigger on pull_request, so the run on this branch exercises the exact paths that were dead

Not verifiable from here, stated plainly

  • gh actions-lock --verify-local — the extension binary is unreachable (release assets are on
    release-assets.githubusercontent.com, outside this sandbox's egress allowlist), so the lock
    gate could not be run locally
  • org/repo Actions settings, branch protection, vars, secrets — 403. The startup_failure
    cause is inferred from its signature plus repo content, not read from the policy that produced it
  • cargo, clippy, just, bun, asciidoctor — absent from the sandbox; no build was run
  • hypatia at da2c748a has its own red Governance run; the green oracle used here is
    rsr-template-repo (banner + lock + da2c748a → Governance success 2026-10-09) and
    gitbot-fleet (success 2026-10-08)

Merge order — one dependency

  1. 87d5bc8 and d506ccd are safe to merge now.
  2. Before merging the pin bumps, re-mint the lock (gh actions-lock → commit
    .github/workflows/actions.lock) for the two remaining drifts:
    hyperpolymath/smtp-notify-action@v0.5.0 (the lock lags an intentional bump; org-owned
    actions are outside the allowlist, so this workflow is green and was left alone) and the
    slsa-github-generator reusable call in release.yml (the lockfile does not record
    reusable-workflow calls). The bump adds a governance / Actions lockfile verify job that
    installs the tool and runs --verify-local; without the re-mint that job will go red.
    It is a true red naming its cause — not a reason to hand-edit a machine-generated file.

Follow-ups (drafted here, not filed)

  1. standards: dependabot-lock-regen.yml exists specifically so a bump cannot land with a
    stale lock — but both "Mint an App installation token" steps are skipped on run
    37859881345 because vars.APP_ID is unset, so it warns and exits 0. The guard is
    fail-open estate-wide; this outage is downstream of that one missing credential (STD-R-3).
  2. this repo: #265 was a Dependabot branch that merged 21 workflow files and 2,136 lines
    of human CI revival. Bot PRs should not be the vehicle for that — the diff is unreviewable
    in the way Dependabot reviews are designed to be skippable, and here it broke four workflows.
  3. 🤖 Rhodibot — RSR Compliance Canary is red and untouched by this branch.
  4. mirror.yml still uses secrets: inherit; the template's mirror caller uses an explicit
    secrets map (WH008). Deliberately not changed — the workflow is green, and changing two
    things at once on a green workflow is how you manufacture an outage.

hyperpolymath and others added 4 commits October 10, 2026 10:38
… allowlist

Three workflows on main died as `startup_failure` - no jobs, no check runs, no
annotations. That signature is a workflow rejected by the organisation's
"specific actions allowed" policy at *validation* time, not a failing step. The
allowlist is derived from `.github/workflows/actions.lock` (standards
`scripts/set-allowed-actions.sh`), and PR #265 bumped workflow refs without a
lock refresh, so those refs stopped being on the list GitHub matches:

  E2E Validation        lock: setup-node@v7.0.0             had: @v7.1.0
  Static Analysis Gate  lock: upload-artifact@v7.0.1,       had: @v7.0.2,
                        download-artifact@v8.0.1                 @v8.0.2
  CodeQL Security ...   lock: codeql-action@v4.38.0         had: @v4.38.2

Each rolled-back tag resolves upstream to exactly the commit the lock records -
checked per ref against the API (setup-node 820762786026, upload-artifact
043fb46d1a93, download-artifact 3e5f45b2cfb9, setup-ocaml e89b2ded52a6,
codeql-action b96794f015df, setup-beam 54075bcc5e24) - so this restores the
state the tooling already trusts instead of inventing a pin set.

c5-regenerate.yml (setup-ocaml v3.9.0 -> v3.8.0) and release.yml
(upload-artifact v7.0.2 -> v7.0.1) carry the same drift and would fail the same
way the moment they are triggered; fixing them removes the class, not just the
three visible symptoms.

CodeQL has a second, independent reason to stay at v4.38.0: v4.38.1+ fails
workflow-startup validation estate-wide (recorded in
hyperpolymath/phantom-metal-taste#71, mirrored by that repo's dependabot hold).

Two drifts are left alone deliberately: `hyperpolymath/smtp-notify-action@v0.5.0`
(the lock lags an intentional bump; the workflow is green and org-owned actions
are outside the allowlist) and the slsa-generator reusable call in release.yml
(the lockfile does not record reusable-workflow calls). Both are the lock's to
catch up with via `gh actions-lock`, not code to roll back.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…c748a

`Governance / Workflow security linter` has been red on main for at least 13
days. It is not a repo bug: `gh actions-lock` prepends

    # This workflow is managed by gh actions-lock.

at line 1 of all 20 workflows, and the pin this repo watches (81dbf2dd) tests
the licence with `head -1`, so the tool and the gate cannot both be satisfied.
standards#727 is that upstream conflict; da2c748a carries the fix (13cb62ac:
scan the leading comment block instead of line 1) and is the pin
rsr-template-repo itself uses.

Evidence, both directions, by running each pin's gate step verbatim against
this tree (extracted from the reusable, executed in a worktree):

    81dbf2dd  head -1 scan      -> exit 1, 20 ERROR lines   (today's red)
    da2c748a  block scan        -> exit 0, "All workflows have SPDX headers"

The old pin's `Check SHA-pinned actions` step is also gone at da2c748a, and that
matters more than it looks: replaying its grep here finds 43 violations, one of
them *inside actions.lock itself* (`        uses:` at line 87). That gate
pre-dates the lockfile model, so it flags the exact symbolic refs the lock exists
to authorise - and it was masked today only because SPDX fails first in the same
job. Reordering the headers instead would have been the wrong fix: it satisfies
a gate the estate has since deleted, re-breaks on every lock refresh, and would
still leave 43 refs to hand-pin.

Grants added with the bump, per standards#1035 - a job-level `permissions:` block
REPLACES the workflow map and a reusable cannot escalate:

  hypatia-scan.yml  workflow level  + actions:read   (uploads SARIF + artifact)
  scorecard.yml     job level       + actions:read   (uploads the report artifact)
  governance.yml    no grant        (its reusable has no upload step; least
                                     privilege beats copying the sibling diff)

mirror.yml moves d135b05b -> da2c748a as well: d135b05b is the pin standards#950
names as "all seven jobs blocking, no advisory job", which is why one dead remote
here would fail the whole mirror.

Two consequences a reviewer must sequence, both called out in the PR body:
this pin adds an `Actions lockfile verify` job that installs `gh actions-lock`
and runs `--verify-local`, so it will red until the lock is re-minted for
`hyperpolymath/smtp-notify-action@v0.5.0` and the slsa reusable call - a true
red naming its cause, not a false one. `Check action pins resolve upstream` was
replayed locally and all six SHA-form pins in this repo resolve.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
… ecosystems that do not exist

Three separate measured defects in one file.

1. The github-actions entry was `patterns: ["*"]` with no hold, which is how the
   refs in 87d5bc8 got moved under the lock. `ignore` alone does not suppress a
   grouped bump (phantom-metal-taste#76 measured exactly that), so the ref is
   excluded from the group as well as ignored. CodeQL carries a second,
   independent reason to stay put: v4.38.1+ fails workflow-startup validation
   estate-wide.

2. `npm` declared `directory: "/"`, where this repo has no package.json, while the
   one manifest it does have - `tools/tree-sitter-twasm/package.json` - sat
   unwatched. The ecosystem was live; the path was wrong.

3. `mix`, `pip` and `nix` had no manifests anywhere in the tree (`git ls-files`
   counts: 0 mix.exs, 0 pyproject/requirements/setup.py, 0 flake.nix/shell.nix).
   An ecosystem with no manifest cannot open a PR; it can only imply coverage it
   does not provide. Same shape as phantom-metal-taste#77's dead `nix`/`npm`
   entries, and that repo's second defect - two `ignore:` keys in one mapping,
   where the parser keeps only the last - is avoided here by using one key.

Parsed and re-read after editing: ecosystems are now `[github-actions, cargo,
npm]`, `ignore` resolves to `[github/codeql-action]`, the group keeps
`patterns: ["*"]` plus `exclude-patterns: [github/codeql-action]`.

The comment in the file records what the real guard is and why it is not holding:
standards' `dependabot-lock-regen.yml` exists precisely to refresh each open
dependabot PR's lockfile, but both of its App-token steps are `skipped` because
`vars.APP_ID` is unset, so it warns and exits 0. That is an owner action on
standards, not something this repo can fix, and it is the reason a hold comment
is doing security-critical work here at all.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
e3fbd1b moved governance, hypatia-scan, scorecard and mirror onto the pin the
template itself uses; secret-scanner was left on c65436ee. That pin is not
wrong - it is one commit behind da2c748a on this file - but a repo whose five
estate reusables sit on four different refs is one `gh actions-lock` sweep away
from the drift that caused this outage, and pin-divergence is exactly what makes
"which gate is running?" unanswerable during an incident.

Checked before swapping: da2c748a's secret-scanner-reusable declares the same
`workflow_call` inputs (only the optional `runs-on`), so the call site needs no
shape change, and its job-level permission grants (contents: read for the
checkout, actions: read for workflow-run metadata) are already in place in this
caller.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 8c158d4b-db43-4d09-9147-f775027a36d8

📥 Commits

Reviewing files that changed from the base of the PR and between f6990b2 and 8bed496.


📒 Files selected for processing (11)
  • .github/dependabot.yml
  • .github/workflows/c5-regenerate.yml
  • .github/workflows/codeql.yml
  • .github/workflows/e2e.yml
  • .github/workflows/governance.yml
  • .github/workflows/hypatia-scan.yml
  • .github/workflows/mirror.yml
  • .github/workflows/release.yml
  • .github/workflows/scorecard.yml
  • .github/workflows/secret-scanner.yml
  • .github/workflows/static-analysis-gate.yml

 __________________________________________________________
< This regex looks like you sneezed on the keyboard-twice. >
 ----------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@hyperpolymath
hyperpolymath merged commit a6615d6 into main Oct 10, 2026
36 of 44 checks passed
@hyperpolymath
hyperpolymath deleted the arena/c04d58de-typed-wasm branch October 10, 2026 10:44
hyperpolymath added a commit that referenced this pull request Oct 10, 2026
#269)

## What this fixes

The other half of the Governance red that #266 left behind. After that
merge, `governance / Workflow security linter` fails one step later than
it used to — the SPDX step passes now, and what remains is:

```
::error file=Justfile::Policy requires a retired descriptile path; use
.machine_readable/descriptiles/ and reconcile existing files
```

This is a **pre-existing repo violation the stale pin was masking**, not
a regression from #266. `0-canon/CANONICAL-NAMES.adoc` (owner mandate
2026-06-30) states the old name `6a2` is DEPRECATED and to *"migrate it
to descriptiles on sight"*; `check-canonical-names.sh` only blocks *new*
occurrences, because the bulk migration was chartered separately. This
is that migration.

## Changes

| | |
|---|---|
| **Moved** |
`.machine_readable/6a2/{STATE,META,ECOSYSTEM,AGENTIC,NEUROSYM,PLAYBOOK}.a2ml`
→ `.machine_readable/descriptiles/…` (`git mv`, content unchanged — five
of six are `R100`) |
| **Rewritten** | 49 path refs across 19 live files: `Justfile`,
`0-AI-MANIFEST.a2ml`, `AUDIT.adoc`, `.github/GOVERNANCE.md`, the PR
template, `.machine_readable/ai/*`, `policies/MAINTENANCE-AXES.a2ml`,
four `docs/` files, `spec/async-convergence-abi.adoc`,
`src/abi/README.adoc` |
| **Deliberately left** | 4 refs in `CHANGELOG.adoc` and
`docs/reports/maintenance/2026-05-25-phase-0-closure.adoc`. The mandate
allows the deprecated spelling in historical records, and rewriting a
dated report would make it lie about what it referenced that day |

## The Justfile half is not cosmetic

`just info`, `just validate-rsr` and `just validate-state` tested
`.machine_readable/STATE.a2ml` and `…/META.a2ml` **bare** — paths that
have never existed in this repo — so their `[ -f ]` guards made the
recipes skip and report success. After retargeting:

```
META.a2ml axis assertions        -> 5 pass / 0 fail   (previously: not executed at all)
STATE.a2ml phase                 -> 0                 (previously: nothing printed)
STATE [metadata] / project key   -> 1 / 1             (previously: "No …STATE.a2ml found")
```

No workflow invokes those recipes (checked), so CI behaviour is
unchanged and local dev checks stop being theatre.

## Verification — the estate's own gates, run locally

```
check-descriptile-policy.sh             -> exit 0    (exit 1 on main)
check-canonical-names.sh origin/main    -> "✅ no deprecated names reintroduced"
every .machine_readable/descriptiles/*.a2ml referenced anywhere -> exists
```

Not verified, and not claimable from here: the real CI run (this PR
triggers `governance / Workflow security linter` on `pull_request`,
which is the test), and `just` itself is absent from the sandbox, so the
recipes were exercised by running their shell bodies directly rather
than through `just`.

## What this does not fix

- `governance / Actions lockfile verify` — needs `gh actions.lock`
re-minted for `hyperpolymath/smtp-notify-action@v0.5.0`. The lockfile
header says *"Do not edit by hand; run `gh actions-lock` to update"*,
and this sandbox cannot fetch the extension (release assets are on
`release-assets.githubusercontent.com`, outside the egress allowlist),
so hand-writing the entry was rejected as the wrong move. One command,
from a machine that has it: `gh extension install github/gh-actions-lock
&& gh actions-lock`. Note CI's own annotation agrees: *"Regenerate with
scripts/update-actions-lock.sh in the same PR as the uses: change."*
Useful correction from #266: the lockfile does **not** record
reusable-workflow calls (0 such entries in `rsr-template-repo`'s lock),
so the `slsa-generator` line is not drift and needs no action.
- `E2E Validation / Structural E2E` — `tests/e2e.sh:68` asserts a
`LEVEL-STATUS.md` containing rows `| 1 |`…`| 12 |`; the repo has
`LEVEL-STATUS.adoc`, whose tables are a version matrix and a feature
matrix and do not enumerate levels at all. Renaming in the test would
turn one honest failure into twelve fake-compliant ones, so the choice
(doc gains a level table, or the check retargets at the ranges the doc
declares) is a spec decision, not a CI fix.
- `Smoke test`, `Cargo build + test (typed-wasm-verify)`, `Hypatia
neurosymbolic scan`, `SonarCloud Code Analysis` — pre-existing content
reds, now genuinely executing.
- `.machine_readable/agent_instructions/` — the other name this mandate
retires (`→ bot_directives`). `bot_directives/` already exists here with
three byte-identical files and a README differing by one explanatory
bullet, so this is a question about whether the two dirs mean the same
thing in this repo, not a rename. Filed rather than guessed.
- `ubuntu-latest` → Ubuntu 26 lands **2026-10-19**, 9 days out; every
job here carries the notice. The estate-wide decision is still pending
and this PR does not pre-empt it.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
hyperpolymath added a commit that referenced this pull request Oct 14, 2026
…k drift from #268) (#270)

## Summary

**E2E Validation** and **Static Analysis Gate** have been
`startup_failure` on `main` since `554345c` (#268). Dependabot bumped
four workflow refs without refreshing `.github/workflows/actions.lock`.
That is the same failure #266 fixed two commits earlier. A ref the lock
doesn't list is one the organisation doesn't allow, so the workflow is
rejected before any job runs.

Getting the workflows to start exposes failures that have been sitting
on `main` the whole time. The tests point at files that have since
moved. This PR fixes both. It's the bottom of the PWL-01 stack (the
grade-kernel PR stacks on it), but it stands alone.

## Changes

- `13d26a5`: reverts exactly #268's four workflow edits, back to the
locked refs: `setup-node` v7.1.0 → v7.0.0, `upload-artifact` v7.0.2 →
v7.0.1, `download-artifact` v8.0.2 → v8.0.1, `setup-ocaml` v3.9.0 →
v3.8.0.
- `145d84b`: points the tests at files that moved:
- `tests/e2e.sh`: `LEVEL-STATUS.md` → `.adoc`. The row pattern changes
from `| N |` (Markdown) to `^|N |` (AsciiDoc), so all 12 rows are
actually checked.
- `tests/aspect/claim-envelope.mjs`: the RSR list and the L11/L12
cross-check now use `LEVEL-STATUS` / `PROOF-NEEDS` / `TEST-NEEDS`
`.adoc`.
- `tests/aspect/security-envelope.mjs`: `.well-known/security.txt` →
`www/.well-known/security.txt`.
  - `README.adoc`, `ROADMAP.adoc`: `docs/WHITEPAPER.md` → `.adoc`.

## RSR Quality Checklist

### Required

- [x] Tests pass (see Testing)
- [x] Code is formatted
- [x] Linter is clean
- [x] No banned language patterns
- [x] No `unsafe` blocks without `// SAFETY:` comments
- [x] No banned functions
- [x] SPDX license headers present on all new/modified source files
- [x] No secrets, credentials, or `.env` files included

### As Applicable

- [ ] `.machine_readable/descriptiles/STATE.a2ml` updated: no
project-state change
- [ ] `CHANGELOG` or release notes updated: CI-only fix

## Testing

Run locally on this branch, with the same commands the jobs run:

```
E2E_BUILD=0 bash tests/e2e.sh          47 passed, 0 failed, 7 skipped   (main: 35 passed, 1 failed)
node tests/aspect/claim-envelope.mjs    57 passed, 0 failed, 0 skipped   (main: 51 passed, 5 failed)
node tests/aspect/security-envelope.mjs 10 passed, 0 failed, 0 skipped   (main:  9 passed, 1 failed)
node tests/proof/regression.mjs        141 passed, 0 failed, 1 skipped
```

**Not fixed here (owner decision):** recurrence. The next grouped
Dependabot actions bump will drift again unless the lock is regenerated
alongside it. That needs `gh actions-lock` plus the org allow-list
publish, or a Dependabot rule holding the `actions` group until the lock
is refreshed. Other drifts are left alone because #268 didn't cause
them. One example: `push-email-notify.yml` uses `smtp-notify-action`
v0.5.0 while the lock says v0.3.0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Y6FD63UrriRBHqmhhEca9t

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Y6FD63UrriRBHqmhhEca9t)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
hyperpolymath pushed a commit that referenced this pull request Oct 14, 2026
governance / Actions lockfile verify fails on main and on every PR:

    workflow uses ref "v0.5.0" but lockfile pins "v0.3.0"
    (.github/workflows/push-email-notify.yml)

Dependabot #265 bumped hyperpolymath/smtp-notify-action v0.3.0 -> v0.5.0
without a lock refresh, the same drift #266 and #270 reverted elsewhere
(#270 left this one alone because #268 did not cause it).  Back to the
locked v0.3.0; recurrence needs the lock refreshed with the bump (owner
decision, as in #270).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6FD63UrriRBHqmhhEca9t
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant