Skip to content

Add a compatibility record model, environment probe and CI check - #18

Merged
P-R-N-D merged 64 commits into
mainfrom
claude/project-thread-6xwx6u
Oct 12, 2026
Merged

P-R-N-D merged 64 commits into
mainfrom
claude/project-thread-6xwx6u

Conversation

@P-R-N-D

@P-R-N-D P-R-N-D commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Summary

Before: the repository had no shared way to say where Claus is expected to run or what a run actually observed. CI checked one Linux x64 lane, and the only record of platform support was the lane table in docs/DEPENDENCY-STRATEGY.md.

After: a compatibility record model describes a profile (what a lane promises) and an environment result (what one run observed) as layers from the outermost host through any guest and container to the workload, with every fact marked observed, inferred, declared, unknown, or not-applicable. A standard-library validator checks records against the schema subset it implements and against semantic rules (OS, CPU, and ABI contradictions, emulation declared as native, CPU fallback declared as accelerator success, stage results, the rootless policy and its exceptions, and support claims against the lane table). A read-only collector writes this runner's record, and one new CI job collects and validates it.

The privilege policy passes only on measured facts (observed or inferred). A declared or unknown value fails a required rule, and an exception can cover only a measured violation inside its rule's scope. A support claim must name a lane of the DEPENDENCY-STRATEGY.md table, match that lane's OS family, architecture, C library and WSL2 layer, meet the version bounds the table gives its status (Linux x86-64 glibc is Checked for glibc 2.39 or newer and CPython 3.12.13 or newer; macOS Apple Silicon is statically resolvable for macOS 15 or newer and not supported for macOS 14 or older), and copy the lane's status only for the backend lock's interpreter (CPython 3.12 with the GIL); any other interpreter, or versions below every bound, records not-assessed. A GitHub Actions record is linked to a profile by its measured workflow ref and job.

This is PR B of the sequence agreed in the project thread (A, B, C, D). Related: #17

Changes / Scope

  • tests/compatibility/schema.json: the record schema, version 1, for profiles and results. It includes workload.isolation.pid_namespace, a profile's accelerator_backend, the real and saved user IDs, and permitted capabilities.
  • tests/compatibility/profiles/linux-x64-gha.json: a static profile for the GitHub-hosted Linux x64 lane. It declares what its job can observe, including a GitHub-hosted runner and the runner's glibc 2.39, and makes no measured claim.
  • tests/compatibility/fixtures/*.json: 9 synthetic fixtures, each marked synthetic: a Linux x86-64 VM, Windows on Arm native, WSL2 on Arm with rootless Podman, a VM with Quadlet-managed Podman and systemd, and Kubernetes on arm64 with CUDA.
  • scripts/ci/validate_compatibility.py: the validator (standard library only).
  • scripts/ci/collect_environment.py: the environment collector (standard library only).
  • scripts/ci/tests/test_compatibility_schema.py (95 tests) and scripts/ci/tests/test_environment_probe.py (72 tests): unittest, no Django initialization.
  • .github/workflows/ci.yml: one new job, Compatibility / Schema & Probe / Linux x64. It validates the record even after a probe failure and uploads it as linux-x64-gha.result.json, kept for 90 days. The six existing jobs are unchanged.
  • docs/CI-MATRIX.md (new), and updates to TESTING.md, SECURITY-ARCHITECTURE.md (the CI section and the CI sentence of the Secrets hygiene bullet), SECURITY-REVIEW.md (the CI trigger row and a check of compatibility policy exceptions), CODE-REVIEW.md, CONTEXT.md, DEPENDENCY-STRATEGY.md (how profiles copy a lane's status, and the version bounds of a status, with the Linux x86-64 glibc Checked cell now naming glibc 2.39 or newer and CPython 3.12.13 or newer), and both READMEs (one link each).

Out of scope, by request: running Python 3.15 or 3.15t, RHEL, Windows, ARM64, or WSL2 lanes; real containers, clusters, GPU, or NPU work; new packages or lock changes; check_test_lock.py (unchanged, still a Linux x86-64 / CPython 3.12.15 tool); backend/pytest.ini and the test parity check (unchanged); Dockerfile, Containerfile, Kubernetes, Helm, systemd, or Quadlet files; required status checks, rulesets, and Issue #15. PR C and PR D are not started.

Implementation Status

N/A for features and SEC-* requirements: no feature status in docs/ARCHITECTURE.md and no SEC-* status changes. The lane table in docs/DEPENDENCY-STRATEGY.md keeps every status it had; no lane is promoted to Checked. The Linux x86-64 glibc row now writes the versions its Checked status covers, the lowest its recorded installs used (glibc 2.39, CPython 3.12.13), so a profile below them, such as RHEL 9 with glibc 2.34, records not-assessed; this narrows what the row covers and adds no evidence. The new CI job's pass means the tooling works on that runner; it is not a support claim for any other platform.

Validation

Head 9d72dd7, base main at fbf173a; GitHub reports the merge state as clean.

  • GitHub Actions run 38202243247 on 9d72dd7: 7 of 7 jobs succeeded. The six existing jobs (Backend / CPython 3.12.15, Backend / pytest, Frontend / Node 24, Frontend / Vitest, E2E / Playwright Chromium dev Next and production Next) are unchanged. The new job's record was a "valid result (0 errors, 0 warnings)" with --require-real, the lane table and the linux-x64-gha profile; the committed profile was valid; the two test files ran 95 and 72 tests, OK, including the test that runs the collector's Node expression on the job's real node (v24.21.0); and it uploaded compatibility-linux-x64-attempt-1 (2,997 bytes, kept 90 days). The earlier fix commits a80178c, c1c0674, ce78b64, 5c83cb5, 840c429, e3a7637, e71b193, 550c8dc, 8caccf4, fb5ff97, ccc35e6, e3b4037, a91839d (run 38142834378), aee6794 (run 38144283810), f63fe18 (run 38145113024), 8ec100e (run 38146152994), 1b37fe7 (run 38147612330), 29270a1 (run 38148616077), e397278 (run 38150962215), 01f26d0 (run 38153161677), 75f014a (run 38155428102), bb2020b (run 38159801448), 5dfa093 (run 38160680573), 904dec9 (run 38163470842), 3c84fab (run 38166698760), cb58456 (run 38168871376), fb45b68 (run 38170596893), b132fe5 (run 38171374932), 13cfade (run 38173233255), 7785ca5 (run 38174432720), 379da1d (run 38175427934), 3643d53 (run 38177557338), a112004 (run 38178579353), f6dfd64 (run 38179466431), 004f43a (run 38180086611), d680250 (run 38180773681), 3de33e0 (run 38181414381), d6d266e (run 38184079757), 7dbc25e (run 38186610739), d75c622 (run 38187955921), 149f0a6 (run 38188776411), 21cc4a8 (run 38189540879), 27f81c2 (run 38190328473), 1802cb6 (run 38192212801), 3bf8ba4 (run 38194115122, with 2c7e857), 70496b1 (run 38195707907), 19ea96a (run 38197704942), 9813f3a (run 38199280792) and 4c03f84 (run 38200906929) also ran 7 of 7 green; the run for 16a19c2 was cancelled when fb45b68 was pushed.
  • Locally on CPython 3.12.15 and 3.13.16, on 9d72dd7: python -I -B scripts/ci/tests/test_compatibility_schema.py ran 95 tests, OK, and python -I -B scripts/ci/tests/test_environment_probe.py ran 72 tests, OK (the real-node test ran against Node 22.22.0); both files also pass on CPython 3.11. scripts/ci/tests/test_check_test_lock.py (unchanged) ran 49 tests, OK, on 3.12.15; on 3.13.16 it fails because that interpreter does not hold the test lock, as the check intends.
  • python -I scripts/ci/validate_compatibility.py --dependency-strategy docs/DEPENDENCY-STRATEGY.md --profile <fixture>.profile.json <fixture>.result.json, for each fixture pair: valid. Applied exceptions are listed as policy-exception-applied warnings.
  • The collector run on this sandbox (a root container, not GitHub Actions) against linux-x64-gha fails as intended: profile-link, profile-mismatch and profile-unverified (the runner), app-non-root (superuser, real and saved user ID 0), no-privileged-container, minimal-devices and minimal-capabilities (effective and permitted), with an engine-rootless warning.
  • Schema parity, scratch check, on the final tree: on 6,000 mutated records, including a real local record, the validator's structural verdict agreed with jsonschema 4.26.0 (Draft 2020-12) every time. jsonschema was already installed in the sandbox's system Python and is not added to the repository.
  • The 8 first-round Codex threads, re-run on the final tree with a scratch script: each original reproduction still fails as it should (profile-mismatch, abi-contradiction, ci-evidence, unknown AppArmor state, evidence-source, platform not recorded, stage-fields).
  • Repository checks: git diff --check clean; the added lines have no em dashes. (DEPENDENCY-STRATEGY.md keeps the em dashes it already had.)
  • Django check and Playwright: not run locally, and not applicable. No backend, frontend or UI file changed; CI ran both.

Reviews (the author's own review is a self-assessment; agent reviews are evidence, not approval; no human has reviewed this PR yet):

  • First round and re-review on earlier heads: see the commit history. Their findings were fixed in d3f1975, f88b36a, 01fe600, b778572, b612554 and 1ac72cb.
  • Codex review: 8 P2 comments, fixed in d3f1975, resolved, and rechecked on the final tree (27f81c2). A later Codex review of 5c83cb5 left 3 P2 comments, fixed in 840c429, answered and resolved (see below). A Codex code review of e71b193 left 4 P2 comments, fixed in 550c8dc, answered and resolved (see below). A Codex code review of e3b4037 left 5 P2 comments, fixed in a91839d, answered and resolved (see below). A Codex code review of 29270a1 left 4 P2 comments, fixed in e397278, answered and resolved (see below). A Codex code review of 5dfa093 left 4 P2 comments, fixed in 904dec9 and 3c84fab, answered and resolved (see below). A Codex code review of 3c84fab left 4 P2 comments, fixed in cb58456 (refined in 16a19c2, fb45b68 and b132fe5), answered and resolved (see below). A Codex code review of b132fe5 left 3 P2 comments, fixed in 13cfade (refined in 7785ca5 and 379da1d), answered and resolved (see below). A Codex code review of 379da1d left 3 P2 comments, fixed in 3643d53 (refined in a112004, f6dfd64, 004f43a, d680250 and 3de33e0), answered and resolved (see below). A Codex code review of 3de33e0 left 6 P2 comments, fixed in d6d266e (refined in 7dbc25e, d75c622, 149f0a6, 21cc4a8 and 27f81c2), answered and resolved (see below). A Codex code review of 27f81c2 left 2 P2 and 1 P3 comments, fixed in 1802cb6, answered and resolved (see below). A Codex security review ran once, when this PR was opened on 3eae426, and posted no comment (the Codex summary comment shows it completed). The security reviews requested later, on 5c83cb5, e71b193, e3b4037, 29270a1, 5dfa093, 3c84fab, b132fe5, 379da1d, 3de33e0 and 27f81c2, did not start; every later Codex review is a code review. A Codex code and security review of the final head 9d72dd7 is requested with this update.
  • Pre-merge review on 1ac72cb (separate agents for the schema and validator, the Rootless policy, the collector, CI, PR C readiness, negative tests and the docs; each finding then checked by a separate verify pass): 46 findings. 1 P1 and 2 P2 blocked merge: support claims bound only to the OS/arch/libc row (a 3.15t or RHEL profile could claim checked), declared facts satisfying the least-privilege rules, and the policy failing open off Linux. 15 more P2 (6 code, 9 test gaps) and 25 P3; 2 known limits; 1 refuted. All P1 and P2 are fixed in 417e7ad, 9653380 and a80178c; 24 P3 are fixed in code or docs, and one (the engine rule across layers) is left open, listed under Notes.
  • Independent checks of each head from a80178c to 27f81c2 (42 checks, by one to four agents each, some with a separate verify pass): no P0 or P1, and four P2, all fixed: plain() still showed a token when the echo escaped the character before it (found on 550c8dc, fixed in 8caccf4); on e397278, no test pinned the lock or the clear before the node wait, and none ran the new timed wait (closed in 01f26d0); and on 904dec9, no test passed a profile document under a looser schema (fixed in 3c84fab). Their P3s were fixed in a following commit or left as optional; the fix commits' messages describe each fix, and the ones left that matter are listed under Notes.
  • Codex review of 5c83cb5: 3 P2, all fixed in 840c429. (1) The ABI tag's C library was not compared with the libc fact, so a glibc record with a musl SOABI was valid; it is now an abi-contradiction. (2) A file name starting with :: could start a workflow command, because plain() broke up only ##[; :: is now printed as : :. The committed CI passes only $RUNNER_TEMP and tests/ paths, so this was not reachable there. (3) The collector reported a cgroup as delegated without write access to cgroup.subtree_control; it now requires it. That fact is informational; no policy rule reads it.
  • Codex review of e71b193: 4 P2, each reproduced and fixed in 550c8dc. (1) A secret-shaped value that also broke the schema was printed in full by the structural findings, which run before the secret check; plain(), which every printed line and summary cell goes through, now replaces such a value and the rest of its line with a note naming the shape. (2) load_json read a whole file before the 2 MiB check; it now reads at most the limit plus one byte. (3) An approved exception's date was checked for its digit shape only, so 2026-99-99 let the exception apply; the date must now be a day that exists, or the profile gets a policy-exception error. (4) The collector printed its --output name as given, so a name starting with :: could start a workflow command; the committed CI passes a fixed name, so this was not reachable there. The collector now has a plain() equal to the validator's, and a test keeps the two equal.
  • Codex review of e3b4037: 5 P2, each reproduced and fixed in a91839d. (1) A documented technology on a layer of another kind was accepted: a VM named podman, a container named kvm, a WSL2 layer named docker. Each documented technology now makes one layer kind (hypervisors a vm, engines a container, wsl2 a wsl2), and any other pairing is a layer-structure error; an unlisted technology still only warns. (2) The AWS access key ID shape ended at \b, so a key followed by _ or a lowercase letter was printed and written; it now ends where the key's own alphabet (uppercase letters and digits) ends, in both scripts. (3) generated_at was checked for its shape only, so 2026-99-99T99:99:99Z passed; it must now be a valid date and time, or the result gets a synthetic-marking error. (4) Support claims read the C library only from a libc fact starting with glibc , so a musl profile with no ABI tag could not claim its lane; a libc fact starting with musl is read too. (5) The collector took Node's ppc64 byte order from its own Python; it now asks Node (os.endianness()), and a test runs the expression on a real node.
  • Codex review of 29270a1: 4 P2, each reproduced and fixed in e397278. (1) The run fields and commit SHAs of a GitHub Actions record only had to be known, so a declared, unevidenced tested_sha or event_sha passed; each run field (event, repository, run_id, run_attempt, run_url, job) and each SHA must now be observed or inferred, or ci-evidence reports it. (2) A known workflow_ref outside the record's repository fell back to comparing the job alone, so a record from another repository with a matching job name was linked to the profile; it is now a profile-link error, and only an unknown ref falls back. (3) The node timer could signal a process group after the main thread had reaped node, and so possibly a reused process ID; the timer now signals only while the read runs, the main thread ends that under a lock before it reaps node, and the wait after the read has its own deadline. (4) A glibc or musl libc fact on a Windows or macOS Python passed whenever the ABI tag named no C library; it is now an abi-contradiction when the innermost layer, the platform tag or the SOABI is Windows, macOS or FreeBSD. Each new test fails on 29270a1. The collector already writes the run fields and SHAs as observed or inferred, so CI's own record is unaffected.
  • Codex review of 5dfa093: 4 P2, each reproduced and fixed in 904dec9. (1) Slack app-level tokens (xapp-) were not a secret shape, so the collector wrote one given as a stage reason and the validator accepted and printed it; the Slack shape in both scripts now includes them. (2) A GitHub Actions record with run attempt 0 was valid; GitHub numbers attempts from 1, so it is now ci-evidence. (3) A schema given with --schema that accepts a document the rules across fields cannot read ended in a traceback; it now gives exit status 2 with a message, and with the default schema such an exception is still raised (decided on the resolved path since 3c84fab). (4) On Windows, node started with an empty environment can fail to start without SystemRoot, which Windows needs to load side-by-side assemblies; the collector now passes SystemRoot alone there (allowlisted, presence-only, not recorded) and still an empty environment elsewhere. Each new test fails on 5dfa093.
  • Codex review of 3c84fab: 4 P2, each reproduced and fixed in cb58456. (1) A --summary that named the record made the collector append Markdown to the JSON and exit 0, and the validator could append to its own inputs. The collector now checks, after writing the record, whether the summary is the same file (Path.samefile, so a link or another spelling counts) and exits 2 without appending; the validator exits 2 before anything else when the summary is its schema, profile, lane table or a document. (2) argparse printed a rejected argument as given, so --bad=##[error]x or a token-shaped value reached the log; both scripts now write argparse's usage error, program name included, through plain(). (3) An empty PATH was replaced by the default directories when looking for node; now only an unset one is. (4) With the innermost layer's OS family unknown, the platform tag and the SOABI were never compared with each other; for CPython they now are, by OS, and by architecture when the arch fact is unknown or not a listed name. Each new test fails on 3c84fab.
  • Codex review of b132fe5: 3 P2, each reproduced and fixed in 13cfade. (1) A CPython version was read from its first two numbers, so a profile declaring 3.12-not-python kept its checked claim and its tags were compared as 3.12. The version is now read whole, as platform.python_version() writes it (3.12.15, 3.13.0rc1, or 3.14.0a1+ for a build after a release) or as major.minor; any other text is an abi-contradiction at the version and names no lane interpreter. (2) A passed stage whose compute record named no backend that ran (effective null) was valid; it is now a backend-reference error, and on a GPU or NPU request it stays a single accelerator-fallback. (3) With a listed arch fact, an ABI tag for an architecture outside the list, such as vax, was not compared with it; it is now an abi-contradiction. Each new test fails on b132fe5.
  • Codex review of 379da1d: 3 P2, each reproduced and fixed in 3643d53. (1) An AppArmor enabled file holding anything but Y, such as a replaced file with garbage, was recorded as observed False; only Y or N now count, and any other value makes enabled and profile unknown. (2) On Windows, shutil.which("node", path=...) also looks in the current directory first, so a node.exe in the checkout could run although PATH did not name it; node is now looked for in each PATH directory by itself, and an empty entry names none. (3) A run URL ending in /attempts/1 was valid on a record with run_attempt 2; such a URL must now name the record's attempt, while a URL with no attempt still names the run. Each new test fails with its fix reverted.
  • Codex review of 3de33e0: 6 P2, each reproduced and fixed in d6d266e, refined in 7dbc25e. (1) The collector read GITHUB_RUN_ATTEMPT only up to 4 digits; it now reads up to the schema's largest integer, 4294967295, and a larger value is unknown. (2) On Windows shutil.which tries every PATHEXT extension, so a node.bat in an earlier PATH directory ran in place of node.exe; the collector now looks for node.exe alone in each PATH directory, with any CPython version. (3) A finding's path repeated every key above it, and walk() built every child path before visiting any, so a 1.82 MiB document with a 1,000,000-character key over 90,000 values ran out of memory; a path is now built when its value is reached and is bounded (see CI-MATRIX.md), and that document validates in 0.2 seconds. (4) An unreadable evidence-source workflow (a name over the system's limit, or a file that is not UTF-8) ended the validator in a traceback; it is now an evidence-source finding. (5) A lane listed twice in the lane table kept its last row; that is now a usage error. (6) --summary could append to a workflow file that a real profile reads as its evidence source; that is now a usage error. Fixes (3), (4) and (6) close limits recorded for fb5ff97, cb58456 and fb45b68, and (2) closes the CPython 3.11 items recorded for 3643d53 to f6dfd64.
  • Codex review of 27f81c2: 2 P2 and 1 P3, each reproduced and fixed in 1802cb6. (1) An evidence-source job was found only as an unquoted key two spaces in with nothing after the colon, so a valid workflow with four-space indentation, a quoted key or a comment after the key failed evidence-source; the validator now reads the block mapping under an unindented jobs: line at the indentation of its first key. (2) A --summary naming a workflow file that a real profile names but that is missing was not refused, so the summary created it; the paths are now also compared once resolved. (3) TESTING.md gave an old test total. 47 of 47 Codex threads are resolved.
  • Independent check of 1802cb6: no P0 to P2, 9 P3. Fixed in 3bf8ba4: lines were split at U+0085, U+2028 and U+2029 as well as \n, so a value holding one could hide or invent a job (the reader now splits at \n only); jobs:#c was read as a jobs key (a comment after the colon now needs a blank); a workflow was split into a list of all its lines before the jobs key was found (128 MB of short lines with no jobs key took 22 s; the key is now found with one search, 0.9 s, and lines after it are read one at a time); and the test gaps it listed (quoted and spaced jobs keys, a tab after a colon, a block scalar, a dot in a job name, a summary written elsewhere next to a missing workflow, a summary path that loops). Recorded, not fixed: the reader takes the first of duplicate top-level jobs keys, where PyYAML takes the last; a profile that already fails evidence-source with a workflow path outside .github/workflows/ can make --summary with that path exit 2; on a case-insensitive file system a missing workflow spelled in another case is not recognized; a summary path holding NUL raises from open() when main() is called directly (not reachable from the command line).
  • Another review, relayed by the owner in the project thread, found 2 P2, fixed in 2c7e857. (1) The macOS Apple Silicon row is statically resolvable only for macOS 15 or newer, but the validator accepted either of its statuses whatever the macOS version; a status the table gives for some versions only now needs those versions declared and met, the validator keeps the bounds (STATUS_CONDITIONS), and a test keeps them equal to the table's cells. The Linux x86-64 glibc row's Checked status now covers glibc 2.39 or newer and CPython 3.12.13 or newer releases; the distribution is not part of a status. (2) A GitHub Actions record without a known workflow_ref was linked to a profile by its job alone; workflow_ref is now a measured run field, and a record is linked by its workflow and job only.
  • Independent check of 3bf8ba4: no P0 or P1, 1 P2 and 8 P3. The P2: since 2c7e857 made the workflow ref a required run field, a manual dispatch from a branch whose name holds a character git allows but the collector did not (anything but letters, digits and ._/+-, such as fix/issue#12, it's or a non-ASCII letter) failed ci-evidence and profile-link, because the collector recorded its workflow ref as unknown. Fixed in 70496b1: the collector records any ref git allows, without the characters the validator refuses in any value, up to the schema's 200 characters. Also fixed there: version bounds read a part with a leading zero (glibc 2.039 met the Checked bound); a record missing only its repository was reported at workflow_ref, with a message naming both fields; the comments and CI-MATRIX.md said workflow lines split at \n only, while the file is read with universal newlines, so \r\n and \r end a line too; a DEPENDENCY-STRATEGY.md sentence said no recorded install used a version above the bounds; and test gaps (a lane without bounds, a repository name that only begins the workflow ref's, a ref holding @, more than one blank after glibc, long version parts, a bound name the validator cannot read). Recorded, not fixed: no test tells the reader's strip(" \t") from strip() or covers a \r in the text given to it, which comes with universal newlines.
  • Independent check of 70496b1: no P0 or P2, 1 P1 and 2 P3. The P1: 70496b1 recorded the workflow ref of a manual dispatch from a branch such as fix/issue#12, but the dispatch still failed ci-evidence. actions/checkout checks a branch out by its name, so .git/HEAD reads ref: refs/heads/fix/issue#12, and the collector followed a HEAD ref only when it was made of letters, digits and ._/+-, so the tested commit was unknown; the 70496b1 test used a detached HEAD and missed it. The cause predates 70496b1. Fixed in 19ea96a: GITHUB_REF and the ref .git/HEAD names take the characters the workflow ref takes, within 200 characters; a HEAD ref git refuses (.., an empty part, a part that starts with .) is still not followed; HEAD is stripped of ASCII blanks only; and a loose ref whose name is not ASCII is cited as git:.git/HEAD, because an evidence reference is printable ASCII. A real-git repro now records the ref, workflow ref and commit for main, fix/issue#12, a non-ASCII branch and one ending in U+3000, with no ci-evidence or profile-link finding; on 70496b1 the fix/issue#12 dispatch gave ci-evidence at /run/revision/tested_sha. The P3s, also fixed there: the ref character class was copied by hand from CONTROL (it is now built from CONTROL, and a test compares it with the validator's on every code point), and test gaps (the bound-name test did not show each bound reads a declared version, no four-digit version part was tested, and the dispatch loop asserted outside subTest).
  • Independent check of 19ea96a: no P0 to P2, 5 P3 (3 introduced, 2 older), all fixed in 9813f3a. .git/HEAD and packed-refs were decoded with replace, so a branch name holding a byte that is not UTF-8 read as U+FFFD and could resolve to another branch's commit, recorded as observed (they are now decoded with surrogateescape, which the ref check refuses); no test exercised the HEAD ref's character and length check (tests now refuse a space, a control or bidi character, more than 200 characters, and git's other forms even when packed-refs lists the ref); the docs said every ref git refuses is refused, while @{ and a part ending with .lock or . were followed (they are now refused, a trailing . also because Windows drops it from a file name); packed-refs was split with splitlines(), which also ends a line at U+2028 and other characters a ref may hold (now at \n only); and summary cells did not escape &, so a value could write an HTML entity (both scripts now escape it).
  • Independent check of 9813f3a: no P0 to P2, 5 P3. Fixed in 4c03f84: no test decoded a packed-refs line that is not UTF-8, so decoding packed-refs alone with replace (reading the branch x/a followed by byte 0x80 as x/a followed by U+FFFD, the checked-out branch) or strict went unnoticed (tests added); refs holding {, @ or .lock elsewhere had no positive case (added); CI-MATRIX.md did not say that a dispatch whose commit alone the collector cannot record fails ci-evidence only, and its read list left out the character and length conditions (both fixed); and, older, under a file system encoding other than UTF-8 a ref that is not ASCII could open the loose ref of another branch, because the path is written in that encoding while git names the file in UTF-8 (such a ref is now left unknown). Kept as it is: a part ending with ., which git allows, is refused on every system, so the commit of such a dispatch is unknown (fails closed; listed under Notes).
  • Independent check of 4c03f84: no P0 to P2, 3 P3, all fixed in 9d72dd7. The test held the non-ASCII ref in packed-refs only, so a guard moved after the loose read still passed (the test now holds a newer loose ref, with its assertions inside subTest); under a file system encoding other than UTF-8, a GITHUB_REF or GITHUB_WORKFLOW_REF that is not ASCII was recorded as other characters, since Python decodes the environment in that encoding (such a value is now unknown, and the docs apply the encoding condition to the ref and workflow ref as well as the commit); and, older, any loose ref that could not be read fell back to packed-refs, so a FIFO or a symbolic ref in its place recorded the older packed commit as observed (only a loose ref that does not exist now falls back).
  • Independent check of 9d72dd7, the final head: no P0 to P2, 2 P3, recorded and not fixed (see Notes). Each of the three 4c03f84 findings is fixed. With real git, a FIFO, a text symbolic ref and an unreadable loose ref now give unknown where 4c03f84 recorded the older packed commit, and every layout git produces (loose only, packed only, a newer loose over a packed ref, a missing parent directory, a directory holding other loose refs) still resolves to git rev-parse HEAD. Under an ISO-8859-1 locale built for the check, a non-ASCII GITHUB_REF and GITHUB_WORKFLOW_REF are unknown where 4c03f84 recorded the two UTF-8 bytes of the accented letter as two Latin-1 characters, and the validator accepts the new facts. Against 4c03f84's collector the new tests fail 5 cases, each new behaviour, and every mutant of the change but one fails them.
  • Every fix was reverted or mutated in a scratch copy, and a test fails on it: each fix in c1c0674, ce78b64, 5c83cb5, 840c429, e3a7637, e71b193, 550c8dc and 8caccf4, and the linear-time search in fb5ff97; the four JWT counts and a leading \b on each of the six shapes (ccc35e6, e3b4037); 16 of 17 mutants for a91839d and aee6794; 6 of 6 for f63fe18; 12 of 12 for 8ec100e; 11 of 11 for 1b37fe7; 4 of 4 for 29270a1; 8 of 8 for e397278; 11 of 11 for 01f26d0; 12 of 13 collector mutants and 7 of 7 libc mutants for 75f014a, whose tests pass, without hanging, on 2 correct variants; the FreeBSD mutant for bb2020b; 6 of 6 for 5dfa093; 9 of 9 for 904dec9 and 3c84fab; 17 of 18 for cb58456, 16a19c2 and fb45b68; 20 of 20 for 13cfade, 7785ca5 and 379da1d; the three fixes of 3643d53; all but one for a112004, and two more with the tests of f6dfd64; 3 of 3 for d680250 and 3de33e0; 10 of 10 for d6d266e; 6 of 6 for 7dbc25e, each within seconds; 2 of 2 for d75c622; and for 21cc4a8, the key limit checked before escaping, no key limit, no depth limit, an elision that keeps 18 keys, a 7-digit digest and no digest, each within a second and with a short message, and the chain block alone fails on the four bound mutants; 16 of 16 for 1802cb6; 24 of 25 for 2c7e857 (the survivor reads 3.12 as a three-part CPython version, which the bound already refuses as too short); 10 of 10 for 3bf8ba4; 12 of 12 for 70496b1; 15 of 15 for 19ea96a; 12 of 12 for 9813f3a; 10 of 11 for 4c03f84; and 9 of 10 for 9d72dd7. Five survivors are equivalent to the code they change: the kind rule skipping the host (the host rule before it already refuses any technology on a host), elif for if in the tag comparison (its two branches need different and equal OSes), the loose read in 4c03f84 skipped on the condition the guard before it already returns on, a second copy of that guard before packed-refs in 9d72dd7, and . dropped from the default directories on every system (os.defpath lists no . outside Windows). The 13th 75f014a mutant, a wait under the lock with the flag cleared after it, leaves a race too narrow for the test to reach. The 21 mutants of the earlier validator mutation script still fail a test.

Security Impact

The change touches the CI trust boundary (.github/workflows/ci.yml) and adds a script that reads runner state. The CODE-REVIEW.md step 9 triggers that match are CI workflows, environment variable handling, logging (the job summary and the uploaded record), and authorization policy (the rootless rules and their exceptions).

  • The new job:

    • runs on ubuntu-24.04 with the workflow's read-only contents permission and persist-credentials: false;
    • pins its actions by commit SHA, at the same versions as the existing jobs;
    • sets UV_NO_CONFIG and gives setup-uv a Python version;
    • installs no package, uses no secret and does not use sudo.
  • The collector reads:

    • /proc (including /proc/self/status for the real, effective and saved user IDs and capabilities, and /proc/sys/kernel/osrelease), /sys, /etc/os-release or /usr/lib/os-release, and two files under /run;
    • whether /.dockerenv, /.singularity.d and /run/systemd/system exist;
    • the first 20 bytes of the interpreter (its ELF header);
    • the checkout's .git/HEAD and the ref it names (loose or in packed-refs), and only the head and base SHAs from the event file;
    • a fixed allowlist of environment variable names. For some of them, such as APPTAINER_CONTAINER, it records only that they are present.

    It reads only regular files, so a FIFO cannot block it. It never iterates the environment, uses no network and changes no system state. It runs one subprocess, node -p, looked for in the PATH directories only (on Windows only as node.exe, whatever PATHEXT lists, and not in the current directory, where shutil.which looks first, unless PATH names it, also when PATH is unset), without a shell, with an empty environment (on Windows, SystemRoot alone) and its stderr discarded, reading at most 4 KiB of output and killing its process group after 30 seconds. The timer can signal only while the read runs; the main thread ends that under a lock and only then reaps node, so no signal reaches a reaped, possibly reused, process ID. A descendant that starts a session of its own can hold the read until the job's 10-minute timeout; the node facts are then unknown.

  • The validator rejects records with control characters, lone surrogates, or values shaped like common tokens or private keys; the collector refuses to write such a record. In every line it prints, the validator replaces control characters, does not show a secret-shaped value (it prints a note naming the shape instead of the rest of the line), and writes the runner's legacy ##[ command marker as ## [ and :: as : :; it also escapes Markdown in the summary. The collector prints its lines through the same function, and both scripts write argparse's usage errors, which repeat a rejected argument, through it. So neither a record value nor a file name given on the command line can start a workflow command or put a secret-shaped value in the log or the summary. The shapes are found wherever they start, also after an escape such as \n or ~1, and the AWS key ID shape ends where the key's alphabet does, so a key followed by _ or a lowercase letter is caught too.

  • The validator reads at most 2 MiB and one byte of each document, profile or schema.

  • Neither script appends its summary to a file it was given: the collector exits 2 without appending when --summary is the record it wrote, and the validator exits 2 before reading anything when --summary is its schema, profile, lane table or a document.

  • A policy exception's approval is a recorded claim. The validator checks that it is recorded, approved and in scope, and prints each use as a policy-exception-applied warning; it cannot check who approved it. SECURITY-REVIEW.md now asks a reviewer to check that the reference points to a maintainer's decision outside the change itself.

  • The collector records GITHUB_WORKFLOW_REF, GITHUB_REF and the ref .git/HEAD names with any ref git allows, so these values can hold characters such as #, quotes or non-ASCII letters. The HEAD ref is read as a path under .git only when it is UTF-8 and has no .. or @{ and no part that is empty, starts with ., or ends with .lock or ., and, when it is not ASCII, only when the file system encoding is UTF-8, in which git names the file; packed-refs is read only when the loose ref does not exist, so a loose ref that cannot be read or holds no commit ID gives unknown, not an older packed commit. A GITHUB_REF or GITHUB_WORKFLOW_REF that is not ASCII is recorded only under a UTF-8 file system encoding. Summary cells in both scripts escape & as well as Markdown, so a value cannot write an HTML entity. It still excludes control and bidirectional formatting characters and lone surrogates, stays within the schema's 200 characters, and goes through the same secret-shape check before writing; the validator's printed lines and summary handle it as any other value.

  • The uploaded artifact holds the runner record only and is kept for 90 days.

  • As with the lock check, a pull request that changes the collector, the validator or their tests runs them as changed. Such a change needs review in the diff.

Notes / Risks

  • Platforms not run: everything except the GitHub-hosted ubuntu-24.04 x86-64 runner and the local sandbox. That covers RHEL, Windows (native and WSL2), macOS, ARM64, CPython 3.15 and 3.15t, real containers and Kubernetes, and GPU or NPU backends. Their fixtures are synthetic, and the probe tests fix interpreter facts and have run only on Linux x86-64 hosts.
  • Off Linux, the collector has no isolation probe and reads user IDs only on FreeBSD and macOS, so a record from Windows or macOS fails app-non-root (Windows), no-privileged-container, minimal-capabilities and minimal-devices and warns on engine-rootless until such a probe exists. Windows token privileges are not modeled.
  • Container lanes cannot pass the default rootless policy until the engine-side facts are recorded. From inside a container the collector cannot see whether it is privileged or which devices were passed in. It infers rootless or rootful only for Podman, from /run/.containerenv.
  • The rule for a container nothing marks cannot see a container that shares the initial PID namespace and leaves no marker, or one inside a WSL2 distribution that leaves no marker.
  • Emulation limits: full-system emulation is not visible from inside the guest; user-mode translators that present the emulated architecture as the kernel's (FEX, box64) were not evaluated; a kernel without /proc/sys/kernel/arch (before 6.1) gives an unknown execution mode, unless /proc/cpuinfo shows another architecture family, which is recorded as emulated (inferred).
  • Policy exceptions have no approver or expiry field, and device exceptions are not scoped per layer. A status is bound to the versions its table cell names, not to the Linux distribution, so a profile on another distribution with glibc 2.39 or newer and CPython 3.12.13 or newer copies the Checked status of Linux x86-64 glibc, and newer glibc and CPython 3.12 releases are taken to keep binary compatibility untested. A statically resolvable Linux row has no glibc bound, though its wheels' manylinux tags set one. The collector does not record the macOS version, so a real macOS profile cannot have a status checked against its run until it does. The engine rule does not yet compare layers (an inner rootless engine under a rootful outer one).
  • Whether sudo or the docker group is available on the runner is not observed.
  • A GitHub Actions record must carry workflow_ref and repository, measured; a record from a runner that does not set GITHUB_WORKFLOW_REF fails as CI evidence and links to no profile. The collector records the ref and the workflow ref for any ref git allows, so a manual dispatch from any branch git allows links, unless its workflow ref holds a character the validator refuses in any value, is longer than 200 characters, or, under a file system encoding other than UTF-8, is not ASCII. It records the checked-out commit too, except for a ref with a part that ends with . (refused on every system because Windows drops it from a file name), or one whose loose ref exists but cannot be read or holds no commit ID; such a dispatch fails ci-evidence. The check that the ref lies in the record's repository and names a workflow file and a ref runs only with --profile.
  • A libc fact names glibc or musl only when it starts with that name (in any ASCII letter case, with no letter after it; a digit or another number may follow). Ubuntu GLIBC 2.39 or GNU libc 2.39 names another C library, so it contradicts a glibc lane and a gnu ABI tag. This fails closed, and the collector writes only glibc and a version. A glibc or musl fact for a Windows, macOS or FreeBSD Python is an abi-contradiction.
  • On an x32 interpreter, sys.implementation._multiarch and the SOABI (x86_64-linux-gnux32) name x86_64, so the collector records x86_64 with 32-bit pointers and the validator rejects its own record. x32 is no lane and this fails closed; the ELF fallback alone checks the class.
  • test_the_expression_runs_on_a_real_node runs the first node on PATH with an empty environment. A version-manager shim that needs HOME fails it on a developer machine; CI uses the real binary from setup-node.
  • The schema pattern check looks for the forms CI-MATRIX.md lists only. Other differences between Python and ECMA-262 remain, such as how . treats line terminators, so a new pattern still needs review.
  • From the checks of cb58456 and fb45b68, recorded and not fixed, each older than cb58456 or outside what it changed: with the arch fact unknown, the pointer size is not compared with the platform tag or the SOABI; with the innermost layer's OS family unknown, the Node platform is not compared with the Python tags; and --help prints the program name, taken from how the script was started, without plain(), though it repeats no argument value.
  • From the check of 13cfade, recorded and not fixed, both older than this PR's last changes: TESTING.md has one non-ASCII character (a multiplication sign) from main before this PR, and an x32 record is rejected, as noted above. From the check of 379da1d, optional and older than it: no accepted version has a two-digit pre-release number, so a pattern that allows one digit there, or a separate rule for b, passes the tests; no CPython release has had one.
  • From the checks of 3643d53 to f6dfd64, recorded and not fixed: the job puts $GITHUB_WORKSPACE/.venv/bin, inside the checkout, first on PATH before the collector runs, which crosses no boundary under pull_request, where the PR's own code runs anyway; and a quoted PATH entry such as "C:\nodejs" finds no node, as before.
  • From the check of d6d266e, recorded and not fixed: a document under 2 MiB with about 200,000 findings under 14 nested keys of 127 characters prints 330 MiB of finding lines (708 MiB of memory, 28 seconds), because each finding gets a line; the path in each line is now bounded, and only a document given to the validator reaches this (in CI, the collector's record and the committed profiles). Capping the findings printed per document would change the output, so it is left for a later change. On POSIX, shutil.which reads an empty PATH entry as the current directory and returns a bare node, which Popen with an empty environment then looks up in os.defpath, so the node found and the node run can differ; this is older than d6d266e, and CI's PATH has no empty entry.
  • From the check of 9d72dd7, recorded and not fixed: a symbolic ref written as a symlink (only with git's deprecated core.preferSymlinkRefs) dangles when the collector opens it, so a branch HEAD names that is such a symlink, with an older packed-refs entry of the same name, still records the packed commit (as before 9d72dd7; actions/checkout does not make this layout); no test pins the default of read_git's absent_ok; an empty directory at a loose ref's path gives unknown where git reads packed-refs (no git command leaves that layout); and the WORKFLOW_REF comment does not repeat the encoding condition.
  • From the check of 7dbc25e, recorded and not fixed: the root and an empty key "" both print as / (older than this PR); and evidence_workflows also names the workflow files of a profile whose structure is invalid, so the summary guard only refuses more.
  • The message of commit 8a5fb95 says the collector runs no subprocess. That is wrong: it runs node -p as described above, which --no-node disables.

Interfaces for PR C:

  • the record schema, version 1, and its fact model, with measured meaning observed or inferred;
  • the validator options --profile, --require-real, --dependency-strategy, --no-policy (which now warns policy-skipped) and --summary (refused with exit 2 when it names the schema, the profile, the lane table or a document), and exit codes 0, 1 and 2;
  • the collector options --profile-id, --stage*, --no-node and --summary (exit 2 without appending when it names the record);
  • the stage names and their order, and the overall verdict;
  • the rootless rule IDs, RULE_SCOPE (exception scope keys per rule), and the exception fields;
  • LANE_TARGETS (kept equal to the lane table by a test) and LOCK_INTERPRETER; a new lane is a new table row plus a LANE_TARGETS entry;
  • TECHNOLOGY_KINDS, the layer kind each documented technology makes; a new hypervisor or engine is a new entry, and a test keeps the collector's tables bound to it;
  • evidence_sources matched as (workflow, job) against run.ci.workflow_ref and run.ci.job; a record without a measured workflow ref, or with one outside its repository or naming no workflow file or ref, links to no profile; and accelerator_backend;
  • STATUS_CONDITIONS, the version bounds of a lane status (kept equal to the table's cells by a test): a new bound is written in the cell as <name> <version> or newer or or older and added there; a macOS lane needs the collector to record the macOS version before a real profile can claim its status;
  • a GitHub Actions record's run fields and commit SHAs, which must be observed or inferred, with a run attempt of 1 or more and a run URL for this run (a URL ending in /attempts/<n>, with at most 10 digits, names that attempt, which must be run_attempt);
  • on Windows, the collector looks for node only as node.exe in the PATH directories, with any CPython version, so a Windows lane puts node.exe itself on PATH (a node.cmd or node.bat shim is not run, and the node facts are then unknown);
  • a CPython version written as platform.python_version() gives it, or as major.minor, with no leading zero in any number; a lane's interpreter is read from it whole.

Open for PR C: recording engine_mode, privileged and devices from the engine side, a WSL2 lane record, isolation probes for systems other than Linux, and a stage interface for accelerator lanes.

🤖 Generated with Claude Code

https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

The schema describes a profile (what a lane promises) and an
environment result (what one run observed) as layered host, guest,
container and workload records, each fact carrying observed, inferred,
declared, unknown or not-applicable provenance. The validator is a
standard-library interpreter for the schema keywords the schema uses,
plus semantic rules for layer structure, OS, CPU and ABI
contradictions, emulation, accelerator fallback, stage ordering and
results, the rootless policy and its exceptions, support claims
against the dependency strategy lane table, and profile linkage.

Fixtures are marked synthetic and cover a Linux x86-64 VM, Windows on
Arm native, WSL2 with rootless Podman, a VM with Quadlet-managed
Podman and Kubernetes on arm64 with CUDA. The profile for the GitHub
hosted Linux x64 lane is static and makes no measured claim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector writes one environment result from the running
interpreter and the Linux /proc and /sys views it can read without
privileges. It runs no subprocess, makes no network call and changes
no system state. It reads only an allowlist of environment variable
names, records presence only for a few more, and never copies host
names, user names, addresses or tokens into the record. Probe errors
are recorded and fail the detect stage instead of being dropped.

Execution mode compares the kernel's architecture with the build
architecture of the interpreter. The container, user namespace,
cgroup, capability and service manager facts are observed from the
current process only; nothing is claimed about the physical host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
One new job, "Compatibility / Schema & Probe / Linux x64", runs the
validator and collector unit tests, collects this runner's environment
record, validates it against the GitHub hosted Linux x64 profile and
uploads the record with a seven day retention. It runs on
ubuntu-24.04 with read-only contents permission, installs nothing and
uses the same pinned actions as the existing jobs. The six existing
jobs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
docs/CI-MATRIX.md describes the record model, the provenance rules,
the stage and support statuses, the rootless defaults and exceptions,
what the validator checks and does not check, and what the GitHub
hosted lane can and cannot observe. TESTING.md lists the new tests,
commands and job; the security documents add the job to the CI
section and the review triggers; CONTEXT.md routes compatibility work
to the new document; DEPENDENCY-STRATEGY.md keeps its lane table as
the only support record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T04:35:36.469716Z 9d72dd7 Manual request
🔒 Security Review ✅ Completed 2026-10-11T02:38:51.687981Z 3eae426 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3eae4263f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/collect_environment.py Resolved
Comment thread tests/compatibility/schema.json Resolved
Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
claude added 12 commits October 11, 2026 03:16
Validator:
- Reject emulation declared native anywhere inside an emulated layer, not
  only next to it, and check the SOABI's architecture and the Windows and
  macOS platform tags.
- Allow not-applicable only for the security and performance stages, so
  overall passed needs detect through correctness to pass; require a log
  or artifact reference for a real stage that ran, and fail detect when
  probes failed.
- Add accelerator_backend to profiles: required execute, correctness and
  performance stages must run on it, so a CPU-only run cannot satisfy a
  GPU or NPU lane.
- Compare every fact a profile declares, including backends by ID and
  the accelerator inventory, and ignore fact-shaped data in extensions.
- Apply the container policy rules when the record does not show the
  workload outside a container, and skip minimal-capabilities only off
  Linux.
- Read per-condition lane statuses, require a real profile of a listed
  lane to copy its status, match workflow jobs only under jobs:, reject
  run facts in synthetic results, and escape summary cells.

Collector:
- Record whether the process is in the initial PID namespace, and do not
  take the kernel layer's OS, service manager or CPU from a process that
  may be in a container, or from uname without /proc/sys/kernel/arch.
- Record armv7 on aarch64 as unknown, EC2 metal and WSL 1 as no VM, and
  ppc64 Node on little-endian hosts as ppc64le.
- Refuse secret-shaped stage reasons and evidence, and write no record
  that holds a secret-shaped value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector writes a partial record and exits 1 when a probe fails, so
the validate step now runs unless the run was cancelled, and the
validator and probe tests run as two steps so one failing does not hide
the other. The six other jobs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
CI-MATRIX.md describes the new rules (inherited emulation, which stages
may be not-applicable, stage evidence, accelerator_backend, the hidden
container rule, lane statuses) and the collector's full read list, and
its lanes table names lanes instead of repeating support statuses.
TESTING.md has the new test counts (61 and 48) and CI steps.
SECURITY-ARCHITECTURE.md fixes the job's line numbers and Node source,
CODE-REVIEW.md and SECURITY-REVIEW.md add scripts/ci/ and
tests/compatibility/ to the CI security review trigger, and both
READMEs list CI-MATRIX.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector now records:
- an Apptainer (Singularity) container from /.singularity.d or its
  environment variables, which only say that one is present, since
  Apptainer keeps the initial PID namespace by default;
- os-release and PID 1 as inferred, not observed, when no container
  marker is found, because a chroot or a marker-less container that
  shares the PID namespace looks the same;
- a WSL 2 distribution's os-release and PID 1 from its own PID
  namespace, which WSL 2 gives every distribution, and older WSL 2
  kernels named "-microsoft-standard" (WSL 1 ends in "-Microsoft");
- EC2 Arm VMs as VMs and EC2 metal instances (".metal" product names)
  as not shown to be VMs;
- the container layer's architecture and execution as unknown, since
  its programs need not share one architecture; each runtime keeps
  its own, so an emulated interpreter is still recorded as emulated;
- Node's ppc64 on a little-endian host as an inferred ppc64le;
- the C library only from os.confstr, which only glibc answers,
  instead of a version scanned from the interpreter's executable.

The validator knows the apptainer technology, and does not read a
WSL 2 distribution's own PID namespace as a sign of a hidden container.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
- The OS and architecture in a CPython ABI tag are checked even when
  the version is unknown.
- A job named in a result must be an evidence source in the workflow
  that run.ci.workflow_ref names, when it names one, not only a job of
  that name in any workflow.
- A comment line inside a workflow's jobs mapping no longer ends it.
- Lone surrogates count as control characters, document strings are
  quoted in messages, and every printed line has its control
  characters replaced, so a value cannot start a workflow command line
  or stop the summary from being written.
- New tests cover a CPU stage request under an accelerator profile, a
  required performance stage, an unknown OS family, and file: evidence
  on a stage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
CI-MATRIX.md now says:
- which containers share the initial PID namespace, that WSL 2 gives
  every distribution its own, and that a WSL 2 innermost layer is not
  read as a hidden container;
- that a container layer's CPU facts are unknown and each runtime
  records its own;
- that off Linux a collected record fails minimal-capabilities until
  a probe for that system exists;
- that evidence sources are checked for real profiles, and the job is
  matched within the workflow that run.ci.workflow_ref names;
- that lone surrogates count as control characters.

TESTING.md has the new test counts (62 validator, 49 collector) and
what they cover. A sentence in SECURITY-ARCHITECTURE.md is mended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…to lanes

Validator:
- A required policy rule passes only on observed or inferred facts; a
  declared or unknown value fails it, and an exception covers only a
  measured violation inside its rule's scope (RULE_SCOPE).
- Each applied exception is printed as a policy-exception-applied
  warning, and --no-policy warns policy-skipped.
- A real or saved user ID of 0 and permitted capabilities fail
  app-non-root and minimal-capabilities.
- A support claim must name a lane of the DEPENDENCY-STRATEGY.md table,
  match its OS family, architecture, C library and WSL2 layer, and copy
  its status only for CPython 3.12 with the GIL; other interpreters
  record not-assessed. LANE_TARGETS is kept equal to the table by a test.
- A real profile must be recorded by one of its GitHub Actions evidence
  sources (profile-link), and declared values in a result do not satisfy
  a profile.
- The schema subset rejects another dialect, nested $schema/$id/$defs,
  non-portable patterns and $ref cycles.

Collector:
- Records the real and saved user IDs (Linux, FreeBSD, macOS; Windows
  not-applicable) and permitted capabilities, and reads the kernel
  release from /proc/sys/kernel/osrelease.
- Reads only regular files, so a FIFO cannot block it.
- Bounds node -p to 4 KiB and 30 seconds and kills its process group.
- Infers the container engine only from markers that agree, and the
  engine mode only for Podman.
- Drops the WSL variables from the allowlist.

Tests: 83 validator and 60 collector tests, including the negative cases
for each rule above.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The record is the lane's only machine-readable evidence and is a few KB,
so the artifact is kept for 90 days instead of 7. The file is named
linux-x64-gha.result.json in the collect, validate and upload steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
- CI-MATRIX.md: the measured-only rule, exception scope per rule and the
  limits of a recorded approval, real and saved user IDs and permitted
  capabilities, support claims bound to the lane and the lock's
  interpreter, the collector's node and file bounds, and the limits that
  remain for PR C.
- TESTING.md: 83 and 60 tests and what they cover, and the record name.
- SECURITY-ARCHITECTURE.md: the compatibility job's line range, the
  collector's bounds and the 90-day artifact.
- SECURITY-REVIEW.md: check a new or changed policy exception's approval
  reference, scope and end condition.
- DEPENDENCY-STRATEGY.md: the lane statuses are for the backend lock's
  interpreter, and how the validator binds a profile to its lane.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Fixes from the independent re-review of a80178c (all minor):
- The validator writes "##[" as "## [" in every line it prints, because
  the GitHub Actions runner reads that form as a workflow command
  anywhere in a line.
- The schema subset also rejects the {,n} and possessive quantifiers,
  which Python and ECMA-262 read differently.
- The FreeBSD platform tag (freebsd-14.1-RELEASE-amd64) is parsed by
  both scripts and recorded by the collector.
- The collector reads user IDs only on Linux, FreeBSD and macOS; under
  Cygwin or MSYS2 they are unknown.
- Tests now fail if the macOS and FreeBSD identity check, the stop after
  node's output limit, or the FreeBSD parsing is removed.
- The docs say that a node descendant with its own session can hold the
  read until the job times out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
From the independent review of c1c0674 (both minor):
- The schema subset rejects any escaped letter other than t, n, r, f, v,
  x and u (adding Python's \a, \N and \U) and a character class that
  opens with ] or ^]. CI-MATRIX.md lists the forms and says a new
  pattern still needs review.
- The collector's docstring says it reads user IDs on FreeBSD and macOS
  only, as the code does.
- SECURITY-ARCHITECTURE.md carries the caveat on node descendants that
  start their own session.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Python reads a \u escape of a surrogate as a code point, while ECMA-262
reads it as a UTF-16 code unit and joins a pair with the u flag, so the
same pattern can match different strings. The pattern check now rejects
\uD800 to \uDFFF. TESTING.md named \A as tested, so a case for it is
added, and both docs list the new form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

P-R-N-D commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review
@codex security review

Requesting a code and security re-review of the current head 5c83cb5. The last Codex review was on 3eae426; the changes since then are described in the PR body under Validation and Reviews.


Generated by Claude Code

@P-R-N-D P-R-N-D self-assigned this Oct 11, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c83cb5827

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/collect_environment.py Resolved
…ation

From the Codex review of 5c83cb5:
- A libc fact and a CPython ABI tag that name different C libraries
  (glibc with -linux-musl, or musl with -linux-gnu) are an
  abi-contradiction. A test had accepted the glibc and musl pair.
- The validator writes "::" as ": :" in every line it prints. The runner
  reads "::" commands only at the start of a line, but a relative file
  name that starts with "::" is printed there.
- The collector records cgroup v2 delegation only when
  cgroup.subtree_control is also writable, since without it no
  controller can be enabled below the process's cgroup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector now also needs write access to cgroup.threads, where the
kernel has it (Linux 4.14 and later), before it records a cgroup as
delegated, as the kernel's delegation model grants all of the
directory, cgroup.procs, cgroup.threads and cgroup.subtree_control.
The evidence names each file checked.

The delegation test now withholds write access to each object in turn
and checks the access mode, so an existence check fails it. The libc
test adds a suffixed ABI tag (gnux32), so the tag's suffix handling is
covered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Path.exists() re-raises a permission error, so a process that could not
search its own cgroup directory lost every isolation fact and the
collector exited 1. The collector now uses its missing() helper, which
counts only an absent file as missing: a cgroup.threads that cannot be
looked up is checked like any other file, and delegation is then false.

The delegation test pins the evidence, with and without cgroup.threads,
and covers a cgroup.threads that cannot be looked up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

P-R-N-D commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review

Head e71b193. Your three comments on 5c83cb5 are fixed in 840c429, and two follow-up independent checks led to e3a7637 and e71b193.


Generated by Claude Code

P-R-N-D commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

@codex security review

Head e71b193.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e71b1936b2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
The validator now replaces a value shaped like a secret, with the rest
of its line, by a note naming the shape in every line it prints and in
its summary. A schema finding that echoes a rejected value, or a file
name, could otherwise show a token in the job log.

It reads at most the size limit and one byte of a document, instead of
the whole file before checking its size.

An approved exception's date must be a day that exists; the schema
checks only its shape, so 2026-99-99 was accepted.

The collector writes its printed lines the same way as the validator,
so an output file name cannot start a workflow command or show a
token-shaped value. A test keeps the two functions equal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The validator's plain() missed a token when the finding escaped the character
before it: repr() writes a newline as \n and a JSON Pointer writes / as ~1, so
the leading \b of each shape failed and the whole token reached the log and
the summary. The collector's check of its JSON text missed a token after a
character JSON escapes for the same reason. No shape starts with \b now, so
all three checks find it.

The tests now cover a value after \n, a key after /, and the collector's
check after \n and U+200E; the summary is checked for the token's body,
which its Markdown escaping no longer hides from the test, and the two
CONTROL patterns are compared. TESTING.md had the old test count, and
CI-MATRIX.md now says any approval date given must exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Without a word boundary before it, the JWT shape was tried at every "eyJ"
and scanned the rest of each run, so a long run of base64url characters
with many "eyJ" in it took time quadratic in its length (the old shape
already did this for "-eyJ" repeated). The shape now starts at the start
of a run and finds the first "eyJ" in it with a lookahead, so each run is
tested once; it matches the same strings.

The tests add a token after a letter and a time limit on long runs, and
the collector's secret subtests each start without an output file, so one
failure no longer fails the subtests after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 379da1dbd2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
…TH directories, and match a run URL's attempt

- collect_environment: an AppArmor `enabled` value other than Y or N is
  unknown rather than observed False.
- collect_environment: node is looked for in each PATH directory by itself,
  so on Windows shutil.which no longer finds a node in the current directory.
- validate_compatibility: a run URL ending in /attempts/<n> must name the
  record's run_attempt; a URL with no attempt names the run.
- Tests for each case (162 in total: 92 validator, 70 collector) and the
  matching CI-MATRIX, SECURITY-ARCHITECTURE and TESTING text.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
… an unset PATH on Windows, and state the CPython 3.12 floor of the Windows node lookup

- validate_compatibility: an attempt in a run URL has at most 10 digits,
  as run_attempt has in the schema, so a longer one names no run instead
  of reaching int() and a traceback under a looser schema.
- collect_environment: with PATH unset, os.defpath is searched without the
  current directory that Windows lists first in it.
- collect_environment and CI-MATRIX: the Windows lookup needs CPython 3.12
  or later; an older shutil.which adds no PATHEXT extension to a name with
  a directory, so node is not found and its facts are unknown.
- Tests for both code changes, and the matching SECURITY-ARCHITECTURE and
  TESTING text.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…ATH on Windows, and say what an older Python finds

- Validator test: the largest run_attempt (4294967295) in a run URL is
  valid, and an 11-digit attempt names no run.
- Collector test: on Windows, a PATH that names the current directory
  still has it searched; only the default directories of an unset PATH
  drop it.
- Collector docstring: on Windows before CPython 3.12, node is not found
  and the node facts are unknown.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…TH names it

The collector docstring, CI-MATRIX, SECURITY-ARCHITECTURE and TESTING said
the current directory is never searched on Windows; a PATH that names it
has it searched, as the collector test pins. The docstring also says an
older Python finds no node.exe rather than no node.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…n Windows Python is older than 3.12

- collect_environment: the AppArmor `enabled` parameter counts only as
  "Y\n" or "N\n", as the kernel writes it, so a padded value or a longer
  one cut at the 16 bytes read is unknown.
- collect_environment: on Windows before CPython 3.12, node is not looked
  for, and its facts are unknown with a reason that names the 3.12 floor
  rather than "node is not on PATH".
- Tests for both, and the matching CI-MATRIX and TESTING text.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The reason for an AppArmor `enabled` value other than "Y\n" or "N\n" said
the file holds neither Y nor N, which is wrong for a padded Y; it now says
the value is not Y or N and a newline, as the kernel writes it, and the
test pins it. The find_on_path docstring says node() does not call it on
Windows before CPython 3.12.

Related: #18

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

P-R-N-D commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

@codex review
@codex security review

Re-review request for the final head 3de33e0. Since 379da1d:

  • 3643d53 fixes the 3 P2 comments of the 379da1d review: an AppArmor enabled value other than Y or N makes enabled and profile unknown, node is looked for in each PATH directory by itself so Windows does not search the current directory unless PATH names it, and a run URL ending in /attempts/<n> must name the record's run_attempt.
  • a112004 reads at most 10 attempt digits (as run_attempt has in the schema), leaves . out of Windows' default directories when PATH is unset, and states the CPython 3.12 floor of the Windows lookup.
  • f6dfd64 adds tests for the 10-digit bound and for an explicit . in PATH on Windows; 004f43a corrects the wording of the docs and the module docstring.
  • d680250 counts AppArmor's enabled only as Y\n or N\n, as the kernel writes it, and on Windows before CPython 3.12 does not look for node and records the 3.12 floor as the reason.
  • 3de33e0 states that rule in the unknown reason, pins it in the test, and corrects the find_on_path docstring.

CI run 38181414381 on 3de33e0: 7 of 7 jobs succeeded. All 38 review threads are resolved.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3de33e0ad6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/collect_environment.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
Validator:
- Build a value's path when the walk reaches it, and bound each path: a key over 128 characters is named by its length, and a path of more than 16 keys keeps its first 8 and last 7 around "...". A long key over many values no longer makes as many long paths.
- Report a workflow file that cannot be read (not UTF-8, or a path the system refuses) as an evidence-source finding instead of a traceback.
- Reject a lane table that lists a lane twice, whose status would depend on the row order.
- Refuse a --summary that is a workflow file a real profile of the run names as its evidence source.

Collector:
- Read GITHUB_RUN_ATTEMPT up to the schema's largest integer, 4294967295.
- On Windows look for node only as node.exe in the PATH directories, without shutil.which, which tries every PATHEXT extension. The CPython 3.12 floor for the node lookup is gone.

Tests and docs follow, and the Windows lookup test now runs on CPython 3.11 too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Follow-ups from the independent check of d6d266e:
- Apply the 128-character key limit after escaping, so a key of "~" or "/" cannot reach twice the limit.
- Name a replaced key by its length and the first 8 hex digits of its SHA-256, so keys of one length stay apart, and mark both a replaced key and the elision with "~", which an escaped key holds only before "0" or "1".
- Treat a profile as real for the summary guard whenever check_evidence_sources reads its workflows, also when a custom schema lets synthetic be a false value other than false.
- Check the message of the non-UTF-8 workflow finding, restore the workflow before each summary subtest, and compare finding paths as sets so a failure is reported quickly.
- Say in CI-MATRIX.md what the bound is, that a shortened path can name more than one place, and that an empty PATH entry names no directory on Windows only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 7dbc25e found five low findings:

- Under a schema given with --schema that allows other keys, the rules
  across fields name places with keys as they are. CI-MATRIX.md now limits
  the bound to the default schema's property names for those findings.
- Only the first 8 hex digits of the digest are printed, so long keys of
  one length are told apart unless those digits agree. CI-MATRIX.md,
  TESTING.md and the test comment now say so, and CI-MATRIX.md says the
  length is taken before escaping and the digest is over the UTF-8 bytes.
- The chain test now uses keys under the limit but over it once escaped,
  with a finding under every long key, so it fails on its own if the limit
  is checked before escaping.
- The long-key test checks the path length before comparing paths, so a
  regression of the limit fails with a short message.
- A blank line before EnvironmentRuleTests is restored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of d75c622 found:

- CI-MATRIX.md said every rule across fields names places by schema
  property names, but the control-character and secret checks name any
  key through pointer(). The sentence now covers the other rules, which
  name a fixed place from property names and array indices, and says a
  custom --schema can also give them more than 16 keys. The comment on
  KEY_LIMIT now speaks of the paths pointer() builds.
- The chain test lost its keys kept near the limit, so on its own it no
  longer caught a missing depth limit. It now runs a chain of kept keys
  next to the chain of replaced ones.
- The direct pointer() checks now run before the 1000-path comparison,
  so a change to the replaced key's name fails with a short message.
- TESTING.md says "the first 8 hexadecimal digits" where it later
  refers to "those digits".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 149f0a6 found:

- The CI-MATRIX paragraph left `schema` findings out of the bounded
  kind, stated the bound before its condition, and limited the --schema
  caveat to schemas that allow other keys. It now says that the path of
  a value the validator walks to is bounded, that any other finding
  names its place by keys and indices of the document, which the default
  schema keeps short and at most 8 deep outside extensions, and that
  under any other schema such a path can be unescaped and longer.
- The chain test checked only the total length of a path. It now checks
  that each path has at most PATH_KEYS keys of at most KEY_LIMIT
  characters, so it also fails when the elision keeps 18 keys.
- The comment on KEY_LIMIT says the bound holds for a path pointer()
  builds from "" or from a path it built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 21cc4a8 found that the CI-MATRIX paragraph
classified a path by the place it names, while whether it is bounded
depends on the rule: the rules that walk the document (schema,
control-character, secret-like-value, fact-provenance,
profile-provenance, and synthetic-marking on an evidence reference)
build bounded paths, and the rules that read keys, such as
synthetic-marking on a synthetic run, profile-mismatch and
profile-unverified, do not. The paragraph now names the rules, and
limits "the default schema allows only the property names it defines"
to places outside extensions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

P-R-N-D commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

@codex review
@codex security review

Please re-review the current head 27f81c2. Since 3de33e0:

  • d6d266e fixes the six P2 findings from the 3de33e0 review: the collector reads GITHUB_RUN_ATTEMPT up to 10 digits and reports values above the schema maximum as unknown; on Windows the collector looks only for node.exe in each PATH directory, whatever PATHEXT lists; the validator builds a finding's path only when it reaches a value and bounds it; an unreadable evidence-source workflow is an evidence-source finding instead of a traceback; a lane listed twice in the lane table is a usage error; and --summary naming a workflow file that a real profile reads as its evidence source is a usage error.
  • 7dbc25e refines those changes: the key limit is measured after JSON Pointer escaping, a shortened key is written as ~(N characters, sha256 XXXXXXXX) and elided keys as ~... (neither can come from an escaped key), and the summary check uses the same synthetic test as the evidence check.
  • d75c622, 149f0a6, 21cc4a8 and 27f81c2 change docs, one comment and tests only: CI-MATRIX.md names the rules whose paths are bounded (those that build the path as they walk the document) and says the other rules' paths stay within the bound only under the default schema; it also says what the length and the 8-hex-digit digest are taken over, and that two long keys of one length can share a path. The path test now runs a chain of kept keys and a chain of replaced ones, checks each path's key count and key length, and fails quickly with a short message if the key limit is checked before escaping, either limit is removed, the elision keeps more keys, or the digest changes.

Validator tests: 94, collector tests: 71, both passing on CPython 3.11, 3.12 and 3.13. CI run 38190328473 is 7/7 green on 27f81c2.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 27f81c2c05

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/ci/validate_compatibility.py Resolved
Comment thread docs/TESTING.md Resolved
Comment thread scripts/ci/validate_compatibility.py Resolved
…ary target

Codex review of 27f81c2:

- The validator found an evidence-source job only as an unquoted key two
  spaces in with nothing after the colon, so a valid workflow with four
  spaces, a quoted key, or a comment after the key reported a false
  evidence-source error. It now reads the block mapping under an
  unindented jobs: line at the indentation of its first key, plain or
  quoted, with a colon followed by a space, a comment or the end of the
  line. Deeper keys, longer names and flow style still name no job.
- A --summary naming a workflow file that a real profile names but that
  is missing was not caught, because neither path named an existing
  file, so the summary created it. The paths are now also compared once
  resolved, which covers another spelling and a dangling link.
- TESTING.md gave the old total of compatibility tests (162); it is 165.

The new cases extend the two existing tests, so the counts stay 94 and
71. Each of 16 mutants of the two fixes fails a test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
A lane table status that holds for some versions only now needs those
versions declared and met. Linux x86-64 glibc is checked for glibc 2.39
or newer and CPython 3.12.13 or newer, the lowest versions its recorded
installs used; macOS Apple Silicon is statically resolvable for macOS 15
or newer and not supported for macOS 14 or older. The validator keeps
the bounds, a test keeps them equal to the table's cells, and a real
profile copies the status its versions meet or records not-assessed.
linux-x64-gha now declares the glibc 2.39 its runner measures.

A GitHub Actions record must carry its workflow ref, measured, and is
linked to a profile by its workflow and job; a record without its
workflow ref or repository, or with a ref that names no workflow file
or ref, links to no profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…s comment

The evidence-source reader now finds the jobs key with one search and
reads the lines after it one at a time, split at "\n" only, so U+0085,
U+2028 and U+2029 inside a value no longer split a line, and a workflow
without a jobs key is not split into lines at all. A comment after the
jobs colon needs a blank first, as YAML reads it, so `jobs:#c:` is not
a jobs key.

Tests cover quoted and spaced jobs keys, a tab after a colon, a block
scalar, a dot in a job name, a summary written elsewhere next to a
profile whose workflow is missing, and a summary path that loops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The workflow ref became a required run field in 2c7e857, but the collector
recorded it only for refs made of letters, digits and ._/+-, so a manual
dispatch from a branch such as fix/issue#12 failed ci-evidence and
profile-link. It now records any ref git allows, without the characters
the validator refuses in any value, up to the schema's 200 characters.

Also from the independent check of 3bf8ba4:
- Version bounds no longer read a part with a leading zero.
- A record missing only its repository is reported at that field.
- The workflow reader's comments and CI-MATRIX.md say lines end at \n,
  \r\n or \r, as the file is read with universal newlines.
- DEPENDENCY-STRATEGY.md names the versions the recorded installs used.
- Tests cover a lane without bounds, a repository name that only begins
  the workflow ref's, a ref holding @, more than one blank after glibc,
  long version parts, and every bound name being one the validator reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
70496b1 recorded the workflow ref of a manual dispatch from a branch
such as fix/issue#12, but the dispatch still failed ci-evidence:
actions/checkout checks a branch out by its name, so .git/HEAD reads
"ref: refs/heads/fix/issue#12", and the collector followed a HEAD ref
only when it was made of letters, digits and ._/+-, leaving the tested
commit unknown. Its test used a detached HEAD and missed this.

- GITHUB_REF and the ref .git/HEAD names now take the characters the
  workflow ref takes, built from CONTROL, within 200 characters. A HEAD
  ref that git refuses (.., an empty part, a part starting with ".") is
  still not followed, and HEAD is stripped of ASCII blanks only.
- A loose ref whose name is not ASCII is cited as git:.git/HEAD, since
  an evidence reference is printable ASCII.
- read_git also catches ValueError, for a ref name the file system
  encoding cannot write.
- Tests dispatch from such branches with a loose ref and with
  packed-refs, check the ref characters against the validator's CONTROL
  on every code point, read four-digit version parts, and check that
  each version bound reads a declared version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
… refs

From the independent check of 19ea96a:
- .git/HEAD and packed-refs were decoded with "replace", so a branch
  name holding a byte that is not UTF-8 was read as U+FFFD and could
  resolve to another branch's commit, recorded as observed. They are now
  decoded with surrogateescape, which the ref check refuses.
- A HEAD ref with "@{", or a part ending with ".lock" or ".", is not
  followed. Windows drops a trailing dot from a file name.
- packed-refs lines end at "\n" only; splitlines() also ended one at
  U+2028 and other characters a ref name may hold.
- Summary cells escape "&", so a value cannot write an HTML entity.
- Tests refuse HEAD refs with a space, a control or bidi character,
  more than 200 characters, or such a form, even when packed-refs lists
  them, and look up a loose ref the file system cannot name in
  packed-refs. CI-MATRIX.md names the refs that are not followed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector opens a loose ref by a path written in the file system
encoding, while git names the file in UTF-8. Under another encoding a ref
that is not ASCII could open another branch's file, and packed-refs alone
could hold an older commit, so such a ref is now left unknown.

Tests now cover a packed-refs line that is not UTF-8 (not read as U+FFFD,
and not stopping the lookup) and refs that hold "{", "@" or ".lock"
elsewhere, which are followed. CI-MATRIX.md lists every condition on the
ref the collector reads and says that a ref whose commit alone it cannot
record fails ci-evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…ronment as UTF-8

From the independent check of 4c03f84:
- Any loose ref that could not be read fell back to packed-refs, so a
  FIFO, an unreadable file or a symbolic ref in its place recorded the
  older packed commit as observed. Only a loose ref that does not exist
  (FileNotFoundError, NotADirectoryError) is now looked up in
  packed-refs; one that cannot be read or holds no commit ID is unknown.
- Python decodes the environment in the file system encoding, so under
  another encoding than UTF-8 a non-ASCII GITHUB_REF or
  GITHUB_WORKFLOW_REF was recorded as other characters. Such a value is
  now unknown, as the checked-out ref already was.
- The encoding test now holds a loose ref with a newer commit, so a
  guard moved after the loose read fails it, and asserts inside subTest.
- CI-MATRIX.md and TESTING.md say which refs fall back to packed-refs
  and apply the encoding condition to the ref and workflow ref too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU

P-R-N-D commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

@codex review

Please run a code review and a security review of the final head 9d72dd7. Since 1802cb6 the changes are in scripts/ci/collect_environment.py, scripts/ci/validate_compatibility.py, their tests, docs/CI-MATRIX.md, docs/DEPENDENCY-STRATEGY.md and docs/TESTING.md: lane status version bounds (STATUS_CONDITIONS), workflow_ref as a required measured run field, and how the collector reads GITHUB_REF, GITHUB_WORKFLOW_REF, .git/HEAD, loose refs and packed-refs.


Generated by Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 9d72dd7c90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@P-R-N-D
P-R-N-D merged commit 5f27b1c into main Oct 12, 2026
7 checks passed
@P-R-N-D
P-R-N-D deleted the claude/project-thread-6xwx6u branch October 12, 2026 04:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants