Repository navigation
Add a compatibility record model, environment probe and CI check - #18
Conversation
The schema describes a profile (what a lane promises) and an environment result (what one run observed) as layered host, guest, container and workload records, each fact carrying observed, inferred, declared, unknown or not-applicable provenance. The validator is a standard-library interpreter for the schema keywords the schema uses, plus semantic rules for layer structure, OS, CPU and ABI contradictions, emulation, accelerator fallback, stage ordering and results, the rootless policy and its exceptions, support claims against the dependency strategy lane table, and profile linkage. Fixtures are marked synthetic and cover a Linux x86-64 VM, Windows on Arm native, WSL2 with rootless Podman, a VM with Quadlet-managed Podman and Kubernetes on arm64 with CUDA. The profile for the GitHub hosted Linux x64 lane is static and makes no measured claim. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector writes one environment result from the running interpreter and the Linux /proc and /sys views it can read without privileges. It runs no subprocess, makes no network call and changes no system state. It reads only an allowlist of environment variable names, records presence only for a few more, and never copies host names, user names, addresses or tokens into the record. Probe errors are recorded and fail the detect stage instead of being dropped. Execution mode compares the kernel's architecture with the build architecture of the interpreter. The container, user namespace, cgroup, capability and service manager facts are observed from the current process only; nothing is claimed about the physical host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
One new job, "Compatibility / Schema & Probe / Linux x64", runs the validator and collector unit tests, collects this runner's environment record, validates it against the GitHub hosted Linux x64 profile and uploads the record with a seven day retention. It runs on ubuntu-24.04 with read-only contents permission, installs nothing and uses the same pinned actions as the existing jobs. The six existing jobs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
docs/CI-MATRIX.md describes the record model, the provenance rules, the stage and support statuses, the rootless defaults and exceptions, what the validator checks and does not check, and what the GitHub hosted lane can and cannot observe. TESTING.md lists the new tests, commands and job; the security documents add the job to the CI section and the review triggers; CONTEXT.md routes compatibility work to the new document; DEPENDENCY-STRATEGY.md keeps its lane table as the only support record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3eae4263f7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Validator: - Reject emulation declared native anywhere inside an emulated layer, not only next to it, and check the SOABI's architecture and the Windows and macOS platform tags. - Allow not-applicable only for the security and performance stages, so overall passed needs detect through correctness to pass; require a log or artifact reference for a real stage that ran, and fail detect when probes failed. - Add accelerator_backend to profiles: required execute, correctness and performance stages must run on it, so a CPU-only run cannot satisfy a GPU or NPU lane. - Compare every fact a profile declares, including backends by ID and the accelerator inventory, and ignore fact-shaped data in extensions. - Apply the container policy rules when the record does not show the workload outside a container, and skip minimal-capabilities only off Linux. - Read per-condition lane statuses, require a real profile of a listed lane to copy its status, match workflow jobs only under jobs:, reject run facts in synthetic results, and escape summary cells. Collector: - Record whether the process is in the initial PID namespace, and do not take the kernel layer's OS, service manager or CPU from a process that may be in a container, or from uname without /proc/sys/kernel/arch. - Record armv7 on aarch64 as unknown, EC2 metal and WSL 1 as no VM, and ppc64 Node on little-endian hosts as ppc64le. - Refuse secret-shaped stage reasons and evidence, and write no record that holds a secret-shaped value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector writes a partial record and exits 1 when a probe fails, so the validate step now runs unless the run was cancelled, and the validator and probe tests run as two steps so one failing does not hide the other. The six other jobs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
CI-MATRIX.md describes the new rules (inherited emulation, which stages may be not-applicable, stage evidence, accelerator_backend, the hidden container rule, lane statuses) and the collector's full read list, and its lanes table names lanes instead of repeating support statuses. TESTING.md has the new test counts (61 and 48) and CI steps. SECURITY-ARCHITECTURE.md fixes the job's line numbers and Node source, CODE-REVIEW.md and SECURITY-REVIEW.md add scripts/ci/ and tests/compatibility/ to the CI security review trigger, and both READMEs list CI-MATRIX.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector now records:
- an Apptainer (Singularity) container from /.singularity.d or its
environment variables, which only say that one is present, since
Apptainer keeps the initial PID namespace by default;
- os-release and PID 1 as inferred, not observed, when no container
marker is found, because a chroot or a marker-less container that
shares the PID namespace looks the same;
- a WSL 2 distribution's os-release and PID 1 from its own PID
namespace, which WSL 2 gives every distribution, and older WSL 2
kernels named "-microsoft-standard" (WSL 1 ends in "-Microsoft");
- EC2 Arm VMs as VMs and EC2 metal instances (".metal" product names)
as not shown to be VMs;
- the container layer's architecture and execution as unknown, since
its programs need not share one architecture; each runtime keeps
its own, so an emulated interpreter is still recorded as emulated;
- Node's ppc64 on a little-endian host as an inferred ppc64le;
- the C library only from os.confstr, which only glibc answers,
instead of a version scanned from the interpreter's executable.
The validator knows the apptainer technology, and does not read a
WSL 2 distribution's own PID namespace as a sign of a hidden container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
- The OS and architecture in a CPython ABI tag are checked even when the version is unknown. - A job named in a result must be an evidence source in the workflow that run.ci.workflow_ref names, when it names one, not only a job of that name in any workflow. - A comment line inside a workflow's jobs mapping no longer ends it. - Lone surrogates count as control characters, document strings are quoted in messages, and every printed line has its control characters replaced, so a value cannot start a workflow command line or stop the summary from being written. - New tests cover a CPU stage request under an accelerator profile, a required performance stage, an unknown OS family, and file: evidence on a stage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
CI-MATRIX.md now says: - which containers share the initial PID namespace, that WSL 2 gives every distribution its own, and that a WSL 2 innermost layer is not read as a hidden container; - that a container layer's CPU facts are unknown and each runtime records its own; - that off Linux a collected record fails minimal-capabilities until a probe for that system exists; - that evidence sources are checked for real profiles, and the job is matched within the workflow that run.ci.workflow_ref names; - that lone surrogates count as control characters. TESTING.md has the new test counts (62 validator, 49 collector) and what they cover. A sentence in SECURITY-ARCHITECTURE.md is mended. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…to lanes Validator: - A required policy rule passes only on observed or inferred facts; a declared or unknown value fails it, and an exception covers only a measured violation inside its rule's scope (RULE_SCOPE). - Each applied exception is printed as a policy-exception-applied warning, and --no-policy warns policy-skipped. - A real or saved user ID of 0 and permitted capabilities fail app-non-root and minimal-capabilities. - A support claim must name a lane of the DEPENDENCY-STRATEGY.md table, match its OS family, architecture, C library and WSL2 layer, and copy its status only for CPython 3.12 with the GIL; other interpreters record not-assessed. LANE_TARGETS is kept equal to the table by a test. - A real profile must be recorded by one of its GitHub Actions evidence sources (profile-link), and declared values in a result do not satisfy a profile. - The schema subset rejects another dialect, nested $schema/$id/$defs, non-portable patterns and $ref cycles. Collector: - Records the real and saved user IDs (Linux, FreeBSD, macOS; Windows not-applicable) and permitted capabilities, and reads the kernel release from /proc/sys/kernel/osrelease. - Reads only regular files, so a FIFO cannot block it. - Bounds node -p to 4 KiB and 30 seconds and kills its process group. - Infers the container engine only from markers that agree, and the engine mode only for Podman. - Drops the WSL variables from the allowlist. Tests: 83 validator and 60 collector tests, including the negative cases for each rule above. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The record is the lane's only machine-readable evidence and is a few KB, so the artifact is kept for 90 days instead of 7. The file is named linux-x64-gha.result.json in the collect, validate and upload steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
- CI-MATRIX.md: the measured-only rule, exception scope per rule and the limits of a recorded approval, real and saved user IDs and permitted capabilities, support claims bound to the lane and the lock's interpreter, the collector's node and file bounds, and the limits that remain for PR C. - TESTING.md: 83 and 60 tests and what they cover, and the record name. - SECURITY-ARCHITECTURE.md: the compatibility job's line range, the collector's bounds and the 90-day artifact. - SECURITY-REVIEW.md: check a new or changed policy exception's approval reference, scope and end condition. - DEPENDENCY-STRATEGY.md: the lane statuses are for the backend lock's interpreter, and how the validator binds a profile to its lane. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Fixes from the independent re-review of a80178c (all minor): - The validator writes "##[" as "## [" in every line it prints, because the GitHub Actions runner reads that form as a workflow command anywhere in a line. - The schema subset also rejects the {,n} and possessive quantifiers, which Python and ECMA-262 read differently. - The FreeBSD platform tag (freebsd-14.1-RELEASE-amd64) is parsed by both scripts and recorded by the collector. - The collector reads user IDs only on Linux, FreeBSD and macOS; under Cygwin or MSYS2 they are unknown. - Tests now fail if the macOS and FreeBSD identity check, the stop after node's output limit, or the FreeBSD parsing is removed. - The docs say that a node descendant with its own session can hold the read until the job times out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
From the independent review of c1c0674 (both minor): - The schema subset rejects any escaped letter other than t, n, r, f, v, x and u (adding Python's \a, \N and \U) and a character class that opens with ] or ^]. CI-MATRIX.md lists the forms and says a new pattern still needs review. - The collector's docstring says it reads user IDs on FreeBSD and macOS only, as the code does. - SECURITY-ARCHITECTURE.md carries the caveat on node descendants that start their own session. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Python reads a \u escape of a surrogate as a code point, while ECMA-262 reads it as a UTF-16 code unit and joins a pair with the u flag, so the same pattern can match different strings. The pattern check now rejects \uD800 to \uDFFF. TESTING.md named \A as tested, so a case for it is added, and both docs list the new form. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
|
@codex review Requesting a code and security re-review of the current head Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c83cb5827
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ation From the Codex review of 5c83cb5: - A libc fact and a CPython ABI tag that name different C libraries (glibc with -linux-musl, or musl with -linux-gnu) are an abi-contradiction. A test had accepted the glibc and musl pair. - The validator writes "::" as ": :" in every line it prints. The runner reads "::" commands only at the start of a line, but a relative file name that starts with "::" is printed there. - The collector records cgroup v2 delegation only when cgroup.subtree_control is also writable, since without it no controller can be enabled below the process's cgroup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector now also needs write access to cgroup.threads, where the kernel has it (Linux 4.14 and later), before it records a cgroup as delegated, as the kernel's delegation model grants all of the directory, cgroup.procs, cgroup.threads and cgroup.subtree_control. The evidence names each file checked. The delegation test now withholds write access to each object in turn and checks the access mode, so an existence check fails it. The libc test adds a suffixed ABI tag (gnux32), so the tag's suffix handling is covered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Path.exists() re-raises a permission error, so a process that could not search its own cgroup directory lost every isolation fact and the collector exited 1. The collector now uses its missing() helper, which counts only an absent file as missing: a cgroup.threads that cannot be looked up is checked like any other file, and delegation is then false. The delegation test pins the evidence, with and without cgroup.threads, and covers a cgroup.threads that cannot be looked up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
|
@codex review Head Generated by Claude Code |
|
@codex security review Head Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e71b1936b2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The validator now replaces a value shaped like a secret, with the rest of its line, by a note naming the shape in every line it prints and in its summary. A schema finding that echoes a rejected value, or a file name, could otherwise show a token in the job log. It reads at most the size limit and one byte of a document, instead of the whole file before checking its size. An approved exception's date must be a day that exists; the schema checks only its shape, so 2026-99-99 was accepted. The collector writes its printed lines the same way as the validator, so an output file name cannot start a workflow command or show a token-shaped value. A test keeps the two functions equal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The validator's plain() missed a token when the finding escaped the character before it: repr() writes a newline as \n and a JSON Pointer writes / as ~1, so the leading \b of each shape failed and the whole token reached the log and the summary. The collector's check of its JSON text missed a token after a character JSON escapes for the same reason. No shape starts with \b now, so all three checks find it. The tests now cover a value after \n, a key after /, and the collector's check after \n and U+200E; the summary is checked for the token's body, which its Markdown escaping no longer hides from the test, and the two CONTROL patterns are compared. TESTING.md had the old test count, and CI-MATRIX.md now says any approval date given must exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Without a word boundary before it, the JWT shape was tried at every "eyJ" and scanned the rest of each run, so a long run of base64url characters with many "eyJ" in it took time quadratic in its length (the old shape already did this for "-eyJ" repeated). The shape now starts at the start of a run and finds the first "eyJ" in it with a lookahead, so each run is tested once; it matches the same strings. The tests add a token after a letter and a time limit on long runs, and the collector's secret subtests each start without an output file, so one failure no longer fails the subtests after it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 379da1dbd2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…TH directories, and match a run URL's attempt - collect_environment: an AppArmor `enabled` value other than Y or N is unknown rather than observed False. - collect_environment: node is looked for in each PATH directory by itself, so on Windows shutil.which no longer finds a node in the current directory. - validate_compatibility: a run URL ending in /attempts/<n> must name the record's run_attempt; a URL with no attempt names the run. - Tests for each case (162 in total: 92 validator, 70 collector) and the matching CI-MATRIX, SECURITY-ARCHITECTURE and TESTING text. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
… an unset PATH on Windows, and state the CPython 3.12 floor of the Windows node lookup - validate_compatibility: an attempt in a run URL has at most 10 digits, as run_attempt has in the schema, so a longer one names no run instead of reaching int() and a traceback under a looser schema. - collect_environment: with PATH unset, os.defpath is searched without the current directory that Windows lists first in it. - collect_environment and CI-MATRIX: the Windows lookup needs CPython 3.12 or later; an older shutil.which adds no PATHEXT extension to a name with a directory, so node is not found and its facts are unknown. - Tests for both code changes, and the matching SECURITY-ARCHITECTURE and TESTING text. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…ATH on Windows, and say what an older Python finds - Validator test: the largest run_attempt (4294967295) in a run URL is valid, and an 11-digit attempt names no run. - Collector test: on Windows, a PATH that names the current directory still has it searched; only the default directories of an unset PATH drop it. - Collector docstring: on Windows before CPython 3.12, node is not found and the node facts are unknown. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…TH names it The collector docstring, CI-MATRIX, SECURITY-ARCHITECTURE and TESTING said the current directory is never searched on Windows; a PATH that names it has it searched, as the collector test pins. The docstring also says an older Python finds no node.exe rather than no node. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…n Windows Python is older than 3.12 - collect_environment: the AppArmor `enabled` parameter counts only as "Y\n" or "N\n", as the kernel writes it, so a padded value or a longer one cut at the 16 bytes read is unknown. - collect_environment: on Windows before CPython 3.12, node is not looked for, and its facts are unknown with a reason that names the 3.12 floor rather than "node is not on PATH". - Tests for both, and the matching CI-MATRIX and TESTING text. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The reason for an AppArmor `enabled` value other than "Y\n" or "N\n" said the file holds neither Y nor N, which is wrong for a padded Y; it now says the value is not Y or N and a newline, as the kernel writes it, and the test pins it. The find_on_path docstring says node() does not call it on Windows before CPython 3.12. Related: #18 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
|
@codex review Re-review request for the final head
CI run 38181414381 on Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3de33e0ad6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Validator: - Build a value's path when the walk reaches it, and bound each path: a key over 128 characters is named by its length, and a path of more than 16 keys keeps its first 8 and last 7 around "...". A long key over many values no longer makes as many long paths. - Report a workflow file that cannot be read (not UTF-8, or a path the system refuses) as an evidence-source finding instead of a traceback. - Reject a lane table that lists a lane twice, whose status would depend on the row order. - Refuse a --summary that is a workflow file a real profile of the run names as its evidence source. Collector: - Read GITHUB_RUN_ATTEMPT up to the schema's largest integer, 4294967295. - On Windows look for node only as node.exe in the PATH directories, without shutil.which, which tries every PATHEXT extension. The CPython 3.12 floor for the node lookup is gone. Tests and docs follow, and the Windows lookup test now runs on CPython 3.11 too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
Follow-ups from the independent check of d6d266e: - Apply the 128-character key limit after escaping, so a key of "~" or "/" cannot reach twice the limit. - Name a replaced key by its length and the first 8 hex digits of its SHA-256, so keys of one length stay apart, and mark both a replaced key and the elision with "~", which an escaped key holds only before "0" or "1". - Treat a profile as real for the summary guard whenever check_evidence_sources reads its workflows, also when a custom schema lets synthetic be a false value other than false. - Check the message of the non-UTF-8 workflow finding, restore the workflow before each summary subtest, and compare finding paths as sets so a failure is reported quickly. - Say in CI-MATRIX.md what the bound is, that a shortened path can name more than one place, and that an empty PATH entry names no directory on Windows only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 7dbc25e found five low findings: - Under a schema given with --schema that allows other keys, the rules across fields name places with keys as they are. CI-MATRIX.md now limits the bound to the default schema's property names for those findings. - Only the first 8 hex digits of the digest are printed, so long keys of one length are told apart unless those digits agree. CI-MATRIX.md, TESTING.md and the test comment now say so, and CI-MATRIX.md says the length is taken before escaping and the digest is over the UTF-8 bytes. - The chain test now uses keys under the limit but over it once escaped, with a finding under every long key, so it fails on its own if the limit is checked before escaping. - The long-key test checks the path length before comparing paths, so a regression of the limit fails with a short message. - A blank line before EnvironmentRuleTests is restored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of d75c622 found: - CI-MATRIX.md said every rule across fields names places by schema property names, but the control-character and secret checks name any key through pointer(). The sentence now covers the other rules, which name a fixed place from property names and array indices, and says a custom --schema can also give them more than 16 keys. The comment on KEY_LIMIT now speaks of the paths pointer() builds. - The chain test lost its keys kept near the limit, so on its own it no longer caught a missing depth limit. It now runs a chain of kept keys next to the chain of replaced ones. - The direct pointer() checks now run before the 1000-path comparison, so a change to the replaced key's name fails with a short message. - TESTING.md says "the first 8 hexadecimal digits" where it later refers to "those digits". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 149f0a6 found: - The CI-MATRIX paragraph left `schema` findings out of the bounded kind, stated the bound before its condition, and limited the --schema caveat to schemas that allow other keys. It now says that the path of a value the validator walks to is bounded, that any other finding names its place by keys and indices of the document, which the default schema keeps short and at most 8 deep outside extensions, and that under any other schema such a path can be unescaped and longer. - The chain test checked only the total length of a path. It now checks that each path has at most PATH_KEYS keys of at most KEY_LIMIT characters, so it also fails when the elision keeps 18 keys. - The comment on KEY_LIMIT says the bound holds for a path pointer() builds from "" or from a path it built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The independent check of 21cc4a8 found that the CI-MATRIX paragraph classified a path by the place it names, while whether it is bounded depends on the rule: the rules that walk the document (schema, control-character, secret-like-value, fact-provenance, profile-provenance, and synthetic-marking on an evidence reference) build bounded paths, and the rules that read keys, such as synthetic-marking on a synthetic run, profile-mismatch and profile-unverified, do not. The paragraph now names the rules, and limits "the default schema allows only the property names it defines" to places outside extensions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
|
@codex review Please re-review the current head
Validator tests: 94, collector tests: 71, both passing on CPython 3.11, 3.12 and 3.13. CI run 38190328473 is 7/7 green on Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27f81c2c05
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ary target Codex review of 27f81c2: - The validator found an evidence-source job only as an unquoted key two spaces in with nothing after the colon, so a valid workflow with four spaces, a quoted key, or a comment after the key reported a false evidence-source error. It now reads the block mapping under an unindented jobs: line at the indentation of its first key, plain or quoted, with a colon followed by a space, a comment or the end of the line. Deeper keys, longer names and flow style still name no job. - A --summary naming a workflow file that a real profile names but that is missing was not caught, because neither path named an existing file, so the summary created it. The paths are now also compared once resolved, which covers another spelling and a dangling link. - TESTING.md gave the old total of compatibility tests (162); it is 165. The new cases extend the two existing tests, so the counts stay 94 and 71. Each of 16 mutants of the two fixes fails a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
A lane table status that holds for some versions only now needs those versions declared and met. Linux x86-64 glibc is checked for glibc 2.39 or newer and CPython 3.12.13 or newer, the lowest versions its recorded installs used; macOS Apple Silicon is statically resolvable for macOS 15 or newer and not supported for macOS 14 or older. The validator keeps the bounds, a test keeps them equal to the table's cells, and a real profile copies the status its versions meet or records not-assessed. linux-x64-gha now declares the glibc 2.39 its runner measures. A GitHub Actions record must carry its workflow ref, measured, and is linked to a profile by its workflow and job; a record without its workflow ref or repository, or with a ref that names no workflow file or ref, links to no profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…s comment The evidence-source reader now finds the jobs key with one search and reads the lines after it one at a time, split at "\n" only, so U+0085, U+2028 and U+2029 inside a value no longer split a line, and a workflow without a jobs key is not split into lines at all. A comment after the jobs colon needs a blank first, as YAML reads it, so `jobs:#c:` is not a jobs key. Tests cover quoted and spaced jobs keys, a tab after a colon, a block scalar, a dot in a job name, a summary written elsewhere next to a profile whose workflow is missing, and a summary path that loops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The workflow ref became a required run field in 2c7e857, but the collector recorded it only for refs made of letters, digits and ._/+-, so a manual dispatch from a branch such as fix/issue#12 failed ci-evidence and profile-link. It now records any ref git allows, without the characters the validator refuses in any value, up to the schema's 200 characters. Also from the independent check of 3bf8ba4: - Version bounds no longer read a part with a leading zero. - A record missing only its repository is reported at that field. - The workflow reader's comments and CI-MATRIX.md say lines end at \n, \r\n or \r, as the file is read with universal newlines. - DEPENDENCY-STRATEGY.md names the versions the recorded installs used. - Tests cover a lane without bounds, a repository name that only begins the workflow ref's, a ref holding @, more than one blank after glibc, long version parts, and every bound name being one the validator reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
70496b1 recorded the workflow ref of a manual dispatch from a branch such as fix/issue#12, but the dispatch still failed ci-evidence: actions/checkout checks a branch out by its name, so .git/HEAD reads "ref: refs/heads/fix/issue#12", and the collector followed a HEAD ref only when it was made of letters, digits and ._/+-, leaving the tested commit unknown. Its test used a detached HEAD and missed this. - GITHUB_REF and the ref .git/HEAD names now take the characters the workflow ref takes, built from CONTROL, within 200 characters. A HEAD ref that git refuses (.., an empty part, a part starting with ".") is still not followed, and HEAD is stripped of ASCII blanks only. - A loose ref whose name is not ASCII is cited as git:.git/HEAD, since an evidence reference is printable ASCII. - read_git also catches ValueError, for a ref name the file system encoding cannot write. - Tests dispatch from such branches with a loose ref and with packed-refs, check the ref characters against the validator's CONTROL on every code point, read four-digit version parts, and check that each version bound reads a declared version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
… refs From the independent check of 19ea96a: - .git/HEAD and packed-refs were decoded with "replace", so a branch name holding a byte that is not UTF-8 was read as U+FFFD and could resolve to another branch's commit, recorded as observed. They are now decoded with surrogateescape, which the ref check refuses. - A HEAD ref with "@{", or a part ending with ".lock" or ".", is not followed. Windows drops a trailing dot from a file name. - packed-refs lines end at "\n" only; splitlines() also ended one at U+2028 and other characters a ref name may hold. - Summary cells escape "&", so a value cannot write an HTML entity. - Tests refuse HEAD refs with a space, a control or bidi character, more than 200 characters, or such a form, even when packed-refs lists them, and look up a loose ref the file system cannot name in packed-refs. CI-MATRIX.md names the refs that are not followed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
The collector opens a loose ref by a path written in the file system
encoding, while git names the file in UTF-8. Under another encoding a ref
that is not ASCII could open another branch's file, and packed-refs alone
could hold an older commit, so such a ref is now left unknown.
Tests now cover a packed-refs line that is not UTF-8 (not read as U+FFFD,
and not stopping the lookup) and refs that hold "{", "@" or ".lock"
elsewhere, which are followed. CI-MATRIX.md lists every condition on the
ref the collector reads and says that a ref whose commit alone it cannot
record fails ci-evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
…ronment as UTF-8 From the independent check of 4c03f84: - Any loose ref that could not be read fell back to packed-refs, so a FIFO, an unreadable file or a symbolic ref in its place recorded the older packed commit as observed. Only a loose ref that does not exist (FileNotFoundError, NotADirectoryError) is now looked up in packed-refs; one that cannot be read or holds no commit ID is unknown. - Python decodes the environment in the file system encoding, so under another encoding than UTF-8 a non-ASCII GITHUB_REF or GITHUB_WORKFLOW_REF was recorded as other characters. Such a value is now unknown, as the checked-out ref already was. - The encoding test now holds a loose ref with a newer commit, so a guard moved after the loose read fails it, and asserts inside subTest. - CI-MATRIX.md and TESTING.md say which refs fall back to packed-refs and apply the encoding condition to the ref and workflow ref too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU
|
@codex review Please run a code review and a security review of the final head Generated by Claude Code |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
Before: the repository had no shared way to say where Claus is expected to run or what a run actually observed. CI checked one Linux x64 lane, and the only record of platform support was the lane table in
docs/DEPENDENCY-STRATEGY.md.After: a compatibility record model describes a profile (what a lane promises) and an environment result (what one run observed) as layers from the outermost host through any guest and container to the workload, with every fact marked observed, inferred, declared, unknown, or not-applicable. A standard-library validator checks records against the schema subset it implements and against semantic rules (OS, CPU, and ABI contradictions, emulation declared as native, CPU fallback declared as accelerator success, stage results, the rootless policy and its exceptions, and support claims against the lane table). A read-only collector writes this runner's record, and one new CI job collects and validates it.
The privilege policy passes only on measured facts (observed or inferred). A declared or unknown value fails a required rule, and an exception can cover only a measured violation inside its rule's scope. A support claim must name a lane of the
DEPENDENCY-STRATEGY.mdtable, match that lane's OS family, architecture, C library and WSL2 layer, meet the version bounds the table gives its status (Linux x86-64 glibc is Checked for glibc 2.39 or newer and CPython 3.12.13 or newer; macOS Apple Silicon is statically resolvable for macOS 15 or newer and not supported for macOS 14 or older), and copy the lane's status only for the backend lock's interpreter (CPython 3.12 with the GIL); any other interpreter, or versions below every bound, recordsnot-assessed. A GitHub Actions record is linked to a profile by its measured workflow ref and job.This is PR B of the sequence agreed in the project thread (A, B, C, D). Related: #17
Changes / Scope
tests/compatibility/schema.json: the record schema, version 1, for profiles and results. It includesworkload.isolation.pid_namespace, a profile'saccelerator_backend, the real and saved user IDs, and permitted capabilities.tests/compatibility/profiles/linux-x64-gha.json: a static profile for the GitHub-hosted Linux x64 lane. It declares what its job can observe, including a GitHub-hosted runner and the runner's glibc 2.39, and makes no measured claim.tests/compatibility/fixtures/*.json: 9 synthetic fixtures, each marked synthetic: a Linux x86-64 VM, Windows on Arm native, WSL2 on Arm with rootless Podman, a VM with Quadlet-managed Podman and systemd, and Kubernetes on arm64 with CUDA.scripts/ci/validate_compatibility.py: the validator (standard library only).scripts/ci/collect_environment.py: the environment collector (standard library only).scripts/ci/tests/test_compatibility_schema.py(95 tests) andscripts/ci/tests/test_environment_probe.py(72 tests):unittest, no Django initialization..github/workflows/ci.yml: one new job,Compatibility / Schema & Probe / Linux x64. It validates the record even after a probe failure and uploads it aslinux-x64-gha.result.json, kept for 90 days. The six existing jobs are unchanged.docs/CI-MATRIX.md(new), and updates toTESTING.md,SECURITY-ARCHITECTURE.md(the CI section and the CI sentence of the Secrets hygiene bullet),SECURITY-REVIEW.md(the CI trigger row and a check of compatibility policy exceptions),CODE-REVIEW.md,CONTEXT.md,DEPENDENCY-STRATEGY.md(how profiles copy a lane's status, and the version bounds of a status, with the Linux x86-64 glibc Checked cell now naming glibc 2.39 or newer and CPython 3.12.13 or newer), and both READMEs (one link each).Out of scope, by request: running Python 3.15 or 3.15t, RHEL, Windows, ARM64, or WSL2 lanes; real containers, clusters, GPU, or NPU work; new packages or lock changes;
check_test_lock.py(unchanged, still a Linux x86-64 / CPython 3.12.15 tool);backend/pytest.iniand the test parity check (unchanged); Dockerfile, Containerfile, Kubernetes, Helm, systemd, or Quadlet files; required status checks, rulesets, and Issue #15. PR C and PR D are not started.Implementation Status
N/A for features and
SEC-*requirements: no feature status indocs/ARCHITECTURE.mdand noSEC-*status changes. The lane table indocs/DEPENDENCY-STRATEGY.mdkeeps every status it had; no lane is promoted to Checked. The Linux x86-64 glibc row now writes the versions its Checked status covers, the lowest its recorded installs used (glibc 2.39, CPython 3.12.13), so a profile below them, such as RHEL 9 with glibc 2.34, recordsnot-assessed; this narrows what the row covers and adds no evidence. The new CI job's pass means the tooling works on that runner; it is not a support claim for any other platform.Validation
Head
9d72dd7, basemainatfbf173a; GitHub reports the merge state asclean.9d72dd7: 7 of 7 jobs succeeded. The six existing jobs (Backend / CPython 3.12.15, Backend / pytest, Frontend / Node 24, Frontend / Vitest, E2E / Playwright Chromium dev Next and production Next) are unchanged. The new job's record was a "valid result (0 errors, 0 warnings)" with--require-real, the lane table and thelinux-x64-ghaprofile; the committed profile was valid; the two test files ran 95 and 72 tests, OK, including the test that runs the collector's Node expression on the job's realnode(v24.21.0); and it uploadedcompatibility-linux-x64-attempt-1(2,997 bytes, kept 90 days). The earlier fix commitsa80178c,c1c0674,ce78b64,5c83cb5,840c429,e3a7637,e71b193,550c8dc,8caccf4,fb5ff97,ccc35e6,e3b4037,a91839d(run 38142834378),aee6794(run 38144283810),f63fe18(run 38145113024),8ec100e(run 38146152994),1b37fe7(run 38147612330),29270a1(run 38148616077),e397278(run 38150962215),01f26d0(run 38153161677),75f014a(run 38155428102),bb2020b(run 38159801448),5dfa093(run 38160680573),904dec9(run 38163470842),3c84fab(run 38166698760),cb58456(run 38168871376),fb45b68(run 38170596893),b132fe5(run 38171374932),13cfade(run 38173233255),7785ca5(run 38174432720),379da1d(run 38175427934),3643d53(run 38177557338),a112004(run 38178579353),f6dfd64(run 38179466431),004f43a(run 38180086611),d680250(run 38180773681),3de33e0(run 38181414381),d6d266e(run 38184079757),7dbc25e(run 38186610739),d75c622(run 38187955921),149f0a6(run 38188776411),21cc4a8(run 38189540879),27f81c2(run 38190328473),1802cb6(run 38192212801),3bf8ba4(run 38194115122, with2c7e857),70496b1(run 38195707907),19ea96a(run 38197704942),9813f3a(run 38199280792) and4c03f84(run 38200906929) also ran 7 of 7 green; the run for16a19c2was cancelled whenfb45b68was pushed.9d72dd7:python -I -B scripts/ci/tests/test_compatibility_schema.pyran 95 tests, OK, andpython -I -B scripts/ci/tests/test_environment_probe.pyran 72 tests, OK (the real-node test ran against Node 22.22.0); both files also pass on CPython 3.11.scripts/ci/tests/test_check_test_lock.py(unchanged) ran 49 tests, OK, on 3.12.15; on 3.13.16 it fails because that interpreter does not hold the test lock, as the check intends.python -I scripts/ci/validate_compatibility.py --dependency-strategy docs/DEPENDENCY-STRATEGY.md --profile <fixture>.profile.json <fixture>.result.json, for each fixture pair: valid. Applied exceptions are listed aspolicy-exception-appliedwarnings.linux-x64-ghafails as intended:profile-link,profile-mismatchandprofile-unverified(the runner),app-non-root(superuser, real and saved user ID 0),no-privileged-container,minimal-devicesandminimal-capabilities(effective and permitted), with anengine-rootlesswarning.jsonschema4.26.0 (Draft 2020-12) every time.jsonschemawas already installed in the sandbox's system Python and is not added to the repository.git diff --checkclean; the added lines have no em dashes. (DEPENDENCY-STRATEGY.mdkeeps the em dashes it already had.)Reviews (the author's own review is a self-assessment; agent reviews are evidence, not approval; no human has reviewed this PR yet):
d3f1975,f88b36a,01fe600,b778572,b612554and1ac72cb.d3f1975, resolved, and rechecked on the final tree (27f81c2). A later Codex review of5c83cb5left 3 P2 comments, fixed in840c429, answered and resolved (see below). A Codex code review ofe71b193left 4 P2 comments, fixed in550c8dc, answered and resolved (see below). A Codex code review ofe3b4037left 5 P2 comments, fixed ina91839d, answered and resolved (see below). A Codex code review of29270a1left 4 P2 comments, fixed ine397278, answered and resolved (see below). A Codex code review of5dfa093left 4 P2 comments, fixed in904dec9and3c84fab, answered and resolved (see below). A Codex code review of3c84fableft 4 P2 comments, fixed incb58456(refined in16a19c2,fb45b68andb132fe5), answered and resolved (see below). A Codex code review ofb132fe5left 3 P2 comments, fixed in13cfade(refined in7785ca5and379da1d), answered and resolved (see below). A Codex code review of379da1dleft 3 P2 comments, fixed in3643d53(refined ina112004,f6dfd64,004f43a,d680250and3de33e0), answered and resolved (see below). A Codex code review of3de33e0left 6 P2 comments, fixed ind6d266e(refined in7dbc25e,d75c622,149f0a6,21cc4a8and27f81c2), answered and resolved (see below). A Codex code review of27f81c2left 2 P2 and 1 P3 comments, fixed in1802cb6, answered and resolved (see below). A Codex security review ran once, when this PR was opened on3eae426, and posted no comment (the Codex summary comment shows it completed). The security reviews requested later, on5c83cb5,e71b193,e3b4037,29270a1,5dfa093,3c84fab,b132fe5,379da1d,3de33e0and27f81c2, did not start; every later Codex review is a code review. A Codex code and security review of the final head9d72dd7is requested with this update.1ac72cb(separate agents for the schema and validator, the Rootless policy, the collector, CI, PR C readiness, negative tests and the docs; each finding then checked by a separate verify pass): 46 findings. 1 P1 and 2 P2 blocked merge: support claims bound only to the OS/arch/libc row (a 3.15t or RHEL profile could claimchecked), declared facts satisfying the least-privilege rules, and the policy failing open off Linux. 15 more P2 (6 code, 9 test gaps) and 25 P3; 2 known limits; 1 refuted. All P1 and P2 are fixed in417e7ad,9653380anda80178c; 24 P3 are fixed in code or docs, and one (the engine rule across layers) is left open, listed under Notes.a80178cto27f81c2(42 checks, by one to four agents each, some with a separate verify pass): no P0 or P1, and four P2, all fixed:plain()still showed a token when the echo escaped the character before it (found on550c8dc, fixed in8caccf4); one397278, no test pinned the lock or the clear before the node wait, and none ran the new timed wait (closed in01f26d0); and on904dec9, no test passed a profile document under a looser schema (fixed in3c84fab). Their P3s were fixed in a following commit or left as optional; the fix commits' messages describe each fix, and the ones left that matter are listed under Notes.5c83cb5: 3 P2, all fixed in840c429. (1) The ABI tag's C library was not compared with the libc fact, so a glibc record with a musl SOABI was valid; it is now anabi-contradiction. (2) A file name starting with::could start a workflow command, becauseplain()broke up only##[;::is now printed as: :. The committed CI passes only$RUNNER_TEMPandtests/paths, so this was not reachable there. (3) The collector reported a cgroup as delegated without write access tocgroup.subtree_control; it now requires it. That fact is informational; no policy rule reads it.e71b193: 4 P2, each reproduced and fixed in550c8dc. (1) A secret-shaped value that also broke the schema was printed in full by the structural findings, which run before the secret check;plain(), which every printed line and summary cell goes through, now replaces such a value and the rest of its line with a note naming the shape. (2)load_jsonread a whole file before the 2 MiB check; it now reads at most the limit plus one byte. (3) An approved exception's date was checked for its digit shape only, so2026-99-99let the exception apply; the date must now be a day that exists, or the profile gets apolicy-exceptionerror. (4) The collector printed its--outputname as given, so a name starting with::could start a workflow command; the committed CI passes a fixed name, so this was not reachable there. The collector now has aplain()equal to the validator's, and a test keeps the two equal.e3b4037: 5 P2, each reproduced and fixed ina91839d. (1) A documented technology on a layer of another kind was accepted: a VM namedpodman, a container namedkvm, a WSL2 layer nameddocker. Each documented technology now makes one layer kind (hypervisors avm, engines acontainer,wsl2awsl2), and any other pairing is alayer-structureerror; an unlisted technology still only warns. (2) The AWS access key ID shape ended at\b, so a key followed by_or a lowercase letter was printed and written; it now ends where the key's own alphabet (uppercase letters and digits) ends, in both scripts. (3)generated_atwas checked for its shape only, so2026-99-99T99:99:99Zpassed; it must now be a valid date and time, or the result gets asynthetic-markingerror. (4) Support claims read the C library only from a libc fact starting withglibc, so a musl profile with no ABI tag could not claim its lane; a libc fact starting withmuslis read too. (5) The collector took Node'sppc64byte order from its own Python; it now asks Node (os.endianness()), and a test runs the expression on a realnode.29270a1: 4 P2, each reproduced and fixed ine397278. (1) The run fields and commit SHAs of a GitHub Actions record only had to be known, so a declared, unevidencedtested_shaorevent_shapassed; each run field (event,repository,run_id,run_attempt,run_url,job) and each SHA must now be observed or inferred, orci-evidencereports it. (2) A knownworkflow_refoutside the record's repository fell back to comparing the job alone, so a record from another repository with a matching job name was linked to the profile; it is now aprofile-linkerror, and only an unknown ref falls back. (3) The node timer could signal a process group after the main thread had reaped node, and so possibly a reused process ID; the timer now signals only while the read runs, the main thread ends that under a lock before it reaps node, and the wait after the read has its own deadline. (4) A glibc or musl libc fact on a Windows or macOS Python passed whenever the ABI tag named no C library; it is now anabi-contradictionwhen the innermost layer, the platform tag or the SOABI is Windows, macOS or FreeBSD. Each new test fails on29270a1. The collector already writes the run fields and SHAs as observed or inferred, so CI's own record is unaffected.5dfa093: 4 P2, each reproduced and fixed in904dec9. (1) Slack app-level tokens (xapp-) were not a secret shape, so the collector wrote one given as a stage reason and the validator accepted and printed it; the Slack shape in both scripts now includes them. (2) A GitHub Actions record with run attempt 0 was valid; GitHub numbers attempts from 1, so it is nowci-evidence. (3) A schema given with--schemathat accepts a document the rules across fields cannot read ended in a traceback; it now gives exit status 2 with a message, and with the default schema such an exception is still raised (decided on the resolved path since3c84fab). (4) On Windows,nodestarted with an empty environment can fail to start withoutSystemRoot, which Windows needs to load side-by-side assemblies; the collector now passesSystemRootalone there (allowlisted, presence-only, not recorded) and still an empty environment elsewhere. Each new test fails on5dfa093.3c84fab: 4 P2, each reproduced and fixed incb58456. (1) A--summarythat named the record made the collector append Markdown to the JSON and exit 0, and the validator could append to its own inputs. The collector now checks, after writing the record, whether the summary is the same file (Path.samefile, so a link or another spelling counts) and exits 2 without appending; the validator exits 2 before anything else when the summary is its schema, profile, lane table or a document. (2) argparse printed a rejected argument as given, so--bad=##[error]xor a token-shaped value reached the log; both scripts now write argparse's usage error, program name included, throughplain(). (3) An emptyPATHwas replaced by the default directories when looking fornode; now only an unset one is. (4) With the innermost layer's OS family unknown, the platform tag and the SOABI were never compared with each other; for CPython they now are, by OS, and by architecture when the arch fact is unknown or not a listed name. Each new test fails on3c84fab.b132fe5: 3 P2, each reproduced and fixed in13cfade. (1) A CPython version was read from its first two numbers, so a profile declaring3.12-not-pythonkept its checked claim and its tags were compared as 3.12. The version is now read whole, asplatform.python_version()writes it (3.12.15,3.13.0rc1, or3.14.0a1+for a build after a release) or as major.minor; any other text is anabi-contradictionat the version and names no lane interpreter. (2) A passed stage whose compute record named no backend that ran (effectivenull) was valid; it is now abackend-referenceerror, and on a GPU or NPU request it stays a singleaccelerator-fallback. (3) With a listed arch fact, an ABI tag for an architecture outside the list, such asvax, was not compared with it; it is now anabi-contradiction. Each new test fails onb132fe5.379da1d: 3 P2, each reproduced and fixed in3643d53. (1) An AppArmorenabledfile holding anything butY, such as a replaced file withgarbage, was recorded as observedFalse; onlyYorNnow count, and any other value makesenabledandprofileunknown. (2) On Windows,shutil.which("node", path=...)also looks in the current directory first, so anode.exein the checkout could run althoughPATHdid not name it; node is now looked for in eachPATHdirectory by itself, and an empty entry names none. (3) A run URL ending in/attempts/1was valid on a record withrun_attempt2; such a URL must now name the record's attempt, while a URL with no attempt still names the run. Each new test fails with its fix reverted.3de33e0: 6 P2, each reproduced and fixed ind6d266e, refined in7dbc25e. (1) The collector readGITHUB_RUN_ATTEMPTonly up to 4 digits; it now reads up to the schema's largest integer, 4294967295, and a larger value isunknown. (2) On Windowsshutil.whichtries everyPATHEXTextension, so anode.batin an earlierPATHdirectory ran in place ofnode.exe; the collector now looks fornode.exealone in eachPATHdirectory, with any CPython version. (3) A finding's path repeated every key above it, andwalk()built every child path before visiting any, so a 1.82 MiB document with a 1,000,000-character key over 90,000 values ran out of memory; a path is now built when its value is reached and is bounded (see CI-MATRIX.md), and that document validates in 0.2 seconds. (4) An unreadable evidence-source workflow (a name over the system's limit, or a file that is not UTF-8) ended the validator in a traceback; it is now anevidence-sourcefinding. (5) A lane listed twice in the lane table kept its last row; that is now a usage error. (6)--summarycould append to a workflow file that a real profile reads as its evidence source; that is now a usage error. Fixes (3), (4) and (6) close limits recorded forfb5ff97,cb58456andfb45b68, and (2) closes the CPython 3.11 items recorded for3643d53tof6dfd64.27f81c2: 2 P2 and 1 P3, each reproduced and fixed in1802cb6. (1) An evidence-source job was found only as an unquoted key two spaces in with nothing after the colon, so a valid workflow with four-space indentation, a quoted key or a comment after the key failedevidence-source; the validator now reads the block mapping under an unindentedjobs:line at the indentation of its first key. (2) A--summarynaming a workflow file that a real profile names but that is missing was not refused, so the summary created it; the paths are now also compared once resolved. (3)TESTING.mdgave an old test total. 47 of 47 Codex threads are resolved.1802cb6: no P0 to P2, 9 P3. Fixed in3bf8ba4: lines were split at U+0085, U+2028 and U+2029 as well as\n, so a value holding one could hide or invent a job (the reader now splits at\nonly);jobs:#cwas read as a jobs key (a comment after the colon now needs a blank); a workflow was split into a list of all its lines before the jobs key was found (128 MB of short lines with no jobs key took 22 s; the key is now found with one search, 0.9 s, and lines after it are read one at a time); and the test gaps it listed (quoted and spaced jobs keys, a tab after a colon, a block scalar, a dot in a job name, a summary written elsewhere next to a missing workflow, a summary path that loops). Recorded, not fixed: the reader takes the first of duplicate top-leveljobskeys, where PyYAML takes the last; a profile that already failsevidence-sourcewith a workflow path outside.github/workflows/can make--summarywith that path exit 2; on a case-insensitive file system a missing workflow spelled in another case is not recognized; a summary path holding NUL raises fromopen()whenmain()is called directly (not reachable from the command line).2c7e857. (1) The macOS Apple Silicon row is statically resolvable only for macOS 15 or newer, but the validator accepted either of its statuses whatever the macOS version; a status the table gives for some versions only now needs those versions declared and met, the validator keeps the bounds (STATUS_CONDITIONS), and a test keeps them equal to the table's cells. The Linux x86-64 glibc row's Checked status now covers glibc 2.39 or newer and CPython 3.12.13 or newer releases; the distribution is not part of a status. (2) A GitHub Actions record without a knownworkflow_refwas linked to a profile by its job alone;workflow_refis now a measured run field, and a record is linked by its workflow and job only.3bf8ba4: no P0 or P1, 1 P2 and 8 P3. The P2: since2c7e857made the workflow ref a required run field, a manual dispatch from a branch whose name holds a character git allows but the collector did not (anything but letters, digits and._/+-, such asfix/issue#12,it'sor a non-ASCII letter) failedci-evidenceandprofile-link, because the collector recorded its workflow ref asunknown. Fixed in70496b1: the collector records any ref git allows, without the characters the validator refuses in any value, up to the schema's 200 characters. Also fixed there: version bounds read a part with a leading zero (glibc 2.039met the Checked bound); a record missing only its repository was reported atworkflow_ref, with a message naming both fields; the comments andCI-MATRIX.mdsaid workflow lines split at\nonly, while the file is read with universal newlines, so\r\nand\rend a line too; aDEPENDENCY-STRATEGY.mdsentence said no recorded install used a version above the bounds; and test gaps (a lane without bounds, a repository name that only begins the workflow ref's, a ref holding@, more than one blank afterglibc, long version parts, a bound name the validator cannot read). Recorded, not fixed: no test tells the reader'sstrip(" \t")fromstrip()or covers a\rin the text given to it, which comes with universal newlines.70496b1: no P0 or P2, 1 P1 and 2 P3. The P1:70496b1recorded the workflow ref of a manual dispatch from a branch such asfix/issue#12, but the dispatch still failedci-evidence. actions/checkout checks a branch out by its name, so.git/HEADreadsref: refs/heads/fix/issue#12, and the collector followed a HEAD ref only when it was made of letters, digits and._/+-, so the tested commit wasunknown; the70496b1test used a detached HEAD and missed it. The cause predates70496b1. Fixed in19ea96a:GITHUB_REFand the ref.git/HEADnames take the characters the workflow ref takes, within 200 characters; a HEAD ref git refuses (.., an empty part, a part that starts with.) is still not followed; HEAD is stripped of ASCII blanks only; and a loose ref whose name is not ASCII is cited asgit:.git/HEAD, because an evidence reference is printable ASCII. A real-git repro now records the ref, workflow ref and commit formain,fix/issue#12, a non-ASCII branch and one ending in U+3000, with noci-evidenceorprofile-linkfinding; on70496b1thefix/issue#12dispatch gaveci-evidenceat/run/revision/tested_sha. The P3s, also fixed there: the ref character class was copied by hand fromCONTROL(it is now built fromCONTROL, and a test compares it with the validator's on every code point), and test gaps (the bound-name test did not show each bound reads a declared version, no four-digit version part was tested, and the dispatch loop asserted outsidesubTest).19ea96a: no P0 to P2, 5 P3 (3 introduced, 2 older), all fixed in9813f3a..git/HEADandpacked-refswere decoded withreplace, so a branch name holding a byte that is not UTF-8 read as U+FFFD and could resolve to another branch's commit, recorded as observed (they are now decoded withsurrogateescape, which the ref check refuses); no test exercised the HEAD ref's character and length check (tests now refuse a space, a control or bidi character, more than 200 characters, and git's other forms even whenpacked-refslists the ref); the docs said every ref git refuses is refused, while@{and a part ending with.lockor.were followed (they are now refused, a trailing.also because Windows drops it from a file name);packed-refswas split withsplitlines(), which also ends a line at U+2028 and other characters a ref may hold (now at\nonly); and summary cells did not escape&, so a value could write an HTML entity (both scripts now escape it).9813f3a: no P0 to P2, 5 P3. Fixed in4c03f84: no test decoded apacked-refsline that is not UTF-8, so decodingpacked-refsalone withreplace(reading the branchx/afollowed by byte 0x80 asx/afollowed by U+FFFD, the checked-out branch) orstrictwent unnoticed (tests added); refs holding{,@or.lockelsewhere had no positive case (added); CI-MATRIX.md did not say that a dispatch whose commit alone the collector cannot record failsci-evidenceonly, and its read list left out the character and length conditions (both fixed); and, older, under a file system encoding other than UTF-8 a ref that is not ASCII could open the loose ref of another branch, because the path is written in that encoding while git names the file in UTF-8 (such a ref is now leftunknown). Kept as it is: a part ending with., which git allows, is refused on every system, so the commit of such a dispatch isunknown(fails closed; listed under Notes).4c03f84: no P0 to P2, 3 P3, all fixed in9d72dd7. The test held the non-ASCII ref inpacked-refsonly, so a guard moved after the loose read still passed (the test now holds a newer loose ref, with its assertions insidesubTest); under a file system encoding other than UTF-8, aGITHUB_REForGITHUB_WORKFLOW_REFthat is not ASCII was recorded as other characters, since Python decodes the environment in that encoding (such a value is nowunknown, and the docs apply the encoding condition to the ref and workflow ref as well as the commit); and, older, any loose ref that could not be read fell back topacked-refs, so a FIFO or a symbolic ref in its place recorded the older packed commit as observed (only a loose ref that does not exist now falls back).9d72dd7, the final head: no P0 to P2, 2 P3, recorded and not fixed (see Notes). Each of the three4c03f84findings is fixed. With real git, a FIFO, a text symbolic ref and an unreadable loose ref now giveunknownwhere4c03f84recorded the older packed commit, and every layout git produces (loose only, packed only, a newer loose over a packed ref, a missing parent directory, a directory holding other loose refs) still resolves togit rev-parse HEAD. Under an ISO-8859-1 locale built for the check, a non-ASCIIGITHUB_REFandGITHUB_WORKFLOW_REFareunknownwhere4c03f84recorded the two UTF-8 bytes of the accented letter as two Latin-1 characters, and the validator accepts the new facts. Against4c03f84's collector the new tests fail 5 cases, each new behaviour, and every mutant of the change but one fails them.c1c0674,ce78b64,5c83cb5,840c429,e3a7637,e71b193,550c8dcand8caccf4, and the linear-time search infb5ff97; the four JWT counts and a leading\bon each of the six shapes (ccc35e6,e3b4037); 16 of 17 mutants fora91839dandaee6794; 6 of 6 forf63fe18; 12 of 12 for8ec100e; 11 of 11 for1b37fe7; 4 of 4 for29270a1; 8 of 8 fore397278; 11 of 11 for01f26d0; 12 of 13 collector mutants and 7 of 7 libc mutants for75f014a, whose tests pass, without hanging, on 2 correct variants; the FreeBSD mutant forbb2020b; 6 of 6 for5dfa093; 9 of 9 for904dec9and3c84fab; 17 of 18 forcb58456,16a19c2andfb45b68; 20 of 20 for13cfade,7785ca5and379da1d; the three fixes of3643d53; all but one fora112004, and two more with the tests off6dfd64; 3 of 3 ford680250and3de33e0; 10 of 10 ford6d266e; 6 of 6 for7dbc25e, each within seconds; 2 of 2 ford75c622; and for21cc4a8, the key limit checked before escaping, no key limit, no depth limit, an elision that keeps 18 keys, a 7-digit digest and no digest, each within a second and with a short message, and the chain block alone fails on the four bound mutants; 16 of 16 for1802cb6; 24 of 25 for2c7e857(the survivor reads3.12as a three-part CPython version, which the bound already refuses as too short); 10 of 10 for3bf8ba4; 12 of 12 for70496b1; 15 of 15 for19ea96a; 12 of 12 for9813f3a; 10 of 11 for4c03f84; and 9 of 10 for9d72dd7. Five survivors are equivalent to the code they change: the kind rule skipping the host (the host rule before it already refuses any technology on a host),elifforifin the tag comparison (its two branches need different and equal OSes), the loose read in4c03f84skipped on the condition the guard before it already returns on, a second copy of that guard beforepacked-refsin9d72dd7, and.dropped from the default directories on every system (os.defpathlists no.outside Windows). The 13th75f014amutant, a wait under the lock with the flag cleared after it, leaves a race too narrow for the test to reach. The 21 mutants of the earlier validator mutation script still fail a test.Security Impact
The change touches the CI trust boundary (
.github/workflows/ci.yml) and adds a script that reads runner state. The CODE-REVIEW.md step 9 triggers that match are CI workflows, environment variable handling, logging (the job summary and the uploaded record), and authorization policy (the rootless rules and their exceptions).The new job:
ubuntu-24.04with the workflow's read-onlycontentspermission andpersist-credentials: false;UV_NO_CONFIGand givessetup-uva Python version;sudo.The collector reads:
/proc(including/proc/self/statusfor the real, effective and saved user IDs and capabilities, and/proc/sys/kernel/osrelease),/sys,/etc/os-releaseor/usr/lib/os-release, and two files under/run;/.dockerenv,/.singularity.dand/run/systemd/systemexist;.git/HEADand the ref it names (loose or inpacked-refs), and only the head and base SHAs from the event file;APPTAINER_CONTAINER, it records only that they are present.It reads only regular files, so a FIFO cannot block it. It never iterates the environment, uses no network and changes no system state. It runs one subprocess,
node -p, looked for in thePATHdirectories only (on Windows only asnode.exe, whateverPATHEXTlists, and not in the current directory, whereshutil.whichlooks first, unlessPATHnames it, also whenPATHis unset), without a shell, with an empty environment (on Windows,SystemRootalone) and its stderr discarded, reading at most 4 KiB of output and killing its process group after 30 seconds. The timer can signal only while the read runs; the main thread ends that under a lock and only then reaps node, so no signal reaches a reaped, possibly reused, process ID. A descendant that starts a session of its own can hold the read until the job's 10-minute timeout; the node facts are thenunknown.The validator rejects records with control characters, lone surrogates, or values shaped like common tokens or private keys; the collector refuses to write such a record. In every line it prints, the validator replaces control characters, does not show a secret-shaped value (it prints a note naming the shape instead of the rest of the line), and writes the runner's legacy
##[command marker as## [and::as: :; it also escapes Markdown in the summary. The collector prints its lines through the same function, and both scripts write argparse's usage errors, which repeat a rejected argument, through it. So neither a record value nor a file name given on the command line can start a workflow command or put a secret-shaped value in the log or the summary. The shapes are found wherever they start, also after an escape such as\nor~1, and the AWS key ID shape ends where the key's alphabet does, so a key followed by_or a lowercase letter is caught too.The validator reads at most 2 MiB and one byte of each document, profile or schema.
Neither script appends its summary to a file it was given: the collector exits 2 without appending when
--summaryis the record it wrote, and the validator exits 2 before reading anything when--summaryis its schema, profile, lane table or a document.A policy exception's approval is a recorded claim. The validator checks that it is recorded, approved and in scope, and prints each use as a
policy-exception-appliedwarning; it cannot check who approved it.SECURITY-REVIEW.mdnow asks a reviewer to check that the reference points to a maintainer's decision outside the change itself.The collector records
GITHUB_WORKFLOW_REF,GITHUB_REFand the ref.git/HEADnames with any ref git allows, so these values can hold characters such as#, quotes or non-ASCII letters. The HEAD ref is read as a path under.gitonly when it is UTF-8 and has no..or@{and no part that is empty, starts with., or ends with.lockor., and, when it is not ASCII, only when the file system encoding is UTF-8, in which git names the file;packed-refsis read only when the loose ref does not exist, so a loose ref that cannot be read or holds no commit ID givesunknown, not an older packed commit. AGITHUB_REForGITHUB_WORKFLOW_REFthat is not ASCII is recorded only under a UTF-8 file system encoding. Summary cells in both scripts escape&as well as Markdown, so a value cannot write an HTML entity. It still excludes control and bidirectional formatting characters and lone surrogates, stays within the schema's 200 characters, and goes through the same secret-shape check before writing; the validator's printed lines and summary handle it as any other value.The uploaded artifact holds the runner record only and is kept for 90 days.
As with the lock check, a pull request that changes the collector, the validator or their tests runs them as changed. Such a change needs review in the diff.
Notes / Risks
ubuntu-24.04x86-64 runner and the local sandbox. That covers RHEL, Windows (native and WSL2), macOS, ARM64, CPython 3.15 and 3.15t, real containers and Kubernetes, and GPU or NPU backends. Their fixtures are synthetic, and the probe tests fix interpreter facts and have run only on Linux x86-64 hosts.app-non-root(Windows),no-privileged-container,minimal-capabilitiesandminimal-devicesand warns onengine-rootlessuntil such a probe exists. Windows token privileges are not modeled./run/.containerenv./proc/sys/kernel/arch(before 6.1) gives an unknown execution mode, unless/proc/cpuinfoshows another architecture family, which is recorded as emulated (inferred).sudoor thedockergroup is available on the runner is not observed.workflow_refandrepository, measured; a record from a runner that does not setGITHUB_WORKFLOW_REFfails as CI evidence and links to no profile. The collector records the ref and the workflow ref for any ref git allows, so a manual dispatch from any branch git allows links, unless its workflow ref holds a character the validator refuses in any value, is longer than 200 characters, or, under a file system encoding other than UTF-8, is not ASCII. It records the checked-out commit too, except for a ref with a part that ends with.(refused on every system because Windows drops it from a file name), or one whose loose ref exists but cannot be read or holds no commit ID; such a dispatch failsci-evidence. The check that the ref lies in the record's repository and names a workflow file and a ref runs only with--profile.Ubuntu GLIBC 2.39orGNU libc 2.39names another C library, so it contradicts a glibc lane and agnuABI tag. This fails closed, and the collector writes onlyglibcand a version. A glibc or musl fact for a Windows, macOS or FreeBSD Python is anabi-contradiction.sys.implementation._multiarchand the SOABI (x86_64-linux-gnux32) namex86_64, so the collector recordsx86_64with 32-bit pointers and the validator rejects its own record. x32 is no lane and this fails closed; the ELF fallback alone checks the class.test_the_expression_runs_on_a_real_noderuns the firstnodeonPATHwith an empty environment. A version-manager shim that needsHOMEfails it on a developer machine; CI uses the real binary fromsetup-node.CI-MATRIX.mdlists only. Other differences between Python and ECMA-262 remain, such as how.treats line terminators, so a new pattern still needs review.cb58456andfb45b68, recorded and not fixed, each older thancb58456or outside what it changed: with the arch fact unknown, the pointer size is not compared with the platform tag or the SOABI; with the innermost layer's OS family unknown, the Node platform is not compared with the Python tags; and--helpprints the program name, taken from how the script was started, withoutplain(), though it repeats no argument value.13cfade, recorded and not fixed, both older than this PR's last changes:TESTING.mdhas one non-ASCII character (a multiplication sign) frommainbefore this PR, and an x32 record is rejected, as noted above. From the check of379da1d, optional and older than it: no accepted version has a two-digit pre-release number, so a pattern that allows one digit there, or a separate rule forb, passes the tests; no CPython release has had one.3643d53tof6dfd64, recorded and not fixed: the job puts$GITHUB_WORKSPACE/.venv/bin, inside the checkout, first onPATHbefore the collector runs, which crosses no boundary underpull_request, where the PR's own code runs anyway; and a quotedPATHentry such as"C:\nodejs"finds no node, as before.d6d266e, recorded and not fixed: a document under 2 MiB with about 200,000 findings under 14 nested keys of 127 characters prints 330 MiB of finding lines (708 MiB of memory, 28 seconds), because each finding gets a line; the path in each line is now bounded, and only a document given to the validator reaches this (in CI, the collector's record and the committed profiles). Capping the findings printed per document would change the output, so it is left for a later change. On POSIX,shutil.whichreads an emptyPATHentry as the current directory and returns a barenode, whichPopenwith an empty environment then looks up inos.defpath, so the node found and the node run can differ; this is older thand6d266e, and CI'sPATHhas no empty entry.9d72dd7, recorded and not fixed: a symbolic ref written as a symlink (only with git's deprecatedcore.preferSymlinkRefs) dangles when the collector opens it, so a branch HEAD names that is such a symlink, with an olderpacked-refsentry of the same name, still records the packed commit (as before9d72dd7; actions/checkout does not make this layout); no test pins the default ofread_git'sabsent_ok; an empty directory at a loose ref's path givesunknownwhere git readspacked-refs(no git command leaves that layout); and theWORKFLOW_REFcomment does not repeat the encoding condition.7dbc25e, recorded and not fixed: the root and an empty key""both print as/(older than this PR); andevidence_workflowsalso names the workflow files of a profile whose structure is invalid, so the summary guard only refuses more.8a5fb95says the collector runs no subprocess. That is wrong: it runsnode -pas described above, which--no-nodedisables.Interfaces for PR C:
measuredmeaning observed or inferred;--profile,--require-real,--dependency-strategy,--no-policy(which now warnspolicy-skipped) and--summary(refused with exit 2 when it names the schema, the profile, the lane table or a document), and exit codes 0, 1 and 2;--profile-id,--stage*,--no-nodeand--summary(exit 2 without appending when it names the record);RULE_SCOPE(exception scope keys per rule), and the exception fields;LANE_TARGETS(kept equal to the lane table by a test) andLOCK_INTERPRETER; a new lane is a new table row plus aLANE_TARGETSentry;TECHNOLOGY_KINDS, the layer kind each documented technology makes; a new hypervisor or engine is a new entry, and a test keeps the collector's tables bound to it;evidence_sourcesmatched as (workflow, job) againstrun.ci.workflow_refandrun.ci.job; a record without a measured workflow ref, or with one outside its repository or naming no workflow file or ref, links to no profile; andaccelerator_backend;STATUS_CONDITIONS, the version bounds of a lane status (kept equal to the table's cells by a test): a new bound is written in the cell as<name> <version> or neweroror olderand added there; a macOS lane needs the collector to record the macOS version before a real profile can claim its status;/attempts/<n>, with at most 10 digits, names that attempt, which must berun_attempt);nodeonly asnode.exein thePATHdirectories, with any CPython version, so a Windows lane putsnode.exeitself onPATH(anode.cmdornode.batshim is not run, and the node facts are thenunknown);platform.python_version()gives it, or as major.minor, with no leading zero in any number; a lane's interpreter is read from it whole.Open for PR C: recording
engine_mode,privilegedanddevicesfrom the engine side, a WSL2 lane record, isolation probes for systems other than Linux, and a stage interface for accelerator lanes.🤖 Generated with Claude Code
https://claude.ai/code/session_012m4sD3rnNy79fxNefxmvAU