Skip to content

Permit verified PR runs for native F/E host attestations - #814

Closed
amaybaum wants to merge 1 commit into
mainfrom
policy/ci-attestation-pr-runs-20261012
Closed

amaybaum wants to merge 1 commit into
mainfrom
policy/ci-attestation-pr-runs-20261012

Conversation

@amaybaum

Copy link
Copy Markdown
Owner

Native rounds currently require manually dispatched verification at F and E even when a green PR run tested the exact candidate tree. This proposal allows that existing PR evidence to qualify after its actual checkouts, coverage and candidate history are established.

The amendment to AGENTS.md §A.39 admits PR runs only when the recorded head is the candidate, every required repository-code job has a recorded checkout with the same tree, a synthetic merge’s base is an ancestor of the candidate, and all required jobs succeed. Event/job profiles and permitted skips are declared before F. Owner designations, frozen control planes, linear execution and reconciliation remain required.

The dispatch-only Act42 exclusion matrix can be inherited only from a named successful full run with an unchanged complete input/dependency closure and declared runtime requirements. The recorded comparison includes entry scripts, transitive imports, data/configuration, job commands and matrix setup, and package/runtime versions. Changed or unresolved inputs require a fresh full matrix run.

The operational rule in §A.40 and its lesson are aligned. V3 S5 clarifies that equivalent-tree PR evidence is a host attestation: commit-history obligations still apply to the actual F/E candidates, and no host event, synthetic merge or job result becomes a verifier predicate input. The workflow edit changes explanatory comments only. Receipt schema, verifier implementation and CI execution behavior are unchanged.

Scope: the policy applies after adoption to rounds whose F has not been designated. Historical receipts and already frozen rounds keep their existing protocol. A changed candidate needs its own qualifying run. #811 and #812 remain held; opening this proposal designates neither F nor E and converts no prior run into an attestation.

Concrete example: run 38228945877 succeeds for #812 head 1f17f1a. Its kernel checkout log identifies merge 1fbc533, whose tree d93b05a1a571f4e3eeb8ec0c4ea27021731a842e equals the candidate tree. It skips the Act42 exclusion matrix. This demonstrates tree matching; the full per-job and inherited-regression evidence would still need to be established before using that run under the amended policy.

Validation: independent policy review; git diff --check; legacy_records_check self-test and all 303 records in 75 closed namespaces intact; ci_gate_presence_test passes; an independent comparison confirms the workflow’s non-comment content is byte-identical. Full CI runs on this draft PR.

@amaybaum amaybaum left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested changes that keep this PR's goal and narrow the policy so it stays sound without tool changes. Each inline suggestion can be applied as it stands. Two points can't be made as line suggestions:

  1. Land this as a V3 round. Every earlier change to architecture.md, and every edit to §A.39 since V3-9 adopted it, went through a round with a frozen preregistration. A round gives this text the same frozen record. With the suggested scoping, each later round is then governed by the §A.39 text at its own D. The S2 sentence at architecture.md line 144 needs amending in the same change; it is outside this diff.
  2. Correct the #812 example in the description. #812's own preregistration commits it to dispatch records at F and E. Run 38228945877 can't attest #812 without an amendment, and an amendment changes the candidate. The run illustrates the checkout-log check only.

With these changes, the saving is the Act42 matrix run at F whenever main hasn't moved since D.


Generated by Claude Code

Comment thread AGENTS.md
Comment on lines +759 to +797
**The check runs at `F` and `E`.** Before the owner designates either candidate, its branch
is held at that exact commit and the applicable verification profile must complete successfully.
A `workflow_dispatch` run on that commit is admissible. A `pull_request` run is also admissible
under all of the following conditions:

- The host run's recorded head is the candidate. For every required job that executes repository
code, its actual checkout commit is recovered from the recorded job logs, recorded separately,
and its tree is exactly the candidate's tree. If the checkout is a synthetic merge, its base
must be an ancestor of the candidate; a tree containing unrelated later base changes does not
certify that candidate. A job whose checkout cannot be established does not qualify.
- The workflow and required-job profile are identified from the tested tree. Every required job,
including the round's own control shard at E, concludes success. Failed, cancelled, missing or
unexpectedly skipped jobs invalidate the attestation. Each permitted skip is named and justified
in the round's control plane before F.
- The dispatch-only Act42 exclusion matrix may be inherited from a named successful full run only
when its complete source/input dependency closure and declared runtime requirements are unchanged.
The compared set includes the entry script, transitive imports, data and configuration, relevant
job commands and matrix setup, and declared package/runtime versions. That comparison and the
prior run are recorded. A change to those inputs or the Act42 result, or an unresolved dependency
comparison, requires the full matrix to run successfully.
- Commit-history obligations are checked on the named candidate itself. Tree equality establishes
the tested source, not an equivalence of repository histories. Receipt validity remains a
separate repository-fact check.

The host record carries the run ID and attempt, event, candidate SHA, each applicable job's actual
checkout SHA and matching tree IDs, the applicable job results and any justified inherited
regression evidence. For regression inheritance it names the prior full run and the complete
compared input set, with mode/blob identities and declared runtime requirements at both subjects.
It is recorded truthfully as a PR or dispatch attestation. Only after these checks pass does the
owner designate the exact F or E. A green PR badge alone is not an attestation, and a moved head
needs its own evidence. Earlier receipts and frozen rounds retain their original protocol.

This policy applies after its adoption to rounds whose F has not yet been designated. Each such
round declares the admitted event/job profile and any regression inheritance in its control plane
before F. An existing run may be considered only for the exact unchanged candidate it tested;
a drafting amendment that changes the candidate needs a qualifying run for the new candidate.

Future workflows may check out the PR head explicitly to make the tested commit itself the
candidate; integration on the reconciled receipt commit remains separately checked before landing.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested replacement for the §A.39 paragraph. Compared with the current text, it:

  • keeps "every job green" for dispatch runs;
  • admits pull-request runs at F only. At E the evidence has nowhere durable to go: E can't contain the record of its own run, Q may add only the receipt, and the receipt tools accept only owner-designation and check-run entries. A pull-request run at E also rarely qualifies once main has moved during execution;
  • replaces the tree comparison and the undefined "its base" with an ancestry check on the logged Merge <F> into <B> line. That check implies the tree match and also covers the history-reading gate steps;
  • admits no skip other than the Act42 matrix;
  • compares Act42 inputs over a measured path set, because the matrix also loads probes through exec (a42/lib42.py, r2a_landed.py, r2b2_census.py) and reads act-41's measurements.json. It also compares the resolved runtime, not only the declared one;
  • says where the evidence is committed, restores the red-badge rule, and fixes the governing text at each round's D;
  • drops the "Future workflows…" sentence, which promises an integration check that no rule or job provides.
Suggested change
**The check runs at `F` and `E`.** Before the owner designates either candidate, its branch
is held at that exact commit and the applicable verification profile must complete successfully.
A `workflow_dispatch` run on that commit is admissible. A `pull_request` run is also admissible
under all of the following conditions:
- The host run's recorded head is the candidate. For every required job that executes repository
code, its actual checkout commit is recovered from the recorded job logs, recorded separately,
and its tree is exactly the candidate's tree. If the checkout is a synthetic merge, its base
must be an ancestor of the candidate; a tree containing unrelated later base changes does not
certify that candidate. A job whose checkout cannot be established does not qualify.
- The workflow and required-job profile are identified from the tested tree. Every required job,
including the round's own control shard at E, concludes success. Failed, cancelled, missing or
unexpectedly skipped jobs invalidate the attestation. Each permitted skip is named and justified
in the round's control plane before F.
- The dispatch-only Act42 exclusion matrix may be inherited from a named successful full run only
when its complete source/input dependency closure and declared runtime requirements are unchanged.
The compared set includes the entry script, transitive imports, data and configuration, relevant
job commands and matrix setup, and declared package/runtime versions. That comparison and the
prior run are recorded. A change to those inputs or the Act42 result, or an unresolved dependency
comparison, requires the full matrix to run successfully.
- Commit-history obligations are checked on the named candidate itself. Tree equality establishes
the tested source, not an equivalence of repository histories. Receipt validity remains a
separate repository-fact check.
The host record carries the run ID and attempt, event, candidate SHA, each applicable job's actual
checkout SHA and matching tree IDs, the applicable job results and any justified inherited
regression evidence. For regression inheritance it names the prior full run and the complete
compared input set, with mode/blob identities and declared runtime requirements at both subjects.
It is recorded truthfully as a PR or dispatch attestation. Only after these checks pass does the
owner designate the exact F or E. A green PR badge alone is not an attestation, and a moved head
needs its own evidence. Earlier receipts and frozen rounds retain their original protocol.
This policy applies after its adoption to rounds whose F has not yet been designated. Each such
round declares the admitted event/job profile and any regression inheritance in its control plane
before F. An existing run may be considered only for the exact unchanged candidate it tested;
a drafting amendment that changes the candidate needs a qualifying run for the new candidate.
Future workflows may check out the PR head explicitly to make the tested commit itself the
candidate; integration on the reconciled receipt commit remains separately checked before landing.
**The check runs at `F` and `E`.** Before the owner designates either candidate, its branch is held
at that exact commit. `E` is attested by a `workflow_dispatch` run whose `head_sha` is `E`, with
every job concluding success. `F` is attested by such a run, or by a `pull_request` run whose head
is `F` in which every job that executes repository code logs its checkout as `Merge <F> into <B>`
with `B` an ancestor of `F`; such a merge has `F`'s tree, and history-reading steps see `F`'s
history. Every job of that run concludes success except the Act42 exclusion matrix, which
pull-request runs skip by design; no other skip is admitted. A job whose checkout line is missing
or names another base does not qualify.
The pull-request attestation may inherit the Act42 result from a named successful full run at
commit `X` only when `git diff --name-only X F` is empty over the matrix's recorded input path set
and both runs record the same resolved runtime: runner image version, Python version and installed
package versions. The input path set is measured by running every exclusion part under an
import-and-open audit hook, committed, and measured again whenever the matrix runs in full on
changed inputs. Otherwise the full matrix runs.
The evidence (run ID and attempt, each job's logged checkout line and, for an inheritance, `X`,
the empty diff and both runtimes) is committed in the round's record directory at the first
execution commit, and the round's control plane lists that file. A green pull-request badge alone
is not an attestation, and a moved head needs its own evidence. A red pull-request badge on a
round held at `F` or `E` is first checked the same way: a run that tested another base is not
evidence about the candidate, and the remedy is a dispatch run on the candidate, never a merge of
main into the round before `E`. Receipt validity remains a separate repository-fact check.
A round is governed by this section as it stands at the round's `D`; earlier receipts and frozen
rounds retain their original protocol.

Generated by Claude Code

Comment thread AGENTS.md
Comment on lines +449 to +452
bridge, the release gate and the full probe suite run in CI. The qualifying host runs for `F`
and `E`, with the candidate, actual checkouts, matching trees and applicable job coverage
recorded under §A.39, are the source of certification; a local run is evidence for the executor
and is never recorded as a `check-run` attestation.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Counts the inherited Act42 run as part of what certifies, and refers to §A.39 instead of restating its contents.

Suggested change
bridge, the release gate and the full probe suite run in CI. The qualifying host runs for `F`
and `E`, with the candidate, actual checkouts, matching trees and applicable job coverage
recorded under §A.39, are the source of certification; a local run is evidence for the executor
and is never recorded as a `check-run` attestation.
bridge, the release gate and the full probe suite run in CI. The qualifying host runs for `F`
and `E` under §A.39, with any inherited Act42 run they cite, are the source of certification; a
local run is evidence for the executor and is never recorded as a `check-run` attestation.

Generated by Claude Code

Comment thread AGENTS.md
Comment on lines +482 to +484
otherwise. Hence §A.39's candidate-source check: compare the recorded PR checkout and its tree
with the held candidate before calling a red badge a research failure, and never cure it by
merging main into a certified execution.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

§A.39 doesn't define a "candidate-source check". This points the lesson at the red-badge rule that the suggestion on the §A.39 paragraph restores.

Suggested change
otherwise. Hence §A.39's candidate-source check: compare the recorded PR checkout and its tree
with the held candidate before calling a red badge a research failure, and never cure it by
merging main into a certified execution.
otherwise. Hence §A.39's red-badge rule: compare the run's logged checkout with the held
candidate before calling a red badge a research failure, and never cure it by merging main into
a certified execution.

Generated by Claude Code

Comment on lines +266 to +271
Admissibility of a host check as evidence for F or E is the operational policy in AGENTS.md
section A.39. That policy may accept a PR checkout with the exact candidate tree and recorded
job coverage; the tested checkout and the candidate remain separately identified. The required
checks on exactly F or E in S2 and S3 refer to the named candidate under that policy. No event,
synthetic merge or job result becomes an input to a V3 predicate, and tree equality does not
replace checks of the candidate's repository history.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches the narrowed §A.39 text:

  • pull-request runs are admitted at F only;
  • the governing text is fixed by each round's D;
  • an inherited Act42 result is described as an attestation about its own commit, as S6 requires.

S2's own sentence at line 144 ("the required checks have passed on exactly F") should be amended in the same change rather than redefined from here. It is outside this diff, so I can't attach a suggestion to it. S3 needs no change under this version, because E keeps dispatch runs.

Suggested change
Admissibility of a host check as evidence for F or E is the operational policy in AGENTS.md
section A.39. That policy may accept a PR checkout with the exact candidate tree and recorded
job coverage; the tested checkout and the candidate remain separately identified. The required
checks on exactly F or E in S2 and S3 refer to the named candidate under that policy. No event,
synthetic merge or job result becomes an input to a V3 predicate, and tree equality does not
replace checks of the candidate's repository history.
Admissibility of a host check as evidence for F or E is the operational policy in AGENTS.md
section A.39 as it stands at the round's D. That policy may accept, for F, a pull-request run
that tested a merge of F into an ancestor of F, which has F's tree and history; the run and the
candidate remain separately identified. An Act42 result inherited under that policy is a host
attestation about the commit where it was produced (S6) and is recorded as such. No event,
synthetic merge or job result becomes an input to a V3 predicate.

Generated by Claude Code

Comment on lines +414 to +416
# run only on workflow_dispatch. Section A.39 permits a PR attestation to inherit a prior full
# exclusion run only after the complete input closure and runtime requirements are compared.
# This event condition alone establishes no inherited evidence.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

§A.39 requires the Act42 inputs to be unchanged, not just compared.

Suggested change
# run only on workflow_dispatch. Section A.39 permits a PR attestation to inherit a prior full
# exclusion run only after the complete input closure and runtime requirements are compared.
# This event condition alone establishes no inherited evidence.
# run only on workflow_dispatch. Section A.39 lets a pull-request attestation of F inherit a
# prior full exclusion run only when its recorded input paths and resolved runtime are unchanged.
# This event condition alone establishes no inherited evidence.

Generated by Claude Code

Copy link
Copy Markdown
Owner Author

Closing without merging, on the owner's decision to keep the current rule: F and E are attested by workflow_dispatch runs on the exact candidate, with every job green.

The review suggestions above are superseded. Three of their claims were wrong:

  • An ancestor-base merge has the candidate's tree, but its first-parent chain runs through the base, not the candidate. So "history-reading steps see F's history" does not hold.
  • An import-and-open audit hook in the parent process misses reads by the separate Python processes the Act42 matrix launches.
  • E evidence can be recorded durably: the receipt's record field can cite a PR comment, as owner designations already do. Limiting PR attestations to F was a policy choice, not a tool limit.

Generated by Claude Code

@amaybaum amaybaum closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant