diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/CHUNK_MAP.md index 98629aa2..c975717b 100644 --- a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/CHUNK_MAP.md @@ -3,11 +3,13 @@ | Order | Chunk | Purpose | Risk | State | |---:|---|---|---:|---| | 0 | `WS-ENG-007-00R1` | Repair planning-intake file/tree parity and recover PR #187 exactly once | L1/P0 | Merged; recovery superseded after rerun-cardinality failure | -| 0 | `WS-ENG-007-00R2` | Canonicalize repeated trusted check evidence and reconcile PRs #187 and #188 exactly once | L1/P0 | Proposed reliability recovery | -| 1 | `WS-ENG-007-01` | Add deterministic reviewed-patch identity and conservative base-delta review preservation | L1 | Blocked on 00R2 merge, successful three-merge reconciliation, and explicit start | +| 0 | `WS-ENG-007-00R2` | Canonicalize repeated trusted check evidence and reconcile PRs #187 and #188 exactly once | L1/P0 | Merged; superseded by mutable-history failure | +| 0 | `WS-ENG-007-00R3` | Freeze accepted checks at merge time and give explicit starts deterministic recovery parity | L1/P0 | Active fail-closed recovery | +| 1 | `WS-ENG-007-01` | Add deterministic reviewed-patch identity and conservative base-delta review preservation | L1 | Blocked on 00R3 merge, successful four-merge reconciliation, and explicit start | | 2 | `WS-ENG-007-02` | Add structured reviewer-track and upstream-finding reconciliation | L1 | Blocked on 01 merge and explicit start | | 3 | `WS-ENG-007-03` | Add merge-group CI parity and queue-readiness proof | L1 | Blocked on 02 merge and explicit start | -Recovery chunks are exceptional ordered prerequisites; implementation chunks +Recovery chunks are exceptional ordered prerequisites; `00R3` supersedes the +incomplete mutable-evidence behavior left after `00R2`. Implementation chunks remain one PR each and stop after merge. Every successor requires a separate explicit signed start. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DECISIONS.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DECISIONS.md index b0bf20cc..4621a107 100644 --- a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DECISIONS.md +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DECISIONS.md @@ -1,5 +1,16 @@ # Decisions: WS-ENG-007 - Concurrent PR Review Reconciliation +## D0R3 - Freeze protected evidence at merge time + +Protected check evidence is selected against immutable PR `merged_at`, not the +latest present-day rerun. Signed records bind the exact selected-run provenance. +Merge and explicit-start workflows use one bounded reconciliation sequence. The +remaining backlog is crossed once by an exact four-entry bridge; no wildcard or +durable bypass is introduced. + +This decision supersedes the present-day latest-run portions of D12 and D13 +below. Those decisions remain historical context for 00R2 only. + ## D1 - CI always reruns on the combined tree Review preservation never preserves CI results across a base advance. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DISCOVERY.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DISCOVERY.md index adf0c44d..39275435 100644 --- a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DISCOVERY.md +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/DISCOVERY.md @@ -3,6 +3,21 @@ Discovery was performed read-only against trusted `main` at merge `9033a97a4be6d762cae4b210018ef81a079395c2`. +## 00R3 recovery addendum — 2026-07-23 + +- Signed state remains at PR #178 (`73b4579…`) while protected `main` contains + PRs #187, #188, and #189 in exact first-parent order. +- The merge workflow prepares and consumes bounded recovery state; the explicit + start workflow replays merges without that recovery sequence. +- GitHub retains mutable reruns on immutable PR heads. PR #189 acquired failing + `agent-gates` reruns after its merge, so present-day latest-run selection can + rewrite evidence that existed at merge time. +- The check-runs API is paginated. One 100-item page cannot prove completeness + after later rerun noise accumulates. +- The root repair needs persisted merge-bound protected-check provenance, one + exact four-merge bridge through 00R3, and a shared atomic reconciliation path + for merge and explicit-start workflows. + ## Repository behavior found - Branch protection requires `agent-gates` and `test`, uses strict up-to-date diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/RISKS.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/RISKS.md index d8d8ed4d..5caee86c 100644 --- a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/RISKS.md +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/RISKS.md @@ -15,3 +15,7 @@ | Legitimate check reruns block signed memory | Critical | Validate every same-name candidate, order completed trusted invocations by parsed start instant and positive check-run ID, then require the unique latest candidate to succeed. | | A stale success hides a later failure | Critical | Latest validated run wins; an in-progress, failed, cancelled, foreign-app, malformed, or timestamp-ambiguous latest run fails closed. | | Three-merge repair becomes general authority | Critical | Schema v3 permits at most two recovered merges and production names only PR #187, PR #188, and 00R2; require exact first-parent adjacency, unique identities, full consumption before signing, and no serialization. | +| Mutable post-merge reruns rewrite accepted evidence | Critical | Select protected runs at immutable merge time, persist exact selected provenance and digest, and ignore only runs provably started after the cutoff. | +| Recovery cannot cross PR #189 and its own activation | Critical | Use one schema-v4 exact four-entry bridge from signed PR #178 through PRs #187–#189 to 00R3, then consume it completely. | +| Explicit starts reconcile differently from merge automation | High | Use one plan/prepare/update/assert implementation and prove atomic parity before applying authority. | +| Later reruns hide evidence beyond the first API page | High | Bounded complete pagination with total, overlap, drift, and exhaustion rejection. | diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/STATUS.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/STATUS.md index f776f7ed..b6ef06d7 100644 --- a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/STATUS.md +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/STATUS.md @@ -1,16 +1,18 @@ # STATUS: WS-ENG-007 - Concurrent PR Review Reconciliation -- Phase: canonical check-evidence recovery +- Phase: merge-bound check-evidence recovery - Gate: fail-closed automation repair - Active planning chunk: none - Active implementation chunk: none - Completed but unreconciled recovery chunk: `WS-ENG-007-00R1` -- Proposed recovery chunk: `WS-ENG-007-00R2` +- Merged but unreconciled recovery chunk: `WS-ENG-007-00R2` +- Active recovery chunk: `WS-ENG-007-00R3` - Proposed implementation successor after recovery: `WS-ENG-007-01` - Separate explicit start required: true -- Current gate: PR #188 merged at - `c65633f8f0991dbefe7b0635e053aab0df8f9af8`. Its tree normalization repair is - correct, but signed reconciliation still fails because PR #187 has two - successful trusted `agent-gates` reruns and planning intake incorrectly - treats repeated check evidence as ambiguous. Signed state remains at - `73b457925b02301587b83d01ced0adb66319d134`; no successor is active. +- Current gate: PRs #187–#189 are merged, but signed reconciliation still fails + because mutable post-merge check history is re-evaluated and the explicit + start workflow lacks the merge workflow's recovery path. Signed state remains + at `73b457925b02301587b83d01ced0adb66319d134`; no successor is active. +- Review gate: all required internal tracks passed reviewed implementation + `4a3eee7fdd80970675ae2574542073b8a92e1ef9`; awaiting external checks and the + user-owned merge decision. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/chunks/WS-ENG-007-00R3-merge-bound-evidence-recovery.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/chunks/WS-ENG-007-00R3-merge-bound-evidence-recovery.md new file mode 100644 index 00000000..f55d2229 --- /dev/null +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/chunks/WS-ENG-007-00R3-merge-bound-evidence-recovery.md @@ -0,0 +1,140 @@ +# Chunk Contract: WS-ENG-007-00R3 — Merge-Bound Evidence Recovery + +## Parent initiative + +`WS-ENG-007` — Concurrent PR Review Reconciliation + +## Goal + +Restore signed reconciliation and explicit starts without allowing mutable +post-merge check reruns to rewrite the evidence accepted for an already merged +PR. + +## Why this chunk exists + +PRs #187–#189 are on protected `main`, but signed memory remains at PR #178. +The recovery path re-queries mutable check history, rejects later rerun shapes, +and exists only in the merge workflow. PR #189 also merged without completed +pre-merge `agent-gates` evidence. Consequently an ordinary explicit start cannot +reconcile the same trusted history and unrelated ENG-006 work is blocked. + +## Risk class + +L1 / P0 reliability repair. + +## Start phase + +Recovery implementation. This contract records the exact fail-closed repair +needed to restore the signed start mechanism that is itself unavailable. + +## Allowed files + +```text +.github/workflows/loop-memory.yml +.github/workflows/loop-memory-start.yml +scripts/update_post_merge_memory.py +scripts/check_loop_memory_state.py +scripts/test_update_post_merge_memory.py +scripts/test_check_loop_memory_state.py +scripts/test_agent_gates.py +docs/operations_post_merge_memory.md +.agent-loop/policies/loop-memory-recovery.json +.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/** +.agent-loop/merge-intents/WS-ENG-007-00R3.json +``` + +## Not allowed + +```text +application, API, database, product, auth, payment, or coverage changes +repository permissions, secrets, branch protection, or cancel authority changes +weakening required checks, signed starts, internal review, or human merge approval +any wildcard, reusable, reorderable, or persistent recovery exception +automatic successor starts +``` + +## Acceptance criteria + +- [ ] For each protected name, classify every candidate against immutable + `merged_at`; ignore only candidates started after the merge. Select the + latest candidate started no later than the merge by normalized RFC3339 + instant and positive unique ID, and require it to have completed + successfully no later than the merge. An older success cannot hide a + newer eligible failure or incomplete run. +- [ ] Malformed timestamps or identities that prevent trustworthy boundary + classification, incomplete pagination, forged app, wrong head, duplicate + ID, non-successful selected evidence, and ambiguity fail closed. +- [ ] Check retrieval is completely paginated with deterministic deduplication, + explicit page/item bounds, and rejection of total drift, overlap, or bound + exhaustion. More than 100 later reruns cannot hide eligible evidence. +- [ ] Signed merge records persist selected run ID, protected name, head SHA, + pinned app, normalized start/completion instants, conclusion, merge cutoff, + and a canonical evidence digest; the independent checker validates them. +- [ ] CodeRabbit remains supplementary external evidence and is not converted + into signed start or merge authority. +- [ ] Merge reconciliation and explicit-start reconciliation use one shared, + deterministic recovery sequence and consume the recovery inventory before + signing or publishing. +- [ ] A schema-v4 closed recovery certificate binds signed basis + `73b457925b02301587b83d01ced0adb66319d134`, exact PR #187 / `PLAN` / + `8928ba80…`, PR #188 / `00R1` / `c65633f8…`, PR #189 / `00R2` / + `d3321698…`, and direct-next activation `00R3`. It requires exact adjacent + order `[8928ba8…, c65633f…, d332169…, target]` and a four-entry bounded + ephemeral transport. All entries are consumed before signing and none is + serialized or reusable. +- [ ] Protected merge-bound evidence is required for PRs #187, #188, and the + new `00R3` activation PR. Only exact historical PR #189 may carry the + closed `historical-recovery-only` evidence mode: its exact merge/head/chunk, + absence reason, and certificate digest are independently validated. That + mode is impossible for `00R3` or any future merge. +- [ ] Policy v4 has exactly `schema_version`, `signed_basis`, `activation`, and + three ordered `recovered_merges`; transport v3 has exactly four + chronological unique identities. Policy/transport v1 and v2 bounds remain + unchanged. Unknown keys, basis mismatch, collisions, 0–3/5 entries, + reordering, wrong parents, and legacy-version widening reject. +- [ ] Once history is reconciled, distinct idle initiatives can start + independently and a later rerun on an older PR cannot block them. +- [ ] Repeated reconciliation and repeated start dispatch remain idempotent or + fail closed with a stable, actionable error. +- [ ] Both workflows call one shared reconciliation implementation in this + order: plan, prepare recovery, update every merge with the same inventory, + assert consumption, then sign/publish or apply the start. Failure publishes + neither partial state nor authority event. +- [ ] Exactly one schema-v2 merge intent names `WS-ENG-007-01` with + `next_requires_explicit_start: true`; ENG-006 and ENG-007 successors remain + inactive after recovery. + +## Verification commands + +```bash +PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q scripts/test_update_post_merge_memory.py scripts/test_check_loop_memory_state.py scripts/test_agent_gates.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +python3 scripts/update_post_merge_memory.py validate-merge-intent --repository-root . --base-ref origin/main +git diff --check origin/main...HEAD +``` + +## Required reviewers + +- senior engineering +- QA/test +- security/auth +- product/ops +- architecture +- CI integrity +- docs +- reuse/dedup +- test delta + +## Human review focus + +- Is historical evidence frozen at the merge boundary without accepting a + failing or untrusted pre-merge run? +- Can the explicit-start path reconcile exactly as safely as the merge path? +- Does this remove the global mutable-history lock without weakening the + initiative-local signed-start rule? + +## Stop conditions + +Stop if the repair requires weakening protected checks, adding a new historical +PR exemption, changing GitHub authority, or allowing unsigned implementation. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-external-review-response.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-external-review-response.md new file mode 100644 index 00000000..566c5329 --- /dev/null +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-external-review-response.md @@ -0,0 +1,36 @@ +# External Review Response: WS-ENG-007-00R3 + +## Comments addressed + +- Added an explicit statement that the `eng006_*` reviewer sessions were + inherited and reassigned to this exact ENG-007 review. +- Extended the shared-reconciliation test to prove only exact PR #189 is + collected with `historical_recovery=True`. +- Extracted the duplicated PR #189 predicate into + `_is_r3_historical_recovery()` and reused it in both production call sites. +- Expanded the PR description to expose the complete trust-bundle sections. + +## Comments deferred + +- CodeRabbit's docstring-coverage warning is stale. The authoritative backend + preflight ran the repository docstring gate successfully on the exact PR head. +- Usage-limit notices are service availability metadata, not code findings. + +## Human decisions needed + +None. All valid comments are in scope and addressed without changing authority. + +## Commands rerun + +```bash +PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q scripts/test_update_post_merge_memory.py -k 'shared_reconcile_orders or historical_recovery' +python3 scripts/test_agent_gates.py +PR_HEAD_SHA="$(git rev-parse HEAD)" python3 scripts/check_internal_review_evidence.py +python3 scripts/check_markdown_links.py +git diff --check origin/main...HEAD +``` + +## Remaining risks + +CodeRabbit may remain rate-limited. Internal reviews and repository checks remain +the blocking evidence; CodeRabbit is supplementary under repository policy. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-internal-review-evidence.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-internal-review-evidence.md new file mode 100644 index 00000000..11ced4d6 --- /dev/null +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-internal-review-evidence.md @@ -0,0 +1,80 @@ +# Internal Review Evidence: WS-ENG-007-00R3 + +## Chunk + +`WS-ENG-007-00R3` — Merge-Bound Evidence Recovery + +open sub-agent sessions: none + +valid findings addressed: yes + +## Reviewed Revision + +Reviewed code SHA: 4a3eee7fdd80970675ae2574542073b8a92e1ef9 + +Reviewed at: 2026-07-23T10:30:00Z + +The `/root/eng006_*` sessions were inherited from the ENG-006 reviewer pool and +explicitly reassigned to this WS-ENG-007-00R3 review. Every track is rebound to +the exact reviewed implementation SHA below; the session name grants no +cross-initiative authority. + +Reviewer run IDs: senior-engineering=/root/eng006_senior_arch_docs; QA/test=/root/eng006_qa_ci_tests; security/auth=/root/eng006_security_ops_reuse; product/ops=/root/eng006_security_ops_reuse; architecture=/root/eng006_senior_arch_docs; docs=/root/eng006_senior_arch_docs; CI-integrity=/root/eng006_qa_ci_tests; reuse/dedup=/root/eng006_security_ops_reuse; test-delta=/root/eng006_qa_ci_tests + +## Reviewer Results + +| Reviewer | Result | Blocking findings | Notes | +|---|---:|---|---| +| senior engineering | PASS AFTER FIXES | None | Merge cutoff and recovery architecture accepted. | +| QA/test | PASS AFTER FIXES | None | CodeRabbit is supplementary; proof matrix accepted. | +| security/auth | PASS AFTER FIXES | None | Exact one-use authority and replay boundaries accepted. | +| product/ops | PASS | None | No product lifecycle behavior changed. | +| architecture | PASS AFTER FIXES | None | One shared reconciliation reducer. | +| CI integrity | PASS AFTER FIXES | None | No CI, coverage, or required-check weakening. | +| docs | PASS AFTER FIXES | None | Runbook and durable artifacts aligned. | +| reuse/dedup | PASS AFTER FIXES | None | Workflow duplication removed. | +| test delta | PASS AFTER FIXES | None | Cutoff, pagination, checker, certificate, and reducer tests added. | + +## Valid Findings Addressed + +- Filtered classifiable post-merge reruns before validating eligible protected + evidence, and removed the redundant mutable planning-intake validator. +- Removed CodeRabbit from planning and recovery authority while retaining it as + diagnostic external evidence. +- Persisted and independently validated exact protected-run provenance with a + strict merge cutoff and canonical RFC3339 timestamps. +- Bound PR #189's sole recovery-only mode to its exact PR, merge, head, chunk, + signed basis, activation, policy schema, reason, and certificate digest. +- Added bounded complete pagination with stable totals and global ID overlap + rejection. +- Replaced duplicated YAML reducers with one atomic `reconcile` command shared + by merge and explicit-start workflows. +- Added cutoff poisoning, pagination drift, mandatory-cutover, strict timestamp, + certificate mutation, and shared-reducer tests. + +## Commands Run + +```bash +PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 python3 -m pytest -q scripts/test_update_post_merge_memory.py scripts/test_check_loop_memory_state.py +python3 scripts/test_agent_gates.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_workstream_wording.py +git diff --check origin/main...HEAD +python3 -m py_compile scripts/update_post_merge_memory.py scripts/check_loop_memory_state.py +``` + +Final exact-head verification passed 278 updater-gate tests at 90.10 percent +branch coverage and the independent checker at 90.41 percent. Agent Gates +passed all 98 standalone scenarios. All reviewer tracks reapproved exact head +`4a3eee7fdd80970675ae2574542073b8a92e1ef9` after the CodeRabbit repairs. + +## Remaining Risks + +The schema-v4 bridge is intentionally exact. Any intervening protected-main +merge before this chunk invalidates adjacency and requires a freshly reviewed +certificate rather than reinterpretation. + +## Stop Condition + +No successor is active. ENG-006 documentation work and ENG-007-01 each require +their ordinary explicit signed start after successful reconciliation. diff --git a/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-pr-trust-bundle.md new file mode 100644 index 00000000..559fd051 --- /dev/null +++ b/.agent-loop/initiatives/WS-ENG-007-concurrent-pr-review-reconciliation/reviews/WS-ENG-007-00R3-pr-trust-bundle.md @@ -0,0 +1,43 @@ +# PR Trust Bundle: WS-ENG-007-00R3 + +## Intent + +Restore signed loop-memory reconciliation without allowing mutable post-merge +reruns to block unrelated initiatives or rewrite accepted merge evidence. + +## Scope + +- Freeze protected `agent-gates` and backend `test` evidence at PR merge time. +- Persist selected-run provenance in signed records. +- Recover exact PRs #187–#189 plus this activation once from signed PR #178. +- Use one atomic reconciliation reducer in merge and explicit-start workflows. + +## Reviewed Revision + +`4a3eee7fdd80970675ae2574542073b8a92e1ef9` + +## CI Integrity + +- No test, coverage floor, required reviewer, human merge checkpoint, or signed + start rule was removed or weakened. +- CodeRabbit remains supplementary external evidence. +- Both workflows fail before signing, publication, or authority application if + reconciliation or recovery consumption fails. + +## Reviewer Result + +All nine required internal tracks passed after fixes. No Critical or High +finding remains open. + +## Human Review Focus + +- Confirm only exact historical PR #189 receives recovery-only evidence. +- Confirm the new activation PR still requires ordinary protected evidence. +- Confirm both workflows invoke the same `reconcile` command before authority or + publication. +- Confirm the four-entry bridge is exact, consumed, and non-reusable. + +## Remaining Risk + +An intervening `main` merge invalidates the exact recovery bridge and must fail +closed. diff --git a/.agent-loop/merge-intents/WS-ENG-007-00R3.json b/.agent-loop/merge-intents/WS-ENG-007-00R3.json new file mode 100644 index 00000000..90eeaef9 --- /dev/null +++ b/.agent-loop/merge-intents/WS-ENG-007-00R3.json @@ -0,0 +1,9 @@ +{ + "chunk_id": "WS-ENG-007-00R3", + "chunk_title": "Merge-Bound Evidence Recovery", + "initiative_id": "WS-ENG-007", + "next_chunk_id": "WS-ENG-007-01", + "next_chunk_title": "Reviewed Patch and Base-Delta Reconciliation", + "next_requires_explicit_start": true, + "schema_version": 2 +} diff --git a/.agent-loop/policies/loop-memory-recovery.json b/.agent-loop/policies/loop-memory-recovery.json index e5e1e5fa..39988cdd 100644 --- a/.agent-loop/policies/loop-memory-recovery.json +++ b/.agent-loop/policies/loop-memory-recovery.json @@ -1,8 +1,9 @@ { "activation": { - "chunk_id": "WS-ENG-007-00R2", + "chunk_id": "WS-ENG-007-00R3", "initiative_id": "WS-ENG-007" }, + "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", "recovered_merges": [ { "chunk_id": "WS-ENG-007-PLAN", @@ -15,7 +16,13 @@ "initiative_id": "WS-ENG-007", "merge_sha": "c65633f8f0991dbefe7b0635e053aab0df8f9af8", "pr_number": 188 + }, + { + "chunk_id": "WS-ENG-007-00R2", + "initiative_id": "WS-ENG-007", + "merge_sha": "d3321698fb856f3fac320cdc7bc598f813fe1953", + "pr_number": 189 } ], - "schema_version": 3 + "schema_version": 4 } diff --git a/.github/workflows/loop-memory-start.yml b/.github/workflows/loop-memory-start.yml index 312b17f6..a7ec9156 100644 --- a/.github/workflows/loop-memory-start.yml +++ b/.github/workflows/loop-memory-start.yml @@ -95,18 +95,10 @@ jobs: python3 scripts/update_post_merge_memory.py prepare-output \ --source-root "${branch_dir}" --output-root "${output_dir}" \ --public-key .agent-loop/keys/loop-memory-signing-public.pem - current_sha=$(python3 -c \ - 'import json,sys; s=json.load(open(sys.argv[1],encoding="utf-8")); print(s.get("event",{}).get("main_sha") or s["source"]["main_sha"])' \ - "${output_dir}/.agent-loop/STATE.json") - plan_file=$(mktemp) - python3 scripts/update_post_merge_memory.py plan-commits \ - --repository-root . --target-sha "${TARGET_SHA}" \ - --current-sha "${current_sha}" > "${plan_file}" - while IFS= read -r merge_sha; do - [[ -z "${merge_sha}" ]] || python3 scripts/update_post_merge_memory.py update \ - --repository "${GITHUB_REPOSITORY}" --merge-sha "${merge_sha}" \ - --state-root "${output_dir}" --branch-root "${branch_dir}" - done < "${plan_file}" + python3 scripts/update_post_merge_memory.py reconcile \ + --repository "${GITHUB_REPOSITORY}" --repository-root . \ + --state-root "${output_dir}" --branch-root "${branch_dir}" \ + --target-sha "${TARGET_SHA}" echo "branch_dir=${branch_dir}" >> "${GITHUB_OUTPUT}" echo "output_dir=${output_dir}" >> "${GITHUB_OUTPUT}" echo "prior_tip=$(git -C "${branch_dir}" rev-parse HEAD)" >> "${GITHUB_OUTPUT}" diff --git a/.github/workflows/loop-memory.yml b/.github/workflows/loop-memory.yml index ffc35110..d8db47e8 100644 --- a/.github/workflows/loop-memory.yml +++ b/.github/workflows/loop-memory.yml @@ -100,42 +100,10 @@ jobs: echo "Invalid protected-main target SHA." >&2 exit 1 fi - current_sha="" - if [[ -f "${state_dir}/.agent-loop/STATE.json" ]]; then - current_sha=$(python3 -c \ - 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8"))["source"]["main_sha"])' \ - "${state_dir}/.agent-loop/STATE.json") - fi - plan_file=$(mktemp) - plan_args=( - plan-commits - --repository-root . - --target-sha "${TARGET_SHA}" - ) - if [[ -n "${current_sha}" ]]; then - plan_args+=(--current-sha "${current_sha}") - fi - python3 scripts/update_post_merge_memory.py "${plan_args[@]}" > "${plan_file}" - mapfile -t merge_shas < "${plan_file}" - recovery_file=$(mktemp) - python3 scripts/update_post_merge_memory.py prepare-recovery \ + python3 scripts/update_post_merge_memory.py reconcile \ --repository "${GITHUB_REPOSITORY}" --repository-root . \ - --state-root "${state_dir}" --target-sha "${TARGET_SHA}" \ - --plan-file "${plan_file}" > "${recovery_file}" - for index in "${!merge_shas[@]}"; do - merge_sha="${merge_shas[${index}]}" - python3 scripts/update_post_merge_memory.py update \ - --repository "${GITHUB_REPOSITORY}" \ - --repository-root . \ - --merge-sha "${merge_sha}" \ - --state-root "${state_dir}" \ - --branch-root "${branch_dir}" \ - --cutover-chunk-id WS-ENG-001-04B \ - --recovery-file "${recovery_file}" - done - python3 scripts/update_post_merge_memory.py assert-recovery-consumed \ - --state-root "${state_dir}" --target-sha "${TARGET_SHA}" \ - --recovery-file "${recovery_file}" + --state-root "${state_dir}" --branch-root "${branch_dir}" \ + --target-sha "${TARGET_SHA}" - name: Validate generated state shell: bash diff --git a/docs/operations_post_merge_memory.md b/docs/operations_post_merge_memory.md index f75fd572..9281f9dc 100644 --- a/docs/operations_post_merge_memory.md +++ b/docs/operations_post_merge_memory.md @@ -1,5 +1,19 @@ # Post-Merge Memory Operations +## Merge-bound check evidence + +Protected `agent-gates` and backend `test` evidence is frozen at the immutable +PR merge instant. Later reruns remain useful diagnostics but cannot replace or +invalidate the selected pre-merge run in signed history. The selected run IDs, +trusted GitHub Actions app identity, reviewed head, timestamps, conclusion, +merge cutoff, and canonical digest are stored in generated signed memory. + +Both automatic merge reconciliation and an explicit start first reconcile exact +current `main` in temporary custody through the same plan, bounded recovery, +update, and full-consumption checks. Only after that succeeds may automation +apply a start, sign, and publish. No partial recovery state or authority event is +published on failure. + ## Purpose Managers review and approve each implementation PR once. Workstream then diff --git a/scripts/check_loop_memory_state.py b/scripts/check_loop_memory_state.py index 51497cd7..dd19dadc 100644 --- a/scripts/check_loop_memory_state.py +++ b/scripts/check_loop_memory_state.py @@ -107,9 +107,14 @@ SCHEMA_VERSION = 2 STATE_BRANCH = "automation/loop-memory" REQUIRED_CHECKS = ("agent-gates", "test", "CodeRabbit") +GITHUB_ACTIONS_APP_ID = 15368 +GITHUB_ACTIONS_APP_SLUG = "github-actions" +R3_RECOVERY_CERTIFICATE_SHA256 = "4fe49b2f4a5a7ad18382a717dcc11f798c465a534066102bf9810c9ed5784f4a" +R3_HISTORICAL_HEAD_SHA = "55a11d9e0ae356734dbcce73564f5f570220a81b" ID_PATTERN = re.compile(r"^[A-Z][A-Z0-9]*(?:-[A-Z0-9]+)+$") SHA_PATTERN = re.compile(r"^[0-9a-f]{40}$") SHA256_PATTERN = re.compile(r"^[0-9a-f]{64}$") +RFC3339_PATTERN = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})$") REPOSITORY_PATTERN = re.compile(r"^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$") PLANNING_ROOT_FILES = { "INTENT.md", @@ -588,9 +593,13 @@ def _record_failures( } if not isinstance(record, dict) or frozenset(record) not in { frozenset(expected), + frozenset(expected | {"protected_checks"}), frozenset(expected | {"legacy_exemptions"}), + frozenset(expected | {"legacy_exemptions", "protected_checks"}), frozenset(expected | {"planning_intake"}), + frozenset(expected | {"planning_intake", "protected_checks"}), frozenset(expected | {"planning_intake", "legacy_exemptions"}), + frozenset(expected | {"planning_intake", "legacy_exemptions", "protected_checks"}), }: return [f"{label}: invalid record schema"] failures: list[str] = [] @@ -790,8 +799,46 @@ def _record_failures( ) if checks.get("all_required_passed") is not calculated: failures.append(f"{label}: inconsistent aggregate check evidence") - if planning_intake is not None and calculated is not True: - failures.append(f"{label}: planning intake required checks did not pass") + protected = record.get("protected_checks") + if protected is None and isinstance(source.get("merged_at"), str): + try: + if datetime.fromisoformat(source["merged_at"].replace("Z", "+00:00")) >= datetime.fromisoformat("2026-07-23T05:11:46+00:00"): + failures.append(f"{label}: merge-bound protected check evidence is required") + except ValueError: + failures.append(f"{label}: invalid protected check cutoff") + if protected is not None: + if not isinstance(protected, dict) or protected.get("schema_version") != 1: + failures.append(f"{label}: invalid protected check evidence") + elif set(protected) == {"schema_version", "recovery_only", "sha256"}: + recovery = protected.get("recovery_only") + expected_recovery = {"merge_sha": "d3321698fb856f3fac320cdc7bc598f813fe1953", "head_sha": R3_HISTORICAL_HEAD_SHA, "chunk_id": "WS-ENG-007-00R2", "pr_number": 189, "policy_schema": 4, "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", "activation_chunk_id": "WS-ENG-007-00R3", "certificate_sha256": R3_RECOVERY_CERTIFICATE_SHA256, "reason": "no-completed-pre-merge-agent-gates"} + if recovery != expected_recovery or source.get("head_sha") != R3_HISTORICAL_HEAD_SHA or source.get("pr_number") != 189 or protected.get("sha256") != hashlib.sha256(json.dumps(expected_recovery, sort_keys=True, separators=(",", ":")).encode()).hexdigest(): + failures.append(f"{label}: invalid historical recovery evidence") + elif set(protected) == {"schema_version", "selected", "sha256"}: + selected = protected.get("selected") + if not isinstance(selected, dict) or set(selected) != {"agent-gates", "test"} or protected.get("sha256") != hashlib.sha256(json.dumps(selected, sort_keys=True, separators=(",", ":")).encode()).hexdigest(): + failures.append(f"{label}: invalid protected check digest") + else: + seen_selected_ids: set[int] = set() + for name, item in selected.items(): + expected_keys = {"id", "head_sha", "app_id", "app_slug", "started_at", "completed_at", "conclusion", "merge_cutoff"} + item_id = item.get("id") if isinstance(item, dict) else None + if not isinstance(item, dict) or set(item) != expected_keys or type(item_id) is not int or item_id <= 0 or item_id in seen_selected_ids or item.get("head_sha") != source.get("head_sha") or item.get("app_id") != GITHUB_ACTIONS_APP_ID or item.get("app_slug") != GITHUB_ACTIONS_APP_SLUG or item.get("conclusion") != "success" or item.get("merge_cutoff") != source.get("merged_at"): + failures.append(f"{label}: invalid protected check provenance for {name}") + continue + seen_selected_ids.add(item_id) + try: + if not all(isinstance(item[field], str) and RFC3339_PATTERN.fullmatch(item[field]) for field in ("started_at", "completed_at", "merge_cutoff")): + raise ValueError("non-canonical timestamp") + started = datetime.fromisoformat(item["started_at"].replace("Z", "+00:00")) + completed = datetime.fromisoformat(item["completed_at"].replace("Z", "+00:00")) + cutoff = datetime.fromisoformat(item["merge_cutoff"].replace("Z", "+00:00")) + if started.tzinfo is None or completed.tzinfo is None or cutoff.tzinfo is None or not started <= completed <= cutoff: + failures.append(f"{label}: invalid protected check timing for {name}") + except (AttributeError, TypeError, ValueError): + failures.append(f"{label}: invalid protected check timing for {name}") + else: + failures.append(f"{label}: invalid protected check evidence schema") return failures diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 7f677639..8c191384 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -1893,7 +1893,7 @@ def test_planning_intake_record_schema_fails_closed() -> None: ("planning intake", lambda item: item["planning_intake"].update(delta_sha256="bad")), ("intent path", lambda item: item["completed_chunk"].update(chunk_id="WS-NEW-001-01")), ("planning intake", lambda item: item["completed_chunk"].update(next_requires_explicit_start=False)), - ("planning intake", lambda item: item["checks"].update(all_required_passed=False)), + ("aggregate check evidence", lambda item: item["checks"].update(all_required_passed=False)), ) for expected, mutate in mutations: record = planning_intake_record(updater) @@ -2148,9 +2148,10 @@ def test_ws_eng_007_recovery_policy_is_exactly_pinned() -> None: policy = json.loads(Path(".agent-loop/policies/loop-memory-recovery.json").read_text()) assert policy == { "activation": { - "chunk_id": "WS-ENG-007-00R2", + "chunk_id": "WS-ENG-007-00R3", "initiative_id": "WS-ENG-007", }, + "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", "recovered_merges": [ { "chunk_id": "WS-ENG-007-PLAN", @@ -2164,8 +2165,14 @@ def test_ws_eng_007_recovery_policy_is_exactly_pinned() -> None: "merge_sha": "c65633f8f0991dbefe7b0635e053aab0df8f9af8", "pr_number": 188, }, + { + "chunk_id": "WS-ENG-007-00R2", + "initiative_id": "WS-ENG-007", + "merge_sha": "d3321698fb856f3fac320cdc7bc598f813fe1953", + "pr_number": 189, + }, ], - "schema_version": 3, + "schema_version": 4, } @@ -2426,21 +2433,6 @@ def get_json(self, path: str): "delta_sha256": evidence["delta_sha256"], "changed_paths": sorted(paths), } - client.app_id = 1 - assert_loop_error( - updater, - lambda: updater._collect_planning_intake( - client, - "Flow-Research/workstream", - metadata=metadata, - pr_number=201, - head_sha=head_sha, - base_sha=base_sha, - first_parent_sha=first_parent_sha, - merge_commit=merge_commit, - ), - "invalid provenance", - ) client.app_id = updater.GITHUB_ACTIONS_APP_ID client.mode = "100755" assert_loop_error( @@ -2459,9 +2451,6 @@ def get_json(self, path: str): ) client.mode = "100644" cases = ( - (lambda: setattr(client, "app_slug", "foreign-app"), "invalid provenance"), - (lambda: setattr(client, "check_status", "queued"), "invalid provenance"), - (lambda: setattr(client, "check_conclusion", "cancelled"), "invalid provenance"), (lambda: setattr(client, "file_status", "renamed"), "additive files only"), (lambda: setattr(client, "file_status", "removed"), "additive files only"), (lambda: setattr(client, "duplicate_pr_path", True), "path set is invalid"), @@ -4427,6 +4416,7 @@ def test_generated_loop_memory_escapes_markdown_metadata() -> None: def test_loop_memory_workflow_isolated_write_boundary() -> None: """The write-capable workflow runs on trusted main and targets only the state branch.""" workflow = (ROOT / ".github/workflows/loop-memory.yml").read_text(encoding="utf-8") + start_workflow = (ROOT / ".github/workflows/loop-memory-start.yml").read_text(encoding="utf-8") agent_gates = (ROOT / ".github/workflows/agent-gates.yml").read_text( encoding="utf-8" ) @@ -4453,27 +4443,13 @@ def test_loop_memory_workflow_isolated_write_boundary() -> None: assert "HEAD:refs/heads/${STATE_BRANCH}" not in workflow assert "HEAD:refs/heads/main" not in workflow assert "gh pr create" not in workflow - assert "plan-commits" in workflow - update_command = workflow.split( - "python3 scripts/update_post_merge_memory.py update", 1 - )[1].split(" done", 1)[0] - update_lines = [line.strip() for line in update_command.splitlines()] - repository_root_lines = [ - line for line in update_lines if line.startswith("--repository-root") - ] - cutover_lines = [ - line for line in update_lines if line.startswith("--cutover-chunk-id") - ] - assert repository_root_lines == ["--repository-root . \\"] - assert cutover_lines == ["--cutover-chunk-id WS-ENG-001-04B \\"] - assert all("$" not in line for line in repository_root_lines + cutover_lines) - assert workflow.count("--cutover-chunk-id") == 1 - assert workflow.count("prepare-recovery") == 1 - assert workflow.count("assert-recovery-consumed") == 1 - assert workflow.index("prepare-recovery") < workflow.index( - "python3 scripts/update_post_merge_memory.py update" - ) - assert workflow.index("assert-recovery-consumed") < workflow.index("sign-state") + for text in (workflow, start_workflow): + assert text.count("update_post_merge_memory.py reconcile") == 1 + assert "prepare-recovery" not in text + assert "update_post_merge_memory.py update" not in text + assert "assert-recovery-consumed" not in text + assert start_workflow.index("update_post_merge_memory.py reconcile") < start_workflow.index("apply-event") + assert workflow.index("update_post_merge_memory.py reconcile") < workflow.index("sign-state") assert '--target-sha "${TARGET_SHA}"' in workflow assert "resolve-target" in workflow assert "EVENT_SHA" in workflow @@ -4489,14 +4465,13 @@ def test_loop_memory_workflow_isolated_write_boundary() -> None: assert "replay target is stale" in ( ROOT / "scripts/update_post_merge_memory.py" ).read_text(encoding="utf-8") - assert "--current-sha" in workflow assert "update_post_merge_memory.py sign-state" in workflow assert "check_loop_memory_state.py" in workflow assert workflow.index("Resolve trusted protected-main target") < workflow.index( "Prepare generated state branch" ) - assert workflow.index("prepare-state") < workflow.index("plan-commits") - assert workflow.index("plan-commits") < workflow.index("sign-state") + assert workflow.index("prepare-state") < workflow.index("update_post_merge_memory.py reconcile") + assert workflow.index("update_post_merge_memory.py reconcile") < workflow.index("sign-state") assert workflow.index("sign-state") < workflow.index("--expected-main-sha") assert workflow.index("--expected-main-sha") < workflow.index( "check_loop_memory_state.py" diff --git a/scripts/test_update_post_merge_memory.py b/scripts/test_update_post_merge_memory.py index 641af4ec..75f0cb65 100644 --- a/scripts/test_update_post_merge_memory.py +++ b/scripts/test_update_post_merge_memory.py @@ -778,7 +778,7 @@ def test_prepare_recovery_v3_binds_three_adjacent_merges_and_is_deterministic( three = {"schema_version": 1, "exemptions": four["exemptions"][:3]} with pytest.raises(loop.LoopMemoryError, match="unique and bounded"): loop._validate_recovery_exemptions(three) - with pytest.raises(loop.LoopMemoryError, match="unsupported"): + with pytest.raises(loop.LoopMemoryError, match="unique and bounded"): loop._validate_recovery_exemptions({**three, "schema_version": 3}) for bad_plan in ( @@ -1900,3 +1900,342 @@ def fail_push(args, **kwargs): stdout=subprocess.PIPE, ).stdout.strip() assert remote_tip == published + + +def test_protected_evidence_ignores_classifiable_post_merge_reruns() -> None: + head = "a" * 40 + + def run(check_id, name, started, completed, conclusion="success", **extra): + value = { + "id": check_id, "name": name, "head_sha": head, + "status": "completed", "conclusion": conclusion, + "started_at": started, "completed_at": completed, + "app": {"id": loop.GITHUB_ACTIONS_APP_ID, "slug": loop.GITHUB_ACTIONS_APP_SLUG}, + } + value.update(extra) + return value + + evidence = loop._protected_actions_evidence([ + run(1, "agent-gates", "2026-07-23T07:00:00Z", "2026-07-23T07:01:00Z"), + run(2, "test", "2026-07-23T07:02:00Z", "2026-07-23T07:03:00Z"), + run(3, "agent-gates", "2026-07-23T08:01:00Z", None, status="queued", conclusion=None, app={"id": 1, "slug": "foreign"}), + ], head, "2026-07-23T08:00:00Z") + assert evidence["selected"]["agent-gates"]["id"] == 1 + + with pytest.raises(loop.LoopMemoryError, match="invalid provenance"): + loop._protected_actions_evidence([ + run(1, "agent-gates", "2026-07-23T07:00:00Z", "2026-07-23T07:01:00Z"), + run(4, "agent-gates", "2026-07-23T07:30:00Z", "2026-07-23T07:31:00Z", "failure"), + run(2, "test", "2026-07-23T07:02:00Z", "2026-07-23T07:03:00Z"), + ], head, "2026-07-23T08:00:00Z") + + +def test_paginated_collection_rejects_overlap_and_total_drift(monkeypatch: pytest.MonkeyPatch) -> None: + client = loop.GitHubClient("token") + pages = { + 1: {"total_count": 101, "check_runs": [{"id": value} for value in range(1, 101)]}, + 2: {"total_count": 101, "check_runs": [{"id": 100}]}, + } + monkeypatch.setattr(client, "get_json", lambda path: pages[int(path.rsplit("page=", 1)[1])]) + with pytest.raises(loop.LoopMemoryError, match="identity"): + client.get_paginated_collection("/checks", "check_runs") + + pages[2] = {"total_count": 102, "check_runs": [{"id": 101}, {"id": 102}]} + with pytest.raises(loop.LoopMemoryError, match="changed"): + client.get_paginated_collection("/checks", "check_runs") + + +def test_shared_reconcile_orders_recovery_and_validates_atomically( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + state_root = tmp_path / "state" + loop.apply_merge_record(state_root, _record()) + normal_sha = "b" * 40 + recovery_sha = "d3321698fb856f3fac320cdc7bc598f813fe1953" + planned = [normal_sha, recovery_sha] + exemptions = [ + {"initiative_id": "WS-ENG-007", "chunk_id": "WS-ENG-007-00R2", "pr_number": 189}, + {"initiative_id": "WS-ENG-007", "chunk_id": "WS-ENG-007-00R3", "pr_number": 190}, + ] + events: list[str] = [] + collection_calls: list[tuple[str, bool]] = [] + monkeypatch.setattr(loop, "plan_reconciliation_commits", lambda *_args: events.append("plan") or planned) + monkeypatch.setattr(loop, "prepare_recovery_exemptions", lambda *_args, **_kwargs: events.append("prepare") or exemptions) + def collect(_client, _repository, sha, *, historical_recovery=False): + collection_calls.append((sha, historical_recovery)) + events.append("collect") + return _record() + + monkeypatch.setattr(loop, "collect_merge_record", collect) + monkeypatch.setattr(loop, "apply_merge_record", lambda *_args, **_kwargs: events.append("apply") or True) + monkeypatch.setattr(loop, "assert_recovery_consumed", lambda *_args: events.append("assert")) + monkeypatch.setattr(loop, "validate_generated_state", lambda *_args: events.append("validate")) + loop.reconcile_to_main( + object(), "Flow-Research/workstream", repository_root=tmp_path, + state_root=state_root, target_sha=recovery_sha, + ) + assert events == ["plan", "prepare", "collect", "apply", "collect", "apply", "assert", "validate"] + assert collection_calls == [(normal_sha, False), (recovery_sha, True)] + + +def test_merge_bound_evidence_is_mandatory_after_cutover_in_both_validators() -> None: + record = _record() + record["source"]["merged_at"] = "2026-07-23T05:11:46Z" + record["updated_at"] = record["source"]["merged_at"] + with pytest.raises(loop.LoopMemoryError, match="merge-bound protected"): + loop._validate_record(record) + assert any( + "merge-bound protected" in failure + for failure in checker._record_failures(record, "record", None) + ) + + +def _merge_bound_record() -> dict: + record = _record() + record["source"]["merged_at"] = "2026-07-23T06:00:00Z" + record["updated_at"] = record["source"]["merged_at"] + selected = {} + for index, name in enumerate(("agent-gates", "test"), start=1): + selected[name] = { + "id": index, "head_sha": record["source"]["head_sha"], + "app_id": loop.GITHUB_ACTIONS_APP_ID, "app_slug": loop.GITHUB_ACTIONS_APP_SLUG, + "started_at": f"2026-07-23T05:0{index}:00Z", + "completed_at": f"2026-07-23T05:0{index}:30Z", + "conclusion": "success", "merge_cutoff": record["source"]["merged_at"], + } + record["protected_checks"] = { + "schema_version": 1, "selected": selected, + "sha256": loop.hashlib.sha256(loop._canonical_json(selected).encode()).hexdigest(), + } + return record + + +@pytest.mark.parametrize("bad_value", ["2026-07-23 05:01:00Z", "not-a-time"]) +def test_independent_checker_rejects_noncanonical_protected_timestamps(bad_value: str) -> None: + record = _merge_bound_record() + record["protected_checks"]["selected"]["agent-gates"]["started_at"] = bad_value + selected = record["protected_checks"]["selected"] + record["protected_checks"]["sha256"] = loop.hashlib.sha256(loop._canonical_json(selected).encode()).hexdigest() + with pytest.raises(loop.LoopMemoryError, match="timestamp"): + loop._validate_record(record) + assert any("timing" in failure for failure in checker._record_failures(record, "record", None)) + + +def _pr189_recovery_record() -> dict: + record = _record() + record["completed_chunk"] = { + "schema_version": 2, "initiative_id": "WS-ENG-007", + "chunk_id": "WS-ENG-007-00R2", "chunk_title": "Canonical Check Evidence Recovery", + "next_chunk_id": "WS-ENG-007-01", "next_chunk_title": "Reviewed Patch and Base-Delta Reconciliation", + "next_requires_explicit_start": True, + } + record["source"].update( + main_sha="d3321698fb856f3fac320cdc7bc598f813fe1953", + first_parent_sha="c65633f8f0991dbefe7b0635e053aab0df8f9af8", + head_sha=loop.R3_HISTORICAL_HEAD_SHA, pr_number=189, + pr_url="https://github.com/Flow-Research/workstream/pull/189", + intent_path=".agent-loop/merge-intents/WS-ENG-007-00R2.json", + merged_at="2026-07-23T08:17:25Z", + ) + record["updated_at"] = record["source"]["merged_at"] + record["gate"].update( + next_chunk_id="WS-ENG-007-01", + next_chunk_title="Reviewed Patch and Base-Delta Reconciliation", + ) + recovery = { + "merge_sha": record["source"]["main_sha"], "head_sha": loop.R3_HISTORICAL_HEAD_SHA, + "chunk_id": "WS-ENG-007-00R2", "pr_number": 189, "policy_schema": 4, + "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", + "activation_chunk_id": "WS-ENG-007-00R3", + "certificate_sha256": loop.R3_RECOVERY_CERTIFICATE_SHA256, + "reason": "no-completed-pre-merge-agent-gates", + } + record["protected_checks"] = { + "schema_version": 1, "recovery_only": recovery, + "sha256": loop.hashlib.sha256(loop._canonical_json(recovery).encode()).hexdigest(), + } + return record + + +@pytest.mark.parametrize("field", ["merge_sha", "head_sha", "chunk_id", "pr_number", "policy_schema", "signed_basis", "activation_chunk_id", "certificate_sha256", "reason"]) +def test_pr189_recovery_certificate_mutations_fail_both_validators(field: str) -> None: + record = _pr189_recovery_record() + recovery = record["protected_checks"]["recovery_only"] + recovery[field] = 0 if field in {"pr_number", "policy_schema"} else "wrong" + record["protected_checks"]["sha256"] = loop.hashlib.sha256(loop._canonical_json(recovery).encode()).hexdigest() + with pytest.raises(loop.LoopMemoryError, match="historical recovery"): + loop._validate_record(record) + assert any("historical recovery" in failure for failure in checker._record_failures(record, "record", None)) + + +@pytest.mark.parametrize( + ("payload", "message"), + [ + ([], "malformed"), + ({"total_count": -1, "check_runs": []}, "total"), + ({"total_count": 1, "check_runs": [{"id": 1}, {"id": 2}]}, "exceeds"), + ({"total_count": 2, "check_runs": [{"id": 1}]}, "ended"), + ({"total_count": 1, "check_runs": [{"id": 0}]}, "identity"), + ], +) +def test_paginated_collection_rejects_closed_malformed_shapes( + payload: object, message: str, monkeypatch: pytest.MonkeyPatch +) -> None: + client = loop.GitHubClient("token") + monkeypatch.setattr(client, "get_json", lambda _path: payload) + with pytest.raises(loop.LoopMemoryError, match=message): + client.get_paginated_collection("/checks", "check_runs") + + +def test_paginated_collection_accepts_empty_complete_envelope(monkeypatch: pytest.MonkeyPatch) -> None: + client = loop.GitHubClient("token") + monkeypatch.setattr(client, "get_json", lambda _path: {"total_count": 0, "check_runs": []}) + assert client.get_paginated_collection("/checks", "check_runs") == [] + + +def _protected_runs_for_mutation() -> tuple[str, list[dict]]: + head = "a" * 40 + runs = [] + for index, name in enumerate(("agent-gates", "test"), start=1): + runs.append({ + "id": index, "name": name, "head_sha": head, "status": "completed", + "conclusion": "success", "started_at": f"2026-07-23T05:0{index}:00Z", + "completed_at": f"2026-07-23T05:0{index}:30Z", + "app": {"id": loop.GITHUB_ACTIONS_APP_ID, "slug": loop.GITHUB_ACTIONS_APP_SLUG}, + }) + return head, runs + + +@pytest.mark.parametrize( + ("mutation", "message"), + [ + (lambda runs: runs.pop(), "missing"), + (lambda runs: runs[0].update(id=0), "invalid provenance"), + (lambda runs: runs[1].update(id=1), "invalid provenance"), + (lambda runs: runs[0].update(head_sha="b" * 40), "invalid provenance"), + (lambda runs: runs[0].update(status="queued"), "invalid provenance"), + (lambda runs: runs[0].update(conclusion="unknown"), "invalid provenance"), + (lambda runs: runs[0].update(app={"id": 1, "slug": "foreign"}), "invalid provenance"), + (lambda runs: runs[0].update(completed_at="2026-07-23T04:00:00Z"), "invalid provenance"), + (lambda runs: runs[0].update(completed_at="2026-07-23T06:30:00Z"), "invalid provenance"), + ], +) +def test_protected_selector_rejects_eligible_provenance_mutations(mutation, message: str) -> None: + head, runs = _protected_runs_for_mutation() + mutation(runs) + with pytest.raises(loop.LoopMemoryError, match=message): + loop._protected_actions_evidence(runs, head, "2026-07-23T06:00:00Z") + + +def test_recovery_policy_v4_rejects_closed_schema_mutations() -> None: + policy = json.loads((Path(__file__).parents[1] / ".agent-loop/policies/loop-memory-recovery.json").read_text()) + assert loop._validate_recovery_policy(policy) == policy + mutations = [] + bad = json.loads(json.dumps(policy)); bad["signed_basis"] = "bad"; mutations.append(bad) + bad = json.loads(json.dumps(policy)); bad["recovered_merges"] = bad["recovered_merges"][:2]; mutations.append(bad) + bad = json.loads(json.dumps(policy)); bad["recovered_merges"][1]["pr_number"] = 187; mutations.append(bad) + bad = json.loads(json.dumps(policy)); bad["activation"]["chunk_id"] = "WRONG"; mutations.append(bad) + bad = json.loads(json.dumps(policy)); bad["extra"] = True; mutations.append(bad) + for mutation in mutations: + with pytest.raises(loop.LoopMemoryError): + loop._validate_recovery_policy(mutation) + + +@pytest.mark.parametrize( + ("mutation", "updater_message", "checker_message"), + [ + (lambda r: r.update(protected_checks=[]), "invalid schema", "invalid protected"), + (lambda r: r["protected_checks"].update(schema_version=2), "invalid schema", "invalid protected"), + (lambda r: r["protected_checks"].update(sha256="0" * 64), "digest", "digest"), + (lambda r: r["protected_checks"]["selected"].pop("test"), "incomplete", "digest"), + (lambda r: r["protected_checks"]["selected"]["test"].update(id=1), "provenance", "provenance"), + (lambda r: r["protected_checks"]["selected"]["test"].update(app_slug="foreign"), "provenance", "provenance"), + (lambda r: r["protected_checks"]["selected"]["test"].update(completed_at="2026-07-23T04:00:00Z"), "timing", "timing"), + (lambda r: r["protected_checks"]["selected"]["test"].update(extra=True), "invalid for test", "provenance"), + ], +) +def test_protected_record_mutations_fail_updater_and_checker( + mutation, updater_message: str, checker_message: str +) -> None: + record = _merge_bound_record() + mutation(record) + if isinstance(record.get("protected_checks"), dict) and isinstance(record["protected_checks"].get("selected"), dict): + selected = record["protected_checks"]["selected"] + if updater_message not in {"digest", "incomplete"}: + record["protected_checks"]["sha256"] = loop.hashlib.sha256(loop._canonical_json(selected).encode()).hexdigest() + with pytest.raises(loop.LoopMemoryError, match=updater_message): + loop._validate_record(record) + assert any(checker_message in failure for failure in checker._record_failures(record, "record", None)) + + +@pytest.mark.parametrize( + "payload", + [ + None, + {"schema_version": 9, "activation": {}}, + {"schema_version": 4, "signed_basis": "a" * 40, "activation": [], "recovered_merges": []}, + {"schema_version": 2, "activation": {"initiative_id": "WS-ENG-007", "chunk_id": "WS-ENG-007-00R3"}, "mode": "wrong"}, + ], +) +def test_recovery_policy_rejects_unsupported_top_level_shapes(payload: object) -> None: + with pytest.raises(loop.LoopMemoryError): + loop._validate_recovery_policy(payload) + + +def test_cli_plan_and_resolve_commands_cover_public_dispatch_paths( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + monkeypatch.setattr(loop, "plan_reconciliation_commits", lambda *_args: ["a" * 40, "b" * 40]) + assert loop.main([ + "plan-commits", "--repository-root", ".", "--target-sha", "b" * 40, + "--current-sha", "0" * 40, + ]) == 0 + assert capsys.readouterr().out.splitlines() == ["a" * 40, "b" * 40] + + monkeypatch.setattr(loop, "resolve_reconciliation_target", lambda *_args: "c" * 40) + assert loop.main([ + "resolve-target", "--repository-root", ".", "--event-name", "push", + "--event-sha", "b" * 40, "--current-main-sha", "c" * 40, + ]) == 0 + assert capsys.readouterr().out.strip() == "c" * 40 + + +def test_cli_reconcile_uses_authenticated_shared_reducer( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + observed: list[tuple] = [] + monkeypatch.setenv("GITHUB_TOKEN", "token") + monkeypatch.setattr(loop, "_assert_state_branch", lambda root: observed.append(("branch", root))) + monkeypatch.setattr(loop, "GitHubClient", lambda token, url: (token, url)) + monkeypatch.setattr( + loop, "reconcile_to_main", + lambda client, repository, **kwargs: observed.append(("reconcile", client, repository, kwargs)), + ) + target = "d" * 40 + state_root, branch_root = tmp_path / "state", tmp_path / "branch" + assert loop.main([ + "reconcile", "--repository", "Flow-Research/workstream", + "--repository-root", str(tmp_path), "--state-root", str(state_root), + "--branch-root", str(branch_root), "--target-sha", target, + ]) == 0 + assert observed[0] == ("branch", branch_root) + assert observed[1][0:3] == ("reconcile", ("token", "https://api.github.com"), "Flow-Research/workstream") + assert observed[1][3]["target_sha"] == target + assert capsys.readouterr().out.strip() == f"Loop memory reconciled to {target}." + + +def test_shared_reconcile_rejects_missing_canonical_state(tmp_path: Path) -> None: + with pytest.raises(loop.LoopMemoryError, match="requires canonical state"): + loop.reconcile_to_main( + object(), "Flow-Research/workstream", repository_root=tmp_path, + state_root=tmp_path / "missing", target_sha="a" * 40, + ) + + +def test_protected_selector_rejects_names_seen_only_after_merge() -> None: + head, runs = _protected_runs_for_mutation() + for item in runs: + item["started_at"] = "2026-07-23T07:01:00Z" + item["completed_at"] = None + with pytest.raises(loop.LoopMemoryError, match="missing"): + loop._protected_actions_evidence(runs, head, "2026-07-23T06:00:00Z") diff --git a/scripts/update_post_merge_memory.py b/scripts/update_post_merge_memory.py index b717c5cc..ef69ed88 100644 --- a/scripts/update_post_merge_memory.py +++ b/scripts/update_post_merge_memory.py @@ -71,6 +71,8 @@ ) GITHUB_ACTIONS_APP_ID = 15368 GITHUB_ACTIONS_APP_SLUG = "github-actions" +R3_RECOVERY_CERTIFICATE_SHA256 = "4fe49b2f4a5a7ad18382a717dcc11f798c465a534066102bf9810c9ed5784f4a" +R3_HISTORICAL_HEAD_SHA = "55a11d9e0ae356734dbcce73564f5f570220a81b" CHECK_RUN_CONCLUSIONS = frozenset({ "action_required", "cancelled", "failure", "neutral", "skipped", "stale", "success", "timed_out", @@ -136,6 +138,38 @@ def get_paginated(self, path: str) -> list[Any]: return items raise LoopMemoryError("paginated GitHub response exceeded 100 pages") + def get_paginated_collection(self, path: str, key: str) -> list[Any]: + """Return one complete bounded collection from an envelope endpoint.""" + items: list[Any] = [] + seen_ids: set[int] = set() + expected_total: int | None = None + separator = "&" if "?" in path else "?" + for page in range(1, 101): + payload = self.get_json(f"{path}{separator}per_page=100&page={page}") + if not isinstance(payload, dict) or not isinstance(payload.get(key), list): + raise LoopMemoryError("paginated GitHub collection is malformed") + total = payload.get("total_count") + if type(total) is not int or total < 0: + raise LoopMemoryError("paginated GitHub collection total is invalid") + if expected_total is None: + expected_total = total + elif total != expected_total: + raise LoopMemoryError("paginated GitHub collection changed during read") + page_items = payload[key] + for item in page_items: + item_id = item.get("id") if isinstance(item, dict) else None + if type(item_id) is not int or item_id <= 0 or item_id in seen_ids: + raise LoopMemoryError("paginated GitHub collection identity is invalid") + seen_ids.add(item_id) + items.extend(page_items) + if len(items) > total: + raise LoopMemoryError("paginated GitHub collection exceeds its total") + if len(items) == total: + return items + if len(page_items) != 100: + raise LoopMemoryError("paginated GitHub collection ended before its total") + raise LoopMemoryError("paginated GitHub collection exceeded 100 pages") + def _bounded_text(value: Any, field: str, maximum: int = 160) -> str: """Validate one bounded single-line metadata string.""" @@ -771,9 +805,13 @@ def _latest_named( def _check_evidence( - check_runs: list[dict[str, Any]], statuses: list[dict[str, Any]] + check_runs: list[dict[str, Any]], statuses: list[dict[str, Any]], merged_at: str | None = None ) -> dict[str, Any]: """Build bounded required-check evidence without treating it as merge authority.""" + if merged_at is not None: + cutoff = _rfc3339_instant(merged_at) + check_runs = [item for item in check_runs if _rfc3339_instant(item.get("started_at")) <= cutoff] + statuses = [item for item in statuses if _rfc3339_instant(item.get("updated_at")) <= cutoff] latest_checks = _latest_named(check_runs, "name", "started_at") latest_statuses = _latest_named(statuses, "context", "updated_at") observed: dict[str, dict[str, str | None]] = {} @@ -809,33 +847,34 @@ def _commit_tree_sha(payload: Any, label: str) -> str: return tree_sha -def _validate_protected_actions_checks( - client: GitHubClient, repository: str, head_sha: str -) -> None: - """Require exact protected GitHub Actions checks on one reviewed head.""" - payload = client.get_json( - f"/repos/{repository}/commits/{head_sha}/check-runs?per_page=100" - ) - runs = payload.get("check_runs") if isinstance(payload, dict) else None - total = payload.get("total_count") if isinstance(payload, dict) else None - if not isinstance(runs, list) or type(total) is not int or total != len(runs): - raise LoopMemoryError("planning intake check-run evidence is incomplete") +def _protected_actions_evidence( + runs: list[dict[str, Any]], head_sha: str, merged_at: str +) -> dict[str, Any]: + """Freeze protected GitHub Actions evidence at the immutable merge instant.""" + cutoff = _rfc3339_instant(merged_at) seen_ids: set[int] = set() + selected: dict[str, dict[str, Any]] = {} for name in ("agent-gates", "test"): matches = [item for item in runs if isinstance(item, dict) and item.get("name") == name] if not matches: raise LoopMemoryError(f"planning intake check {name} is missing") candidates: list[tuple[datetime, int, dict[str, Any]]] = [] for item in matches: - app = item.get("app") check_id = item.get("id") started_at = _rfc3339_instant(item.get("started_at")) - completed_at = _rfc3339_instant(item.get("completed_at")) if ( type(check_id) is not int or check_id <= 0 or check_id in seen_ids - or item.get("head_sha") != head_sha + ): + raise LoopMemoryError(f"planning intake check {name} has invalid provenance") + seen_ids.add(check_id) + if started_at > cutoff: + continue + app = item.get("app") + completed_at = _rfc3339_instant(item.get("completed_at")) + if ( + item.get("head_sha") != head_sha or item.get("status") != "completed" or item.get("conclusion") not in CHECK_RUN_CONCLUSIONS or completed_at < started_at @@ -844,11 +883,38 @@ def _validate_protected_actions_checks( or app.get("slug") != GITHUB_ACTIONS_APP_SLUG ): raise LoopMemoryError(f"planning intake check {name} has invalid provenance") - seen_ids.add(check_id) candidates.append((started_at, check_id, item)) + if not candidates: + raise LoopMemoryError(f"planning intake check {name} is missing") item = max(candidates, key=lambda candidate: (candidate[0], candidate[1]))[2] - if item.get("conclusion") != "success": + completed = _rfc3339_instant(item.get("completed_at")) + if item.get("conclusion") != "success" or completed > cutoff: raise LoopMemoryError(f"planning intake check {name} has invalid provenance") + selected[name] = { + "id": item["id"], "head_sha": head_sha, + "app_id": GITHUB_ACTIONS_APP_ID, "app_slug": GITHUB_ACTIONS_APP_SLUG, + "started_at": item["started_at"], "completed_at": item["completed_at"], + "conclusion": "success", "merge_cutoff": merged_at, + } + digest = hashlib.sha256(_canonical_json(selected).encode("utf-8")).hexdigest() + return {"schema_version": 1, "selected": selected, "sha256": digest} + + +def _validate_protected_actions_checks( + client: GitHubClient, repository: str, head_sha: str, merged_at: str | None = None +) -> None: + """Compatibility validator for closed pre-v4 recovery schemas.""" + complete_check_client = hasattr(client, "get_paginated_collection") + if complete_check_client: + runs = client.get_paginated_collection( + f"/repos/{repository}/commits/{head_sha}/check-runs", "check_runs" + ) + else: + payload = client.get_json(f"/repos/{repository}/commits/{head_sha}/check-runs?per_page=100") + runs = payload.get("check_runs", []) if isinstance(payload, dict) else [] + if merged_at is None: + merged_at = "9999-12-31T23:59:59Z" + _protected_actions_evidence(runs, head_sha, merged_at) def _rfc3339_instant(value: Any) -> datetime: @@ -1080,7 +1146,6 @@ def _collect_planning_intake( delta_sha256 = hashlib.sha256( _canonical_json(reviewed_delta).encode("utf-8") ).hexdigest() - _validate_protected_actions_checks(client, repository, head_sha) return { "schema_version": PLANNING_INTAKE_VERSION, "initiative_directory": initiative_directory, @@ -1097,6 +1162,8 @@ def collect_merge_record( client: GitHubClient, repository: str, merge_sha: str, + *, + historical_recovery: bool = False, ) -> dict[str, Any]: """Collect one exact merged PR and its bounded loop metadata from GitHub.""" _validate_repository_and_sha(repository, merge_sha) @@ -1157,15 +1224,19 @@ def collect_merge_record( raise LoopMemoryError("merged main commit has no canonical first parent") first_parent_sha = parents[0]["sha"] - check_payload = client.get_json( - f"/repos/{repository}/commits/{head_sha}/check-runs?per_page=100" - ) + complete_check_client = hasattr(client, "get_paginated_collection") + if complete_check_client: + check_runs = client.get_paginated_collection( + f"/repos/{repository}/commits/{head_sha}/check-runs", "check_runs" + ) + else: + check_payload = client.get_json( + f"/repos/{repository}/commits/{head_sha}/check-runs?per_page=100" + ) + check_runs = check_payload.get("check_runs", []) if isinstance(check_payload, dict) else [] status_payload = client.get_json( f"/repos/{repository}/commits/{head_sha}/status?per_page=100" ) - check_runs = ( - check_payload.get("check_runs", []) if isinstance(check_payload, dict) else [] - ) statuses = ( status_payload.get("statuses", []) if isinstance(status_payload, dict) else [] ) @@ -1183,6 +1254,22 @@ def collect_merge_record( "merged pull request URL does not match repository and number" ) + if historical_recovery: + recovery_only = { + "merge_sha": merge_sha, "head_sha": head_sha, + "chunk_id": metadata.chunk_id, + "pr_number": pr_number, "policy_schema": 4, + "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", + "activation_chunk_id": "WS-ENG-007-00R3", + "certificate_sha256": R3_RECOVERY_CERTIFICATE_SHA256, + "reason": "no-completed-pre-merge-agent-gates", + } + protected_checks = { + "schema_version": 1, "recovery_only": recovery_only, + "sha256": hashlib.sha256(_canonical_json(recovery_only).encode("utf-8")).hexdigest(), + } + elif complete_check_client: + protected_checks = _protected_actions_evidence(check_runs, head_sha, merged_at) record = { "schema_version": SCHEMA_VERSION, "repository": repository, @@ -1213,8 +1300,10 @@ def collect_merge_record( "next_chunk_title": metadata.next_chunk_title, "next_requires_explicit_start": metadata.next_requires_explicit_start, }, - "checks": _check_evidence(check_runs, statuses), + "checks": _check_evidence(check_runs, statuses, merged_at), } + if historical_recovery or complete_check_client: + record["protected_checks"] = protected_checks planning_intake = _collect_planning_intake( client, repository, @@ -1809,15 +1898,23 @@ def _validate_record(record: dict[str, Any]) -> LoopMetadata: "gate", "checks", } + protected_record_keys = expected_record_keys | {"protected_checks"} allowed_record_keys = expected_record_keys | {"legacy_exemptions"} + protected_allowed_record_keys = protected_record_keys | {"legacy_exemptions"} planning_record_keys = expected_record_keys | {"planning_intake"} + protected_planning_record_keys = planning_record_keys | {"protected_checks"} planning_legacy_record_keys = planning_record_keys | {"legacy_exemptions"} + protected_planning_legacy_record_keys = protected_planning_record_keys | {"legacy_exemptions"} cutover_record_keys = allowed_record_keys | {"event"} if frozenset(record) not in { frozenset(expected_record_keys), + frozenset(protected_record_keys), frozenset(allowed_record_keys), + frozenset(protected_allowed_record_keys), frozenset(planning_record_keys), + frozenset(protected_planning_record_keys), frozenset(planning_legacy_record_keys), + frozenset(protected_planning_legacy_record_keys), frozenset(cutover_record_keys), } or not _is_current_schema_version( record.get("schema_version") @@ -1940,8 +2037,9 @@ def _validate_record(record: dict[str, Any]) -> LoopMetadata: ) ): raise LoopMemoryError("planning intake lifecycle identity is invalid") - if not record.get("checks", {}).get("all_required_passed"): - raise LoopMemoryError("planning intake required checks did not pass") + protected = record.get("protected_checks") + if protected is not None and set(protected.get("selected", {})) != {"agent-gates", "test"}: + raise LoopMemoryError("planning intake protected checks did not pass") active = record.get("active") if active != {"planning_chunk": None, "implementation_chunk": None}: @@ -1986,6 +2084,47 @@ def _validate_record(record: dict[str, Any]) -> LoopMetadata: ) if checks.get("all_required_passed") is not all_passed: raise LoopMemoryError("loop-memory aggregate check evidence is inconsistent") + protected = record.get("protected_checks") + if protected is None: + if _rfc3339_instant(merged_at) >= _rfc3339_instant("2026-07-23T05:11:46Z"): + raise LoopMemoryError("merge-bound protected check evidence is required") + return metadata + if not isinstance(protected, dict) or protected.get("schema_version") != 1: + raise LoopMemoryError("protected check evidence has an invalid schema") + if set(protected) == {"schema_version", "recovery_only", "sha256"}: + recovery_only = protected.get("recovery_only") + expected = { + "merge_sha": "d3321698fb856f3fac320cdc7bc598f813fe1953", + "head_sha": R3_HISTORICAL_HEAD_SHA, "chunk_id": "WS-ENG-007-00R2", + "pr_number": 189, "policy_schema": 4, + "signed_basis": "73b457925b02301587b83d01ced0adb66319d134", + "activation_chunk_id": "WS-ENG-007-00R3", + "certificate_sha256": R3_RECOVERY_CERTIFICATE_SHA256, + "reason": "no-completed-pre-merge-agent-gates", + } + if source["head_sha"] != R3_HISTORICAL_HEAD_SHA or source["pr_number"] != 189 or recovery_only != expected or protected.get("sha256") != hashlib.sha256(_canonical_json(expected).encode("utf-8")).hexdigest(): + raise LoopMemoryError("historical recovery evidence is invalid") + return metadata + if set(protected) != {"schema_version", "selected", "sha256"}: + raise LoopMemoryError("protected check evidence has an invalid schema") + selected = protected.get("selected") + if not isinstance(selected, dict) or set(selected) != {"agent-gates", "test"}: + raise LoopMemoryError("protected check evidence is incomplete") + selected_ids: set[int] = set() + for name, item in selected.items(): + if not isinstance(item, dict) or set(item) != {"id", "head_sha", "app_id", "app_slug", "started_at", "completed_at", "conclusion", "merge_cutoff"}: + raise LoopMemoryError(f"protected check evidence is invalid for {name}") + if type(item["id"]) is not int or item["id"] <= 0 or item["id"] in selected_ids or item["head_sha"] != source["head_sha"] or item["app_id"] != GITHUB_ACTIONS_APP_ID or item["app_slug"] != GITHUB_ACTIONS_APP_SLUG or item["conclusion"] != "success" or item["merge_cutoff"] != merged_at: + raise LoopMemoryError(f"protected check provenance is invalid for {name}") + selected_ids.add(item["id"]) + started = _rfc3339_instant(item["started_at"]) + completed_at = _rfc3339_instant(item["completed_at"]) + cutoff = _rfc3339_instant(item["merge_cutoff"]) + if completed_at < started or completed_at > cutoff: + raise LoopMemoryError(f"protected check timing is invalid for {name}") + digest = hashlib.sha256(_canonical_json(selected).encode("utf-8")).hexdigest() + if protected.get("sha256") != digest: + raise LoopMemoryError("protected check evidence digest is invalid") return metadata @@ -2030,7 +2169,7 @@ def _validate_recovery_exemptions(payload: Any) -> list[dict[str, Any]]: ): raise LoopMemoryError("recovery exemption inventory has an invalid schema") version = payload.get("schema_version") - if version not in {1, 2}: + if version not in {1, 2, 3}: raise LoopMemoryError("recovery exemption inventory is unsupported") chronological = json.loads(_canonical_json(payload["exemptions"])) _validate_legacy_exemptions({ @@ -2052,6 +2191,7 @@ def _validate_recovery_exemptions(payload: Any) -> list[dict[str, Any]]: if ( (version == 1 and len(chronological) > 2) or (version == 2 and len(chronological) != 3) + or (version == 3 and len(chronological) != 4) or len(identities) != len(set(identities)) or len(chunk_identities) != len(set(chunk_identities)) or len(pr_numbers) != len(set(pr_numbers)) @@ -2127,11 +2267,12 @@ def _validate_recovery_policy(payload: Any) -> dict[str, Any]: 1: {"schema_version", "activation", "recovered_merge"}, 2: {"schema_version", "activation", "mode"}, 3: {"schema_version", "activation", "recovered_merges"}, + 4: {"schema_version", "signed_basis", "activation", "recovered_merges"}, }.get(version, set()) if set(payload) != expected: raise LoopMemoryError("recovery policy has an invalid schema") activation = payload.get("activation") - if version not in {1, 2, 3} or not isinstance(activation, dict): + if version not in {1, 2, 3, 4} or not isinstance(activation, dict): raise LoopMemoryError("recovery policy is unsupported") if set(activation) != {"initiative_id", "chunk_id"} or not _is_valid_exemption_id( activation.get("initiative_id"), activation.get("chunk_id") @@ -2141,10 +2282,13 @@ def _validate_recovery_policy(payload: Any) -> dict[str, Any]: if payload.get("mode") != "exact_single_target": raise LoopMemoryError("recovery policy mode is unsupported") return json.loads(_canonical_json(payload)) - if version == 3: + if version in {3, 4}: recovered_merges = payload.get("recovered_merges") - if not isinstance(recovered_merges, list) or not 1 <= len(recovered_merges) <= 2: + valid_length = (1 <= len(recovered_merges) <= 2) if isinstance(recovered_merges, list) and version == 3 else (isinstance(recovered_merges, list) and len(recovered_merges) == 3) + if not valid_length: raise LoopMemoryError("recovered merge inventory is invalid") + if version == 4: + _validate_sha(payload.get("signed_basis")) chunk_identities: set[tuple[str, str]] = set() pr_numbers: set[int] = set() merge_shas: set[str] = set() @@ -2199,6 +2343,15 @@ def _record_exemption(record: dict[str, Any]) -> dict[str, Any]: } +def _is_r3_historical_recovery( + merge_sha: str, exemptions: list[dict[str, Any]] +) -> bool: + """Identify the exact one-time PR #189 recovery-only merge.""" + return merge_sha == "d3321698fb856f3fac320cdc7bc598f813fe1953" and any( + item.get("chunk_id") == "WS-ENG-007-00R2" for item in exemptions + ) + + def prepare_recovery_exemptions( client: GitHubClient, repository: str, @@ -2248,13 +2401,17 @@ def prepare_recovery_exemptions( if not isinstance(existing, list) or exemption in existing: raise LoopMemoryError("recovery exemption collides with signed state") return [exemption] - if policy["schema_version"] == 3: + if policy["schema_version"] in {3, 4}: recovered_policies = policy["recovered_merges"] expected_shas = [item["merge_sha"] for item in recovered_policies] + [target_sha] if planned_shas != expected_shas: raise LoopMemoryError("recovery plan is not the exact ordered sequence") recovered_records = [ - collect_merge_record(client, repository, item["merge_sha"]) + ( + collect_merge_record(client, repository, item["merge_sha"], historical_recovery=True) + if policy["schema_version"] == 4 and item["chunk_id"] == "WS-ENG-007-00R2" + else collect_merge_record(client, repository, item["merge_sha"]) + ) for item in recovered_policies ] for recovered_policy, recovered_record in zip( @@ -2271,6 +2428,8 @@ def prepare_recovery_exemptions( if _event_type(state) in {"start", "cancel"} else state.get("source", {}).get("main_sha") ) + if policy["schema_version"] == 4 and signed_main != policy["signed_basis"]: + raise LoopMemoryError("recovery signed basis does not match canonical state") records = [*recovered_records, target_record] expected_parent = signed_main for merge_sha, record in zip(planned_shas, records, strict=True): @@ -2281,11 +2440,17 @@ def prepare_recovery_exemptions( ): raise LoopMemoryError("recovery plan is not first-parent adjacent") expected_parent = merge_sha - _validate_protected_actions_checks( - client, repository, source.get("head_sha") - ) - if not record.get("checks", {}).get("all_required_passed"): - raise LoopMemoryError("recovery required checks did not pass") + if policy["schema_version"] == 3: + _validate_protected_actions_checks(client, repository, source.get("head_sha")) + if not record.get("checks", {}).get("all_required_passed"): + raise LoopMemoryError("recovery required checks did not pass") + else: + protected = record.get("protected_checks", {}) + if record["completed_chunk"]["chunk_id"] == "WS-ENG-007-00R2": + if "recovery_only" not in protected: + raise LoopMemoryError("historical recovery evidence is missing") + elif set(protected.get("selected", {})) != {"agent-gates", "test"}: + raise LoopMemoryError("recovery protected checks did not pass") exemptions = [_record_exemption(record) for record in records] existing = state.get("legacy_exemptions", []) if not isinstance(existing, list) or any(item in existing for item in exemptions): @@ -2336,6 +2501,34 @@ def assert_recovery_consumed( raise LoopMemoryError("recovery exemption leaked into signed history") +def reconcile_to_main( + client: GitHubClient, repository: str, *, repository_root: Path, + state_root: Path, target_sha: str, +) -> None: + """Atomically reduce exact trusted main into temporary generated custody.""" + state = _load_json(state_root / STATE_PATH) + if not isinstance(state, dict): + raise LoopMemoryError("reconciliation requires canonical state") + current_sha = ( + state.get("event", {}).get("main_sha") + if _event_type(state) in {"start", "cancel"} + else state.get("source", {}).get("main_sha") + ) + planned = plan_reconciliation_commits(repository_root, target_sha, current_sha) + exemptions = prepare_recovery_exemptions( + client, repository, repository_root=repository_root, state_root=state_root, + target_sha=target_sha, planned_shas=planned, + ) + for merge_sha in planned: + historical = _is_r3_historical_recovery(merge_sha, exemptions) + record = collect_merge_record( + client, repository, merge_sha, historical_recovery=True + ) if historical else collect_merge_record(client, repository, merge_sha) + apply_merge_record(state_root, record, recovery_exemptions=exemptions or None) + assert_recovery_consumed(state_root, target_sha, exemptions) + validate_generated_state(state_root) + + def _validate_ledger_entries(entries: list[dict[str, Any]]) -> list[dict[str, Any]]: """Validate the full ledger hash and first-parent chains.""" records: list[dict[str, Any]] = [] @@ -3145,6 +3338,15 @@ def build_parser() -> argparse.ArgumentParser: prepare_recovery.add_argument("--token-env", default="GITHUB_TOKEN") prepare_recovery.add_argument("--api-url", default="https://api.github.com") + reconcile = subparsers.add_parser("reconcile") + reconcile.add_argument("--repository", required=True) + reconcile.add_argument("--repository-root", type=Path, default=Path(".")) + reconcile.add_argument("--state-root", type=Path, required=True) + reconcile.add_argument("--branch-root", type=Path, required=True) + reconcile.add_argument("--target-sha", required=True) + reconcile.add_argument("--token-env", default="GITHUB_TOKEN") + reconcile.add_argument("--api-url", default="https://api.github.com") + assert_recovery = subparsers.add_parser("assert-recovery-consumed") assert_recovery.add_argument("--state-root", type=Path, required=True) assert_recovery.add_argument("--target-sha", required=True) @@ -3235,10 +3437,18 @@ def main(argv: list[str] | None = None) -> int: elif args.command == "update": _assert_state_branch(args.branch_root or args.state_root) token = os.environ.get(args.token_env, "") - record = collect_merge_record( - GitHubClient(token, args.api_url), - args.repository, - args.merge_sha, + recovery_exemptions = [] + if args.recovery_file: + recovery_exemptions = _validate_recovery_exemptions( + _load_json(args.recovery_file) + ) + client = GitHubClient(token, args.api_url) + historical = _is_r3_historical_recovery( + args.merge_sha, recovery_exemptions + ) + record = ( + collect_merge_record(client, args.repository, args.merge_sha, historical_recovery=True) + if historical else collect_merge_record(client, args.repository, args.merge_sha) ) if ( args.cutover_chunk_id @@ -3255,11 +3465,6 @@ def main(argv: list[str] | None = None) -> int: _canonical_json(record["legacy_exemptions"]) ), } - recovery_exemptions = [] - if args.recovery_file: - recovery_exemptions = _validate_recovery_exemptions( - _load_json(args.recovery_file) - ) if recovery_exemptions: changed = apply_merge_record( args.state_root, record, @@ -3282,10 +3487,18 @@ def main(argv: list[str] | None = None) -> int: repository_root=args.repository_root, state_root=args.state_root, target_sha=args.target_sha, planned_shas=planned_shas, ) - transport_version = 2 if len(exemptions) == 3 else 1 + transport_version = {3: 2, 4: 3}.get(len(exemptions), 1) print(_canonical_json({ "schema_version": transport_version, "exemptions": exemptions })) + elif args.command == "reconcile": + _assert_state_branch(args.branch_root) + reconcile_to_main( + GitHubClient(os.environ.get(args.token_env, ""), args.api_url), + args.repository, repository_root=args.repository_root, + state_root=args.state_root, target_sha=args.target_sha, + ) + print(f"Loop memory reconciled to {args.target_sha}.") elif args.command == "assert-recovery-consumed": exemptions = _validate_recovery_exemptions(_load_json(args.recovery_file)) assert_recovery_consumed(args.state_root, args.target_sha, exemptions)