fix(queue): recover source callbacks and asynchronous queue advance

README.md

@@ -105,6 +105,32 @@ mirrors skipped validation as success. `actions/queue-completion` notifies this
 trusted reconciler from dispatched queue runs. Retain the caller's existing
 `Queue completion callback` name and event guard.
 
+Package-source callers can also notify the reconciler when validation starts and
+finishes. Add optional `source_run_attempt` and `source_activity` dispatch inputs
+to the default-branch reconciler, forwarding them to `source-run-attempt` and
+`source-activity` on `actions/reconcile-queue`. Source callback jobs use
+`actions/queue-completion` with `source-activity: in_progress` at start and
+`source-activity: completed` after all validation jobs. Restrict these jobs to
+same-repository dispatched `automation/package-updates-` branches and retain
+only their existing `actions: write` permission. Queue callbacks can omit the
+new input and keep their original payload.
+
+The reconciler validates the producer run's repository, workflow identity,
+path, branch, head and attempt before using its metadata. Start notifications
+invalidate previous mirrored jobs and admission without waiting for builds;
+completion notifications wait briefly for their own producer to finish and
+revalidate its actual attempt. A delayed start for a completed attempt reads
+the completed results. Verified obsolete attempts or advanced/deleted producer
+branches stop without any status or queue writes; tampered identities and future
+attempts remain errors. An already queued source PR cannot bypass this check.
+Statuses never replace the required native jobs or relax branch protection.
+After publishing new complete queue success, reconciliation waits at most 55
+seconds for GitHub's asynchronous merge to expose a different live front ref.
+It verifies that ref and dispatches its missing validation through the existing
+handler once. An empty queue returns immediately without a warning. Running or
+failed candidates do not start this additional poll;
+there is no recursive recovery dispatch.
+
 The actions read the default branch from GitHub's repository event metadata.
 Use that same default branch in the caller's callback guard. A package updater
 can opt in to admission with `automation-branch`; it accepts only that exact
@@ -128,8 +154,8 @@ workflow and required job passes on the exact source SHA. A blocked approval run
 can be ignored only when its jobs endpoint proves zero jobs. Failed, pending,
 skipped, missing, stale and untrusted evidence never becomes success; newer
 attempts invalidate older source adapter results. Actual failed required jobs
-receive failure statuses linking the current failed job. Normal required checks and
-expected-head merge-queue admission still apply. Unchanged completed runs reuse
+receive failure statuses linking the current failed job. Normal required checks
+and expected-head merge-queue admission still apply. Unchanged completed runs reuse
 only this bot's previously verified job attestations for the same run and attempt;
 latest-run metadata remains live. Immutable Git objects and workflow identity are
 cached only within one reconciliation. Opted-in callers should also reconcile
@@ -173,8 +199,9 @@ release. Do not provide privileged tokens to PR build steps.
 `actions/discover-lockfiles` emits a JSON `lockfiles` output for a workflow matrix.
 It requires a tracked `flake.lock` for each tracked `flake.nix` in the caller
 repository. Deleting only the lockfile fails discovery; deleting the entire
-partition removes its matrix entry. Added, renamed and deleted partitions change that inventory
-without edits to the workflow. Submodules own their lockfile health jobs.
+partition removes its matrix entry. Added, renamed and deleted partitions change
+that inventory without edits to the workflow. Submodules own their lockfile
+health jobs.
 Require a stable aggregate job that waits for the complete matrix and fails for
 any failed, cancelled or skipped matrix. Protect that aggregate instead of
 individual lockfile names, so adding coverage cannot be bypassed and deleting a

actions/queue-completion/action.yml

@@ -1,5 +1,9 @@
 name: Notify queue completion
 description: Notify the trusted default-branch reconciler after manually dispatched validation.
+inputs:
+  source-activity:
+    description: Optional package-source notification phase, in_progress or completed. Requires matching reconciler dispatch inputs.
+    default: ''
 runs:
   using: composite
   steps:
@@ -9,9 +13,17 @@ runs:
         GH_TOKEN: ${{ github.token }}
         GH_REPO: ${{ github.repository }}
         DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+        SOURCE_ACTIVITY: ${{ inputs.source-activity }}
       run: |
+        args=(-f source_run_id="$GITHUB_RUN_ID")
+        case "$SOURCE_ACTIVITY" in
+          '') ;;
+          in_progress|completed)
+            args+=(-f source_run_attempt="$GITHUB_RUN_ATTEMPT" -f source_activity="$SOURCE_ACTIVITY") ;;
+          *) echo '::error::Invalid package-source notification phase'; exit 1 ;;
+        esac
         for attempt in 1 2 3; do
-          if gh workflow run reconcile-merge-queue.yml --ref "$DEFAULT_BRANCH" -f source_run_id="$GITHUB_RUN_ID"; then
+          if gh workflow run reconcile-merge-queue.yml --ref "$DEFAULT_BRANCH" "${args[@]}"; then
             exit 0
           fi
           echo "::warning::Queue completion notification attempt $attempt failed"

actions/reconcile-queue/action.yml

@@ -16,6 +16,12 @@ inputs:
   source-run-id:
     description: Optional dispatch completion notification.
     default: ''
+  source-run-attempt:
+    description: Producer run attempt for a package-source start or completion callback.
+    default: ''
+  source-activity:
+    description: Package-source callback phase, in_progress or completed.
+    default: ''
 runs:
   using: composite
   steps:
@@ -27,6 +33,8 @@ runs:
         BASE_SHA_WORKFLOWS: ${{ inputs.base-sha-workflows }}
         QUEUE_WORKFLOWS: ${{ inputs.workflows }}
         SOURCE_RUN_ID: ${{ inputs.source-run-id }}
+        SOURCE_RUN_ATTEMPT: ${{ inputs.source-run-attempt }}
+        SOURCE_NOTIFICATION_ACTIVITY: ${{ inputs.source-activity }}
         SOURCE_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
         SOURCE_ACTIVITY: ${{ github.event.action }}
       run: |
@@ -90,6 +98,8 @@ runs:
         BASE_SHA_WORKFLOWS: ${{ inputs.base-sha-workflows }}
         QUEUE_WORKFLOWS: ${{ inputs.workflows }}
         SOURCE_RUN_ID: ${{ inputs.source-run-id }}
+        SOURCE_RUN_ATTEMPT: ${{ inputs.source-run-attempt }}
+        SOURCE_NOTIFICATION_ACTIVITY: ${{ inputs.source-activity }}
         SOURCE_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
         SOURCE_ACTIVITY: ${{ github.event.action }}
       run: python3 "$ACTION_PATH/../../scripts/reconcile-queue.py"

docs/releases/v2.12.0.md

@@ -0,0 +1,39 @@
+# Nix Forge CI v2.12.0
+
+## Changelog
+
+- Add explicit package-source start and completion notifications using the
+  existing repository token and narrowly scoped callback jobs. Validate actual
+  producer identity and run attempt before deriving its source head or activity.
+- Invalidate all previous required source results and admission when a rerun
+  starts. Completion waits for its producer, revalidates current attempts and
+  preserves actual failed-job links. Verified obsolete callbacks make no writes;
+  tampered producer metadata and future attempts remain errors.
+- After publishing complete queue success, wait at most 55 seconds for GitHub's
+  asynchronous merge to expose the next verified front ref, then dispatch its
+  missing validation once. Empty queues return quietly. Running, failed or
+  incomplete candidates do not start this additional poll.
+
+Existing queue callbacks retain their original payload. Opted-in package
+callers declare optional `source_run_attempt` and `source_activity` reconciler
+dispatch inputs and forward them to `source-run-attempt` and `source-activity`.
+Use `source-activity: in_progress` in a source-only start callback, then
+`source-activity: completed` after all source validation jobs. Retain the queue
+completion callback name and restrict callbacks to their approved ref namespaces.
+Update source branches only after the new caller workflow contract is merged.
+
+No token permissions, required job gates, signing trust or release-builder pins
+change. Full expected-head PR and merge-group validation remains required.
+The 161 repository regression tests and offline workflow/security checks passed
+locally; required hosted validation must pass before publishing this release.
+Coverage includes actual callback payloads, immutable producer identity,
+completion races, obsolete notifications, 16-job source invalidation, request
+budgets and asynchronous next-candidate dispatch.
+
+The source implementation was reviewed at
+`8cc10aa7a626997ef82bb46b6c3e91e0544869ef`. The signed tag and verified release
+manifest identify the protected source commit including these notes. Publication
+uses the separately pinned SLSA source builder and verifies archive checksums,
+all provenance subjects and protected-main ancestry. Restore the v2.11.0 pin and
+remove the optional source notification inputs to roll back. Compatible v2.x
+consumers have no scheduled support end date.