From e6dbdeddc6e154b5f45ac28f9daf67abf832d309 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 12:03:34 +1000 Subject: [PATCH 01/14] docs: design a pull-request preview for config changes Sample one state branch per configured prefix and run the existing process job on it when a pull request changes a policy config, without publishing anything. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- ...2026-09-22-config-change-preview-design.md | 72 +++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-22-config-change-preview-design.md diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md new file mode 100644 index 0000000..efe4320 --- /dev/null +++ b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md @@ -0,0 +1,72 @@ +# Config-change preview for the GitHub Actions example + +## Goal + +When a pull request changes a policy's control configuration (`.github/tf-version-bump/nonproduction.yml` or `production.yml`), the GitHub Actions example previews the change on a sample of real state branches: one branch per configured branch prefix, processed exactly as a live run would process it, with nothing published. The reviewer sees in the run summary how the changed configuration would land on those branches before it is merged. + +## Scope and assumptions + +- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request from a fork skips the preview rather than failing. +- Anyone who can open such a pull request can already edit the workflows, so running Terraform with the registry token at pull-request time adds no new exposure. +- Merging still starts the full live run through each caller's existing `push` trigger. The config-validation workflow is unchanged. +- Only the policy whose configuration the pull request changes is previewed, because each caller triggers on its own configuration path. + +## Design + +### Discovery: sampling and the caller-ref guard + +`discover-state-branches.sh` gains one optional input, `DISCOVERY_SAMPLE_SEED`. When it is empty or unset, discovery behaves exactly as it does today. + +When it is set: + +- **Guard.** The caller ref must match `refs/pull//merge` instead of `refs/heads/`. The seed must be a positive decimal integer. A pull-request ref without a seed is still refused, so the guard is relaxed only in a run that cannot publish. +- **Selection.** Prefix validation, exclusions and matching are unchanged: each branch belongs to the first prefix it matches, as the existing loop assigns it. +- **Sampling.** For each prefix with at least one matched branch, the branches are sorted and one is picked at index `sha256("\t") mod count`. The pick is stable for a pull request across pushes and reruns, and different prefixes and pull requests land on different branches. +- **Empty prefixes.** A prefix that matches no branch produces a `::warning::` annotation on stderr, as an unmatched exclusion does. If no prefix matches any branch, the existing "no remote branches matched" failure applies. +- **Output.** The matrix JSON shape is unchanged, so the `process` job needs no change to consume it. The 256-branch limit applies to the sample. + +A manual `branch_prefix` does not arise, because pull-request runs pass none. + +### Reusable workflow + +- New input `preview` (boolean, default `false`). +- The discovery step passes `DISCOVERY_SAMPLE_SEED: ${{ inputs.preview && github.event.number || '' }}`. +- The `publish` job's `if` adds `!inputs.preview`, so a preview never changes refs, pull requests or issues. +- In a preview, the `Report processing result` step also writes `candidate.patch`, when it exists, to the job summary in a fenced block, truncated at 64 KiB with the same fence and truncation handling as the updater log. The patch holds only `.tf`, formatting and lock-file changes, never Terraform's logs, so it cannot carry a credential Terraform echoed. + +### Callers + +Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml`: + +- Adds `pull_request: paths:` naming its own configuration file. +- Extends its job `if` to also accept `github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository`. +- Passes `preview: ${{ github.event_name == 'pull_request' }}`. +- Suffixes its concurrency group with `-pr-` on pull-request events, so a preview never queues behind or blocks a live run. +- Cancels an older preview for the same pull request when a newer push arrives, if GitHub's documentation confirms that `cancel-in-progress` can be combined with the `queue: max` key the callers use. If it cannot, previews queue per pull request instead. This is checked against GitHub's documentation during implementation, not assumed. + +A pull-request run checks out the merge commit as the control checkout, so the preview uses the pull request's configuration and scripts. Each state branch is still checked out at its discovered commit. + +### Failure behaviour + +A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, as it does today, so the pull request's check fails. The summary names the branch, classification, failed stage and root as it does now. + +## Testing + +Following TDD, in `examples/github-actions/test.sh`: + +- Discovery with a seed picks the same branches for the same seed and one branch per prefix. +- Exclusions are honoured before sampling. +- A prefix with no branches produces the warning and the run continues; no matches at all still fails. +- A pull-request ref is accepted with a seed and refused without one; a malformed seed is refused; the default-branch path is unchanged. +- Workflow structure: `publish` is gated on `!inputs.preview`; each caller triggers on pull requests for its own configuration path and passes `preview`. + +A separate test-cleanup pass follows implementation. + +## Documentation + +- A README section, "Preview on pull requests", covering what runs, how the sample is chosen, that nothing is published and that a failed sample fails the check. +- The install section notes that previews use the same secrets and that fork pull requests skip the preview. + +## Size + +About 30 lines of discovery shell, 15 to 20 lines of workflow YAML, a comparable amount of harness tests and one README section, in keeping with the example's brevity. From 2d0a9752a0edcb41f0b2a6bb6bcb58eabaa60b90 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 12:17:20 +1000 Subject: [PATCH 02/14] docs: revise the config-change preview design after review Run previews as a separate caller job sharing the live job's inputs through YAML aliases, so they can cancel stale runs without the forbidden queue/cancel combination. Add an independent publication guard, derive the sample seed from the pull request ref, and spell out the tests and documentation edits the change requires. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- ...2026-09-22-config-change-preview-design.md | 145 +++++++++++++----- 1 file changed, 109 insertions(+), 36 deletions(-) diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md index efe4320..a658926 100644 --- a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md +++ b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md @@ -2,71 +2,144 @@ ## Goal -When a pull request changes a policy's control configuration (`.github/tf-version-bump/nonproduction.yml` or `production.yml`), the GitHub Actions example previews the change on a sample of real state branches: one branch per configured branch prefix, processed exactly as a live run would process it, with nothing published. The reviewer sees in the run summary how the changed configuration would land on those branches before it is merged. +When a pull request changes a policy's control configuration or its caller workflow, the GitHub Actions example previews the result on a sample of real state branches: one branch per configured branch prefix, processed exactly as a live run processes it, with nothing published. The reviewer sees in the run summary the candidate that the merged configuration would produce on those branches before the pull request is merged. + +The preview is a smoke test, not coverage. It shows the full candidate the merged configuration would produce on each sampled branch, which on a branch that lags the default branch's configuration includes changes the current configuration would already make. It does not isolate the pull request's own delta. ## Scope and assumptions -- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request from a fork skips the preview rather than failing. -- Anyone who can open such a pull request can already edit the workflows, so running Terraform with the registry token at pull-request time adds no new exposure. -- Merging still starts the full live run through each caller's existing `push` trigger. The config-validation workflow is unchanged. -- Only the policy whose configuration the pull request changes is previewed, because each caller triggers on its own configuration path. +- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request from a fork, or into a branch other than the default branch, skips the preview rather than failing. +- Anyone who can open such a pull request can already edit the workflows, so running Terraform with the registry token at pull-request time adds no new exposure. Accidental publication from an unmerged pull request is a new failure mode, so publication gains an independent guard (see [Publication guard](#publication-guard)). +- A pull-request run executes the merge commit's workflows and scripts. A pull request that also edits the scripts or the reusable workflow is previewed with the edited versions. +- Each caller previews only its own policy, triggered by its own configuration file or its own caller workflow file. A change to the scripts or the reusable workflow alone does not trigger a preview. +- Merging still starts the live run through each caller's existing `push` trigger, which remains limited to the configuration file. A merged change to the caller workflow alone takes effect at the next scheduled or manual run, as today. The config-validation workflow is unchanged. -## Design +## Prerequisites to verify before implementation + +The design depends on two GitHub Actions behaviours the documentation does not settle. Both are verified in a disposable private repository before any implementation work, with a probe that mirrors the caller shape below. The probe's structure has already passed the pinned actionlint 1.7.12 and `yq` alias resolution locally. + +1. **Job-level `queue: max`.** GitHub's documentation describes `queue` only for workflow-level `concurrency`. The design moves each caller's live concurrency to the job level. If job-level `queue: max` is rejected or ignored, the fallback keeps workflow-level concurrency with an event-aware group, `github.repository_id` plus a `-pr-` suffix on pull-request events, and `queue: max`. Previews then queue per pull request instead of cancelling, and the README says so. +2. **A read-only preview job calling a workflow whose skipped `publish` job requests write permissions.** GitHub documents that a called workflow can only downgrade the caller's `GITHUB_TOKEN` permissions. It does not say whether a job the `if` skips is still checked. If the preview job cannot declare `contents: read`, the fallback gives it the caller's existing permissions, and publication relies on the `publish` job's `if` together with the publication guard. + +GitHub's documentation confirms that `queue: max` combined with `cancel-in-progress: true` is a validation error, so the two are never combined in one `concurrency` block. It also confirms YAML anchors and aliases are supported in workflows. It says nothing about merge keys (`<<`), so the design does not use them. -### Discovery: sampling and the caller-ref guard +## Design -`discover-state-branches.sh` gains one optional input, `DISCOVERY_SAMPLE_SEED`. When it is empty or unset, discovery behaves exactly as it does today. +### Callers -When it is set: +Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml` changes as follows: -- **Guard.** The caller ref must match `refs/pull//merge` instead of `refs/heads/`. The seed must be a positive decimal integer. A pull-request ref without a seed is still refused, so the guard is relaxed only in a run that cannot publish. -- **Selection.** Prefix validation, exclusions and matching are unchanged: each branch belongs to the first prefix it matches, as the existing loop assigns it. -- **Sampling.** For each prefix with at least one matched branch, the branches are sorted and one is picked at index `sha256("\t") mod count`. The pick is stable for a pull request across pushes and reruns, and different prefixes and pull requests land on different branches. -- **Empty prefixes.** A prefix that matches no branch produces a `::warning::` annotation on stderr, as an unmatched exclusion does. If no prefix matches any branch, the existing "no remote branches matched" failure applies. -- **Output.** The matrix JSON shape is unchanged, so the `process` job needs no change to consume it. The 256-branch limit applies to the sample. +- **Triggers.** Adds `pull_request: paths:` naming its own configuration file and its own workflow file. The `push`, `workflow_dispatch` and `schedule` triggers are unchanged. +- **Workflow-level concurrency.** Removed. It moves to the live job. +- **Two jobs.** Both call the reusable workflow, with the same `with:` and `secrets:` blocks. The live job defines them as anchors (`with: &policy-inputs`, `secrets: &policy-secrets`) and the preview job uses the aliases (`with: *policy-inputs`, `secrets: *policy-secrets`). Every policy value is therefore written once, and a pull request that edits a value, such as a tool pin, a prefix or a root, is previewed with it. + - `automation`, the live job: `if: github.ref == refs/heads/ && github.event_name != 'pull_request'`, with job-level `concurrency: {group: tf-version-bump--, cancel-in-progress: false, queue: max}`. Its behaviour is otherwise unchanged. + - `preview`: `if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch`, with job-level `concurrency: {group: tf-version-bump--preview--, cancel-in-progress: true}` and `permissions: contents: read` (subject to prerequisite 2). +- **The `preview` input.** The shared `with:` block includes `preview: ${{ github.event_name == 'pull_request' }}`, which is `false` for the live job and `true` for the preview job without either block differing. `branch_prefix`, `dry_run` and `terraform_init_upgrade` keep their existing event-guarded expressions, which evaluate to `''`, `false` and `false` on pull-request events. -A manual `branch_prefix` does not arise, because pull-request runs pass none. +A newer push to a pull request cancels that pull request's older preview, and a preview never waits behind or blocks a live run. ### Reusable workflow -- New input `preview` (boolean, default `false`). -- The discovery step passes `DISCOVERY_SAMPLE_SEED: ${{ inputs.preview && github.event.number || '' }}`. -- The `publish` job's `if` adds `!inputs.preview`, so a preview never changes refs, pull requests or issues. -- In a preview, the `Report processing result` step also writes `candidate.patch`, when it exists, to the job summary in a fenced block, truncated at 64 KiB with the same fence and truncation handling as the updater log. The patch holds only `.tf`, formatting and lock-file changes, never Terraform's logs, so it cannot carry a credential Terraform echoed. +- **New input.** `preview` (boolean, default `false`). +- **Discovery.** The discovery step passes `DISCOVERY_PREVIEW: ${{ inputs.preview }}`. +- **Publication.** The `publish` job's `if` becomes `${{ always() && needs.discover.result == 'success' && !inputs.preview }}`. +- **Summary.** The `Report processing result` step receives `PREVIEW: ${{ inputs.preview }}` through `env`, because the local harness runs step bodies without evaluating expressions. When `PREVIEW` is `true` and `candidate.patch` exists, the step appends it to the job summary under `#### Candidate patch`. The patch exists only for a changed, validated candidate, so there is none for failures or no-change results. +- **Shared rendering.** The existing fence-escalation and 64 KiB truncation code becomes one shell function in the step, `append_fenced_file `, used for each updater log and for the patch. The patch holds only `.tf` changes, formatting changes and root lock files (`write_candidate`, `process-state-branch.sh`), never Terraform's logs, so it cannot carry a credential Terraform echoed. -### Callers +### Discovery: sampling + +`discover-state-branches.sh` gains one optional input, `DISCOVERY_PREVIEW`. Unset or `false` means discovery behaves exactly as it does today, including for the report workflow, which does not set it. Any value other than exact `true` or `false` is refused. + +When it is `true`: -Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml`: +- **Guard.** The caller ref must match `^refs/pull/([1-9][0-9]*)/merge$`, instead of `refs/heads/`. The captured pull request number is the sampling seed, so seed and ref cannot disagree. A pull-request ref without `DISCOVERY_PREVIEW=true` is still refused. +- **No manual prefix.** A non-empty `DISCOVERY_MANUAL_PREFIX` is refused, because a manual prefix only arises from `workflow_dispatch`. +- **Selection.** Prefix validation, exclusions and matching are unchanged. Each branch belongs to the first prefix it matches, as the existing loop assigns it. +- **Branch limit.** The 256-branch limit is checked against the full matched set before sampling, so a policy the live run would refuse fails its preview too. +- **Sampling.** For each prefix that owns at least one branch, its branches are sorted (the script already sets `LC_ALL=C`), and one is picked. The index is the first 15 hexadecimal digits of `sha256("\t")`, read as an integer, modulo the number of branches the prefix owns. Fifteen digits stay within bash's signed 64-bit arithmetic, so the index is never negative. +- **Stability.** The pick is deterministic while the prefix's set of branches is unchanged. Adding, deleting or excluding a branch under the prefix may move it. Different pull requests are spread across a prefix's branches but may land on the same one. +- **Prefixes that own no branch.** In preview mode only, each such prefix produces `::warning::branch prefix '' owns no branch to preview` on stderr, escaped as the existing exclusion warning is. A prefix owns no branch when none of its matches remain after exclusions and earlier prefixes. If no branch is matched at all, the existing "no remote branches matched" failure applies. +- **Output.** The matrix JSON shape is unchanged, so the `process` job consumes it unmodified. `prepare_contract` checks the control checkout's `HEAD` against `control_oid`, which is the merge commit in a pull-request run. -- Adds `pull_request: paths:` naming its own configuration file. -- Extends its job `if` to also accept `github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository`. -- Passes `preview: ${{ github.event_name == 'pull_request' }}`. -- Suffixes its concurrency group with `-pr-` on pull-request events, so a preview never queues behind or blocks a live run. -- Cancels an older preview for the same pull request when a newer push arrives, if GitHub's documentation confirms that `cancel-in-progress` can be combined with the `queue: max` key the callers use. If it cannot, previews queue per pull request instead. This is checked against GitHub's documentation during implementation, not assumed. +### Publication guard -A pull-request run checks out the merge commit as the control checkout, so the preview uses the pull request's configuration and scripts. Each state branch is still checked out at its discovered commit. +`reconcile-state-branch.sh publish` gains required inputs `RECONCILE_CALLER_REF` (`${{ github.ref }}`) and `RECONCILE_DEFAULT_BRANCH` (`${{ github.event.repository.default_branch }}`). Before any Git or GitHub operation, it refuses to publish unless the caller ref is `refs/heads/`, dry run or not. Publication is then guarded twice: by the `publish` job's `if` and by this check. Discovery does not claim to know whether a run can publish. ### Failure behaviour A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, as it does today, so the pull request's check fails. The summary names the branch, classification, failed stage and root as it does now. +Because the sample is stable per pull request, a branch that already fails on the default branch's configuration fails every push and rerun of the preview. The README says that a red preview may be pre-existing, points at the branch's marked failure issue, and advises against making the preview a required check. An invalid configuration fails every sampled branch at the update stage with classification `branch-update`, while the config-validation workflow reports the cause. + ## Testing -Following TDD, in `examples/github-actions/test.sh`: +Following TDD, in `examples/github-actions/test.sh` unless noted. + +**Discovery** + +- A known-answer case: a fixed pull request number, prefix and branch list select a named branch, pinning the formula. +- One branch per prefix is selected, and exclusions are applied before sampling. +- A prefix that owns no branch warns in preview mode and not in live mode. The run continues when another prefix selects a branch, and fails when none does. +- The 256-branch limit fails a preview whose full matched set exceeds it. +- A pull-request ref is accepted with `DISCOVERY_PREVIEW=true` and refused without it. A malformed ref (`refs/pull/0/merge`, `refs/pull/7/head`) is refused, as are a non-boolean `DISCOVERY_PREVIEW` and a manual prefix in preview mode. +- The default-branch path and its matrix output are unchanged. + +**Reusable workflow** + +- Replace the exact `.publish.if` assertion in `workflow_publishes_after_processing_failures` with one requiring `always()`, `needs.discover.result == 'success'` and `!inputs.preview`. The existing mutants that drop `always()` still fail, and a new mutant that drops `!inputs.preview` fails too. +- `DISCOVERY_PREVIEW` is wired from `inputs.preview`, and `PREVIEW` reaches `Report processing result` through `env`. +- The summary step, run through `run_workflow_step`: + - appends the patch in preview mode when `candidate.patch` exists; + - omits it outside preview mode and when the file is absent; + - escalates the fence past backticks in the patch; + - truncates past 64 KiB with the existing notice. -- Discovery with a seed picks the same branches for the same seed and one branch per prefix. -- Exclusions are honoured before sampling. -- A prefix with no branches produces the warning and the run continues; no matches at all still fails. -- A pull-request ref is accepted with a seed and refused without one; a malformed seed is refused; the default-branch path is unchanged. -- Workflow structure: `publish` is gated on `!inputs.preview`; each caller triggers on pull requests for its own configuration path and passes `preview`. +**Callers** + +- Each caller has the pull-request trigger for its own configuration and workflow file. +- The live job's `if` refuses pull-request events and non-default refs. +- The preview job's `if` requires a same-repository head and a default-branch base. +- The live job has job-level `queue: max` without `cancel-in-progress: true`, and the preview job has `cancel-in-progress: true` without `queue`. +- Both jobs resolve to identical `with:` and `secrets:` blocks, and the shared `preview` expression is event-derived. + +**Publication guard** (`reconcile-test.sh`) + +- Publication is refused for a pull-request ref and for another branch, and in dry-run mode too. +- The refusal happens before any Git or GitHub command. +- The existing default-branch cases pass with the new inputs. A separate test-cleanup pass follows implementation. ## Documentation -- A README section, "Preview on pull requests", covering what runs, how the sample is chosen, that nothing is published and that a failed sample fails the check. -- The install section notes that previews use the same secrets and that fork pull requests skip the preview. +Add a README section, "Preview on pull requests", that covers: +- what runs, and that nothing is published; +- how the sample is chosen and why it is stable; +- that the patch is the full candidate the merged configuration would produce, not the pull request's delta; +- that a failed sample fails the check and may be pre-existing, and that the check should not be required; +- that pull requests with a merge conflict run no preview, because GitHub does not run `pull_request` workflows for them; +- that the preview is not refreshed when the base or state branches move, only on `opened`, `synchronize` and `reopened`; +- that a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled; +- that a pull request editing the scripts is previewed with the edited scripts; +- that an invalid configuration appears as `branch-update` on every sampled branch. + +Edit the passages the change makes false: +- `examples/github-actions/README.md`, the Install section's "Both callers run only from the default branch." The live job still does; the preview job runs for same-repository pull requests into the default branch. +- `examples/github-actions/README.md`, the Configure updates section's "Pull requests changing these files run a read-only config validation check; they do not process state branches or run Terraform." +- `examples/github-actions/README.md`, the Run and inspect section's description of the `process` summary, which gains the patch in a preview. +- `docs/ADVANCED-USAGE.md`, the GitHub Actions paragraph's "callers that run only from the default branch" and "full state-branch dry runs remain deferred". + +Also in the README: +- The Install section notes that previews use the same secrets. +- The environment-variable section notes that a GitHub Environment with deployment-branch rules may withhold its secrets from `refs/pull//merge`, which fails or holds the preview. ## Size -About 30 lines of discovery shell, 15 to 20 lines of workflow YAML, a comparable amount of harness tests and one README section, in keeping with the example's brevity. +Roughly: +- 40 lines of discovery shell; +- 10 lines of publication guard; +- 30 lines of caller and reusable workflow YAML, net of the shared rendering function; +- a comparable amount of harness tests; +- one README section, plus four passage edits. + +This is in keeping with the example's brevity. From 10bf995d868d38fa138164675e101f1cd0db5d59 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 13:10:37 +1000 Subject: [PATCH 03/14] docs: record the verified Actions behaviour for the preview design A disposable probe repository confirmed that job-level queue: max serialises live runs, that job-level cancel-in-progress cancels stale previews, and that a read-only caller job fails at startup when the called workflow's skipped publish job requests write permissions, so the preview job keeps the caller's permissions. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../specs/2026-09-22-config-change-preview-design.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md index a658926..dc29734 100644 --- a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md +++ b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md @@ -14,12 +14,13 @@ The preview is a smoke test, not coverage. It shows the full candidate the merge - Each caller previews only its own policy, triggered by its own configuration file or its own caller workflow file. A change to the scripts or the reusable workflow alone does not trigger a preview. - Merging still starts the live run through each caller's existing `push` trigger, which remains limited to the configuration file. A merged change to the caller workflow alone takes effect at the next scheduled or manual run, as today. The config-validation workflow is unchanged. -## Prerequisites to verify before implementation +## Verified GitHub Actions behaviour -The design depends on two GitHub Actions behaviours the documentation does not settle. Both are verified in a disposable private repository before any implementation work, with a probe that mirrors the caller shape below. The probe's structure has already passed the pinned actionlint 1.7.12 and `yq` alias resolution locally. +The design depends on GitHub Actions behaviours that the documentation leaves open. Each was checked in a disposable private repository (`yesdevnull/tfvb-preview-probe`) on 2026-09-22 with sleep-only jobs that mirror the caller shape below. The probe's workflows also pass the pinned actionlint 1.7.12, and `yq` resolves their aliases. -1. **Job-level `queue: max`.** GitHub's documentation describes `queue` only for workflow-level `concurrency`. The design moves each caller's live concurrency to the job level. If job-level `queue: max` is rejected or ignored, the fallback keeps workflow-level concurrency with an event-aware group, `github.repository_id` plus a `-pr-` suffix on pull-request events, and `queue: max`. Previews then queue per pull request instead of cancelling, and the README says so. -2. **A read-only preview job calling a workflow whose skipped `publish` job requests write permissions.** GitHub documents that a called workflow can only downgrade the caller's `GITHUB_TOKEN` permissions. It does not say whether a job the `if` skips is still checked. If the preview job cannot declare `contents: read`, the fallback gives it the caller's existing permissions, and publication relies on the `publish` job's `if` together with the publication guard. +1. **Job-level `queue: max` works.** GitHub's documentation describes `queue` only for workflow-level `concurrency`. Three dispatches of a job carrying `concurrency: {group, cancel-in-progress: false, queue: max}` that calls a reusable workflow ran one after another, each starting after the previous finished, and none was cancelled. The default `queue: single` would have cancelled the middle one while it was pending. +2. **A read-only preview job fails at startup.** A caller job declaring `permissions: contents: read` that calls a workflow whose `publish` job requests write permissions fails with `startup_failure`, even though `preview: true` means `publish` would be skipped. GitHub checks a called job's permissions whatever its `if`. The preview job therefore keeps the caller's permissions, and publication relies on two guards: the `publish` job's `if` and the [publication guard](#publication-guard). +3. **Job-level `cancel-in-progress: true` cancels an older preview.** On a pull request, a second push cancelled the in-progress preview for the first push, and the second run completed. In that run the live job and `publish` were both skipped, and the called workflow received `preview=true` through the aliased `with:` block. GitHub's documentation confirms that `queue: max` combined with `cancel-in-progress: true` is a validation error, so the two are never combined in one `concurrency` block. It also confirms YAML anchors and aliases are supported in workflows. It says nothing about merge keys (`<<`), so the design does not use them. @@ -33,7 +34,7 @@ Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml` - **Workflow-level concurrency.** Removed. It moves to the live job. - **Two jobs.** Both call the reusable workflow, with the same `with:` and `secrets:` blocks. The live job defines them as anchors (`with: &policy-inputs`, `secrets: &policy-secrets`) and the preview job uses the aliases (`with: *policy-inputs`, `secrets: *policy-secrets`). Every policy value is therefore written once, and a pull request that edits a value, such as a tool pin, a prefix or a root, is previewed with it. - `automation`, the live job: `if: github.ref == refs/heads/ && github.event_name != 'pull_request'`, with job-level `concurrency: {group: tf-version-bump--, cancel-in-progress: false, queue: max}`. Its behaviour is otherwise unchanged. - - `preview`: `if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch`, with job-level `concurrency: {group: tf-version-bump--preview--, cancel-in-progress: true}` and `permissions: contents: read` (subject to prerequisite 2). + - `preview`: `if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch`, with job-level `concurrency: {group: tf-version-bump--preview--, cancel-in-progress: true}`. It declares no job-level `permissions`, because a read-only caller job fails at startup (see [Verified GitHub Actions behaviour](#verified-github-actions-behaviour)). - **The `preview` input.** The shared `with:` block includes `preview: ${{ github.event_name == 'pull_request' }}`, which is `false` for the live job and `true` for the preview job without either block differing. `branch_prefix`, `dry_run` and `terraform_init_upgrade` keep their existing event-guarded expressions, which evaluate to `''`, `false` and `false` on pull-request events. A newer push to a pull request cancels that pull request's older preview, and a preview never waits behind or blocks a live run. From 674bc607a4dcc54753d6d142609a31f521f0097d Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 13:38:46 +1000 Subject: [PATCH 04/14] docs: plan the config-change preview implementation Seven TDD tasks: the discovery preview guard, per-prefix sampling, the publication guard, the reusable workflow preview mode, the caller preview jobs, documentation, and full validation with a separate test cleanup pass. The spec now keeps the live job's existing if and names the shared renderer's truncation noun. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../plans/2026-09-22-config-change-preview.md | 1105 +++++++++++++++++ ...2026-09-22-config-change-preview-design.md | 4 +- 2 files changed, 1107 insertions(+), 2 deletions(-) create mode 100644 docs/superpowers/plans/2026-09-22-config-change-preview.md diff --git a/docs/superpowers/plans/2026-09-22-config-change-preview.md b/docs/superpowers/plans/2026-09-22-config-change-preview.md new file mode 100644 index 0000000..d0b4ac6 --- /dev/null +++ b/docs/superpowers/plans/2026-09-22-config-change-preview.md @@ -0,0 +1,1105 @@ +# Config-change preview Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** When a same-repository pull request into the default branch changes a policy's control config or caller workflow, the GitHub Actions example processes one sampled state branch per configured prefix, exactly as a live run would, and publishes nothing. + +**Architecture:** +- Discovery gains a preview mode. It accepts only a `refs/pull//merge` caller ref, seeds a deterministic per-prefix sample from ``, and otherwise emits the same matrix as today. +- The reusable workflow gains a `preview` input. It skips `publish` and adds the candidate patch to the `process` summary. +- Each caller gains a `preview` job. It reuses the live job's `with:`/`secrets:` blocks through YAML aliases and carries its own cancelling concurrency. +- `reconcile-state-branch.sh publish` independently refuses any caller ref other than the default branch. + +**Tech Stack:** Bash (the scripts run under `set -euo pipefail` with `LC_ALL=C`), GitHub Actions YAML, `jq`, mikefarah `yq`, the example's own harnesses (`examples/github-actions/test.sh`, `reconcile-test.sh`, `report-test.sh`), Go documentation tests, actionlint via `make actionlint`, shellcheck 0.11.0 via `make shellcheck`. + +**Spec:** `docs/superpowers/specs/2026-09-22-config-change-preview-design.md`. Read it before starting any task; it carries the reasoning and the verified GitHub Actions behaviour. + +## Global Constraints + +**Prose and Markdown** +- Use Australian / British spelling in prose and comments. +- Write each Markdown paragraph, list item and quote on one line. `TestDocumentationProseIsNotHardWrapped` fails on hard-wrapped prose. + +**Process** +- Follow TDD in every task: write the failing test, watch it fail, implement, watch it pass. +- Test output must be pristine. Capture and assert expected diagnostics. + +**Commits and pushes** +- Commit with the GenAI wrapper, passing the message through a file: `/Users/dan/.claude/bin/claude-git -C /Users/dan/Code/tf-version-bump commit -F `. Write the message file in the session scratchpad. +- End every commit message with these two lines: + - `Co-Authored-By: Claude Opus 5 (1M context) ` + - `Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ` +- Never `git add -A`. Stage named paths. +- Never push without Dan's confirmation. + +**Harness** +- Run the example harness with the GenAI signer so fixture commits never reach Dan's 1Password signer: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh ` (from the repository root). +- Discovery and workflow-step tests need no Docker; `test_processing_*` tests do. + +**Workflow YAML** +- Keep each caller's `with:` keys indented six spaces. `scripts/update-actions-release-pin.sh` matches ` tf_version_bump_version:` and ` tf_version_bump_archive_sha256:` exactly once per caller. +- Never combine `queue: max` with `cancel-in-progress: true` in one `concurrency` block; GitHub rejects it. +- Do not use YAML merge keys (`<<`). Only anchors (`&name`) and whole-node aliases (`*name`). +- The preview job must not declare job-level `permissions`. A read-only caller job fails at startup when the called workflow's `publish` job requests write, even when it is skipped (verified; see the spec). +- Pass values into `run:` bodies only through `env:`, never `${{ }}` inside a script body. + +**Size** +- Keep scripts and workflows small; consumers copy them. + +--- + +## File Structure + +| File | Change | Responsibility | +| --- | --- | --- | +| `examples/github-actions/.github/scripts/discover-state-branches.sh` | Modify | Preview guard (`DISCOVERY_PREVIEW`, PR merge ref, seed) and per-prefix sampling | +| `examples/github-actions/.github/scripts/reconcile-state-branch.sh` | Modify | Refuse publication from any ref but the default branch | +| `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml` | Modify | `preview` input, discovery and publish wiring, `publish` gate, shared fenced renderer, preview patch | +| `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` | Modify | PR trigger, job-level concurrency, `preview` job via aliases | +| `examples/github-actions/.github/workflows/tf-version-bump-production.yml` | Modify | Same as non-production | +| `examples/github-actions/test.sh` | Modify | Discovery, reusable-workflow and caller tests | +| `examples/github-actions/reconcile-test.sh` | Modify | Publication guard tests | +| `examples/github-actions/README.md` | Modify | New "Preview on pull requests" section; four passage edits | +| `docs/ADVANCED-USAGE.md` | Modify | GitHub Actions paragraph | + +--- + +### Task 1: Discovery preview guard + +**Files:** +- Modify: `examples/github-actions/.github/scripts/discover-state-branches.sh:83-85` (the caller-ref check) +- Modify: `examples/github-actions/test.sh:362-430` (`setup_discovery_repository`, `run_discovery`) +- Test: `examples/github-actions/test.sh` (new `test_discovery_preview_accepts_only_pull_request_merge_refs`) + +**Interfaces:** +- Consumes: nothing new. +- Produces: + - Env input `DISCOVERY_PREVIEW`: unset means `false`. Otherwise it must be exactly `true` or `false`. + - Shell variables `preview` (`true`/`false`) and `preview_seed` (the decimal PR number, set only when `preview=true`). Task 2 reads both. + - Diagnostics, exactly: + - `discovery input error: DISCOVERY_PREVIEW must be true or false` + - `discovery caller error: preview caller ref must be refs/pull//merge` + - `discovery input error: a preview cannot take a manual prefix` + +- [ ] **Step 1: Let the harness pass `DISCOVERY_PREVIEW` only when a test sets it** + +In `setup_discovery_repository` (`test.sh:362`), add this as its second line, after `cleanup_discovery_repository`: + +```bash + unset DISCOVERY_PREVIEW +``` + +In `run_discovery` (`test.sh:414`), export `DISCOVERY_PREVIEW` inside the existing subshell only when a test has set it. An unset `DISCOVERY_PREVIEW` then stays unset for the script, which is the report workflow's path. Do not use `env` for this: `test_discovery_uses_runner_git_not_a_workstation_shim` runs discovery with a `PATH` that has no `env`. The subshell keeps the export from leaking. Change the line after `cd "$DISCOVERY_REPO"` to: + +```bash + cd "$DISCOVERY_REPO" + [[ -z "${DISCOVERY_PREVIEW+set}" ]] || export DISCOVERY_PREVIEW +``` + +- [ ] **Step 2: Write the failing test** + +Add after `test_discovery_rejects_invalid_inputs_by_stage` in `test.sh`: + +```bash +test_discovery_preview_accepts_only_pull_request_merge_refs() { + # Production break caught: relaxing the default-branch guard for previews lets any ref drive + # discovery, a pull-request ref passes without asking for a preview, or a preview's seed can + # disagree with the pull request it runs for. + setup_discovery_repository + add_discovery_branch "state/prod/example" + DISCOVERY_ALLOWED_PREFIXES="state/prod/" + + DISCOVERY_CALLER_REF="refs/pull/42/merge" + assert_discovery_failure "discovery caller error: caller ref must be refs/heads/main" \ + "a pull-request ref without a preview" + + DISCOVERY_PREVIEW=true + local output + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/prod/example"]' <<<"$output" >/dev/null \ + || fail "preview discovery refused a pull-request merge ref: $output" + + local ref + for ref in refs/heads/main refs/pull/0/merge refs/pull/042/merge refs/pull/7/head refs/pull/x/merge; do + DISCOVERY_CALLER_REF=$ref + assert_discovery_failure "discovery caller error: preview caller ref must be refs/pull//merge" \ + "a preview from $ref" + done + + DISCOVERY_CALLER_REF="refs/pull/42/merge" + DISCOVERY_MANUAL_PREFIX="state/prod/" + assert_discovery_failure "discovery input error: a preview cannot take a manual prefix" \ + "a preview with a manual prefix" + + DISCOVERY_MANUAL_PREFIX="" + local value + for value in "" TRUE yes; do + DISCOVERY_PREVIEW=$value + assert_discovery_failure "discovery input error: DISCOVERY_PREVIEW must be true or false" \ + "a preview flag of '$value'" + done +} +``` + +`test_discovery_*` functions are collected automatically by `compgen` in the harness's no-argument run, so there is no list to update. + +- [ ] **Step 3: Run it and watch it fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_accepts_only_pull_request_merge_refs` + +Expected: the harness exits non-zero without printing `PASS:`. Discovery still demands `refs/heads/main`, so `output=$(run_discovery)` fails under the harness's `set -e`, and discovery's own `discovery caller error: caller ref must be refs/heads/main` is the last line on stderr. + +- [ ] **Step 4: Implement the guard** + +In `discover-state-branches.sh`, replace lines 83–85: + +```bash +expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" +[[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ + || fail_discovery caller "caller ref must be $expected_caller_ref" +``` + +with: + +```bash +preview=${DISCOVERY_PREVIEW-false} +[[ "$preview" == true || "$preview" == false ]] \ + || fail_discovery input "DISCOVERY_PREVIEW must be true or false" +if [[ "$preview" == true ]]; then + # A preview runs from a pull request's merge ref. Its number seeds the sample, so the seed + # cannot disagree with the pull request, and a manual prefix only arises from a dispatch. + [[ "$DISCOVERY_CALLER_REF" =~ ^refs/pull/([1-9][0-9]*)/merge$ ]] \ + || fail_discovery caller "preview caller ref must be refs/pull//merge" + preview_seed=${BASH_REMATCH[1]} + [[ -z "$DISCOVERY_MANUAL_PREFIX" ]] \ + || fail_discovery input "a preview cannot take a manual prefix" +else + expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" + [[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ + || fail_discovery caller "caller ref must be $expected_caller_ref" +fi +``` + +Also add `DISCOVERY_PREVIEW` to `usage()` so `--help` documents it. Replace the heredoc body line `Discover immutable inputs for configured Terraform state branches.` with: + +```text +Discover immutable inputs for configured Terraform state branches. +With DISCOVERY_PREVIEW=true, run from refs/pull//merge and select one +sampled branch per prefix instead of every matching branch. +``` + +- [ ] **Step 5: Run the new test and every discovery test** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_accepts_only_pull_request_merge_refs` + +Expected: `PASS: test_discovery_preview_accepts_only_pull_request_merge_refs` + +Then run every existing discovery test by name (list them with `grep -o '^test_discovery_[a-z_]*' examples/github-actions/test.sh`), passing all names as arguments in one call. Expected: one `PASS:` line per test and nothing else. + +Then run `examples/github-actions/report-test.sh`, whose report workflow calls discovery without `DISCOVERY_PREVIEW`. Expected: all `PASS:`. + +- [ ] **Step 6: Shellcheck and commit** + +Run: `make shellcheck`. Expected: no output, exit 0. + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/discover-state-branches.sh examples/github-actions/test.sh +``` + +Commit message: `feat(actions-example): accept pull-request refs in preview discovery`. + +--- + +### Task 2: Discovery per-prefix sampling + +**Files:** +- Modify: `examples/github-actions/.github/scripts/discover-state-branches.sh`, from the branch-matching loop (~line 143) to the sort (~line 174) +- Test: `examples/github-actions/test.sh` (new `test_discovery_preview_samples_one_branch_per_prefix`, `test_discovery_preview_warns_for_prefixes_that_own_no_branch`, `test_discovery_preview_keeps_the_live_matrix_limit`) + +**Interfaces:** +- Consumes: `preview` and `preview_seed` from Task 1; the existing `selection_prefixes` array and `branch_records` array (`"\t"` entries). +- Produces: + - In preview mode, `branch_records` holds one sorted record per prefix that owns a branch. The matrix JSON shape is unchanged. + - Preview-only warning on stderr, exactly: `::warning::branch prefix '' owns no branch to preview`, with `%` escaped as `%25`. + +- [ ] **Step 1: Write the failing tests** + +Add after the Task 1 test in `test.sh`: + +```bash +test_discovery_preview_samples_one_branch_per_prefix() { + # Production break caught: a preview processes every branch, picks differently on each push, + # or samples a branch its caller excludes. + setup_discovery_repository + local name + for name in alpha bravo charlie; do + add_discovery_branch "state/staging/$name" + add_discovery_branch "state/production/$name" + done + add_discovery_branch "state/production/excluded" + DISCOVERY_ALLOWED_PREFIXES=$'state/staging/\nstate/production/\n!state/production/excluded' + DISCOVERY_PREVIEW=true + local output stderr_file="$DISCOVERY_TMP_ROOT/sample.stderr" + + # Known answers: the first 15 hex digits of sha256("\t"), modulo the three + # branches each prefix owns, index the sorted branches. Pull request 42 picks index 2 of + # state/production/ and 0 of state/staging/; pull request 43 picks index 1 of both. + DISCOVERY_CALLER_REF="refs/pull/42/merge" + output=$(run_discovery 2>"$stderr_file") + jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ + <<<"$output" >/dev/null || fail "pull request 42 did not sample its known branches: $output" + [[ ! -s "$stderr_file" ]] || fail "sampling reported a warning: $(<"$stderr_file")" + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ + <<<"$output" >/dev/null || fail "a repeat preview of pull request 42 sampled differently: $output" + + DISCOVERY_CALLER_REF="refs/pull/43/merge" + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/production/bravo", "state/staging/bravo"]' \ + <<<"$output" >/dev/null || fail "pull request 43 did not sample its known branches: $output" +} + + +test_discovery_preview_warns_for_prefixes_that_own_no_branch() { + # Production break caught: a prefix with nothing to preview passes silently, a narrower prefix + # listed after a broader one is reported as matching nothing without saying why, a repeated + # prefix previews its branch twice, or live runs gain the warning. + setup_discovery_repository + add_discovery_branch "state/staging/alpha" + DISCOVERY_ALLOWED_PREFIXES=$'state/\nstate/staging/\nstate/\naws-state/100%/' + local output stderr_file="$DISCOVERY_TMP_ROOT/unowned.stderr" + + output=$(run_discovery 2>"$stderr_file") + [[ ! -s "$stderr_file" ]] || fail "a live run reported unowned prefixes: $(<"$stderr_file")" + + DISCOVERY_PREVIEW=true + DISCOVERY_CALLER_REF="refs/pull/42/merge" + output=$(run_discovery 2>"$stderr_file") + jq -e '.include | map(.branch) == ["state/staging/alpha"]' <<<"$output" >/dev/null \ + || fail "a preview did not sample the one owned branch exactly once: $output" + [[ "$(<"$stderr_file")" == "::warning::branch prefix 'state/staging/' owns no branch to preview"$'\n'"::warning::branch prefix 'aws-state/100%25/' owns no branch to preview" ]] \ + || fail "a preview did not name each prefix that owns no branch once: $(<"$stderr_file")" +} + + +test_discovery_preview_keeps_the_live_matrix_limit() { + # Production break caught: a policy the live run refuses for exceeding the matrix limit + # previews green because the sample itself is small. + setup_discovery_repository + add_numbered_discovery_branches 0 256 + DISCOVERY_ALLOWED_PREFIXES="state/limit/" + DISCOVERY_PREVIEW=true + DISCOVERY_CALLER_REF="refs/pull/42/merge" + assert_discovery_failure "discovery matrix error:" "a preview of more than 256 branches" +} +``` + +- [ ] **Step 2: Run them and watch them fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_samples_one_branch_per_prefix test_discovery_preview_warns_for_prefixes_that_own_no_branch test_discovery_preview_keeps_the_live_matrix_limit` + +Expected: +- The first fails with `pull request 42 did not sample its known branches`, because all six branches are returned. +- The third already passes: the limit check runs before any sampling exists. It stays as the regression guard for the ordering in Step 3. + +- [ ] **Step 3: Implement sampling** + +In `discover-state-branches.sh`: +- Change `branch_records=()` (just before the `while IFS=$'\t' read -r oid ref` loop) to also declare the owner map: + +```bash +branch_records=() +declare -A branch_owners=() +``` + +- Inside the prefix loop, record the owning prefix next to the record: + +```bash + for prefix in "${selection_prefixes[@]}"; do + if [[ "$branch" == "$prefix"* ]]; then + branch_records+=("$branch"$'\t'"$oid") + branch_owners[$branch]=$prefix + break + fi + done +``` + +- Define this function directly after `branch_is_excluded()`: + +```bash +# Keeps one branch per prefix, so a preview processes a sample rather than the whole policy. A +# branch belongs to the first prefix it matches; the pick is the first 15 hex digits of +# sha256("\t") modulo the prefix's branch count, which stays within bash's signed +# 64-bit arithmetic and is stable while that prefix's branches are unchanged. +sample_branch_records() { + local prefix record digest + local -a owned sampled=() + declare -A seen_prefixes=() + for prefix in "${selection_prefixes[@]}"; do + [[ -z "${seen_prefixes[$prefix]-}" ]] || continue + seen_prefixes[$prefix]=1 + owned=() + for record in "${branch_records[@]}"; do + [[ "${branch_owners[${record%%$'\t'*}]}" != "$prefix" ]] || owned+=("$record") + done + if [[ ${#owned[@]} -eq 0 ]]; then + echo "::warning::branch prefix '${prefix//'%'/%25}' owns no branch to preview" >&2 + continue + fi + digest=$(printf '%s\t%s' "$preview_seed" "$prefix" | sha256sum) + sampled+=("${owned[$((16#${digest:0:15} % ${#owned[@]}))]}") + done + readarray -t branch_records < <(printf '%s\n' "${sampled[@]}" | sort) +} +``` + +- Directly after the existing line `readarray -t branch_records < <(printf '%s\n' "${branch_records[@]}" | sort)`, add: + +```bash +[[ "$preview" != true ]] || sample_branch_records +``` + +This order keeps the selection and 256-branch checks on the full matched set. It also means `owned` is built from sorted records. + +`sampled` is never empty here. Selection already failed if nothing matched, and every matched record has an owner. + +- [ ] **Step 4: Run the tests and watch them pass** + +Run the Step 2 command again. Expected: three `PASS:` lines and nothing else. + +Then run every discovery test by name, as in Task 1 Step 5, plus `examples/github-actions/report-test.sh`. Expected: all `PASS:`. + +- [ ] **Step 5: Shellcheck and commit** + +Run: `make shellcheck`. Expected: no output. + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/discover-state-branches.sh examples/github-actions/test.sh +``` + +Commit message: `feat(actions-example): sample one state branch per prefix in preview discovery`. + +--- + +### Task 3: Publication guard + +**Files:** +- Modify: `examples/github-actions/.github/scripts/reconcile-state-branch.sh` (`usage()`, new `require_default_branch_caller`, first line of `publish_result`) +- Modify: `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml` (`Publish processing result` step `env`) +- Modify: `examples/github-actions/reconcile-test.sh` (`run_publish`, new test, test list) +- Test: `examples/github-actions/test.sh` (`test_workflow_runs_three_jobs_with_current_attempt_results`) + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: + - Required `publish` inputs `RECONCILE_CALLER_REF` and `RECONCILE_DEFAULT_BRANCH`. + - Diagnostic `reconciliation error: publication runs only from refs/heads/`. + +- [ ] **Step 1: Give the reconcile harness a default-branch caller** + +In `reconcile-test.sh` `run_publish`, add two lines after `RECONCILE_RUN_URL=...`: + +```bash + RECONCILE_CALLER_REF=${RECONCILE_CALLER_REF-refs/heads/main} \ + RECONCILE_DEFAULT_BRANCH=main \ +``` + +- [ ] **Step 2: Write the failing reconcile test** + +Add before the `if [[ $# -eq 0 ]]` block in `reconcile-test.sh`: + +```bash +test_publication_runs_only_from_the_default_branch() { + # Production break caught: a preview whose publish gate is lost, or any other run from a + # non-default ref, pushes update refs or changes pull requests and issues from unmerged config. + local caller_ref dry_run remote + for caller_ref in refs/pull/7/merge refs/heads/feature; do + for dry_run in true false; do + setup_success_fixture + remote=$FIXTURE_REMOTE + if RECONCILE_CALLER_REF=$caller_ref RECONCILE_DRY_RUN=$dry_run run_publish \ + >"$FIXTURE_ROOT/stdout" 2>"$FIXTURE_ROOT/stderr"; then + fail "publication from $caller_ref (dry run $dry_run) was accepted" + fi + [[ "$(<"$FIXTURE_ROOT/stderr")" == 'reconciliation error: publication runs only from refs/heads/main' ]] \ + || fail "publication from $caller_ref reported: $(<"$FIXTURE_ROOT/stderr")" + [[ ! -s "$FIXTURE_ROOT/stdout" ]] || fail 'refused publication emitted stdout' + [[ ! -f "$FIXTURE_GH_CAPTURE/calls" ]] || fail "publication from $caller_ref called GitHub" + [[ -z "$("$TEST_GIT" --git-dir "$remote" for-each-ref --format='%(refname)' refs/heads/update_)" ]] \ + || fail "publication from $caller_ref pushed an update ref" + done + done +} +``` + +Append `test_publication_runs_only_from_the_default_branch` to the `tests=(...)` list at the bottom of `reconcile-test.sh`. + +- [ ] **Step 3: Run it and watch it fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/reconcile-test.sh test_publication_runs_only_from_the_default_branch` + +Expected: `FAIL: publication from refs/pull/7/merge (dry run true) was accepted`. + +- [ ] **Step 4: Implement the guard** + +In `reconcile-state-branch.sh`, add after `require_common_identity()`: + +```bash +# Publication must not depend on the publish job's `if` alone: a pull-request run executes the +# workflows it changes, so this refuses every ref but the default branch, dry run or not. +require_default_branch_caller() { + : "${RECONCILE_CALLER_REF:?RECONCILE_CALLER_REF must be set}" + : "${RECONCILE_DEFAULT_BRANCH:?RECONCILE_DEFAULT_BRANCH must be set}" + [[ "$RECONCILE_CALLER_REF" == "refs/heads/$RECONCILE_DEFAULT_BRANCH" ]] \ + || reconcile_error "publication runs only from refs/heads/$RECONCILE_DEFAULT_BRANCH" +} +``` + +Make it the first line of `publish_result()`, before `validate_result`: + +```bash +publish_result() { + require_default_branch_caller + validate_result +``` + +In `usage()`, change `PRs and failure issues. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP,` to: + +```text +PRs and failure issues, only when RECONCILE_CALLER_REF is refs/heads/ followed by +RECONCILE_DEFAULT_BRANCH. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP, +``` + +- [ ] **Step 5: Run the reconcile harness** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/reconcile-test.sh` + +Expected: every test prints `PASS:`, including the new one. + +- [ ] **Step 6: Wire the workflow and pin it with a failing check** + +In `test.sh` `test_workflow_runs_three_jobs_with_current_attempt_results`, add this clause before the closing `'` of its jq program (after the `RECONCILE_TF_VERSION_BUMP_VERSION` clause, joined with `and`): + +```text + and ([.publish.steps[] | select(.env.RECONCILE_RUN_URL != null) + | .env | {RECONCILE_CALLER_REF, RECONCILE_DEFAULT_BRANCH}] + == [{RECONCILE_CALLER_REF: "${{ github.ref }}", + RECONCILE_DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}"}]) +``` + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_runs_three_jobs_with_current_attempt_results` + +Expected: `FAIL: workflow does not wire the three-job current-attempt result contract`. + +In `tf-version-bump-reusable.yml`, in the `Publish processing result` step's `env`, add after `RECONCILE_RUN_URL`: + +```yaml + RECONCILE_CALLER_REF: ${{ github.ref }} + RECONCILE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} +``` + +Run the same test again. Expected: `PASS:`. + +- [ ] **Step 7: Lint and commit** + +Run: `make shellcheck` and `make actionlint`. Expected: no findings. + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/reconcile-state-branch.sh examples/github-actions/.github/workflows/tf-version-bump-reusable.yml examples/github-actions/reconcile-test.sh examples/github-actions/test.sh +``` + +Commit message: `feat(actions-example): refuse publication from any ref but the default branch`. + +--- + +### Task 4: Reusable workflow preview mode + +**Files:** +- Modify: `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml`: + - `workflow_call.inputs` + - `discover` step `env` + - `publish.if` + - `Report processing result` step +- Test: `examples/github-actions/test.sh`: + - `workflow_publishes_after_processing_failures` and `test_workflow_publishes_every_branch_after_processing_failures` + - `test_workflow_reports_the_processing_result` + - new `test_workflow_previews_without_publishing` + - new `run_preview_report_step` and `test_workflow_summarises_the_preview_patch` + - the `tests=(...)` list + +**Interfaces:** +- Consumes: `DISCOVERY_PREVIEW` (Task 1). +- Produces: + - Reusable input `preview` (boolean, default `false`), which callers pass in Task 5. + - `PREVIEW` env on `Report processing result`. + - Shell function `append_fenced_file ` inside that step. + +- [ ] **Step 1: Write the failing wiring and gate tests** + +In `test.sh`, change the condition in `workflow_publishes_after_processing_failures`: + +```bash + | jq -e --arg condition "\${{ always() && needs.discover.result == 'success' && !inputs.preview }}" ' +``` + +In `test_workflow_publishes_every_branch_after_processing_failures`, add this mutant after the `publish-condition.yml` block: + +```bash + mutant="$TEST_TMP_ROOT/publish-without-preview-gate.yml" + yq ".jobs.publish.if = \"\${{ always() && needs.discover.result == 'success' }}\"" \ + "$REUSABLE_WORKFLOW" >"$mutant" + ! workflow_publishes_after_processing_failures "$mutant" \ + || fail 'the guard passes when a preview can reach publication' +``` + +In `test_workflow_reports_the_processing_result`, extend the first jq check. Change `and .[0].env.RESULT_MANIFEST == $result + "/result.json"` to: + +```text + and .[0].env.RESULT_MANIFEST == $result + "/result.json" + and .[0].env.PREVIEW == "${{ inputs.preview }}" +``` + +Add a new test after `test_workflow_fails_discovery_when_the_script_fails`: + +```bash +test_workflow_previews_without_publishing() { + # Production break caught: callers cannot ask for a preview, or discovery never hears of it and + # processes every branch from a pull request. + yq -o=json '.' "$REUSABLE_WORKFLOW" | jq -e ' + .on.workflow_call.inputs.preview == {type: "boolean", default: false} + and ([.jobs.discover.steps[] | select(.id == "discover") | .env.DISCOVERY_PREVIEW] + == ["${{ inputs.preview }}"]) + ' >/dev/null || fail 'the reusable workflow does not offer a preview input wired into discovery' +} +``` + +Add `test_workflow_previews_without_publishing` to the `tests=(...)` list, after `test_workflow_fails_discovery_when_the_script_fails`. + +- [ ] **Step 2: Run them and watch them fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_publishes_every_branch_after_processing_failures test_workflow_reports_the_processing_result test_workflow_previews_without_publishing` + +Expected: `FAIL: a processing failure cancels other branches or skips their publication`, the gate string no longer matching. Run the other two individually and see them fail too. + +- [ ] **Step 3: Implement the input, discovery wiring and gate** + +In `tf-version-bump-reusable.yml`: +- Under `workflow_call.inputs`, after `dry_run`: + +```yaml + preview: + type: boolean + default: false +``` + +- In the `discover` step `env`, after `DISCOVERY_MANUAL_PREFIX`: + +```yaml + DISCOVERY_PREVIEW: ${{ inputs.preview }} +``` + +- Replace the `publish` job `if`: + +```yaml + if: ${{ always() && needs.discover.result == 'success' && !inputs.preview }} +``` + +- In the `Report processing result` step `env`, after `RESULT_MANIFEST`: + +```yaml + PREVIEW: ${{ inputs.preview }} +``` + +- [ ] **Step 4: Run the Step 2 tests and watch them pass** + +Expected: three `PASS:` lines. + +- [ ] **Step 5: Write the failing patch-summary test** + +Add after `test_workflow_summarises_update_logs` in `test.sh`: + +```bash +run_preview_report_step() { + run_workflow_step process 'Report processing result' "$3" \ + PROCESS_OUTCOME="$2" RESULT_MANIFEST="$1" PREVIEW="$4" +} + + +test_workflow_summarises_the_preview_patch() { + # Production break caught: a preview hides the candidate it produced, a live run's summary + # gains the patch, or backticks or size in the patch break the summary. + local work="$TEST_TMP_ROOT/report-patch" + rm -rf -- "$work" + mkdir -p "$work/logs" + local manifest="$work/result.json" summary="$work/summary.md" diagnostics="$work/stderr" + local report + + write_report_manifest "$manifest" success null '["."]' + printf '%s\n' '- version = "1.0.0"' '+ version = "2.0.0"' '```' >"$work/candidate.patch" + assert_silent_success 'summarising a preview patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + report=$(<"$summary") + [[ "$report" == *'#### Candidate patch'* && "$report" == *'+ version = "2.0.0"'* ]] \ + || fail "the preview summary omits the candidate patch: $report" + [[ $(grep -cx '````' "$summary") -eq 2 ]] \ + || fail "the patch's fence does not outlast the backticks inside it: $report" + + assert_silent_success 'summarising a live result' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" false + [[ "$(<"$summary")" != *'Candidate patch'* ]] \ + || fail 'a live run summary shows the candidate patch' + + awk 'BEGIN { for (i = 1; i <= 1200; i++) printf "+line %04d %060d\n", i, 0 }' \ + >"$work/candidate.patch" + assert_silent_success 'summarising an oversized preview patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + [[ "$(<"$summary")" == *$'\n```\n\nThis patch was truncated'* ]] \ + || fail "an oversized patch was not truncated with its closing fence on its own line: $(<"$summary")" + + rm -f -- "$work/candidate.patch" + write_report_manifest "$manifest" no-change + assert_silent_success 'summarising a preview without a patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + [[ "$(<"$summary")" != *'Candidate patch'* ]] \ + || fail 'a preview without a candidate patch claims one' +} +``` + +Add `test_workflow_summarises_the_preview_patch` to the `tests=(...)` list, after `test_workflow_summarises_update_logs`. + +- [ ] **Step 6: Run it and watch it fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_summarises_the_preview_patch` + +Expected: `FAIL: the preview summary omits the candidate patch`. + +- [ ] **Step 7: Implement the shared renderer and the patch** + +In the `Report processing result` step's `run:` body, replace everything from `# Only the updater's log is shown: ...` through `done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST")` with: + +```bash + # Appends a file in a fence one backtick longer than any run inside it, truncating past + # 64 KiB at a line boundary so the closing fence stays on a line of its own. + append_fenced_file() { + local heading=$1 file=$2 noun=$3 fence='```' longest + longest=$({ grep -aoE '`+' "$file" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }') + while [[ ${#fence} -le $longest ]]; do fence+='`'; done + printf '\n#### %s\n\n%s\n' "$heading" "$fence" >>"$GITHUB_STEP_SUMMARY" + if [[ $(wc -c <"$file") -gt 65536 ]]; then + head -c 65536 "$file" | sed '$d' >>"$GITHUB_STEP_SUMMARY" + printf '%s\n\nThis %s was truncated; the processing artefact holds it in full.\n' "$fence" "$noun" >>"$GITHUB_STEP_SUMMARY" + else + awk 1 "$file" >>"$GITHUB_STEP_SUMMARY" + printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY" + fi + } + # Only the updater's log is shown: Terraform's logs can carry a credential a provider echoed. + index=0 + while IFS= read -r root; do + index=$((index + 1)) + log="${RESULT_MANIFEST%/*}/logs/update-$index.log" + [[ -f "$log" ]] || continue + append_fenced_file "Updates in \`$root\`" "$log" log + done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST") + # The patch holds only Terraform files and root lock files, never Terraform's logs. + patch="${RESULT_MANIFEST%/*}/candidate.patch" + if [[ "$PREVIEW" == true && -f "$patch" ]]; then + append_fenced_file 'Candidate patch' "$patch" patch + fi +``` + +`PREVIEW` is unset when the harness's existing `run_report_step` runs the body. The step runs under `bash -e` without `-u`, so `"$PREVIEW" == true` is simply false there. + +- [ ] **Step 8: Run the summary tests** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_summarises_the_preview_patch test_workflow_summarises_update_logs test_workflow_reports_the_processing_result` + +Expected: three `PASS:` lines. `test_workflow_summarises_update_logs` still passes unchanged, which proves the refactor kept the log rendering byte-for-byte, including `This log was truncated`. + +- [ ] **Step 9: Lint and commit** + +Run: `make actionlint` and `make shellcheck`. actionlint also runs shellcheck over the `run:` body. Expected: no findings. + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/workflows/tf-version-bump-reusable.yml examples/github-actions/test.sh +``` + +Commit message: `feat(actions-example): add a preview mode to the reusable workflow`. + +--- + +### Task 5: Callers preview pull requests + +**Files:** +- Modify: `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` +- Modify: `examples/github-actions/.github/workflows/tf-version-bump-production.yml` +- Test: `examples/github-actions/test.sh` (new `test_callers_preview_pull_requests`, `tests=(...)` list) + +**Interfaces:** +- Consumes: reusable input `preview` (Task 4). +- Produces: caller jobs `automation` and `preview`, and YAML anchors `policy-inputs` and `policy-secrets`. `report-test.sh` and `test_workflow_offers_input_and_secret_terraform_environment` keep reading `.jobs.automation.with` and `.jobs.automation.secrets`, which stay in place. + +- [ ] **Step 1: Write the failing test** + +Add after `test_workflow_previews_without_publishing` in `test.sh`: + +```bash +test_callers_preview_pull_requests() { + # Production break caught: a preview runs with inputs that differ from its live run, queues + # behind or blocks live runs, previews a fork or a non-default base, narrows its permissions so + # GitHub refuses to start it, or a live run starts from a pull request. + local policy workflow + for policy in nonproduction production; do + workflow="$SCRIPT_DIR/.github/workflows/tf-version-bump-$policy.yml" + yq -o=json '.' "$workflow" | jq -e --arg policy "$policy" ' + .jobs as $jobs + | (has("concurrency") | not) + and .on.pull_request == {paths: [$jobs.automation.with.config_path, + ".github/workflows/tf-version-bump-\($policy).yml"]} + and ($jobs | keys) == ["automation", "preview"] + and $jobs.automation.if == "${{ github.ref == format('"'"'refs/heads/{0}'"'"', github.event.repository.default_branch) }}" + and $jobs.automation.concurrency == {group: "tf-version-bump-\($policy)-${{ github.repository_id }}", + "cancel-in-progress": false, queue: "max"} + and $jobs.preview.if == "${{ github.event_name == '"'"'pull_request'"'"' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }}" + and $jobs.preview.concurrency == {group: "tf-version-bump-\($policy)-preview-${{ github.repository_id }}-${{ github.event.number }}", + "cancel-in-progress": true} + and ($jobs.preview | has("permissions") | not) + and $jobs.preview.uses == $jobs.automation.uses + and $jobs.preview.with == $jobs.automation.with + and $jobs.preview.secrets == $jobs.automation.secrets + and $jobs.automation.with.preview == "${{ github.event_name == '"'"'pull_request'"'"' }}" + ' >/dev/null || fail "the $policy caller does not preview pull requests with its live inputs" + # The aliases keep one copy of each policy value, so a pull request's edits reach its preview. + grep -qxF ' with: *policy-inputs' "$workflow" \ + && grep -qxF ' secrets: *policy-secrets' "$workflow" \ + || fail "the $policy caller's preview does not reuse the live job's inputs by alias" + done +} +``` + +Add `test_callers_preview_pull_requests` to the `tests=(...)` list, after `test_workflow_previews_without_publishing`. + +- [ ] **Step 2: Run it and watch it fail** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_callers_preview_pull_requests` + +Expected: `FAIL: the nonproduction caller does not preview pull requests with its live inputs`. + +- [ ] **Step 3: Rewrite the non-production caller** + +Replace `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` with the following. Everything under `with:` except the new `preview:` line is copied unchanged from the current file: + +```yaml +name: Terraform version bump (non-production) + +on: + push: + paths: + - .github/tf-version-bump/nonproduction.yml + pull_request: + paths: + - .github/tf-version-bump/nonproduction.yml + - .github/workflows/tf-version-bump-nonproduction.yml + workflow_dispatch: + inputs: + branch_prefix: + description: Optional literal prefix that narrows the configured branch policy + type: string + default: "" + dry_run: + description: Run all local stages without publishing Git or GitHub changes + type: boolean + default: false + terraform_init_upgrade: + description: Upgrade dependencies within their version constraints during preparation + type: boolean + default: false + schedule: + - cron: "17 4 * * 1" + timezone: Australia/Melbourne + +permissions: + contents: write + pull-requests: write + issues: write + +jobs: + automation: + if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} + concurrency: + group: tf-version-bump-nonproduction-${{ github.repository_id }} + cancel-in-progress: false + queue: max + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: &policy-secrets + TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} + TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} + with: &policy-inputs + automation_policy_id: nonproduction + allowed_branch_prefixes: | + state/nonproduction/ + state/staging/ + aws-state/nonproduction/ + aws-state/staging/ + branch_prefix: ${{ github.event_name == 'workflow_dispatch' && inputs.branch_prefix || '' }} + config_path: .github/tf-version-bump/nonproduction.yml + terraform_directories: . + terraform_fmt: true + terraform_init_upgrade: ${{ github.event_name == 'workflow_dispatch' && inputs.terraform_init_upgrade || false }} + terraform_version: 1.15.5 + tf_version_bump_version: v1.0.0-rc.17 + tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d + dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} + preview: ${{ github.event_name == 'pull_request' }} + max_parallel: 4 + commit_author_name: github-actions[bot] + commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com + # Previews a same-repository pull request into the default branch on one sampled branch per + # prefix, without publishing. The aliases give it the live job's inputs, including any the pull + # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects + # both settings in one concurrency block. It keeps the workflow's permissions because GitHub + # refuses to start a read-only caller of a workflow whose publish job requests write. + preview: + if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} + concurrency: + group: tf-version-bump-nonproduction-preview-${{ github.repository_id }}-${{ github.event.number }} + cancel-in-progress: true + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: *policy-secrets + with: *policy-inputs +``` + +- [ ] **Step 4: Rewrite the production caller** + +Replace `examples/github-actions/.github/workflows/tf-version-bump-production.yml` with: + +```yaml +name: Terraform version bump (production) + +on: + push: + paths: + - .github/tf-version-bump/production.yml + pull_request: + paths: + - .github/tf-version-bump/production.yml + - .github/workflows/tf-version-bump-production.yml + workflow_dispatch: + inputs: + branch_prefix: + description: Optional literal prefix that narrows the configured branch policy + type: string + default: "" + dry_run: + description: Run all local stages without publishing Git or GitHub changes + type: boolean + default: false + terraform_init_upgrade: + description: Upgrade dependencies within their version constraints during preparation + type: boolean + default: false + schedule: + - cron: "43 4 * * 0" + timezone: Australia/Melbourne + +permissions: + contents: write + pull-requests: write + issues: write + +jobs: + automation: + if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} + concurrency: + group: tf-version-bump-production-${{ github.repository_id }} + cancel-in-progress: false + queue: max + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: &policy-secrets + TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} + TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} + with: &policy-inputs + automation_policy_id: production + allowed_branch_prefixes: | + state/production/ + aws-state/production/ + branch_prefix: ${{ github.event_name == 'workflow_dispatch' && inputs.branch_prefix || '' }} + config_path: .github/tf-version-bump/production.yml + terraform_directories: . + terraform_fmt: true + terraform_init_upgrade: ${{ github.event_name == 'workflow_dispatch' && inputs.terraform_init_upgrade || false }} + terraform_version: 1.15.5 + tf_version_bump_version: v1.0.0-rc.17 + tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d + dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} + preview: ${{ github.event_name == 'pull_request' }} + max_parallel: 2 + commit_author_name: github-actions[bot] + commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com + # Previews a same-repository pull request into the default branch on one sampled branch per + # prefix, without publishing. The aliases give it the live job's inputs, including any the pull + # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects + # both settings in one concurrency block. It keeps the workflow's permissions because GitHub + # refuses to start a read-only caller of a workflow whose publish job requests write. + preview: + if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} + concurrency: + group: tf-version-bump-production-preview-${{ github.repository_id }}-${{ github.event.number }} + cancel-in-progress: true + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: *policy-secrets + with: *policy-inputs +``` + +Before replacing each file, diff its current `with:` block against the block above (`git diff` after the edit). Only the `preview:` line may differ. + +- [ ] **Step 5: Run the caller tests and everything that reads the callers** + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_callers_preview_pull_requests test_workflow_offers_input_and_secret_terraform_environment` + +Expected: two `PASS:` lines. + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/report-test.sh`. Expected: all `PASS:`. Its report matrix still matches `.jobs.automation.with`. + +Run: `go test -C /Users/dan/Code/tf-version-bump -run 'TestUpdateActionsReleasePin' -v ./...`. Expected: `ok` with every `TestUpdateActionsReleasePin*` test passing. These are in `release_workflow_test.go`, and together they show the release-pin updater still finds exactly one pin line per caller. + +- [ ] **Step 6: Lint and commit** + +Run: `make actionlint`. Expected: no findings. The launcher already ignores actionlint 1.7.12's stale `queue` diagnostic. The spec's probe confirmed the job-level form passes. + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml examples/github-actions/.github/workflows/tf-version-bump-production.yml examples/github-actions/test.sh +``` + +Commit message: `feat(actions-example): preview config changes on pull requests`. + +--- + +### Task 6: Documentation + +**Files:** +- Modify: `examples/github-actions/README.md`: + - line 35 (Install) + - line 50 (secret) + - line 78 (Environments) + - line 89 (Configure updates) + - line 167 (Run and inspect) + - a new section before `## Version report` +- Modify: `docs/ADVANCED-USAGE.md:9` + +**Interfaces:** +- Consumes: the behaviour from Tasks 1–5. +- Produces: the anchor `#preview-on-pull-requests`. + +- [ ] **Step 1: Edit the existing passages** + +In `examples/github-actions/README.md`: +- **Line 35.** Replace `Both callers run only from the default branch.` with: + + ```markdown + Both callers' live runs start only from the default branch; each caller also previews same-repository pull requests into the default branch (see [Preview on pull requests](#preview-on-pull-requests)). + ``` +- **Line 50.** Append this sentence: `Pull-request previews run the same processing job, so they receive the same token.` +- **Line 78.** Append this sentence: + + ```markdown + An Environment's deployment-branch rules also apply to previews, which run from `refs/pull//merge`, so a rule that admits only the default branch may withhold the secret from a preview or hold it for approval. + ``` +- **Line 89.** Replace `Pull requests changing these files run a read-only config validation check; they do not process state branches or run Terraform.` with: + + ```markdown + Pull requests changing these files run a read-only config validation check and, for same-repository pull requests into the default branch, a preview that processes one sampled state branch per prefix without publishing anything (see [Preview on pull requests](#preview-on-pull-requests)). + ``` +- **Line 167.** After the sentence ending `kept in full in the artefact.`, insert: `In a preview, the summary also shows each changed branch's candidate patch, truncated the same way.` + +- [ ] **Step 2: Add the new section** + +Insert before `## Version report`: + +```markdown +## Preview on pull requests + +A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests from forks and pull requests into another branch skip the preview. + +The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. It also runs the pull request's own scripts and reusable workflow, so a pull request that edits those is previewed with the edited versions. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. + +The branch for each prefix is picked from the pull request number, so every push to the same pull request previews the same branches while that prefix's branches are unchanged. Adding, deleting or excluding a branch under a prefix may move the pick, and different pull requests may land on the same branch. A branch belongs to the first prefix it matches, and a prefix that owns no branch is named in a warning annotation. The preview is a smoke test, not coverage: a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled. + +Each `process` job's summary shows the updater's log and, for a changed candidate, the candidate patch. The patch is the full candidate the merged configuration would produce, not only what the pull request changes: on a branch whose update pull request has not been merged, it also includes the changes the current configuration would already make. + +A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, and with it the pull request's check. The failure may already exist on that branch, so check the branch's marked failure issue before blaming the pull request, and do not make the preview a required check. An invalid configuration fails every sampled branch at the update stage as `branch-update`, while the configuration validation check reports the cause. + +GitHub runs pull-request workflows only for the `opened`, `synchronize` and `reopened` activities, and not at all while the pull request has a merge conflict. A preview is therefore not refreshed when the default branch or the state branches move; push to the pull request or re-run the workflow to refresh it. +``` + +- [ ] **Step 3: Edit `docs/ADVANCED-USAGE.md` line 9** + +Replace `callers that run only from the default branch, plus config-change triggers, read-only pull-request config validation,` with `callers whose live runs start only from the default branch, plus config-change triggers, read-only pull-request config validation, pull-request previews on one sampled state branch per prefix,`. + +Replace this sentence: + +```markdown +The pull-request check validates only the control config with `tf-version-bump`; full state-branch dry runs remain deferred. +``` + +with: + +```markdown +The pull-request config check validates only the control config with `tf-version-bump`; the preview processes a sample of state branches and publishes nothing. +``` + +- [ ] **Step 4: Run the documentation checks** + +Run: `make -C /Users/dan/Code/tf-version-bump docs-check` + +Expected: `ok` with every test passing. `TestDocumentationLocalLinksResolve` resolves `#preview-on-pull-requests`, and `TestDocumentationProseIsNotHardWrapped` passes. + +Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_readme_documents_the_reserved_environment_names`. Expected: `PASS:`. + +- [ ] **Step 5: Commit** + +```bash +git -C /Users/dan/Code/tf-version-bump add examples/github-actions/README.md docs/ADVANCED-USAGE.md +``` + +Commit message: `docs(actions-example): document pull-request previews`. + +--- + +### Task 7: Full validation and test cleanup + +**Files:** none new. This task verifies the branch and prunes tests. + +- [ ] **Step 1: Run the full example harness (needs Docker)** + +Run: `make -C /Users/dan/Code/tf-version-bump test-github-actions TEST_GIT=/Users/dan/.claude/bin/claude-git` + +Expected: every test prints `PASS:`, then the reconcile and report harnesses print theirs, and nothing else. + +- [ ] **Step 2: Run the repository's CI-equivalent validation** + +Run each and expect success with no warnings: + +```bash +go -C /Users/dan/Code/tf-version-bump mod verify +go test -C /Users/dan/Code/tf-version-bump -race -coverprofile=coverage.out -covermode=atomic ./... +golangci-lint run --timeout=5m +make -C /Users/dan/Code/tf-version-bump actionlint +make -C /Users/dan/Code/tf-version-bump shellcheck +make -C /Users/dan/Code/tf-version-bump docs-check +``` + +Run `golangci-lint` from the repository root. Remove `coverage.out` afterwards with `make -C /Users/dan/Code/tf-version-bump clean`. + +- [ ] **Step 3: Run the test-cleanup pass as a separate subagent** + +Dispatch a fresh subagent with the `test-cleanup` skill against `git diff main...HEAD -- examples/github-actions/test.sh examples/github-actions/reconcile-test.sh`. The implementer must not do this pass. Apply only removals it justifies, re-run Step 1, and commit with message `test(actions-example): trim preview tests`. + +- [ ] **Step 4: Report** + +Tell Dan: +- which checks passed; +- anything that failed, with its output; +- that the branch `wip/config-change-preview` is ready for review. + +Do not push without his confirmation. diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md index dc29734..0640902 100644 --- a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md +++ b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md @@ -33,7 +33,7 @@ Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml` - **Triggers.** Adds `pull_request: paths:` naming its own configuration file and its own workflow file. The `push`, `workflow_dispatch` and `schedule` triggers are unchanged. - **Workflow-level concurrency.** Removed. It moves to the live job. - **Two jobs.** Both call the reusable workflow, with the same `with:` and `secrets:` blocks. The live job defines them as anchors (`with: &policy-inputs`, `secrets: &policy-secrets`) and the preview job uses the aliases (`with: *policy-inputs`, `secrets: *policy-secrets`). Every policy value is therefore written once, and a pull request that edits a value, such as a tool pin, a prefix or a root, is previewed with it. - - `automation`, the live job: `if: github.ref == refs/heads/ && github.event_name != 'pull_request'`, with job-level `concurrency: {group: tf-version-bump--, cancel-in-progress: false, queue: max}`. Its behaviour is otherwise unchanged. + - `automation`, the live job: its existing `if` (`github.ref` is `refs/heads/`) is unchanged, because a pull-request run's ref is `refs/pull//merge` and never matches; it gains job-level `concurrency: {group: tf-version-bump--, cancel-in-progress: false, queue: max}`. Its behaviour is otherwise unchanged. - `preview`: `if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch`, with job-level `concurrency: {group: tf-version-bump--preview--, cancel-in-progress: true}`. It declares no job-level `permissions`, because a read-only caller job fails at startup (see [Verified GitHub Actions behaviour](#verified-github-actions-behaviour)). - **The `preview` input.** The shared `with:` block includes `preview: ${{ github.event_name == 'pull_request' }}`, which is `false` for the live job and `true` for the preview job without either block differing. `branch_prefix`, `dry_run` and `terraform_init_upgrade` keep their existing event-guarded expressions, which evaluate to `''`, `false` and `false` on pull-request events. @@ -45,7 +45,7 @@ A newer push to a pull request cancels that pull request's older preview, and a - **Discovery.** The discovery step passes `DISCOVERY_PREVIEW: ${{ inputs.preview }}`. - **Publication.** The `publish` job's `if` becomes `${{ always() && needs.discover.result == 'success' && !inputs.preview }}`. - **Summary.** The `Report processing result` step receives `PREVIEW: ${{ inputs.preview }}` through `env`, because the local harness runs step bodies without evaluating expressions. When `PREVIEW` is `true` and `candidate.patch` exists, the step appends it to the job summary under `#### Candidate patch`. The patch exists only for a changed, validated candidate, so there is none for failures or no-change results. -- **Shared rendering.** The existing fence-escalation and 64 KiB truncation code becomes one shell function in the step, `append_fenced_file `, used for each updater log and for the patch. The patch holds only `.tf` changes, formatting changes and root lock files (`write_candidate`, `process-state-branch.sh`), never Terraform's logs, so it cannot carry a credential Terraform echoed. +- **Shared rendering.** The existing fence-escalation and 64 KiB truncation code becomes one shell function in the step, `append_fenced_file `, used for each updater log (`log`) and for the patch (`patch`); the noun names the file in the truncation notice. The patch holds only `.tf` changes, formatting changes and root lock files (`write_candidate`, `process-state-branch.sh`), never Terraform's logs, so it cannot carry a credential Terraform echoed. ### Discovery: sampling From 3b9bbd1a809cac6b85de6945b55dbed2b03a89a8 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 13:45:44 +1000 Subject: [PATCH 05/14] feat(actions-example): accept pull-request refs in preview discovery Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../scripts/discover-state-branches.sh | 22 ++++++++-- examples/github-actions/test.sh | 42 +++++++++++++++++++ 2 files changed, 61 insertions(+), 3 deletions(-) diff --git a/examples/github-actions/.github/scripts/discover-state-branches.sh b/examples/github-actions/.github/scripts/discover-state-branches.sh index f312acd..20a602a 100755 --- a/examples/github-actions/.github/scripts/discover-state-branches.sh +++ b/examples/github-actions/.github/scripts/discover-state-branches.sh @@ -9,6 +9,8 @@ usage() { Usage: discover-state-branches.sh --help Discover immutable inputs for configured Terraform state branches. +With DISCOVERY_PREVIEW=true, run from refs/pull//merge and select one +sampled branch per prefix instead of every matching branch. EOF } @@ -80,9 +82,23 @@ branch_is_excluded() { : "${DISCOVERY_ALLOWED_PREFIXES?DISCOVERY_ALLOWED_PREFIXES must be set}" : "${DISCOVERY_MANUAL_PREFIX?DISCOVERY_MANUAL_PREFIX must be set}" -expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" -[[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ - || fail_discovery caller "caller ref must be $expected_caller_ref" +preview=${DISCOVERY_PREVIEW-false} +[[ "$preview" == true || "$preview" == false ]] \ + || fail_discovery input "DISCOVERY_PREVIEW must be true or false" +if [[ "$preview" == true ]]; then + # A preview runs from a pull request's merge ref. Its number seeds the sample, so the seed + # cannot disagree with the pull request, and a manual prefix only arises from a dispatch. + [[ "$DISCOVERY_CALLER_REF" =~ ^refs/pull/([1-9][0-9]*)/merge$ ]] \ + || fail_discovery caller "preview caller ref must be refs/pull//merge" + # shellcheck disable=SC2034 # Consumed by the sampling this preview adds in a later task. + preview_seed=${BASH_REMATCH[1]} + [[ -z "$DISCOVERY_MANUAL_PREFIX" ]] \ + || fail_discovery input "a preview cannot take a manual prefix" +else + expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" + [[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ + || fail_discovery caller "caller ref must be $expected_caller_ref" +fi # shellcheck disable=SC2153 # Set by the reusable workflow or the focused harness. [[ "$CONTROL_CHECKOUT" == /* && -d "$CONTROL_CHECKOUT" ]] \ || fail_discovery input "CONTROL_CHECKOUT must be an absolute existing directory" diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index 522a4e9..d547e62 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -361,6 +361,7 @@ assert_processing_failure() { setup_discovery_repository() { cleanup_discovery_repository + unset DISCOVERY_PREVIEW DISCOVERY_ALLOWED_PREFIXES="" DISCOVERY_MANUAL_PREFIX="" DISCOVERY_POLICY_ID="nonproduction" @@ -414,6 +415,7 @@ add_numbered_discovery_branches() { run_discovery() { ( cd "$DISCOVERY_REPO" + [[ -z "${DISCOVERY_PREVIEW+set}" ]] || export DISCOVERY_PREVIEW DISCOVERY_ALLOWED_PREFIXES=${DISCOVERY_ALLOWED_PREFIXES-} \ DISCOVERY_MANUAL_PREFIX=${DISCOVERY_MANUAL_PREFIX-} \ DISCOVERY_POLICY_ID=${DISCOVERY_POLICY_ID-nonproduction} \ @@ -725,6 +727,46 @@ test_discovery_rejects_invalid_inputs_by_stage() { } +test_discovery_preview_accepts_only_pull_request_merge_refs() { + # Production break caught: relaxing the default-branch guard for previews lets any ref drive + # discovery, a pull-request ref passes without asking for a preview, or a preview's seed can + # disagree with the pull request it runs for. + setup_discovery_repository + add_discovery_branch "state/prod/example" + DISCOVERY_ALLOWED_PREFIXES="state/prod/" + + DISCOVERY_CALLER_REF="refs/pull/42/merge" + assert_discovery_failure "discovery caller error: caller ref must be refs/heads/main" \ + "a pull-request ref without a preview" + + DISCOVERY_PREVIEW=true + local output + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/prod/example"]' <<<"$output" >/dev/null \ + || fail "preview discovery refused a pull-request merge ref: $output" + + local ref + for ref in refs/heads/main refs/pull/0/merge refs/pull/042/merge refs/pull/7/head refs/pull/x/merge; do + DISCOVERY_CALLER_REF=$ref + assert_discovery_failure "discovery caller error: preview caller ref must be refs/pull//merge" \ + "a preview from $ref" + done + + DISCOVERY_CALLER_REF="refs/pull/42/merge" + DISCOVERY_MANUAL_PREFIX="state/prod/" + assert_discovery_failure "discovery input error: a preview cannot take a manual prefix" \ + "a preview with a manual prefix" + + DISCOVERY_MANUAL_PREFIX="" + local value + for value in "" TRUE yes; do + DISCOVERY_PREVIEW=$value + assert_discovery_failure "discovery input error: DISCOVERY_PREVIEW must be true or false" \ + "a preview flag of '$value'" + done +} + + test_discovery_treats_hostile_refs_as_inert_data() { # Production break caught: shell evaluation, word splitting, option parsing, locale filtering, # or JSON interpolation executes or corrupts a Git-valid remote branch name. From d148bd724b5ec1c73e152c3cb6a14e2ab8546111 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 13:50:49 +1000 Subject: [PATCH 06/14] feat(actions-example): sample one state branch per prefix in preview discovery Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../scripts/discover-state-branches.sh | 29 +++++++- examples/github-actions/test.sh | 67 +++++++++++++++++++ 2 files changed, 95 insertions(+), 1 deletion(-) diff --git a/examples/github-actions/.github/scripts/discover-state-branches.sh b/examples/github-actions/.github/scripts/discover-state-branches.sh index 20a602a..bdf4110 100755 --- a/examples/github-actions/.github/scripts/discover-state-branches.sh +++ b/examples/github-actions/.github/scripts/discover-state-branches.sh @@ -71,6 +71,31 @@ branch_is_excluded() { return 1 } +# Keeps one branch per prefix, so a preview processes a sample rather than the whole policy. A +# branch belongs to the first prefix it matches; the pick is the first 15 hex digits of +# sha256("\t") modulo the prefix's branch count, which stays within bash's signed +# 64-bit arithmetic and is stable while that prefix's branches are unchanged. +sample_branch_records() { + local prefix record digest + local -a owned sampled=() + declare -A seen_prefixes=() + for prefix in "${selection_prefixes[@]}"; do + [[ -z "${seen_prefixes[$prefix]-}" ]] || continue + seen_prefixes[$prefix]=1 + owned=() + for record in "${branch_records[@]}"; do + [[ "${branch_owners[${record%%$'\t'*}]}" != "$prefix" ]] || owned+=("$record") + done + if [[ ${#owned[@]} -eq 0 ]]; then + echo "::warning::branch prefix '${prefix//'%'/%25}' owns no branch to preview" >&2 + continue + fi + digest=$(printf '%s\t%s' "$preview_seed" "$prefix" | sha256sum) + sampled+=("${owned[$((16#${digest:0:15} % ${#owned[@]}))]}") + done + readarray -t branch_records < <(printf '%s\n' "${sampled[@]}" | sort) +} + : "${DISCOVERY_DEFAULT_BRANCH:?DISCOVERY_DEFAULT_BRANCH must be set}" : "${DISCOVERY_CALLER_REF:?DISCOVERY_CALLER_REF must be set}" : "${CONTROL_CHECKOUT:?CONTROL_CHECKOUT must be set}" @@ -90,7 +115,6 @@ if [[ "$preview" == true ]]; then # cannot disagree with the pull request, and a manual prefix only arises from a dispatch. [[ "$DISCOVERY_CALLER_REF" =~ ^refs/pull/([1-9][0-9]*)/merge$ ]] \ || fail_discovery caller "preview caller ref must be refs/pull//merge" - # shellcheck disable=SC2034 # Consumed by the sampling this preview adds in a later task. preview_seed=${BASH_REMATCH[1]} [[ -z "$DISCOVERY_MANUAL_PREFIX" ]] \ || fail_discovery input "a preview cannot take a manual prefix" @@ -157,6 +181,7 @@ if ! git -C "$control_checkout" ls-remote --heads --refs origin >"$remote_heads_ fi branch_records=() +declare -A branch_owners=() declare -A matched_exclusions=() while IFS=$'\t' read -r oid ref; do branch=${ref#refs/heads/} @@ -167,6 +192,7 @@ while IFS=$'\t' read -r oid ref; do for prefix in "${selection_prefixes[@]}"; do if [[ "$branch" == "$prefix"* ]]; then branch_records+=("$branch"$'\t'"$oid") + branch_owners[$branch]=$prefix break fi done @@ -188,6 +214,7 @@ done || fail_discovery matrix "more than 256 branches matched; narrow or partition the prefix policy" readarray -t branch_records < <(printf '%s\n' "${branch_records[@]}" | sort) +[[ "$preview" != true ]] || sample_branch_records # Git ref names cannot contain a tab (or any control character), so a tab-separated record per # branch is a safe carrier into the single jq pass below that replaces one jq -cn per branch. tsv_records=() diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index d547e62..2bc6c5d 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -767,6 +767,73 @@ test_discovery_preview_accepts_only_pull_request_merge_refs() { } +test_discovery_preview_samples_one_branch_per_prefix() { + # Production break caught: a preview processes every branch, picks differently on each push, + # or samples a branch its caller excludes. + setup_discovery_repository + local name + for name in alpha bravo charlie; do + add_discovery_branch "state/staging/$name" + add_discovery_branch "state/production/$name" + done + add_discovery_branch "state/production/excluded" + DISCOVERY_ALLOWED_PREFIXES=$'state/staging/\nstate/production/\n!state/production/excluded' + DISCOVERY_PREVIEW=true + local output stderr_file="$DISCOVERY_TMP_ROOT/sample.stderr" + + # Known answers: the first 15 hex digits of sha256("\t"), modulo the three + # branches each prefix owns, index the sorted branches. Pull request 42 picks index 2 of + # state/production/ and 0 of state/staging/; pull request 43 picks index 1 of both. + DISCOVERY_CALLER_REF="refs/pull/42/merge" + output=$(run_discovery 2>"$stderr_file") + jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ + <<<"$output" >/dev/null || fail "pull request 42 did not sample its known branches: $output" + [[ ! -s "$stderr_file" ]] || fail "sampling reported a warning: $(<"$stderr_file")" + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ + <<<"$output" >/dev/null || fail "a repeat preview of pull request 42 sampled differently: $output" + + DISCOVERY_CALLER_REF="refs/pull/43/merge" + output=$(run_discovery) + jq -e '.include | map(.branch) == ["state/production/bravo", "state/staging/bravo"]' \ + <<<"$output" >/dev/null || fail "pull request 43 did not sample its known branches: $output" +} + + +test_discovery_preview_warns_for_prefixes_that_own_no_branch() { + # Production break caught: a prefix with nothing to preview passes silently, a narrower prefix + # listed after a broader one is reported as matching nothing without saying why, a repeated + # prefix previews its branch twice, or live runs gain the warning. + setup_discovery_repository + add_discovery_branch "state/staging/alpha" + DISCOVERY_ALLOWED_PREFIXES=$'state/\nstate/staging/\nstate/\naws-state/100%/' + local output stderr_file="$DISCOVERY_TMP_ROOT/unowned.stderr" + + output=$(run_discovery 2>"$stderr_file") + [[ ! -s "$stderr_file" ]] || fail "a live run reported unowned prefixes: $(<"$stderr_file")" + + DISCOVERY_PREVIEW=true + DISCOVERY_CALLER_REF="refs/pull/42/merge" + output=$(run_discovery 2>"$stderr_file") + jq -e '.include | map(.branch) == ["state/staging/alpha"]' <<<"$output" >/dev/null \ + || fail "a preview did not sample the one owned branch exactly once: $output" + [[ "$(<"$stderr_file")" == "::warning::branch prefix 'state/staging/' owns no branch to preview"$'\n'"::warning::branch prefix 'aws-state/100%25/' owns no branch to preview" ]] \ + || fail "a preview did not name each prefix that owns no branch once: $(<"$stderr_file")" +} + + +test_discovery_preview_keeps_the_live_matrix_limit() { + # Production break caught: a policy the live run refuses for exceeding the matrix limit + # previews green because the sample itself is small. + setup_discovery_repository + add_numbered_discovery_branches 0 256 + DISCOVERY_ALLOWED_PREFIXES="state/limit/" + DISCOVERY_PREVIEW=true + DISCOVERY_CALLER_REF="refs/pull/42/merge" + assert_discovery_failure "discovery matrix error:" "a preview of more than 256 branches" +} + + test_discovery_treats_hostile_refs_as_inert_data() { # Production break caught: shell evaluation, word splitting, option parsing, locale filtering, # or JSON interpolation executes or corrupts a Git-valid remote branch name. From fa81c80501b831c90bc0db9f0cd655d7b1dc628c Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 13:57:13 +1000 Subject: [PATCH 07/14] feat(actions-example): refuse publication from any ref but the default branch A preview run on a pull request executes the PR's own workflows, so publication must not rely on the publish job's `if` alone: a lost or tampered gate there would still push update refs and change pull requests and issues from unmerged config. reconcile-state-branch.sh publish now independently refuses any caller ref other than refs/heads/, dry run or not. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../.github/scripts/reconcile-state-branch.sh | 14 +++++++++- .../workflows/tf-version-bump-reusable.yml | 2 ++ examples/github-actions/reconcile-test.sh | 27 ++++++++++++++++++- examples/github-actions/test.sh | 6 ++++- 4 files changed, 46 insertions(+), 3 deletions(-) diff --git a/examples/github-actions/.github/scripts/reconcile-state-branch.sh b/examples/github-actions/.github/scripts/reconcile-state-branch.sh index a264cbb..9ab1a45 100755 --- a/examples/github-actions/.github/scripts/reconcile-state-branch.sh +++ b/examples/github-actions/.github/scripts/reconcile-state-branch.sh @@ -17,7 +17,8 @@ RECONCILE_STATE_BRANCH, RECONCILE_BASE_OID, RECONCILE_REF_HASH. publish additionally checks the clean exact-base RECONCILE_TARGET_CHECKOUT and RECONCILE_TERRAFORM_ROOTS, creates one owned update commit, and reconciles marked -PRs and failure issues. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP, +PRs and failure issues, only when RECONCILE_CALLER_REF is refs/heads/ followed by +RECONCILE_DEFAULT_BRANCH. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP, RECONCILE_RUN_URL, RECONCILE_GIT_REMOTE, RECONCILE_REPOSITORY, GH_TOKEN, RECONCILE_COMMIT_AUTHOR_NAME, RECONCILE_COMMIT_AUTHOR_EMAIL, and the RECONCILE_TERRAFORM_VERSION and RECONCILE_TF_VERSION_BUMP_VERSION their bodies name. @@ -235,6 +236,7 @@ reconcile_lifecycle() { } publish_result() { + require_default_branch_caller validate_result [[ "$CLASSIFICATION" != automation ]] || return 0 : "${RECONCILE_DRY_RUN:?RECONCILE_DRY_RUN must be set}" @@ -293,6 +295,16 @@ require_common_identity() { } +# Publication must not depend on the publish job's `if` alone: a pull-request run executes the +# workflows it changes, so this refuses every ref but the default branch, dry run or not. +require_default_branch_caller() { + : "${RECONCILE_CALLER_REF:?RECONCILE_CALLER_REF must be set}" + : "${RECONCILE_DEFAULT_BRANCH:?RECONCILE_DEFAULT_BRANCH must be set}" + [[ "$RECONCILE_CALLER_REF" == "refs/heads/$RECONCILE_DEFAULT_BRANCH" ]] \ + || reconcile_error "publication runs only from refs/heads/$RECONCILE_DEFAULT_BRANCH" +} + + github_marker() { printf '\n' \ "$RECONCILE_AUTOMATION_POLICY_ID" "$RECONCILE_REF_HASH" diff --git a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml index 4f2e6e3..038fa22 100644 --- a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml +++ b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml @@ -260,6 +260,8 @@ jobs: RECONCILE_TARGET_CHECKOUT: ${{ github.workspace }}/target RECONCILE_TERRAFORM_ROOTS: ${{ inputs.terraform_directories }} RECONCILE_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + RECONCILE_CALLER_REF: ${{ github.ref }} + RECONCILE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} RECONCILE_GIT_REMOTE: ${{ github.server_url }}/${{ github.repository }}.git RECONCILE_REPOSITORY: ${{ github.repository }} RECONCILE_DRY_RUN: ${{ inputs.dry_run }} diff --git a/examples/github-actions/reconcile-test.sh b/examples/github-actions/reconcile-test.sh index a546704..46b22b6 100755 --- a/examples/github-actions/reconcile-test.sh +++ b/examples/github-actions/reconcile-test.sh @@ -126,6 +126,8 @@ run_publish() { PATH="$FIXTURE_BIN:$PATH" \ GH_CAPTURE_DIR="$FIXTURE_GH_CAPTURE" \ RECONCILE_RUN_URL=https://github.com/yesdevnull/reconciliation-test/actions/runs/100 \ + RECONCILE_CALLER_REF=${RECONCILE_CALLER_REF-refs/heads/main} \ + RECONCILE_DEFAULT_BRANCH=main \ RECONCILE_RUN_ID=${RECONCILE_RUN_ID-100} \ RECONCILE_RUN_ATTEMPT=${RECONCILE_RUN_ATTEMPT-1} \ RECONCILE_AUTOMATION_POLICY_ID=nonproduction \ @@ -447,13 +449,36 @@ test_failure_issue_create_reopen_and_invalid_status() { done } +test_publication_runs_only_from_the_default_branch() { + # Production break caught: a preview whose publish gate is lost, or any other run from a + # non-default ref, pushes update refs or changes pull requests and issues from unmerged config. + local caller_ref dry_run remote + for caller_ref in refs/pull/7/merge refs/heads/feature; do + for dry_run in true false; do + setup_success_fixture + remote=$FIXTURE_REMOTE + if RECONCILE_CALLER_REF=$caller_ref RECONCILE_DRY_RUN=$dry_run run_publish \ + >"$FIXTURE_ROOT/stdout" 2>"$FIXTURE_ROOT/stderr"; then + fail "publication from $caller_ref (dry run $dry_run) was accepted" + fi + [[ "$(<"$FIXTURE_ROOT/stderr")" == 'reconciliation error: publication runs only from refs/heads/main' ]] \ + || fail "publication from $caller_ref reported: $(<"$FIXTURE_ROOT/stderr")" + [[ ! -s "$FIXTURE_ROOT/stdout" ]] || fail 'refused publication emitted stdout' + [[ ! -f "$FIXTURE_GH_CAPTURE/calls" ]] || fail "publication from $caller_ref called GitHub" + [[ -z "$("$TEST_GIT" --git-dir "$remote" for-each-ref --format='%(refname)' refs/heads/update_)" ]] \ + || fail "publication from $caller_ref pushed an update ref" + done + done +} + if [[ $# -eq 0 ]]; then tests=(test_publishes_one_owned_commit_from_exact_base test_result_validation_prevents_mutation test_rejects_unsafe_candidate_paths_and_modes test_publication_refuses_moved_base_and_foreign_update test_stale_results_do_not_reconcile_lifecycle test_reconciles_no_change_and_failure_in_order test_api_failures_stop_followup_actions test_dry_run_and_automation_do_not_mutate test_exact_lease_rejects_racing_update_ref - test_failure_issue_create_reopen_and_invalid_status) + test_failure_issue_create_reopen_and_invalid_status + test_publication_runs_only_from_the_default_branch) else tests=("$@"); fi for test_name in "${tests[@]}"; do "$test_name" diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index 2bc6c5d..0416a76 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -1204,7 +1204,11 @@ test_workflow_runs_three_jobs_with_current_attempt_results() { ([.publish.steps[] | select(.env.TF_TOKEN_app_terraform_io != null)] | length == 0) and ([.publish.steps[] | select(.env.RECONCILE_RUN_URL != null)] | length == 1) and ([.publish.steps[] | select(.env.RECONCILE_TERRAFORM_VERSION == "${{ inputs.terraform_version }}" - and .env.RECONCILE_TF_VERSION_BUMP_VERSION == "${{ inputs.tf_version_bump_version }}")] | length == 1) + and .env.RECONCILE_TF_VERSION_BUMP_VERSION == "${{ inputs.tf_version_bump_version }}")] | length == 1) and + ([.publish.steps[] | select(.env.RECONCILE_RUN_URL != null) + | .env | {RECONCILE_CALLER_REF, RECONCILE_DEFAULT_BRANCH}] + == [{RECONCILE_CALLER_REF: "${{ github.ref }}", + RECONCILE_DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}"}]) ' >/dev/null || fail 'workflow does not wire the three-job current-attempt result contract' } From b654468a06c8e32fd5662feff1a03e591430d610 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:04:09 +1000 Subject: [PATCH 08/14] feat(actions-example): add a preview mode to the reusable workflow Add a `preview` input to the reusable workflow, wire it into discovery's `DISCOVERY_PREVIEW`, gate the publish job behind `!inputs.preview` so a preview never reaches publication, and have the processing report step append the candidate patch to the run summary whenever a preview produced one. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../workflows/tf-version-bump-reusable.yml | 39 +++++++---- examples/github-actions/test.sh | 68 ++++++++++++++++++- 2 files changed, 93 insertions(+), 14 deletions(-) diff --git a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml index 038fa22..a3864b4 100644 --- a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml +++ b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml @@ -39,6 +39,9 @@ on: dry_run: type: boolean default: false + preview: + type: boolean + default: false max_parallel: type: number default: 4 @@ -74,6 +77,7 @@ jobs: CONTROL_CHECKOUT: ${{ github.workspace }}/control DISCOVERY_ALLOWED_PREFIXES: ${{ inputs.allowed_branch_prefixes }} DISCOVERY_MANUAL_PREFIX: ${{ inputs.branch_prefix }} + DISCOVERY_PREVIEW: ${{ inputs.preview }} DISCOVERY_POLICY_ID: ${{ inputs.automation_policy_id }} DISCOVERY_RUN_ID: ${{ github.run_id }} DISCOVERY_RUN_ATTEMPT: ${{ github.run_attempt }} @@ -166,6 +170,7 @@ jobs: env: PROCESS_OUTCOME: ${{ steps.process.outcome }} RESULT_MANIFEST: ${{ runner.temp }}/result/result.json + PREVIEW: ${{ inputs.preview }} run: | if [[ ! -f "$RESULT_MANIFEST" ]]; then echo "Processing wrote no result manifest; publication will not change managed refs, pull requests or issues." >&2 @@ -178,24 +183,34 @@ jobs: else "\n- Automation failed with status \(.failure.status); the `Process candidate` step log names the cause, and `logs/` in the processing artefact holds any command output" end)' \ "$RESULT_MANIFEST" >>"$GITHUB_STEP_SUMMARY" + # Appends a file in a fence one backtick longer than any run inside it, truncating past + # 64 KiB at a line boundary so the closing fence stays on a line of its own. + append_fenced_file() { + local heading=$1 file=$2 noun=$3 fence='```' longest + longest=$({ grep -aoE '`+' "$file" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }') + while [[ ${#fence} -le $longest ]]; do fence+='`'; done + printf '\n#### %s\n\n%s\n' "$heading" "$fence" >>"$GITHUB_STEP_SUMMARY" + if [[ $(wc -c <"$file") -gt 65536 ]]; then + head -c 65536 "$file" | sed '$d' >>"$GITHUB_STEP_SUMMARY" + printf '%s\n\nThis %s was truncated; the processing artefact holds it in full.\n' "$fence" "$noun" >>"$GITHUB_STEP_SUMMARY" + else + awk 1 "$file" >>"$GITHUB_STEP_SUMMARY" + printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY" + fi + } # Only the updater's log is shown: Terraform's logs can carry a credential a provider echoed. index=0 while IFS= read -r root; do index=$((index + 1)) log="${RESULT_MANIFEST%/*}/logs/update-$index.log" [[ -f "$log" ]] || continue - fence='```' - longest=$({ grep -aoE '`+' "$log" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }') - while [[ ${#fence} -le $longest ]]; do fence+='`'; done - printf '\n#### Updates in %s\n\n%s\n' "\`$root\`" "$fence" >>"$GITHUB_STEP_SUMMARY" - if [[ $(wc -c <"$log") -gt 65536 ]]; then - head -c 65536 "$log" | sed '$d' >>"$GITHUB_STEP_SUMMARY" - printf '%s\n\nThis log was truncated; the processing artefact holds it in full.\n' "$fence" >>"$GITHUB_STEP_SUMMARY" - else - awk 1 "$log" >>"$GITHUB_STEP_SUMMARY" - printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY" - fi + append_fenced_file "Updates in \`$root\`" "$log" log done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST") + # The patch holds only Terraform files and root lock files, never Terraform's logs. + patch="${RESULT_MANIFEST%/*}/candidate.patch" + if [[ "$PREVIEW" == true && -f "$patch" ]]; then + append_fenced_file 'Candidate patch' "$patch" patch + fi classification=$(jq -er '.classification' "$RESULT_MANIFEST") if [[ "$PROCESS_OUTCOME" == success \ && "$classification" != success && "$classification" != no-change ]]; then @@ -206,7 +221,7 @@ jobs: publish: name: publish (${{ matrix.branch }}) needs: [discover, process] - if: ${{ always() && needs.discover.result == 'success' }} + if: ${{ always() && needs.discover.result == 'success' && !inputs.preview }} strategy: fail-fast: false max-parallel: ${{ inputs.max_parallel }} diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index 0416a76..af5793a 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -1675,6 +1675,7 @@ test_workflow_reports_the_processing_result() { | length == 1 and .[0].if == "${{ always() }}" and .[0].env.PROCESS_OUTCOME == "${{ steps.process.outcome }}" and .[0].env.RESULT_MANIFEST == $result + "/result.json" + and .[0].env.PREVIEW == "${{ inputs.preview }}" ' >/dev/null || fail 'the report step is not wired to the processing step and its result' local work="$TEST_TMP_ROOT/report-step" rm -rf -- "$work" @@ -1784,6 +1785,52 @@ test_workflow_summarises_update_logs() { } +run_preview_report_step() { + run_workflow_step process 'Report processing result' "$3" \ + PROCESS_OUTCOME="$2" RESULT_MANIFEST="$1" PREVIEW="$4" +} + + +test_workflow_summarises_the_preview_patch() { + # Production break caught: a preview hides the candidate it produced, a live run's summary + # gains the patch, or backticks or size in the patch break the summary. + local work="$TEST_TMP_ROOT/report-patch" + rm -rf -- "$work" + mkdir -p "$work/logs" + local manifest="$work/result.json" summary="$work/summary.md" diagnostics="$work/stderr" + local report + + write_report_manifest "$manifest" success null '["."]' + printf '%s\n' '- version = "1.0.0"' '+ version = "2.0.0"' '```' >"$work/candidate.patch" + assert_silent_success 'summarising a preview patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + report=$(<"$summary") + [[ "$report" == *'#### Candidate patch'* && "$report" == *'+ version = "2.0.0"'* ]] \ + || fail "the preview summary omits the candidate patch: $report" + [[ $(grep -cx '````' "$summary") -eq 2 ]] \ + || fail "the patch's fence does not outlast the backticks inside it: $report" + + assert_silent_success 'summarising a live result' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" false + [[ "$(<"$summary")" != *'Candidate patch'* ]] \ + || fail 'a live run summary shows the candidate patch' + + awk 'BEGIN { for (i = 1; i <= 1200; i++) printf "+line %04d %060d\n", i, 0 }' \ + >"$work/candidate.patch" + assert_silent_success 'summarising an oversized preview patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + [[ "$(<"$summary")" == *$'\n```\n\nThis patch was truncated'* ]] \ + || fail "an oversized patch was not truncated with its closing fence on its own line: $(<"$summary")" + + rm -f -- "$work/candidate.patch" + write_report_manifest "$manifest" no-change + assert_silent_success 'summarising a preview without a patch' "$work/stdout" "$diagnostics" \ + run_preview_report_step "$manifest" success "$summary" true + [[ "$(<"$summary")" != *'Candidate patch'* ]] \ + || fail 'a preview without a candidate patch claims one' +} + + run_dry_run_report_step() { run_workflow_step publish 'Report dry-run outcome' "$2" RESULT_MANIFEST="$1" } @@ -1858,7 +1905,7 @@ test_workflow_fails_the_process_job_on_processing_failure() { workflow_publishes_after_processing_failures() { local workflow=$1 yq -o=json '.jobs' "$workflow" \ - | jq -e --arg condition "\${{ always() && needs.discover.result == 'success' }}" ' + | jq -e --arg condition "\${{ always() && needs.discover.result == 'success' && !inputs.preview }}" ' .process.strategy["fail-fast"] == false and .publish.strategy["fail-fast"] == false and .publish.if == $condition @@ -1881,6 +1928,11 @@ test_workflow_publishes_every_branch_after_processing_failures() { yq 'del(.jobs.publish.if)' "$REUSABLE_WORKFLOW" >"$mutant" ! workflow_publishes_after_processing_failures "$mutant" \ || fail 'the guard passes when publication runs only after every process leg succeeds' + mutant="$TEST_TMP_ROOT/publish-without-preview-gate.yml" + yq ".jobs.publish.if = \"\${{ always() && needs.discover.result == 'success' }}\"" \ + "$REUSABLE_WORKFLOW" >"$mutant" + ! workflow_publishes_after_processing_failures "$mutant" \ + || fail 'the guard passes when a preview can reach publication' } test_workflow_offers_input_and_secret_terraform_environment() { @@ -1928,6 +1980,16 @@ test_workflow_fails_discovery_when_the_script_fails() { || fail "the discover step emitted a matrix after discovery failed: $(<"$work/output")" } +test_workflow_previews_without_publishing() { + # Production break caught: callers cannot ask for a preview, or discovery never hears of it and + # processes every branch from a pull request. + yq -o=json '.' "$REUSABLE_WORKFLOW" | jq -e ' + .on.workflow_call.inputs.preview == {type: "boolean", default: false} + and ([.jobs.discover.steps[] | select(.id == "discover") | .env.DISCOVERY_PREVIEW] + == ["${{ inputs.preview }}"]) + ' >/dev/null || fail 'the reusable workflow does not offer a preview input wired into discovery' +} + cleanup_test_repositories() { cleanup_discovery_repository cleanup_processing_workspace @@ -1965,7 +2027,9 @@ if [[ $# -eq 0 ]]; then test_workflow_publishes_every_branch_after_processing_failures test_workflow_offers_input_and_secret_terraform_environment test_workflow_fails_discovery_when_the_script_fails - test_workflow_summarises_update_logs test_workflow_reports_the_dry_run_outcome) + test_workflow_previews_without_publishing + test_workflow_summarises_update_logs test_workflow_summarises_the_preview_patch + test_workflow_reports_the_dry_run_outcome) while IFS= read -r test_name; do tests+=("$test_name"); done < <(compgen -A function test_discovery_) for test_name in "${tests[@]}"; do "$test_name" From acfe7fbf4499f34e6fd8e2c63d199240e81e4c43 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:12:08 +1000 Subject: [PATCH 09/14] feat(actions-example): preview config changes on pull requests Give each caller a preview job that runs a same-repository pull request into the default branch on the reusable workflow's new preview input, without publishing, so a config change can be seen before it merges. Move each caller's concurrency block to job level so the live job can keep queue: max while the preview job cancels its own older run; GitHub rejects both settings in one concurrency block. Anchors share the live job's with/secrets so the preview always runs with the pull request's edits. The preview job declares no permissions, because GitHub refuses to start a read-only caller when the called workflow's publish job requests write, even when that job is skipped. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../tf-version-bump-nonproduction.yml | 31 +++++++++++++---- .../workflows/tf-version-bump-production.yml | 31 +++++++++++++---- examples/github-actions/test.sh | 33 +++++++++++++++++++ 3 files changed, 81 insertions(+), 14 deletions(-) diff --git a/examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml b/examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml index 5dbfa48..d218b8c 100644 --- a/examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml +++ b/examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml @@ -4,6 +4,10 @@ on: push: paths: - .github/tf-version-bump/nonproduction.yml + pull_request: + paths: + - .github/tf-version-bump/nonproduction.yml + - .github/workflows/tf-version-bump-nonproduction.yml workflow_dispatch: inputs: branch_prefix: @@ -27,19 +31,18 @@ permissions: pull-requests: write issues: write -concurrency: - group: tf-version-bump-nonproduction-${{ github.repository_id }} - cancel-in-progress: false - queue: max - jobs: automation: if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} + concurrency: + group: tf-version-bump-nonproduction-${{ github.repository_id }} + cancel-in-progress: false + queue: max uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: + secrets: &policy-secrets TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} - with: + with: &policy-inputs automation_policy_id: nonproduction allowed_branch_prefixes: | state/nonproduction/ @@ -55,6 +58,20 @@ jobs: tf_version_bump_version: v1.0.0-rc.17 tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} + preview: ${{ github.event_name == 'pull_request' }} max_parallel: 4 commit_author_name: github-actions[bot] commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com + # Previews a same-repository pull request into the default branch on one sampled branch per + # prefix, without publishing. The aliases give it the live job's inputs, including any the pull + # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects + # both settings in one concurrency block. It keeps the workflow's permissions because GitHub + # refuses to start a read-only caller of a workflow whose publish job requests write. + preview: + if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} + concurrency: + group: tf-version-bump-nonproduction-preview-${{ github.repository_id }}-${{ github.event.number }} + cancel-in-progress: true + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: *policy-secrets + with: *policy-inputs diff --git a/examples/github-actions/.github/workflows/tf-version-bump-production.yml b/examples/github-actions/.github/workflows/tf-version-bump-production.yml index 5dc249c..d6c52f9 100644 --- a/examples/github-actions/.github/workflows/tf-version-bump-production.yml +++ b/examples/github-actions/.github/workflows/tf-version-bump-production.yml @@ -4,6 +4,10 @@ on: push: paths: - .github/tf-version-bump/production.yml + pull_request: + paths: + - .github/tf-version-bump/production.yml + - .github/workflows/tf-version-bump-production.yml workflow_dispatch: inputs: branch_prefix: @@ -27,19 +31,18 @@ permissions: pull-requests: write issues: write -concurrency: - group: tf-version-bump-production-${{ github.repository_id }} - cancel-in-progress: false - queue: max - jobs: automation: if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} + concurrency: + group: tf-version-bump-production-${{ github.repository_id }} + cancel-in-progress: false + queue: max uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: + secrets: &policy-secrets TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} - with: + with: &policy-inputs automation_policy_id: production allowed_branch_prefixes: | state/production/ @@ -53,6 +56,20 @@ jobs: tf_version_bump_version: v1.0.0-rc.17 tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} + preview: ${{ github.event_name == 'pull_request' }} max_parallel: 2 commit_author_name: github-actions[bot] commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com + # Previews a same-repository pull request into the default branch on one sampled branch per + # prefix, without publishing. The aliases give it the live job's inputs, including any the pull + # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects + # both settings in one concurrency block. It keeps the workflow's permissions because GitHub + # refuses to start a read-only caller of a workflow whose publish job requests write. + preview: + if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} + concurrency: + group: tf-version-bump-production-preview-${{ github.repository_id }}-${{ github.event.number }} + cancel-in-progress: true + uses: ./.github/workflows/tf-version-bump-reusable.yml + secrets: *policy-secrets + with: *policy-inputs diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index af5793a..20223f4 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -1990,6 +1990,38 @@ test_workflow_previews_without_publishing() { ' >/dev/null || fail 'the reusable workflow does not offer a preview input wired into discovery' } +test_callers_preview_pull_requests() { + # Production break caught: a preview runs with inputs that differ from its live run, queues + # behind or blocks live runs, previews a fork or a non-default base, narrows its permissions so + # GitHub refuses to start it, or a live run starts from a pull request. + local policy workflow + for policy in nonproduction production; do + workflow="$SCRIPT_DIR/.github/workflows/tf-version-bump-$policy.yml" + yq -o=json '.' "$workflow" | jq -e --arg policy "$policy" ' + .jobs as $jobs + | (has("concurrency") | not) + and .on.pull_request == {paths: [$jobs.automation.with.config_path, + ".github/workflows/tf-version-bump-\($policy).yml"]} + and ($jobs | keys) == ["automation", "preview"] + and $jobs.automation.if == "${{ github.ref == format('"'"'refs/heads/{0}'"'"', github.event.repository.default_branch) }}" + and $jobs.automation.concurrency == {group: "tf-version-bump-\($policy)-${{ github.repository_id }}", + "cancel-in-progress": false, queue: "max"} + and $jobs.preview.if == "${{ github.event_name == '"'"'pull_request'"'"' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }}" + and $jobs.preview.concurrency == {group: "tf-version-bump-\($policy)-preview-${{ github.repository_id }}-${{ github.event.number }}", + "cancel-in-progress": true} + and ($jobs.preview | has("permissions") | not) + and $jobs.preview.uses == $jobs.automation.uses + and $jobs.preview.with == $jobs.automation.with + and $jobs.preview.secrets == $jobs.automation.secrets + and $jobs.automation.with.preview == "${{ github.event_name == '"'"'pull_request'"'"' }}" + ' >/dev/null || fail "the $policy caller does not preview pull requests with its live inputs" + # The aliases keep one copy of each policy value, so a pull request's edits reach its preview. + grep -qxF ' with: *policy-inputs' "$workflow" \ + && grep -qxF ' secrets: *policy-secrets' "$workflow" \ + || fail "the $policy caller's preview does not reuse the live job's inputs by alias" + done +} + cleanup_test_repositories() { cleanup_discovery_repository cleanup_processing_workspace @@ -2028,6 +2060,7 @@ if [[ $# -eq 0 ]]; then test_workflow_offers_input_and_secret_terraform_environment test_workflow_fails_discovery_when_the_script_fails test_workflow_previews_without_publishing + test_callers_preview_pull_requests test_workflow_summarises_update_logs test_workflow_summarises_the_preview_patch test_workflow_reports_the_dry_run_outcome) while IFS= read -r test_name; do tests+=("$test_name"); done < <(compgen -A function test_discovery_) From 70979cf982ff2e921235faf4c2a663b9a941a286 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:17:16 +1000 Subject: [PATCH 10/14] docs(actions-example): document pull-request previews Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- docs/ADVANCED-USAGE.md | 2 +- examples/github-actions/README.md | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/docs/ADVANCED-USAGE.md b/docs/ADVANCED-USAGE.md index ec2317d..ffe880c 100644 --- a/docs/ADVANCED-USAGE.md +++ b/docs/ADVANCED-USAGE.md @@ -6,7 +6,7 @@ This is intentionally separate from the core CLI. `tf-version-bump` edits one ch ## GitHub Actions state-branch automation POC -For a scheduled or manually dispatched GitHub Actions proof of concept, see [`examples/github-actions`](../examples/github-actions/README.md). It copies into a consuming repository's `.github` directory and provides separate production and non-production callers that run only from the default branch, plus config-change triggers, read-only pull-request config validation, dry-run support, update pull requests, and marked failure issues. The reusable workflow has three jobs: `discover`, `process`, and `publish`. The pull-request check validates only the control config with `tf-version-bump`; full state-branch dry runs remain deferred. The same copy adds a read-only version report workflow that audits every state branch against its policy's config, and each caller can leave individual branches out with a `!` exclusion; the example's README describes both. +For a scheduled or manually dispatched GitHub Actions proof of concept, see [`examples/github-actions`](../examples/github-actions/README.md). It copies into a consuming repository's `.github` directory and provides separate production and non-production callers whose live runs start only from the default branch, plus config-change triggers, read-only pull-request config validation, pull-request previews on one sampled state branch per prefix, dry-run support, update pull requests, and marked failure issues. The reusable workflow has three jobs: `discover`, `process`, and `publish`. The pull-request config check validates only the control config with `tf-version-bump`; the preview processes a sample of state branches and publishes nothing. The same copy adds a read-only version report workflow that audits every state branch against its policy's config, and each caller can leave individual branches out with a `!` exclusion; the example's README describes both. The example targets trusted official providers and organisation-authored private modules, including the HCP Terraform registry. It checks for configuration mistakes and unexpected file changes; it does not sandbox malicious Terraform or providers. diff --git a/examples/github-actions/README.md b/examples/github-actions/README.md index cd206b5..25a0921 100644 --- a/examples/github-actions/README.md +++ b/examples/github-actions/README.md @@ -32,7 +32,7 @@ Review and commit the copied files. The example supplies separate callers for: The same copy adds `tf-version-bump-report.yml`, a read-only version report described [below](#version-report). -Both callers run only from the default branch. Their schedules are Monday 04:17 and Sunday 04:43 respectively in `Australia/Melbourne`. They also run when their control configuration changes, and can be started manually. A manual `branch_prefix` can narrow the configured prefixes but cannot select another branch family. +Both callers' live runs start only from the default branch; each caller also previews same-repository pull requests into the default branch (see [Preview on pull requests](#preview-on-pull-requests)). Their schedules are Monday 04:17 and Sunday 04:43 respectively in `Australia/Melbourne`. They also run when their control configuration changes, and can be started manually. A manual `branch_prefix` can narrow the configured prefixes but cannot select another branch family. To leave one branch out of a caller's policy, add a line naming it with a leading `!`: @@ -47,7 +47,7 @@ An exclusion names one exact branch (wildcard characters fail validation), so `s Allow the workflow's `contents`, `pull-requests` and `issues` write permissions. Enable **Settings → Actions → General → Workflow permissions → Allow GitHub Actions to create and approve pull requests** before live publication. -Create an Actions secret named `TF_API_TOKEN` with read access to your HCP Terraform registry modules and providers. The workflow exposes it as `TF_TOKEN_app_terraform_io` only during processing. The processing checkouts disable persisted Git credentials. +Create an Actions secret named `TF_API_TOKEN` with read access to your HCP Terraform registry modules and providers. The workflow exposes it as `TF_TOKEN_app_terraform_io` only during processing. The processing checkouts disable persisted Git credentials. Pull-request previews run the same processing job, so they receive the same token. ## Supply environment variables @@ -75,7 +75,7 @@ Each of the secret's values is registered with `::add-mask::`, which redacts it A name may appear only once across both sources. Names the automation or the runner sets are rejected, as are names that would redirect the programs Terraform runs or its configuration, credentials, logging or plug-in sources. The reserved prefixes are `PROCESS_`, `RECONCILE_`, `DISCOVERY_`, `RUNNER_`, `ACTIONS_`, `LD_`, `DYLD_`, `TF_CLI_ARGS`, `TF_LOG`, `TF_PLUGIN_CACHE` and `GIT_`. The reserved exact names are `PATH`, `IFS`, `ENV`, `BASH_ENV`, `SHELLOPTS`, `BASHOPTS`, `TF_DATA_DIR`, `TF_IN_AUTOMATION`, `CHECKPOINT_DISABLE`, `TF_CLI_CONFIG_FILE`, `TERRAFORM_CONFIG`, `TF_WORKSPACE`, `HOME`, `TMPDIR`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `GITHUB_ENV`, `GITHUB_PATH`, `GITHUB_OUTPUT`, `GITHUB_STEP_SUMMARY`, `GITHUB_STATE` and `TF_TOKEN_app_terraform_io`. The last of those is reserved in any letter case or other spelling Terraform maps to the `app.terraform.io` host, because it would silently shadow the registry token the workflow injects from the `TF_API_TOKEN` secret; `TF_TOKEN_*` names for other registries remain allowed. The five `GITHUB_*` names are the runner's own command channels rather than provider configuration, which is why `GITHUB_APP_ID` and the provider's other `GITHUB_` variables are accepted while `GITHUB_ENV` is not. Treat the list as best effort rather than exhaustive. The structural protection is a separate rule: a file newly created during a run is only ever publishable if it is a `.terraform.lock.hcl` directly inside a configured Terraform root. A rejected entry never prints its value, and is rejected before any Terraform command runs and before any file in the checkout is modified. -Both supplied callers forward the same repository `TERRAFORM_ENV` secret, so production credentials are also available to non-production state-branch jobs. To separate them, either give each policy its own secret name — mapping, say, `TERRAFORM_ENV_PRODUCTION` and `TERRAFORM_ENV_NONPRODUCTION` onto the reusable workflow's `TERRAFORM_ENV` secret in each caller — or hold the secret in a GitHub Environment named for the caller's `automation_policy_id`. A caller cannot pass an environment's secrets to a reusable workflow, so the reusable workflow's `process` job must then also declare that environment with its job-level `environment` key; GitHub then gives the job the environment's secret rather than the one the caller passes. +Both supplied callers forward the same repository `TERRAFORM_ENV` secret, so production credentials are also available to non-production state-branch jobs. To separate them, either give each policy its own secret name — mapping, say, `TERRAFORM_ENV_PRODUCTION` and `TERRAFORM_ENV_NONPRODUCTION` onto the reusable workflow's `TERRAFORM_ENV` secret in each caller — or hold the secret in a GitHub Environment named for the caller's `automation_policy_id`. A caller cannot pass an environment's secrets to a reusable workflow, so the reusable workflow's `process` job must then also declare that environment with its job-level `environment` key; GitHub then gives the job the environment's secret rather than the one the caller passes. An Environment's deployment-branch rules also apply to previews, which run from `refs/pull//merge`, so a rule that admits only the default branch may withhold the secret from a preview or hold it for approval. ## Configure updates @@ -86,7 +86,7 @@ Edit the control configurations on the default branch: .github/tf-version-bump/production.yml ``` -These are strict `tf-version-bump` configuration files. The workflow owns file selection, so do not add a `pattern` key. Pull requests changing these files run a read-only config validation check; they do not process state branches or run Terraform. +These are strict `tf-version-bump` configuration files. The workflow owns file selection, so do not add a `pattern` key. Pull requests changing these files run a read-only config validation check and, for same-repository pull requests into the default branch, a preview that processes one sampled state branch per prefix without publishing anything (see [Preview on pull requests](#preview-on-pull-requests)). The callers process the repository root by default: @@ -143,6 +143,20 @@ Unchanged candidates still run validation. Before any PR or issue reconciliation Publication uses the built-in `GITHUB_TOKEN`. The helper respects Git's signing configuration; the example does not provision a signing key. If signing is enabled, the runner must have a working key. Do not assume token-created branches and PRs will automatically run your downstream checks; review GitHub's [GITHUB_TOKEN workflow behaviour](https://docs.github.com/en/actions/how-tos/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow#triggering-a-workflow-from-a-workflow) when configuring required checks. +## Preview on pull requests + +A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests from forks and pull requests into another branch skip the preview. + +The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. It also runs the pull request's own scripts and reusable workflow, so a pull request that edits those is previewed with the edited versions. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. + +The branch for each prefix is picked from the pull request number, so every push to the same pull request previews the same branches while that prefix's branches are unchanged. Adding, deleting or excluding a branch under a prefix may move the pick, and different pull requests may land on the same branch. A branch belongs to the first prefix it matches, and a prefix that owns no branch is named in a warning annotation. The preview is a smoke test, not coverage: a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled. + +Each `process` job's summary shows the updater's log and, for a changed candidate, the candidate patch. The patch is the full candidate the merged configuration would produce, not only what the pull request changes: on a branch whose update pull request has not been merged, it also includes the changes the current configuration would already make. + +A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, and with it the pull request's check. The failure may already exist on that branch, so check the branch's marked failure issue before blaming the pull request, and do not make the preview a required check. An invalid configuration fails every sampled branch at the update stage as `branch-update`, while the configuration validation check reports the cause. + +GitHub runs pull-request workflows only for the `opened`, `synchronize` and `reopened` activities, and not at all while the pull request has a merge conflict. A preview is therefore not refreshed when the default branch or the state branches move; push to the pull request or re-run the workflow to refresh it. + ## Version report `tf-version-bump-report.yml` compares every state branch with its policy's control configuration without changing anything. It runs on Mondays at 06:17 `Australia/Melbourne`, after both scheduled update runs, and can be started manually from the default branch. It has read-only repository access, runs no Terraform and receives no secrets. @@ -164,6 +178,6 @@ Both CSVs keep values as the audit reports them: as written, except that when a Start with a manual run from the default branch and select `dry_run`. This processes and validates candidates and checks publication locally, without pushing refs or changing PRs or issues. Inspect the candidate patch and logs before enabling live publication. -Open the workflow run to inspect `discover`, `process` and `publish`. Each `process` job's summary includes the updater's log for every configured root, so the run overview page shows which versions changed; a long log is truncated there and kept in full in the artefact. Terraform's own logs appear only in the artefact, because a provider can echo a credential into them. In a dry run, each `publish` job's summary states what a live run would have done to the branch's pull request and failure issue. Download its result artefacts to see `result.json`, `candidate.patch` and captured command logs. Artefacts are retained for seven days. Results belong to one run attempt; use **Re-run all jobs**, as partial job reruns are unsupported. +Open the workflow run to inspect `discover`, `process` and `publish`. Each `process` job's summary includes the updater's log for every configured root, so the run overview page shows which versions changed; a long log is truncated there and kept in full in the artefact. In a preview, the summary also shows each changed branch's candidate patch, truncated the same way. Terraform's own logs appear only in the artefact, because a provider can echo a credential into them. In a dry run, each `publish` job's summary states what a live run would have done to the branch's pull request and failure issue. Download its result artefacts to see `result.json`, `candidate.patch` and captured command logs. Artefacts are retained for seven days. Results belong to one run attempt; use **Re-run all jobs**, as partial job reruns are unsupported. Before enabling schedules, test in a disposable private repository: run a dry run, publish a valid change twice and check that the same PR is refreshed, then introduce a validation failure and confirm that the PR closes and one failure issue is maintained. Finally, test a valid no-change result and confirm that the issue closes. No live GitHub repository is created or mutated by this repository's local component tests. From 1e91b56c45118f17163027a93058a07d28e5871f Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:23:19 +1000 Subject: [PATCH 11/14] test(actions-example): check each caller alias separately Split the combined `grep && grep || fail` alias check in test_callers_preview_pull_requests into two independent checks, each with its own `|| fail`, naming which alias (with/secrets) is missing. shellcheck's SC2015 flags `A && B || C` because C also runs when A succeeds and B fails, masking which alias check actually failed. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- examples/github-actions/test.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index 20223f4..7f07c83 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -2017,8 +2017,9 @@ test_callers_preview_pull_requests() { ' >/dev/null || fail "the $policy caller does not preview pull requests with its live inputs" # The aliases keep one copy of each policy value, so a pull request's edits reach its preview. grep -qxF ' with: *policy-inputs' "$workflow" \ - && grep -qxF ' secrets: *policy-secrets' "$workflow" \ || fail "the $policy caller's preview does not reuse the live job's inputs by alias" + grep -qxF ' secrets: *policy-secrets' "$workflow" \ + || fail "the $policy caller's preview does not reuse the live job's secrets by alias" done } From f1fe1334a0ce412bf2ea4860200c6d6cda3e88c0 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:35:11 +1000 Subject: [PATCH 12/14] docs(actions-example): correct preview refresh and fork wording A final review of the preview feature found three wording problems: the refresh paragraph claimed a re-run picks up a moved default branch, but GitHub re-runs reuse the original event's GITHUB_SHA, so only a moved state branch is picked up; the Environment paragraph conflated branch rules (which reject) with required reviewers (which hold for approval); and the fork-skip claim was never verified, since a fork or Dependabot pull request's read-only token may make GitHub fail the run at startup instead. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../specs/2026-09-22-config-change-preview-design.md | 2 +- examples/github-actions/README.md | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md index 0640902..0078c74 100644 --- a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md +++ b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md @@ -8,7 +8,7 @@ The preview is a smoke test, not coverage. It shows the full candidate the merge ## Scope and assumptions -- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request from a fork, or into a branch other than the default branch, skips the preview rather than failing. +- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request into a branch other than the default branch skips the preview; one from a fork is skipped by the preview job's `if`, although GitHub may instead fail it at startup because its token is read-only (unverified). - Anyone who can open such a pull request can already edit the workflows, so running Terraform with the registry token at pull-request time adds no new exposure. Accidental publication from an unmerged pull request is a new failure mode, so publication gains an independent guard (see [Publication guard](#publication-guard)). - A pull-request run executes the merge commit's workflows and scripts. A pull request that also edits the scripts or the reusable workflow is previewed with the edited versions. - Each caller previews only its own policy, triggered by its own configuration file or its own caller workflow file. A change to the scripts or the reusable workflow alone does not trigger a preview. diff --git a/examples/github-actions/README.md b/examples/github-actions/README.md index 25a0921..c593e88 100644 --- a/examples/github-actions/README.md +++ b/examples/github-actions/README.md @@ -75,7 +75,7 @@ Each of the secret's values is registered with `::add-mask::`, which redacts it A name may appear only once across both sources. Names the automation or the runner sets are rejected, as are names that would redirect the programs Terraform runs or its configuration, credentials, logging or plug-in sources. The reserved prefixes are `PROCESS_`, `RECONCILE_`, `DISCOVERY_`, `RUNNER_`, `ACTIONS_`, `LD_`, `DYLD_`, `TF_CLI_ARGS`, `TF_LOG`, `TF_PLUGIN_CACHE` and `GIT_`. The reserved exact names are `PATH`, `IFS`, `ENV`, `BASH_ENV`, `SHELLOPTS`, `BASHOPTS`, `TF_DATA_DIR`, `TF_IN_AUTOMATION`, `CHECKPOINT_DISABLE`, `TF_CLI_CONFIG_FILE`, `TERRAFORM_CONFIG`, `TF_WORKSPACE`, `HOME`, `TMPDIR`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `GITHUB_ENV`, `GITHUB_PATH`, `GITHUB_OUTPUT`, `GITHUB_STEP_SUMMARY`, `GITHUB_STATE` and `TF_TOKEN_app_terraform_io`. The last of those is reserved in any letter case or other spelling Terraform maps to the `app.terraform.io` host, because it would silently shadow the registry token the workflow injects from the `TF_API_TOKEN` secret; `TF_TOKEN_*` names for other registries remain allowed. The five `GITHUB_*` names are the runner's own command channels rather than provider configuration, which is why `GITHUB_APP_ID` and the provider's other `GITHUB_` variables are accepted while `GITHUB_ENV` is not. Treat the list as best effort rather than exhaustive. The structural protection is a separate rule: a file newly created during a run is only ever publishable if it is a `.terraform.lock.hcl` directly inside a configured Terraform root. A rejected entry never prints its value, and is rejected before any Terraform command runs and before any file in the checkout is modified. -Both supplied callers forward the same repository `TERRAFORM_ENV` secret, so production credentials are also available to non-production state-branch jobs. To separate them, either give each policy its own secret name — mapping, say, `TERRAFORM_ENV_PRODUCTION` and `TERRAFORM_ENV_NONPRODUCTION` onto the reusable workflow's `TERRAFORM_ENV` secret in each caller — or hold the secret in a GitHub Environment named for the caller's `automation_policy_id`. A caller cannot pass an environment's secrets to a reusable workflow, so the reusable workflow's `process` job must then also declare that environment with its job-level `environment` key; GitHub then gives the job the environment's secret rather than the one the caller passes. An Environment's deployment-branch rules also apply to previews, which run from `refs/pull//merge`, so a rule that admits only the default branch may withhold the secret from a preview or hold it for approval. +Both supplied callers forward the same repository `TERRAFORM_ENV` secret, so production credentials are also available to non-production state-branch jobs. To separate them, either give each policy its own secret name — mapping, say, `TERRAFORM_ENV_PRODUCTION` and `TERRAFORM_ENV_NONPRODUCTION` onto the reusable workflow's `TERRAFORM_ENV` secret in each caller — or hold the secret in a GitHub Environment named for the caller's `automation_policy_id`. A caller cannot pass an environment's secrets to a reusable workflow, so the reusable workflow's `process` job must then also declare that environment with its job-level `environment` key; GitHub then gives the job the environment's secret rather than the one the caller passes. An Environment's deployment-branch rules also apply to previews, which run from `refs/pull//merge`, so a rule that admits only the default branch may withhold the secret from a preview, and an Environment with required reviewers holds a preview for approval. ## Configure updates @@ -145,7 +145,7 @@ Publication uses the built-in `GITHUB_TOKEN`. The helper respects Git's signing ## Preview on pull requests -A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests from forks and pull requests into another branch skip the preview. +A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests into another branch skip the preview. The preview job also skips pull requests from forks, although GitHub may instead fail such a run, or a Dependabot pull request's, at startup, because their token is read-only. The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. It also runs the pull request's own scripts and reusable workflow, so a pull request that edits those is previewed with the edited versions. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. @@ -155,7 +155,7 @@ Each `process` job's summary shows the updater's log and, for a changed candidat A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, and with it the pull request's check. The failure may already exist on that branch, so check the branch's marked failure issue before blaming the pull request, and do not make the preview a required check. An invalid configuration fails every sampled branch at the update stage as `branch-update`, while the configuration validation check reports the cause. -GitHub runs pull-request workflows only for the `opened`, `synchronize` and `reopened` activities, and not at all while the pull request has a merge conflict. A preview is therefore not refreshed when the default branch or the state branches move; push to the pull request or re-run the workflow to refresh it. +GitHub runs pull-request workflows only for the `opened`, `synchronize` and `reopened` activities, and not at all while the pull request has a merge conflict. Re-running the workflow picks up moved state branches, because discovery lists them live, but a GitHub re-run reuses the original event's `GITHUB_SHA`, so reflecting a moved default branch needs a new pull-request event instead, such as a push (for example updating the branch from the default branch) or closing and reopening the pull request. ## Version report From 9f0585d60588bab691b99b861b7c09f2b7b48b77 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 14:41:33 +1000 Subject: [PATCH 13/14] docs: remove the config-change preview design and plan The design and implementation plan served the branch; their reasoning stays in its history, and the README documents the behaviour. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../plans/2026-09-22-config-change-preview.md | 1105 ----------------- ...2026-09-22-config-change-preview-design.md | 146 --- 2 files changed, 1251 deletions(-) delete mode 100644 docs/superpowers/plans/2026-09-22-config-change-preview.md delete mode 100644 docs/superpowers/specs/2026-09-22-config-change-preview-design.md diff --git a/docs/superpowers/plans/2026-09-22-config-change-preview.md b/docs/superpowers/plans/2026-09-22-config-change-preview.md deleted file mode 100644 index d0b4ac6..0000000 --- a/docs/superpowers/plans/2026-09-22-config-change-preview.md +++ /dev/null @@ -1,1105 +0,0 @@ -# Config-change preview Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** When a same-repository pull request into the default branch changes a policy's control config or caller workflow, the GitHub Actions example processes one sampled state branch per configured prefix, exactly as a live run would, and publishes nothing. - -**Architecture:** -- Discovery gains a preview mode. It accepts only a `refs/pull//merge` caller ref, seeds a deterministic per-prefix sample from ``, and otherwise emits the same matrix as today. -- The reusable workflow gains a `preview` input. It skips `publish` and adds the candidate patch to the `process` summary. -- Each caller gains a `preview` job. It reuses the live job's `with:`/`secrets:` blocks through YAML aliases and carries its own cancelling concurrency. -- `reconcile-state-branch.sh publish` independently refuses any caller ref other than the default branch. - -**Tech Stack:** Bash (the scripts run under `set -euo pipefail` with `LC_ALL=C`), GitHub Actions YAML, `jq`, mikefarah `yq`, the example's own harnesses (`examples/github-actions/test.sh`, `reconcile-test.sh`, `report-test.sh`), Go documentation tests, actionlint via `make actionlint`, shellcheck 0.11.0 via `make shellcheck`. - -**Spec:** `docs/superpowers/specs/2026-09-22-config-change-preview-design.md`. Read it before starting any task; it carries the reasoning and the verified GitHub Actions behaviour. - -## Global Constraints - -**Prose and Markdown** -- Use Australian / British spelling in prose and comments. -- Write each Markdown paragraph, list item and quote on one line. `TestDocumentationProseIsNotHardWrapped` fails on hard-wrapped prose. - -**Process** -- Follow TDD in every task: write the failing test, watch it fail, implement, watch it pass. -- Test output must be pristine. Capture and assert expected diagnostics. - -**Commits and pushes** -- Commit with the GenAI wrapper, passing the message through a file: `/Users/dan/.claude/bin/claude-git -C /Users/dan/Code/tf-version-bump commit -F `. Write the message file in the session scratchpad. -- End every commit message with these two lines: - - `Co-Authored-By: Claude Opus 5 (1M context) ` - - `Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ` -- Never `git add -A`. Stage named paths. -- Never push without Dan's confirmation. - -**Harness** -- Run the example harness with the GenAI signer so fixture commits never reach Dan's 1Password signer: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh ` (from the repository root). -- Discovery and workflow-step tests need no Docker; `test_processing_*` tests do. - -**Workflow YAML** -- Keep each caller's `with:` keys indented six spaces. `scripts/update-actions-release-pin.sh` matches ` tf_version_bump_version:` and ` tf_version_bump_archive_sha256:` exactly once per caller. -- Never combine `queue: max` with `cancel-in-progress: true` in one `concurrency` block; GitHub rejects it. -- Do not use YAML merge keys (`<<`). Only anchors (`&name`) and whole-node aliases (`*name`). -- The preview job must not declare job-level `permissions`. A read-only caller job fails at startup when the called workflow's `publish` job requests write, even when it is skipped (verified; see the spec). -- Pass values into `run:` bodies only through `env:`, never `${{ }}` inside a script body. - -**Size** -- Keep scripts and workflows small; consumers copy them. - ---- - -## File Structure - -| File | Change | Responsibility | -| --- | --- | --- | -| `examples/github-actions/.github/scripts/discover-state-branches.sh` | Modify | Preview guard (`DISCOVERY_PREVIEW`, PR merge ref, seed) and per-prefix sampling | -| `examples/github-actions/.github/scripts/reconcile-state-branch.sh` | Modify | Refuse publication from any ref but the default branch | -| `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml` | Modify | `preview` input, discovery and publish wiring, `publish` gate, shared fenced renderer, preview patch | -| `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` | Modify | PR trigger, job-level concurrency, `preview` job via aliases | -| `examples/github-actions/.github/workflows/tf-version-bump-production.yml` | Modify | Same as non-production | -| `examples/github-actions/test.sh` | Modify | Discovery, reusable-workflow and caller tests | -| `examples/github-actions/reconcile-test.sh` | Modify | Publication guard tests | -| `examples/github-actions/README.md` | Modify | New "Preview on pull requests" section; four passage edits | -| `docs/ADVANCED-USAGE.md` | Modify | GitHub Actions paragraph | - ---- - -### Task 1: Discovery preview guard - -**Files:** -- Modify: `examples/github-actions/.github/scripts/discover-state-branches.sh:83-85` (the caller-ref check) -- Modify: `examples/github-actions/test.sh:362-430` (`setup_discovery_repository`, `run_discovery`) -- Test: `examples/github-actions/test.sh` (new `test_discovery_preview_accepts_only_pull_request_merge_refs`) - -**Interfaces:** -- Consumes: nothing new. -- Produces: - - Env input `DISCOVERY_PREVIEW`: unset means `false`. Otherwise it must be exactly `true` or `false`. - - Shell variables `preview` (`true`/`false`) and `preview_seed` (the decimal PR number, set only when `preview=true`). Task 2 reads both. - - Diagnostics, exactly: - - `discovery input error: DISCOVERY_PREVIEW must be true or false` - - `discovery caller error: preview caller ref must be refs/pull//merge` - - `discovery input error: a preview cannot take a manual prefix` - -- [ ] **Step 1: Let the harness pass `DISCOVERY_PREVIEW` only when a test sets it** - -In `setup_discovery_repository` (`test.sh:362`), add this as its second line, after `cleanup_discovery_repository`: - -```bash - unset DISCOVERY_PREVIEW -``` - -In `run_discovery` (`test.sh:414`), export `DISCOVERY_PREVIEW` inside the existing subshell only when a test has set it. An unset `DISCOVERY_PREVIEW` then stays unset for the script, which is the report workflow's path. Do not use `env` for this: `test_discovery_uses_runner_git_not_a_workstation_shim` runs discovery with a `PATH` that has no `env`. The subshell keeps the export from leaking. Change the line after `cd "$DISCOVERY_REPO"` to: - -```bash - cd "$DISCOVERY_REPO" - [[ -z "${DISCOVERY_PREVIEW+set}" ]] || export DISCOVERY_PREVIEW -``` - -- [ ] **Step 2: Write the failing test** - -Add after `test_discovery_rejects_invalid_inputs_by_stage` in `test.sh`: - -```bash -test_discovery_preview_accepts_only_pull_request_merge_refs() { - # Production break caught: relaxing the default-branch guard for previews lets any ref drive - # discovery, a pull-request ref passes without asking for a preview, or a preview's seed can - # disagree with the pull request it runs for. - setup_discovery_repository - add_discovery_branch "state/prod/example" - DISCOVERY_ALLOWED_PREFIXES="state/prod/" - - DISCOVERY_CALLER_REF="refs/pull/42/merge" - assert_discovery_failure "discovery caller error: caller ref must be refs/heads/main" \ - "a pull-request ref without a preview" - - DISCOVERY_PREVIEW=true - local output - output=$(run_discovery) - jq -e '.include | map(.branch) == ["state/prod/example"]' <<<"$output" >/dev/null \ - || fail "preview discovery refused a pull-request merge ref: $output" - - local ref - for ref in refs/heads/main refs/pull/0/merge refs/pull/042/merge refs/pull/7/head refs/pull/x/merge; do - DISCOVERY_CALLER_REF=$ref - assert_discovery_failure "discovery caller error: preview caller ref must be refs/pull//merge" \ - "a preview from $ref" - done - - DISCOVERY_CALLER_REF="refs/pull/42/merge" - DISCOVERY_MANUAL_PREFIX="state/prod/" - assert_discovery_failure "discovery input error: a preview cannot take a manual prefix" \ - "a preview with a manual prefix" - - DISCOVERY_MANUAL_PREFIX="" - local value - for value in "" TRUE yes; do - DISCOVERY_PREVIEW=$value - assert_discovery_failure "discovery input error: DISCOVERY_PREVIEW must be true or false" \ - "a preview flag of '$value'" - done -} -``` - -`test_discovery_*` functions are collected automatically by `compgen` in the harness's no-argument run, so there is no list to update. - -- [ ] **Step 3: Run it and watch it fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_accepts_only_pull_request_merge_refs` - -Expected: the harness exits non-zero without printing `PASS:`. Discovery still demands `refs/heads/main`, so `output=$(run_discovery)` fails under the harness's `set -e`, and discovery's own `discovery caller error: caller ref must be refs/heads/main` is the last line on stderr. - -- [ ] **Step 4: Implement the guard** - -In `discover-state-branches.sh`, replace lines 83–85: - -```bash -expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" -[[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ - || fail_discovery caller "caller ref must be $expected_caller_ref" -``` - -with: - -```bash -preview=${DISCOVERY_PREVIEW-false} -[[ "$preview" == true || "$preview" == false ]] \ - || fail_discovery input "DISCOVERY_PREVIEW must be true or false" -if [[ "$preview" == true ]]; then - # A preview runs from a pull request's merge ref. Its number seeds the sample, so the seed - # cannot disagree with the pull request, and a manual prefix only arises from a dispatch. - [[ "$DISCOVERY_CALLER_REF" =~ ^refs/pull/([1-9][0-9]*)/merge$ ]] \ - || fail_discovery caller "preview caller ref must be refs/pull//merge" - preview_seed=${BASH_REMATCH[1]} - [[ -z "$DISCOVERY_MANUAL_PREFIX" ]] \ - || fail_discovery input "a preview cannot take a manual prefix" -else - expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH" - [[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \ - || fail_discovery caller "caller ref must be $expected_caller_ref" -fi -``` - -Also add `DISCOVERY_PREVIEW` to `usage()` so `--help` documents it. Replace the heredoc body line `Discover immutable inputs for configured Terraform state branches.` with: - -```text -Discover immutable inputs for configured Terraform state branches. -With DISCOVERY_PREVIEW=true, run from refs/pull//merge and select one -sampled branch per prefix instead of every matching branch. -``` - -- [ ] **Step 5: Run the new test and every discovery test** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_accepts_only_pull_request_merge_refs` - -Expected: `PASS: test_discovery_preview_accepts_only_pull_request_merge_refs` - -Then run every existing discovery test by name (list them with `grep -o '^test_discovery_[a-z_]*' examples/github-actions/test.sh`), passing all names as arguments in one call. Expected: one `PASS:` line per test and nothing else. - -Then run `examples/github-actions/report-test.sh`, whose report workflow calls discovery without `DISCOVERY_PREVIEW`. Expected: all `PASS:`. - -- [ ] **Step 6: Shellcheck and commit** - -Run: `make shellcheck`. Expected: no output, exit 0. - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/discover-state-branches.sh examples/github-actions/test.sh -``` - -Commit message: `feat(actions-example): accept pull-request refs in preview discovery`. - ---- - -### Task 2: Discovery per-prefix sampling - -**Files:** -- Modify: `examples/github-actions/.github/scripts/discover-state-branches.sh`, from the branch-matching loop (~line 143) to the sort (~line 174) -- Test: `examples/github-actions/test.sh` (new `test_discovery_preview_samples_one_branch_per_prefix`, `test_discovery_preview_warns_for_prefixes_that_own_no_branch`, `test_discovery_preview_keeps_the_live_matrix_limit`) - -**Interfaces:** -- Consumes: `preview` and `preview_seed` from Task 1; the existing `selection_prefixes` array and `branch_records` array (`"\t"` entries). -- Produces: - - In preview mode, `branch_records` holds one sorted record per prefix that owns a branch. The matrix JSON shape is unchanged. - - Preview-only warning on stderr, exactly: `::warning::branch prefix '' owns no branch to preview`, with `%` escaped as `%25`. - -- [ ] **Step 1: Write the failing tests** - -Add after the Task 1 test in `test.sh`: - -```bash -test_discovery_preview_samples_one_branch_per_prefix() { - # Production break caught: a preview processes every branch, picks differently on each push, - # or samples a branch its caller excludes. - setup_discovery_repository - local name - for name in alpha bravo charlie; do - add_discovery_branch "state/staging/$name" - add_discovery_branch "state/production/$name" - done - add_discovery_branch "state/production/excluded" - DISCOVERY_ALLOWED_PREFIXES=$'state/staging/\nstate/production/\n!state/production/excluded' - DISCOVERY_PREVIEW=true - local output stderr_file="$DISCOVERY_TMP_ROOT/sample.stderr" - - # Known answers: the first 15 hex digits of sha256("\t"), modulo the three - # branches each prefix owns, index the sorted branches. Pull request 42 picks index 2 of - # state/production/ and 0 of state/staging/; pull request 43 picks index 1 of both. - DISCOVERY_CALLER_REF="refs/pull/42/merge" - output=$(run_discovery 2>"$stderr_file") - jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ - <<<"$output" >/dev/null || fail "pull request 42 did not sample its known branches: $output" - [[ ! -s "$stderr_file" ]] || fail "sampling reported a warning: $(<"$stderr_file")" - output=$(run_discovery) - jq -e '.include | map(.branch) == ["state/production/charlie", "state/staging/alpha"]' \ - <<<"$output" >/dev/null || fail "a repeat preview of pull request 42 sampled differently: $output" - - DISCOVERY_CALLER_REF="refs/pull/43/merge" - output=$(run_discovery) - jq -e '.include | map(.branch) == ["state/production/bravo", "state/staging/bravo"]' \ - <<<"$output" >/dev/null || fail "pull request 43 did not sample its known branches: $output" -} - - -test_discovery_preview_warns_for_prefixes_that_own_no_branch() { - # Production break caught: a prefix with nothing to preview passes silently, a narrower prefix - # listed after a broader one is reported as matching nothing without saying why, a repeated - # prefix previews its branch twice, or live runs gain the warning. - setup_discovery_repository - add_discovery_branch "state/staging/alpha" - DISCOVERY_ALLOWED_PREFIXES=$'state/\nstate/staging/\nstate/\naws-state/100%/' - local output stderr_file="$DISCOVERY_TMP_ROOT/unowned.stderr" - - output=$(run_discovery 2>"$stderr_file") - [[ ! -s "$stderr_file" ]] || fail "a live run reported unowned prefixes: $(<"$stderr_file")" - - DISCOVERY_PREVIEW=true - DISCOVERY_CALLER_REF="refs/pull/42/merge" - output=$(run_discovery 2>"$stderr_file") - jq -e '.include | map(.branch) == ["state/staging/alpha"]' <<<"$output" >/dev/null \ - || fail "a preview did not sample the one owned branch exactly once: $output" - [[ "$(<"$stderr_file")" == "::warning::branch prefix 'state/staging/' owns no branch to preview"$'\n'"::warning::branch prefix 'aws-state/100%25/' owns no branch to preview" ]] \ - || fail "a preview did not name each prefix that owns no branch once: $(<"$stderr_file")" -} - - -test_discovery_preview_keeps_the_live_matrix_limit() { - # Production break caught: a policy the live run refuses for exceeding the matrix limit - # previews green because the sample itself is small. - setup_discovery_repository - add_numbered_discovery_branches 0 256 - DISCOVERY_ALLOWED_PREFIXES="state/limit/" - DISCOVERY_PREVIEW=true - DISCOVERY_CALLER_REF="refs/pull/42/merge" - assert_discovery_failure "discovery matrix error:" "a preview of more than 256 branches" -} -``` - -- [ ] **Step 2: Run them and watch them fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_discovery_preview_samples_one_branch_per_prefix test_discovery_preview_warns_for_prefixes_that_own_no_branch test_discovery_preview_keeps_the_live_matrix_limit` - -Expected: -- The first fails with `pull request 42 did not sample its known branches`, because all six branches are returned. -- The third already passes: the limit check runs before any sampling exists. It stays as the regression guard for the ordering in Step 3. - -- [ ] **Step 3: Implement sampling** - -In `discover-state-branches.sh`: -- Change `branch_records=()` (just before the `while IFS=$'\t' read -r oid ref` loop) to also declare the owner map: - -```bash -branch_records=() -declare -A branch_owners=() -``` - -- Inside the prefix loop, record the owning prefix next to the record: - -```bash - for prefix in "${selection_prefixes[@]}"; do - if [[ "$branch" == "$prefix"* ]]; then - branch_records+=("$branch"$'\t'"$oid") - branch_owners[$branch]=$prefix - break - fi - done -``` - -- Define this function directly after `branch_is_excluded()`: - -```bash -# Keeps one branch per prefix, so a preview processes a sample rather than the whole policy. A -# branch belongs to the first prefix it matches; the pick is the first 15 hex digits of -# sha256("\t") modulo the prefix's branch count, which stays within bash's signed -# 64-bit arithmetic and is stable while that prefix's branches are unchanged. -sample_branch_records() { - local prefix record digest - local -a owned sampled=() - declare -A seen_prefixes=() - for prefix in "${selection_prefixes[@]}"; do - [[ -z "${seen_prefixes[$prefix]-}" ]] || continue - seen_prefixes[$prefix]=1 - owned=() - for record in "${branch_records[@]}"; do - [[ "${branch_owners[${record%%$'\t'*}]}" != "$prefix" ]] || owned+=("$record") - done - if [[ ${#owned[@]} -eq 0 ]]; then - echo "::warning::branch prefix '${prefix//'%'/%25}' owns no branch to preview" >&2 - continue - fi - digest=$(printf '%s\t%s' "$preview_seed" "$prefix" | sha256sum) - sampled+=("${owned[$((16#${digest:0:15} % ${#owned[@]}))]}") - done - readarray -t branch_records < <(printf '%s\n' "${sampled[@]}" | sort) -} -``` - -- Directly after the existing line `readarray -t branch_records < <(printf '%s\n' "${branch_records[@]}" | sort)`, add: - -```bash -[[ "$preview" != true ]] || sample_branch_records -``` - -This order keeps the selection and 256-branch checks on the full matched set. It also means `owned` is built from sorted records. - -`sampled` is never empty here. Selection already failed if nothing matched, and every matched record has an owner. - -- [ ] **Step 4: Run the tests and watch them pass** - -Run the Step 2 command again. Expected: three `PASS:` lines and nothing else. - -Then run every discovery test by name, as in Task 1 Step 5, plus `examples/github-actions/report-test.sh`. Expected: all `PASS:`. - -- [ ] **Step 5: Shellcheck and commit** - -Run: `make shellcheck`. Expected: no output. - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/discover-state-branches.sh examples/github-actions/test.sh -``` - -Commit message: `feat(actions-example): sample one state branch per prefix in preview discovery`. - ---- - -### Task 3: Publication guard - -**Files:** -- Modify: `examples/github-actions/.github/scripts/reconcile-state-branch.sh` (`usage()`, new `require_default_branch_caller`, first line of `publish_result`) -- Modify: `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml` (`Publish processing result` step `env`) -- Modify: `examples/github-actions/reconcile-test.sh` (`run_publish`, new test, test list) -- Test: `examples/github-actions/test.sh` (`test_workflow_runs_three_jobs_with_current_attempt_results`) - -**Interfaces:** -- Consumes: nothing from earlier tasks. -- Produces: - - Required `publish` inputs `RECONCILE_CALLER_REF` and `RECONCILE_DEFAULT_BRANCH`. - - Diagnostic `reconciliation error: publication runs only from refs/heads/`. - -- [ ] **Step 1: Give the reconcile harness a default-branch caller** - -In `reconcile-test.sh` `run_publish`, add two lines after `RECONCILE_RUN_URL=...`: - -```bash - RECONCILE_CALLER_REF=${RECONCILE_CALLER_REF-refs/heads/main} \ - RECONCILE_DEFAULT_BRANCH=main \ -``` - -- [ ] **Step 2: Write the failing reconcile test** - -Add before the `if [[ $# -eq 0 ]]` block in `reconcile-test.sh`: - -```bash -test_publication_runs_only_from_the_default_branch() { - # Production break caught: a preview whose publish gate is lost, or any other run from a - # non-default ref, pushes update refs or changes pull requests and issues from unmerged config. - local caller_ref dry_run remote - for caller_ref in refs/pull/7/merge refs/heads/feature; do - for dry_run in true false; do - setup_success_fixture - remote=$FIXTURE_REMOTE - if RECONCILE_CALLER_REF=$caller_ref RECONCILE_DRY_RUN=$dry_run run_publish \ - >"$FIXTURE_ROOT/stdout" 2>"$FIXTURE_ROOT/stderr"; then - fail "publication from $caller_ref (dry run $dry_run) was accepted" - fi - [[ "$(<"$FIXTURE_ROOT/stderr")" == 'reconciliation error: publication runs only from refs/heads/main' ]] \ - || fail "publication from $caller_ref reported: $(<"$FIXTURE_ROOT/stderr")" - [[ ! -s "$FIXTURE_ROOT/stdout" ]] || fail 'refused publication emitted stdout' - [[ ! -f "$FIXTURE_GH_CAPTURE/calls" ]] || fail "publication from $caller_ref called GitHub" - [[ -z "$("$TEST_GIT" --git-dir "$remote" for-each-ref --format='%(refname)' refs/heads/update_)" ]] \ - || fail "publication from $caller_ref pushed an update ref" - done - done -} -``` - -Append `test_publication_runs_only_from_the_default_branch` to the `tests=(...)` list at the bottom of `reconcile-test.sh`. - -- [ ] **Step 3: Run it and watch it fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/reconcile-test.sh test_publication_runs_only_from_the_default_branch` - -Expected: `FAIL: publication from refs/pull/7/merge (dry run true) was accepted`. - -- [ ] **Step 4: Implement the guard** - -In `reconcile-state-branch.sh`, add after `require_common_identity()`: - -```bash -# Publication must not depend on the publish job's `if` alone: a pull-request run executes the -# workflows it changes, so this refuses every ref but the default branch, dry run or not. -require_default_branch_caller() { - : "${RECONCILE_CALLER_REF:?RECONCILE_CALLER_REF must be set}" - : "${RECONCILE_DEFAULT_BRANCH:?RECONCILE_DEFAULT_BRANCH must be set}" - [[ "$RECONCILE_CALLER_REF" == "refs/heads/$RECONCILE_DEFAULT_BRANCH" ]] \ - || reconcile_error "publication runs only from refs/heads/$RECONCILE_DEFAULT_BRANCH" -} -``` - -Make it the first line of `publish_result()`, before `validate_result`: - -```bash -publish_result() { - require_default_branch_caller - validate_result -``` - -In `usage()`, change `PRs and failure issues. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP,` to: - -```text -PRs and failure issues, only when RECONCILE_CALLER_REF is refs/heads/ followed by -RECONCILE_DEFAULT_BRANCH. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP, -``` - -- [ ] **Step 5: Run the reconcile harness** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/reconcile-test.sh` - -Expected: every test prints `PASS:`, including the new one. - -- [ ] **Step 6: Wire the workflow and pin it with a failing check** - -In `test.sh` `test_workflow_runs_three_jobs_with_current_attempt_results`, add this clause before the closing `'` of its jq program (after the `RECONCILE_TF_VERSION_BUMP_VERSION` clause, joined with `and`): - -```text - and ([.publish.steps[] | select(.env.RECONCILE_RUN_URL != null) - | .env | {RECONCILE_CALLER_REF, RECONCILE_DEFAULT_BRANCH}] - == [{RECONCILE_CALLER_REF: "${{ github.ref }}", - RECONCILE_DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}"}]) -``` - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_runs_three_jobs_with_current_attempt_results` - -Expected: `FAIL: workflow does not wire the three-job current-attempt result contract`. - -In `tf-version-bump-reusable.yml`, in the `Publish processing result` step's `env`, add after `RECONCILE_RUN_URL`: - -```yaml - RECONCILE_CALLER_REF: ${{ github.ref }} - RECONCILE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} -``` - -Run the same test again. Expected: `PASS:`. - -- [ ] **Step 7: Lint and commit** - -Run: `make shellcheck` and `make actionlint`. Expected: no findings. - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/scripts/reconcile-state-branch.sh examples/github-actions/.github/workflows/tf-version-bump-reusable.yml examples/github-actions/reconcile-test.sh examples/github-actions/test.sh -``` - -Commit message: `feat(actions-example): refuse publication from any ref but the default branch`. - ---- - -### Task 4: Reusable workflow preview mode - -**Files:** -- Modify: `examples/github-actions/.github/workflows/tf-version-bump-reusable.yml`: - - `workflow_call.inputs` - - `discover` step `env` - - `publish.if` - - `Report processing result` step -- Test: `examples/github-actions/test.sh`: - - `workflow_publishes_after_processing_failures` and `test_workflow_publishes_every_branch_after_processing_failures` - - `test_workflow_reports_the_processing_result` - - new `test_workflow_previews_without_publishing` - - new `run_preview_report_step` and `test_workflow_summarises_the_preview_patch` - - the `tests=(...)` list - -**Interfaces:** -- Consumes: `DISCOVERY_PREVIEW` (Task 1). -- Produces: - - Reusable input `preview` (boolean, default `false`), which callers pass in Task 5. - - `PREVIEW` env on `Report processing result`. - - Shell function `append_fenced_file ` inside that step. - -- [ ] **Step 1: Write the failing wiring and gate tests** - -In `test.sh`, change the condition in `workflow_publishes_after_processing_failures`: - -```bash - | jq -e --arg condition "\${{ always() && needs.discover.result == 'success' && !inputs.preview }}" ' -``` - -In `test_workflow_publishes_every_branch_after_processing_failures`, add this mutant after the `publish-condition.yml` block: - -```bash - mutant="$TEST_TMP_ROOT/publish-without-preview-gate.yml" - yq ".jobs.publish.if = \"\${{ always() && needs.discover.result == 'success' }}\"" \ - "$REUSABLE_WORKFLOW" >"$mutant" - ! workflow_publishes_after_processing_failures "$mutant" \ - || fail 'the guard passes when a preview can reach publication' -``` - -In `test_workflow_reports_the_processing_result`, extend the first jq check. Change `and .[0].env.RESULT_MANIFEST == $result + "/result.json"` to: - -```text - and .[0].env.RESULT_MANIFEST == $result + "/result.json" - and .[0].env.PREVIEW == "${{ inputs.preview }}" -``` - -Add a new test after `test_workflow_fails_discovery_when_the_script_fails`: - -```bash -test_workflow_previews_without_publishing() { - # Production break caught: callers cannot ask for a preview, or discovery never hears of it and - # processes every branch from a pull request. - yq -o=json '.' "$REUSABLE_WORKFLOW" | jq -e ' - .on.workflow_call.inputs.preview == {type: "boolean", default: false} - and ([.jobs.discover.steps[] | select(.id == "discover") | .env.DISCOVERY_PREVIEW] - == ["${{ inputs.preview }}"]) - ' >/dev/null || fail 'the reusable workflow does not offer a preview input wired into discovery' -} -``` - -Add `test_workflow_previews_without_publishing` to the `tests=(...)` list, after `test_workflow_fails_discovery_when_the_script_fails`. - -- [ ] **Step 2: Run them and watch them fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_publishes_every_branch_after_processing_failures test_workflow_reports_the_processing_result test_workflow_previews_without_publishing` - -Expected: `FAIL: a processing failure cancels other branches or skips their publication`, the gate string no longer matching. Run the other two individually and see them fail too. - -- [ ] **Step 3: Implement the input, discovery wiring and gate** - -In `tf-version-bump-reusable.yml`: -- Under `workflow_call.inputs`, after `dry_run`: - -```yaml - preview: - type: boolean - default: false -``` - -- In the `discover` step `env`, after `DISCOVERY_MANUAL_PREFIX`: - -```yaml - DISCOVERY_PREVIEW: ${{ inputs.preview }} -``` - -- Replace the `publish` job `if`: - -```yaml - if: ${{ always() && needs.discover.result == 'success' && !inputs.preview }} -``` - -- In the `Report processing result` step `env`, after `RESULT_MANIFEST`: - -```yaml - PREVIEW: ${{ inputs.preview }} -``` - -- [ ] **Step 4: Run the Step 2 tests and watch them pass** - -Expected: three `PASS:` lines. - -- [ ] **Step 5: Write the failing patch-summary test** - -Add after `test_workflow_summarises_update_logs` in `test.sh`: - -```bash -run_preview_report_step() { - run_workflow_step process 'Report processing result' "$3" \ - PROCESS_OUTCOME="$2" RESULT_MANIFEST="$1" PREVIEW="$4" -} - - -test_workflow_summarises_the_preview_patch() { - # Production break caught: a preview hides the candidate it produced, a live run's summary - # gains the patch, or backticks or size in the patch break the summary. - local work="$TEST_TMP_ROOT/report-patch" - rm -rf -- "$work" - mkdir -p "$work/logs" - local manifest="$work/result.json" summary="$work/summary.md" diagnostics="$work/stderr" - local report - - write_report_manifest "$manifest" success null '["."]' - printf '%s\n' '- version = "1.0.0"' '+ version = "2.0.0"' '```' >"$work/candidate.patch" - assert_silent_success 'summarising a preview patch' "$work/stdout" "$diagnostics" \ - run_preview_report_step "$manifest" success "$summary" true - report=$(<"$summary") - [[ "$report" == *'#### Candidate patch'* && "$report" == *'+ version = "2.0.0"'* ]] \ - || fail "the preview summary omits the candidate patch: $report" - [[ $(grep -cx '````' "$summary") -eq 2 ]] \ - || fail "the patch's fence does not outlast the backticks inside it: $report" - - assert_silent_success 'summarising a live result' "$work/stdout" "$diagnostics" \ - run_preview_report_step "$manifest" success "$summary" false - [[ "$(<"$summary")" != *'Candidate patch'* ]] \ - || fail 'a live run summary shows the candidate patch' - - awk 'BEGIN { for (i = 1; i <= 1200; i++) printf "+line %04d %060d\n", i, 0 }' \ - >"$work/candidate.patch" - assert_silent_success 'summarising an oversized preview patch' "$work/stdout" "$diagnostics" \ - run_preview_report_step "$manifest" success "$summary" true - [[ "$(<"$summary")" == *$'\n```\n\nThis patch was truncated'* ]] \ - || fail "an oversized patch was not truncated with its closing fence on its own line: $(<"$summary")" - - rm -f -- "$work/candidate.patch" - write_report_manifest "$manifest" no-change - assert_silent_success 'summarising a preview without a patch' "$work/stdout" "$diagnostics" \ - run_preview_report_step "$manifest" success "$summary" true - [[ "$(<"$summary")" != *'Candidate patch'* ]] \ - || fail 'a preview without a candidate patch claims one' -} -``` - -Add `test_workflow_summarises_the_preview_patch` to the `tests=(...)` list, after `test_workflow_summarises_update_logs`. - -- [ ] **Step 6: Run it and watch it fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_summarises_the_preview_patch` - -Expected: `FAIL: the preview summary omits the candidate patch`. - -- [ ] **Step 7: Implement the shared renderer and the patch** - -In the `Report processing result` step's `run:` body, replace everything from `# Only the updater's log is shown: ...` through `done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST")` with: - -```bash - # Appends a file in a fence one backtick longer than any run inside it, truncating past - # 64 KiB at a line boundary so the closing fence stays on a line of its own. - append_fenced_file() { - local heading=$1 file=$2 noun=$3 fence='```' longest - longest=$({ grep -aoE '`+' "$file" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }') - while [[ ${#fence} -le $longest ]]; do fence+='`'; done - printf '\n#### %s\n\n%s\n' "$heading" "$fence" >>"$GITHUB_STEP_SUMMARY" - if [[ $(wc -c <"$file") -gt 65536 ]]; then - head -c 65536 "$file" | sed '$d' >>"$GITHUB_STEP_SUMMARY" - printf '%s\n\nThis %s was truncated; the processing artefact holds it in full.\n' "$fence" "$noun" >>"$GITHUB_STEP_SUMMARY" - else - awk 1 "$file" >>"$GITHUB_STEP_SUMMARY" - printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY" - fi - } - # Only the updater's log is shown: Terraform's logs can carry a credential a provider echoed. - index=0 - while IFS= read -r root; do - index=$((index + 1)) - log="${RESULT_MANIFEST%/*}/logs/update-$index.log" - [[ -f "$log" ]] || continue - append_fenced_file "Updates in \`$root\`" "$log" log - done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST") - # The patch holds only Terraform files and root lock files, never Terraform's logs. - patch="${RESULT_MANIFEST%/*}/candidate.patch" - if [[ "$PREVIEW" == true && -f "$patch" ]]; then - append_fenced_file 'Candidate patch' "$patch" patch - fi -``` - -`PREVIEW` is unset when the harness's existing `run_report_step` runs the body. The step runs under `bash -e` without `-u`, so `"$PREVIEW" == true` is simply false there. - -- [ ] **Step 8: Run the summary tests** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_workflow_summarises_the_preview_patch test_workflow_summarises_update_logs test_workflow_reports_the_processing_result` - -Expected: three `PASS:` lines. `test_workflow_summarises_update_logs` still passes unchanged, which proves the refactor kept the log rendering byte-for-byte, including `This log was truncated`. - -- [ ] **Step 9: Lint and commit** - -Run: `make actionlint` and `make shellcheck`. actionlint also runs shellcheck over the `run:` body. Expected: no findings. - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/workflows/tf-version-bump-reusable.yml examples/github-actions/test.sh -``` - -Commit message: `feat(actions-example): add a preview mode to the reusable workflow`. - ---- - -### Task 5: Callers preview pull requests - -**Files:** -- Modify: `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` -- Modify: `examples/github-actions/.github/workflows/tf-version-bump-production.yml` -- Test: `examples/github-actions/test.sh` (new `test_callers_preview_pull_requests`, `tests=(...)` list) - -**Interfaces:** -- Consumes: reusable input `preview` (Task 4). -- Produces: caller jobs `automation` and `preview`, and YAML anchors `policy-inputs` and `policy-secrets`. `report-test.sh` and `test_workflow_offers_input_and_secret_terraform_environment` keep reading `.jobs.automation.with` and `.jobs.automation.secrets`, which stay in place. - -- [ ] **Step 1: Write the failing test** - -Add after `test_workflow_previews_without_publishing` in `test.sh`: - -```bash -test_callers_preview_pull_requests() { - # Production break caught: a preview runs with inputs that differ from its live run, queues - # behind or blocks live runs, previews a fork or a non-default base, narrows its permissions so - # GitHub refuses to start it, or a live run starts from a pull request. - local policy workflow - for policy in nonproduction production; do - workflow="$SCRIPT_DIR/.github/workflows/tf-version-bump-$policy.yml" - yq -o=json '.' "$workflow" | jq -e --arg policy "$policy" ' - .jobs as $jobs - | (has("concurrency") | not) - and .on.pull_request == {paths: [$jobs.automation.with.config_path, - ".github/workflows/tf-version-bump-\($policy).yml"]} - and ($jobs | keys) == ["automation", "preview"] - and $jobs.automation.if == "${{ github.ref == format('"'"'refs/heads/{0}'"'"', github.event.repository.default_branch) }}" - and $jobs.automation.concurrency == {group: "tf-version-bump-\($policy)-${{ github.repository_id }}", - "cancel-in-progress": false, queue: "max"} - and $jobs.preview.if == "${{ github.event_name == '"'"'pull_request'"'"' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }}" - and $jobs.preview.concurrency == {group: "tf-version-bump-\($policy)-preview-${{ github.repository_id }}-${{ github.event.number }}", - "cancel-in-progress": true} - and ($jobs.preview | has("permissions") | not) - and $jobs.preview.uses == $jobs.automation.uses - and $jobs.preview.with == $jobs.automation.with - and $jobs.preview.secrets == $jobs.automation.secrets - and $jobs.automation.with.preview == "${{ github.event_name == '"'"'pull_request'"'"' }}" - ' >/dev/null || fail "the $policy caller does not preview pull requests with its live inputs" - # The aliases keep one copy of each policy value, so a pull request's edits reach its preview. - grep -qxF ' with: *policy-inputs' "$workflow" \ - && grep -qxF ' secrets: *policy-secrets' "$workflow" \ - || fail "the $policy caller's preview does not reuse the live job's inputs by alias" - done -} -``` - -Add `test_callers_preview_pull_requests` to the `tests=(...)` list, after `test_workflow_previews_without_publishing`. - -- [ ] **Step 2: Run it and watch it fail** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_callers_preview_pull_requests` - -Expected: `FAIL: the nonproduction caller does not preview pull requests with its live inputs`. - -- [ ] **Step 3: Rewrite the non-production caller** - -Replace `examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml` with the following. Everything under `with:` except the new `preview:` line is copied unchanged from the current file: - -```yaml -name: Terraform version bump (non-production) - -on: - push: - paths: - - .github/tf-version-bump/nonproduction.yml - pull_request: - paths: - - .github/tf-version-bump/nonproduction.yml - - .github/workflows/tf-version-bump-nonproduction.yml - workflow_dispatch: - inputs: - branch_prefix: - description: Optional literal prefix that narrows the configured branch policy - type: string - default: "" - dry_run: - description: Run all local stages without publishing Git or GitHub changes - type: boolean - default: false - terraform_init_upgrade: - description: Upgrade dependencies within their version constraints during preparation - type: boolean - default: false - schedule: - - cron: "17 4 * * 1" - timezone: Australia/Melbourne - -permissions: - contents: write - pull-requests: write - issues: write - -jobs: - automation: - if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} - concurrency: - group: tf-version-bump-nonproduction-${{ github.repository_id }} - cancel-in-progress: false - queue: max - uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: &policy-secrets - TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} - TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} - with: &policy-inputs - automation_policy_id: nonproduction - allowed_branch_prefixes: | - state/nonproduction/ - state/staging/ - aws-state/nonproduction/ - aws-state/staging/ - branch_prefix: ${{ github.event_name == 'workflow_dispatch' && inputs.branch_prefix || '' }} - config_path: .github/tf-version-bump/nonproduction.yml - terraform_directories: . - terraform_fmt: true - terraform_init_upgrade: ${{ github.event_name == 'workflow_dispatch' && inputs.terraform_init_upgrade || false }} - terraform_version: 1.15.5 - tf_version_bump_version: v1.0.0-rc.17 - tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d - dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} - preview: ${{ github.event_name == 'pull_request' }} - max_parallel: 4 - commit_author_name: github-actions[bot] - commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com - # Previews a same-repository pull request into the default branch on one sampled branch per - # prefix, without publishing. The aliases give it the live job's inputs, including any the pull - # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects - # both settings in one concurrency block. It keeps the workflow's permissions because GitHub - # refuses to start a read-only caller of a workflow whose publish job requests write. - preview: - if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} - concurrency: - group: tf-version-bump-nonproduction-preview-${{ github.repository_id }}-${{ github.event.number }} - cancel-in-progress: true - uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: *policy-secrets - with: *policy-inputs -``` - -- [ ] **Step 4: Rewrite the production caller** - -Replace `examples/github-actions/.github/workflows/tf-version-bump-production.yml` with: - -```yaml -name: Terraform version bump (production) - -on: - push: - paths: - - .github/tf-version-bump/production.yml - pull_request: - paths: - - .github/tf-version-bump/production.yml - - .github/workflows/tf-version-bump-production.yml - workflow_dispatch: - inputs: - branch_prefix: - description: Optional literal prefix that narrows the configured branch policy - type: string - default: "" - dry_run: - description: Run all local stages without publishing Git or GitHub changes - type: boolean - default: false - terraform_init_upgrade: - description: Upgrade dependencies within their version constraints during preparation - type: boolean - default: false - schedule: - - cron: "43 4 * * 0" - timezone: Australia/Melbourne - -permissions: - contents: write - pull-requests: write - issues: write - -jobs: - automation: - if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }} - concurrency: - group: tf-version-bump-production-${{ github.repository_id }} - cancel-in-progress: false - queue: max - uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: &policy-secrets - TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }} - TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }} - with: &policy-inputs - automation_policy_id: production - allowed_branch_prefixes: | - state/production/ - aws-state/production/ - branch_prefix: ${{ github.event_name == 'workflow_dispatch' && inputs.branch_prefix || '' }} - config_path: .github/tf-version-bump/production.yml - terraform_directories: . - terraform_fmt: true - terraform_init_upgrade: ${{ github.event_name == 'workflow_dispatch' && inputs.terraform_init_upgrade || false }} - terraform_version: 1.15.5 - tf_version_bump_version: v1.0.0-rc.17 - tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d - dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }} - preview: ${{ github.event_name == 'pull_request' }} - max_parallel: 2 - commit_author_name: github-actions[bot] - commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com - # Previews a same-repository pull request into the default branch on one sampled branch per - # prefix, without publishing. The aliases give it the live job's inputs, including any the pull - # request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects - # both settings in one concurrency block. It keeps the workflow's permissions because GitHub - # refuses to start a read-only caller of a workflow whose publish job requests write. - preview: - if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }} - concurrency: - group: tf-version-bump-production-preview-${{ github.repository_id }}-${{ github.event.number }} - cancel-in-progress: true - uses: ./.github/workflows/tf-version-bump-reusable.yml - secrets: *policy-secrets - with: *policy-inputs -``` - -Before replacing each file, diff its current `with:` block against the block above (`git diff` after the edit). Only the `preview:` line may differ. - -- [ ] **Step 5: Run the caller tests and everything that reads the callers** - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_callers_preview_pull_requests test_workflow_offers_input_and_secret_terraform_environment` - -Expected: two `PASS:` lines. - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/report-test.sh`. Expected: all `PASS:`. Its report matrix still matches `.jobs.automation.with`. - -Run: `go test -C /Users/dan/Code/tf-version-bump -run 'TestUpdateActionsReleasePin' -v ./...`. Expected: `ok` with every `TestUpdateActionsReleasePin*` test passing. These are in `release_workflow_test.go`, and together they show the release-pin updater still finds exactly one pin line per caller. - -- [ ] **Step 6: Lint and commit** - -Run: `make actionlint`. Expected: no findings. The launcher already ignores actionlint 1.7.12's stale `queue` diagnostic. The spec's probe confirmed the job-level form passes. - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/.github/workflows/tf-version-bump-nonproduction.yml examples/github-actions/.github/workflows/tf-version-bump-production.yml examples/github-actions/test.sh -``` - -Commit message: `feat(actions-example): preview config changes on pull requests`. - ---- - -### Task 6: Documentation - -**Files:** -- Modify: `examples/github-actions/README.md`: - - line 35 (Install) - - line 50 (secret) - - line 78 (Environments) - - line 89 (Configure updates) - - line 167 (Run and inspect) - - a new section before `## Version report` -- Modify: `docs/ADVANCED-USAGE.md:9` - -**Interfaces:** -- Consumes: the behaviour from Tasks 1–5. -- Produces: the anchor `#preview-on-pull-requests`. - -- [ ] **Step 1: Edit the existing passages** - -In `examples/github-actions/README.md`: -- **Line 35.** Replace `Both callers run only from the default branch.` with: - - ```markdown - Both callers' live runs start only from the default branch; each caller also previews same-repository pull requests into the default branch (see [Preview on pull requests](#preview-on-pull-requests)). - ``` -- **Line 50.** Append this sentence: `Pull-request previews run the same processing job, so they receive the same token.` -- **Line 78.** Append this sentence: - - ```markdown - An Environment's deployment-branch rules also apply to previews, which run from `refs/pull//merge`, so a rule that admits only the default branch may withhold the secret from a preview or hold it for approval. - ``` -- **Line 89.** Replace `Pull requests changing these files run a read-only config validation check; they do not process state branches or run Terraform.` with: - - ```markdown - Pull requests changing these files run a read-only config validation check and, for same-repository pull requests into the default branch, a preview that processes one sampled state branch per prefix without publishing anything (see [Preview on pull requests](#preview-on-pull-requests)). - ``` -- **Line 167.** After the sentence ending `kept in full in the artefact.`, insert: `In a preview, the summary also shows each changed branch's candidate patch, truncated the same way.` - -- [ ] **Step 2: Add the new section** - -Insert before `## Version report`: - -```markdown -## Preview on pull requests - -A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests from forks and pull requests into another branch skip the preview. - -The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. It also runs the pull request's own scripts and reusable workflow, so a pull request that edits those is previewed with the edited versions. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. - -The branch for each prefix is picked from the pull request number, so every push to the same pull request previews the same branches while that prefix's branches are unchanged. Adding, deleting or excluding a branch under a prefix may move the pick, and different pull requests may land on the same branch. A branch belongs to the first prefix it matches, and a prefix that owns no branch is named in a warning annotation. The preview is a smoke test, not coverage: a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled. - -Each `process` job's summary shows the updater's log and, for a changed candidate, the candidate patch. The patch is the full candidate the merged configuration would produce, not only what the pull request changes: on a branch whose update pull request has not been merged, it also includes the changes the current configuration would already make. - -A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, and with it the pull request's check. The failure may already exist on that branch, so check the branch's marked failure issue before blaming the pull request, and do not make the preview a required check. An invalid configuration fails every sampled branch at the update stage as `branch-update`, while the configuration validation check reports the cause. - -GitHub runs pull-request workflows only for the `opened`, `synchronize` and `reopened` activities, and not at all while the pull request has a merge conflict. A preview is therefore not refreshed when the default branch or the state branches move; push to the pull request or re-run the workflow to refresh it. -``` - -- [ ] **Step 3: Edit `docs/ADVANCED-USAGE.md` line 9** - -Replace `callers that run only from the default branch, plus config-change triggers, read-only pull-request config validation,` with `callers whose live runs start only from the default branch, plus config-change triggers, read-only pull-request config validation, pull-request previews on one sampled state branch per prefix,`. - -Replace this sentence: - -```markdown -The pull-request check validates only the control config with `tf-version-bump`; full state-branch dry runs remain deferred. -``` - -with: - -```markdown -The pull-request config check validates only the control config with `tf-version-bump`; the preview processes a sample of state branches and publishes nothing. -``` - -- [ ] **Step 4: Run the documentation checks** - -Run: `make -C /Users/dan/Code/tf-version-bump docs-check` - -Expected: `ok` with every test passing. `TestDocumentationLocalLinksResolve` resolves `#preview-on-pull-requests`, and `TestDocumentationProseIsNotHardWrapped` passes. - -Run: `TEST_GIT=/Users/dan/.claude/bin/claude-git examples/github-actions/test.sh test_readme_documents_the_reserved_environment_names`. Expected: `PASS:`. - -- [ ] **Step 5: Commit** - -```bash -git -C /Users/dan/Code/tf-version-bump add examples/github-actions/README.md docs/ADVANCED-USAGE.md -``` - -Commit message: `docs(actions-example): document pull-request previews`. - ---- - -### Task 7: Full validation and test cleanup - -**Files:** none new. This task verifies the branch and prunes tests. - -- [ ] **Step 1: Run the full example harness (needs Docker)** - -Run: `make -C /Users/dan/Code/tf-version-bump test-github-actions TEST_GIT=/Users/dan/.claude/bin/claude-git` - -Expected: every test prints `PASS:`, then the reconcile and report harnesses print theirs, and nothing else. - -- [ ] **Step 2: Run the repository's CI-equivalent validation** - -Run each and expect success with no warnings: - -```bash -go -C /Users/dan/Code/tf-version-bump mod verify -go test -C /Users/dan/Code/tf-version-bump -race -coverprofile=coverage.out -covermode=atomic ./... -golangci-lint run --timeout=5m -make -C /Users/dan/Code/tf-version-bump actionlint -make -C /Users/dan/Code/tf-version-bump shellcheck -make -C /Users/dan/Code/tf-version-bump docs-check -``` - -Run `golangci-lint` from the repository root. Remove `coverage.out` afterwards with `make -C /Users/dan/Code/tf-version-bump clean`. - -- [ ] **Step 3: Run the test-cleanup pass as a separate subagent** - -Dispatch a fresh subagent with the `test-cleanup` skill against `git diff main...HEAD -- examples/github-actions/test.sh examples/github-actions/reconcile-test.sh`. The implementer must not do this pass. Apply only removals it justifies, re-run Step 1, and commit with message `test(actions-example): trim preview tests`. - -- [ ] **Step 4: Report** - -Tell Dan: -- which checks passed; -- anything that failed, with its output; -- that the branch `wip/config-change-preview` is ready for review. - -Do not push without his confirmation. diff --git a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md b/docs/superpowers/specs/2026-09-22-config-change-preview-design.md deleted file mode 100644 index 0078c74..0000000 --- a/docs/superpowers/specs/2026-09-22-config-change-preview-design.md +++ /dev/null @@ -1,146 +0,0 @@ -# Config-change preview for the GitHub Actions example - -## Goal - -When a pull request changes a policy's control configuration or its caller workflow, the GitHub Actions example previews the result on a sample of real state branches: one branch per configured branch prefix, processed exactly as a live run processes it, with nothing published. The reviewer sees in the run summary the candidate that the merged configuration would produce on those branches before the pull request is merged. - -The preview is a smoke test, not coverage. It shows the full candidate the merged configuration would produce on each sampled branch, which on a branch that lags the default branch's configuration includes changes the current configuration would already make. It does not isolate the pull request's own delta. - -## Scope and assumptions - -- The consuming repository is never forked; pull requests come from branches in the same repository, so the `TF_API_TOKEN` and `TERRAFORM_ENV` secrets are available to them. A pull request into a branch other than the default branch skips the preview; one from a fork is skipped by the preview job's `if`, although GitHub may instead fail it at startup because its token is read-only (unverified). -- Anyone who can open such a pull request can already edit the workflows, so running Terraform with the registry token at pull-request time adds no new exposure. Accidental publication from an unmerged pull request is a new failure mode, so publication gains an independent guard (see [Publication guard](#publication-guard)). -- A pull-request run executes the merge commit's workflows and scripts. A pull request that also edits the scripts or the reusable workflow is previewed with the edited versions. -- Each caller previews only its own policy, triggered by its own configuration file or its own caller workflow file. A change to the scripts or the reusable workflow alone does not trigger a preview. -- Merging still starts the live run through each caller's existing `push` trigger, which remains limited to the configuration file. A merged change to the caller workflow alone takes effect at the next scheduled or manual run, as today. The config-validation workflow is unchanged. - -## Verified GitHub Actions behaviour - -The design depends on GitHub Actions behaviours that the documentation leaves open. Each was checked in a disposable private repository (`yesdevnull/tfvb-preview-probe`) on 2026-09-22 with sleep-only jobs that mirror the caller shape below. The probe's workflows also pass the pinned actionlint 1.7.12, and `yq` resolves their aliases. - -1. **Job-level `queue: max` works.** GitHub's documentation describes `queue` only for workflow-level `concurrency`. Three dispatches of a job carrying `concurrency: {group, cancel-in-progress: false, queue: max}` that calls a reusable workflow ran one after another, each starting after the previous finished, and none was cancelled. The default `queue: single` would have cancelled the middle one while it was pending. -2. **A read-only preview job fails at startup.** A caller job declaring `permissions: contents: read` that calls a workflow whose `publish` job requests write permissions fails with `startup_failure`, even though `preview: true` means `publish` would be skipped. GitHub checks a called job's permissions whatever its `if`. The preview job therefore keeps the caller's permissions, and publication relies on two guards: the `publish` job's `if` and the [publication guard](#publication-guard). -3. **Job-level `cancel-in-progress: true` cancels an older preview.** On a pull request, a second push cancelled the in-progress preview for the first push, and the second run completed. In that run the live job and `publish` were both skipped, and the called workflow received `preview=true` through the aliased `with:` block. - -GitHub's documentation confirms that `queue: max` combined with `cancel-in-progress: true` is a validation error, so the two are never combined in one `concurrency` block. It also confirms YAML anchors and aliases are supported in workflows. It says nothing about merge keys (`<<`), so the design does not use them. - -## Design - -### Callers - -Each of `tf-version-bump-nonproduction.yml` and `tf-version-bump-production.yml` changes as follows: - -- **Triggers.** Adds `pull_request: paths:` naming its own configuration file and its own workflow file. The `push`, `workflow_dispatch` and `schedule` triggers are unchanged. -- **Workflow-level concurrency.** Removed. It moves to the live job. -- **Two jobs.** Both call the reusable workflow, with the same `with:` and `secrets:` blocks. The live job defines them as anchors (`with: &policy-inputs`, `secrets: &policy-secrets`) and the preview job uses the aliases (`with: *policy-inputs`, `secrets: *policy-secrets`). Every policy value is therefore written once, and a pull request that edits a value, such as a tool pin, a prefix or a root, is previewed with it. - - `automation`, the live job: its existing `if` (`github.ref` is `refs/heads/`) is unchanged, because a pull-request run's ref is `refs/pull//merge` and never matches; it gains job-level `concurrency: {group: tf-version-bump--, cancel-in-progress: false, queue: max}`. Its behaviour is otherwise unchanged. - - `preview`: `if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch`, with job-level `concurrency: {group: tf-version-bump--preview--, cancel-in-progress: true}`. It declares no job-level `permissions`, because a read-only caller job fails at startup (see [Verified GitHub Actions behaviour](#verified-github-actions-behaviour)). -- **The `preview` input.** The shared `with:` block includes `preview: ${{ github.event_name == 'pull_request' }}`, which is `false` for the live job and `true` for the preview job without either block differing. `branch_prefix`, `dry_run` and `terraform_init_upgrade` keep their existing event-guarded expressions, which evaluate to `''`, `false` and `false` on pull-request events. - -A newer push to a pull request cancels that pull request's older preview, and a preview never waits behind or blocks a live run. - -### Reusable workflow - -- **New input.** `preview` (boolean, default `false`). -- **Discovery.** The discovery step passes `DISCOVERY_PREVIEW: ${{ inputs.preview }}`. -- **Publication.** The `publish` job's `if` becomes `${{ always() && needs.discover.result == 'success' && !inputs.preview }}`. -- **Summary.** The `Report processing result` step receives `PREVIEW: ${{ inputs.preview }}` through `env`, because the local harness runs step bodies without evaluating expressions. When `PREVIEW` is `true` and `candidate.patch` exists, the step appends it to the job summary under `#### Candidate patch`. The patch exists only for a changed, validated candidate, so there is none for failures or no-change results. -- **Shared rendering.** The existing fence-escalation and 64 KiB truncation code becomes one shell function in the step, `append_fenced_file `, used for each updater log (`log`) and for the patch (`patch`); the noun names the file in the truncation notice. The patch holds only `.tf` changes, formatting changes and root lock files (`write_candidate`, `process-state-branch.sh`), never Terraform's logs, so it cannot carry a credential Terraform echoed. - -### Discovery: sampling - -`discover-state-branches.sh` gains one optional input, `DISCOVERY_PREVIEW`. Unset or `false` means discovery behaves exactly as it does today, including for the report workflow, which does not set it. Any value other than exact `true` or `false` is refused. - -When it is `true`: - -- **Guard.** The caller ref must match `^refs/pull/([1-9][0-9]*)/merge$`, instead of `refs/heads/`. The captured pull request number is the sampling seed, so seed and ref cannot disagree. A pull-request ref without `DISCOVERY_PREVIEW=true` is still refused. -- **No manual prefix.** A non-empty `DISCOVERY_MANUAL_PREFIX` is refused, because a manual prefix only arises from `workflow_dispatch`. -- **Selection.** Prefix validation, exclusions and matching are unchanged. Each branch belongs to the first prefix it matches, as the existing loop assigns it. -- **Branch limit.** The 256-branch limit is checked against the full matched set before sampling, so a policy the live run would refuse fails its preview too. -- **Sampling.** For each prefix that owns at least one branch, its branches are sorted (the script already sets `LC_ALL=C`), and one is picked. The index is the first 15 hexadecimal digits of `sha256("\t")`, read as an integer, modulo the number of branches the prefix owns. Fifteen digits stay within bash's signed 64-bit arithmetic, so the index is never negative. -- **Stability.** The pick is deterministic while the prefix's set of branches is unchanged. Adding, deleting or excluding a branch under the prefix may move it. Different pull requests are spread across a prefix's branches but may land on the same one. -- **Prefixes that own no branch.** In preview mode only, each such prefix produces `::warning::branch prefix '' owns no branch to preview` on stderr, escaped as the existing exclusion warning is. A prefix owns no branch when none of its matches remain after exclusions and earlier prefixes. If no branch is matched at all, the existing "no remote branches matched" failure applies. -- **Output.** The matrix JSON shape is unchanged, so the `process` job consumes it unmodified. `prepare_contract` checks the control checkout's `HEAD` against `control_oid`, which is the merge commit in a pull-request run. - -### Publication guard - -`reconcile-state-branch.sh publish` gains required inputs `RECONCILE_CALLER_REF` (`${{ github.ref }}`) and `RECONCILE_DEFAULT_BRANCH` (`${{ github.event.repository.default_branch }}`). Before any Git or GitHub operation, it refuses to publish unless the caller ref is `refs/heads/`, dry run or not. Publication is then guarded twice: by the `publish` job's `if` and by this check. Discovery does not claim to know whether a run can publish. - -### Failure behaviour - -A sampled branch whose update, initialisation, formatting or validation fails fails its `process` job, as it does today, so the pull request's check fails. The summary names the branch, classification, failed stage and root as it does now. - -Because the sample is stable per pull request, a branch that already fails on the default branch's configuration fails every push and rerun of the preview. The README says that a red preview may be pre-existing, points at the branch's marked failure issue, and advises against making the preview a required check. An invalid configuration fails every sampled branch at the update stage with classification `branch-update`, while the config-validation workflow reports the cause. - -## Testing - -Following TDD, in `examples/github-actions/test.sh` unless noted. - -**Discovery** - -- A known-answer case: a fixed pull request number, prefix and branch list select a named branch, pinning the formula. -- One branch per prefix is selected, and exclusions are applied before sampling. -- A prefix that owns no branch warns in preview mode and not in live mode. The run continues when another prefix selects a branch, and fails when none does. -- The 256-branch limit fails a preview whose full matched set exceeds it. -- A pull-request ref is accepted with `DISCOVERY_PREVIEW=true` and refused without it. A malformed ref (`refs/pull/0/merge`, `refs/pull/7/head`) is refused, as are a non-boolean `DISCOVERY_PREVIEW` and a manual prefix in preview mode. -- The default-branch path and its matrix output are unchanged. - -**Reusable workflow** - -- Replace the exact `.publish.if` assertion in `workflow_publishes_after_processing_failures` with one requiring `always()`, `needs.discover.result == 'success'` and `!inputs.preview`. The existing mutants that drop `always()` still fail, and a new mutant that drops `!inputs.preview` fails too. -- `DISCOVERY_PREVIEW` is wired from `inputs.preview`, and `PREVIEW` reaches `Report processing result` through `env`. -- The summary step, run through `run_workflow_step`: - - appends the patch in preview mode when `candidate.patch` exists; - - omits it outside preview mode and when the file is absent; - - escalates the fence past backticks in the patch; - - truncates past 64 KiB with the existing notice. - -**Callers** - -- Each caller has the pull-request trigger for its own configuration and workflow file. -- The live job's `if` refuses pull-request events and non-default refs. -- The preview job's `if` requires a same-repository head and a default-branch base. -- The live job has job-level `queue: max` without `cancel-in-progress: true`, and the preview job has `cancel-in-progress: true` without `queue`. -- Both jobs resolve to identical `with:` and `secrets:` blocks, and the shared `preview` expression is event-derived. - -**Publication guard** (`reconcile-test.sh`) - -- Publication is refused for a pull-request ref and for another branch, and in dry-run mode too. -- The refusal happens before any Git or GitHub command. -- The existing default-branch cases pass with the new inputs. - -A separate test-cleanup pass follows implementation. - -## Documentation - -Add a README section, "Preview on pull requests", that covers: -- what runs, and that nothing is published; -- how the sample is chosen and why it is stable; -- that the patch is the full candidate the merged configuration would produce, not the pull request's delta; -- that a failed sample fails the check and may be pre-existing, and that the check should not be required; -- that pull requests with a merge conflict run no preview, because GitHub does not run `pull_request` workflows for them; -- that the preview is not refreshed when the base or state branches move, only on `opened`, `synchronize` and `reopened`; -- that a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled; -- that a pull request editing the scripts is previewed with the edited scripts; -- that an invalid configuration appears as `branch-update` on every sampled branch. - -Edit the passages the change makes false: -- `examples/github-actions/README.md`, the Install section's "Both callers run only from the default branch." The live job still does; the preview job runs for same-repository pull requests into the default branch. -- `examples/github-actions/README.md`, the Configure updates section's "Pull requests changing these files run a read-only config validation check; they do not process state branches or run Terraform." -- `examples/github-actions/README.md`, the Run and inspect section's description of the `process` summary, which gains the patch in a preview. -- `docs/ADVANCED-USAGE.md`, the GitHub Actions paragraph's "callers that run only from the default branch" and "full state-branch dry runs remain deferred". - -Also in the README: -- The Install section notes that previews use the same secrets. -- The environment-variable section notes that a GitHub Environment with deployment-branch rules may withhold its secrets from `refs/pull//merge`, which fails or holds the preview. - -## Size - -Roughly: -- 40 lines of discovery shell; -- 10 lines of publication guard; -- 30 lines of caller and reusable workflow YAML, net of the shared rendering function; -- a comparable amount of harness tests; -- one README section, plus four passage edits. - -This is in keeping with the example's brevity. From 340937e8b0722735f4da9563bdbc11614ddc76c0 Mon Sep 17 00:00:00 2001 From: Dan Barrett Date: Tue, 22 Sep 2026 15:03:40 +1000 Subject: [PATCH 14/14] fix(actions-example): address PR review of the preview Fail preview discovery when sampling loses its branches: a failing sort inside a process substitution is invisible to set -e, and an empty matrix runs as zero jobs. Run the processing report step under shell: bash so pipefail catches a failed read in its summary pipelines. Correct the README: Dependabot pull requests do run the preview but receive Dependabot secrets rather than Actions secrets, and editing the scripts or reusable workflow alone does not start a preview. Make the reconcile usage text say publish refuses everything from another ref. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ --- .../scripts/discover-state-branches.sh | 4 ++++ .../.github/scripts/reconcile-state-branch.sh | 9 +++---- .../workflows/tf-version-bump-reusable.yml | 1 + examples/github-actions/README.md | 4 ++-- examples/github-actions/test.sh | 24 ++++++++++++++++++- 5 files changed, 35 insertions(+), 7 deletions(-) diff --git a/examples/github-actions/.github/scripts/discover-state-branches.sh b/examples/github-actions/.github/scripts/discover-state-branches.sh index bdf4110..6b667c5 100755 --- a/examples/github-actions/.github/scripts/discover-state-branches.sh +++ b/examples/github-actions/.github/scripts/discover-state-branches.sh @@ -94,6 +94,10 @@ sample_branch_records() { sampled+=("${owned[$((16#${digest:0:15} % ${#owned[@]}))]}") done readarray -t branch_records < <(printf '%s\n' "${sampled[@]}" | sort) + # A failing sort inside the process substitution is not seen by set -e, and an empty matrix + # runs as zero jobs, so the preview would pass having previewed nothing. + [[ ${#sampled[@]} -gt 0 && ${#branch_records[@]} -eq ${#sampled[@]} ]] \ + || fail_discovery selection "preview sampling did not keep one branch per owning prefix" } : "${DISCOVERY_DEFAULT_BRANCH:?DISCOVERY_DEFAULT_BRANCH must be set}" diff --git a/examples/github-actions/.github/scripts/reconcile-state-branch.sh b/examples/github-actions/.github/scripts/reconcile-state-branch.sh index 9ab1a45..b362a09 100755 --- a/examples/github-actions/.github/scripts/reconcile-state-branch.sh +++ b/examples/github-actions/.github/scripts/reconcile-state-branch.sh @@ -15,10 +15,11 @@ the classification. Requires RECONCILE_RESULT_DIR and RECONCILE_RUN_ID, RECONCILE_RUN_ATTEMPT, RECONCILE_AUTOMATION_POLICY_ID, RECONCILE_CONTROL_OID, RECONCILE_STATE_BRANCH, RECONCILE_BASE_OID, RECONCILE_REF_HASH. -publish additionally checks the clean exact-base RECONCILE_TARGET_CHECKOUT and -RECONCILE_TERRAFORM_ROOTS, creates one owned update commit, and reconciles marked -PRs and failure issues, only when RECONCILE_CALLER_REF is refs/heads/ followed by -RECONCILE_DEFAULT_BRANCH. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP, +publish refuses to do anything, dry run or not, unless RECONCILE_CALLER_REF is +refs/heads/ followed by RECONCILE_DEFAULT_BRANCH. It then additionally checks the clean +exact-base RECONCILE_TARGET_CHECKOUT and RECONCILE_TERRAFORM_ROOTS, creates one owned +update commit, and reconciles marked PRs and failure issues. Requires RECONCILE_DRY_RUN +(true/false), RUNNER_TEMP, RECONCILE_RUN_URL, RECONCILE_GIT_REMOTE, RECONCILE_REPOSITORY, GH_TOKEN, RECONCILE_COMMIT_AUTHOR_NAME, RECONCILE_COMMIT_AUTHOR_EMAIL, and the RECONCILE_TERRAFORM_VERSION and RECONCILE_TF_VERSION_BUMP_VERSION their bodies name. diff --git a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml index a3864b4..cb2c340 100644 --- a/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml +++ b/examples/github-actions/.github/workflows/tf-version-bump-reusable.yml @@ -167,6 +167,7 @@ jobs: retention-days: 7 - name: Report processing result if: ${{ always() }} + shell: bash env: PROCESS_OUTCOME: ${{ steps.process.outcome }} RESULT_MANIFEST: ${{ runner.temp }}/result/result.json diff --git a/examples/github-actions/README.md b/examples/github-actions/README.md index c593e88..353f153 100644 --- a/examples/github-actions/README.md +++ b/examples/github-actions/README.md @@ -145,9 +145,9 @@ Publication uses the built-in `GITHUB_TOKEN`. The helper respects Git's signing ## Preview on pull requests -A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests into another branch skip the preview. The preview job also skips pull requests from forks, although GitHub may instead fail such a run, or a Dependabot pull request's, at startup, because their token is read-only. +A same-repository pull request into the default branch that changes a caller's configuration file or the caller workflow itself starts that caller's `preview` job. It discovers the policy's branches exactly as a live run does, then processes one branch per configured prefix, with its update, `terraform init`, optional formatting and `terraform validate`, and publishes nothing: the `publish` job is skipped, and publication also refuses any ref but the default branch. Pull requests into another branch skip the preview. The preview job skips pull requests from forks. A Dependabot pull request that changes a watched file, such as an action pin in a caller, does run the preview, because Dependabot's branches live in this repository; GitHub gives Dependabot runs Dependabot secrets rather than the repository's Actions secrets, so such a preview has no `TF_API_TOKEN` or `TERRAFORM_ENV` unless you also store them as Dependabot secrets. -The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. It also runs the pull request's own scripts and reusable workflow, so a pull request that edits those is previewed with the edited versions. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. +The preview job shares the live job's inputs through YAML aliases, so a pull request that edits a pin, a prefix or a root is previewed with its edits. When a preview runs, it uses the pull request's own scripts and reusable workflow, including any edits to them; editing those files alone does not start a preview. A newer push to the pull request cancels its older preview; previews never wait behind or block a live run. The branch for each prefix is picked from the pull request number, so every push to the same pull request previews the same branches while that prefix's branches are unchanged. Adding, deleting or excluding a branch under a prefix may move the pick, and different pull requests may land on the same branch. A branch belongs to the first prefix it matches, and a prefix that owns no branch is named in a warning annotation. The preview is a smoke test, not coverage: a branch-scoped `ignore_modules` entry is previewed only if its branch happens to be sampled. diff --git a/examples/github-actions/test.sh b/examples/github-actions/test.sh index 7f07c83..5c5c0b0 100755 --- a/examples/github-actions/test.sh +++ b/examples/github-actions/test.sh @@ -834,6 +834,25 @@ test_discovery_preview_keeps_the_live_matrix_limit() { } +test_discovery_preview_fails_when_sampling_loses_its_branches() { + # Production break caught: a failing sort inside a process substitution leaves discovery with + # an empty matrix, which GitHub runs as zero jobs, so the preview passes having previewed + # nothing. + setup_discovery_repository + add_discovery_branch "state/staging/alpha" + DISCOVERY_ALLOWED_PREFIXES="state/staging/" + DISCOVERY_PREVIEW=true + DISCOVERY_CALLER_REF="refs/pull/42/merge" + local failing_sort="$DISCOVERY_TMP_ROOT/failing-sort" + mkdir -m 700 "$failing_sort" + printf '#!/bin/sh\nexit 1\n' >"$failing_sort/sort" + chmod 700 "$failing_sort/sort" + DISCOVERY_PATH="$failing_sort:$PATH" assert_discovery_failure \ + "discovery selection error: preview sampling did not keep one branch per owning prefix" \ + "a preview whose sampling lost its branches" +} + + test_discovery_treats_hostile_refs_as_inert_data() { # Production break caught: shell evaluation, word splitting, option parsing, locale filtering, # or JSON interpolation executes or corrupts a Git-valid remote branch name. @@ -1668,11 +1687,14 @@ run_report_step() { test_workflow_reports_the_processing_result() { # The report runs whatever processing did, judges it by the processing step's own - # outcome, and reads the manifest from the directory processing writes. + # outcome, and reads the manifest from the directory processing writes. `shell: bash` adds + # pipefail, so a failed read inside the summary's pipelines fails the step instead of + # leaving an empty section. yq -o=json '.jobs.process.steps' "$REUSABLE_WORKFLOW" | jq -e ' (.[] | select(.id == "process") | .env.PROCESS_RESULT_DIR) as $result | [.[] | select(.name == "Report processing result")] | length == 1 and .[0].if == "${{ always() }}" + and .[0].shell == "bash" and .[0].env.PROCESS_OUTCOME == "${{ steps.process.outcome }}" and .[0].env.RESULT_MANIFEST == $result + "/result.json" and .[0].env.PREVIEW == "${{ inputs.preview }}"