Skip to content

feat(actions-example): preview config changes on pull requests - #170

Merged
yesdevnull merged 14 commits into
mainfrom
wip/config-change-preview
Sep 22, 2026
Merged

feat(actions-example): preview config changes on pull requests#170
yesdevnull merged 14 commits into
mainfrom
wip/config-change-preview

Conversation

@yesdevnull

@yesdevnull yesdevnull commented Sep 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Adds a pull-request preview to the GitHub Actions example. When a same-repository pull request into the default branch changes a policy's control configuration or its caller workflow, that caller's new preview job processes one sampled state branch per configured prefix exactly as a live run would — update, terraform init, optional formatting and terraform validate — and publishes nothing. Each process summary shows the updater's log and, for a changed candidate, the candidate patch.

How it works

  • Discovery (discover-state-branches.sh) gains DISCOVERY_PREVIEW. In preview mode it accepts only a refs/pull/<n>/merge caller ref, refuses a manual prefix, checks the 256-branch limit on the full matched set, then keeps one branch per prefix: a branch belongs to the first prefix it matches, and the pick is the first 15 hex digits of sha256("<n>\t<prefix>") modulo that prefix's branch count, so every push to the same pull request previews the same branches. A prefix that owns no branch gets a warning annotation. Unset, discovery behaves exactly as before, so the version report is unchanged.
  • Reusable workflow gains a preview input. publish is skipped when it is set, and the Report processing result step appends the candidate patch through a shared append_fenced_file renderer that the updater logs now use too.
  • Publication guard: reconcile-state-branch.sh publish refuses any caller ref other than refs/heads/<default branch>, dry run or not, before any Git or GitHub operation. A pull-request run executes the workflows it changes, so publication no longer rests on the publish job's if alone.
  • Callers each gain a preview job alongside the live automation job. The two share their with: and secrets: blocks through YAML anchors and aliases, so a pull request that edits a pin, prefix or root is previewed with its edits. Concurrency moves to the job level: the live job keeps queue: max, while the preview's per-pull-request group uses cancel-in-progress: true, which GitHub rejects in the same block as queue: max.

Verified GitHub Actions behaviour

The design depends on behaviour GitHub's documentation leaves open, so it was checked in a disposable private repository (since deleted) with sleep-only jobs mirroring the caller shape:

  • Job-level queue: max on a job calling a reusable workflow serialises runs without cancelling any.
  • Job-level cancel-in-progress: true cancels an in-progress preview when the pull request gets a new push.
  • The aliased with: block reaches the called workflow intact (preview=true), with the live job and publish skipped.
  • A preview job declaring permissions: contents: read fails at startup, because GitHub checks the called workflow's publish permissions even when that job would be skipped. The preview job therefore keeps the workflow's permissions and relies on the two publication guards.

A Dependabot pull request that touches a caller does run the preview, because its branch lives in this repository; per GitHub's Dependabot documentation it receives Dependabot secrets rather than Actions secrets, so it has no TF_API_TOKEN unless one is also stored as a Dependabot secret. The README says so. Not verified: whether the skipped live job ever waits in its queue during a live run.

Verification

  • go test -race over the module (95.4% coverage), golangci-lint run (0 issues), make actionlint, make shellcheck and make docs-check all pass.
  • Every non-Docker test in examples/github-actions/test.sh, plus reconcile-test.sh and report-test.sh, passes locally (56/56). The Docker-backed test_processing_* tests could not run locally, because the pinned linux/amd64 Terraform image fails with exec format error on this Mac's Docker, as it does on main; CI runs them. process-state-branch.sh is unchanged.
  • New harness tests pin the discovery guard and its diagnostics, a known-answer sample for two pull request numbers, the preview-only warning, the matrix limit, a sampling failure that would otherwise leave an empty matrix, the reconcile refusal with no GitHub calls or pushed refs, the publish gate (with a mutant that drops it), the patch summary's fencing and truncation, and each caller's triggers, concurrency and alias-shared inputs.

Documentation

examples/github-actions/README.md gains a "Preview on pull requests" section and corrects the passages the change made false (callers running only from the default branch, pull requests never processing state branches); docs/ADVANCED-USAGE.md is updated to match.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ

yesdevnull and others added 14 commits September 22, 2026 12:03
@yesdevnull @claude
docs: design a pull-request preview for config changes
e6dbded 
Sample one state branch per configured prefix and run the existing
process job on it when a pull request changes a policy config, without
publishing anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs: revise the config-change preview design after review
2d0a975 
Run previews as a separate caller job sharing the live job's inputs
through YAML aliases, so they can cancel stale runs without the
forbidden queue/cancel combination. Add an independent publication
guard, derive the sample seed from the pull request ref, and spell
out the tests and documentation edits the change requires.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs: record the verified Actions behaviour for the preview design
10bf995 
A disposable probe repository confirmed that job-level queue: max
serialises live runs, that job-level cancel-in-progress cancels stale
previews, and that a read-only caller job fails at startup when the
called workflow's skipped publish job requests write permissions, so
the preview job keeps the caller's permissions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs: plan the config-change preview implementation
674bc60 
Seven TDD tasks: the discovery preview guard, per-prefix sampling, the
publication guard, the reusable workflow preview mode, the caller
preview jobs, documentation, and full validation with a separate test
cleanup pass. The spec now keeps the live job's existing if and names
the shared renderer's truncation noun.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
feat(actions-example): accept pull-request refs in preview discovery
3b9bbd1 
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
feat(actions-example): sample one state branch per prefix in preview …
d148bd7 
…discovery

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
feat(actions-example): refuse publication from any ref but the defaul…
fa81c80 
…t branch

A preview run on a pull request executes the PR's own workflows, so
publication must not rely on the publish job's `if` alone: a lost or
tampered gate there would still push update refs and change pull
requests and issues from unmerged config. reconcile-state-branch.sh
publish now independently refuses any caller ref other than
refs/heads/<default branch>, dry run or not.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
feat(actions-example): add a preview mode to the reusable workflow
b654468 
Add a `preview` input to the reusable workflow, wire it into
discovery's `DISCOVERY_PREVIEW`, gate the publish job behind
`!inputs.preview` so a preview never reaches publication, and have
the processing report step append the candidate patch to the run
summary whenever a preview produced one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
feat(actions-example): preview config changes on pull requests
acfe7fb 
Give each caller a preview job that runs a same-repository pull
request into the default branch on the reusable workflow's new
preview input, without publishing, so a config change can be seen
before it merges.

Move each caller's concurrency block to job level so the live job can
keep queue: max while the preview job cancels its own older run;
GitHub rejects both settings in one concurrency block. Anchors share
the live job's with/secrets so the preview always runs with the pull
request's edits. The preview job declares no permissions, because
GitHub refuses to start a read-only caller when the called workflow's
publish job requests write, even when that job is skipped.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs(actions-example): document pull-request previews
70979cf 
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
test(actions-example): check each caller alias separately
1e91b56 
Split the combined `grep && grep || fail` alias check in
test_callers_preview_pull_requests into two independent checks, each
with its own `|| fail`, naming which alias (with/secrets) is missing.
shellcheck's SC2015 flags `A && B || C` because C also runs when A
succeeds and B fails, masking which alias check actually failed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs(actions-example): correct preview refresh and fork wording
f1fe133 
A final review of the preview feature found three wording problems:
the refresh paragraph claimed a re-run picks up a moved default
branch, but GitHub re-runs reuse the original event's GITHUB_SHA, so
only a moved state branch is picked up; the Environment paragraph
conflated branch rules (which reject) with required reviewers (which
hold for approval); and the fork-skip claim was never verified, since
a fork or Dependabot pull request's read-only token may make GitHub
fail the run at startup instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
docs: remove the config-change preview design and plan
9f0585d 
The design and implementation plan served the branch; their reasoning
stays in its history, and the README documents the behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull @claude
fix(actions-example): address PR review of the preview
340937e 
Fail preview discovery when sampling loses its branches: a failing sort
inside a process substitution is invisible to set -e, and an empty
matrix runs as zero jobs. Run the processing report step under
shell: bash so pipefail catches a failed read in its summary pipelines.

Correct the README: Dependabot pull requests do run the preview but
receive Dependabot secrets rather than Actions secrets, and editing the
scripts or reusable workflow alone does not start a preview. Make the
reconcile usage text say publish refuses everything from another ref.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmjqpPp4yfPeg11EgGETuQ
@yesdevnull
yesdevnull merged commit 02e4df0 into main Sep 22, 2026
5 checks passed
@yesdevnull
yesdevnull deleted the wip/config-change-preview branch September 22, 2026 05:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@yesdevnull