Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ADVANCED-USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is intentionally separate from the core CLI. `tf-version-bump` edits one ch

## GitHub Actions state-branch automation POC

For a scheduled or manually dispatched GitHub Actions proof of concept, see [`examples/github-actions`](../examples/github-actions/README.md). It copies into a consuming repository's `.github` directory and provides separate production and non-production callers that run only from the default branch, plus config-change triggers, read-only pull-request config validation, dry-run support, update pull requests, and marked failure issues. The reusable workflow has three jobs: `discover`, `process`, and `publish`. The pull-request check validates only the control config with `tf-version-bump`; full state-branch dry runs remain deferred. The same copy adds a read-only version report workflow that audits every state branch against its policy's config, and each caller can leave individual branches out with a `!` exclusion; the example's README describes both.
For a scheduled or manually dispatched GitHub Actions proof of concept, see [`examples/github-actions`](../examples/github-actions/README.md). It copies into a consuming repository's `.github` directory and provides separate production and non-production callers whose live runs start only from the default branch, plus config-change triggers, read-only pull-request config validation, pull-request previews on one sampled state branch per prefix, dry-run support, update pull requests, and marked failure issues. The reusable workflow has three jobs: `discover`, `process`, and `publish`. The pull-request config check validates only the control config with `tf-version-bump`; the preview processes a sample of state branches and publishes nothing. The same copy adds a read-only version report workflow that audits every state branch against its policy's config, and each caller can leave individual branches out with a `!` exclusion; the example's README describes both.

The example targets trusted official providers and organisation-authored private modules, including the HCP Terraform registry. It checks for configuration mistakes and unexpected file changes; it does not sandbox malicious Terraform or providers.

Expand Down
53 changes: 50 additions & 3 deletions examples/github-actions/.github/scripts/discover-state-branches.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ usage() {
Usage: discover-state-branches.sh --help

Discover immutable inputs for configured Terraform state branches.
With DISCOVERY_PREVIEW=true, run from refs/pull/<number>/merge and select one
sampled branch per prefix instead of every matching branch.
EOF
}

Expand Down Expand Up @@ -69,6 +71,35 @@ branch_is_excluded() {
return 1
}

# Keeps one branch per prefix, so a preview processes a sample rather than the whole policy. A
# branch belongs to the first prefix it matches; the pick is the first 15 hex digits of
# sha256("<seed>\t<prefix>") modulo the prefix's branch count, which stays within bash's signed
# 64-bit arithmetic and is stable while that prefix's branches are unchanged.
sample_branch_records() {
local prefix record digest
local -a owned sampled=()
declare -A seen_prefixes=()
for prefix in "${selection_prefixes[@]}"; do
[[ -z "${seen_prefixes[$prefix]-}" ]] || continue
seen_prefixes[$prefix]=1
owned=()
for record in "${branch_records[@]}"; do
[[ "${branch_owners[${record%%$'\t'*}]}" != "$prefix" ]] || owned+=("$record")
done
if [[ ${#owned[@]} -eq 0 ]]; then
echo "::warning::branch prefix '${prefix//'%'/%25}' owns no branch to preview" >&2
continue
fi
digest=$(printf '%s\t%s' "$preview_seed" "$prefix" | sha256sum)
sampled+=("${owned[$((16#${digest:0:15} % ${#owned[@]}))]}")
done
readarray -t branch_records < <(printf '%s\n' "${sampled[@]}" | sort)
# A failing sort inside the process substitution is not seen by set -e, and an empty matrix
# runs as zero jobs, so the preview would pass having previewed nothing.
[[ ${#sampled[@]} -gt 0 && ${#branch_records[@]} -eq ${#sampled[@]} ]] \
|| fail_discovery selection "preview sampling did not keep one branch per owning prefix"
}

: "${DISCOVERY_DEFAULT_BRANCH:?DISCOVERY_DEFAULT_BRANCH must be set}"
: "${DISCOVERY_CALLER_REF:?DISCOVERY_CALLER_REF must be set}"
: "${CONTROL_CHECKOUT:?CONTROL_CHECKOUT must be set}"
Expand All @@ -80,9 +111,22 @@ branch_is_excluded() {
: "${DISCOVERY_ALLOWED_PREFIXES?DISCOVERY_ALLOWED_PREFIXES must be set}"
: "${DISCOVERY_MANUAL_PREFIX?DISCOVERY_MANUAL_PREFIX must be set}"

expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH"
[[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \
|| fail_discovery caller "caller ref must be $expected_caller_ref"
preview=${DISCOVERY_PREVIEW-false}
[[ "$preview" == true || "$preview" == false ]] \
|| fail_discovery input "DISCOVERY_PREVIEW must be true or false"
if [[ "$preview" == true ]]; then
# A preview runs from a pull request's merge ref. Its number seeds the sample, so the seed
# cannot disagree with the pull request, and a manual prefix only arises from a dispatch.
[[ "$DISCOVERY_CALLER_REF" =~ ^refs/pull/([1-9][0-9]*)/merge$ ]] \
|| fail_discovery caller "preview caller ref must be refs/pull/<number>/merge"
preview_seed=${BASH_REMATCH[1]}
[[ -z "$DISCOVERY_MANUAL_PREFIX" ]] \
|| fail_discovery input "a preview cannot take a manual prefix"
else
expected_caller_ref="refs/heads/$DISCOVERY_DEFAULT_BRANCH"
[[ "$DISCOVERY_CALLER_REF" == "$expected_caller_ref" ]] \
|| fail_discovery caller "caller ref must be $expected_caller_ref"
fi
# shellcheck disable=SC2153 # Set by the reusable workflow or the focused harness.
[[ "$CONTROL_CHECKOUT" == /* && -d "$CONTROL_CHECKOUT" ]] \
|| fail_discovery input "CONTROL_CHECKOUT must be an absolute existing directory"
Expand Down Expand Up @@ -141,6 +185,7 @@ if ! git -C "$control_checkout" ls-remote --heads --refs origin >"$remote_heads_
fi

branch_records=()
declare -A branch_owners=()
declare -A matched_exclusions=()
while IFS=$'\t' read -r oid ref; do
branch=${ref#refs/heads/}
Expand All @@ -151,6 +196,7 @@ while IFS=$'\t' read -r oid ref; do
for prefix in "${selection_prefixes[@]}"; do
if [[ "$branch" == "$prefix"* ]]; then
branch_records+=("$branch"$'\t'"$oid")
branch_owners[$branch]=$prefix
break
fi
done
Expand All @@ -172,6 +218,7 @@ done
|| fail_discovery matrix "more than 256 branches matched; narrow or partition the prefix policy"

readarray -t branch_records < <(printf '%s\n' "${branch_records[@]}" | sort)
[[ "$preview" != true ]] || sample_branch_records
# Git ref names cannot contain a tab (or any control character), so a tab-separated record per
# branch is a safe carrier into the single jq pass below that replaces one jq -cn per branch.
tsv_records=()
Expand Down
19 changes: 16 additions & 3 deletions examples/github-actions/.github/scripts/reconcile-state-branch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ the classification. Requires RECONCILE_RESULT_DIR and RECONCILE_RUN_ID,
RECONCILE_RUN_ATTEMPT, RECONCILE_AUTOMATION_POLICY_ID, RECONCILE_CONTROL_OID,
RECONCILE_STATE_BRANCH, RECONCILE_BASE_OID, RECONCILE_REF_HASH.

publish additionally checks the clean exact-base RECONCILE_TARGET_CHECKOUT and
RECONCILE_TERRAFORM_ROOTS, creates one owned update commit, and reconciles marked
PRs and failure issues. Requires RECONCILE_DRY_RUN (true/false), RUNNER_TEMP,
publish refuses to do anything, dry run or not, unless RECONCILE_CALLER_REF is
refs/heads/ followed by RECONCILE_DEFAULT_BRANCH. It then additionally checks the clean
exact-base RECONCILE_TARGET_CHECKOUT and RECONCILE_TERRAFORM_ROOTS, creates one owned
update commit, and reconciles marked PRs and failure issues. Requires RECONCILE_DRY_RUN
(true/false), RUNNER_TEMP,
RECONCILE_RUN_URL, RECONCILE_GIT_REMOTE, RECONCILE_REPOSITORY, GH_TOKEN,
RECONCILE_COMMIT_AUTHOR_NAME, RECONCILE_COMMIT_AUTHOR_EMAIL, and the
RECONCILE_TERRAFORM_VERSION and RECONCILE_TF_VERSION_BUMP_VERSION their bodies name.
Expand Down Expand Up @@ -235,6 +237,7 @@ reconcile_lifecycle() {
}

publish_result() {
require_default_branch_caller
validate_result
[[ "$CLASSIFICATION" != automation ]] || return 0
: "${RECONCILE_DRY_RUN:?RECONCILE_DRY_RUN must be set}"
Expand Down Expand Up @@ -293,6 +296,16 @@ require_common_identity() {
}


# Publication must not depend on the publish job's `if` alone: a pull-request run executes the
# workflows it changes, so this refuses every ref but the default branch, dry run or not.
require_default_branch_caller() {
: "${RECONCILE_CALLER_REF:?RECONCILE_CALLER_REF must be set}"
: "${RECONCILE_DEFAULT_BRANCH:?RECONCILE_DEFAULT_BRANCH must be set}"
[[ "$RECONCILE_CALLER_REF" == "refs/heads/$RECONCILE_DEFAULT_BRANCH" ]] \
|| reconcile_error "publication runs only from refs/heads/$RECONCILE_DEFAULT_BRANCH"
}


github_marker() {
printf '<!-- tf-version-bump:%s:%s -->\n' \
"$RECONCILE_AUTOMATION_POLICY_ID" "$RECONCILE_REF_HASH"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
push:
paths:
- .github/tf-version-bump/nonproduction.yml
pull_request:
paths:
- .github/tf-version-bump/nonproduction.yml
- .github/workflows/tf-version-bump-nonproduction.yml
workflow_dispatch:
inputs:
branch_prefix:
Expand All @@ -27,19 +31,18 @@ permissions:
pull-requests: write
issues: write

concurrency:
group: tf-version-bump-nonproduction-${{ github.repository_id }}
cancel-in-progress: false
queue: max

jobs:
automation:
if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
concurrency:
group: tf-version-bump-nonproduction-${{ github.repository_id }}
cancel-in-progress: false
queue: max
uses: ./.github/workflows/tf-version-bump-reusable.yml
secrets:
secrets: &policy-secrets
TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }}
TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }}
with:
with: &policy-inputs
automation_policy_id: nonproduction
allowed_branch_prefixes: |
state/nonproduction/
Expand All @@ -55,6 +58,20 @@ jobs:
tf_version_bump_version: v1.0.0-rc.17
tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }}
preview: ${{ github.event_name == 'pull_request' }}
max_parallel: 4
commit_author_name: github-actions[bot]
commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com
# Previews a same-repository pull request into the default branch on one sampled branch per
# prefix, without publishing. The aliases give it the live job's inputs, including any the pull
# request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects
# both settings in one concurrency block. It keeps the workflow's permissions because GitHub
# refuses to start a read-only caller of a workflow whose publish job requests write.
preview:
if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }}
concurrency:
group: tf-version-bump-nonproduction-preview-${{ github.repository_id }}-${{ github.event.number }}
cancel-in-progress: true
uses: ./.github/workflows/tf-version-bump-reusable.yml
secrets: *policy-secrets
with: *policy-inputs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ on:
push:
paths:
- .github/tf-version-bump/production.yml
pull_request:
paths:
- .github/tf-version-bump/production.yml
- .github/workflows/tf-version-bump-production.yml
workflow_dispatch:
inputs:
branch_prefix:
Expand All @@ -27,19 +31,18 @@ permissions:
pull-requests: write
issues: write

concurrency:
group: tf-version-bump-production-${{ github.repository_id }}
cancel-in-progress: false
queue: max

jobs:
automation:
if: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}
concurrency:
group: tf-version-bump-production-${{ github.repository_id }}
cancel-in-progress: false
queue: max
uses: ./.github/workflows/tf-version-bump-reusable.yml
secrets:
secrets: &policy-secrets
TF_API_TOKEN: ${{ secrets.TF_API_TOKEN }}
TERRAFORM_ENV: ${{ secrets.TERRAFORM_ENV }}
with:
with: &policy-inputs
automation_policy_id: production
allowed_branch_prefixes: |
state/production/
Expand All @@ -53,6 +56,20 @@ jobs:
tf_version_bump_version: v1.0.0-rc.17
tf_version_bump_archive_sha256: 137eac8239d08e2a93409e9eda1fd59b52abcf3d774fa7c3882ce53534f56a4d
dry_run: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || false }}
preview: ${{ github.event_name == 'pull_request' }}
max_parallel: 2
commit_author_name: github-actions[bot]
commit_author_email: 41898282+github-actions[bot]@users.noreply.github.com
# Previews a same-repository pull request into the default branch on one sampled branch per
# prefix, without publishing. The aliases give it the live job's inputs, including any the pull
# request edits. A newer push cancels an older preview, while live runs queue: GitHub rejects
# both settings in one concurrency block. It keeps the workflow's permissions because GitHub
# refuses to start a read-only caller of a workflow whose publish job requests write.
preview:
if: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.base.ref == github.event.repository.default_branch }}
concurrency:
group: tf-version-bump-production-preview-${{ github.repository_id }}-${{ github.event.number }}
cancel-in-progress: true
uses: ./.github/workflows/tf-version-bump-reusable.yml
secrets: *policy-secrets
with: *policy-inputs
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ on:
dry_run:
type: boolean
default: false
preview:
type: boolean
default: false
max_parallel:
type: number
default: 4
Expand Down Expand Up @@ -74,6 +77,7 @@ jobs:
CONTROL_CHECKOUT: ${{ github.workspace }}/control
DISCOVERY_ALLOWED_PREFIXES: ${{ inputs.allowed_branch_prefixes }}
DISCOVERY_MANUAL_PREFIX: ${{ inputs.branch_prefix }}
DISCOVERY_PREVIEW: ${{ inputs.preview }}
DISCOVERY_POLICY_ID: ${{ inputs.automation_policy_id }}
DISCOVERY_RUN_ID: ${{ github.run_id }}
DISCOVERY_RUN_ATTEMPT: ${{ github.run_attempt }}
Expand Down Expand Up @@ -163,9 +167,11 @@ jobs:
retention-days: 7
- name: Report processing result
if: ${{ always() }}
shell: bash
env:
PROCESS_OUTCOME: ${{ steps.process.outcome }}
RESULT_MANIFEST: ${{ runner.temp }}/result/result.json
PREVIEW: ${{ inputs.preview }}
run: |
if [[ ! -f "$RESULT_MANIFEST" ]]; then
echo "Processing wrote no result manifest; publication will not change managed refs, pull requests or issues." >&2
Expand All @@ -178,24 +184,34 @@ jobs:
else "\n- Automation failed with status \(.failure.status); the `Process candidate` step log names the cause, and `logs/` in the processing artefact holds any command output"
end)' \
"$RESULT_MANIFEST" >>"$GITHUB_STEP_SUMMARY"
# Appends a file in a fence one backtick longer than any run inside it, truncating past
# 64 KiB at a line boundary so the closing fence stays on a line of its own.
append_fenced_file() {
local heading=$1 file=$2 noun=$3 fence='```' longest
longest=$({ grep -aoE '`+' "$file" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }')
while [[ ${#fence} -le $longest ]]; do fence+='`'; done
printf '\n#### %s\n\n%s\n' "$heading" "$fence" >>"$GITHUB_STEP_SUMMARY"
if [[ $(wc -c <"$file") -gt 65536 ]]; then
head -c 65536 "$file" | sed '$d' >>"$GITHUB_STEP_SUMMARY"
printf '%s\n\nThis %s was truncated; the processing artefact holds it in full.\n' "$fence" "$noun" >>"$GITHUB_STEP_SUMMARY"
else
awk 1 "$file" >>"$GITHUB_STEP_SUMMARY"
printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY"
fi
}
# Only the updater's log is shown: Terraform's logs can carry a credential a provider echoed.
index=0
while IFS= read -r root; do
index=$((index + 1))
log="${RESULT_MANIFEST%/*}/logs/update-$index.log"
[[ -f "$log" ]] || continue
fence='```'
longest=$({ grep -aoE '`+' "$log" || true; } | awk '{ if (length > max) max = length } END { print max + 0 }')
while [[ ${#fence} -le $longest ]]; do fence+='`'; done
printf '\n#### Updates in %s\n\n%s\n' "\`$root\`" "$fence" >>"$GITHUB_STEP_SUMMARY"
if [[ $(wc -c <"$log") -gt 65536 ]]; then
head -c 65536 "$log" | sed '$d' >>"$GITHUB_STEP_SUMMARY"
printf '%s\n\nThis log was truncated; the processing artefact holds it in full.\n' "$fence" >>"$GITHUB_STEP_SUMMARY"
else
awk 1 "$log" >>"$GITHUB_STEP_SUMMARY"
printf '%s\n' "$fence" >>"$GITHUB_STEP_SUMMARY"
fi
append_fenced_file "Updates in \`$root\`" "$log" log
done < <(jq -r '(.roots // [])[]' "$RESULT_MANIFEST")
# The patch holds only Terraform files and root lock files, never Terraform's logs.
patch="${RESULT_MANIFEST%/*}/candidate.patch"
if [[ "$PREVIEW" == true && -f "$patch" ]]; then
append_fenced_file 'Candidate patch' "$patch" patch
fi
classification=$(jq -er '.classification' "$RESULT_MANIFEST")
if [[ "$PROCESS_OUTCOME" == success \
&& "$classification" != success && "$classification" != no-change ]]; then
Expand All @@ -206,7 +222,7 @@ jobs:
publish:
name: publish (${{ matrix.branch }})
needs: [discover, process]
if: ${{ always() && needs.discover.result == 'success' }}
if: ${{ always() && needs.discover.result == 'success' && !inputs.preview }}
strategy:
fail-fast: false
max-parallel: ${{ inputs.max_parallel }}
Expand Down Expand Up @@ -260,6 +276,8 @@ jobs:
RECONCILE_TARGET_CHECKOUT: ${{ github.workspace }}/target
RECONCILE_TERRAFORM_ROOTS: ${{ inputs.terraform_directories }}
RECONCILE_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
RECONCILE_CALLER_REF: ${{ github.ref }}
RECONCILE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
RECONCILE_GIT_REMOTE: ${{ github.server_url }}/${{ github.repository }}.git
RECONCILE_REPOSITORY: ${{ github.repository }}
RECONCILE_DRY_RUN: ${{ inputs.dry_run }}
Expand Down
Loading
Loading