Repository navigation
ci: guard against ADR files introduced inside merge commits (evil merges) #17577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
christian-byrne
wants to merge
4
commits into
main
Choose a base branch
from
rush/phs48-adr-evil-merge-guard
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
30c4788
ci: fail PRs whose merge commits introduce docs/adr files present in …
christian-byrne 6752b3f
Merge remote-tracking branch 'comfyorg/main' into wt/17577-adr-guard
christian-byrne 6efeaa0
ci: stop the evil-merge fixtures from tripping the ADR link guard
christian-byrne 7fd611a
ci: stop the evil-merge guard passing when it did not run
christian-byrne File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| # Description: Fails when a merge commit in the PR's history introduces a | ||
| # docs/adr/*.md file present in neither of its parents ("evil merge" of | ||
| # governance content — invisible to per-commit review tooling). | ||
| name: 'CI: ADR Evil Merge Guard' | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
| pull_request: | ||
|
|
||
| jobs: | ||
| adr-evil-merge: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| fetch-depth: 0 | ||
| - name: Check for ADR files introduced inside merge commits | ||
| run: | | ||
| base="${{ github.event.pull_request.base.sha }}" | ||
| if [ -z "$base" ]; then | ||
| base="${{ github.event.before }}" | ||
| fi | ||
| ./scripts/cicd/check-adr-evil-merge.sh "$base" HEAD |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| #!/bin/bash | ||
| # Description: Fails when a merge commit in the scanned range introduces a | ||
| # `docs/adr/*.md` file that is present in NEITHER of its parents ("evil merge" | ||
| # of governance content). Per-commit review tooling (`gh pr diff --patch`, | ||
| # incremental bot reviews) cannot see content that only exists inside a merge | ||
| # commit, so governance artifacts smuggled in that way take zero review. | ||
| # Added by phs-48 after AGENT-SESSION-0028 landed inside PR #16768's | ||
| # "merge main" commit (own commit on main: #17483). | ||
| # | ||
| # Usage: check-adr-evil-merge.sh <base-rev> [head-rev] | ||
| # <base-rev> exclusive range start (e.g. PR base sha, or `github.event.before`) | ||
| # [head-rev] inclusive range end, defaults to HEAD | ||
| # An all-zero base (new-branch push) is a no-op success. The range scan needs | ||
| # merge-commit history: run the workflow with `fetch-depth: 0`. | ||
| set -euo pipefail | ||
|
|
||
| base="${1:?usage: check-adr-evil-merge.sh <base-rev> [head-rev]}" | ||
| head="${2:-HEAD}" | ||
| path='docs/adr/' | ||
| # The contract in the header is `docs/adr/*.md`. Matching the whole directory | ||
| # instead would fail a merge that adds a diagram or a fixture under it and | ||
| # report that as smuggled governance content. | ||
| pathspec='docs/adr/*.md' | ||
|
|
||
| if [[ "$base" =~ ^0+$ ]]; then | ||
| exit 0 | ||
| fi | ||
|
|
||
| # A range this script cannot resolve is not an empty range. `|| true` turned an | ||
| # unavailable base - which is what a force-push leaves behind in | ||
| # `github.event.before` - into a silent pass, so the guard reported green | ||
| # precisely when it had not looked. Exit 2 to keep "I could not run" distinct | ||
| # from the exit 1 that means "I found one". | ||
| rev_list_err="$(mktemp)" | ||
| trap 'rm -f "$rev_list_err"' EXIT | ||
| if ! merges="$(git rev-list --merges "$base..$head" 2>"$rev_list_err")"; then | ||
| echo "ERROR: cannot resolve the revision range ${base}..${head}" >&2 | ||
| sed 's/^/ git: /' "$rev_list_err" >&2 | ||
| echo " The guard did not run, so this is NOT evidence that no ADR was" >&2 | ||
| echo " evil-merged. Check out with fetch-depth: 0, or pass a base that" >&2 | ||
| echo " still exists (a force-push can orphan github.event.before)." >&2 | ||
| exit 2 | ||
| fi | ||
| if [ -z "$merges" ]; then | ||
| exit 0 | ||
| fi | ||
|
|
||
| fail=0 | ||
| for merge in $merges; do | ||
| # `rev-list --parents -n 1` prints "<merge> <parent1> <parent2> ..." | ||
| parents="$(git rev-list --parents -n 1 "$merge")" | ||
| read -r _ first_parent other_parents <<< "$parents" | ||
| # Files ADDED relative to parent 1 under docs/adr/; then reject when no | ||
| # other parent contains the file either, i.e. it exists only in the merge. | ||
| while IFS= read -r file; do | ||
| [ -n "$file" ] || continue | ||
| in_some_parent=0 | ||
| for parent in $other_parents; do | ||
| if git cat-file -e "${parent}:${file}" 2>/dev/null; then | ||
| in_some_parent=1 | ||
| break | ||
| fi | ||
| done | ||
| if [ "$in_some_parent" -eq 0 ]; then | ||
| echo "ERROR: merge $(git rev-parse --short "$merge") introduces ${file}" >&2 | ||
| echo " present in neither of its parents — ADR governance content must land" >&2 | ||
| echo " on its own commit so per-commit review tools can see it." >&2 | ||
| fail=1 | ||
| fi | ||
| done < <(git diff-tree -r --name-only --diff-filter=A --no-commit-id "$first_parent" "$merge" -- "$pathspec") | ||
| done | ||
|
|
||
| if [ "$fail" -ne 0 ]; then | ||
| echo "One or more merge commits evil-merged files under ${path}." >&2 | ||
| fi | ||
| exit "$fail" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,174 @@ | ||
| import { execFileSync, spawnSync } from 'node:child_process' | ||
| import fs from 'node:fs' | ||
| import os from 'node:os' | ||
| import path from 'node:path' | ||
| import { afterEach, describe, expect, it } from 'vitest' | ||
|
|
||
| const SCRIPT = path.join(import.meta.dirname, 'check-adr-evil-merge.sh') | ||
| const temporaryDirectories: string[] = [] | ||
|
|
||
| const GIT_ENV = { | ||
| GIT_CONFIG_GLOBAL: '/dev/null', | ||
| GIT_CONFIG_SYSTEM: '/dev/null' | ||
| } | ||
|
|
||
| // The script walks merge-commit history, so fixtures need a hermetic throwaway | ||
| // repo with real (non-shallow) history. | ||
| function tempGitRepo(): { dir: string; git: (...args: string[]) => string } { | ||
| const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'adr-evil-merge-')) | ||
| temporaryDirectories.push(dir) | ||
| const env = { ...process.env, ...GIT_ENV } | ||
| const git = (...args: string[]) => | ||
| execFileSync('git', args, { cwd: dir, encoding: 'utf8', env }) | ||
| git('init', '--initial-branch=main') | ||
| git('config', 'user.email', 'ci@example.com') | ||
| git('config', 'user.name', 'CI') | ||
| return { dir, git } | ||
| } | ||
|
|
||
| function write(dir: string, rel: string, contents: string): void { | ||
| const filePath = path.join(dir, rel) | ||
| fs.mkdirSync(path.dirname(filePath), { recursive: true }) | ||
| fs.writeFileSync(filePath, contents) | ||
| } | ||
|
|
||
| function commitAll( | ||
| git: (...args: string[]) => string, | ||
| message: string | ||
| ): string { | ||
| git('add', '.') | ||
| git('commit', '-m', message) | ||
| return git('rev-parse', 'HEAD').trim() | ||
| } | ||
|
|
||
| function runScript(dir: string, base: string, head = 'HEAD') { | ||
| return spawnSync('bash', [SCRIPT, base, head], { | ||
| cwd: dir, | ||
| encoding: 'utf8', | ||
| env: { ...process.env, ...GIT_ENV } | ||
| }) | ||
| } | ||
|
|
||
| afterEach(() => { | ||
| for (const directory of temporaryDirectories.splice(0)) { | ||
| fs.rmSync(directory, { recursive: true, force: true }) | ||
| } | ||
| }) | ||
|
|
||
| describe('check-adr-evil-merge', () => { | ||
| // A guard that exits 0 while printing to stderr reads as a pass with a | ||
| // warning, and nobody reads CI warnings. Exit status alone cannot tell the | ||
| // two apart, so every clean case asserts silence as well. | ||
| function expectSilentPass(result: ReturnType<typeof runScript>): void { | ||
| expect({ status: result.status, stderr: result.stderr }).toEqual({ | ||
| status: 0, | ||
| stderr: '' | ||
| }) | ||
| expect(result.stdout).toBe('') | ||
| } | ||
|
|
||
| it('passes when the range has no merge commits', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'docs/adr/TOPIC-0001.md', '# adr\n') | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| const base = commitAll(git, 'base') | ||
| write(dir, 'src/index.ts', 'export {}\n') | ||
| commitAll(git, 'ordinary commit adds an unrelated file') | ||
|
|
||
| expectSilentPass(runScript(dir, base)) | ||
| }) | ||
|
|
||
| it('passes when the ADR arrived on a branch commit and the merge only integrates it', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'src/index.ts', 'export {}\n') | ||
| const base = commitAll(git, 'base') | ||
| git('checkout', '-b', 'feature') | ||
| write(dir, 'docs/adr/TOPIC-0002.md', '# adr\n') | ||
| commitAll(git, 'branch commit adds the ADR') | ||
| git('checkout', 'main') | ||
| git('merge', '--no-ff', '--no-edit', 'feature') | ||
|
|
||
| expectSilentPass(runScript(dir, base)) | ||
| }) | ||
|
|
||
| it('fails when a merge commit introduces a docs/adr file present in neither parent', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'src/index.ts', 'export {}\n') | ||
| commitAll(git, 'base') | ||
| git('checkout', '-b', 'feature') | ||
| write(dir, 'src/feature.ts', 'export {}\n') | ||
| commitAll(git, 'branch commit') | ||
| git('checkout', 'main') | ||
| write(dir, 'src/main-drift.ts', 'export {}\n') | ||
| const mainDrift = commitAll(git, 'main drifts independently') | ||
| git('merge', '--no-ff', '--no-commit', 'feature') | ||
| // Smuggle governance content into the merge itself: present in NEITHER parent. | ||
| write(dir, 'docs/adr/TOPIC-0003-evil.md', '# smuggled\n') | ||
| git('add', '.') | ||
| git('commit', '-m', 'merge main and smuggle an ADR') | ||
|
|
||
| const result = runScript(dir, mainDrift) | ||
| expect(result.status).toBe(1) | ||
| expect(result.stderr).toContain('docs/adr/TOPIC-0003-evil.md') | ||
| }) | ||
|
|
||
| it('does not flag a merge that only resolves conflicts in files both parents already have', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'docs/adr/TOPIC-0004.md', '# adr\n') | ||
| commitAll(git, 'base') | ||
| git('checkout', '-b', 'feature') | ||
| write(dir, 'docs/adr/TOPIC-0004.md', '# adr with branch edit\n') | ||
| commitAll(git, 'branch edits the ADR') | ||
| git('checkout', 'main') | ||
| write(dir, 'docs/adr/TOPIC-0004.md', '# adr with main edit\n') | ||
| const mainEdit = commitAll(git, 'main edits the ADR') | ||
| expect(() => git('merge', '--no-ff', '--no-edit', 'feature')).toThrow() | ||
| write(dir, 'docs/adr/TOPIC-0004.md', '# adr resolved\n') | ||
| git('add', '.') | ||
| git('commit', '--no-edit') | ||
|
|
||
| expectSilentPass(runScript(dir, mainEdit)) | ||
| }) | ||
|
|
||
| it('is a no-op when the base is the all-zero new-branch sentinel', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'docs/adr/TOPIC-0005.md', '# adr\n') | ||
| commitAll(git, 'base') | ||
|
|
||
| expectSilentPass(runScript(dir, '0'.repeat(40))) | ||
| }) | ||
|
|
||
| it('ignores a non-Markdown file smuggled into a merge', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'src/index.ts', 'export {}\n') | ||
| commitAll(git, 'base') | ||
| git('checkout', '-b', 'feature') | ||
| write(dir, 'src/feature.ts', 'export {}\n') | ||
| commitAll(git, 'branch commit') | ||
| git('checkout', 'main') | ||
| write(dir, 'src/main-drift.ts', 'export {}\n') | ||
| const mainDrift = commitAll(git, 'main drifts independently') | ||
| git('merge', '--no-ff', '--no-commit', 'feature') | ||
| // The documented contract is `docs/adr/*.md`. A diagram is not governance | ||
| // content taking zero review, and reporting it as an evil-merged ADR | ||
| // teaches people to ignore the guard. | ||
| write(dir, 'docs/adr/diagram.svg', '<svg />\n') | ||
| git('add', '.') | ||
| git('commit', '-m', 'merge main and add an asset') | ||
|
|
||
| expectSilentPass(runScript(dir, mainDrift)) | ||
| }) | ||
|
|
||
| it('refuses to report a pass when the range cannot be resolved', () => { | ||
| const { dir, git } = tempGitRepo() | ||
| write(dir, 'docs/adr/TOPIC-0006.md', '# adr\n') | ||
| commitAll(git, 'base') | ||
|
|
||
| // What a force-push leaves in `github.event.before`: a real-looking sha | ||
| // that no longer exists. Silently treating it as an empty range is the | ||
| // one outcome the guard must never produce. | ||
| const result = runScript(dir, 'f'.repeat(40)) | ||
| expect(result.status).not.toBe(0) | ||
| expect(result.status).not.toBe(1) | ||
| expect(result.stderr).toContain('cannot resolve the revision range') | ||
| }) | ||
| }) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.