Skip to content

test: stop cloud backport tag tests depending on an ambient GH_TOKEN - #19544

Closed
huang47 wants to merge 1 commit into
mainfrom
glary/fix-cloud-backport-tag-test-gh-token
Closed

huang47 wants to merge 1 commit into
mainfrom
glary/fix-cloud-backport-tag-test-gh-token

Conversation

@huang47

@huang47 huang47 commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

PR Created by the Glary-Bot Agent


scripts/cicd/cloud-backport-tag.test.ts is red on main and on every PR whose merge ref includes #19523, regardless of what the PR changes. Reported from #17413 (a camera-angle PR that touches no CI scripts).

Root cause

#19523 added a credential precondition to the script:

: "${GH_TOKEN:?PR_GH_TOKEN is required for cloud tag reconciliation}"

The harness never supplied one. runTagScript stubs gh on PATH and sets every other variable the script reads — EVENT_NAME, BRANCH, GITHUB_REPOSITORY, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY — but GH_TOKEN arrived only through the ...process.env spread. A developer shell with a token exported satisfies it; ci-tests-unit.yaml has none, so the script exits at line 5 before reaching the stub and all six scenario tests fail.

Reproduced on main with the token unset — 6 failed | 1 passed, and the [6/18] failure counter is 6 tests across retry: 2, which is why the report showed one test name three times.

Fix

Pin GH_TOKEN in the spawned environment next to the stub gh, placed after the spread so an ambient token cannot change the outcome either, and add a case that runs with the credential blank to pin the guard #19523 introduced. That guard had no coverage, which is why a script change requiring a secret could ship green.

Verification

  • vitest run --project tooling scripts/cicd/cloud-backport-tag.test.ts under CI=true with GH_TOKEN and GITHUB_TOKEN unset (the Actions environment): 8/8 pass, no retries.
  • Same file with GH_TOKEN exported: 8/8 pass — hermetic in both directions.
  • Mutation check: deleting the guard line from cloud-backport-tag.sh turns the new test red (exit 0, ::notice::Tag cloud/v1.54.16 already exists); restoring it turns it green. It pins behavior, not wording.
  • Whole scripts/cicd/ tooling suite with no token: 203 passed. The one remaining red file, upsert-comment-section-large-content.test.ts, fails identically on main in this container — it probes for the E2BIG environment-size limit, which this sandbox does not enforce — and is untouched here.
  • Guard exercised directly in a terminal: scripts/cicd/cloud-backport-tag.sh: line 5: GH_TOKEN: PR_GH_TOKEN is required for cloud tag reconciliation, exit 1.
  • oxfmt, oxlint (type-aware), eslint, typecheck:scripts clean.

No Playwright screenshots: the change is a CI shell-script test with no browser surface.

Unblocks #17413 and every other PR currently failing on this file.

The credential guard added in #19523 made the script exit before any
scenario ran, but the harness only inherited GH_TOKEN from the shell, so
the suite passed on a developer machine with a token exported and failed
on every CI job.

The harness now supplies its own stub token alongside the stub `gh`, and
a new case pins the guard by running with the credential blank.
@coderabbitai

coderabbitai Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Comfy-Org/ComfyUI_frontend/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: e4919363-e4d2-4080-b122-6eb125dc79f8

📥 Commits

Reviewing files that changed from the base of the PR and between e71d2fa and 99585b3.

📒 Files selected for processing (1)
  • scripts/cicd/cloud-backport-tag.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The backport-tag test helper now accepts an optional token and sets it as GH_TOKEN. A test checks that an empty token causes the script to report PR_GH_TOKEN is required.

Changes

Backport tag token test

Layer / File(s) Summary
Configure token and check missing-token error
scripts/cicd/cloud-backport-tag.test.ts
The test helper accepts an optional token and sets GH_TOKEN, defaulting to stub-release-token. A new test checks the error returned when the token is empty.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Suggested reviewers: christian-byrne

Merge Risk: ⚪ Minimal · up to 99585

The test no longer depends on an ambient GH_TOKEN and covers the missing-token error. No actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 7
✅ Passed checks (7 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: removing the cloud backport tag tests' dependence on an ambient GH_TOKEN.
Description check ✅ Passed The description provides a clear summary, root cause, fix, verification results, and screenshot applicability. It does not use the repository template headings, but it contains the required informatio…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
End-To-End Regression Coverage For Fixes ✅ Passed The PR title and commit subject do not use the specified bug-fix signals. The PR changes only scripts/cicd/cloud-backport-tag.test.ts, which is a CI test file and is not under src/ or packages/.…
Website End-To-End Regression Coverage ✅ Passed The pull request changes only scripts/cicd/cloud-backport-tag.test.ts. It does not change website runtime files under apps/website/src/ or apps/website/public/, so this check does not apply.
Adr Compliance For Entity/Litegraph Changes ✅ Passed The changed-file information shows only scripts/cicd/cloud-backport-tag.test.ts. The PR does not modify src/lib/litegraph/, src/ecs/, or files related to graph entities. The ADR-LAYOUT and ADR-E…
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🌾 Fallow: ✅ Passed

Fallow audits the diff for duplication, complexity and dead code. It gates on new findings only, so when the audit completes, a red is something this PR introduced or moved — never inherited backlog. A red can also mean the audit itself did not finish; the details below say which happened.

No new findings in the changed files.

Reproduce locally
# the base the gate compared against
npx fallow audit --root . --quiet --format json \
  --changed-since e71d2fa088ed561327479243c63cf0c7ef07c4ab

Full docs: ADR-DEVEX-LINT-0015

🧩 Custom Nodes Ecosystem Matrix: 🟡 Withheld — could not measure

Executes the frontend JS of every registry pack that ships any, against this change, in both renderers. The eight ecosystem-matrix shard checks are a harness self-test; the pack verdict is matrix-verdict.

This candidate changes nothing the matrix measures, so every expensive stage was skipped.

Reproduce locally
# build one shard's fixture, then run it
python3 scripts/registry-census/build_matrix.py --shard 1/4
MATRIX_RENDERER=vue MATRIX_VUE=1 \
  npx vitest run --config vitest.matrix.config.mts

Full docs: scripts/registry-census/README.md

@github-actions github-actions Bot added the risk:high PR risk grade (advisory shadow check; grader-owned) label Sep 30, 2026
@datadog-official

Copy link
Copy Markdown

Pipelines

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 2 Pipeline jobs failed

CI: Lint Format | lint-and-format — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

CI: Lint Format | typecheck-aux — 🔧 Needs a code fix, caused by this PR

View in Datadog · View in GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 99585b3 | Docs | View more details | Give us feedback!

@glary-bot

glary-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown

Superseded — please close without merging.

#19532 ("test: isolate cloud tag token contract") landed on main at 02:21 UTC, one minute before the CI failure was reported and ten minutes before I branched. I branched from a stale local main and did not re-fetch, so I diagnosed and fixed a problem that was already fixed. Same root cause, same remedy:

#19532 (on main) this PR
token?: string on runTagScript options yes yes (byte-identical hunk)
stub token after the ...process.env spread options.token ?? 'test-token' options.token ?? 'stub-release-token'
test pinning the absent-credential guard fails closed when the release token is absent refuses to reconcile tags without the release credential

The one red check here is caused by that duplication, not by a defect in either change. The two branches insert GH_TOKEN at different positions in the same object literal — after EVENT_NAME on main, after FAKE_SCENARIO here — so git merges both insertions cleanly and the merge ref ends up with the key twice:

      EVENT_NAME: options.event ?? 'workflow_dispatch',
      GH_TOKEN: options.token ?? 'test-token',          # from #19532
      FAKE_CONTAINMENT: options.containment ?? 'behind',
      FAKE_GH_COUNT_FILE: count,
      FAKE_SCENARIO: scenario,
      GH_TOKEN: options.token ?? 'stub-release-token',  # from this PR

That is typecheck-aux's TS1117: An object literal cannot have multiple properties with the same name at scripts/cicd/cloud-backport-tag.test.ts:98. Rebasing does not help: the correct resolution leaves an empty diff.

main at 3d2c82db4e is verified green — scripts/cicd/cloud-backport-tag.test.ts passes 8/8 under CI=true with both GH_TOKEN and GITHUB_TOKEN unset. Nothing further is needed for the failure this PR set out to fix.

CodeRabbit's approval on this branch is accurate in isolation and changes nothing here; the diff is redundant regardless of its quality.

@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

@@            Coverage Diff             @@
##             main   #19544      +/-   ##
==========================================
+ Coverage   84.31%   85.04%   +0.72%     
==========================================
  Files        3294     2340     -954     
  Lines      158986   133925   -25061     
  Branches    43239    36088    -7151     
==========================================
- Hits       134051   113892   -20159     
+ Misses      23060    19668    -3392     
+ Partials     1875      365    -1510     
Flag Coverage Δ
billing-web-unit ?
unit 81.48% <ø> (ø)
website-unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 954 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:high PR risk grade (advisory shadow check; grader-owned)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants