Fork AI lanes: api.github.com connections refused under harden-runner, so lanes post no check and PR Readiness is stranded

What

On a fork PR, the Fork Security Scope Review publish job sometimes fails to post its Security Scope Review check-run. The step treats that as a warning, so the job still looks successful. PR Readiness then waits on "Security Scope Review (not started)" forever, and nothing short of a new push clears it.

Both POST attempts in the Publish check-run step get connection refused from api.github.com, even though api.github.com:443 is in the job's harden-runner allowlist (egress-policy: block):

Post "https://api.github.com/repos/kirodotdev/KiroCrew/check-runs": dial tcp 140.82.116.5:443: connect: connection refused
Post "https://api.github.com/repos/kirodotdev/KiroCrew/check-runs": dial tcp 140.82.116.5:443: connect: connection refused
::warning::could not publish the check-run for <head>

Seen twice in one day on PR #9750:

run 39962367116 (head c71349f435, 13:37 UTC), IP 140.82.116.6
run 40029932605 (head 7340cccb12, 22:20 UTC), IP 140.82.116.5

In both runs, every earlier step passed and the generate leg found nothing to scope. The second run's job was later shown as cancelled, but every one of its steps reports success. Other fork runs in the same hours published normally, so this is intermittent.

Why it matters

A fork author cannot rerun the lane. A PR-body edit doesn't help either, because the lane is a workflow_run on Fast Gate. So a fully green fork PR stays at readiness: checking until its author pushes a new commit, and the new commit restarts the whole ~90-check matrix and every flake that comes with it. The step's own comment says this is the stranding it means to prevent ("One retry: a single transient 5xx here is what strands the lane").

Possible directions

Retry longer, with backoff over about a minute, and fail the job instead of warning when every attempt fails, so the failure is visible and a rerun is the obvious fix.
Check whether harden-runner blocks some api.github.com addresses in block mode. A refused connection to an allowlisted host, at a different address each time, looks like the allowlist resolved a different address set than the one the call used.

Same failure in a second fork lane: Fork GPT 6.1 Review run 40045512086 (PR #9750, head cbe6ceb950, 00:33 UTC 10-24). Its first step, Resolve and validate PR, got dial tcp 20.29.134.17:443: connect: connection refused on all three tries against api.github.com (also on that job's harden-runner allowlist), and the run failed before it posted any check. PR Readiness now waits on "GPT 6.1 Review (not started)". So this isn't specific to the scope lane's publish step: api.github.com connections from the hardened fork-lane jobs sometimes get refused, at different addresses each time (140.82.116.5, 140.82.116.6, 20.29.134.17). In the same hour, other fork GPT runs went through.

Thanks for the report. This issue has been triaged by Crew Captain and will be assigned to a maintainer, who will follow up here.

Fourth occurrence, third lane: Fork Internal Content Scan run 40054618195 (PR #9750, head d84a781219, 02:33 UTC 10-24). Resolve and validate PR got dial tcp 140.82.116.5:443: connect: connection refused on all 3 tries against api.github.com. The job sat until it was cancelled 20 minutes later, so no Internal Content Scan check was posted, and PR Readiness waits on it while the other 89 checks are green. A maintainer can clear this one with gh run rerun 40054618195 --repo kirodotdev/KiroCrew, since the fork author can't.

Claimed by issue-triage (Wave N11). A worker session is starting a fix now.

The worker will pick the most conservative fix, write the chosen decision here and in the PR body, and keep the PR small so a maintainer can review the decision directly.

🤖 AI-assisted comment 🤖

This is covered by open PR #23705 (famanson, head 8d2602f). Per that PR, systemd-resolved resolves api.github.com through Azure DNS, so harden-runner's DNS proxy never sees the IP and block mode refuses it for about one DNS cache period (~57 s). The old lanes retried for only 3-15 s. #23705 retries every API call on the verdict path in all 7 fork lanes, waiting 5/10/20/40 s (75 s total), and does not touch the egress allowlist.

One gap remains against this issue: after every try fails, the Security Scope Publish check-run step still only warns, so the job stays green. Making that step fail would be a small follow-up on top of #23705.

Maintainers: please link #23705 to this issue (for example Closes #23596 in its body) if you agree. We are not opening a competing PR, and the claimed label is removed.

🤖 AI-assisted comment 🤖

Again on fork PR #9749 (40dd276862, 12:03 UTC 10-25), this time in the Fork Security Scope Review generate job: Resolve and validate PR got dial tcp 140.82.116.6:443: connect: connection refused on all 3 tries (run 40191404842), so the check posted [scope-floor:unsettled] and PR Readiness blocks. Waiting on #23705.

Again on fork PR #9749 (07b80f55bb, 21:03 UTC 10-25): the Fork Security Scope Review generate job got dial tcp 140.82.116.6:443: connect: connection refused on GET /pulls/9749 (run 40233612157) and posted [scope-floor:unsettled]. #23705 still open.

Again on fork PR #24197 (9d79a56554, 03:48 UTC 10-26): both Fork First Principles Review (run 40265811918) and Fork UX Review (run 40265811904) failed Resolve and validate PR with dial tcp 20.29.134.17:443: connect: connection refused on all 3 tries of GET /pulls?state=open. The sibling Fork Design Review from the same Fast Gate event published normally. PR Readiness then held at "UX Review (not started), First Principles Review (not started)". Re-ran both once.

Again on fork PR #9750 (1c804904c0, 2026-10-26 18:53Z): Fork Security Scope Review's generate job could not reach github.com:443 in "Checkout base (trusted)" (Failed to connect to github.com port 443 after 4 ms: Couldn't connect to server, retried until the job was cancelled at 19:27). Publish then reported [scope-floor:unsettled]. Run: https://github.com/kirodotdev/KiroCrew/actions/runs/40380517135

— Sent by an AI Agent on behalf of Amad

Again on fork PR #9750 (0e72233863, 2026-10-26 21:53Z): Fork GPT 6.1 Review could not resolve the PR (dial tcp 140.82.116.6:443: connect: connection refused on all 3 resolver attempts), so it never posted a GPT 6.1 Review check for the head. Run: https://github.com/kirodotdev/KiroCrew/actions/runs/40407031348

— Sent by an AI Agent on behalf of Amad

Again on the Fork Internal Content Scan lane, this time with the check posted as a red NO VERDICT. Fork PR #23415 (head 338af42e7e, 23:19 UTC 10-27), run 40601269523: the job opened its Internal Content Scan check at 23:19:26 and finalized it at 23:22:55 as "NO VERDICT — the scan did not run to completion", so the Scan step never produced a result. GitHub's step list for the job stops at Open check-run (in progress), still in progress, and the job log was never uploaded (the logs endpoint returns BlobNotFound). The job was cancelled at 23:39:02, 20 minutes after it started, which is the job's 15-minute timeout plus 5 minutes. With no log I can't name the failing step. Refused runner traffic to GitHub would explain the frozen step list, the missing log and the missing result together.

These runs of the lane show the same signature (step list frozen at Open check-run or Checkout base (trusted), no job log, cancelled 20 minutes after start): 39966631207 (10-23), then 40516821713, 40553296533, 40561052642, 40586880795, 40586935692 and 40607287285 (10-27 to 10-28).

These runs on 10-27 reached Complete job, which succeeded within about a minute of the start, and the job still stayed open until the same 20-minute cancel: 40508290967, 40512153536, 40554215311, 40563803987, 40566052472, 40578126255 and 40586792532. That matches the Complete job hang that the closing note on #23063 left open for an owner decision on runner egress, now on a second lane.

Again in the Fork Security Scope Review generate job, on fork PR #21031 (head 8f8a08395a, 02:29 UTC 10-28): Resolve and validate PR got dial tcp 140.82.116.5:443: connect: connection refused on all 3 tries against GET /pulls?state=open, so no leg ran and the lane posted [scope-floor:unsettled] (run 40621195189). The job then stayed open until it was cancelled at 03:04.

An earlier one, from before this issue was filed: fork PR #16481 (head be6eace1c2, 16:45 UTC 10-23), same step, 140.82.116.6 (run 39989157891).

Navigation Menu

Repository navigation

Fork AI lanes: api.github.com connections refused under harden-runner, so lanes post no check and PR Readiness is stranded #23596

What

Why it matters

Possible directions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fork AI lanes: api.github.com connections refused under harden-runner, so lanes post no check and PR Readiness is stranded #23596

Description

What

Why it matters

Possible directions

Activity

amadsalmon commented on Oct 24, 2026

KiroCrewCaptain commented on Oct 24, 2026

amadsalmon commented on Oct 24, 2026

iamwhatever commented on Oct 24, 2026

iamwhatever commented on Oct 24, 2026

amadsalmon commented on Oct 25, 2026

amadsalmon commented on Oct 25, 2026

IngridMorstrad commented on Oct 26, 2026

amadsalmon commented on Oct 26, 2026

amadsalmon commented on Oct 26, 2026

jeeshofone commented on Oct 28, 2026

jeeshofone commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions