Skip to content

Fix Windows flake in notifier dead-endpoint e2e test - #1392

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/notifier-dead-endpoint-grace
Oct 11, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/notifier-dead-endpoint-grace

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

update_notifier_e2e::dead_endpoint_never_fails_the_command flakes on the Windows test leg:

panicked at crates\socket-patch-cli\tests\update\update_notifier_e2e.rs:515:5:
a failed check must still rate-limit itself to once a day; lastCheckAt=1791482144 now=1791572146

Seen on run 37972832939 (test (windows-latest, 2), --test update: 52 passed, 1 failed) on an unrelated docs PR. lastCheckAt is the STALE value the test seeds (~25h old), so the state file was never rewritten. It has hit a PR so far, not the merge queue. But this test runs on the Windows leg in merge_group too, and there each flake evicts an entry and costs a full rebuild.

Root cause

The lastCheckAt write happens in refresh_latest, which runs in a detached tokio::spawn. finish joins it under the production 500 ms grace, and then the child calls process::exit. This row used eligible_kit, which keeps the 500 ms grace. On a loaded Windows runner the task can miss the window and get killed before save_state lands. The comment on eligible_kit_await_fetch describes this same race. Every other row that asserts on the background task's effect already uses that helper; this one was missed.

Fix

Use eligible_kit_await_fetch("http://127.0.0.1:1") in this test, and add a doc note explaining why. No assertion changes:

  • The fetch is still capped at 2 s, so the run waits at most about that long.
  • The test still proves that a failed check advances lastCheckAt. Only the pre-fetch write exists on the failure path.
  • grace_budget_bounds_command_latency still pins the real 500 ms ceiling.

Proof

To simulate a runner where the task misses the window, I added SOCKET_UPDATE_GRACE_MS=0 to eligible_kit locally (the change is not committed), then looped the test 20x:

pass fail
main 5 15
this PR 20 0
  • With the real config, the fixed test passed 30/30 with every core saturated by busy loops.
  • cargo test -p socket-patch-cli --test update update_notifier_e2e: 19 passed.
  • cargo clippy -p socket-patch-cli --test update -- -D warnings: clean.
  • rustfmt --check on the touched file: clean.

No tests were removed or moved.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UWAaQAvuYc8RATeyXGm8rs


Generated by Claude Code


Note

Low Risk
Test-only change to e2e harness timing; no production notifier or CLI behavior is modified.

Overview
Fixes intermittent Windows CI failures in dead_endpoint_never_fails_the_command by aligning it with other notifier e2e tests that assert on background-check side effects.

The test now uses eligible_kit_await_fetch (lifted SOCKET_UPDATE_GRACE_MS) instead of eligible_kit, so the child process waits for the detached update check to finish writing lastCheckAt before exit. Assertions are unchanged—the unreachable host still uses a 2s fetch cap; only the join grace is relaxed to avoid reading the seeded STALE timestamp when the runner is slow.

Doc comments on the test explain why the lifted grace is required.

Reviewed by Cursor Bugbot for commit 25c1633. Configure here.


Generated by Claude Code

dead_endpoint_never_fails_the_command asserts that the failed update
check rewrote lastCheckAt, but that write happens in the detached
background task. The row used eligible_kit, which keeps the production
500 ms join grace, so on a loaded Windows runner the child could exit
before the task's save_state ran and the test read back the seeded
STALE value (Windows test leg on run 37972832939).

Use eligible_kit_await_fetch like every other row that asserts on the
background task's effect. The fetch itself stays capped at 2 s, and
grace_budget_bounds_command_latency still covers the real 500 ms
ceiling.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWAaQAvuYc8RATeyXGm8rs
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Oct 11, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 25c1633. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 11, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at 25c1633.

  • CI: 35/35 non-skipped checks green on the head (18 skipped by path filters); mergeable, no conflicts; no main-wide failures.
  • Bugbot: reviewed 25c1633, no findings.
  • Reviewer focus: test-only change in crates/socket-patch-cli/tests/update/update_notifier_e2e.rs; check the dead-endpoint assertion no longer depends on the detached task finishing inside the 500 ms join grace.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. Test-only fix for a Windows flake. dead_endpoint_never_fails_the_command asserts that a failed update check still advances lastCheckAt, but that write happens in the detached check task. Under the default 500 ms join grace, a loaded runner could exit before the write. The test now uses eligible_kit_await_fetch (SOCKET_UPDATE_GRACE_MS=30000), like the other rows that assert on the task's effect.

Risk: low. One test changed, no product code. The endpoint is 127.0.0.1:1 (connection refused), so the longer grace adds no runtime in practice.

Look here

  • update_notifier_e2e.rs#L503-L517:`` the change and its rationale
  • update_notifier_e2e.rs#L105-L109:``` eligible_kit_await_fetch, already used by the five other effect-asserting rows

Verified. Read the diff and the helper. The test's assertions are unchanged; only the grace is lifted, which matches the rows at L136, L203, L235, L301 and L332. CI 53/53 green (35 pass, 18 skipped) on 25c1633, Windows included; Cursor Bugbot: no new issues. CHANGELOG untouched.

Changes I made. None.

Open questions. None.

Auto-merge is armed: approving sends it straight to the merge queue.


Generated by Claude Code

Merged via the queue into main with commit 5d7cd39 Oct 11, 2026
53 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/notifier-dead-endpoint-grace branch October 11, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants