Skip to content

test: write the idle-seed fixtures off the loop so the replace retry applies - #25779

Closed
Mening-dotcom wants to merge 1 commit into
kirodotdev:mainfrom
Mening-dotcom:test/idle-seed-writes-off-loop
Closed

Mening-dotcom wants to merge 1 commit into
kirodotdev:mainfrom
Mening-dotcom:test/idle-seed-writes-off-loop

Conversation

@Mening-dotcom

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The idle-seed tests stop failing a Windows shard when a scanner briefly holds a freshly written temp file.

test_private_transcript_is_not_seeded[temporary] failed Backend Tests (Windows) (2) with PermissionError: [WinError 5] from atomic_write.py's replace, on a PR that changes label text only (#24915; run 40569051980). The traceback runs test → _restarted → update_metadata → _update_metadata_locked → the replace. The cause is the test setup's position, not the writer: replace_with_retry deliberately skips its bounded Windows retry when a loop is running in the thread (re-raising instead of sleeping on the gateway loop), and _restarted performs its transcript writes synchronously inside an async test under allow_on_loop_persist() — on the loop. One transient AV/indexer handle on the fresh temp therefore fails the test on the first attempt, where the same write off the loop would have retried through the 10 × 50 ms budget and passed. (#24977)

Why it matters

The shard was red on a PR that touched nothing near history consolidation — exactly the kind of unrelated-red that trains people to re-run CI instead of reading it, on the OS where rename contention is routine.

Not a goal

  • No change to replace_with_retry or its on-loop re-raise: that behavior is the repo's no-blocking-call-on-event-loop anchor working as documented, and the documented remedy is for the caller to offload.
  • No change to the file's other three allow_on_loop_persist blocks: _write_row's single append and the one-row delete are each the behavior under test (same-process writer semantics, the dashboard delete), and the no-op mark_consolidated(key, 0) is asserted byte-identical — none is a fixture bulk-write, and none is the frame the flake fired in.

What changed (motivation → approach → change)

Observed: the two fixture writers (_restarted, _live) persist transcripts on the event loop, where the Windows replace retry cannot apply. Root cause: the setup stands for "an earlier process wrote these transcripts", and an earlier process has no event loop — the on-loop position is an artifact of calling the sync helper from an async test. Change: each helper's write block now runs in a bounded worker (ThreadPoolExecutor, result(timeout=30) so exceptions propagate and a hang fails rather than deadlocks). Off the loop, the writes earn replace_with_retry's bounded retry, and the allow_on_loop_persist() hatch becomes unnecessary for these blocks and is dropped there (it remains where the on-loop position is the point).

Backwards compatibility

Test-only; fixture semantics unchanged (same records, same files — written by a worker thread exactly as the "earlier process" they model would have). All 29 tests in the file pass on native Windows; the flake-pattern ratchet passes (test_flake_pattern_ratchet, 62 passed).

Tests

  • test/test_history_consolidation_idle_seed.py on native Windows (the failing platform): 29 passed.
  • python -m pytest -n0 test/test_flake_pattern_ratchet.py: 62 passed.
  • scripts/check_comment_history.py, black --check, isort --check-only clean.

Manual verification

N/A — test-only change, verified by running the file on the platform the flake reports.

Screenshots / video

N/A — no user-facing surface.

Related Issues

Fixes #24977

Pattern harvest

Rule candidate: review-prompt
Pattern: a sync fixture helper called from an async test runs its file writes ON the event loop, where replace_with_retry deliberately re-raises instead of retrying — any allow_on_loop_persist() around fixture bulk-writes is a sign the setup should be a bounded worker-thread call instead.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions accepted under the project's Apache-2.0 license per CONTRIBUTING.md.

@Mening-dotcom
Mening-dotcom requested review from a team and KiroCrewCaptain as code owners October 29, 2026 13:20
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Oct 29, 2026
@Mening-dotcom

Copy link
Copy Markdown
Contributor Author

Closing: current main already settles this — #23788 moved the idle-seed fixture writes into an _off_loop worker helper (same diagnosis: on the loop, replace_with_retry re-raises instead of retrying the Windows rename), so this branch now duplicates it. Note for #24977 triage: the fix is on main via #23788 and the issue can likely be closed.

@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky: test_private_transcript_is_not_seeded PermissionError on Windows

1 participant