Repository navigation
Flaky: test_two_concurrent_releases_each_prune_their_own_view sees a lingering view since #23913 #23965
Description
test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view failed twice on 2026-10-25, on two PRs that touch neither skill projection nor this test. Both runs tested trees that contain #23913, merged at 06:03 UTC.
- Run: https://github.com/kirodotdev/KiroCrew/actions/runs/40169359209/job/121099126068 (Backend Tests (Windows) (2), PR feat(chat): take back a steer the running turn has not read yet #23388)
- OS: Windows, windows-latest, Python 3.12
- Run: https://github.com/kirodotdev/KiroCrew/actions/runs/40168203653/job/121096654929 (Backend Tests (3.12, 2), PR fix(desktop): discover the system proxy for a GUI-launched gateway #19996, merged with main at 03f3b2a)
- OS: Linux, ubuntu-latest, Python 3.12
- Failing line, both runs:
test/test_native_skill_projection_rewrites.py:1533,assert not first_alias.exists(), "the first release's view lingered"
The cause is a test synchronization gap that #23913 opened. With _coalesced_end_prune (src/kiro_crew/acp/skill_projection_parts/alias_reclaim.py), an end that arrives while another end's prune runs only records a request. The running pass then starts the follow-up pass on a new kirocrew-skill-view-prune thread. The test builds its join list once, right after both release() calls, so it never joins that thread and can assert before the follow-up pass prunes the view. Before #23913 each release pruned on its own thread, and the join list held both.
Reproduced locally by forcing that order with delays only: the first release waits 0.4 s before its end, the first end prune stays open 1.5 s after its walk, and the follow-up pass waits 0.5 s before it walks.
| Tree | Result |
|---|---|
| main at 4a50525, delays on | fails 3 of 3 at :1533, same message as CI |
| 95640ac (parent of #23913's merge commit 47e0b8f), delays on | passes 3 of 3 |
| main at 4a50525, no delays | passes 5 of 5 |
The view is not leaked. After the forced failure, waiting for every kirocrew-skill-view-prune thread leaves no view on disk (2 of 2 runs). So the fix belongs in the test: wait until no end prune is running or requested for the target (_END_PRUNE_RUNNING and _END_PRUNE_REQUESTED empty) instead of joining a thread list captured once.
Activity
IngridMorstrad commentedon Oct 25, 2026
Third occurrence, on PR #23946 (touches only
cloud/aws.pysubprocess decoding, not skill projection):test/test_native_skill_projection_rewrites.py:1498--AssertionError: the first release's view lingeredPearcekieser commentedon Oct 25, 2026
Occurrence on PR #23738 (head a0c2491), Linux, Backend Tests 3.12 shard 2: https://github.com/kirodotdev/KiroCrew/actions/runs/40174914309. AssertionError: the second concurrent release skipped its prune on the busy in-proc view. It passes 3/3 locally on that head, and the PR does not touch the skill-projection module.
IngridMorstrad commentedon Oct 25, 2026
Seen again on PR #23951, head e507272, run https://github.com/kirodotdev/KiroCrew/actions/runs/40181510127 attempt 1. It failed on both Linux (Backend Tests (3.12, 2)) and Windows (Backend Tests (Windows) (2)):
AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered. The PR's diff only touchesensure-python.sh, its test, anddocs/guides/install.md.unstablebrainiac commentedon Oct 25, 2026
Two more occurrences, on PRs that touch neither skill projection nor this test:
test/test_native_skill_projection_rewrites.py:1533, "the first release's view lingered"test/test_native_skill_projection_rewrites.py:1534, "the second concurrent release skipped its prune on the busy in-process lock and its view lingered"The Windows run fails the second assertion rather than the first. The cause is the same: whichever release arrives while the other end's prune pass runs is pruned by the follow-up pass on a third
kirocrew-skill-view-prunethread, and the test never joins that thread.IngridMorstrad commentedon Oct 25, 2026
Another hit on PR #23961 (head 3c1e063, which touches only apps/registry_pipeline/manifests.py). Run: https://github.com/kirodotdev/KiroCrew/actions/runs/40181405024/job/121136036534. OS: ubuntu-latest, Python 3.12, Backend Tests (3.12, 2). Failing line: test/test_native_skill_projection_rewrites.py:1498, AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered.
CrysisDeu commentedon Oct 25, 2026
Two more hits on PR #23615 (head a6cca6f, touches only the dashboard usage/identity handlers and auth/bridge, not skill projection), Linux, Backend Tests (3.12, 2): run https://github.com/kirodotdev/KiroCrew/actions/runs/40181606209 attempts 1 and 2. Same assertion: the first release's view lingered.
bolichen97 commentedon Oct 25, 2026
Seen again on #23333 head
4b17d70492(basea33907dfe0), a PR that does not touch skill projection:-n autoshard 2/8test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the first release's view lingeredPasses 5/5 serially (
-n0) on the same head locally. Rerunning that one job once.[operator: bolichen97]
IngridMorstrad commentedon Oct 25, 2026
Seen again on PR #23951, run https://github.com/kirodotdev/KiroCrew/actions/runs/40181510127 attempt 2 (re-run of the failed jobs only). Backend Tests (3.12, 2) on Linux failed with the same
AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered. The re-run of Backend Tests (Windows) (2) passed.amadsalmon commentedon Oct 25, 2026
Two more hits from fork PRs on 10-25: #9749 (
a99b1329aa) in Backend Tests (3.12, 2) (job 121140703861) and Backend Tests (Windows) (2) (job 121140703829), and #9750 (c9a5067719) in Backend Tests (3.12, 2) (job 121146632378). Same assertions ("the first release's view lingered" / "the second concurrent release skipped its prune on the busy in-process lock"). Neither PR touches skill projection.bolichen97 commentedon Oct 25, 2026
[operator: bolichen97] Another hit on an unrelated PR (#23995, backend-only change to
slot_note.py): https://github.com/kirodotdev/KiroCrew/actions/runs/40186187992/job/121149658941 -- Linux, Backend Tests (3.12, 2),test/test_native_skill_projection_rewrites.py:1533AssertionError: the first release's view lingered. Passes 30/30 locally in isolation on main 27cb084. Rerunning the failed job once.IngridMorstrad commentedon Oct 25, 2026
Seen again on PR #23951, head b4ee7fe, run https://github.com/kirodotdev/KiroCrew/actions/runs/40191837996 attempt 1. Windows (Backend Tests (Windows) (2)) failed with the same lingering-view assertion.
Pearcekieser commentedon Oct 25, 2026
Another report from a PR that does not touch native skill projection.
test/test_native_skill_projection_rewrites.py:1498AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingeredPR #23937 changes only
src/kiro_crew/apps/proxy_auth.py,src/kiro_crew/apps/routes_proxy.py, their tests and docs.26 remaining items
IngridMorstrad commentedon Oct 25, 2026
The one allowed rerun on #24109 hit it again: https://github.com/kirodotdev/KiroCrew/actions/runs/40230512331/job/121296323206 (Backend Tests (3.12, 2), attempt 2, same head 45b2986, Linux / Ubuntu 24.04 / Python 3.12). Same line,
test/test_native_skill_projection_rewrites.py:1534, 1 failed / 31014 passed. The test passes 10/10 when run alone on that branch, and shard 2 is green on main push run 40232491507 (badd8f5). So it is intermittent and depends on load, not on this diff. I am not re-running again.IngridMorstrad commentedon Oct 25, 2026
Again on #24101 (head 5e70966), both Backend Tests (3.12, 2) https://github.com/kirodotdev/KiroCrew/actions/runs/40234505787/job/121296187401 and Backend Tests (Windows) (2) https://github.com/kirodotdev/KiroCrew/actions/runs/40234505787/job/121296187504, same assertion. It passes 5/5 locally in isolation on that head; the PR touches only dev_fleet pod-guard code.
Pearcekieser commentedon Oct 25, 2026
Another occurrence on #17624, head 6e1abcb, Backend Tests (3.12, 2), 2026-10-25 22:06Z. The second release's view lingered at
test/test_native_skill_projection_rewrites.py:1534. 31,015 other tests in the shard passed, and the PR changes no skill projection code.Job: https://github.com/kirodotdev/KiroCrew/actions/runs/40233509139/job/121300344808
IngridMorstrad commentedon Oct 25, 2026
Hit on PR #24121 (head 58ba2e8, which touches only taskrunner_parts/run_lifecycle.py, a new taskrunner test file and the sync-io baseline): https://github.com/kirodotdev/KiroCrew/actions/runs/40236900084/job/121303827754 (Backend Tests (3.12, 2), ubuntu-latest, Python 3.12). Failing line: test/test_native_skill_projection_rewrites.py:1534
AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered. It was the only failure in the group (1 failed, 31012 passed).IngridMorstrad commentedon Oct 25, 2026
Another hit, on PR #24118 (touches only
apps/registry_pipeline/install_clone.py, its test and the sync-io baseline): https://github.com/kirodotdev/KiroCrew/actions/runs/40236527445/job/121302657128 (Backend Tests (3.12, 2), ubuntu-latest, Python 3.12, head 29d2709 on main badd8f5). Failing line this time istest/test_native_skill_projection_rewrites.py:1534: "the second concurrent release skipped its prune on the busy in-process lock and its view lingered".IngridMorstrad commentedon Oct 25, 2026
Another hit, on PR #24119 (touches only
dashboard/handlers/agent_templates.pyand its endpoint test): https://github.com/kirodotdev/KiroCrew/actions/runs/40236728343/job/121303466339 (Backend Tests (3.12, 2), ubuntu-latest, Python 3.12, head c26d480 on main badd8f5). Same failure:test_two_concurrent_releases_each_prune_their_own_view, "the second concurrent release skipped its prune on the busy in-process lock and its view lingered". It was the only failure in the group (1 failed, 31012 passed). #24142 looks like the fix.IngridMorstrad commentedon Oct 25, 2026
Second hit on #24111, on its one rerun (attempt 2, same head 29017a9), Linux, Backend Tests (3.12, 2): https://github.com/kirodotdev/KiroCrew/actions/runs/40232969063/job/121306625841. Same assertion at
test/test_native_skill_projection_rewrites.py:1498("the second concurrent release skipped its prune on the busy in-process lock and its view lingered").I ran it locally in isolation (
-n0), interleaving the PR's parentbadd8f5aa8with the PR head, 25 runs each: 7/25 red on the parent without the PR's commit, 2/25 red on the head. So it reproduces without the PR. I will not rerun again. #24142 is open to fix it.IngridMorstrad commentedon Oct 25, 2026
The one allowed rerun on #24101 (attempt 2, same head 5e70966) hit it again on Backend Tests (Windows) (2): https://github.com/kirodotdev/KiroCrew/actions/runs/40234505787/job/121310137808. Linux shard 2 passed on the rerun.
IngridMorstrad commentedon Oct 25, 2026
Another hit on PR #24126 (head a0f5ad4), Backend Tests (3.12, 2), ubuntu: https://github.com/kirodotdev/KiroCrew/actions/runs/40239091301/job/121310145075
FAILED test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingeredThat PR only touches
dashboard/handlers/updates_channel.pyandtest/test_changelog_handler.py. Re-running the failed job once.IngridMorstrad commentedon Oct 25, 2026
The one allowed rerun on #24118 (attempt 2, same head 29d2709) hit it again, Linux, Backend Tests (3.12, 2): https://github.com/kirodotdev/KiroCrew/actions/runs/40236527445/job/121315920987
IngridMorstrad commentedon Oct 25, 2026
Another hit on PR #24154 (head 63b0ad6, diff touches only
slack/events_home_tab.pyand its test), Backend Tests (3.12, 2), ubuntu: https://github.com/kirodotdev/KiroCrew/actions/runs/40242604739/job/121320140478FAILED test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingeredIngridMorstrad commentedon Oct 25, 2026
Another hit on PR #24152 (head 32ec7ba), Backend Tests (3.12, 2), ubuntu: https://github.com/kirodotdev/KiroCrew/actions/runs/40242441094/job/121320432484
FAILED test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered#24152 changes only
src/kiro_crew/apps/routes_listing.pyandtest/test_app_config_self_heal.py.IngridMorstrad commentedon Oct 25, 2026
The one allowed rerun on #24126 (attempt 2, same head a0f5ad4) hit it again, Linux, Backend Tests (3.12, 2): https://github.com/kirodotdev/KiroCrew/actions/runs/40239091301/job/121323506453. Same assertion ("the second concurrent release skipped its prune on the busy in-process lock and its view lingered"), 1 failed / 31016 passed. That PR touches only
dashboard/handlers/updates_channel.pyandtest/test_changelog_handler.py.IngridMorstrad commentedon Oct 25, 2026
Another hit on PR #24164 (head 6fc1b58, diff touches only
ops_mission_control/backend/dispatch.pyand its test), Backend Tests (3.12, 2), ubuntu: https://github.com/kirodotdev/KiroCrew/actions/runs/40243510759/job/121323655863FAILED test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered(test_native_skill_projection_rewrites.py:1498).IngridMorstrad commentedon Oct 26, 2026
Another hit on PR #24176 (head 39a7bd5, diff touches only
ops_mission_control/backend/providers/github_issues.pyand its test), Backend Tests (3.12, 2), ubuntu: https://github.com/kirodotdev/KiroCrew/actions/runs/40245106848/job/121327593579FAILED test/test_native_skill_projection_rewrites.py::test_two_concurrent_releases_each_prune_their_own_view - AssertionError: the second concurrent release skipped its prune on the busy in-process lock and its view lingered(test_native_skill_projection_rewrites.py:1498).