Repository navigation
polis-api: Treevite waves lock tests wait for what they test, not for the runner - #3329
Merged
Merged
Conversation
… the runner Two database tests in post_treevite_waves_tests.rs failed intermittently with PostgreSQL timeouts, on hosted runners and on a quiet x86 box: - allocator_lock_and_real_fk_locks_do_not_cycle timed out (57014) in fixture DDL, before either lock probe ran: setup used the serving connection with its 500 ms statement budget. - concurrent_creates_and_owner_budget_are_serialized raced: the second create could reach the owner lock after the first had committed, or time out waiting for it, so the test sometimes observed no serialization at all. Setup and lock probes now use a test-only connection with 30 s / 10 s statement and lock timeouts; connect() and the real writer pool keep the serving settings. The concurrent test holds the owner advisory lock in an observer, sees both creates waiting on it before releasing, and joins both workers before asserting. Test-only; no handler, schema or serving change. Under 8-process load plus 8 CPU workers, 600 pairs: concurrent creates passed 348 before and 600 after. Handler mutations still fail the tests: a real lock cycle or a member-row FOR UPDATE fails the allocator test; removing the owner lock, or taking it after the cap check, fails the concurrent test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Observable changes: none. Test-only: one file,
queue-rs/polis-api/src/post_treevite_waves_tests.rs. No handler, schema, migration or serving-setting change.What failed
Two database tests failed intermittently with PostgreSQL timeouts, on hosted runners (PR #3317's run, twice) and on a quiet x86 box:
allocator_lock_and_real_fk_locks_do_not_cycle: SqlState 57014 at line 83, the fixture DDL, before either lock probe ran. Setup used the serving connection with its 500 ms statement budget.concurrent_creates_and_owner_budget_are_serialized: a lock timeout (55P03) while the second create waited on the owner lock under the servinglock_timeout. Sampled wait edges showed owner-lock waiting, never a cycle.The change
long_timeout_connect(), a test-only connection with 30 s statement / 10 s lock timeouts.connect()and the real writer pool keep the serving settings.Evidence
FOR UPDATEfails the allocator test 10/10; removing the owner lock, or taking it after the cap check, fails the concurrent test.cargo fmt --checkand clippy (with and withoutdb-tests) clean.Note: in production
lock_timeoutstays 100 ms, so a second concurrent create for the same owner is refused (500create_wave) rather than waiting; the existing pool test covers that and is unchanged.Correction to the commit message: the concurrent test's failures were lock timeouts on the owner lock; "observed no serialization" there overstates it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ss9NfEMZS9Qm4FZ2npXDrx