Repository navigation
fix: races in watch-connection seed, partial sig-share batches, and ChainLock shutdown - #8117
Conversation
|
No actionable comments were generated in the recent review. WalkthroughThe changes update chainlock shutdown so the destructor calls Stop(), which joins the scheduler thread when joinable. The signing-share collection function clears its output maps when a required quorum is missing. Watch-connection seed initialization now uses a function-local static initialized by GetRandHash(). Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk:
|
e6e52d1 to
a3933fa
Compare
thepastaclaw
left a comment
There was a problem hiding this comment.
Final validation — Phase 2 only (queue backlog)
The reviewed changes correctly address the three stated concurrency and shutdown defects, and no blocking correctness issue is evident. Add deterministic regression coverage for the missing-quorum batch contract; the relevant build, lint, and sanitizer checks were still queued in the supplied CI snapshot.
🟡 1 suggestion(s)
Approved: nothing here blocks the merge. The non-blocking findings are optional: fix them forward, in this PR or a follow-up, without waiting for another review round.
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: dash-core-commit-history); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
normalbygpt-6.1-sol(effort low) — Although small, the diff changes concurrent initialization, batch failure handling, and shutdown thread joining, requiring reasoning about caller invariants and lifecycle ordering rather than straightforward local verification. - Phase 1 reviewers: not run (skipped for throughput: 12 PRs queued, above the 10 limit)
- Second Phase-2 pass: on this first full review every Phase-2 reviewer also ran a second, independent time, and the verifier weighed both sets
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— dash-core-commit-history (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort high); agentphase2-reviewer,gpt-6.1-sol— dash-core-commit-history (completed, effort high); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review (PastaClaw). How to work through them:
- Verify each finding against the code; fix only what is real, as narrowly as the finding.
- If a finding is wrong, decline it on its thread in one or two sentences; the bot answers once and stops.
- To defer one, link a tracking issue on its thread (or reply asking the bot to file one).
- Suggestions don't block the merge, but most are worth fixing in this PR; judge each on its merits.
- Keep the PR's scope; don't grow it to satisfy hardening suggestions.
- Carried items were raised in an earlier round: read the thread first, and don't re-fix what you already fixed, declined or deferred there. Disputed and deferred items are already handled and are not listed.
In `src/llmq/signing_shares.cpp`:
- [SUGGESTION] src/llmq/signing_shares.cpp:599-600: Add deterministic regression coverage for incomplete batch suppression
The fix correctly clears both output maps when quorum lookup fails, which prevents NetSigning::WorkThreadDispatcher from dispatching an incomplete batch. However, no test exercises this contract. Add a focused regression that collects a batch containing a quorum unavailable to GetQuorum, verifies that both output maps are empty, and confirms that a later valid batch can still be collected. This is important because the caller uses the boolean return only as a more-work indicator and dispatches whenever the shares map is nonempty; returning false alone would not prevent the original quorums.at() failure.
… static Two first callers of CalcDeterministicWatchConnections could both see the generated flag unset, and one could regenerate the seed while the other was already reading it. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
CollectPendingSigSharesToVerify returned with the shares collected so far but without their quorums. The caller dispatches whenever the share map is non-empty, so verification then hit quorums.at() for a missing key. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Stop() only asked the scheduler to stop, and the thread was joined in the destructor at NodeContext destruction. A task already running could keep going through the rest of shutdown, after mn_sync and the mempool were destroyed, or block in ActivateBestChain on the validation queue of the already stopped main scheduler. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
a3933fa to
9707616
Compare
…atches, and ChainLock shutdown 9707616 fix(chainlock): join the scheduler thread in ChainlockHandler::Stop() (pasta) 547392c fix(llmq): drop the whole sig-share batch when a quorum lookup fails (pasta) 5e524df fix(llmq): initialize the watch-connection seed with a function-local static (pasta) Pull request description: ## Issue being fixed or feature implemented Concurrent first calls can regenerate the watch-connection seed while another caller reads it. A missing quorum during signature-share collection leaves a partial batch that the caller dispatches to code using `quorums.at()`. ChainLock shutdown also returns before its scheduler thread exits. The thread was only joined when the node context was destroyed, so a task already running could continue after `mn_sync` and the mempool were destroyed, or block shutdown in `ActivateBestChain` waiting on the validation queue of the already stopped main scheduler. ## What was done? - Initialize the immutable watch-connection seed with thread-safe function-local static initialization. - Clear both batch outputs when quorum lookup fails, so the caller cannot dispatch an incomplete batch. - Join the ChainLock scheduler thread in `Stop()` and reuse that method from the destructor, guarding repeated joins with `joinable()`. Each fix is its own commit. Reviewed every handler stop/destruction path: production shutdown joins before stopping the main scheduler, with no chain or handler lock held; the full unit fixture calls `Stop()` before later destruction; reindex resets the handler without those locks while the main scheduler runs. The simpler base fixture leaves the handler dormant. No stop/destructor runs on the handler's own scheduler thread. ## How Has This Been Tested? - Debug build with warnings treated as errors: `make -C src -j4 dashd test/test_dash`. - `./src/test/test_dash --run_test='llmq*' --log_level=message`: all 84 cases passed. - `./src/test/test_dash --run_test='validation_chainstatemanager_tests/chainstatemanager,chainstatemanager_snapshot_discarded_on_reindex' --log_level=message`: both cases passed, covering base-fixture teardown and reindex destruction/recreation. - RAM-disk functional runner with an isolated port range and `-j1 --timeout-factor=3`: `feature_llmq_chainlocks.py`, `feature_llmq_signing.py`, and its `--spork21` variant all passed. - Whitespace lint, `git diff --check`, and self-review passed. The seed and partial-batch failure paths were verified by code inspection. No TSAN run or deterministic forced-race reproducer is claimed. ## Breaking Changes None. ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have made corresponding changes to the documentation (not needed for these internal fixes) - [x] I have assigned this pull request to a milestone 🤖 Generated with [Claude Code](https://claude.com/claude-code) Top commit has no ACKs. Tree-SHA512: 0d518624c79f2cdee35b2daa341a5905de3e0a27b63105ca06941baa6e620f53ba1f26a0eebe0c843a5118c4615f680b0da7e4f5798eb22c1aefadd717c02baa (cherry picked from commit 92ed025)
b534385 chore: prepare v24.0.2 release (pasta) 70e06ae Merge #8117: fix: races in watch-connection seed, partial sig-share batches, and ChainLock shutdown (merge-script) d50ea34 Merge #8115: fix: free pooled secure allocations with their allocated size (merge-script) 824e58a Merge #8113: fix(wallet): schedule CoinJoin auto-denomination per wallet and guard an empty round list (merge-script) 068529f Merge #8112: fix: answer govsync only to the requesting peer and index replayed orphan votes (merge-script) 484ec63 Merge #8119: fix(stats): wrong inputValue gauge, counter data race, and null-tip crash (merge-script) 28d5711 Merge #8120: fix(utils): accept -printcrashinfo in dash-cli and dash-tx (merge-script) 578b914 Merge bitcoin-core/gui#770: Revert "gui: provide wallet controller context to wallet actions" (Konstantin Akimov) d6902fa Merge bitcoin-core/gui#765: Fix wallet list hover crash on shutdown (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented Backports and release prep for v24.0.2, on top of `v24.0.x` at 137d999 (v24.0.1). One PR carries both, so the release branch gets a single merge. Two fixes are the reason for the release: - **Memory leak (#8115).** Since v24.0.0 a node's locked-memory use grows for as long as it runs. On `v24.0.x` BLS still uses relic, where a G2 element is 296 bytes; the pooled secure allocator frees such an allocation with one chunk fewer than it allocated. The batch-verification weighting added in v24.0.0 makes one such allocation per verified message, so the pool keeps growing until restart. On develop (blst, 288-byte elements) the same code path does not leak, which is why #8115 describes the bug as latent. - **macOS crash on close (#8073's bitcoin-core/gui#765 and gui#770).** On macOS the menu bar stays usable after the main window is hidden for shutdown. `File > Open Wallet`, `Close Wallet` and the other wallet items then call the wallet controller after it has been deleted. The rest are small fixes that were already merged on develop and are low risk to carry. ## What was done? ### Picked In develop merge order. The Dash PRs are picked from their develop merge commits with `git cherry-pick -m1 -x`; the two GUI backports are picked with `-x` from the commits #8073 merged, like earlier partial picks on this branch. - bitcoin-core/gui#765 Fix wallet list hover crash on shutdown, and bitcoin-core/gui#770, which reverts its first commit (from #8073). Together they leave one change: `appMenuBar->clear()` when shutdown starts. - #8120 fix(utils): accept -printcrashinfo in dash-cli and dash-tx - #8119 fix(stats): wrong inputValue gauge, counter data race, and null-tip crash - #8112 fix: answer govsync only to the requesting peer and index replayed orphan votes - #8113 fix(wallet): schedule CoinJoin auto-denomination per wallet and guard an empty round list - #8115 fix: free pooled secure allocations with their allocated size - #8117 fix: races in watch-connection seed, partial sig-share batches, and ChainLock shutdown Then one commit, `chore: prepare v24.0.2 release`: - `configure.ac`: `_CLIENT_VERSION_BUILD` 1 → 2. - `doc/release-notes.md`: v24.0.2 notes. The v24.0.1 notes are archived verbatim to `doc/release-notes/dash/release-notes-24.0.1.md` and added to "Older releases". - `contrib/flatpak/org.dash.dash-core.metainfo.xml`: `24.0.2` entry dated 2026-11-13. The date needs changing if the tag is cut on another day. - `doc/man/*.1`: version string only. `-printcrashinfo` (#8120) is a hidden option, so no help text changed. ### Adaptations Every pick changes the same lines as its develop counterpart except: - **#8073:** only the two GUI commits are taken. The other two backports in that PR (bitcoin#28629, partial bitcoin#28639) are test and refactor changes for code `v24.0.x` doesn't need. - **#8112:** `test/functional/feature_governance.py` differs on `v24.0.x` (no pruned node from #7860). The `v24.0.x` file is kept and only #8112's additions are applied: the imports it needs, `GovernanceSyncPeer`, `check_govsync_is_peer_local()` and its call. The source change is identical. - **#8113:** one line of the new unit test. `v24.0.x` has the five-argument `CWallet` constructor and `CreateMockWalletDatabase()`, so the second wallet is built the way the fixture in the same file builds the first. The source change is identical. - **#8117:** in `src/llmq/utils.cpp` the neighbouring includes differ; the pick removes `<atomic>` and adds nothing. The source change is otherwise identical. ### Not picked - #8111 (check every BIP39 checksum bit): it can refuse a mnemonic that v24.0.1 accepts, so it should come with a major or minor release and a release note, not a patch release. - #8118 (RPC argument and locking fixes): changes RPC error behaviour and needs a hand-port to `rpc/masternode.cpp`. - #8122 (CacheMultiMap): the use-after-free can't be reached on `v24.0.x`, and the capacity part is hardening with no behaviour change. - #8047 (index startup after an unclean stop): on `v24.0.x` a synced node never records the uncommitted tip it handles, and the governance index it mainly covers doesn't exist here. - #8092 (wording): changes two translatable strings. - #8101, #8102, #8098, #8095 with #8121, #8116 with #7815, #8085: unchanged from the v24.0.1 decision in #8128. - Features, refactors, Bitcoin Core backports, CI and test-only changes merged since the branch point. #8103 (EvoDB left ahead of the coins DB after a reindex) is a candidate that is still open on develop. If it merges before this PR does, its pick will be added here. ## How Has This Been Tested? Linux x86_64, depends build with `--enable-debug --enable-werror`, GUI enabled. Head is b534385. - `dashd`, `dash-qt`, `test_dash` and `test_dash-qt` build cleanly; `dashd -version` reports `v24.0.2`. - `test_dash` (1106 cases), `test_dash-qt` and `test/util/test_runner.py` pass. That includes the picked unit tests: `allocator_tests/mt_pooled_secure_allocator_reuses_all_chunks` (#8115), `coinjoin_tests/coinjoin_maintenance_runs_for_every_wallet` (#8113) and the orphan-vote case in `governance_vote_processing_tests` (#8112). - The full functional suite (`test_runner.py -j14`): 312 passed, 12 skipped, 1 failed. The skipped ones are the usual environment-dependent tests (`feature_bind_port_*`, `rpc_bind.py --ipv6`, `interface_usdt_*`, previous-release compatibility and `feature_unsupported_utxo_db.py`). The run includes the scenarios the picks add or extend: `feature_governance.py` (#8112), `feature_stats.py` (#8119) and `interface_bitcoin_cli.py` (#8120). - The failure was `feature_llmq_is_cl_conflicts.py`, in `mine_cycle_quorum`: the node logs show the block for the final commitments being built about 40 ms before the second quorum's final commitment reached the mining node, so that quorum was not mined. It is a timing race in the test helper under load, in code none of the picks touch (develop reworked those waits in #7892). Run again on its own four times in parallel, it passed all four. - **Leak, end to end.** A regtest script (not part of the tree) mines a single-member InstantSend quorum, sends InstantSend-locked transactions and reads `getmemoryinfo` `locked.used` on each node every 50 locks. - The published v24.0.1 Linux binary: the four non-masternode nodes go from 39,568 bytes to 467,664 bytes over 600 locks, and are still growing at the end. - This branch: the same nodes stay at 20,096 bytes for all 600 locks. - The man pages were regenerated with `contrib/devtools/gen-manpages.py` from this build. They are byte-identical to the committed ones except for the `lock` debug category in `dashd.1` and `dash-qt.1`, which only an `--enable-debug` build lists. - The archived v24.0.1 notes are byte-identical to the previous `doc/release-notes.md`. Not tested: the macOS shutdown crash itself. The GUI change is the upstream one-liner and was built and run through `test_dash-qt` on Linux only. ## Breaking Changes None. No consensus change, and no change to which P2P messages a node accepts or how it scores peers. A node that syncs governance from a v24.0.2 node receives the same inventories as before; other peers of that node no longer receive them. ## Checklist: - [x] I have performed a self-review of my own code - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(if you have permission to set one)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Top commit has no ACKs. Tree-SHA512: 6d9861f4a8c2341889bb0ce5dd500a7083ea59395777a21c5e5d01e70ec8e8b489cf6425bc26ca79f15b1919610ceab10baada361043322dc689b7dfcae7cb5d
Issue being fixed or feature implemented
Concurrent first calls can regenerate the watch-connection seed while another caller reads it. A missing quorum during signature-share collection leaves a partial batch that the caller dispatches to code using
quorums.at(). ChainLock shutdown also returns before its scheduler thread exits. The thread was only joined when the node context was destroyed, so a task already running could continue aftermn_syncand the mempool were destroyed, or block shutdown inActivateBestChainwaiting on the validation queue of the already stopped main scheduler.What was done?
Stop()and reuse that method from the destructor, guarding repeated joins withjoinable().Each fix is its own commit.
Reviewed every handler stop/destruction path: production shutdown joins before stopping the main scheduler, with no chain or handler lock held; the full unit fixture calls
Stop()before later destruction; reindex resets the handler without those locks while the main scheduler runs. The simpler base fixture leaves the handler dormant. No stop/destructor runs on the handler's own scheduler thread.How Has This Been Tested?
make -C src -j4 dashd test/test_dash../src/test/test_dash --run_test='llmq*' --log_level=message: all 84 cases passed../src/test/test_dash --run_test='validation_chainstatemanager_tests/chainstatemanager,chainstatemanager_snapshot_discarded_on_reindex' --log_level=message: both cases passed, covering base-fixture teardown and reindex destruction/recreation.-j1 --timeout-factor=3:feature_llmq_chainlocks.py,feature_llmq_signing.py, and its--spork21variant all passed.git diff --check, and self-review passed.The seed and partial-batch failure paths were verified by code inspection. No TSAN run or deterministic forced-race reproducer is claimed.
Breaking Changes
None.
Checklist:
🤖 Generated with Claude Code