Skip to content

backport: bitcoin#26684 (partial), #28921, #29458, #28339, #28770, #30344, #28263, #28209 - #8178

Open
PastaPastaPasta wants to merge 8 commits into
dashpay:developfrom
PastaPastaPasta:backport-cmake-registrations
Open

PastaPastaPasta wants to merge 8 commits into
dashpay:developfrom
PastaPastaPasta:backport-cmake-registrations

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

The CMake import in #7481 (bitcoin#30454) had to leave out several test, bench and fuzz registrations because the upstream PRs that add those source files had not been backported. Most notably, the IPC test block in src/test/CMakeLists.txt was commented out, so -DWITH_MULTIPROCESS=ON built no IPC test at all. This PR backports the small, self-contained upstream PRs behind those gaps, so the registrations now exist where upstream's CMake tree has them.

What was done?

One commit per upstream PR (cherry-pick -m1 -x of the upstream merge). Upstream's Makefile.am/Makefile.*.include hunks become edits to the matching CMakeLists.txt, in the place upstream's CMake tree has them.

No new Dash-specific files are added, so test/util/data/non-backported.txt is unchanged.

How Has This Been Tested?

Two builds, each with the depends toolchain, Ninja, -DWERROR=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON:

  • depends NO_QT=1 (GCC 15):
    • ctest: 184/184 passed.
    • bench_dash -sanity-check passed for ReadBlockFromDiskTest, HexParse and CheckBlockIndex.
    • The new fuzz targets and validation_load_mempool passed against batches of random inputs. For block_index, a breakpoint confirmed that inputs reached LoadBlockIndexGuts and every assert.
    • Functional tests passed: mempool_persist, feature_reindex, feature_reindex_readonly, feature_loadblock, rpc_invalidateblock, rpc_getchaintips, rpc_blockchain, feature_block, p2p_unrequested_blocks, p2p_invalid_block, p2p_sendheaders, feature_assumevalid, feature_llmq_chainlocks, feature_llmq_is_cl_conflicts, feature_dip3_deterministicmns, mempool_reorg, mining_basic, feature_shutdown. On regtest these all run the reworked CheckBlockIndex after every block-tree change.
  • depends NO_QT=1 MULTIPROCESS=1 with clang-19, plus -DWITH_MULTIPROCESS=ON: the build passes, ipc_tests passes, and ctest passed 185/185.
  • test/lint/lint-circular-dependencies.py and lint-includes.py pass.

Breaking Changes

-checkblockindex (a debug-only option) now takes a frequency, as upstream: -checkblockindex=<n> checks on average every n operations, 0 disables it, and a bare -checkblockindex still means 1. Defaults are unchanged.

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone (if you have permission to set one)

🤖 Generated with Claude Code

achow101 and others added 8 commits November 16, 2026 06:06
1c4b9cb bench: add readblock benchmark (Andrew Toth)

Pull request description:

  Requested in bitcoin#13151 (comment).
  See bitcoin#26415 and bitcoin#21319.

  Benchmarking shows a >50x increase in speed on both nvme and spinning disk.

  Benchmark results:
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        5,377,375.00 |              185.96 |    0.2% |   60,125,513.00 |   11,633,676.00 |  5.168 |   3,588,800.00 |    0.4% |      0.09 | `ReadBlockFromDiskTest`

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |           89,945.58 |           11,117.83 |    0.7% |       12,743.90 |       64,530.33 |  0.197 |       2,595.20 |    0.2% |      0.01 | `ReadRawBlockFromDiskTest`

ACKs for top commit:
  maflcko:
    lgtm ACK 1c4b9cb
  achow101:
    ACK 1c4b9cb
  TheCharlatan:
    ACK 1c4b9cb

Tree-SHA512: 71dbcd6c7e2be97eb3001e35d0a95ef8e0c9b10dc9193025c7f8e11a09017fa2fbf89489b686353cd88fb409fb729fe2c4a25c567d2988f64c9c164ab09fba9f
(cherry picked from commit 00bf4a1)

Dash adaptation: the benchmark reads Dash's bundled block813851 instead of
block413567, deserializes it without TX_WITH_WITNESS, and is registered in
src/bench/CMakeLists.txt instead of Makefile.bench.include.

Partial: ReadRawBlockFromDiskTest is left out because Dash's BlockManager has
no ReadRawBlockFromDisk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6acec6b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fc multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee23 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)

Pull request description:

  Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

  The non-test code in this PR was previously part of bitcoin#10102 and has been split off for easier review, but the test code is new.

  ---

  This PR is part of the [process separation project](bitcoin#28722).

ACKs for top commit:
  achow101:
    ACK 6acec6b
  dergoegge:
    reACK 6acec6b

Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
(cherry picked from commit 2f218c6)

Dash adaptations:
- The Makefile.am / Makefile.test.include hunks become the IPC test block in
  src/test/CMakeLists.txt, restored as upstream's CMake tree has it (bitcoin#30454).
  The build_msvc hunk is dropped; Dash has no MSVC build.
- Dash's SpanReader still takes a serialization type and version
  (bitcoin#28451 is not backported), so CustomReadField passes
  SER_NETWORK, CLIENT_VERSION.
- Dash has no Txid type (bitcoin#28107), so the test builds the COutPoint
  from a plain uint256.
- Dash's libmultiprocess pin already carries the update from bitcoin#31105,
  whose ipc_test.cpp hunk (m_rpc_system is now a std::optional) Dash's
  partial bitcoin#31105 left out because the file did not exist yet; it is applied
  here so the test builds against that pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…void resizing

a19235c Preallocate result in `TryParseHex` to avoid resizing (Lőrinc)
b7489ec Add benchmark for TryParseHex (Lőrinc)

Pull request description:

  This pull request introduces optimizations to the `TryParseHex` function, focusing primarily on the ideal case (valid hexadecimal input without spaces).
  A new benchmark, `HexParse` was introduced in a separate commit.

  The main optimization preallocates the result vector based on the input string's length. This aims to completely avoid costly dynamic reallocations when no spaces are present.

  ------------

  Before:
  ```
  |           ns/base16 |            base16/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |                1.60 |      623,238,893.11 |    0.3% |      0.01 | `HexParse`
  |                1.65 |      606,747,566.34 |    0.6% |      0.01 | `HexParse`
  |                1.60 |      626,149,544.07 |    0.3% |      0.01 | `HexParse`
  ```

  After:
  ```
  |           ns/base16 |            base16/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |                0.68 |    1,465,555,976.27 |    0.8% |      0.01 | `HexParse`
  |                0.68 |    1,472,962,920.18 |    0.3% |      0.01 | `HexParse`
  |                0.68 |    1,476,159,423.00 |    0.3% |      0.01 | `HexParse`
  ```

ACKs for top commit:
  achow101:
    ACK a19235c
  hebasto:
    ACK a19235c.
  andrewtoth:
    Re-ACK a19235c
  Empact:
    Re-ACK bitcoin@a19235c

Tree-SHA512: e09a59791104be3fd1026862ce98de9efafa1f949626fa01e3b7d58e6a2ef02a11f0de55ddba5c43230a53effd24e6d368c1e12848b17e8ce91d7908a59333f0
(cherry picked from commit 6dda050)

Dash adaptation: the benchmark is registered in src/bench/CMakeLists.txt
instead of Makefile.bench.include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5bc2077 validation: allow to specify frequency for -checkblockindex (Martin Zumsande)
d5a631b validation: improve performance of CheckBlockIndex (Martin Zumsande)
32c8041 bench: add benchmark for checkblockindex (Martin Zumsande)

Pull request description:

  `CheckBlockIndex() ` are consistency checks that are currently enabled by default on regtest.

  The function is rather slow, which is annoying if you
  * attempt to run it on other networks, especially if not fully synced
  * want to generate a long chain on regtest and see block generation slow down because you forgot to disable `-checkblockindex` or don't know it existed.

  One reason why it's slow is that in order to be able to traverse the block tree depth-first from genesis, it inserts pointers to all block indices into a `std::multimap` - for which inserts and lookups become slow once there are hundred thousands of entries.
  However, typically the block index is mostly chain-like with just a few forks so a multimap isn't really needed for the most part. This PR suggests to store the block indices of the chain ending in the best header in a vector instead, and store only the rest of the indices in a multimap. This does not change the actual consistency checks that are being performed for each index, just the way the block index tree is stored and traversed.

  This adds a bit of complication to make sure each block is visited (note that there are asserts that check it), making sure that the two containers are traversed correctly, but it speeds up the function considerably:

  On master, a single invocation of `CheckBlockIndex` takes ~1.4s on mainnet for me (4.9s on testnet which has >2.4 million blocks).
  With this branch, the runtime goes down to ~0.27s (0.85s on testnet).This is a speedup by a factor ~5.

ACKs for top commit:
  achow101:
    ACK 5bc2077
  furszy:
    ACK 5bc2077
  ryanofsky:
    Code review ACK 5bc2077. Just added suggested assert and simplification since last review

Tree-SHA512: 6b9c3e3e5069d6152b45a09040f962380d114851ff0f9ff1771cf8cad7bb4fa0ba25cd787ceaa3dfa5241fb249748e2ee6987af0ccb24b786a5301b2836f8487
(cherry picked from commit 891e4bf)

Dash adaptations:
- Dash's CheckBlockIndex has no snapshot (assumeutxo) bookkeeping, so the
  snap_update_firsts() call next to the reworded comment is not added.
- The -checkblockindex help text takes upstream's new wording, which also
  repairs Dash's garbled "for the block tree, and  occasionally".
- The benchmark is registered in src/bench/CMakeLists.txt instead of
  Makefile.bench.include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ncy from validation.cpp

fa7d319 refactor: Remove unused circular include dependency from validation.cpp (MarcoFalke)

Pull request description:

  Also, sort includes

ACKs for top commit:
  ismaelsadeeq:
    ACK fa7d319
  hebasto:
    ACK fa7d319
  TheCharlatan:
    ACK fa7d319

Tree-SHA512: ea5e0001644d70ecfbccf87e27b393786a0eda79af4923ff68a0096d4d5b910cf6eeed8667ecbf55f3a164f500d3f5aeaf9d81bb190296c30ce0cc93c165717d
(cherry picked from commit b2240f6)

Dash adaptation: Dash's validation.cpp never included kernel/chain.h, so only
kernel/coinstats.h moves into the sorted include list.

Prerequisite for bitcoin#30344, which moves mempool_persist to node/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
f1478c0 mempool: move LoadMempool/DumpMempool to node (Cory Fields)
6d242ff kernel: remove mempool_persist.cpp (Cory Fields)

Pull request description:

  DumpMempool/LoadMempool are not necessary for the kernel.

  Noticed while working on instantiated logging.

  I suppose these could have been left in on purpose, but I'm assuming it was probably just an oversight.

ACKs for top commit:
  TheCharlatan:
    Re-ACK f1478c0
  glozow:
    ACK f1478c0
  stickies-v:
    ACK f1478c0

Tree-SHA512: 5825da0cf2e67470524eb6ebe397eb90755a368469a25f184df99ab935b3eb6d89eb802b41a6c3661e869bba3bbfa8ba9d95281bc75ebbf790ec5d9d1f79c66f
(cherry picked from commit d2c8d16)

Dash adaptation: the Makefile.am hunk becomes the matching move in
src/CMakeLists.txt (bitcoin_node) and the removal of mempool_persist.cpp from
the chainstate kernel library in src/kernel/CMakeLists.txt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…20Poly1305

8607773 Add fuzz test for FSChaCha20Poly1305 (stratospher)
c807f33 Add fuzz test for AEADChacha20Poly1305 (stratospher)

Pull request description:

  This PR adds fuzz tests for `AEADChaCha20Poly1305` and `FSChaCha20Poly1305` introduced in bitcoin#28008.

  Run using:
  ```
  $ FUZZ=crypto_aeadchacha20poly1305 src/test/fuzz/fuzz
  $ FUZZ=crypto_fschacha20poly1305 src/test/fuzz/fuzz
  ```

ACKs for top commit:
  dergoegge:
    tACK 8607773
  marcofleon:
    Tested ACK 8607773. Ran both targets for ~200 CPU hours. Coverage of intended targets looks good to me. The simulation of damaged keys and checks that follow seem useful as well.

Tree-SHA512: b6b85661d896e653caeed330f941fde665fc2bbd97ecd340808a3f365c469fe9134aa77316569a771dc36d1158cac1a5f76700bcfc45fff12aef07562e48feb9
(cherry picked from commit 1db0be8)

Dash adaptations:
- Dash has no InsecureRandomContext (bitcoin#29625 and its follow-ups are not
  backported), so both targets seed a FastRandomContext from a fuzzer-chosen
  uint256.
- The target is registered in src/test/fuzz/CMakeLists.txt instead of
  Makefile.test.include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
86b3852 qa: a fuzz target for the block index database (Antoine Poinsot)

Pull request description:

  This introduces a small fuzz target for `CBlockTreeDB` which asserts a few invariants by using an in-memory LevelDb.

ACKs for top commit:
  achow101:
    ACK 86b3852
  TheCharlatan:
    Re-ACK 86b3852
  maflcko:
    review ACK 86b3852 🥒
  brunoerg:
    utACK 86b3852

Tree-SHA512: ab75b4ae1c7e0a4b15f8a6ceffdf509fbc79833e6ea073ecef68558d53b83663d1b30362aaa2d77c22b8890a572f5b1d4b1c5abbca483c8c8f9b1fb5b276a59a
(cherry picked from commit ee57737)

Dash adaptations:
- Dash still calls the block tree database CBlockTreeDB; it becomes
  kernel::BlockTreeDB with bitcoin#28191, which Dash has not backported.
- Dash's CBlockTreeDB::LoadBlockIndexGuts() takes no interrupt argument
  (it checks ShutdownRequested(); the SignalInterrupt argument comes with
  bitcoin#27861), so the target calls it without m_interrupt.
- The target is registered in src/test/fuzz/CMakeLists.txt instead of
  Makefile.test.include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PastaPastaPasta PastaPastaPasta added this to the 24.1 milestone Nov 16, 2026
@coderabbitai

coderabbitai Bot commented Nov 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 30 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 473849ab-7d52-4c7f-bb18-332559f7df34
Commits

Reviewing files that changed from the base of the PR and between 5bce2e7 and 0e291d3.

Files selected for processing: 28

  • src/CMakeLists.txt
  • src/bench/CMakeLists.txt
  • src/bench/checkblockindex.cpp
  • src/bench/parse_hex.cpp
  • src/bench/readblock.cpp
  • src/init.cpp
  • src/ipc/capnp/common-types.h
  • src/kernel/CMakeLists.txt
  • src/kernel/chainstatemanager_opts.h
  • src/node/chainstatemanager_args.cpp
  • src/node/mempool_persist.cpp
  • src/node/mempool_persist.h
  • src/rpc/mempool.cpp
  • src/test/.gitignore
  • src/test/CMakeLists.txt
  • src/test/fuzz/CMakeLists.txt
  • src/test/fuzz/block_index.cpp
  • src/test/fuzz/crypto_chacha20poly1305.cpp
  • src/test/fuzz/validation_load_mempool.cpp
  • src/test/ipc_test.capnp
  • src/test/ipc_test.cpp
  • src/test/ipc_test.h
  • src/test/ipc_tests.cpp
  • src/test/util/setup_common.cpp
  • src/util/strencodings.cpp
  • src/validation.cpp
  • src/validation.h
  • test/lint/lint-circular-dependencies.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@thepastaclaw

thepastaclaw commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

🔍 Review in progress (commit 0e291d3) · live progress

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants