Skip to content

Add getSerializedBlock() to skip CBlock deserialization - #154

Draft
Sjors wants to merge 5 commits into
stratum-mining:masterfrom
Sjors:2026/10/ipc-serialized
Draft

Sjors wants to merge 5 commits into
stratum-mining:masterfrom
Sjors:2026/10/ipc-serialized

Conversation

@Sjors

@Sjors Sjors commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

While working on an alternative approach to bitcoin/bitcoin#36525, I noticed that CBlock deserialization takes several milliseconds, and that this blocks the main IPC thread.

Since we want to write the block straight to disk (for diagnostics), there's no need to deserialize it.

Introduce ipc::Serialized<T> as a general way to bypass this and then have getBlock() wrap around getSerializedBlock().

Add a manual hot-cache mp::ReadField benchmark comparing CBlock with
ipc::Serialized<CBlock>. Use an empty control and a synthetic large-witness
block, select production SHA256 acceleration, and alternate conversion
order across 101 samples after five warmups.

GCC 15.2, RelWithDebInfo -O2, x86 SHA-NI, median milliseconds:

payload CBlock Serialized
142 bytes, 1 transaction 0.00020 0.00002
3,713,460 bytes, 1,021 txs 1.829 0.053

The interval includes field conversion and destruction, excluding encoding,
transport, and disk I/O.

Sjors added 5 commits October 29, 2026 10:03
Exercise the existing raw byte-vector binding through mp::BuildField and
mp::ReadField. Check that received bytes survive mutation and destruction
of the Cap'n Proto message, and that an empty field clears an existing
payload. Share the checks so other owned payload types can reuse them.
Allow clients to receive a Data field as ipc::Serialized<T>, retaining an
owned byte copy without constructing T on the IPC event loop. Provide
build and read hooks using the existing libmultiprocess customization
points.

Run the existing ownership and empty-payload checks for the new wrapper,
including arbitrary bytes that cannot be decoded as a block.
Add a manual hot-cache mp::ReadField benchmark comparing CBlock with
ipc::Serialized<CBlock>. Use an empty control and a synthetic large-witness
block, select production SHA256 acceleration, and alternate conversion
order across 101 samples after five warmups.

GCC 15.2, RelWithDebInfo -O2, x86 SHA-NI, median milliseconds:

  payload                     CBlock  Serialized<CBlock>
  142 bytes, 1 transaction    0.00020 0.00002
  3,713,460 bytes, 1,021 txs  1.829   0.053

The interval includes field conversion and destruction, excluding encoding,
transport, and disk I/O. Run with:

  IPC_SERIALIZED_BENCH=1 build/bin/test_sv2 -t ipc_serialized_tests

See doc/benchmark-ipc-serialized.md for methodology.
Check that accepted and rejected submissions both save the complete
block returned by getBlock, including transaction witness data. Allow
the mock to return a known block and compare the saved file byte for
byte. This test passes with the existing typed getBlock implementation.
Keep getBlock returning CBlock and add getSerializedBlock for owned IPC
bytes. Map the serialized accessor to RPC ordinal 2 and provide the typed
getBlock through the client adapter, decoding on the calling thread.

Use the serialized accessor for diagnostic saves. Parse only the header
for the filename and write the exact returned bytes, avoiding transaction
construction and hashing on the IPC loop. This can save several
milliseconds for large blocks.

Extend the diagnostic-save test with malformed transaction data after
accepted and rejected submissions. Check both block accessors agree and
that typed decoding failures remain local. All 64 test_sv2 cases pass.
@Sjors

Sjors commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator Author

@ryanofsky do you think ipc::Serialized<T> would be useful for Bitcoin Core too, e.g. for the chain interface? If so, I might first PR it there and then backport here.

@Sjors
Sjors force-pushed the 2026/10/ipc-serialized branch 2 times, most recently from cb34bff to 9d71796 Compare October 29, 2026 11:52

@xyephy xyephy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK. Tested 9d71796: each commit builds and passes the unit tests on macOS (clang), and the head also on Linux (GCC). The two new template provider tests are clean under repeated TSan runs.

End-to-end against Bitcoin Core v31.1 and v32.1 on regtest, with segwit transactions in the mempool:

  • pool + mining_device: every saved .dat file is byte-identical to getblock <hash> 0, including blocks with witness transactions.
  • pool with embedded JDS + JDC on sv2-tp: every RequestTransactionData succeeded and the JDS accepted every declared job.

I ran the same JDS + JDC setup against v32.1 on signet and on mainnet for a few minutes each. There were no RequestTransactionData errors and every declaration was accepted, including mainnet templates with ~7,600-7,900 transactions. On signet the CPU miner reached the proof-of-work target and Core rejected the blocks for the missing signet signature (bad-signet-blksig). sv2-tp still saved them, and all 31 unique files parse fully, with header hash and merkle root consistent.

I also timed the CBlock decode on six recent mainnet blocks (479 to 7,580 transactions). Full blocks took about as long as the synthetic 3.7 MB benchmark block, up to roughly twice as long, on both an arm64/clang and an x86/GCC machine. The synthetic block is mostly a few large witness items, so "several milliseconds" holds for real blocks.

Test coverage at 9d71796:

  • No mock template transaction carries a witness (neither the coinbase nor MakeDummyTx()). Switching the client decode in mining.cpp to TX_NO_WITNESS still passes every test. In 7a67a91, save_getblock sent a witness through the typed getBlock(), but 9d71796 switched that override to raw bytes, so the coverage was lost. Real Core's coinbase carries the 32-byte witness reserved value, which RequestTransactionData sends as excess_data. I tried adding it to the mock coinbase: client_tests then needs its RequestTransactionData.Success size updated, and with an assertion on excess_data the mutation is caught by get_block_accessors. Happy to share the diff.
  • MockBlockTemplate::getBlock(), including its override branch, is never reached over IPC, since @2 now dispatches to getSerializedBlock() (an abort() there still passes).

With Apple clang the benchmark reports Serialized_ms=0. The timed loop only uses value.bytes.size(), so the copy is elided, while GCC still performs it. Reading one byte of the result makes it measurable.

An optional alternative that may suit sv2-tp, as a pure IPC client: instead of renaming the capnp method and adding the ProxyClientCustom override, interfaces::BlockTemplate::getBlock() could return ipc::Serialized<CBlock> directly, with RequestTransactionData decoding via a SpanReader. That keeps the BlockTemplate capnp interface identical to Core's and drops mining-client.h and the $Proxy.include. The difference moves to the return type in interfaces/mining.h, which already diverges from Core. I tried it: it builds, the remaining tests pass (get_block_accessors no longer applies), and the end-to-end results above are the same on v31.1 and v32.1. The cost is that callers decode by hand. In Core, local callers such as miner_tests want a CBlock, so if ipc::Serialized<T> goes upstream first, the current shape is probably the better fit and this can be ignored.

// interfaces/mining.h
virtual ipc::Serialized<CBlock> getBlock() = 0;

// RequestTransactionData
const auto serialized{(*cached_block->second.second).getBlock()};
SpanReader reader{MakeByteSpan(serialized.bytes)};
reader >> TX_WITH_WITNESS(block);

Nits:

  • "getBlock RPC" (the mining.capnp comment and the get_block_accessors comment) and "RPC ordinal 2" (the 9d71796 message) could read as JSON-RPC; "IPC method" would avoid that.
  • CustomReadField for Serialized<T> leaves existing bytes in place when the field is absent. That's unreachable from clientInvoke, which default-constructs the result, but clearing first would match the Data hook.

@Sjors

Sjors commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

An optional alternative that may suit sv2-tp, as a pure IPC client: instead of renaming the capnp method and adding the ProxyClientCustom override, interfaces::BlockTemplate::getBlock() could return ipc::Serialized<CBlock> directly,

I prefer to keep the regular getBlock() around even if we currently don't use it. After #155 I might even drop getSerializedBlock() and only use ipc::Serialized<CBlock> to handle submitSolution()'s return.

@ryanofsky

Copy link
Copy Markdown
Contributor

re: #154 (comment)

@ryanofsky do you think ipc::Serialized<T> would be useful for Bitcoin Core too, e.g. for the chain interface? If so, I might first PR it there and then backport here.

I think these changes are reasonable workarounds for libmultiprocess performance problems, but I'd rather not add ipc::Serialized<T> to Bitcoin Core's interfaces. I think both problems could be fixed more generally in libmultiprocess without changing C++ interface definitions, which keeps the interfaces cleaner and works for every method without special cases:

  • For callers that only want the raw bytes, like SaveBlockAsync writing the block to disk, libmultiprocess proxy client objects could provide an escape hatch for making Cap'n Proto calls directly without the C++ conversions.
  • For decoding blocking the event loop: libmultiprocess clients currently build parameters and read results on the event loop thread. They could do those conversions on the calling thread instead, which would help every method with large parameters or results.

@Sjors
Sjors marked this pull request as draft October 31, 2026 02:49
@Sjors

Sjors commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks. I'm marking this as draft pending the direction in bitcoin/bitcoin#36525.

@xyephy

xyephy commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

I prefer to keep the regular getBlock() around even if we currently don't use it. After #155 I might even drop getSerializedBlock() and only use ipc::Serialized<CBlock> to handle submitSolution()'s return.

Thanks. One correction to my test-coverage note: with #141 / sv2-spec#241 moving excess_data to empty, asserting on it is the wrong direction. A witness item in MakeDummyTx() (mtx.vin[0].scriptWitness.stack.emplace_back(4, 0x77);) covers it instead: the suite still passes at 9d71796 and on top of #141, and the TX_NO_WITNESS mutation then fails client_tests on the RequestTransactionData.Success size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants