Skip to content

Add nonunix platform support (libmultiprocess v14) - #110

Merged
Sjors merged 22 commits into
stratum-mining:masterfrom
Sjors:2026/06/nonunix
Sep 28, 2026
Merged

Sjors merged 22 commits into
stratum-mining:masterfrom
Sjors:2026/06/nonunix

Conversation

@Sjors

@Sjors Sjors commented Jun 17, 2026 •

Copy link
Copy Markdown
Collaborator

This tracks:

The subtree update is not strictly needed in this PR, because of the shims, but I prefer to it over with. Shims are dropped in the last commit.

@Sjors
Sjors force-pushed the 2026/06/nonunix branch 5 times, most recently from 11ac490 to 14a8fe7 Compare June 23, 2026 16:05
@Sjors Sjors changed the title Add nonunix platform support Add nonunix platform support (libmultiprocess v14) Jul 21, 2026
@Sjors
Sjors force-pushed the 2026/06/nonunix branch from 14a8fe7 to 1f12ef2 Compare July 21, 2026 10:56
@Sjors

Sjors commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Rebased after #113. The subtree update here is now from v13 to v14, in order to pull in bitcoin-core/libmultiprocess#274.

Just waiting for the Bitcoin Core PR to be merged, in order to minimize drift with our implementation.

@Sjors

Sjors commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased after bitcoin/bitcoin#35084 landed and pulled in its latest changes.

@Sjors
Sjors marked this pull request as ready for review September 28, 2026 11:28
ryanofsky and others added 10 commits September 28, 2026 13:30
Pass exe_name parameter to ipc::Protocol class constructor instead. It never
really made sense to have exe parameters as part of the protocol interface and
removing them makes adding new features like Windows support easier.

The exe name values are only used for logging and debugging purposes to
distinguish log messages from different processes.

This commit is based on 33d37f3c35efaac136863253b91799bf2711fd46 with
the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted.

Backport of bitcoin/bitcoin#35084.
This just changes Protocol class field order to make sure class members are not
destroyed before the event loop thread exits. There is no change in behavior.
The change is just being made to clarify intent and avoid potential bugs.

Backport of bitcoin/bitcoin#35084.
Keep standard headers separate from posix headers

Backport of bitcoin/bitcoin#35084.
Avoid compile error from MSVC:

D:\a\bitcoin\bitcoin\src\ipc\interfaces.cpp(24,1): error C1083: Cannot open include file: 'unistd.h': No such file or directory

MinGW provides this header but MSVC does not. Header is unneeded on windows
because HandleCtrlC code that uses it is not compiled on windows.

Backport of bitcoin/bitcoin#35084.
Use ProcessId type instead of int to represent process ids to be
compatible with an upcoming version of libmultiprocess which adds
windows support.

Backport of bitcoin/bitcoin#35084.
Use SocketId type instead of int to represent socket ids to be
compatible with an upcoming version of libmultiprocess which adds
Windows support.

This commit is based on 3859805f05e6fb642140c89e6d7378203e04df2c with
the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted.

Backport of bitcoin/bitcoin#35084.
Use Stream type to abstract socket ids and be compatible with updated
mp::ConnectStream() and mp::ServeStream() functions that use streams
instead of socket ids in an upcoming version of libmultiprocess which
adds Windows support.

Since creating Stream objects from socket ids can require the event loop
to be running, the ipc::Protocol::serve() method is also updated to
accept the server stream through a callback parameter instead of a normal
parameter.

This commit is based on e9f19815caa3e129020f116bef43d409cb50a475 with
the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted.

Backport of bitcoin/bitcoin#35084.
Use new SpawnProcess and StartSpawned functions to be compatible with an
upcoming version of libmultiprocess which adds windows support.

Backport of bitcoin/bitcoin#35084.
Wrap socketpair descriptors with libmultiprocess streams before passing
them to ConnectStream and ServeStream. This updates the sv2 test harness
after cherry-picking the Bitcoin Core IPC stream refactor.
Use new CurrentThread function to be compatible with windows mingw bug
workaround bitcoin-core/libmultiprocess#318

Backport of bitcoin/bitcoin#35084.

Omit the IPC fuzz target, which is not present in sv2-tp. The API
refactoring landed separately in bitcoin-core/libmultiprocess#323.
@Sjors
Sjors force-pushed the 2026/06/nonunix branch 2 times, most recently from 9e62ca4 to 21af003 Compare September 28, 2026 11:39
Sjors and others added 5 commits September 28, 2026 13:41
libmultiprocess#339 uses C++23 lambda attribute syntax in a public
header. Clang diagnoses this extension in C++20 mode, which breaks
WERROR builds after updating the subtree.

Keep the warning visible but non-fatal for libmultiprocess and targets
consuming its headers. Probe compiler support before adding the option,
falling back to c++23-extensions on older Clang versions such as
AppleClang 16. Preserve the upstream subtree unchanged.
2dba130478 Merge bitcoin-core/libmultiprocess#363: ci: use LLVM 23 in Bitcoin Core CI
161197a5c7 Merge bitcoin-core/libmultiprocess#352: ci: add cmake debug output
fb4ac7eb8c ci: use LLVM 23 in Bitcoin Core CI
7bac69de1f Merge bitcoin-core/libmultiprocess#360: pull latest .clang-tidy from downstream
79ddc44eb3 Merge bitcoin-core/libmultiprocess#359: ci: bump cmake version to 4.3.4 in newdeps job
bf229bf82a Merge bitcoin-core/libmultiprocess#351: ci: do not ignore NIXPKGS_CHANNEL in local ci runs
073ac4f197 Merge bitcoin-core/libmultiprocess#347: refactor: Replace EventLoop::post() with sync() taking kj::FunctionParam
fa1db7a9e3 pull latest .clang-tidy from downstream
5c49666a1d refactor: rename EventLoop::m_post_fn to m_sync_fn
2330fbe81e refactor: replace EventLoop::post() with sync() taking kj::FunctionParam
4d454a81da Merge bitcoin-core/libmultiprocess#358: refactor: Enable readability-container-contains
dba99582b3 Merge bitcoin-core/libmultiprocess#357: doc: Update Cap'n Proto version to match minimum
00923922ac Merge bitcoin-core/libmultiprocess#356: ci: Remove hard-coded -j4 from sanitize config
766867fb32 ci: clarify CAPNP_CHECKOUT=master is the v1.x release branch
cc3675280b ci: bump cmake version to 4.3.4 in newdeps job
fa101113b5 refactor: Enable readability-container-contains
81f824b025 doc: Update Cap'n Proto version to match minimum
fa30e2093c ci: Remove hard-coded -j4 from sanitize config
a7ff9d5da2 ci: add cmake debug output
f2e8df82ed Merge bitcoin-core/libmultiprocess#350: cmake: add type-unordered-set.h and version.h to public headers
0e146c0468 Merge bitcoin-core/libmultiprocess#349: type-context: fix async disconnect race condition found by antithesis
49f95e2632 Merge bitcoin-core/libmultiprocess#212: ci: add newdeps job testing newer versions of cmake and capnproto
7c73cceda9 ci: rename CI-internal variables to use CI_ prefix
fe1b8339f0 ci: do not ignore NIXPKGS_CHANNEL in local ci runs
914dc839f2 proxy: fix data race between server request threads and disconnect handling
275c8eefdf Merge bitcoin-core/libmultiprocess#345: Remove trailing whitespace and Add -Wtrailing-whitespace to default ci config
cd7162fb83 Merge bitcoin-core/libmultiprocess#304: proxy: fix BuildList to use non-const iteration for interface types
9b136782af ci: Add -Wtrailing-whitespace to default config
2f4be9ec6e refactor: Remove trailing whitespace
2448d282cc cmake: add type-unordered-set.h and version.h to public headers
b3fc922ee4 ci: add newdeps job testing newest versions of cmake and capnproto
390b5f901f Merge bitcoin-core/libmultiprocess#344: test: listen_tests and connect_tests follow-ups
d6f8588d1a proxy: fix BuildList to use non-const iteration for interface types
e18ca520f4 Merge bitcoin-core/libmultiprocess#343: test: fix race in connect_tests disconnect-deferred-failure test
c39c7850c6 doc: note construct() call in valid init interface test
b9c36c6175 test: close sockets unconditionally and check errors with KJ_SYSCALL
7eb741e635 test: drop unnecessary KJ_EXPECT(true)
113f1d4d28 test: join server thread unconditionally in connect tests
44bc4630bc test: drop mp:: prefixes in connect tests
038d33eb31 test: share DefaultLogHandler between test files
b54a163308 test: drop TestSetup socket members in connect tests
70467c5a72 test: add m_ prefix to TestSetup members in connect tests
cc260f2526 test: replace capnp fix link with upstream PR
137a6e4e03 test: fix race in connect_tests disconnect-deferred-failure test
8dab0d4bde Merge bitcoin-core/libmultiprocess#341: ci: add -Wextra-semi to llvm config
b3b134eed8 ci: add -Wextra-semi to llvm config
bdd0cd6941 Merge bitcoin-core/libmultiprocess#339: refactor: add `[[noreturn]]` attributes
a779a09764 ci: add -Wmissing-noreturn
636aaff576 refactor: add missing [[noreturn]] attributes
cc11c2b1b4 Merge bitcoin-core/libmultiprocess#338: test: check ReadList return value
2d6e863c77 Merge bitcoin-core/libmultiprocess#334: ci: Set CMAKE_BUILD_PARALLEL_LEVEL to enable parallelism by default
d4d10ff98a Merge bitcoin-core/libmultiprocess#332: ci: add -Wextra-semi to default config
b540e70f25 Merge bitcoin-core/libmultiprocess#324: proxy: Name threads spawned by the event loop
e5e367e785 Merge bitcoin-core/libmultiprocess#312: util: report back child errors to parent and throw
2220df68c9 Merge bitcoin-core/libmultiprocess#298: Fix error handling when creating clients (`mp::ConnectStream`)
51defb79ef Merge bitcoin-core/libmultiprocess#340: ci: Update `capnproto` prerequisites on NetBSD
7e94790b08 ci: Update `capnproto` prerequisites on NetBSD
9f25ffca5b test: Cover OS thread names for worker, pool, and async threads
648a18589c proxy: Name threads spawned by the event loop
49834b2609 ci: add -Wextra-semi to default config
fae9a637e3 example: Remove unused kj/async.h include
bb473690c9 Fix error handling when creating clients
44d191420c Add test coverage for ConnectStream
231361ae5a Correct stale UnixListener doc comment
060c1a50d0 Extract `UnixListener` class to a dedicated file
62f25af06c test: check ReadList return value
ce51d73725 ci: Set CMAKE_BUILD_PARALLEL_LEVEL to enable parallism in build jobs by default
67302cd132 Merge bitcoin-core/libmultiprocess#331: Remove code for Cap'n Proto versions before 0.9
f13c64ab54 Merge bitcoin-core/libmultiprocess#330: ci: Compile with minimum supported g++ in olddeps
8e026f6625 Merge bitcoin-core/libmultiprocess#327: build: avoid unnecessary capnp-rpc dependency for mpgen
e5206e9eb5 Merge bitcoin-core/libmultiprocess#325: cmake: Remove `QUIET` option from `find_package(CapnProto ...)`
879efea2bc Merge bitcoin-core/libmultiprocess#321: ci: Roll NetBSD releases to 11.0, drop 9.4
abf127a314 Merge bitcoin-core/libmultiprocess#317: ipc: Fix mpgen capnp tool path for vcpkg/Windows builds
c437d7f107 Merge bitcoin-core/libmultiprocess#310: test: cover immediate client disconnects for `ListenConnections`
31bff8a673 Merge bitcoin-core/libmultiprocess#307: refactor: memcpy -> std::ranges::copy
f355108b0a Merge bitcoin-core/libmultiprocess#303: type-chrono: Add CustomBuildField/CustomReadField overloads for std::chrono::time_point
2d678177c1 Merge bitcoin-core/libmultiprocess#296: ci: Bump channel to nixos-26.05
3f05b11624 util: kill and reap child on SpawnProcess error
4a56c1837a util: report back child error to parent and throw
a9e70dbe77 ci: Add NetBSD release 11.0
2d33b14fb0 ci: Switch to default compiler on NetBSD 9.4
36f7400277 ci: Drop NetBSD release 9.4
bd508311b5 refactor: Drop stray semicolons after function definitions
788f17a850 Remove code for Cap'n Proto versions before 0.9
7402affd0c ci: Pin oldeps config to older nixpkgs channel to compile older cmake with older gcc
edf6343562 ci: Compile with minimum supported g++-11 in olddeps
fa47449afe cmake: avoid unnecessary capnp-rpc dependency for mpgen
a494b764de cmake: Remove `QUIET` option from `find_package(CapnProto ...)`
26452e02d7 refactor: memcpy -> std::ranges::copy
e1dcc6eb18 Merge bitcoin-core/libmultiprocess#316: cmake: Fix stale codegen when mpgen binary changes
7a72df02e2 type-chrono: Add CustomBuildField/CustomReadField overloads for std::chrono::time_point
45b685c3f5 type-number, type-chrono: Fix static assert signed/unsigned comparisons
45f6255975 type-number: exclude bool from the integral overload
8d6d464948 Merge bitcoin-core/libmultiprocess#315: Fix startup race in example
a6fc80d254 Merge bitcoin-core/libmultiprocess#311: bugfix: clear FD_CLOEXEC in child instead of parent before fork
496fb84e69 test: cover immediate client disconnects for `ListenConnections`
36c6c63520 doc: Document reference-counted EventLoop lifetime
3a997e113c Fix startup race in mpexample
f5c15ce33f Merge bitcoin-core/libmultiprocess#323: refactor: access ThreadContext through CurrentThread(), ci: switch Bitcoin Core to master
66298c737f ci: Switch back to Bitcoin Core's master branch
86b4810501 refactor: access ThreadContext through CurrentThread()
eea9c64f6e cmake: Fix stale codegen when mpgen binary changes
a26a08496b cmake: Fix mpgen capnp tool path for vcpkg/Windows builds
140d9ba6ff test: allow custom log handler in `ListenSetup`
1e0c7ff9a5 util: Clear FD_CLOEXEC in child instead of parent before fork
8550ee6a31 util, refactor: Add ChildFail helper for post-fork child errors
17eab90b52 test: Fix typo in listen_tests.cpp
ce865a9ba8 refactor: Directly use value in CustomBuildField
3f221b5bfd Merge bitcoin-core/libmultiprocess#274: Add nonunix platform support
1b0f605606 doc: Remove trailing whitespace
d8f8ca3119 ipc: Wrap mpgen main() in try-catch to print errors
fbe5a14ad4 ci: Check out bitcoin/bitcoin PR #35084 instead of master
39d3690d83 types: Replace SFINAE with requires clauses to avoid MSVC C2039 error
ba68520203 proxy, refactor: Fix C4305 truncation warning in Accessor on MSVC
1d81d47811 util, refactor: Fix PtrOrValue constructor for move-only types on MSVC
b883fe1e52 proxy: Fix shutdownWrite() exception handling on macOS with dynamic libraries
0012411ccc proxy: Call shutdownWrite() in Connection destructor
38312ad191 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects
e96d5d742a proxy, refactor: Replace EventLoop wakeup fd integers with KJ stream objects
db4f9a3d73 cmake: Bump minimum required Cap'n Proto version to 0.9
652934fb79 util, refactor: Add SocketPair() and use it in SpawnProcess
1c6ef7a26c util, refactor: Do not fork() and exec() separately
1389cf3132 util, refactor: Add SpawnConnectInfo type alias and use it
c7ca1f00b6 util, refactor: Add SocketId type alias and use it
be46a35203 util, refactor: Add ProcessId type alias and use it
91a78db780 doc: Bump version 13 > 14
fa2c56ec27 ci: Bump channel to nixos-26.05

git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 2dba130478ab71a745fe96c5726719fe357e6d17
Import upstream master through September 9, 2026 (v14 development).
Includes the nonunix API changes and subsequent fixes.
The embedded libmultiprocess subtree now provides the v14 ProcessId,
SocketId, Stream, process spawning, and CurrentThread APIs directly.
Backport of bitcoin/bitcoin#35887.

Keep the direct mp/util.h include because the compatibility shims have
already been removed.
ViniciusCestarii and others added 7 commits September 28, 2026 13:42
Backport of bitcoin/bitcoin#35887.

Omit the bitcoin wrapper change and its lint exception, which do not
apply to sv2-tp.
Backport of bitcoin/bitcoin#35887.

Move the test into test_sv2 because the Bitcoin Core IPC test harness
is not present. Only build it when IPC is enabled.
Backport of bitcoin/bitcoin#33905.

Apply the macOS cache change; sv2-tp has no native Windows job.
Backport of bitcoin/bitcoin#34802.

Apply the cache action updates to the macOS job. Leave unrelated
Docker and Windows action changes out of this backport.
This avoids code duplication and improves readability.

Backport of bitcoin/bitcoin#35024.

Apply the macOS cache change; sv2-tp has no native Windows job.
zmq seems to be an alias for pyzmq, and the project page,
https://pypi.org/project/zmq/, states "You are probably looking for
pyzmq.". So switch to pyzmq, which is what we document, and use in all
other jobs.

Backport of bitcoin/bitcoin#35446.
@Sjors
Sjors merged commit ea2e8a4 into stratum-mining:master Sep 28, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants