Repository navigation
Add nonunix platform support (libmultiprocess v14) - #110
Merged
Merged
Conversation
Sjors
force-pushed
the
2026/06/nonunix
branch
5 times, most recently
from
June 23, 2026 16:05
11ac490 to
14a8fe7
Compare
Collaborator
Author
|
Rebased after #113. The subtree update here is now from v13 to v14, in order to pull in bitcoin-core/libmultiprocess#274. Just waiting for the Bitcoin Core PR to be merged, in order to minimize drift with our implementation. |
Sjors
force-pushed
the
2026/06/nonunix
branch
from
September 28, 2026 11:28
1f12ef2 to
7056400
Compare
Collaborator
Author
|
Rebased after bitcoin/bitcoin#35084 landed and pulled in its latest changes. |
Sjors
marked this pull request as ready for review
September 28, 2026 11:28
Pass exe_name parameter to ipc::Protocol class constructor instead. It never really made sense to have exe parameters as part of the protocol interface and removing them makes adding new features like Windows support easier. The exe name values are only used for logging and debugging purposes to distinguish log messages from different processes. This commit is based on 33d37f3c35efaac136863253b91799bf2711fd46 with the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted. Backport of bitcoin/bitcoin#35084.
This just changes Protocol class field order to make sure class members are not destroyed before the event loop thread exits. There is no change in behavior. The change is just being made to clarify intent and avoid potential bugs. Backport of bitcoin/bitcoin#35084.
Keep standard headers separate from posix headers Backport of bitcoin/bitcoin#35084.
Avoid compile error from MSVC: D:\a\bitcoin\bitcoin\src\ipc\interfaces.cpp(24,1): error C1083: Cannot open include file: 'unistd.h': No such file or directory MinGW provides this header but MSVC does not. Header is unneeded on windows because HandleCtrlC code that uses it is not compiled on windows. Backport of bitcoin/bitcoin#35084.
Use ProcessId type instead of int to represent process ids to be compatible with an upcoming version of libmultiprocess which adds windows support. Backport of bitcoin/bitcoin#35084.
Use SocketId type instead of int to represent socket ids to be compatible with an upcoming version of libmultiprocess which adds Windows support. This commit is based on 3859805f05e6fb642140c89e6d7378203e04df2c with the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted. Backport of bitcoin/bitcoin#35084.
Use Stream type to abstract socket ids and be compatible with updated mp::ConnectStream() and mp::ServeStream() functions that use streams instead of socket ids in an upcoming version of libmultiprocess which adds Windows support. Since creating Stream objects from socket ids can require the event loop to be running, the ipc::Protocol::serve() method is also updated to accept the server stream through a callback parameter instead of a normal parameter. This commit is based on e9f19815caa3e129020f116bef43d409cb50a475 with the bitcoin/bitcoin changes to src/ipc/test/ipc_tests.cpp omitted. Backport of bitcoin/bitcoin#35084.
Use new SpawnProcess and StartSpawned functions to be compatible with an upcoming version of libmultiprocess which adds windows support. Backport of bitcoin/bitcoin#35084.
Wrap socketpair descriptors with libmultiprocess streams before passing them to ConnectStream and ServeStream. This updates the sv2 test harness after cherry-picking the Bitcoin Core IPC stream refactor.
Use new CurrentThread function to be compatible with windows mingw bug workaround bitcoin-core/libmultiprocess#318 Backport of bitcoin/bitcoin#35084. Omit the IPC fuzz target, which is not present in sv2-tp. The API refactoring landed separately in bitcoin-core/libmultiprocess#323.
Sjors
force-pushed
the
2026/06/nonunix
branch
2 times, most recently
from
September 28, 2026 11:39
9e62ca4 to
21af003
Compare
libmultiprocess#339 uses C++23 lambda attribute syntax in a public header. Clang diagnoses this extension in C++20 mode, which breaks WERROR builds after updating the subtree. Keep the warning visible but non-fatal for libmultiprocess and targets consuming its headers. Probe compiler support before adding the option, falling back to c++23-extensions on older Clang versions such as AppleClang 16. Preserve the upstream subtree unchanged.
2dba130478 Merge bitcoin-core/libmultiprocess#363: ci: use LLVM 23 in Bitcoin Core CI 161197a5c7 Merge bitcoin-core/libmultiprocess#352: ci: add cmake debug output fb4ac7eb8c ci: use LLVM 23 in Bitcoin Core CI 7bac69de1f Merge bitcoin-core/libmultiprocess#360: pull latest .clang-tidy from downstream 79ddc44eb3 Merge bitcoin-core/libmultiprocess#359: ci: bump cmake version to 4.3.4 in newdeps job bf229bf82a Merge bitcoin-core/libmultiprocess#351: ci: do not ignore NIXPKGS_CHANNEL in local ci runs 073ac4f197 Merge bitcoin-core/libmultiprocess#347: refactor: Replace EventLoop::post() with sync() taking kj::FunctionParam fa1db7a9e3 pull latest .clang-tidy from downstream 5c49666a1d refactor: rename EventLoop::m_post_fn to m_sync_fn 2330fbe81e refactor: replace EventLoop::post() with sync() taking kj::FunctionParam 4d454a81da Merge bitcoin-core/libmultiprocess#358: refactor: Enable readability-container-contains dba99582b3 Merge bitcoin-core/libmultiprocess#357: doc: Update Cap'n Proto version to match minimum 00923922ac Merge bitcoin-core/libmultiprocess#356: ci: Remove hard-coded -j4 from sanitize config 766867fb32 ci: clarify CAPNP_CHECKOUT=master is the v1.x release branch cc3675280b ci: bump cmake version to 4.3.4 in newdeps job fa101113b5 refactor: Enable readability-container-contains 81f824b025 doc: Update Cap'n Proto version to match minimum fa30e2093c ci: Remove hard-coded -j4 from sanitize config a7ff9d5da2 ci: add cmake debug output f2e8df82ed Merge bitcoin-core/libmultiprocess#350: cmake: add type-unordered-set.h and version.h to public headers 0e146c0468 Merge bitcoin-core/libmultiprocess#349: type-context: fix async disconnect race condition found by antithesis 49f95e2632 Merge bitcoin-core/libmultiprocess#212: ci: add newdeps job testing newer versions of cmake and capnproto 7c73cceda9 ci: rename CI-internal variables to use CI_ prefix fe1b8339f0 ci: do not ignore NIXPKGS_CHANNEL in local ci runs 914dc839f2 proxy: fix data race between server request threads and disconnect handling 275c8eefdf Merge bitcoin-core/libmultiprocess#345: Remove trailing whitespace and Add -Wtrailing-whitespace to default ci config cd7162fb83 Merge bitcoin-core/libmultiprocess#304: proxy: fix BuildList to use non-const iteration for interface types 9b136782af ci: Add -Wtrailing-whitespace to default config 2f4be9ec6e refactor: Remove trailing whitespace 2448d282cc cmake: add type-unordered-set.h and version.h to public headers b3fc922ee4 ci: add newdeps job testing newest versions of cmake and capnproto 390b5f901f Merge bitcoin-core/libmultiprocess#344: test: listen_tests and connect_tests follow-ups d6f8588d1a proxy: fix BuildList to use non-const iteration for interface types e18ca520f4 Merge bitcoin-core/libmultiprocess#343: test: fix race in connect_tests disconnect-deferred-failure test c39c7850c6 doc: note construct() call in valid init interface test b9c36c6175 test: close sockets unconditionally and check errors with KJ_SYSCALL 7eb741e635 test: drop unnecessary KJ_EXPECT(true) 113f1d4d28 test: join server thread unconditionally in connect tests 44bc4630bc test: drop mp:: prefixes in connect tests 038d33eb31 test: share DefaultLogHandler between test files b54a163308 test: drop TestSetup socket members in connect tests 70467c5a72 test: add m_ prefix to TestSetup members in connect tests cc260f2526 test: replace capnp fix link with upstream PR 137a6e4e03 test: fix race in connect_tests disconnect-deferred-failure test 8dab0d4bde Merge bitcoin-core/libmultiprocess#341: ci: add -Wextra-semi to llvm config b3b134eed8 ci: add -Wextra-semi to llvm config bdd0cd6941 Merge bitcoin-core/libmultiprocess#339: refactor: add `[[noreturn]]` attributes a779a09764 ci: add -Wmissing-noreturn 636aaff576 refactor: add missing [[noreturn]] attributes cc11c2b1b4 Merge bitcoin-core/libmultiprocess#338: test: check ReadList return value 2d6e863c77 Merge bitcoin-core/libmultiprocess#334: ci: Set CMAKE_BUILD_PARALLEL_LEVEL to enable parallelism by default d4d10ff98a Merge bitcoin-core/libmultiprocess#332: ci: add -Wextra-semi to default config b540e70f25 Merge bitcoin-core/libmultiprocess#324: proxy: Name threads spawned by the event loop e5e367e785 Merge bitcoin-core/libmultiprocess#312: util: report back child errors to parent and throw 2220df68c9 Merge bitcoin-core/libmultiprocess#298: Fix error handling when creating clients (`mp::ConnectStream`) 51defb79ef Merge bitcoin-core/libmultiprocess#340: ci: Update `capnproto` prerequisites on NetBSD 7e94790b08 ci: Update `capnproto` prerequisites on NetBSD 9f25ffca5b test: Cover OS thread names for worker, pool, and async threads 648a18589c proxy: Name threads spawned by the event loop 49834b2609 ci: add -Wextra-semi to default config fae9a637e3 example: Remove unused kj/async.h include bb473690c9 Fix error handling when creating clients 44d191420c Add test coverage for ConnectStream 231361ae5a Correct stale UnixListener doc comment 060c1a50d0 Extract `UnixListener` class to a dedicated file 62f25af06c test: check ReadList return value ce51d73725 ci: Set CMAKE_BUILD_PARALLEL_LEVEL to enable parallism in build jobs by default 67302cd132 Merge bitcoin-core/libmultiprocess#331: Remove code for Cap'n Proto versions before 0.9 f13c64ab54 Merge bitcoin-core/libmultiprocess#330: ci: Compile with minimum supported g++ in olddeps 8e026f6625 Merge bitcoin-core/libmultiprocess#327: build: avoid unnecessary capnp-rpc dependency for mpgen e5206e9eb5 Merge bitcoin-core/libmultiprocess#325: cmake: Remove `QUIET` option from `find_package(CapnProto ...)` 879efea2bc Merge bitcoin-core/libmultiprocess#321: ci: Roll NetBSD releases to 11.0, drop 9.4 abf127a314 Merge bitcoin-core/libmultiprocess#317: ipc: Fix mpgen capnp tool path for vcpkg/Windows builds c437d7f107 Merge bitcoin-core/libmultiprocess#310: test: cover immediate client disconnects for `ListenConnections` 31bff8a673 Merge bitcoin-core/libmultiprocess#307: refactor: memcpy -> std::ranges::copy f355108b0a Merge bitcoin-core/libmultiprocess#303: type-chrono: Add CustomBuildField/CustomReadField overloads for std::chrono::time_point 2d678177c1 Merge bitcoin-core/libmultiprocess#296: ci: Bump channel to nixos-26.05 3f05b11624 util: kill and reap child on SpawnProcess error 4a56c1837a util: report back child error to parent and throw a9e70dbe77 ci: Add NetBSD release 11.0 2d33b14fb0 ci: Switch to default compiler on NetBSD 9.4 36f7400277 ci: Drop NetBSD release 9.4 bd508311b5 refactor: Drop stray semicolons after function definitions 788f17a850 Remove code for Cap'n Proto versions before 0.9 7402affd0c ci: Pin oldeps config to older nixpkgs channel to compile older cmake with older gcc edf6343562 ci: Compile with minimum supported g++-11 in olddeps fa47449afe cmake: avoid unnecessary capnp-rpc dependency for mpgen a494b764de cmake: Remove `QUIET` option from `find_package(CapnProto ...)` 26452e02d7 refactor: memcpy -> std::ranges::copy e1dcc6eb18 Merge bitcoin-core/libmultiprocess#316: cmake: Fix stale codegen when mpgen binary changes 7a72df02e2 type-chrono: Add CustomBuildField/CustomReadField overloads for std::chrono::time_point 45b685c3f5 type-number, type-chrono: Fix static assert signed/unsigned comparisons 45f6255975 type-number: exclude bool from the integral overload 8d6d464948 Merge bitcoin-core/libmultiprocess#315: Fix startup race in example a6fc80d254 Merge bitcoin-core/libmultiprocess#311: bugfix: clear FD_CLOEXEC in child instead of parent before fork 496fb84e69 test: cover immediate client disconnects for `ListenConnections` 36c6c63520 doc: Document reference-counted EventLoop lifetime 3a997e113c Fix startup race in mpexample f5c15ce33f Merge bitcoin-core/libmultiprocess#323: refactor: access ThreadContext through CurrentThread(), ci: switch Bitcoin Core to master 66298c737f ci: Switch back to Bitcoin Core's master branch 86b4810501 refactor: access ThreadContext through CurrentThread() eea9c64f6e cmake: Fix stale codegen when mpgen binary changes a26a08496b cmake: Fix mpgen capnp tool path for vcpkg/Windows builds 140d9ba6ff test: allow custom log handler in `ListenSetup` 1e0c7ff9a5 util: Clear FD_CLOEXEC in child instead of parent before fork 8550ee6a31 util, refactor: Add ChildFail helper for post-fork child errors 17eab90b52 test: Fix typo in listen_tests.cpp ce865a9ba8 refactor: Directly use value in CustomBuildField 3f221b5bfd Merge bitcoin-core/libmultiprocess#274: Add nonunix platform support 1b0f605606 doc: Remove trailing whitespace d8f8ca3119 ipc: Wrap mpgen main() in try-catch to print errors fbe5a14ad4 ci: Check out bitcoin/bitcoin PR #35084 instead of master 39d3690d83 types: Replace SFINAE with requires clauses to avoid MSVC C2039 error ba68520203 proxy, refactor: Fix C4305 truncation warning in Accessor on MSVC 1d81d47811 util, refactor: Fix PtrOrValue constructor for move-only types on MSVC b883fe1e52 proxy: Fix shutdownWrite() exception handling on macOS with dynamic libraries 0012411ccc proxy: Call shutdownWrite() in Connection destructor 38312ad191 proxy, refactor: Change ConnectStream and ServeStream to accept stream objects e96d5d742a proxy, refactor: Replace EventLoop wakeup fd integers with KJ stream objects db4f9a3d73 cmake: Bump minimum required Cap'n Proto version to 0.9 652934fb79 util, refactor: Add SocketPair() and use it in SpawnProcess 1c6ef7a26c util, refactor: Do not fork() and exec() separately 1389cf3132 util, refactor: Add SpawnConnectInfo type alias and use it c7ca1f00b6 util, refactor: Add SocketId type alias and use it be46a35203 util, refactor: Add ProcessId type alias and use it 91a78db780 doc: Bump version 13 > 14 fa2c56ec27 ci: Bump channel to nixos-26.05 git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: 2dba130478ab71a745fe96c5726719fe357e6d17
Import upstream master through September 9, 2026 (v14 development). Includes the nonunix API changes and subsequent fixes.
The embedded libmultiprocess subtree now provides the v14 ProcessId, SocketId, Stream, process spawning, and CurrentThread APIs directly.
Backport of bitcoin/bitcoin#35887. Keep the direct mp/util.h include because the compatibility shims have already been removed.
Backport of bitcoin/bitcoin#35887. Omit the bitcoin wrapper change and its lint exception, which do not apply to sv2-tp.
Backport of bitcoin/bitcoin#35887. Move the test into test_sv2 because the Bitcoin Core IPC test harness is not present. Only build it when IPC is enabled.
Backport of bitcoin/bitcoin#33905. Apply the macOS cache change; sv2-tp has no native Windows job.
Backport of bitcoin/bitcoin#34802. Apply the cache action updates to the macOS job. Leave unrelated Docker and Windows action changes out of this backport.
This avoids code duplication and improves readability. Backport of bitcoin/bitcoin#35024. Apply the macOS cache change; sv2-tp has no native Windows job.
zmq seems to be an alias for pyzmq, and the project page, https://pypi.org/project/zmq/, states "You are probably looking for pyzmq.". So switch to pyzmq, which is what we document, and use in all other jobs. Backport of bitcoin/bitcoin#35446.
Sjors
force-pushed
the
2026/06/nonunix
branch
from
September 28, 2026 11:50
21af003 to
ea2e8a4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This tracks:
The subtree update is not strictly needed in this PR, because of the shims, but I prefer to it over with. Shims are dropped in the last commit.