libstore: add a minimal derivation builder for Windows - #16347
Open
awsmadi wants to merge 3 commits into
Open
Conversation
awsmadi
marked this pull request as ready for review
August 21, 2026 19:25
Member
|
Thanks @awsmadi! Can you check out https://github.com/nix-windows/nix-windows-demo (which used another Windows builder, but one blocked on a refactor that hasn't landed) and see if it still works with your version? |
Ericson2314
reviewed
Aug 21, 2026
Ericson2314
reviewed
Aug 21, 2026
Comment on lines
+305
to
+320
| PROCESS_INFORMATION procInfo = {0}; | ||
| if (CreateProcessW( | ||
| /* The executable is given in the command line. */ | ||
| NULL, | ||
| cmdline.data(), | ||
| NULL, | ||
| NULL, | ||
| /* Inherit handles, so the child gets the pipe. */ | ||
| TRUE, | ||
| CREATE_UNICODE_ENVIRONMENT | CREATE_SUSPENDED, | ||
| envBlock.data(), | ||
| tmpDir.native().c_str(), | ||
| &startInfo, | ||
| &procInfo) | ||
| == 0) | ||
| throw windows::WinError("CreateProcessW failed for builder '%s'", drv.builder); |
Member
There was a problem hiding this comment.
Our Pid abstraction should already have a Windows implementation, can you use it here?
`buildLocally` threw `UnimplementedError("building derivations is not yet
implemented on Windows")`. This replaces that with a working, deliberately
minimal builder, so `nix build` does something on Windows.
Verified end to end under Wine 11.0: `nix.exe` builds a derivation whose
builder is `cmd.exe`, registers the output in a Windows store, and the built
output is then executed:
STEP 1 -- nix.exe, under Wine, builds a Windows PE into the store
building 'C:\ProgramData\nix\store/qn9waj...-winhostname.exe.drv'...
C:\ProgramData\nix\store/yyga9i...-winhostname.exe
STEP 2 -- run that binary, from the store, under Wine
output: DB3705BCAF67
The output is byte-identical to its source (sha256 bedfdf07...), is mode 0555
after canonicalisation, and `nix path-info` reports it as a valid store path
with a narHash and the correct deriver.
What this builder does NOT do, all deliberate: no sandbox, no chroot, no build
user, no network isolation, no recursive Nix, no content-addressed or
fixed-output derivations, no hash rewriting (so no self-references), and no
output checks. It rejects what it cannot handle rather than silently
mis-building: non-input-addressed outputs and builtin builders throw, as do
recursive Nix and the post-build hook.
It is a separate class rather than a subclass of the Unix
`DerivationBuilderImpl` because that class is under `unix/` and so unreachable
from a Windows build. Sharing it means moving ~1850 lines of platform-neutral
logic out of `unix/`, which is a refactor of the core build path and wants
agreement on the module split first. Only about forty lines of that file are
actually POSIX-bound. This class is small enough to delete once that happens.
Three Windows-specific things were needed:
* The worker waits with I/O completion ports, so the builder's log pipe has to
be a `MuxablePipe` tied to the port, and `MuxablePipePollState::iterate`
reads the pipe's `OVERLAPPED` state -- a `Descriptor` is not enough to
register a child. Hence the new `commChannel` member and the extra `ioport`
parameter to `makeDerivationBuilder`.
* The builder environment must pass `PATH` through. Unix builds can start from
an empty environment because the store closure supplies every executable by
absolute path; on Windows the system tools live outside the store and are
found via `PATH`, so without it `cmd /c xcopy ...` fails with "not
recognized as an internal or external command". This is impure and is called
out in the code.
* Deleting a stale output needs the read-only attribute cleared first. Store
paths are canonicalised to read-only, and Windows honours that attribute on
delete where POSIX governs unlink by directory permission.
Controls: cross-compiled nix-util-tests unmoved under Wine at 782 passed /
8 skipped / 0 failed; native at 802 passed / 2 skipped / 0 failed. Off
Windows the only change is the removal of an `#ifdef _WIN32` throw whose
`#else` branch was already the entire function body, so non-Windows behaviour
is unchanged.
Assisted-by: Claude Code (claude-opus-5)
Review feedback on NixOS#16347: * OS_STR widens the fixed environment-variable names at compile time, so the runtime string_to_os_string lambda is only needed for values that are not literals. * Pid already has a Windows implementation. Pid::wait returns the exit code and reaps; Pid::kill terminates then reaps; the destructor kills a child that outlives the builder, which the hand-rolled version did not. The process handle is held in a local AutoCloseFD until the job object is set up and the thread resumed, so the failure paths there can tear the child down without Pid also doing so; ownership moves into Pid only once the child is fully started. Also sorts the sources list in windows/meson.build, which is what meson-format wants and what the failing pre-commit check was about. Assisted-by: Claude Code (claude-opus-5)
awsmadi
force-pushed
the
pr/windows-derivation-builder
branch
from
August 24, 2026 15:28
8956dcf to
029af44
Compare
Cut them to the facts that are not already in the code. 489 lines to 444. Assisted-by: Claude Code (claude-opus-5)
Ericson2314
reviewed
Aug 24, 2026
Comment on lines
973
to
974
| ? makeExternalDerivationBuilder( | ||
| localBuildCap.localStore, |
Member
There was a problem hiding this comment.
Suggested change
| ? makeExternalDerivationBuilder( | |
| localBuildCap.localStore, | |
| ? | |
| #ifdef _WIN32 | |
| /* No external-builder support on Windows yet, and the Windows | |
| builder additionally needs the worker's I/O completion port. */ | |
| throw UnimplementedError("external builders are not yet supported on Windows") | |
| #else | |
| makeExternalDerivationBuilder( | |
| localBuildCap.localStore, |
do the CPP like this, so we don't duplicate the condition
Ericson2314
reviewed
Aug 24, 2026
Comment on lines
+961
to
+971
| #ifdef _WIN32 | ||
| /* No external-builder support on Windows yet, and the Windows | ||
| builder additionally needs the worker's I/O completion port. */ | ||
| if (localBuildCap.externalBuilder) | ||
| throw UnimplementedError("external builders are not yet supported on Windows"); | ||
| builder = makeDerivationBuilder( | ||
| localBuildCap.localStore, | ||
| std::make_shared<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile), | ||
| std::move(params), | ||
| worker.ioport.get()); | ||
| #else |
Member
There was a problem hiding this comment.
Suggested change
| #ifdef _WIN32 | |
| /* No external-builder support on Windows yet, and the Windows | |
| builder additionally needs the worker's I/O completion port. */ | |
| if (localBuildCap.externalBuilder) | |
| throw UnimplementedError("external builders are not yet supported on Windows"); | |
| builder = makeDerivationBuilder( | |
| localBuildCap.localStore, | |
| std::make_shared<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile), | |
| std::move(params), | |
| worker.ioport.get()); | |
| #else |
Ericson2314
reviewed
Aug 24, 2026
| localBuildCap.localStore, | ||
| std::make_shared<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile), | ||
| std::move(params)); | ||
| #endif |
Ericson2314
reviewed
Aug 24, 2026
| #ifndef _WIN32 // TODO enable `DerivationBuilder` on Windows | ||
| #ifndef _WIN32 | ||
| DerivationBuilderUnique makeDerivationBuilder( | ||
| LocalStore & store, std::shared_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params); |
Member
There was a problem hiding this comment.
Suggested change
| LocalStore & store, std::shared_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params); | |
| LocalStore & store, std::shared_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params | |
| #ifdef _WIN32 | |
| , HANDLE ioport | |
| #endif | |
| ); |
Ericson2314
reviewed
Aug 24, 2026
| using DerivationBuilderUnique = std::unique_ptr<DerivationBuilder, DerivationBuilderDeleter>; | ||
|
|
||
| #ifndef _WIN32 // TODO enable `DerivationBuilder` on Windows | ||
| #ifndef _WIN32 |
Member
There was a problem hiding this comment.
Suggested change
| #ifndef _WIN32 | |
| /** | |
| * @param ioport The worker's I/O completion port, which the log pipe is tied to. | |
| */ |
Ericson2314
reviewed
Aug 24, 2026
| #ifndef _WIN32 | ||
| DerivationBuilderUnique makeDerivationBuilder( | ||
| LocalStore & store, std::shared_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params); | ||
|
|
Member
There was a problem hiding this comment.
Suggested change
| #ifndef _WIN32 // TODO enable `ExternalDerivationBuilder` on Windows |
Ericson2314
reviewed
Aug 24, 2026
Comment on lines
+269
to
+279
| #else | ||
| /** | ||
| * @param ioport The worker's I/O completion port, which the log pipe is tied to. | ||
| * | ||
| * @note No `makeExternalDerivationBuilder` counterpart yet. | ||
| */ | ||
| DerivationBuilderUnique makeDerivationBuilder( | ||
| LocalStore & store, | ||
| std::shared_ptr<DerivationBuilderCallbacks> miscMethods, | ||
| DerivationBuilderParams params, | ||
| HANDLE ioport); |
Member
There was a problem hiding this comment.
Suggested change
| #else | |
| /** | |
| * @param ioport The worker's I/O completion port, which the log pipe is tied to. | |
| * | |
| * @note No `makeExternalDerivationBuilder` counterpart yet. | |
| */ | |
| DerivationBuilderUnique makeDerivationBuilder( | |
| LocalStore & store, | |
| std::shared_ptr<DerivationBuilderCallbacks> miscMethods, | |
| DerivationBuilderParams params, | |
| HANDLE ioport); |
Member
|
(And don't forget to keep the history clean when you accept those changes too!) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
buildLocallythrewUnimplementedError("building derivations is not yet implemented on Windows"). This replaces it with a working but deliberately small builder, sonix builddoes something on Windows.Verified end to end under Wine 11.0.
nix.exebuilds a derivation whose builder iscmd.exe, registers the output in a Windows store, and the built binary then runs:Running the store copy printed the machine's hostname, which is what the copied
binary should do. The output is byte-identical to its source by sha256, is mode 0555
after canonicalisation, and
nix path-inforeports it valid with a narHash and theright deriver.
What it does not do
No sandbox, chroot or filesystem isolation. No build user, so the builder runs as
whoever ran Nix. No network isolation. No recursive Nix. No content-addressed or
fixed-output derivations. No hash rewriting, so no self-references. No output checks.
It rejects what it cannot handle rather than mis-building: non-input-addressed
outputs, builtin builders, recursive Nix and the post-build hook all throw.
It is a separate class rather than a subclass of
DerivationBuilderImplbecause thatlives under
unix/. Sharing it means moving the platform-neutral part out first, atwhich point this can go. That is most of the file: of its 1893 lines, the genuinely
POSIX-bound primitives are about forty (
fork/startProcesstwice,execveonce,chroot6, uid/gid 12,chown11, signals 8).Three things Windows forced
The log pipe has to be a
MuxablePipetied to the worker's completion port. Theworker waits with IOCP and
MuxablePipePollState::iteratereads the pipe'sOVERLAPPEDstate, so aDescriptorcannot register a child. Hence the newcommChannelmember and the extraioportparameter.PATHhas to be passed into the builder environment. Unix builds can start from anempty one because the store closure supplies every executable by absolute path; on
Windows the system tools live outside the store and are found through
PATH, sowithout it
cmd /c xcopy ...fails with "not recognized as an internal or externalcommand". This is impure and is called out in the code.
Deleting a stale output needs the read-only attribute cleared first. Store paths are
canonicalised read-only, and Windows honours that attribute on delete where POSIX
goes by directory permission.
One dependency, not included here
The run above additionally needs
getFSSourceAccessor()to stop passing a hardcoded"/"root, orassert(root.empty() || root.is_absolute())inposix-source-accessor.ccfires before any build starts. On Windows a leadingseparator carries no root name, so
"/"is not absolute.That is a separate question about what absoluteness should mean on Windows and is not
part of this PR. A/B on this tree, differing only in that change:
I did think the corrected
hashPathcall here had removed that dependency, sinceactualPathis drive-rooted and satisfies the assert by itself. It has not; theassert is reached from elsewhere. This PR compiles and stands alone without it, but
the end-to-end run does not.
Controls
nix-util-testsunmoved under Wine at 782 passed / 8 skipped, native clean at 802passed / 2 skipped. Off Windows the only change is removing an
#ifdef _WIN32throwwhose
#elsewas already the whole function body.