Repository navigation
util: set FD_CLOEXEC atomically in SocketPair when SOCK_CLOEXEC is available - #394
Conversation
…ailable SocketPair() created the sockets with socketpair() and then set FD_CLOEXEC on each end with separate fcntl() calls. If another thread called fork() in between, the child inherited the descriptors without FD_CLOEXEC and could keep them open across exec. Pass SOCK_CLOEXEC to socketpair() where it is defined so the flag is set atomically at creation. SpawnProcess relies on FD_CLOEXEC to close its error-reporting socket on a successful exec, and SOCK_CLOEXEC sets the same descriptor flag, so behavior is unchanged apart from closing the race. Platforms without SOCK_CLOEXEC (e.g. macOS) keep using fcntl(), but now close both descriptors before throwing if fcntl() fails, instead of leaking them.
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline and AI policy for information on the review process.
If your review is incorrectly listed, please copy-paste |
ryanofsky
left a comment
There was a problem hiding this comment.
Code review ACK e8c449e. Thanks for following up on this and for the strace check!
I'm a little ambivalent about this change. The race only happens if another thread calls fork() between the socketpair() and fcntl() calls, and right now the only Bitcoin Core caller of SpawnProcess() is the echoipc test RPC, so I don't think it can happen in practice today. Bitcoin Core also leaves FD_CLOEXEC unset on most of its own descriptors, so those leak into every child process all the time, not only in a narrow window. On the other hand, a leaked socket pair is worse than most leaked descriptors, because SpawnProcess() relies on EOF from the error-reporting socket, so a long-running unrelated child holding it could make SpawnProcess() wait until that child exits. And the SOCK_CLOEXEC part of the change is simple.
I'm planning to merge this, but will leave it open for a few days in case anyone else has thoughts on whether it is worth it to try to fix these theoretical races because it seems like there a lot of them and changes like this only scratch the surface.
| if (fcntl(pair[0], F_SETFD, FD_CLOEXEC) == -1 || fcntl(pair[1], F_SETFD, FD_CLOEXEC) == -1) { | ||
| const int err = errno; | ||
| (void)close(pair[0]); | ||
| (void)close(pair[1]); | ||
| KJ_FAIL_SYSCALL("fcntl", err); | ||
| } |
There was a problem hiding this comment.
In commit "util: set FD_CLOEXEC atomically in SocketPair when SOCK_CLOEXEC is available" (e8c449e)
Note: not asking for a change here, this looks fine. But it is a good example of a tradeoff this file keeps running into: either use KJ_SYSCALL and leak descriptors when it throws, or add manual close() calls and hand-written error reporting like this. The same choice comes up in other places in this file, and it gets a lot worse with the Windows code in #231, which has many more error paths.
I think a better direction for followups would be to own descriptors with RAII from the moment they are created, so KJ_SYSCALL can throw freely without leaking, and only release them when ownership is handed off. KJ already provides kj::AutoCloseFd and, on Windows, kj::AutoCloseHandle for this. Here it could look like:
int pair[2];
KJ_SYSCALL(socketpair(AF_UNIX, SOCK_STREAM, 0, pair));
kj::AutoCloseFd fd0{pair[0]}, fd1{pair[1]};
KJ_SYSCALL(fcntl(fd0, F_SETFD, FD_CLOEXEC));
KJ_SYSCALL(fcntl(fd1, F_SETFD, FD_CLOEXEC));
return {fd0.release(), fd1.release()};There was a problem hiding this comment.
re: #394 (comment)
I'd be happy to work on a follow-up with
kj::AutoCloseFdafter this merges, if you think that would be useful.
Yes, a follow-up would be welcome, thanks! Besides SocketPair(), SpawnProcess() is a good place for it: if the second SocketPair() call or connect_info_to_args() throws, the first socket pair is never closed.
|
Thanks! That makes sense. Using RAII here would be cleaner than handling each error path manually. I'd be happy to work on a follow-up with |
SocketPair()currently callssocketpair()and then uses twofcntl()calls to setFD_CLOEXEC. If another thread forks between those calls, the descriptors can be inherited by the child and remain open across exec.This change uses
SOCK_CLOEXECwhen available, setting the flag on both descriptors atomically when the socket pair is created. Platforms without support for it still use the existingfcntl()approach. If either call fails, both descriptors are closed.The race came up in #311 and #312. It was recently brought up again by ValeraFinebits in bitcoin/bitcoin#36747, where ryanofsky expressed interest in reviewing an upstream fix.
Testing:
SOCK_CLOEXECdisabled at compile time.-Werror,-Wundefand-Wunused.straceconfirms that the spawn tests usesocketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, ...)without the subsequentfcntl(F_SETFD, FD_CLOEXEC)calls. On master, the same test produces two socket pairs followed by fourfcntl()calls.This modifies the same function as #231, but only in the POSIX branch.