Repository navigation
feat(executor): reuse SSH connections instead of dialing per command - #5
Merged
Merged
Conversation
Both transports previously opened a fresh connection (dial + handshake + auth) for every command and closed it immediately. Add connection reuse. Native transport (golang.org/x/crypto/ssh): - Concurrency-safe per-host connection pool keyed by resolved target shape (host, port, user, ProxyJump, identity files); concurrent dials for the same key are single-flighted, different keys proceed independently. - Pooled clients keep the existing nativeClient closer chain, so direct clients, ProxyJump tunnels, and jump clients still close in order. - keepalive@openssh.com heartbeat per pooled connection (default 30s); a failed heartbeat evicts and closes the client. - Transparent one-shot reconnect when a stale pooled client fails NewSession. Never retries after session.Run has started, preserving remote side-effect and exit-code semantics. - NativeExecutor.Close() drains the pool; Probe stays one-shot so diagnostics never leave a long-lived socket. Shell transport (system ssh binary): - Enable OpenSSH multiplexing by default: ControlMaster=auto, ControlPersist=60s, stable per-target ControlPath under a 0700 per-run temp dir; command and hidden OS probe share the socket. - SSHOptions allows disabling or customizing persist/dir; BuildSSHArgv stays backward-compatible, multiplexing rides on BuildSSHArgvWithOptions. - SSHExecutor.Close() removes the owned temp dir / tracked sockets. Tests cover pool reuse, concurrent single-flight dialing, reconnect on a dead pooled client, keepalive+close, closer-chain shutdown, ControlMaster argv construction, shared command/probe socket, disable path, and temp-dir cleanup. gofmt/go build/go vet/go test all pass. Implemented via Codex. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correctness (native pool): - Bound pooled NewSession with ConnectTimeout and make session.Run ctx-aware (close transport on cancel, no command timeout), so a half-open pooled connection fails fast instead of hanging until kernel TCP timeout. Wait for the Run goroutine after cancel so output buffers are never read while its copies still write. - Run keepalive SendRequest under a short timeout; timeout/error evicts and closes, restoring dead-connection detection. - Track in-flight sessions per pool entry: suspicious post-Run errors de-pool immediately but defer Close until in-flight sessions drain, so one command's channel-level error no longer kills concurrent sessions. Provably-dead paths still close immediately. - Broadcast dial errors to all waiters via entry.dialErr, so N concurrent commands to an unreachable host fail in parallel. - Fall back to a one-off unpooled dial when NewSession is rejected with Prohibited/ResourceShortage (sshd MaxSessions), without evicting the healthy shared client. Shell transport (multiplexing): - Use a stable per-user control dir (UserCacheDir/agentssh/ssh-mux, 0700); Close no longer removes live sockets, so masters are actually reused across invocations instead of leaking orphaned authenticated masters with unlinked sockets. - Do not inject ControlMaster/ControlPath/ControlPersist for ssh_config alias targets - the user's ssh_config governs. - Fall back to the non-multiplexed argv when the ControlPath would exceed ~90 bytes (sun_path limit), caching the decision. Config & lifecycle: - Expose ssh.multiplexing / ssh.control_persist / ssh.keepalive_interval in config and wire through newExecutor. - Cache the normalized OS per host (pool entry / mux key) instead of re-probing uname -s on every command. - Add Close() error to the Executor interface; drop the ad-hoc io.Closer type assertion in runDirect. All fixes verified with gofmt, go vet, and go test -count=1 -race ./... (full suite green). Implemented via Codex; the ctx-cancel buffer race was found by the race detector and fixed on top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Previously both transports opened a fresh SSH connection (TCP dial + handshake + auth) for every command and closed it immediately. This PR adds connection reuse to both:
Native transport (
golang.org/x/crypto/ssh)keepalive@openssh.comheartbeat per pooled connection (default 30s, configurable) under a short timeout; a failed/hung heartbeat evicts and closes the client.NewSessionis bounded by ConnectTimeout;session.Runis ctx-aware (closes transport on cancel, no command timeout). A stale pooled client is evicted and transparently re-dialed once — never aftersession.Runhas started (double-execution risk).MaxSessionschannel-open rejections fall back to a one-off unpooled dial.Close()drains the pool;Probestays one-shot.Shell transport (system
ssh)ControlMaster=auto+ControlPersist=60s+ per-targetControlPathunder a stable per-user 0700 dir (UserCacheDir/agentssh/ssh-mux), so masters are reused across invocations;Close()leaves live sockets alone.sun_pathlimit.Config & lifecycle
ssh.multiplexing,ssh.control_persist,ssh.keepalive_interval.uname -severy command.Close() erroradded to theExecutorinterface.Review
The initial implementation went through a high-effort code review (8 finder angles, adversarial verification): 10 findings confirmed (3 CONFIRMED / 7 PLAUSIBLE) — including indefinite hangs on half-open pooled connections, a keepalive that itself blocked forever, evictions killing concurrent in-flight sessions, and orphaned ControlPersist masters. All 10 are fixed in the second commit, plus a ctx-cancel output-buffer data race caught by the race detector.
Testing
gofmt -l .clean,go build ./...,go vet ./...passgo test -count=1 ./...— all packages greengo test -count=1 -race ./...— full suite green under the race detector🤖 Generated with Claude Code