feat(discovery): report probe timeouts instead of "not installed" - #31
Merged
Conversation
The `<command> --version` probe in codex-discovery hardcoded a 2s budget with no way to change, extend, or cancel it. 2s is generous for a native binary (~0.3s) but marginal for an npm shim: `codex.cmd` is a `cmd.exe -> node -> shim` chain measuring ~1.5s warm on Windows, so a loaded machine crosses it. Overrunning was not reported as slow — it was indistinguishable from "not installed". The candidate came back with no version, a version- gating consumer demoted it, nothing was selected, and `resolve()` threw `CodexCliNotInstalledError`. Nothing retried. `discoverCommands` is generic, so this hit `git` and `gh` lookups too. - Budget is configurable (`versionTimeoutMs`) with an exported `DEFAULT_COMMAND_VERSION_TIMEOUT_MS`, raised to 10s for the shim chain. - `readCommandVersion` is exported and returns a `CommandVersionProbeOutcome` (ok / version_not_reported / not_found / not_executable / timed_out / aborted / failed). The outcome rides on every candidate and on `ResolvedCommandCandidate`, so a caller can tell an unfinished measurement from a missing CLI and re-probe on its own budget. A timed-out candidate is no longer labelled `not_executable` nor filtered out of the snapshot. - `CodexCliNotInstalledError` keeps its type but carries `timedOutCommands` / `probeTimedOut`. - `AbortSignal` on every entry point; an aborted run yields `error: COMMAND_DISCOVERY_ABORTED` / `CodexDiscoveryAbortedError` rather than a false "not installed". Also fixes three latent hangs of the same class: - Both probes now settle within their budget even when the child cannot be killed. Killing the `cmd.exe` wrapper of a `.cmd` shim leaves the `node` grandchild holding the stdio pipes, so `execFile` never calls back and no timeout value would have helped. - `collectCodexStatus` had no budget at all and could hang a caller forever. It now has one (`DEFAULT_CODEX_STATUS_TIMEOUT_MS`, 10s) and reports `outcome` / `timedOut` so a slow check is not read as a signed-out profile. - `AcpConnection.request` accepted `timeoutMs` and ignored it, making the 30s `initialize` and 1h `session/prompt` budgets `AcpAgentClient` passes dead code. Now enforced. agent-acp local discovery gains `probeTimeoutMs` (default unchanged at 5s — its candidate loop is serial) plus `AbortSignal`, and reports an overrun as `reason: "probe-timed-out"` rather than `version-probe-failed`. Everything is additive and backward compatible so PwrAgent can adopt it and delete its desktop-owned re-probe workaround.
The new timeout tests passed on macOS for the wrong reason. Their shim was `#!/bin/sh` + `sleep 3`, but the probe env sets `PATH` to just the temp dir, so the shell could not resolve `sleep`, printed "command not found", and fell straight through to `echo` — answering instantly. macOS spawn latency (~1.6s for a freshly written script) still exceeded the 150ms budget, so the assertions held; on a fast-spawn Linux runner the shim answered inside the budget and nine tests failed. Run the delay inside `process.execPath` instead: no PATH lookup at all, and the child dies cleanly on kill rather than orphaning a `sleep`. Also update the Windows-only `.cmd` shim test, whose exact `toEqual` now sees the new `versionProbeOutcome` key on the resolved candidate.
`CodexThreadClient` and `CodexOneShotClient` resolve their binary through codex-discovery on first connect, but passed no options along — a host could only take the default budget, and an overrun surfaced as `CodexCliNotInstalledError` from the very first call. Both now accept `commandVersionTimeoutMs` and forward it as `versionTimeoutMs`. Named for the client's vocabulary (it sits beside `requestTimeoutMs` / `turnTimeoutMs`, and `clientVersion` is right above it) rather than mirroring the discovery option name. Covered by a test on the REAL discovery path, since `transportFactory` short-circuits discovery entirely: a `codex` shim that cannot answer for 3s must still come back inside a second when the budget is 150ms. Verified the test fails (4158ms) without the pass-through.
Ten review angles over the PR diff; fifteen findings, all applied. Correctness: - acp-local-discovery and codex-login clamped their budgets without a `Number.isFinite` guard. `Infinity` is the natural spelling of "no budget", and both consumers reject it: `setTimeout` collapses a non-finite delay to 1ms, and `execFile`'s `timeout` throws ERR_OUT_OF_RANGE synchronously. `probeTimeoutMs: Infinity` therefore reported every installed ACP agent as not installed. - ACP discovery only recorded a timed-out candidate behind `includeRejectedCandidates` (default false), so on the default path the group was dropped entirely — the very "slow machine looks like an empty machine" bug this PR fixes on the codex side, still live on the ACP one. - `AcpAgentClient.initialize` registered its notification listeners before the `initialize` request, which this PR made rejectable. A retry after a timeout left the first listener registered and unreachable, so every later `session/update` dispatched twice and duplicated the transcript. - `resolveCodexCommand` returned the selected candidate before testing for abort, making `CodexDiscoveryAbortedError` unreachable and handing back a command whose version was never measured. - The `skipVersionProbe` branch hardcoded its outcome, swallowing a preflight that said `not_found` and reporting `not_executable` instead — a regression against main. - `collectCodexStatus` left its stdout/stderr handlers attached after abandoning a child, so a surviving child grew `output` without bound. - Abort was labelled from a trailing `signal.aborted` read, so a signal firing after every probe finished marked a complete snapshot abandoned and masked a `codex_too_old` verdict. Now driven by the evidence. - `resolveDiscoveredCommand` pinned some other candidate's outcome onto the bare-name fallback, sending callers to re-probe the wrong command. - `checkCodexAuthStatus` dropped the `aborted` outcome, painting a self-cancelled check as a broken profile. `CodexAuthStatusResponse` now carries the full `CodexStatusOutcome` instead of a `timedOut` flag. - `withAcpRequestTimeout` treated `Infinity` as "no budget" but `0` as a 1ms budget; both now mean unbounded. - Both agent-client clients dropped `versionProbeOutcome`, hiding that the minimum-version gate had been skipped. They now warn. Tests: - Two tests inherited the 10s default budget under vitest's 5s limit, so they could never report the outcome they assert — these are the "transient flakes" seen earlier; they were a real defect. - The login timeout tests were slow only because `collectCodexStatus` happens to spread `process.env` so `sleep` resolves; switched to the execPath shim so the slowness is a property of the test. - Added the first real-spawn coverage of the default ACP probe, which had none: deleting its budget race left every test green.
Harold Hunt (huntharo)
force-pushed
the
claude/gifted-jones-7ba9fd
branch
from
August 18, 2026 00:08
e8c37a7 to
d589cf7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
command-discovery.tsprobed<command> --versionwith a hardcodedtimeout: 2_000— no way for a caller to change, extend, or cancel it.2s is generous for a native binary and marginal for a shim. Measured against a real Codex 0.146.0 install on a Windows VM:
codex.execodex.cmd(cmd.exe → node → shim)A loaded machine goes straight past 2s.
Worse than a slow path: an overrun was not reported as slow — it was indistinguishable from "this tool is not installed". The candidate came back with no version, a version-gating consumer demoted it (a version is required for protocol-compatibility gating), no candidate was selected, and
resolve()threwCodexCliNotInstalledError. Nothing retried. One slow moment stranded the caller with Codex reported missing.discoverCommandsis generic, so the same path servesgitandghlookups.What changed
1. The budget is configurable.
versionTimeoutMsonDiscoverCommandOptions,discoverCodexCommands, andresolveCodexCommand, with a namedDEFAULT_COMMAND_VERSION_TIMEOUT_MSexport. Shape follows the siblings that already got this right (DEFAULT_SHELL_PATH_TIMEOUT_MS,requestTimeoutMs,warmTimeoutMs).2. The default is raised to 10s, sized for the shim chain rather than the native binary — configurability alone doesn't help a caller who doesn't know to tune it, and the symptom points away from the cause.
3. A timeout is distinguishable in the result shape — the load-bearing item.
readCommandVersionis now exported and returns aCommandVersionProbeOutcome:That outcome rides on every discovery candidate and on
ResolvedCommandCandidate, so a consumer can tell an unfinished measurement from a missing CLI and re-probe on its own budget or surface an honest message.isUnprovenVersionProbe()names the "no evidence" set.Two places were still swallowing the signal and are fixed here:
includeFailedAutoCandidateswas set — the evidence vanished one level up;failureReason: "not_executable"when the access check had proven nothing. Nowversion_probe_timed_out.CodexCliNotInstalledErrorkeeps its type (existing catch sites still work) but carriestimedOutCommands/probeTimedOutwhen a timeout — not a missing binary — is why nothing resolved.4. Cancellation.
AbortSignalon every entry point, for a renderer stampede or a quit mid-probe. An aborted run returns a snapshot witherror: COMMAND_DISCOVERY_ABORTED;resolveCodexCommandthrowsCodexDiscoveryAbortedError, deliberately not aCodexCliNotInstalledError— a run that stopped looking is not evidence about what is installed.Three latent hangs of the same class, also fixed
cmd.exewrapper of a.cmdshim leaves thenodegrandchild holding the stdio pipes open, soexecFile'sclosenever fires and no timeout value would have helped. Both probes now race their own budget and resolve regardless.collectCodexStatushad no budget at all — a wedgedcodex login statushung the caller forever. NowDEFAULT_CODEX_STATUS_TIMEOUT_MS(10s) withoutcome/timedOut, so a slow check isn't read as a signed-out profile.AcpConnection.requestacceptedtimeoutMsand ignored it (_timeoutMs), making the 30sinitializeand 1hsession/promptbudgetsAcpAgentClientpasses dead code. Now enforced. This is the only behavior change that can turn a previously-infinite wait into a rejection.agent-acplocal discovery gainsprobeTimeoutMs+AbortSignal. Its default stays 5s:discoverStrategyInstanceswalks candidates serially, so the worst case is candidates × budget, not one budget — raising it there is a knowing choice, not a default.Reviewer notes
version_probe_timed_outinstead ofnot_executable. PwrAgent can adopt this and delete its desktop-owned 10s re-probe workaround.discoverCommandsnow runs fixed and auto candidates in one parallel wave. They were awaited in sequence, so a fully-hung machine cost two full budgets — which matters much more at 10s than at 2s.codex-discovery.test.tsfail on any machine with a real Codex CLI at a well-known path (they passplatform: "linux", whose auto-candidates include/usr/local/bin/codex, and the host binary beats the temp shim). They fail identically on a pristineb7cd7bc; there's no seam to override the well-known paths, so fixing it is filed separately rather than widening this change.agent-client'sCodexThreadClient/CodexOneShotClientcallresolveCodexCommandinternally with no options seam. They inherit the raised default but can't tune it — easy to thread through if wanted.Testing
pnpm typecheck— clean across all 7 packages.pnpm test— 271 pass (24 new: 19 incommand-discovery-timeout.test.ts, 4 incodex-login.test.ts, plus ACP connection and discovery cases), alongside the 4 pre-existing failures above.lint:boundaries,lint:deps,lint:licenses— all pass.🤖 Generated with Claude Code