Skip to content

feat(discovery): report probe timeouts instead of "not installed" - #31

Merged
Harold Hunt (huntharo) merged 4 commits into
mainfrom
claude/gifted-jones-7ba9fd
Aug 18, 2026
Merged

feat(discovery): report probe timeouts instead of "not installed"#31
Harold Hunt (huntharo) merged 4 commits into
mainfrom
claude/gifted-jones-7ba9fd

Conversation

@huntharo

Copy link
Copy Markdown
Contributor

The problem

command-discovery.ts probed <command> --version with a hardcoded timeout: 2_000 — no way for a caller to change, extend, or cancel it.

2s is generous for a native binary and marginal for a shim. Measured against a real Codex 0.146.0 install on a Windows VM:

launch path time
native codex.exe ~0.3s
npm codex.cmd (cmd.exe → node → shim) ~1.5s warm

A loaded machine goes straight past 2s.

Worse than a slow path: an overrun was not reported as slow — it was indistinguishable from "this tool is not installed". The candidate came back with no version, a version-gating consumer demoted it (a version is required for protocol-compatibility gating), no candidate was selected, and resolve() threw CodexCliNotInstalledError. Nothing retried. One slow moment stranded the caller with Codex reported missing. discoverCommands is generic, so the same path serves git and gh lookups.

What changed

1. The budget is configurable. versionTimeoutMs on DiscoverCommandOptions, discoverCodexCommands, and resolveCodexCommand, with a named DEFAULT_COMMAND_VERSION_TIMEOUT_MS export. Shape follows the siblings that already got this right (DEFAULT_SHELL_PATH_TIMEOUT_MS, requestTimeoutMs, warmTimeoutMs).

2. The default is raised to 10s, sized for the shim chain rather than the native binary — configurability alone doesn't help a caller who doesn't know to tune it, and the symptom points away from the cause.

3. A timeout is distinguishable in the result shape — the load-bearing item. readCommandVersion is now exported and returns a CommandVersionProbeOutcome:

type CommandVersionProbeOutcome =
  | "ok" | "version_not_reported"
  | "not_found" | "not_executable"   // verdicts about the command
  | "timed_out" | "aborted"          // no evidence either way
  | "failed";

That outcome rides on every discovery candidate and on ResolvedCommandCandidate, so a consumer can tell an unfinished measurement from a missing CLI and re-probe on its own budget or surface an honest message. isUnprovenVersionProbe() names the "no evidence" set.

Two places were still swallowing the signal and are fixed here:

  • a timed-out candidate was filtered out of the snapshot entirely unless includeFailedAutoCandidates was set — the evidence vanished one level up;
  • it was labelled failureReason: "not_executable" when the access check had proven nothing. Now version_probe_timed_out.

CodexCliNotInstalledError keeps its type (existing catch sites still work) but carries timedOutCommands / probeTimedOut when a timeout — not a missing binary — is why nothing resolved.

4. Cancellation. AbortSignal on every entry point, for a renderer stampede or a quit mid-probe. An aborted run returns a snapshot with error: COMMAND_DISCOVERY_ABORTED; resolveCodexCommand throws CodexDiscoveryAbortedError, deliberately not a CodexCliNotInstalledError — a run that stopped looking is not evidence about what is installed.

Three latent hangs of the same class, also fixed

  • Neither probe could be trusted to settle. Killing the cmd.exe wrapper of a .cmd shim leaves the node grandchild holding the stdio pipes open, so execFile's close never fires and no timeout value would have helped. Both probes now race their own budget and resolve regardless.
  • collectCodexStatus had no budget at all — a wedged codex login status hung the caller forever. Now DEFAULT_CODEX_STATUS_TIMEOUT_MS (10s) with outcome / timedOut, so a slow check isn't read as a signed-out profile.
  • AcpConnection.request accepted timeoutMs and ignored it (_timeoutMs), making the 30s initialize and 1h session/prompt budgets AcpAgentClient passes dead code. Now enforced. This is the only behavior change that can turn a previously-infinite wait into a rejection.

agent-acp local discovery gains probeTimeoutMs + AbortSignal. Its default stays 5s: discoverStrategyInstances walks candidates serially, so the worst case is candidates × budget, not one budget — raising it there is a knowing choice, not a default.

Reviewer notes

  • Backward compatible throughout — every addition is optional; the only semantic changes are the raised default, the newly-enforced ACP request timeout, and a timed-out candidate reporting version_probe_timed_out instead of not_executable. PwrAgent can adopt this and delete its desktop-owned 10s re-probe workaround.
  • Bonus: discoverCommands now runs fixed and auto candidates in one parallel wave. They were awaited in sequence, so a fully-hung machine cost two full budgets — which matters much more at 10s than at 2s.
  • Pre-existing test failures, not from this PR: 4 tests in codex-discovery.test.ts fail on any machine with a real Codex CLI at a well-known path (they pass platform: "linux", whose auto-candidates include /usr/local/bin/codex, and the host binary beats the temp shim). They fail identically on a pristine b7cd7bc; there's no seam to override the well-known paths, so fixing it is filed separately rather than widening this change.
  • Deliberately untouched: agent-client's CodexThreadClient / CodexOneShotClient call resolveCodexCommand internally with no options seam. They inherit the raised default but can't tune it — easy to thread through if wanted.

Testing

  • pnpm typecheck — clean across all 7 packages.
  • pnpm test — 271 pass (24 new: 19 in command-discovery-timeout.test.ts, 4 in codex-login.test.ts, plus ACP connection and discovery cases), alongside the 4 pre-existing failures above.
  • lint:boundaries, lint:deps, lint:licenses — all pass.
  • New tests pin the distinction end to end: probe outcome, discovery candidate, and resolved-command shape; that the budget is authoritative against a sleeping shim; and that abort never masquerades as "not installed".

🤖 Generated with Claude Code

The `<command> --version` probe in codex-discovery hardcoded a 2s budget
with no way to change, extend, or cancel it. 2s is generous for a native
binary (~0.3s) but marginal for an npm shim: `codex.cmd` is a
`cmd.exe -> node -> shim` chain measuring ~1.5s warm on Windows, so a
loaded machine crosses it.

Overrunning was not reported as slow — it was indistinguishable from
"not installed". The candidate came back with no version, a version-
gating consumer demoted it, nothing was selected, and `resolve()` threw
`CodexCliNotInstalledError`. Nothing retried. `discoverCommands` is
generic, so this hit `git` and `gh` lookups too.

- Budget is configurable (`versionTimeoutMs`) with an exported
  `DEFAULT_COMMAND_VERSION_TIMEOUT_MS`, raised to 10s for the shim chain.
- `readCommandVersion` is exported and returns a
  `CommandVersionProbeOutcome` (ok / version_not_reported / not_found /
  not_executable / timed_out / aborted / failed). The outcome rides on
  every candidate and on `ResolvedCommandCandidate`, so a caller can tell
  an unfinished measurement from a missing CLI and re-probe on its own
  budget. A timed-out candidate is no longer labelled `not_executable`
  nor filtered out of the snapshot.
- `CodexCliNotInstalledError` keeps its type but carries
  `timedOutCommands` / `probeTimedOut`.
- `AbortSignal` on every entry point; an aborted run yields
  `error: COMMAND_DISCOVERY_ABORTED` / `CodexDiscoveryAbortedError`
  rather than a false "not installed".

Also fixes three latent hangs of the same class:

- Both probes now settle within their budget even when the child cannot
  be killed. Killing the `cmd.exe` wrapper of a `.cmd` shim leaves the
  `node` grandchild holding the stdio pipes, so `execFile` never calls
  back and no timeout value would have helped.
- `collectCodexStatus` had no budget at all and could hang a caller
  forever. It now has one (`DEFAULT_CODEX_STATUS_TIMEOUT_MS`, 10s) and
  reports `outcome` / `timedOut` so a slow check is not read as a
  signed-out profile.
- `AcpConnection.request` accepted `timeoutMs` and ignored it, making the
  30s `initialize` and 1h `session/prompt` budgets `AcpAgentClient`
  passes dead code. Now enforced.

agent-acp local discovery gains `probeTimeoutMs` (default unchanged at
5s — its candidate loop is serial) plus `AbortSignal`, and reports an
overrun as `reason: "probe-timed-out"` rather than
`version-probe-failed`.

Everything is additive and backward compatible so PwrAgent can adopt it
and delete its desktop-owned re-probe workaround.
The new timeout tests passed on macOS for the wrong reason. Their shim
was `#!/bin/sh` + `sleep 3`, but the probe env sets `PATH` to just the
temp dir, so the shell could not resolve `sleep`, printed "command not
found", and fell straight through to `echo` — answering instantly. macOS
spawn latency (~1.6s for a freshly written script) still exceeded the
150ms budget, so the assertions held; on a fast-spawn Linux runner the
shim answered inside the budget and nine tests failed.

Run the delay inside `process.execPath` instead: no PATH lookup at all,
and the child dies cleanly on kill rather than orphaning a `sleep`.

Also update the Windows-only `.cmd` shim test, whose exact `toEqual`
now sees the new `versionProbeOutcome` key on the resolved candidate.
`CodexThreadClient` and `CodexOneShotClient` resolve their binary through
codex-discovery on first connect, but passed no options along — a host
could only take the default budget, and an overrun surfaced as
`CodexCliNotInstalledError` from the very first call.

Both now accept `commandVersionTimeoutMs` and forward it as
`versionTimeoutMs`. Named for the client's vocabulary (it sits beside
`requestTimeoutMs` / `turnTimeoutMs`, and `clientVersion` is right
above it) rather than mirroring the discovery option name.

Covered by a test on the REAL discovery path, since `transportFactory`
short-circuits discovery entirely: a `codex` shim that cannot answer for
3s must still come back inside a second when the budget is 150ms.
Verified the test fails (4158ms) without the pass-through.
Ten review angles over the PR diff; fifteen findings, all applied.

Correctness:
- acp-local-discovery and codex-login clamped their budgets without a
  `Number.isFinite` guard. `Infinity` is the natural spelling of "no
  budget", and both consumers reject it: `setTimeout` collapses a
  non-finite delay to 1ms, and `execFile`'s `timeout` throws
  ERR_OUT_OF_RANGE synchronously. `probeTimeoutMs: Infinity` therefore
  reported every installed ACP agent as not installed.
- ACP discovery only recorded a timed-out candidate behind
  `includeRejectedCandidates` (default false), so on the default path the
  group was dropped entirely — the very "slow machine looks like an empty
  machine" bug this PR fixes on the codex side, still live on the ACP one.
- `AcpAgentClient.initialize` registered its notification listeners before
  the `initialize` request, which this PR made rejectable. A retry after a
  timeout left the first listener registered and unreachable, so every
  later `session/update` dispatched twice and duplicated the transcript.
- `resolveCodexCommand` returned the selected candidate before testing for
  abort, making `CodexDiscoveryAbortedError` unreachable and handing back a
  command whose version was never measured.
- The `skipVersionProbe` branch hardcoded its outcome, swallowing a
  preflight that said `not_found` and reporting `not_executable` instead —
  a regression against main.
- `collectCodexStatus` left its stdout/stderr handlers attached after
  abandoning a child, so a surviving child grew `output` without bound.
- Abort was labelled from a trailing `signal.aborted` read, so a signal
  firing after every probe finished marked a complete snapshot abandoned
  and masked a `codex_too_old` verdict. Now driven by the evidence.
- `resolveDiscoveredCommand` pinned some other candidate's outcome onto the
  bare-name fallback, sending callers to re-probe the wrong command.
- `checkCodexAuthStatus` dropped the `aborted` outcome, painting a
  self-cancelled check as a broken profile. `CodexAuthStatusResponse` now
  carries the full `CodexStatusOutcome` instead of a `timedOut` flag.
- `withAcpRequestTimeout` treated `Infinity` as "no budget" but `0` as a
  1ms budget; both now mean unbounded.
- Both agent-client clients dropped `versionProbeOutcome`, hiding that the
  minimum-version gate had been skipped. They now warn.

Tests:
- Two tests inherited the 10s default budget under vitest's 5s limit, so
  they could never report the outcome they assert — these are the
  "transient flakes" seen earlier; they were a real defect.
- The login timeout tests were slow only because `collectCodexStatus`
  happens to spread `process.env` so `sleep` resolves; switched to the
  execPath shim so the slowness is a property of the test.
- Added the first real-spawn coverage of the default ACP probe, which had
  none: deleting its budget race left every test green.
@huntharo
Harold Hunt (huntharo) merged commit 2c5f28f into main Aug 18, 2026
2 checks passed
@huntharo
Harold Hunt (huntharo) deleted the claude/gifted-jones-7ba9fd branch August 18, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant