Skip to content

test(windows): run the stdio probe's stderr stubs as Python scripts - #23799

Merged
KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/windows-probe-stub-xfails
Oct 30, 2026
Merged

KiroCrewCaptain merged 1 commit into
kirodotdev:mainfrom
leonlaiyc:claude/windows-probe-stub-xfails

Conversation

@leonlaiyc

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: the stdio probe's stderr-capture tests read a real child's stderr on Windows too, so the two listed as Windows xfails pass there and the bound and redaction tests stop passing on an error that has no stderr.

TestProbeServerStderrCapture in test/test_mcp_discovery_probe_stdio.py passes a /bin/sh script as the MCP server command in four tests. Windows cannot execute a script file directly, so on a Windows host each spawn fails with [WinError 193] %1 is not a valid Win32 application before the child writes anything:

  • test_stderr_captured_when_child_exits_before_response and test_successful_probe_does_not_mention_stderr are strict xfails in test/windows-expected-failures.txt.
  • test_stderr_tail_is_bounded and test_credential_in_stderr_is_redacted are not listed, and they pass: each asserts that the error is short and holds no secret, which an error with no stderr tail satisfies trivially. From main on a Windows host the probe logs MCP probe failed [verbose]: [WinError 193] ... and MCP probe failed [leaky]: [WinError 193] ..., and both pass.

Why it matters

The stderr tail is what turns an opaque "no response" into the server's real failure, such as a traceback or a missing module. Its redaction keeps a credential the server printed out of doctor output and the dashboard. On Windows none of the four tests reaches that code: two are strict xfails and two pass without it.

Not a goal

  • How probe_server drains, bounds or redacts stderr.
  • test_real_grandchild_is_reaped's /bin/sh launcher, which models a POSIX process group and is skipped off POSIX.

What changed (motivation → approach → change)

  • Symptom: on Windows the four stubs fail to spawn with WinError 193. Two tests fail, and two pass without reading any stderr.
  • Root cause: each test passes a shell script as the command, which only a POSIX execve can run, through its shebang.
  • Change, in test/test_mcp_discovery_probe_stdio.py: each stub is a Python script that does what the shell one did, run as command=sys.executable, args=[str(stub)]. That is the spelling test_mcp_discovery_probe_spawn_policy.py already uses for a real probe child. The healthy stub answers initialize and tools/list with the request's own id and ignores the initialized notification, as the shell case did. chmod 0o755 goes, because a script handed to the interpreter needs no execute bit.
  • test_stderr_tail_is_bounded and test_credential_in_stderr_is_redacted also assert that stderr: is in the error. The bound and the redaction are then measured on a captured tail on every platform, and a stub that cannot spawn fails them instead of passing.
  • The two strict-xfail lines are gone from test/windows-expected-failures.txt.

Backwards compatibility

Compatible: test-only; no product code changes.

Removes nothing: two strict-xfail lines go and two assertions are added; no assertion is dropped.

Tests

Proof: not provable -- test-only change

Manual verification

N/A — test-only change; the Windows runs above are the check, and the Windows matrix runs these tests on this PR.

Related Issues

Fixes #23798

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

TestProbeServerStderrCapture spawned four /bin/sh stubs as the MCP
server command. Windows cannot execute a script file directly, so each
spawn failed with WinError 193 before the child wrote anything: two
tests were strict xfails there, and the bound and redaction tests
passed on an error that carried no stderr at all.

Each stub is now a Python script run by sys.executable, so the probe
reads a real child's stderr on every platform. The bound and redaction
tests also assert that a tail was captured, and the two xfail lines go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leonlaiyc
leonlaiyc requested review from a team and KiroCrewCaptain as code owners October 24, 2026 17:22
@leonlaiyc
leonlaiyc requested a review from buluoray October 24, 2026 17:22
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision d3fbe1f13e3ca48e46c7d8ad5f3d9e0a081536c0 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision d3fbe1f13e3ca48e46c7d8ad5f3d9e0a081536c0 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed d3fbe1f13e3ca48e46c7d8ad5f3d9e0a081536c0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d3fbe1f

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed d3fbe1f13e3ca48e46c7d8ad5f3d9e0a081536c0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d3fbe1f

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 24, 2026
@leonlaiyc

Copy link
Copy Markdown
Contributor Author

PR Readiness on d3fbe1f13e is still pending, waiting on Design Review (not started), and that lane stopped on infrastructure, not on the code:

  • Fork Design Review run 40116728973 for this head: Resolve and validate PR (authoritative from GitHub) failed, so Stage the base-owned redactor had no base commit (scripts/scope_redact.py is missing or empty on the base commit ()), and the job was cancelled at 35 minutes without publishing a verdict.

The Security Scope, Opus 5.5, GPT 6.1, First Principles and UX lanes all published for this head. A fork cannot re-run workflows: could a maintainer re-run run 40116728973?

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of d3fbe1f13e3ca48e46c7d8ad5f3d9e0a081536c0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This is a test-only change. It fixes the root cause (shell-script stubs Windows can't spawn), and the added stderr: assertions match the product's \nstderr: prefix in mcp_discovery_probe.py:879.

[DESIGN-REVIEWED] d3fbe1f

Description read

[DESCRIPTION-READ] 39fde9c4cd15ec8837e341b18d51547fb2fea76dd0f90fe03c012f80783f4596

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 30, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) October 30, 2026 09:06

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: test-only change that runs the stdio probe's stderr stubs as Python scripts so they work on Windows.

@KiroCrewCaptain
KiroCrewCaptain merged commit b5eeaa6 into kirodotdev:main Oct 30, 2026
109 of 110 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stdio probe stderr tests spawn a /bin/sh stub, so Windows never reads the child's stderr

2 participants