Skip to content

test(mcp): make live MCP server tests opt-in - #1354

Merged
akramcodez merged 3 commits into
Nano-Collective:mainfrom
Piyushrathoree:fix/mcp-live-tests-opt-in-1348
Sep 28, 2026
Merged

akramcodez merged 3 commits into
Nano-Collective:mainfrom
Piyushrathoree:fix/mcp-live-tests-opt-in-1348

Conversation

@Piyushrathoree

@Piyushrathoree Piyushrathoree commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1348.

Description

The HTTP transport tests in source/mcp/mcp-client.spec.ts connect to real third-party servers (mcp.deepwiki.com, mcp.context7.com) and were only skipped when CI was set. Running the suite locally without network access, or when one of those hosts is down, fails with an unhandled TypeError: fetch failed.

The live tests are now gated behind RUN_LIVE_MCP_TESTS=true instead. The invalid URL and websocket protocol checks only hit localhost / validation, so they run unconditionally now.

Without the flag: 63 passed, 8 skipped. With RUN_LIVE_MCP_TESTS=true: all 71 pass.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Changeset

  • Added a changeset (pnpm changeset) describing this change for the changelog

Test-only change, no changeset.

Testing

Automated Tests

  • New features include passing tests in .spec.ts/tsx files
  • All existing tests pass (pnpm test:all completes successfully)
  • Tests cover both success and error scenarios

Manual Testing

  • Tested with Ollama
  • Tested with OpenRouter
  • Tested with OpenAI-compatible API
  • Tested MCP integration (if applicable)

Checklist

  • If this was for an open issue, I was assigned to it
  • Code follows project style guidelines
  • Self-review completed
  • Documentation updated (if needed)
  • No breaking changes (or clearly documented)
  • Appropriate logging added using structured logging (see CONTRIBUTING.md)

@github-actions

Copy link
Copy Markdown
Contributor

nc-review: comments — 1 nit

@Piyushrathoree — a few things worth a look, none blocking.

PR #1354 gates the eight live HTTP-transport integration tests in source/mcp/mcp-client.spec.ts behind RUN_LIVE_MCP_TESTS=true, directly resolving the symptom of issue #1348. The two error-path tests that don't need external network (invalid URL via localhost:99999 and websocket protocol validation, both of which fail locally before any fetch) are correctly converted to unconditional test. The change is small, targeted, and does not touch any public contract, so no changeset is needed.

⚪ nit · completeness · source/mcp/mcp-client.spec.ts:68

The new opt-in flag RUN_LIVE_MCP_TESTS is only documented in the inline comment and PR description. CONTRIBUTING's testing section and the devcontainer README mention pnpm test:ava without flagging that MCP HTTP integration coverage is opt-in. Contributors running the suite to verify HTTP transport will get a silent skip and may not realise they need to set the flag. A short note in CONTRIBUTING.md (or the test file's comment) pointing at the flag would make the new gate discoverable, but the behaviour itself is correct.


🔴 blocking · 🟠 a reviewer would ask for a change · ⚪ optional

Automated code review — correctness, security, design, tests, plus duplicates and scope. A human still decides; this is not a substitute for review and is not exhaustive. The required status checks separately cover lint, formatting, types, unused dependencies, the test suite and the build. This bot never merges. Maintainers can rerun with /re-review.

@github-actions github-actions Bot added the agent:comments nc-review left non-blocking findings label Sep 16, 2026
@akramcodez

Copy link
Copy Markdown
Member

Hey @Piyushrathoree, the PR looks good! Could you add a changeset as well?

@Piyushrathoree

Copy link
Copy Markdown
Contributor Author

Hey @Piyushrathoree, the PR looks good! Could you add a changeset as well?

hey not sure about the change set , its just a test change , if I still have to add a changeset which one should i go for? patch ig

The HTTP transport tests in mcp-client.spec.ts hit real third-party
servers (mcp.deepwiki.com, mcp.context7.com) and were only skipped when
CI was set. Running the suite locally without network access, or when
one of those hosts is down, fails with an unhandled "TypeError: fetch
failed" and takes the whole test run down with it.

Gate them behind RUN_LIVE_MCP_TESTS=true instead. The invalid URL and
websocket protocol checks only hit localhost / validation, so they now
run unconditionally.

Fixes Nano-Collective#1348.
@Piyushrathoree
Piyushrathoree force-pushed the fix/mcp-live-tests-opt-in-1348 branch from 7be7d1b to a97a926 Compare September 22, 2026 19:36
@Piyushrathoree

Copy link
Copy Markdown
Contributor Author

fixed the conflicts too @akramcodez

@Piyushrathoree

Copy link
Copy Markdown
Contributor Author

@akramcodez added the changeset too

@akramcodez
akramcodez merged commit 2269460 into Nano-Collective:main Sep 28, 2026
16 checks passed
@akramcodez

Copy link
Copy Markdown
Member

Thanks for the PR @Piyushrathoree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:comments nc-review left non-blocking findings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] MCP test suite fails locally with TypeError: fetch failed on external DeepWiki server

2 participants