Skip to content

test(mcp): make live MCP server tests opt-in #1354

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
akramcodez merged 3 commits into from
Sep 28, 2026
Merged

test(mcp): make live MCP server tests opt-in #1354

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a9c34af
test(mcp): make live MCP server tests opt-in
Piyushrathoree Sep 16, 2026
a97a926
docs: document opt-in live MCP tests
Piyushrathoree Sep 16, 2026
355b63b
chore: add changeset for MCP client test updates
Piyushrathoree Sep 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/smooth-coins-teach.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@nanocollective/nanocoder": patch
---

update test - mcp-client-spec.ts
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,12 @@ All new features and bug fixes should include appropriate tests:

This command runs: Biome formatting checks, type checks, lint checks, AVA tests, Knip, security scans.

MCP live HTTP integration tests are skipped by default. Run them explicitly with:

```bash
RUN_LIVE_MCP_TESTS=true pnpm test:ava source/mcp/mcp-client.spec.ts
```

4. **Test Requirements for PRs**:
- New features **must** include passing tests in `.spec.ts/tsx` files
- Bug fixes should include regression tests when possible
Expand Down
78 changes: 40 additions & 38 deletions source/mcp/mcp-client.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,36 +65,38 @@ const mockTransportFactory = {

console.log(`\nmcp-client.spec.ts`);

// Skip integration tests in CI. These tests hit real third-party MCP servers
// (mcp.deepwiki.com, remote.mcpservers.org, mcp.context7.com) — running them in
// CI would couple our pipeline to those services' uptime. Run them locally to
// verify HTTP transport against live servers.
const isCI = process.env.CI === 'true' || process.env.CI === '1';
const testOrSkip = (title: string, impl: (t: any) => Promise<void> | void) => {
if (isCI) {
// Live integration tests are opt-in because they depend on third-party MCP
// servers and network availability. Run them with RUN_LIVE_MCP_TESTS=true.
const runLiveMcpTests =
process.env.RUN_LIVE_MCP_TESTS === 'true' ||
process.env.RUN_LIVE_MCP_TESTS === '1';
const liveTest = (
title: string,
impl: (t: any) => Promise<void> | void,
) => {
if (!runLiveMcpTests) {
test.skip(title, impl as any);
} else {
test.serial(title, async t => {
let lastErr: any;
for (let i = 0; i < 3; i++) {
const result = await (t as any).try(impl);
if (result.passed) {
result.commit();
return;
}

result.discard();
lastErr = result.errors[0] || new Error('Unknown test failure');
// These are remote integration tests, so we retry any failure
// (usually fetch failures or socket hang ups wrapped in AssertionErrors)
if (i < 2) {
await new Promise(r => setTimeout(r, 2000 * (i + 1)));
continue;
}
}
throw lastErr;
});
return;
}

test.serial(title, async t => {
let lastErr: any;
for (let i = 0; i < 3; i++) {
const result = await (t as any).try(impl);
if (result.passed) {
result.commit();
return;
}

result.discard();
lastErr = result.errors[0] || new Error('Unknown test failure');
// These are remote integration tests, so retry transient failures.
if (i < 2) {
await new Promise(resolve => setTimeout(resolve, 2000 * (i + 1)));
}
}
throw lastErr;
});
};

// ============================================================================
Expand Down Expand Up @@ -734,7 +736,7 @@ test('MCPClient.getServerInfo: returns undefined when only tools exist', t => {
// These tests use real remote MCP servers via HTTP transport
// They test the actual connection, tool listing, and tool execution flow

testOrSkip('MCPClient.connectToServer: connects to remote HTTP MCP server', async t => {
liveTest('MCPClient.connectToServer: connects to remote HTTP MCP server', async t => {
const client = new MCPClient();

// Use DeepWiki public MCP server (no auth required)
Expand Down Expand Up @@ -769,7 +771,7 @@ testOrSkip('MCPClient.connectToServer: connects to remote HTTP MCP server', asyn
t.is(client.getServerTools('test-deepwiki').length, 0);
});

testOrSkip('MCPClient.connectToServer: connects to context7 HTTP server and executes a tool', async t => {
liveTest('MCPClient.connectToServer: connects to context7 HTTP server and executes a tool', async t => {
// Pair with the DeepWiki test above so a single host going dark doesn't
// nuke all HTTP-transport integration coverage. context7 was picked after
// remote.mcpservers.org disappeared at DNS level around mid-May 2026.
Expand Down Expand Up @@ -805,7 +807,7 @@ testOrSkip('MCPClient.connectToServer: connects to context7 HTTP server and exec
t.false(client.isServerConnected('test-context7'));
});

testOrSkip('MCPClient.connectToServers: connects to multiple HTTP servers', async t => {
liveTest('MCPClient.connectToServers: connects to multiple HTTP servers', async t => {
const client = new MCPClient();

const servers = [
Expand Down Expand Up @@ -841,7 +843,7 @@ testOrSkip('MCPClient.connectToServers: connects to multiple HTTP servers', asyn
await client.disconnect();
});

testOrSkip('MCPClient.getAllTools: builds tools registry from connected HTTP server', async t => {
liveTest('MCPClient.getAllTools: builds tools registry from connected HTTP server', async t => {
const client = new MCPClient();

const server = {
Expand Down Expand Up @@ -872,7 +874,7 @@ testOrSkip('MCPClient.getAllTools: builds tools registry from connected HTTP ser
await client.disconnect();
});

testOrSkip('MCPClient.getNativeToolsRegistry: creates registry from connected HTTP server', async t => {
liveTest('MCPClient.getNativeToolsRegistry: creates registry from connected HTTP server', async t => {
const client = new MCPClient();

const server = {
Expand Down Expand Up @@ -903,7 +905,7 @@ testOrSkip('MCPClient.getNativeToolsRegistry: creates registry from connected HT
await client.disconnect();
});

testOrSkip('MCPClient.callTool: executes tool on connected HTTP server', async t => {
liveTest('MCPClient.callTool: executes tool on connected HTTP server', async t => {
const client = new MCPClient();

const server = {
Expand Down Expand Up @@ -934,7 +936,7 @@ testOrSkip('MCPClient.callTool: executes tool on connected HTTP server', async t
await client.disconnect();
});

testOrSkip('MCPClient.getToolMapping: returns mapping from connected HTTP server', async t => {
liveTest('MCPClient.getToolMapping: returns mapping from connected HTTP server', async t => {
const client = new MCPClient();

const server = {
Expand Down Expand Up @@ -966,7 +968,7 @@ testOrSkip('MCPClient.getToolMapping: returns mapping from connected HTTP server
await client.disconnect();
});

testOrSkip('MCPClient.getToolEntries: returns entries from connected HTTP server', async t => {
liveTest('MCPClient.getToolEntries: returns entries from connected HTTP server', async t => {
const client = new MCPClient();

const server = {
Expand Down Expand Up @@ -996,7 +998,7 @@ testOrSkip('MCPClient.getToolEntries: returns entries from connected HTTP server
// Error Handling Tests with Real Servers
// ============================================================================

testOrSkip('MCPClient.connectToServer: handles invalid URL gracefully', async t => {
test('MCPClient.connectToServer: handles invalid URL gracefully', async t => {
const client = new MCPClient();

const server = {
Expand All @@ -1009,7 +1011,7 @@ testOrSkip('MCPClient.connectToServer: handles invalid URL gracefully', async t
await t.throwsAsync(async () => await client.connectToServer(server));
});

testOrSkip('MCPClient.connectToServer: validates websocket URL protocol', async t => {
test('MCPClient.connectToServer: validates websocket URL protocol', async t => {
const client = new MCPClient();

const server = {
Expand Down
Loading