Paginate MCP tools/list and prompts/list

[DivyaMaddipudi]

Paginate MCP tools/list and prompts/list

McpServerProxy dropped the MCP nextCursor, so a server that returned more than
one page (servers commonly page at ~30 items) was silently truncated to its first
page. This change pages through nextCursor with three termination guards — a blank
or absent cursor ends pagination, a repeated or cycling cursor aborts, and a hard
page cap backstops a runaway server — returns an immutable list, and logs the number
of items/pages fetched.

It also hardens the surrounding paths:

• Isolation — each proxy's listing in McpService is wrapped so one proxy's
  failure can't abort tool/prompt discovery for the rest.
• No reader-thread deadlock — tools/list_changed refreshes now run off the
  transport's reader thread (a synchronous refresh calls listTools() whose response
  is read by that same thread, deadlocking it), using an add-then-prune swap so a
  concurrent tools/list never sees a gap.
• No unbounded hang — StdioProxy gets a per-request timeout (default 5 min,
  builder-configurable, symmetric with HttpMcpProxy) so a server that stays alive
  but never responds can't block the caller forever.

What behavior changes?

Before: listTools() / listPrompts() issued a single JSON-RPC call and ignored
nextCursor, returning only the first page (~30 tools) for any multi-page server.
After: they return the union of all pages. Single-page servers are unchanged — still
exactly one round-trip. Secondary observable differences: a failing/misbehaving proxy
no longer wipes out discovery for other proxies; a tools/list_changed refresh no
longer risks freezing a stdio connection; and a silent stdio server now fails after
the timeout instead of hanging indefinitely.

Why is this change needed?

Bug. MCP tools/list and prompts/list are cursor-paginated, but the client only
ever read the first page, so any server/gateway federating more than ~30 tools had
its tool set truncated everywhere McpServerProxy is used (offline enumeration and
the runtime proxy federation in McpService) — with no error and no log, making it
hard to notice.

How was this validated?

./gradlew :mcp:mcp-server:test (all green) plus Spotless. Tests added:

• McpServerProxyTest (12): multi-page union with exact nextCursor -> cursor
  threading, single-page regression, blank-cursor termination, repeated-cursor abort,
  cycling (A -> B -> A) abort, page-cap boundary, error on the first page, error on a
  later page, null-result handling, listPrompts guard path, and immutable return.
• HttpMcpProxyTest (+1): real wire round-trip — a localhost HttpServer pages
  tools/list with a nextCursor and asserts listTools() follows it, reading the
  cursor off the actually-deserialized response.
• McpServiceTest (2): a tools/list_changed refresh runs off the notifying thread
  (deadlock regression), and add-then-prune correctly adds new tools while pruning
  stale ones.
• StdioProxyTest (1): a silent server times out instead of hanging.

What should reviewers focus on?

• McpServerProxy#listPaginated(...) — the pagination loop and its three termination
  guards, the immutable return, and the null-result guard.
• mcp/mcp-schemas/model/main.smithy — the additive optional nextCursor on
  ListToolsResult / ListPromptsResult.
• McpService — the two listTools() call sites now isolated per-proxy, and the
  tools/list_changed handler: async dispatch off the reader thread + refreshProxyTools
  add-then-prune swap.
• StdioProxy#rpc — the per-request timeout and pending-request cleanup on completion.

Additional Links

• MCP specification — Server Utilities: Pagination (cursor-based nextCursor /
  cursor): https://modelcontextprotocol.io/specification

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.