Skip to content

[bot] Missing instrumentation for the official Model Context Protocol (MCP) Java SDK #141

Description

@braintrust-bot

Summary

The Braintrust Java SDK does not instrument the official Model Context Protocol (MCP) Java SDK (io.modelcontextprotocol.sdk:mcp). MCP is the open standard for connecting LLM applications to external tools/data sources, and its Java SDK is maintained by the Model Context Protocol org in collaboration with Spring AI — a framework this repo already instruments (springai_1_0_0). The MCP client's tool-invocation execution surface (the mechanism models actually use to call tools) is completely uninstrumented, even though Spring AI chat model spans are captured.

What is missing

The io.modelcontextprotocol.sdk:mcp SDK provides these execution-oriented surfaces, none of which are instrumented:

API Class Description
callTool() / callToolAsync() McpSyncClient / McpAsyncClient Executes a tool on an MCP server and returns its result — the core "tool-calling during model execution" surface
listTools() McpSyncClient / McpAsyncClient Discovers available tools from a server
getPrompt() / listPrompts() McpSyncClient Retrieves/executes server-defined prompt templates
readResource() / listResources() McpSyncClient Reads server-exposed resources referenced during agent execution
Server-side CallToolResult handlers McpServer (sync/async) Server-side tool execution when this JVM acts as an MCP server

Because Spring AI's MCP integration (spring-ai-mcp) wraps this same SDK to expose MCP tools as ToolCallbacks to ChatModel, any tool call made through MCP during an instrumented Spring AI chat completion still produces no dedicated tool-execution span — the model call is traced, but the tool invocation it triggers is not.

Why existing instrumentation does not cover this

Braintrust docs status

Upstream sources

Local files inspected

  • settings.gradle (lines 28–33) — lists all instrumentation modules (openai_2_15_0, anthropic_2_2_0, genai_1_18_0, langchain_1_8_0, springai_1_0_0, aws_bedrock_2_30_0); no MCP module present
  • braintrust-sdk/instrumentation/springai_1_0_0/src/main/java/dev/braintrust/instrumentation/springai/v1_0_0/BraintrustSpringAI.java — only handles OpenAiChatModel$Builder / AnthropicChatModel$Builder; no MCP tool-callback handling
  • Full-repo grep for modelcontextprotocol, io.modelcontextprotocol — zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions