Skip to content

feat(claude_sdk): add CSDK-019, tool prints to stdout for diagnostics - #83

Open
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/claude-sdk-observability
Open

feat(claude_sdk): add CSDK-019, tool prints to stdout for diagnostics#83
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/claude-sdk-observability

Conversation

@bradAGI

@bradAGI bradAGI commented Aug 24, 2026

Copy link
Copy Markdown

Claude SDK and MCP were the two mature packs with no observability rule. OpenAI ships OAI-010 and ADK ships ADK-009 for the same print() pattern.

Two Claude-SDK-specific consequences go beyond the lost-log-line framing OAI-010 uses:

  • A Python SDK tool is commonly served to the agent over an MCP stdio transport, where stdout carries the JSON-RPC frames. A loose print interleaves with them and the client hits a parse error on a line that isn't JSON.
  • When the SDK is driven programmatically, the host application is already reading the SDK's own message stream — so tool prints land interleaved with it rather than in the application's logs, attributable to no particular turn or tool call.

The fix reflects the first point: where the tool may be served over stdio, the log handler needs to write to stderr, which the transport leaves alone. Swapping print for a logger that still defaults to stdout doesn't fix that case.

Severity/confidence match OAI-010 (low / 0.65) since the dominant case here is still the lost diagnostic. The sharper stdio-corruption case is the primary reading in MCP-023 (#82), where stdio is the default transport.

Verification — engine built at main:

$ trustabl rules validate .
OK: 86 rule pack(s), 207 rule(s) valid under rule schema version 14

Fire (print(...) in a @tool-decorated handler): CSDK-002, CSDK-019, CSDK-203
Silent (logger.info(...) with basicConfig(stream=sys.stderr)): CSDK-002, CSDK-203

(CSDK-002 and CSDK-203 are pre-existing rules firing on the fixture's untyped args param and missing CLAUDE.md.)

has_print_call matches a bare print callee, so pprint and other attribute calls don't false-positive.

No new predicates, so no schema_version bump.

Claude SDK and MCP were the two mature packs with no observability rule;
OpenAI ships OAI-010 and ADK ships ADK-009 for the same pattern.

Two Claude-SDK-specific consequences go beyond the lost log line OAI-010
describes. A tool defined in a Python SDK server is commonly served to the
agent over an MCP stdio transport, where stdout carries the JSON-RPC
frames and a loose print makes the client hit a parse error on a line that
is not JSON. And when the SDK is driven programmatically, the host
application is already reading the SDK's own message stream, so tool
prints land interleaved with it rather than in the application's logs,
attributable to no particular turn or tool call.
@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @bradAGI, this slots into the pack cleanly. CSDK-019 is unclaimed on main and by any other open PR, the policy header mirrors openai_sdk/observability.yaml, and low / 0.65 matches OAI-010, which is the right anchor for the same predicate. has_print_call already ships and is already dispatched at tool scope, so you are right that no schema_version bump is needed. The engine half in trustabl/trustabl#157 carries a byte-identical fixture copy plus fire and silent cases, so the pair is complete as long as both land together.

One question on the explanation. Discovery classifies a tool registered with @server.tool or @mcp.tool as mcp_tool, not claude_sdk_tool, so the standalone stdio server shape belongs to MCP-023 in #82. What actually reaches this rule is the claude_agent_sdk @tool decorator, the in-process SDK MCP server shape, where stdout is not the transport. Would you hedge that first consequence the way OAI-010 does, or lead with the host message stream point, which holds for every tool this rule can fire on?

Worth adding an observability.yaml line to the claude_sdk/ tree in the README, since that tree lists every topic file. Nothing to do about attestations.json or the count badge, both regenerate after merge. Before this ships it also needs a rationale doc in trustabl-rulebook, which we can handle or pair on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants