Skip to content

fix(sensor): attribute MCP servers and failed tool calls in Claude Code - #167

Open
vschs007 wants to merge 1 commit into
uber:mainfrom
vschs007:fix/claude-parser-mcp-attribution
Open

vschs007 wants to merge 1 commit into
uber:mainfrom
vschs007:fix/claude-parser-mcp-attribution

Conversation

@vschs007

Copy link
Copy Markdown
Contributor

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • Optimization
  • Documentation Update

Related issue: Supersedes and closes #38 (incorporating review feedback from @pengyuzhang)

What changed?

  1. Tool Classification & MCP Attribution:

    • Added _classify_tool() to ClaudeParser:
      • Classifies Bash shell executions as tool_type="terminal_command".
      • Detects mcp__<server>__<tool> invocations, classifying them as tool_type="mcp_tool" and populating server_name="<server>".
      • Defaults built-in inspection tools (Read, Edit, Glob, Grep, View) to tool_type="tool_use".
      • Includes isinstance(tool_name, str) safety guard and defaults present-but-null names ("name": null) to "unknown", preventing session drop crashes.
    • Wired _classify_tool into assistant tool extraction so initial ToolUsage objects carry both tool_type and server_name.
  2. Accurate Error & Completion Status Tracking:

    • Extracted is_error across both tool_result content blocks (item.get("is_error")) and top-level toolUseResult metadata objects (tur.get("is_error") / tur.get("status") == "error").
    • Preserves fallback error strings from toolUseResult["error"] if content is empty.
    • Followed reviewer feedback from fix(sensor): attribute MCP servers and failed tool calls in Claude Code sessions #38: status = "error" if is_error else "success". Because a tool_result is an authoritative completion signal, successful tools returning empty strings (e.g. ToolSearch with 0 matches) are correctly recorded as status="success" rather than ambiguous "unknown".
    • Preserved tool_type and server_name during the replace(old_tool, ...) result merge.
  3. Documentation:

    • Updated Sensor/README.md to document tool classification, MCP attribution, and error retention under the Claude Code section.
  4. Unit Tests:

    • Added 7 unit tests in Sensor/tests/test_claude_parser.py:
      • Direct tool classification across Bash, mcp__*, built-ins, and malformed prefixes.
      • Multi-tool transcript session parsing.
      • server_name and tool_type surviving user turn result merge.
      • Failed commands (is_error: True) setting status="error" and populating error.
      • Top-level toolUseResult envelope error fallback.
      • Empty non-error outputs correctly recording status="success".
      • Null/missing tool names ("name": null) safely defaulting to "unknown" without discarding sessions.

Why?

PR #38 originally proposed MCP attribution and is_error handling for Claude Code, and review verified the premise against real corpora (is_error living on tool_result blocks; ~18k MCP calls needing attribution). However, PR #38 was abandoned and blocked by CLA before addressing review comments:

  1. _classify_tool crashed with AttributeError on "name": null, discarding whole sessions.
  2. Empty non-error results were mislabeled as status="unknown".
  3. Value-based matching could swap results between duplicate pending calls.

While the post-#130 telemetry refactor (abc12c2) solved identity indexing with pending_tools[tool_use_id] = (msg_idx, tool_idx), the tool classification and is_error handling were never carried into main. Currently:

  • tool_type is hardcoded to "tool_use" for all tools, obscuring arbitrary shell commands.
  • server_name is always None, breaking MCP policy governance in downstream detectors.
  • Tool errors/denials are ignored, causing failed attacks to be recorded as status="success".

This PR brings the verified improvements to main while addressing all review items from #38.

How did you test it?

  1. Ruff Linter:
    ruff check Sensor/adr_sensor/parsers/claude_parser.py Sensor/tests/test_claude_parser.py
    # All checks passed!

@vschs007
vschs007 force-pushed the fix/claude-parser-mcp-attribution branch from 0edda36 to 885ed06 Compare November 1, 2026 04:14

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant