fix(anthropic): drop redundant bt_safe_deep_copy passes in tracing#583
Merged
Abhijeet Prasad (AbhiPrasad) merged 1 commit intoJul 16, 2026
Merged
Conversation
Braintrust already deep-copies via bt_safe_deep_copy in SpanImpl.log_internal,
so the eager copies in the Anthropic integration were duplicating work on every
span open/log. Remove them from _normalize_anthropic_input,
_get_input_from_kwargs, and _redact_server_tool_output; the remaining walks
(_normalize_anthropic_data, _process_input_attachments, and the inlined
redaction) already build fresh dicts/lists and never mutate user data.
This mirrors what google_genai/tracing.py does and matches the sdk-integrations
skill guidance ("Do not over-serialize. Braintrust serializes at send/log
time."). All existing cassette-backed tests pass on both matrix versions
(latest=0.116.0 and 0.48.0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Abhijeet Prasad (AbhiPrasad)
approved these changes
Jul 16, 2026
Abhijeet Prasad (AbhiPrasad)
enabled auto-merge (squash)
July 16, 2026 21:28
Abhijeet Prasad (AbhiPrasad)
deleted the
fix/anthropic-drop-redundant-deep-copies
branch
July 16, 2026 21:35
2 tasks
Abhijeet Prasad (AbhiPrasad)
pushed a commit
that referenced
this pull request
Jul 17, 2026
…586) ## Summary - `_iter_tool_calls` ran `_try_to_dict(output)` (Pydantic `model_dump("python")`) on every chat/stream response just to probe for `tool_calls`, even when none were present — an extra recursive walk on top of the one `SpanImpl.log_internal` already runs via `bt_safe_deep_copy`. - Replaced the eager conversion with `_get_field(output, "tool_calls")`, which already handles both dict and Pydantic-object shapes. Downstream tool-call helpers (`_tool_call_name`, `_tool_call_input`, `_tool_call_metadata`) all read via `_get_field`, so behavior is preserved. - Matches the `sdk-integrations` skill guidance ("Do not over-serialize. Braintrust serializes at send/log time.") and mirrors #583 (anthropic). ## Test plan - [x] `nox -s "test_cohere(latest)"` (cohere==7.0.5) — 21 passed - [x] `nox -s "test_cohere(5.0.0)"` — 10 passed, 11 legitimately skipped for v2/audio surfaces not present on 5.0.0 - No cassettes re-recorded, no new tests — used the existing VCR-backed coverage in `test_cohere.py`, which already asserts tool-call span parent/child structure for both v1 (`test_wrap_cohere_chat_v1_tool_call_spans`) and v2 (`test_wrap_cohere_chat_v2_tool_call_spans`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- sfk:created-approved-by --> Created by abhijeet <!-- sfk:slack-thread --> [Slack thread](https://starfolkai.slack.com/archives/C0AQDETAVT3/p1784304169602969?thread_ts=1784304169.602969&cid=C0AQDETAVT3) Co-authored-by: Starfolk <noreply@starfolk.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
7 tasks
Abhijeet Prasad (AbhiPrasad)
pushed a commit
that referenced
this pull request
Jul 17, 2026
…er serialization (#587) ## Summary Align the CrewAI integration with `.agents/skills/sdk-integrations/SKILL.md`. Audit turned up five issues: - **Missing `metadata.provider` on `crewai.llm` spans** — spec requires every `llm` span to carry both `metadata.model` and `metadata.provider`. Added `_provider_from_source()`, which reads the `provider` string off the emitting CrewAI `LLM` object (`"openai"` / `"anthropic"` / `"bedrock"` / ...). - **Tools in `input` instead of `metadata.tools`** — spec says tool definitions live in `metadata.tools`, not `input`. Moved both the LLM-call `tools` and the agent `tools` onto `metadata`. - **Token double-counting on `crewai.llm`** — the previous rule only skipped tokens when the LiteLLM integration was patched, but CrewAI 1.x routes `gpt-*` to a native `openai` client, `claude-*` to a native `anthropic` client, etc. When those integrations are patched (as `auto_instrument()` does by default), they emit a leaf span with tokens and `crewai.llm` double-counted at every ancestor in the trace-tree rollup. The SKILL is explicit here: "Do not add 'if OpenAI is patched, skip metrics' checks — define clear ownership instead." CrewAI is always an orchestration layer that delegates to a provider SDK, so the clear rule is: **`crewai.llm` never owns tokens**. Same pattern `pydantic_ai` already uses for its wrapper spans. Dropped `_litellm_owns_leaf_span` + the token-name/prefix maps + the `extra_metrics` plumbing on `_end_span` accordingly. - **Excess serialization** — dropped eager `_try_to_dict` / `_normalize_output` / `_normalize_tools` calls across kickoff, task, agent, LLM, and tool output paths. `bt_json._to_bt_safe` at log time already handles Pydantic v2 (`model_dump`), Pydantic v1 (`dict`), and dataclasses; the eager pre-pass was wasted work (same pattern as #585 for claude_agent_sdk and #583 for anthropic). All other metadata extractors (`_causal_metadata`, `_crew_metadata`, `_task_metadata`, `_LLM_CONFIG_FIELDS`) were already allowlist-based — no changes needed there. ## Test plan - [x] Extended `test_kickoff_llm_event_tree_parents_and_shape` to emit with a real `LLM` source and assert `metadata.provider == "openai"`. - [x] Added `test_llm_tools_route_to_metadata_not_input` (positive-in-metadata + negative-not-in-input). - [x] Replaced the two conditional token tests with a single `test_llm_never_emits_token_metrics` that asserts no token key (`tokens`, `prompt_tokens`, `completion_tokens`, `prompt_cached_tokens`, `completion_reasoning_tokens`) leaks onto `crewai.llm` regardless of which provider integrations are patched. - [x] **No new cassettes.** The test file docstring documents why VCR is impractical for CrewAI (pytest-vcr + httpcore + CrewAI's native `OpenAICompletion` `model_post_init` interaction bug). Direct-event tests remain the source of truth here; the LiteLLM `mock_response` smoke test still exercises the full `crew.kickoff()` path. - [x] `cd py && BRAINTRUST_TEST_PACKAGE_VERSION=latest pytest src/braintrust/integrations/crewai/test_crewai.py -v` — 17 passed - [x] `cd py && nox -s pylint` — success - [x] `pre-commit run` on the changed files — passed (ruff format, ruff check, codespell, EOF/trailing-whitespace) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- sfk:created-approved-by --> Created by abhijeet <!-- sfk:slack-thread --> [Slack thread](https://starfolkai.slack.com/archives/C0AQDETAVT3/p1784304223300019?thread_ts=1784304223.300019&cid=C0AQDETAVT3) --------- Co-authored-by: Starfolk <noreply@starfolk.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SpanImpl.log_internalalready runsbt_safe_deep_copyon every logged event (seelogger.py:4699), so the eager copies in_normalize_anthropic_input,_get_input_from_kwargs, and_redact_server_tool_outputwere duplicating work on every span open/log.bt_safe_deep_copycalls and dropped the now-unused import. The remaining walks (_normalize_anthropic_data,_process_input_attachments, inlined redaction) already build fresh dicts/lists and never mutate user data, so behavior is preserved.google_genai/tracing.pydoes today and matches thesdk-integrationsskill guidance: "Do not over-serialize. Braintrust serializes at send/log time."Test plan
pytest src/braintrust/integrations/anthropic/test_anthropic.pyon anthropic==0.116.0 (matrixlatest) — 48 passedpytest src/braintrust/integrations/anthropic/test_anthropic.pyon anthropic==0.48.0 — 39 passed, 9 legitimately skipped for features unavailable on that version🤖 Generated with Claude Code
Created by abhijeet
Slack thread