Skip to content

fix(adk): fix llm span metadata (provider, tools, errors)#580

Merged
Abhijeet Prasad (AbhiPrasad) merged 11 commits into
mainfrom
adk/skill-compliance-fixes
Jul 16, 2026
Merged

fix(adk): fix llm span metadata (provider, tools, errors)#580
Abhijeet Prasad (AbhiPrasad) merged 11 commits into
mainfrom
adk/skill-compliance-fixes

Conversation

@starfolkai

@starfolkai starfolkai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Brings the Google ADK integration in line with .agents/skills/sdk-integrations/SKILL.md and cuts the test suite over to real VCR-backed coverage.

Integration behavior (tracing.py)

  • metadata.provider="google" and metadata.tools on every llm span. Tool definitions are lifted out of input.config into metadata.tools, preserving Google-native shape.
  • _capture_config is now a strict allowlist — no more dict(config) pass-through. Fields outside the allowlist (including tools/tool_config) are dropped from input.config automatically. _extract_tool_metadata is the single source of truth for what goes into metadata.tools.
  • error=e instead of error=str(e) in the tool and MCP-tool wrappers — the Exception instance flows through to the log pipeline.
  • Trim raw ADK objects from span metadata. invocation_context, model_response_event, parent_context, and the **kwargs spread across every wrapper are dropped; only small named fields (app_name, user_id, session_id, state_delta, agent_name, flow_class, tool_class, model, provider, tools) remain.
  • Drop _get_field and bt_safe_deep_copy(tools). Production ADK always passes Pydantic objects, so plain getattr(x, field, default) is enough. The log pipeline serializes Pydantic tool objects at send time (SKILL: "do not over-serialize").

Tests

  • test_adk_braintrust_integration now filters spans by context.span_origin.instrumentation.name == "adk-auto", asserts {task, llm, tool} types are covered, and validates metadata.provider, metadata.tools, and no tool leakage into input.config.
  • test_adk_captures_metrics asserts metadata.provider="google" and the direct_response call-type span name.
  • test_adk_max_tokens_captures_content now also asserts the sampling config (max_output_tokens, temperature) is captured in the LLM span input.
  • test_adk_sync_runner_run_does_not_duplicate_invocation_spans now also asserts thread-bridge context propagation — every adk-auto span shares the outer parent's root_span_id.
  • Delete test_adk_mcp_tool.py entirely (the two remaining patcher-wiring tests were duplicative of the auto-instrument subprocess test) and drop its line from noxfile.py.
  • Delete fake-LLM, mock, and non-ADK unit tests — 15 tests removed. Every branch they covered (_determine_llm_call_type, _capture_config allowlist, _serialize_content/_serialize_part, _serialize_pydantic_schema, bt_safe_deep_copy, sampling config, thread bridge, PDF attachment, MCP-in-flow parenting, async context) is exercised by an existing VCR test.

Final test count: 13 tests, 12 VCR-backed + one plain-Python regression for the _create_thread_wrapper double-invoke bug.

Test plan

  • nox -s "test_google_adk(latest)" — 13 passed
  • nox -s "test_google_adk(1.14.1)" — 13 passed
  • No cassettes re-recorded (one orphaned cassette, test_adk_agent_metadata_with_attachment.yaml, removed on both matrix versions)

🤖 Generated with Claude Code

Created by abhijeet

Slack thread

starfolkbot and others added 3 commits July 16, 2026 15:28
- Add metadata.provider="google" and metadata.tools on llm spans; lift
  tools/tool_config out of input.config via _capture_config(exclude=...)
- Pass Exception instance to span.log(error=...) instead of str(e) in the
  tool and mcp_tool wrappers
- Drop raw ADK objects (invocation_context, model_response_event,
  parent_context, kwargs spread) from span metadata; keep only the small
  named fields that are actually useful
- Assert span_origin coverage across task/llm/tool types and validate
  metadata.provider / metadata.tools in the existing VCR-backed tests
- Prune mock-heavy MCP tests; real behavior is covered end-to-end by the
  VCR suite in test_adk.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The two remaining tests only asserted patcher wiring (marker set,
setup_adk returns True). Patcher idempotence is enforced by the base
patcher and covered indirectly through the auto-instrument subprocess
test; real MCP tool span behavior is covered end-to-end by the
VCR-backed suite in test_adk.py.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…y on tools

- _capture_config no longer seeds from dict(config); only fields in
  _CAPTURED_CONFIG_FIELDS are returned. Removes the pass-through
  behavior that let tools/tool_config leak into input.config (previously
  patched with an `exclude=` bandaid).
- _extract_tool_metadata no longer wraps tools/tool_config in
  bt_safe_deep_copy — the log pipeline serializes Pydantic objects at
  send time (SKILL: "do not over-serialize").
- Update the two _capture_config unit tests to reflect the strict
  allowlist behavior.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@starfolkai starfolkai Bot changed the title fix(adk): align integration with sdk-integrations SKILL fix(adk): fix llm span metadata (provider, tools, errors) Jul 16, 2026
starfolkbot and others added 6 commits July 16, 2026 15:36
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Production ADK always passes Pydantic objects (LlmRequest, Content,
Event, Part, GenerateContentConfig) — the dict-vs-attribute duality
existed only for unit tests. Rewrite the _determine_llm_call_type and
_capture_config unit tests to build fixtures with SimpleNamespace via
a small _ns() helper so getattr works uniformly. Also drop the now-
unused Mapping import.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 6 test_determine_llm_call_type_* and 2 test_capture_config_* tests
built SimpleNamespace fixtures to exercise internal helpers. Every path
they covered is already asserted through real ADK calls in the
VCR-backed suite:

- tool_selection / response_generation: test_adk_braintrust_integration
- direct_response: test_adk_captures_metrics (now asserts the name)
- response_schema Pydantic: test_adk_structured_output_pydantic
- response_schema nested: test_adk_complex_nested_schema
- input_schema: test_adk_input_schema_serialization
- response_json_schema dict: test_adk_response_json_schema_dict

Also drops the _ns() helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Removes tests that used fake LlmRegistry entries, MagicMock, or exercised
code that isn't ADK-specific:

- test_adk_thread_context_propagation: folded into
  test_adk_sync_runner_run_does_not_duplicate_invocation_spans by
  wrapping runner.run() in an outer parent span and asserting every
  adk-auto span shares its root_span_id.
- test_adk_generation_config_is_logged: folded into
  test_adk_max_tokens_captures_content (asserts max_output_tokens and
  temperature against the real logged span input).
- test_adk_document_inline_data_attachment_conversion: same code path as
  test_adk_binary_data_attachment_conversion (VCR-backed image).
- test_llm_call_span_wraps_child_spans (MagicMock Flow), and the
  test_serialize_*, test_bt_safe_deep_copy_*, test_serialize_pydantic_
  schema_direct, test_capture_config_preserves_none,
  test_adk_agent_metadata_with_attachment (patched Attachment.upload),
  test_adk_bytes_and_attachment_in_structure, and
  test_async_context_preservation_across_yields tests: covered by the
  VCR suite or not ADK-specific.

The one non-VCR test kept is
test_create_thread_wrapper_exception_does_not_double_invoke_target -
a plain-Python regression test for a wrapper bug (no mocks, no HTTP).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
starfolkbot and others added 2 commits July 16, 2026 16:10
pydantic_ai_wrap_openai(0.1.9) shard 3 failure looks unrelated to this
PR — pydantic_ai 0.1.9 imports opentelemetry._events (added in
opentelemetry-api 1.30.0) but the session resolved an older api. Same
session passed on main at c056762 (this branch's base) an hour before.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
pydantic-ai 0.1.9 unconditionally imports
``from opentelemetry._events import Event``. The ``_events`` module was
removed from ``opentelemetry-api`` in 1.40+, so once opentelemetry-api
1.44.0 landed on PyPI the fresh resolution in the
test_pydantic_ai_wrap_openai(0.1.9) session picked a version that fails
at import time.

Verified locally: opentelemetry-api 1.39.1 exposes _events, 1.44.0 does
not. Constraint only applies to the 0.1.9 install; newer matrix
versions (1.0.1, 2.9.0) remain unpinned.

Unrelated to this PR's ADK work — same session passed on main at
c056762 (this branch's base) before the opentelemetry release.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) merged commit 5c93bd7 into main Jul 16, 2026
82 checks passed
@AbhiPrasad
Abhijeet Prasad (AbhiPrasad) deleted the adk/skill-compliance-fixes branch July 16, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants