[bot] AWS Bedrock Runtime: embedding models via `invoke_model` (Titan/Cohere Embed) get no embedding-specific instrumentation

[braintrust-bot]

<!-- provider-gap-audit: bedrock-invoke-model-embeddings-not-instrumented -->

Summary

The Bedrock Runtime integration (py/src/braintrust/integrations/bedrock_runtime/) instruments converse, converse_stream, invoke_model, and invoke_model_with_response_stream, but invoke_model has no embedding-specific handling. When invoke_model is called against an embedding model hosted on Bedrock (e.g. Amazon Titan Text Embeddings G1/V2, or Cohere Embed on Bedrock), the response is logged as a generic raw JSON blob with no embedding_length, no embedding-specific token/usage metrics, and no truncation of the raw vector — a materially lower level of detail than every other embeddings-capable integration in this repo.

What is missing

_invoke_model_wrapper (py/src/braintrust/integrations/bedrock_runtime/tracing.py:82-110) always:

1. Parses the raw JSON response body and logs it verbatim as output (tracing.py:96-102).
2. Computes metrics via _converse_usage_metrics(output_usage) (tracing.py:106), which looks for usage.inputTokens/usage.outputTokens — fields that do not exist on Titan/Cohere embedding responses — so this always returns {} for embedding calls.
3. Computes _anthropic_invoke_usage_metrics(model_id, output) (tracing.py:107), which is explicitly gated to "anthropic.claude" in model_id.lower() (tracing.py:463-465) and returns {} for any non-Claude model, including embedding models.

Net effect for an embedding invoke_model call: no usage metrics of any kind, and the full embedding vector (e.g. 1024 floats for Titan V2, or embeddingsByType for both float and binary variants) is logged directly as the raw span output instead of being summarized.

Comparison with other providers in this repo

Integration	Embedding metrics extracted	Location
OpenAI	embedding_length	integrations/openai/tracing.py:1460-1472
LiteLLM	embedding_length	integrations/litellm/tracing.py:409-442
Google GenAI	embedding_length, embeddings_count, per-embedding token_count, billable_characters	integrations/google_genai/tracing.py:357-395
Cohere	embedding_length/count	integrations/cohere/tracing.py:292-322
Mistral	embeddings coverage (sync/async)	integrations/mistral/tracing.py
HuggingFace Hub	feature_extraction/sentence_similarity handling	integrations/huggingface_hub/tracing.py
OpenRouter	embeddings coverage	integrations/openrouter/tracing.py:92-107,185-192,516-543
Bedrock Runtime	None	integrations/bedrock_runtime/tracing.py:82-110

Bedrock is the only embeddings-capable provider integration in this repo with zero embedding-aware output/metric handling.

Upstream API details

• Endpoint: bedrock-runtime InvokeModel — same API surface already instrumented for chat models, just a different modelId (e.g. amazon.titan-embed-text-v2:0, amazon.titan-embed-text-v1, cohere.embed-english-v3).
• Titan Embeddings response body (both G1 and V2): {"embedding": [float, ...], "inputTextTokenCount": int}; V2 additionally returns embeddingsByType: {"float": [...], "binary": [...]}.
• AWS docs: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-embed-text.html (confirms exact response fields embedding and inputTextTokenCount, invoked via InvokeModel: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html)
• Cohere Embed on Bedrock returns {"embeddings": [[float,...],...], "id": ..., "response_type": "embeddings_floats", "texts": [...]} via the same invoke_model call.

What should be instrumented

For invoke_model calls where the parsed response body contains an embedding/embeddingsByType (Titan) or embeddings (Cohere) field instead of output.message/stopReason:

Span field	Content
output	embedding_length (vector dimensionality) instead of the raw vector, matching the pattern used by every other provider integration
metrics	prompt_tokens from inputTextTokenCount (Titan) or equivalent Cohere usage field
metadata	provider: "bedrock", model, endpoint: "invoke_model" (already present)

Braintrust docs status

not_found. The AWS Bedrock integration page (https://www.braintrust.dev/docs/integrations/ai-providers/bedrock) states that token usage is captured "for InvokeModel calls to Anthropic Claude models" — explicitly scoping usage capture to Claude and not mentioning embedding models, Titan Embeddings, or Cohere Embed at all.

Upstream sources

• AWS Titan Text Embeddings request/response reference: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-embed-text.html
• Bedrock Runtime InvokeModel API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html
• py/pyproject.toml [tool.braintrust.matrix.boto3]: latest = "boto3==1.43.46", floor "1.34.116" = "boto3==1.34.116" (both include invoke_model)

Local repo files inspected

• py/src/braintrust/integrations/bedrock_runtime/tracing.py:82-110 — _invoke_model_wrapper, no embedding-shape detection
• py/src/braintrust/integrations/bedrock_runtime/tracing.py:367-386 — _converse_usage_metrics (expects inputTokens/outputTokens, absent from embedding responses)
• py/src/braintrust/integrations/bedrock_runtime/tracing.py:463-485 — _anthropic_invoke_usage_metrics (gated to "anthropic.claude" model IDs only)
• py/src/braintrust/integrations/openai/tracing.py:1460-1472, integrations/litellm/tracing.py:409-442, integrations/google_genai/tracing.py:357-395, integrations/cohere/tracing.py:292-322 — comparable embedding metric extraction in other providers
• py/pyproject.toml — [tool.braintrust.matrix.boto3] matrix pins