[integrations][bedrock] Apply Bedrock native structured output - #1097
Open
weiqingy wants to merge 3 commits into
Open
[integrations][bedrock] Apply Bedrock native structured output#1097weiqingy wants to merge 3 commits into
weiqingy wants to merge 3 commits into
Conversation
The Bedrock connection built its ConverseRequest inline in chat(), and every request-building helper was private, so nothing could assert what the connection actually sends. Move the construction into a package-private buildRequest() and cover it. The extraction is behavior-preserving: the moved block is unchanged, and resolveModel() still runs first so a missing model still fails before anything else happens. Five tests, none of which had an equivalent before: model-id resolution across both the configured default and a per-call override, tool config, the system/conversation split, inference config, and message merging. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
Converse exposes outputConfig, its native structured-output surface, from 2.41.22 onward; 2.41.21 does not have it. Pin the minimum version that carries the surface rather than the latest, to keep the change reviewable. The property is shared, so this also moves the Bedrock embedding model, the OpenSearch vector store and the S3 Vectors vector store. All four modules compile and test unchanged, with no source edits. NOTICE follows the resulting set: 27 AWS entries and 10 netty entries, which move to 4.1.130.Final transitively through netty-nio-client, plus a new entry for utils-lite. Nothing leaves the set. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
Bedrock inherited the base 4-arg chat(), which rejects any non-null output schema, so it was the last Java chat model with no native path. Wire Converse's outputConfig so the provider constrains the response, falling back to prompt engineering when the schema, the derivation or the model cannot support it. Capability is an exact match against the model ids AWS documents as supporting structured output, retried once after stripping a leading inference-profile segment. Support is per model rather than per family, so a prefix match would claim a capability the provider denies for one Qwen model while granting it for eight siblings. Anything unrecognised, ARNs included, answers false and takes the fallback: an ARN carries no model information, and a prompt router does not choose its model until the request runs. The AWS SDK ships no schema generator, so the schema is derived with victools and serialised, JsonSchemaDefinition taking a string rather than a JSON value. JacksonModule keeps @JsonProperty names and drops @JsonIgnore fields; without a required check victools emits no required key at all, which would let an empty document satisfy every schema. additionalProperties is never emitted with a non-false value because Bedrock rejects that, which leaves a map's values undescribed. No collision guard: this connection reads only model, temperature and max_tokens, so there is no channel for a caller to have set outputConfig already. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
github-actions
Bot
added
doc-not-needed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #280
Purpose of change
Bedrock was the last Java chat model with no native structured-output path. Asking it for a schema did not fall back quietly, it threw, because the connection never overrode the foundation's hook. This wires Converse's
outputConfigso the provider constrains the response, with the existing prompt-engineered path still handling everything else.Bedrock has no Python connection in this repo, so this is Java-only.
The feature needed an SDK bump.
outputConfigappears onConverseRequestatbedrockruntime2.41.22 and does not exist at 2.41.21, so the pin moves to the first version that has it rather than to the latest. That property is shared with three other modules, so the bump moves them too. They compile and test unchanged, andNOTICEfollows the resulting dependency set.It is three commits: a refactor that adds a seam for testing the request, the bump on its own, then the feature. Each is meant to be readable without the other two.
Two decisions are worth a reviewer's attention.
Which models get the native path. Support on Bedrock is documented per model, not per family, so the check is an exact match against the ids AWS lists, retried once after stripping a leading inference-profile segment such as
us.oreu.. A family-prefix match would be wrong in a way that matters: AWS documents one Qwen model as unsupported while eight of its siblings are supported. Anything unrecognised takes the prompt fallback, and that includes ARNs, which carry no model information at all, and prompt routers, which do not pick a model until the request runs. Guessing wrong in that direction costs a fallback; guessing wrong in the other costs a runtime error.How the schema is built. The AWS SDK has no schema generator, unlike the OpenAI and Anthropic ones, so the schema is derived locally the way the Ollama connection does it. A couple of constraints there are easy to get wrong and are covered by tests: without Jackson awareness the generator ignores
@JsonPropertynames and emits@JsonIgnorefields, and without a required check it marks nothing required, which would let an empty document satisfy any schema.Two limitations to be aware of rather than surprised by. Recursive types cannot be used as output schemas, since the generated self-reference is something Bedrock rejects up front. And Bedrock warns that the first request against a new schema can spend up to a few minutes compiling it, which here happens inside an operator, so the first record through a new schema may be far slower than the rest.
Tests
The Bedrock chat-model module goes from 12 tests to 54, all offline.
Most of that is coverage this module simply never had. Nothing previously asserted what the connection actually sends, so tools, system messages, inference config and message merging are all tested for the first time. The rest covers the new behaviour: which model ids are accepted and rejected, that the derived schema keeps Jackson property names and marks fields required, and that the native path engages and disengages when it should. Tools and a schema on the same request are also covered, since Bedrock allows both.
The three other modules affected by the SDK bump run unchanged, and the full build was run so the packaging step is exercised, since a dependency move is exactly what can break it.
What is not covered: none of this calls Bedrock. Those tests need credentials CI does not have, so the request shape is verified and the service's acceptance of it is not. I am verifying that separately against a real endpoint and will report the results here before this is merged. The open questions are whether
outputConfigis accepted as built, which schema details Bedrock tolerates, and whether an unsupported model errors or quietly ignores the request.API
No public API is added or changed, and no new configuration. The visible difference is that a Bedrock connection given an output schema on a supported model now returns a schema-constrained response instead of throwing.
Documentation
doc-neededdoc-not-neededdoc-includedNothing user-facing changes in configuration, and the repo has no structured-output docs page for any provider today.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code 2.1.259 (Claude Opus 5)