[integrations][gemini] Apply Gemini native structured output - #1098
Open
weiqingy wants to merge 2 commits into
Open
[integrations][gemini] Apply Gemini native structured output#1098weiqingy wants to merge 2 commits into
weiqingy wants to merge 2 commits into
Conversation
GeminiChatModelConnection.buildConfig assembles the GenerateContentConfig for every Gemini request -- system instruction, sampling parameters, additional kwargs and tool declarations -- and had no test coverage. Widen it to package-private, matching the convention already used by five other members of the class, and add four tests. Three production writes were reachable but unasserted, so the tests assert the tool declaration's description and parametersJsonSchema, and the part count that detects a user turn leaking into the system instruction. The dispatch from buildConfig to applyAdditionalKwargs was previously uncovered: the existing applyAdditionalKwargs tests each call that method directly on a builder they construct. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
Report native structured-output capability and honour an output schema: a POJO class is rendered to a JSON Schema and sent as the SDK's responseJsonSchema, alongside responseMimeType application/json. A schema reaching this connection previously raised UnsupportedOperationException from the base class. The typed responseSchema field is not used. It routes through a Jackson round-trip into a Schema message that drops every keyword that message cannot model, on both the Developer API and Vertex paths, so a request would carry fewer constraints than the caller declared. Capability is reported for the gemini- family with seven non-text modality markers denied first, rather than an enumerated allowlist: support covers all actively supported models, but the image, speech, audio, Live, transcription and embedding variants share the family prefix and reject a schema. The schema is skipped when the request carries tools, because that combination is rejected outside a documented preview, and skipping keeps the prompt fallback rather than failing at the provider. Sibling keywords beside a $ref are removed from the derived schema: Gemini forbids them, and the generator emits them when a type is reused. The guard tests that $ref holds a string, so a property legitimately named $ref is not mistaken for a reference. Generated-by: Claude Code 2.1.259 (Claude Opus 5)
github-actions
Bot
added
doc-not-needed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #280
Stacked on #1096. The first commit here belongs to that PR; only the second is under review. I will rebase once it merges.
Purpose of change
The Gemini connection now supports native structured output. When a caller asks for a response matching a class, that class is turned into a JSON schema and sent to Gemini as a real schema parameter, so the model is constrained by the provider rather than by instructions in the prompt. Until now any output schema reaching this connection raised an error, because the connection declared no native support.
This is the sixth provider in the #280 series, after OpenAI, Azure OpenAI, Anthropic and Ollama. Gemini is Java-only because there is no Python Gemini connection, so nothing is missing on the Python side.
Three choices are worth calling out, since a reader would reasonably expect otherwise.
The schema is sent through the SDK's raw JSON-schema field, not its typed one. The typed field converts what it is given into an internal message type and silently discards anything that type cannot express. A rich schema can arrive at the service as little more than "this is an object". The raw field is passed through untouched. Sending fewer constraints than the caller declared, with no error, seemed worse than the alternative.
Support is decided by model family, not by a list of model names. Google states that JSON schema support covers all actively supported Gemini models, and new Flash models appear every few weeks, so a fixed list would refuse models that work. Support does still depend on the model: the image, speech, audio, Live, transcription and embedding variants share the same name prefix and reject a schema, so those are excluded by name first.
The schema is skipped when the request also carries tools. Gemini rejects that combination outright except on a small set of preview models, and the two documentation pages disagree about which. Rather than risk a failed request, the connection falls back to the existing prompt-based approach for that case. This differs from the OpenAI and Azure connections, which always send the schema. That is deliberate: OpenAI documents the combination as generally available, Gemini publishes it behind a preview warning.
One limit worth stating plainly. Gemini supports a subset of JSON schema and ignores the rest without reporting it. So a response will follow the schema's shape, but a constraint such as a string pattern may be accepted and then have no effect. Values still need validating in application code.
Tests
Fourteen unit tests added, covering which models report support, that a schema reaches the request, that a non-class schema is skipped rather than rejected, that no schema is sent when tools are present, and that the generated schema honours the usual Jackson annotations. Module suite is at 65, no failures.
Every behaviour these tests describe was checked by breaking the production code and confirming a test fails, including the guards that make the two skip cases work. That is how three problems were found during review that a passing suite did not show.
Not tested: anything requiring a live call. There is no API key in CI and no live structured-output test exists for any provider in this repository, so this does not add the first one. Everything about the service's behaviour here comes from Google's documentation, not from observation.
API
No public API change. The connection implements two methods the base class already defines, which every other native-capable provider also implements. Callers passing no schema are unaffected.
Adds the victools JSON schema generator to this module, the same library and version the Ollama connection already uses, with the version managed centrally. The distribution's NOTICE already lists it at that version, so no licensing metadata changes.
Documentation
doc-neededdoc-not-neededdoc-includedWas this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code 2.1.259 (Claude Opus 5)