Skip to content

[Fix] Vertex Gemini requests fail when resuming model turns - #1289

Open
ghost wants to merge 2 commits into
mainfrom
fix/vertex-gemini-model-turn-06ixuqqflvtqk
Open

ghost wants to merge 2 commits into
mainfrom
fix/vertex-gemini-model-turn-06ixuqqflvtqk

Conversation

@ghost

@ghost ghost commented Aug 19, 2026 •

Copy link
Copy Markdown

​Opened on behalf of @navedmerchant. View the task or mention @roomote for follow-up asks.

What changed

Vertex Gemini requests now preserve assistant history while adding a minimal user continuation when converted conversation content would otherwise end with a model turn. Focused request-level tests cover the reported gemini-3.7-flash failure and verify that already-valid user-ending histories are unchanged.

Why this change was made

Vertex AI rejects Gemini requests that end with a model turn, causing resumed tasks to fail with Requests ending with a model turn are not supported. The direct Gemini provider tolerates this shape, so the compatibility handling is intentionally limited to Vertex.

Impact

Users can resume Gemini 3.7 tasks through Vertex AI without receiving the 400 invalid-argument error, while prior assistant and tool-call context remains intact.

Closes #1287

Related PRs

@codecov

codecov Bot commented Aug 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/api/providers/gemini.ts 90.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@navedmerchant
navedmerchant marked this pull request as ready for review August 19, 2026 15:28
@github-actions github-actions Bot added the awaiting-review PR changes are ready and waiting for maintainer re-review label Aug 19, 2026

@edelauna edelauna left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had a question on the logic for getting the model to continue / go on

Comment thread src/api/providers/gemini.ts Outdated
// Vertex rejects requests that end with a model turn, which can occur when
// resuming after an interrupted response. Preserve that turn and explicitly
// ask the model to continue rather than dropping conversation history.
if (this.isVertex && contents.at(-1)?.role === "model") {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guard keys only on role === "model", not the trailing part type -- so if the last model turn ends with an unanswered functionCall (interrupted mid-tool-call), appending a plain-text Continue. turn breaks Gemini's functionCall->functionResponse pairing and Vertex 400s again. Task.ts normalizes dangling tool_use on the UI resume path, so is a functionCall-ending turn still reachable here (condensation, cross-model import)? If so, detect a trailing functionCall and synthesize a functionResponse instead of text.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — yes, a functionCall-ending turn is still reachable here: Task.ts's normalization only covers the UI resume path, while condensed history and cross-model imports can hand this handler a conversation that ends in an unanswered tool call, so the request layer is the right place to close the gap.

Updated in f64b508: the guard now inspects the trailing model turn's parts instead of keying only on role === "model". If it contains unanswered functionCall parts, we append a user turn of synthesized functionResponse parts — one per call, name taken from the functionCall part itself, with the payload in the same { name, response: { name, content } } shape convertAnthropicContentToGemini produces for real tool results — instead of the plain-text Continue. turn. Model turns that end in plain text keep the existing Continue. behavior.

Added a test for the interrupted-mid-tool-call case that asserts the request contents pair the trailing functionCall (name read_file) with a synthesized functionResponse of the same name. All 10 vertex.spec.ts tests pass, along with the full pnpm test suite, ESLint, and tsc --noEmit.

@github-actions github-actions Bot added awaiting-author PR is waiting for the author to address requested changes and removed awaiting-review PR changes are ready and waiting for maintainer re-review labels Aug 19, 2026
@myk1yt

myk1yt commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

PR Review: [Fix] Vertex Gemini requests fail when resuming model turns

Thanks for this fix!

Summary & Validation

  • Root Cause Resolution: Resuming interrupted tasks or tasks where the last turn in history was an assistant message caused Vertex AI to reject requests with Requests ending with a model turn are not supported.
  • Implementation: Appending an explicit user continuation turn ({ role: "user", parts: [{ text: "Continue." }] }) when contents.at(-1)?.role === "model" cleanly satisfies the API requirement while preserving complete conversational history.
  • Test Coverage: The tests in vertex.spec.ts nicely verify both the continuation insertion for trailing model turns and the untouched behavior for already valid user-ending histories.

Suggestions / Observations

  • Trailing Function Call Handling: As noted by @edelauna, if the trailing model turn ends with an unanswered functionCall (e.g., interrupted mid-tool-execution), appending a plain text Continue. turn will break Gemini's required functionCall -> functionResponse pair contract. Detecting trailing functionCall parts and synthesizing a corresponding functionResponse avoids 400 validation failures on resume.
  • Rate Limit (429) & Error Preservation: In gemini.ts, routing stream and prompt generation errors through handleProviderError (from ./utils/error-handler) ensures HTTP status codes (such as 429 Quota Exceeded and 503 Service Unavailable) are preserved. This allows Zoo Code's exponential backoff and retry mechanism to handle rapid multi-turn tool bursts gracefully instead of abruptly failing.
  • Broader Gemini API Applicability: Google AI Studio Gemini API endpoints (direct Gemini provider) also benefit from trailing model turn continuation when resuming interrupted tasks. Generalizing the check beyond this.isVertex (or unifying it across GeminiHandler) ensures consistent resumption behavior across all Gemini backends.
  • Reasoning Model Resolution: For Vertex Gemini reasoning variants (e.g., gemini-3.7-flash:thinking), ensure the model identifier is normalized in VertexHandler.getModel() so reasoning parameters are mapped properly without triggering default model fallbacks.

LGTM! 🚀
📌 Reference Implementation: For the complete working fix, 429 error handling, and test cases, check out the fix/vertexgemini branch on my fork.

@github-actions

github-actions Bot commented Aug 29, 2026 •

Copy link
Copy Markdown
Contributor

Review status

Thanks for contributing. This comment tracks the review sequence and the next action.

Current step: Required CI passed. Waiting for automated review of the latest commit.

If automated review does not start, a maintainer must restart it.

Review-state labels are managed by this workflow; do not edit them manually. community-approved is managed the same way — do not add or remove it manually. It signals a fresh community code approval for the current head as an advisory priority only; maintainer review is still required.

@github-actions github-actions Bot added coderabbit-review-active Required CI passed; CodeRabbit review is active awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit awaiting-author PR is waiting for the author to address requested changes and removed awaiting-author PR is waiting for the author to address requested changes coderabbit-review-active Required CI passed; CodeRabbit review is active awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit labels Aug 29, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has been awaiting author changes for 14 days and will be automatically closed in 7 days. Please address the review comments or leave a comment if you need more time.

@github-actions github-actions Bot added the stale-awaiting-author PR is stale while waiting for requested author changes label Sep 18, 2026
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 0f890205-0fe6-470c-a2d8-afaef43715cb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot removed awaiting-author PR is waiting for the author to address requested changes stale-awaiting-author PR is stale while waiting for requested author changes labels Sep 22, 2026
@github-actions github-actions Bot added the awaiting-author PR is waiting for the author to address requested changes label Sep 22, 2026
@ghost
ghost requested a review from edelauna September 22, 2026 23:28
@github-actions github-actions Bot added coderabbit-review-active Required CI passed; CodeRabbit review is active awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit and removed awaiting-author PR is waiting for the author to address requested changes labels Sep 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has had no activity for 60 days and will be automatically closed in 7 days unless there is new activity. If you'd like to continue, please rebase and leave a comment.

@github-actions github-actions Bot added the stale-inactive PR closed due to 60+ days of inactivity label Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-coderabbit Waiting for CodeRabbit to approve the latest commit coderabbit-review-active Required CI passed; CodeRabbit review is active stale-inactive PR closed due to 60+ days of inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Gemini 3.7 Support in Vertex - Error

4 participants