Repository navigation
fix(model-gateway): terminate downstream streams when passthrough or Grok upstreams close mid-response - #820
Merged
Eigenwise merged 2 commits intoNov 12, 2026
Conversation
chris7iu
force-pushed
the
fix/model-gateway-passthrough-stream-abort
branch
from
November 12, 2026 07:50
4fcf54a to
2ea24f8
Compare
…Grok upstreams close mid-response
chris7iu
force-pushed
the
fix/model-gateway-passthrough-stream-abort
branch
from
November 12, 2026 07:58
2ea24f8 to
12964f6
Compare
Owner
|
Merged, thanks. The fix reads right: with an error listener already on the upstream response, nothing told the client side that the body was cut short, so the stream just hung. Destroying the downstream response on error, abort or an incomplete close covers the passthrough and Grok paths, and the two-hop test proves it through the shim too. It goes out with the next model-gateway release. The red Windows check on the first run was ours, not yours: a gateway lock test that times out on a slow runner. It's filed and the rerun passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
upRes.pipe(clientRes)) inrequest-worker.js:claude-sonnet-4-6,claude-opus-4-7),request-worker.jsattachesupRes.once('error', () => routeTelemetry?.finish(upRes.statusCode, 'upstream_error'))before pipingupRes.pipe(clientRes). BecauseupResalready has an'error'listener, Node'shttpclient does not re-emit mid-body chunked framing/socket errors (HPE_INVALID_CHUNK_SIZE, premature socket close) onupReq, andReadable.prototype.pipe()forwards neither'error'nor'aborted'toclientRes. When an upstream aborts mid-stream after flushingHTTP 200headers,clientRespreviously hung open indefinitely instead of propagatingUND_ERR_SOCKETtoserve-shim/ Claude Code'swithRetry.terminateDownstreamonupRes'error','aborted', and non-complete'close'(!upRes.complete) soclientResis destroyed immediately when headers are already sent (or answered with502 Bad Gatewayif headers were not yet sent).forwardGrok:terminateGrokDownstreamon GrokupRes'error','aborted', and non-complete'close'so truncated Grok SSE responses after200headers also destroyclientResrather than stalling or finalizing cleanly.withRetryacross the two-hopserve-shim->serve-worker-> upstream path.Test Plan
node --require ./plugins/model-gateway/test/isolate-home.js --test plugins/model-gateway/test/relay-stream-abort.test.js plugins/model-gateway/test/request-worker.test.js plugins/model-gateway/test/grok-backend.test.jsrelay-stream-abort.test.jscovering bothserve-workerand two-hopserve-shimpropagation when a passthrough Anthropic SSE upstream (claude-sonnet-4-6) or a Grok SSE upstream (grok-4.20) closes mid-chunk after flushing200headers.node scripts/release/guard.mjs --mode dev --publish-ref origin/main --default-branch main --integration-branch develop ...