Repository navigation
fix(gateway): report a failed upstream body read instead of a fabricated 200 - #211
Merged
Merged
Conversation
…ted 200 `forward` read the upstream response body with `resp.bytes().await.unwrap_or_default()`, which collapses a body-read failure into an empty vector, and then let the status line alone decide the outcome. A 2xx whose body never arrived was therefore handed to the client as `200 OK` + `content-type: application/json` with **0 bytes** - not a response any client can parse. Measured against a scripted upstream that claims `content-length: 999` and sends 47 bytes before closing: `status=200 body_len=0`. The damage is not just the empty body: `parse_usage` sees no JSON and returns `(0, 0, 0)`, `settle_usage` returns early (its contract is "only settle when tokens > 0"), so the call leaves no `usage_records` row and no `transactions` row, the balance stays unchanged and the key is not cooled. The client gets a fabricated success while the call disappears from both the ledger and the operational counter (`month_calls = COUNT(*) FROM usage_records`) - even though the upstream really spent tokens. This is the same rule the tree already states elsewhere: missing data is not the value zero. The failure is reachable: the non-stream client deliberately keeps a **total** 120 s deadline (`UPSTREAM_TIMEOUT`), so any non-stream generation whose body outlives it lands here, as does a mid-body connection reset or a proxy cutting a large response. The streaming path already handles the identical situation the opposite way - a truncated body reaches the client as an error and is not settled (see `sse_slow_stream_is_cut_by_a_total_deadline`) - and `passthrough` exists to forward an upstream body, not to invent one. Keep the status-based dispatch exactly where it is, but stop discarding the error: log it, and when the status was 2xx answer **502** instead of a fabricated empty 200. A non-2xx keeps today's behaviour byte-for-byte (empty body, then the existing 401/403/429/5xx failover or the 4xx passthrough), so a 5xx whose body is unreadable still switches keys. A 2xx is deliberately **not** retried: the upstream may already have done the work, and retrying could charge the key owner twice. Who pays is unchanged (nobody, before and after) - the point is to stop reporting a failure as a success, and deliberately not to estimate tokens. Tests (2 new, 207 -> 209): a raw scripted TCP upstream (one pre-built HTTP response per connection, `connection: close`) is required here, because the existing axum fakes can only emit well-formed bodies and therefore cannot express "200 whose body never arrives". - `forward_body_read_failure_is_not_a_success` carries its own positive control: a complete 200 + usage must still answer 200, bill exactly once and spend one upstream request; the truncated 200 must answer 502 with no `usage_records` row, an unchanged balance and still exactly one upstream request (no failover on a 2xx). - `unreadable_body_on_a_server_error_still_fails_over` pins the unchanged non-2xx path: a 500 with an unreadable body still fails over, and with every key failed the answer is still 503 with both keys cooled. A/B: reverting only the new error arm (back to `unwrap_or_default()`) leaves exactly `forward_body_read_failure_is_not_a_success` red with `left: 200, right: 502`; relaxing the new arm's condition to every status leaves exactly `unreadable_body_on_a_server_error_still_fails_over` red with `left: 502, right: 503`. Each mutation was applied alone and restored byte-identically.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
forwardread the upstream response body withresp.bytes().await.unwrap_or_default(), which collapses a body-read failure into an empty vector, and then let the status line alone decide the outcome. A 2xx whose body never arrived was handed to the client as200 OK+content-type: application/jsonwith 0 bytes — not a response any client can parse. Measured against a scripted upstream that claimscontent-length: 999and sends 47 bytes before closing:status=200 body_len=0.The damage is not only the empty body.
parse_usagesees no JSON and returns(0, 0, 0);settle_usagereturns early by its own contract ("only settle whentokens > 0"), so the call leaves nousage_recordsrow and notransactionsrow, the balance stays unchanged and the key is not cooled. The client gets a fabricated success while the call disappears from both the ledger and the operational counter (month_calls = COUNT(*) FROM usage_records) — even though the upstream really spent tokens. That is the same rule the tree states elsewhere: missing data is not the value zero.Reachability: the non-stream client deliberately keeps a total 120 s deadline (
UPSTREAM_TIMEOUT), so any non-stream generation whose body outlives it lands here, as does a mid-body connection reset or a proxy cutting a large response. The streaming path already handles the identical situation the opposite way — a truncated body reaches the client as an error and is not settled (sse_slow_stream_is_cut_by_a_total_deadline) — andpassthroughexists to forward an upstream body, not to invent one.Related Issue
Changes
src/gateway.rs::forwardkeeps the status-based dispatch exactly where it is but stops discarding the read error: it is logged, and when the status was 2xx the response is 502 instead of a fabricated empty200.ui/change; no new i18n keys.Tests
cargo test— 209 passed (207 before, +2 new).cargo fmt --checkclean;cargo clippy --all-targets -- -D warningsclean.A raw scripted TCP upstream (one pre-built HTTP response per connection,
connection: close) is required here: the existing axum fakes can only emit well-formed bodies, so they cannot express "200 whose body never arrives".forward_body_read_failure_is_not_a_success— carries its own positive control: a complete200+ usage must still answer200, bill exactly once and spend one upstream request; the truncated200must answer502with nousage_recordsrow, an unchanged balance and still exactly one upstream request (no failover on a 2xx).unreadable_body_on_a_server_error_still_fails_over— pins the unchanged non-2xx path: a500with an unreadable body still fails over; with every key failed the answer is still503with both keys cooled and nothing settled.A/B teeth proof (each mutation applied alone, restored byte-identically): reverting only the new error arm (back to
unwrap_or_default()) leaves exactlyforward_body_read_failure_is_not_a_successred, and the failure output is the defect —left: 200, right: 502; relaxing the new arm's condition to every status leaves exactlyunreadable_body_on_a_server_error_still_fails_overred —left: 502, right: 503.Checklist
fix/)fix(gateway): ...)