Skip to content

fix(gateway): report a failed upstream body read instead of a fabricated 200 - #211

Merged
argszero merged 1 commit into
mainfrom
fix/forward-body-read-error
Sep 13, 2026
Merged

argszero merged 1 commit into
mainfrom
fix/forward-body-read-error

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

forward read the upstream response body with resp.bytes().await.unwrap_or_default(), which collapses a body-read failure into an empty vector, and then let the status line alone decide the outcome. A 2xx whose body never arrived was handed to the client as 200 OK + content-type: application/json with 0 bytes — not a response any client can parse. Measured against a scripted upstream that claims content-length: 999 and sends 47 bytes before closing: status=200 body_len=0.

The damage is not only the empty body. parse_usage sees no JSON and returns (0, 0, 0); settle_usage returns early by its own contract ("only settle when tokens > 0"), so the call leaves no usage_records row and no transactions row, the balance stays unchanged and the key is not cooled. The client gets a fabricated success while the call disappears from both the ledger and the operational counter (month_calls = COUNT(*) FROM usage_records) — even though the upstream really spent tokens. That is the same rule the tree states elsewhere: missing data is not the value zero.

Reachability: the non-stream client deliberately keeps a total 120 s deadline (UPSTREAM_TIMEOUT), so any non-stream generation whose body outlives it lands here, as does a mid-body connection reset or a proxy cutting a large response. The streaming path already handles the identical situation the opposite way — a truncated body reaches the client as an error and is not settled (sse_slow_stream_is_cut_by_a_total_deadline) — and passthrough exists to forward an upstream body, not to invent one.

Related Issue

Changes

  • src/gateway.rs::forward keeps the status-based dispatch exactly where it is but stops discarding the read error: it is logged, and when the status was 2xx the response is 502 instead of a fabricated empty 200.
  • A non-2xx keeps today's behaviour byte-for-byte (empty body, then the existing 401/403/429/5xx failover or the 4xx passthrough) — a 5xx whose body is unreadable still switches keys.
  • A 2xx is deliberately not retried: the upstream may already have done the work, and retrying could charge the key owner twice. Who pays is unchanged (nobody, before and after); tokens are deliberately not estimated.
  • No config / schema / ui/ change; no new i18n keys.

Tests

  • cargo test — 209 passed (207 before, +2 new).
  • cargo fmt --check clean; cargo clippy --all-targets -- -D warnings clean.
  • New tests added.

A raw scripted TCP upstream (one pre-built HTTP response per connection, connection: close) is required here: the existing axum fakes can only emit well-formed bodies, so they cannot express "200 whose body never arrives".

  • forward_body_read_failure_is_not_a_success — carries its own positive control: a complete 200 + usage must still answer 200, bill exactly once and spend one upstream request; the truncated 200 must answer 502 with no usage_records row, an unchanged balance and still exactly one upstream request (no failover on a 2xx).
  • unreadable_body_on_a_server_error_still_fails_over — pins the unchanged non-2xx path: a 500 with an unreadable body still fails over; with every key failed the answer is still 503 with both keys cooled and nothing settled.

A/B teeth proof (each mutation applied alone, restored byte-identically): reverting only the new error arm (back to unwrap_or_default()) leaves exactly forward_body_read_failure_is_not_a_success red, and the failure output is the defect — left: 200, right: 502; relaxing the new arm's condition to every status leaves exactly unreadable_body_on_a_server_error_still_fails_over red — left: 502, right: 503.

Checklist

  • Branch name follows the convention (fix/)
  • Commit message uses Conventional Commits (fix(gateway): ...)
  • Single responsibility, minimal change

…ted 200

`forward` read the upstream response body with `resp.bytes().await.unwrap_or_default()`,
which collapses a body-read failure into an empty vector, and then let the status line
alone decide the outcome. A 2xx whose body never arrived was therefore handed to the
client as `200 OK` + `content-type: application/json` with **0 bytes** - not a response
any client can parse. Measured against a scripted upstream that claims
`content-length: 999` and sends 47 bytes before closing: `status=200 body_len=0`.

The damage is not just the empty body: `parse_usage` sees no JSON and returns
`(0, 0, 0)`, `settle_usage` returns early (its contract is "only settle when
tokens > 0"), so the call leaves no `usage_records` row and no `transactions` row,
the balance stays unchanged and the key is not cooled. The client gets a fabricated
success while the call disappears from both the ledger and the operational counter
(`month_calls = COUNT(*) FROM usage_records`) - even though the upstream really spent
tokens. This is the same rule the tree already states elsewhere: missing data is not
the value zero.

The failure is reachable: the non-stream client deliberately keeps a **total** 120 s
deadline (`UPSTREAM_TIMEOUT`), so any non-stream generation whose body outlives it
lands here, as does a mid-body connection reset or a proxy cutting a large response.
The streaming path already handles the identical situation the opposite way - a
truncated body reaches the client as an error and is not settled (see
`sse_slow_stream_is_cut_by_a_total_deadline`) - and `passthrough` exists to forward an
upstream body, not to invent one.

Keep the status-based dispatch exactly where it is, but stop discarding the error:
log it, and when the status was 2xx answer **502** instead of a fabricated empty 200.
A non-2xx keeps today's behaviour byte-for-byte (empty body, then the existing
401/403/429/5xx failover or the 4xx passthrough), so a 5xx whose body is unreadable
still switches keys.

A 2xx is deliberately **not** retried: the upstream may already have done the work, and
retrying could charge the key owner twice. Who pays is unchanged (nobody, before and
after) - the point is to stop reporting a failure as a success, and deliberately not to
estimate tokens.

Tests (2 new, 207 -> 209): a raw scripted TCP upstream (one pre-built HTTP response per
connection, `connection: close`) is required here, because the existing axum fakes can
only emit well-formed bodies and therefore cannot express "200 whose body never
arrives".

- `forward_body_read_failure_is_not_a_success` carries its own positive control: a
  complete 200 + usage must still answer 200, bill exactly once and spend one upstream
  request; the truncated 200 must answer 502 with no `usage_records` row, an unchanged
  balance and still exactly one upstream request (no failover on a 2xx).
- `unreadable_body_on_a_server_error_still_fails_over` pins the unchanged non-2xx
  path: a 500 with an unreadable body still fails over, and with every key failed the
  answer is still 503 with both keys cooled.

A/B: reverting only the new error arm (back to `unwrap_or_default()`) leaves exactly
`forward_body_read_failure_is_not_a_success` red with `left: 200, right: 502`; relaxing
the new arm's condition to every status leaves exactly
`unreadable_body_on_a_server_error_still_fails_over` red with `left: 502, right: 503`.
Each mutation was applied alone and restored byte-identically.
@argszero
argszero merged commit e885cd6 into main Sep 13, 2026
1 check passed
@argszero
argszero deleted the fix/forward-body-read-error branch September 13, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant