Skip to content

Inline provider error frames bypass every retry budget: the turn dies on the first frame #6795

Description

@7jrxt42BxFZo4iAnN4CX

Goal / Why

An OpenAI-compatible provider can report a transient upstream failure inside a successful
response: HTTP 200, with a chunk-level {"error":{"message": …}} frame (OpenRouter does exactly
this when the routed upstream returns nothing — the text is literally
Provider returned an empty response). Codewhale makes that frame terminal on the first
occurrence: it is not counted as a stream error, it does not arm any resume/retry path, and it is
not covered by any configured budget. The turn dies with the provider's text as its outcome and
the operator has to resend by hand.

Two things make this look like a bug rather than a policy:

  1. The envelope built for the frame declares recoverable = true (hard-coded), so the UI shows an
    amber "warn" card for a frame the engine treats as terminal — the card promises retryability
    that does not exist.
  2. Every other no-content stream death is retried: a transport Err(_) before content is
    transparently re-issued, and a network-class drop is resumed. The same failure delivered as a
    frame inside a 200 is the only one with no recovery at all.

Observed

Five interactive sessions on OpenRouter (stealth/space-bunny-alpha) recorded 32 failed turns
whose outcome is exactly the provider text Provider returned an empty response
(~/.codewhale/sessions/*.json, turn_outcomes[].error). In each of those sessions every recorded
turn failed (one recorded 13/13 over several hours). The configured budgets were
[tui] stream_max_transparent_retries = 10, stream_max_resumes = 10, stream_max_errors = 10,
[retry] max_retries = 35 — none of them was consulted, and no retry is observable in the TUI.

Scope / Plan

  1. crates/tui/src/core/engine/turn_loop.rs:6248-6267 (v0.8.58: Native Anthropic Messages API adapter — cache_control, thinking blocks, tool streaming #3014) — the StreamEvent::Error arm:
    count the frame as a stream error and record its reason, exactly as the Err(_) arm does at
    :5843 (stream_errors = stream_errors.saturating_add(1)), instead of only setting
    stream_error and breaking (:6267).
  2. crates/tui/src/core/engine/turn_loop.rs:2168 — the re-issue decision. With the frame counted,
    stream_died_with_nothing becomes true when nothing actionable streamed, so the existing
    transparent retry / resume budget can fire; verify the classification gate below is what admits
    it for this class.
  3. crates/tui/src/error_taxonomy.rs:337 (classify_error_message) — add the transient
    provider-error vocabulary so the resume gate and the envelope agree with the retry decision:
    provider returned error, provider returned an empty response, and upstream 5xx wording →
    Network (or RateLimit for 429-style wording). Today nothing matches and the message falls
    through to ErrorCategory::Internal (:437), which is why no resume path can see it.
  4. crates/tui/src/error_taxonomy.rs:175 — stop hard-coding recoverable = true for a frame the
    engine is about to treat as terminal (turn_loop.rs:6264): derive it from the classification, or
    make the frame's treatment consistent with the envelope it emits.
  5. Keep genuinely terminal frames terminal: an auth or invalid-model rejection must still fail
    exactly once, with error severity, without a retry.

Key files

crates/tui/src/core/engine/turn_loop.rs
crates/tui/src/error_taxonomy.rs
crates/tui/src/core/engine/streaming.rs
crates/tui/src/client/chat.rs

Acceptance criteria

  • A stream whose first frame is an error object, with nothing actionable streamed, is re-issued
    up to [tui] stream_max_transparent_retries; a successful retry ends the turn normally, with
    no terminal error left in the transcript.
  • When the budget is exhausted the turn fails once with the provider's reason — and with the
    same "gave up" semantics the other stream paths already have.
  • A non-transient frame (auth, invalid model) still issues exactly one request and fails with
    error severity.
  • [tui] stream_max_errors, stream_max_resumes and stream_max_transparent_retries are all
    consulted on this path.
  • The envelope severity matches the outcome: a retried frame is a warning, a frame that ends
    the turn is an error.

Verification

cargo check -p codewhale-tui
cargo test -p codewhale-tui -- error_taxonomy
cargo test -p codewhale-tui -- stream
cargo clippy -p codewhale-tui -- -D warnings

Plus a new engine test: a scripted stream whose first frame is
{"error":{"message":"Provider returned an empty response"}}, followed by a healthy stream —
assert N+1 requests for max_transparent_retries = N and a Completed turn; and a terminal-class
frame asserting exactly one request.

Out of scope

  • The provider-side cause of the empty response (upstream behaviour, not something this repo can fix).
  • Replaying a request after actionable content already streamed — duplicate side effects stay
    forbidden, this issue only covers the no-content case.
  • The visibility of retry attempts in the transcript (filed separately).
  • The empty-stop budget being a hard-coded const (turn_loop.rs:7357, EMPTY_STOP_MAX_RETRIES = 2,
    used at :3090) — raised on Expose stream retry budgets and transport timeouts as configuration #6700.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageNew external report awaiting maintainer triage; repro, logs and version output help

    Projects

    • Status
      Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions