[Compatibility] Add an opt-in Responses stream/terminal repair escape hatch for custom providers

Summary

Custom openai-responses providers can hit compatibility failures where the upstream turn completes but Native Codex does not receive a terminal lifecycle it can commit. PR #1367 reproduced this class of failure and explored a per-model bounded-JSON fallback, but that PR cannot land as written and should be treated as evidence/design exploration rather than an implementation to salvage.

The remaining requirement is still valid: custom Responses providers need a narrow, explicit compatibility escape hatch that uses the current Responses repair architecture instead of adding another broad transport/configuration path.

Why this issue exists

PR #1367 reproduced the following boundary on both macOS arm64 and GNU/Linux x86_64:

the same Responses upstream completed when Native Codex connected directly;
through OpenCodex, the upstream/internal path reached response.completed;
Native Codex did not commit the turn and later entered its stream-idle retry path;
the reproduction did not require the reporter's separate OCI relay;
minimal calls could succeed, while richer turns reproduced the delivery failure.

That is useful evidence that there is a real Responses compatibility class after upstream completion and before the client commits the terminal event. It does not prove one specific Bun primitive is the sole cause.

The maintainer triage on #1367 also makes the current direction explicit:

The failure class is real and still uncovered. Current terminal repair is a narrower registry-controlled case, and no custom-provider escape hatch exists on current dev. A redesign around the current registry-only compatibility hint + terminal-repair architecture would be welcome.

PR #1367 itself is conflicting, far behind dev, and has unresolved blockers, so this issue intentionally does not copy its implementation surface.

Current architecture to build on

Current dev already has a registry-only Responses terminal-repair path in:

src/providers/registry.ts
src/server/responses-terminal-repair.ts
src/server/responses/core.ts

That path was designed for the narrow DeepSeek case where output items are structurally complete but a real terminal is missing or indefinitely delayed. It keeps upstream streaming, relays healthy streams unchanged, and can synthesize one terminal only after strict structural checks and a grace period.

This issue should extend that architecture rather than create a second broad Responses validation/reframing subsystem.

Proposed direction

Add a custom-provider, per-model compatibility hint for effective openai-responses routes. The hint should resolve through the same model/wire policy machinery used by registry compatibility policies and should be inactive unless explicitly configured.

The implementation should prefer the smallest repair that matches the observed failure:

Terminal repair first. Allow an eligible custom Responses model to opt into the existing bounded terminal-repair state machine, with the same fail-closed structural requirements used by registry-controlled repair.
Preserve healthy streams byte-for-byte where possible. A real upstream terminal remains authoritative and must never be duplicated or replaced.
Do not silently apply repair to unrelated providers/models. Resolve after the effective per-model wire is known and reject the setting when that wire is not openai-responses.
Keep forward-auth/canonical OpenAI routes protected. Do not allow a custom compatibility setting to mutate the canonical forward-auth contract unless a separate design explicitly approves that.
If the feat(responses): add an opt-in bounded JSON fallback for custom providers #1367 reproduction remains broken because the upstream terminal is present but is lost after the relay/inspection split, evaluate a bounded non-streaming fallback as a second, explicit compatibility mode. That fallback should live behind the same narrow compatibility-policy boundary. It should not reintroduce feat(responses): add an opt-in bounded JSON fallback for custom providers #1367's broad shared WebSocket validation changes or make non-streaming the default.

A shape such as the following is illustrative, not prescribed:

{
  "providers": {
    "<custom-provider>": {
      "adapter": "openai-responses",
      "modelResponsesCompatibility": {
        "<model>": "terminal-repair"
      }
    }
  }
}

If more than one repair mode is eventually justified, prefer an explicit enum/policy object over multiple overlapping booleans.

Acceptance criteria

Design guardrails

Do not use this issue to:

expose the old feat(responses): add an opt-in bounded JSON fallback for custom providers #1367 implementation wholesale;
globally tighten validation for every successful Responses JSON/WebSocket snapshot;
force all custom Responses providers to stream:false;
add provider/model name heuristics;
change the default streamMode;
bypass the Bun/runtime safety gate for eager relay;
replace the broader protocol-safe one-reader work tracked by [architecture][memory] Make 32 concurrent tool-recall sessions protocol-safe and memory-bounded #820;
claim terminal repair alone fixes the feat(responses): add an opt-in bounded JSON fallback for custom providers #1367 relay-loss reproduction without reproducing it on current dev.

The important distinction is that terminal repair and downstream relay loss are related compatibility symptoms, but they are not automatically the same bug. The implementation should prove which repair mode covers the current reproduction instead of assuming it.

feat(responses): add an opt-in bounded JSON fallback for custom providers #1367 - original draft and reproduction evidence; do not use as the implementation template
[architecture][memory] Make 32 concurrent tool-recall sessions protocol-safe and memory-bounded #820 - long-term runtime-qualified, protocol-safe one-reader/stream architecture
#893 is not fixed on macOS: #1025 repairs snapshots but the transport gate from closed #928 is still missing — Codex hangs on third-party Responses gateways #1127 - prior macOS Responses delivery failure with upstream success and missing client SSE
src/server/responses-terminal-repair.ts - current bounded terminal-repair state machine
docs/superpowers/plans/2026-08-06-deepseek-responses-streaming-terminal-repair.md - design rationale for the current registry-only repair path

🔗 Related PRs

#749 - fix(responses): reject expired forward continuations [merged]
#928 - fix(streaming): repair sparse Responses snapshots [open]
#1095 - fix(deepseek): restore progressive Responses streaming [open]
#1361 - fix(responses): restore DeepSeek streaming and routed code-mode tools [merged]
#1788 - fix(responses): fail closed when a routed provider calls an undeclared tool [open]

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

Create Plan

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

Observation: on the uncovered custom-Responses class, a clean `[DONE]` completion is also mis-accounted as a routing-health `failure`

This is a follow-on in support of #1809. The gap #1809 describes (a custom openai-responses
provider that completes output items but ends a turn without a Responses terminal —
response.completed / response.failed / response.incomplete — i.e. closes on a bare
data: [DONE] sentinel) affects not just client commit behavior but routing-health accounting.
On current dev that class is also recorded as an incomplete failure even though the stream
contents were a clean success.

Concrete, traced end-to-end against main @ v2.22.0:

src/server/relay.ts:1121-1126 — consumeForInspection’s onCleanEof fires
onTerminal("incomplete") whenever the SSE inspector hasn’t reported a terminal ([DONE] is not a
terminal — terminalStatusFromParsed(null) for [DONE] at relay.ts:280). So a [DONE]-only
stream is accounted incomplete, never completed.
src/server/responses/core.ts:2797-2816 — that synthetic terminal calls terminalRecorder("incomplete")
- options.onNativePassthroughTerminal("incomplete").
src/server/index.ts:1220-1224 (HTTP namespace) — onNativePassthroughTerminal →
addFinalRequestLog(..., { terminalStatus: "incomplete", closeReason: "terminal" }).
src/server/request-log.ts:319-332 — persisted to usage.jsonl with terminalStatus: "incomplete"
(kept because it is not "completed").
src/routing/history/indexer.ts:176 — requests.terminal_status = "incomplete" in the routing
index.
src/routing/health.ts:210 — classifySample: terminalStatus === "incomplete" ⇒ "failure"
(and :304 increments incompleteStreams, and :212 already treats any terminal ≠ completed as
failure). The health score is reduced both by the failure count and by the
INCOMPLETE_WEIGHT * (1 - incompleteRate) term (health.ts:392-395).
This surfaces in the documented routing-analytics incomplete-stream rate (src/routing/analytics.ts:356;
docs-site/.../routing.md:229).

Reachable class (and why it’s not a standalone issue)

The step above only runs when the upstream has no terminal repair, and current repair is strictly
registry-controlled:

providerModelResponsesTerminalRepair (src/providers/registry.ts:2635) resolves against the
registry entry (:2641); a policy exists only for deepseek-v4-flash / deepseek-v4-pro
(registry.ts:1519).
core.ts:2642-2650 wraps the upstream body in relayResponsesSseWithTerminalRepair only when that
policy is non-empty; a custom provider passes its body through unrepaired.
Built-in DeepSeek V4 is repaired before the relays → never hits this. Canonical OpenAI forward-pool
turns also don’t health-penalize: codexForwardTerminalOutcomeRecorder records an incomplete as a
200 success (core.ts:654-667) — though the request-log/requests.terminal_status would still read
incomplete.

So the mis-accounting is reachable only for a custom, unrepaired openai-responses provider that
closes a structurally-complete turn on a bare [DONE] — exactly the class this issue says is still
uncovered. I’m not filing it separately; it belongs here as implementation context, because the repair
escape hatch #1809 proposes is the natural place to also normalize the terminal-status accounting.

Ask for #1809

When adding the custom escape hatch, please require the repaired terminal to be recorded as
completed (not incomplete/failure) when the repaired stream is structurally complete, so a healthy
custom provider is not penalized in routing health / analytics on what is, from the client’s
perspective, a successful turn. (For contrast, the repair relay already emits the terminal into the
client body — responses-terminal-repair.ts:272-273 — whereas the passthrough relays emit nothing to
the client, only to accounting.)

Not a client-visible bug

To be explicit so nobody mis-files this: no response.incomplete SSE frame is injected into the
client body on the passthrough / eager (relay-eager.ts) / default tee paths. The raw [DONE] still
reaches and closes the client stream. The only contamination is the routing-health / request-log /
analytics status. (This corrects the earlier standalone draft’s headline.)

리뷰 · 우선순위 48 / 80

커스텀 openai-responses 프로바이더에서 업스트림은 response.completed까지 갔는데 Native Codex가 터미널 라이프사이클을 커밋하지 못하는 호환 실패를 다룹니다. PR #1367은 재현 증거로만 보고, 그 구현을 그대로 살리지 말라는 이슈입니다. 기본 네이티브 경로가 설치 직후 깨지는 문제는 아니고, 커스텀 Responses를 쓰는 일일 호환 이슈입니다.

현재 dev에는 레지스트리 전용 터미널 리페어가 이미 있습니다. src/providers/registry.ts의 providerModelResponsesTerminalRepair()는 getProviderRegistryEntry(id)의 modelResponsesTerminalRepair만 보고, 전송이 레지스트리와 맞을 때만 graceMs를 돌려줍니다. src/server/responses/core.ts가 그 정책을 받아 src/server/responses-terminal-repair.ts 상태머신에 넘깁니다. 커스텀 프로바이더가 설정으로 이 경로에 들어가는 탈출구는 없습니다.

원하는 것은 유효 와이어가 openai-responses인 모델만, 명시적으로 기존 리페어에 옵트인하는 좁은 힌트입니다. 건강한 SSE의 실제 터미널은 권위가 있어야 하고, 불완전·오염·취소된 출력을 성공으로 만들면 안 됩니다. #1367의 광범위한 WebSocket 검증이나 기본 stream:false는 이 이슈 범위가 아닙니다. 터미널 리페어와 릴레이 손실은 증상이 비슷해도 같은 버그라고 단정하면 안 됩니다.

해결방안: effective per-model wire를 확인한 뒤, 예시처럼 modelResponsesCompatibility 같은 설정을 레지스트리 정책과 같은 해석기에 연결하면 됩니다. 기본값은 그대로 두고, 정방향 인증/캐논 OpenAI 경로는 보호하며, 구조 검사와 grace를 통과할 때만 터미널 하나를 합성하면 됩니다. #1367 재현이 릴레이 손실이면 두 번째 모드를 같은 정책 경계 뒤에만 두고, 재현 없이 패치하지 않는 편이 맞습니다.

이 댓글은 grok-bot이 작성했습니다

리뷰 · 우선순위 48 / 80 · 유지

커스텀 openai-responses는 업스트림이 response.completed까지 갔는데 Codex가 턴을 못 닫고 idle 재시도로 감. DeepSeek용 수리기(src/server/responses-terminal-repair.ts)는 레지스트리만. 커스텀 옵트인이 없음. #1367은 증거일 뿐, 살리지 말 것.

유지함. 기본은 그대로. 설정한 모델만 기존 수리 상태기계에 태움. 건강한 스트림은 그대로. canonical OpenAI 포워드는 손대지 말 것.

해결방안: modelResponsesCompatibility: { "<model>": "terminal-repair" }. 실단말이 있으면 합성 금지. #1367 재현을 현재 dev에서 다시 보기.

이 댓글은 grok-bot이 작성했습니다

fix(providers): normalize configured model keys in terminal repair lo…

fix(providers): fail closed on invalid explicit grace values in compa…

feat(providers): wire terminal repair config validation and safeConfi…

feat(providers): add Responses terminal repair escape hatch for custo…

fix(providers): normalize configured model keys in terminal repair lo…

fix(providers): fail closed on invalid explicit grace values in compa…

feat(providers): wire terminal repair config validation and safeConfi…

fix(providers): harden custom terminal repair resolver against forwar…