Skip to content
Merged
28 changes: 27 additions & 1 deletion docs-site/src/content/docs/reference/proxy-formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,26 @@ returns an error instead of partial JSON with HTTP 200. Streaming callers are un
When a provider filters or truncates a response, an unfinished tool call remains `incomplete`
in both JSON and SSE. Partial output is preserved, and the bridge does not emit an argument
completion event for that open call. Calls already completed keep their status. This preserves
the provider outcome; client retry behavior for incomplete responses is unchanged.
the provider outcome.

On the Responses streaming wire, an explicit non-retryable content-filter stop from
Anthropic (`refusal` / `content_filter`) or Kiro (`CONTENT_FILTERED` /
`GUARDRAIL_INTERVENED`) ends with `response.failed`, `error.code: "invalid_prompt"`,
and `retryable: false`. This prevents clients that retry incomplete streams from
resending the refused context. Usage and partial output are preserved. Web-search
and image sidecars stop without executing calls from that refused iteration or
asking the model again, on every inbound wire.

Buffered Responses JSON retains `status: "incomplete"`; Chat Completions retains
`finish_reason: "content_filter"`, and Messages retains `stop_reason: "refusal"`.
A content filter without an explicit non-retryable verdict, token limits, stalls,
and EOFs keep their existing incomplete behavior.

Explicit incomplete reasons use the same vocabulary in adapter-backed responses and native
Responses JSON, SSE, and WebSocket delivery:
output-limit aliases such as `length` and `max_tokens` become `max_output_tokens`, and refusal/filter
aliases such as `refusal` become `content_filter`. Other reasons remain unchanged, along with the
provider's message, retryability, and usage. Direct Chat and Messages encoders apply the same mapping.

On the pending `dev` implementation for #4112, a final upstream HTTP 413 on this surface
is classified as `invalid_request_error` / `context_length_exceeded`. Non-streaming callers
Expand Down Expand Up @@ -859,3 +878,10 @@ An ordinary 400 and a 429 are never retried this way, and neither is a 5xx, with
exception: a 502 whose body is the exact encrypted tool-output decrypt rejection, sent for a request
that carries encrypted tool output, gets the same single resend. A second rejection reaches the
client unchanged. If that happens, start a new conversation on the destination provider.

### Malformed tool-call history

Routed requests tolerate malformed serialized function arguments as an empty object.
Two bounded history repairs preserve usable arguments: a missing JSON object prefix,
and MiMo's trailing `</function></tool_call>` envelope with a missing closing quote or brace.
A repair must decode to a JSON object. Valid JSON values containing those tags are preserved.
8 changes: 7 additions & 1 deletion scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@
"version": 1,
"root": "tests",
"explicitShards": ["layout-clients.json"],
"explicit": { "catalog-auto-review-override.test.ts": "codex-integration", "management-auto-review-settings.test.ts": "server", "management-auto-review-auth.test.ts": "server", "prefer-model-alias-ids.test.ts": "server", "gcp-credential-boundary.test.ts": "adapters/google", "gcp-sa-marker.test.ts": "adapters/google", "gcp-sa-adapter.test.ts": "adapters/google", "management-gcp-credential-input.test.ts": "server", "codex-preserve-signed-in.test.ts": "codex-integration", "cli-account-preservation.test.ts": "cli", "service-state-isolation.test.ts": "service", "compaction-history-framing.test.ts": "responses", "catalog-native-max-context.test.ts": "codex-integration", "protocol-argon-thought-boundary.test.ts": "responses", "service-log.test.ts": "service", "messages-native-empty-inline.test.ts": "claude-integration",
"explicit": { "catalog-auto-review-override.test.ts": "codex-integration", "management-auto-review-settings.test.ts": "server", "management-auto-review-auth.test.ts": "server", "prefer-model-alias-ids.test.ts": "server", "responses-mimo-envelope.test.ts": "responses", "responses-call-id-linkage.test.ts": "responses",
"bridge-terminal-refusal.test.ts": "adapters",
"loop-terminal-refusal.test.ts": "images",
"web-search-terminal-refusal.test.ts": "web-search",
"combo-terminal-refusal.test.ts": "routing",
"server-terminal-refusal-wiring.test.ts": "server",
"incomplete-reason-normalization.test.ts": "responses", "gcp-credential-boundary.test.ts": "adapters/google", "gcp-sa-marker.test.ts": "adapters/google", "gcp-sa-adapter.test.ts": "adapters/google", "management-gcp-credential-input.test.ts": "server", "codex-preserve-signed-in.test.ts": "codex-integration", "cli-account-preservation.test.ts": "cli", "service-state-isolation.test.ts": "service", "compaction-history-framing.test.ts": "responses", "catalog-native-max-context.test.ts": "codex-integration", "protocol-argon-thought-boundary.test.ts": "responses", "service-log.test.ts": "service", "messages-native-empty-inline.test.ts": "claude-integration",
"owner-registry-isolation.test.ts": "config",
"codex-history-pending-wal.test.ts": "codex-integration", "responses-anthropic-billing-send-budget.test.ts": "responses", "anthropic-billing-ownership.test.ts": "adapters/anthropic", "server-anthropic-billing.test.ts": "server", "server-anthropic-billing-boundaries.test.ts": "server", "router-anthropic-billing-failover.test.ts": "routing", "anthropic-billing-refusal.test.ts": "adapters/anthropic", "messages-request-id-headers.test.ts": "claude-integration", "bounded-file-read.test.ts": "lib", "responses-undeclared-tool-guard-inbound.test.ts": "responses", "explicit-empty-tool-catalog.test.ts": "responses", "explicit-empty-tool-catalog-history.test.ts": "responses",
"messages-request-id-endpoint.test.ts": "claude-integration",
Expand Down
17 changes: 9 additions & 8 deletions src/adapters/anthropic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -313,11 +313,12 @@ function isAnthropicContentFilterStopReason(
}

/**
* Anthropic `refusal` / `content_filter` is a permanent sampling decision, not a disconnect.
* Emitting `done` with that stopReason used to surface as `response.incomplete` without
* `retryable`, which Codex treats as a dropped stream and retries five times (#4312).
* The explicit incomplete event is what the bridge already forwards into
* `incomplete_details.retryable`. Usage is preserved: a filtered turn still consumed tokens.
* Anthropic `refusal` / `content_filter` is a permanent sampling decision, not a disconnect
* (#4312). Codex does not read `incomplete_details.retryable`: it retries every
* `response.incomplete`. The explicit `retryable: false` verdict here is the marker the bridge
* uses to deliver the turn to a Responses client as a terminal `response.failed` instead
* (`src/bridge/terminal-refusal.ts`); a `done` with that stopReason carries no verdict and
* stays a plain incomplete. Usage is preserved: a filtered turn still consumed tokens.
* `max_tokens` stays a `done` so the client can continue from a legitimate truncation.
*/
function anthropicContentFilterIncomplete(
Expand Down Expand Up @@ -1237,8 +1238,8 @@ export function createAnthropicAdapter(provider: OcxProviderConfig, cacheRetenti
return;
}
// Refusal / content_filter must not look like a dropped stream. `done` with that
// stopReason becomes `response.incomplete` without `retryable`, and Codex retries
// the same refusal five times (#4312).
// stopReason carries no verdict, so the bridge could not tell it from a retryable
// incomplete and Codex would re-send the same refusal (#4312).
if (isAnthropicContentFilterStopReason(pendingStopReason)) {
yield anthropicContentFilterIncomplete(pendingStopReason, usageFromAnthropic(pendingUsage));
return;
Expand Down Expand Up @@ -1433,7 +1434,7 @@ export function createAnthropicAdapter(provider: OcxProviderConfig, cacheRetenti
}
// Same rule as emitDone: refusal / content_filter is a permanent decision, not a
// disconnect. This branch bypasses emitDone, so the check has to be repeated here
// or the EOF route still emits `done` and Codex retries the refusal (#4312).
// or the EOF route emits a verdict-less `done` the bridge cannot make terminal (#4312).
if (isAnthropicContentFilterStopReason(pendingStopReason)) {
emittedDone = true;
yield anthropicContentFilterIncomplete(pendingStopReason, usageFromAnthropic(pendingUsage));
Expand Down
10 changes: 7 additions & 3 deletions src/adapters/openai-responses/passthrough.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { truncationReasonFor } from "../../responses/truncated-stop-reason";
import { normalizeRoutedAgentMessages } from "../routed-agent-messages";
import { frameCompactionHistory } from "../../responses/compaction-history-framing";
import { nameRoutedIdentity, repairIdentityInResponsesBody, stripRoutedIdentity } from "../identity";
Expand Down Expand Up @@ -662,9 +663,11 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig):
case "error":
yield { type: "error", message: responsesErrorMessage(payload.response ?? payload) };
return;
case "response.incomplete":
yield { type: "incomplete", reason: responsesErrorMessage(payload.response ?? payload) };
case "response.incomplete": {
const reason = responsesErrorMessage(payload.response ?? payload);
yield { type: "incomplete", reason: truncationReasonFor(reason) ?? reason };
return;
}
case "response.completed":
{
completedSeen = true;
Expand Down Expand Up @@ -757,7 +760,8 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig):
return [{ type: "error", message: responsesErrorMessage(payload) }];
}
if (payload.status === "incomplete") {
return [{ type: "incomplete", reason: responsesErrorMessage(payload) }];
const reason = responsesErrorMessage(payload);
return [{ type: "incomplete", reason: truncationReasonFor(reason) ?? reason }];
}
const usage = usageFromResponsesPayload(payload);
const output = Array.isArray(payload.output) ? payload.output : [];
Expand Down
2 changes: 1 addition & 1 deletion src/bridge/response-json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -657,7 +657,7 @@ function buildResponseJSONWithBudget(
: errorEvent?.retryable !== undefined ? { retryable: errorEvent.retryable } : {}),
...(incompleteEvent ? {
incomplete_details: {
reason: incompleteEvent.reason,
reason: truncationReasonFor(incompleteEvent.reason) ?? incompleteEvent.reason,
...(incompleteEvent.message ? { message: incompleteEvent.message } : {}),
...(incompleteEvent.retryable !== undefined ? { retryable: incompleteEvent.retryable } : {}),
},
Expand Down
28 changes: 24 additions & 4 deletions src/bridge/sse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import {
classifyError,
cyberPolicyErrorType,
CYBER_POLICY_ERROR_CODE,
isCyberPolicyCode,
isTerminalRefusalCode,
type OcxErrorPayload,
} from "../lib/errors";
import { redactSecretString } from "../lib/redact";
Expand Down Expand Up @@ -51,6 +51,7 @@ import {
} from "../lib/translator-budget";
import { adapterFailureFromEvent, emptyChunks, joinChunks, ownedBudgetAbandonedMs, responsesUsage, toolCallArgumentsCouldBeJson, toolCallArgumentsUsable, uuid, webSearchAction } from "./internal";
import type { OutputItem, StringChunks } from "./internal";
import { terminalRefusalFailure } from "./terminal-refusal";

function sseEvent(name: string, data: Record<string, unknown>): string {
return `event: ${name}\ndata: ${JSON.stringify(data)}\n\n`;
Expand Down Expand Up @@ -121,6 +122,12 @@ export function bridgeToResponsesSSE(
* would reach the client under the invented name instead of the normalized one.
*/
enforceDeclaredToolNames?: boolean;
/**
* Deliver a non-retryable upstream refusal as `response.failed` with a terminal-refusal code
* instead of `response.incomplete` (see `./terminal-refusal`). Set only for the Responses
* inbound wire: the chat and Anthropic encoders read the incomplete to report a filter stop.
*/
terminalRefusalAsFailed?: boolean;
/** Declared parameter schema per tool name; repairs integral-float integer args (#1611). */
toolParameterSchemas?: ReadonlyMap<string, Record<string, unknown>>;
/**
Expand Down Expand Up @@ -1312,12 +1319,25 @@ export function bridgeToResponsesSSE(
flushHiddenReasoningEnvelope();
options?.onUsage?.(event.usage);
await awaitThoughtSignatureDurability();
const refusal = options?.terminalRefusalAsFailed ? terminalRefusalFailure(event) : undefined;
if (refusal) {
emit("response.failed", {
response: {
...responseSnapshot("failed", finishedItems, event.endTurn),
usage: responsesUsage(event.usage),
...refusal,
},
});
reportTerminal("failed");
terminalEvent = true;
break;
}
emit("response.incomplete", {
response: {
...responseSnapshot("incomplete", finishedItems, event.endTurn),
usage: responsesUsage(event.usage),
incomplete_details: {
reason: event.reason,
reason: truncationReasonFor(event.reason) ?? event.reason,
...(event.message ? { message: event.message } : {}),
...(event.retryable !== undefined ? { retryable: event.retryable } : {}),
},
Expand Down Expand Up @@ -1350,7 +1370,7 @@ export function bridgeToResponsesSSE(
...(event.usage ? { usage: responsesUsage(event.usage) } : {}),
error: failure.error,
last_error: failure.error,
...(isCyberPolicyCode(failure.error.code)
...(isTerminalRefusalCode(failure.error.code)
? { retryable: false }
: event.retryable !== undefined ? { retryable: event.retryable } : {}),
},
Expand Down Expand Up @@ -1391,7 +1411,7 @@ export function bridgeToResponsesSSE(
...responseSnapshot("failed", finishedItems),
error: failure,
last_error: failure,
...(isCyberPolicyCode(failure.code) ? { retryable: false } : {}),
...(isTerminalRefusalCode(failure.code) ? { retryable: false } : {}),
},
});
reportTerminal("failed");
Expand Down
59 changes: 59 additions & 0 deletions src/bridge/terminal-refusal.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import type { AdapterEvent } from "../types";
import type { OcxErrorPayload } from "../lib/errors";
import { redactSecretString } from "../lib/redact";

/**
* The terminal-refusal code Codex ends a turn on. It is a member of `TERMINAL_REFUSAL_CODES`
* in `src/lib/errors.ts`, and the one that says "this prompt, as sent, is refused" without
* claiming a specific policy family the upstream never named.
*/
const TERMINAL_REFUSAL_FAILED_CODE = "invalid_prompt";

interface TerminalRefusalFailure {
error: OcxErrorPayload;
last_error: OcxErrorPayload;
retryable: false;
}

/**
* Client-visible copy: what happened, and that resending the same context cannot help. It points
* at a new conversation rather than a new turn, because a later turn replays the refused context.
*/
function terminalRefusalMessage(detail: string | undefined): string {
const reason = detail?.trim() ? ` (${redactSecretString(detail.trim())})` : "";
return `The upstream model refused this request${reason}. `
+ "Retrying with the same conversation context will not succeed; change the request or start a new conversation.";
}

/**
* An adapter `incomplete` that is a non-retryable upstream refusal: a content filter carrying an
* explicit `retryable: false` verdict. A filter without the verdict, an output cap, a stall and an
* adapter EOF do not qualify. The sidecar loops read this too: a refused iteration ends the turn,
* so a sidecar call it carried is neither run nor answered with another model request.
*/
export function isTerminalRefusalIncomplete(event: AdapterEvent | undefined): boolean {
return event?.type === "incomplete" && event.reason === "content_filter" && event.retryable === false;
}

/**
* `response.failed` fields for an adapter `incomplete` that is a non-retryable upstream refusal
* (#4312), or `undefined` when the event is anything else. The Anthropic and Kiro adapters are
* the ones that emit it today.
*
* codex-rs maps every `response.incomplete` to a retryable stream error and never reads
* `incomplete_details.retryable`, so a refusal delivered that way is re-sent through the whole
* stream retry budget. It only ends the turn on `response.failed` carrying a terminal-refusal
* `error.code`. Only an explicit `retryable: false` content filter qualifies: a filter without
* the verdict, an output cap, a stall and an adapter EOF stay `response.incomplete`.
*/
export function terminalRefusalFailure(
event: Extract<AdapterEvent, { type: "incomplete" }>,
): TerminalRefusalFailure | undefined {
if (!isTerminalRefusalIncomplete(event)) return undefined;
const error: OcxErrorPayload = {
type: "invalid_request_error",
code: TERMINAL_REFUSAL_FAILED_CODE,
message: terminalRefusalMessage(event.message),
};
return { error, last_error: error, retryable: false };
}
9 changes: 8 additions & 1 deletion src/images/loop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import { isAnthropicInstanceId } from "../providers/anthropic-instance-id";
import { cloneProviderOpaqueToolCallMetadata } from "../responses/provider-opaque-metadata";
import type { AttemptRecoveryKind } from "../usage/log";
import { bridgeToResponsesSSE } from "../bridge";
import { isTerminalRefusalIncomplete } from "../bridge/terminal-refusal";
import { clearableDeadline, idleDeadline } from "../lib/abort";
import { readBoundedResponseBody } from "../lib/bounded-body";
import { applyUpstreamRecoveryInit, fetchWithResetRetry, prepareSameTarget429Wait } from "../lib/upstream-retry";
Expand Down Expand Up @@ -359,6 +360,8 @@ export interface ImageBridgeDeps {
onCompletedResponse?: (response: Record<string, unknown>, providerState?: OcxProviderContinuationState) => void;
/** WebSocket Responses path only — leave response id empty for protocol compatibility. */
forceEmptyResponseId?: boolean;
/** Responses inbound only: deliver a non-retryable upstream refusal as `response.failed`. */
terminalRefusalAsFailed?: boolean;
}

/**
Expand Down Expand Up @@ -869,7 +872,10 @@ export async function runWithImageBridge(deps: ImageBridgeDeps): Promise<Respons
// Loop (fulfill + re-ask) ONLY when the model's actionable output is purely image_gen. A
// real tool call means this turn is terminal for Codex — finalize so those calls reach
// Codex. forceFinal also finalizes.
const shouldLoop = split.calls.length > 0 && !split.hasRealToolCall && !forceFinal;
// A refused iteration also finalizes: fulfilling its media call would re-ask the model
// with the refused context (#7148).
const shouldLoop = split.calls.length > 0 && !split.hasRealToolCall && !forceFinal
&& !split.passthrough.some(isTerminalRefusalIncomplete);
if (!shouldLoop) {
if (hiddenUsage) {
for (let i = split.passthrough.length - 1; i >= 0; i--) {
Expand Down Expand Up @@ -1097,6 +1103,7 @@ export async function runWithImageBridge(deps: ImageBridgeDeps): Promise<Respons
replayCacheScope: parsed._reasoningReplayScope,
...(deps.clientToolCatalogRestriction ? { ...deps.clientToolCatalogRestriction, enforceDeclaredToolNames: false } : {}),
...(deps.forceEmptyResponseId ? { responseId: "" } : {}),
...(deps.terminalRefusalAsFailed ? { terminalRefusalAsFailed: true } : {}),
hideThinkingSummary: parsed.options.hideThinkingSummary,
hideRawReasoning: parsed.options.hideRawReasoning,
stallTimeoutSec: deps.stallTimeoutSec,
Expand Down
2 changes: 1 addition & 1 deletion src/protocols/encoders/adapter-events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -729,7 +729,7 @@ export function encodeAdapterEventStream(
terminal = {
status: "incomplete",
incomplete: {
reason: event.reason,
reason: truncationReasonFor(event.reason) ?? event.reason,
...(event.message ? { message: event.message } : {}),
...(event.retryable !== undefined ? { retryable: event.retryable } : {}),
},
Expand Down
Loading
Loading