inline document bytes do not survive the inbound parse

Client or integration

Codex CLI

Provider or upstream service

openai-compatible, anthropic

OpenCodex version

2.59.0 / commit 134c92a01b120162f00c7275189cc47858720379

Endpoint or capability

message content part translation

Current behaviour

Two inbound parsers lose an attached document before any adapter runs, so no adapter can carry it even to a target that has a representation for it.

#	inbound	caller sent	what reaches the adapter
1	`/v1/chat/completions`	`file` part with inline `file_data`	nothing, the part is dropped
2	`/v1/messages`	`document` block with base64 bytes	a text marker `[document: title]`

Case 1. src/chat/inbound.ts:75 walks the content parts and recognises three shapes:

if (typeof raw === "string") { ... continue; }
if (!isRec(raw)) continue;
if ((raw.type === "text" || raw.type === "input_text" || raw.type === "output_text") && typeof raw.text === "string") {
  blocks.push({ type: "input_text", text: raw.text });
  continue;
}
const imageUrl = imageUrlFromPart(raw);
if (imageUrl) { ... continue; }
const videoUrl = videoUrlFromPart(raw);
if (videoUrl) blocks.push({ type: "input_video", video_url: videoUrl });

A {"type": "file", "file": {"filename": ..., "file_data": ...}} part matches none of these. The loop reaches its end with nothing pushed and no else branch, so the part is dropped in silence:

grep -c "file_data" src/chat/inbound.ts
0

Case 2. src/claude/inbound.ts:210 replaces the block with a marker:

case "document":
  // No Responses equivalent for raw document blocks; surface the title so the
  ...
  pending.push({ type: "input_text", text: `[document${typeof raw.title === "string" ? `: ${raw.title}` : ""}]` });

The same substitution appears at src/claude/inbound.ts:57 for the other block position.

The marker is a real improvement over dropping the block, which is what #939 fixed, and this report does not ask for that to be reverted. The remaining gap is narrower. For a target that can hold the bytes, the marker is now the only thing sent, and the caller cannot distinguish
"the model read the document" from "the model was told a document existed".

Because both losses happen at the inbound, they apply to every routed target, not only to the ones that cannot represent a document.

Expected behaviour

Case 1. A file part carrying inline file_data should be represented internally rather than dropped, so that adapters targeting a format with a counterpart can forward it. If the part cannot be carried to the selected target, the caller should be told rather than receiving a confident answer about a document the model never saw.

Case 2. Keep the marker for targets that have no representation, and carry the payload where the target does have one. The Chat Completions file part with file_data is the direct counterpart for an Anthropic document block with a base64 source.

Minimal redacted request or reproduction

Case 1:


curl http://127.0.0.1:10100/v1/chat/completions \
  -H 'content-type: application/json' \
  -H 'authorization: Bearer REDACTED' \
  -d '{
    "model": "REDACTED_PROVIDER/REDACTED_MODEL",
    "messages": [{"role": "user", "content": [
      {"type": "file", "file": {"filename": "doc.pdf", "file_data": "data:application/pdf;base64,JVBERi0xLjQK"}}
    ]}]
  }'


Case 2:


curl http://127.0.0.1:10100/v1/messages \
  -H 'content-type: application/json' \
  -H 'x-api-key: REDACTED' \
  -H 'anthropic-version: 2023-06-01' \
  -d '{
    "model": "REDACTED_PROVIDER/REDACTED_MODEL",
    "max_tokens": 64,
    "messages": [{"role": "user", "content": [
      {"type": "document", "title": "spec",
       "source": {"type": "base64", "media_type": "application/pdf", "data": "JVBERi0xLjQK"}}
    ]}]
  }'


Route case 2 to an OpenAI-compatible provider so the translated path is used.

Actual response or error

Case 1, the forwarded request carries a user message with no content derived from the file part.

Case 2, the forwarded request carries the text `[document: spec]` in place of the bytes.

Upstream documentation

https://developers.openai.com/api/reference/resources/chat.md

The Chat Completions content part set includes a file part whose file_data carries inline bytes, at $.messages[*].content[*].file.file_data. That is both the construct dropped in case 1 and the counterpart representation for the Anthropic document block in case 2.

Suggested mapping or implementation notes

Case 1. Add a branch to the content loop in src/chat/inbound.ts for a file part, alongside the existing image and video branches, and carry it on the internal block set.

Case 2. Once an internal representation for inline document bytes exists, the Messages inbound can populate it and keep the marker as the fallback for targets with no counterpart. The decision of which targets qualify is already made per adapter elsewhere, so the marker path does
not need to change for those.

The two cases share a dependency. Both need one internal carrier for inline document bytes, which is why they are filed together.

Additional context and attachments

#939 introduced the [document] marker to replace silent dropping. That was an improvement and this report builds on it rather than reversing it. Nothing was found covering the Chat inbound file part.

Checks

I searched existing provider and compatibility issues.
The request and response were redacted.
The expected behaviour is based on an upstream specification or a concrete client requirement.

Confirmed against current dev (2a2c01a7): Chat inbound has no file/file_data branch, and Claude inbound replaces base64 document blocks with a title-only marker before destination selection.

The implementation needs a bounded internal document representation rather than passing arbitrary base64 strings through ad hoc:

validate media type, decoded-size projection and aggregate request budget before materializing/copying bytes;
preserve filename/title and source bytes only for destination adapters with an explicit document/file capability;
keep the existing marker only as an explicit lossy fallback where product policy permits it, otherwise return a stable unsupported-media error before upstream dispatch;
never log or include document bytes in usage/debug records; and
cover Chat→Chat, Messages→Chat, unsupported target, malformed base64, oversized single document and aggregate multi-document limits.

A success response must not imply that the model saw bytes which were discarded. Keeping this open as a verified transport-loss issue; it should not be fixed by globally forwarding unbounded inline payloads.

fix: preserve caller constraints on the request path (#5211, #5210, #…

Fixed on dev by #5271 (41b66be6d880625055adb319aef9bce75300a42b).

The four losses on this path were fixed together because they share one failure mode: the request arrives carrying an explicit constraint, the proxy drops it, and the response is a normal success. A test that asserts a successful call or a 200 sees none of them. The governing rule the branch applies is to preserve on a supported path, and to refuse or apply the operator's conversion policy on an unsupported one, rather than dropping silently and reporting success.

Inline document bytes now survive the inbound parse into the internal representation and outbound, covered by tests/responses/chat-inline-document-bytes.test.ts. A title-only forward is no longer reported as a successful delivery.

Closing on dev. PRs here target dev, so GitHub does not auto-close.

Merge dev into feat/gemini-agentic-video-3271

@lidge-jun

@lidge-jun Thanks for your quick fix and support! I rechecked at 4cb43cb. The fix covers base64 documents. Three inputs from the same path are still lost.

1. A URL source document block still becomes the [document] marker

/v1/messages to anthropic and every other target. documentBlockToInputFile returns null for any source other than base64, so the block falls back to the text marker and the URL is not carried, even to an Anthropic target that accepts URL documents. The response is a normal success with no notice (src/claude/inbound.ts:52 and :238).

sent:      "content": [{"type": "document", "source": {"type": "url", "url": "http://internal-probe:8082/internal/document-marker"}}]
forwarded: "messages": [{"role": "user", "content": "[document]"}]

2. container_upload blocks are dropped by the Messages inbound

/v1/messages to anthropic and to openai-compatible. userMessageToItems has no container_upload case and the default branch drops it, so the file_id is lost on every target (src/claude/inbound.ts:246). When it is the only block, the anthropic adapter sends a synthetic (continue) turn (src/adapters/anthropic.ts:878) and the openai-compatible request goes out with an empty messages array. claudeCode.compatibility: "enforce" flags it as unknown_content_block, but that setting is opt in.

sent:      "messages": [{"role": "user", "content": [{"type": "container_upload", "file_id": "file-g1-controlled"}]}]
anthropic: "messages": [{"role": "user", "content": "(continue)"}]
openai:    "messages": []

3 oneOf is removed and the property is left unconstrained

OpenAI Chat to Gemini and Anthropic Messages to Gemini. oneOf is in the unsupported list (src/adapters/google-tool-schema.ts:17) and not in MERGED_SCHEMA_KEYS (:43). Nothing rewrites it, not even a type disjoint oneOf into the kept anyOf, so the property is sent as {}. compileTools fills only parameters (src/adapters/google-wire-compiler.ts:140). Under the default compatible policy the loss goes only to a debug diagnostic (src/adapters/google.ts:766), and the caller gets a normal response unless googleToolSchemaPolicy: "reject-lossy" is set.

sent:      "payment": {"oneOf": [{"type": "string"}, {"type": "number"}]}
forwarded: "payment": {}

The Anthropic input_schema gives the same result.

Please correct me if I am wrong. Thanks a lot

Navigation Menu

Repository navigation

inline document bytes do not survive the inbound parse #5212

Client or integration

Provider or upstream service

OpenCodex version

Endpoint or capability

Current behaviour

Expected behaviour

Minimal redacted request or reproduction

Actual response or error

Upstream documentation

Suggested mapping or implementation notes

Additional context and attachments

Checks

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

inline document bytes do not survive the inbound parse #5212

Description

Client or integration

Provider or upstream service

OpenCodex version

Endpoint or capability

Current behaviour

Expected behaviour

Minimal redacted request or reproduction

Actual response or error

Upstream documentation

Suggested mapping or implementation notes

Additional context and attachments

Checks

Activity

Ingwannu commented on Sep 20, 2026

lidge-jun commented on Sep 20, 2026

Android-Login-Analysis commented on Sep 24, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions