Skip to content

[Provider compatibility] Custom Kimi/vLLM Responses routes silently blank plaintext reasoning without a separate opt-in #6675

Closed

Description

@shch-y

Client or integration

Codex App and a direct Responses HTTP client on macOS.

Provider or upstream service

Custom self-hosted Kimi K3 (vLLM), using the openai-responses adapter. This report is about the plaintext-reasoning contract on a custom endpoint, not a claim that the official Kimi presets lose all reasoning.

OpenCodex version

@bitkyc08/opencodex 2.78.0. The installed reasoning.ts and passthrough.ts are byte-identical to the files fetched from main on 2026-10-06 (HEAD observed: 93cdffd6ac7f125d4b932a6d45af5c3ab1de5156).

Endpoint or capability

POST /v1/responses: replay of reasoning.content[].reasoning_text in multi-turn/tool-call histories.

Current behaviour

A custom Responses provider without preserveResponsesReasoningContent: true turns a nonempty plaintext reasoning item's content into [] before sending it upstream. Setting the similarly named preserveReasoningContentModels does not opt this wire into plaintext preservation.

This is deterministic at request construction, requires no GPU/model, and can silently remove history on a backend that accepts the request instead of rejecting it. Kimi K3's model card says that multi-turn/tool use requires preserved thinking history.

Relevant code:

  • reasoning.ts: blankContent depends on !opts?.preserveRawReasoningContent.
  • passthrough.ts: supplies that option only when provider.preserveResponsesReasoningContent === true.

Expected behaviour

An explicitly configured plaintext-replay endpoint should retain its reasoning history. For custom providers, please make this wire capability discoverable/configurable during setup, or warn when nonempty plaintext reasoning with no usable native encrypted payload will be discarded.

I understand that the existing default is intentional for native OpenAI compatibility. This is a custom-provider compatibility/configuration trap, not a request to preserve raw reasoning globally or forward it indiscriminately across providers.

Minimal redacted request or reproduction

Provider shape before the workaround:

{
  "adapter": "openai-responses",
  "baseUrl": "http://127.0.0.1:8000/v1",
  "allowPrivateNetwork": true
}

Save the standalone adapter reproduction below as reasoning-minimal-repro.ts. With the npm package installed, run it with Bun; for a global install, set OCX_PACKAGE_DIR to the installed package directory. It performs no network or model requests.

import { createRequire } from 'node:module';
import { dirname, join } from 'node:path';
const require=createRequire(import.meta.url);
const root=process.env.OCX_PACKAGE_DIR || dirname(require.resolve('@bitkyc08/opencodex/package.json'));
const {createResponsesPassthroughAdapter}=await import(join(root,'src/adapters/openai-responses/passthrough.ts'));
const {createTranslatorBudget}=await import(join(root,'src/lib/translator-budget.ts'));
const marker='Synthetic replay marker: ALPHA=217, BETA=684.';
for(const [name,flags] of Object.entries({
  default:{},
  chatFlagOnly:{preserveReasoningContentModels:['Kimi-K3-MXFP4']},
  responsesFlag:{preserveResponsesReasoningContent:true},
})){
 const provider={adapter:'openai-responses',baseUrl:'http://127.0.0.1:8000/v1',allowPrivateNetwork:true,...flags};
 const raw={model:'Kimi-K3-MXFP4',store:false,stream:true,input:[
  {role:'user',content:'Remember two labels.'},
  {type:'reasoning',summary:[],content:[{type:'reasoning_text',text:marker}]},
  {role:'assistant',content:'Recorded.'},
  {role:'user',content:'What are the label values?'},
 ]};
 const budget=createTranslatorBudget();
 const result=createResponsesPassthroughAdapter(provider).buildRequest(
  {modelId:raw.model,options:{},context:{tools:[]},_rawBody:structuredClone(raw)},
  {headers:new Headers(),translatorBudget:budget,providerName:'kimi-local'}
 );
 const body=JSON.parse(result.body);
 const item=body.input.find((x:any)=>x.type==='reasoning');
 console.log(JSON.stringify({case:name,content:item.content,markerPreserved:JSON.stringify(item).includes(marker)}));
 result.releaseBodyObservation?.();budget.dispose();
}

Actual response or error

The offline request-builder test prints:

{"case":"default","content":[],"markerPreserved":false}
{"case":"chatFlagOnly","content":[],"markerPreserved":false}
{"case":"responsesFlag","content":[{"type":"reasoning_text","text":"Synthetic replay marker: ALPHA=217, BETA=684."}],"markerPreserved":true}

The confirmed defect surface is the outgoing payload transformation, not an HTTP error.

Upstream documentation

Kimi K3 model usage / preserved thinking example.

The model's Chat example requires the complete prior assistant message, including reasoning and tool calls. On the self-hosted Responses backend, that reasoning is represented by plaintext Responses reasoning items.

Suggested mapping or implementation notes

Current workaround: set preserveResponsesReasoningContent: true on the custom provider. Applying it through the provider management API updates both live routing and persisted configuration; changing a Chat replay list alone is insufficient.

Please consider a documented/editor-exposed plaintext Responses replay capability and a regression test for custom Kimi/vLLM providers. Do not infer the wire contract from model names alone; native encrypted reasoning, plaintext reasoning, and cross-provider histories need different policies.

The built-in routes are meaningfully different:

Related reports: #875 and #1477 (DeepSeek Responses replay), #1193 (Chat replay-cache misses), #5421 (plaintext backfill). I found no issue specifically covering this custom Kimi/vLLM Responses setup.

Additional context and limits of the evidence

This was found while investigating repetitive output in a long Kimi session. A reconstructed, cold-cache replay at roughly 90k input tokens looped without tool calls at the 2048-token output bound when historical reasoning was blanked; the same seed with preserved history completed with a tool call in 254 output tokens. A different seed completed in both conditions. The replay is not byte-identical to the original session, and sampling/inference effects are not ruled out, so these observations do not establish that this flag is the sole cause or a complete fix for repetition.

A separate end-to-end test based on the official five-number example still answered incorrectly after enabling the flag, although the measured input increased from 146 to 176 tokens. Thus the report intentionally separates the proven adapter-level history loss from downstream model/parser behaviour that remains under investigation.

No private conversation, endpoint, credential, or raw production log is attached.

Checks

  • I searched existing provider and compatibility issues.
  • The request and response were redacted; the reproduction is synthetic.
  • The expected behaviour is based on an upstream specification and a concrete client requirement.

Activity

  1. Ingwannu commented on Oct 7, 2026

    @Ingwannu
    Owner

    Independently reproduced offline on current dev c30b522 with the production Responses request builder and synthetic four-item history: default => content:[]; preserveReasoningContentModels alone => content:[]; preserveResponsesReasoningContent:true => the original reasoning_text survives. All three expected comparisons passed, with no model/network requests, under CPU75%/RAM1536M/no-swap limits and private homes.

    The Responses-specific opt-in already exists and is described in the provider configuration reference. Your report therefore identifies a setup/discoverability trap, not a reason to change native OpenAI defaults or infer plaintext support from Kimi model names. Keep the option scoped to an endpoint whose replay contract you control; the Chat list is a different wire capability.

    Leaving this open for a clearer custom-Responses setup/diagnostic contract. Exposing a new editor control or warning is an owner-facing design choice, not implemented by this triage. The synthetic builder result does not establish that preserved history fixes long-session repetition or the five-number example.

  2. lidge-jun commented on Oct 8, 2026

    @lidge-jun
    Owner

    Reproduced on current dev (a12bb1f) with the production request builder and no network: by default, and with only preserveReasoningContentModels, a custom openai-responses provider sends plaintext reasoning items as content: []. With preserveResponsesReasoningContent: true, the original reasoning_text survives. Proxy-owned envelopes are still stripped.

    The default blanking is intentional. The ChatGPT backend can reject echoed raw reasoning, and #2229 / #2258 recorded the token and privacy cost of forwarding plaintext. Changing it for every unconfigured Responses route would alter established native behavior, so the default stays as it is.

    Workaround for Kimi K3 on vLLM today: set "preserveResponsesReasoningContent": true on that provider. preserveReasoningContentModels controls only Chat-wire reasoning_content replay and does not affect the Responses wire.

    Two follow-ups remain open as a product decision: surfacing this control in custom-provider setup, and warning when reasoning is discarded.

  3. lidge-jun commented on Oct 12, 2026

    @lidge-jun
    Owner

    Fixed on dev by #6961 (commit 6c27087518), landed through #7009 (merge 8f10f238e3). Ships in 2.84.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    providerProvider adapters, OpenAI-compat presets, upstream API quirksprovider-compatibilityProvider compatibility reports

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions