Skip to content

fix!: send reasoning_effort as reasoning.effort on flex Responses requests - #54

Merged
Crown421 merged 1 commit into
mainfrom
claude/flex-reasoning-effort-33dcc6
Oct 21, 2026
Merged

Crown421 merged 1 commit into
mainfrom
claude/flex-reasoning-effort-33dcc6

Conversation

@Crown421

@Crown421 Crown421 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What

Since 0.11.0 (#48), text calls without completion_window="24h" go to flex /v1/responses. Two kinds of reasoning control were forwarded in a shape that /v1/responses doesn't accept:

  • responses.create(reasoning_effort=...), top-level or in extra_body, went out as a top-level reasoning_effort. A test asserted this as a supported extension.
  • chat_template_kwargs, thinking and thinking_token_budget in extra_body were copied onto the Responses request from both chat.completions.create and responses.create.

Today the gateway ignores these on /v1/responses, so the caller's reasoning setting is silently lost. control-layer#1999 (DW-974) makes the gateway refuse all four with 400 unsupported_parameter ("use 'reasoning.effort'"). The refused list comes from onwards/src/reasoning.rs (ApiSurface::Responses).

Change (Python and TypeScript, same behaviour)

  • reasoning_effort on the Responses flex path now becomes reasoning: {"effort": ...}. It merges with other reasoning fields (e.g. summary). If reasoning.effort is also set to a different value, the call raises instead of picking one. Chat calls already mapped reasoning_effort this way. A new end-to-end test covers that.
  • chat_template_kwargs, thinking and thinking_token_budget now raise client-side (ValueError / Error) before anything is submitted. The error names the field and points to reasoning_effort (chat) or reasoning={"effort": ...} (Responses).

Why raise, not translate

Mapping enable_thinking: false to reasoning.effort: "none" would be faithful for that one key. chat_template_kwargs can hold arbitrary template keys, though, and thinking / thinking_token_budget have no exact reasoning.effort equivalent. Translating only some of them means guessing effort levels, which changes behaviour silently. Raising is simpler and matches the gateway everywhere else:

  • realtime /v1/chat/completions already returns a 400 for these fields;
  • 24h batch uploads already refuse lines that contain them;
  • flex will refuse them after #1999.

The difference is that the error comes before a background job is created, and it names the replacement. Callers who already use reasoning_effort are unaffected.

The 24h batch path is unchanged.

Breaking change

Flex calls (any text call without completion_window="24h") now raise before submitting in two cases:

  • the request carries chat_template_kwargs, thinking or thinking_token_budget;
  • reasoning_effort conflicts with reasoning.effort.

Before this change, these requests were sent and the fields were silently ignored. After control-layer#1999 they'd fail with a 400 anyway. bump-minor-pre-major is set, so release-please bumps the minor version: Python 0.12.0 and TypeScript 0.7.0.

Tests

  • Python (python/tests/test_flex_polling.py):
    • The old test that expected reasoning_effort in extra_body now checks that the other extensions still go through extra_body.
    • New TestFlexReasoningControls, using public create() with only the upstream responses.create mocked:
      • top-level, extra_body and merged reasoning_effort all become reasoning;
      • a conflicting reasoning.effort raises;
      • each native thinking field raises on both the chat and Responses flex paths, and nothing is submitted;
      • chat reasoning_effort reaches the submitted Responses request as reasoning.effort.
  • TypeScript (typescript/test/client.test.ts): the same cases.

Run locally:

  • pytest tests/test_flex_polling.py tests/test_responses.py: 59 passed (Python 3.11, openai latest).
  • ty check: clean.
  • npm test: 57 passed.
  • npm run typecheck: clean.

Python 3.12 and the openai-version matrix are left to CI.

Known follow-ups (not in this PR; tracked in DW-981)

  • Flex tools without a description get a 400. dwctl's Responses tool type requires description: String (dwctl/src/inference/translation/responses/types.rs:557), but Chat Completions accepts tools without one. This is the default path for the PydanticAI, Agno, Microsoft Agent Framework, Agents SDK and Arize recipes. Confirmed in source, not run against a gateway.
  • autobatcher drops cache_control on content parts and assistant reasoning_content on the flex path. Neither translator references them. As a result, the cache_control option on ChatDoublewordAsync / DoublewordLLMAsync has no effect, and interleaved reasoning isn't replayed on tool turns.
  • Docs pages call this path "batch". The PydanticAI, Agno, Microsoft Agent Framework, Agents SDK and Arize recipes say BatchOpenAI() without completion_window="24h" "submits batch jobs", but it runs as flex. This came from a docs read on 21 Oct; I didn't re-check it here.

View guided diff

…uests

Flex text calls are submitted to /v1/responses. responses.create(reasoning_effort=...)
was forwarded as a top-level reasoning_effort, and extra_body thinking fields
(chat_template_kwargs, thinking, thinking_token_budget) were forwarded as-is.
Doubleword ignores these on /v1/responses today and will refuse them with a 400.

Map reasoning_effort into reasoning.effort (merging with any other reasoning
fields, and refusing a conflicting reasoning.effort), and raise a ValueError /
Error naming the field for the native thinking fields on both the chat and
Responses flex paths. Same change in the Python and TypeScript SDKs.

BREAKING CHANGE: flex chat.completions.create and responses.create now raise
before submitting when the request carries chat_template_kwargs, thinking or
thinking_token_budget, or a reasoning_effort that conflicts with
reasoning.effort. These were previously sent and silently ignored. Use
reasoning_effort (chat) or reasoning={"effort": ...} (Responses) instead.
@Crown421 Crown421 changed the title fix: send reasoning_effort as reasoning.effort on flex Responses requests fix!: send reasoning_effort as reasoning.effort on flex Responses requests Oct 21, 2026
@Crown421
Crown421 force-pushed the claude/flex-reasoning-effort-33dcc6 branch from 054cdf9 to b51b6a0 Compare October 21, 2026 15:24
@Crown421
Crown421 merged commit 0d270e7 into main Oct 21, 2026
4 checks passed
@fergusfinn fergusfinn mentioned this pull request Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant