From 4e9284038d63eb345b3af101c2acd4f219ac5af9 Mon Sep 17 00:00:00 2001 From: 6Kmfi6HP <179412085+6Kmfi6HP@users.noreply.github.com> Date: Mon, 28 Sep 2026 15:35:43 +0200 Subject: [PATCH 1/4] =?UTF-8?q?fix(protocol):=20sub2api-parity=20audit=20?= =?UTF-8?q?=E2=80=94=209=20conversion=20gaps=20across=20chat/responses/ant?= =?UTF-8?q?hropic?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - chat->responses request: passthrough parallel_tool_calls/service_tier (top-level keys backfilled from raw body via preserveChatPassthroughKeys); map response_format to text.format (json_schema flatten / json_object type) - responses->chat stream/aggregate/nonstream: accept custom_tool_call + custom_tool_call_input delta/done on shared tool index (done reads input key); split incomplete reason (content_filter vs length); echo upstream service_tier on chat top level; treat response.done alias like completed - chat->anthropic request: strip temperature/top_p when thinking enabled - claude responses: content_filter stop_reason refusal -> end_turn (enum); stop branch falls back to tool_use when content holds tool_use blocks - usage/finish closed sets: cache_creation_input_tokens -> details 归位; normalizeFinishReason unknown -> stop - 11 regression tests in protocol_parity_sub2api_test.go; docs/API.md + CHANGELOG updated - live E2E (mimo-v2.6-flash direct + muse-spark responses-rule): 9/9 pass Co-Authored-By: Claude Code --- CHANGELOG.md | 8 + docs/API.md | 2 + internal/app/chat.go | 30 ++ internal/app/chat_protocol.go | 22 +- internal/app/chat_to_anthropic.go | 39 ++- internal/app/chat_to_responses_upstream.go | 165 +++++++++- internal/app/claude.go | 14 +- internal/app/claude_responses.go | 4 +- internal/app/protocol_parity_sub2api_test.go | 308 +++++++++++++++++++ 9 files changed, 554 insertions(+), 38 deletions(-) create mode 100644 internal/app/protocol_parity_sub2api_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index e53b5ba..7b36903 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ ## v0.14.12 (unreleased) +- Protocol parity vs sub2api `apicompat` (`fix(chat)`, `fix(claude)`):对照 Wei-Shaw/sub2api `backend/internal/pkg/apicompat` 全量审计三条转换链路并补齐 9 项差异,官方 Responses 流事件文档确认事件语义,真实流量三协议 9/9 验证通过。 + - chat→responses 请求体:`parallel_tool_calls` / `service_tier` 透传上游(顶层键经 `preserveChatPassthroughKeys` 从原始 body 回填 ExtraBody,类型化结构装不下;`extra_body` 显式键优先);`response_format`(`json_schema` 展平 / `json_object` 透传)映射为 `text.format`(此前直接丢弃,structured-output 约束在该路径失效)。 + - responses→chat 流/聚合/非流式:`custom_tool_call`(custom/freeform 工具)纳入工具槽位,`custom_tool_call_input.delta/done` 与 function_call 同形累积(done 读 `input` 键);`incomplete` 按 `reason` 细分(`content_filter`→`content_filter`,其余→`length`,此前一律 `length`);上游 `service_tier` 回写 chat 顶层(流 chunk + 聚合 + 非流式);`response.done`(Realtime/WS 别名)与 `completed` 同等终结(此前聚合路径直接原样回吐上游体)。 + - chat→anthropic 请求体:thinking 生效即剥离 `temperature`/`top_p`(此前无条件透传,上游 400;与本项目 `convertClaudeRequest` 同口径)。 + - claude 响应:`content_filter` 的 `stop_reason` 由非法枚举 `refusal` 改为 `end_turn`(拒绝文本已进内容);`stop` 分支加 tool_use block 存在性回退(防止客户端不回传工具结果、对话卡死)。 + - usage/finish 闭集合:`anthropicUsageToChat` 补 `cache_creation_input_tokens`→`prompt_tokens_details.cache_creation_tokens` 归位;`normalizeFinishReason` 未知原因闭集合回退 `stop`(此前透传污染)。 + - 回归测试:`protocol_parity_sub2api_test.go` 新增 11 用例(custom 工具同 index/`input` 去重、`content_filter` 双路径、`response.done` 哨兵、`service_tier` 双路径、thinking 剥参、`cache_creation` 归位、未知 finish 闭集合、ExtraBody 回填等)。`docs/API.md` 同步。 + - 真实流量(新二进制,18358 direct + 18359 muse-spark responses-rule):direct chat 工具单 index、claude/responses 工具、chat 流 `[DONE]`、responses 流 `completed`、messages 流 `message_stop` 9/9 通过;thinking+temperature 同传 200。 - Fix muse-spark via Claude protocol never hitting upstream prefix-cache (`fix(cache)`): `claudeToResponsesBody` never injected `prompt_cache_key` / `prompt_cache_retention` — the chat→responses, native passthrough, and claude→anthropic paths all had the injection, only the claude→responses path missed it, so `/v1/messages` traffic on muse-spark models never warmed the upstream cache (`stats.json` showed ~3M prompt tokens with zero `cache_read_tokens` while chat-path `big-pickle` cached normally). The body now goes through `applyResponsesCacheHintsToRawBody` (responses-scoped: retention + session-derived key, no top-level `cache_control`), signature gains a `ctx` param at all 4 call sites. Also drops the non-spec top-level `stop` field the converter used to emit (chat path already dropped it). Regression tests in `claude_responses_cache_test.go`. - Fix non-stream upstream error-in-200 swallowed as empty message (`fix(claude)`): some upstreams return HTTP 200 with an error-shaped body (`{"type":"error","error":{...}}`, e.g. model-side generation failure); `convertResponsesToClaude` failed to parse it as a success response and fell back to an empty text message with no usage, hiding the cause. The non-stream 2xx branches (forward first-try + same-protocol retry + probe) now detect the error shape via `isResponsesErrorBody` and relay it through the existing `convertResponsesErrorToClaude` as HTTP 502 with the upstream message preserved. Stream path already relayed `response.failed`/`error` events. diff --git a/docs/API.md b/docs/API.md index d062f2f..d8c1e38 100644 --- a/docs/API.md +++ b/docs/API.md @@ -100,6 +100,8 @@ - 显式零值的 `temperature`(闭区间 `0..2`)、`top_p`、`frequency_penalty`、`presence_penalty` - `max_output_tokens`、`stop`、`user`、`parallel_tool_calls`、`stream_options`、`store` - 函数工具、项目已有的内置工具、`tool_choice`、`reasoning`、`metadata` +- Chat 经 `protocol_rules` 走原生 Responses 上游时:`parallel_tool_calls` / `service_tier` 透传上游;`response_format`(`json_schema` 展平 / `json_object` 透传 `type`)映射为 `text.format`;`custom_tool_call`(custom/freeform 工具)的 `input` 增量与 `done` 按同一 `tool_calls` index 累积;`incomplete` 按 `reason` 细分 `finish_reason`(`max_output_tokens`→`length`,`content_filter`→`content_filter`);上游 `service_tier` 回写 chat 顶层;`response.done`(Realtime/WS 别名)与 `completed` 同等终结流 +- Chat 经 `protocol_rules` 走原生 Anthropic 上游时:thinking 生效即剥离 `temperature`/`top_p`(与 Claude 入站同口径,避免上游 400) - Anthropic-style `tool_result`(`call_id`,缺省时用 `tool_use_id`;`content` 支持 string、字符串数组、`{type:"text"|"input_text"|"output_text",text}` blocks;`is_error:true` 加 `Error: ` 前缀) - 正常终态 `response.completed`;长度截断终态 `response.incomplete`,reason 为 `max_output_tokens` diff --git a/internal/app/chat.go b/internal/app/chat.go index d3118cc..7d75f01 100644 --- a/internal/app/chat.go +++ b/internal/app/chat.go @@ -356,6 +356,31 @@ func clientStreamUsageWanted(body []byte) bool { return false } +// preserveChatPassthroughKeys 把 OpenAIRequest 类型化结构装不下的顶层直通 +// 字段从原始 body 回填进 ExtraBody(只补缺,不覆盖 extra_body 内显式键),供 +// chat→responses 上游体透传(对齐 sub2api ChatCompletionsToResponses 的 +// ParallelToolCalls/ServiceTier)。没有这些键时不建 ExtraBody。 +func preserveChatPassthroughKeys(body []byte, req *OpenAIRequest) { + var raw map[string]any + if json.Unmarshal(body, &raw) != nil || req == nil { + return + } + for _, key := range []string{"parallel_tool_calls", "service_tier"} { + v, ok := raw[key] + if !ok || v == nil { + continue + } + if req.ExtraBody != nil { + if _, exists := req.ExtraBody[key]; exists { + continue + } + } else { + req.ExtraBody = map[string]any{} + } + req.ExtraBody[key] = v + } +} + // convertStreamChunkWithUsage 转换流式 chunk,并在同一次解析中顺带返回 usage。 // 注意:流循环(chat.go 的 stream 处理)仍会为流统计单独解析一次 chunk; // 这里的 "顺带提取" 只是免去了 usage 的第三次解析。 @@ -492,6 +517,11 @@ func chatCompletionsHandler(w http.ResponseWriter, r *http.Request) { http.Error(w, "Invalid JSON", http.StatusBadRequest) return } + // OpenAIRequest 类型化结构装不下的顶层直通字段(parallel_tool_calls / + // service_tier)从原始 body 回填进 ExtraBody,供 chat→responses 上游体 + // 透传(对齐 sub2api ChatCompletionsToResponses)。只补缺,不覆盖客户端 + // 已显式放在 extra_body 内的同名键。 + preserveChatPassthroughKeys(body, &req) modelIn := req.Model req.Model = resolveModelForAuth(auth, req.Model) if req.Model == "" { diff --git a/internal/app/chat_protocol.go b/internal/app/chat_protocol.go index 8415949..0f5e52c 100644 --- a/internal/app/chat_protocol.go +++ b/internal/app/chat_protocol.go @@ -10,7 +10,9 @@ import ( ) // normalizeFinishReason maps Anthropic stop reasons onto the closed set used -// by Chat Completions. +// by Chat Completions. Unknown reasons fall back to "stop":Chat +// finish_reason 是闭集合,透传上游新枚举(如 pause_turn)会污染下游(对齐 +// sub2api 各映射器的闭集合输出,无透传分支)。 func normalizeFinishReason(reason string) string { switch reason { case "end_turn", "stop_sequence", "stop": @@ -22,7 +24,7 @@ func normalizeFinishReason(reason string) string { case "refusal", "content_filter": return "content_filter" default: - return reason + return "stop" } } @@ -51,8 +53,8 @@ func anthropicUsageToChat(usage map[string]any) map[string]any { } } // Anthropic 缓存读/写 token 顶层键透传,并同时归入 chat 约定位置 - // prompt_tokens_details.cached_tokens(对齐 sub2api 对 Chat Completions - // usage 的形状;原有顶层键透传保留,不改已有调用方行为)。 + // prompt_tokens_details.cached_tokens / .cache_creation_tokens(对齐 sub2api + // 对 Chat Completions usage 的形状;原有顶层键透传保留,不改已有调用方行为)。 if v, ok := numberAsFloat(usage["cache_read_input_tokens"]); ok { details, _ := out["prompt_tokens_details"].(map[string]any) if details == nil { @@ -63,6 +65,18 @@ func anthropicUsageToChat(usage map[string]any) map[string]any { } out["prompt_tokens_details"] = details } + // cache_creation_input_tokens 同样归位到 details(对齐 sub2api + // promptDetailsFromResponses),否则按 OpenAI 形状计费/统计时写缓存 token 丢失。 + if v, ok := numberAsFloat(usage["cache_creation_input_tokens"]); ok { + details, _ := out["prompt_tokens_details"].(map[string]any) + if details == nil { + details = map[string]any{} + } + if existing, eok := numberAsFloat(details["cache_creation_tokens"]); !eok || existing == 0 { + details["cache_creation_tokens"] = v + } + out["prompt_tokens_details"] = details + } // 上游发 output_tokens_details.thinking_tokens 时归位到 chat 的 // completion_tokens_details.reasoning_tokens。 if outDetails, ok := usage["output_tokens_details"].(map[string]any); ok { diff --git a/internal/app/chat_to_anthropic.go b/internal/app/chat_to_anthropic.go index a5e0969..74d748c 100644 --- a/internal/app/chat_to_anthropic.go +++ b/internal/app/chat_to_anthropic.go @@ -373,11 +373,26 @@ func chatToAnthropicBodyWithRaw(req *OpenAIRequest, modelID string, rawBody map[ } maxTokens := resolveMaxTokens(rawBody, req, modelID) body["max_tokens"] = maxTokens - if req.Temperature != nil { - body["temperature"] = *req.Temperature + // thinking 与采样参数互斥(对齐 sub2api 与本项目 convertClaudeRequest + // anthropic_protocol.go:119-144):thinking 生效(budget>0 将写入)时剥离 + // temperature/top_p,避免上游 Anthropic 400。判定先行,写入分支据此跳过。 + thinkingBudget := 0 + if !config.ForceDisableThinking() && !isThinkingDisabled(req.Thinking) { + effort := req.ReasoningEffort + if effort == "" { + effort = reasoningEffortFromThinking(req.Thinking) + } + if effort != "" && effort != "none" { + thinkingBudget = effortToThinkingBudget(mappedReasoningEffort(effort)) + } } - if req.TopP != nil { - body["top_p"] = *req.TopP + if thinkingBudget <= 0 { + if req.Temperature != nil { + body["temperature"] = *req.Temperature + } + if req.TopP != nil { + body["top_p"] = *req.TopP + } } if stop := extraBodyValue(req, "stop"); stop != nil { if arr, ok := stop.([]any); ok { @@ -417,18 +432,10 @@ func chatToAnthropicBodyWithRaw(req *OpenAIRequest, modelID string, rawBody map[ body["tool_choice"] = choice } } - // thinking:effort 映射为预算;ForceDisableThinking 或客户端显式禁用则省略。 - if !config.ForceDisableThinking() && !isThinkingDisabled(req.Thinking) { - effort := req.ReasoningEffort - if effort == "" { - effort = reasoningEffortFromThinking(req.Thinking) - } - if effort != "" && effort != "none" { - effort = mappedReasoningEffort(effort) - if budget := effortToThinkingBudget(effort); budget > 0 { - body["thinking"] = map[string]any{"type": "enabled", "budget_tokens": budget} - } - } + // thinking 写入:预算已在上方预计算(thinkingBudget),与采样参数剥离 + // 用同一判定,避免两处推导不一致。 + if thinkingBudget > 0 { + body["thinking"] = map[string]any{"type": "enabled", "budget_tokens": thinkingBudget} } b, err := json.Marshal(body) if err != nil { diff --git a/internal/app/chat_to_responses_upstream.go b/internal/app/chat_to_responses_upstream.go index d43bb6b..eb00952 100644 --- a/internal/app/chat_to_responses_upstream.go +++ b/internal/app/chat_to_responses_upstream.go @@ -138,6 +138,31 @@ func chatToResponsesBodyWithRaw(req *OpenAIRequest, modelID string, rawBody map[ if instructions != "" { body["instructions"] = instructions } + // parallel_tool_calls / service_tier 透传到 Responses 上游(对齐 sub2api + // ChatCompletionsToResponses:ParallelToolCalls/ServiceTier)。 + // rawBody(客户端原始顶层)优先,ExtraBody 次之;缺省不写,不改变上游默认。 + if v, ok := rawBody["parallel_tool_calls"]; ok { + body["parallel_tool_calls"] = v + } else if req.ExtraBody != nil { + if v, ok := req.ExtraBody["parallel_tool_calls"]; ok { + body["parallel_tool_calls"] = v + } + } + if v, ok := rawBody["service_tier"].(string); ok && v != "" { + body["service_tier"] = v + } else if req.ExtraBody != nil { + if v, ok := req.ExtraBody["service_tier"].(string); ok && v != "" { + body["service_tier"] = v + } + } + // response_format(json_schema/json_object)映射为 Responses text.format + // (对齐 sub2api chatResponseFormatToResponsesTextFormat);非 json 形态 + // (如 json_object 无 schema)透传 type 由上游按 Responses 语义解释。 + if req.ResponseFormat != nil { + if format := chatResponseFormatToResponsesTextFormat(req.ResponseFormat); len(format) > 0 { + body["text"] = map[string]any{"format": format} + } + } if req.Stream { body["stream_options"] = map[string]any{"include_usage": true} } @@ -211,6 +236,42 @@ func chatToResponsesBodyWithRaw(req *OpenAIRequest, modelID string, rawBody map[ return b } +// chatResponseFormatToResponsesTextFormat 把 Chat response_format 映射为 +// Responses text.format(对齐 sub2api chatResponseFormatToResponsesTextFormat): +// json_schema 形态展平为 Responses 侧 {type:"json_schema",name,schema,...}; +// json_object / text 等其它形态原样透传(由上游按 Responses 语义解释)。 +// 返回 nil 表示无可用格式(调用方省略 text 字段)。 +func chatResponseFormatToResponsesTextFormat(raw any) map[string]any { + if raw == nil { + return nil + } + b, err := json.Marshal(raw) + if err != nil || len(bytes.TrimSpace(b)) == 0 || string(bytes.TrimSpace(b)) == "null" { + return nil + } + var obj map[string]any + if err := json.Unmarshal(b, &obj); err != nil { + return nil + } + if typ, _ := obj["type"].(string); typ != "json_schema" { + return map[string]any{"type": obj["type"]} + } + schemaRaw, ok := obj["json_schema"] + if !ok { + return map[string]any{"type": obj["type"]} + } + var schema map[string]any + sb, err := json.Marshal(schemaRaw) + if err != nil { + return map[string]any{"type": obj["type"]} + } + if err := json.Unmarshal(sb, &schema); err != nil { + return map[string]any{"type": obj["type"]} + } + schema["type"] = "json_schema" + return schema +} + // responsesReasoningBody 构造上行 Responses 请求体的 reasoning 字段。 // 只带 effort 时上游(muse-spark 系实测如此)静默思考、summary 恒为空数组, // 可见思考必须有 reasoning.summary;上行该字段只负责"请求 summary",敏感 @@ -368,7 +429,7 @@ func forwardChatViaResponses(w http.ResponseWriter, r *http.Request, auth Upstre // 是 JSON、空体或流中带 error 事件)时原样返回,交给上层既有处理(含 // convertResponsesToChat 的 JSON 解析),因此幂等。 func aggregateResponsesStreamToChat(body []byte, model string, wantReasoning bool) []byte { - var id, outModel string + var id, outModel, serviceTier string var contentBuilder, reasoningBuilder strings.Builder var refusal string type toolAcc struct { @@ -460,7 +521,7 @@ func aggregateResponsesStreamToChat(body []byte, model string, wantReasoning boo continue } it, _ := item["type"].(string) - if it != "function_call" && it != "tool_call" { + if it != "function_call" && it != "tool_call" && it != "custom_tool_call" { continue } callID, _ := item["call_id"].(string) @@ -478,25 +539,32 @@ func aggregateResponsesStreamToChat(body []byte, model string, wantReasoning boo if args, _ := item["arguments"].(string); args != "" && acc.args == "" { acc.args = args } - case "response.function_call_arguments.delta", "response.tool_call_arguments.delta": + case "response.function_call_arguments.delta", "response.tool_call_arguments.delta", "response.custom_tool_call_input.delta": oi, _ := evt["output_index"].(float64) itemID, _ := evt["item_id"].(string) acc := ensureTool(itemID, formatOutputIndex(oi)) if pj, _ := evt["delta"].(string); pj != "" { acc.args += pj } - case "response.function_call_arguments.done", "response.tool_call_arguments.done": + case "response.function_call_arguments.done", "response.tool_call_arguments.done", "response.custom_tool_call_input.done": oi, _ := evt["output_index"].(float64) itemID, _ := evt["item_id"].(string) acc := ensureTool(itemID, formatOutputIndex(oi)) - if completed, _ := evt["arguments"].(string); completed != "" { + completed, _ := evt["arguments"].(string) + if completed == "" { + completed, _ = evt["input"].(string) + } + if completed != "" { acc.args = completed } - case "response.completed", "response.incomplete": + case "response.completed", "response.incomplete", "response.done": if resp, ok := evt["response"].(map[string]any); ok { if u, ok := resp["usage"].(map[string]any); ok { usage = u } + if tier, _ := resp["service_tier"].(string); tier != "" { + serviceTier = tier + } if rid, _ := resp["id"].(string); rid != "" { id = rid } @@ -504,7 +572,15 @@ func aggregateResponsesStreamToChat(body []byte, model string, wantReasoning boo outModel = m } if status, _ := resp["status"].(string); status == "incomplete" { - finishReason = "length" + reason := "" + if details, ok := resp["incomplete_details"].(map[string]any); ok { + reason, _ = details["reason"].(string) + } + if reason == "content_filter" { + finishReason = "content_filter" + } else { + finishReason = "length" + } } } } @@ -569,6 +645,11 @@ func aggregateResponsesStreamToChat(body []byte, model string, wantReasoning boo "index": 0, "message": msg, "finish_reason": finishReason, }}, } + // 上游计费层级透传到 chat 顶层(对齐流式 emitChunk 与 sub2api + // ChatCompletionsChunk.ServiceTier)。 + if serviceTier != "" { + resp["service_tier"] = serviceTier + } if usage != nil { resp["usage"] = responsesUsageToChatBridge(usage) } @@ -611,7 +692,17 @@ func convertResponsesToChat(respBody []byte, model string, wantReasoning bool) [ var toolCalls []map[string]any finishReason := "stop" if status, _ := raw["status"].(string); status == "incomplete" { - finishReason = "length" + // incomplete 原因细分(对齐流式/聚合两条路径与 sub2api): + // max_output_tokens → length,content_filter → content_filter。 + reason := "" + if details, ok := raw["incomplete_details"].(map[string]any); ok { + reason, _ = details["reason"].(string) + } + if reason == "content_filter" { + finishReason = "content_filter" + } else { + finishReason = "length" + } } output, _ := raw["output"].([]any) for _, itemRaw := range output { @@ -650,13 +741,17 @@ func convertResponsesToChat(respBody []byte, model string, wantReasoning bool) [ } } } - case "function_call", "tool_call": + case "function_call", "tool_call", "custom_tool_call": callID, _ := item["call_id"].(string) if callID == "" { callID, _ = item["id"].(string) } name, _ := item["name"].(string) + // custom_tool_call 的完整输入在 "input" 键(对齐流式 done 路径)。 args, _ := item["arguments"].(string) + if args == "" { + args, _ = item["input"].(string) + } toolCalls = append(toolCalls, map[string]any{ "id": callID, "type": "function", @@ -695,6 +790,10 @@ func convertResponsesToChat(respBody []byte, model string, wantReasoning bool) [ "index": 0, "message": msg, "finish_reason": finishReason, }}, } + // 上游计费层级透传到 chat 顶层(对齐流式/聚合路径与 sub2api)。 + if tier, _ := raw["service_tier"].(string); tier != "" { + resp["service_tier"] = tier + } if u, ok := raw["usage"]; ok && u != nil { resp["usage"] = responsesUsageToChatBridge(u.(map[string]any)) } @@ -785,6 +884,7 @@ type responsesToChatState struct { stats *logging.StreamStats id string model string + serviceTier string // 上游 response.service_tier,回写 chat chunk 顶层(对齐 sub2api) keepReasoning bool includeUsage bool sentRole bool @@ -959,6 +1059,11 @@ func (st *responsesToChatState) emitChunk(delta map[string]any, finishReason str "finish_reason": finishReasonOr(finishReason), }}, } + // 上游 service_tier 透传到 chat chunk 顶层(对齐 sub2api + // makeChatDeltaChunk/ChatCompletionsChunk.ServiceTier)。 + if st.serviceTier != "" { + chunk["service_tier"] = st.serviceTier + } if usage != nil { chunk["usage"] = usage } @@ -1073,6 +1178,11 @@ func (st *responsesToChatState) handleLine(line string) { if m, _ := resp["model"].(string); m != "" { st.model = m } + // 上游计费层级:后续所有 chunk 顶层回写(对齐 sub2api + // resToChatHandleCompleted 的 ServiceTier 状态)。 + if tier, _ := resp["service_tier"].(string); tier != "" { + st.serviceTier = tier + } if u, ok := resp["usage"].(map[string]any); ok { mergeUsage(st.fullUsage, u) } @@ -1105,8 +1215,11 @@ func (st *responsesToChatState) handleLine(line string) { if item == nil { return } + // custom_tool_call(custom/freeform 工具,如新版 apply_patch)的参数 + // 增量(response.custom_tool_call_input.delta)与 function_call 同形, + // 按同一工具槽位注册(对齐 sub2api resToChatHandleOutputItemAdded)。 switch item["type"] { - case "function_call", "tool_call": + case "function_call", "tool_call", "custom_tool_call": st.sawTool = true callID, toolIdx := st.registerToolKeys(evt, item) st.toolAnnounced[toolIdx] = true @@ -1116,7 +1229,7 @@ func (st *responsesToChatState) handleLine(line string) { "function": map[string]any{"name": restoreToolNameCase(toString(item["name"])), "arguments": ""}, }}}, "", nil) } - case "response.function_call_arguments.delta", "response.tool_call_arguments.delta": + case "response.function_call_arguments.delta", "response.tool_call_arguments.delta", "response.custom_tool_call_input.delta": st.ensureRole() toolIdx := st.eventToolIdx(evt) if pj, _ := evt["delta"].(string); pj != "" { @@ -1126,12 +1239,17 @@ func (st *responsesToChatState) handleLine(line string) { "function": map[string]any{"name": "", "arguments": pj}, }}}, "", nil) } - case "response.function_call_arguments.done", "response.tool_call_arguments.done": + case "response.function_call_arguments.done", "response.tool_call_arguments.done", "response.custom_tool_call_input.done": st.ensureRole() toolIdx := st.eventToolIdx(evt) // done 携带完整 arguments JSON:只补发已下发前缀之后的差量, // 避免客户端 concat 后重复(对齐 sub2api resToChatHandleFuncArgsDone)。 - if completed, _ := evt["arguments"].(string); completed != "" { + // custom_tool_call_input.done 的完整输入在 "input" 键而非 "arguments"。 + completed, _ := evt["arguments"].(string) + if completed == "" { + completed, _ = evt["input"].(string) + } + if completed != "" { emitted := st.arguments[toolIdx] if completed != emitted && strings.HasPrefix(completed, emitted) { remainder := completed[len(emitted):] @@ -1148,7 +1266,7 @@ func (st *responsesToChatState) handleLine(line string) { return } switch item["type"] { - case "function_call", "tool_call": + case "function_call", "tool_call", "custom_tool_call": st.sawTool = true callID, toolIdx := st.registerToolKeys(evt, item) // 没有 add/delta 出现过(罕见):补一次首 chunk 宣告工具调用, @@ -1170,13 +1288,27 @@ func (st *responsesToChatState) handleLine(line string) { } } } - case "response.completed", "response.incomplete": + case "response.completed", "response.incomplete", "response.done": if resp, ok := evt["response"].(map[string]any); ok { if u, ok := resp["usage"].(map[string]any); ok { mergeUsage(st.fullUsage, u) } + // response.done(Realtime/WS 别名)同样可能携带 usage/service_tier。 + if tier, _ := resp["service_tier"].(string); tier != "" { + st.serviceTier = tier + } if status, _ := resp["status"].(string); status == "incomplete" { - st.finishReason = "length" + // incomplete 原因细分(对齐 sub2api resToChatHandleCompleted): + // max_output_tokens → length,content_filter → content_filter。 + reason := "" + if details, ok := resp["incomplete_details"].(map[string]any); ok { + reason, _ = details["reason"].(string) + } + if reason == "content_filter" { + st.finishReason = "content_filter" + } else { + st.finishReason = "length" + } } else if st.sawTool { st.finishReason = "tool_calls" } else { @@ -1184,6 +1316,7 @@ func (st *responsesToChatState) handleLine(line string) { } } st.finalize() + // response.failed 走错误分支:上游错误体在 evt["response"]["error"]。 case "response.failed", "error": em, _ := evt["response"].(map[string]any) message := "upstream error" diff --git a/internal/app/claude.go b/internal/app/claude.go index cfc332d..b444097 100644 --- a/internal/app/claude.go +++ b/internal/app/claude.go @@ -794,12 +794,24 @@ func openAIToClaudeResponse(chatBody []byte, model string, wantReasoning bool) [ switch fr { case "stop": stopReason = "end_turn" + // block 存在性回退:finish_reason 缺失/错配但 content 已含 tool_use + // 时仍报 tool_use,否则客户端不回传工具结果、对话卡死(对齐 sub2api + // chatFinishReasonToAnthropicStopReason 的 default 分支)。 + for _, c := range content { + if c.Type == "tool_use" { + stopReason = "tool_use" + break + } + } case "length": stopReason = "max_tokens" case "tool_calls", "function_call": stopReason = "tool_use" case "content_filter": - stopReason = "refusal" + // refusal 不在 Anthropic stop_reason 枚举内(end_turn/max_tokens/ + // stop_sequence/tool_use),严格客户端会拒收;拒绝文本已进 message + // 内容,终态按完成归一(对齐 sub2api responsesStatusToAnthropicStopReason)。 + stopReason = "end_turn" } } diff --git a/internal/app/claude_responses.go b/internal/app/claude_responses.go index 2dad13a..2c2a39e 100644 --- a/internal/app/claude_responses.go +++ b/internal/app/claude_responses.go @@ -843,7 +843,9 @@ func responsesOutputToClaudeBlocks(output []any, wantReasoning bool) ([]ClaudeCo } if refusalText != "" && !hasToolUse { - stopReason = "refusal" + // 同 claude.go:refusal 不在 Anthropic stop_reason 枚举内,终态按 + // 完成归一(对齐 sub2api)。 + stopReason = "end_turn" } else if hasToolUse { stopReason = "tool_use" } diff --git a/internal/app/protocol_parity_sub2api_test.go b/internal/app/protocol_parity_sub2api_test.go new file mode 100644 index 0000000..10653b8 --- /dev/null +++ b/internal/app/protocol_parity_sub2api_test.go @@ -0,0 +1,308 @@ +package app + +import ( + "context" + "encoding/json" + "net/http" + "strings" + "testing" +) + +// TestPreserveChatPassthroughKeys 顶层 parallel_tool_calls/service_tier +// 必须从原始 body 回填进 ExtraBody(类型化结构装不下),extra_body 显式键优先。 +func TestPreserveChatPassthroughKeys(t *testing.T) { + req := &OpenAIRequest{Model: "gpt-x"} + preserveChatPassthroughKeys([]byte(`{"model":"gpt-x","parallel_tool_calls":false,"service_tier":"flex"}`), req) + if req.ExtraBody["parallel_tool_calls"] != false { + t.Fatalf("ExtraBody = %#v", req.ExtraBody) + } + if req.ExtraBody["service_tier"] != "flex" { + t.Fatalf("ExtraBody = %#v", req.ExtraBody) + } + // extra_body 显式键优先,不被顶层覆盖。 + req2 := &OpenAIRequest{Model: "gpt-x", ExtraBody: map[string]any{"service_tier": "auto"}} + preserveChatPassthroughKeys([]byte(`{"model":"gpt-x","service_tier":"flex"}`), req2) + if req2.ExtraBody["service_tier"] != "auto" { + t.Fatalf("ExtraBody = %#v", req2.ExtraBody) + } + // 无键时不建 ExtraBody。 + req3 := &OpenAIRequest{Model: "gpt-x"} + preserveChatPassthroughKeys([]byte(`{"model":"gpt-x"}`), req3) + if req3.ExtraBody != nil { + t.Fatalf("ExtraBody = %#v, want nil", req3.ExtraBody) + } +} + +// TestChatToResponsesBody_ParityFields chat→responses 请求体必须透传 +// parallel_tool_calls / service_tier,并把 response_format 映射为 +// text.format(对齐 sub2api ChatCompletionsToResponses)。 +func TestChatToResponsesBody_ParityFields(t *testing.T) { + mt := 256 + req := &OpenAIRequest{ + Model: "gpt-x", + Messages: []Message{{Role: "user", Content: "hi"}}, + MaxTokens: &mt, + ResponseFormat: map[string]any{ + "type": "json_schema", + "json_schema": map[string]any{ + "name": "ans", + "schema": map[string]any{"type": "object"}, + }, + }, + ExtraBody: map[string]any{ + "parallel_tool_calls": false, + "service_tier": "flex", + }, + } + body := chatToResponsesBodyWithRaw(req, "gpt-x", map[string]any{}) + var got map[string]any + if err := json.Unmarshal(body, &got); err != nil { + t.Fatal(err) + } + if v, _ := got["parallel_tool_calls"].(bool); v != false { + t.Fatalf("parallel_tool_calls = %#v, want false passthrough", got["parallel_tool_calls"]) + } + if got["service_tier"] != "flex" { + t.Fatalf("service_tier = %#v, want flex", got["service_tier"]) + } + text, _ := got["text"].(map[string]any) + format, _ := text["format"].(map[string]any) + if format["type"] != "json_schema" { + t.Fatalf("text.format = %#v, want json_schema", text) + } + schema, _ := format["schema"].(map[string]any) + if schema["type"] != "object" { + t.Fatalf("text.format.schema = %#v", format) + } +} + +// TestChatToAnthropicBody_ThinkingStripsSampling thinking 生效时必须剥离 +// temperature/top_p(对齐 sub2api 与本项目 convertClaudeRequest),否则上游 +// Anthropic 400。thinking 关闭时采样参数保留。 +func TestChatToAnthropicBody_ThinkingStripsSampling(t *testing.T) { + temp, topP := 0.7, 0.9 + req := &OpenAIRequest{ + Model: "claude-x", Messages: []Message{{Role: "user", Content: "hi"}}, + Temperature: &temp, TopP: &topP, + ReasoningEffort: "high", + } + var got map[string]any + if err := json.Unmarshal(chatToAnthropicBody(req, "claude-x"), &got); err != nil { + t.Fatal(err) + } + if _, exists := got["thinking"]; !exists { + t.Fatal("want thinking injected") + } + if _, exists := got["temperature"]; exists { + t.Fatalf("temperature must be stripped when thinking on: %#v", got["temperature"]) + } + if _, exists := got["top_p"]; exists { + t.Fatalf("top_p must be stripped when thinking on: %#v", got["top_p"]) + } + + plain := &OpenAIRequest{ + Model: "claude-x", Messages: []Message{{Role: "user", Content: "hi"}}, + Temperature: &temp, TopP: &topP, + } + var got2 map[string]any + if err := json.Unmarshal(chatToAnthropicBody(plain, "claude-x"), &got2); err != nil { + t.Fatal(err) + } + if got2["temperature"] != temp || got2["top_p"] != topP { + t.Fatalf("sampling params must be kept when thinking off: %#v", got2) + } +} + +// TestAnthropicUsageToChat_CacheCreation 写缓存 token 必须归位到 +// prompt_tokens_details.cache_creation_tokens(对齐 sub2api +// promptDetailsFromResponses),否则 OpenAI 形状下游丢失该分量。 +func TestAnthropicUsageToChat_CacheCreation(t *testing.T) { + out := anthropicUsageToChat(map[string]any{ + "input_tokens": float64(100), "output_tokens": float64(20), + "cache_read_input_tokens": float64(30), "cache_creation_input_tokens": float64(10), + }) + details, _ := out["prompt_tokens_details"].(map[string]any) + if details["cached_tokens"] != float64(30) { + t.Fatalf("cached_tokens = %#v", details) + } + if details["cache_creation_tokens"] != float64(10) { + t.Fatalf("cache_creation_tokens = %#v", details) + } +} + +// TestNormalizeFinishReason_UnknownClosed 未知 stop_reason 必须闭集合 +// 回退为 stop(对齐 sub2api),不得透传污染 finish_reason。 +func TestNormalizeFinishReason_UnknownClosed(t *testing.T) { + if normalizeFinishReason("pause_turn") != "stop" { + t.Fatalf("pause_turn → %q, want stop", normalizeFinishReason("pause_turn")) + } + if normalizeFinishReason("model_context_window_exceeded") != "stop" { + t.Fatal("unknown reason must close to stop") + } + if normalizeFinishReason("tool_use") != "tool_calls" { + t.Fatal("known mapping must be stable") + } +} + +// TestChatResponseFormatToResponsesTextFormat_JsonObject 非 schema 形态 +// (json_object)透传 type,不伪造 schema。 +func TestChatResponseFormatToResponsesTextFormat_JsonObject(t *testing.T) { + got := chatResponseFormatToResponsesTextFormat(map[string]any{"type": "json_object"}) + if got["type"] != "json_object" { + t.Fatalf("got = %#v", got) + } + if chatResponseFormatToResponsesTextFormat(nil) != nil { + t.Fatal("nil input must yield nil") + } +} + +// TestResponsesSSEToChatStream_CustomToolCall custom/freeform 工具的 +// input 增量与 done 必须落到同一 tool_calls index(对齐 sub2api +// resToChatHandleOutputItemAdded/custom_tool_call 分支)。 +func TestResponsesSSEToChatStream_CustomToolCall(t *testing.T) { + sse := "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_c\",\"model\":\"gpt-x\"}}\n\n" + + "event: response.output_item.added\ndata: {\"type\":\"response.output_item.added\",\"output_index\":0,\"item\":{\"type\":\"custom_tool_call\",\"id\":\"ct_1\",\"call_id\":\"call_c1\",\"name\":\"apply_patch\"}}\n\n" + + "event: response.custom_tool_call_input.delta\ndata: {\"type\":\"response.custom_tool_call_input.delta\",\"output_index\":0,\"item_id\":\"ct_1\",\"delta\":\"{\\\"op\\\":\"}}\n\n" + + "event: response.custom_tool_call_input.done\ndata: {\"type\":\"response.custom_tool_call_input.done\",\"output_index\":0,\"item_id\":\"ct_1\",\"input\":\"{\\\"op\\\":\\\"create\\\"}\"}\n\n" + + "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp_c\"}}\n\n" + body := drainSSEFromHandler(func(w http.ResponseWriter) { + committed, err := responsesSSEToChatStream(context.Background(), w, strings.NewReader(sse), "gpt-x", false, false, nil, nil) + if err != nil || !committed { + t.Fatalf("responsesSSEToChatStream = (%v, %v), want (true, nil)", committed, err) + } + }) + names := map[int]string{} + args := map[int]string{} + for _, line := range strings.Split(body, "\n") { + payload, ok := strings.CutPrefix(line, "data: ") + if !ok || payload == "[DONE]" { + continue + } + var chunk struct { + Choices []struct { + Delta struct { + ToolCalls []struct { + Index int `json:"index"` + Function struct { + Name string `json:"name"` + Arguments string `json:"arguments"` + } `json:"function"` + } `json:"tool_calls"` + } `json:"delta"` + } `json:"choices"` + } + if err := json.Unmarshal([]byte(payload), &chunk); err != nil { + t.Fatalf("chunk is not JSON: %q", payload) + } + for _, c := range chunk.Choices { + for _, tc := range c.Delta.ToolCalls { + names[tc.Index] += tc.Function.Name + args[tc.Index] += tc.Function.Arguments + } + } + } + if len(names) != 1 || names[0] != "apply_patch" { + t.Fatalf("names = %v, body: %s", names, body) + } + // delta 前缀 + done 差量补齐 == 完整 input,不重复。 + if args[0] != `{"op":"create"}` { + t.Fatalf("args[0] = %q, want full input without duplication, body: %s", args[0], body) + } +} + +// TestResponsesSSEToChatStream_IncompleteContentFilter incomplete + +// content_filter 原因必须映射为 finish_reason content_filter(对齐 sub2api +// resToChatHandleCompleted),而非一律 length。 +func TestResponsesSSEToChatStream_IncompleteContentFilter(t *testing.T) { + sse := "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_f\",\"model\":\"gpt-x\"}}\n\n" + + "event: response.incomplete\ndata: {\"type\":\"response.incomplete\",\"response\":{\"id\":\"resp_f\",\"status\":\"incomplete\",\"incomplete_details\":{\"reason\":\"content_filter\"}}}\n\n" + body := drainSSEFromHandler(func(w http.ResponseWriter) { + committed, err := responsesSSEToChatStream(context.Background(), w, strings.NewReader(sse), "gpt-x", false, false, nil, nil) + if err != nil || !committed { + t.Fatalf("responsesSSEToChatStream = (%v, %v), want (true, nil)", committed, err) + } + }) + if !strings.Contains(body, `"finish_reason":"content_filter"`) { + t.Fatalf("want content_filter finish reason, body: %s", body) + } +} + +// TestResponsesSSEToChatStream_ResponseDoneAlias response.done +// (Realtime/WS 别名)必须像 completed 一样终结流并补 [DONE](对齐 sub2api)。 +func TestResponsesSSEToChatStream_ResponseDoneAlias(t *testing.T) { + sse := "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_d\",\"model\":\"gpt-x\"}}\n\n" + + "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"delta\":\"hi\"}\n\n" + + "event: response.done\ndata: {\"type\":\"response.done\",\"response\":{\"id\":\"resp_d\",\"status\":\"completed\"}}\n\n" + body := drainSSEFromHandler(func(w http.ResponseWriter) { + committed, err := responsesSSEToChatStream(context.Background(), w, strings.NewReader(sse), "gpt-x", false, false, nil, nil) + if err != nil || !committed { + t.Fatalf("responsesSSEToChatStream = (%v, %v), want (true, nil)", committed, err) + } + }) + if !strings.Contains(body, "data: [DONE]") { + t.Fatalf("want [DONE] sentinel, body: %s", body) + } + if !strings.Contains(body, `"finish_reason":"stop"`) { + t.Fatalf("want stop finish chunk, body: %s", body) + } +} + +// TestResponsesSSEToChatStream_ServiceTierChunk 上游 service_tier 必须 +// 回写到 chat chunk 顶层(对齐 sub2api ChatCompletionsChunk.ServiceTier)。 +func TestResponsesSSEToChatStream_ServiceTierChunk(t *testing.T) { + sse := "event: response.created\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_s\",\"model\":\"gpt-x\",\"service_tier\":\"flex\"}}\n\n" + + "event: response.output_text.delta\ndata: {\"type\":\"response.output_text.delta\",\"delta\":\"hi\"}\n\n" + + "event: response.completed\ndata: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp_s\"}}\n\n" + body := drainSSEFromHandler(func(w http.ResponseWriter) { + committed, err := responsesSSEToChatStream(context.Background(), w, strings.NewReader(sse), "gpt-x", false, false, nil, nil) + if err != nil || !committed { + t.Fatalf("responsesSSEToChatStream = (%v, %v), want (true, nil)", committed, err) + } + }) + if !strings.Contains(body, `"service_tier":"flex"`) { + t.Fatalf("want service_tier passthrough, body: %s", body) + } +} + +// TestConvertResponsesToChat_IncompleteContentFilter 非流式路径同样细分 +// incomplete 原因。 +func TestConvertResponsesToChat_IncompleteContentFilter(t *testing.T) { + resp := `{"id":"resp_9","model":"gpt-x","status":"incomplete", + "incomplete_details":{"reason":"content_filter"}, + "output":[{"type":"message","id":"m1","content":[{"type":"output_text","text":"x"}]}]}` + out := convertResponsesToChat([]byte(resp), "gpt-x", false) + var got map[string]any + if err := json.Unmarshal(out, &got); err != nil { + t.Fatal(err) + } + choices := got["choices"].([]any) + fr := choices[0].(map[string]any)["finish_reason"] + if fr != "content_filter" { + t.Fatalf("finish_reason = %#v, want content_filter", fr) + } +} + +// TestConvertResponsesToChat_CustomToolCall 非流式 function_call 外的 +// custom_tool_call 必须转为 chat tool_calls(读 input 键)。 +func TestConvertResponsesToChat_CustomToolCall(t *testing.T) { + resp := `{"id":"resp_8","model":"gpt-x","status":"completed", + "output":[{"type":"custom_tool_call","id":"ct_1","call_id":"call_c1","name":"apply_patch","input":"{\"op\":\"create\"}"}]}` + out := convertResponsesToChat([]byte(resp), "gpt-x", false) + var got map[string]any + if err := json.Unmarshal(out, &got); err != nil { + t.Fatal(err) + } + choices := got["choices"].([]any) + msg := choices[0].(map[string]any)["message"].(map[string]any) + tcs, _ := msg["tool_calls"].([]any) + if len(tcs) != 1 { + t.Fatalf("tool_calls = %#v", msg["tool_calls"]) + } + fn := tcs[0].(map[string]any)["function"].(map[string]any) + if fn["name"] != "apply_patch" || fn["arguments"] != `{"op":"create"}` { + t.Fatalf("function = %#v", fn) + } + if choices[0].(map[string]any)["finish_reason"] != "tool_calls" { + t.Fatalf("finish = %#v", choices[0].(map[string]any)["finish_reason"]) + } +} From 339ff36cb2b946e6cafe5d86ce43e9cb140f446c Mon Sep 17 00:00:00 2001 From: 6Kmfi6HP <179412085+6Kmfi6HP@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:01:08 +0200 Subject: [PATCH 2/4] =?UTF-8?q?fix(keypool):=20sticky=20=E6=8C=89=E5=AE=8C?= =?UTF-8?q?=E6=95=B4=E4=BC=9A=E8=AF=9D=E9=94=AE=E6=95=A3=E5=88=97=EF=BC=8C?= =?UTF-8?q?=E4=BC=9A=E8=AF=9D=E7=BA=A7=E8=B4=9F=E8=BD=BD=E5=9D=87=E8=A1=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 同一客户端 token 下所有请求曾恒选同一池 key(仅哈希 token 部分), 1811 把 key 实际只有 k36 在服务。现 sticky 复用完整 egress sticky 键 (token + 客户端会话后缀),同一会话仍粘定同 key(缓存亲和),不同 会话按哈希散开;failover 重试混入常量后缀跳离故障 key。 Co-Authored-By: Claude Code --- docs/keypool-design.md | 8 ++++-- internal/app/auth_test.go | 2 +- internal/app/keypool.go | 39 ++++++++++++++++++------- internal/app/keypool_test.go | 56 +++++++++++++++++++++++++++--------- internal/app/opencode.go | 2 +- 5 files changed, 80 insertions(+), 27 deletions(-) diff --git a/docs/keypool-design.md b/docs/keypool-design.md index d5975d3..85eee39 100644 --- a/docs/keypool-design.md +++ b/docs/keypool-design.md @@ -36,13 +36,17 @@ ## 3. 选择器(`internal/app/keypool.go`,内存状态,仿 `socks5RRIndex` 原子模式) * `keyRRIndex atomic.Uint64`:`round_robin` 取 `Add % len`;`weighted` 取 `Add % totalWeight` 走权重区间; - `sticky` 取 `fnv32a(stickyKeyForRequest 同源串) % len`(与 egress sticky 同键,保证 prompt cache 亲和)。 + `sticky` 取 `fnv32a(完整 egress sticky 键) % len`:`stickySessionBase(auth, bodyMap, headers, scope)` + = `stickyKeyForRequest` 全键(`tok:<客户端token>|cli:<客户端会话哈希>` 等,与 egress + 完全同源),同一会话同时粘定同一池 key 与同一出口路径(prompt cache 亲和), + 不同会话按哈希散开(会话级负载均衡)。`attempt>0` 的池 failover 重试混入常量 + 后缀 `|pool-retry`,跳离刚失败的 key(重试之间仍粘同一备选 key,保持亲和)。 * 候选过滤:`enabled && group匹配 && now > cooldownUntil`;全冷却 → 放行最早过期的那把(不断服)。 * 状态表(`keypoolMu` 守卫):`{cooldownUntil, consecutiveFails}`;成功清零。 ## 4. Failover(改 `opencode.go:582` 重试循环内两点,不建新子系统) -* 每 attempt:`attemptAuth, keyID := selectPoolKey(auth, modelID, attempt)`, +* 每 attempt:`attemptAuth, keyID := selectPoolKey(auth, modelID, bodyMap, headers, scope, attempt)`, 以值拷贝传入 `selectUpstreamTarget` + `buildOCRequestWithSubpath` + `invalidateUpstreamTarget` (三处都收 `auth` 值类型,race-free;sticky egress 按池 key 绑定)。 * 记账(`ReportKeyResult`): diff --git a/internal/app/auth_test.go b/internal/app/auth_test.go index f1c398e..14af5c8 100644 --- a/internal/app/auth_test.go +++ b/internal/app/auth_test.go @@ -72,7 +72,7 @@ func TestSelectPoolKey_AdminAutoPicked(t *testing.T) { keypoolMu.Unlock() }() - auth, keyID, pooled := selectPoolKey(UpstreamAuth{Mode: AuthRouteAdmin, Source: "admin"}, "big-pickle", 0) + auth, keyID, pooled := selectPoolKey(UpstreamAuth{Mode: AuthRouteAdmin, Source: "admin"}, "big-pickle", nil, nil, "", 0) if !pooled || keyID != "k1" { t.Fatalf("admin auth should be picked from pool: pooled=%v keyID=%q auth=%+v", pooled, keyID, auth) } diff --git a/internal/app/keypool.go b/internal/app/keypool.go index 0c921d4..aa0c08d 100644 --- a/internal/app/keypool.go +++ b/internal/app/keypool.go @@ -131,13 +131,21 @@ func poolEnabled() bool { return keypoolCfg.Enabled && len(keypoolEntries) > 0 } -// stickySessionBase mirrors stickyKeyForRequest's token part without -// importing body context: account token wins, else the public fallback. -func stickySessionBase(auth UpstreamAuth) string { - if auth.Token != "" { - return "tok:" + auth.Token +// stickySessionBase is the pool's sticky routing key. It reuses the full +// egress sticky key (stickyKeyForRequest: token + client-session suffix), +// so keypool stickiness and egress stickiness share one identity: same +// session pins both the same pooled key and the same egress path (prompt +// cache affinity), while different sessions spread across the pool (load +// balancing). isRetry selects the alternate slot for pool-failover retries +// so a retry hashes away from the just-failed key; the alternate is itself +// sticky (constant suffix) to preserve cache affinity among retries of the +// same request. +func stickySessionBase(auth UpstreamAuth, bodyMap map[string]any, headers http.Header, ocScope string, isRetry bool) string { + key := stickyKeyForRequest(auth, bodyMap, headers, ocScope) + if isRetry { + key += keyPoolRetrySuffix } - return stickyPublicFallback + return key } func keyPoolGroupOK(group string, goSurface bool) bool { @@ -153,11 +161,22 @@ func keyPoolGroupOK(group string, goSurface bool) bool { } } -// selectPoolKey picks a pooled key for this request. +// keyPoolRetrySuffix is the constant sticky-hash salt for pool-failover +// retries; see stickySessionBase. +const keyPoolRetrySuffix = "|pool-retry" + +// selectPoolKey picks a pooled key for this request. scope carries the +// already-derived transport scope (normalizedTransportScope) so sticky +// hashing can reuse it without re-hashing the session; empty scope falls +// back to the token-only form. attempt>0 (pool failover) shifts the sticky +// slot so a retry does not re-land on the just-failed key. // ok=false means caller falls back to client-token passthrough // (pool disabled, public auth, or no surface candidate). -func selectPoolKey(auth UpstreamAuth, modelID string, attempt ...int) (UpstreamAuth, string, bool) { - _ = attempt +func selectPoolKey(auth UpstreamAuth, modelID string, bodyMap map[string]any, headers http.Header, scope string, attempt ...int) (UpstreamAuth, string, bool) { + try := 0 + if len(attempt) > 0 { + try = attempt[0] + } if auth.Mode == AuthRoutePublic { return auth, "", false } @@ -236,7 +255,7 @@ func selectPoolKey(auth UpstreamAuth, modelID string, attempt ...int) (UpstreamA } case "sticky": h := fnv.New32a() - _, _ = h.Write([]byte(stickySessionBase(auth))) + _, _ = h.Write([]byte(stickySessionBase(auth, bodyMap, headers, scope, try > 0))) picked = pool[int(h.Sum32()%uint32(len(pool)))] default: // round_robin picked = pool[int(keypoolRRIndex.Add(1)-1)%len(pool)] diff --git a/internal/app/keypool_test.go b/internal/app/keypool_test.go index 613edd7..91ff0c0 100644 --- a/internal/app/keypool_test.go +++ b/internal/app/keypool_test.go @@ -3,6 +3,7 @@ package app import ( "encoding/json" "errors" + "net/http" "testing" ) @@ -19,7 +20,7 @@ func TestKeyPool_RoundRobin(t *testing.T) { auth := UpstreamAuth{Mode: AuthRouteAuto, Token: "client"} var got []string for range 6 { - _, id, ok := selectPoolKey(auth, "m") + _, id, ok := selectPoolKey(auth, "m", nil, nil, "") if !ok { t.Fatal("want ok") } @@ -32,7 +33,7 @@ func TestKeyPool_RoundRobin(t *testing.T) { } } // Pooled auth replaces client token. - a, _, _ := selectPoolKey(auth, "m") + a, _, _ := selectPoolKey(auth, "m", nil, nil, "") if a.Token == "client" { t.Fatal("pooled auth must replace client token") } @@ -47,7 +48,7 @@ func TestKeyPool_Weighted(t *testing.T) { auth := UpstreamAuth{Mode: AuthRouteAuto, Token: "c"} counts := map[string]int{} for range 8 { - _, id, ok := selectPoolKey(auth, "m") + _, id, ok := selectPoolKey(auth, "m", nil, nil, "") if !ok { t.Fatal("want ok") } @@ -63,17 +64,46 @@ func TestKeyPool_Sticky(t *testing.T) { setKeyPool(KeyPool{Enabled: true, Strategy: "sticky", Keys: []UpstreamKey{{Key: "ka"}, {Key: "kb"}}}) a1 := UpstreamAuth{Mode: AuthRouteAuto, Token: "user1"} a2 := UpstreamAuth{Mode: AuthRouteAuto, Token: "user2"} - _, id1a, _ := selectPoolKey(a1, "m") - _, id1b, _ := selectPoolKey(a1, "m") + _, id1a, _ := selectPoolKey(a1, "m", nil, nil, "") + _, id1b, _ := selectPoolKey(a1, "m", nil, nil, "") if id1a != id1b { t.Fatal("sticky must return same key for same session") } - if got := stickySessionBase(a1); got != "tok:user1" { + // 同一 token、不同客户端会话必须散开(会话级负载均衡)。 + hdr := func(sess string) http.Header { + return http.Header{headerClaudeSession: []string{sess}} + } + seen := map[string]bool{id1a: true} + for _, sess := range []string{"sess-a", "sess-b", "sess-c", "sess-d", "sess-e", "sess-f", "sess-g", "sess-h"} { + _, id, _ := selectPoolKey(a1, "m", nil, hdr(sess), "") + seen[id] = true + } + if len(seen) < 2 { + t.Fatalf("same token with different sessions must spread across pool, got only %v", seen) + } + // 同一会话必须稳定命中同一 key(会话内缓存亲和)。 + _, idA1, _ := selectPoolKey(a1, "m", nil, hdr("sess-a"), "") + _, idA2, _ := selectPoolKey(a1, "m", nil, hdr("sess-a"), "") + if idA1 != idA2 { + t.Fatal("sticky must return same key for same session headers") + } + // failover 重试必须跳离首选 key。 + _, idR, _ := selectPoolKey(a1, "m", nil, hdr("sess-a"), "", 1) + if idR == idA1 { + t.Fatalf("pool retry should hash away from first-attempt key %q", idA1) + } + if got := stickySessionBase(a1, nil, nil, "", false); got != "tok:user1" { t.Fatalf("stickySessionBase = %q", got) } - if got := stickySessionBase(UpstreamAuth{}); got != stickyPublicFallback { + if got, want := stickySessionBase(a1, nil, hdr("sess-a"), "", false), "tok:user1|cli:"+hashSessionRouteKey("sess-a"); got != want { + t.Fatalf("stickySessionBase with session = %q, want %q", got, want) + } + if got := stickySessionBase(UpstreamAuth{}, nil, nil, "", false); got != stickyPublicFallback { t.Fatalf("public fallback = %q", got) } + if got := stickySessionBase(a1, nil, hdr("sess-a"), "", true); got != "tok:user1|cli:"+hashSessionRouteKey("sess-a")+keyPoolRetrySuffix { + t.Fatalf("stickySessionBase retry = %q", got) + } _ = a2 } @@ -83,7 +113,7 @@ func TestKeyPool_Disabled(t *testing.T) { if poolEnabled() { t.Fatal("pool must be disabled") } - if _, _, ok := selectPoolKey(UpstreamAuth{Mode: AuthRouteAuto, Token: "c"}, "m"); ok { + if _, _, ok := selectPoolKey(UpstreamAuth{Mode: AuthRouteAuto, Token: "c"}, "m", nil, nil, ""); ok { t.Fatal("disabled pool must return ok=false") } setKeyPool(KeyPool{Enabled: true}) @@ -98,14 +128,14 @@ func TestKeyPool_CooldownSkip(t *testing.T) { reportKeyResult("a", 429, nil) auth := UpstreamAuth{Mode: AuthRouteAuto, Token: "c"} for range 4 { - _, id, ok := selectPoolKey(auth, "m") + _, id, ok := selectPoolKey(auth, "m", nil, nil, "") if !ok || id != "b" { t.Fatalf("cooling key must be skipped, got %q ok=%v", id, ok) } } // All cooling → earliest expiry still serves. reportKeyResult("b", 500, nil) - if _, _, ok := selectPoolKey(auth, "m"); !ok { + if _, _, ok := selectPoolKey(auth, "m", nil, nil, ""); !ok { t.Fatal("all-cooldown must fall back to earliest expiry, not fail") } // Success clears fails. @@ -146,7 +176,7 @@ func TestKeyPool_GroupFilter(t *testing.T) { goAuth := UpstreamAuth{Mode: AuthRouteGo, Token: "c"} // zen surface: go-only key must never be picked. for range 10 { - _, id, ok := selectPoolKey(zenAuth, "some-model") + _, id, ok := selectPoolKey(zenAuth, "some-model", nil, nil, "") if !ok { t.Fatal("want ok") } @@ -167,7 +197,7 @@ func TestKeyPool_GroupFilter(t *testing.T) { }) goModel := "go-only-model" for range 10 { - _, id, ok := selectPoolKey(goAuth, goModel) + _, id, ok := selectPoolKey(goAuth, goModel, nil, nil, "") if !ok { t.Fatal("want ok") } @@ -180,7 +210,7 @@ func TestKeyPool_GroupFilter(t *testing.T) { func TestKeyPool_PublicNeverPooled(t *testing.T) { resetPool(t) setKeyPool(KeyPool{Enabled: true, Keys: []UpstreamKey{{Key: "ka"}}}) - if _, _, ok := selectPoolKey(UpstreamAuth{Mode: AuthRoutePublic}, "m"); ok { + if _, _, ok := selectPoolKey(UpstreamAuth{Mode: AuthRoutePublic}, "m", nil, nil, ""); ok { t.Fatal("public must never use pool") } } diff --git a/internal/app/opencode.go b/internal/app/opencode.go index 683b35e..adaad54 100644 --- a/internal/app/opencode.go +++ b/internal/app/opencode.go @@ -621,7 +621,7 @@ func callOpenCodeEndpoint(ctx context.Context, endpointSubpath string, upstreamB if strings.TrimSpace(ocSession) == "" { ocSession = newOCSessionID() } - attemptAuth, keyID, pooled := selectPoolKey(auth, modelID, attempt) + attemptAuth, keyID, pooled := selectPoolKey(auth, modelID, bodyMap, upstreamHeadersFromContext(ctx), normalizedTransportScope(ocSession), attempt) targetAuth := auth if pooled { targetAuth = attemptAuth From 16ce153bfc3adca486f18f2bd2711b1bb0b86172 Mon Sep 17 00:00:00 2001 From: 6Kmfi6HP <179412085+6Kmfi6HP@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:24:19 +0200 Subject: [PATCH 3/4] =?UTF-8?q?fix(keypool):=20token-less=20=E8=AF=B7?= =?UTF-8?q?=E6=B1=82=E6=8C=89=E4=BC=9A=E8=AF=9D=E6=95=A3=E5=88=97=EF=BC=8C?= =?UTF-8?q?=E9=81=BF=E5=85=8D=E5=85=B1=E7=94=A8=E4=B8=80=E6=A7=BD?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit stickySessionBase 对无 token(admin 密码触发池)的请求剥离共享 public fallback 前缀,以 sess:<会话后缀> 散列:不同网关会话散开, 同一会话仍粘定;有 token 路径保持 tok:|cli:<会话> 不变。 回归测试覆盖 admin scope 散开 + 同 scope 粘定。 Co-Authored-By: Claude Code --- docs/keypool-design.md | 9 +++++---- internal/app/keypool.go | 32 +++++++++++++++++++++++--------- internal/app/keypool_test.go | 20 ++++++++++++++++++-- 3 files changed, 46 insertions(+), 15 deletions(-) diff --git a/docs/keypool-design.md b/docs/keypool-design.md index 85eee39..45d5186 100644 --- a/docs/keypool-design.md +++ b/docs/keypool-design.md @@ -36,10 +36,11 @@ ## 3. 选择器(`internal/app/keypool.go`,内存状态,仿 `socks5RRIndex` 原子模式) * `keyRRIndex atomic.Uint64`:`round_robin` 取 `Add % len`;`weighted` 取 `Add % totalWeight` 走权重区间; - `sticky` 取 `fnv32a(完整 egress sticky 键) % len`:`stickySessionBase(auth, bodyMap, headers, scope)` - = `stickyKeyForRequest` 全键(`tok:<客户端token>|cli:<客户端会话哈希>` 等,与 egress - 完全同源),同一会话同时粘定同一池 key 与同一出口路径(prompt cache 亲和), - 不同会话按哈希散开(会话级负载均衡)。`attempt>0` 的池 failover 重试混入常量 + `sticky` 取 `fnv32a(stickySessionBase) % len`。身份先按下游凭证、再按客户端会话 + 划分:同一下游 token 的不同客户端会话(`tok:|cli:<会话哈希>`,与 egress + 完全同源)散开——会话级负载均衡;同一会话粘定同一 key——prompt cache 亲和。 + 无 token 的 admin 请求以 `sess:`(`|` 去掉的会话后缀)散列,会话后缀 + 为空时所有 token-less 流量仍共用一槽。`attempt>0` 的池 failover 重试混入常量 后缀 `|pool-retry`,跳离刚失败的 key(重试之间仍粘同一备选 key,保持亲和)。 * 候选过滤:`enabled && group匹配 && now > cooldownUntil`;全冷却 → 放行最早过期的那把(不断服)。 * 状态表(`keypoolMu` 守卫):`{cooldownUntil, consecutiveFails}`;成功清零。 diff --git a/internal/app/keypool.go b/internal/app/keypool.go index aa0c08d..6a6faee 100644 --- a/internal/app/keypool.go +++ b/internal/app/keypool.go @@ -131,20 +131,34 @@ func poolEnabled() bool { return keypoolCfg.Enabled && len(keypoolEntries) > 0 } -// stickySessionBase is the pool's sticky routing key. It reuses the full -// egress sticky key (stickyKeyForRequest: token + client-session suffix), -// so keypool stickiness and egress stickiness share one identity: same -// session pins both the same pooled key and the same egress path (prompt -// cache affinity), while different sessions spread across the pool (load -// balancing). isRetry selects the alternate slot for pool-failover retries -// so a retry hashes away from the just-failed key; the alternate is itself -// sticky (constant suffix) to preserve cache affinity among retries of the -// same request. +// stickySessionBase is the pool's sticky routing key. Identity is scoped by +// downstream credential first, then by client session: same credential's +// different sessions spread across the pool (session-level balancing), while +// same session pins one pooled key (prompt cache affinity). Paid-token +// requests use the full egress sticky key (token + client-session suffix) so +// keypool stickiness and egress stickiness share one identity (same session +// pins both the same pooled key and the same egress path). Token-less +// (admin) requests have no per-credential signal, so the session suffix +// alone drives the spread — seeded by the scope (x-opencode-session hash, +// unique per gateway session) to avoid all token-less traffic collapsing +// onto one pool slot. isRetry selects the alternate slot for pool-failover +// retries so a retry hashes away from the just-failed key; the alternate is +// itself sticky (constant suffix) to preserve cache affinity among retries +// of the same request. func stickySessionBase(auth UpstreamAuth, bodyMap map[string]any, headers http.Header, ocScope string, isRetry bool) string { key := stickyKeyForRequest(auth, bodyMap, headers, ocScope) if isRetry { key += keyPoolRetrySuffix } + if auth.Token != "" { + return key + } + // Token-less: strip the shared public fallback so the session suffix + // alone decides the slot ("sess:" instead of + // "cli://public-shared|oc:"). + if suffix, ok := strings.CutPrefix(key, stickyPublicFallback); ok { + key = "sess:" + strings.TrimPrefix(suffix, "|") + } return key } diff --git a/internal/app/keypool_test.go b/internal/app/keypool_test.go index 91ff0c0..a803635 100644 --- a/internal/app/keypool_test.go +++ b/internal/app/keypool_test.go @@ -98,12 +98,28 @@ func TestKeyPool_Sticky(t *testing.T) { if got, want := stickySessionBase(a1, nil, hdr("sess-a"), "", false), "tok:user1|cli:"+hashSessionRouteKey("sess-a"); got != want { t.Fatalf("stickySessionBase with session = %q, want %q", got, want) } - if got := stickySessionBase(UpstreamAuth{}, nil, nil, "", false); got != stickyPublicFallback { - t.Fatalf("public fallback = %q", got) + if got := stickySessionBase(UpstreamAuth{}, nil, nil, "", false); got != "sess:" { + t.Fatalf("token-less fallback = %q", got) } if got := stickySessionBase(a1, nil, hdr("sess-a"), "", true); got != "tok:user1|cli:"+hashSessionRouteKey("sess-a")+keyPoolRetrySuffix { t.Fatalf("stickySessionBase retry = %q", got) } + // Admin(无 token)请求:不同网关会话必须散开(会话后缀驱动均衡)。 + admin := UpstreamAuth{Mode: AuthRouteAdmin, Source: "admin"} + seenAdmin := map[string]bool{} + for _, sc := range []string{"scope-1", "scope-2", "scope-3", "scope-4", "scope-5", "scope-6", "scope-7", "scope-8"} { + _, id, _ := selectPoolKey(admin, "m", nil, nil, sc) + seenAdmin[id] = true + } + if len(seenAdmin) < 2 { + t.Fatalf("token-less admin requests with different scopes must spread, got only %v", seenAdmin) + } + // 同一 token-less 会话必须稳定命中同一 key。 + _, idS1, _ := selectPoolKey(admin, "m", nil, nil, "scope-1") + _, idS2, _ := selectPoolKey(admin, "m", nil, nil, "scope-1") + if idS1 != idS2 { + t.Fatal("sticky must return same key for same token-less scope") + } _ = a2 } From 07b54f3327dc23a13cdccdcccfb44000c9c3c1b4 Mon Sep 17 00:00:00 2001 From: 6Kmfi6HP <179412085+6Kmfi6HP@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:35:30 +0200 Subject: [PATCH 4/4] =?UTF-8?q?refactor(server):=20=E4=B8=AD=E9=97=B4?= =?UTF-8?q?=E4=BB=B6=E6=94=B9=E5=90=8D=E4=BB=A5=E5=8F=8D=E6=98=A0=E5=85=B6?= =?UTF-8?q?=E5=BF=AB=E7=85=A7=E5=85=A8=E9=83=A8=E5=AE=A2=E6=88=B7=E7=AB=AF?= =?UTF-8?q?=E5=A4=B4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit withSessionFromRequest 实际克隆全部请求头供 sticky 路由使用, 而不仅是 x-opencode-session;改名为 withClientHeadersFromRequest 避免读者误以为客户端会话头被丢弃。无行为变更。 Co-Authored-By: Claude Code --- internal/app/opencode.go | 9 ++++++++- internal/app/server.go | 2 +- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/internal/app/opencode.go b/internal/app/opencode.go index adaad54..cbb05b8 100644 --- a/internal/app/opencode.go +++ b/internal/app/opencode.go @@ -101,7 +101,14 @@ func sessionFromRequestContext(ctx context.Context, fallback string) string { return fallback } -func withSessionFromRequest(r *http.Request) *http.Request { +// withClientHeadersFromRequest snapshots the downstream request's headers +// into the context for sticky routing (upstreamHeadersFromContext), and +// carries x-opencode-session as the transport scope. The snapshot is taken +// before any body read so handlers that replace r.Body (readJSONRequestBody) +// don't lose the headers; all client headers are preserved because +// sessionHeaderValue matches the known session header names +// case-insensitively (X-Claude-Code-Session-Id, Thread-Id, Session-Id…). +func withClientHeadersFromRequest(r *http.Request) *http.Request { headers := r.Header.Clone() session := strings.TrimSpace(headers.Get(headerOpencodeSession)) ctx := context.WithValue(r.Context(), opencodeUpstreamHeadersContextKey{}, headers) diff --git a/internal/app/server.go b/internal/app/server.go index 5763c38..94ceff6 100644 --- a/internal/app/server.go +++ b/internal/app/server.go @@ -115,7 +115,7 @@ func Run() { func sessionContextMiddleware(next http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { - next(w, withSessionFromRequest(r)) + next(w, withClientHeadersFromRequest(r)) } }