Activity

1. github-actions

   added

   enhancementNew feature or requestNew feature or request

   proxyHTTP proxy, routing, reverse-proxy / management authHTTP proxy, routing, reverse-proxy / management auth

   on Sep 2, 2026
2. lidge-jun commented on Sep 2, 2026

   

   lidge-jun

   on Sep 2, 2026

   Owner

   More actions

   리뷰 · 우선순위 48 / 80

   이 이슈는 combo 레인에서 한 타깃이 TCP/연결은 받아 두고 응답 헤더(첫 바이트)를 안 보내는 멈춤(stall)일 때, 전역 connectTimeoutMs(기본 200초)를 다 태우기 전에 다음 타깃으로 넘어가게 하자는 제안이다. 작성자 시나리오는 strategy: round-robin 콤보를 ocx claude 에이전트 앞에 두고, 키만 다른 같은 패밀리 업스트림 세 개를 넣는 형태다. 지금 구조에서는 멈춘 타깃 하나가 턴 전체를 약 200초 잡아먹고, 그제야 comboFailureDecision → advanceComboAfterFailure 로 넘어간다.

   현재 dev HEAD 272ff6b11(패키지 2.41.0) 기준으로 본문이 가리킨 코드 경로는 맞다. src/server/chat-native.ts 171줄, src/server/claude-messages.ts 380줄, src/server/responses/core.ts 3925줄 부근에서 시도마다 config.connectTimeoutMs ?? 200_000 을 헤더 도착 예산으로 쓴다. OcxComboConfig(src/types/config.ts 808–840줄) 필드에는 targets · strategy · stickyLimit · defaultEffort · reasoningEffortMode · imageInput · alias · nativeAlias · displayName 만 있고, 시도당 first-byte 데드라인 필드는 없다. 전역 connectTimeoutMs 를 줄이는 우회는 콤보 아닌 단일 제공자 요청까지 같이 짧아져서, 작성자가 말한 대로 안전하지 않다.

   이미 가까운 기차가 하나 있다. #3236 이 콤보 스트림의 zero-output incomplete 를 페일오버한다(src/server/responses/combo-stream-preflight.ts). 그건 헤더는 왔는데 본문이 비는 쪽이고, 이 이슈는 헤더 자체가 안 오는 stall 이다. 겹치지만 같은 패치가 아니다. 그래서 zero-output 머지를 이유로 이 이슈를 닫으면 안 된다. types.ts/config.ts 분할 캠페인과도 직접 충돌하지 않는다. 새 필드를 OcxComboConfig 에 넣으면 설정 스키마 쪽이 조금 건드려질 수는 있다.

   우선순위 48인 이유. 실사용 고통(에이전트 턴 타임아웃)은 분명하고 재현 서술이 구체적이다. 다만 설계 선택이 남아 있다. 시도당 예산을 콤보 필드에 둘지, 라우팅 프로필에 둘지, 헤더 대기만 자를지 connect 전체와 공유할지, 스트림 경로(chat-native/claude-messages/responses/core) 세 곳에 같은 계약을 어떻게 심을지. 기본값을 전역보다 짧게 두면 느린 정상 업스트림을 오탐으로 넘길 수 있다. enhancement + proxy 라벨은 맞다. 지금 당장 릴리즈 블로커는 아니다.

   경로 src/types/config.ts OcxComboConfig - 시도당 first-byte/헤더 예산 필드가 없다. 전역 connectTimeoutMs 만으로는 콤보 홉을 짧게 못 만든다.
   경로 src/server/chat-native.ts:171 · claude-messages.ts:380 · responses/core.ts:3925 부근 - 시도마다 전역 200s 헤더 예산을 그대로 쓴다. stall 한 번이 턴을 잠근다.
   경로 comboFailureDecision / advanceComboAfterFailure - 실패 판정 후에야 다음 타깃으로 간다. 예산이 길면 홉도 늦다.
   경로 #3236 zero-output 페일오버 - 헤더 이후 빈 본문 문제. 이 이슈(헤더 미도착 stall)와 겹치되 대체재가 아니다.
   경로 전역 connectTimeoutMs 축소 - 콤보 밖 요청까지 같이 짧아져 우회로 부적합하다.

   메인테이너의 판단이 필요한 지점

   • 새 필드를 OcxComboConfig 에 둘지(attemptHeaderTimeoutMs 같은 이름) 전역 옆에 콤보 전용 오버라이드만 둘지
   • 기본값을 얼마로 할지(예: 5–15s) 그리고 sticky RR 배치와 어떻게 맞출지
   • chat / claude-messages / responses 세 경로에 동일 계약을 한 패치로 심을지, 가장 아픈 경로만 먼저 할지
   • help wanted 로 열어 둘지, 메인테이너가 스키마부터 짧게 스펙을 박고 받을지
   • types/config 분할 진행 중이라 스키마 PR은 close-don't-rebase 대상이 될 수 있는지(필드 추가면 보통 독립 가능)

   너의 추천
   열어 두고 needs-design 성격으로 짧게 스펙을 먼저 박는 편이 좋다. 최소 스펙 예시는 OcxComboConfig 에 optional 시도당 헤더 예산(전역보다 짧을 때만 적용) + 세 서버 경로에서 그 값을 connect 예산 상한으로 쓰기 + stall/timeout을 기존 combo failure 경로로 넘기기. #3236 과 이슈를 합치지 마라. 중복·분할 무효로 닫을 대상이 아니다. 당장 머지할 PR은 없다.

   이 댓글은 grok-bot이 작성했습니다
3. Veritas-7 commented on Sep 2, 2026

   

   Veritas-7

   on Sep 2, 2026

   ContributorAuthor

   More actions

   Proposed spec for #3266 (separate from #3236):

   Add two optional per-combo fields:

   • attemptFirstByteTimeoutMs: maximum time for one target to produce final response headers.
   • requestBudgetMs: optional wall-clock budget shared by all target attempts in one logical combo request.

   Both are positive integer milliseconds (1..600000). When omitted, behavior is unchanged. If both are present, require requestBudgetMs >= attemptFirstByteTimeoutMs.

   For each target, the effective header deadline is:

   min(global connectTimeoutMs, attemptFirstByteTimeoutMs, remaining requestBudgetMs)

   The request budget starts once when combo execution begins and is never reset on hop or same-target recovery. If it is exhausted before a send, no further target is dispatched. A header deadline produces synthetic HTTP 504 with error.code = "first_byte_timeout"; this is target-local and therefore enters the existing combo hop path. That path records the failed attempt, cools the target through advanceComboAfterFailure/coolComboTarget, and selects the next eligible target. Client cancellation remains 499 and never hops.

   Suggested log line:

   [combo] <comboId>: <provider/model> first-byte-timeout after <elapsed>ms; hopping

   Implementation should reuse the existing header-timeout machinery across Chat, Anthropic Messages, Responses passthrough, and adapter paths, with only minimal CODEOWNED core.ts plumbing. Tests should cover validation/default compatibility, deadline precedence, stalled-header failover/cooldown, all transport paths, and budget exhaustion before a later dispatch.
4. Veritas-7 commented on Sep 3, 2026

   

   Veritas-7

   on Sep 3, 2026

   ContributorAuthor

   More actions

   Measured data now argues against our own earlier proposal of a global per-attempt first-byte deadline.

   What we measured

   • Source: six days of local proxy attempt records, covering 134,716 attempts.
   • Stall criterion: zero first byte plus duration clustered at the global timeout (200.008–200.019 s).

   Measure	Result
   Timeout-shaped stalls	23 attempts (0.17 per 1000); 4 of 6 days
   Attempts in the timeout band	28; failover rescued 5
   Successful attempts at 190 s or more to first byte	44 (0.33 per 1000)
   Successful attempts at 30–120 s to first byte	12,249 (9.1%)
   p99 time-to-first-byte	57 s

   Why the proposed fix is net-negative

   A 190 s first-byte deadline would abort 44 healthy successful responses to rescue 23 stalled ones, roughly a 1.9:1 loss. At 120 s, the trade-off is about 7:1 against. The mechanism cannot distinguish a dead target from a model still thinking until the first byte arrives; connection-level failures overwhelmingly fail fast (under 5 s), so the 200 s stall is a rare tail.

   Alternatives

   A. Hedged failover

   After a configurable no-first-byte delay, start the next eligible target in parallel without aborting the first; whichever produces the first byte wins and the loser is aborted.
   Default off, with the delay set well above p99 first-byte time; duplicated upstream usage is bounded by aborting the loser, and a healthy slow response is never killed.

   B. Per-target opt-in deadline

   Allow each combo target to declare its own first-byte deadline, absent by default; the measurement makes this more defensible for a fast non-reasoning target than as a global policy.
   Long-but-healthy first-byte times concentrated on reasoning-heavy targets, so a target-specific policy avoids imposing their latency on every target.

   Would you rather see the connect/TLS phase get its own shorter timeout? That is a cheaper change, but it would not catch these particular stalls because the connection had already been accepted before the silence began.

   We are happy to implement whichever direction you prefer; the branch for the original design is parked, not abandoned.
5. Veritas-7 commented on Sep 4, 2026

   

   Veritas-7

   on Sep 4, 2026

   ContributorAuthor

   More actions

   A correction to a number in my earlier comment, found while writing this up internally.

   I wrote that 12,249 attempts (9.1%) took 30 to 120 seconds to first byte and succeeded. That label is wrong. Re-running the same sample — the first 134,716 attempts in the log, ending 2026-09-03T13:31Z — separates cleanly:

   measure, successful attempts in the 30-120 s band	count	share
   time to first byte	2,057	1.5%
   total attempt duration	12,249	9.1%

   So 12,249 was a total-duration figure presented as a first-byte one. The underlying cause is that the log carries both a request-level and an attempt-level record with the same field names, and I conflated the two.

   The conclusion is unaffected, and if anything the corrected figure argues for it more strongly. The decision rests on how many healthy responses a first-byte deadline would abort versus how many stalls it would rescue. On first-byte timing at that cutoff: 2,057 successful attempts waited 30-120 seconds for their first byte, against 21 attempts that produced no first byte at all and ran to the 200-second timeout. I originally gave the stall count as 23; the number moves between 21 and 23 depending on how tightly "at the timeout" is banded, since the durations cluster between 198 and 204 seconds rather than landing on 200000 ms exactly — no record matches that value exactly, which is worth knowing for anyone reproducing this.

   Either way the ratio is roughly two orders of magnitude against the deadline, so the recommendation stands: hedging a second attempt without aborting the first, or a per-target opt-in deadline, rather than a global first-byte cutoff.
6. Veritas-7 commented on Sep 5, 2026

   

   Veritas-7

   on Sep 5, 2026

   ContributorAuthor

   More actions

   Second correction — opencodex#3266

   This is my second correction to the measurements I posted on this issue. The issue is lidge-jun/opencodex#3266, titled feat(combo): per-combo attempt first-byte deadline so a stalled target hops before the global 200s connectTimeoutMs; the proposed feature is combo-route-only.

   I apologize for reversing myself a second time on this thread. The numbers below were re-measured from scratch on the same sample, and the full reproduction script and its output are included so the figures can be checked independently rather than taken on trust.

   The file-order prefix of the first 134,716 attempts — the same sample size as the original post — yields 19 timeout-shaped stalls. On this same prefix: 23 is the count of all attempts with 198,000 <= durationMs < 200,000; 28 is the count of no-first-byte attempts with 198,000 <= durationMs < 202,000; the 21 I gave in my previous correction is the count of no-first-byte attempts with 199,500 <= durationMs < 200,500; and 19 is the count of no-first-byte attempts in the tight 200,000 <= durationMs < 200,020 window that the original comment actually described (200.008–200.019 s). The published counts are band-sensitive, and that accounts for most of the spread I reported. I am not claiming those bands are the populations the original numbers came from: the 23-count is 20 successful (status 200) attempts and 3 with status 502, and 19 of the 23 recorded a first byte — 14 of them under 60 seconds, the earliest at 1,178 ms — so that set is not "timeout-shaped stalls" under any reading. What does line up is the rescue count: both the 28 band and the tight window contain exactly 5 attempts whose parent request still ended 200, matching the "failover rescued 5" I published. I am standardizing on the tight 200,000 <= durationMs < 200,020 window because it is the criterion the original comment stated (200.008-200.019 s), and I am reporting the band sensitivity beside it rather than claiming the earlier counts are reconstructed. The prior cutoff label was also wrong: 2026-09-03T13:31Z was Korean local time written with a Z suffix. The measured boundary is 2026-09-03T04:31:35.971000Z (epoch 1788409895971, = 13:31:35.971+09:00).

   Values in the published (verbatim) column are copied from issue comments 5520470506 and 5534802008; the four rows labelled as my previous correction preserve the latter comment's published claims. Corrected values carry an explicit population label; the new route decomposition is measured on the same 134,716-attempt prefix.

   measure	published (verbatim)	corrected	corrected population
   Timeout-shaped stalls	23 attempts (0.17 per 1000); 4 of 6 days	19 (status 502 = 18, status 504 = 1); 0.141 per 1000; 3 of 6 days	all attempts
   Attempts in the timeout band	28; failover rescued 5	19 in the tight 200,000 <= durationMs < 200,020 window; failover rescued 5; every attempt in that window had no first byte. Widening the window to 199,900-200,100 gives 21 no-first-byte attempts (20 of them combo); the published 28 corresponds to no-first-byte attempts in 198,000-202,000.	all attempts
   Timeout-shaped stalls (my previous correction)	21 attempts that produced no first byte at all and ran to the 200-second timeout	reproduces as no-first-byte attempts with 199,500 <= durationMs < 200,500; the tight 200,000-200,020 window gives 19	all attempts
   Stall-count stability (my previous correction)	the number moves between 21 and 23 depending on how tightly "at the timeout" is banded	confirmed; each of 23/28/21/19 reproduces as a band on this prefix, though I cannot confirm which band the original counts used	all attempts
   Duration spread (my previous correction)	durations cluster between 198 and 204 seconds rather than landing on 200000 ms exactly	confirmed: 0 records at exactly 200,000 ms; 30 no-first-byte attempts fall in 198-204 s, of which 19 fall in 200.008-200.019 s	all attempts
   Recommendation (my previous correction)	the recommendation stands: hedging ... rather than a global first-byte cutoff	retracted as stated: the cost side that produced it is non-combo, so it cannot decide a combo-only feature	published versus corrected route attribution
   Successful attempts at 190 s or more to first byte	44 (0.33 per 1000)	44 (status 200 with a recorded first byte); route split: explicit-provider 42, native 2, combo 0	all successful attempts with status 200 and firstOutputMs >= 190,000
   Successful attempts at 30–120 s to first byte	12,249 (9.1%)	2,057 (status 200 with a recorded first byte; 1.5% of all 134,716 attempts); 12,249 is the total-duration count, not the first-byte count	all successful attempts with status 200 and firstOutputMs in 30,000–120,000 ms
   p99 time-to-first-byte	57 s	56,582.820 ms (linear interpolation) / 56,664 ms (nearest-rank); both round to 57 s	all attempts with status 200 and a recorded first byte
   190-second conclusion	A 190 s first-byte deadline would abort 44 healthy successful responses to rescue 23 stalled ones, roughly a 1.9:1 loss.	Invalid as a combo-route cost estimate: the 44 are non-combo, while the feature is combo-route-only.	published comparison versus corrected route attribution
   120-second conclusion	At 120 s, the trade-off is about 7:1 against.	Combo-only: 1 success (status 200 with a recorded first byte) at or above 120 s versus 18 stalls; this is not 7:1 against.	combo only

   Why the original loss calculation was invalid

   The original 190-second argument used 44 slow successful attempts as the cost side. On the corrected prefix, those 44 are explicit-provider 42 + native 2 + combo 0. The proposed change is combo-route-only, so those 44 responses are outside the feature's reach. I therefore imported the cost from routes the feature cannot affect.

   Combo-only trade-off

   For the route the feature can actually change, the corrected prefix contains 14,400 combo attempts, 18 combo stalls in the tight 200,000 <= durationMs < 200,020 window, and 0 combo successes with first byte at or after 190 seconds. Every combo stall ended at 200,008–200,019 ms, so a 190-second deadline would have an observed cost of 0 combo successes and would save only 10.008–10.019 s per stall, 180.195 s across the six-day sample. This is a small benefit, not the earlier claimed rescue-versus-loss trade-off. Because the stalls are in the tight window, 190 s leaves about 10 s of the 200 s cap before the observed stall boundary.

   The anti-deadline recommendation in my previous correction is withdrawn: its cost side was non-combo, so it cannot decide a combo-only feature. Threshold choice is the material decision: at 60 s, 8 combo successes are in the observed danger zone while a stalled attempt would be cut roughly 140 s earlier; at 120 s, 1 combo success is in that zone; at 190 s, 0 are. These measurements leave that policy choice to the maintainer rather than presenting a global 44-versus-23 loss calculation that does not apply to combo.

   Method

   • The reproducible sample is the file-order prefix. The 134,716th attempt is on 1-indexed log line 155,335, with boundary timestamp 1788409895971 (2026-09-03T04:31:35.971Z).
   • routeDecision.routeKind is a request-level field; it is absent from all 134,716 attempt objects, so each attempt inherits routeKind from its parent request record.
   • An attempt record is an element of the attempts array on a request-level line. Lines without an attempts array are compact request records and are excluded.
   • A missing firstOutputMs is a measurement gap, not a stall: 45,604 of the 49,074 no-first-byte attempts in this prefix (93%) ended with status 200. Only the tight window is all-5xx (502 x 18, 504 x 1); the wider bands mix in completed successes.
   • The timeout-shaped stall criterion is no first byte plus 200,000 <= durationMs < 200,020. The band-sensitivity controls additionally count all attempts in 198,000 <= durationMs < 200,000, no-first-byte attempts in 198,000 <= durationMs < 202,000, and no-first-byte attempts in 199,500 <= durationMs < 200,500. The first-byte controls use successful attempts with firstOutputMs; the total-duration control uses successful attempts with durationMs.
   • p99 is reported by both methods: linear interpolation at 0.99 * (n - 1) and nearest-rank at index ceil(0.99 * n) - 1. The method is named beside every p99 result.
   • The output block below was generated by redirecting the corresponding command to a file and then diffed against the block.

   Reproduction

   The required source-document reproduction script was run with the following command. It reads only request-level attempts array elements and excludes compact records without that array.

   python3 - "$HOME/.opencodex/usage.jsonl" 134716 <<'PY'
   import collections
   import datetime
   import json
   import math
   import statistics
   import sys
   
   path, target_attempts = sys.argv[1], int(sys.argv[2])
   attempts = []
   last_line = None
   last_timestamp = None
   with open(path, encoding="utf-8") as stream:
       for line_number, line in enumerate(stream, 1):
           record = json.loads(line)
           attempt_field = record.get("attempts")
           if not isinstance(attempt_field, list):
               continue
           route_kind = (record.get("routeDecision") or {}).get("routeKind")
           if not isinstance(route_kind, str):
               raise SystemExit(f"missing parent routeKind at log line {line_number}")
           for attempt in attempt_field:
               attempts.append((attempt, record))
               last_line = line_number
               last_timestamp = record.get("timestamp")
               if len(attempts) == target_attempts:
                   break
           if len(attempts) == target_attempts:
               break
   
   if len(attempts) != target_attempts:
       raise SystemExit(f"expected {target_attempts} attempts, got {len(attempts)}")
   
   attempt_values = [attempt for attempt, _ in attempts]
   
   def succeeded(attempt):
       return attempt.get("status") == 200
   
   def has_first_output(attempt):
       return attempt.get("firstOutputMs") is not None
   
   def is_number(value):
       return isinstance(value, (int, float)) and not isinstance(value, bool)
   
   no_first_byte_near_timeout = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
       and is_number(attempt.get("durationMs"))
       and 200_000 <= attempt["durationMs"] < 200_020
   ]
   first_byte_successes = sorted(
       attempt["firstOutputMs"]
       for attempt in attempt_values
       if succeeded(attempt) and has_first_output(attempt)
   )
   
   p99_position = 0.99 * (len(first_byte_successes) - 1)
   p99_lower_index = math.floor(p99_position)
   p99_linear_ms = (
       first_byte_successes[p99_lower_index]
       + (first_byte_successes[p99_lower_index + 1] - first_byte_successes[p99_lower_index])
       * (p99_position - p99_lower_index)
   )
   p99_nearest_rank_ms = first_byte_successes[math.ceil(0.99 * len(first_byte_successes)) - 1]
   
   band_attempts = [
       (attempt, record) for attempt, record in attempts
       if is_number(attempt.get("durationMs"))
       and 200_000 <= attempt["durationMs"] < 200_020
   ]
   all_attempts_198000_200000 = [
       (attempt, record) for attempt, record in attempts
       if is_number(attempt.get("durationMs"))
       and 198_000 <= attempt["durationMs"] < 200_000
   ]
   no_first_byte_198000_202000 = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
       and is_number(attempt.get("durationMs"))
       and 198_000 <= attempt["durationMs"] < 202_000
   ]
   no_first_byte_199500_200500 = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
       and is_number(attempt.get("durationMs"))
       and 199_500 <= attempt["durationMs"] < 200_500
   ]
   no_first_byte_198000_204000 = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
       and is_number(attempt.get("durationMs"))
       and 198_000 <= attempt["durationMs"] < 204_000
   ]
   no_first_byte_199900_200100 = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
       and is_number(attempt.get("durationMs"))
       and 199_900 <= attempt["durationMs"] < 200_100
   ]
   combo_no_first_byte_199900_200100 = [
       (attempt, record) for attempt, record in no_first_byte_199900_200100
       if (record.get("routeDecision") or {}).get("routeKind") == "combo"
   ]
   no_first_byte_all_prefix = [
       (attempt, record) for attempt, record in attempts
       if not has_first_output(attempt)
   ]
   no_first_byte_status_counts = collections.Counter(
       attempt.get("status") for attempt, _ in no_first_byte_all_prefix
   )
   all_attempts_198000_200000_status_counts = collections.Counter(
       attempt.get("status") for attempt, _ in all_attempts_198000_200000
   )
   all_attempts_198000_200000_route_counts = collections.Counter(
       (record.get("routeDecision") or {}).get("routeKind")
       for _, record in all_attempts_198000_200000
   )
   all_attempts_198000_200000_first_byte_values = [
       attempt["firstOutputMs"] for attempt, _ in all_attempts_198000_200000
       if has_first_output(attempt)
   ]
   all_attempts_198000_200000_first_byte_under_60_seconds = sum(
       value < 60_000 for value in all_attempts_198000_200000_first_byte_values
   )
   failover_rescued = [
       (attempt, record) for attempt, record in band_attempts
       if attempt.get("status") != 200 and record.get("status") == 200
   ]
   failover_rescued_wide = [
       (attempt, record) for attempt, record in no_first_byte_198000_202000
       if attempt.get("status") != 200 and record.get("status") == 200
   ]
   combo_successes = sorted(
       attempt["firstOutputMs"]
       for attempt, record in attempts
       if (record.get("routeDecision") or {}).get("routeKind") == "combo"
       and succeeded(attempt)
       and has_first_output(attempt)
   )
   combo_stalls = [
       (attempt, record)
       for attempt, record in no_first_byte_near_timeout
       if (record.get("routeDecision") or {}).get("routeKind") == "combo"
   ]
   combo_savings_ms = [attempt["durationMs"] - 190_000 for attempt, _ in combo_stalls]
   combo_p99_position = 0.99 * (len(combo_successes) - 1)
   combo_p99_lower_index = math.floor(combo_p99_position)
   combo_p99_linear_ms = (
       combo_successes[combo_p99_lower_index]
       + (combo_successes[combo_p99_lower_index + 1] - combo_successes[combo_p99_lower_index])
       * (combo_p99_position - combo_p99_lower_index)
   )
   stall_days_utc = {
       datetime.datetime.fromtimestamp(record["timestamp"] / 1000, datetime.timezone.utc).date().isoformat()
       for attempt, record in no_first_byte_near_timeout
       if record.get("timestamp") is not None
   }
   client_cancel_attempts_2 = sum(
       record.get("closeReason") == "client_cancel"
       and len(record.get("attempts") or []) == 2
       for attempt, record in no_first_byte_near_timeout
   )
   error_status_stalls = sum(
       record.get("status") in (429, 502, 504)
       for attempt, record in no_first_byte_near_timeout
   )
   parent_outcomes = {}
   for attempt, record in no_first_byte_near_timeout:
       key = (
           record.get("status"),
           record.get("closeReason"),
           len(record.get("attempts") or []),
       )
       parent_outcomes[key] = parent_outcomes.get(key, 0) + 1
   
   print("attempts", len(attempt_values))
   print("prefix_boundary_1_indexed_line", last_line)
   print("prefix_boundary_timestamp_ms", last_timestamp)
   print("route_kind_source", "request_parent")
   print("route_kind_counts")
   for route_kind in ("native", "explicit-provider", "combo", "default-provider"):
       print(route_kind, sum(
           (record.get("routeDecision") or {}).get("routeKind") == route_kind
           for _, record in attempts
       ))
   print("attempt_route_kind_fields", sum("routeKind" in attempt for attempt in attempt_values))
   print("timeout_band_attempts", len(band_attempts))
   print("all_attempts_198000_200000", len(all_attempts_198000_200000))
   print("all_attempts_198000_200000_status_counts", dict(sorted(all_attempts_198000_200000_status_counts.items())))
   print("all_attempts_198000_200000_route_counts", dict(sorted(all_attempts_198000_200000_route_counts.items())))
   print("all_attempts_198000_200000_first_byte_count", len(all_attempts_198000_200000_first_byte_values))
   print("all_attempts_198000_200000_first_byte_under_60_seconds", all_attempts_198000_200000_first_byte_under_60_seconds)
   print("all_attempts_198000_200000_first_byte_min_ms", min(all_attempts_198000_200000_first_byte_values))
   print("all_attempts_exact_200000", sum(
       is_number(a.get("durationMs")) and a["durationMs"] == 200_000
       for a, _ in attempts
   ))
   print("no_first_byte_all_prefix", len(no_first_byte_all_prefix))
   print("no_first_byte_status_counts", dict(sorted(no_first_byte_status_counts.items())))
   print("no_first_byte_198000_202000", len(no_first_byte_198000_202000))
   print("no_first_byte_198000_202000_failover_rescued", len(failover_rescued_wide))
   print("no_first_byte_199500_200500", len(no_first_byte_199500_200500))
   print("no_first_byte_200000_200020", len(no_first_byte_near_timeout))
   print("no_first_byte_198000_204000", len(no_first_byte_198000_204000))
   print("no_first_byte_199900_200100", len(no_first_byte_199900_200100))
   print("combo_no_first_byte_199900_200100", len(combo_no_first_byte_199900_200100))
   print("stalls", len(no_first_byte_near_timeout))
   print("stall_status_502", sum(a.get("status") == 502 for a, _ in no_first_byte_near_timeout))
   print("stall_status_504", sum(a.get("status") == 504 for a, _ in no_first_byte_near_timeout))
   print("combo_stalls", len(combo_stalls))
   print("combo_stall_status_502", sum(a.get("status") == 502 for a, _ in combo_stalls))
   print("combo_stall_status_504", sum(a.get("status") == 504 for a, _ in combo_stalls))
   print("band_failover_rescued", len(failover_rescued))
   print("tight_failover_rescued", len(failover_rescued))
   print("slow_successes_at_or_after_190_seconds", sum(
       succeeded(a) and has_first_output(a) and a["firstOutputMs"] >= 190_000
       for a in attempt_values
   ))
   print("route_kind_cross_table slow_successes_at_or_after_190_seconds stalls")
   for route_kind in ("native", "explicit-provider", "combo", "default-provider"):
       print(route_kind, sum(
           (record.get("routeDecision") or {}).get("routeKind") == route_kind
           and succeeded(a) and has_first_output(a) and a["firstOutputMs"] >= 190_000
           for a, record in attempts
       ), sum(
           (record.get("routeDecision") or {}).get("routeKind") == route_kind
           for a, record in no_first_byte_near_timeout
       ))
   print("successes_by_first_byte_between_30_and_120_seconds", sum(
       succeeded(a) and has_first_output(a)
       and 30_000 <= a["firstOutputMs"] <= 120_000
       for a in attempt_values
   ))
   print("successes_by_total_duration_between_30_and_120_seconds", sum(
       succeeded(a) and is_number(a.get("durationMs"))
       and 30_000 <= a["durationMs"] <= 120_000
       for a in attempt_values
   ))
   print("successful_attempts_with_first_byte_status_200_recorded", len(first_byte_successes))
   print("p99_first_byte_linear_interpolation_ms", f"{p99_linear_ms:.3f}")
   print("p99_first_byte_nearest_rank_ms", p99_nearest_rank_ms)
   print("combo_successes_with_first_output_status_200_recorded", len(combo_successes))
   print("combo_p99_first_byte_linear_interpolation_ms", f"{combo_p99_linear_ms:.3f}")
   print("combo_p99_first_byte_nearest_rank_ms", combo_successes[math.ceil(0.99 * len(combo_successes)) - 1])
   print("combo_max_first_byte_ms", max(combo_successes))
   for threshold_seconds in (60, 120, 190):
       print(f"combo_successes_at_or_after_seconds_{threshold_seconds}", sum(
           value >= threshold_seconds * 1000 for value in combo_successes
       ))
   print("combo_stall_duration_min_ms", min(a["durationMs"] for a, _ in combo_stalls))
   print("combo_stall_duration_max_ms", max(a["durationMs"] for a, _ in combo_stalls))
   print("combo_savings_vs_190_seconds_min", f"{min(combo_savings_ms) / 1000:.3f}")
   print("combo_savings_vs_190_seconds_max", f"{max(combo_savings_ms) / 1000:.3f}")
   print("combo_savings_vs_190_seconds_total", f"{sum(combo_savings_ms) / 1000:.3f}")
   print("stall_days_utc", sorted(stall_days_utc))
   print("client_cancel_parent_attempts_2", client_cancel_attempts_2)
   print("error_status_stalls", error_status_stalls)
   print("stall_parent_outcomes", sorted((str(key), value) for key, value in parent_outcomes.items()))
   print("boundary_utc", datetime.datetime.fromtimestamp(last_timestamp / 1000, datetime.timezone.utc).isoformat().replace("+00:00", "Z"))
   PY

   The status fields sit at two levels. stall_status_502 = 18 and stall_status_504 = 1 are the attempt-level statuses of the 19 stalled attempts. stall_parent_outcomes is the parent request outcome for those same 19: 5 were rescued by failover (parent 200), 11 were cancelled by the client (parent 499) on requests that all carried exactly two attempts, meaning the stalled attempt had already hit the 200 s cap and failover had already started a second attempt before the client gave up, and 3 ended in an error status (429/502/504). The client cancellations follow the stall; they do not replace it.

   Output from the script above was generated by redirecting its stdout to a file and then diffed against this block.

   attempts 134716
   prefix_boundary_1_indexed_line 155335
   prefix_boundary_timestamp_ms 1788409895971
   route_kind_source request_parent
   route_kind_counts
   native 97290
   explicit-provider 22980
   combo 14400
   default-provider 46
   attempt_route_kind_fields 0
   timeout_band_attempts 19
   all_attempts_198000_200000 23
   all_attempts_198000_200000_status_counts {200: 20, 502: 3}
   all_attempts_198000_200000_route_counts {'combo': 5, 'explicit-provider': 6, 'native': 12}
   all_attempts_198000_200000_first_byte_count 19
   all_attempts_198000_200000_first_byte_under_60_seconds 14
   all_attempts_198000_200000_first_byte_min_ms 1178
   all_attempts_exact_200000 0
   no_first_byte_all_prefix 49074
   no_first_byte_status_counts {200: 45604, 400: 8, 429: 1715, 499: 62, 500: 4, 502: 1671, 503: 5, 504: 3, 520: 2}
   no_first_byte_198000_202000 28
   no_first_byte_198000_202000_failover_rescued 5
   no_first_byte_199500_200500 21
   no_first_byte_200000_200020 19
   no_first_byte_198000_204000 30
   no_first_byte_199900_200100 21
   combo_no_first_byte_199900_200100 20
   stalls 19
   stall_status_502 18
   stall_status_504 1
   combo_stalls 18
   combo_stall_status_502 18
   combo_stall_status_504 0
   band_failover_rescued 5
   tight_failover_rescued 5
   slow_successes_at_or_after_190_seconds 44
   route_kind_cross_table slow_successes_at_or_after_190_seconds stalls
   native 2 0
   explicit-provider 42 1
   combo 0 18
   default-provider 0 0
   successes_by_first_byte_between_30_and_120_seconds 2057
   successes_by_total_duration_between_30_and_120_seconds 12249
   successful_attempts_with_first_byte_status_200_recorded 85083
   p99_first_byte_linear_interpolation_ms 56582.820
   p99_first_byte_nearest_rank_ms 56664
   combo_successes_with_first_output_status_200_recorded 6144
   combo_p99_first_byte_linear_interpolation_ms 25727.730
   combo_p99_first_byte_nearest_rank_ms 25895
   combo_max_first_byte_ms 155929
   combo_successes_at_or_after_seconds_60 8
   combo_successes_at_or_after_seconds_120 1
   combo_successes_at_or_after_seconds_190 0
   combo_stall_duration_min_ms 200008
   combo_stall_duration_max_ms 200019
   combo_savings_vs_190_seconds_min 10.008
   combo_savings_vs_190_seconds_max 10.019
   combo_savings_vs_190_seconds_total 180.195
   stall_days_utc ['2026-08-30', '2026-09-02', '2026-09-03']
   client_cancel_parent_attempts_2 11
   error_status_stalls 3
   stall_parent_outcomes [('(200, None, 2)', 5), ("(429, 'non_stream', 2)", 1), ("(499, 'client_cancel', 2)", 11), ("(502, 'non_stream', 1)", 1), ("(504, 'non_stream', 1)", 1)]
   boundary_utc 2026-09-03T04:31:35.971000Z
   

   The output block above is the same stdout produced by the command. Its route counts, cross-table, combo thresholds, combo p99 values, combo maximum, and 190-second savings are all generated values; no output was hand-entered.

   The p99 estimators are explicit: the all-attempt successful-first-byte p99 (status 200 with a recorded first byte) is linear interpolation at 0.99 * (n - 1) = 56,582.820 ms and nearest-rank at ceil(0.99 * n) - 1 = 56,664 ms; the combo-only p99 (status 200 with a recorded first byte) uses the same two methods and is 25,727.730 ms (linear interpolation) / 25,895 ms (nearest-rank). Both all-attempt estimators round to the published 57 s.

   The published 23, 28, and 21 counts are copied verbatim from the issue comments above; their populations are now identified by the reproduced bands, and the tight-window 19 is the standardized criterion.
7. lidge-jun commented on Sep 8, 2026

   

   lidge-jun

   on Sep 8, 2026

   Owner

   More actions

   Closing this as a maintainer decision on the size of the win, and I want to be explicit that it is my call rather than a concession you made — your final comment withdrew the earlier anti-deadline argument precisely because its cost data was non-combo.

   Thank you for correcting the measurements publicly twice rather than letting the first numbers stand. That is unusually careful reporting, and the route decomposition is what made this decidable.

   The final figures put timeout-shaped stalls at 19 in 134,716 attempts (0.141 per 1000, across 3 of 6 days), and your combo-only decomposition narrows that to 18 stalls against 14,400 combo attempts. Failover rescued 5; the other 14 ran the full 200 s cap.

   On combo-only data a 190 s deadline would abort 0 healthy responses, but every observed stall landed at 200.008–200.019 s, so it would save about 10 s per stall — roughly 180 s across a six-day sample. A tighter threshold buys more and starts costing real responses: 1 combo success sits at or above 120 s and 8 at or above 60 s. Against that, a second per-attempt deadline alongside connectTimeoutMs — documented as a header-arrival budget at src/types/config.ts:672 and again at :1058 — leaves operators with two interacting deadlines to reason about for a 0.13% event that already partially self-heals. That trade is not worth the configuration surface today.

   If the stall rate changes materially — a provider that regularly holds connections without sending headers, or a window where failover rescues few of them — please reopen with the new sample. The measurement methodology you built here would make that a quick decision.
8. lidge-jun

   closed this as not plannedon Sep 8, 2026
9. stephen-drew commented on Sep 18, 2026

   

   stephen-drew

   on Sep 18, 2026

   More actions

   Additional provider sample for the reopen condition in this issue.

   On OpenCodex 2.58.0 on Windows, a healthy proxy ran an Astra failover combo containing xai/grok-4.6. Two recent attempts returned HTTP 502 only after 78,727 ms and 82,356 ms. Nearby xAI attempts also produced transient HTTP 503 responses and connection resets. Each failure eventually triggered combo failover, but one slow target could consume most of a scheduled automation's bounded run budget before the next target started.

   This is not the same magnitude as the 200-second global-cap stall, but it is a provider-specific slow failure: the target holds the attempt for roughly 80 seconds before returning the terminal 5xx. Removing the target from scheduled combos avoids the delay, but it also removes usable fallback capacity.

   A per-target opt-in no-progress or first-byte deadline, or hedged failover, would let an operator keep a provider such as this in a lower-priority combo slot without letting one slow 5xx consume the run budget. The existing cooldown does not help before the first failure.

   No secrets, account identifiers, request bodies, or raw configuration are included.