Repository navigation
Show estimated decode tok/s alongside end-to-end throughput in Logs #4038
Description
This was generated by AI during triage.
Area
Dashboard
What are you trying to accomplish?
Compare routed model generation throughput without conflating first-token latency with post-first-output delivery speed. The Logs page already records TTFT and output token counts, so users should be able to inspect an estimated decode-rate metric alongside the existing end-to-end tok/s value.
What prevents this today?
The current tok/s metric is intentionally calculated as outputTokens / full request duration. A request with a large TTFT therefore appears slow even when tokens arrive quickly after the first output. TTFT is shown separately in the detail dialog, but the dashboard does not derive or display the post-TTFT rate.
What should OpenCodex do?
Keep the existing end-to-end tok/s metric and its filter semantics unchanged. Add an explicitly estimated decode-rate value calculated as:
outputTokens / (durationMs - firstOutputMs)
Use the request-relative TTFT for the top-level row and each attempt's own attempt-relative TTFT for combo rows. Return the metric as unavailable when usage, positive output tokens, TTFT, or a positive post-TTFT duration is unavailable. Label the value as estimated because completion framing, network pauses, and stream finalization can still be included after the first output.
Example usage or interface
For a log entry with 240 output tokens, 10,000 ms total duration, and 2,000 ms TTFT:
- Existing end-to-end rate:
24 tok/s - Estimated decode rate:
30 tok/s
The Logs table and request detail should show both values without changing existing speed filters.
Alternatives or workarounds
Users can manually export the log fields and compute the value, but this is slow and error-prone, especially for combo attempts whose TTFT is relative to the attempt rather than the parent request.
Additional context
This is a display-time derived metric only. It should not mutate persisted request logs or usage history.
Checks
- I searched existing issues and documentation.
- This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
- I removed secrets and personal data.
Proposed acceptance criteria
- Keep the current end-to-end
tok/svalue and speed-filter behavior unchanged. - Add
displayMetrics.decodeTokPerSecondonly at management-API response time; do not changeRequestLogEntry,usage.jsonl, or persisted schemas. - Use the parent request's
firstOutputMsfor the parent metric and each combo attempt's ownfirstOutputMsfor the attempt metric. - Return
ttft_missingwhen TTFT was not recorded, andinvalid_durationwhen the post-TTFT interval is non-positive or invalid. - Mark every decode-rate value as estimated, including when token usage itself was reported exactly.
- Stack end-to-end and decode values in the existing Logs rate column to avoid widening the fixed-layout table; show separate labeled values in request details and combo attempt details.
- Keep this metric Logs-only; routing analytics remain unchanged.
- Cover parent/attempt separation, unavailable cases, persistence immutability, locale parity, and unchanged speed-filter semantics with tests.
Activity
lidge-jun commentedon Sep 8, 2026
리뷰 · 우선순위 38 / 80
이 이슈는 Logs 화면의 기존 end-to-end
tok/s는 그대로 두고, 첫 출력 이후 구간만 보는 추정 decode 속도를 옆에 보여 달라는 요청입니다. 지금devHEAD7dc7dc99e에서 상위 행·상세·combo attempt의tok/s는src/server/management/shared.ts의tokPerSecondResult→tokensPerSecond(outputTokens, durationMs)로 만들어지고, GUI는gui/src/pages/Logs.tsx와logs-filter.ts가displayMetrics.tokPerSecond만 읽습니다. TTFT(firstOutputMs)는 이미src/usage/log.ts/src/server/request-log.ts에 쌓이고, 상세 다이얼로그에서logs.detail.ttft로 보이지만, 그 값을 빼서 속도를 다시 계산하는 표시는 없습니다. 그래서 TTFT가 긴 요청은 실제로 디코드가 빨라도 표의 tok/s가 낮게 보입니다.이슈가 제안한 식은
outputTokens / (durationMs - firstOutputMs)이고, 상위 행은 요청 기준 TTFT, combo 행은 attempt 기준 TTFT를 쓰라고 합니다. usage·양수 output·TTFT·양수 post-TTFT 구간이 없으면 unavailable, 라벨은 estimated로 두라고 했고, 영속 로그/usage history는 바꾸지 않는 display-time 파생이라고 못 박았습니다. 속도 필터(logs.filter.speed.*,minTokPerSec/maxTokPerSec) 의미는 기존 e2e tok/s에 그대로 두라는 점도 분명합니다. Logs 쪽에는 이미 #3508 필터 엔진, #3290 테이블 안정화, #3800 델타 폴링 등이 HEAD에 있어서, 새 열을 넣을 자리는 있습니다. 다만 제품 tip(#4002 근처)이나 런타임/할당량 버그보다 우선은 낮습니다.구현할 때 맞춰야 할 HEAD 좌표는 대략 이렇습니다. DTO는
requestLogDto가displayMetrics에tokPerSecond와cost만 넣는 모양이므로, decode 지표를 넣으려면tokPerSecondResult옆 형제 필드(예:decodeTokPerSecond)를 추가하고 attempt 매핑에도 같은 함수를 태우는 편이 자연스럽습니다.MetricSource/tokPerSecondResult입력이 지금은durationMs|usageStatus|usage만 받으므로firstOutputMs를 Pick에 넣고, GUIformatTokPerSecond·i18n(logs.col.tokPerSec,logs.detail.attempt.e2eNote)·필터는 e2e만 보도록 유지해야 합니다. types.ts/config.ts 분할과는 무관합니다.경로/심볼 shared.ts tokPerSecondResult - e2e 계산은 유지하고 decode용 형제 결과를 추가하는 쪽이 이슈 의도와 맞다
경로/심볼 Logs.tsx / logs-filter.ts - 표·상세에 decode 열을 넣되 speed 필터는 기존 tokPerSecond만 쓰게 분리할 것
경로/심볼 firstOutputMs (usage/log.ts, request-log.ts, responses/core.ts attempt-relative) - combo는 attempt-relative TTFT를 이미 쌓고 있으니 그 값을 재사용할 것
경로/심볼 i18n logs.col.tokPerSec / e2eNote - “추정 decode” 라벨과 e2e 주석을 헷갈리지 않게 키를 나눌 것
경로/심볼 영속 RequestLogEntry - display-time만 deriv하므로 디스크 스키마 변경은 피해야 한다(이슈 본문과 동일)
메인테이너의 판단이 필요한 지점
missing_ttft/nonpositive_decode_window처럼 새 reason을 둘지너의 추천
받아 둘 만한 enhancement입니다.
help wanted로 두고, 구현 PR은shared.tsdisplayMetrics 형제 필드 + Logs/i18n/테스트만 건드리는 작은 PR로 받으세요. 지금 당장 메인테이너가 직접 넣을 필수는 아니고, e2e 필터 의미는 절대 바꾸지 마세요. 닫을 중복도 아닙니다.이 댓글은 grok-bot이 작성했습니다
docs(devlog): lock the round-2 two-lane delivery roadmap
lidge-jun commentedon Sep 10, 2026
Closed by #4166, merged to
devas386b6a0d9a8acef818b9c40ebd472e4974750199.The Logs view now shows an estimated decode rate stacked alongside end-to-end throughput, so a slow first token is no longer indistinguishable from slow delivery. The estimate is shown only when the underlying timing is actually available; filters continue to read end-to-end throughput.
PRs here target
dev, so GitHub does not auto-close the linked issue; this is the manual close with the merge commit as evidence.feat(logs): show an estimated decode rate alongside end-to-end throug…
docs(devlog): lock the round-2 two-lane delivery roadmap
feat(logs): show an estimated decode rate alongside end-to-end throug…