27B contended throughput, recorded as contended (#440/#441) - #2340
Closed
joelteply wants to merge 1 commit into
Closed
27B contended throughput, recorded as contended (#440/#441)#2340joelteply wants to merge 1 commit into
joelteply wants to merge 1 commit into
Conversation
…441) MEASURED 2026-08-20, 6 samples of 150 tok on the live M5 lane: 4.74 / 5.25 / 6.55 / 6.57 / 6.66 / 6.67 tok/s, median 6.56, at a 22,528 PER-SLOT window with `busy_slots = 2` of 4 on EVERY sample. NOT written into `tokens_per_second`, deliberately. That field is what #441's collapse alarm reads as the SINGLE-STREAM expectation, and 6.56 is a contended rate — `ThroughputBaseline`'s own doc says "single sequence". Putting it there would drop the alarm's floor from ~4.3 to ~1.6 tok/s and let a genuine collapse pass silently. The pinned 17.2 stays. Both numbers are true; they measure different machines. THE GAP IS THE POINT: the sentinel cannot currently tell "contended" from "degraded" because nothing records concurrency at measurement time. That is the sibling of the window axis landed in #2339, and it is #441's remaining half. MTP held 83.9-84.8% draft acceptance across all six, and 80.3-84.3% across three earlier samples on a different prompt. Acceptance is FLAT while t/s moves 1.4x, so the draft head is not the variance source. Also corrects my own earlier reading: I took `-c 89280` off the lane argv and called it the live window. It is the TOTAL across 4 slots; per-slot is 22,528 against the catalog's 19,712 — 1.14x, not the 4.5x I claimed. The window was never the explanation. cargo check exit 0; model_registry tests exit 0 (both read by exit code). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
6 samples on the live M5 lane: median 6.56 tok/s at a 22,528 per-slot window with
busy_slots = 2of 4 every time. MTP held 83.9–84.8% draft acceptance throughout — flat while t/s moved 1.4×, so the draft head is not the variance source.Deliberately NOT written into
tokens_per_second. That field is #441's single-stream expectation; 6.56 is contended. Recording it would drop the collapse floor from ~4.3 to ~1.6 tok/s and let a real collapse pass silently. The pinned 17.2 stays.Both numbers are true and measure different machines. The gap is the finding: the sentinel can't distinguish contended from degraded, because nothing records concurrency at measurement time — the sibling of the window axis landed in #2339, and #441's remaining half.
Also corrects my own earlier claim:
-c 89280is the total across 4 slots, not the live window. Per-slot is 22,528 vs the catalog's 19,712 — 1.14×, not 4.5×.Comment-only change to one file. cargo check 0, model_registry tests 0.
🤖 Generated with Claude Code
https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo