Warmup times itself - #2
Merged
Merged
Conversation
The file's own summary said the timing was yours to measure "because this file does not time itself", and noted that twill has had mono_ns since 1.7 and nothing here called it. docs/needs.md entry 3 said the same thing more plainly: delivered, not taken up. Every pass is timed now, and the report gives the first pass against the median of the rest. That difference is the number warmup exists to produce. The median rather than the mean because one stall in the middle of a warmup would drag a mean far enough to hide the first pass, which is the thing being measured against it. It claims nothing from fewer than three passes, and nothing when the first pass was no slower than the others: a model with no lazy work in it gets an honest silence rather than a saving made of noise. tests/warmup_test.tw is new, and the file had none. It asserts the refusals rather than the speed, because a timing assertion is a claim about the machine the tests run on. One of its own assertions was wrong first: it checked that a failed warmup's report contains no "ms", which the word "warms" contains. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The file's own summary said the timing was yours to measure "because this file does not time itself", and noted twill has had
mono_nssince 1.7 with nothing here calling it.docs/needs.mdentry 3 said the same: delivered, not taken up.Every pass is timed now, and the report gives the total plus the first pass against the median of the rest. That difference is the number warmup exists to produce — what it moved off the first request. The median rather than the mean, because one stall mid-warmup drags a mean far enough to hide the first pass, which is the thing being measured against it.
It claims nothing from fewer than three passes, and nothing when the first pass was no slower than the others: a model with no lazy work in it gets an honest silence rather than a saving made of noise.
tests/warmup_test.twis new — the file had none. It asserts the refusals rather than the speed, because a timing assertion is a claim about the machine the tests run on, and a CI runner that stalls in the wrong place would turn it into a failure that says nothing.One of its own assertions was wrong first: it checked that a failed warmup's report contains no
"ms", which the word warms contains. It now asserts the whole report instead, which is what a "says only this and nothing appended" check should have been.Also bumps the pin, README and CI to 1.9.0. Seven suites pass.
🤖 Generated with Claude Code