bench: register the HTTP/3 samples so their numbers survive - #174
Merged
Conversation
Http3/Managed and Http3/Streamed were benchmarked all evening through throwaway shell loops, which is how a body-size mismatch and two h3x flag mistakes got into the record before being caught. Neither had a baseline, so nothing would have noticed them regressing. Both are in bench/samples.tsv now, which asserts full coverage of the Playground - 40 samples, 40 registered - so a sample nobody can benchmark shows up as a gap rather than as silence. First run, 2 reactors, 64 conns, 10s: Http3/Nghttp3 480371 req/s 4.17us +0.9% vs baseline Http3/Buffered 516707 req/s 3.87us +3.2% Http3/Managed 665936 req/s 2.96us (new) Http3/Streamed 89906 req/s 22.03us (new) The two pre-existing samples are unchanged within noise, which is the point of having them there.
Three panes had drifted or were missing. Http3/Nghttp3's pane went stale when #171 gave that sample a body knob - the exact drift the generator exists to prevent, and it only stayed hidden because nothing regenerates on merge. Http3/Managed and Http3/Streamed had no pane at all, having been added the same evening they were written. All 40 samples are generated now. The only hand-written panes left are `vs` and `pxmatrix`, which are prose comparisons with no sample to derive from. Verified: tabs, labels, panes and CSS agree; both generators are idempotent; the new panes compile standalone against the real packages.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Http3/ManagedandHttp3/Streamedwere benchmarked all evening through throwaway shell loops. That is how a body-size mismatch (8 KiB against 13 bytes) and twoh3xflag mistakes got into the record before being caught — and neither sample had a baseline, so nothing would have noticed them regressing afterwards.Both are in
bench/samples.tsvnow. That file asserts full coverage of the Playground — 40 samples, 40 registered — so a sample nobody can benchmark shows up as a gap rather than as silence.First run
2 reactors, 64 connections, 10s:
The two pre-existing samples are unchanged within noise, which is the point of having them in there.
Http3/Streamedis registered withPLAYGROUND_CHUNKS=8 PLAYGROUND_CHUNK_BYTES=1024so its cell measures a genuinely chunked response rather than a single-chunk one.