http2: the write queue, a pure-C# client, streaming both ways, and nghttp2 retired - #176
Merged
Conversation
The response half only, as agreed: an h2 handler can now push body bytes as it produces them, each flush becoming a DATA frame, instead of returning a finished Http2Response. That is what unblocks a streaming IResponseContent, and what makes an endless response expressible at all. The part HTTP/3 did not have is flow control. Every stream here rides one TCP connection and a write needs credit in BOTH the stream's window and the connection's, so FlushAsync waits for a WINDOW_UPDATE rather than resetting the stream the way the buffered path does - a whole body was always in hand there, so running out of credit could only mean the peer had stopped reading. Verified correct: 8192 bytes exact on an 8 x 1KiB response, and 1 MiB delivered whole - sixteen times the default 65535 window, which is what proves the wait works rather than the stream dying at the first window. One leak found and fixed on the way. The streamed dispatch skips the buffered path's tail, and that tail is what retires the stream, so every response was leaking its PendingRequest arena: 20 GB of RSS over eight seconds of load. Retiring the stream in the writer's finally brings that to 97 MB, and doubles throughput as a side effect. NOT READY. Streamed still runs 93990 req/s against the buffered path's 1773623 on the same 8 KiB - a 19x gap I have not diagnosed. Eight flushes per response against one is part of it, but not obviously all of it, and shipping a streaming path that slow would mislead anyone who reached for it. No playground pane, no registry entry, no version bump until that is understood. Unit 29, E2E 46, Http 35 pass.
A streamed response measured the same whether it carried 2 bytes or 8 KiB -
408k against 355k req/s - and that flatness is the shape of a syscall, not of
work. Each response was forcing its own transport write, where the buffered
path spreads ONE write across every stream in flight on the connection. That
alone was the whole gap, and at 2 bytes it was 12.8x.
So a streamed writer no longer writes for itself while the read loop still
owes a flush: it stages, and the pass carries it out alongside every other
response. Coalescing is the reason buffered h2 is fast and there is no reason
a streamed response cannot share it.
It has to stay bounded, though not for memory. A producer that loops without
awaiting anything of its own has no yield point except that write, so skipping
it unconditionally spins the reactor and the response never moves at all -
which is exactly how two earlier attempts at this broke the endless case, both
times silently, because a benchmark only measures responses that END. Past
16 KiB staged the write happens for real and hands the thread back. A paced
producer never reaches the limit: it parks, the pass flush takes its chunk out
immediately, and it resumes outside the pass where the write is unconditional.
before after buffered
2 bytes 408,483 3,873,462 5,235,162
1 x 8192 355,209 585,830 1,801,105
8 x 1024 93,006 569,832 1,801,105
Chunk count stopped mattering - 8x1024 and 1x8192 now sit within 3% where they
differed 3.8x, because writes follow bytes rather than how often the handler
asks for a flush.
What remains is per-byte, not per-write: raising the limit to 128 KiB changed
nothing, and streamed still falls off with body size faster than buffered.
That points at the staging copy - the handler fills an ArrayPool block that is
then copied into the pipe, where the buffered path writes the body once.
Framing the DATA header directly into the pipe's span would remove it.
Verified unchanged: 8192 exact, 1 MiB through a 65535 window, endless /feed
still trickling, RSS flat at 98 MB. Unit 29, E2E 46, Http 35 pass.
DispatchReadyAsync awaited each handler in turn, so one request that parked -
a database, an upstream, a disk - held up every other stream on that TCP
connection, including responses already produced and staged with nowhere to
go. Two requests on one connection, one sleeping a second:
/slow /fast
before 1.02s 1.02s <- waited for /slow
after 1.02s 20.99ms
That is the whole point of multiplexing, and h2 is the worst place to lose it:
QUIC streams are independent, so an h3 handler that parks inconveniences only
itself, but on h2 everything shares one connection and one dispatch loop. Both
h3 modules already dispatch this way, buffered and streamed alike. h2 was the
outlier.
A handler that answers synchronously - nearly all of them - stays inline, so
its response is still staged in time for the pass flush and still leaves with
every other one in a single write, and there is no Task to allocate. Only a
handler that actually parks is detached, and it writes its own bytes when it
finishes, since the pass flush has gone by.
Detaching means nothing is awaiting the tail, so the tail has to do the work
the loop used to do in its finally: retire the request, and flush. Skipping
exactly that is what leaked 20 GB in the streamed path. An escaping exception
would also vanish silently and leave the peer waiting on a stream that never
comes, so it is caught, logged and answered with a 500 - the buffered path
previously let it kill the connection.
No cost to the fast path: 2 bytes 5353304 req/s (was 5235162), 8 KiB 1783354
(was 1801105) - both inside noise. Unit 29, E2E 46, Http 35 pass.
nghttp2 has the same defect at RunBuffered.cs:94 and is deliberately left
alone; the managed stack is where the effort goes.
…e turns Non-blocking dispatch made handlers complete outside the pass, and the write path could not accept them: a PipeWriter permits no Write while a flush is outstanding, so every asynchronous handler faulted with "Cannot write while flush is in progress" and served nothing at all. Frames produced during a flush now land in a queue; when the flush completes the whole queue moves into the pipe and leaves as the next one. So responses that completed during one transport write share the single write after it, which extends the pass coalescing to handlers that finish outside the pass - which is every handler that touches a database or an upstream. Callers who queued await that turn, keeping backpressure and the yield a real flush gave them. Kestrel's Http2FrameWriter and Go's net/http2 writer take this shape. h2load -t4 -c32 -m16 -D8, 2 reactors, 3 reps: async handler, Task.Yield 0 bytes (faulted) -> 1928617 req/s async handler, Task.Delay(1) 0 bytes (faulted) -> 352000 req/s streamed 8x1KiB 551000 -> 1346000 (2.44x) buffered 2 B 5180000 -> 5110000 (-1%) buffered 8 KiB 1754280 -> 1768103 The coalesce limit is now per writer rather than per pass: many responses coalescing into one large write is the point, and the per-pass version split that write and cost a third of streamed throughput. First tests for any of this - the fake transport enforces the real contract, that a write during a flush throws and so does a second flush, so the case that used to break is the case under test.
The h2 client was the last thing holding the nghttp2 binding in the tree. It is the same framing, HPACK and flow control ioxide.http2 already runs for the server, pointed the other way round: the preface, odd stream ids, requests instead of responses, and the retry rules that decide whether a failed exchange may be sent again. Those types stay internal to ioxide.http2 and reach the client through InternalsVisibleTo. Duplicating them was the alternative, and two copies of an HPACK encoder is how the two drift; promoting them to public API would have made the package's surface bigger than its job. Kept from the binding, because the hazards are the client's own rather than nghttp2's: completions are recorded during a parse and resumed after it unwinds, so a resumed caller that submits again - or retries through the pool, which may dispose this connection - never re-enters the parser from inside itself. Verified against nginx over h2c and over TLS with ALPN, not just against our own server: GET, a POST body that reaches the origin, 25 requests multiplexed onto one connection, MaxResponseBytes, and a trailered response. Those five were skipping for want of a sidecar; they run now. Two more cover what a 1 KiB GET cannot reach - a 1 MiB body that has to park on the flow-control window and resume on each WINDOW_UPDATE, and a header block that has to leave as HEADERS plus CONTINUATION. Http 37 pass, 0 skipped.
ioxide.http2 started as the drop-in that needed no native library and ended as the only HTTP/2 here. It measured level with the binding (0.98x-1.09x on a small body, the ordering depending on the connection-to-reactor ratio rather than the codec), then grew past it: streamed responses, streamed request bodies and non-blocking dispatch all landed on the managed side, while the binding kept the blocking dispatch loop where one slow handler held up every other stream on the connection. Two implementations of one protocol is a tax paid in samples, docs, tests and benchmark fixtures, and the second one had stopped buying coverage of the protocol's darker corners - it was buying a native build step. dropped/ is where retired code goes: out of ioxide.slnx, out of CI, off NuGet, kept readable because the reasoning is easier to follow with the thing itself still there. Its README says so. The five samples that used it move to ioxide.http2. Four are the three-name swap the packages were designed for - Proxy/H2ToH1, H2ToH2, H2ToH3 and Http2/SslStream. The fifth, Http2/Tls, was already ported: Http2/ManagedTls is that same server on the managed stack, so keeping both would have been one sample twice. Also corrects comments the drop made false - the pure-C# module described itself as an alternative to nghttp2, its response type claimed bytes were copied into nghttp2 at submit, and the h2c sample said its read loop fed it. Unit 33, Http 37, E2E 46 pass. Solution builds with no reference to the binding left outside dropped/.
h2 could stream a response but never a request: Http2Request carried only Body, so the whole upload was assembled before the handler saw any of it, and MaxRequestBytes was the only thing standing between a hostile peer and the arena. h3 has had Http3Request.BodyReader all along; this is its counterpart. Http2Options.StreamRequestBodies dispatches at the HEADERS and hands the handler an Http2BodyReader. The stream then stays in _streams while it runs - DATA frames still have somewhere to go - and is retired when the handler is done rather than when the request ends. What makes it worth having is where the credit goes. A chunk opens the peer's window only as the handler READS it, so a slow consumer stops replenishing and the peer stops sending: memory is bound by one window instead of by the body. Crediting on arrival, which is what the buffered path does and should, would leave the bound off. Unlike h3 the credit is shared - every stream is on one TCP connection - so a handler that never reads holds down the connection window for every other stream too, and the comment says so. Wakes are deferred to after the parser unwinds, the same discipline the h3 reader and the write queue already use, so a resumed handler cannot re-enter the parser mid-frame. Tests: 1 MiB uploaded through the pure-C# client into a streaming origin, and three unit tests on the part end-to-end cannot see - that no WINDOW_UPDATE is emitted while a handler holds the body unread, that a bodyless request reads empty instead of parking forever, and that buffered dispatch still assembles and still credits on arrival. Unit 36, Http 38, both 0 failed.
… ways "Managed" stopped meaning anything when nghttp2 left - all HTTP/2 here is pure C# now - and "streamed" never said WHICH direction, which is the thing a reader actually needs to know. Samples are named for what they do, and the library appears only where two of them still exist: Http2/Managed -> Http2/Buffered Http2/ManagedStreamed-> Http2/StreamedResponse Http2/ManagedTls -> Http2/Tls Http3/Managed -> Http3/Buffered Http3/ManagedStreamed-> Http3/StreamedBoth Http3/Nghttp3 -> Http3/Nghttp3Request Http3/Buffered -> Http3/Nghttp3Buffered Http3/Streamed -> Http3/Nghttp3Response Two new h2 samples close the ladder against h3's: StreamedRequest, where the body arrives a chunk at a time and reading it is what credits the peer, and StreamedBoth, whose /echo reads and writes at once - the shape a proxy needs, and the reason the two directions are separate switches rather than one. Writing them found two real bugs, which is the argument for samples that run: ReleaseAllCreditWaiters enumerated _creditWaiters while waking writers, and those wake INLINE and re-register at once - "collection was modified", thrown from the teardown finally, so it escaped the catch that exists to stop a malformed peer looking like a server fault. The stream-0 path had the mirror of it, clearing after releasing and so discarding the waiter a resumed writer had just added, parking it forever. Both take the waiters out before waking any. And PendingRequest.SendWindow opened at the RFC default of 65535 instead of what the peer's SETTINGS advertised. Streams are created long after those SETTINGS arrive, so a response longer than 65535 bytes stalled waiting for a WINDOW_UPDATE the peer had no reason to send - it believed we still held its whole window. Streamed responses had been dodging it by dropping the stream from _streams, which also dropped the per-stream window from the credit calculation; keeping the stream for a streamed request is what exposed it. /feed went from 64 KB in three seconds to 926 MB. Verified by running them: 8 MiB through /echo exactly, 50 MiB uploaded to /upload, /feed endless. Unit 36, Http 38, E2E 46, Chaos 37.
The http/3 menu had TWO tabs reading "nghttp3 · streamed" - one was request
streaming, one was response streaming, and nothing on the page distinguished
them. The rest were named after a library ("pure c#"), which says what a sample
is built from and not what it does.
Every tab now names the direction, and the library appears only in http/3 where
two implementations still exist:
h2c · buffered h3 · buffered
h2c · response streamed h3 · request + response streamed
h2c · request streamed h3 · buffered (nghttp3)
h2c · both streamed h3 · request streamed (nghttp3)
h2 · tls & alpn h3 · response streamed (nghttp3)
h2 · over sslstream
Three of those h2 tabs are new: response streaming was never on the site at all,
and request streaming and both-directions did not exist until this branch. Their
notes are about the trade rather than the API - buffered bounds nothing but
MaxRequestBytes, streamed bounds one flow-control window because a chunk credits
the peer only as the handler reads it.
The two nghttp2 panes are gone with the package. The proxy panes said they
needed ioxide.nghttp2, which would now fail to restore; they take ioxide.http2.
The h2-over-TLS pane no longer explains itself as a diff against a tab that does
not exist, and the learn pages stop offering a choice between two h2 packages.
Panes regenerate idempotently from the samples, no reference to a removed tab is
left in the page or the stylesheet, and no mention of nghttp2 survives anywhere
under docs/.
MDA2AV
changed the title
All ten packable projects move together, as they always have. The gate on bumping was the async-workload numbers, and those exist now: an asynchronous handler went from serving 0 bytes to serving, streamed responses measure 2.45x, and buffered 8 KiB is unchanged. ioxide.nghttp2 is not among them any more - it stays at 0.4.169 in dropped/, which is genuinely its last published version. That also means 0.4.176 is the first release where ioxide.httpclient does not pull it in: its dependencies are ioxide, ioxide.http2, ioxide.nghttp3 and ioxide.ngtcp2, so the HTTP/2 half of the client no longer ships a native library. The h3 half still does. Verified by packing: every inter-package dependency resolves to 0.4.176, and CI's pack steps match the ten projects exactly.
MDA2AV
marked this pull request as ready for review
August 10, 2026 15:04
Both were found by asking what nghttp2 was buying us beyond speed. The answer was hardening, and these are the two most-published HTTP/2 denial-of-service vectors of recent years - neither of which the managed server defended against. MaxConcurrentStreams was advertised in SETTINGS and never enforced: the option appeared exactly three times, none of which compared it to _streams.Count. So a peer could open unbounded streams, each costing a PendingRequest and a pooled arena, and "open a stream, reset it, repeat" (CVE-2023-44487) cost the peer nothing. Streams past the limit are now refused with REFUSED_STREAM, which RFC 9113 8.7 makes safe for the peer to retry elsewhere. The header block was unbounded. MaxFrameSize caps one frame at 16 KiB, but nothing capped how many CONTINUATION frames follow a HEADERS that never sets END_HEADERS, so the accumulated block grew until the process died - the CONTINUATION flood. MaxHeaderListSize bounds it, is advertised, and exceeding it is a CONNECTION error rather than a stream one, because a block that stops being decoded desynchronises HPACK for everything after it. The subtlety in both: a refused or over-long block still has to be DECODED. HPACK is one stream across the whole connection, so skipping a block would desynchronise the table for every later request. A refused stream's block decodes into a shared scratch and is thrown away, and that scratch is bounded too - otherwise refusing a stream would itself be the way in. The tests fail without the fix, which is the only reason to trust them: the flood test wedges for the full 120s timeout, and the stream test sees no RST_STREAM at all. Chaos 39 pass, 0 failed.
Retiring it traded away something the benchmark could not see. It is the reference implementation: continuously fuzzed, patched by people whose job it is when the next HTTP/2 CVE lands, and carrying a decade of interop against every other stack. The two DoS vectors closed in the previous commit are exactly the class of thing that buys - both were vectors nghttp2 had defended against for years and the managed server never had. So it is back in src/, in the solution, in CI's pack list and at 0.4.176, and dropped/ is gone with it - an empty folder documenting a decision that was reversed is worse than no folder. What is NOT restored is the client. ioxide.httpclient stays on the managed stack, which measured 1.35x-1.39x the binding as a client and is where the features are. So nothing depends on ioxide.nghttp2 now: it is a standalone server-side option a user opts into, not something pulled in transitively. The sample comes back as Playground/Http2/Nghttp2Buffered - the naming scheme puts the library back on the tab now that there are two h2 implementations again, and Buffered is the honest suffix because buffered is all it does. Its pane says so, and says what that costs: no streamed response, no streamed request, and the dispatch loop still waits for each handler in turn. Unit 36, Chaos 39, Http 38 pass. The sample serves.
The binding kept the dispatch loop the managed stack had already been fixed
out of: DispatchReadyAsync awaited each handler in turn, so one request that
parked held up every other stream on that TCP connection - including responses
already submitted with nowhere to go. Two requests on one connection, one
sleeping a second, over nghttp:
/slow /fast
before 1.02s 1.02s
after 1.01s 17.01ms
Handlers that answer synchronously stay inline, so they still submit in time
for the pass drain and allocate no Task.
The drain needed the guard first, and for the reason the managed stack needed a
whole write queue: a handler finishing late submits and drains from outside the
read loop, and two drains interleaving would write out of the single _egress
buffer while a flush was outstanding - which a PipeWriter refuses outright.
Here one flag is enough, because nghttp2 holds the queued frames itself: a
caller that arrives mid-drain sets _drainAgain and the in-flight drain loops
once more to pull what was just submitted.
The detached tail also has to do what the loop would have: submit, answer 500
and log if the handler threw, retire the request, and drain - nothing observes
that Task, so an escaping exception would leave the peer waiting on a stream
that never comes.
This is the prerequisite for streaming. Streaming on a dispatch loop that
blocks would mean a streamed response parking the whole connection.
Unit 36, Chaos 39, Http 38, E2E 46 pass.
Chaos covered only the managed Http2Connection, and the Http suite's h2 tests moved to the managed client - so a package we just committed to supporting had essentially no automated coverage, and the dispatch fix in the previous commit had none at all. The wire is identical, so H2cClient is shared and the same assaults point at Nghttp2Connection: bad preface, oversize frame, unknown frame types, a frame truncated mid-payload, and a CONTINUATION flood. That last one is worth stating, because it answers what "battle-tested" buys in something other than adjectives: nghttp2 refuses the flood with NO configuring, where the managed server had to be taught MaxHeaderListSize this morning. The same test, the same client, two implementations, one of which had the defence already. The head-of-line test asserts on ORDER rather than elapsed time - the first response to come back must be /fast, not the /slow stream dispatched before it. That is deterministic where a stopwatch is not, and it fails against the loop it replaced: made blocking again, it reports "expected [3], got [1]". Chaos 46, Unit 36, Http 38, E2E 46.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HTTP/2 becomes the only HTTP/2 here: nghttp2 is retired, the client is pure C#, both directions stream, and the samples say what they do.
Async handlers could not write at all (
b771050)The non-blocking dispatch fix made handlers complete outside the dispatch pass, and the write path could not accept them - a
PipeWriterpermits noWritewhile a flush is outstanding. Every asynchronous handler faulted withCannot write while flush is in progressand served nothing. Which is every handler that touches a database or an upstream.Staging and flushing take turns now: frames produced during a flush queue, and the whole queue moves into the pipe when it completes, so responses that finished during one transport write share the single write after it. That extends the pass coalescing past the pass. Kestrel's
Http2FrameWriterand Go'snet/http2writer take the same shape.h2load -t4 -c32 -m16 -D8, 2 reactors, 3 reps:Task.YieldTask.Delay(1ms)The 2.44x is not the headline; the first two rows are. Buffered's -1% is the two branches added to
Stage, and worth it.A pure-C# HTTP/2 client (
778fb62)The h2 client was the last caller of the nghttp2 binding. It is now the same framing, HPACK and flow control the server runs, pointed the other way: preface, odd stream ids, and the RFC 9113 8.7 rules for when a failed exchange may be resent. Those types stay internal to
ioxide.http2and reach the client throughInternalsVisibleTo- two copies of an HPACK encoder is how the two drift.Verified against nginx, over h2c and over TLS with ALPN. The five h2 client tests had been skipping for want of a sidecar; they run. Two more cover what a 1 KiB GET cannot: a 1 MiB body that parks on the window and resumes per
WINDOW_UPDATE, and a header block that leaves as HEADERS + CONTINUATION.nghttp2 retired to
dropped/(b7fafdf)It measured level with the managed stack (0.98x-1.09x, ordering depending on connections-per-reactor rather than the codec) and then stopped keeping up: streamed responses, streamed requests and non-blocking dispatch all landed on the managed side, while the binding kept the blocking dispatch loop. Two implementations of one protocol is a tax on every sample, doc, test and benchmark fixture, and this one had stopped buying protocol coverage - it was buying a native build step.
dropped/is out of the solution, out of CI, off NuGet, and readable, with a README saying why. Its five samples moved toioxide.http2.Request bodies stream too (
0964f98)Http2Options.StreamRequestBodiesdispatches at the HEADERS and hands the handler anHttp2BodyReader, the counterpart of h3's. A chunk credits the peer's window only as the handler reads it, so memory is bound by one window instead of the body, and a slow consumer makes the peer slow down. Unlike h3 the credit is shared, so a handler that never reads holds the connection window down for every stream on it.Samples named for what they do (
86c44ba,d7e2120)"Managed" meant nothing once nghttp2 left, and the site had two tabs both reading "nghttp3 · streamed" - one request, one response. Every tab now names the direction; the library appears only in h3, where two implementations still exist. Three h2 tabs are new: response streaming was never on the site, and request/both did not exist.
Writing the samples found two real bugs, which is the argument for samples that run:
ReleaseAllCreditWaitersenumerated_creditWaiterswhile waking writers that wake inline and re-register at once. Thrown from the teardownfinally, so it escaped the catch that keeps a malformed peer from looking like a server fault. The stream-0 path had the mirror, clearing after releasing and discarding the waiter a resumed writer had just added.PendingRequest.SendWindowopened at the RFC default 65535 rather than what the peer's SETTINGS advertised. Streams are created long after those SETTINGS arrive, so any response past 65535 bytes stalled waiting for aWINDOW_UPDATEthe peer had no reason to send. Streamed responses had been dodging it by dropping the stream from_streams- which also dropped the per-stream window from the credit calculation - and keeping the stream for a streamed request exposed it./feedwent from 64 KB in three seconds to 926 MB.Verification
Unit 36, Http 38, E2E 46, Chaos 37 - all 0 failed, 0 skipped. Solution builds with no reference to the binding outside
dropped/. Panes regenerate idempotently and no mention of nghttp2 survives underdocs/. Samples run: 8 MiB through/echoexactly, 50 MiB uploaded,/feedendless.The h2 client tests need nginx sidecars on 14464/14465/14466; their configs are not in the repo, so they skip silently without them. Worth committing.
0.4.176 across all ten packages.
ioxide.nghttp2is not among them - it stays at 0.4.169 indropped/, its last published version - so this is the first release whereioxide.httpclientdoes not pull a native library for HTTP/2.