Repository navigation
Stream the HttpURLConnection body off the wire - #37
Merged
Merged
Conversation
clj-http-lite's `:as :stream` did not return until the whole body had arrived: perform! read the response buffered, so getInputStream handed back a stream over a body already read to EOF and an SSE feed came out in one piece at the end. perform! now asks read-response for a live body, the same way the java.net.http path does, and getInputStream / getErrorStream wrap its pull in make-body-stream. The connection goes back to the pool when the body is read to its end and is closed when the stream is closed early, on disconnect, or when a read fails. Redirect hops drain their body before following. The socket read timeout stays armed for the body here, which is HttpURLConnection's contract and what lets a reader bound the gap between chunks. Every other `:as` drains the same stream, so gzip/deflate decoding is unchanged. openStream now throws on an error status like the JVM's. The persistent test server closed each connection twice, once from stop and once from its serving thread, and the second close hit whatever socket had reused that fd in the next test. stop now shuts connections down and only the serving thread closes them.
Each test stood up its own server on port 18110. stop closes the listener, but on Linux a thread parked in accept() keeps it listening, so every test after the first failed with bind() failed on CI. One server for the namespace, through a :once fixture.
Tracking nightly surfaces a jolt change that breaks this library the day it lands. The floor in deps.edn stays at 0.8.16, but CI no longer tests it.
with-redefs is process-wide, and the in-process TLS server closes and frees its end of the same connection on its own thread. On Linux that landed inside the counting window and read as a second close, failing closing-a-tls-stream-twice-releases-once on the nightly leg.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
clj-http-lite's
:as :streamreturned only after the whole body had arrived: the HttpURLConnection shim read the response buffered, so an SSE feed came back in one piece at the end. The shim now hands back a live body the same way the java.net.http path does, and returns at the headers. The connection is pooled once the body is read to its end and closed when the stream is closed early, on disconnect, or when a read fails. Every other:asdrains the same stream, so decoding is unchanged. openStream now throws on an error status like the JVM's.Measured against a server sending one chunk a second for 3 s: headers at 3 ms and a read per chunk, where it was headers and the whole body at 3011 ms.
Also: