[DO NOT MERGE] WIP, add/backport generic server-sent events support to Azure Core - #50081
[DO NOT MERGE] WIP, add/backport generic server-sent events support to Azure Core#50081Xiaofei Cao (XiaofeiCao) wants to merge 28 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff0a367c-664e-4de0-902b-ddaae2c86546
|
Azure Pipelines: Successfully started running 1 pipeline(s). 34 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ff0a367c-664e-4de0-902b-ddaae2c86546
Weidong Xu (weidongxu-microsoft)
left a comment
There was a problem hiding this comment.
The incremental parser needs a bounded pending-event size before this is safe to ship.
appendByte doubles lineBytes without a maximum, and completed data: lines are accumulated in data until a blank line. A server (or intermediary) can therefore send either one unterminated line or an unlimited sequence of data: lines without an empty-line dispatch and make the client retain the entire stream until heap exhaustion. This affects both the async and listener paths.
Please enforce a documented maximum line/event size and terminate the stream with a clear exception when it is exceeded. Add tests for both an oversized single line and many bounded lines without a terminating blank line.
Some thoughts(evidence collected by agent):
Spring's 256kb limit doesn't seem to fit our knowledgebase retrieval case. Since currently we don't have configuration for response body size(correct me if I'm wrong), I'd say we don't enforce this. |
|
Three minor follow-ups from the API/protocol pass:
These are non-blocking compared with the pending-event size concern. |
|
Agent-generated Java SSE interface design comparison and recommendations: https://gist.github.com/weidongxu-microsoft/f99f792a78f16cef8c9c1674c68e7d1b |
Summary aside, this is more or less the "alternative design" we've discussed. A closable stream pattern for sync. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
| * Parsing an event stream</a> | ||
| */ | ||
| @Immutable | ||
| public final class ServerSentEvent<T> { |
There was a problem hiding this comment.
I'd prefer we kept closer to the protocol layer with the typing here and remove <T>. Replacing T data with String data and letting the SSE consumer handle String -> T.
There was a problem hiding this comment.
There's ServerSentEventFrame for this: https://github.com/XiaofeiCao/azure-sdk-for-java/blob/5fa41d4a0f0c64386c5930c5c676a88a6039e32d/sdk/core/azure-core/src/main/java/com/azure/core/implementation/util/ServerSentEventStream.java#L469
In this draft, I'm exposing ServerSentEvent to user via
Flux<ServerSentEvent<KnowledgeBaseRetrievalStreamEvent>> retrieveStream();, maybe a generic one would be more suitable?
| }); | ||
| } | ||
|
|
||
| private ServerSentEvent(String id, String event, T data, String comment, Duration retryAfter) { |
There was a problem hiding this comment.
Let's not hide this constructor, it really doesn't benefit us to do so. Let's scope the constructor to properties that are required to create an event and add setters and getters for optional properties. Or, just keep everything in the constructor and pass null accordingly.
There was a problem hiding this comment.
If I'm understanding correctly, ServerSentEvent is kinda like an immutable response model.
If it's exposed to user, I'd prefer hiding it. But we could have more discussion on whether to expose it, or directly return, e.g. Flux<KnowledegBaseRetrievalStreamEvent. Would like to hear your opinion on this one.
| return true; | ||
| } | ||
|
|
||
| private static boolean hasPositiveQuality(List<String> mediaRange) { |
There was a problem hiding this comment.
Do we have any cases where Azure services use multiple Accept header values and use this part of the HTTP spec?
There was a problem hiding this comment.
Originally I thought I saw one in recording files:
https://github.com/Azure/azure-sdk-assets/blob/9e19803afadaf24ee00ff403a60eae684667944f/java/sdk/compute/azure-resourcemanager-compute/src/test/resources/session-records/ManagedDiskOperationsTests.canCopyStartIncrementalSnapshot.json#L705
checked again and it turns out to be fallback logic by us..
https://github.com/microsoft/typespec/blob/main/packages/http-client-java/generator/http-client-generator-core/src/main/java/com/microsoft/typespec/http/client/generator/core/util/MethodUtil.java#L58
Then no, haven't seen other cases. Let me remove this check.
| if (equalsIndex > 0 | ||
| && "charset".equalsIgnoreCase(parameter.substring(0, equalsIndex).trim()) | ||
| && !isUtf8(parameter.substring(equalsIndex + 1).trim())) { |
There was a problem hiding this comment.
Why do we fail if the charset isn't UTF-8? We are capable of supporting other character encodings, even have CoreUtils.bomAwareToString which returns a String based on a leading byte order mark or the charset defined in the Content-Type header.
There was a problem hiding this comment.
good question, it is for simplicity here.. yeah, we probably don't want to restrict the encoding here.
CoreUtils.bomAwareToString seems to require reading the whole stream, which may not be suitable for sse stream here, CMIIW.
I'm letting agent figure this out..
There was a problem hiding this comment.
Done by agent. Hopefully the correctness is ensured by the added tests..
There was a problem hiding this comment.
Seems sse can only be decoded as utf8:
Event streams are always decoded as UTF-8. There is no way to specify another character encoding.
See https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events-intro
| private static <T> Flux<ServerSentEvent<T>> toFlux(Response<BinaryData> response, | ||
| BiFunction<String, String, T> deserializer, TerminalEventPolicy<T> terminalPolicy) { | ||
| AtomicBoolean subscribed = new AtomicBoolean(); | ||
| return Flux.defer(() -> { |
There was a problem hiding this comment.
Should we instead write a Subscriber / Flux type for this, similar to what we did for FluxUtil.writeFile and FluxUtil.readFile?
There was a problem hiding this comment.
agent produced additional 250+ lines of code and pretty hard to reason about.
Compared to FluxUtil.writeFile, there's extra complexity which standard Reactor operator already handles(below is from agent):
For SSE it had to additionally handle parser state across buffers, zero-to-many events per buffer, queued frames, downstream demand, EOF flushing, synchronous errors, cancellation, and reentrant requests. It duplicated Reactor’s demand, cancellation, and signal-ordering machinery. Standard Reactor operators express those guarantees more clearly, so I simplified it to a per-subscription decoder pipeline with bounded read-ahead.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: daa3657a-91d5-48fa-b7ae-6983091cab6a
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c6a36a59-cfcf-4a11-b7d3-967de8071bd3
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c6a36a59-cfcf-4a11-b7d3-967de8071bd3
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 607c904a-42fc-440a-b53d-dbb61a5cbd4e
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 607c904a-42fc-440a-b53d-dbb61a5cbd4e
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 607c904a-42fc-440a-b53d-dbb61a5cbd4e
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 48a66eea-bdde-4a8f-95c6-a745a6bf01c0
Replace the custom Flux and Subscriber state machine with standard Reactor operators while preserving bounded read-ahead and response-body lifecycle behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c769c73d-fd59-442e-b297-7f98b28db34b
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 50f0c93b-2784-4012-9f6d-2f666e82a4e0
Keep terminal predicates as inclusive early-stop signals without treating EOF or HTTP 204 as an error. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 50f0c93b-2784-4012-9f6d-2f666e82a4e0
Remove the private terminal-event policy wrapper and pass the optional predicate directly through internal stream processing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 50f0c93b-2784-4012-9f6d-2f666e82a4e0
9ab9302 to
147b1ba
Compare
Scope
Adds Azure Core support for single-response generic SSE consumption. It parses one established
Response<BinaryData>; it never reconnects, replays, or sendsLast-Event-Id.ServerSentEvent<T>,ServerSentEventListener<T>,ServerSentEventStreams, incremental SSE parsing, body-owned response lifecycle, and optional typed terminal-event predicates.BinaryDatastreaming.Contract
ServerSentEventStreamsis exported fromcom.azure.core.http; parser and response-validation types remain internal. REST proxy methods return an ordinary, non-CloseableResponse<BinaryData>. Its non-replayable body owns the physicalHttpResponseand releases it when body consumption completes, fails, is interrupted, or is cancelled. Validation failures cancel an available body before propagating the error.Non-204 responses require
text/event-streamcontent. Event streams are decoded as UTF-8 regardless of an optional declared charset. Existing overloads complete on EOF or HTTP 204. Predicate overloads deliver the matching typed terminal event and complete; EOF before a match is an incomplete-stream error.Accept: text/event-streampreserves a live body only when itsqvalue is syntactically valid and greater than zero.Validation
FluxInputStream, andBinaryDatatests