Skip to content

fix(chat): Stream clause deltas instead of whole lines - #877

Draft
stickerdaniel wants to merge 28 commits into
mainfrom
fix/stream-clause-chunking
Draft

stickerdaniel wants to merge 28 commits into
mainfrom
fix/stream-clause-chunking

Conversation

@stickerdaniel

Copy link
Copy Markdown
Owner

A streamed reply stalls before it moves. Line chunking buffers text until a newline arrives, so a paragraph of prose produces no delta at all while the model is generating it, and the reader watches a still screen until the paragraph lands whole. Measured on a three-paragraph answer at 45 tokens/s, 1244ms passed with no update; on a single paragraph containing no newline, nothing is written until the reply ends. The presentation scheduler on the base branch paces the words inside a burst, but it cannot fill the silence before one.

Both call sites now share one constant that names the agent's own default. Omitting the option would not reach it, because the agent forwards undefined to the AI SDK's smoothStream, whose default is word chunking instead. The cost is more writes, and it is bounded by the throttle rather than by the chunk size: DeltaStreamer.addParts coalesces everything arriving inside one window, so the same answer costs 30 writes rather than 2 and can never exceed ten per second of generation. The guard measures the stall on newline-free prose instead of asserting the value, since the value alone says nothing about which setting stalls.

Two suites had replaced all of @convex-dev/agent with the calls they assert on, which failed them at load as soon as anything new was imported from it; they now use the importOriginal shape a third suite already used. Verified with the full unit suite, the Convex type check, and the compat check.

Adds the transitions.dev recipe vocabulary as a shared `t-*` block in
layout.css, and puts the first two to work: the checkbox fills its box
then strokes the tick in, and the switch thumb overshoots and settles
instead of sliding flat.

The tick has to stay mounted to have a dash offset to draw from, and the
thumb keyframes are gated behind a first-toggle class so a page full of
switches does not play its return bounce at mount.

Switch and Toggle come from the shadcn-svelte registry; Toggle picks up
the registry's current icon-padding variants along the way.
Switch has no call site in the app yet, so the shadcn demo harness is
the only place its thumb travel can be seen. Checkbox and Toggle join it
so the three controls can be compared side by side.
Shimmer text, thinking states, icon swap, the learn-more chevron and the
avatar-group hover action, each wired to the `t-*` rules in layout.css.

Timings are read back from the CSS custom properties rather than
repeated in code, so retuning a recipe in layout.css does not leave the
orchestration behind on the old numbers.
The badge slides onto its trigger and pops its dot on a clock of its
own, so the trigger underneath never moves, and it stays mounted at
zero so the closing leg can play.

Only a per-thread boolean is stored and only the last admin reply is
retained, so several replies collapse into one flag and a message-level
count is unrecoverable. The badge therefore counts conversations off
the existing unread index, capped one above the 9+ label. The per-row
indicator has no count to show and stays a bare dot.

The separating ring is gone with it.
Both icons stay mounted in one grid cell and trade opacity, blur and
scale. An {#if}/{:else} destroys the outgoing icon, so there is nothing
left to fade and the swap reads as a hard cut; the password toggle was
built that way and could not animate at all.

The theme toggle is left alone: mode-watcher suppresses every
transition on the page while it flips the theme class, and the circular
reveal already covers the icon change.
Connecting, Thinking and the closing duration were three separate
mounts, so each state arrived as a cut. They are now one line that
carries its own swap: the outgoing copy leaves upward through a blur
while the incoming one rises from below, and the shimmer stops once the
model does.

Streamed words fade in from a blur instead of appearing. svelte-
streamdown types its easing as a keyword union and its blur is not a
prop, so the timing is overridden in CSS against the inline animation
it writes per word.

Both surfaces share this, the AI chat and the support agent.
The faces on the support greeting say who will answer; they are not a
choice the user can make. Hovering one lifts it and its neighbours by a
distance falloff, and the row carries no cursor, focus ring or click
target.

A trailing chevron on a link now slides along the reading direction and
opens its two arms into an arrow. Drawn as two strokes rather than one
polyline because the arms rotate independently about the apex.
The recipes drive their motion through class hooks the components spell
out by hand, and a rule whose hook nobody applies fails silently in the
worst direction: the badge's closing rule targeted a class the markup
never set, so a red zero sat on the support launcher on every page
while every check passed.

Reading a token is renamed with it, since the values are milliseconds,
pixels and bare scale factors rather than durations alone.
The badge stays in the DOM at zero so its closing leg can play, so
counting elements now counts one that is deliberately collapsed. The
two checks that meant "no unread reply" filter on what is visible
instead. Confirmed against a real Playwright run: closed resolves to
one mounted and zero shown, open to one shown.
Selecting the markup on the entering flag put an already-entered label
in the entrance branch forever, so from the second swap on the outgoing
one never got is-exit: it sat under the incoming label and vanished
when the settle timer removed it. The label is one element now and the
start state is written by the action, which is the only place that
knows the line is new. Measured on a live chat: both Connecting and
Thinking now exit.

The reduced-motion block missed the switch thumb's transition-transform,
which covers the same translate property the keyframes animate, so the
thumb still slid the length of the track.

The hook guard compared substrings, so it passed for t-stream on
application/octet-stream and for t-switch on t-switch-thumb. It compares
whole class tokens now, and both probes fail without their class.
The status line and the shimmer both paint their label more than once:
a swap keeps the outgoing and incoming copies in the DOM together, and
the shimmer repeats the string as generated content. Chromium exposes
both, so a swap read as "Connecting Thinking" and a shimmering label
read twice. The painted copies are hidden now and the label reaches
assistive technology from one node holding only the current text.

A box with both flags set reports itself as mixed, and the tick is
dashed out in that state, so gating the minus on !checked left it
visibly empty while telling assistive technology otherwise. The
harness now carries that state so it cannot go unnoticed again.

Also: the badge holds its last real number through the close instead
of flashing a red zero, its dot no longer replaces the caller's grid
with block and pushes the number off centre, --think-gap now holds the
entrance back as its name claims, and the hook guard no longer accepts
a comment or a CSS selector as proof that a class is applied.
Hiding the painted status text left the accordion trigger it fills with
no accessible name, because a role=status child does not contribute to
name-from-content. Measured in Chromium on the real component: the same
button is named with a plain aria-live node and unnamed with the role.
The label keeps aria-live and loses the role.

The switch thumb moved on keyframes, which restart from their own first
frame, so reversing a switch mid-travel snapped it to the far end first.
It is a transition now, and the overshoot lives in the easing curve,
which interpolates from wherever the thumb is: measured reversing at 13
of 14px, it continues down instead of jumping, and a plain toggle still
passes 104% before settling.

Under reduced motion the status entrance still parked the incoming
label at zero opacity, so the line went blank for the gap and then
blinked. Also: hover lift only where hover exists, since a tap emits
mouseenter and never mouseleave; the harness card is marked English
like its neighbour; will-change is gone from five settled elements that
held compositor layers for good; --think-hold had no consumer; and
twelve em dashes should never have been written.
The unread badge now shows a number to sighted users and names that same
count on both launcher variants, including a localized more-than-nine
label at the query cap. A real seeded anonymous thread exposed the
button as "Open feedback, 1 unread support conversation".

Switch travel is owned by one unlayered rule in both directions. The
positive Tailwind checked variant previously beat its separate RTL
variant and pushed both thumb sizes outside the track; measured after
the change, both checked thumbs stay within their RTL tracks. The same
interruptible overshoot is preserved.

Convertible pointer capability is observed for the avatar lift rather
than sampled once: folding into touch mode now removes its listeners and
resets a lifted row. The unused Checkbox snippet field is gone too, so
changed-file validation has no warning.
The production guard still returns 404 by default. The draft PR preview
accepts an explicit query so Switch, Toggle and Checkbox can be tested
before the temporary bypass is reverted for merge.
svelte-streamdown 3.0.1 leaves its animated-text context unmounted
when animateOnMount is omitted, even with animation enabled. The old
callers therefore produced no word spans, so the streaming transition
had nothing to animate.

Only the trailing text part of an in-progress message is live now. It
mounts with both dependency flags, while settled text and text before an
active tool stay static when history is reopened. Reasoning follows the
same contract. The component harness measured zero spans at rest, one
animated span on start, three after a chunk, and zero again once
settled; the computed animation is t-stream-in at 350ms with the source
cubic-bezier.

The provenance comment now describes the actual fidelity policy rather
than claiming selector-for-selector copying.
step-start is intentionally not rendered, but it still means the prose
before it has settled. Selecting the last renderable part therefore
marked old text live while the next step had already begun. Only a text
part at the actual array tail is animated now, with a regression guard
for both step-start and tool tails.
Clamping each reveal at the 900ms lookahead landed the tail of a large
batch on one timestamp, so those words appeared together: the burst the
scheduler exists to break up. Dividing the remaining lookahead by the
batch keeps the gaps equal and keeps the cap by construction.

Dropping the cap instead is what the 12ms gap floor forces, and that is
unbounded: 20 words every 100ms adds 140ms of lag per batch, measured at
28s of drift after 200 batches. So the floor is what gives.
The scheduler finds its spans through an inline style svelte-streamdown
writes, which is an implementation detail of a locked dependency and part
of no public API. A bump can drop it while every other test stays green:
the spans go unfound, pacing stops, and the stream reverts to revealing
whole Convex batches at once.

Reads the installed package so a future release fails the check, rather
than a fixture that would agree with itself through exactly that release.
Both suites replaced all of @convex-dev/agent with the two calls they
assert on, so the double silently claimed the module has nothing else.
Any new import from it fails these suites at load with a missing-export
error that points at the importing file rather than at the mock.

Adopts the importOriginal shape messageListing.test.ts already uses.
Line chunking buffers until a newline arrives, so a paragraph of prose
produces no delta at all while it is being generated: the reply stalls
on screen and then lands in one piece. Measured on a three-paragraph
answer at 45 tokens/s, 1244ms passed with no update; on a single
paragraph with no newline, nothing is written until the reply ends.

Both call sites now share one constant that names the agent default.
Omitting the option would not reach it: the agent forwards undefined to
smoothStream, whose own default is word chunking. Writes stay bounded by
the throttle rather than the chunk size, since addParts coalesces
everything inside a window, so the same answer costs 30 writes instead
of 2 and can never exceed ten per second of generation.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 31, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
saas-starter ec66fb2 Commit Preview URL

Branch Preview URL
Aug 31 2026, 12:18 PM

@stickerdaniel

Copy link
Copy Markdown
Owner Author

Blocked on a defect this change would expose, found in cross-model review and confirmed against the sources.

The consumer requests deltas from cursor zero on every subscription update (src/lib/chat/ui/ChatRoot.svelte:148), and the agent component returns at most MAX_DELTAS_PER_STREAM = 100 documents per stream (@convex-dev/agent/src/component/streams.ts:31,67-74). Nothing merges delta documents: compressUIMessageChunks only concatenates parts inside one delta, so document count equals addDelta count. During a stream the rendered text comes solely from those deltas (deriveUIMessagesFromDeltas via src/lib/chat/ui/streaming-display.ts:54), with no second path supplying it until the final message is saved.

At one delta per 100ms throttle window, a reply that generates for more than ten seconds exceeds 100 deltas; from there the query keeps returning the same first 100, the visible reply freezes, and the text appears in full only when generation ends. Line chunking rarely reached that count, so the limit is pre-existing but this change makes it the ordinary case for long replies.

Holding this until the cursor is advanced and pages accumulated, which is its own change and needs its own review.

Base automatically changed from feat/transitions-motion-upgrade to main August 31, 2026 12:49
@stickerdaniel

Copy link
Copy Markdown
Owner Author

Measured, not only read. Wrote 150 deltas straight through the component into one stream, then read them back on the path ChatRoot uses (listDeltas with cursor: 0):

written:  150
returned: 100
lastEnd:  796   # delta 99 of 150

The query returns exactly 100 regardless of how many exist, so the rendered text stops at the 100th delta and only completes when the final message is saved. Confirms the finding above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant