Skip to content

fix(watch): subscribe auto audio at the estimator's ceiling, replay through the estimator - #4117

Merged
kixelated merged 2 commits into
quest/main/audio-jitter-target/READMEfrom
quest/m0/audio-jitter-target/watch
Sep 25, 2026
Merged

kixelated merged 2 commits into
quest/main/audio-jitter-target/READMEfrom
quest/m0/audio-jitter-target/watch

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Works /quest/m0/audio-jitter-target/watch.md. Draft: the quest's manual Chrome/Safari run and the recorded-trace replay remain, and neither can be done from here.

Problem

In "auto" the audio subscription's max age followed the playout target. The transport drops groups older than that budget before Container.Consumer sees them, both at the relay and in the receiver's own Track.Subscriber. So when a flush started after the target had settled, the estimator never saw the late frames. The target crept up one 20 ms bucket at a time, and the frames it should have measured went unplayed.

Replaying a minute of paced audio followed by 100 ms flushes through the real subscription and consumer:

Subscription budget Target over the first 2.4 s of flushes Frames delivered
follows the target (before) 40 -> 80, reaches 140 after 4.6 s 3235 / 3300
at least 2 s (after) 40 -> 140 after 1.4 s 3300 / 3300

replay.test.ts also sized the rings from a p95 approximation instead of from the estimator.

Approach

  • audio/decoder.ts: in "auto" the subscription asks for max(sync.out.maxAge, AUTO_MAX_AGE), where AUTO_MAX_AGE is the estimator's 2 s ceiling. This matches moq play --delay auto. Container.Consumer keeps sync.out.maxAge as its local skip budget, since it observes each frame before applying it, so playback latency is unchanged.
  • replay.test.ts: measure() feeds a trace through Track.Producer -> subscribe -> Container.Consumer on a stubbed performance.now. Both rings are now sized from the target that settles, and every ring case now asserts zero underruns and zero skips (previously up to 2 underruns were allowed). Two new cases pin the budget behavior.
  • doc/concept/audio-jitter.md states the rule for both implementations.
  • The quest's Plan now lists only what remains.

Findings that are not code changes:

Impact

  • No public API or wire change. AUTO_MAX_AGE lives in the internal audio/latency.ts module, which is not re-exported.
  • Behavior: in "auto", the audio subscription's max age is now at least 2 s (audio only; video is unchanged). Under congestion, up to 2 s of stale audio groups are delivered instead of being reset. That is a few KB at Opus rates, and the ring drops anything already behind the playhead.
  • Sync is untouched. This change only reads sync.in.delay and sync.out.maxAge, so it does not affect the open question on quest(audio-jitter-target): Audio jitter target #3952 about whether feat(watch): size the auto delay from the measured arrival estimator #3954's Sync semantics belong on dev.

Alternatives

  • Raise Sync.out.maxAge itself in auto. Rejected: Sync.received() also uses it as the lookahead cap, so this would let playback drift up to 2 s behind the live edge.
  • Apply the ceiling to video too. Not done: video groups are GoPs, so holding 2 s of them costs real bandwidth under congestion. When audio is present its target already governs Sync.

Follow-ups

  • Manual run on Chrome and Safari against the public relay (real microphone, 40 ms or more of added RTT, both ring paths), plus the publisher encoder-lag measurement. Both stay in the quest.
  • The watch: "Real-time" audio cuts in and out: RTT-sized delay and a ring with no hysteresis or re-buffer (measured locally and against cdn.moq.pro) #3477 traces need re-attaching by the reporter, or re-recording with /quest/m1/audio-quality-harness/browser.md.
  • Branch naming: the line PR's branch is quest/main/audio-jitter-target/README, but quest branch now names it quest/m0/.... This PR uses quest/m0/audio-jitter-target/watch and targets the existing line branch.
  • The line branch is 95 commits behind main. Merging main in conflicts in js/watch/src/sync.ts (and rs/moq-cli), so I left that for whoever settles the Sync question.

(written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits September 25, 2026 00:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hrough the estimator

In "auto" the audio subscription's max age followed the playout target, so the
transport dropped every group later than the target before the container
consumer could observe it, and the estimate crept up one bucket at a time while
those frames went unplayed. Subscribe with at least the 2 s ceiling, as moq play
does, and keep the target as the consumer's local skip budget.

replay.test.ts now sizes the rings from the estimator itself, measured through a
transport subscription and container consumer on a stubbed clock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review September 25, 2026 13:36
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-25T13:39:17.174018Z 36443e7 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated
kixelated merged commit 157996f into quest/main/audio-jitter-target/README Sep 25, 2026
3 checks passed
@kixelated
kixelated deleted the quest/m0/audio-jitter-target/watch branch September 25, 2026 13:40

Copy link
Copy Markdown
Collaborator Author

No code changes. Marked the draft ready. Codex completed on 36443e7 with no findings, and Check and Test were already green. Squash-merged onto quest/main/audio-jitter-target/README as 157996f (not main).

(Written by Grok 4.7)

@kixelated
kixelated restored the quest/m0/audio-jitter-target/watch branch September 26, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant