Skip to content

quest(audio-jitter-target): Audio jitter target - #3952

Closed
kixelated wants to merge 5 commits into
mainfrom
quest/main/audio-jitter-target/README
Closed

kixelated wants to merge 5 commits into
mainfrom
quest/main/audio-jitter-target/README

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Questline branch for /quest/main/audio-jitter-target/README.md. Child quests merge here; this stays a draft until the line's Quests list is empty.

(written by Opus 5.5)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review September 23, 2026 18:00
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-25T13:44:28.029657Z 157996f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 50 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 880299d7-a013-4976-aa4c-ac9c985aeabe

📥 Commits

Reviewing files that changed from the base of the PR and between 386aa12 and 157996f.

📒 Files selected for processing (36)
  • demo/web/src/index.ts
  • demo/web/src/viz.ts
  • doc/concept/audio-jitter.md
  • doc/concept/audio-jitter/corpus.test.ts
  • doc/lib/js/watch.md
  • js/hang/src/container/consumer.test.ts
  • js/hang/src/container/consumer.ts
  • js/hang/src/container/jitter.test.ts
  • js/hang/src/container/jitter.ts
  • js/hang/src/util/opus.test.ts
  • js/hang/src/util/opus.ts
  • js/moq-boy/src/game.ts
  • js/watch/src/audio/buffer.ts
  • js/watch/src/audio/config.test.ts
  • js/watch/src/audio/config.ts
  • js/watch/src/audio/decoder.ts
  • js/watch/src/audio/latency.test.ts
  • js/watch/src/audio/latency.ts
  • js/watch/src/audio/render-worklet.ts
  • js/watch/src/audio/render.ts
  • js/watch/src/audio/replay.test.ts
  • js/watch/src/audio/ring-buffer.test.ts
  • js/watch/src/audio/ring-buffer.ts
  • js/watch/src/audio/shared-ring-buffer.test.ts
  • js/watch/src/audio/shared-ring-buffer.ts
  • js/watch/src/audio/source.ts
  • js/watch/src/element.ts
  • js/watch/src/player.ts
  • js/watch/src/sync.test.ts
  • js/watch/src/sync.ts
  • js/watch/src/ui/components/buffering-indicator.ts
  • js/watch/src/ui/components/latency.ts
  • js/watch/src/ui/stats.ts
  • js/watch/src/video/decoder.ts
  • quest/m0/audio-jitter-target/README.md
  • quest/m0/audio-jitter-target/watch.md

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: f8e84a2f-1c81-440f-85d1-27797b4307a8

📥 Commits

Reviewing files that changed from the base of the PR and between a227ea3 and 386aa12.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (14)
  • doc/bin/cli.md
  • doc/concept/audio-jitter.md
  • quest/m0/audio-jitter-target/README.md
  • quest/m0/audio-jitter-target/native.md
  • quest/m1/audio-quality-harness/native.md
  • rs/moq-audio/Cargo.toml
  • rs/moq-audio/src/decode/consumer.rs
  • rs/moq-audio/src/jitter.rs
  • rs/moq-audio/src/lib.rs
  • rs/moq-cli/src/args.rs
  • rs/moq-cli/src/play/args.rs
  • rs/moq-cli/src/play/media.rs
  • rs/moq-cli/src/play/timeline.rs
  • rs/moq-cli/src/play/window.rs
💤 Files with no reviewable changes (2)
  • quest/m1/audio-quality-harness/native.md
  • quest/m0/audio-jitter-target/native.md

Included review availability: Your plan provides up to 4 included reviews per hour; 1 remains after this review.


Walkthrough

The change adds automatic audio playout-delay estimation and keeps fixed-duration delay options. Playback uses the estimated or configured delay to set audio buffering and video timing. Audio writes now fit the sink’s buffered level by adding silence or skipping incoming samples. Documentation and quest status describe the estimator and its remaining native replay comparison.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 386aa

The reported rewind and audio-buffering failures do not occur on the checked playback paths. No actionable merge-blocking risk remains after normal checks.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to 386aa

A publisher whose audio a user plays may be able to drive excessive work in the new delay estimator. The potential impact appears confined to the receiving playback client; broader exposure has not been established.

Retained concerns

  • Medium · security · inferred: A publisher able to supply many packets with strictly increasing timestamps inside the estimator's media-time window could grow its arrival history and make each subsequent observation scan more entries, exhausting playback-client CPU or memory. Observation occurs before audio decoding can reject the packet.
Security review details

Security Blast Radius

  • inferred — The identified resource-exhaustion path affects a client playing the supplied audio. Evidence does not establish that it changes relay authority, other tenants' state, or a shared service; the publisher privileges needed to deliver such a stream are unresolved.

Security Findings and Attack Paths

  • inferred — A stream of accepted container packets with dense, increasing media timestamps can repeatedly extend and scan estimator history before audio payload validation. This is a newly introduced client resource-exhaustion path, conditional on upstream limits not shown in the available evidence.

Trust Boundaries and Controls

  • observed — Publisher timing and the catalog's advertised jitter enter client delay selection. Playback caps the effective target at its age budget, and sink overflow drops excess audio rather than growing its queue; neither control bounds the estimator's per-packet history scan.

Resilience and Maintainability Implications

  • observed — Playback continues past a decode error, resets the sink at a timeline discontinuity, and bounds the final drain wait. Thus malformed packets can continue reaching the pre-decode estimator observation point while playback remains active.

Hardening Proposals

  • proposed — Bound estimator history and per-observation work independently of media time, and verify the bound with densely timestamped packets, including packets whose audio payload is rejected.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 78.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 51 functions across 8 files. (4 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the audio jitter target questline, which matches the pull request objectives and the implementation changes.
Description check ✅ Passed The description explains that this is a questline branch for the audio-jitter-target work and that child quests merge into it. This matches the pull request objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 78.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 51 functions across 8 files. (4 skipped: 4 unsupported.)

✨ Finishing Touches 💡 1
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch quest/main/audio-jitter-target/README
✨ Simplify code
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

ITERATE

Positive improvement: not yet. Against main this PR has an empty diff (files: [], 0+/0−): only the questline open + merge commits, with no remaining file delta. There is nothing product-facing to approve into main.

Worth the complexity: N/A for an empty change set. As a questline integration branch (child work like #3954 targets quest/main/audio-jitter-target/README), keeping the PR open as a draft merge target is fine; marking it ready/mergeable while empty is not.

Different approach: leave this draft until the line’s Quests list is empty and children have landed, or close it if main already absorbed the quest scaffolding. Do not merge a no-op; review #3954 (and siblings) against this base instead.

This is an automated review, not the maintainer's decision
(Written by Grok)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 386aa120ab

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +204 to +206
let level = buffered + pad + incoming;
let skip = if level > target + slack {
(level - target).min(incoming)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve real frames while priming a dry burst

When a new or underrun sink receives several codec frames in one publisher flush, the first call pads the ring to target, then later calls treat the remaining frames as excess and discard them here. For example, with target = 4800, slack = 960, and five immediately delivered 960-sample frames, only the first two survive, whereas the same samples supplied as one incoming buffer would all survive. Playback timestamps still advance across the discarded 60 ms, producing an audible jump and an incorrect A/V anchor precisely for the bursty publishers that auto delay is intended to support; priming needs to accumulate the dry burst or replace padding instead of dropping real samples.

Useful? React with 👍 / 👎.

Comment on lines +268 to +271
let arrival = jitter::millis(now().duration_since(self.origin));
let timestamp = mux_frame.timestamp;
let media = timestamp.value() as f64 * 1000.0 / timestamp.scale().as_u64() as f64;
self.target.observe(arrival, media);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reset media-order state at declared discontinuities

When a declared container discontinuity rewinds timestamps, apply_discontinuity resets the decoder timeline but leaves the estimator's newest timestamp intact. Every packet in the new epoch is consequently rejected by observe as reordered until its media timestamp surpasses the old epoch's high-water mark, so a long-running stream that reconnects at timestamp zero can keep a stale playout target for hours and suffer persistent underruns or excess latency. Reset the estimator's media-order/history epoch when the container discontinuity changes.

Useful? React with 👍 / 👎.

…3954)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision needed before this lands: #3954 changed published @moq/watch Sync semantics. A numeric delay is now literal (no rendition delay added), and register() takes a track's full target. Existing callers compile unchanged but get a shallower clock. Per CONTRIBUTING, that argues for dev rather than main.

The Quests list still has Watch, the real-browser proof, so the line isn't finished yet.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32b0c47c2b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Observed here, before any gap handling or age budget: a target derived from
// what survives the budget would only ever confirm the budget it was cut to.
if (!marker) {
this.#jitter.observe(arrival, Moq.Time.Milli.fromMicro(sample.timestamp));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reset browser jitter state at playhead discontinuities

When a marker or latency skip starts an epoch whose timestamps rewind, every subsequent frame is still passed to this Jitter, but its previous epoch's #newest remains intact and rejects those frames as reordered until they exceed the old high-water mark. The browser's auto target can therefore remain stale for hours, either preserving excess latency or repeatedly underrunning instead of adapting. Fresh evidence beyond the native finding is this separate browser observation path and the absence of any jitter reset when #markPlayhead() increments the discontinuity. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

});
effect.proxy(this.#out.timestamp, active.timestamp);
effect.proxy(this.#out.buffered, active.buffered);
effect.proxy(this.#spread, active.spread);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the pending rendition's measured spread

During a video rendition switch, only the active track's spread is proxied here, while out.jitter already includes both active and pending advertised values. If the pending rendition measures a larger arrival spread than either advertised floor, Sync keeps the smaller delay while that track catches up and promotes it; promotion then suddenly raises the global delay and causes an avoidable video pause or audio rebuffer. Include the pending track's measured spread in the switch target before promotion. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

…hrough the estimator (#4117)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 157996f719

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// Ran dry mid-playback. Re-stall so `insert` refills to the target before playback
// resumes: the alternative is playing the next chunk on an empty cushion, which
// underruns again on the very next quantum. See STALLED for the write discipline.
Atomics.store(this.#control, STALLED, 1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid re-stalling after a concurrent final refill

In the SharedArrayBuffer path, the reader can observe an empty ring here, then race with insert(), which publishes enough samples and clears STALLED, before this unconditional store sets it back to 1. The comment assumes a later insert will recover, but if this was the final packet, the fully refilled tail remains parked forever and playback never drains it. Recheck the current write/read state when raising the flag, or use an atomic transition that cannot overwrite a writer's successful refill. (Written by GPT-5.6 Sol)

Useful? React with 👍 / 👎.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Superseded by #4162, which carries the same commits under the renamed branch quest/m0/audio-jitter-target/README.

(Written by Opus 5.5)

@kixelated kixelated closed this Sep 25, 2026
@kixelated
kixelated deleted the quest/main/audio-jitter-target/README branch September 25, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant