Skip to content

bench(moq-net): model the dash aggregator's stats load in the session bench - #4233

Merged
kixelated merged 2 commits into
mainfrom
bench/session-dash
Sep 26, 2026
Merged

kixelated merged 2 commits into
mainfrom
bench/session-dash

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

The moq.pro dash aggregator is one session following every node's .stats/<project>/node/<node> broadcast: ~34 nodes x >=5 projects x ~24 small tracks, one single-frame group per track per 1 s tick, max_age 0, all announced by peer relays. On 2026-09-25, moving that session off edge0.dal0 dropped the relay from ~129% to 40-78% of one core. Nothing benchmarked that shape.

Approach

session_dash_* in rs/moq-net/benches/session.rs (from #4160): relay 0 hosts the dash session, and every other relay in the full mesh is a node publishing its per-project stats broadcasts into its own origin, so relay 0 reaches each one over a peer session. Lite 06 (what the dash and peers negotiate today) and IETF 22. Throughput counts tracks, so flat elements/s means a constant cost per track.

  • projects: 1 / 4 / 16 projects on 4 nodes (96 to 1536 tracks).
  • tracks: 6 / 24 / 96 tracks per broadcast at 16 broadcasts.
  • relays: 1 to 32 peer nodes carrying the same 768 tracks, then the production point (34 x 5 x 24 = 4080).

Cluster::join now only connects, and Viewer::watch subscribes to a broadcast's tracks, so one viewer can follow many tracks per broadcast. The viewer's relay view includes hidden routes, like the dash's .stats/ scope.

Results

Measured on main with #4216, macOS M-series, one core for the whole mesh. The machine was shared (load 8-12), so treat these as +-30%.

Sweep lite-06 us/track IETF-22 us/track
projects 96 / 384 / 1536 / 6144 (6144 since trimmed) 17 / 24 / 27 / 34 15 / 19 / 21 / 20
tracks 96 / 384 / 1536 18 / 24 / 27 15 / 19 / 22
relays 1 -> 32 nodes, 768 tracks 27 -> 23 21 -> 19
production 34 x 5 x 24 24.6 (100 ms per tick) 20.6 (84 ms per tick)
  • Peer relay count costs nothing: flat to slightly faster from 1 to 32 nodes.
  • Per-track cost is roughly flat above a few hundred tracks. The lite rise to 34 us at 6144 didn't reproduce: a rerun under heavier load read 25 / 31 / 33 / 33 for lite and 37 / 34 / 35 for IETF above 96 tracks. I'm not calling it super-linear.

Top costs (xctrace Time Profiler, lite, 384 vs 6144 tracks; the two profiles have the same shape):

  • 55% in the lite publisher's Control::poll_serve, mostly poll_recv_next -> resume::Subscriber::poll_recv_group -> poll_sync/refresh_anchor (17%) and the LiveEdge max fold (5% self).
  • 28% in the lite SubscriberDriver (UniServe ingest).
  • About 15% in malloc/free/memmove, 10% in kio WaiterList::register/wake, and 10% in pthread_mutex lock/unlock.
  • kio::Tasks::poll swaps every bitset word on each owner poll. At 6144 in-flight groups that adds about 0.5 us per track, an O(n^2) term per tick. Skipping zero words with a relaxed load made no measurable difference, so I didn't file it.

Finding

The model doesn't account for the production cost. The whole mesh, including every node, spends about 100 ms of one core per tick at the production shape. Production shows 50-90% of a core on the dash's relay alone, 5-9x more. That points at the relay layer or the transport (a QUIC stream per group, 4080 per second), not moq-net's model. I added this to the Relay session bench quest as its first shape. No new quests: no super-linear finding survived a rerun.

Impact

  • No public API or wire change. Bench and quest text only.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

… bench

One session on relay 0 following every peer node's per-project stats
broadcast, ~24 single-frame tracks each, swept over projects, tracks per
broadcast, and peer relays, plus the 34 x 5 x 24 production point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-26T14:40:40.379354Z 51e466b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 55 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 3d9f8ded-81fe-40b2-b2a9-621a1fc0e05e

📥 Commits

Reviewing files that changed from the base of the PR and between ea7db01 and 51e466b.

📒 Files selected for processing (2)
  • quest/m1/bench-relay.md
  • rs/moq-net/benches/session.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

MERGE

Positive improvement. The dash aggregator's stats fan-in is a real production shape (the 2026-09-25 edge0.dal0 incident), and nothing in the session bench covered it. Modeling one session on relay 0 following every peer node's .stats/<project>/node/<node> broadcasts, with throughput in tracks, is the right unit: flat elements/s means constant cost per track, which is what you want when hunting for super-linearity.

Worth the complexity. The join → connect / Viewer::watch split is a small, reusable cleanup (multi-track per broadcast was awkward before), and Dash / DashRoom stay local to this shape. Sweep design is careful: projects and tracks scale independently; the relays sweep holds ~768 tracks via projects: 32 / (relays - 1) then adds the 34×5×24 production point. Using lite-06 (what production negotiates) alongside IETF-22, single-frame 128-byte groups, and with_hidden(true) on the dash viewer match the real session. The finding that the model is ~5–9× cheaper than production, and pointing that gap at the relay/transport quest instead of inventing a new one, is the correct conclusion.

No better approach jumps out. A separate bench file would duplicate Cluster for little gain; extending the existing mesh is the right reuse. One small note, not blocking: Cluster::join now always includes hidden routes, so every session_* viewer sees them. Harmless for room/* publishers, and consistent with how peer links already peer with with_hidden(true), but if you ever want a non-hidden viewer shape, a flag on join would be cleaner than forking.

Ship it.

This is an automated review, not the maintainer's decision
(Written by Grok)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main to resolve a conflict in rs/moq-net/benches/session.rs: kept main's paced rounds and allocation counter alongside the dash bench. cargo clippy -p moq-net --benches -- -D warnings passes and session_dash_tracks runs in --test mode. No inline review findings. Enabling auto-merge.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge (squash) September 26, 2026 14:37
@kixelated
kixelated merged commit b08ba83 into main Sep 26, 2026
5 checks passed
@kixelated
kixelated deleted the bench/session-dash branch September 26, 2026 15:06
This was referenced Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant