Skip to content

chore(moq-net): bench lite-06, smoke-run benches nightly, refresh perf quests - #4229

Merged
kixelated merged 1 commit into
mainfrom
claude/bench-session-followup
Sep 26, 2026
Merged

kixelated merged 1 commit into
mainfrom
claude/bench-session-followup

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

Follow-up to #4160. Its session bench compiled in CI but never ran, so a stale version string (moq-lite-07) broke it silently: nextest never executes harness = false targets. It also only benched lite-07-wip, but production negotiates lite-06. And #4216 resolved one of its perf quests without deleting it.

Approach

  • session.rs benches moq-lite-06, moq-lite-07-wip, and moq-transport-22.
  • Nightly runs every moq-net Criterion routine once (cargo bench -p moq-net --features fuzz --bench '*' -- --test): 7 targets, 192 routines, about 2 minutes on an M4 including the build. fuzz enables the announce bench, which the glob otherwise refuses. This only catches failures. Timing regressions stay with the Bench CI quest.
  • Quests, re-measured on current main (2026-09-25, Apple M4):
Quest Result
Lite route rescan Deleted, done by #4216. session_delivery_broadcasts is flat on all three versions (~365 µs at 16/256/4096), and session_delivery_scale 256x256 is at parity (lite-06 12.5 ms, IETF 13.2 ms).
Announce replay Still reproduces. Join at 1/64/1024 routes: lite-06 128 µs / 317 µs / 5.0 ms, IETF 95 µs / 311 µs / 4.05 ms. initial.retain is still in AnnounceRun::init.
Group cost Still reproduces: ~26 µs per viewer-group (~420 µs at 16 viewers). The 256-viewer point is too noisy on this machine to quote.
Cache expiry growth Premise corrected. With the default pool, RSS climbs past the 30 s window on both wires: IETF 1.9 GB at 30 s to 4.4 GB at 120 s, lite-06 2.3 GB to 3.5 GB. It looked IETF-only before because lite ran 30x slower per round. Retitled.

Impact

  • No public API or wire change. Bench target, nightly workflow, and quest docs only.

Notes

just check hits one unrelated failure on macOS: moq-relay::auth_lifetime an_outage_keeps_the_session_until_expires times out on connect in 8 of 9 runs on main too.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

…f quests

The session bench now covers moq-lite-06, the draft production negotiates,
beside moq-lite-07-wip and moq-transport-22. Nightly runs every moq-net
Criterion routine once, since nextest never executes harness = false targets
and a bench that panics otherwise goes unnoticed.

Lite route rescan is done by #4216 (delivery flat at ~365 us from 16 to 4096
announced routes on every version), so its quest is deleted. The remaining
findings are re-measured on current main: announce replay and group cost still
reproduce; cache growth under the default pool now shows on lite too, so that
quest drops its IETF-only premise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: eb522ce2-8476-4591-94d6-11d2cedf90b8

📥 Commits

Reviewing files that changed from the base of the PR and between b99cad9 and 88b2885.

📒 Files selected for processing (8)
  • .github/workflows/nightly.yml
  • quest/m1/README.md
  • quest/m1/cache-expiry-growth.md
  • quest/m1/perf/README.md
  • quest/m1/perf/announce-replay.md
  • quest/m1/perf/group-cost.md
  • quest/m1/perf/lite-route-rescan.md
  • rs/moq-net/benches/session.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-26T02:29:27.485863Z 88b2885 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Code Review — MERGE

(a) Positive improvement? Yes. The session bench was compiled but never exercised in CI (harness = false, nextest skips it), and it only covered moq-lite-07-wip + moq-transport-22 while production negotiates lite-06. Adding lite-06 to VERSIONS, smoke-running every moq-net Criterion target nightly, and refreshing/closing the perf quests against current numbers closes a real blind spot and keeps the quest board honest.

(b) Worth the complexity? Yes — small, focused, low surface area.

(c) Different approach better? Not really. Nightly is the right place for a ~2 min Criterion smoke; putting it on every PR would be the worse trade. Archiving lite-route-rescan as a done quest instead of deleting is an alternative, but #4216 already landed the fix and dropping the open quest matches how the board is maintained.

Concrete notes from the diff

nightly.yml — moq-net benchmark smoke

  • Command: nix develop --command cargo bench --locked -p moq-net --features fuzz --bench '*' -- --test
  • --test = Criterion runs each routine once (no timing compare) — correct for panic / unparsable-version / deadlock detection.
  • --features fuzz is required so the announce bench (which has required-features = ["fuzz"]) is not refused by the '*' glob.
  • Placement after the JS origin broadcast-map bench, before Audit, with if: ${{ !cancelled() }} is fine.
  • Comment correctly documents why nextest never covers these harness = false targets.
  • Risk: a hanging bench holds the nightly runner (that is partly the point of the smoke). Timing flake is not a concern under --test. Claimed ~2 min is acceptable for nightly.

rs/moq-net/benches/session.rs

  • VERSIONS becomes ["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"] — production + next opt-in + IETF. Correct and overdue; this is the highest-value line in the PR.

Quest docs

  • Deleting quest/m1/perf/lite-route-rescan.md is justified: fix(moq-net): serve only the lite routes a request woke, and bound kio waiter lists #4216 (serve only the lite routes a request woke…) is merged and matches the quest goal. README link removed cleanly.
  • announce-replay.md / group-cost.md refreshed with 2026-09-25 Apple M4 numbers (lite-06 + lite-07-wip + IETF); machine called out.
  • cache-expiry-growth.md correctly reframes the bug: growth on both wires under the default unbounded pool; earlier “IETF-only” reading was an apples-to-oranges wall-time compare when lite was ~30× slower. Goal text updated to “every version.”

Minor watch-items (not blockers)

  • Nightly smoke will fail the job on the first bad bench — good — but if a routine deadlocks rather than panics, expect a stuck runner until timeout.
  • Quest numbers are M4-local; fine for quest docs, just don’t treat them as cross-machine baselines.

Recommendation: MERGE.

This is an automated review, not the maintainer's decision
(Written by Grok)

@kixelated
kixelated merged commit 7461eca into main Sep 26, 2026
5 checks passed
@kixelated
kixelated deleted the claude/bench-session-followup branch September 26, 2026 03:29
@moq-bot moq-bot Bot mentioned this pull request Sep 26, 2026
@moq-bot moq-bot Bot mentioned this pull request Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant