test(audio-quality): grade browser audio playout against a budget, nightly

.github/workflows/nightly.yml

@@ -18,6 +18,13 @@ name: Nightly
 # the wheel builds in a manylinux container, the Kotlin/Swift/Go/Dart packages
 # link the FFI on every platform) and nothing notices until the tag is pushed.
 #
+# `audio-quality` is here because of what it measures. It plays sixteen rows of
+# audio in real time, a minute each, which no merge gate should carry, and every
+# number it grades is a timing one that moves with whatever else the runner is
+# doing. A per-PR arm would fail on unrelated load often enough to be ignored;
+# nightly it competes with nothing, and a regression surfaces a day late against
+# a budget that still means something.
+#
 # A failure here lands on main rather than being caught in review, which is the
 # accepted trade. Anything that must block a merge belongs in check.yml.
 
@@ -176,6 +183,54 @@ jobs:
           NEXTEST_PROFILE: ci
           MOQ_STRICT: 1
 
+  # Plays the matrix in headless Chromium through moq-shaper and grades what the
+  # listener heard against test/audio-quality/budgets.json. ARM64 like wasm.yml,
+  # the only architecture cache.yml warms a Rust store for.
+  audio-quality:
+    name: Audio quality
+    runs-on: ubuntu-24.04-arm
+    # Sixteen rows at a minute each, plus the relay, CLI, and shaper built from source.
+    timeout-minutes: 60
+    steps:
+      - name: Free disk space
+        uses: jlumbroso/free-disk-space@ceedf095f4ec1a097402bc6bd80831f2e1a6fde6 # main
+        with:
+          tool-cache: false
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - uses: DeterminateSystems/nix-installer-action@1d87d45818068401a10cf16bdc5f00b24994a83f # main
+        with:
+          determinate: false
+          extra-conf: |
+            extra-substituters = https://kixelated.cachix.org
+            extra-trusted-public-keys = kixelated.cachix.org-1:CmFcV0lyM6KuVM2m9mih0q4SrAa0XyCsiM7GHrz3KKk=
+      - name: Rust cache
+        uses: ./.github/actions/rust-cache
+      # Playwright's Chromium isn't in the nix devShell; `--with-deps` installs
+      # its apt runtime libs, and run.sh's own install then no-ops.
+      - name: Playwright Chromium
+        run: nix develop --command bash -c "bun install --frozen-lockfile && bunx playwright install --with-deps chromium"
+        shell: bash -leo pipefail {0}
+      - name: Audio quality
+        run: nix develop --command just test audio-quality --enforce
+        shell: bash -leo pipefail {0}
+        env:
+          # Somewhere upload-artifact can reach; the harness default is under $TMPDIR.
+          MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
+      # The run directory is the diagnosis: each process's log, the shaper's
+      # counters, the raw samples, and every row's summary. The harness keeps it
+      # on a failure without being asked.
+      - name: Keep the failing run
+        if: ${{ failure() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: audio-quality-run
+          path: ${{ runner.temp }}/moq-test-runs/
+          retention-days: 7
+          if-no-files-found: ignore
+
   # Every tag-driven release workflow, run against main with publishing skipped:
   # the workflows gate their publish jobs on `github.ref_type == 'tag'`, and no
   # secrets are passed, so a dry-run cannot publish even by accident.

js/justfile

@@ -9,7 +9,7 @@ set working-directory := '.'
 # Paths a JS check depends on: every bun workspace (they are not all under js/,
 # see the root package.json) plus the config shared by all of them. `check`,
 # `fix`, and `ci` all skip themselves when a diff misses this.
-scope := '^(js/|doc/|drafts/|demo/(boy|web)/|test/interop/clients/js|test/wasm/|package\.json$|bun\.lock(b)?$|biome\.jsonc$)'
+scope := '^(js/|doc/|drafts/|demo/(boy|web)/|test/(audio-quality|interop)/clients/js|test/wasm/|package\.json$|bun\.lock(b)?$|biome\.jsonc$)'
 
 # Run all checks by default.
 default:

package.json

@@ -46,6 +46,7 @@
 		"demo/boy",
 		"demo/web",
 		"js/moq-boy",
+		"test/audio-quality/clients/js",
 		"test/interop/clients/js",
 		"test/interop/clients/js-native",
 		"test/wasm"

quest/m0/audio-jitter-target/watch.md

@@ -75,8 +75,8 @@ against it is likely cheaper than patching the branch's:
   that down: a stored preference silently winning over the default is its own
   bug, and it also means auto gets far less real exposure than it looks like.
 - Replay recorded traces rather than synthetic ones of the same shape. The
-  #3477 traces are gone, so record fresh ones with the [browser
-  harness](/quest/m0/audio-quality-harness/browser.md) (decided with the
+  #3477 traces are gone, so record fresh ones with the [audio quality
+  harness](/quest/m0/audio-quality-harness/traces.md) (decided with the
   maintainer instead of asking the reporter). Trim a copy into the repository
   and replay it through both rings in `replay.test.ts`.
 - Manual run against the public relay on Chrome and Safari, the two rows the
@@ -97,7 +97,7 @@ that quest can adopt it without a second estimator change.
 
 ## Required
 
-- [Browser harness](/quest/m0/audio-quality-harness/browser.md) - records the arrival traces this quest replays
+- [Traces](/quest/m0/audio-quality-harness/traces.md) - records the arrival traces this quest replays
 
 ## Related
 

quest/m0/audio-quality-harness/README.md

@@ -23,14 +23,13 @@ are. The native lane moved to m1 as a standalone quest (decided in the
 same budgets and metric schema so the two implementations can be compared
 rather than merely both passing.
 
-The starting point is not a blank page. The reporter on #3477 already built a
-working browser lane on their fork (`fperex/moq`, branch
-`debug-findings-solution`, under `test/audio-quality/`): a Playwright-driven
-matrix over `moq-shaper`, a budget file graded under `--enforce`, a nightly
-job, and a replay runtime. Upstream that rather than reinventing it. The raw traces it shipped with are gone, so
-this harness records fresh ones, and the [jitter target's watch
-quest](/quest/m0/audio-jitter-target/watch.md) replays them (decided with the
-maintainer during the merged-PR audit).
+The browser lane has landed as `test/audio-quality/`, upstreamed from the
+reporter's fork on #3477 (`fperex/moq`, branch `debug-findings-solution`) on
+its own, without the fork's player, estimator, or shaper changes. That leaves
+the two things that needed them: real arrival traces, which the [jitter
+target's watch quest](/quest/m0/audio-jitter-target/watch.md) replays, and the
+bursty and step profiles, which need shaper features the fork has and `main`
+does not.
 
 Jitter comes from the seeded userspace UDP shaper the transport drills run
 under (`rs/moq-shaper`, documented in `test/drill/README.md`), not from a fake
@@ -49,7 +48,8 @@ each of those dimensions moves the expected floor.
 
 ## Required
 
-- [Browser](/quest/m0/audio-quality-harness/browser.md) - upstream the fork's harness, grade it against a budget, run it nightly
+- [Traces](/quest/m0/audio-quality-harness/traces.md) - real arrival traces recorded, checked in, and graded as replay rows
+- [Shaper profiles](/quest/m0/audio-quality-harness/shaper-profiles.md) - the bursty and mid-run step profiles, once moq-shaper can express them
 
 ## Related