From 4d03a79a0781c3a0733d8c3e1f04abad803f51fb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 2 Sep 2026 14:20:49 -0700 Subject: [PATCH 1/2] add moxygen interoperability smoke lane --- .github/workflows/smoke.yml | 23 ++++++ README.md | 8 +- freshness.sh | 18 +++++ justfile | 5 ++ moxygen.sh | 146 ++++++++++++++++++++++++++++++++++++ 5 files changed, 199 insertions(+), 1 deletion(-) create mode 100755 moxygen.sh diff --git a/.github/workflows/smoke.yml b/.github/workflows/smoke.yml index f2191f6..476f5b3 100644 --- a/.github/workflows/smoke.yml +++ b/.github/workflows/smoke.yml @@ -251,3 +251,26 @@ jobs: - name: Cloudflare interoperability run: ./cloudflare.sh + + moxygen: + name: Moxygen interoperability + runs-on: ubuntu-latest + # Current moq-dev HEAD negotiates successfully but fails moxygen's + # announce-subscribe case. Keep the full diagnostic running without making + # every unrelated PR red; remove this once that protocol mismatch is fixed. + continue-on-error: true + permissions: + contents: read + packages: read + # The published client image and moq-dev relay both follow Git HEAD. Building + # the Rust relay is the long pole; the 12 protocol cases are bounded and quick. + timeout-minutes: 45 + + steps: + - name: Checkout + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 + with: + persist-credentials: false + + - name: Moxygen interoperability + run: ./moxygen.sh diff --git a/README.md b/README.md index 449dc37..49d0d37 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # moq smoke -Cross-language interop smoke test for the **public** [Media over QUIC](https://github.com/moq-dev/moq) packages, plus source-head interoperability coverage between [`moq-dev/moq`](https://github.com/moq-dev/moq) and [Cloudflare's `moq-rs`](https://github.com/cloudflare/moq-rs). +Cross-language interop smoke test for the **public** [Media over QUIC](https://github.com/moq-dev/moq) packages, plus source-head interoperability coverage with [Cloudflare's `moq-rs`](https://github.com/cloudflare/moq-rs) and Meta's [`moxygen`](https://github.com/facebookexperimental/moxygen). The [moq-dev/moq](https://github.com/moq-dev/moq) monorepo has its own in-tree smoke test, but it builds every client from workspace source. That proves the code in the tree works; it does **not** prove a real user can install the published artifacts and have them talk to each other. A missing wheel, a stale Homebrew formula, a broken `.deb`, an export that didn't survive packaging, a Go module missing its header. none of that shows up until someone installs from a registry. @@ -28,6 +28,8 @@ We check that bytes move across implementations, not that H.264 decodes. Cloudflare's fork is tested separately because it uses a different media catalog and track layout, so putting its fMP4 tools into the Hang media matrix would create application-format failures rather than transport interop coverage. [`cloudflare.sh`](cloudflare.sh) builds both projects' relays from their latest default branches (honoring each checkout's committed Rust dependency lock), then drives them with a purpose-built Cloudflare client over WebTransport and raw QUIC. Against both relays it validates 32 complete 64 KiB subgroup objects (2 MiB total) byte-for-byte; against Cloudflare's relay it also validates 32 complete QUIC datagrams per transport. This is deliberately stronger than a setup-only or single-object protocol probe. +Meta's moxygen is also tested separately from the Hang media matrix because its media samples use MoQ Media Interop packaging. [`moxygen.sh`](moxygen.sh) pulls moxygen's source-head `moxygen-interop-client` image and runs all six of its self-contained relay cases through the latest `moq-dev/moq` relay over WebTransport and raw QUIC: setup, namespace publication and withdrawal, expected subscription failure, announced publish/subscribe routing, and subscribe-before-announce behavior. The lane deliberately negotiates draft-16, which moxygen currently recommends for new integrations while its draft-18 support remains experimental. The published image is Linux/amd64 and the test uses Docker host networking, so this lane runs on Linux CI rather than in the package-channel matrix. + The **Native JS** client runs the JS packages *outside* a browser, where there's no native WebTransport, using moq's own `@moq/web-transport` polyfill (a prebuilt NAPI QUIC/HTTP3 addon). It runs as two cells, `js-native-node` and `js-native-bun`, to catch runtime-specific breakage. Subscribe only here too: publishing media needs a WebCodecs encoder, which a native JS runtime lacks (reading raw container frames doesn't). Swift, Kotlin, C, and GStreamer **subscribe only**. The FFI wrappers (Swift/Kotlin/C) publish through the streaming importer (`publish_media_stream`), which isn't in the published 0.2.x FFI yet, so they can only subscribe until it ships; the GStreamer cell drives `moqsrc` (publishing via `moqsink` needs an encoder + request-pad muxing — a follow-up). Rust and the browser publish today. @@ -57,6 +59,8 @@ cargo install moq-relay moq-cli # installs moq-relay + moq (or brew / apt) just full # full matrix, --timeout 30 # Cloudflare client/relay self-test + Cloudflare client against moq-dev relay: just cloudflare +# Moxygen's protocol interop client against the moq-dev relay (Linux Docker): +just moxygen # ...or use the moq flake as the channel (builds moq, no install needed): just nix-channel --publishers rust,js-vite --subscribers rust,python,js-jsdelivr --timeout 30 ``` @@ -89,6 +93,7 @@ RELAY_BIN=/path/to/moq-relay MOQ_BIN=/path/to/moq ./smoke.sh ``` smoke.sh orchestrator: relay + media interop matrix cloudflare.sh orchestrator: Cloudflare client through both projects' relays +moxygen.sh orchestrator: moxygen protocol client through the moq-dev relay smoke.toml relay config (anonymous, self-signed localhost) token.sh orchestrator: moq-token generate/verify interop matrix clients/ @@ -183,5 +188,6 @@ This test tracks the **latest published** packages, so it sometimes runs ahead o - **Token interop** (`token.sh`): working on **cargo / apt / nix** plus the **`moqdev/moq-token-cli` Docker image** (Linux). The published `moq-token` binary (from crates.io / apt / nix / Docker Hub) and `@moq/token` (npm, under both node and bun) cross-verify every token across `HS256`, `EdDSA`, `ES256`, and `RS256`, and each verifier rejects tampered tokens and the wrong key. The Docker cell (`rust-docker`) proves the image — built `FROM nixos/nix`, so it carries the libiconv the brew bottle used to leak — runs cleanly. Subscriber-only languages don't ship token tooling yet, so the matrix is rust (binary + Docker) + the two JS runtimes for now. - **Token interop on the Homebrew bottle** (`rust` cells, macOS `brew`): working. The `moq-dev/tap/moq-token-cli` package's `moq-token` binary used to abort on launch — it baked in a `/nix/store/…-libiconv/lib/libiconv.2.dylib` rpath from the build sandbox that doesn't exist on a user's Mac (`dyld: Library not loaded`). The 0.5.31 bottle fixes it: its only `LC_RPATH` is now `/usr/lib`, so `@rpath/libiconv.2.dylib` resolves to the system libiconv and the binary runs (verified locally — `generate --algorithm HS256` succeeds, no leaked `/nix/store` rpath). `token.sh` still probes the binary once at startup, so a relapse would be caught again. Exactly the break-then-fix this repo exists to surface. - **Cloudflare interoperability**: the Cloudflare client publishes and subscribes over WebTransport and raw QUIC through both `cloudflare/moq-rs`'s `moq-relay-ietf` and `moq-dev/moq`'s `moq-relay`, with sustained subgroup payloads checked byte-for-byte. Cloudflare's relay additionally exercises datagrams in both directions. This is a source-head smoke test, so a later upstream commit can intentionally turn it red. +- **Moxygen interoperability**: currently **red**. Moxygen's published source-head interop client negotiates draft-16 and passes 5/6 relay scenarios through `moq-dev/moq`, but `announce-subscribe` closes the subscriber session instead of routing it to the announced publisher. The failure reproduces over WebTransport and raw QUIC with the published relay, and over WebTransport with current moq-dev HEAD. CI runs the full Linux/amd64 Docker lane as non-blocking diagnostic coverage until the mismatch is fixed; `just moxygen` still exits nonzero locally. A broken published package fails only its own matrix cells (see `mark_broken` in `smoke.sh` / `token.sh`); it never aborts the rest of the run. diff --git a/freshness.sh b/freshness.sh index a177fca..a32361d 100755 --- a/freshness.sh +++ b/freshness.sh @@ -107,6 +107,24 @@ else note FAIL "moq-dev/moq relay is no longer resolved from the unpinned Git default branch" fail=1 fi +# Moxygen publishes its source-head interop client as an amd64 image. Keep the +# moving tag and pull it on every run; the wire draft is pinned separately for +# deliberate protocol compatibility. +# shellcheck disable=SC2016 # the grep checks for literal variable references in moxygen.sh +if grep -qF 'ghcr.io/facebookexperimental/moxygen-interop-client:latest-amd64}' moxygen.sh && + grep -qF '"$DOCKER" pull --platform linux/amd64 "$MOXYGEN_IMAGE"' moxygen.sh; then + note ok "facebookexperimental/moxygen interop client -> latest-amd64 (pulled each run)" +else + note FAIL "moxygen.sh no longer pulls the latest source-head interop client image" + fail=1 +fi +# shellcheck disable=SC2016 # the grep checks for the literal variable reference in moxygen.sh +if grep -q 'cargo install --quiet --locked --git "$MOQ_REPO"' moxygen.sh; then + note ok "moxygen lane moq-dev/moq relay -> unpinned Git default branch" +else + note FAIL "moxygen lane no longer resolves moq-dev/moq from the unpinned Git default branch" + fail=1 +fi # Swift: `from: "x"` floats to the newest compatible; an `.exact(` pin would not. if grep -q '\.exact(' clients/swift/Package.swift; then note FAIL "moq-swift pinned with .exact( (want from:, which floats to latest)" diff --git a/justfile b/justfile index 2551873..ca00948 100644 --- a/justfile +++ b/justfile @@ -25,6 +25,11 @@ full: cloudflare: ./cloudflare.sh +# Run moxygen's published protocol interop client through the latest moq-dev +# relay over WebTransport and raw QUIC. Linux Docker is required. +moxygen: + ./moxygen.sh + # Token interop: install moq-token in each published flavour and cross-verify. # The Rust moq-token binary comes from a channel (PATH); @moq/token comes from npm, # driven under both node and bun. Default: rust only. Pass flags through, e.g. diff --git a/moxygen.sh b/moxygen.sh new file mode 100755 index 0000000..7a28b71 --- /dev/null +++ b/moxygen.sh @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# Protocol-level interoperability smoke test using moxygen's published client. +# +# This is separate from smoke.sh's Hang media matrix: moxygen uses MoQ Media +# Interop packaging, while this client exercises transport setup, namespace +# publication, subscription errors, and publisher/subscriber routing directly. +set -euo pipefail + +MOQ_REPO="https://github.com/moq-dev/moq" +MOQ_RELAY="${MOQ_RELAY_BIN:-}" +DOCKER="${MOXYGEN_DOCKER:-docker}" +MOXYGEN_IMAGE="${MOXYGEN_IMAGE:-ghcr.io/facebookexperimental/moxygen-interop-client:latest-amd64}" +PORT="${MOXYGEN_SMOKE_PORT:-4444}" +TIMEOUT="${MOXYGEN_SMOKE_TIMEOUT:-45}" +VERSION="${MOXYGEN_MOQT_VERSION:-16}" +HTTPS_URL="https://127.0.0.1:${PORT}" +MOQT_URL="moqt://127.0.0.1:${PORT}" +CONTAINER_PREFIX="moxygen-smoke-$$" + +TMP=$(mktemp -d) +RELAY_PID="" + +have() { command -v "$1" >/dev/null 2>&1; } + +# shellcheck disable=SC2329 # invoked indirectly via 'trap cleanup EXIT' +kill_tree() { + local pid="$1" child + for child in $(pgrep -P "$pid" 2>/dev/null || true); do kill_tree "$child"; done + kill -KILL "$pid" 2>/dev/null || true +} + +# shellcheck disable=SC2329 # invoked indirectly via 'trap cleanup EXIT' +cleanup() { + "$DOCKER" rm -f "${CONTAINER_PREFIX}-webtransport" "${CONTAINER_PREFIX}-quic" >/dev/null 2>&1 || true + [[ -n "$RELAY_PID" ]] && kill_tree "$RELAY_PID" + rm -rf "$TMP" +} +trap cleanup EXIT + +missing=() +for tool in curl openssl pgrep timeout "$DOCKER"; do + have "$tool" || missing+=("$tool") +done +if [[ -z "$MOQ_RELAY" ]]; then + have cargo || missing+=("cargo") +fi +if [[ ${#missing[@]} -gt 0 ]]; then + echo "error: missing required tools: ${missing[*]}" >&2 + exit 1 +fi +if ! "$DOCKER" info >"$TMP/docker-info.log" 2>&1; then + echo "error: $DOCKER daemon is not running" >&2 + sed 's/^/ /' "$TMP/docker-info.log" >&2 || true + exit 1 +fi + +if [[ -z "$MOQ_RELAY" ]]; then + echo "building moq-relay from $MOQ_REPO (latest default branch)..." + if cargo install --quiet --locked --git "$MOQ_REPO" \ + --root "$TMP/moq-install" --target-dir "$TMP/moq-target" moq-relay \ + >"$TMP/moq-relay-build.log" 2>&1; then + MOQ_RELAY="$TMP/moq-install/bin/moq-relay" + else + echo "error: failed to build moq-relay" >&2 + sed 's/^/ /' "$TMP/moq-relay-build.log" >&2 || true + exit 1 + fi +fi + +[[ -x "$MOQ_RELAY" ]] || { + echo "error: moq-dev relay is not executable: $MOQ_RELAY" >&2 + exit 1 +} + +echo "pulling moxygen interop client: $MOXYGEN_IMAGE" +if ! "$DOCKER" pull --platform linux/amd64 "$MOXYGEN_IMAGE" >"$TMP/docker-pull.log" 2>&1; then + echo "error: failed to pull moxygen interop client" >&2 + sed 's/^/ /' "$TMP/docker-pull.log" >&2 || true + exit 1 +fi + +# The relay requires certificate files. The moxygen client disables verification +# only for this loopback test. +if ! openssl req -x509 -newkey rsa:2048 -nodes \ + -keyout "$TMP/localhost.key" -out "$TMP/localhost.crt" -days 1 \ + -subj /CN=localhost -addext subjectAltName=DNS:localhost,IP:127.0.0.1 \ + >"$TMP/openssl.log" 2>&1; then + echo "error: failed to generate the temporary relay certificate" >&2 + sed 's/^/ /' "$TMP/openssl.log" >&2 || true + exit 1 +fi + +echo "starting moq-dev relay on 127.0.0.1:${PORT}..." +"$MOQ_RELAY" --server-bind "127.0.0.1:${PORT}" \ + --tls-cert "$TMP/localhost.crt" --tls-key "$TMP/localhost.key" \ + --web-https-listen "127.0.0.1:${PORT}" \ + --web-https-cert "$TMP/localhost.crt" --web-https-key "$TMP/localhost.key" \ + --auth-public "" >"$TMP/relay.log" 2>&1 & +RELAY_PID=$! + +ready=0 +for _ in $(seq 1 60); do + if curl --cacert "$TMP/localhost.crt" -sf "$HTTPS_URL/certificate.sha256" >/dev/null 2>&1; then + ready=1 + break + fi + kill -0 "$RELAY_PID" 2>/dev/null || break + sleep 0.5 +done +if [[ "$ready" -ne 1 ]]; then + echo "error: moq-dev relay never became ready" >&2 + sed 's/^/ relay: /' "$TMP/relay.log" >&2 || true + exit 1 +fi + +overall=0 +run_transport() { + local transport="$1" url="$2" name + name="${CONTAINER_PREFIX}-${transport}" + echo "=== moxygen / moq-dev / $transport / draft-$VERSION ===" + if timeout -k 3 "$TIMEOUT" "$DOCKER" run --rm \ + --name "$name" --network host --platform linux/amd64 \ + "$MOXYGEN_IMAGE" --relay "$url" --tls_disable_verify \ + --versions "$VERSION" >"$TMP/$transport.tap" 2>&1; then + sed 's/^/ /' "$TMP/$transport.tap" + else + echo " FAIL moxygen interop client" >&2 + sed 's/^/ client: /' "$TMP/$transport.tap" >&2 || true + sed 's/^/ relay: /' "$TMP/relay.log" >&2 || true + overall=1 + fi + "$DOCKER" rm -f "$name" >/dev/null 2>&1 || true +} + +# moxygen recommends draft-16 for new integrations while its draft-18 support +# remains experimental. Pinning the wire draft is intentional; both source-head +# implementations are still refreshed on every run. +run_transport webtransport "$HTTPS_URL" +run_transport quic "$MOQT_URL" + +if [[ "$overall" -eq 0 ]]; then + echo "moxygen interop smoke: all checks passed" +else + echo "moxygen interop smoke: FAILURES detected" >&2 +fi +exit "$overall" From e6ac89b0e7705962086b28da8d171d1b98d479de Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 4 Sep 2026 18:27:23 -0700 Subject: [PATCH 2/2] make moxygen diagnostic non-blocking --- .github/workflows/smoke.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/smoke.yml b/.github/workflows/smoke.yml index 476f5b3..99524ba 100644 --- a/.github/workflows/smoke.yml +++ b/.github/workflows/smoke.yml @@ -255,10 +255,6 @@ jobs: moxygen: name: Moxygen interoperability runs-on: ubuntu-latest - # Current moq-dev HEAD negotiates successfully but fails moxygen's - # announce-subscribe case. Keep the full diagnostic running without making - # every unrelated PR red; remove this once that protocol mismatch is fixed. - continue-on-error: true permissions: contents: read packages: read @@ -273,4 +269,8 @@ jobs: persist-credentials: false - name: Moxygen interoperability + # Current moq-dev HEAD negotiates successfully but fails moxygen's + # announce-subscribe case. Keep the full diagnostic visible without + # failing the job; remove this once that protocol mismatch is fixed. + continue-on-error: true run: ./moxygen.sh