moq-net: anonymous peers each get a random origin, so an anonymous subscriber cannot resolve an anonymous publisher's broadcast (regression in #3042)

[riedlse]

Since 4cb5fc2 ("fix(net): assign anonymous server peers an origin", #3042), a relay that
allows anonymous publish and anonymous subscribe can no longer connect the two: a SUBSCRIBE
from one anonymous session for a broadcast announced by a different anonymous session is
rejected with 404 dropped.

Bisected to 4cb5fc2 over the 139 commits in 260a3a47..fd477082, 7 steps, no skips.

Repro

ghcr.io/eyevinn/mlmtest:latest (moqlivemock 0.14.0, draft-18) against moq-relay with
anonymous subscribe on the root path:

MOQ_AUTH_KEY=auth.jwk MOQ_AUTH_PUBLIC_SUBSCRIBE="" \
moq-relay --server-bind 0.0.0.0:4443 --web-http-listen 0.0.0.0:4443 \
          --tls-cert tls.crt --tls-key tls.key

ok 1 - setup-only
ok 2 - announce-only
ok 3 - publish-namespace-done
ok 4 - subscribe-error
not ok 5 - announce-subscribe
  error: SUBSCRIBE: request rejected: unknown request error code: 0x194: dropped
not ok 6 - subscribe-before-announce
  error: subscriber timed out (neither OK nor ERROR received)

rev	result
260a3a4	6/6
4cb5fc2^ (356df39)	6/6
4cb5fc2	4/6
fd47708	4/6
7bcee5d	4/6

Tests 1–4 pass because none of them cross two anonymous sessions; 5 and 6 are the two that do.

Mechanism

Before #3042 an anonymous peer carried peer_origin: None. The commit assigns each request a
fresh crate::Origin::random() in rs/moq-net/src/server.rs, and with_peer_origin's own
doc comment gives the contract:

Two sessions given the same origin are treated as one endpoint: routes learned from either
are kept off both, and content arriving on either is interchangeable with the other's.

A random origin per session makes every anonymous connection a distinct endpoint, so the
announce from the publisher session is not resolvable from the subscriber session. The 404
comes out of rs/moq-net/src/ietf/publisher.rs:546, carrying Error::Dropped as the reason.

The intent of #3042 looks right for cluster loop-prevention — the issue is that an
unauthenticated peer has no established identity, and giving it a unique one makes it its own
island. Two anonymous sessions to the same relay probably want the same (or no) origin.

Impact

This is currently costing 8 cells in the nightly interop matrix (imquic, moq-playa,
moqlivemock, moqtopus, xquic-draft-18 against our relay went 6/6 → 4/6 the night we picked up
the bump). Happy to test a patch against the same harness.

[riedlse]

Correcting myself: the mechanism I gave above is wrong, and the impact is broader than I described. The symptom and the bisect stand.

What I got wrong. I said distinct random origins make each anonymous session "its own island" whose routes are filtered from each other. That isn't it — advertisable_routes(routes, self_origin, exclude) is called with exclude = O_B while A's route is stamped [O_A], and O_A != O_B, so A's announce is advertised to B normally. The announce filter is fine.

What actually happens, from a --log-level debug run rather than from reading:

1. moq-relay lingers a broadcast 5s past its last publisher (DEFAULT_LINGER, rs/moq-relay/src/cluster.rs:29).
2. A new connection is advertised that lingering front before it announces anything.
3. That advertisement goes through Consumer::excluding(O_n), and AnnounceConsumerNotify::announce registers an ExclusionGuard for the freshly minted O_n in FrontState::excluded.
4. The peer then announces the same path; Subscriber::session_route stamps hops = [O_n].
5. attach_source's reflection gate FrontState::taints_a_reader fires (rs/moq-net/src/model/origin.rs:1955) — we put O_n in the chain and in excluded ourselves — and logs path already live with a different publisher; parking this source until it ends.
6. The real publisher's source is parked, so the SUBSCRIBE finds nothing live and gets the 404 dropped.

The gate is circular for an assigned identity: the relay excludes the peer under an id it minted, then reads the peer's own announce under that id as its own content reflected back. For a peer we dialed or authenticated that reading is sound; for an anonymous one, an ordinary reconnect is indistinguishable from a reflection.

It also isn't permanent, and it isn't interop-specific. Recovery happens when the linger expires. In a clean run of the 6-test suite I see exactly three parking events, on connections 2, 4 and 7 — every publisher after the first:

conn{id=2}: path already live with a different publisher; parking this source until it ends broadcast=moq-test/interop
conn{id=4}: path already live with a different publisher; parking this source until it ends broadcast=moq-test/interop
conn{id=7}: path already live with a different publisher; parking this source until it ends broadcast=moq-test/interop

So the production shape is: for linger seconds after any publisher of a path goes away, the next publisher that arrives without an established identity is stranded and its subscribers get 404. That's every reconnect and every publisher handoff, not just a test suite. Since moq-relay never calls with_peer_origin, every one of its peers is in that category.

On the fix — I have a patch that takes assigned_origin to Option<Origin>, None unless a caller establishes one, which restores what with_peer_origin's own doc comment describes ("Only for a peer whose identity the server has actually established"). It gives 6/6 on the suite with zero parking events, and all 960 moq-net tests plus all 8 of #3042's own tests stay green.

But I don't think that's obviously the right call, and it's yours to make. It reopens exactly what #3042 fixed — the PR body notes anonymous sessions stamping 0 caused Cloudflare moq-rs to receive its own PUBLISH_NAMESPACE and reject it as a duplicate — and it declines to do what the draft text you added says ("an endpoint accepting an anonymous session SHOULD assign a distinct id per session"). My reading is that the draft endorses the announce filter and the implementation extended the minted id to the reflection gate too, which is where it breaks. If that's right, the narrower fix is to carry an established-vs-minted bit through to FrontState so attach_source counts only established exclusions while prefer_untainted keeps counting all — but that's ~5 files in code you reasoned about carefully, and the existing tests wouldn't catch a subtle error, so I didn't want to guess at it.

Happy to send whichever shape you prefer as a PR, or just hand over the repro. Repro is the moqlivemock suite against a relay with MOQ_AUTH_PUBLIC_SUBSCRIBE=""; the parking lines only show at debug.

[kixelated]

okay DEFAULT_LINGER is removed on dev so maybe it just fixes the issue.

[riedlse]

Confirmed — it does. moq-relay built from dev @ 7f593a3, same anonymous-subscribe config: the moqlivemock suite goes 6/6 on two consecutive runs with zero parking this source events, versus 4/6 and three parks per run on main @ 7bcee5d. With no lingering front to advertise before the peer announces, the exclusion guard is never registered under the minted id, so there's nothing for taints_a_reader to trip on.

Two narrowings to my earlier comments, both in your favour:

• The blast radius is smaller than I said. A peer that declares its identity in SETUP (cluster::peer_into_setup) never takes the minted origin, so every moq-net-based client — including our own publishers — was unaffected. It only bites wire-anonymous peers: third-party IETF clients without the cluster extension, and lite clients that omit origin. That's the interop matrix, not production reconnects. I overstated that.
• So on main the fix is either "wait for dev" or a small server.rs change; on dev it's already done, and feat(moq-net)!: ban Hop ID 0 from hop chains #3583 banning hop 0 goes the other way from what I'd have patched, so I'm glad I asked rather than sent it.

The practical question for us is timing: main still carries both the linger and the minting, and it's what moq-relay 0.14.16 on crates.io ships. Is the dev line heading for a release before mid-October? If not, we'll stay on main and eat the interop cells until it does — I'd rather that than carry a patch that runs against where you're taking it.