quest(m1): cluster routing as generic per-node routes and path-less announces

quest/m1/README.md

@@ -23,7 +23,9 @@ QUIC studies there on that rule.
 ## Required
 
 - [moq-transport peers are plain clients](/quest/m1/ietf-cluster-off.md) - no moq-transport session negotiates the cluster extension, so none can splice a pool's members on a Hop ID label
-- [Cluster routing](/quest/m1/cluster-routing/README.md) - edge and core tiers carry each broadcast into a region once over path vector, with the backbone hidden from end users
+- [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@<uuidv7>` path segment, shared by e2ee and broadcast epochs
+- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - each publish of a name gets a fresh `@<uuidv7>` epoch, viewers follow the newest live one at once, and bare names still resolve on every version
+- [Cluster routing](/quest/m1/cluster-routing/README.md) - any node routes toward a broadcast's origin over CDN and P2P links alike, with per-origin routes and path-less announces
 - [Remove `--hop`](/quest/m1/hop-removal.md) - on `dev`, redundant publishers share an explicit `@<epoch>`, and `--hop` and the publisher's Hop ID are gone
 - [One route cost](/quest/m1/route-cost.md) - on `dev`, Warm and Cold collapse to one static route cost
 - [Delete removed cluster flags](/quest/m1/cluster-shims.md) - on `dev`, once a release has carried their refusals, `mesh` and `linger` leave `cluster::Config`
@@ -147,9 +149,7 @@ QUIC studies there on that rule.
 - [Admission bench](/quest/m1/admission-bench.md) - the driver's per-event admission walk is benchmarked over tracks and copies
 - [Publish channel count](/quest/m1/publish-audio-channel-count.md) - forcing a channel count on an Audio.Capture stops costing the subscriber gaps of silence
 - [JS abandonment](/quest/m1/js-subscribe-abandonment.md) - a viewer returning during IETF subscribe setup keeps its track across microtasks
-- [Epoch primitive](/quest/m1/epoch.md) - one `Epoch` type in moq-net and @moq/net, carried as a trailing `@<uuidv7>` path segment, shared by e2ee and broadcast epochs
 - [E2EE](/quest/m1/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt
-- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - each publish of a name gets a fresh `@<uuidv7>` epoch, viewers follow the newest live one at once, and bare names still resolve on every version
 - [#3056](/quest/m1/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: the video decoder resets on a declared discontinuity
 - [#933](/quest/m1/933-video-rotation-metadata-not-propagated-from-mobile-camera.md) - the catalog rotation follows the live camera's orientation
 - [#2848](/quest/m1/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md) - the Opus producer follows its bandwidth grant through the settled `moq_mux::rate::Control`

quest/m1/auth/peer-grant.md

@@ -67,3 +67,4 @@ Additive.
 ## Related
 
 - [Signaling and policy](/quest/m2/p2p/signal.md) - the first consumer
+- [Route trust](/quest/m1/cluster-routing/route-trust.md) - extends the grant with the node ids its holder may advertise routes for, a second purpose beside serving paths

quest/m1/cluster-routing/link-quality.md

@@ -0,0 +1,34 @@
+# [M] Link quality
+
+## Goal
+
+A link can take its cost from its measured quality instead of configuration,
+so a drone mesh steers around a fading radio link without manual tuning,
+while a small wobble never flips a route. Static configured cost stays the
+default and is what the CDN runs; measured cost is opt-in per link. This is
+local policy feeding the one metric on the wire, with no wire change.
+
+## Plan
+
+Decided 2026-10-01: the wire carries one additive metric and each node
+computes its own link costs.
+
+- Derive the cost from what QUIC already measures (smoothed RTT, loss); an
+  ETX-style estimate (Babel RFC 8966 Appendix A, B.A.T.M.A.N.'s TQ) is the
+  reference. Pick the formula from the harness, not from intuition.
+- Hysteresis: only advertise a cost change past a threshold or after it
+  holds, so the route layer sees a few changes per real shift. Report how
+  many ROUTE changes a fading link causes.
+- Never mix it into business pricing: moq.pro's
+  [priced topology](https://github.com/moq-dev/moq.pro/blob/main/quest/m3/priced-topology.md)
+  keeps RTT out of monetary costs, and that stays true because measured cost
+  is opt-in on links the operator chooses.
+- A lossy-link fixture over the mock transport with mocked time, reusable by
+  the line's drone-mesh end-to-end test.
+
+Public API: a per-link way to ask for measured cost in the peer entry.
+Wire: none.
+
+## Required
+
+- [Routes and announces](/quest/m1/cluster-routing/routes.md) - a cost change costs one ROUTE per origin there, instead of a re-announce per broadcast

quest/m1/cluster-routing/multi-cdn.md

@@ -0,0 +1,49 @@
+# [M] Multi-CDN endpoints
+
+## Goal
+
+A client or publisher holds sessions to several CDNs at once, ranked by its
+own preference: moq.pro as primary, Cloudflare as secondary. A subscription
+goes over the most preferred link that has a route and moves to the next one
+when that route goes away; a publisher pushes the same epoch to every CDN so
+the move splices instead of restarting. The secondary needs nothing from us:
+a moq-transport relay that speaks no ROUTE is just a link.
+
+## Plan
+
+Decided 2026-10-01 (moq-dev/moq#4694):
+
+- Ranking: longest prefix, then the link's preference, then the metric, then
+  the rendezvous hash. Preference is a per-link rank the node sets (lower
+  wins, equal by default), separate from the additive link cost, so metrics
+  from different operators are never added together or compared; within one
+  preference the metric decides as before. Babel permits any choice among
+  feasible routes, so loop freedom is unchanged. The ranking lives in
+  [Selection](/quest/m1/cluster-routing/selection.md); this quest adds the
+  preference input and the endpoint side.
+- A session that speaks no ROUTE (moq-transport, older lite) is a link whose
+  announces resolve to a session-scoped origin at that link's cost.
+- Identity: two CDNs never share node ids, so failover across them splices
+  only for an epoch-qualified path; a bare path is a discontinuity. The
+  publisher side is the redundant pair of
+  [Selection](/quest/m1/cluster-routing/selection.md), from one process.
+- API: the Rust client config and `js/net` accept several upstream URLs, each
+  with its preference (and cost), feeding one shared origin. Mirror the shape
+  in the bindings that expose connect config. Pick the smallest shape that
+  covers both publish and subscribe.
+
+Test with mocked time: a subscriber linked to a lite relay (preferred) and a
+moq-transport relay (secondary), with a publisher pushing one epoch to both;
+losing the preferred relay moves the subscription at a group boundary with
+no timestamp rewind, and its return moves it back. A bare path ends and
+resubscribes instead.
+
+Docs: a multi-CDN section in the routing concept page and the client
+connect docs.
+
+Public API: per-upstream preference in client config, in Rust, JS, and the
+bindings. Wire: none.
+
+## Required
+
+- [Selection](/quest/m1/cluster-routing/selection.md) - the ranking this adds the preference input to

quest/m1/cluster-routing/route-trust.md

@@ -0,0 +1,39 @@
+# [M] Route trust
+
+## Goal
+
+A session may advertise routes only to nodes it is trusted for, so an
+authorized publisher cannot claim a cheap route to a CDN relay's node and
+attract its subscriptions. Cluster-peer links may advertise any node. A
+client link's node ids stay scoped to that session, unless its peer grant
+names the nodes it may speak for: then a Starlink drone carries its fleet's
+nodes as their real identities, and two uplinks of one swarm reach one origin
+instead of two unrelated ones.
+
+## Plan
+
+Decided 2026-10-01:
+
+- Session scoping is the default and ships with
+  [Routes and announces](/quest/m1/cluster-routing/routes.md): a scoped id
+  resolves only that session's own authorized announces and never shadows a
+  route learned on a trusted link.
+- Promotion rides [Peer grants](/quest/m1/auth/peer-grant.md), the
+  hop-bound credential a direct session already presents; extend it to name
+  the node ids (or an id range or prefix of them) the holder may advertise.
+  Decide the shape with the auth line.
+- A promoted link is still subject to its upstream attribute and the
+  down-only bit.
+
+Test: a client advertising a cheap route to a cluster node does not move any
+other session's subscription; two uplinks of one mesh holding grants for the
+same drone node fail over between each other at a group boundary instead of
+ending the subscription.
+
+Public API: the grant's node scope (`moq-token`, `js/token`, and the token
+CLI docs). Wire: none beyond the grant.
+
+## Required
+
+- [Routes and announces](/quest/m1/cluster-routing/routes.md) - the node ids this scopes
+- [Peer grants](/quest/m1/auth/peer-grant.md) - the credential this extends

quest/m1/cluster-routing/routes.md

@@ -0,0 +1,117 @@
+# [XL] Routes and announces
+
+## Goal
+
+Routing info splits into two layers on the session's announce stream, in the
+wip lite version, for every session. A ROUTE advertises reachability of one
+origin node; an ANNOUNCE says a prefix lives at a route's node and carries no
+path. A link flap or relay loss sends one ROUTE change per origin whose best
+route changed, never a re-announce per broadcast; ending a broadcast reaches
+each node about once and nothing hunts; failover to another neighbour is
+seamless; and no state of disagreeing neighbours keeps a dead broadcast
+alive.
+
+## Plan
+
+Decided 2026-10-01 (see the [line's decisions](/quest/m1/cluster-routing/README.md)):
+
+- Wire sketch, flexible in detail. ROUTE_START carries the node id, a seqno,
+  the metric, and a down-only bit, and implicitly takes the next stream-local
+  Route ID, as ANNOUNCE_START takes the next Announce ID today. ROUTE_UPDATE
+  (Route ID, seqno, metric, down-only bit) and ROUTE_END (Route ID) reference
+  it; the bit rides UPDATE so a route moving between a mesh path and a CDN
+  fallback changes it without retracting its announces.
+  ANNOUNCE_START carries the prefix (keeping today's Path Base/Keep
+  compression), a Route ID, and the origin's cost for that prefix.
+  ANNOUNCE_UPDATE re-prices it and ANNOUNCE_END ends one broadcast while the
+  route stays up. An announce never changes its Route ID: another origin
+  serving the same path is another ANNOUNCE. An ANNOUNCE naming an unknown
+  Route ID is a protocol violation, and ROUTE_END ends that stream's
+  ANNOUNCEs on the route. The hop list leaves this version.
+
+  ```text
+  ROUTE_START  node=0x7a3f seqno=41 metric=12   -> route 0
+  ANNOUNCE_START prefix=transcode/ route=0 cost=10    -> announce 0
+  ANNOUNCE_START prefix=transcode/foobar route=0 cost=1 -> announce 1
+  ROUTE_UPDATE route=0 seqno=42 metric=15
+  ANNOUNCE_END announce=1                       # one broadcast ends
+  ROUTE_END    route=0                          # ends announce 0 too
+  ```
+- A stream carries a ROUTE only while some ANNOUNCE on it references it: sent
+  just before the first, ended after the last. That bounds a client session
+  to the origins of what it asked for (announce interest already scopes it),
+  not the CDN's whole node table.
+- The announce cost is the origin's per-prefix seed (`transcode/**` at 10 and
+  `transcode/foobar` at 1 from one node, decided 2026-10-01) and nobody
+  changes it in transit; a path's cost through a route is that seed plus the
+  route metric. Link changes therefore touch only ROUTE, and feasibility runs
+  on the route metric alone. This is [One route cost](/quest/m1/route-cost.md)'s
+  single cost, which stops accumulating per hop once this lands.
+- Each hop adds its link cost plus one to the metric (decided 2026-10-01), so
+  the metric strictly increases as Babel requires while operators keep
+  configuring cost 0 for a free link.
+- The node id is global and opaque: a relay's `cluster.id` or a random id,
+  and an app's handshake id. It is needed so routes from two neighbours to
+  one origin are recognized as one, which carries loop freedom,
+  deduplication, the reply's serving node, and P2P dialing (an app maps an
+  announce's node to a roster peer).
+- Loop freedom is Babel's feasibility condition (RFC 8966) keyed by node:
+  accept a route if its seqno is newer, or equal with a metric below the
+  feasibility distance. Only the origin advances its seqno. Retraction is an
+  infinite metric or ROUTE_END; there is no count to infinity.
+- Next-hop authority: store every neighbour's ANNOUNCEs, but a prefix is live
+  at a node only as its current next hop toward the origin announces it, the
+  neighbour a subscribe for that path would go to (metric, then the path-keyed
+  rendezvous hash among equal next hops). When the next hop changes, adopt
+  the new neighbour's stored set and send downstream only the difference.
+  No per-announce seqno.
+- Down-only bit: set on a route learned on an upstream link, kept across
+  other links, and a route carrying it is never sent on an upstream link
+  ([Upstream links](/quest/m1/cluster-routing/transit.md)).
+- A plain client with one link advertises a ROUTE for itself and the
+  ANNOUNCEs it publishes; its node id is scoped to its session (shared
+  identity across sessions is [Route trust](/quest/m1/cluster-routing/route-trust.md)).
+  A relay advertising routes to a client sends them as usual; node ids reveal
+  nothing about the backbone.
+- Every hop re-selects; SUBSCRIBE names no origin. The reply names the
+  serving node, which is the identity Selection's splice rule uses.
+- Mixed versions: a lite-06 peer keeps today's path vector, translated at the
+  relay that speaks both, for the rollout window only.
+
+Why not a hold-down: #4644 retracted at once and re-announced a replacement
+after 1 s, which ended a downstream front whose only path ran through the
+relay, dropping subscribers for about a second while the relay kept serving;
+holding without retracting brought hunting back (moq-dev/moq.pro#2116 has the
+measurements, and #4642's 300 ms cursor hold is the production mitigation).
+Separating "the path changed" (ROUTE) from "the broadcast ended" (ANNOUNCE)
+removes that trade, and the failover tests must show no front loses its
+route while a working one exists. Decide whether #4642's hold can then go.
+
+Open, for the implementer to settle and record:
+
+- Seqno lifetime across restarts of a node with a configured stable id
+  (persist it, fold an incarnation into the id, or Babel's seqno request).
+- Whether starvation recovery needs a request message, or session restart
+  covers it.
+- The window where an origin's ANNOUNCE_END and a next-hop change cross in
+  flight, and whether it needs more than the next END to arrive.
+- How much of today's route trie, fronts, and `route_order` survives intact.
+
+Tests, time mocked: #4644's live-graph withdrawal and failover tests
+(`quest/m0/path-hunting` branch) as acceptance, run on the wip version; a
+link flap with many broadcasts behind one origin costs one ROUTE change per
+link; a ring of three where a stale neighbour cannot revive an ended
+broadcast; a two-uplink mesh that never carries CDN routes back to the CDN;
+an island reached through the CDN after a partition; an unknown Route ID
+refused.
+
+Wire: `drafts/draft-lcurley-moq-lite.md` in the same PR, and `js/net`
+encodes, decodes, and resolves it (JS transit stays in
+[P2P](/quest/m2/p2p/README.md)). Public API: the route-change surface on
+`broadcast::Route` and its bindings will likely change; report it. This may
+split at start (Rust and draft, then JS), as long as both land in one
+release.
+
+## Required
+
+- [Simulate the split](/quest/m1/cluster-routing/sim.md) - the numbers that confirm the design before the wire is written