Skip to content

fix(net)!: the application owns a replacement track's start sequence - #5068

Draft
kixelated wants to merge 12 commits into
mainfrom
quest/m0/js-track-takeover
Draft

kixelated wants to merge 12 commits into
mainfrom
quest/m0/js-track-takeover

Conversation

@kixelated

@kixelated kixelated commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A track name always means the same content, so a producer that replaces an ended one under the same name must never reuse its group or datagram sequences. #4929 handled that in the library: Rust's producers, and JS's on-demand producers, silently continued from wherever the name's last producer stopped. JS createTrack didn't, so @moq/publish restarted its catalog at group 0 on every re-announce. It also restarted every rendition track whenever a rendition was registered or removed.

Direction from the maintainer: the application owns the start sequence. The library should not quietly start somewhere else. It should fail loudly when an app reuses a name in a way that would repeat sequences.

Approach

  • Rust and JS, same rule. A broadcast still records, per track name, one past the highest sequence any producer of the name wrote. An append no longer jumps ahead to that value. It takes one past the producer's own writes and refuses if that would reuse a written sequence: Error::Duplicate in Rust, Error("duplicate sequence: ...") in JS. A replacement picks its start by writing its first group with an explicit sequence (create_group / writeGroup), and later appends continue from there. Explicit writes stay unchecked, so a relay can still forward upstream sequences, including a group it had before. A new broadcast starts over.
  • JS createTrack answers a queued request, as Rust's create_track does. This is separate from sequences: without it, subscribers queued for a name stay stuck once the app creates the track.
  • JS insertTrack is removed. It has no Rust counterpart and no production caller. It would also bypass the name's sequence record.
  • JS sweeps unused names like Rust. Names that never wrote and that no open producer holds are swept as the record grows. There is no cap and no runtime refusal.
  • @moq/publish owns its sequences. Each track, catalog included, lives as long as the network broadcast. A re-announce, a rendition change, or re-registering a name keeps writing to the same producer. A maxAge change publishes a new network broadcast, because a track's retention is fixed when it is created. A rendition name keeps its kind within one network broadcast. A rendition track that closed (an encoder error) can't be continued, so replacing it publishes a new network broadcast, where the name starts over. This absorbs publish catalog restart, which is deleted.
  • JS Request.accept rejects on any failure, invalid info included, so a request is never left open with subscribers waiting.

Decisions

Settled 2026-10-09, still holding:

  1. A new name past the bound is a plain Error("too many track names"). Superseded by 8.
  2. The bound is 1024 names per broadcast. Superseded by 8.
  3. insertTrack is removed in this PR, not in a separate quest (confirmed 2026-10-11). ✅
  4. Taking over a request a handler already pulled stays duplicate, as in Rust. ✅

Settled 2026-10-11 (maintainer direction: the application owns the start sequence):

  1. Library behaviour when a replacement appends. Options: (a) continue the name's sequences silently (fix(net): one dynamic track per name, sequences continue across replacements #4929); (b) refuse an append that would reuse a written sequence, and leave explicit writes unchecked ✅; (c) refuse to re-create a written name at all. (c) breaks relays, which replace producers under the same name and write upstream sequences.
  2. How @moq/publish complies. Options: (a) keep each track for the network broadcast's life, and move to a new broadcast on maxAge ✅; (b) mint a fresh broadcast on every re-announce or rendition change; (c) carry sequences by hand into re-created tracks.

Settled 2026-10-11 by the maintainer:

  1. Replacing a rendition track that closed after writing. Options: (a) let the replacement's first keyframe throw; (b) have @moq/publish move to a new network broadcast, since a new broadcast is new content, in this PR ✅; (c) carry the closed track's next sequence into the replacement.
  2. The JS name bound. Options: (a) keep the 1024-name cap; (b) drop it and sweep unused names like Rust ✅. A runtime cap only adds a runtime error.

Impact

  • Public API (Rust moq-net): no signature change. Behavior change: track::Producer::append_group and append_datagram return Error::Duplicate instead of continuing past an earlier producer of the same name. This applies to create_track, reserve_track, and on-demand requests.
  • Public API (@moq/net): breaking. BroadcastProducer.insertTrack is removed. Track.Producer.appendGroup / appendDatagram throw duplicate sequence instead of continuing past an earlier producer of the name. createTrack answers a queued request instead of throwing duplicate track.
  • @moq/publish: tracks persist for the network broadcast's life. Rendition.close() no longer closes the network track. A maxAge change, or replacing a closed rendition track, recreates broadcast.net. Registering a name under a different kind on the same network broadcast throws.
  • @moq/net behavior: Request.accept closes the request's producer with the error whenever it throws.
  • Wire: none.

Testing

  • Rust: replacement_refuses_reused_sequences, stale_producer, and route_change.rs producer_replaced_* (an app replacing an aborted producer continues explicitly).
  • JS net: the replacement and created/requested namespace tests now assert the throw and the explicit continuation. The sweep forgets names that never wrote and keeps written and held ones.
  • JS net: createTrack with invalid info rejects the queued request it answers.
  • JS publish: a re-announce continues the catalog's group sequence, a rendition keeps its track across rendition changes and re-registration, replacing a closed track publishes a new network broadcast, a name keeps its kind only within one network broadcast, and a maxAge change swaps the network broadcast. The re-announce, rendition-change, and closed-track tests fail without the fix.
  • just check. There is no wire change, so the interop suite is not affected.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits October 8, 2026 16:29
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quences

createTrack answers a request still queued for its name, and createTrack and
insertTrack continue the name's group and datagram sequences within the
broadcast, as Rust's create_track does. A broadcast remembers written
sequences for up to 1024 names; unwritten, unheld names are swept and a new
name past the bound is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest run outcome: implemented and just check passes locally. Left as a draft, with these maintainer decisions open:

  1. How a new name is refused past the bound. Options: (a) a plain Error("too many track names") thrown by createTrack/insertTrack/Request.accept, with the request rejected to its subscribers (implemented); (b) an exported error class or stream code so callers can match on it; (c) no hard bound, only Rust's sweep of unused entries. Recommend (a). Rust has no bound today, so (c) would match it exactly, but the quest asked for a bound.
  2. The bound value. 1024 names per broadcast, counting only names that wrote or are held open. Recommend keeping 1024.
  3. What insertTrack does with a queued request. It still throws duplicate track, because the waiting subscribers belong to the request's producer. Options: (a) keep it (implemented); (b) remove insertTrack, which has no Rust counterpart and no production caller (a breaking change); (c) move subscribers between producers (complex). Recommend (a) now, and consider (b) in a separate API-cleanup quest.
  4. Taking over a request a handler already pulled. It stays duplicate, as in Rust, so a later accept/reject from the handler never conflicts. Recommend keeping this.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 9, 2026 11:17
…eover

Keep this branch's track-name behavior in the JS net doc and main's timestamp wording. The broadcast tests import both the sequence bound and textFrame, and write frames with textFrame.

Co-Authored-By: Grok 4.7 <noreply@x.ai>
createTrack is the only way to add a track by hand. It still answers a queued request, continues the name's sequences, and refuses a new name past 1024 with a plain Error. A request a handler already pulled stays duplicate.

Co-Authored-By: Grok 4.7 <noreply@x.ai>
@kixelated kixelated changed the title fix(net): JS createTrack takes over a queued request and continues sequences fix(net)!: JS createTrack takes over a queued request and continues sequences Oct 9, 2026
kixelated and others added 2 commits October 11, 2026 14:27
…eover

# Conflicts:
#	quest/m0/js-track-takeover.md
#	quest/m1/README.md
An append on a producer that replaces an ended one under the same name no
longer silently continues the name's sequences. It refuses (Rust
Error::Duplicate, JS "duplicate sequence") when it would reuse a sequence the
name already wrote; the application starts the replacement with an explicit
create_group/writeGroup. Explicit writes stay unchecked so relays keep
forwarding upstream numbering.

@moq/publish now keeps each track for its network broadcast's life, so a
re-announce or rendition change continues the same producer, and a maxAge
change publishes a new network broadcast. Absorbs the publish catalog
restart quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title fix(net)!: JS createTrack takes over a queued request and continues sequences fix(net)!: the application owns a replacement track's start sequence Oct 11, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T01:55:34.487853Z fcf9cf4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 611c590637

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tracks.cleanup(() => track.close());
tracks.run((demand) => {
demand.set(signal, demand.get(track.demand().used) ? track : undefined);
let track = tracks.get(rendition.name);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject rendition names that collide with catalog tracks

When a rendition is registered as catalog.json or catalog.json.z after catalog serving begins, this lookup returns the catalog producer instead of reporting that the name is taken. The rendition then publishes media into the same immutable track used for JSON snapshots, with catalog timing, retention, and priority, corrupting both streams. Keep catalog tracks separate from reusable rendition tracks, or reject these reserved names.

AGENTS.md reference: AGENTS.md:L64-L64

Useful? React with 👍 / 👎.

@kixelated kixelated Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a8b2a07: catalog tracks are no longer in the reusable rendition map, so a rendition named catalog.json or catalog.json.z fails as a duplicate track, as it did before this PR.

(Written by Claude Opus 5.5)

A rendition named like a catalog track would otherwise reuse the catalog's
producer instead of failing as a duplicate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8b2a07ae3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/net/src/broadcast.ts
throw new Error(`duplicate track: ${track.name}`);
const request = takeQueued(this.#state, name);
if (request) {
const producer = request.accept(info);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject queued requests when info validation fails

If a request for this name is queued and info is malformed, such as priority: 256, takeQueued removes the only handler-visible request before request.accept calls Producer.accept, whose validation throws without closing the producer. The producer remains open in state.requests, so the waiting subscriber never resolves and a later valid createTrack sees a duplicate indefinitely; validate before dequeuing or reject the request on this failure path. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L17-L17

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 62f0672: Request.accept now closes the producer with the error whenever accepting throws, invalid info included, so the queued request is rejected to its subscribers and no longer holds the name. A regression test covers it.

(Written by Claude Opus 5.5)

Comment on lines +147 to 148
// The network track stays with the broadcast, so registering the name again continues it.
track.set(undefined);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Correct the Rendition.close contract

This path deliberately stops closing the network track, but the exported Rendition.close() documentation in js/publish/src/rendition.ts still promises to close any active track. Consumers following that contract can now wait for track termination that never occurs; document that closing a rendition only unregisters and detaches it while the broadcast retains the track. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L28-L28

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 62f0672: Rendition.close() now says it detaches the track, and that the broadcast keeps the network track open so registering the name again continues it.

(Written by Claude Opus 5.5)

Invalid info left the request's producer open, so a queued request answered
by createTrack stranded its subscribers and held the name. Also document that
Rendition.close leaves the network track with the broadcast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62f0672298

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +291 to +292
let track = tracks.get(rendition.name);
if (!track) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recreate a terminal track when restarting a rendition

When an active rendition track is closed after an encoder error or teardown, then the rendition is unregistered and registered again under the same name, this cache hit returns the closed producer. A closed producer can never report demand again, so new subscribers do not cause the replacement rendition's track getter to become defined; before this change, re-registration created a replacement producer. Discard terminal entries or recreate the network broadcast rather than reusing the closed track.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f46a96d: a closed track is replaced when its name is registered again. Under this PR's rule, the replacement throws on an append that would reuse the closed track's sequences, rather than stalling silently. A test covers it.

(Written by Claude Opus 5.5)

Comment on lines +291 to +296
let track = tracks.get(rendition.name);
if (!track) {
track = broadcast.createTrack(
rendition.name,
Container.trackInfo({ maxAge, priority: Catalog.PRIORITY[rendition.kind] }),
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject changing the kind of a reused rendition name

If a caller closes video("x") and later registers audio("x"), this cache hit reuses the video producer, so the catalog advertises audio while the immutable track info retains PRIORITY.video instead of PRIORITY.audio, and the same track name carries two media kinds. Preserve and validate the original kind, or require a new broadcast when it changes.

AGENTS.md reference: AGENTS.md:L64-L64

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f46a96d: a rendition name keeps the kind it was first registered as for the broadcast's life, so audio("x") after video("x") throws at registration.

(Written by Claude Opus 5.5)

A rendition track closed by an encoder error is recreated on re-register
instead of reused, and registering a name under a different kind throws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f46a96dbe6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/publish/src/broadcast.ts Outdated
Comment on lines +138 to +142
const previous = this.#kinds.get(name);
if (previous !== undefined && previous !== kind) {
throw new Error(`rendition ${name} was registered as ${previous}, not ${kind}`);
}
this.#kinds.set(name, kind);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope remembered rendition kinds to the network broadcast

When no network broadcast exists, or after enabled, name, origin, or maxAge recreates net, closing video("x") and registering audio("x") still throws because #kinds persists for the entire component lifetime. No old track can be reused in those cases, and the new broadcast/epoch is precisely the boundary where changing content is safe, so this unnecessarily removes a previously valid reconfiguration path. Record the kind only for names materialized in the current network broadcast, or clear this state when that broadcast is replaced.

AGENTS.md reference: AGENTS.md:L64-L64

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 054b563: a kind is recorded only when a rendition track is created in the current network broadcast, and forgotten when that broadcast is replaced. A test covers changing the kind on a new broadcast.

(Written by Claude Opus 5.5)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 054b563c32

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/publish/src/broadcast.ts Outdated
Comment on lines +305 to +309
if (!track || track.closed.peek() !== undefined) {
track = broadcast.createTrack(
rendition.name,
Container.trackInfo({ maxAge, priority: Catalog.PRIORITY[rendition.kind] }),
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Publish failed rendition replacements on a new broadcast

When a rendition track aborts after writing at least one group and the rendition is registered again, this branch creates its replacement inside the same broadcast. Fresh evidence beyond the earlier closed-track finding is that Container.Legacy.Producer.encode() always starts a keyframe with appendGroup() (js/hang/src/container/legacy.ts:97), while the new shared namespace makes that append throw for this replacement, and neither built-in audio nor video encoder supplies an explicit initial sequence. Recovery therefore produces a track that fails on its first encoded keyframe; recreate the network broadcast/epoch or supply the preserved starting sequence instead. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L64-L64

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining for this PR. Throwing here is the intended result: under the maintainer's rule, the library refuses an append that would reuse a name's sequences rather than renumber. On main, this path silently restarts the replacement at group 0, which is the bug. Recovering properly means @moq/publish either moves to a new network broadcast/epoch or carries the sequence forward. That is an app-level design choice, so I am raising it with the maintainer as a follow-up rather than deciding it here.

(Written by Claude Opus 5.5)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: per the maintainer, fcf9cf4 now handles this in @moq/publish. Replacing a rendition track that closed moves the tracks to a new network broadcast, where the name starts over, instead of letting the replacement's first keyframe throw. A regression test covers a track that wrote, failed, and was registered again.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 11, 2026 18:40
…ew broadcast

@moq/net no longer refuses a broadcast's 1025th track name. Like Rust, it
sweeps names that never wrote and no producer holds as the record grows.

@moq/publish moves to a new network broadcast when a closed rendition track
must be replaced, since a replacement under the same name would restart its
sequences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: fcf9cf4901

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant