Skip to content
5 changes: 5 additions & 0 deletions quest/m1/archive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ DVR expiry recover from the next checkpoint; `Reader::finish` applies out-of-ban

### Format

[Per-track timelines](/quest/m1/archive/track-timeline/README.md) replaces the
aligned segments below with one timeline per track.

The format is the draft's
[Recording section](/drafts/draft-lcurley-moq-hang.md#recording).
The application chooses the object prefix, selected tracks, retention, and credentials; `moq-archive` owns the
Expand Down Expand Up @@ -126,6 +129,8 @@ owned by that prerequisite, not duplicated in archive storage.

## Quests

- [Per-track timelines](/quest/m1/archive/track-timeline/README.md) - every track segments and expires on its own timeline, and HLS is derived from group timestamps at the edge
- [Replay catalog](/quest/m1/archive/replay-catalog.md) - `moq import archive` publishes the recorded catalog live with `store` set, so stock `moq export hls` serves the whole replay
- [Browser archive](/quest/m1/archive/browser.md) - the same contract for browser-published broadcasts
- [DVR rewind](/quest/m1/archive/dvr.md) - seek through a bounded archive and return to live playback
- [Archive proof](/quest/m1/archive/proof.md) - prove persistence ordering, selective reads, exact FETCH replay, and timeline-only HLS generation
Expand Down
21 changes: 11 additions & 10 deletions quest/m1/archive/browser.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ layout, and FETCH behavior as native `moq-archive` users.

Port the archive contract to the JS packages with memory and OPFS storage. The
application explicitly enrolls video, audio, catalog, or arbitrary data tracks;
the archive does not infer them from Hang. The JS timeline already publishes
through the Window (`js/hang/src/container/timeline.ts:134`, `:165`, backed by
`js/json/src/window/`) with pacing tracks and application-driven cuts; add the
deferred commit Rust has (`Producer::deferred`,
`rs/moq-mux/src/timeline.rs:917`).

Persist one range-named object per track per segment after its groups complete,
then publish the archive timeline record. Match the 19-digit group-bound keys,
ascending delta-encoded IDs, and sequential timeline discovery used by Rust. A typical audio segment contains many
one-group-per-frame audio groups. Match the Rust binary envelope bytes and `.info` property values, per the [Recording section](/drafts/draft-lcurley-moq-hang.md#recording), without inferring catalog-to-group applicability.
the archive does not infer them from Hang. Record against the per-track
timelines from [JS per-track timelines](/quest/m1/archive/track-timeline/js.md).

Commit each track independently: persist each stored span, including a frame
range of a still-open group, then publish that track's timeline record. Match
the object keys, envelope bytes, `.info` property values, and timeline
discovery the Rust writer uses after
[Rust per-track timelines](/quest/m1/archive/track-timeline/core.md), per the
[Recording section](/drafts/draft-lcurley-moq-hang.md#recording), without
inferring catalog-to-group applicability.
[Catalog track identity](/quest/m2/catalog-tracks.md) addresses that separately.

Use [JavaScript FETCH](/quest/m1/js-fetch.md)'s on-demand group requests to
Expand All @@ -34,3 +34,4 @@ remains downstream (moq.pro) work.
## Required

- [JavaScript FETCH](/quest/m1/js-fetch.md) - generic on-demand group serving and IETF FETCH support
- [JS per-track timelines](/quest/m1/archive/track-timeline/js.md) - the timeline this archive records against
Comment thread
kixelated marked this conversation as resolved.
9 changes: 9 additions & 0 deletions quest/m1/archive/dvr.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,15 @@ The reader evicts popped spans from its object cache, but a group it already
served stays in `moq_net`'s track cache until the pool reclaims it. Decide
whether expiry during a seek needs a group eviction API in `moq-net`.

`moq-hls` reads a timeline from the catalog's own broadcast, and an
`archive.replay` path only marks it non-durable. Decide whether a live
broadcast's exporter follows `replay` to the recording for rewind, or whether
viewers address the replay broadcast directly.

## Required

- [Rust per-track timelines](/quest/m1/archive/track-timeline/core.md) - seeks through per-track timelines

## Closes

- [#2275](https://github.com/moq-dev/moq/issues/2275) - close this issue when the quest finishes
9 changes: 7 additions & 2 deletions quest/m1/archive/proof.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ reads, and timeline-only HLS generation from one multi-rendition broadcast.

## Plan

Record explicitly enrolled video, audio, catalog, and non-media tracks. Cut at
aligned keyframe boundaries, including multiple audio groups per segment, then
Record explicitly enrolled video, audio, catalog, and non-media tracks. Cut each
track on its own timeline, including many audio groups per object and a group
split across objects by frame, then
replay their original sequences, timestamps, and payloads through
`track::Dynamic`.

Expand Down Expand Up @@ -60,3 +61,7 @@ restart must finish this cleanup before accepting new groups.
Finally render and reload HLS playlists while rejecting every media-object GET
until a segment URI is requested. A segment request must resolve one object
from the replayed timeline without any listing or separate index object.

## Required

- [Rust per-track timelines](/quest/m1/archive/track-timeline/core.md) - proves the per-track format, not the aligned one
48 changes: 48 additions & 0 deletions quest/m1/archive/replay-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# [S] Replay catalog

## Goal

A replayed recording plays as VOD HLS from the stock server: `moq export
archive`, then `moq import archive`, then `moq export hls` lists the whole
recording with no embedder supplying a catalog. The replay broadcast publishes
its recorded catalog live, stamped with the recording's `store` and `version`,
so the exporter treats its timeline as durable and lists past `--window`. A
`--follow` replay grows like an event playlist.

Choosing which catalog applies to which media group stays with
[Catalog track identity](/quest/m2/catalog-tracks.md).

## Plan

Today the reader writes live groups only for timelines; the recorded catalog
is FETCH-only, so a SUBSCRIBE to it on the replay broadcast never sees a group
and `moq-hls` never finds the `archive` entry. The HLS archive tests work around
this by hand-building a catalog.

- Republish each recorded catalog group live in its timeline's order, so the newest
one is at the live edge and a `--follow` replay picks up catalogs recorded
after it opened.
- Stamp `store` with the URL passed to `import archive`, and `version` with the
recording format. Refuse a URL carrying userinfo so credentials never land in
a catalog. `replay` stays unset: the timelines live on this broadcast.
- Keep the logic in `moq-cli`; `moq-archive` stays catalog-agnostic. Select the
catalog track with the CLI's catalog format as `export archive` does: hang
and hang.z are stamped, MSF is refused.
- The recorded catalog's `archive` entry describes the source's live
timelines, not the recording's, so replace it with the timelines the reader
replays rather than trusting the recorded ones.

Add a CI test that records a broadcast longer than the default window, replays
it, and asserts the stock exporter lists segment 0 with a durable
`timeShiftBufferDepth`. Consider moving the HLS archive tests onto the
republished catalog instead of their hand-built one.

Update `doc/bin/cli.md` and `doc/bin/hls.md` inline with an export, import, and
serve example.

Cover a DVR recording whose catalog outlived its first video segment, and fail
loudly on a recording with no catalog.

## Required

- [Rust per-track timelines](/quest/m1/archive/track-timeline/core.md) - a DVR keeps its catalog's newest group
46 changes: 46 additions & 0 deletions quest/m1/archive/track-timeline/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Per-track timelines

## Goal

Every track carries its own timeline, so tracks segment, commit, and expire
independently. A DVR keeps each track's newest group, such as a catalog that
never changes, and an append-only group stays addressable as frames arrive. An
edge like `moq-hls` derives HLS and DASH from group timestamps, so a publisher
never needs to know about HLS.

Nothing on `main` has users yet: break the timeline, catalog `archive` entry,
and recording format in place, with no compatibility path, even though hang
0.21 released the current shape. The children merge into this line's branch,
so Rust and JS reach `main` together.

## Plan

Today one timeline numbers aligned segments for every track. A DVR pops whole
segments, so a static catalog expires with the first video segment, and the
writer only records a non-pacing group once it completes, so a group that never
closes is never stored. `moq_json::window` pops from the front, so one shared
timeline cannot keep an old catalog record while dropping newer video.

Decisions:

- One timeline track per track, live and recorded. `moq-mux` publishes them for
every broadcast; an unsubscribed track costs nothing.
- The catalog's root `archive` entry maps each track to its timeline, including
the catalog track itself. `replay`, `store`, and `version` stay beside it.
- Each track cuts on its own: automatically at a group boundary between a
minimum and maximum duration (roughly 1s and 10s), splitting a long-lived
group by frame at the maximum. Manual cuts stay as an optimization, such as a
video keyframe cutting audio so derived segments need fewer objects.
- A stored object may hold a frame range of a group, not only whole groups.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add a distinct object key for each frame slice

The revised browser plan now defers to the Rust writer's keys, but this line still leaves the recording layout as groups/<largest>.<smallest> in quest/m1/archive/README.md:74-90. When one long-lived group is split at successive maximum-duration cuts, every immutable slice has the same <g>.<g> key, so the second PUT must either collide or overwrite the earlier frames, and recovery cannot identify the requested frame range. Define a frame-aware or per-record object identity in the core format and draft before making sliced groups a requirement.

Useful? React with 👍 / 👎.

- HLS and DASH segments are derived at the edge from group timestamps, not
from storage objects. Fetching extra objects is fine when they land in the
reader's cache for the next request.

## Quests

- [Rust per-track timelines](/quest/m1/archive/track-timeline/core.md) - the draft, `hang`, `moq-mux`, `moq-archive`, and `moq-hls` move to per-track timelines in one change
- [JS per-track timelines](/quest/m1/archive/track-timeline/js.md) - `@moq/hang` publishes and reads the same per-track timelines as Rust

## Related

- [Catalog track identity](/quest/m2/catalog-tracks.md) - the catalog's own timeline gives it timestamps, but which catalog applies to a group stays there
46 changes: 46 additions & 0 deletions quest/m1/archive/track-timeline/core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# [XL] Rust per-track timelines

## Goal

The moq-hang draft, `hang`, `moq-mux`, `moq-archive`, and `moq-hls` use one
timeline per track, per the [line's decisions](/quest/m1/archive/track-timeline/README.md).
A DVR recording keeps every track's newest group, an append-only group is
stored as its frames arrive, and `moq-hls` serves aligned HLS and DASH from
live or recorded timelines without the publisher cutting for it.

## Plan

These crates compile together, so they change in one PR. Update the draft's
timeline and Recording sections in the same change, in a new draft revision;
most of their rules assume one aligned segment counter (one timeline per
broadcast, cross-track boundaries, whole-segment retention, reading segment N
of track T through record N).

- **Timeline:** a record describes one stored span of its own track: sequence,
pts, duration, and the group and frame range. Drop cross-track pacing and the
completeness wait; keep manual cuts. Retention pops each track's window
independently and always keeps its newest record.
- **Catalog:** replace the root `archive.track` with a map from track to
timeline, covering the catalog track too.
- **Writer and reader:** commit each track independently, and store frame
ranges so a never-closing group is recorded. Bump the recording `version` and
refuse the old one. Recovery, grace deletion, and FETCH replay follow the
per-track index.
Comment on lines +25 to +28

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Support extending fetched groups before promising replay

When import archive --follow first serves an early slice of an open group and later frames of that same sequence are recorded, the current moq-net cache cannot expose the new slice: TrackState::covering_group treats a cached group whose first frame precedes frame_start as complete (rs/moq-net/src/model/track.rs:442-451), claim_sequence rejects replacing it (:793-812), and the archive reader finishes every fetched producer (rs/moq-archive/src/reader/mod.rs:319-332). A later FETCH therefore returns the already-finished prefix and never reaches storage. The quest needs to include a source-level moq-net cache/replay change or choose a representation that does not reuse a group sequence across independently arriving slices.

Useful? React with 👍 / 👎.

- **HLS and DASH:** take segment boundaries from a reference video rendition
(an audio one when the broadcast has no video), near a target duration, and
number them so every edge and every reload agrees, including after DVR pops. Every other video rendition snaps each
boundary to its nearest group start within a tolerance (around 1s); a
Comment on lines +29 to +32

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep reference boundaries stable across catalog changes

When the selected reference rendition is removed, reconfigured, or superseded by a newly added rendition, an existing edge and an edge joining afterward can select different reference timelines and derive different segment boundaries and numbers for the same media. Rendition membership is explicitly dynamic through Added and Removed events in rs/moq-hls/src/export/renditions.rs:452-477, while a late catalog subscriber does not have the prior selection, so the stated guarantee that every edge and reload agrees cannot follow from an unspecified current "reference video rendition." Persist a stable reference or boundary epoch, or derive boundaries independently of current rendition membership, and test reference removal/replacement.

Useful? React with 👍 / 👎.

segment with no start in range becomes a gap (`EXT-X-GAP` in HLS), so a
player switching renditions lands on the next real segment. Gaps are a
best-effort fallback: the draft and HLS docs say a publisher wanting HLS
export SHOULD align video GOPs across renditions. Audio and other
renditions take the groups and frames whose timestamps fall in each span,
possibly from more than one object. The reader's cache absorbs the overlap.

Carry over the existing tests (DVR trim, durable listing, archive replay) and
add a static catalog outliving its first video segment, an append-only group
spanning several objects, audio cut independently of video, renditions that
start at different times, video renditions whose group starts never
coincide, and an audio-only broadcast.

Update `doc/concept/hang.md` and the HLS and CLI docs this makes stale.
17 changes: 17 additions & 0 deletions quest/m1/archive/track-timeline/js.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# [M] JS per-track timelines

## Goal

`@moq/hang` publishes one timeline per track with the same records, cuts, and
catalog `archive` map as Rust, and parses them identically.

## Plan

Port the landed Rust shape to `js/hang/src/timeline.ts` and its catalog schema,
replacing the aligned timeline and cross-track pacing rather than keeping both.
Comment thread
kixelated marked this conversation as resolved.
Cover the same cut rules and a static catalog outliving other tracks' records,
and check the records against Rust output in the interop suite.

## Required

- [Rust per-track timelines](/quest/m1/archive/track-timeline/core.md) - the format this mirrors
Loading