From f39492e0fb59e6d16f2fc8972d52ad92eb1633bb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 10:41:10 -0700 Subject: [PATCH 1/6] quest(m1): carry #5290's system-buffer detail into psi-runs Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/psi-runs.md | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index cba4f19d17..614ca85acd 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -1,4 +1,4 @@ -# [S] A TS slot never repeats a PSI run back to back +# [M] A TS slot never repeats a PSI run back to back ## Goal @@ -6,7 +6,8 @@ When keyframes on several video PIDs align in one slot, the export does not lay identical PAT and PMT runs back to back, so the system buffer (512 bytes, ISO 13818-1 2.4.2.3) never overflows at 25 Mb/s and up. Each send-ahead unit still leads with its own tables as a tune-in point, and changed tables (a new -PMT version) always go out, in order. +PMT version) always go out, in order. `compliance.py` grades the PAT and PMT +buffers too, so every TS Interop arm catches a regression. ## Plan @@ -15,12 +16,26 @@ Found by Codex on #5207 (`schedule.rs`, repeated PSI groups). It exists on `aac_program_config_follows_each_table`, because send-ahead units need their own tables. +At 25 Mb/s, three PAT and PMT pairs back to back deliver 564 B of PAT in about +301 µs, of which about 38 B drains at Rxsys = 1 Mb/s, peaking near 526 B. Two +renditions' 376 B fit. + - Start from the author's narrower diff on #5207 (https://github.com/moq-dev/moq/pull/5207#discussion_r4239051134): drop a run only when the same flush would lay it right after an identical run, and pad - the dropped packets back with nulls in a padded slot. -- Test against system-buffer fill with the T-STD model, not a packet count: - three or more aligned video keyframes at 25 Mb/s overflow today, and pass - after the fix. The AAC tune-in test still passes. + the dropped packets back with nulls in a padded slot. `Slot::nulls` becomes + `Option` so an unpadded slot gains none. A slot still opens with its + clock packet and keeps its length, so no PCR moves. +- A new table version that bunches with the old one is not spaced out. +- Test against system-buffer fill with the T-STD model, not a packet count: a + unit test runs laid slots' PAT and each PMT PID through its own 512-byte + buffer drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on three video + PIDs (fails without the fix) and on two (passes either way). The AAC tune-in + test still passes. +- `test/ts/compliance.py` gains a hard-severity system-buffer check for the PAT + and every PMT PID: TBsys of 512 B drained at Rxsys, then Bsys, per ISO + 13818-1 2.4.2.3. Its `tstd` check models only elementary streams today. DVB SI + (SDT, EIT) and SCTE-35 stay out, since ISO defines these buffers for + program-specific information. Every existing TS arm still passes. Public API: none. Wire: none. From 5148ea46f46e42504f226e70c24fd84593b465ad Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 10:50:46 -0700 Subject: [PATCH 2/6] quest(m1): model the PAT and PMT in one shared TBsys Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/psi-runs.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 614ca85acd..83975e4d91 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -16,9 +16,11 @@ Found by Codex on #5207 (`schedule.rs`, repeated PSI groups). It exists on `aac_program_config_follows_each_table`, because send-ahead units need their own tables. -At 25 Mb/s, three PAT and PMT pairs back to back deliver 564 B of PAT in about -301 µs, of which about 38 B drains at Rxsys = 1 Mb/s, peaking near 526 B. Two -renditions' 376 B fit. +The PAT and the PMT share one TBsys: it takes PIDs 0 to 3 and the selected +program's PMT PID (ISO 13818-1 2.4.2.3). One PAT and PMT pair (376 B) fits. At +25 Mb/s, two pairs back to back deliver 752 B in about 180 µs, of which about +23 B drains at Rxsys = 1 Mb/s, peaking near 729 B; three peak near 1090 B. Two +distinct pairs fit only about 1.9 ms apart, at any mux rate. - Start from the author's narrower diff on #5207 (https://github.com/moq-dev/moq/pull/5207#discussion_r4239051134): drop a run @@ -28,14 +30,14 @@ renditions' 376 B fit. clock packet and keeps its length, so no PCR moves. - A new table version that bunches with the old one is not spaced out. - Test against system-buffer fill with the T-STD model, not a packet count: a - unit test runs laid slots' PAT and each PMT PID through its own 512-byte - buffer drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on three video - PIDs (fails without the fix) and on two (passes either way). The AAC tune-in - test still passes. -- `test/ts/compliance.py` gains a hard-severity system-buffer check for the PAT - and every PMT PID: TBsys of 512 B drained at Rxsys, then Bsys, per ISO - 13818-1 2.4.2.3. Its `tstd` check models only elementary streams today. DVB SI - (SDT, EIT) and SCTE-35 stay out, since ISO defines these buffers for - program-specific information. Every existing TS arm still passes. + unit test runs laid slots' PAT and PMT packets through one shared 512-byte + TBsys drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on two and three + video PIDs (both fail without the fix) and on one (passes either way). The AAC + tune-in test still passes. +- `test/ts/compliance.py` gains a hard-severity system-buffer check: for each + program, one 512 B TBsys fed by PIDs 0 to 3 and that program's PMT PID, + drained at Rxsys into Bsys. Its `tstd` check models only elementary streams + today. Other SI (NIT, SDT, EIT) and SCTE-35 never enter TBsys. Every existing + TS arm still passes. Public API: none. Wire: none. From 41ebb4e51595bc69f512ef120c6f74151ca729ca Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 11:06:03 -0700 Subject: [PATCH 3/6] quest(m1): keep PSI pacing in psi-runs, building on #5220 Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/README.md | 2 +- quest/m1/tstd/psi-runs.md | 42 ++++++++++++++++++++++++--------------- 2 files changed, 27 insertions(+), 17 deletions(-) diff --git a/quest/m1/tstd/README.md b/quest/m1/tstd/README.md index 10a4d62c11..c405e70802 100644 --- a/quest/m1/tstd/README.md +++ b/quest/m1/tstd/README.md @@ -37,7 +37,7 @@ nightly. - [Send-ahead within the delay](/quest/m1/tstd/send-ahead.md) - total lag is `--delay`, send-ahead included, with a 1 s default - [Mux-rate hold](/quest/m1/tstd/mux-rate-hold.md) - import publishes its catalog once the mux rate is measured, so export is constant-rate from the start -- [PSI runs](/quest/m1/tstd/psi-runs.md) - aligned keyframes never lay identical PAT/PMT runs back to back, so the system buffer holds +- [PSI runs](/quest/m1/tstd/psi-runs.md) - the PAT and PMT are deduplicated and paced so the shared system buffer never overflows ## Related diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 83975e4d91..99a27915be 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -1,13 +1,14 @@ -# [M] A TS slot never repeats a PSI run back to back +# [L] TS export paces PSI within the system buffer ## Goal -When keyframes on several video PIDs align in one slot, the export does not -lay identical PAT and PMT runs back to back, so the system buffer (512 bytes, -ISO 13818-1 2.4.2.3) never overflows at 25 Mb/s and up. Each send-ahead unit -still leads with its own tables as a tune-in point, and changed tables (a new -PMT version) always go out, in order. `compliance.py` grades the PAT and PMT -buffers too, so every TS Interop arm catches a regression. +The export's PAT and PMT never overflow the program's system transport buffer +(TBsys, 512 bytes drained at Rxsys = 1 Mb/s, ISO 13818-1 2.4.2.3) at any mux +rate. Identical runs that the same flush would lay back to back go out once. +Distinct tune-in runs (each send-ahead unit leads with its own tables) and +changed tables (a new PMT version) are paced, never dropped, and changed tables +go out in order. `compliance.py` grades TBsys too, so every TS Interop arm +catches a regression. ## Plan @@ -19,21 +20,30 @@ own tables. The PAT and the PMT share one TBsys: it takes PIDs 0 to 3 and the selected program's PMT PID (ISO 13818-1 2.4.2.3). One PAT and PMT pair (376 B) fits. At 25 Mb/s, two pairs back to back deliver 752 B in about 180 µs, of which about -23 B drains at Rxsys = 1 Mb/s, peaking near 729 B; three peak near 1090 B. Two -distinct pairs fit only about 1.9 ms apart, at any mux rate. +23 B drains, peaking near 729 B; three peak near 1090 B. Two distinct pairs fit +only about 1.9 ms apart, at any mux rate. - Start from the author's narrower diff on #5207 (https://github.com/moq-dev/moq/pull/5207#discussion_r4239051134): drop a run only when the same flush would lay it right after an identical run, and pad the dropped packets back with nulls in a padded slot. `Slot::nulls` becomes `Option` so an unpadded slot gains none. A slot still opens with its - clock packet and keeps its length, so no PCR moves. -- A new table version that bunches with the old one is not spaced out. -- Test against system-buffer fill with the T-STD model, not a packet count: a - unit test runs laid slots' PAT and PMT packets through one shared 512-byte - TBsys drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on two and three - video PIDs (both fail without the fix) and on one (passes either way). The AAC - tune-in test still passes. + clock packet and keeps its length, so no PCR moves. #5304 implements this + step and stays a draft until pacing lands. +- Then pace what suppression leaves: a distinct tune-in run or a changed + version goes out only once TBsys has drained enough to take it, within the + slot budget and the units' decode deadlines. #5304 found equal-weight aligned + units on four and eight video PIDs still overflow after suppression. +- #5220 (multi-packet PMT) adds PMT pacing at the system-buffer rate, since a + PMT section of up to 1,024 B spans several packets and can exceed 512 B alone. + Build on that pacing rather than adding a second one, pacing the PAT and PMT + together since they share TBsys. +- Test against TBsys fill with the T-STD model, not a packet count: a unit test + runs laid slots' PAT and PMT packets through one shared 512-byte TBsys + drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on two and three video + PIDs (both fail without the fix) and on one (passes either way). Add a + changed-version regression and #5304's distinct-run cases. The AAC tune-in + test still passes. - `test/ts/compliance.py` gains a hard-severity system-buffer check: for each program, one 512 B TBsys fed by PIDs 0 to 3 and that program's PMT PID, drained at Rxsys into Bsys. Its `tstd` check models only elementary streams From 8120d6a9db8343e04d2088dfe5edc70af54df25f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 11:12:52 -0700 Subject: [PATCH 4/6] quest(m1): pace every TBsys PID together, owning PMT pacing if needed Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/psi-runs.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 99a27915be..16e422ab9a 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -36,8 +36,10 @@ only about 1.9 ms apart, at any mux rate. units on four and eight video PIDs still overflow after suppression. - #5220 (multi-packet PMT) adds PMT pacing at the system-buffer rate, since a PMT section of up to 1,024 B spans several packets and can exceed 512 B alone. - Build on that pacing rather than adding a second one, pacing the PAT and PMT - together since they share TBsys. + Build on that pacing rather than adding a second one, or own it here if #5220 + lands without it. Pace every PID that enters TBsys as one lane: the PAT, the + PMT, and any `mpegts.si` passthrough on PIDs 1 to 3, which the scheduler gives + its own `Buffer::SYSTEM` lane today. - Test against TBsys fill with the T-STD model, not a packet count: a unit test runs laid slots' PAT and PMT packets through one shared 512-byte TBsys drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on two and three video From 9771125f3125d8167e82493166cb56d7b05a87d3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 11:21:00 -0700 Subject: [PATCH 5/6] quest(m1): bound Bsys too and test PIDs 1 to 3 in psi-runs Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/README.md | 2 +- quest/m1/tstd/psi-runs.md | 55 +++++++++++++++++++++------------------ 2 files changed, 30 insertions(+), 27 deletions(-) diff --git a/quest/m1/tstd/README.md b/quest/m1/tstd/README.md index c405e70802..16ddc27252 100644 --- a/quest/m1/tstd/README.md +++ b/quest/m1/tstd/README.md @@ -37,7 +37,7 @@ nightly. - [Send-ahead within the delay](/quest/m1/tstd/send-ahead.md) - total lag is `--delay`, send-ahead included, with a 1 s default - [Mux-rate hold](/quest/m1/tstd/mux-rate-hold.md) - import publishes its catalog once the mux rate is measured, so export is constant-rate from the start -- [PSI runs](/quest/m1/tstd/psi-runs.md) - the PAT and PMT are deduplicated and paced so the shared system buffer never overflows +- [PSI runs](/quest/m1/tstd/psi-runs.md) - the PAT and PMT are deduplicated and paced so the system buffers never overflow ## Related diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index 16e422ab9a..a6d27c75bb 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -1,21 +1,22 @@ -# [L] TS export paces PSI within the system buffer +# [L] TS export paces PSI within the system buffers ## Goal -The export's PAT and PMT never overflow the program's system transport buffer -(TBsys, 512 bytes drained at Rxsys = 1 Mb/s, ISO 13818-1 2.4.2.3) at any mux -rate. Identical runs that the same flush would lay back to back go out once. -Distinct tune-in runs (each send-ahead unit leads with its own tables) and -changed tables (a new PMT version) are paced, never dropped, and changed tables -go out in order. `compliance.py` grades TBsys too, so every TS Interop arm -catches a regression. +The export's program tables never overflow the program's system buffers at any +mux rate: TBsys (512 bytes drained at Rxsys = 1 Mb/s) and Bsys (1,536 bytes +drained at Rsys = max(80 kb/s, mux rate / 500)), per ISO 13818-1 2.4.2.3. +Identical runs that the same flush would lay back to back go out once. Distinct +tune-in runs (each send-ahead unit leads with its own tables) and changed tables +(a new PMT version) are paced, never dropped, and changed tables go out in +order. `compliance.py` grades both buffers too, so every TS Interop arm catches +a regression. ## Plan Found by Codex on #5207 (`schedule.rs`, repeated PSI groups). It exists on `main` before #5207 too. Keeping each table once per slot was tried and broke -`aac_program_config_follows_each_table`, because send-ahead units need their -own tables. +`aac_program_config_follows_each_table`, because send-ahead units need their own +tables. The PAT and the PMT share one TBsys: it takes PIDs 0 to 3 and the selected program's PMT PID (ISO 13818-1 2.4.2.3). One PAT and PMT pair (376 B) fits. At @@ -28,28 +29,30 @@ only about 1.9 ms apart, at any mux rate. only when the same flush would lay it right after an identical run, and pad the dropped packets back with nulls in a padded slot. `Slot::nulls` becomes `Option` so an unpadded slot gains none. A slot still opens with its - clock packet and keeps its length, so no PCR moves. #5304 implements this - step and stays a draft until pacing lands. -- Then pace what suppression leaves: a distinct tune-in run or a changed - version goes out only once TBsys has drained enough to take it, within the - slot budget and the units' decode deadlines. #5304 found equal-weight aligned - units on four and eight video PIDs still overflow after suppression. + clock packet and keeps its length, so no PCR moves. #5304 implements this step + and stays a draft until pacing lands. +- Then pace what suppression leaves: a distinct tune-in run or a changed version + goes out only once TBsys and Bsys have drained enough to take it, within the + slot budget and the units' decode deadlines. Bsys caps sustained PSI at Rsys, + 80 kb/s below 40 Mb/s. #5304 found equal-weight aligned units on four and + eight video PIDs still overflow after suppression. - #5220 (multi-packet PMT) adds PMT pacing at the system-buffer rate, since a PMT section of up to 1,024 B spans several packets and can exceed 512 B alone. Build on that pacing rather than adding a second one, or own it here if #5220 lands without it. Pace every PID that enters TBsys as one lane: the PAT, the PMT, and any `mpegts.si` passthrough on PIDs 1 to 3, which the scheduler gives its own `Buffer::SYSTEM` lane today. -- Test against TBsys fill with the T-STD model, not a packet count: a unit test - runs laid slots' PAT and PMT packets through one shared 512-byte TBsys - drained at 1 Mb/s, with keyframes aligned at 25 Mb/s on two and three video - PIDs (both fail without the fix) and on one (passes either way). Add a - changed-version regression and #5304's distinct-run cases. The AAC tune-in - test still passes. +- Test against buffer fill with the T-STD model, not a packet count: a unit test + runs laid slots' PIDs 0 to 3 and PMT packets through one shared TBsys and + Bsys, with keyframes aligned at 25 Mb/s on two and three video PIDs (both fail + without the fix) and on one (passes either way). Add a changed-version + regression, #5304's distinct-run cases, a PID 1 to 3 passthrough packet beside + a pair (fails while it keeps its own lane), and a sustained burst that fits + TBsys but overflows Bsys. The AAC tune-in test still passes. - `test/ts/compliance.py` gains a hard-severity system-buffer check: for each - program, one 512 B TBsys fed by PIDs 0 to 3 and that program's PMT PID, - drained at Rxsys into Bsys. Its `tstd` check models only elementary streams - today. Other SI (NIT, SDT, EIT) and SCTE-35 never enter TBsys. Every existing - TS arm still passes. + program, one TBsys fed by PIDs 0 to 3 and that program's PMT PID, draining + into one Bsys, both checked for overflow. Its `tstd` check models only + elementary streams today. Other SI (NIT, SDT, EIT) and SCTE-35 never enter + TBsys. Every existing TS arm still passes. Public API: none. Wire: none. From 7b20815480ebfaa0ad215480db56f796d126bc06 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 11 Oct 2026 11:27:14 -0700 Subject: [PATCH 6/6] quest(m1): derive PSI spacing from the mux rate and check TBsys empties Co-Authored-By: Claude Opus 5.5 --- quest/m1/tstd/psi-runs.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/quest/m1/tstd/psi-runs.md b/quest/m1/tstd/psi-runs.md index a6d27c75bb..9877a57401 100644 --- a/quest/m1/tstd/psi-runs.md +++ b/quest/m1/tstd/psi-runs.md @@ -21,8 +21,9 @@ tables. The PAT and the PMT share one TBsys: it takes PIDs 0 to 3 and the selected program's PMT PID (ISO 13818-1 2.4.2.3). One PAT and PMT pair (376 B) fits. At 25 Mb/s, two pairs back to back deliver 752 B in about 180 µs, of which about -23 B drains, peaking near 729 B; three peak near 1090 B. Two distinct pairs fit -only about 1.9 ms apart, at any mux rate. +23 B drains, peaking near 729 B; three peak near 1090 B. At high rates two +distinct pairs fit only about 1.9 ms apart; pacing derives the spacing from +simulated occupancy at the actual mux rate. - Start from the author's narrower diff on #5207 (https://github.com/moq-dev/moq/pull/5207#discussion_r4239051134): drop a run @@ -51,8 +52,9 @@ only about 1.9 ms apart, at any mux rate. TBsys but overflows Bsys. The AAC tune-in test still passes. - `test/ts/compliance.py` gains a hard-severity system-buffer check: for each program, one TBsys fed by PIDs 0 to 3 and that program's PMT PID, draining - into one Bsys, both checked for overflow. Its `tstd` check models only - elementary streams today. Other SI (NIT, SDT, EIT) and SCTE-35 never enter - TBsys. Every existing TS arm still passes. + into one Bsys, both checked for overflow, and TBsys for emptying at least + once a second (2.4.2.6), with a negative control for each. Its `tstd` check + models only elementary streams today. Other SI (NIT, SDT, EIT) and SCTE-35 + never enter TBsys. Every existing TS arm still passes. Public API: none. Wire: none.