moq import ts warns when an audio stream loses frame sync (#3372),
and Import::stats exposes that per PID as the counters an operator alarms on. There is no equivalent
for a stream that simply stops, and for video there is nothing at all.
That gap is measurable from outside, and the reason it matters is that no downstream check catches the
case either.
What was measured
A stimulus was built from a 1080i25 contribution capture (PCR on the video PID, as in most muxes) by
suppressing the video PES for 60 s while keeping its PCR, substituting null packets so the mux rate is
unchanged to the byte, and adjusting the continuity counters so the surviving sequence stays legal.
That is what an encoder whose video path has died behind a running mux emits.
The stimulus is indistinguishable from healthy before MoQ sees it, which is the point:
|
unmodified clip |
video suppressed 60 s |
| packets |
3,967,645 |
3,967,645 |
| PCRs |
24,574 |
24,574 |
| worst PCR interval |
24.951 ms |
24.951 ms |
continuity errors (TSDuck continuity) |
0 |
0 |
pcrverify --absolute --jitter-max 500 |
pass |
pass |
| video access units |
19,629 |
17,609 |
Run through moq import ts → relay → moq export ts --latency-max 500ms → a CBR groomer at
11 Mb/s, against a control arm on the same build (moq 0.10.0 / moq-relay 0.14.15), the delivered
output carries a 57.22 s hole in the video and:
| check |
control |
video suppressed |
| mux rate |
11,001,940 b/s |
11,001,354 b/s |
| continuity errors |
0 |
0 |
| PCR intervals > 40 ms |
0 |
0 |
| worst PCR interval |
30.080 ms |
30.080 ms |
| publisher / relay / exporter log lines |
— |
identical to control once broadcast names and connection ids are normalised |
So the whole of TR 101 290 P1 passes over a service with no pictures in it, and the transport says
nothing. The same experiment with the audio PIDs suppressed instead is visible in the publisher's
log, purely because of #3372:
WARN moq_mux::container::ts::import: audio stream lost frame sync and resynced pid=121 track=".mp2" discarded=466 resyncs=1
Two things are worth separating there. That line fires on recovery, not on the outage, so it is
late by the outage's duration — useful for "this feed is losing audio", not for "audio is off air
now". And it exists for audio only.
Why the importer is the right place for this
Downstream detection is possible but proportional to the dead stream's share of the mux, which we also
measured. The stuffing ratio at the groomer's output went from 13.7 % to 95.2 % when the video died —
unmissable, within a second. When the two audio streams and the subtitles died instead, about 440 kb/s
of a 9.5 Mb/s programme, it peaked at 27.0 % against a control that peaks at 27.1 %. Not
separable. A threshold sensitive enough for audio would fire on healthy VBR video.
The importer has no such problem, because it is already parsing each elementary stream in order to
demux it. It is the only component in the chain that knows a specific track has gone quiet without
doing any extra work — and this is a property of media-aware carriage specifically, since a relay
forwarding an opaque TS payload cannot know.
Suggested shape
Adding to the existing surface rather than a new one, and deliberately not proposing a behaviour
change:
- Extend
StreamStats to every elementary stream, not only audio, and add something equivalent to
time since the last access unit was published on this PID — or, if a monotonic counter suits the
existing style better, published access units, from which a poller derives liveness itself.
- Optionally a
tracing::warn! once a PID exceeds a configurable quiet threshold, matching the audio
resync line's shape (pid, track, and the gap). A threshold is needed because a legitimately
sparse stream — SCTE-35 sections arrive ~1.3 s apart in this clip — must not warn.
- No change to what is published or how tracks are mapped.
Happy to test a branch against the stimulus set; the generator and the per-PID grader are small and
can be attached or upstreamed if useful.
Scope of the measurement
One clip, one PID layout, loopback relay, single host, one run per arm plus a control. PCR shares a
PID with the video here, which is common but not universal; a mux carrying PCR on its own PID gives
the same result for the same reason, since the P1 checks are checks on the PCR either way. The
--on-stall mute groomer policy was in force, which matters only for the total-loss arm.
Related: #1838 (TR 101 290 monitoring requirements),
#3372 (the audio half this asks to generalise).
moq import tswarns when an audio stream loses frame sync (#3372),and
Import::statsexposes that per PID as the counters an operator alarms on. There is no equivalentfor a stream that simply stops, and for video there is nothing at all.
That gap is measurable from outside, and the reason it matters is that no downstream check catches the
case either.
What was measured
A stimulus was built from a 1080i25 contribution capture (PCR on the video PID, as in most muxes) by
suppressing the video PES for 60 s while keeping its PCR, substituting null packets so the mux rate is
unchanged to the byte, and adjusting the continuity counters so the surviving sequence stays legal.
That is what an encoder whose video path has died behind a running mux emits.
The stimulus is indistinguishable from healthy before MoQ sees it, which is the point:
continuity)pcrverify --absolute --jitter-max 500Run through
moq import ts→ relay →moq export ts --latency-max 500ms→ a CBR groomer at11 Mb/s, against a control arm on the same build (
moq0.10.0 /moq-relay0.14.15), the deliveredoutput carries a 57.22 s hole in the video and:
So the whole of TR 101 290 P1 passes over a service with no pictures in it, and the transport says
nothing. The same experiment with the audio PIDs suppressed instead is visible in the publisher's
log, purely because of #3372:
Two things are worth separating there. That line fires on recovery, not on the outage, so it is
late by the outage's duration — useful for "this feed is losing audio", not for "audio is off air
now". And it exists for audio only.
Why the importer is the right place for this
Downstream detection is possible but proportional to the dead stream's share of the mux, which we also
measured. The stuffing ratio at the groomer's output went from 13.7 % to 95.2 % when the video died —
unmissable, within a second. When the two audio streams and the subtitles died instead, about 440 kb/s
of a 9.5 Mb/s programme, it peaked at 27.0 % against a control that peaks at 27.1 %. Not
separable. A threshold sensitive enough for audio would fire on healthy VBR video.
The importer has no such problem, because it is already parsing each elementary stream in order to
demux it. It is the only component in the chain that knows a specific track has gone quiet without
doing any extra work — and this is a property of media-aware carriage specifically, since a relay
forwarding an opaque TS payload cannot know.
Suggested shape
Adding to the existing surface rather than a new one, and deliberately not proposing a behaviour
change:
StreamStatsto every elementary stream, not only audio, and add something equivalent totime since the last access unit was published on this PID — or, if a monotonic counter suits the
existing style better, published access units, from which a poller derives liveness itself.
tracing::warn!once a PID exceeds a configurable quiet threshold, matching the audioresync line's shape (
pid,track, and the gap). A threshold is needed because a legitimatelysparse stream — SCTE-35 sections arrive ~1.3 s apart in this clip — must not warn.
Happy to test a branch against the stimulus set; the generator and the per-PID grader are small and
can be attached or upstreamed if useful.
Scope of the measurement
One clip, one PID layout, loopback relay, single host, one run per arm plus a control. PCR shares a
PID with the video here, which is common but not universal; a mux carrying PCR on its own PID gives
the same result for the same reason, since the P1 checks are checks on the PCR either way. The
--on-stall mutegroomer policy was in force, which matters only for the total-loss arm.Related: #1838 (TR 101 290 monitoring requirements),
#3372 (the audio half this asks to generalise).