What happens
moq import ts, and the SRT gateway through it, end the whole ingest on the first packet they cannot
parse. One damaged packet at 20 s into an otherwise clean 60 s off-air capture (H.264, MP2, AC-3 and
teletext, 9.95 Mb/s, PCR-paced through tsp -P regulate into moq import ts):
| Damage at 20 s |
dev @ 9a80e87 |
main @ 6a01640 |
One video PES header: flag and timestamp bytes zeroed, transport_error_indicator clear |
exits 1: InvalidInput: Unexpected marker bits |
same |
One H.264 NAL header: forbidden_zero_bit set |
exits 1: h264: forbidden zero bit is not zero |
same |
The same PES damage with transport_error_indicator set |
carries on |
carries on |
| About two dozen lost 7-packet datagrams over 60 s (continuity gaps) |
carries on |
carries on |
The importer is unchanged from 9a80e875 to the current dev tip. The SRT gateway ends the
connection on the same errors (SRT ingest ended with error … mux: …). A caller that redials turns
that into an outage of about a second, and a subscriber that does not --linger loses its output
for good.
Why it matters
transport_error_indicator covers the errors a demodulator detected. It does not cover a bit error
that got past FEC, a multiplexer bug, or a sender quirk. On a 24/7 contribution feed, one such packet
in days of input is enough to end the ingest.
One sender quirk showed how easy it is to hit. srt-live-transmit 1.5.6, fed from a pipe, pads every
short stdin read with zeros out to its full chunk. Its default chunk is 1456 bytes, which is not a
multiple of 188. The result is packets with a valid header over zeros or over the next packet's bytes.
In 4-minute runs on a loaded host, the gateway ended 10 to 15 sessions on Unexpected marker bits,
Expected stuffing byte 0xFF, Expected packet start code prefix, CRC32 mismatch and
h264: forbidden zero bit is not zero. That is a defect in the sender, not here, and a libsrt
receiver gets the same zero-filled payloads. But it is exactly the kind of damage a broadcast
demultiplexer is expected to survive.
The watch side already follows this principle: quest/m1/watch-decoder-recovery.md rebuilds the
decoder on one malformed packet instead of ending playback. This asks for the ingest equivalent.
Where
In rs/moq-mux/src/container/ts/import.rs, decode runs
while let Some(packet) = self.reader.read_ts_packet()? { self.handle_packet(packet)?; }. So a
PES-header or adaptation-field error from mpeg2ts, and a codec error raised from handle_packet,
both end the import. The transport_error_indicator branch in the same loop already drops a flagged
packet, and it is the model for the rest.
Suggestion
Treat a parse error confined to one packet, PES or access unit as damage to that unit:
- drop the unit;
- count it in the importer's stats, alongside the audio resync counters;
- where the codec needs one, have the track wait for its next keyframe;
- carry on.
Keep a fatal error for input that cannot be TS at all, such as sync that never comes back.
Reproduce
Zero one PES header's flag and timestamp bytes on the video PID after 20 s (TEI left clear):
import sys
data = bytearray(open(sys.argv[1], "rb").read())
pid, at = int(sys.argv[3], 0), int(float(sys.argv[2]) * len(data) / 188 / 72.3) # 72.3 s clip
for i in range(at, len(data) // 188):
p = i * 188
if ((data[p + 1] & 0x1F) << 8 | data[p + 2]) == pid and data[p + 1] & 0x40:
off = 4 if (data[p + 3] >> 4) & 3 == 1 else 5 + data[p + 4]
if data[p + off : p + off + 3] == b"\0\0\1":
data[p + off + 6 : p + off + 19] = bytes(13)
break
sys.stdout.buffer.write(data)
python3 corrupt.py clip.ts 20 0x6f \
| tsp -I file - -P regulate --pcr-synchronous -O file - \
| moq --connect <relay> --broadcast damaged.hang import ts
# exits 1 at 20 s: Error: InvalidInput: Unexpected marker bits
What happens
moq import ts, and the SRT gateway through it, end the whole ingest on the first packet they cannotparse. One damaged packet at 20 s into an otherwise clean 60 s off-air capture (H.264, MP2, AC-3 and
teletext, 9.95 Mb/s, PCR-paced through
tsp -P regulateintomoq import ts):dev@ 9a80e87main@ 6a01640transport_error_indicatorclearInvalidInput: Unexpected marker bitsforbidden_zero_bitseth264: forbidden zero bit is not zerotransport_error_indicatorsetThe importer is unchanged from
9a80e875to the currentdevtip. The SRT gateway ends theconnection on the same errors (
SRT ingest ended with error … mux: …). A caller that redials turnsthat into an outage of about a second, and a subscriber that does not
--lingerloses its outputfor good.
Why it matters
transport_error_indicatorcovers the errors a demodulator detected. It does not cover a bit errorthat got past FEC, a multiplexer bug, or a sender quirk. On a 24/7 contribution feed, one such packet
in days of input is enough to end the ingest.
One sender quirk showed how easy it is to hit.
srt-live-transmit1.5.6, fed from a pipe, pads everyshort stdin read with zeros out to its full chunk. Its default chunk is 1456 bytes, which is not a
multiple of 188. The result is packets with a valid header over zeros or over the next packet's bytes.
In 4-minute runs on a loaded host, the gateway ended 10 to 15 sessions on
Unexpected marker bits,Expected stuffing byte 0xFF,Expected packet start code prefix,CRC32 mismatchandh264: forbidden zero bit is not zero. That is a defect in the sender, not here, and a libsrtreceiver gets the same zero-filled payloads. But it is exactly the kind of damage a broadcast
demultiplexer is expected to survive.
The watch side already follows this principle:
quest/m1/watch-decoder-recovery.mdrebuilds thedecoder on one malformed packet instead of ending playback. This asks for the ingest equivalent.
Where
In
rs/moq-mux/src/container/ts/import.rs,decoderunswhile let Some(packet) = self.reader.read_ts_packet()? { self.handle_packet(packet)?; }. So aPES-header or adaptation-field error from
mpeg2ts, and a codec error raised fromhandle_packet,both end the import. The
transport_error_indicatorbranch in the same loop already drops a flaggedpacket, and it is the model for the rest.
Suggestion
Treat a parse error confined to one packet, PES or access unit as damage to that unit:
Keep a fatal error for input that cannot be TS at all, such as sync that never comes back.
Reproduce
Zero one PES header's flag and timestamp bytes on the video PID after 20 s (TEI left clear):