Corruption-pattern survey tool for FEC design

[josephnef]

Summary

Follow-up B from #83 (depends on #84's phy soft metrics): a chip-side DEVOURER_RX_DUMP_ALL=1 env var that emits one line per RX frame with the chip's full integrity + phy soft-metric vector, plus an aggregate analyser that turns those into FEC-design-grade statistics.

The previous work showed that the chip-corrupt pipeline now reaches the application layer (#83) and that per-frame phy metrics let the analyser correlate BER with SNR (#84). This PR is the third leg: a long-capture survey tool that characterises the actual corruption-pattern distribution real-world deployments face, so a FEC layer on top of the stream link can be sized empirically rather than guessed.

Changes

• demo/main.cpp — new DEVOURER_RX_DUMP_ALL=1 knob emits <devourer-corrupt-any>len=L crc_err=X icv_err=Y rate=R rssi=A,B evm=A,B snr=A,B. Body bytes are deliberately omitted (a hot survey would inflate the log past usable size); the aggregate report only needs length + flags + phy.
• tools/precoder/corruption_survey.py — new tool that reads those lines and reports:
  • headline chip-clean vs chip-corrupt counts
  • corruption rate broken down by DESC_RATE (the CCK-vs-OFDM split — without this the headline is dominated by always-clean CCK ACKs/beacons and underestimates what OFDM data faces)
  • frame-size distribution for each population
  • phy-metric stats per population, filtered to frames where the chip populated phy stats (CCK reports 0/0; we treat as "no measurement" instead of "0 dB" so the buckets don't collapse)
  • per-SNR-bucket corruption rate (where measurable)
  • temporal clustering (live captures only)
  • a heuristic FEC recommendation based on median-vs-peak corruption rate

Bench finding

60-second ch6 capture in a busy office environment with several APs in range:

=== corruption survey (2266 frames, file/pipe) ===
chip-clean       :   1663 ( 73.4%)
chip-corrupt     :    603 ( 26.6%)
corruption rate  : 26.61%
no-phy-measurement:  2103  (CCK/short frames, chip reports 0/0)

Corruption rate by DESC_RATE:
   idx name            count      %    corrupt    rate
  0x00 1M CCK           2075  91.6%        412  19.9%
  0x02 5.5M CCK            2   0.1%          2 100.0%
  0x03 11M CCK             1   0.0%          1 100.0%
  0x04 6M OFDM            17   0.8%         17 100.0%
  0x05 9M OFDM            19   0.8%         19 100.0%
  0x06 12M OFDM           20   0.9%         20 100.0%
  0x07 18M OFDM           31   1.4%         31 100.0%
  0x08 24M OFDM           22   1.0%         22 100.0%
  0x09 36M OFDM           30   1.3%         30 100.0%
  0x0a 48M OFDM           31   1.4%         31 100.0%
  0x0b 54M OFDM           18   0.8%         18 100.0%

Reading the result

• 1M CCK loses ~20% even at this location — CCK is robust but background interference still nukes one in five ACKs/beacons.
• Every OFDM rate above CCK is 100% corrupt because we're hearing distant APs at marginal SNR — the chip detects them, decodes them, fails the FCS, and now (with Surface CRC/ICV-corrupted RX frames + analysis tool #83's RCR change) surfaces them.

The FEC-design takeaway:

• The PoC's 6M OFDM stream link only works because TX and RX are co-located. At any real range the chip will surface FCS failures at high rate.
• The stream layer needs inter-frame parity (Reed-Solomon over N frames + K parity, Raptor, etc.) to recover from blocks of lost frames, not just per-frame FEC.
• For a P2P link's typical "moderate range" use case (e.g. OpenIPC long-range video), expect frame loss rates in the 30–70% range. FEC overhead has to be sized accordingly — at 50% loss you need K/N ≈ 0.5 to be reliable.

Follow-ups (for whoever picks up the FEC layer)

• Pick a parity scheme (Reed-Solomon is simplest, Raptor scales better) and parametrise N, K against captures from realistic ranges.
• Decide where parity rides: in-band on the same SA (current TX path) vs. on a dedicated SA / frame type. In-band keeps the link simple but eats stream airtime.
• Consider degrading rate gracefully (rateless codes) so the receiver can decode at whatever fraction of N+K frames it actually receives.

Builds on #83 (chip-level filter open, merged) and #84 (phy soft metrics, open).

🤖 Generated with Claude Code