docs(0111): the live window is not starved — 0215 already fixed it - #243
Conversation
…ping up This file asserted the live window "keeps up fine (2.81K unpriced)" and used that to frame Phase 1 as purely a scan-cost fix. The 2.81K is XLM-quoted only, taken from the by-year table and read as if it shared scope with the era table's all-quotes figure of 10.44M of 17.05M (61.2%). Re-measured on price_ohlcv_1m FINAL over 48h in 4h buckets, partition-bounded: the unpriced share is flat at 50-58% with no age-decay. A bucket 44h old carries the same share as one 4h old, so enrichment never returns to those rows. Roughly 45% of the 26,900 rolling-4h total is the permanent exotic-quote floor and is not this task's to fix, but ~2,370 per 4h are USDC(3)- and XLM(4)-quoted and should be priceable - about 14,200/day. Also records why no alarm caught it: pass_metrics publishes only after pass.run() returns, the Lambda is killed mid-pass, and EnrichmentBacklogAlarm is treatMissingData NOT_BREACHING - so it reads OK on no data, the same pattern as the seven alarms in 0204. The chosen design and phase ordering are unchanged; the payoff is larger. AC 6 is annotated because its "floor" was the wrong number - judging against 2.81K would have read a working fix as a failure.
Runs the leg-level discriminator the 2026-08-21 entry called for and it refutes that entry's own conclusion. Legs 3 (USDC) and 4 (XLM) decay to zero within ~4h: over 40h excluding the freshest 8h, XLM is 12 unpriced of 138,722 and USDC is 0 of 90,672. Every other quote leg is unpriced at exactly 100% — the signature of a structural gap, not a backlog. Both measurements are correct. ~42 of the 08-21 window's 48h predate the 0215 Caddy fix, when the pass died at 30.0s and the live window really was starved, and the aggregates reconcile because the floor is half the candidates and masks a priceable half that goes to zero. Phase 1's payoff reverts to the original case; every reason it is still needed is untouched. AC 6 is rewritten leg-scoped, because EnrichmentRowsRemainingRecent is ~100% exotic floor and would have passed a starved pass.
⛔ Correction — this PR's original conclusion is refuted by its own discriminatorThe first version of this PR closed with "whether legs 3 and 4 decay with age is the one query not run — flat ⇒ starved, decaying ⇒ ordinary lag." That query has now been run. They decay. Measured 2026-08-24, 48h in 4h buckets, partition-bounded, using enrichment's real
There is no middle ground — 100.0% on every exotic leg, not 99% or 95%. That is a structural gap (no oracle, no pivot reference), not a backlog. It matches 0056's exotic-quote floor: ~88K rows/day against ~138K/day priceable. Both measurements were correct — different regimesThe original 48h window ran 08-19 20:00 → 08-21 20:45. The 0215 Caddy fix landed before 14:00 on 08-21, so ~42 of those 48 hours are pre-fix, when the pass died at 30.0s on every invocation and the live window genuinely was starved. The aggregates reconcile exactly: 08-22 20:00 gives 17,038 of 33,268 = 51.2%, squarely inside the original 50–58% band. The aggregate looks flat because the floor is half the candidates at ~99.7% and never moves, masking a priceable half that goes to zero. The original analysis identified that masking effect and then drew the opposite conclusion from it without testing.
What changed in the file
Every reason Phase 1 is still needed is untouched: the alarm at 300,000/300,338 ms, 5.4 of 40 budgeted batches, the ~322-day drain, and 0218 blocked behind it. |
Summary
pass_metricspublishes only afterpass.run()returns, the Lambda is killed mid-pass, andEnrichmentBacklogAlarmistreatMissingData: NOT_BREACHING— it reads OK on no data, the 0204 patternDocs-only. No code, no schema, no infra. The chosen design and phase ordering are unchanged — Phase 1's payoff is larger than recorded, because it un-starves the live window rather than merely speeding it up.
One thing deliberately left open and marked ⏳ in the file: whether legs 3 and 4 decay with age once the floor is stripped out. That query has not been run, so the starvation reading is stated as the likelier of two readings, not as established.