Skip to content

Far-field anomaly Phases 0+1 (#46): the pixel floor falls, and 'silent' is almost never absent - #99

Open
jonfroehlich wants to merge 4 commits into
mainfrom
analysis/farfield-forensics-46
Open

Far-field anomaly Phases 0+1 (#46): the pixel floor falls, and 'silent' is almost never absent#99
jonfroehlich wants to merge 4 commits into
mainfrom
analysis/farfield-forensics-46

Conversation

@jonfroehlich

@jonfroehlich jonfroehlich commented Jul 31, 2026

Copy link
Copy Markdown
Member

Phases 0 and 1 of the four-phase far-field study designed on #46 (2026-07-31): before the 34/36-visible verdicts are allowed to mean anything, check the sample they were made on and the pixel-starvation framing against the model's own behaviour (Phase 0), then read the model's heatmap inside every silent miss's match window to ask whether "silent" ever meant "saw nothing" (Phase 1). Phase 0 is committed data only — no GPU, no network, no imagery. Phase 1 ran the published checkpoint (projectsidewalk/rampnet-model, the same weights every committed cache came from) on a local RTX 3070, single-pass fp32 matching op_cache.

Findings

1. Survivorship (H1) is real, mild, and now quantified. The rated 37 are the ≥30-source-px slice of the 46 unwitnessed far-field silent misses, and the floor is not one floor — it is 7.5 model px on the 16384-px splits and 30 on morgantown, so the deck's composition follows stored width (27 GSV / 10 Mapillary; gainesville alone contributes 14). Net bias: AUC(rated px vs unrated far-silent px) = 0.600, rated median at the 62nd percentile. The 9 below-floor exclusions are the extreme tail (median 117 m, 3 above-horizon clamps). So the 94% visible rate generalizes to the far-silent core (~18–50 m) and says nothing about the deep tail.

2. The model's own hits refute a hard pixel floor. The decisive readout needs no reviewer:

band GT recall silent rated visible
18–25 m 395 0.777 25 12
25–40 m 226 0.549 32 13
40–150 m 72 0.292 22 9

For each visible miss, the model's recall over all far-field GT within ±20% of that miss's apparent size is median 0.57 (q1 0.31, q3 0.74). A hard pixel floor would put these near zero. Far-field failure is graded sensitivity, not a cliff — a silent far miss is the unlucky tail of a process that succeeds on most same-sized ramps.

3. Guards hold. Zero above-horizon clamps among the rated 37; the deck is majority GSV, where flat-ground distance is trustworthy (Spearman 0.95).

4. Phase 1: silent was peak bookkeeping, not blindness. Decomposing the in-window heatmap mass for all 128 pooled silent misses: absent (act < 0.01) 10 — the only true "saw nothing" cases; faint local (0.01–0.05) 39 — a real on-site response below the floor (mass at the ramp in 30/39, nothing else within ~3.8 R), the sub_threshold continuum; tail (≥ 0.05) 79 — a confident adjacent mode's tail (nearest cached floor peak ~1.4 R away at median score 0.685; argmax in the window's outer quarter in 75/79), the σ/representation family whichever of neighbour-TP / FP / self-just-outside-radius it is. The anomaly population itself — the 34 far-field rated-visible — splits 3 / 12 / 19: the model responds at or next to ~91% of the far ramps a human called resolvable. And of the 45 near-field silent misses under the 0.013 sourcing estimate, only 6 are heatmap-absent — the rest are the calibration and σ families §0b already prices, so 0.013 stays unrevised in either direction until Phases 2–3.

The takeaway (now stated explicitly at the end of §0c): distance is a stressor on failure families the taxonomy already prices, not a new category of failure — 62% σ/representation, 30% sub-threshold attenuation, 8% genuine absence. The implied lever is decoder- and representation-side (target σ, peak spacing, threshold calibration, Phase 2's scale question for the residual), not far-field training vocabulary; multi-view remains the one remedy that sidesteps all three mechanisms at once.

What this changes in the docs

  • New §0c in docs/curb_ramp_data_sourcing.md with the tables above and the boundaries of the claim: §0a's far/near measurement stands, its "fixable by" binary falls (the 18 m boundary marks difficulty, not reachability); the §0b sourcing bracket's exclusion of all 83 far-field silent misses is no longer safe, but 0.013 is deliberately not revised until Phases 2–3 run (scale counterfactual, decoy control); multi-view's remedy logic is untouched, though the "MV ceiling" column shares the binary assumption.
  • §0a gets an in-place qualifier pointing to §0c.
  • Two stale §0b claims fixed: the reviewer pass is done (one rater), and the point estimate exists (0.013, docs/replication.md).
  • Phase 3 design consequence recorded: the decoy deck should stratify by distance band and oversample 40–150 m, where visible went 9-of-9 down to 10.5 model px — the spot where pointed-verification bias would show most.

Mechanics

Phase 0: scripts/analysis/farfield_forensics.py + 21 tests (tests/test_farfield_forensics.py), result JSON at analysis_out/farfield_forensics.json. Phase 1: scripts/analysis/silent_activation.py + 14 tests (tests/test_silent_activation.py), result JSON at analysis_out/silent_activation.json (both whitelisted in .gitignore like the other result files). The integration tests pin the population arithmetic (83 = 37 rated + 9 below-floor + 37 witnessed) and cross-check the gallery manifest's widths against the imagery manifests. One instrument catch for the record: the first azimuth-null implementation could read a site's own faint bump back as "chance" — a unit test caught it before any numbers existed, and the null now self-excludes a 2 R zone around the site. Full suite: 533 passed.

🤖 Generated with Claude Code (claude-fable-5)

…d the pixel floor is refuted by the model's own hits

The reviewer pass rated 34/36 rateable far-field silent-miss crops visible
on the model-resolution panel, contradicting E1's pixel-starvation framing.
Before letting the verdicts mean anything, this checks the sample and the
framing from committed data alone (no GPU, no network, no imagery).

- scripts/analysis/farfield_forensics.py (+21 tests): populations from the
  low-floor caches, the witness list, the gallery manifest/verdicts, and the
  imagery manifests' width fields.
- Survivorship (H1) is real but mild: AUC(rated vs unrated far-silent px)
  0.600, rated median at the 62nd percentile. The 9 below-floor exclusions
  are the extreme tail (median 117 m, 3 above-horizon clamps), so the 94%
  generalizes to the 18-50 m far-silent core, not the deep tail.
- The decisive readout needs no reviewer: matched-size detection rate is
  median 0.57 (q1 0.31, q3 0.74) - the model detects most OTHER far-field
  ramps at the same apparent size as its silent misses. Band recall declines
  0.777 / 0.549 / 0.292 (18-25 / 25-40 / 40-150 m), never cliffs to zero.
  Far-field failure is graded sensitivity, not a floor.
- docs/curb_ramp_data_sourcing.md: new section 0c with the tables and what
  it does and does not change (0.013 deliberately NOT revised - Phases 1-3
  first); section 0a's "fixable by" binary qualified in place; two stale
  0b claims fixed (the reviewer pass IS done; the point estimate exists).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
jonfroehlich and others added 2 commits July 31, 2026 12:44
…f-excluding azimuth null

silent is a statement about PEAKS (nothing >= 0.05 via peak_local_max), not
about the heatmap - 0.04 and 0.0001 are different failures. For all 128
pooled silent misses this reads the max heatmap value inside the match
radius (the scaled matcher space IS the 512x1024 heatmap grid, asserted)
and compares it to the pano's own elevation-band null.

The null rejects draws whose window overlaps the site's (wrapped column
distance < 2R): a draw reading the site's own faint bump back would
contaminate exactly the sparse-heatmap case the analysis exists to detect
- caught by the unit test, fixed before any numbers were produced.

Model: the published projectsidewalk/rampnet-model weights via
threshold_sweep.load_model() - the same checkpoint every committed cache
came from; single-pass fp32, matching analysis_out/op_cache.

farfield_forensics.load_rated grows a field=None mode so Phase 1 can join
the near-field verdicts too. 10 new tests; suite 529.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
…t; 62% are an adjacent confident mode's tail

Ran silent_activation.py over all 128 pooled silent misses on the local
RTX 3070 (published checkpoint, single-pass fp32, matching op_cache).

- absent (act < 0.01): 10 of 128 (8%). "Silent = the model saw nothing"
  is wrong for 92% of the bucket - silent was peak bookkeeping.
- tail (act >= 0.05): 79 (62%) - argmax in the window's outer quarter in
  75/79, nearest cached floor peak ~1.4R away at median score 0.685: a
  CONFIDENT adjacent detection. The sigma/representation family again,
  not vocabulary. (A floor-level annulus peak leaves a miss "silent" -
  the localization bucket only inspects kept peaks.)
- faint local (0.01-0.05): 39 (30%) - mass on-site in 30/39, nothing
  else within ~3.8R: the sub_threshold continuum below the floor.
- The anomaly population (far-field rated-visible, 34): 3 absent /
  12 faint-local / 19 tail - the model responds at or next to ~91% of
  the far ramps a human called resolvable.
- Near-field consequence for 0.013: of the 45 near silent misses, only
  6 are heatmap-absent. Not revised either direction until Phases 2-3.

Instrument additions since the first run: site_profile (argmax offset +
value at the site) and nearest_peak (matcher-unit distance to cached
floor peaks, wrapped) separate a site response from a neighbour's tail;
4 new tests (suite 533). Doc: Phase 1 section added to 0c.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
@jonfroehlich jonfroehlich changed the title Far-field anomaly Phase 0 (#46): mild survivorship; the pixel floor is refuted by the model's own hits Far-field anomaly Phases 0+1 (#46): the pixel floor falls, and 'silent' is almost never absent Jul 31, 2026
@jonfroehlich

Copy link
Copy Markdown
Member Author

Scope grew from Phase 0 to Phases 0+1 in this branch — Phase 1 (activation forensics over all 128 silent misses, local GPU) landed as 59d5a9e + 64763d0. Headline: only 10/128 silent misses are heatmap-absent; 62% are a confident adjacent mode's tail (~1.4R, median score 0.685); the far-field rated-visible population splits 3 absent / 12 faint-local / 19 tail. Full numbers in the #46 comment and docs §0c. Suite 533 passed.

🤖 Generated with Claude Code (claude-fable-5)

…eady-priced families, not a new failure category

Phases 0-1 established every piece of this but the section ended on
caveats; the synthesis itself - the sentence a reader should leave
with - was never stated. Now it is: far-field recall degrades threefold
but decomposes into 62% sigma/representation, 30% sub_threshold, 8%
absent, so the lever is decoder-side (target sigma, peak spacing,
calibration, Phase 2 scale), not far-field training vocabulary, and
multi-view sidesteps all three mechanisms at once.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant