Far-field anomaly Phases 0+1 (#46): the pixel floor falls, and 'silent' is almost never absent - #99
Open
jonfroehlich wants to merge 4 commits into
Open
Far-field anomaly Phases 0+1 (#46): the pixel floor falls, and 'silent' is almost never absent#99jonfroehlich wants to merge 4 commits into
jonfroehlich wants to merge 4 commits into
Conversation
…d the pixel floor is refuted by the model's own hits The reviewer pass rated 34/36 rateable far-field silent-miss crops visible on the model-resolution panel, contradicting E1's pixel-starvation framing. Before letting the verdicts mean anything, this checks the sample and the framing from committed data alone (no GPU, no network, no imagery). - scripts/analysis/farfield_forensics.py (+21 tests): populations from the low-floor caches, the witness list, the gallery manifest/verdicts, and the imagery manifests' width fields. - Survivorship (H1) is real but mild: AUC(rated vs unrated far-silent px) 0.600, rated median at the 62nd percentile. The 9 below-floor exclusions are the extreme tail (median 117 m, 3 above-horizon clamps), so the 94% generalizes to the 18-50 m far-silent core, not the deep tail. - The decisive readout needs no reviewer: matched-size detection rate is median 0.57 (q1 0.31, q3 0.74) - the model detects most OTHER far-field ramps at the same apparent size as its silent misses. Band recall declines 0.777 / 0.549 / 0.292 (18-25 / 25-40 / 40-150 m), never cliffs to zero. Far-field failure is graded sensitivity, not a floor. - docs/curb_ramp_data_sourcing.md: new section 0c with the tables and what it does and does not change (0.013 deliberately NOT revised - Phases 1-3 first); section 0a's "fixable by" binary qualified in place; two stale 0b claims fixed (the reviewer pass IS done; the point estimate exists). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
…f-excluding azimuth null silent is a statement about PEAKS (nothing >= 0.05 via peak_local_max), not about the heatmap - 0.04 and 0.0001 are different failures. For all 128 pooled silent misses this reads the max heatmap value inside the match radius (the scaled matcher space IS the 512x1024 heatmap grid, asserted) and compares it to the pano's own elevation-band null. The null rejects draws whose window overlaps the site's (wrapped column distance < 2R): a draw reading the site's own faint bump back would contaminate exactly the sparse-heatmap case the analysis exists to detect - caught by the unit test, fixed before any numbers were produced. Model: the published projectsidewalk/rampnet-model weights via threshold_sweep.load_model() - the same checkpoint every committed cache came from; single-pass fp32, matching analysis_out/op_cache. farfield_forensics.load_rated grows a field=None mode so Phase 1 can join the near-field verdicts too. 10 new tests; suite 529. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
…t; 62% are an adjacent confident mode's tail Ran silent_activation.py over all 128 pooled silent misses on the local RTX 3070 (published checkpoint, single-pass fp32, matching op_cache). - absent (act < 0.01): 10 of 128 (8%). "Silent = the model saw nothing" is wrong for 92% of the bucket - silent was peak bookkeeping. - tail (act >= 0.05): 79 (62%) - argmax in the window's outer quarter in 75/79, nearest cached floor peak ~1.4R away at median score 0.685: a CONFIDENT adjacent detection. The sigma/representation family again, not vocabulary. (A floor-level annulus peak leaves a miss "silent" - the localization bucket only inspects kept peaks.) - faint local (0.01-0.05): 39 (30%) - mass on-site in 30/39, nothing else within ~3.8R: the sub_threshold continuum below the floor. - The anomaly population (far-field rated-visible, 34): 3 absent / 12 faint-local / 19 tail - the model responds at or next to ~91% of the far ramps a human called resolvable. - Near-field consequence for 0.013: of the 45 near silent misses, only 6 are heatmap-absent. Not revised either direction until Phases 2-3. Instrument additions since the first run: site_profile (argmax offset + value at the site) and nearest_peak (matcher-unit distance to cached floor peaks, wrapped) separate a site response from a neighbour's tail; 4 new tests (suite 533). Doc: Phase 1 section added to 0c. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
Member
Author
|
Scope grew from Phase 0 to Phases 0+1 in this branch — Phase 1 (activation forensics over all 128 silent misses, local GPU) landed as 59d5a9e + 64763d0. Headline: only 10/128 silent misses are heatmap-absent; 62% are a confident adjacent mode's tail (~1.4R, median score 0.685); the far-field rated-visible population splits 3 absent / 12 faint-local / 19 tail. Full numbers in the #46 comment and docs §0c. Suite 533 passed. 🤖 Generated with Claude Code (claude-fable-5) |
…eady-priced families, not a new failure category Phases 0-1 established every piece of this but the section ended on caveats; the synthesis itself - the sentence a reader should leave with - was never stated. Now it is: far-field recall degrades threefold but decomposes into 62% sigma/representation, 30% sub_threshold, 8% absent, so the lever is decoder-side (target sigma, peak spacing, calibration, Phase 2 scale), not far-field training vocabulary, and multi-view sidesteps all three mechanisms at once. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017KCSfZm3MjoJ62LVtkvNeb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phases 0 and 1 of the four-phase far-field study designed on #46 (2026-07-31): before the 34/36-
visibleverdicts are allowed to mean anything, check the sample they were made on and the pixel-starvation framing against the model's own behaviour (Phase 0), then read the model's heatmap inside every silent miss's match window to ask whether "silent" ever meant "saw nothing" (Phase 1). Phase 0 is committed data only — no GPU, no network, no imagery. Phase 1 ran the published checkpoint (projectsidewalk/rampnet-model, the same weights every committed cache came from) on a local RTX 3070, single-pass fp32 matchingop_cache.Findings
1. Survivorship (H1) is real, mild, and now quantified. The rated 37 are the ≥30-source-px slice of the 46 unwitnessed far-field silent misses, and the floor is not one floor — it is 7.5 model px on the 16384-px splits and 30 on morgantown, so the deck's composition follows stored width (27 GSV / 10 Mapillary; gainesville alone contributes 14). Net bias: AUC(rated px vs unrated far-silent px) = 0.600, rated median at the 62nd percentile. The 9 below-floor exclusions are the extreme tail (median 117 m, 3 above-horizon clamps). So the 94% visible rate generalizes to the far-silent core (~18–50 m) and says nothing about the deep tail.
2. The model's own hits refute a hard pixel floor. The decisive readout needs no reviewer:
visibleFor each
visiblemiss, the model's recall over all far-field GT within ±20% of that miss's apparent size is median 0.57 (q1 0.31, q3 0.74). A hard pixel floor would put these near zero. Far-field failure is graded sensitivity, not a cliff — a silent far miss is the unlucky tail of a process that succeeds on most same-sized ramps.3. Guards hold. Zero above-horizon clamps among the rated 37; the deck is majority GSV, where flat-ground distance is trustworthy (Spearman 0.95).
4. Phase 1:
silentwas peak bookkeeping, not blindness. Decomposing the in-window heatmap mass for all 128 pooled silent misses: absent (act < 0.01) 10 — the only true "saw nothing" cases; faint local (0.01–0.05) 39 — a real on-site response below the floor (mass at the ramp in 30/39, nothing else within ~3.8 R), thesub_thresholdcontinuum; tail (≥ 0.05) 79 — a confident adjacent mode's tail (nearest cached floor peak ~1.4 R away at median score 0.685; argmax in the window's outer quarter in 75/79), the σ/representation family whichever of neighbour-TP / FP / self-just-outside-radius it is. The anomaly population itself — the 34 far-field rated-visible— splits 3 / 12 / 19: the model responds at or next to ~91% of the far ramps a human called resolvable. And of the 45 near-field silent misses under the 0.013 sourcing estimate, only 6 are heatmap-absent — the rest are the calibration and σ families §0b already prices, so 0.013 stays unrevised in either direction until Phases 2–3.The takeaway (now stated explicitly at the end of §0c): distance is a stressor on failure families the taxonomy already prices, not a new category of failure — 62% σ/representation, 30% sub-threshold attenuation, 8% genuine absence. The implied lever is decoder- and representation-side (target σ, peak spacing, threshold calibration, Phase 2's scale question for the residual), not far-field training vocabulary; multi-view remains the one remedy that sidesteps all three mechanisms at once.
What this changes in the docs
docs/curb_ramp_data_sourcing.mdwith the tables above and the boundaries of the claim: §0a's far/near measurement stands, its "fixable by" binary falls (the 18 m boundary marks difficulty, not reachability); the §0b sourcing bracket's exclusion of all 83 far-field silent misses is no longer safe, but 0.013 is deliberately not revised until Phases 2–3 run (scale counterfactual, decoy control); multi-view's remedy logic is untouched, though the "MV ceiling" column shares the binary assumption.docs/replication.md).visiblewent 9-of-9 down to 10.5 model px — the spot where pointed-verification bias would show most.Mechanics
Phase 0:
scripts/analysis/farfield_forensics.py+ 21 tests (tests/test_farfield_forensics.py), result JSON atanalysis_out/farfield_forensics.json. Phase 1:scripts/analysis/silent_activation.py+ 14 tests (tests/test_silent_activation.py), result JSON atanalysis_out/silent_activation.json(both whitelisted in.gitignorelike the other result files). The integration tests pin the population arithmetic (83 = 37 rated + 9 below-floor + 37 witnessed) and cross-check the gallery manifest's widths against the imagery manifests. One instrument catch for the record: the first azimuth-null implementation could read a site's own faint bump back as "chance" — a unit test caught it before any numbers existed, and the null now self-excludes a 2 R zone around the site. Full suite: 533 passed.🤖 Generated with Claude Code (claude-fable-5)