Skip to content

Delete the ghost-paths fallback the record reader now makes loud - #394

Merged
typeless merged 1 commit into
mainfrom
fix/391-remove-ghost-paths-fallback
Aug 12, 2026
Merged

Delete the ghost-paths fallback the record reader now makes loud#394
typeless merged 1 commit into
mainfrom
fix/391-remove-ghost-paths-fallback

Conversation

@typeless

Copy link
Copy Markdown
Owner

merge_out_of_scope_commands scanned the new index for entries whose
non-empty path was missing from path_to_id, and its path lookup fell back
to that scan. Every producer of such an entry registers it: the two
serialization arms that skip registration are the ones that write an empty
path deliberately, keeping a slot so ids stay dense, and the loop's own
guard excluded exactly those. The two predicates cannot drift because
interning the empty string yields the empty handle the guard tests for --
which is now stated beside is_empty and pinned, since the guards rest on it
permanently while the comment that mentioned it goes with the map.

The map is not merely unexercised in practice but dominated in principle.
It can only matter on a run where a lookup consults path_to_id for an
unregistered non-empty path, and on exactly those runs deleting it is
better: resolve_file takes its miss branch instead, mints a second entry
for that path copying the old entry's type, and prior_paths has refused a
record naming one path twice since #382 -- usually through its per-list
repeats scan rather than the cross-class overlaps one, because the mint
copies the type of the entry it duplicates. So the deletion never reduces
detection and strictly increases it: what the fallback absorbed in silence
now reaches a reader.

The qualification worth stating: that loudness arrives one build late, and
as a record reported unreadable rather than as the producer that forgot to
register. It is a better failure than none, not a diagnosis.

Measured rather than argued: the function runs 234 times across the test
suite, walking 3126 file entries, and the map receives an entry zero times.
It is entered on every build that has a prior record -- the condition is
if (old_index), with no scope test -- so despite its name it is not
scoped-build-only machinery, and the incremental scenarios exercise it far
more broadly than the issue's suggested net implies.

The pin is discharged by proof by drift: deleting intern()'s empty-string
special case makes it fail. The pair review found no blockers.

Closes #391

merge_out_of_scope_commands scanned the new index for entries whose
non-empty path was missing from path_to_id, and its path lookup fell back
to that scan. Every producer of such an entry registers it: the two
serialization arms that skip registration are the ones that write an empty
path deliberately, keeping a slot so ids stay dense, and the loop's own
guard excluded exactly those. The two predicates cannot drift because
interning the empty string yields the empty handle the guard tests for --
which is now stated beside is_empty and pinned, since the guards rest on it
permanently while the comment that mentioned it goes with the map.

The map is not merely unexercised in practice but dominated in principle.
It can only matter on a run where a lookup consults path_to_id for an
unregistered non-empty path, and on exactly those runs deleting it is
better: resolve_file takes its miss branch instead, mints a second entry
for that path copying the old entry's type, and prior_paths has refused a
record naming one path twice since #382 -- usually through its per-list
repeats scan rather than the cross-class overlaps one, because the mint
copies the type of the entry it duplicates. So the deletion never reduces
detection and strictly increases it: what the fallback absorbed in silence
now reaches a reader.

The qualification worth stating: that loudness arrives one build late, and
as a record reported unreadable rather than as the producer that forgot to
register. It is a better failure than none, not a diagnosis.

Measured rather than argued: the function runs 234 times across the test
suite, walking 3126 file entries, and the map receives an entry zero times.
It is entered on every build that has a prior record -- the condition is
`if (old_index)`, with no scope test -- so despite its name it is not
scoped-build-only machinery, and the incremental scenarios exercise it far
more broadly than the issue's suggested net implies.

Closes #391

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Xkm9ce9gsgq41k4k6pvc3
@github-actions

Copy link
Copy Markdown

PR metrics

Performance (gcc example, Linux)

Workload Instructions CPU time Page faults D1 miss LL miss Wall Peak RSS
parse 1762 M 0.64 s 14.5 k 0.6% (-0.1pp) 0% 0.649 s 32.3 MB (+0.1MB)
dry-run 2255 M 0.72 s 15.5 k 0.8% 0% 0.727 s 36.1 MB

Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1).

Internal statistics (gcc example, up-to-date dry run)

Metric Value
Tupfiles parsed 24
Commands 3545
Commands scheduled 0
Files checked 5834
Files changed 0
Files in index 6102
Graph edges 381415
Index size (bytes) 7683630
Implicit deps 344126
Hash computations 197
Hashes skipped (stat cache) 5636
Stat calls 5885
Parse time (ms) 573.5
Total time (ms) 714.6
Runner CPU AMD EPYC 7763 64-Core Processor

Counters from putup -n --stat on the fully-built gcc example (up-to-date dry run): deterministic work measures — a jump in commands scheduled, hash computations, or stat calls is a real behavior change, not noise. Timings are the minimum over repeated runs, compared only against a baseline from the same CPU model; the counters are the regression signal.
Timing deltas suppressed: baseline ran on different hardware (AMD EPYC 9V74 80-Core Processor).

Binary size (Linux)

Binary .text .data .bss File
putup 562 KB (-0.1%) 2.3 KB 98.7 KB 666.4 KB (-0.6%)

Code churn (whole codebase, last 30d)

Files Lines written Still present Churned Churn rate
144 11185 9579 1606 14.4%

Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 5206 lines were deleted in the window in total, most of them older than it.

Where the churn is
File Lines written then discarded
src/cli/cmd_build.cpp 653
src/graph/scanners/clang_cl.cpp 110
src/cli/context.cpp 80
src/graph/builder.cpp 72
src/core/layout.cpp 69
src/core/new_delete.cpp 63
src/graph/scanners/dep_words.cpp 61
src/platform/file_io-posix.cpp 42
include/pup/index/format.hpp 35
src/cli/cmd_clean.cpp 35

Test coverage (lines)

Overall Median file Min file Max file
88% 96.45% 14.7% include/pup/parser/token.hpp 100.0% include/pup/core/arena.hpp

102 files · 16430/18666 lines covered

Deltas vs main@e65214da5.

Updated for 8f20d29

@typeless
typeless merged commit 42912d0 into main Aug 12, 2026
13 checks passed
@typeless
typeless deleted the fix/391-remove-ghost-paths-fallback branch August 12, 2026 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the dead ghost_paths map and its fallback in merge_out_of_scope_commands

1 participant