Skip to content

Announce record damage that is not a layout failure - #396

Merged
typeless merged 2 commits into
mainfrom
fix/383-announce-record-damage
Aug 12, 2026
Merged

Announce record damage that is not a layout failure#396
typeless merged 2 commits into
mainfrom
fix/383-announce-record-damage

Conversation

@typeless

Copy link
Copy Markdown
Owner

Summary

Two commits, split because two files carry both changes:

  1. Announce record damage that is not a layout failure — a record too
    small to hold a header, and one whose magic says it is not a putup index
    at all, are damage by any reading, but both carried InvalidFormat, the
    same code as "this record's version is one I do not read" (the expected,
    benign outcome on every upgrade). Each open-time rejection now gets a code
    that says which happened: IndexDamaged for damage by any reading,
    IndexVersionMismatch for a version outside the readable window. The
    announcement is inverted so silence is the enumerated case
    (IndexVersionMismatch alone) instead of the announced set, so a
    rejection the reader gains later is loud by default. InvalidFormat and
    IndexTruncated are deleted (no producers, no consumers, no tests).
    REQ-READ-ANNOUNCE-DAMAGE is widened by complement rather than
    enumeration.

  2. Delete is_valid_index, a third encoding of the readable window — no
    production caller, only its own test; it re-answered the same question as
    open_index with two different definitions (exact version match instead
    of the [INDEX_LAYOUT_FLOOR, INDEX_VERSION] window, and a size floor of
    header alone without the footer).

RED was quoted (unit + E2E failures showing the wrong error codes and a
silent rebuild) before any production code changed. The whole suite passes
in one process: 427 ~[e2e] cases (167220 assertions) + 394 [e2e] cases
(2936 assertions). format-check, spec-check, and iwyu are clean;
tidy shows only pre-existing warnings on lines this change does not touch.

Pair-partner review (Sonnet, cross-model per charter) found and fixed one
real issue: two added comments were two lines each, over CLAUDE.md's
one-line why-comment cap — both compressed, with the dropped context moved
into this commit's body. Everything else it checked (check ordering, the
switch's reachable-code enumeration, the InvalidFormat/IndexTruncated/
is_valid_index grep, three trap claims, spec-check, the DESIGN.md row,
and the commit split staging) came back sound.

make check is red on this branch, but it is also red on stock main at
42912d05e on this machine (verified against a git archive control tree):
the 32-way sharded E2E runner fails one shard per run, alternating between
two tests that each pass standalone and pass when the whole suite runs in
one process. Pre-existing, unrelated to this change.

Closes #383

Test plan

  • RED quoted for all three unit cases and both E2E scenarios before any
    production code changed
  • GREEN: all seven pinned tests pass individually
  • Whole suite passes in one process (~[e2e] + [e2e])
  • make format-check, make spec-check, make iwyu clean
  • make tidy — only pre-existing warnings, none on touched lines
  • Pair-partner (Sonnet) review, one finding fixed
  • make check red-on-main confirmed via stock control tree — pre-existing sharded-runner flake, not this change

A record too small to hold a header, and one whose magic says it is not a putup
index at all, are damage by any reading -- but both carried InvalidFormat, the
same code as "this record's version is one I do not read", which is the expected
benign outcome on every upgrade. load_old_index announced only checksum and
layout failures, so a build over a truncated or overwritten record printed
nothing and read as a first build: the failure IndexDamaged was introduced to
end (#381), still live at three of its five sites.

Give each open-time rejection a code that says which of the two happened: damage
by any reading takes IndexDamaged, and a version outside the readable window
takes IndexVersionMismatch, a code that was declared and never produced. The
too-small and magic checks stay ahead of the version check because no version of
the format is shorter than header plus footer and none lacks the magic, so
neither can be a record of a version this binary merely does not read.

Then invert the announcement: silence is now the enumerated case
(IndexVersionMismatch alone) rather than the announced set, so a rejection the
reader gains later is loud by default instead of silent by default -- the
property whose absence is why #381 fixed one row and left three. The remaining
arm covers IoError, where the record may be intact and the path or its
permissions is the problem, and deliberately does not call it damage.

InvalidFormat and IndexTruncated have no producers after this and never had a
consumer or a test; ErrorCode is in-process only, so deleting them costs
nothing. InvalidState moves to the general block, where it belongs.

REQ-READ-ANNOUNCE-DAMAGE is widened by complement rather than by enumeration:
it names the benign exception and lets damage be everything else, so it stays
true as damage modes are added.

Closes #383.
No production caller; its only use was its own test. It answered a different
question from open_index in two ways -- an exact version match rather than the
[INDEX_LAYOUT_FLOOR, INDEX_VERSION] window, and a size floor of the header alone
without the footer -- so keeping it meant maintaining a third definition of "a
record this putup can read" that no build consults.
@github-actions

Copy link
Copy Markdown

PR metrics

Performance (gcc example, Linux)

Workload Instructions CPU time Page faults D1 miss LL miss Wall Peak RSS
parse 1762 M 0.65 s 14.5 k 0.6% (-0.1pp) 0% 0.673 s 32 MB (-0.3MB)
dry-run 2204 M (-2.3%) 0.7 s 15.5 k 0.8% 0% 0.743 s 36.1 MB (+0.3MB)

Deterministic signals: instructions (cachegrind-simulated instruction reads — exact across runs, no PMU needed), page faults, peak RSS, and the cachegrind D1/LL miss rates. CPU time is user+sys from time(1).

Internal statistics (gcc example, up-to-date dry run)

Metric Value
Tupfiles parsed 24
Commands 3545
Commands scheduled 0
Files checked 5834
Files changed 0
Files in index 6102
Graph edges 381415
Index size (bytes) 7683630
Implicit deps 344126
Hash computations 0 (-100.0%)
Hashes skipped (stat cache) 5833 (+3.5%)
Stat calls 5885
Parse time (ms) 578.2
Total time (ms) 720
Runner CPU AMD EPYC 9V74 80-Core Processor

Counters from putup -n --stat on the fully-built gcc example (up-to-date dry run): deterministic work measures — a jump in commands scheduled, hash computations, or stat calls is a real behavior change, not noise. Timings are the minimum over repeated runs, compared only against a baseline from the same CPU model; the counters are the regression signal.
Timing deltas suppressed: baseline ran on different hardware (AMD EPYC 7763 64-Core Processor).

Binary size (Linux)

Binary .text .data .bss File
putup 562.1 KB 2.3 KB 98.7 KB 666.4 KB

Code churn (whole codebase, last 30d)

Files Lines written Still present Churned Churn rate
144 11202 9594 1608 14.4%

Of the lines written across the codebase in the last 30 days, how many are already gone — work that was written and then discarded or rewritten inside the same window. This is the state of the tree including this PR, not a measure of the PR itself. Only code we write is counted: tests, examples, vendored and generated files, CI plumbing and prose are excluded. 5233 lines were deleted in the window in total, most of them older than it.

Where the churn is
File Lines written then discarded
src/cli/cmd_build.cpp 653
src/graph/scanners/clang_cl.cpp 110
src/cli/context.cpp 82
src/graph/builder.cpp 72
src/core/layout.cpp 69
src/core/new_delete.cpp 63
src/graph/scanners/dep_words.cpp 61
src/platform/file_io-posix.cpp 42
include/pup/index/format.hpp 35
src/cli/cmd_clean.cpp 35

Test coverage (lines)

Overall Median file Min file Max file
88% 96.45% 14.7% include/pup/parser/token.hpp 100.0% include/pup/core/arena.hpp

102 files · 16431/18666 lines covered

Deltas vs main@42912d05e.

Updated for 8c578d2

@typeless
typeless merged commit bb2c787 into main Aug 12, 2026
13 checks passed
@typeless
typeless deleted the fix/383-announce-record-damage branch August 12, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Damage that is not a layout failure is still silent: too-small and bad-magic records share a code with the benign version case

1 participant