Skip to content

Delete the batch 2 redundant tests (test audit TA5, #1734) - #1735

Merged
tamashi095 merged 8 commits into
mainfrom
codex/batch-test-policy-4
Oct 20, 2026
Merged

tamashi095 merged 8 commits into
mainfrom
codex/batch-test-policy-4

Conversation

@tamashi095

@tamashi095 tamashi095 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Test audit phase B, PR 5 (TA5). This is the last PR of the audit. It deletes the tests of the batch 2 sample (200 random KEEP tests) that are redundant both by measurement and on reading.

Closes #1734. Adds the spec for #1733. Refers to #1728 and does not close it.

Summary

  • 97 tests deleted. Batch 2 measured 162 of the 200 sampled tests as redundant. Each of the 162 was then read against the AGENTS.md test-value question. 97 are deleted, and each deleted test's answer names the remaining test that defends its claim. 65 are kept, each with a one-line reason:
    • the acked-batch question;
    • the only red test of a MUTATIONS.md row;
    • known AArch64 defects;
    • ABI or wire layouts;
    • allocation, count or ceiling claims;
    • exact edges;
    • randomized differentials;
    • reproducers.
  • 31 helpers deleted (465 lines). Each went only after clippy reported it unused, and no #[allow] was added. 2 test files are deleted.
  • MUTATIONS.md: rows that named a deleted test now name the remaining catcher. The "TA4 deletions" ledgers that TA4 appended are removed. The list lives in git history and in the comments on Delete the tests that mutation measurement found redundant (test audit TA4) #1731.
  • Give the workspace debug test job a real time margin #1728: the test-debug-a timeout goes from 15 to 22 minutes, which is 1.5 times the slowest recent green run (14:41). It still took 14m29s after TA4.
  • 12 survivors. 3 are equivalent, 1 unreachable and 2 dead code. 6 are real gaps, which go to Close the batch 2 mutation-survivor gaps #1733 with no new tests in this PR. The most important gap: bench-support record_peak_window could be a no-op and the Shrink the peak heap of a live-only commit #1615 heap ceilings would still pass.

The test-by-test list with each answer and its catchers is in the comments on #1734.

Finding for any later cut

About 12 random mutants per test overstate redundancy. 81% of the sample measured redundant, but only 49% was safe to delete after reading. B2's pool-wide projection does not stand. The spec's last section states what a reopening brief must check first.

Gates (author, final tree)

  • All 31 steps of the qualification lint job: PASS.
  • cargo test --locked --all-features on the touched packages: PASS (2,276 tests after the review fold; parametric-eq 122 after the last restore).
  • A fresh verifier ran two rounds. Round 1 restored 5 tests and kept 28 more. Round 2 restored 1 reproducer. Final verdict: READY.
  • Root removed the TA4 ledgers and numbered the specs. check-conformance-boundaries.sh and the spec lint scripts PASS.

-5.2k / +0.3k lines over 88 files.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

tamashi095 and others added 8 commits October 20, 2026 10:17
Batch 2 (B2) measured 162 of 200 sampled KEEP tests as mutation-redundant at
81a3074. Each was read against the AGENTS.md test-value question; 31 are
kept by reading (named edge cases, recorded sole catchers, ABI and wire rules,
allocation, count and ceiling claims, a randomized differential), 131 are
deleted. Helpers deleted only after clippy reported them unused; two test
files left with no test are deleted. MUTATIONS.md rows and open specs that
name a deleted test say so in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The slowest recent green run of the job took 14:41 against a 15-minute
timeout. 22 minutes is 1.5 times that run. Only this job's timeout moves;
#1728 stays open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The TA5 spec records the owner approval, the batch 2 method and result
(162/200, 81%, Wilson 75-86%), the blind spot and how reading handled it,
the per-package counts, the 31 tests kept by reading with their reasons,
the deleted helpers and the #1728 timeout change. The per-test list goes to
issue comments. The successor spec records two leads TA5 did not fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, add the gap spec

Remove the "TA5 deletions" appendices from the MUTATIONS.md files (a list of
deleted tests is a ledger row). Each row that named a deleted test now names a
remaining same-crate catcher or says "no catcher recorded". Notes in specs and
comments stay only where the name would be a stale reference. Add T5S's
survivor-gap spec and fill the TA5 spec's "Survivor gaps" section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The verifier found five deleted tests that were the only defenders of their
claims. Re-reading the 73 DELETE answers that named siblings only by file kept
28 more where no remaining test, by full name, asserts the same claim. 64 of the
162 are now kept and 98 deleted; every DELETE answer names the remaining test.
MUTATIONS rows name a catcher that can see the row's mutant or say none is
recorded; the 1340 spec's gate 3 now asks for a new test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
the_low_shelf_fixed_point_reaches_exact_rest reproduces the 10 Hz slope-0.1
shelf that stuck at 6.01e-20 and holds a ceiling (exact rest within
REST_BOUND_SAMPLES); the named defender does not reproduce the shelf. Restored
with its helpers; MUTATIONS 1328-M1 is as at ad5c7ae. Counts: 97 deleted,
65 kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Root ruling: a list of deleted tests in a tracked file is a ledger row.
Git history and the comments on #1731 keep the list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The successor-leads draft held no work, so its lesson moves into the
TA5 spec as its last section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tamashi095
tamashi095 merged commit 64fe78c into main Oct 20, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delete the redundant tests of the batch 2 sample (test audit TA5)

1 participant