Skip to content

test: make exact validation sensitive to rare branches - #620

Draft
bachase-agent wants to merge 2 commits into
mainfrom
codex/rare-branch-validation
Draft

bachase-agent wants to merge 2 commits into
mainfrom
codex/rare-branch-validation

Conversation

@bachase-agent

@bachase-agent bachase-agent commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Exact validation could accept a missing or misweighted outcome below its 1e-9 absolute tolerance, skip conditional expectation checks on that outcome, and discard joint weights below 1e-15 during aggregation. The new negative controls reproduced all four failures before the helper changes.

Compare outcome and discarded probabilities relatively, check conditional probes on every positive cluster, retain positive representable joint weights, and reject invalid distributions or underflowed probability products. The dense oracle now reports an explicit omitted-mass bound, uses a configurable conditional Born cutoff defaulting to 1e-18, and retains positive Pauli/readout noise branches. Comparisons report inconclusive references when the combined omission bound exceeds the fixture's budget. Production numerical policies are unchanged.

The compact analytic corpus covers rare Born outcomes and their joint probabilities down to about 6e-32, categorical/correlated/readout noise, postselection, aggregation, invalid inputs, and corrupt conditional expectations. Python enumeration gains configurable pruning and signal-relative precision checks, with analytic population/loss/transition fixtures and explicit checks against existing statistical bands. The contributor guide explains the precision contracts and their limits.

Closes #612.

Test plan

  • Full C++ CTest suites passed: 1,200 Debug cases and 1,201 Release cases, including expensive. Local builds: GCC 13.3 Debug and Clang 18.1.3 Release with fast-math and assertions enabled; generic CPU baseline.
  • Affected Python suites passed: 231 tests across test_noncomp_fixed.py, test_noncomp_oracle.py, test_noncomp_trajectory_tvd.py, and test_noncomp_enumerator_precision.py, including the expensive trajectory cases.
  • Reproduced the three nonfinite-input CI failures locally with the pinned Clang 22.1.8 compiler, ThinLTO, fast-math, x86-64-v2 baseline, OpenMP, and assertions. After using the existing runtime-opaque NaN/infinity helpers and removing an unnecessary infinity constant from omitted-mass rounding, all 1,202 C++ cases pass in that configuration. The affected Debug selection also passes all 31 cases.
  • Pre-commit passed with the repository-required command.
  • Hosted CI passed on e1f0edcf, including release-cpp-smoke, every cross-platform job, and ci-gate: successful run.
  • Per-job cost review below remains a merge prerequisite.

The reference cutoff calibration covers both FP64 and FP32 policies without claiming GPU execution. Omitted-mass bounds account for deliberate pruning, not arbitrary floating-point roundoff; the rare fixtures use independent, well-conditioned analytic weights. Larger/nonrepresentable cases fail instead of silently certifying lost support.

Runtime evidence and merge prerequisite

Local measurements on an 8-vCPU AMD EPYC 9554P Linux VM, three wall-clock repetitions per command:

Selection Baseline seconds Changed seconds Median increment
Non-expensive pass-equivalence C++ cases, plus the new corpus on the changed side; GCC 13.3 Debug, generic, OpenMP 3.267, 3.228, 3.228 3.274, 3.313, 3.233 +0.046 s
Affected non-expensive Python files, plus the new file on the changed side; CPython 3.14.2, NumPy 2.4.2, same Release extension 6.855, 6.895, 6.924 6.746, 6.768, 6.972 -0.127 s; no measurable increase

The original ten new C++ cases alone took 0.0074, 0.0066, and 0.0068 seconds. All new cases remain in regular CI; no new matrix axes or scheduled jobs are introduced. Baseline is 5fc87ec2; these are local subset timings, not hosted job totals or build-time measurements.

Keep this PR in draft until #612's hosted-cost requirement is satisfied: compare baseline and incremental wall time, runner/build configuration, and repetitions for build-and-test (default/scalar), release-cpp-smoke, gcc-release-cpp, cross-platform (Linux arm64/macOS/Windows), package-smoke, and scheduled coverage/sanitizer jobs as applicable. Set numeric additional-time budgets from those measurements and include any other validation-stream additions on the same jobs. This local result does not establish a Windows Debug budget or a universal CI limit.

Contributor agreement

  • I confirm this contribution is original work and license it under the project's Apache-2.0 license.
  • Human review of the AI-assisted changes. The commit includes an Assisted-by: trailer.

Compare outcome probabilities relatively, retain tiny joint outcomes, and bound reference omissions. Add analytic rare-event and negative-control coverage without changing production precision policies.

Assisted-by: Codex (GPT-6) <noreply@openai.com>
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://unitaryfoundation.github.io/clifft/pr-preview/pr-620/

Built to branch gh-pages at 2026-10-26 19:39 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Use the existing opaque NaN and infinity helpers so Clang cannot fold invalid test constants away. Round omitted-mass bounds toward a finite maximum and check their upward rounding under optimized arithmetic.

Assisted-by: Codex (GPT-6) <noreply@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make exact validation sensitive to rare branches

2 participants