Repository navigation
feat(analysis): exact counts and rationals with an explicit numeric policy - #49
Merged
Merged
Conversation
The live-R guard asked a different question from the record it stood in front of: it probed an empty package list, its consumer probed the four in R_PACKAGES, so on any checkout where R answers but the packages are absent -- every bare checkout -- the guard reported ready and the consumer threw outside any try, erroring the testset before its first @test (#43). The @info branch could not fire for the case it was written for. provenance.jl now answers the question once. RProbeStatus carries a status (:ok, :packages_missing, :r_unreachable, :no_lockfile) beside the record, and r_probe_status performs the single probe whose record the suite then asserts on, so guard and consumer cannot diverge. The failure's component names travel as a ProbeFailure field rather than as prose, so classification is a field read, and probe_r collects every absent package instead of stopping at the first: four missing packages report four. The skip is loud and named: the testset is titled with the status and the reason -- "r - not probed (packages_missing): ... dada2, Biostrings, ShortRead, vegan" -- and recorded with @test_skip, so it lands in the summary's own Broken column rather than in an @info that scrolls past. On CI, where ci.yml installs R and restores all 79 packages from renv.lock, the same block is a hard assertion: a skip there is a provisioning regression, not a bare checkout. The negative control is pure and runs everywhere the live probe skips: :r_unreachable, :packages_missing and :no_lockfile are asserted to be distinguishable in both symbol and text. Verified against the real module on a box with R 4.5.0 and none of the four packages: the :packages_missing path is exercised live, names all four, and records Broken rather than passing or erroring. Closes #43
Issue #15 asks for the Milestone 2 record to be audited. Re-ran the claims rather than reading them back, and two of them no longer described the repository: - the ">10% regression gate fails CI" claim, for the frontend script and for each bench/*.jl. The gate was built, then deliberately replaced by an informational comparison: ci.yml records two consecutive runs of identical benchmark code producing -16%..+52% deltas, because the workloads import no application code and so measure the runner. bench/table_loading/benchmark.jl prints NOTE and warns instead of failing. Corrected, with the reasoning kept. - the test-suite figures: 586 frontend tests across 16 files are now 604 (599 pass, 5 todo, 0 fail, 3368 expects) and 27 Julia unit files / 6830 lines are now 30 / 8310. Corrected as dated audit notes beside the original snapshot rather than overwritten, so the record shows both what was measured then and what holds now. Verified and left as recorded: coverage 40.23% funcs / 47.50% lines (claim 40/47), all five benchmark categories plus the comprehensive runner and analysis_config, the seven frontend workloads one-for-one, Codecov residue gone and gitar absent, CI runtime 14-29 min (claim 15-20). The project board is recorded as unverifiable with the credentials this audit had, not assumed. docs/audit/milestone-2-close-out.md carries the claim-by-claim evidence and the reproducible commands behind each verdict. Closes #15
…olicy First foundation for the validated statistics layer (#1): the numeric integrity half, which the methods half builds on. No method is added here and nothing existing changes behaviour -- :ordinary, the default mode, is Float64 throughout and leaves saved analyses untouched. The module keeps three claims apart instead of letting one representation carry all of them: exact (integer counts, rationals), approximate (any float, including arbitrary precision), rounded (display). Higher precision is not exactness, and the contract says so. What it refuses, each because the alternative is a wrong number that looks right: - `1//0`. Julia answers `1//0` for a non-zero numerator -- a non-finite Rational that renders as a number -- so a zero denominator is refused before the division, and a sample of zero reads reports `nothing` rather than a proportion of 0.0. - Counts arriving as Float64 beyond 2^53, which can hold a value but cannot prove which integer it was, and whose history at an earlier boundary (R, CSV, a JavaScript client) cannot be inspected. - Overflow: checked sums raise CountOverflowError unless the caller asks for :widen, rather than wrapping to a wrong total. - Unbounded rational growth, which is a resource decision and now raises ResourceLimitError against a stated budget. - NaN/Inf at the JSON boundary, which has no representation for them. Concurrency: BigFloat precision is a process global, not a task-local, so with_precision serialises callers and reads the achieved precision inside the scope. Six concurrent tasks at six precisions are asserted to each observe their own, with the default restored afterwards. Boundaries are published in docs/statistics/numeric-contracts.md: the JSON rules (integers beyond 2^53 and rationals travel as strings, because a JSON number in a browser is a Float64), CSV, R, DuckDB, and the provenance fingerprint that makes two runs comparable. The bounded method catalogue is drafted for approval in the same directory, since #1 requires it before methods are implemented. .gitignore whitelists docs/statistics/ the same way its sibling documentation directories are whitelisted. Tests: 144, run under the real module. Known-answer, adversarial (overflow, all-zero tables, denominator exhaustion, non-finite input), metamorphic (proportions of a table sum to exactly 1 across 50 random tables), concurrency, and negative controls that assert the lossy alternatives really are lossy. Refs #1
|
Warning Review limit reachedNext included review available in 53 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (11)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
14 tasks
|
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1 — the numeric-integrity half of the validated statistics layer, as a first
foundation. No method is added, and nothing existing changes behaviour: the
default
:ordinarymode is Float64 throughout, so saved analyses are untouched(criterion: "verify existing scientific behaviour remains unchanged when the new
analysis layer is not selected").
What it does
Keeps three claims apart rather than letting one representation carry all three —
exact (integer counts, rationals), approximate (any float, including
arbitrary precision), rounded (display) — and refuses the conversions where the
alternative is a wrong number that looks right:
1//0. Julia answers1//0for a non-zero numerator: a non-finiteRationalthat flows into charts looking like a number. A zero denominator is refused before
the division, and a sample of zero reads reports
nothing— an unsuccessful statethe presentation layer must render as such, not
0.0.which integer it was, and its history at an earlier boundary (R, CSV, a JavaScript
client) cannot be inspected. Refused rather than trusted.
CountOverflowErrorunless the caller asks for:widen; a total that wraps is worse than a failure because nothing downstream candetect it.
max_denominator_bitsbudget, withResourceLimitErrorwhen exhausted.Concurrency
BigFloatprecision is a process global in Julia, not task-local, sowith_precisionserialises callers (the same trade the R runtime already makes) andreads the achieved precision inside the scope. Six concurrent tasks at six
precisions are asserted to each observe their own, default restored afterwards.
Contracts published
docs/statistics/numeric-contracts.md: the JSON rules (integers beyond 2^53 andrationals travel as strings, because a JSON number in a browser is a Float64), CSV,
R and DuckDB boundaries, the mode-downgrade refusal, and the provenance fingerprint
that makes two runs comparable.
docs/statistics/method-catalogue-v1-draft.mddraftsthe bounded method catalogue for approval — #1 requires it before methods are
implemented, and approval is the owner's call, not this PR's.
Verification
144 tests against the real module: known-answer, adversarial (overflow, all-zero
tables, denominator exhaustion, non-finite input), metamorphic (proportions of a table
sum to exactly 1 across 50 random tables), concurrency, and negative controls that
assert the lossy alternatives really are lossy — e.g. that
Float64(2^53 + 1) != 2^53 + 1, so the refusal at that boundary is not a formality.scripts/check-spdx.sh,check-format.sh,check-lint.sh,check-blob-hygiene.shand
config/ci/lint_source.jlall clean.What this does not do
It does not make any method exact, and it claims no accuracy: precision is not
accuracy, and measurement error, sampling uncertainty and model misspecification are
untouched. The remaining #1 criteria — the catalogue's approval, method-level
known-answer and simulation validation, independent statistical review — stay open, and
the symbolic gate (#2) stays blocked.