diff --git a/proofs/agda/MetaManifold/All.agda b/proofs/agda/MetaManifold/All.agda index 7cedd39a..ff5c18c0 100644 --- a/proofs/agda/MetaManifold/All.agda +++ b/proofs/agda/MetaManifold/All.agda @@ -12,3 +12,4 @@ module MetaManifold.All where import MetaManifold.Prelude import MetaManifold.BenjaminiHochberg import MetaManifold.Scaling +import MetaManifold.Permanova diff --git a/proofs/agda/MetaManifold/Permanova.agda b/proofs/agda/MetaManifold/Permanova.agda new file mode 100644 index 00000000..97ed741d --- /dev/null +++ b/proofs/agda/MetaManifold/Permanova.agda @@ -0,0 +1,473 @@ +-- SPDX-License-Identifier: AGPL-3.0-only +-- SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) +-- +-- PERMANOVA's permutation bookkeeping: the two things `run_permanova` in +-- `src/analysis/analysis.jl` decides for itself rather than taking from vegan +-- (branch fix/permanova-terms, PR #35), proved. +-- +-- 1. `min_p_value = 1 / (n_perm + 1)`. vegan's permutation p-value is +-- (k + 1)/(B + 1), where B is the number of permutations performed and k +-- how many of them gave a statistic at least the observed one (0 ≤ k ≤ B). +-- The theorems say that 1/(B + 1) is the smallest p-value B permutations +-- can produce, that it is attained, and that no threshold below it can be +-- reached. +-- 2. Which terms have their p-value withheld. With permutations restricted +-- to blocks, Julia withholds the p-value of a term when +-- `_varies_within_any_block(values, blocks)` is false. The theorems say +-- that this happens exactly when every within-block permutation leaves the +-- term's labels where they were, so the permutations never exchange them. +-- They also say that when it is true, some within-block permutation does +-- exchange them. This characterises the block-constancy filter only; +-- it does not establish statistical testability or a reported p-value. +-- +-- What is NOT modelled, and why: +-- +-- * The pseudo-F statistic and vegan's distribution of it. The p-value +-- theorems take k as given; they are about the arithmetic of (k + 1)/(B + 1), +-- not about which permutations exceed the observed F. +-- * How many within-block permutations exist. B is whatever `permute` +-- reports (`attr(perm_res, "control")$nperm`). Six blocks of two samples +-- admit 2⁶ = 64 arrangements, the identity and 63 others, which is the 63 in +-- test_permanova.jl. That count is the product of the blocks' factorials; +-- it is not proved here. +-- * Unblocked runs. Julia only asks `_varies_within_any_block` when some +-- block label repeats (`blocked`). Without repeated blocks the +-- permutations are unrestricted and nothing is withheld; that branch is +-- Julia's `if`, not a theorem. +-- * Julia's `isequal` on NaN or `missing`. Equality of labels and of blocks +-- is an arbitrary decidable equality here. +-- * Floating point. `1 / (n_perm + 1)` is exact here and a Float64 in Julia. +-- +-- Theorems (names as in proofs/agda/README.md): +-- +-- minP-attained (0 + 1)/(B + 1) is 1/(B + 1) +-- p-floor every permutation p-value is at least 1/(B + 1) +-- p-at-most-one ... and at most 1 when k ≤ B +-- below-floor-unreachable a threshold below 1/(B + 1) is below every p-value +-- minP-anti more permutations never raise the floor +-- minP-63 with 63 permutations the floor is 1/64 +-- no-p-at-1%-with-63 with 63 permutations no p-value is at or below 0.01 +-- never-exchanged a block-constant term is fixed by every +-- within-block permutation +-- exchanged a term with two labels in one block is moved by a +-- within-block permutation (a transposition) +-- untestable⇔fixed block-constant ⇔ fixed by every within-block +-- permutation +-- varies-false⇔Constant the model of `_varies_within_any_block` returns +-- false exactly on block-constant input +-- withheld⇔never-exchanged ... so its false answer, which withholds the +-- p-value, holds exactly when no within-block +-- permutation exchanges the term's labels +-- tested⇔exchanged ... and its true answer, which passes this filter, +-- holds exactly when some within-block permutation +-- does +-- constant-in-blocks, varies-in-a-block +-- the two direct cases in test_permanova.jl +-- fixture-group-withheld, fixture-run-tested +-- the twelve-sample fixture in test_permanova.jl: group +-- is never exchanged, run is + +{-# OPTIONS --safe --without-K #-} + +module MetaManifold.Permanova where + +open import Data.Bool.Base using (Bool; true; false) +open import Data.Empty using (⊥-elim) +open import Data.Fin.Base using (Fin) +open import Data.Fin.Properties using (_≟_) +open import Data.Fin.Permutation using (Permutation′; _⟨$⟩ʳ_; transpose) +open import Data.Integer.Base as ℤ using (+_; +[1+_]) +import Data.Integer.Properties as ℤₚ +open import Data.List.Base using (List; []; _∷_; _++_; [_]; length; lookup) +open import Data.List.Membership.Propositional using (_∈_) +open import Data.List.Membership.Propositional.Properties + using (∈-lookup; ∈-++⁺ˡ; ∈-++⁺ʳ; ∈-++⁻) +open import Data.List.Properties using (++-assoc; ++-identityʳ) +open import Data.List.Relation.Unary.Any using (here; index) +open import Data.List.Relation.Unary.Any.Properties using (lookup-index) +open import Data.Maybe.Base using (Maybe; just; nothing) +open import Data.Nat.Base as ℕ using (ℕ; suc; z≤n; s≤s) +import Data.Nat.Properties as ℕₚ +open import Data.Product.Base using (Σ; ∃₂; _×_; _,_; proj₁; proj₂) +open import Data.Rational.Base as ℚ using (ℚ; _/_; 1ℚ; _≤_; _<_) +import Data.Rational.Properties as ℚₚ +open import Data.Rational.Unnormalised.Base using (mkℚᵘ; *≤*) +open import Data.Sum.Base using (inj₁; inj₂) +open import Function.Bundles using (_⇔_; mk⇔; Equivalence) +open import Relation.Binary.Definitions using (DecidableEquality) +open import Relation.Binary.PropositionalEquality + using (_≡_; _≢_; refl; sym; trans; cong; subst) +open import Relation.Nullary using (¬_; yes; no) +open import Relation.Nullary.Decidable using (toWitness) +open import Relation.Nullary.Negation using (contradiction) + +open import MetaManifold.Prelude using (numerator-monotone-≤; fromℚᵘ-mono-≤) + +------------------------------------------------------------------------ +-- 1. The smallest p-value B permutations can produce + +-- vegan's permutation p-value after B permutations, k of which gave a +-- statistic at least the observed one. +permP : ℕ → ℕ → ℚ +permP B k = + suc k / suc B + +-- Julia's `min_p_value = 1 / (Int(n_perm) + 1)`. +minP : ℕ → ℚ +minP B = + 1 / suc B + +-- The floor is a p-value: the one reported when no permutation reaches the +-- observed statistic. +minP-attained : ∀ B → permP B 0 ≡ minP B +minP-attained B = refl + +-- No permutation p-value is below the floor. +p-floor : ∀ B k → minP B ≤ permP B k +p-floor B k = fromℚᵘ-mono-≤ {mkℚᵘ (+ 1) B} {mkℚᵘ (+ suc k) B} + (numerator-monotone-≤ (+ 1) (+ suc k) B (ℤ.+≤+ (s≤s z≤n))) + +-- With k ≤ B, none is above 1. +p-at-most-one : ∀ B k → k ℕ.≤ B → permP B k ≤ 1ℚ +p-at-most-one B k k≤B = fromℚᵘ-mono-≤ {mkℚᵘ (+ suc k) B} {mkℚᵘ (+ 1) 0} + (*≤* (ℤₚ.≤-trans (ℤₚ.≤-reflexive (ℤₚ.*-identityʳ (+ suc k))) + (ℤₚ.≤-trans (ℤ.+≤+ (s≤s k≤B)) + (ℤₚ.≤-reflexive (sym (ℤₚ.*-identityˡ +[1+ B ])))))) + +-- A significance threshold below the floor is below every p-value, so no term +-- can be significant at it, whatever the data. +below-floor-unreachable : ∀ B k {α} → α < minP B → α < permP B k +below-floor-unreachable B k α --> -# Machine-checked proofs: Benjamini–Hochberg and size-factor scaling +# Machine-checked proofs: Benjamini–Hochberg, size-factor scaling and PERMANOVA permutations `proofs/agda/` holds Agda proofs of the exact-arithmetic properties of three functions in `src/analysis/differential.jl`: `bh_adjust`, `tss_factors` and @@ -12,6 +12,12 @@ differential-abundance change (branch `feat/differential-abundance-nb`, commits `6c3e8b2` and `23dc89c`). These proofs depend on that change being merged first. +`MetaManifold/Permanova.agda` covers the two things `run_permanova` in +`src/analysis/analysis.jl` decides for itself rather than taking from vegan: +the reported `min_p_value`, and which terms have their p-value withheld +(`_varies_within_any_block`). Both arrive with the PERMANOVA change (branch +`fix/permanova-terms`, PR #35), which these proofs depend on. + Every module is checked with `--safe --without-K`. There are no postulates, no holes and no termination or positivity pragmas. `proofs/tests/axiom-audit.sh` enforces this. It also fails if any module is @@ -62,6 +68,18 @@ vector. - **Input validation** (finite and in `[0, 1]`) is the hypothesis `All Valid ps`. The Julia `ArgumentError` paths are therefore assumptions of the theorems, not theorems. +- **`variesWithinAnyBlock`** follows the shape of + `_varies_within_any_block`. A list of `(value, block)` pairs stands in for + `zip(values, blocks)` and an association list for the `first_seen` + dictionary. As with `get!`, a block's value is stored only the first time + the block is seen, and the loop returns `true` at the first sample whose + value differs from its block's stored one. Equality of values and of blocks + is any decidable equality, standing in for `isequal`. +- **Within-block permutations** are permutations of the samples + (`Permutation′ n` from the standard library) that keep every sample in its + block. A term's labels are a function from samples to values. The + p-value is vegan's `(k + 1)/(B + 1)` for `B` permutations, `k` of which + reached the observed statistic. ## Theorem map @@ -90,6 +108,24 @@ proportion theorems are the relative-abundance reading of TSS, proved because the requirement names them. They do not describe a value the Julia code returns. +### PERMANOVA (`Permanova`) + +| Agda theorem | Plain statement | Julia | Test (`test/unit/test_permanova.jl`) | +|---|---|---|---| +| `minP-attained`, `p-floor` | `1/(B + 1)` is the smallest p-value `B` permutations can give, and it is given when no permutation reaches the observed statistic. | `min_p_value = 1 / (Int(n_perm) + 1)` | lines 73, 84 | +| `p-at-most-one` | With `k ≤ B`, no p-value is above 1. | (vegan) | – | +| `below-floor-unreachable` | A threshold below `1/(B + 1)` is below every p-value, so no term can meet it. | `min_p_value` | – | +| `minP-anti` | More permutations never raise the floor. | – | – | +| `minP-63`, `no-p-at-1%-with-63` | With 63 permutations the floor is `1/64`, and no p-value is at or below 0.01. | – | lines 83–84 | +| `never-exchanged` | A term that is constant within every block is left in place by every within-block permutation, so its labels are never exchanged. | the reason for withholding | – | +| `exchanged` | A term with two values in one block is moved by a within-block permutation: the swap of those two samples. | – | – | +| `untestable⇔fixed` | Constant within every block ⇔ fixed by every within-block permutation. | – | – | +| `varies-false⇔Constant` | The model of `_varies_within_any_block` returns `false` exactly when no block holds two different values. | `_varies_within_any_block` | lines 47–51 | +| `withheld⇔never-exchanged` | So `untestable` (the function returned `false`) holds exactly when no within-block permutation exchanges the term's labels. | `untestable`, `withheld` | lines 92–93 | +| `tested⇔exchanged` | The block-constancy filter admits a term exactly when some within-block permutation changes its labels; this does not guarantee a reported p-value. | `untestable`, `withheld` | line 98 | +| `constant-in-blocks`, `varies-in-a-block` | The two direct cases in the test, by evaluation. | `_varies_within_any_block` | lines 49–50 | +| `fixture-group-withheld`, `fixture-run-tested` | On the test's twelve samples (six individuals, two runs each, three individuals per group), `group` is never exchanged and `run` is. | `run_permanova` | lines 83–98 | + ## Corrections to the requirement wording - **R-BH-2a is true for the procedure.** An earlier archived note recorded @@ -105,6 +141,11 @@ returns. ## What is not proved, and why +- **Statistical validity and successful reporting.** Label exchangeability here + characterises only the block-constancy filter. It does not prove model + identifiability, assumptions for inference, or that vegan returns a p-value. + Julia also withholds missing p-values independently of this filter. + - **Geometric means, `exp` and `log`.** A geometric mean of rationals is in general irrational, so `_geomean` cannot be computed over `ℚ`. Wherever Julia divides by a geometric mean, the theorems divide by an arbitrary @@ -118,6 +159,16 @@ returns. proved. - **Floating point.** Every statement is exact. `Float64` rounding, and a refusal caused by underflow to `0.0`, are outside the model. +- **PERMANOVA: the pseudo-F statistic, and how many within-block + permutations exist.** The p-value theorems take `B` and `k` as given. `B` + is what `permute` reports, and `k` depends on the F distribution, neither of + which is modelled. Six blocks of two samples admit `2⁶ = 64` + arrangements, the identity and 63 others, which is the 63 in the test. + That count (the product of the blocks' factorials) is not proved. +- **PERMANOVA: the unblocked case.** Julia asks `_varies_within_any_block` + only when some block label repeats (`blocked`). Otherwise the permutations + are unrestricted and nothing is withheld; that is Julia's `if`, not a + theorem. - **Equivalence of the Agda and Julia code** is argued, not formalised. The model mirrors the loop, and the two differences above are each discharged by a proved lemma plus a short argument.