Skip to content

fix(analysis): method names compare in lower case; offsets wired in - #62

Merged
hyperpolymath merged 1 commit into
mainfrom
arena/01a0da3a-metamanifold-webui
Sep 25, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
arena/01a0da3a-metamanifold-webui

Conversation

@arena-ai-coding-agent

Copy link
Copy Markdown

Two things in one PR, because they have one cause: the layer that claimed to compute
TSS/CSS/RSS offsets did not, and the layer that accepted them refused them.

1. The refusal — main is red because of this

NormalizationConfig canonicalises its method to lower case ("TSS" → "tss"), while
VALID_NORMALIZATION_FOR_METHOD listed "TSS", "CSS", "RSS". The membership test
therefore failed for those three spellings and AnalysisConfig raised
normalization.method 'tss' incompatible with method 'nb_glm' — including for the
existing test/unit/test_execution.jl testset TSS offset for NB_GLM, which constructs
method="TSS" exactly as the documentation instructs. That is the failure behind the red
Julia tests job on the TSS-offsets commits; it is not a flaky test and it is not the
new estimation layer.

The table and both comparisons (constructor, validate_config) are lower case now, the
JSON-schema enum still accepts the upper-case spellings for existing documents, and
test/unit/test_scaling.jl asserts that every admissible spelling of every method name
is accepted.

2. The wiring — issue #16's offsets are computed, not aliased

prepare_analysis_table now computes scaling factors through src/analysis/scaling.jl
and hands the offset to the fit:

declared response offset
tss / none (count model) counts log(library size)
css counts log cumulative sum at the declared css_quantile
rss (TMM) counts log trimmed weighted mean of log-ratios to a reference sample
size_factors counts log median-of-ratios (RLE) — the estimator the name claimed
relative proportions none (a transform, not an offset)

Previously css/rss were aliased to relative under @warn, tss did that for every
method but nb_glm, and size_factors computed library-size centring wearing DESeq2's
name. css/rss are now refused for a response with no counts to offset — in the
configuration layer and again in the execution path.

css_quantile (0.75), tmm_ref_column (chosen the way edgeR chooses it, and recorded),
tmm_log_ratio_trim (0.3) and tmm_sum_trim (0.05) are validated in the constructor,
included in the config hash and canonical JSON, round-tripped through
to_json/from_json/Nickel/DEED, mirrored in the JSON schema and Nickel contracts,
exposed through the server's configuration route, and explained in context_help and the
frontend help. A run that declared a different quantile is a different run.

diagnostics.checks["scaling"] and the manifest provenance record the kind, the
definition, the parameters, the raw quantities before centring and a SHA-256 of the
offset vector.

Evidence

  • test/unit/test_scaling.jl: known answers with the arithmetic written beside them;
    properties a wrong implementation fails (feature permutation, a sample that is an exact
    multiple of another, invariance to a shared constant); the CSS outlier-robustness
    property; negative controls for every refusal (zero-total sample, zero CSS cumulative
    sum, unknown reference, disjoint supports, out-of-range trims); the integration path
    through prepare_analysis_table; and a base-R transcription of the same conditions,
    which skips loudly by name where R is unreachable.
  • Conditions document, published before the implementation and merged in feat(analysis): exact TSS/CSS/RSS offsets, and no silent substitutions #61:
    docs/statistics/method-conditions/scaling-and-offsets.md.
  • Local gates: check-spdx.sh OK (284 files), check-format.sh OK (331 files),
    check-lint.sh OK (tsc --noEmit clean over frontend/src).

Not claimed

Parity with metagenomeSeq::cumNorm and edgeR::calcNormFactors at 1e-6 is not
verified: neither package is in renv.lock, and this PR does not add an unpinned R
dependency to make a test pass. The base-R transcription catches implementation slips, not
specification errors. That outstanding condition stays recorded in issue #16 and in the
conditions document rather than being asserted as met.

Closes #16.

Two things, one cause: the layer that claimed to compute TSS/CSS/RSS offsets
did not, and the layer that accepted them refused them.

The refusal. NormalizationConfig canonicalises its method to lower case while
VALID_NORMALIZATION_FOR_METHOD listed "TSS", "CSS", "RSS", so the membership
test failed for those three spellings and AnalysisConfig raised
"normalization.method 'tss' incompatible with method 'nb_glm'". That is what
reddened the CI run for the merged TSS-offsets commit — the existing
test_execution.jl testset constructs method="TSS" exactly as documented. The
table and both comparisons are lower case now, and the new tests assert that
every admissible spelling of every method name is accepted.

The wiring. prepare_analysis_table now computes scaling factors through
src/analysis/scaling.jl and hands over the offset, rather than aliasing
TSS/CSS/RSS to `relative` (which fed a count model a table of fractions) and
calling library-size centring "DESeq2 median-of-ratios":

  tss          log library size (counts stay counts)
  css          log cumulative sum at the declared css_quantile
  rss (TMM)    log trimmed weighted mean of log-ratios to a reference sample
  size_factors median-of-ratios (RLE), the estimator the name claimed
  none         the plain log library size, as it always meant for a count model

css and rss are refused for a response with no counts to offset, in the
configuration layer and again in the execution path. css_quantile,
tmm_ref_column, tmm_log_ratio_trim and tmm_sum_trim are validated in the
constructor, part of the config hash and canonical JSON, round-tripped through
to_json/from_json/Nickel/DEED, mirrored in the JSON schema and Nickel
contracts, exposed through the server route, and explained in context_help and
the frontend help.

What was computed is recorded in diagnostics.checks["scaling"] and the
manifest provenance, including the parameters, the raw quantities before
centring and a SHA-256 of the offset vector.

Evidence: test/unit/test_scaling.jl — known answers with the arithmetic written
beside them, properties a wrong implementation fails (feature permutation,
sample-multiple invariance, shared-constant invariance), the CSS outlier
robustness property, negative controls for every refusal, the integration path
through prepare_analysis_table, and a base-R transcription of the same
conditions. Parity with metagenomeSeq::cumNorm and edgeR::calcNormFactors is
not claimed: neither is in renv.lock, and the outstanding condition stays
recorded in issue #16 and in
docs/statistics/method-conditions/scaling-and-offsets.md.

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Sep 25, 2026

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9bc858b1-5d6b-4943-a683-5f7aeec1debd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hyperpolymath
hyperpolymath merged commit 11c9f54 into main Sep 25, 2026
5 of 6 checks passed
@hyperpolymath
hyperpolymath deleted the arena/01a0da3a-metamanifold-webui branch September 25, 2026 20:53
hyperpolymath added a commit that referenced this pull request Sep 25, 2026
## Why

With #65's annotations, the red Julia job on `main` is readable for the
first time. Run 36193210557 names 11 failures and 20 errors, **all in
`test/unit/test_estimation.jl`**. The Scaling suite (#16) passes. There
are three causes:

1. **Every parametric fit came back `not_run`.** R writes the fits with
`write.csv(..., na = "NA")`, and Julia read them with `CSV.File(path)`,
whose default `missingstring` is `""`. Any numeric column containing an
`NA` was read as `String31`, and `_num` then threw `MethodError: no
method matching isfinite(::String31)`. The catch-all turned that into
`status = :not_run`, which produced the cascade of `KeyError`s
(`n_tested`, `n_failed`, `correction`, `t_up`) and `0 == 3` results in
five testsets.
2. **`glmGamPoi` never reached its #21 refusal.** `AdvancedConfig`
lower-cases `dispersion_method` and then checks it against
`VALID_DISPERSION_METHODS`, which spells it `glmGamPoi`. So
`nb_config(dispersion = "glmGamPoi")` was refused as an unknown name.
This is the same class of bug as the `tss`/`TSS` one fixed in #62.
3. **The "no placeholder statistics" scan flagged comments.** Comments
in `estimation.jl` and `Execution.jl` quoted the removed code literally
(`hash(taxon_id)`).

## What

- `estimation.jl`: `R_NA_STRINGS = ["NA", ""]` is passed as
`missingstring` to both `CSV.File` reads. `_refused_dispersion` looks up
`REFUSED_DISPERSION` case-insensitively, and the error message uses the
canonical name.
- `AnalysisConfig.jl`: the dispersion compare is now lower-case against
`lowercase.(VALID_DISPERSION_METHODS)`.
- The comments are reworded, and the source-scan test is unchanged and
still strict.
- `test_estimation.jl`: three extra spellings (`glmgampoi`, `GLMGAMPOI`,
` glmGamPoi `) must each hit the #21 refusal.
- `CHANGELOG.md` entry.

## Verified / not verified

- spdx, format and blob-hygiene gates are clean.
- There is no Julia runtime in the authoring sandbox, so **this PR's CI
run is the first execution**. Diagnosis 1 comes from the `reason` string
in the annotation. Once the fits run, some numeric assertions that were
never reached may surface in turn. If so, the annotations will name
them.

Co-authored-by: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com>
hyperpolymath added a commit that referenced this pull request Oct 20, 2026
…rast (#192)

## Change and reason

Prepares the next push to upstream
[JoshuaJewell#62](JoshuaJewell#62),
the glmGamPoi dispersion option for JoshuaJewell#37.
The branch is upstream PR 62's reviewed head (`efe1df1`) plus three new
commits, so the update fast-forwards. Under upstream CONTRIBUTING item
8, nothing is rebased once review has started.

1. **Report glm_gp's negative-binomial overdispersion as theta.** Your
`wip/pr62-theta` commit, cherry-picked unchanged. It covers both of
Joshua's code requests from Oct 19 (theta is `1 / overdispersions`; the
Poisson boundary is read from it).
2. **Pass the glmGamPoi contrast to test_de as numeric weights.**
`glmGamPoi::test_de` parses and evaluates a text contrast as R code. The
contrast label is built from run, group and sub-group names joined by
`/`, and those names may contain `-`. So the glmGamPoi path failed every
taxon for comparisons such as `run-A` vs `run-B` or `grpA/run_1` vs
`grpB/run_2`, and label text that reads as R code was executed. The
commit adds a regression test that covers both dispersions.
3. **State what glmGamPoi's theta, standard error and boundary mean.**
These are precision fixes, checked against the glmGamPoi 1.22.0 source:
- `overdispersions` is the per-taxon estimate before shrinkage. The
coefficients and the F-test use the cross-taxon trend, and only `lfc_se`
reads `overdispersions`.
   - The boundary note no longer says the fit is Poisson.
- The README now lists `dispersion` with the other study-scoped
settings.

**Do not merge this PR on GitHub.** Merging would put unsigned commits,
or a merge commit, onto upstream PR 62. Re-sign on your machine and push
instead:

```sh
git fetch origin claude/elegant-rubin-z92rx4 upstream/glmgampoi-r
git switch -c pr62-next origin/claude/elegant-rubin-z92rx4
git rebase --exec 'git commit --amend --no-edit -S' efe1df1   # re-signs only the 3 new commits
git log --format='%h %G? %s' efe1df1..HEAD                     # expect G or E, never N
git push origin pr62-next:upstream/glmgampoi-r                  # fast-forward: no --force needed
```

When Joshua asks for a rebase (upstream `main` is now `ce2c645`), the
five-commit stack rebases cleanly onto it. That was checked in a scratch
worktree.

## Validation

Environment: Julia 1.12.5 (the CI pin, from the official image), R
4.3.3, MASS 7.3-60.0.1, glmGamPoi 1.14.3, bun 1.4.2.

glmGamPoi 1.14.3 lacks `compute_lfc_se`, so 1.22.0's pure-R `test_de`,
`test_de_q` and `predict.glmGamPoi` were loaded over it. Diffing the
1.14.0 and 1.22.0 sources shows no other change on these paths. Upstream
CI runs the real pins (R 4.5, glmGamPoi 1.22.0).

- **`CI=true julia --project=. test/runtests.jl test_differential.jl`:**
- **head:** 253 passed, 0 failed, 5 skipped. The skips are `clr_welch`
tests that need zCompositions. `CI=true` makes a missing glmGamPoi fail
rather than skip, so every glmGamPoi test ran.
- **commits 1, 2 and 3 on their own:** 202, 253 and 253 passed, 0
failed.
- **the new label test on the pre-fix code:** 5 failures, as expected.
The four label sets fail every taxon, and the R-code label ran.

- **Label experiments** (the `fit_nb` R block, run directly):
- **Before:** `run-B`, `grpB/run_2` and `2024-02` failed every taxon,
and a label of `[file.create(...)]` created the file.
- **After:** every label set gives rows identical to `A`/`B`, the file
is not created, and `glm_nb` is unchanged.
- **Null calibration** (share of p < 0.05, binomial SE about 0.003):

  | samples per group | glm_nb (Wald) | glmGamPoi (QL F) |
  |---|---|---|
  | 4 | 0.132 | 0.046 |
  | 12 | 0.069 | 0.048 |

- **Frontend:** `bun test src` 39 pass; typecheck clean; build succeeds.

Not run: the integration and server suites, the DADA2 taxonomy and
phylogeny paths, and the `clr_welch` tests (zCompositions unavailable;
they skip loudly).

## Scope

No lockfile change. The upstream PR description still has to be
replaced; Joshua's third request asks for that. A draft follows,
together with the summary comment for upstream issue 37.

<details><summary>Draft replacement description for upstream PR
62</summary>

~~~~markdown
## Change and reason

Adds an optional `analysis.differential.dispersion` setting for the
`nb_glm` method (issue #37). `glm_nb` stays the default and is
unchanged: `MASS::glm.nb` fits each taxon with its own
maximum-likelihood theta, and the group coefficient gets a Wald test.

`glmGamPoi` fits one `glmGamPoi::glm_gp` across the taxa that pass
`min_prevalence`, with `overdispersion_shrinkage = TRUE`. It reports
`glmGamPoi::test_de`'s quasi-likelihood F-test of the group contrast
(Lund et al., 2012), which is glmGamPoi's own published inference. The
trend-plus-Wald design in the first version of this PR has been
replaced. In that path:

- **estimate and se**: `lfc` and `lfc_se`, converted from log2 to the
natural-log scale. `lfc_se` comes from `predict()` and does not feed the
p-value.
- **statistic and pvalue**: the F statistic and its p-value. The F-test
refits the reduced model at the cross-taxon dispersion trend and divides
by the shrunken quasi-likelihood dispersion.
- **dispersion_theta**: `1 / overdispersions`. This is glm_gp's
negative-binomial overdispersion, the taxon's own estimate before
shrinkage, and the same parameter `MASS::glm.nb`'s theta inverts
(requested in review).
- **Poisson boundary**: based on that overdispersion (requested in
review). A zero estimate marks the row `boundary` and keeps its
statistics, since the test does not read that estimate. A negative or
non-finite estimate fails only its own taxon.
- **contrast**: passed to `test_de` as numeric weights on the design's
columns. `test_de` parses and evaluates a text contrast as R code.
Before this change, a contrast label containing `-` or `/` (e.g.
`run-B`, or `grpB/run_2`, which the app builds when conditions differ by
group and run) made every taxon fail. A label that reads as R code was
executed.
- **failures**: a missing glmGamPoi raises `GlmGamPoiUnavailable` (HTTP
503). A `glm_gp` or `test_de` error fails every taxon, with no fallback
to `glm_nb`. `glmGamPoi` is refused for `clr_welch`.

Config and UI: the config editor and `PATCH /api/v1/config` accept the
new enum (`glm_nb`, `glmGamPoi`). The constructor refuses an invalid
method/dispersion pair when the analysis runs (HTTP 400
`invalid_config`). There is no other config rewrite; the first version's
description wrongly claimed one. The README lists the option with the
other study-scoped settings.

Commits, oldest first (the last three are new since your review):

1. `02ee915` Add glmGamPoi to the pinned R dependencies
2. `efe1df1` Add glmGamPoi dispersion option to nb_glm differential
abundance
3. Report glm_gp's negative-binomial overdispersion as theta (review:
theta and boundary)
4. Pass the glmGamPoi contrast to test_de as numeric weights (labels
with `-` or `/`; label text evaluated as R)
5. State what glmGamPoi's theta, standard error and boundary mean
(comments, note text, README, result type)

Closes #37.

## Validation

Run in a cloud container with Julia 1.12.5 (the CI pin), R 4.3.3, MASS
7.3-60.0.1, glmGamPoi 1.14.3, and bun 1.4.2. glmGamPoi 1.14.3 lacks
`compute_lfc_se`, so 1.22.0's `test_de`, `test_de_q` and
`predict.glmGamPoi` (pure R) were loaded over it. Diffing the 1.14.0 and
1.22.0 sources shows no other change on these code paths: `glm_gp_impl`,
`parse_contrast`, the overdispersion estimate and the quasi-likelihood
shrinkage are the same. CI on this PR runs the pinned R 4.5 with
glmGamPoi 1.22.0.

- `CI=true julia --project=. test/runtests.jl test_differential.jl`:
- **branch head:** 253 passed, 0 failed, 5 skipped. The skips are the
`clr_welch` tests, which need zCompositions. Under `CI=true` a missing
glmGamPoi fails rather than skips, so the glmGamPoi tests all ran.
  - **each new commit on its own:** 202, 253 and 253 passed, 0 failed.
- **the new label test against the pre-fix code:** it fails as intended.
Each of the four label sets fails every taxon, and the R-code label's
code runs.
- The `fit_nb` R block, run directly in R on the `effect` fixture:
- **labels**: `run-A`/`run-B`, `grpA/run_1`/`grpB/run_2` and
`2024-01`/`2024-02` failed every taxon before the fix. With it, every
label set, including one containing R code, gives rows identical to
`A`/`B`.
- **injection**: a label of `[file.create(...)]` created the file before
the fix and does not after it.
  - **glm_nb**: rows are unchanged.
- **field check**: setting `overdispersions[1] <- 0` changes `lfc_se[1]`
only; `pval`, `f_statistic` and `lfc` stay the same.
- Null calibration through the same R block (no taxon differs; TSS
offsets; dispersion falling with the mean). Share of p < 0.05:

  | samples per group | taxa tested | glm_nb (Wald) | glmGamPoi (QL F) |
  |---|---|---|---|
  | 4 | ~5,900 | 0.132 | 0.046 |
  | 12 | 4,000 | 0.069 | 0.048 |

  The binomial SE is about 0.003.
- Frontend (`frontend/`): `bun test src` 39 pass; `bun run typecheck`
clean; `bun run build` succeeds.

Not run here: the integration suite (`--integration`, `--server`), the
DADA2 taxonomy and phylogeny paths, and the `clr_welch` tests
(zCompositions was not available in the container; they skip loudly).

## Scope

Left out: a native Julia gamma-Poisson fit (#37 settled on R),
zero-inflated and occupancy models, any change to the `glm_nb` default
or its results, and any UI beyond the one enum.

Lockfile: `renv.lock` adds glmGamPoi 1.22.0 and the eleven packages it
needs that were not already pinned. Existing pins are unchanged. The
option cannot run without them, and CI and `install.jl` check that
glmGamPoi loads.

Noted for a separate issue, not this PR: in the calibration above, the
default `glm_nb` Wald test rejects too often at small n.

## Contribution checks

- [x] I followed
[CONTRIBUTING.md](https://github.com/JoshuaJewell/MetaManifold-WebUI/blob/main/CONTRIBUTING.md).
- [ ] This branch starts at current upstream main and contains no merges
from fork main. It starts at `029fa61` and contains no merges. Since
review has started, it gains commits rather than being rebased (item 8).
It rebases cleanly onto `ce2c645` whenever you ask.
- [ ] Every commit is signed and shows as Verified. *(tick after
re-signing)*
- [ ] CI has run and passes. *(tick after the push)*
- [x] Existing licensing, citation and acknowledgement text is
unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01HtyPfZ1MWBB94RawgzQgz7
~~~~

</details>

<details><summary>Draft summary comment for upstream issue 37</summary>

~~~~markdown
**Summary for the record**

**Question (Oct 15):** take glmGamPoi's dispersion shrinkage for
`nb_glm` (a) in R through RCall, or (b) as a native Julia gamma-Poisson
fit with R kept as the oracle?

**Answer (@JoshuaJewell, Oct 17):** (a). "glmGamPoi stays R, since there
is no reputable Julia equivalent."

**What follows from that, in #62:**

- **Option:** `analysis.differential.dispersion: glmGamPoi` is optional.
The default stays `glm_nb` (`MASS::glm.nb`), and its results are
unchanged.
- **Test:** inference is glmGamPoi's own quasi-likelihood F-test
(`glm_gp` + `test_de`), not the trend-plus-Wald plan from Oct 15.
- **Review fixes:** theta is `1 / overdispersions` (glm_gp's NB
overdispersion), and the Poisson boundary is read from it, as requested.
The contrast is now passed as numeric weights. Before that, any label
containing `-` or `/` (e.g. `grpB/run_2`) made every taxon fail, and
label text was evaluated as R code.
- **Dependencies:** `renv.lock` adds glmGamPoi 1.22.0 and the eleven
packages it needs; existing pins are unchanged.
- **Evidence:** in a null simulation, the share of p < 0.05 was:

  | samples per group | `glm_nb` (Wald) | glmGamPoi (QL F) |
  |---|---|---|
  | 4 | 13.2% | 4.6% |
  | 12 | 6.9% | 4.8% |

The `glm_nb` excess at small n is outside this issue's scope and is
worth its own issue.

**Good to go.** The R/RCall route is settled, and #62 is ready for
re-review once the three new commits are signed and pushed.
~~~~

</details>

---
_Generated by [Claude
Code](https://claude.ai/code/session_01HtyPfZ1MWBB94RawgzQgz7)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(analysis): TSS/CSS/RSS offsets — exact normalization with DESeq2-style offsets, not alias to relative

1 participant