Repository navigation
fix(analysis): method names compare in lower case; offsets wired in - #62
Merged
Merged
Conversation
Two things, one cause: the layer that claimed to compute TSS/CSS/RSS offsets did not, and the layer that accepted them refused them. The refusal. NormalizationConfig canonicalises its method to lower case while VALID_NORMALIZATION_FOR_METHOD listed "TSS", "CSS", "RSS", so the membership test failed for those three spellings and AnalysisConfig raised "normalization.method 'tss' incompatible with method 'nb_glm'". That is what reddened the CI run for the merged TSS-offsets commit — the existing test_execution.jl testset constructs method="TSS" exactly as documented. The table and both comparisons are lower case now, and the new tests assert that every admissible spelling of every method name is accepted. The wiring. prepare_analysis_table now computes scaling factors through src/analysis/scaling.jl and hands over the offset, rather than aliasing TSS/CSS/RSS to `relative` (which fed a count model a table of fractions) and calling library-size centring "DESeq2 median-of-ratios": tss log library size (counts stay counts) css log cumulative sum at the declared css_quantile rss (TMM) log trimmed weighted mean of log-ratios to a reference sample size_factors median-of-ratios (RLE), the estimator the name claimed none the plain log library size, as it always meant for a count model css and rss are refused for a response with no counts to offset, in the configuration layer and again in the execution path. css_quantile, tmm_ref_column, tmm_log_ratio_trim and tmm_sum_trim are validated in the constructor, part of the config hash and canonical JSON, round-tripped through to_json/from_json/Nickel/DEED, mirrored in the JSON schema and Nickel contracts, exposed through the server route, and explained in context_help and the frontend help. What was computed is recorded in diagnostics.checks["scaling"] and the manifest provenance, including the parameters, the raw quantities before centring and a SHA-256 of the offset vector. Evidence: test/unit/test_scaling.jl — known answers with the arithmetic written beside them, properties a wrong implementation fails (feature permutation, sample-multiple invariance, shared-constant invariance), the CSS outlier robustness property, negative controls for every refusal, the integration path through prepare_analysis_table, and a base-R transcription of the same conditions. Parity with metagenomeSeq::cumNorm and edgeR::calcNormFactors is not claimed: neither is in renv.lock, and the outstanding condition stays recorded in issue #16 and in docs/statistics/method-conditions/scaling-and-offsets.md. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
requested a review
from hyperpolymath
as a code owner
September 25, 2026 20:52
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
hyperpolymath
approved these changes
Sep 25, 2026
hyperpolymath
added a commit
that referenced
this pull request
Sep 25, 2026
## Why With #65's annotations, the red Julia job on `main` is readable for the first time. Run 36193210557 names 11 failures and 20 errors, **all in `test/unit/test_estimation.jl`**. The Scaling suite (#16) passes. There are three causes: 1. **Every parametric fit came back `not_run`.** R writes the fits with `write.csv(..., na = "NA")`, and Julia read them with `CSV.File(path)`, whose default `missingstring` is `""`. Any numeric column containing an `NA` was read as `String31`, and `_num` then threw `MethodError: no method matching isfinite(::String31)`. The catch-all turned that into `status = :not_run`, which produced the cascade of `KeyError`s (`n_tested`, `n_failed`, `correction`, `t_up`) and `0 == 3` results in five testsets. 2. **`glmGamPoi` never reached its #21 refusal.** `AdvancedConfig` lower-cases `dispersion_method` and then checks it against `VALID_DISPERSION_METHODS`, which spells it `glmGamPoi`. So `nb_config(dispersion = "glmGamPoi")` was refused as an unknown name. This is the same class of bug as the `tss`/`TSS` one fixed in #62. 3. **The "no placeholder statistics" scan flagged comments.** Comments in `estimation.jl` and `Execution.jl` quoted the removed code literally (`hash(taxon_id)`). ## What - `estimation.jl`: `R_NA_STRINGS = ["NA", ""]` is passed as `missingstring` to both `CSV.File` reads. `_refused_dispersion` looks up `REFUSED_DISPERSION` case-insensitively, and the error message uses the canonical name. - `AnalysisConfig.jl`: the dispersion compare is now lower-case against `lowercase.(VALID_DISPERSION_METHODS)`. - The comments are reworded, and the source-scan test is unchanged and still strict. - `test_estimation.jl`: three extra spellings (`glmgampoi`, `GLMGAMPOI`, ` glmGamPoi `) must each hit the #21 refusal. - `CHANGELOG.md` entry. ## Verified / not verified - spdx, format and blob-hygiene gates are clean. - There is no Julia runtime in the authoring sandbox, so **this PR's CI run is the first execution**. Diagnosis 1 comes from the `reason` string in the annotation. Once the fits run, some numeric assertions that were never reached may surface in turn. If so, the annotations will name them. Co-authored-by: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com>
hyperpolymath
added a commit
that referenced
this pull request
Oct 20, 2026
…rast (#192) ## Change and reason Prepares the next push to upstream [JoshuaJewell#62](JoshuaJewell#62), the glmGamPoi dispersion option for JoshuaJewell#37. The branch is upstream PR 62's reviewed head (`efe1df1`) plus three new commits, so the update fast-forwards. Under upstream CONTRIBUTING item 8, nothing is rebased once review has started. 1. **Report glm_gp's negative-binomial overdispersion as theta.** Your `wip/pr62-theta` commit, cherry-picked unchanged. It covers both of Joshua's code requests from Oct 19 (theta is `1 / overdispersions`; the Poisson boundary is read from it). 2. **Pass the glmGamPoi contrast to test_de as numeric weights.** `glmGamPoi::test_de` parses and evaluates a text contrast as R code. The contrast label is built from run, group and sub-group names joined by `/`, and those names may contain `-`. So the glmGamPoi path failed every taxon for comparisons such as `run-A` vs `run-B` or `grpA/run_1` vs `grpB/run_2`, and label text that reads as R code was executed. The commit adds a regression test that covers both dispersions. 3. **State what glmGamPoi's theta, standard error and boundary mean.** These are precision fixes, checked against the glmGamPoi 1.22.0 source: - `overdispersions` is the per-taxon estimate before shrinkage. The coefficients and the F-test use the cross-taxon trend, and only `lfc_se` reads `overdispersions`. - The boundary note no longer says the fit is Poisson. - The README now lists `dispersion` with the other study-scoped settings. **Do not merge this PR on GitHub.** Merging would put unsigned commits, or a merge commit, onto upstream PR 62. Re-sign on your machine and push instead: ```sh git fetch origin claude/elegant-rubin-z92rx4 upstream/glmgampoi-r git switch -c pr62-next origin/claude/elegant-rubin-z92rx4 git rebase --exec 'git commit --amend --no-edit -S' efe1df1 # re-signs only the 3 new commits git log --format='%h %G? %s' efe1df1..HEAD # expect G or E, never N git push origin pr62-next:upstream/glmgampoi-r # fast-forward: no --force needed ``` When Joshua asks for a rebase (upstream `main` is now `ce2c645`), the five-commit stack rebases cleanly onto it. That was checked in a scratch worktree. ## Validation Environment: Julia 1.12.5 (the CI pin, from the official image), R 4.3.3, MASS 7.3-60.0.1, glmGamPoi 1.14.3, bun 1.4.2. glmGamPoi 1.14.3 lacks `compute_lfc_se`, so 1.22.0's pure-R `test_de`, `test_de_q` and `predict.glmGamPoi` were loaded over it. Diffing the 1.14.0 and 1.22.0 sources shows no other change on these paths. Upstream CI runs the real pins (R 4.5, glmGamPoi 1.22.0). - **`CI=true julia --project=. test/runtests.jl test_differential.jl`:** - **head:** 253 passed, 0 failed, 5 skipped. The skips are `clr_welch` tests that need zCompositions. `CI=true` makes a missing glmGamPoi fail rather than skip, so every glmGamPoi test ran. - **commits 1, 2 and 3 on their own:** 202, 253 and 253 passed, 0 failed. - **the new label test on the pre-fix code:** 5 failures, as expected. The four label sets fail every taxon, and the R-code label ran. - **Label experiments** (the `fit_nb` R block, run directly): - **Before:** `run-B`, `grpB/run_2` and `2024-02` failed every taxon, and a label of `[file.create(...)]` created the file. - **After:** every label set gives rows identical to `A`/`B`, the file is not created, and `glm_nb` is unchanged. - **Null calibration** (share of p < 0.05, binomial SE about 0.003): | samples per group | glm_nb (Wald) | glmGamPoi (QL F) | |---|---|---| | 4 | 0.132 | 0.046 | | 12 | 0.069 | 0.048 | - **Frontend:** `bun test src` 39 pass; typecheck clean; build succeeds. Not run: the integration and server suites, the DADA2 taxonomy and phylogeny paths, and the `clr_welch` tests (zCompositions unavailable; they skip loudly). ## Scope No lockfile change. The upstream PR description still has to be replaced; Joshua's third request asks for that. A draft follows, together with the summary comment for upstream issue 37. <details><summary>Draft replacement description for upstream PR 62</summary> ~~~~markdown ## Change and reason Adds an optional `analysis.differential.dispersion` setting for the `nb_glm` method (issue #37). `glm_nb` stays the default and is unchanged: `MASS::glm.nb` fits each taxon with its own maximum-likelihood theta, and the group coefficient gets a Wald test. `glmGamPoi` fits one `glmGamPoi::glm_gp` across the taxa that pass `min_prevalence`, with `overdispersion_shrinkage = TRUE`. It reports `glmGamPoi::test_de`'s quasi-likelihood F-test of the group contrast (Lund et al., 2012), which is glmGamPoi's own published inference. The trend-plus-Wald design in the first version of this PR has been replaced. In that path: - **estimate and se**: `lfc` and `lfc_se`, converted from log2 to the natural-log scale. `lfc_se` comes from `predict()` and does not feed the p-value. - **statistic and pvalue**: the F statistic and its p-value. The F-test refits the reduced model at the cross-taxon dispersion trend and divides by the shrunken quasi-likelihood dispersion. - **dispersion_theta**: `1 / overdispersions`. This is glm_gp's negative-binomial overdispersion, the taxon's own estimate before shrinkage, and the same parameter `MASS::glm.nb`'s theta inverts (requested in review). - **Poisson boundary**: based on that overdispersion (requested in review). A zero estimate marks the row `boundary` and keeps its statistics, since the test does not read that estimate. A negative or non-finite estimate fails only its own taxon. - **contrast**: passed to `test_de` as numeric weights on the design's columns. `test_de` parses and evaluates a text contrast as R code. Before this change, a contrast label containing `-` or `/` (e.g. `run-B`, or `grpB/run_2`, which the app builds when conditions differ by group and run) made every taxon fail. A label that reads as R code was executed. - **failures**: a missing glmGamPoi raises `GlmGamPoiUnavailable` (HTTP 503). A `glm_gp` or `test_de` error fails every taxon, with no fallback to `glm_nb`. `glmGamPoi` is refused for `clr_welch`. Config and UI: the config editor and `PATCH /api/v1/config` accept the new enum (`glm_nb`, `glmGamPoi`). The constructor refuses an invalid method/dispersion pair when the analysis runs (HTTP 400 `invalid_config`). There is no other config rewrite; the first version's description wrongly claimed one. The README lists the option with the other study-scoped settings. Commits, oldest first (the last three are new since your review): 1. `02ee915` Add glmGamPoi to the pinned R dependencies 2. `efe1df1` Add glmGamPoi dispersion option to nb_glm differential abundance 3. Report glm_gp's negative-binomial overdispersion as theta (review: theta and boundary) 4. Pass the glmGamPoi contrast to test_de as numeric weights (labels with `-` or `/`; label text evaluated as R) 5. State what glmGamPoi's theta, standard error and boundary mean (comments, note text, README, result type) Closes #37. ## Validation Run in a cloud container with Julia 1.12.5 (the CI pin), R 4.3.3, MASS 7.3-60.0.1, glmGamPoi 1.14.3, and bun 1.4.2. glmGamPoi 1.14.3 lacks `compute_lfc_se`, so 1.22.0's `test_de`, `test_de_q` and `predict.glmGamPoi` (pure R) were loaded over it. Diffing the 1.14.0 and 1.22.0 sources shows no other change on these code paths: `glm_gp_impl`, `parse_contrast`, the overdispersion estimate and the quasi-likelihood shrinkage are the same. CI on this PR runs the pinned R 4.5 with glmGamPoi 1.22.0. - `CI=true julia --project=. test/runtests.jl test_differential.jl`: - **branch head:** 253 passed, 0 failed, 5 skipped. The skips are the `clr_welch` tests, which need zCompositions. Under `CI=true` a missing glmGamPoi fails rather than skips, so the glmGamPoi tests all ran. - **each new commit on its own:** 202, 253 and 253 passed, 0 failed. - **the new label test against the pre-fix code:** it fails as intended. Each of the four label sets fails every taxon, and the R-code label's code runs. - The `fit_nb` R block, run directly in R on the `effect` fixture: - **labels**: `run-A`/`run-B`, `grpA/run_1`/`grpB/run_2` and `2024-01`/`2024-02` failed every taxon before the fix. With it, every label set, including one containing R code, gives rows identical to `A`/`B`. - **injection**: a label of `[file.create(...)]` created the file before the fix and does not after it. - **glm_nb**: rows are unchanged. - **field check**: setting `overdispersions[1] <- 0` changes `lfc_se[1]` only; `pval`, `f_statistic` and `lfc` stay the same. - Null calibration through the same R block (no taxon differs; TSS offsets; dispersion falling with the mean). Share of p < 0.05: | samples per group | taxa tested | glm_nb (Wald) | glmGamPoi (QL F) | |---|---|---|---| | 4 | ~5,900 | 0.132 | 0.046 | | 12 | 4,000 | 0.069 | 0.048 | The binomial SE is about 0.003. - Frontend (`frontend/`): `bun test src` 39 pass; `bun run typecheck` clean; `bun run build` succeeds. Not run here: the integration suite (`--integration`, `--server`), the DADA2 taxonomy and phylogeny paths, and the `clr_welch` tests (zCompositions was not available in the container; they skip loudly). ## Scope Left out: a native Julia gamma-Poisson fit (#37 settled on R), zero-inflated and occupancy models, any change to the `glm_nb` default or its results, and any UI beyond the one enum. Lockfile: `renv.lock` adds glmGamPoi 1.22.0 and the eleven packages it needs that were not already pinned. Existing pins are unchanged. The option cannot run without them, and CI and `install.jl` check that glmGamPoi loads. Noted for a separate issue, not this PR: in the calibration above, the default `glm_nb` Wald test rejects too often at small n. ## Contribution checks - [x] I followed [CONTRIBUTING.md](https://github.com/JoshuaJewell/MetaManifold-WebUI/blob/main/CONTRIBUTING.md). - [ ] This branch starts at current upstream main and contains no merges from fork main. It starts at `029fa61` and contains no merges. Since review has started, it gains commits rather than being rebased (item 8). It rebases cleanly onto `ce2c645` whenever you ask. - [ ] Every commit is signed and shows as Verified. *(tick after re-signing)* - [ ] CI has run and passes. *(tick after the push)* - [x] Existing licensing, citation and acknowledgement text is unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HtyPfZ1MWBB94RawgzQgz7 ~~~~ </details> <details><summary>Draft summary comment for upstream issue 37</summary> ~~~~markdown **Summary for the record** **Question (Oct 15):** take glmGamPoi's dispersion shrinkage for `nb_glm` (a) in R through RCall, or (b) as a native Julia gamma-Poisson fit with R kept as the oracle? **Answer (@JoshuaJewell, Oct 17):** (a). "glmGamPoi stays R, since there is no reputable Julia equivalent." **What follows from that, in #62:** - **Option:** `analysis.differential.dispersion: glmGamPoi` is optional. The default stays `glm_nb` (`MASS::glm.nb`), and its results are unchanged. - **Test:** inference is glmGamPoi's own quasi-likelihood F-test (`glm_gp` + `test_de`), not the trend-plus-Wald plan from Oct 15. - **Review fixes:** theta is `1 / overdispersions` (glm_gp's NB overdispersion), and the Poisson boundary is read from it, as requested. The contrast is now passed as numeric weights. Before that, any label containing `-` or `/` (e.g. `grpB/run_2`) made every taxon fail, and label text was evaluated as R code. - **Dependencies:** `renv.lock` adds glmGamPoi 1.22.0 and the eleven packages it needs; existing pins are unchanged. - **Evidence:** in a null simulation, the share of p < 0.05 was: | samples per group | `glm_nb` (Wald) | glmGamPoi (QL F) | |---|---|---| | 4 | 13.2% | 4.6% | | 12 | 6.9% | 4.8% | The `glm_nb` excess at small n is outside this issue's scope and is worth its own issue. **Good to go.** The R/RCall route is settled, and #62 is ready for re-review once the three new commits are signed and pushed. ~~~~ </details> --- _Generated by [Claude Code](https://claude.ai/code/session_01HtyPfZ1MWBB94RawgzQgz7)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things in one PR, because they have one cause: the layer that claimed to compute
TSS/CSS/RSS offsets did not, and the layer that accepted them refused them.
1. The refusal — main is red because of this
NormalizationConfigcanonicalises its method to lower case ("TSS"→"tss"), whileVALID_NORMALIZATION_FOR_METHODlisted"TSS","CSS","RSS". The membership testtherefore failed for those three spellings and
AnalysisConfigraisednormalization.method 'tss' incompatible with method 'nb_glm'— including for theexisting
test/unit/test_execution.jltestset TSS offset for NB_GLM, which constructsmethod="TSS"exactly as the documentation instructs. That is the failure behind the redJulia testsjob on the TSS-offsets commits; it is not a flaky test and it is not thenew estimation layer.
The table and both comparisons (constructor,
validate_config) are lower case now, theJSON-schema enum still accepts the upper-case spellings for existing documents, and
test/unit/test_scaling.jlasserts that every admissible spelling of every method nameis accepted.
2. The wiring — issue #16's offsets are computed, not aliased
prepare_analysis_tablenow computes scaling factors throughsrc/analysis/scaling.jland hands the offset to the fit:
tss/none(count model)log(library size)csslogcumulative sum at the declaredcss_quantilerss(TMM)logtrimmed weighted mean of log-ratios to a reference samplesize_factorslogmedian-of-ratios (RLE) — the estimator the name claimedrelativePreviously
css/rsswere aliased torelativeunder@warn,tssdid that for everymethod but
nb_glm, andsize_factorscomputed library-size centring wearing DESeq2'sname.
css/rssare now refused for a response with no counts to offset — in theconfiguration layer and again in the execution path.
css_quantile(0.75),tmm_ref_column(chosen the way edgeR chooses it, and recorded),tmm_log_ratio_trim(0.3) andtmm_sum_trim(0.05) are validated in the constructor,included in the config hash and canonical JSON, round-tripped through
to_json/from_json/Nickel/DEED, mirrored in the JSON schema and Nickel contracts,exposed through the server's configuration route, and explained in
context_helpand thefrontend help. A run that declared a different quantile is a different run.
diagnostics.checks["scaling"]and the manifest provenance record the kind, thedefinition, the parameters, the raw quantities before centring and a SHA-256 of the
offset vector.
Evidence
test/unit/test_scaling.jl: known answers with the arithmetic written beside them;properties a wrong implementation fails (feature permutation, a sample that is an exact
multiple of another, invariance to a shared constant); the CSS outlier-robustness
property; negative controls for every refusal (zero-total sample, zero CSS cumulative
sum, unknown reference, disjoint supports, out-of-range trims); the integration path
through
prepare_analysis_table; and a base-R transcription of the same conditions,which skips loudly by name where R is unreachable.
docs/statistics/method-conditions/scaling-and-offsets.md.check-spdx.shOK (284 files),check-format.shOK (331 files),check-lint.shOK (tsc --noEmitclean overfrontend/src).Not claimed
Parity with
metagenomeSeq::cumNormandedgeR::calcNormFactorsat 1e-6 is notverified: neither package is in
renv.lock, and this PR does not add an unpinned Rdependency to make a test pass. The base-R transcription catches implementation slips, not
specification errors. That outstanding condition stays recorded in issue #16 and in the
conditions document rather than being asserted as met.
Closes #16.