Repository navigation
fix(analysis): the analysis path stops returning placeholder statistics - #60
Merged
Merged
Conversation
run_analysis derived its p-values from 0.01 + (hash(taxon_id) % 100)/1000.0, padj from pvalue * 1.5 and log2FoldChange from (hash % 20)/10 - 1: every one of those is a function of the feature's NAME, not of the counts, and they were returned to callers as results. The placeholder is deleted rather than deprecated. src/analysis/estimation.jl runs the declared model for real (MASS::glm.nb with a required offset for nb_glm, stats::lm for CLR/ILR, stats::glm binomial for logistic on a 0/1 response), records R and MASS versions, offset hash and result hashes, excludes failed fits from the BH family while counting them, and returns an explicit not_run with a reason — never an empty table that reads as 'nothing was significant' — when no design or no runtime is available. Conditions published in docs/statistics/method-conditions/parametric-fits.md (catalogue item 2). Evidence in test/unit/test_estimation.jl: written-in known effects, a direct R reference compared coefficient by coefficient, BH against R's p.adjust, negative controls for every refusal, and a source-level guard that the placeholder does not come back. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
arena-ai-coding-agent
Bot
requested a review
from hyperpolymath
as a code owner
September 25, 2026 20:31
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
hyperpolymath
approved these changes
Sep 25, 2026
hyperpolymath
added a commit
that referenced
this pull request
Sep 25, 2026
## Why Julia tests on `main` have been red since #60, and the reason was never visible: the job log is served from a blob host that some environments cannot reach, and the one annotation the step posted (the last 6000 bytes of output) was truncated by the checks API at 4096 bytes — which cut it off exactly before the failing testset. For `e26d6f2` (#63) all it shows is that `Scaling — TSS, CSS, RSS/TMM and size factors (issue #16)` has a failure. ## What The "Run tests" step, on failure only, now posts: - one `::error` annotation per failing assertion (the Test.jl block starting at `Test Failed at` / `Error During Test at`, up to 60 lines / 3500 bytes, max 8), and - one summary annotation holding only the `Test Summary` rows whose Fail or Error column is non-empty (found by the header's column positions), plus the `ERROR:` line. If no failure block is found (e.g. a load error), the old tail annotation is posted as a fallback. The test command and its exit status are unchanged. ## Verified - YAML re-parses; spdx/format/blob-hygiene gates clean. - The extraction shell was run against a synthetic Test.jl log (one `Test Failed`, one `Error During Test` with `Got exception outside of a @test`, nested summary) and produced exactly two failure annotations and a summary with only the failing rows. ## Not claimed This does **not** fix the red test. It makes the failure readable so the next PR can fix it rather than guess. This PR's own CI run should fail the same way as `main` and name the assertion. Co-authored-by: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com>
hyperpolymath
added a commit
that referenced
this pull request
Oct 17, 2026
… work (#178) ## Purpose Make the fork's anticipated upstream state concrete while Joshua's reviews/merges remain pending. This is a **draft baseline proposal**, not evidence that upstream has merged anything. The proposed file tree is exactly JoshuaJewell/main at `52feb605176a4b140f4b40c2b74d2f18adec1f72` plus: - upstream #59 at `3c9d352e1f6c85353c2d78ef1ebf28e5fb64c263` - upstream #60 at `ea974a2328f7f44ad9afeb2f10e5803cd4e31e62` - upstream #61 at `d9ea982dc97c958405e1ff6f57f1b0d8d9b221d1` Resulting tree: `b27daa7f1aa5fa84f320ddae522fa5f2a15ebef5`. ## Effect on fork main This deliberately removes fork-only content from the proposed active baseline: 74 files differ from current main. The existing main commit `491a3e0702ca8ca40e411860d6166523b298b3ac` is the first parent, and all three repair heads are additional parents, preserving the work in history. No existing branch is deleted or rewritten. Do not submit this integration branch upstream. **Before merging:** decide whether main should be the upstream-aligned baseline, with unfinished work isolated onto feature branches. This PR removes real fork features from the active tree, not just stale merge metadata. Recheck upstream and repair heads after Joshua's final decisions; their eventual file tree may change. ## Branch audit There were seven total branches when inspected: main and six additional branches. | Branches | Finding | Disposition | | --- | --- | --- | | repair/permanova-proofs | Clean pending upstream #59 | Keep until merged | | repair/documentation | Clean pending upstream #60 | Keep until merged | | repair/dada2-base-r | Clean pending upstream #61 | Keep until merged | | upstream/permanova-proofs, upstream/docs-community, upstream/dada2-base-r | All three have identical tree 2361ab8; old contaminated submissions, not independent WIP | Retire after baseline decision; no unique feature work needs completing on these branches | | main | Contains additional work beyond the repaired upstream contributions | Preserve and isolate selected changes | Compared with the old three branches, main changes only .gitignore, the differential fixture, the restored deferred-issues specification and removal of the duplicate root code of conduct. Their substantive work is preserved. ## Genuine work to isolate and finish These are code-review findings, not claims that the features have passed a fresh full validation. | Work | Evidence / remaining work | | --- | --- | | glmGamPoi dispersion | Actual implementation and configuration exist, but glmGamPoi is absent from renv.lock. Existing tests do not establish successful reference fits through this path. Pin dependencies and validate against trusted R fits before treating it as complete. | | Significance proof model | The Agda model makes noncomputed numbers unrepresentable, while the Julia documentation explicitly permits retained pairwise rows after an omnibus failure. Reconcile model and implementation; do not claim the proof establishes the Julia representation's behavior. | | Canonical JSON/JCS | Utility and fixtures exist; no production callers found beyond module inclusion/export. Establish conformance and choose an actual integration point before claiming end-to-end provenance. | | Memory occupancy | Measurement, algebra/proof integration, tests and scripts exist. Isolate the work, verify sibling dependency pins and real runtime measurements. This is distinct from biological occupancy modelling in issue #18. | | Wildcard-IP origin handling | Separate behavioral change in server middleware. Review address/port parsing and trust assumptions, then run host/origin, DNS-rebinding and CSRF regressions before submission. | | Plot title markup stripping | Small independent helper and usage change; suitable for a focused bugfix after checking its tests and behavior. | Repository layout changes, frontend major upgrades, community policies, automation and other tooling are separate fork-maintenance decisions, not prerequisites for the three upstream repairs. Do not carry forward the obsolete claim in fork AGENTS.md that locked ggplot2 requires tibble: the retained tibble dependency is used by installation/CI tooling. Open planning issues are not evidence of implemented branch WIP; several overlap or describe functionality already upstream. ## Validation - All six repair commits applied together without conflicts. - git diff --cached --check passes. - Local combined tree and GitHub-created tree match exactly. - Latest individual PR CI runs passed: #59 run 38976014775; #60 run 38998820680; #61 run 38998823755. - Combined baseline CI is separate and must be inspected before merging. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Joshua Jewell <developer@joshuajewell.dev> Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem this fixes
Execution.run_analysisreturned, per feature:Every number is a deterministic function of the feature's name. Nothing in the
run touches the counts. The rows render in a table as p-values, fold changes and
means, and a caller has no way to tell them from a fit. The source called it a stub;
the caller received it as a result.
What replaces it
src/analysis/estimation.jl(catalogue item 2 — conditions published before theimplementation, in
docs/statistics/method-conditions/parametric-fits.md):nb_glmMASS::glm.nb, per featureglmGamPoi/local/mean/pooledrefused by name (issue #21)clr_lm,ilr_lmstats::lm, per featurelogisticstats::glm(binomial), per featurestatus = "failed", anotesaying why and
nullfor every statistic; it is excluded from the BH family andcounted. Nothing that cannot be fitted gets a number.
grouping,
RBusyErrorpast the timeout, R unreachable →status = "not_run"with areason and an empty result set, and a warning on the diagnostics. An empty result is
never dressed up as "no significant features".
in a formula,
$/backticks/;in a formula, offsets on non-count models, count modelswithout one, binomial fits on proportions.
offset kind and its SHA-256, BH family size, exclusion count, and the SHA-256 of the
fits and coefficients tables as they crossed the R boundary.
p.adjust(method = "BH").Evidence (
test/unit/test_estimation.jl)than recorded from a previous run.
coefficient, SE by SE, p by p.
not_run/failedshapes.hash(taxon_id),hash(andrand(are gone from theexecution path.
Refs #1 (catalogue item 2 delivered; item 3's nonparametric tests and the independent
statistical review are still outstanding there). Issue #21 stays open:
glmGamPoiisrefused rather than aliased, and the refusal says so.