Repository navigation
Land #217 (exact DuckDB counts) on main, and fix the Wald tail - #226
Conversation
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui # Conflicts: # frontend/src/api/types.ts # frontend/src/components/PipelineStages.test.ts # frontend/src/components/differentialTable.ts # frontend/src/views/StudyView.tsx # test/unit/test_numeric_policy.jl Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com>
Resolve the conflicts between fix(stats): aggregate DuckDB counts exactly (arena/dd990535) and main since 7bb716d. - src/server/routes/validated_statistics.jl: keep the PR's implementation. Main's #218 fixed the same 500 with an ORDER BY + LIMIT window and a counts-only token scan; the PR supersedes it with a streamed query that bounds features/cells as rows arrive and with a whole-body integer-token scan, so the file is taken from the PR unchanged. - test/unit/test_validated_statistics.jl: union of both sides. Main's (1.0, 1.5, -1, 2^53+1) Dict cases and decimal/exponent raw token cases are kept alongside the PR's byte-backed request-body cases and its 8.0 policy rejection. - test/unit/test_exact_summaries.jl: main's wording, same assertions. - .github/workflows/ci.yml: main's log path and error title, with the PR's ordering that runs the validated-statistics suite first. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
#222 fixes the same CladeCumulus defects as this PR's d5f8060 and adds the route/loading repairs on top, so the clade conflicts take main's wording and main's extra fixes. The PR's remaining semantic repairs are kept because main still carries them: - `PhyloNode.edge_number` in `_graft_jplace_queries` (main still reads the non-existent `edge_num` field there); - the connecting branch in `_reaggregate_after_move`, so a subtree PD agrees with `clade_faith_pd`. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
`_wald` called `erfc`, which `CompositionalModels` never imported or
defined: every multinomial and Dirichlet-multinomial fit raised
`UndefVarError`, so five Differential testsets errored on any tree where
they could load at all. The module deliberately carries its own special
functions ("so the models add no dependency"), so `_erfc` follows the same
pattern: erfc(t) = Q(1/2, t^2), the power series below the switch and the
continued fraction above it, in the modified-Lentz form. Measured against
libm's erfc over t in [0, 30], the worst relative error is 1.5e-13, and
the two-sided tail reproduces the p-value reference values in
test/unit/test_differential.jl to 1.5e-12.
Also fix a test that had never been run to completion: it passed
`min_prevalence` as a keyword to `DifferentialConfig`, which takes it
positionally (as the route does).
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 58 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Fork governance
🟣 critical · commit
Hypatia scan: 133 finding(s)
panic-attack assail: 19 finding(s)
pons asinorum: 10 finding(s)
rhodibot (RSR compliance): not_run — enrolled, but no report-back was received
panicbot (panic recovery, error handling): not_run — enrolled, but no report-back was received
echidnabot (formal verification: proofs/): not_run — enrolled, but no report-back was received
seambot (integration health: API routes): not_run — enrolled, but no report-back was received
finishingbot (release readiness): not_run — enrolled, but no report-back was received
accessibilitybot (WCAG on the React UI): not_run — enrolled, but no report-back was received
glambot (presentation quality): not_run — enrolled, but no report-back was received
cipherbot: not_applicable — no cryptography is implemented in this repository
sustainabot: not_applicable — fleet adapter slot reserved; no adapter exists yet (gitbot-fleet README)
Scope: static findings reported by each row's tool at the named commit. Purple, red, yellow and blue grade the worst finding present; orange is evidence missing, malformed or a scanner that did not run; grey is an explicit scope exclusion with its reason. No row is a security certification. report.json, badge.json and the raw findings are in the verdict-grid artifact on the linked run. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fe57d8e184
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| end | ||
| # Match clade_faith_pd: an active non-root clade includes the branch | ||
| # connecting it to its parent as well as its active descendant branches. | ||
| any_active && !is_root && (sub_pd += max(0.0, u.branch_length)) |
There was a problem hiding this comment.
Avoid double-counting internal branches after a move
When an accepted drag-and-drop proposal has an active internal child, this addition puts that child's incoming branch into rk.subtree_pd, but the parent still adds rk.branch_length + rk.subtree_pd at line 1820. Consequently every active non-root internal edge is counted twice in the proposed tree—for example, a 0.3 internal branch with a 0.1 tip branch contributes 0.7 at the root instead of 0.4—corrupting the displayed and persisted subtree PD. The recursion needs one consistent invariant: either child subtree_pd includes its incoming branch and parents add only that value, or parents remain responsible for adding child branches.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Correct — and now fixed in 0cb9f37. The addition at the node and the parent's rk.branch_length + rk.subtree_pd were two conventions in one pass, so every active non-root internal edge was counted twice.
subtree_pd is now node-local throughout, which is the convention clade_faith_pd already uses in the build path: a leaf puts its own branch into the value it returns, and the parent adds only rk.subtree_pd. Verified by running both recursions over 800 random trees — the patched form agrees with clade_faith_pd exactly, the previous form disagreed on 409 of them.
Codex review (P2) on #226: `_reaggregate_after_move` added a clade's own branch at the node and its parent also added `rk.branch_length`, so every active non-root internal edge was counted twice in a proposed tree. Make `subtree_pd` node-local throughout, which is the convention the build path already uses (`clade_faith_pd` adds a node's own branch where the node is visited): a leaf's own branch goes into the value it returns, and the parent adds only `rk.subtree_pd`. Checked against `clade_faith_pd` on 800 random trees, the patched recursion agrees exactly; the previous parent-local form disagreed on 409 of them. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
The empty-composition case built a four-sample table but compared it against `groups8[1:4]`, which is every sample in group 'A', and it passed `min_prevalence` where the constructor takes it positionally. Both errors were masked by the `erfc` UndefVarError that aborted the testset before this block ran; with that fixed, CI reports the group error instead of the prevalence one. Use all eight samples with a table in which no taxon appears in every sample, so min_prevalence = 1.0 leaves nothing to fit and the engine raises "no composition to model". Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
|
…pe widening, and route deduplication Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Main landed #226 while this branch was in review, and the two found three of the same defects independently. Main's resolution is taken for all three, since it is already on main: - `erfc`: main's private `_erfc` replaces the `compositional_erfc` added here. Same algorithm (incomplete gamma at a = 1/2, series below x^2 = 1.5 and a Lentz continued fraction above), so only the name and the tests differ; `compositional.jl` and `test_differential.jl` are taken wholesale from main. - `subtree_pd` in `_reaggregate_after_move`: identical fix, main's `own_pd` naming and one-line post-loop form kept. Git had merged the two variants additively, which would have counted every active internal branch twice; that is resolved to a single addition. - The `Main.SV` collision that was aborting the include loop: main renames this file's binding to `CLADE_SV` instead of dropping `const`. Main's rename is kept. - `min_prevalence` in the multinomial engine test: main's eight-sample table is better than the four-sample one here, and is kept. Kept from this branch, because main does not have them: - The raw like-weight ratio in the diversity block. Main's change to that line only renames `c.weight` to `c.like_weight` and keeps `cand.like_weight / tw`, which is the renormalisation Codex objected to on #223: it hands a lone 0.6 placement a weight of 1.0. The divisor is removed, and the mass the tool never placed is reported as `unplaced_reads`. - `_sample_total` and the `total_a`/`total_b`/`totals` arguments on `weighted_unifrac`, `branch_unifrac_contributions` and `unifrac_matrix`, with the route passing each sample's library size. Without them the metric divides by the placed-only mass and undoes the line above. - Per-file testsets in `runtests.jl`. Main fixed the `SV` collision but not the reason one bad binding could silently skip thirty-five unit files and the occupancy probe. - `_cc_find`, and the drag-and-drop assertions that pin the exact post-move PD values and total-PD conservation rather than comparing a positional index against an inequality.
Main landed #226, which contains equivalents of this branch's _erfc implementation, min_prevalence test fix, edge_number graft fix and stem-inclusive subtree PD, plus #217's validated-statistics implementation and the CLADE_SV rename that removes the Main.SV const collision. Conflicts resolved in favour of main for those; this branch keeps its unique additions: AbstractString tree helpers and the structured clade_cumulus_failed catch-all with the like_weight diversity fixes. Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>



What this is
#217 (
fix(stats): aggregate DuckDB counts exactly) has beenCONFLICTINGsince main moved: #218 fixed the same run-summaries 500 independently, #210
repaired the differential/analysis modules, #220 removed the Evidence UI, and
#222 repaired the CladeCumulus tree/route code. This branch merges the current
main into #217's head, resolves every conflict, and repairs the two main-line
defects that were failing the required
Julia testscheck on both branches.It contains #217's content in full, so merging it lands #217.
Conflict resolution (4 files)
src/server/routes/validated_statistics.jl— the PR's implementation iskept unchanged. Main's Fix validated-statistics run summaries 500 (invalid GROUP BY aggregation) #218 kept a
GROUP BY-freeORDER BY … LIMIT MAX_CELLS+1window and scanned only thecountsmember for float tokens;fix(stats): aggregate DuckDB counts exactly #217 streams the raw rows, bounds features/cells as rows arrive, and scans
the whole direct body. Nothing main added after 7bb716d was lost: the two
versions differed only in those four hunks.
test/unit/test_validated_statistics.jl— union. Main's(1.5, -1, 2^53+1)Dict cases and its
1.0/1e0raw-token loop are kept next to fix(stats): aggregate DuckDB counts exactly #217'sbyte-backed
Vector{UInt8}cases and itsround_digits: 8.0rejection, soboth sides' contracts are now tested.
test/unit/test_exact_summaries.jl— main's wording; same assertions..github/workflows/ci.yml— main's log path/title with fix(stats): aggregate DuckDB counts exactly #217's ordering thatruns the validated-statistics suite first.
The CladeCumulus merge takes main's #222 implementation (real tables only
via
duckdb_tables(),SubStringnormalisation at the request boundary, NHXcapture,
PLACEMENT_WEIGHT_SUM_TOL = 0.05) and keeps the two repairs #222still lacks:
PhyloNode.edge_numberin_graft_jplace_queries(main reads afield that does not exist) and the connecting branch in
_reaggregate_after_move, so a subtree PD agrees withclade_faith_pd.Two main-line fixes needed to get the required checks green
UndefVarError: erfc—CompositionalModels._waldcallederfcwithout importing or defining it, so every multinomial / Dirichlet-
multinomial fit raised on any tree where it could load. The module states it
carries its own special functions so the models add no dependency, so
_erfcis added in the same style:erfc(t) = Q(1/2, t²), power seriesbelow the switch, continued fraction above it (modified Lentz).
DifferentialConfig(…, min_prevalence=1.0)— the test passedmin_prevalenceas a keyword; it is positional in the constructor and inthe route that builds it. Fixed in the test.
Validation
column "s1" must appear in the GROUP BY clause…; the streamed query returnsG → [3, 2^127],H → [4, 1], and theprefix=s2+Pident ≥ 99filterreturns
s2 → 2^126, exactly the values the tests assert.bodies:
1.0,1e0,3.0e-1andround_digits: 8.0are refused; integertokens, quoted
"1.0", escapes and digits inside strings are accepted; therun route (which legitimately takes
Pident: 99.0) is not scanned._erfcwas checked against libm'serfcont ∈ [0, 30]: worst relativeerror 1.5e-13, and the two-sided tail reproduces this file's p-value
references to 1.5e-12 (the test asserts
rtol = 1e-6).src/**.jlandtest/**.jlfiles parse with a Julia grammar; thethree files that still show grammar nodes (
remote_exec.jl,occupancy_disagreement_test.jl,test_determinism.jl) are untouched here.git diff --checkclean;.github/workflows/ci.ymlparses as YAML.is blocked), so the pinned suite has to be the authority: this PR exists to
give CI the merged tree to run on.
Relationship to #217
Merging this lands #217's content. To make #217 itself mergeable instead, its
head must be advanced to this branch; that is a single command for someone who
can push to it: