Skip to content

fix: resolve PR #219 conflicts and address CladeCumulus & validated-statistics findings - #228

Closed
hyperpolymath wants to merge 13 commits into
mainfrom
arena/7805092e-metamanifold-webui
Closed

hyperpolymath wants to merge 13 commits into
mainfrom
arena/7805092e-metamanifold-webui

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Change and reason

Resolves the merge conflicts on PR #219 (fix: restore validated statistics and CladeCumulus CI) against main and addresses all review and CI findings raised by apps on PR #219:

  • CladeCumulus (src/analysis/clade_cumulus.jl, src/server/routes/clade_cumulus.jl):
    • Reconciles read_label_and_quoted(capture_nhx=..., capture_edge=...) and PLACEMENT_WEIGHT_SUM_TOL = 0.05 from main with PR fix: restore validated statistics and CladeCumulus CI #219's PhyloNode.edge_number fixes in _graft_jplace_queries and PlacementCandidate.like_weight in /api/v1/phylo/validate-jplace.
    • Preserves PR fix: restore validated statistics and CladeCumulus CI #219's fix for CodeRabbit's review finding on _reaggregate_after_move so nested-clade drag-and-drop counts each active non-root branch once (sub_pd += rk.subtree_pd plus incoming branch on active non-root clades) matching clade_faith_pd.
    • Opens results.duckdb via with_results_db (readonly=true) in _load_run_table_df and /api/v1/studies/{study}/runs/{run}/clade-cumulus/sources while filtering duckdb_tables() WHERE NOT internal and converting SubString route parameters (tree_file, table) to String for _valid_tree_file/_valid_name.
  • Validated statistics (src/server/routes/validated_statistics.jl):
    • Consolidates raw JSON count-token checking in _statistics_body(req) via _reject_float_count_tokens(raw) so integral decimal/exponent tokens (1.0, 1e0) fail closed before JSON3.read.
    • Addresses CodeRabbit's review comment on _validated_duckdb_counts by checking the SELECT COUNT(*) AS n row/cell budget (MAX_CELLS) before materialising raw rows into a DataFrame, then aggregating per-feature counts in BigInt.
  • Tests and CI (test/unit/test_clade_cumulus.jl, test/unit/test_exact_summaries.jl, test/unit/test_validated_statistics.jl, .github/workflows/ci.yml):
    • Fixes LoadError: invalid assignment to constant Main.SV in test/unit/test_clade_cumulus.jl by binding SV = MetaManifold.Server (non-const, matching the other test files included in Main) and const VS_SERVER = MetaManifold.Server in test/unit/test_validated_statistics.jl.
    • Checkpoints and closes the test DuckDB.DB (CHECKPOINT + DBInterface.close!(con) + DBInterface.close!(db)) before computing before_db_sha so read-only CladeCumulus routes leave results.duckdb byte-for-byte identical.
    • Preserves structured GitHub Actions error annotations in Run validated-statistics tests first.

Validation

  • Verified zero conflict markers and clean whitespace formatting with git diff --check.
  • Verified independent high-precision decimal references with python3 test/references/check_high_precision_references.py --check test/unit/test_high_precision.jl.
  • Verified .github/workflows/ci.yml shell step syntax with bash -n.
  • CI workflow triggered on branch arena/7805092e-metamanifold-webui.

Scope

Contribution checks

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits include Signed-off-by: (Developer Certificate of Origin) or I agree to license my contribution under the project license (AGPL-3.0-only)

hyperpolymath and others added 13 commits October 23, 2026 00:11
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
…nifold-webui

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 33 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: d06774cf-19ec-42d9-b958-7a32440114c7
📥 Commits

Reviewing files that changed from the base of the PR and between 8df1041 and b501c01.

📒 Files selected for processing: 6
  • .github/workflows/ci.yml
  • src/analysis/clade_cumulus.jl
  • src/server/routes/clade_cumulus.jl
  • src/server/routes/validated_statistics.jl
  • test/unit/test_clade_cumulus.jl
  • test/unit/test_validated_statistics.jl
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T07:52:15.106938Z b501c01 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

Fork governance

Lane Status Run
DEED syntax (repo deed) 🟢 conformant run 39920290745
Hypatia verdict grid purple (job: success) same run

Hypatia grid purple

🟣 critical · commit b501c01c259b · run 39920290745

Reporter Status Critical High Medium Minor Since previous revision
Hypatia scan 🟣 critical 43 59 31 0 — no comparable baseline
panic-attack assail 🔴 high 0 6 13 0 — no comparable baseline
pons asinorum 🟡 medium 0 0 10 0 — no comparable baseline
rhodibot (RSR compliance) 🟠 indeterminate 0 0 0 0 — no comparable baseline
panicbot (panic recovery, error handling) 🟠 indeterminate 0 0 0 0 — no comparable baseline
echidnabot (formal verification: proofs/) 🟠 indeterminate 0 0 0 0 — no comparable baseline
seambot (integration health: API routes) 🟠 indeterminate 0 0 0 0 — no comparable baseline
finishingbot (release readiness) 🟠 indeterminate 0 0 0 0 — no comparable baseline
accessibilitybot (WCAG on the React UI) 🟠 indeterminate 0 0 0 0 — no comparable baseline
glambot (presentation quality) 🟠 indeterminate 0 0 0 0 — no comparable baseline
cipherbot ⚪ not applicable 0 0 0 0 — no comparable baseline
sustainabot ⚪ not applicable 0 0 0 0 — no comparable baseline
Hypatia scan: 133 finding(s)
  • critical cicd_rules/banned_language_file — frontend/src/views/run/parseFilterYaml.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/encode.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/charts.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/export.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/figure.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/legend.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/figure/types.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/download.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/text.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/timeago.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/utils/text.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useUnsavedGuard.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useApi.test.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useApi.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useAnalysis.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useSSE.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useTabParam.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useNavRefresh.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/hooks/useJobEvents.ts — TypeScript file detected -- banned language
  • critical cicd_rules/banned_language_file — frontend/src/about.ts — TypeScript file detected -- banned language
  • 113 more; see the structured report in the artifact.
panic-attack assail: 19 finding(s)
  • high DynamicCodeExecution — src/pipeline/dada2/context.jl — eval/Meta.parse in src/pipeline/dada2/context.jl
  • high DynamicCodeExecution — src/analysis/analysis.jl — eval/Meta.parse in src/analysis/analysis.jl
  • high DynamicCodeExecution — src/analysis/differential.jl — eval/Meta.parse in src/analysis/differential.jl
  • high DynamicCodeExecution — src/core/provenance.jl — eval/Meta.parse in src/core/provenance.jl
  • high DynamicCodeExecution — scripts/check-julia-syntax.jl — eval/Meta.parse in scripts/check-julia-syntax.jl
  • high DynamicCodeExecution — test/unit/test_permanova.jl — eval/Meta.parse in test/unit/test_permanova.jl
  • medium UnsafeFFI — src/core/memory_probe.jl — 3 ccall/FFI calls in src/core/memory_probe.jl
  • medium InputBoundary — src/server/routes/runs.jl — 2 JSON3.read/JSON.parse call(s) with 0 try block(s) in src/server/routes/runs.jl — these throw on malformed input; wrap in try/catch
  • medium InputBoundary — src/server/routes/composition.jl — 6 JSON3.read/JSON.parse call(s) with 2 try block(s) in src/server/routes/composition.jl — these throw on malformed input; wrap in try/catch
  • medium InputBoundary — src/server/routes/studies.jl — 4 JSON3.read/JSON.parse call(s) with 2 try block(s) in src/server/routes/studies.jl — these throw on malformed input; wrap in try/catch
  • medium InputBoundary — src/server/routes/analysis.jl — 9 JSON3.read/JSON.parse call(s) with 4 try block(s) in src/server/routes/analysis.jl — these throw on malformed input; wrap in try/catch
  • medium UnsafeFFI — install.jl — 1 ccall/FFI calls in install.jl
  • medium ProofDrift — test/unit/test_occupancy_grade.jl — 2 `@test x isa Y` assertion(s) with no value check — type-only tests may substitute for an unwritten formal theorem in test/unit/test_occupancy_grade.jl
  • medium ProofDrift — test/unit/test_evidence.jl — 1 `@test x isa Y` assertion(s) with no value check — type-only tests may substitute for an unwritten formal theorem in test/unit/test_evidence.jl
  • medium InputBoundary — bench/advanced_zero_handling/benchmark.jl — 1 JSON3.read/JSON.parse call(s) with 0 try block(s) in bench/advanced_zero_handling/benchmark.jl — these throw on malformed input; wrap in try/catch
  • medium InputBoundary — frontend/src/figure/charts.ts — 1 JSON.parse call(s) with 0 try block(s) in frontend/src/figure/charts.ts — JSON.parse throws SyntaxError on malformed input; wrap in try-catch
  • medium InputBoundary — frontend/src/tree/model.ts — 1 JSON.parse call(s) with 0 try block(s) in frontend/src/tree/model.ts — JSON.parse throws SyntaxError on malformed input; wrap in try-catch
  • medium InputBoundary — frontend/src/api/events.ts — 2 JSON.parse call(s) with 0 try block(s) in frontend/src/api/events.ts — JSON.parse throws SyntaxError on malformed input; wrap in try-catch
  • medium InputBoundary — frontend/src/components/alphaMetrics.tsx — 1 JSON.parse call(s) with 0 try block(s) in frontend/src/components/alphaMetrics.tsx — JSON.parse throws SyntaxError on malformed input; wrap in try-catch
pons asinorum: 10 finding(s)
  • medium string-concat-in-loop — ./frontend/src/tree/model.ts:122 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/model.ts:118 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:69 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:66 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:61 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:53 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:43 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium string-concat-in-loop — ./frontend/src/tree/TreeShapes.tsx:42 — string built by repeated concatenation in a loop (T0, HEURISTIC)
  • medium unreachable-after-jump — ./frontend/src/figure/PaneEditor.tsx:45 — statement is unreachable after an unconditional jump (T0, HEURISTIC)
  • medium unreachable-after-jump — ./frontend/src/components/PublicationTablesPanel.tsx:208 — statement is unreachable after an unconditional jump (T0, HEURISTIC)
rhodibot (RSR compliance): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
panicbot (panic recovery, error handling): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
echidnabot (formal verification: proofs/): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
seambot (integration health: API routes): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
finishingbot (release readiness): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
accessibilitybot (WCAG on the React UI): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
glambot (presentation quality): not_run — enrolled, but no report-back was received
  • 🟠 indeterminate: enrolled, but no report-back was received
cipherbot: not_applicable — no cryptography is implemented in this repository
  • ⚪ not applicable: no cryptography is implemented in this repository
sustainabot: not_applicable — fleet adapter slot reserved; no adapter exists yet (gitbot-fleet README)
  • ⚪ not applicable: fleet adapter slot reserved; no adapter exists yet (gitbot-fleet README)

Scope: static findings reported by each row's tool at the named commit. Purple, red, yellow and blue grade the worst finding present; orange is evidence missing, malformed or a scanner that did not run; grey is an explicit scope exclusion with its reason. No row is a security certification. report.json, badge.json and the raw findings are in the verdict-grid artifact on the linked run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b501c01c25

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/analysis/clade_cumulus.jl
@hyperpolymath
hyperpolymath enabled auto-merge (squash) October 23, 2026 07:54
@hyperpolymath
hyperpolymath enabled auto-merge (squash) October 23, 2026 13:51
@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Add Carrot credits or activate Agent usage billing to use Autofix

Copy link
Copy Markdown
Owner Author

Resolution: this branch is fully superseded by main, and the one open finding is fixed in #244

I went through every app comment on this pull request and re-ran the conflict
resolution against current main (d69d4f9). Summary of where each thread
landed:

Merge conflicts - resolved, and the result is a no-op

This branch's merge base is 8df1041, and all six files it touches have since
been rewritten on main (#231, #233, #234, #236, #237, #239). Resolving the
six conflicted files in favour of main yields a tree that is byte-for-byte
identical to main
- verified with git diff --stat origin/main after the
merge commit - so there is nothing left here to land.

File Status on main
.github/workflows/ci.yml richer per-assertion digest of failing validated-statistics tests + log tail
src/analysis/clade_cumulus.jl PhyloNode.edge_number already used in _graft_jplace_queries; _reaggregate_after_move already adds rk.subtree_pd + the node's own branch, and rebuilds emptied clades from direct_count
src/server/routes/clade_cumulus.jl with_results_db and String(strip(...)) already in place; like-weight renormalisation deliberately not applied (see #236/#237: unplaced mass is reported as unplaced_reads)
src/server/routes/validated_statistics.jl whole-body raw-token validation after ijson_read, and streamed bounded rows instead of a materialised DataFrame - both supersets of this branch
test/unit/* main's suites are much larger; the one test this branch had that main lacked ("Nested-clade drag-and-drop counts each active branch once") is restored in #244

App findings

  • Codex P2 - "Keep leaf subtree PD consistent after a move": this one was
    still open against main, and it was right. _reaggregate_after_move
    credits an active leaf with its own incoming branch while build_node
    hard-coded 0.0, so validating a proposal changed the PD of tips whose
    topology had not moved. Fixed in fix: resolve PR #228, unify leaf subtree PD, and answer malformed JSON with 400 #244: both builders now derive a leaf's PD
    from clade_faith_pd, with a regression test that asserts the node-local
    invariant and compares untouched tips across a proposal.
  • CodeRabbit: the review run on this PR was rate-limited ("Review limit
    reached") and the Autofix request returned "Add Carrot credits or activate
    Agent usage billing", so no new CodeRabbit findings were produced against
    this head. Its earlier findings on fix: restore validated statistics and CladeCumulus CI #219 (the nested-clade double count, the
    results.duckdb connection handling, the count-token checking) are all
    addressed on main, and the double-count fix now has a test.
  • SonarCloud: quality gate passed (0 new issues, 0 hotspots).
  • panic-attack: the InputBoundary finding was real - 21 unguarded
    JSON3.read(String(req.body)) sites turned a malformed body into a 500.
    They now go through Server._request_json and answer 400. The
    DynamicCodeExecution findings are false positives (RCall.reval evaluates
    R source strings, not Julia eval).
  • pons asinorum: the unreachable-after-jump hits point at plain
    arrow-function bodies, and the string-concat-in-loop hits at
    parts.push(...) / parts.join(''). No code change warranted.
  • Hypatia: 43 criticals are all cicd_rules/banned_language_file on
    frontend/src/**/*.ts, an upstream-authored React/TypeScript frontend. That
    rule's only carve-outs are narrow path prefixes (bindings/typescript/,
    .d.ts, vite.config.ts, ...) and it cannot be suppressed repo-side, so
    migrating it is an estate-level AffineScript project rather than a fix for
    this fork. The scan stays advisory here by design (scripts/check-governance-contract.sh).
  • DEED: green, and reproduced locally with the exact linter the reusable
    lane uses (deed_lint.js at hyperpolymath/standards@75bd5994). The
    stdlib Python port beside it is older and rejects #u8; the JS linter that
    CI runs accepts it, so the deed should not be "fixed" to satisfy the port.

Recommendation

Close this pull request as superseded. The conflict resolution is recorded in
#244 (merge commit 77376ef, tree identical to main) together with the
Codex P2 fix and the input-boundary hardening, so nothing from this branch is
lost by closing it.

Thanks @coderabbitai and @chatgpt-codex-connector - the subtree-PD finding in
particular was a real inconsistency that survived several rounds of review on
main.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -u
for number in 228 244; do
  gh pr view "$number" --repo hyperpolymath/MetaManifold-WebUI \
    --json number,state,url,title,baseRefName,headRefName,mergeCommit,mergedAt
done
fd -t f '^(clade_cumulus|validated_statistics|test_clade_cumulus|test_validated_statistics|test_exact_summaries)\.jl$'
for file in src/analysis/clade_cumulus.jl src/server/routes/clade_cumulus.jl src/server/routes/validated_statistics.jl test/unit/test_clade_cumulus.jl test/unit/test_validated_statistics.jl test/unit/test_exact_summaries.jl; do
  if [ -f "$file" ]; then
    ast-grep outline "$file" --items all 2>/dev/null | head -60
    rg -n -C 4 'clade_faith_pd|subtree_pd|with_results_db|unplaced_reads|_statistics_body|_reject_float_count_tokens|MAX_CELLS|Nested-clade|untouched|VS_SERVER|Main.SV' "$file" | head -120
  fi
done
for file in .github/workflows/ci.yml scripts/check-governance-contract.sh; do
  if [ -f "$file" ]; then
    printf '\n--- %s ---\n' "$file"
    sed -n '1,220p' "$file"
  fi
done

Length of output: 27888


@hyperpolymath Thanks for the detailed resolution report. Based on your reported tree comparison, closing #228 as superseded is appropriate.

I confirmed that #228 and #244 are both still open. The fixes in #244 are therefore not yet merged into main. Keep #244 as the active PR for the leaf-PD fix, restored regression test, and malformed-JSON handling.

I have not independently verified the tree equality, #244’s implementation, or its CI results. Before merging #244, verify those changes and its required checks. The fact that RCall.reval evaluates R rather than Julia does not alone establish safety; that assessment also depends on whether untrusted input can reach the R source string.

I have not closed #228 or changed its approval status.

You are interacting with an AI system.

Copy link
Copy Markdown
Owner Author

Superseded by #242, which is now fully green and mergeable as CLEAN on ed24c9c. The replacement fixes the leaf-PD inconsistency from the Codex review in 162545b, verifies every fixture tip before and after an accepted move, and adds exact nested-clade aggregation coverage in ed24c9c. CI, Julia/validated-statistics, governance/DEED/Hypatia, secret scanning, CodeQL, Semgrep, and Sonar all pass on the replacement. Closing this conflicting stale PR in favour of #242.

auto-merge was automatically disabled October 23, 2026 16:45

Pull request was closed

hyperpolymath added a commit that referenced this pull request Oct 23, 2026
## Change and reason

Supersedes #219 and the earlier reconciliation attempt #228. The
validated-statistics and CladeCumulus changes on those stale branches
have already landed through later work on `main`; conflict comparison
showed that taking either stale side would now revert newer strict
I-JSON, streamed `BigInt`, read-only DuckDB, tree-rooting, and
unplaced-read fixes. This branch is therefore based directly on current
`main` and carries only changes that are still needed.

- **CladeCumulus:** active non-root tips now use the same node-local PD
convention before and after drag-and-drop reaggregation. Regression
tests cover every fixture tip before and after an accepted move, plus
exact nested-clade totals (0.60 and 1.40) that reject parent-side
double-counting. This addresses the remaining Codex finding on #228 and
CodeRabbit's aggregation finding on #219.
- **CI:** the full Julia job has a 60-minute hang bound, and
validated-statistics diagnostics no longer use `|| true`; the captured
test exit status remains authoritative.
- **Frontend quality:** SVG path fragments are batched and joined once
rather than repeatedly concatenated, and `PaneEditor` uses the direct
hook dependency.
- **Security and conformance:** adds the missing security policy and
EditorConfig, plus pinned OpenSSF Scorecard and full-history
secret-scanning workflows. Scorecard retains least-privilege
default-branch SARIF upload without a self-referential PR alert. The
repository gitleaks config backports two upstream rules for
biological-data false positives and allowlists only ten exact public
schema/callback identifiers—no path or commit exclusions. The existing
DEED lane is conformant and unchanged.

## Validation

- `npx --yes bun@1.3.10 install --frozen-lockfile`
- `npx --yes bun@1.3.10 run typecheck`
- `npx --yes bun@1.3.10 test src` — **72 passed, 0 failed** (200
assertions)
- `npx --yes bun@1.3.10 run build`
- Parsed every workflow as YAML and every inline `run` block with `bash
-n`
- `bash scripts/check-governance-contract.sh` — all 20 checks passed
- `python3 test/references/check_high_precision_references.py --check
test/unit/test_high_precision.jl`
- `bash scripts/lint-sonar-reliability.sh`
- `git diff --check`
- Reproduced the legacy gitleaks rules over all 188 reachable commits:
211,428 biological/generated-code false positives reduced to zero;
synthetic AWS, Facebook, and generic credential canaries still produced
exactly three findings
- Fresh-head `Secret history and working tree / gitleaks` — **passed**
with the CI-pinned gitleaks 8.18.4
- [CI run
39986326202](https://github.com/hyperpolymath/MetaManifold-WebUI/actions/runs/39986326202)
on `ed24c9c` — **passed**: Julia 1.12.5 tests, validated statistics,
frontend build/tests, R/RCall checks, high-precision references,
benchmarks, Agda, and both occupancy lanes

## Deliberate review triage

- No baseline or exclusion was added to hide Hypatia findings.
- TypeScript remains the settled frontend stack and the independent
Python numerical oracles remain in place; the banned-language findings
are policy debt, not vulnerabilities introduced here.
- XML namespace `http://www.w3.org/...` strings were not changed to
HTTPS: they are identifiers, not network requests.
- Existing licence headers were left unchanged as `CONTRIBUTING.md`
requires.
- No placeholder CodeQL, Hypatia, mirror, or AI-manifest files were
added. CodeQL default setup and the Hypatia governance lane are already
active; external mirroring and the legacy A2ML-to-DEED migration require
explicit owner configuration/decisions.

## Scope

- [x] Does not alter dependencies, lockfiles, generated assets, or
vendored files
- [x] Secrets, local paths, and run outputs are not committed
- [x] Commits include `Signed-off-by:` (Developer Certificate of Origin)

---------

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>

hyperpolymath commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner Author

Independent audit and supersession of #228

Closing this stale, conflicting PR as superseded, not merging it and not claiming all outstanding work is on main.

Security/governance follow-up

The earlier claim that RCall.reval findings are false positives merely because the language is R is not sufficient. Inspection found actual source-construction boundaries: PERMANOVA parses concatenated metadata column names as a formula, and provenance interpolates package names into R source. Commit 1be406d on #243 adds fail-closed identifier/package validation, control/escape/expression regressions and documentation. Ordinary formula identifiers retain existing semantics; non-identifier metadata columns must be renamed. This is targeted hardening, not a complete R execution-safety certification.

#243 also carries the independent K9/DEED/Hypatia lanes, pinned full-history secret scanning with redacted reporting and boundary tests, least-privilege workflows, and dependency/security tooling from the #227 audit. Local contract/mutation/logging/scanner tests and Julia syntax parsing passed after this update. Julia/R are not installed locally; runtime validation is not claimed. #243 was subsequently rebased with explicit user permission and merged at 16:54:51 UTC as a9fb609. Its security/scanner changes have landed. A duplicate Julia timeout key from the rebase also landed before its correction could be pushed; current main CI fails workflow parsing. The minimal repair is #247, not yet merged. Advisory scans are not clean-bill-of-health attestations; remaining findings and two moderate Router advisories have not been suppressed.

No other contributor's branch was rewritten, no checks were waived, and no stale count-loading or placement-normalisation code was restored. The leaf-PD work has landed through #242 and the security follow-up through #243. The immediate CI repair is #247; malformed-JSON follow-up remains in #244. #228 was already closed by the time the explicit close command ran; its closed/unmerged state is confirmed.

Latest validation / repair status

#247 (46665dd) starts from merged main and removes only the duplicate three-line timeout block, retaining the 90-minute budget. All 8 workflows parse with unique keys, all 84 embedded Bash scripts parse, 39 governance checks/8 mutation cases and 4 pipeline-exit cases pass locally. Its actual hosted CI now loads and starts jobs: https://github.com/hyperpolymath/MetaManifold-WebUI/actions/runs/39990855274 . Required runtime checks are pending; this is not an all-green or merge-complete claim.

hyperpolymath added a commit that referenced this pull request Oct 23, 2026
## Scope
Security/governance follow-up to #227. All four Codex findings on that
old PR already landed in #229/#234 and follow-ups; I answered/resolved
its threads and closed it as superseded, rather than merging a
conflicting branch that could undo newer fixes. CodeRabbit's remaining
notices are account limits/billing, not code questions.

## Changes
- Add pinned, least-privileged full-history/working-tree secret scanning
and default-branch Scorecard publication; no inherited secrets or new
alert suppressions.
- Add Dependabot, SECURITY.md, editor settings and a repository-specific
AI manifest without changing existing licensing/architecture.
- Add independent pinned K9 conformance and its artifact to the PR
verdict. The current empty corpus is explicitly grey, not a semantic
pass. Missing semantic evidence fails closed. Keep DEED syntax and
advisory Hypatia as separate evidence families.
- Bound Julia CI at 90 minutes and preserve producer **and** log-writer
failures. Add 8 negative/mutation tests and 4 exit-combination tests.
- Refresh the bun lockfile within current major ranges for real security
advisories, notably DOMPurify, Vite and transitive build dependencies;
raise minimum Vite to 6.4.5 and Router DOM to 6.30.6. This is why the
lockfile changes. No generated assets are included.
- Audit high/critical frontend dependencies in CI.

## Verification
Bun **1.3.10**: frozen install, TypeScript checking, **72 frontend tests
/ 200 assertions**, production build, and high/critical audit pass. Full
audit reduced from **31 advisories (1 critical, 13 high, 14 moderate, 3
low) to 2 moderate Router advisories**. This is not a zero-vulnerability
claim.

Canonical pinned DEED lint, all 39 governance contract checks, 8
weakening mutations, 4 pipeline-exit combinations, unique-key YAML
parsing, all embedded shell syntax, Sonar reliability lint, proof axiom
audit (7 modules), and git diff checks pass locally.

Julia/R/Agda/Nickel are not installed in this sandbox; hosted CI must
supply those results. No local runtime/semantic pass is claimed.

## Hosted wrapper fixes
The first run found two issues in the pinned standards wrappers.
Fork-local copies retain the same immutable standards
validator/renderer/config sources and all gates, but fix the terminal
optional-log copy for an empty K9 corpus and upgrade Gitleaks from
8.18.4 to 8.30.1 (upstream release-asset digest). The old rules falsely
matched historical DNA/FASTQ and schema labels; no history exclusions or
local rule suppressions were added. A synthetic AWS credential must
still fail in CI. Three K9 copy cases and a credential-sensitivity test
also pass locally. Hosted checks are rerunning.

## Remaining / intentionally not done
See `docs/release-notes/2026-10-23-governance-security.md` for the
source Hypatia run and complete triage. Two moderate React Router
advisories need a v7 migration review. Estate language bans conflict
with the settled React/TS stack and independent Python reference
oracles; W3C SVG namespace URIs are not HTTP requests. Do not fix those
by rewriting the architecture or breaking namespaces. No configured
mirror target exists. Existing upstream licensing headers remain
protected. Fleet bots without report-backs remain not_run. No fake
conformance reports have been added.

The integration returns 403 for code-scanning/Dependabot alert listing
and the sandbox blocks Actions artifact/log storage hosts; no alerts
have been dismissed. Published grids/check APIs are still readable.
Scorecard's default-branch publication can only be verified after
landing; K9 has no committed contracts and therefore cannot claim
substantive contract coverage yet.


## PR #228 security follow-up (`1be406d`)
Independent verification found that the recorded conflict-resolution
tree for #228 is identical to main; #228 is being closed as superseded,
not conflict-merged. Its valid leaf-PD finding is already implemented in
**unmerged** #242/#244 and is not duplicated here. #242's integration
run 39986326202 passed all six jobs; #244's Julia validation remains
pending at review time.

The R-evaluation warning was not dismissed as a language-detector false
positive. This follow-up validates metadata column identifiers before
they enter PERMANOVA's parsed R formula and validates package names
before provenance interpolates them into R source. Expressions,
quotes/escapes, controls and reserved formula terms are covered by
regression cases. Plain identifiers preserve the existing formula and
scientific tests; non-identifier metadata columns must be renamed, as
documented in the function contract.

Local verification for the new commit: 39 governance assertions, 8
weakening mutations, 3 K9 copy cases, 4 CI logging cases,
scanner-boundary tests, complete working-tree/history shell scanning,
git diff checks and syntax parsing of all four changed Julia files pass.
**Syntax parsing is not execution**: the new Julia/R regressions await
hosted CI. This does not certify all remaining R call sites or dismiss
outstanding advisory findings.

### Main advanced during review
#242 has now merged as `b656d2a99e53267108d51ad87d7eaee13222d368`
(16:46:42 UTC), after its full integration suite passed. The leaf-PD fix
is therefore on main. #228 is confirmed closed/unmerged. This PR (#243)
now has conflicts with the newly updated main and is **not
merge-ready**; no main-merge commit or unrequested history rewrite has
been made. The new R-boundary/scanner tests are running at `1be406d`;
earlier passing checks must not be mistaken for validation of a future
rebased head.

---------

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
hyperpolymath added a commit that referenced this pull request Oct 23, 2026
…N with 400 (#244)

## Change and reason

Three things, in the order the linked review asked for them.

### 1. PR #228's merge conflicts, resolved (and shown to be a no-op)

PR #228 (`arena/7805092e`) branched from `8df1041`; all six files it
touched
have since been rewritten on `main` by #231, #233, #234, #236, #237 and
#239.
Resolving the six conflicted files in favour of `main` produces a tree
that is
**byte-identical** to `main` (`git diff --stat origin/main` is empty),
so the
merge commit records the resolution and changes no content. Concretely:

* `ci.yml` - main publishes a per-assertion digest of every failing
validated-statistics test plus a log tail; #228's awk windows are
superseded.
* `src/analysis/clade_cumulus.jl` - main already reads
`PhyloNode.edge_number`
in `_graft_jplace_queries`, already sums `rk.subtree_pd` plus the node's
own
branch, and additionally recomputes an emptied clade from
`direct_count`.
* `src/server/routes/clade_cumulus.jl` - main already uses
`with_results_db`
and `String(strip(...))`, and deliberately does **not** renormalise
placement
like-weights by the retained total: `parse_jplace` already rejects
ratios
outside [0, 1], a multi-candidate placement already partitions its mass,
and
dividing a lone 0.6 candidate by its own total would move the 0.4 the
tool
  left unplaced onto the reported edge (#236/#237 report it as
  `unplaced_reads` instead).
* `src/server/routes/validated_statistics.jl` - main validates raw
tokens for
the whole body after `ijson_read` proves it I-JSON, and streams bounded
rows
  instead of materialising a DataFrame. Both are strict supersets.
* `test/unit/*` - main's suites are far larger; the one test #228 had
that main
lacked ("Nested-clade drag-and-drop counts each active branch once") is
  restored here.

### 2. Codex P2 on PR #228: leaf subtree PD

`_reaggregate_after_move` credits an active leaf with its own incoming
branch,
so a parent can sum `subtree_pd` alone and count each active edge once.
The
ordinary builder (`build_node`) hard-coded `0.0` for leaves, so
validating a
drag-and-drop changed the reported PD of leaves whose topology had not
changed
and made the proposed tree's metrics incomparable with the original.

Both builders now take a leaf's PD from `clade_faith_pd` - the function
the
internal-node branch already used - so the conventions cannot drift
again.
Internal nodes and inactive tips are unchanged.

### 3. panic-attack `InputBoundary`: malformed bodies are 400s

35 unguarded `JSON3.read(String(req.body))` call sites across eight
route files
raised a JSON3 parse error out of the handler, so a truncated or hostile
body
produced a 500. Every route now parses through `Server._request_json`,
which
throws the existing `BadRequest`. The helper copies `req.body`, because
`String(::Vector{UInt8})` can consume the buffer.

Rescuing that turned out to need one more step, which CI caught: Oxygen
builds
its chain as `custom(serializer(router))` and its serializer catches
every
handler exception and answers 500 before our middleware runs, so
`_bad_request_middleware` never saw the throw. Oxygen's error mapper is
the
hook inside the serializer, and it is already a dispatch point
(`handle_error(::ValidationError)` answers 400). Extending it for
`BadRequest`
keeps this server's `{"error", "message"}` shape and makes the test's
`status == 400` true.

### 4. pons asinorum: tree path building is linear

The six `string-concat-in-loop` hits in `TreeShapes.tsx` were real:
every node
appended to four shared strings inside a `useMemo` over the whole
layout, which
is quadratic in the node count on a view that renders every node. The
fragments
are collected per node and joined once; the emitted order is unchanged.

### 5. panic-attack `ProofDrift` and a guarded benchmark baseline

The `@test x isa Y` assertions the rule flagged were real tautologies:
they
passed for any string. The occupancy pair now asserts the text the code
emits
(`measured 201 KiB above the predicted 101 KiB`) and the evidence round
trip
asserts the ISO-8601 UTC stamp `save_document` writes. The advanced
zero-handling benchmark parsed its baseline outside any guard, so a
truncated
baseline surfaced as a JSON3 failure from inside the timing loop; it now
names
the file it could not read.

## Validation

- `cd frontend && npx tsc --noEmit -p .` - clean.
- `cd frontend && npx vite build` - clean (chunk-size warning only,
pre-existing).
- `cd frontend && bun test src` - **72 pass, 0 fail** (11 files).
- `bash scripts/check-governance-contract.sh` - **all 20 checks
passed**.
- `bash scripts/lint-sonar-reliability.sh` - clean.
- `bash -n` on every `scripts/*.sh` - clean.
- All four workflows parse as YAML; the DEED lints clean with the pinned
  `deed_lint.js` from `hyperpolymath/standards@75bd5994` (see below).
- **Not run locally:** the Julia suite (no Julia runtime in this
environment;
  the two changed Julia files are covered by CI's `Julia tests` job).

## Hypatia and K9/DEED conformance

* **DEED** - `MetaManifold-WebUI_chora.deed` lints clean with the exact
linter
the reusable lane runs (`bun 1-formats/deed/tools/deed_lint.js`). Note
the
stdlib Python port in the same directory is older and rejects `#u8`; the
JS
linter the workflow uses accepts it, so the deed is correct as written.
* **K9** - this repository has no `.machine_readable/contractiles/`
registry or
Mustfile, and `governance.yml` does not call `k9-contractile.yml`, so
the K9
contractile lane is not in scope here. What *was* wrong is that the deed
declared `:report-backs ".gitbot-fleet/report-backs/"` while the
directory
did not exist; it now does, with the envelope contract documented, and
the
grid keeps rendering those members orange until a real report-back
lands.
* **Hypatia** - the 43 critical findings are all
`cicd_rules/banned_language_file` for `frontend/src/**/*.ts`. That rule
is a
repo-side-unfixable policy finding: TypeScript's only carve-outs are
narrow
path prefixes (`bindings/typescript/`, `.d.ts`, `vite.config.ts`, ...),
none
of which cover an upstream-authored React frontend, and the rule cannot
be
  suppressed from the repository. Migrating upstream's frontend to
  AffineScript is an estate project, not a change this fork can make.
* **panic-attack / pons asinorum** - most remaining findings are stale
or
false positives at `main`: `api/events.ts` and `tree/model.ts` already
wrap
`JSON.parse` in try/catch (hardened in #239), the `DynamicCodeExecution`
  hits are `RCall.reval` (R source strings, not Julia `eval`), and the
`unreachable-after-jump` hits point at plain arrow-function bodies. The
two
classes that were real - unguarded request-body parsing and quadratic
path
building - are fixed above. The grid's residual after this PR: Hypatia
45
critical (all `banned_language_file` on upstream TypeScript), 61 high,
35
medium; panic-attack 6 high (all `RCall.reval` false positives), 7
medium;
pons asinorum 2 medium (`unreachable-after-jump` on a `useMemo` return
and a `useEffect` cleanup, both false positives).

## Scope

- [x] Focused on the linked review plus conformance evidence
- [x] Does not alter dependencies, lockfiles, generated assets, or
vendored files
- [x] Secrets, local paths, and run outputs are not committed

## Contribution checks

- [x] I followed CONTRIBUTING.md
- [x] My commits include `Signed-off-by:` (DCO) and the licence is
unchanged

---------

Co-authored-by: arena-agent <297053741+arena-agent@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
@hyperpolymath
hyperpolymath deleted the arena/7805092e-metamanifold-webui branch October 23, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant