Skip to content

Fix CladeCumulus testset: field names, NHX capture, LWR tolerance, route loading #222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyperpolymath merged 2 commits into from
Oct 23, 2026
+32 −14
Merged

Fix CladeCumulus testset: field names, NHX capture, LWR tolerance, route loading #222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b6ab932
fix(clade-cumulus): repair field names, NHX support capture, LWR tole…
hyperpolymath Oct 23, 2026
0e409ed
fix(doi): restore missing module DOI opener
hyperpolymath Oct 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions src/analysis/clade_cumulus.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@ const DEFAULT_TAXONOMIC_RANKS = [
"Class", "Order", "Family", "Genus", "Species",
]

const PLACEMENT_WEIGHT_SUM_TOL = 1e-5
# Placement tools may truncate a query's candidate list, so retained
# like-weight ratios can legitimately sum to slightly less than 1 (a single
# retained placement at 0.98 is complete evidence). Sums far from 1 indicate
# a malformed or partial record and stay undecided.
const PLACEMENT_WEIGHT_SUM_TOL = 0.05

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject placement weights that exceed one

When candidate LWRs total between 1.0 and 1.05 (for example, 0.60 + 0.44), the existing abs(weight_sum - 1.0) check now marks the query valid. Truncating a candidate list can only lower its retained mass, so the widened tolerance should apply only below 1.0; otherwise malformed overweight records are accepted and subsequently normalized into apparently valid placement evidence.

Useful? React with 👍 / 👎.


## Phylogenetic tree structures & Newick / jplace parsers
mutable struct PhyloNode
Expand Down Expand Up @@ -202,8 +206,12 @@ function parse_newick(src::AbstractString; warnings::Vector{String} = String[]):
end
end

function read_label_and_quoted()::Tuple{String,Bool}
skip_ws_and_leading_comments!()
function read_label_and_quoted(; capture_nhx::Union{Ref{Union{Float64,Nothing}},Nothing} = nothing,
capture_edge::Union{Ref{Union{Int,Nothing}},Nothing} = nothing)::Tuple{String,Bool}
# An NHX comment immediately after ')' precedes the (usually empty)
# label, so the capture refs must be threaded through this first skip
# or the support tag is consumed as an anonymous comment.
skip_ws_and_leading_comments!(capture_nhx = capture_nhx, capture_edge = capture_edge)
pos[] > n && return ("", false)
if bytes[pos[]] == UInt8('\'')
pos[] += 1
Expand Down Expand Up @@ -260,7 +268,7 @@ function parse_newick(src::AbstractString; warnings::Vector{String} = String[]):

nhx_sup = Ref{Union{Float64,Nothing}}(nothing)
edge_num = Ref{Union{Int,Nothing}}(nothing)
raw_lbl, was_quoted = read_label_and_quoted()
raw_lbl, was_quoted = read_label_and_quoted(capture_nhx = nhx_sup, capture_edge = edge_num)
skip_ws_and_leading_comments!(capture_nhx = nhx_sup, capture_edge = edge_num)

branch_len = 0.0
Expand Down Expand Up @@ -991,7 +999,7 @@ function _graft_jplace_queries(

for q in jd.queries
q.status === :valid && !isempty(q.candidates) || continue
total_w = sum((c.weight for c in q.candidates); init = 0.0)
total_w = sum((c.like_weight for c in q.candidates); init = 0.0)

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate the corrected LWR field into the HTTP route

For a CladeCumulus POST using a valid .jplace file and a table with sample columns, this fix lets _graft_jplace_queries complete, but the ensuing diversity block in src/server/routes/clade_cumulus.jl:279-285 still reads c.weight and cand.weight. QueryCandidate has only like_weight, so the advertised HTTP flow still raises a FieldError and returns a 500 instead of the analysis response.

Useful? React with 👍 / 👎.

total_w > 0.0 || continue
multi = length(q.candidates) > 1
for (qname, mult) in q.names
Expand All @@ -1000,7 +1008,7 @@ function _graft_jplace_queries(
tx = get(asv_deepest_taxon, qname, nothing)
tconf = get(asv_tax_conf, qname, 1.0)
for (ci, cand) in enumerate(q.candidates)
share = cand.weight / total_w
share = cand.like_weight / total_w
tip_id = multi ? string(qname, "@edge", cand.edge_num, "#", ci) : qname
if multi
asv_reads[tip_id] = get(asv_reads, tip_id, 0.0) + base_cnt * share
Expand All @@ -1009,9 +1017,9 @@ function _graft_jplace_queries(
elseif !haskey(asv_reads, tip_id)
asv_reads[tip_id] = base_cnt
end
query_lwr[tip_id] = cand.weight
query_lwr[tip_id] = cand.like_weight
push!(get!(by_edge, cand.edge_num, _GraftedQuerySpec[]),
_GraftedQuerySpec(tip_id, qname, share, max(0.0, cand.distal_length), max(0.0, cand.pendant_length), cand.weight))
_GraftedQuerySpec(tip_id, qname, share, max(0.0, cand.distal_len), max(0.0, cand.pendant_len), cand.like_weight))
end
end
end
Expand Down
2 changes: 2 additions & 0 deletions src/core/doi.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
# * Durable write-ahead journal; immutable DOI/provenance receipts
# * DOI insertion never rewrites scientific objects

module DOI

using Dates, SHA, HTTP, JSON3, Logging

export DOIConfig, Deposition, PublicationReceipt,
Expand Down
12 changes: 10 additions & 2 deletions src/server/routes/clade_cumulus.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,11 @@ function _probe_optional_phylo_tool(tool_key::String, fallbacks::Vector{String})
end
end

function _load_run_table_df(study::String, run::String, table::String; group::Union{String,Nothing}=nothing)
function _load_run_table_df(study::AbstractString, run::AbstractString, table::AbstractString; group::Union{AbstractString,Nothing}=nothing)
# JSON3 hands request fields back as SubStrings; the DuckDB helpers are
# String-typed, so normalise once at this boundary.
study = String(study); run = String(run); table = String(table)
group = isnothing(group) ? nothing : String(group)
Comment on lines +70 to +71

@chatgpt-codex-connector chatgpt-codex-connector Bot Oct 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Normalize the selected tree filename too

When either POST route uses treeSource: "study_tree", tree_file = strip(string(...)) in _resolve_tree_inputs_from_request is a SubString, while _valid_tree_file and _tree_format in routes/trees.jl accept only String. Normalizing the table arguments only inside this loader allows the request to advance to that next MethodError, so existing .nwk and .jplace selections still fail before the tree is read; normalize tree_file as well or broaden those helper signatures.

Useful? React with 👍 / 👎.

dir = _require_duckdb(study, run; group)
isnothing(dir) && return (nothing, nothing, json_error(404, "no_results",
"No results for run '$run' - run the pipeline first"))
Expand Down Expand Up @@ -142,7 +146,11 @@ end
db = DuckDB.DB(db_path)
con = DBInterface.connect(db)
try
names_df = DataFrame(DBInterface.execute(con, "SHOW TABLES"))
# Real tables only: SHOW TABLES also lists views, and a view backed
# by a session-registered DataFrame cannot be bound (let alone
# counted) from this fresh connection.
names_df = DataFrame(DBInterface.execute(con,
"SELECT table_name AS name FROM duckdb_tables() WHERE NOT internal"))
tlist = sort(String[string(r.name) for r in eachrow(names_df)])
for t in tlist
cols = _duckdb_columns(con, t)
Expand Down
8 changes: 4 additions & 4 deletions test/unit/test_clade_cumulus.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -252,8 +252,8 @@ done
echo "(MMTIP_000001:0.05,(MMTIP_000002:0.03,MMTIP_000003:0.04)0.910:0.12,MMTIP_000004:0.08);" > "$OUT"
""")
chmod(mock_ft, 0o755)
old_ft = get(PH.Tools._tools, "fasttree", nothing)
PH.Tools._tools["fasttree"] = mock_ft
old_ft = get(MetaManifold.Tools._tools, "fasttree", nothing)
MetaManifold.Tools._tools["fasttree"] = mock_ft
try
s4 = [
"ASV_1" => "ACGTACGTACGTACGTACGT",
Expand All @@ -267,9 +267,9 @@ echo "(MMTIP_000001:0.05,(MMTIP_000002:0.03,MMTIP_000003:0.04)0.910:0.12,MMTIP_0
r["sha256"] == bytes2hex(open(sha256, mock_ft)), ft_res.tool_records)
finally
if old_ft === nothing
delete!(PH.Tools._tools, "fasttree")
delete!(MetaManifold.Tools._tools, "fasttree")
else
PH.Tools._tools["fasttree"] = old_ft
MetaManifold.Tools._tools["fasttree"] = old_ft
end
end
end
Expand Down
Loading