Home

Scoring and Internals

This page covers Hifimizer's scoring mechanics, pruning, Hi-C phasing internals, walltime controls, objective tuning, and the full output directory layout. For installation and everyday usage, see the README.

Scoring

Fold change against the default assembly, scaled by measured movement

Trial 0 is always hifiasm with default parameters, and it is the reference every other trial is measured against. Each metric is scored as a log2 fold change relative to that baseline, signed so that positive always means better, and then divided by how far that metric actually moves when the parameters change:

score = Σ  importance × log2FC / scale

Location comes from a fixed reference. Trial 0 scores exactly 0, so the sign of a trial's score tells you directly whether it beat plain hifiasm, and scores stay comparable for the whole study and across resumes because the reference never drifts.
Scale comes from the burn-in. Without it, a metric that happens to swing proportionally more dominates the objective regardless of whether those swings matter — fold change removes the unit problem (bp vs %) but replaces it with a volatility problem.

The scale is the root mean square of each metric's log2FC across the burn-in trials, measured about the baseline rather than about the burn-in mean. That matters: if every random parameter set lands 4× worse than default but tightly clustered, the standard deviation about the mean is near zero and an entirely typical trial scores −28. The RMS about the baseline is 2.0 and the same trial scores −1.0, which is the honest reading. Scaled scores are clipped to ±5 so a metric with a small scale cannot produce a huge one.

A metric that did not move at all during the burn-in has no scale and is dropped from the objective for the rest of the study, with a warning — it cannot tell one parameter set from another, so scoring it only adds noise.

Before the scales exist (i.e. during the burn-in itself) trials are scored on the raw fold change. They are re-scored on the way into the main study, so every value the optimizer sees is on one stationary scale.

The fold change itself

+1.00 log2FC = twice as good, -1.00 = twice as bad, 0.00 = no change.

log2 rather than a plain ratio because a plain ratio is asymmetric — a doubling moves 1.0 away from "no change" while a halving moves only 0.5 — so summing plain ratios quietly rewards the metrics that got worse. Fold changes are computed on raw values, never the internally stored log(value + 1) ones: a ratio of logarithms is not a fold change.

Three metrics are not scored on the number they report, because a ratio of that number would mislead:

Metric	Compared on	Why
`qv`	the error rate it encodes, `10^(−qv/10)`	Phred is already a logarithm, so a ratio of QVs is a ratio of logarithms. A +3.01 dB gain is exactly one halving of the errors.
`kmer_completeness`	the deficit, `100 − value`	Bounded at 100%, so the percentages barely move: 99.0 → 99.5 is a factor of 1.005, but the deficit halved.
`trans_hap_rate`	the rate itself, floored	A percentage that is legitimately zero on a perfectly phased assembly.

Counts get a +1 pseudocount, since several are legitimately zero on a good assembly (missing markers, duplicated markers) and 0 → 0 must mean "no change" rather than a division by zero. Fold changes are clipped to ±5 log2 units (32×) so one collapsed metric cannot swamp the rest.

A dataset where the parameters change nothing — an assembly that is already a single contig, say — stops cleanly after the burn-in with a message saying so, rather than optimizing noise.

The burn-in

--burn-in-trials (default 10) random parameter sets run after trial 0, before the optimization proper. They exist to measure the scales. Three things worth knowing:

They are never pruned. They define the spread everything else is divided by, so cutting the bad tail out of them would understate it.
They are kept. They are copied into the main study as its starting observations, re-scored, so TPE begins with ten real data points rather than spending its own startup budget rediscovering them.
The .bin files trial 0 writes are reused by every burn-in trial and by the optimization that follows, so the burn-in costs assembly time but not error-correction time.

Below about 5 the scale estimates are too noisy to divide by.

What gets evaluated

Evaluation is alignment-free, apart from Hi-C phasing

Mapping the read set back to every candidate assembly bought a consensus-accuracy estimate that qv already provides from k-mers without reference bias, and a missing-sequence estimate that kmer_completeness already provides — at the cost of a minimap2 pass, a sort, a samtools stats pass and a structural-variant call on every single trial, plus a full pass over the reads at startup to draw the subset.

What that trades away is the misassembly signal. Split-read counts and SV calls were the only metrics that could see a chimeric join, and a misjoin has all the right k-mers in the right abundances: qv and kmer_completeness cannot detect it, and gene-space completeness only catches it when a marker happens to straddle the breakpoint. Since n50 is being maximised, watch fragmented and multi_copy, and do not weight n50 far above everything else — that is what stops the search buying contiguity with misjoins.

Per-haplotype scoring

Hi-C and ultra-long runs produce two haplotypes. Scoring only hap1 leaves half the assembly out of the objective, so a parameter set that builds a good hap1 and a shattered hap2 would score the same as one that builds two good haplotypes.

These are therefore scored separately per haplotype when a second haplotype exists:

Metric	Single haplotype	Two haplotypes
`num_contigs`	`num_contigs`	`num_contigs_hap1`, `num_contigs_hap2`
`n50`	`n50`	`n50_hap1`, `n50_hap2`
`length_diff`	`length_diff`	`length_diff_hap1`, `length_diff_hap2`
`kmer_completeness`	`kmer_completeness`	per haplotype and combined (see below)

Splitting rather than averaging is the point: a mean N50 lets one contiguous haplotype hide a shattered one, and a mean length deviation lets +10 Mb on hap1 cancel −10 Mb on hap2. Each per-haplotype metric carries half the single-haplotype importance, so splitting does not silently double how much that property counts for.

qv is also measured per haplotype (qv_hap1, qv_hap2); the scored qv is their mean.

k-mer completeness is scored both ways on purpose. Per-haplotype completeness is structurally capped by heterozygosity and is higher for a collapsed assembly that packed both alleles into one haplotype — on its own it rewards exactly the failure mode Hi-C phasing exists to avoid. The combined (hap1 + hap2) figure is the only metric that penalises that collapse, so both are kept.

Gene-space completeness stays on hap1. It is the most expensive stage in the run, and each haplotype should carry the full gene set once, so one haplotype already answers the question; running it twice would double the cost of every trial to average two nearly identical numbers. Collapsed phasing still surfaces here as multi_copy on hap1.

Hi-C phasing consistency

Active only with --hic1/--hic2, and the only place reads are still aligned. The two haplotype assemblies are concatenated with their contigs renamed h1_* / h2_*, a fixed subset of Hi-C pairs is aligned to that, and pairs whose two mates land on different contigs are counted by whether those contigs belong to the same haplotype:

trans_hap_rate = 100 × trans / (cis + trans)

Hi-C contacts occur within a physical chromosome, so cross-haplotype links mean contigs were assigned to the wrong haplotype. This is the only metric that responds directly to --s-base, --f-perturb and --l-msjoin; without it those parameters are tuned against an objective that cannot see phasing quality at all.

Two things worth knowing:

The MAPQ filter (--hic-min-mapq, default 20) discards most reads by design. Two haplotypes of one individual are nearly identical, so reads that do not overlap a distinguishing variant map ambiguously and carry no phasing information. What survives is the informative subset — but it is a minority, so --num-hic-reads needs to be generous. hic_pairs_informative is reported every trial so a suspiciously good rate computed from a handful of pairs is visible rather than silently trusted.
It is a fit statistic, not an independent test. hifiasm phased using these same reads. It still separates parameter sets that phase from ones that do not, which is what the optimizer needs.

Disable with --no-hic-phasing.

Other implementation details

k-mer metrics (yak). The read k-mer hash is built once during setup from the full read set and reused by every trial. Disable with --no-kmer-eval.
Gene-space completeness is scored with compleasm by default, falling back to BUSCO with metaeuk and then BUSCO with augustus. Each attempt gets --busco-walltime, and the first backend that succeeds is cached and reused. BUSCO's own miniprot backend is deliberately not in the chain: compleasm is miniprot, so re-running it under BUSCO would spend another walltime to learn nothing.

compleasm and BUSCO cannot share a conda environment (their pinned dependencies conflict), so compleasm is installed into its own environment and invoked by absolute path. The container sets $COMPLEASM_BIN; outside it, create the environment from environment.compleasm.yml next to the main one and hifimizer will find it, or point at it with --compleasm-bin. Use --no-compleasm to force BUSCO.
Bin-file compatibility. Every trial shares one hifiasm -o prefix so the expensive .bin files are reused. hifiasm resumes from whatever it finds there without checking that the flags match, so hifimizer records them in work/hifiasm/.fingerprint.json and refuses to start if they changed. Use --force-rerun.

Early trial pruning

gfastats costs seconds; yak and gene-space completeness cost hours. Since gfastats runs first, a trial whose contiguity has already collapsed can be recognised and abandoned before any of that is paid for.

After gfastats, a trial is pruned when at least --prune-min-metrics (default 2) of contig count, N50 and length difference are worse than the default assembly by more than --prune-fc (default 1.0 log2 units, i.e. twice as bad). With two haplotypes each haplotype is checked separately, so a collapse in hap2 alone still counts.

Two deliberate properties:

The bar never moves. It is measured against trial 0, not the best trial so far. Measured against the running best it ratcheted: every improvement tightened the gate on every later trial, so a study that found one good assembly early then pruned nearly everything after it.
One metric is never enough. Contig count, N50 and length difference are strongly coupled, so one drifting out is ordinary variation; several going at once is a collapse.

Only the gfastats metrics can trigger this — a poor QV or BUSCO score does not. Pruned trials do not count towards --max-metric-skips, which tracks a different failure (a metric tool that worked on the baseline and then broke).

If --max-consecutive-pruned (default 10) trials are pruned back to back without one completing, the study stops and reports the best trial so far. Prunes scattered across a long study are expected and do not count.

Set --prune-fc 0 or --prune-min-metrics 0 to disable pruning.

Reruns and resuming

Hifimizer persists its study in optuna_study.db, so runs are resumable and specific results can be reproduced without re-optimizing:

--force-rerun — discard any existing study and artifacts and start fresh.
--rerun-best — skip optimization and rebuild the assembly from the best parameters in an existing study.
--rerun-trial N — skip optimization and rebuild the assembly from the parameters of trial N.

Without --force-rerun, an existing study in the output directory is resumed. The reference lives in work/cache/baseline_metrics.json and work/cache/metric_scales.json; a study cannot be resumed without both, since its trials would not be comparable with new ones.

Walltime controls

Long-running external steps are individually bounded. When a step exceeds its limit the entire process group is killed, and the trial is pruned.

--trial-walltime HOURS (default 24) — per-trial hifiasm limit; also applied to the final assembly.
--gfastats-walltime HOURS (default 0.5)
--yak-walltime HOURS (default 2) — the one-off yak count and each per-trial yak qv.
--busco-walltime HOURS (default 6) — per gene-prediction-backend attempt.
--hic-phasing-walltime HOURS (default 4) — Hi-C subsetting and the phasing alignment.

Tuning the objective

weights.json holds an importance per metric. Because every metric is already signed so that positive means better, these are all positive and answer one question: what is one scale unit of movement on this metric worth, relative to the others? One scale unit is "about as far as this metric typically moved during the burn-in", so equal importances mean equal say in the objective regardless of how volatile each metric is.

Any metric absent from the file falls back to a built-in default. The optimization direction of each metric is defined in src/optim_directions.json.

Use --multi-objective to optimize a Pareto front over individual metrics instead of a single score. In that mode the objectives are the raw metric values, not fold changes, since each Pareto axis is independent.

Each trial logs a metric report showing, per metric, the raw measurement in its natural units, its log2 fold change against the baseline, and that fold change scaled by the metric's burn-in movement (the last is what the importance multiplies).

Convergence status is reported after every trial so a study closing in on convergence is distinguishable from one that has not moved. Pruned trials count as "no improvement" rather than being skipped, so a run that has stopped finding anything converges instead of grinding on.

Note

--sensitive can improve assemblies occasionally, but will significantly increase runtime.

Output directory layout

By default an output/ directory is created in the current working directory. Final results live directly under the output directory; all intermediates live under work/ and can be deleted to reclaim space (at the cost of a full hifiasm recompute on the next run).

output/
├── final_assembly/              # Optimized assembly (best parameters)
│
├── default_assembly/            # Baseline hifiasm run (default parameters, trial 0)
│
├── logs/                        # main.log + per-command, per-trial logs
│
├── optuna_output/               # Optuna plots + per-metric history
│
├── optuna_study.db              # Optuna study database (all trials, params and metrics)
│
├── best_params_checkpoint.json  # Best parameters seen so far
│
└── work/                        # Intermediates — safe to delete
    ├── hifiasm/                 # Shared hifiasm prefix (.bin files + .fingerprint.json)
    ├── reads/                   # Hi-C subset for phasing (Hi-C runs only)
    ├── kmers/                   # yak read k-mer hash (built once, reused by all trials)
    ├── busco_downloads/         # BUSCO lineage datasets (fallback backend)
    ├── compleasm_downloads/     # compleasm lineage library (primary backend)
    ├── cache/                   # baseline_metrics.json, metric_scales.json, stage state
    └── trials/trial_<id>/       # Per-trial completeness / yak scratch

If the best trial turns out to be trial 0, no final assembly is built: the default assembly already on disk is the result, and the log says so rather than spending another hifiasm run reproducing it.

Metric history plots are written on each metric's natural scale — N50 in bp, length difference in Mb — not in the log(value + 1) space used internally.

Navigation Menu

Repository navigation