tracking: 2026-09-13 codebase review — 35 specs, tiered

[jayhesselberth]

Tracking issue for the 2026-09-13 codebase review (leech at ae73280, escapepod v0.25.0). Every linked issue is a spec: acceptance criteria, target paths, test names, non-goals. They were written to be executed by an agent without further design; if one turns out to need a decision, comment on it rather than guessing.

Standing rule from the review: leech_core defers to the upstream escapepod crate whenever it provides the primitive (#258). Do not optimise a local copy of something escapepod_signal exports; replace it.

Tiers

Tier	#	Title	Effort	Depends on
P1	#258	rust: replace the local chunk pipeline with escapepod_signal::chunk	M	#259 first; one seam needs rnabioco/escapepod-rs#380
P1	#259	rust: hand the k-mer table and move tables to leech_core once	S	—
P1	#260	dataset/rust: encode signal_kmer per batch	M	rnabioco/escapepod-rs#380 (interim leech_core loop allowed)
P1	#261	ci: a leech_core build failure must fail CI	S	—
P1	#262	eval: remove the three commands that import a nonexistent module	S	—
P1	#263	predict: --workers bypasses Rust, swallows config errors; --backend python partial	S–M	—
P1	#264	numerics: fp16 probabilities under AMP feed AUROC and selection	S	— (measure the shipped scores first)
P1	#265	prepare: a run that loses every batch exits 0 (+ #166)	S	—
P1	#266	pipeline: rules pass options the CLI does not have; default context 225 vs 200	S–M	Snakemake agent
P2	#267	rust: struct-of-arrays return + per-read outcomes	M	#258
P2	#268	inference: bundle predict through the streaming pipeline	L	#269
P2	#269	inference: one InferenceSpec resolver	M	—
P2	#270	training: one TrainConfig; grid search forwards every key	M–L	—
P2	#271	cli: shared option decorators, one device/workers/seed convention	M	#270
P2	#272	training: measured hot-path fixes	S each	coordinate with #260
P2	#273	ddp: BatchNorm under --gpus N	S	—
P2	#274	models: tier the registry, structural predicates	M	—
P2	#275	prepare: one dispatcher	M	—
P2	#276	signal_refine: remove the never-run DP path, inert sidecar settings	M	coordinate with #258
P2	#277	dead code sweep	S	—
P2	#278	prepare: base-defined signal window	M	after #258 ideally
P2	#279	training: label-noise-aware loss	M	#270 helpful
P2	#280	training: low-FPR checkpoint metric and loss	S–M	#270 helpful
P2	#281	dataset: time-stretch augmentation	M	—
P2	#282	prepare/predict: junction-disruption field + abstention	S–M	—
P2	#283	models: non-causal TCN knob, recorded feature standardisation	S	—
P3	#284	deployment helpers dedup (+ #141)	M	—
P3	#285	dataset: TensorCorpus split, block-only twins	L	#260
P3	#286	trainer hygiene	S	#270
P3	#287	models: TOML-port the hand-written classes	M	#274
P3	#288	chunk format: one CHUNK_FIELDS spec	M	—
P3	#289	merge: streaming through the spill	M	—
P3	#290	ty: re-enable the nine ignored rules	M	—
P3	#291	docs: generated CLI reference, changelog line, retire --reference-anchored	S	#262
P3	#292	tests: hygiene and coverage gaps	M	—

Suggested execution order

1. Independent small fixes, any order, one PR each: eval: remove compare, importance and ablation, which import a module that does not exist #262, ci: a leech_core build failure must fail CI, not skip the parity suite #261, numerics: fp16 probabilities under AMP feed AUROC and checkpoint selection; eval and predict use different precision policies #264, prepare: a run that loses every batch exits 0; a backend divergence hides in one config corner; torch is imported before the pool forks #265, rust: hand the k-mer table and move tables to leech_core once, not per batch #259, dead code sweep: constants, dependencies, io/encoding/features helpers, bundling shims #277, ddp: BatchNorm variants under --gpus N use per-rank statistics; convert to SyncBatchNorm or refuse #273, pipeline: rules pass options the CLI does not have, config keys are unread, and the default signal context is 225 while every doc says 200 #266.
2. Structural, one at a time per area: rust: replace the local chunk pipeline with escapepod_signal::chunk #258 (Rust), inference: one InferenceSpec resolver for model config, CLI overrides and the feature window #269 then inference: run bundle predict through the single-model streaming pipeline #268 (inference), training: one TrainConfig replacing six hand-written parameter lists; grid search must forward every key #270 then cli: shared option decorators, one --device/--num-workers/--seed convention, lazy startup #271 (training/CLI), models: tier the registry, derive feature/wide/vmap predicates structurally, fix the stale docs #274 (models), prepare: one dispatcher — pool initializer, retire the sequential path, one read-split rule, one write step #275 and signal_refine: remove the never-run DP reference path and stop recording settings escapepod does not honour #276 (prepare), dataset/rust: encode signal_kmer per batch, not per sample #260 and training: measured hot-path fixes (non-blocking H2D, static compile shapes, int32 map, batch probe, DDP/compile order, pool matrix cache) #272 (loader).
3. Model features for the next experiments: models: symmetric (non-causal) TCN padding and recorded per-channel feature standardisation #283, prepare: a base-defined signal window (--signal-context-bases L,R) #278, training: select checkpoints and shape the loss for the low-FPR operating regime #280, training: a label-noise-aware loss with per-sample noise rates #279, dataset: time-stretch augmentation (speed invariance) #281, prepare/predict: a junction-disruption chunk field and an abstention rule that uses it #282 — each is a leech feature with a test; the experiments themselves are run from escapepod-models with three paired seeds.
4. Remaining P3 as capacity allows.

Every PR: /code-review before /land, tests/test_backend_parity.py green, and a CHANGELOG line.

Model-side context (for the P2 feature issues)

The production charging model reads basecall disruption at the adduct; raw signal alone reaches 0.9947 AUROC on the pure cohort against the full model's 0.9961, so the feature branch is nearly idle. Per-base feature engineering, alignment channels, isotype balancing and recipe ports are measured nulls in escapepod-models' dev notes — do not reopen them. The open levers are geometry (the arm must reach +24 and keep the body, #278), labels (#279), the operating regime (#280), and robustness to speed, construct and basecaller (#281, #282, #283).

[jayhesselberth]

v0.12.0 shipped: #258 #259 #260 #261 #262 #264 #265 #266 #268 #269 #270 #272 #273 #274 #275 #276 #277 #278 #279 #280 #281 #282 #283 (#264 and #273 were fixed by #297 and #295 respectively but not auto-closed — closed manually just now).

Still open: #263, #267, #271 (P1/P2), and the P3 batch #284–#292.