Skip to content

Reuse indexed readers per worker thread - #111

Merged
sstadick merged 1 commit into
masterfrom
perf/reuse-indexed-readers
Oct 31, 2026
Merged

sstadick merged 1 commit into
masterfrom
perf/reuse-indexed-readers

Conversation

@sstadick

Copy link
Copy Markdown
Owner

Summary

Closes #110.

Reuse a lazily opened, thread-local indexed BAM/CRAM reader across regions in base-depth and both only-depth paths. The cache is keyed by input and reference paths. RegionProcessor and counting logic are unchanged.

Regression coverage exercises repeated fetches, backward seeks, empty regions, contig changes, input-file changes, and CRAM reference switching. The reference-switch test was verified to fail when reference-path cache invalidation was temporarily disabled; only that test disables CRAM MD5 validation to make the selected reference observable.

Benchmark results

Synthetic warm-cache local-file benchmarks on Apple M3, release builds, four processing workers, two warmups and seven measured runs. Input contains 39,992 reads over two 1 Mb contigs.

BAM with 20,000 short BED intervals:

Mode Before After Speedup
base-depth 442.3 ms 203.8 ms 2.17x
only-depth 337.2 ms 97.5 ms 3.46x
only-depth --fast-mode 346.7 ms 98.1 ms 3.54x

CRAM with 200 short intervals improved 4–6%. Whole-input controls were approximately unchanged. These are workload-specific measurements, not production-data performance guarantees.

Validation

  • cargo test --locked: 24 library tests, 98 binary tests, and one doctest passed.
  • 47 before/after output comparisons were byte-for-byte identical, including all 12 timed workloads.
  • cargo fmt --check passed.
  • cargo clippy --locked --all-targets completed with existing warnings.

An existing reference/zero-depth TSV column-count bug was reproduced in both versions and is addressed in a separate, stacked draft PR.

AI disclosure

Implemented with Amp. Draft for human review.

Reuse a lazily opened reader keyed by input and reference paths across regions in all depth-processing modes. Cover repeated fetches, input changes, and reference switching with regression tests.

Synthetic short-interval BAM benchmarks improved 2.17x for base-depth and 3.46-3.54x for only-depth; CRAM improvements were 4-6%.

Implemented with Amp; pending human review.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18c50-92d6-73c8-af62-326ba8f97fd3
Co-authored-by: Amp <amp@ampcode.com>
@sstadick
sstadick marked this pull request as ready for review October 31, 2026 09:44
@sstadick
sstadick merged commit c2620d4 into master Oct 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reuse one IndexedReader per worker thread instead of opening one per region

1 participant