Repository navigation
Conversation
Reuse ordered output buffers and read-name groups, incorporate safe mate and formatting fast paths, and stream only-depth from reusable records. Use published Seqair 0.10.1 without the experimental fork or unchecked vector operations. Preserve depth admission and reject unsupported BAM long-CIGAR pileups explicitly. Amp-Thread-ID: https://ampcode.com/threads/T-01a1ca76-7f80-72d6-ac92-b1b4e5bacdcf Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Bring the useful application-level results from the Goose/Rust experiments into Perbase, using published upstream Seqair 0.10.1, not the experimental fork. HTSlib remains the default. Build with
--features seqairand select--seqairon either depth command to opt in.-Cbudget constrains queues rather than reducing counting to one worker.Compatibility and limitations
Seqair's generic depth cap does not match HTSlib's whole-read admission at boundaries; the adapter preserves Perbase's existing rule instead. The comparison tests include low and zero caps, ties, filters, and all nine mate strategies.
The optional backend requires Rust 1.98.1 or newer and an external FASTA for CRAM, including embedded-reference CRAM. BAM long-CIGAR records expose an upstream
RecordStorelimitation: it counts the two placeholder CIGAR operations rather than expandingCG. Base-depth now fails with an actionable instruction to rerun without--seqair, rather than returning wrong counts. HTSlib and the Seqair only-depth raw reader handle the tested long-CIGAR fixture correctly.Validation
cargo +1.98.1 test --locked: 132 unit tests and one doctest passed.cargo +1.98.1 test --locked --features seqair: 137 unit tests and one doctest passed.cargo fmt --checkandgit diff --checkpassed. Clippy completed; existing warnings in examples/older tests and dependency future-compatibility warnings remain.Performance
Fresh measurements of this PR's build versus a clean build of released Perbase 1.4.1. These are not the experimental fork's R7 numbers.
Chr20 BAM on the same computer,
-t 32 -c 1000000, default queue budgets, uncompressed TSV written to a regular file (no fsync). One warmup followed by two measured runs for each variant/workload, run sequentially in rotating order without concurrent builds or tests. All 27 outputs matched the corresponding workload's SHA-256. These are preliminary two-run measurements, not confidence intervals.base-depth -monly-depthonly-depth -mThe application changes improve both backends. Seqair gives another approximately 2.71× speedup over the optimized HTSlib base-depth path here; only-depth is essentially tied between the two new backends.
base-depth -monly-depthonly-depth -mRSS is the maximum over the two timed runs, not an average. Seqair's base-depth speed comes with more memory than the PR's HTSlib path because it retains each active region's reads.
Samples and build provenance
Seconds in measured order:
[49.47, 49.38]; PR HTSlib[19.55, 20.20]; PR Seqair[7.33, 7.35].[9.78, 10.35]; PR HTSlib[4.22, 4.13]; PR Seqair[4.18, 4.10].[10.19, 10.16]; PR HTSlib[4.16, 4.14]; PR Seqair[4.11, 4.13].Both source revisions built with Rust 1.98.1, their release profile,
RUSTFLAGS='-C target-cpu=native',CC=clang, andCFLAGS='-O3 -march=native'. Candidate built with--features seqair; both PR columns use that same executable, changing only--seqair. Baseline built from clean 1.4.1 with no features. Both commands used--lockedand their respective Cargo.lock files.a3527928f449355c693275c95fdf2712efaa6279ed57e461652b758d78ff6804.136131baf86623e9d55110318e691878ade0ce98704f6612a57b9ed076044060.b10b1722946bea9a8b952bf416579b94ba46c390b86a5b863fd3965f850c1eab.b4e1e41f38b69f5ae99c2a9b5fa652773dde07df41f4467572a6a2bc8de93ed7.825dd615bcba2d4e841e46e775a7b4a9ab75d146ddfb15f8b10a1956e5d20012.24556d4290684b96765e16bb8b0750f6cd8c1766438475d910224ce07fd61060.Deferred Seqair work
No Seqair fork changes or upstream submissions are included. Follow-up candidates:
CGlong CIGARs inRecordStore, not only raw records; investigate operation-count width for other formats too.The experiment narrative contains the historical checkpoints. These library follow-ups are deliberately separate from this application PR.
AI disclosure
Prepared with Amp, including code and tests. This is a draft for Seth's full human review before merge, as required by CONTRIBUTING.md. No merge or release is requested by this PR.