Repository navigation
v0.0.4 — exact GPU speed-ups, multi-GPU work-unit scheduling, many queries against one reference - #8
Merged
Merged
Conversation
Sources are alejandrogzi/hspZ `experimental` @ dd1b1d2 byte for byte, with the production packaging re-applied: file headers, the clap attribute, the hspZ spelling in usage and messages, and the help text sanitised of references to the experimental tree's planning file (restores what v0.0.3 shipped). New in this sync, on top of what the earlier release branches carried: * `hspZ index` / `run --index DIR` — on-disk reference tables, loaded instead of rebuilt, validated against the parsed reference and the resolved plan before anything reaches the GPU (src/index.rs). * `--query-list` on two or more GPUs — the batch's work units are spread over the workers by the same count partition a single query uses, and the receiver reorders each job's results, so per-job output is unchanged. Exactness on this tree: 174 unit tests; chr20 digest 70152625c1a75cb3; the 5x3 plan at --gpus 2 with the unit partition cb38f81e9dc4ee9e; a three-query batch byte-identical to the standalone runs at --gpus 1, at --gpus 2, and with --index.
…ries per run One release note covering rounds 83-96: reference-locality bucketing with its runtime autotune, the sparse chunk walk, the unit-level static partition across GPUs, `--query-list` (now multi-GPU), the on-disk reference index, `hspZ hits-estimate`, the opt-in `-B 0` layout, the ledger and receiver checks, and the foundations under them. Every arm of every measurement quoted there produced byte-identical output at the same plan and cap.
Retitles the knob table to 0.0.4, documents multi-GPU batching and the `hspZ index` / `--index` workflow with their measured numbers, adds `index` to the CLI reference generator and the covered-commands list, and regenerates docs/src/generated/cli-reference.txt from the 0.0.4 binary (all five subcommands, no references to the experimental planning file).
Three leaks the 0.0.4-0.0.6 release branches had introduced, all of which v0.0.3 was free of: * `--ld-preload` and `--ld-library-path` defaulted to one workstation's ZLUDA shim and library directory, so the paths appeared in `--help` and every other host inherited a broken oracle invocation. They default to empty again, and `compare` overrides only what the caller actually set — passing an empty value through `Command::env` *clears* the inherited variable, which breaks the oracle wherever LD_LIBRARY_PATH is how it finds its CUDA runtime. * 71 source comments and one printed census line cited the experimental tree's planning file by section. Each sentence is kept; the citation is gone. * Two comments carried an internal convention marker. The fixture-backed tests no longer hardcode scratch directories: they read HSPZ_FIXTURE_CHR20 / HSPZ_FIXTURE_R90 and skip when those are unset. Re-verified: 174 tests; chr20 70152625c1a75cb3; 5x3 at --gpus 2 with the unit partition cb38f81e9dc4ee9e; the three-query batch byte-identical to the standalone runs at --gpus 1, at --gpus 2 and with --index; hits-estimate 192,899,566; hspZ --version 0.0.4; the regenerated CLI reference names no path and no planning section.
… index/hits-estimate; docs audit
…, verified in background
…ilder needs libffi-dev
….1% wall, output identical)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Single release for everything tested since 0.0.3: rounds 83–96 of the experimental tree, collapsed into one branch, one changelog entry and this PR. It replaces the earlier stack (#5, #6, #7, now closed) and adds two mechanisms those did not carry: the on-disk reference index and multi-GPU batching.
Sources are
alejandrogzi/hspZexperimental@dd1b1d2byte for byte, with the production packaging re-applied (file headers, clap attribute,hspZspelling, and help text free of references to the experimental tree's planning file — the last of those restores what v0.0.3 shipped and the earlier stack had lost).What a user gets
HSPZ_REF_BUCKETS=1|0forces either path.--query-list: one reference, many queries, one process. 100 queries of 5 Mbp against hg38 take 10,400–11,600 s as separate runs and 1,890–1,900 s batched on one L4 (−82%); on two 4090s the batch is 497 s at--gpus 1and 255 s at--gpus 2(−48%).hspZ index/run --index DIR: build a reference's tables once, load them afterwards. The same 100 queries as separate processes fall from 10,800 s to 2,680 s on an L4 (−75%), after a one-off 120 s index of 8.9 GB.hspZ hits-estimate: exact per-unit seed-hit counts before any GPU work, for planning a multi-GPU schedule.-B 0layout, the per-unit ledger,schedule:lines, receiver completeness checks, and the--max-hits/ physical-capacity separation that admits the whole-genome tail.Every arm of every measurement above produced byte-identical output at the same plan and cap.
Verification on this branch
cargo oxide build -- --releasecargo oxide test -- --releasehspZ --versionhspZ 0.0.4(matches the tag the release workflow asserts)--max-hits 2000000 --gpus 170152625c1a75cb3, 3,638 HSPs--gpus 2, unit partitioncb38f81e9dc4ee9e--gpus 1, at--gpus 2, and with--indexhits-estimateon chr20--timeledger--helpNative measurements behind the numbers: NVIDIA L4 (Delta), 2× and 4× RTX 4090 (rented), 2× Tesla T4 (Kaggle), AMD via ZLUDA for exactness.