Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 98 additions & 58 deletions skills/bionemo-parabricks/BENCHMARK.md

Large diffs are not rendered by default.

12 changes: 7 additions & 5 deletions skills/bionemo-parabricks/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: >-
license: CC-BY-4.0 AND Apache-2.0
metadata:
author: "Angel Pizarro <apizarro@nvidia.com>"
version: "1.3.0"
version: "1.4.0"
tags:
- parabricks
- genomics
Expand Down Expand Up @@ -49,13 +49,13 @@ or wiring optional GPU branches. For pipeline-level work, use
- **Execution:** The documented Docker commands require a supported Linux host,
an NVIDIA GPU with sufficient memory, a compatible NVIDIA driver, Docker,
NVIDIA Container Toolkit, and access to the selected Parabricks image.
Pulling the image may require an NGC API key for authentication to `nvcr.io`.
Pulling the image may require an NGC API key for authentication to [nvcr.io](https://nvcr.io).
See [runtime-environment.md](references/runtime-environment.md) for
version-specific requirements and checks.

Tool scope: read this skill's references and user-supplied manifests; write only
requested command artifacts. Use the bundled helpers for translation or an
authorized host inspection. Documentation lookups use `docs.nvidia.com`; image
authorized host inspection. Documentation lookups use [docs.nvidia.com](https://docs.nvidia.com); image
access uses the selected registry only for authorized container execution.
These instructions do not grant tool permissions; the host's permission policy
still applies to shell, file, network and optional documentation MCP access.
Expand Down Expand Up @@ -85,9 +85,9 @@ clinical interpretation, or promise exact runtime without benchmark data.

## Instructions

1. Identify the requested release. The option references cover v4.7.1; examples
1. Identify the requested release. The option references cover v4.8.0; examples
from another release are labeled. A software version and image tag can
differ: NVIDIA's 4.7.1 image is tagged `4.7.1-1`. Preserve the user's verified
differ: NVIDIA's 4.8.0 image is tagged `4.8.0-1`. Preserve the user's verified
image selection; use [command-conventions.md](references/command-conventions.md)
for the shared launch and validation contract.
2. Classify the request:
Expand Down Expand Up @@ -209,3 +209,5 @@ the `pbrun` portion; the shared container envelope is documented once in
<https://docs.nvidia.com/clara/parabricks/get-started>
- Overview:
<https://docs.nvidia.com/clara/parabricks/about-parabricks>
- 4.8.0 release notes (changes from 4.7.1):
<https://docs.nvidia.com/clara/parabricks/v4.8.0/about-parabricks/release-notes/4-8-0-1-release-notes>
157 changes: 131 additions & 26 deletions skills/bionemo-parabricks/evals/evals.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion skills/bionemo-parabricks/evals/files/rna-batch.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"parabricks_version": "4.7.1",
"parabricks_version": "4.8.0",
"reference_assets": "The FASTA and STAR library use the same reference sequences and annotation; the library was built by STAR 2.7.2a. Input data paths are deployment paths, not local files to probe.",
"jobs": [
{
Expand Down
2 changes: 1 addition & 1 deletion skills/bionemo-parabricks/evals/files/rna-contracts.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"parabricks_version": "4.7.1",
"parabricks_version": "4.8.0",
"reference_assets": "The FASTA and STAR library use the same reference sequences and annotation; the library was built by STAR 2.7.2a. Input data paths are deployment paths, not local files to probe.",
"jobs": [
{
Expand Down
14 changes: 8 additions & 6 deletions skills/bionemo-parabricks/references/command-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ their Tools column for the selected command.

Tool examples give the command inside the container. Add one Docker envelope
only when requested; preserve a requested bare `pbrun` command or argument array.
Use an exact NVIDIA release tag (4.7.1 uses `4.7.1-1`), never `latest` or an
Use an exact NVIDIA release tag (4.8.0 uses `4.8.0-1`), never `latest` or an
implicit tag. For reproducible execution, resolve that release to its trusted
NGC manifest digest and record the digest with the run. An image's local
`RepoDigests` after an authorized pull can identify the downloaded manifest;
Expand Down Expand Up @@ -48,11 +48,11 @@ consent rules. Preparing a workload command does not authorize executing it.

## Translating options

The mapping tables primarily describe Parabricks 4.7.1. Check each selected
release's NVIDIA tool manual before finalizing syntax, defaults, optional flags,
input modes, or output naming. An upstream option without a documented mapping
has no assumed Parabricks equivalent; do not pass it through or borrow an option
from another `pbrun` tool. Flag availability does not establish biological parity.
Check each selected release's NVIDIA tool manual before finalizing syntax,
defaults, optional flags, input modes, or output naming. An upstream option
without a documented mapping has no assumed Parabricks equivalent; do not pass
it through or borrow an option from another `pbrun` tool. Flag availability
does not establish biological parity.
In option tables, **—** in the baseline column denotes a Parabricks-only control.
If an upstream option is absent from both the tool-specific table and its
scoped rows in `shared-options.md`, treat it as unmapped until the selected
Expand Down Expand Up @@ -106,6 +106,8 @@ or outputs. For example, `rna_fq2bam` needs both `--output-dir` for generated
STAR outputs and `--out-bam` for the final BAM. A Docker output mount does not
supply either argument. Retain explicit placeholders when a required path is
unknown. Check this from the command text without launching a container.
Appending `--dry-run` has `pbrun` validate the arguments and input paths
without a GPU; it is still a container launch that needs authorization.

Before execution, check that every input and required index resolves through
the mounts and that reference, known-sites, model, and graph resources are
Expand Down
14 changes: 14 additions & 0 deletions skills/bionemo-parabricks/references/minimap2-controls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Minimap2 pipeline controls

These v4.8.0 controls apply only to the commands named in the **Tools** column.
Combine them with the selected tool's option mapping and its scoped rows in
[shared-options.md](shared-options.md) for inputs, intervals, model files,
output, and memory limits. Use the release and execution rules in
[command-conventions.md](command-conventions.md).

| Option | Role | Tools |
| --- | --- | --- |
| `--pbmm2`, `--pbmm2-unmapped` | Compatibility controls for pbmm2-style output and unmapped records. | `minimap2`, `pacbio_germline` |
| `--standalone-bqsr` | Pipeline mode control; for `minimap2`, runs BQSR after sorted BAM generation. | `minimap2`, `ont_germline`, `pacbio_germline` |
| `--nstreams`, `--max-queue-*`, `--chunk-size`, `--estimated-*`, `--stage2-large-chaining-size` | GPU pipeline tuning, including `--max-queue-chunks`, `--max-queue-reads`, `--estimated-read-length`, and `--estimated-anchor-per-read`. For `minimap2`, streams and chunk size default to `auto`, tuned from device memory and preset. Set values only for benchmarked tuning. | `minimap2`, `ont_germline`, `pacbio_germline` |
| `--seeding-chaining-on-gpu on\|off\|auto` | GPU seeding and chaining; `on` may help CPU-bound alignment. `auto` leaves ONT and PacBio presets on CPU; `minimap2` selects GPU only for `splice:sr`. Replaces 4.7.1's bare `--chaining-on-gpu` switch, now a deprecated alias that requires a value. | `minimap2`, `ont_germline`, `pacbio_germline` |
2 changes: 1 addition & 1 deletion skills/bionemo-parabricks/references/pbrun-bam2fq.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

## Preserving reads during conversion

For Parabricks 4.7.0, paired reads use `--out-suffixF` and `--out-suffixF2`.
Paired reads use `--out-suffixF` and `--out-suffixF2`.
Orphan first mates, orphan second mates, and unpaired reads are **ignored**
unless `--out-suffixO`, `--out-suffixO2`, and `--out-suffixS`, respectively, are
supplied. A single unpaired output does not capture both orphan categories.
Expand Down
3 changes: 2 additions & 1 deletion skills/bionemo-parabricks/references/pbrun-deepsomatic.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ selected version before finalizing.

## Gotchas

Parabricks DeepSomatic `--gvcf` option actually produces both .g.vcf and .vcf files with the same name. Do not generate separate commands for gvcf and vcf outputs.
Parabricks `deepsomatic` has no `--gvcf` option; write the VCF with
`--out-variants`.

## DeepSomatic Option Mapping

Expand Down
7 changes: 4 additions & 3 deletions skills/bionemo-parabricks/references/pbrun-deepvariant.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,12 @@ version.
Parabricks DeepVariant `--gvcf` option actually produces both .g.vcf and .vcf files with the same name. Do not generate separate commands for gvcf and vcf outputs.

The small model is enabled by default. Do not add `--enable-small-model`; that
flag is not in the current tool reference. For WES, pass `--disable-small-model`
together with `--use-wes-model` when matching Google DeepVariant.
flag is not in the current tool reference. For WES, `--use-wes-model` also
disables the small model, matching Google DeepVariant; in 4.7.1, also pass
`--disable-small-model`.

With both `--proposed-variants` and `--gvcf`, Parabricks warns and ignores
`--run-partition`; the [4.7.1 manual](https://docs.nvidia.com/clara/parabricks/tool-reference/tools/deepvariant)
`--run-partition`; the [manual](https://docs.nvidia.com/clara/parabricks/tool-reference/tools/deepvariant)
also warns of a slowdown for that combination. Preserve the requested candidate
input and gVCF output, and omit the ignored partition flag. Removing it does not
prove a speedup, and increasing GPU streams cannot restore partitioning. Measure
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Use this when translating BWA-MEM plus Google DeepVariant germline workflows to
`pbrun deepvariant_germline`. Preprocessing and shared DeepVariant mappings for
`deepvariant_germline` are in [shared-options.md](shared-options.md). The small
model is enabled by default; use `--disable-small-model` to turn it off. For WES,
pass `--disable-small-model` with `--use-wes-model` to match Google DeepVariant.
`--use-wes-model` also disables it, matching Google DeepVariant; in 4.7.1, also
pass `--disable-small-model`.

| Baseline option | `pbrun deepvariant_germline` equivalent | Notes |
| --- | --- | --- |
Expand Down
1 change: 1 addition & 0 deletions skills/bionemo-parabricks/references/pbrun-fq2bam.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ markdup/BQSR, intervals, GPU write controls, and similar) are in
| Single-end FASTQ input | `--in-se-fq` | Single-end FASTQ input. |
| FASTQ manifest or file list | `--in-fq-list`, `--in-se-fq-list` | Parabricks manifest; no direct BWA-MEM flag. |
| `bwa mem -t` | `--bwa-cpu-thread-pool`, `--num-cpu-threads-per-stage`, `--bwa-primary-cpus` | Partial; CPU controls split across stages. |
| `bwa mem -SP5M` for Hi-C | `--bwa-options="-SP5M"` | Add other supported BWA-MEM options to the same string. |
| `bwa mem` alignment-only output | `--align-only` | Stops after BWA-MEM; no sort/markdup. |
| GATK/Picard `SortSam --SORT_ORDER coordinate` | Default `fq2bam` behavior | Coordinate sorting is part of the normal workflow. |
| GATK/Picard `MarkDuplicates -I` | Implicit sorted intermediate | Sorted alignments feed duplicate marking unless disabled. |
Expand Down
8 changes: 4 additions & 4 deletions skills/bionemo-parabricks/references/pbrun-germline.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
4. For preprocessing-only questions, route to
`pbrun-fq2bam.md` and related FASTQ/BAM references; for runtime readiness, use
`runtime-environment.md`.
For prepared BAM/CRAM input, use `pbrun-haplotypecaller.md`; the 4.7.1
germline pipeline does not document `--in-bam`.
For prepared BAM/CRAM input, use `pbrun-haplotypecaller.md`; the germline
pipeline has no `--in-bam` option.

## Command Shape

Expand All @@ -42,7 +42,7 @@ markdup/BQSR, intervals, and shared execution controls) for `germline` are in
| GATK `HaplotypeCaller --output`, `-O` | `--out-variants` | VCF/gVCF output. |
| GATK `HaplotypeCaller --emit-ref-confidence GVCF` | `--gvcf` | gVCF mode. |
| GATK `HaplotypeCaller --sample-ploidy` | `--ploidy` | Haploid/diploid where documented. |
| GATK `HaplotypeCaller` annotation/output-mode/pruning controls | `--haplotypecaller-options` | Pass supported HaplotypeCaller options as one string. |
| GATK `HaplotypeCaller` annotation/output-mode/pruning controls | `--haplotypecaller-options` | Pass supported HaplotypeCaller options as one string. `AS_*` annotations require `--gvcf`. |
| `--java-options`, GATK engine/common flags | No direct equivalent | Not exposed as GATK engine controls. |

## germline Options Without BWA-MEM/GATK Equivalents
Expand All @@ -54,7 +54,7 @@ Shared Parabricks pipeline and GPU controls for `germline` are in
| --- | --- |
| `--run-partition` | Genome partitions for HaplotypeCaller (one process per partition). Not fq2bam GPU-process partitioning. |
| `--bwa-gpu-num-per-partition` | BWA GPUs per worker when `--run-partition` is set (default: 2; must divide `--num-gpus`). |
| `--num-streams-per-gpu` | HaplotypeCaller streams (default: 1). Not a partition flag. |
| `--num-streams-per-gpu` | HaplotypeCaller streams (default: `auto`). Not a partition flag. |

## Validation

Expand Down
20 changes: 18 additions & 2 deletions skills/bionemo-parabricks/references/pbrun-giraffe.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,29 @@ pbrun giraffe \
Always verify exact graph resource flags and output options against the selected
version before finalizing. Do not guess graph/index file combinations.

For **4.7.0**, the GBZ graph, distance index, minimizer index, and zipcodes file
The GBZ graph, distance index, minimizer index, and zipcodes file
are required inputs. Supply `--zipcodes-name` explicitly; do not infer a
compatible zipcodes file from a minimizer filename. Use `--ref-paths` when the
output BAM needs a supplied path order or sequence dictionary.

Giraffe uses vg v1.75.1: generate index files with vg v1.75.1, regenerating any
built with an older vg, and supply a GBZ version 1 graph. vg 1.72.0 and later
write version 2 by default, which fails to load; convert it with
`vg gbwt -Z graph.gbz -g graph-v1.gbz --gbz-v1` (vg 1.74.0 or later). Indexes
built from the same graph stay valid after that conversion. In 4.7.1, giraffe
uses vg v1.70.0; build its indexes with that version.

To produce a **coordinate-sorted BAM without duplicate marking**, use
`--no-markdups`. `--align-only` stops before coordinate sorting and therefore
does not meet that output requirement. Preserve the supplied sample and all
read-group fields using the Giraffe-specific flags in the mapping below.

An `--out-bam` path ending in `.cram` writes CRAM. It requires a
`--ref` FASTA that matches the graph and the paths in `--ref-paths`, cannot be
combined with `--align-only`, and `--run-ref-verification` is recommended the
first time a reference is used. Dropping `--align-only` for CRAM also turns on
duplicate marking; add `--no-markdups` if duplicates should stay unmarked.

## Performance Guidance

Prefer the documented automatic stream selection for general commands: leave
Expand Down Expand Up @@ -73,8 +86,9 @@ equivalent.
| `vg giraffe` fragment length controls | `--max-fragment-length`, `--fragment-mean`, `--fragment-stdev` | Same fragment-distribution role. |
| `vg giraffe --max-multimaps`, `-M` | `--max-multimaps` | Max alignments per read (default: 1). |
| `vg giraffe --copy-comment` | `--copy-comment` | Appends FASTQ comment to BAM output via auxiliary tag. |
| `vg giraffe --supplementary` | `--supplementary` | Identify and report supplementary alignments (off by default). |
| Alignment-only output | `--align-only` | Stops after `vg giraffe` alignment output; does not coordinate-sort. |
| GATK/Picard `SortSam --OUTPUT`, `-O` | `--out-bam` | Final BAM output path. |
| GATK/Picard `SortSam --OUTPUT`, `-O` | `--out-bam` | Final BAM output path, or CRAM for a `.cram` path. |
| GATK/Picard duplicate-marking behavior | `--markdups-*`, `--optical-duplicate-pixel-distance`, `--no-markdups` | Parabricks exposes the duplicate-marking controls documented for this pipeline. |
| GATK/Picard `--version` | `--version` | Same version-reporting role. |
| Upstream `vg giraffe` options not listed here | No direct equivalent | Not exposed by current Parabricks docs for `pbrun giraffe`. |
Expand All @@ -88,6 +102,8 @@ equivalent.
| `--nstreams`, `--num-cpu-threads-per-gpu`, `--batch-size`, `--write-threads`, `--work-queue-capacity` | Parabricks GPU/CPU pipeline scheduling controls. Prefer `--nstreams auto` for default guidance; use integer stream counts only for benchmarked/manual tuning. |
| `--minimizers-gpu` | Parabricks GPU offload for minimizers/seeds in supported single-end runs. |
| `--memory-limit`, `--low-memory` | Parabricks host/GPU memory controls for the pipeline. |
| `--in-se-cram`, `--in-cram-ref`, `--cram-read-threads` | Single-end CRAM input; aligned reads are re-aligned and need `--in-cram-ref` to decode. Paired-end CRAM is not supported. |
| `--ultima-tags`, `--tp-output-order` | Carry Ultima per-base `tp`/`t0` tags from `--in-se-cram` input to the output. |

## Validation

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,11 @@ GPU-accelerated HaplotypeCaller counterpart, but the CLI is not one-to-one:
some GATK flags are direct Parabricks flags, while other supported original
HaplotypeCaller options must be passed through `--haplotypecaller-options`.
Use the spelling documented for that inner parser, which differs from GATK's
top-level CLI for some options. For example, 4.7.1 accepts one quoted value
`'-min-pruning 4 -pcr-indel-model NONE'`; those are single-hyphen inner flags.
top-level CLI for some options. For example, the inner parser takes one quoted
value `'-min-pruning 4 -pcr-indel-model NONE'`; those are single-hyphen inner
flags. Allele-specific (`AS_*`) annotations, whether from `-G` or from `-A`
inside `--haplotypecaller-options`, require `--gvcf`; Parabricks rejects them
during argument checking.
`--in-recal-file` applies the supplied recalibration table during calling;
creating that table earlier did not itself rewrite the input BAM. A separate
ApplyBQSR BAM is unnecessary when using this caller input.
Expand Down Expand Up @@ -66,7 +69,7 @@ ApplyBQSR BAM is unnecessary when using this caller input.
| `--adaptive-pruning` | Parabricks-exposed graph pruning control. |
| `--force-call-filtered-alleles` | Parabricks force-calling behavior tied to its documented allele input. |
| `--filter-reads-too-long`, `--no-alt-contigs` | Parabricks read/contig filtering conveniences. |
| `--htvc-low-memory`, `--num-htvc-threads`, `--num-streams-per-gpu`, `--run-partition`, `--gpu-num-per-partition` | Parabricks GPU/partition performance controls. `--num-streams-per-gpu` default 1. |
| `--htvc-low-memory`, `--num-htvc-threads`, `--num-streams-per-gpu`, `--run-partition`, `--gpu-num-per-partition` | Parabricks GPU/partition performance controls. `--num-streams-per-gpu` defaults to `auto`. |

## Validation

Expand Down
Loading
Loading