An end-to-end Nextflow DSL2 pipeline for Oxford Nanopore (ONT) metagenomics of low-biomass microbiomes
Built for respiratory samples (sputum, BAL): host removal, taxonomy, negative-control-aware contamination
assessment, resistome / virulome / plasmid profiling and a MERIDIAN downstream report, with multi-batch pooling built in.

---

✨ Why PNEUMA

PNEUMA targets low-biomass microbiomes, for example respiratory/airway samples such as sputum and bronchoalveolar lavage (BAL). These samples are dominated by host DNA and carry little microbial signal, so reagent and environmental contaminants can rival the real community, and cohorts are often sequenced across several flow cells and batches. Turning ONT reads into a defensible taxonomic and resistome picture normally means stitching together a dozen tools by hand and re-deriving contamination and pooling decisions for every study. PNEUMA builds the low-biomass safeguards into the workflow:

• Host depletion: Hostile removes host reads before classification, so the remaining reads and the depth statistics describe the microbiome rather than the host.
• Negative-control-aware contamination assessment: when the samplesheet marks negative controls, Recentrifuge subtracts control taxa from the samples.
• Low-depth flagging: samples with fewer than --min_reads reads after host removal receive a LOW_DEPTH flag instead of being silently over-interpreted.

PNEUMA is an end-to-end Nextflow DSL2 pipeline for ONT long-read metagenomics, built on the nf-core template. It targets cohort research: one samplesheet describes every sample × run, reads from the same sample are pooled across runs and batches, and the results are handed straight to MERIDIAN for statistics and a publication-ready report. Databases and tool versions are recorded in pipeline_info/ for provenance.

---

🗺️ Pipeline overview

The diagram is generated by docs/images/make_metro_map.py (standard library only). Each labelled block is a pipeline stage; the grey dashed block is planned, not yet implemented.

---

🚀 Features

Phase 1 (implemented)

• Input: a samplesheet plus ONT FASTQ, with multi-run / multi-batch pooling per sample.
• QC & host removal: cat (pool runs per sample), porechop_abi (optional adapter trimming), nanoq (length/quality filter), NanoPlot and seqkit stats (QC), Hostile host removal, and a LOW_DEPTH flag for samples with fewer than --min_reads reads after host removal. MultiQC summarises the whole run (read QC, host removal, Kraken2/Bracken, Recentrifuge, ABRicate hits and software versions) in one report (multiqc/multiqc_report.html).
• Taxonomic classification: Kraken2 (PlusPF index).
• Abundance estimation: Bracken, KrakenTools report combination and Krona.
• Contamination assessment: Recentrifuge in plain mode and, when the samplesheet marks negative controls, in control mode (negative-control subtraction).
• Functional profiling: ABRicate on Kraken-tagged reads with card (AMR), vfdb (virulence) and plasmidfinder (MGE) at 80 % identity / 80 % coverage.
• Downstream analysis: MERIDIAN config generation plus a run in the container ghcr.io/julio92-c/meridian:v1.1.0 (diversity, differential abundance, resistome/virulome/mobilome, report).
• Profiles: docker, singularity and test (add your cluster's scheduler settings with -c). In-pipeline database mode (--build_databases, then --db_dir) with a checksummed database_manifest.tsv; bin/prepare_databases.sh remains as a fallback.

Phase 2 (planned)

• Metagenome reconstruction: priority-pathogen binning/assembly, CheckM2, Bakta, MLST, and AMR/VF/MGE calls collected in a pathogen report.
• A WHO-priority respiratory panel in tiers 1, 2, 3 and D, plus a reservoir commensals list.

Phase 3 (planned)

• Validation: regression tests, spike-ins and a full cohort run.

---

📦 Installation

Requirements:

• Nextflow >= 25.10.4
• Docker or Singularity / Apptainer
• Reference databases: Kraken2 PlusPF index, NCBI taxdump, Hostile index and ABRicate databases

Run directly from GitHub (no clone needed). Pin a release tag with -r so the exact pipeline code is recorded and the run is reproducible:

nextflow pull Julio92-C/pneuma -r v0.2.2

Build the databases once, with the pipeline itself (--build_databases; no samplesheet and no analysis, only the download). On an HPC cluster, write to a large scratch filesystem rather than your home directory (home quotas are usually small):

nextflow run Julio92-C/pneuma -r v0.2.2 -profile <docker|singularity>   --build_databases --db_outdir /scratch/pneuma_db   --kraken_release <YYYYMMDD> --taxdump_month <YYYY-MM>

This downloads the Kraken2 PlusPF index (--kraken_release, see the Kraken2 index list), the NCBI taxdump (--taxdump_month, match the Kraken2 release month), the Hostile human-t2t-hla index and current ABRicate card, vfdb and plasmidfinder databases, each in a pinned container. Everything is copied into --db_outdir together with database_manifest.tsv (name, version, path, sha256 per database). Analysis runs then need a single parameter, --db_dir /scratch/pneuma_db, which fills --kraken2_db, --taxdump, --host_index, --abricate_db and --database_manifest; a parameter you pass explicitly (for example your own --kraken2_db) still wins. Use --skip_kraken_download, --skip_taxdump_download, --skip_hostile_download or --skip_abricate_download to reuse a database you already have. See docs/usage.md for details and an example HPC job script.

Fallback script. bin/prepare_databases.sh does the same outside Nextflow (it needs singularity, apptainer or docker on the node). Download it from the release tag you use rather than looking for it in the Nextflow assets directory (newer Nextflow keeps per-commit clones under ~/.nextflow/assets/.repos/<org>/<repo>/clones/<sha>/, so the path varies):

curl -fsSL -o prepare_databases.sh https://raw.githubusercontent.com/Julio92-C/pneuma/v0.2.0/bin/prepare_databases.sh
bash prepare_databases.sh --outdir /scratch/pneuma_db --kraken-release <YYYYMMDD> --taxdump-month <YYYY-MM>

It prints the --kraken2_db, --taxdump, --host_index, --abricate_db and --database_manifest parameters to pass (or point --db_dir at its --outdir: the layout is the same). --help lists the options, including --skip-kraken, --skip-taxdump, --skip-hostile and --skip-abricate.

Database sizes (approximate; Kraken2 PlusPF figures from the Kraken2 index list, the rest from the download URLs):

Database	Download	Disk after setup	Needed at run time
Kraken2 PlusPF (--kraken2_db)	85 GB (.tar.gz)	110 GB index (+ 85 GB tarball, deletable once extracted)	about 110 GB RAM (the index is loaded into memory; on a cluster set KRAKEN2_KRAKEN2 to about 110 GB, see docs/usage.md)
NCBI taxdump (--taxdump)	70 MB	under 1 GB	negligible
Hostile human-t2t-hla (--host_index)	1 GB	a few GB (minimap2 index built locally)	a few GB
ABRicate card, vfdb, plasmidfinder (--abricate_db)	tens of MB	under 1 GB	negligible

Plan for about 250 GB of free space while --build_databases runs (Nextflow's work directory holds the extracted index and the published copy lands in --db_outdir; the downloaded Kraken2 tarball is deleted once extracted). Use -work-dir on a big filesystem; once published, the work directory can be removed. The final --db_outdir is about 115 GB.

For development, clone the repository instead:

git clone https://github.com/Julio92-C/pneuma.git
cd pneuma

---

⚡ Quickstart

Check the installation with the bundled test profile (small public test data, MERIDIAN skipped):

nextflow run Julio92-C/pneuma -r v0.2.2 -profile test,docker --outdir results_test

On an HPC cluster use singularity (or apptainer) and pass your scheduler settings in a config file with -c (see docs/usage.md):

nextflow run Julio92-C/pneuma -r v0.2.2 -profile test,singularity -c my_cluster.config --outdir results_test

Run on your own data, choosing the profile docker or singularity:

nextflow run Julio92-C/pneuma -r v0.2.2 -profile <docker|singularity>   --input samplesheet.csv --outdir results   --db_dir <db_outdir>   --meridian_metadata metadata.csv --meridian_group <group_column> --meridian_random_effect <patient_column>   -resume

Always pin -r <tag> (for example -r v0.2.2) for reproducibility; without it Nextflow runs the default branch. --input and --outdir are required for an analysis run; --kraken2_db, --taxdump and --host_index are required unless --db_dir provides them, and --meridian_metadata / --meridian_group are required unless you pass --skip_meridian. Pass pipeline parameters on the command line or with -params-file, not in a custom -c config (keep -c for executor and resource settings). See docs/usage.md for HPC notes.

Nextflow 26.04 note. Its default strict-syntax parser passes command-line values as strings, so boolean and numeric flags (such as --build_databases or --skip_meridian) fail schema validation ("Value is [string] but should be [boolean]"). Put those parameters in a -params-file, or set NXF_SYNTAX_PARSER=v1 for the run; Nextflow 25.10 is not affected.

From a development clone, replace Julio92-C/pneuma -r v0.2.2 with ..

With any other Singularity/Apptainer setup (-profile singularity), set the image cache first so the containers are pulled once and reused (otherwise Nextflow warns that the cache directory has not been defined):

export NXF_SINGULARITY_CACHEDIR=/scratch/singularity_cache   # any writable, persistent directory

🧰 Help & version

nextflow run Julio92-C/pneuma -r v0.2.2 --help                 # all parameters, grouped
nextflow run Julio92-C/pneuma -r v0.2.2 --help kraken2_db      # details of one parameter
nextflow run Julio92-C/pneuma -r v0.2.2 --help_full --show_hidden   # also hidden parameters
nextflow run Julio92-C/pneuma -r v0.2.2 --version              # prints "Julio92-C/pneuma v0.2.0"
nextflow info Julio92-C/pneuma                                 # local revisions (tags, branches) of the pipeline
nextflow pull Julio92-C/pneuma -r v0.2.2                       # fetch or update a release (use the new tag to update)

(--help_full and --show_hidden are this pipeline's parameter names; the plugin's own help text mentions --helpFull and --showHidden, which are not wired up here.)

The software versions of every process in a run are recorded in <outdir>/pipeline_info/pneuma_software_mqc_versions.yml (and in the MultiQC report).

What -profile test uses. The test is a plumbing check of the workflow, not a biologically meaningful analysis: it runs three tiny samples (SARS-CoV-2 Illumina test reads, with one flagged as a negative control) against the nf-core test datasets, namely the sarscov2 Kraken2/Bracken database (kraken2_bracken.tar.gz), a human genome fragment (genome.fasta) as the Hostile host reference, and a small synthetic NCBI taxdump in tests/data/taxdump. ABRicate uses the databases bundled in the container (no --abricate_db), and MERIDIAN is skipped. Expect empty or near-empty taxonomy and ABRicate results.

Database parameters. --abricate_db is the directory that holds the ABRicate databases (the <db_outdir>/abricate folder written by --build_databases or prepare_databases.sh, containing card/, vfdb/ and plasmidfinder/). It is not a database name. The separate --abricate_dbs parameter is the comma-separated list of database names to search (default card,vfdb,plasmidfinder).

---

📥 Inputs (samplesheet)

A CSV with one row per sample × run (schema: assets/schema_input.json). A sample sequenced on several flow cells gets several rows with the same sample; their reads are concatenated.

Column	Required	Description
sample	yes	Sample name; must match ^[A-Za-z0-9][A-Za-z0-9-]*$ (no _ or .)
run_id	yes	Sequencing run / flow cell identifier (no spaces); sample + run_id must be unique
batch	yes	Batch identifier (no spaces)
site	no	Sampling site or cohort label
fastq	yes	Path to an existing .fastq.gz / .fq.gz file
is_control	yes	true for negative controls (used by Recentrifuge control mode), otherwise false

sample,run_id,batch,site,fastq,is_control
SAMPLE-01,RUN1,BATCH1,SITEA,/path/to/SAMPLE-01_run1.fastq.gz,false
SAMPLE-01,RUN2,BATCH1,SITEA,/path/to/SAMPLE-01_run2.fastq.gz,false
CONTROL-01,RUN1,BATCH1,SITEA,/path/to/CONTROL-01_run1.fastq.gz,true

Optional study metadata for MERIDIAN is a separate CSV passed with --meridian_metadata.

---

📤 Outputs

Results are written under --outdir:

Directory	Contents
preprocess/	nanoq stats, host-free reads (clean_reads/), NanoPlot, seqkit stats
taxonomy/	Kraken2 reports and per-read assignments, Bracken, combined reports, bracken_arranged.csv, Krona
contamination/	pneuma.rcf.data.csv, pneuma.rcf.html, pneuma_controls.rcf.* (when controls are present)
functional/	per-sample ABRicate tables and summary_reportClean.csv
meridian/	MERIDIAN config.yaml, Datasets/, Figures/, Reports/
multiqc/	MultiQC report of the whole pipeline, plus its custom tables
pipeline_info/	Nextflow reports, software versions, database_manifest.tsv

A file-by-file description is in docs/output.md.

---

🔗 MERIDIAN hand-off

PNEUMA ends where MERIDIAN begins. The MERIDIAN subworkflow rearranges the combined Kraken2/Bracken reports, the Recentrifuge table and the ABRicate summary into MERIDIAN's input layout, renders a config.yaml from assets/meridian_config.template.yaml (checking the metadata against the samples), and runs MERIDIAN in the container ghcr.io/julio92-c/meridian:v1.1.0 (the tag is the single parameter --meridian_container, so a MERIDIAN bump is a one-line change).

Parameter	Purpose
--meridian_metadata	study metadata CSV (one row per sample)
--meridian_group	metadata column holding the comparison groups; a comma-separated list (sample_type,treatment) runs one MERIDIAN comparison per column in parallel, each in meridian/<column>/ (study id <study_id>_<column>)
--meridian_random_effect	metadata column holding the repeated-measure / patient identifier
--meridian_study_id	study label (default pneuma)
--meridian_container	MERIDIAN image (default ghcr.io/julio92-c/meridian:v1.1.0); the only place the tag lives
--skip_meridian	stop after the PNEUMA stages

---

🧭 Roadmap

• Phase 1, core: input, QC and host removal, taxonomic classification, abundance estimation, contamination assessment, functional profiling, downstream analysis (MERIDIAN), HPC (Singularity) support and database staging.
• Phase 2, metagenome reconstruction: priority-pathogen binning/assembly, CheckM2, Bakta, MLST and an AMR/VF/MGE pathogen report and the WHO-priority respiratory panel (tiers 1, 2, 3, D) with a reservoir commensals list.
• Phase 3, validation: regression tests, spike-ins and a full cohort.

---

🗂️ Repository layout

main.nf, nextflow.config, nextflow_schema.json   entry point, defaults and parameter schema
workflows/pneuma.nf                              top-level workflow
subworkflows/local/                              preprocess, taxonomy, contamination, functional_reads, meridian
modules/local/ , modules/nf-core/                pipeline-specific and nf-core modules
conf/                                            profiles (test, containers) and publishDir rules
bin/                                             helper scripts (prepare_databases.sh, read tagging, ABRicate merge, ...)
assets/                                          samplesheet schema and examples, MERIDIAN config template
docs/                                            usage, output, images (metro map, logos)
tests/                                           nf-test files, pytest suite (tests/python), fixtures (tests/data)

---

🤝 Contributing

Development happens on feature branches; the pipeline follows nf-core conventions. See the contributing guidelines. Run the unit tests with:

pip install pytest && pytest tests/python -q
nf-test test --profile docker

The tiny ABRicate fixture database in tests/data/abricate_db/ was built with abricate --setupdb inside quay.io/biocontainers/abricate:1.0.1--ha8f3691_1. It must be regenerated if the ABRicate image is bumped (the BLAST index version can change). Fixture provenance is listed in tests/data/README.md.

---

📚 Citation

Manuscript in preparation.

Please also cite the tools used by the pipeline; they are listed with DOIs in CITATIONS.md. PNEUMA uses code and infrastructure from the nf-core community, reused under the MIT license.

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.

---

📬 Contact

👤 Julio C. Ortega Cambara

🎓 PhD Candidate — Computational Bioinformatics

🏛️ School of Biomedical Sciences, University of West London

📍 St Mary's Rd, London W5 5RF

✉️ Email: 32104617@student.uwl.ac.uk

---

📜 License

Released under the MIT License.