Skip to content

BAM submission for long-read raw data #117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
3 commits merged into from
Oct 27, 2026
Merged

BAM submission for long-read raw data #117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
63640bf
first draft for bam file as raw read input, uses draft containers and…
Vkale1 Oct 15, 2026
16962f7
change test datasets path, symlink read files instead of copy, update…
Vkale1 Oct 16, 2026
d2f952d
Merge branch 'dev' into feature/long-read-submission
KateSakharova Oct 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 74 additions & 7 deletions assets/schema_input_reads.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,21 @@
"meta": ["sample_accession"]
},
"fastq_1": {
"type": "string",
"format": "file-path",
"exists": true,
"pattern": "^\\S+\\.(fq|fastq)(\\.gz)?$",
"anyOf": [
{
"type": "string",
"format": "file-path",
"exists": true,
"pattern": "^\\S+\\.(fq|fastq)(\\.gz)?$"
},
{
"type": "string",
"maxLength": 0
}
],
"default": null,
"errorMessage": "FASTQ file must have extension '.fq' or '.fastq' (optionally gzipped)",
"description": "Path to single-end/paired-end forward reads in FASTQ format (optionally gzipped)"
"description": "Path to single-end/paired-end forward reads in FASTQ format (optionally gzipped). Leave empty if reads are provided as BAM"
},
"fastq_2": {
"anyOf": [
Expand All @@ -46,6 +55,23 @@
"errorMessage": "FASTQ file for reverse reads must have extension '.fq' or '.fastq' (optionally gzipped)",
"description": "Path to reverse reads in FASTQ format for paired-end data (optionally gzipped). Leave empty for single-end reads"
},
"bam": {
"anyOf": [
{
"type": "string",
"format": "file-path",
"exists": true,
"pattern": "^\\S+\\.bam$"
},
{
"type": "string",
"maxLength": 0
}
],
"default": null,
"errorMessage": "BAM file must have extension '.bam'",
"description": "Path to reads in unaligned or aligned BAM format. Use instead of fastq_1/fastq_2"
},
"platform": {
"type": "string",
"enum": [
Expand Down Expand Up @@ -223,12 +249,53 @@
"required": [
"id",
"sample_accession",
"fastq_1",
"platform",
"instrument",
"library_source",
"library_selection",
"library_strategy"
]
],
"oneOf": [
{
"properties": {
"fastq_1": {
"type": "string",
"minLength": 1
}
},
"required": ["fastq_1"]
},
{
"properties": {
"bam": {
"type": "string",
"minLength": 1
}
},
"required": ["bam"]
}
],
"if": {
"properties": {
"bam": {
"type": "string",
"minLength": 1
}
},
"required": ["bam"]
},
"then": {
"properties": {
"fastq_1": {
"maxLength": 0,
"errorMessage": "fastq_1 must be empty when reads are provided as BAM"
},
"fastq_2": {
"maxLength": 0,
"errorMessage": "fastq_2 must be empty when reads are provided as BAM"
}
}
},
"errorMessage": "Reads must be provided either as FASTQ (fastq_1, optionally fastq_2) or as a single BAM (bam, with fastq_1 and fastq_2 empty)"
}
}
6 changes: 6 additions & 0 deletions conf/modules.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,12 @@ process {
]
}

// Reads can be multi or 100s of GB (e.g. long-read BAMs): symlink instead of copying them before upload.
// assemblies, bins and mags modes keep the default stageInMode 'copy'.
withName: '.*:READSUBMIT:SUBMIT' {
stageInMode = 'symlink'
}

withName: 'REGISTERSTUDY|CREATE_READS_MANIFEST' {
publishDir = [
enabled: false
Expand Down
34 changes: 34 additions & 0 deletions conf/test_reads_long_bam.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/seqsubmit -profile test_reads_long_bam,<docker/singularity> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {
resourceLimits = [
cpus: 2,
memory: '8.GB',
time: '1.h'
]
}

params {
config_profile_name = 'Test --mode reads long-read BAM profile'
config_profile_description = 'Minimal test profile for long-read BAM reads submission'

// Input data
input = params.pipelines_testdata_base_path + 'seqsubmit/samplesheets/samplesheet_reads_bam.csv'
outdir = 'test_output'

mode = "reads"
submission_study = "PRJEB98843"
centre_name = "TEST_CENTER"

test_upload = true
}
39 changes: 21 additions & 18 deletions docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,25 +156,27 @@ You will need to create a samplesheet with information about the raw reads you w
Example:

```csv title="samplesheet_reads.csv"
id,sample_accession,fastq_1,fastq_2,platform,instrument,library_source,library_selection,library_strategy,insert_size,library_name,description
illumina_run_001,SAMEA1234567,data/reads_R1.fastq.gz,data/reads_R2.fastq.gz,ILLUMINA,Illumina HiSeq 2000,GENOMIC,RANDOM,WGS,500,HiSeq_library_001,Illumina sequencing of sample XYZ
pacbio_run_001,SAMEA7654321,data/pacbio_reads.fastq.gz,,PACBIO_SMRT,PacBio Sequel,GENOMIC,RANDOM,WGS,,PacBio_library_002,Long-read sequencing
id,sample_accession,fastq_1,fastq_2,bam,platform,instrument,library_source,library_selection,library_strategy,insert_size,library_name,description
illumina_run_001,SAMEA1234567,data/reads_R1.fastq.gz,data/reads_R2.fastq.gz,,ILLUMINA,Illumina HiSeq 2000,GENOMIC,RANDOM,WGS,500,HiSeq_library_001,Illumina sequencing of sample XYZ
pacbio_run_001,SAMEA7654321,data/pacbio_reads.fastq.gz,,,PACBIO_SMRT,PacBio Sequel,GENOMIC,RANDOM,WGS,,PacBio_library_002,Long-read sequencing
nanopore_run_001,SAMEA7654322,,,data/nanopore_reads.bam,OXFORD_NANOPORE,MinION,METAGENOMIC,RANDOM,WGS,,ONT_library_003,Long-read sequencing provided as BAM
```

| Column | Required | Description |
| ------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Yes | Unique identifier of this particular data entry. Used as an experiment name. |
| `sample_accession` | Yes | ENA sample accession (starting with SAMEA) of the sample used to generate raw reads. Samples must already be submitted to ENA. |
| `fastq_1` | Yes | Path to single-end/paired-end forward reads in FASTQ format (optionally gzipped). |
| `fastq_2` | No | Path to reverse reads in FASTQ format for paired-end data (optionally gzipped). Leave empty for single-end reads. |
| `platform` | Yes | Sequencing platform. Must be one of the ENA controlled vocabulary values listed in [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation). Supported values: `ILLUMINA`, `PACBIO_SMRT`, `OXFORD_NANOPORE`, `ION_TORRENT`, `CAPILLARY`, `DNBSEQ`, `ELEMENT`, `GENAPSYS`, `GENEMIND`, `HELICOS`, `LS454`, `BGISEQ`, `ULTIMA`, `VELA_DIAGNOSTICS`. |
| `instrument` | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-instrument). |
| `library_source` | Yes | Library source type. Options: `GENOMIC`, `METAGENOMIC`, `TRANSCRIPTOMIC`, `METAGENOMIC SINGLE CELL`, `TRANSCRIPTOMIC SINGLE CELL`, `SYNTHETIC`, `VIRAL RNA`, `OTHER`. ENA [documantation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-source). |
| `library_selection` | Yes | Library selection method. Options: `RANDOM`, `PCR`, `RANDOM PCR`, `RT-PCR`, `MF`, `cDNA`, `cDNA_randomPriming`, `cDNA_oligo_dT`, `PolyA`, `Inverse rRNA`, `ChIP`, `MNase`, `DNase`, `Hybrid Selection`, etc. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-selection) for complete list. |
| `library_strategy` | Yes | Library strategy. Options: `WGS`, `WGA`, `WXS`, `RNA-Seq`, `miRNA-Seq`, `ncRNA-Seq`, `EST`, `Hi-C`, `ATAC-seq`, `WCS`, `RAD-Seq`, `CLONE`, `AMPLICON`, `POOLCLONE`, `etc`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-strategy) for complete list. |
| `insert_size` | No | Fragment/insert size for paired-end reads (e.g., 500 for 500 bp inserts). Leave empty if not applicable. |
| `library_name` | No | Descriptive library name (optional). |
| `description` | No | Free-text description of the experiment (optional). |
| Column | Required | Description |
| ------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id` | Yes | Unique identifier of this particular data entry. Used as an experiment name. |
| `sample_accession` | Yes | ENA sample accession (starting with SAMEA) of the sample used to generate raw reads. Samples must already be submitted to ENA. |
| `fastq_1` | Conditional | Path to single-end/paired-end forward reads in FASTQ format (optionally gzipped). Required unless `bam` is provided. |
| `fastq_2` | No | Path to reverse reads in FASTQ format for paired-end data (optionally gzipped). Leave empty for single-end reads and if `bam` is provided. |
| `bam` | Conditional | Path to reads in BAM format (`.bam`). Required unless `fastq_1` is provided. |
| `platform` | Yes | Sequencing platform. Must be one of the ENA controlled vocabulary values listed in [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation). Supported values: `ILLUMINA`, `PACBIO_SMRT`, `OXFORD_NANOPORE`, `ION_TORRENT`, `CAPILLARY`, `DNBSEQ`, `ELEMENT`, `GENAPSYS`, `GENEMIND`, `HELICOS`, `LS454`, `BGISEQ`, `ULTIMA`, `VELA_DIAGNOSTICS`. |
| `instrument` | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-instrument). |
| `library_source` | Yes | Library source type. Options: `GENOMIC`, `METAGENOMIC`, `TRANSCRIPTOMIC`, `METAGENOMIC SINGLE CELL`, `TRANSCRIPTOMIC SINGLE CELL`, `SYNTHETIC`, `VIRAL RNA`, `OTHER`. ENA [documantation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-source). |
| `library_selection` | Yes | Library selection method. Options: `RANDOM`, `PCR`, `RANDOM PCR`, `RT-PCR`, `MF`, `cDNA`, `cDNA_randomPriming`, `cDNA_oligo_dT`, `PolyA`, `Inverse rRNA`, `ChIP`, `MNase`, `DNase`, `Hybrid Selection`, etc. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-selection) for complete list. |
| `library_strategy` | Yes | Library strategy. Options: `WGS`, `WGA`, `WXS`, `RNA-Seq`, `miRNA-Seq`, `ncRNA-Seq`, `EST`, `Hi-C`, `ATAC-seq`, `WCS`, `RAD-Seq`, `CLONE`, `AMPLICON`, `POOLCLONE`, `etc`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-library-strategy) for complete list. |
| `insert_size` | No | Fragment/insert size for paired-end reads (e.g., 500 for 500 bp inserts). Leave empty if not applicable. |
| `library_name` | No | Descriptive library name (optional). |
| `description` | No | Free-text description of the experiment (optional). |

### `reads` parameters and example

Expand All @@ -197,7 +199,8 @@ nextflow run nf-core/seqsubmit \

`reads` mode does not support the following:

- **Reads in non-FASTQ formats**: Only reads provided as FASTQ (`.fastq.gz`) can be submitted. Other formats (e.g. BAM) are not supported as input.
- **Other read formats**: Reads must be provided as FASTQ (`.fastq`/`.fq`, optionally gzipped) or BAM (`.bam`). Other formats (e.g. CRAM or `.tar.gz` archives) are not supported as input.
- **Mixed formats in one row**: A row cannot combine FASTQ and BAM files. Put them in separate rows, which are submitted as separate ENA runs.

## `metagenomic_assemblies` mode

Expand Down
Loading
Loading