Skip to content

Allow BAM as input for long-reads submission #98

Closed

Description

@Vkale1

Description of feature

1. Long-read raw read submission (--mode reads)

Samplesheet schema (assets/schema_input_reads.json)

  • fastq_1 only accepts FASTQ. Add a new bam column, and require either fastq_1 or bam. A separate column is clearer than widening the fastq_1 pattern, since ENA treats BAM as its own manifest field.
  • Only allow bam when platform is OXFORD_NANOPORE or PACBIO_SMRT, and require fastq_2 to be empty for those platforms.
  • To confirm: leave out .tar.gz until ENA confirms whether it will still be supported.
    Workflow (workflows/readsubmit.nf)
  • Update the samplesheet mapping for the new bam column and record the file type (FASTQ or BAM) in meta.
    Manifest creation (modules/local/create_reads_manifest/main.nf)
  • Add BAM support so the manifest gets a BAM line instead of FASTQ lines.
  • To confirm: whether create_reads_manifest in mgnify-pipelines-toolkit (1.5.1) supports BAM.

2. Make BAM uploads retry cleanly

  • ENA_WEBIN_CLI_WRAPPER only retries on a few exit codes, once. Network failures during upload usually aren't retried. Add a withName block in conf/modules.config that retries on any failure, at least 3 times, with more time per attempt.
  • If an upload succeeds but the task fails afterwards, the retry will be rejected by ENA as already submitted. Check that the wrapper treats this as success and recovers the accession; if not, fix upstream.
  • The wrapper copies input files into the work directory. Switch to symlinks if Webin-CLI allows it, to save disk and time.
  • Optional: expose Webin-CLI's Aspera option for faster uploads (needs Aspera in the container).

3. Raw read tests

  • Add a small ONT FASTQ and a small long-read BAM to nf-core/test-datasets.
  • Add tests test_reads_long_fastq and test_reads_long_bam.

Activity

  1. changed the title [-]Pipeline should support long-read submissions end-to-end[/-] [+]Allow BAM as input for long-reads submission[/+] on Oct 16, 2026
  2. KateSakharova commented on Oct 28, 2026

    @KateSakharova
    Contributor

    resolved in #119 and #117. Retry strategy left to test in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions