Skip to content

Replace legacy groupTuple and transpose operators with typed groupBy and flatMap - #156

Merged
maxulysse merged 3 commits into
devfrom
feat/typed-workflows
Sep 24, 2026
Merged

Replace legacy groupTuple and transpose operators with typed groupBy and flatMap#156
maxulysse merged 3 commits into
devfrom
feat/typed-workflows

Conversation

@maxulysse

@maxulysse maxulysse commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

PR checklist

  • This comment contains a description of changes (with reason).
  • CHANGELOG.md is updated.

Description

Prepare the pipeline for static typing (Phase 3 Part A of the nf-core/rnaseq typed-workflows plan) by replacing legacy operators that are not type-checked with their typed equivalents:

  • main.nf ncbi_download_input: replace groupTuple() with groupBy(), collecting source_version-keyed [reference, meta] entries instead of positional groupTuple columns.
  • subworkflows/local/prepare_genome_rnaseq: replace groupTuple() with groupBy() when merging the input GTF with GFFREAD output.
  • subworkflows/local/datasheet_to_channel: replace transpose() with flatMap to expand files(...) VCF holders into one [meta, vcf] per path.

Operator behaviour is equivalent (verified both semantically and with nf-test); no output changes are expected.

Verification

  • nf-core pipelines lint: 366 passed, 0 failed
  • nf-core pipelines lint (nextflow lint .): 54 files, 0 errors
  • nf-test test tests/default.nf.test --profile=+docker: PASSED (71.5s)

Generated by opencode
Verified by @maxulysse

@maxulysse
refactor: replace groupTuple and transpose with typed operators
dbc9723 
Replace the legacy groupTuple and transpose operators with groupBy and
flatMap so the workflow code is ready for static typing, where the
legacy operators are not type-checked.

- ncbi_download_input: collect source_version-keyed [reference, meta]
  entries with groupBy instead of groupTuple
- prepare_genome_rnaseq: merge input gtf and GFFREAD output with groupBy
  instead of groupTuple
- datasheet_to_channel: flatten VCF file holders with flatMap instead of
  transpose

Generated by opencode
@maxulysse
chore: fix CHANGELOG PR number
61f3be4
@github-actions

github-actions Bot commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 44beddc

+| ✅ 216 tests passed       |+
#| ❔  12 tests were ignored |#
!| ❗   8 tests had warnings |!
Details

❗ Test warnings:

  • files_exist - File not found: .github/workflows/awsfulltest.yml
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 4.1.0
  • Run at 2026-09-24 10:09:03

@maxulysse
fix: handle single-file VCF entries in vcf flatMap
44beddc 
The vcf channel mixes files() holders (dbsnp) with single file() entries
(germline_resource, known_indels, known_snps, pon). Path.toList() splits
an http/https path into its segments, so calling toList() on a single
Path broke the flatMap replacement of transpose and fed non-existent
path segments to HTSLIB_BGZIPTABIX, causing "Cannot invoke method
provider() on null object".

Pass single Path entries through unchanged, matching the transpose
behaviour; only expand files() holders per file.

Generated by opencode
@nf-core nf-core deleted a comment from github-actions Bot Sep 24, 2026
@maxulysse
maxulysse merged commit 662b5e2 into dev Sep 24, 2026
20 checks passed
@maxulysse
maxulysse deleted the feat/typed-workflows branch September 24, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@maxulysse