Repository navigation
Conversation
Get rid of lib to merge tools v2.13.1
…rge-template-2.13.1
Merge template 2.13.1
Wip clarification usage
…rge-template-v2.13.1
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Merge template v2.14.1
Synchronize example samplesheets with the usage.md
Fix save_unaligned parameter description in schema
Adopt @maxulysse suggestion on #446: shorten the changelog line.
Modernize pipeline code for Nextflow strict (v2) syntax
Merges the nf-core-bot template branch (#449, c26ac89) following the pattern used for 4.0.2 in #443: merge into a branch off dev and resolve there, rather than pushing to the bot's template branch. Only ro-crate-metadata.json conflicted. It is wholly regenerated on each build -- none of the six UUID @ids are shared between the two sides, and each is both a definition and a cross-reference -- so resolving it hunk-by-hunk risks references pointing at nodes taken from the other side. Taking the template side wholesale is also wrong: the bot generates its RO-Crate from the template README, so its description is boilerplate carrying "TODO nf-core" placeholders (7,179 chars against the real 13,620). Resolved by taking dev's file and regenerating with `nf-core pipelines rocrate`. Verified: description is the real pipeline documentation, 36 nodes, all 6 UUID references resolve, no dangling ids. .nf-core.yml and README.md auto-merged correctly, keeping the pipeline's lint configuration and real documentation while taking nf_core_version 4.0.3.
Also fix the #443 entry, which linked to nf-core/chipseq instead of nf-core/atacseq. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`collect` emits in task-completion order, and plot_macs3_qc.r / plot_homer_annotatepeaks.r derive sample ids from the order of the file list they are handed. In plot_homer_annotatepeaks.r the summary column order follows the `feature` factor levels, which accumulate by first appearance across the input files, so a reshuffle changes the md5 of macs3_annotatePeaks.*.summary.txt, its _mqc.tsv, and the MultiQC peak annotation plot table. Observed as an intermittent snapshot mismatch in tests/skip_trimming.nf.test on #453. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`toSortedList` emits an empty list when its source channel is empty, whereas `collect` emits nothing at all, so the previous version started PLOT_MACS3_QC with no files when no merged-replicate peaks exist and plot_macs3_qc.r died on `flag "i" requires an argument`. Seen in the `bowtie2 with stub` test. Sort the staged file list in the process script instead, which leaves the channel semantics untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ot-order Sort peak QC plot inputs deterministically
Bump every nf-core module and subworkflow via `nf-core modules update --all`
/ `nf-core subworkflows update --all` and reconcile the resulting
call-signature changes in first-party code. No intended behaviour change.
- Reconcile call sites in workflows/atacseq.nf and the affected
subworkflows/{local,nf-core}/*.
- Fix the ch_fasta_fai reference channel: add .first() so the genome
reference broadcasts to every sample instead of being consumed once.
- Consolidate singleton reference-channel plumbing.
- Remove the orphaned custom/getchromsizes module, replaced by
SAMTOOLS_FAIDX(..., get_sizes=true).
- Fix a terminal MultiQC hang: MultiQC must not publish to the `versions`
topic because its own input depends on that topic being resolved, which
creates a cycle that leaves the run hanging forever. Emit the version as
a named output instead, matching the nf-core/multiqc convention.
- Replace fragile '2>&1 | tr | sed;sed' version extraction in local modules
with concise single-command forms (still emitting topic: versions).
- Ignore a merge_markers false positive in the bowtie2 test snapshot: SAM
base-quality strings contain runs of '<' that trip the check.
Chromap BAMs carried no @rg header, so Picard 3.4.0 MarkDuplicates aborted with a NullPointerException (SAMRecord.getReadGroup() is null) once the module update bumped Picard from 3.1.1. Update the fastq_align_chromap subworkflow to d92e0f8, which adds PICARD_ADDORREPLACEREADGROUPS and an update_readgroups input, install the picard/addorreplacereadgroups module, enable read-group updating at the call site in workflows/atacseq.nf, and configure the process in conf/modules.config to mirror the existing BWA/Bowtie2 read-group mapping (RGID=meta.id technical id, RGSM=biological sample, RGPL=ILLUMINA, RGLB=meta.id, RGPU=1, RGCN=params.seq_center). The linked bam_sort_stats_samtools dependency tree also advanced samtools 1.23.1 -> 1.24 as part of the subworkflow update.
The module update bumped subread to 2.1.1, which strictly validates library pairing per BAM: it rejects single-end reads when run in paired mode (-p) and rejects paired-end reads when run in single-end mode. Consensus-peak quantification passes a mixed single-end/paired-end cohort to one SUBREAD_FEATURECOUNTS invocation, so 2.1.1 aborts with "No paired-end reads were detected in paired-end read library". Pin subread back to 2.0.1, which auto-detects pairing per BAM in a single invocation and reproduces the read/fragment counts the existing snapshots encode. This keeps the fix free of custom SE/PE-splitting and matrix-merging code. Remove the pin once mixed-library counting is handled explicitly (follow-up). The pin makes the local module diverge from its registry git_sha, so it is recorded as a tracked nf-core module patch (container + environment.yml) for `nf-core modules lint` check_local_copy.
… order The consensus BAM list is assembled from an unordered channel collect, so the featureCounts invocation received its BAMs (and emitted its count columns) in a run- and host-dependent order. The counts were identical, but the output file's byte content - and therefore its snapshot md5 - varied between runs, making the consensus featureCounts snapshot non-reproducible. Sort the BAM list by filename before featureCounts so the column order (and the output file) is deterministic and identical on any x86_64 host, matching CI.
…ates metrics Regenerate all 7 pipeline nf-test snapshots against the updated modules (subread 2.0.1 pin, samtools bumps, chromap read-group path, deterministic consensus featureCounts column order). The previous snapshots were ~2 months stale, from before the module update. Generated on GitHub-hosted ubuntu-latest x86_64 runners via the update-snapshots workflow, so container digests and architecture match CI. Also extend .nftignore: Picard MarkDuplicates metrics headers carry a per-run "Started on: <timestamp>" line, so their md5 is never reproducible. The existing pattern only covered the default filename (*.MarkDuplicates.metrics.txt), but this pipeline renames them via ext.prefix to *.mkD.sorted.metrics.txt (library) and *.mRp.clN.sorted.metrics.txt (replicate). These were the only non-deterministic files left once the consensus featureCounts inputs were sorted.
Add entries for the Chromap read-group fix, the subread 2.0.1 pin and the deterministic consensus featureCounts column order. The template 4.0.3 bump is left to #449.
seq_center is free text in the schema, so a value containing whitespace was
split by the shell. On the Chromap path this reached Picard
AddOrReplaceReadGroups as `--RGCN Broad Institute`, and Picard took `Broad` as
the value and rejected the remainder:
Illegal argument value: Positional arguments were provided ',Institute}'
but no positional argument is defined for this tool.
Reproduced against the picard 3.4.0 container this PR pins. Quoting the value
makes Picard receive it as a single argument.
Bowtie2 built `--rg CN:${params.seq_center}` the same way and had the same
defect; quoted too. BWA is unaffected because input_check.nf embeds the quotes
in meta.read_group at construction.
No test set seq_center, which is why the matrix stayed green over this. The
Chromap pipeline test now sets a value containing whitespace, so the path is
covered.
The updated subread module publishes *.featureCounts.tsv (and .tsv.summary); the docs still described *.featureCounts.txt. Drop the matching .nftignore patterns rather than renaming them: they are dead, and leaving the .tsv files hashed by the pipeline snapshots is what gives the deterministic column order useful coverage.
…HANGELOG Addresses review from @Kevin-Brockers. Old versions are those shipped in 2.1.2, matching the table's 'compared to the last release' framing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop emit names on single-emit workflows (INPUT_CHECK, ATACSEQ, NFCORE_ATACSEQ) now that the versions emit is gone, and name the toSorted closure parameter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nf-core modules update, with a pin of subread to 2.0.1 due to SE/PE mixing being discontinued in later versions.
Add `manifest.diagram` to `nextflow.config`
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>
Run the test_controls profile with --with_control in CI
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is to resolve the merge conflicts on your branch.