Report duplicated library sequences actionably, and record unmapped pairs

[davidliwei]

issues related to #27.

What changed

1. The duplicate-sequence report says what happened. mageckcount_checklists drops a library row whose sequence is already used by an earlier row. It now names the library file, the dropped IDs, and the representative ID that carries their reads:

1 sgRNAs in gRNA_1.txt share a sequence with an earlier sgRNA and were dropped.
Reads carrying those sequences are counted under the earlier (representative) ID: hgRNA_3 -> hgRNA_2

2. It is silent when nothing was dropped. The old message fired at WARNING on every clean run announcing There are 0 sgRNAs with duplicated sequences, which trains users to ignore it.

3. The --pg-pair-only warning names the cause and the remedy. The loader now returns a {dropped_id: representative_id} map, so the pair-file check can distinguish a dropped duplicate from a genuine mismatch:

sgRNA ID hgRNA_3 was dropped from the first library because its sequence is
already used by hgRNA_2 (a duplicate sequence). This pair will be excluded from
the paired-guide counts; use hgRNA_2 instead.

Genuinely absent IDs keep the original wording.

4. Pair-level unmapped output. mageckcount_printpgdict already accepted an unmapped file handle but was called with None, so a pair excluded from pg_count.txt left no trace and a filtered pair was indistinguishable from one that was never sequenced. Those rows now go to <prefix>.pg_unmapped.txt under the existing --unmapped-to-file flag.

One deviation from the issue: this writes a separate file rather than reusing <prefix>.unmapped.txt. The read-level file holds bare sequences and the pair-level one holds id1_id2 labels with a Gene1_Gene2 column; interleaving two row shapes in one file would be worse than a second file. The pair-level file carries a header, matching pg_count.txt (the read-level file has none, unchanged).

5. A latent crash that item 4 exposed. In --pairguide secondpair mode the second-read window is recorded even when the first read matched nothing, under a None key (mageckCountIO.py:504-509). Labelling such a row raises TypeError on None + '_' + seq. It was invisible only because the file handle was None. Those pairs identify no first guide and are already reported in the read-level unmapped.txt, so they are skipped.

Tests

Six new tests, each written before the change it covers and watched fail first:

• the duplicate report names the file, the dropped ID, and its representative
• a clean library produces no warning at all
• the --pg-pair-only warning explains the duplicate drop and names the replacement
• end-to-end paired-guide counting — a two-construct dual-guide library, one construct allowed by the pair file, asserting pg_count.txt contents
• the filtered pair appears in pg_unmapped.txt
• a run with unmatched first reads completes, writes no None row, and still reports those reads at the read level

The end-to-end fixture is the first test to exercise --pairguide from FASTQ through to pg_count.txt; previously only the pair-file parser was covered. It is a partial down payment on #29, which should extend it to --reverse-complement-2, --pairguide auto, and --pg-min-read.

Full suite: 25 passed (19 before this branch).

[Copilot]

Pull request overview

Improves paired-guide diagnostics and unmapped-pair reporting.

Changes:

• Reports duplicate library sequences with dropped and representative IDs.
• Adds actionable pair-file warnings.
• Writes paired-guide unmapped output with regression tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
mageck2/mageckCount.py	Implements diagnostics and pair-level unmapped output.
tests/test_smoke.py	Adds duplicate and paired-guide tests.
CHANGELOG.md	Documents the new behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[davidliwei]

Correct, and this was a real defect. Confirmed at mageckCount.py:399-400: the unmatched-second-guide branch wrote seq1_seq2 in column 1 and id1_seq2 in column 2, under a header declaring sgRNA1_sgRNA2 / Gene1_Gene2. The pair-filter branch was serialized correctly, which is exactly why the tests missed it — they only covered that path.

Fixed in 954af82. Such a pair has no second ID and no second gene, so the row now reports the first guide by ID and gene, keeps the unmatched second sequence as its only available identifier, and marks the absent gene as None (the marker mageckcount_printdict already uses for an unknown gene):

sg1_CCCCCCCCCCCCCCCCCCCC    GENE1_None    2

Added test_pg_unmapped_rows_match_their_header, which asserts both cases in one file — the pair-filter row (sg1_sg2b / GENE1_GENE3) and the unmatched-R2 row above — so the two serializations cannot drift apart again.

One thing I considered and did not do: adding a reason column to distinguish "second guide unmatched" from "excluded by --pg-pair-only". It would be more useful than inferring the cause from whether a field looks like a sequence, but it changes the file's shape away from pg_count.txt. Deferred rather than rejected.