Skip to content

Per-pair state machines in the multiple aligner, and make it deterministic - #10

Closed
glennhickey wants to merge 2 commits into
masterfrom
pecan-params
Closed

glennhickey wants to merge 2 commits into
masterfrom
pecan-params

Conversation

@glennhickey

Copy link
Copy Markdown

Two changes to the multiple aligner, both for cactus's bar. ComparativeGenomicsToolkit/cactus#2024 pins this branch.

Per-pair state machines (ccdbc32).

  • makeAlignmentWithPairStateMachines takes a callback that picks the state machine for each pair of sequences it aligns. That lets one multiple alignment score different pairs with different models; in cactus, by how diverged the pair's genomes are.
  • A NULL callback, or a NULL from it, means the default machine. makeAlignment, makeAlignmentUsingAllPairs and makeAllPairwiseAlignments keep their signatures and behaviour.
  • The new test checks three things: a chooser returning the default changes nothing; it's asked once per pairwise alignment, about a real pair; and another machine gives a valid but different alignment.

Deterministic multiple alignment (2ec01d1). The same sequences, models and parameters now always give the same alignment. Before, the result could depend on:

  • sonLib's random number generator, drawn from for each aligned pair's tie-breaking jitter, and for a coin flip between equally good pairs when choosing spanning-tree alignments. So the result depended on whatever had drawn from it before.
  • Where memory had been allocated. Weights with equal scores were ordered by address, and which of a weight and its reverse went into the ordered set was decided by comparing column addresses.

Now the jitter is a hash of the aligned pair, with the same magnitude and the same value for both directions. Ties go by the columns' positions, and the spanning-tree tie-break goes by a hash of the pair of sequences.

Testing:

  • Old behaviour: in 400 random cases, two identical calls of the old code gave different aligned pairs in 16 of the 201 that took the spanning-tree path, and none of the 199 all-pairs cases.
  • New tests: the tests now require identical output (aligned pairs, chosen pairwise alignments and columns, in order). A new test advances the generator and holds extra memory between two calls. The old code fails both under each of 24 seeds; this passes them all. All 42 tests pass.
  • In cactus: on evolver mammals with pecan, two runs are identical on every genome. Against faster-pecan, 17 of 4.5 million leaf-pair alignments changed, with accuracy the same to four decimals.

🤖 Generated with Claude Code

glennhickey and others added 2 commits October 1, 2026 14:58
makeAlignmentWithPairStateMachines takes a callback that picks the machine
for each pair of sequences it aligns, so one multiple alignment can score
different pairs with different models -- for cactus, by how diverged the
pair's genomes are.  A NULL callback, or a NULL from it, means the default
machine.  makeAlignment, makeAlignmentUsingAllPairs and
makeAllPairwiseAlignments keep their signatures and behaviour.

The new test checks that a chooser returning the default changes nothing,
that it is asked once per pairwise alignment about a real pair, and that
another machine gives a valid and different alignment.  It also turned up
that makeAlignment is not always repeatable: one random case in 50, on the
spanning-tree path without progressive merging, aligned the same sequences
differently on a second call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The same sequences, models and parameters now always give the same
multiple alignment.  Before, the result could depend on:

- the state of sonLib's random number generator, which the aligner drew
  from for each aligned pair's tie-breaking jitter, and for a coin flip
  between equally good pairs when choosing spanning-tree alignments --
  so on whatever had drawn from it before (in cactus, on the other ends
  of the flower aligned before this one);
- where memory had been allocated: weights with equal scores were
  ordered by address, and which of a weight and its reverse went in the
  ordered set, and so whose jitter counted, was chosen by comparing
  column addresses.

Now the jitter is a hash of the aligned pair (same magnitude, and the
same for both directions of the pair), weight ties go by the columns'
positions, and the spanning-tree tie-break goes by a hash of the pair of
sequences.  Where the jitter decided a near-tie, an alignment can differ
from the old code's, as the old code's could from one run to the next.

In 400 random cases, two identical calls of the old code gave different
aligned pairs in 16 of the 201 on the spanning-tree path (8 of 95 greedy,
8 of 106 progressive), and in none of the 199 all-pairs cases.  The
tests now require identical output -- aligned pairs, chosen pairwise
alignments and columns, in order -- and a new one moves the generator
on and holds memory between two calls.  The old code fails both under
each of 24 seeds; this passes them all.

Cactus already reseeds the generator per flower, so its runs were
reproducible; what changes there is that an end's alignment no longer
depends on the ends aligned before it, so comparing two configurations
is not confounded by the random stream shifting between them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant