IR passes don't preserve successor order, and collapse_prefix has soundness holes #421

Problem

For a backtracking VM, transition order is the semantics — successors are tried first-match-wins. The IR pipeline (crates/plotnik-compiler/src/compile/compiler.rs::compile: eliminate_epsilons → remove_unreachable → collapse_prefix → collapse_up → lower) verifies only epsilon elimination, and with an order-insensitive BTreeSet fingerprint. The other passes run unchecked. Confirmed consequences:

1. check passes, dump/exec/infer panic — collapse_prefix (crates/plotnik-compiler/src/compile/collapse_prefix.rs) deletes an instruction that is still referenced → "label not in layout" (verified 2026-06-12):

   cargo run -p plotnik -- check -q 'Q = (program (comment)? (comment)? (comment)?)' -l javascript  # exit 0
   cargo run -p plotnik -- dump  -q 'Q = (program (comment)? (comment)? (comment)?)' -l javascript  # panic: label not in layout

2. collapse_prefix violates branch priority: merging non-adjacent identical successors hoists a later alternation branch above an earlier one — the wrong branch wins (confirmed with a repro).
3. lower.rs silently drops successors: on >28-way branch overflow the cascade entry replaces the kept successors instead of appending — 27 successors lost (crates/plotnik-compiler/src/compile/lower.rs).
4. Sequences with a leading skippable item in tagged alternations duplicate Enum/EndEnum per item and emit Null Set outside the variant scope (sequences.rs) → dropped captures / panics.

Related: the debug-build IR fingerprint (crates/plotnik-compiler/src/compile/verify.rs) enumerates paths exponentially — ~9 optionals already takes minutes (a 300-capture infer ran >3 min before being killed, 2026-06-12).

Approach

• Make the fingerprint order-sensitive (ordered path list, not BTreeSet) and run it around every pass, not just epsilon elimination.
• Add cheap structural invariants after each pass: no dangling labels, no duplicate labels, scope-effect balance per path.
• Constrain collapse_prefix to adjacent, dedup-first, single-predecessor merges — or delete it; it is a size optimization with two confirmed soundness holes (items 1, 2).
• Fix the lower cascade to append rather than replace (item 3).
• Bound the fingerprint cost (memoized hashing or path sampling) so debug builds stay usable.