Skip to content

multi: Register authenticated provisional lineage end to end - #990

Closed
ellemouton wants to merge 17 commits into
reorg-batchcanon-corefrom
reorg-lineage-producers
Closed

ellemouton wants to merge 17 commits into
reorg-batchcanon-corefrom
reorg-lineage-producers

Conversation

@ellemouton

@ellemouton ellemouton commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does

Connects the real VTXO creation and receipt paths to the batchcanon authority
introduced by #980. Without this PR the authority exists but production paths do
not consistently feed it.

It handles three production sources:

  1. Locally received round VTXOs.
  2. OOR-received VTXOs.
  3. Explicit custom VTXOs supplied as OOR inputs.

The indexer ancestry response supplies the serialized commitment transaction,
ordered commitment prevouts, CSV-delay evidence, and the parent lineage required
to validate a received VTXO. The recipient treats that response as untrusted
and authenticates: the commitment txid, the referenced output, every commitment
input, the input prevouts and their ordering, and the relationship between the
received VTXO and the supplied batch.

Registration happens before the VTXO is saved or exposed:

sequenceDiagram
    participant R as OOR receiver
    participant I as Indexer
    participant B as batchcanon
    participant V as VTXO store

    R->>I: Request ancestry
    I-->>R: Commitment, prevouts, and lineage
    R->>R: Authenticate evidence
    R->>B: Register complete lineage
    B-->>R: Registration durable
    R->>V: Materialize and save VTXO
Loading

Once registered, the client responds to later reorgs using its local chain
watches; it does not requery the indexer.

Wallet reporting uses the same authority: a lifecycle-live VTXO whose lineage is
reorged, conflicted, unseen, or still reconciling is excluded from spendable
balance and included in temporarily_unavailable_sat. Terminally invalidated
value is excluded from both buckets.

Review focus

  • Treat the indexer response as an untrusted input — are all bindings checked?
  • Are every TxIn and ordered prevout included and authenticated?
  • Is registration durably complete before exposure?
  • Are retries idempotent across every crash boundary?
  • Does legacy or incomplete evidence fail closed?

Full design & diagrams: REORG_SAFETY_PR_REVIEW_GUIDE.md §4.3. Overall
architecture and PR stack: lumos#454.

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@ellemouton

Copy link
Copy Markdown
Contributor Author

@claude review this

@claude

claude Bot commented Jul 17, 2026 •

Copy link
Copy Markdown

Claude finished @ellemouton's task in 4m 34s —— View job


Review: PR #990 — Register authenticated provisional lineage end to end

I reviewed the full diff against origin/reorg-batchcanon-core. This is a well-structured, defensively-written PR. The trust boundary in batchcanon.EvidenceFromAncestryPaths is the centerpiece and it is thorough — it binds every server-supplied field (tx bytes → txid, tree output value/script → tx.TxOut[idx], each input outpoint/prevout → tx.TxIn[i]) rather than trusting caller assertions, and the fail-closed rollout story (legacy indexers → empty evidence → gate keeps lineage unusable) is coherent. Test coverage is good across all five producers.

Below are observations, ordered roughly by significance. None are blockers.


1. commitment_csv_expiry_delta is trusted, not authenticated

In batchcanon/rpc_evidence.go, every other evidence field is cross-checked against the serialized commitment tx, but CSVExpiryDelta (path.GetCommitmentCsvExpiryDelta()) is taken directly from the operator with only a > 0 check. A malicious operator can inflate/deflate it, which flows into Record.EffectiveExpiry() (ConfirmationHeight + CSVExpiryDelta). This is likely acceptable (the operator already controls tree construction, and CSV isn't derivable from the tx alone), but it's the one piece of "evidence" that isn't actually evidence. Worth a one-line comment at the trust boundary making that explicit so a future reader doesn't assume it's authenticated like the rest.

2. Asymmetric CSV validation between producer and receive paths

The receive/incoming paths enforce CSVExpiryDelta > 0 in three places (evidenceFromAncestryPath, encodeBatchEvidence/decodeBatchEvidence, validateIncomingEvidenceCoverage, registerIncomingBatchEvidence), but the round producer path in roundBatchRegistration (round/transitions.go) sets CSVExpiryDelta: int32(state.SweepDelay) with no positivity guard, and validateRegistration in batchcanon/manager.go does not check CSV either. SweepDelay is realistically always positive, so this is defensive-only, but given how consistently the receive side guards it, a matching guard (or a check inside validateRegistration) would close the gap and keep the invariant in one place. Fix this →

3. batchOutputIndex diverges from confirmationWatchScript on the out-of-range branch

batchOutputIndex (round/transitions.go) is documented as mirroring confirmationWatchScript so the persisted evidence watches the same output the chain backend does. They agree on the happy path, but when len(vtxoTrees) > 0 and the computed min index is out of range, confirmationWatchScript falls back to TxOut[0] while batchOutputIndex returns an error. So in that (shouldn't-happen) case the confirmation watch keys on output 0 while registration fails outright. The fail-closed direction is arguably safer, but the "mirrors confirmationWatchScript" comment slightly overstates the equivalence — either reconcile the fallback or note the intentional divergence.

4. Duplicated coverage-validation logic

oor.validateIncomingEvidenceCoverage (oor/incoming_batch_registration.go) and the inline coverage loop in vtxo.registerIncomingBatchEvidence (vtxo/incoming_batch_registration.go) are near-identical: per-item Validate(), CSV>0, "names commitment outside ancestry", duplicate detection, and the final len(seen) != len(ancestry) count. They live in different packages so a shared helper would need a home (perhaps batchcanon, alongside EvidenceFromAncestryPaths, taking the ancestry txid set). Not urgent, but this is exactly the kind of security-relevant check you don't want to drift between two copies.

5. Minor: emitReceived on registration failure

In vtxo/incoming_handler.go, a registration failure emits h.emitReceived(ctx, "failed") and returns an error. If the caller retries the receive, you can get failed then later received for the same transfer. Since RegisterBatch is idempotent this is only a metrics-cardinality nuance, not a correctness issue — flagging only for awareness.


Things I checked that are correct

  • OOR dependent-outpoint derivation is right. RegisterIncomingBatchEvidence builds the dependent as {Hash: chainhash.Hash(sessionID), Index: match.OutputIndex}. I confirmed SessionID is the Ark txid (see matchesIncomingVTXO comparing sessionID[:] to the candidate txid, and the events.go doc comments), and the materialized VTXO outpoint is {arkTxid, OutputIndex} — so these match. The vtxo path uses desc.Outpoint directly, which is the same value.
  • No double registration in OOR. Durable path registers in session_actor.driveOutboxEvents (outside the commit tx); the sync path registers in LocalPersistenceOutboxHandler gated on !hasActorDBTx(ctx). Mutually exclusive.
  • Registration-before-exposure ordering holds in round (before VTXOStore persist) and in vtxo (before save/notify), consistent with the fail-closed gate.
  • context.WithoutCancel in the round path correctly detaches the durable registration from request cancellation.
  • Proto changes (fields 6/7/8 on AncestryPath, new CommitmentInputEvidence) are purely additive with no field-number collisions.
  • Duplicate/conflict handling for same-commitment multi-leaf ancestry (equalBatchEvidence) is correct and tested.

Nice work — the authentication layer and the two-graph separation (actual inputs vs logical consumer edges) are clearly reasoned throughout.
· branch reorg-lineage-producers

@ellemouton

Copy link
Copy Markdown
Contributor Author

Review disposition for the current head: the CSV delay is intentionally operator-authored policy input because it is not derivable from the commitment transaction; all transaction-bound fields are authenticated, and the client still requires a positive delay. The output-index error path intentionally fails closed instead of copying the legacy confirmation-watch fallback. The duplicated receive-path coverage checks and failed-then-retried metric sequence are non-blocking cleanup notes; no correctness finding was left open in this PR.

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from b1f9ab8 to 9655bde Compare July 17, 2026 23:27
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch 2 times, most recently from 8dbd209 to c566338 Compare July 17, 2026 23:38
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 9655bde to 659e6aa Compare July 17, 2026 23:38
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from c566338 to 899c934 Compare July 18, 2026 00:00
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 659e6aa to b2a5d3d Compare July 18, 2026 00:00
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 899c934 to 9d3b6b0 Compare July 18, 2026 00:20
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from b2a5d3d to c121b30 Compare July 18, 2026 00:20
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 9d3b6b0 to 095039e Compare July 18, 2026 00:38
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch 2 times, most recently from 49e2003 to 9d19e5d Compare July 20, 2026 16:10
@ellemouton

Copy link
Copy Markdown
Contributor Author

@claude review this — the current head adds canonicality-aware spendable and temporarily-unavailable balance reporting plus a real reorg/reconfirmation systest.

@claude

claude Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 56s —— View job


Review: canonicality-aware balance reporting + reorg systest

Reviewed the head commits 3859c34 (proto fields) and 9d19e5d (classification + wiring + systest) against origin/reorg-batchcanon-core. This is a clean, well-tested increment. The core idea — a third balance bucket for reversible lineage limbo that is carved out of both vtxo_balance_sat and total_confirmed_sat — is implemented consistently end to end. No blockers.

What's correct

  • Spendable classification mirrors the coin-selection gate. ClassifyCanonicalityBalance (vtxo/filter.go:130) counts a VTXO as spendable iff availability.Usable(), which is exactly the predicate gateUnavailableLineage uses to admit candidates. So the reported spendable figure can't drift from what coin selection will actually let you spend — the invariant you'd most want held here.
  • No double counting. The three live buckets are disjoint by construction: ClassifyCanonicalityBalance and SumPendingBalance both key off Status, and only SumPendingBalance touches PendingForfeit/Forfeiting/Spending while classification only touches VTXOStatusLive. Exiting VTXOs come from a separate ListVTXOsByStatusLight query. Nothing overlaps.
  • total_confirmed_sat correctly excludes limbo. rpc_server.go:868 sums BoardingConfirmedSat + VtxoBalanceSat (now the spendable-only figure), matching both proto docs.
  • Terminal Invalidated contributes to neither bucket — right call; it's no longer wallet liquidity, and the case availability != batchcanon.Invalidated guard keeps it out of the temporary bucket.
  • Worst-parent semantics flow through LineageCommitmentTxIDs → LineageAvailability → CombineAvailability, and the multi-parent test (TestCanonicalityAwareVTXOBalanceUsesWorstParent) pins it: one reorged parent drags the whole VTXO into limbo.
  • Fail-closed error propagation — a broken reader surfaces an RPC Internal error rather than a misleading zero/spendable figure (TestCanonicalityAwareVTXOBalancePropagatesReaderError), consistent with the strictness rationale already documented above the boarding/VTXO fetches.
  • Nil-reader legacy path is behavior-neutral (SumSpendableBalance), and the systest asserts the real durable store across a live reorg → reconfirm cycle at all three states (Provisional → ReorgedOut → reconfirmed Provisional), using the same ListLiveVTXOs + ClassifyCanonicalityBalance path the RPC uses.
  • Proto changes are purely additive (field 6 on BalanceResponse, field 11 on GetBalanceResponse), no field-number collisions; generated .pb.go was regenerated, not hand-edited.

Observations (all non-blocking)

1. Proto docs omit the unseen case that the bucket actually includes. The temporary bucket is everything that is !Usable() && != Invalidated, which per the test (unseenBatch → 50k in unavailable) and the commit message ("reorged, conflicted, unseen, and reconciling") deliberately includes AvailabilityUnknown (unseen lineage) and LineageReconciling. But the field docs enumerate a narrower set:

  • wallet.proto:955 — "while its batch lineage is in reorg/conflict limbo or is being reconciled"
  • daemon.proto:564 — "This includes reorg/conflict limbo and lineage reconciliation"

Neither mentions unseen/unregistered lineage. Since a client rendering this figure will see freshly-created-but-unconfirmed live VTXOs land here, it's worth a word in the doc so the wire contract matches the behavior the test locks in. Fix this →

2. UX consideration for the unseen window. Following from (1): a just-received round/OOR VTXO whose commitment isn't confirmed yet is Live but AvailabilityUnknown, so it will briefly appear as temporarily_unavailable rather than as inbound-pending. That's the fail-closed-correct answer (it truly isn't spendable yet), but "temporarily unavailable" reads more alarming than "pending" for a normal first-confirmation wait. Worth a conscious confirmation that this is the intended surface, since it's the one state here that isn't actually a fault condition.

3. Minor nil-guard asymmetry. The non-nil path guards desc == nil (filter.go:117), but the nil-reader path delegates to SumSpendableBalance → FilterDescriptors, which dereferences d.Status without a nil check and would panic on a nil element. ListLiveVTXOs never yields nil entries so this isn't reachable today; flagging only because the two paths now differ on an input the new one explicitly defends against.

4. Per-RPC store fan-out. GetBalance now issues one GetBatch per (live VTXO × distinct parent) via LineageAvailability. This matches the gate's existing per-candidate pattern and is fine at current scales, but it does turn a previously in-memory sum into O(N·P) store reads on a hot read RPC. Not worth changing now — noting for when wallet VTXO counts grow.

Nice, tight change — the classification predicate reusing Usable() is exactly the right way to keep the balance figure and the admission gate from ever disagreeing.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 9d19e5d to 97bd459 Compare July 20, 2026 16:24
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 095039e to 7f636e7 Compare July 20, 2026 16:24
@ellemouton

Copy link
Copy Markdown
Contributor Author

@claude review this — final head is now rebased onto current main and includes canonicality-aware temporarily-unavailable balance reporting plus the real reorg systest.

@claude

claude Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 8s —— View job


Review: PR #990 — final head (rebased onto main)

  • Read diff vs origin/reorg-batchcanon-core
  • Review evidence authentication (batchcanon)
  • Review producer wiring (round, oor, vtxo)
  • Review balance classification + RPC wiring (vtxo/filter.go, waved)
  • Review reorg systest
  • Post review

Re-reviewed the full stack (7 commits) against origin/reorg-batchcanon-core after the rebase onto current main. The rebase is clean — no regressions in the producer wiring or the two-graph separation that the two prior passes signed off on, and the balance-reporting increment is now folded in coherently. No blockers. Everything flagged below is non-blocking, and most of it is a status check on prior observations rather than anything new.

Rebase / correctness confirmations

  • Balance classification still mirrors the coin-selection gate. ClassifyCanonicalityBalance (vtxo/filter.go:130) counts value as spendable iff availability.Usable() — the same predicate coin selection admits on — and buckets !Usable() && != Invalidated as temporarily unavailable. The reported figure can't drift from what a spend will actually let you use. Terminal Invalidated contributes to neither bucket. ✅
  • total_confirmed_sat excludes limbo. GetBalance now sums the spendable-only VtxoBalanceSat (rpc_server.go), so both the new vtxo_temporarily_unavailable_sat (daemon field 11) and temporarily_unavailable_sat (wallet field 6) are carved out of the confirmed total. Proto changes are purely additive. ✅
  • Explicit-OOR-input gate is symmetric with wallet coin selection. requireCustomInputLineage (rpc_server.go) runs batchcanon.LineageBlocked over LineageCommitmentTxIDs(desc) before checkpoint signing, mapping Invalidated → FailedPrecondition and reorg/reconcile limbo → Unavailable, and no-ops when batchCanonStore is nil (legacy). Closes the bypass where a reorged-out VTXO could reach signing through the custom-input path. ✅
  • Registration-before-exposure ordering still holds: round registers inside InputSigSentState.ProcessEvent before the VTXOStore persist, under context.WithoutCancel, and fails the round at its pre-exposure checkpoint on registration error (round/transitions.go). oor/vtxo register before materialize/save. Fail-closed direction is preserved throughout. ✅
  • New systest exercises the real durable store. assertCanonicalityBalance (systest/batch_canonicality_gate_test.go) drives ListLiveVTXOs + ClassifyCanonicalityBalance against the real batchcanon.Reader across Provisional → ReorgedOut → reconfirmed-Provisional, asserting the exact spendable/unavailable split at each state — the same path GetBalance uses. ✅

Non-blocking observations (status of prior notes)

1. Proto docs still omit the "unseen" bucket. ClassifyCanonicalityBalance puts AvailabilityUnknown (a freshly-created, not-yet-confirmed live VTXO) into the temporary bucket, but wallet.proto:952 ("reorg/conflict limbo or is being reconciled") and daemon.proto:562 ("reorg/conflict limbo and lineage reconciliation") don't mention unseen/unregistered lineage. A client will briefly render a normal first-confirmation wait as temporarily_unavailable; worth one word so the wire contract matches the behavior the systest locks in. (Carried over from the balance-PR review; still open.) Fix this →

2. CSVExpiryDelta positivity is still only guarded on the receive path. The receive paths reject CSVExpiryDelta <= 0 (batchcanon/rpc_evidence.go:176, oor/incoming_batch_registration.go:116), but the round producer sets CSVExpiryDelta: int32(state.SweepDelay) (round/transitions.go) with no guard, and validateRegistration (batchcanon/manager.go:305) checks every field except CSV. So a zero/negative SweepDelay would register and make EffectiveExpiry() collapse to ConfirmationHeight. SweepDelay is realistically always positive — this is defensive-only, and you've already dispositioned it as intentional. Flagging only that the rebase didn't change it; a one-line CSVExpiryDelta > 0 check in validateRegistration would keep the invariant in one place for both producers.

3. Minor nil-guard asymmetry. The non-nil path guards desc == nil (vtxo/filter.go:117), but the nil-reader path delegates to SumSpendableBalance → FilterDescriptors, which dereferences d.Status without a nil check. ListLiveVTXOs never yields nil entries, so unreachable today — noting only because the two paths now differ on an input the new one explicitly defends against.

Note on validation

I could not run go build / go test in this environment (build command not permitted here), so the confirmations above are from code reading against the diff. The PR body reports make fmt-changed, make lint-changed-local, make tidy-module-check, go test ./vtxo ./waved ./oor ./round, and F2/F3/F6/F7 real-chain systests all passing.

The authentication layer and the reuse of Usable() as the single admission-and-balance predicate remain the strongest parts of this change — balance and coin selection are structurally incapable of disagreeing.
· branch reorg-lineage-producers

@ellemouton

Copy link
Copy Markdown
Contributor Author

@claude review this — addressed the two actionable non-blocking notes: temporary-balance docs now include unseen/unregistered lineage, and all batch registrations now enforce positive CSV expiry with regression coverage.

@ellemouton

Copy link
Copy Markdown
Contributor Author

Review disposition: temporary-balance wire docs and positive CSV registration validation are fixed in the current head. The nil-reader descriptor note is unreachable because ListLiveVTXOs does not return nil descriptors; no correctness finding is left open.

@claude

claude Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 55s —— View job


Review: follow-up fixes (CSV positivity + temporary-balance docs)

  • Review CSV-expiry positivity commit (c4cd79b)
  • Review temporary-balance docs commit (37ab56c)
  • Confirm regression coverage
  • Post review

Reviewed the two new commits on top of the previously-signed-off stack. Both notes are addressed correctly. One small formatting issue that will fail CI.

✅ CSV-expiry positivity (c4cd79b) — correctly centralized

Putting the req.CSVExpiryDelta <= 0 check in validateRegistration (batchcanon/manager.go:317) is the right fix, and better than the per-producer guard I suggested. validateRegistration is the single chokepoint every registration flows through — both the actor path (handleRegisterBatch, manager.go:225) and the store path (record.go:65) — so this now covers the round producer (SweepDelay), the receive/OOR import paths, and any future producer with one invariant, exactly the "keep it in one place" outcome. EffectiveExpiry() can no longer collapse to ConfirmationHeight from a zero/negative delta. Regression coverage is good: the new "non-positive CSV expiry delta" table case in registration_validation_test.go:39 asserts the rejection, and the two harness defaults (manager_test.go, validRegistrationRequest) keep existing fixtures valid.

✅ Temporary-balance docs (37ab56c) — matches behavior

Both field docs now read "unseen, unregistered, in reorg/conflict limbo, or being reconciled" (wallet.proto:954, daemon.proto:562), which correctly describes what ClassifyCanonicalityBalance actually buckets (AvailabilityUnknown / LineageReconciling → temporary). The wire contract now matches the behavior the systest locks in, closing the doc gap from the prior two reviews. .pb.go was regenerated (not hand-edited), and proto and generated comments agree.


🔴 One actionable: registration_validation_test.go:138 is not gofmt-clean

The new field breaks the struct's alignment block:

BatchOutputIndex:     1,
ConfirmationPkScript: watchScript,
CSVExpiryDelta:        144,   // <- one extra space; value is shifted a column right

ConfirmationPkScript is the longest key in the group, so gofmt wants 7 spaces after CSVExpiryDelta: (aligning 144 under watchScript), but there are 8. This will fail make fmt-changed-check / the lint gate in CI. A make fmt-changed pass fixes it. Fix this →

(I couldn't run make fmt-changed-check directly in this environment — permission-gated — but the misalignment is visible in the source: 144 sits one column right of watchScript above it.)


Status of remaining prior notes (all non-blocking, unchanged)

  • Nil-guard asymmetry (vtxo/filter.go:117 guards desc == nil; the nil-reader path via SumSpendableBalance → FilterDescriptors does not) — still present, still unreachable since ListLiveVTXOs yields no nil entries. Informational only.
  • Per-RPC store fan-out in GetBalance (O(N·P) GetBatch reads) — unchanged, fine at current scales.

Everything else from the three prior passes still holds. Nice, focused follow-up — the CSV fix in particular lands the invariant in the cleanest possible spot.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 37ab56c to 6c63040 Compare July 20, 2026 16:49
Carry the commitment transaction and every input prevout in ancestry
responses. Validate the transaction, output, and input bindings before
receive paths register canonicality watches.
Persist authenticated evidence in the durable receive event and await
canonicality registration before staging incoming VTXOs. Legacy indexers
may omit the full additive extension, while partial or conflicting
evidence fails closed.
Carry authenticated evidence through the round-receive ancestry lookup.
Await canonicality registration before saving or notifying the VTXO
manager. Missing legacy evidence remains blocked by the gate.
Custom OOR inputs bypass wallet coin selection and its fail-closed
lineage gate. Reuse the full-lineage reader before maturity checks so
reorged or invalidated inputs never reach checkpoint signing.
Expose reversible lineage limbo separately from spendable and pending
VTXO balances so clients can explain why value is temporarily unusable.
Keep reorged, conflicted, unseen, and reconciling lineage out of
the spendable figure while preserving it in a temporary-unavailability
bucket. Exercise the classification across a real reorg and
reconfirmation.
Reject malformed local or imported registration evidence before it can
produce a height-derived expiry at the confirmation block itself.
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 6b3483a to 72dfa1d Compare August 17, 2026 15:46
Include unseen and unregistered lineage in the balance field contract so
clients can explain the same fail-closed bucket the daemon reports.
Pass the local round start height and indexed commitment height into
batch canonicality registration. Persist the indexer hint through OOR
durable messages so delayed and replayed receive paths scan across
already-mined confirmations.
The wired VTXO admission gate reads lineage availability straight from
the durable store via LineageBlocked. When a reorg/conflict observation
could not be persisted (a transient ApplyObservation failure), the
in-memory watch was marked not-ready but the stale durable row stayed
Ready(), so the gate kept admitting a VTXO whose commitment had just
left the best chain. The overlay the code comments promised only ever
protected the unwired actor QueryLineage path.

Make the manager itself a batchcanon.Reader whose GetBatch overlays the
in-memory watch: a not-ready watch forces the returned record not-ready,
even when the durable row still reads Ready. Thread the manager (not the
raw store) into the gate. The overlay only downgrades ready->not-ready,
so it is strictly fail-closed. Add a wired-path test proving admission
closes after a failed reorg persist.
A consumer batch can reach a terminal state (ConflictFinalized) before
its ForfeitedBy(consumer) business-revision marker is durably persisted:
the round FSM registers the business revision before the outbox delivers
ForfeitConfirmedToVTXO, which is what drives MarkForfeited. When the
terminal transition wins that race, restoreProvisionalConsumers runs the
restore compare-and-swap against a VTXO not yet at the expected 
revision,
so it defers. No further batchcanon event is then guaranteed on the
consumer, leaving the consumed VTXO stranded until the next restart
redrive.

Have the VTXO actor send ConsumerForfeitPersistedMsg the moment
MarkForfeited succeeds, carrying the consuming batch. The manager
redrives that consumer's terminal lifecycle, gated on it being Ready and
terminal, so the marker's arrival becomes the evidence change that
completes the deferred restore without a restart. The redrive is wired
as an optional VTXO-manager callback, so harness paths and deployments
without batch canonicality are unaffected.
The receive paths authenticated the commitment transaction (self-hash)
and its batch output, but never proved the received VTXO actually
descends from it. Nothing tied the pre-signed tree to that authenticated
output, so a malicious or buggy indexer could name a decoy commitment
(real, deeply buried, never reorging) and have the client arm its reorg
watch there while the VTXO's true lineage silently left the chain.

Running the tree signature check alone does not close this: signatures
verify against a per-node aggregate key recomputed from the tree's own
(indexer-supplied) cosigners, so a fabricated tree signed with the
attacker's own keys passes. The load-bearing check is that each node's
aggregate key must equal the taproot key of the output it spends. At the
root that output is the authenticated commitment output, so the root key
is pinned to the real round key (unforgeable); taproot's SIGHASH_DEFAULT
commits to each tx's outputs, so the pin cascades to every leaf
(verifyAuthenticatedTree). It is key-identity-agnostic, so it never
demands the operator identity key and never false-rejects a genuine 
tree.

Two receive paths consume it:
  - In-round (waved/incoming_ancestry_fetcher): 
VerifyReceivedVTXOBinding
    runs verifyAuthenticatedTree then requires a leaf to pay the 
received
    VTXO script (an in-round VTXO IS a commitment-tree leaf).
  - OOR (oor/incoming_batch_registration): an OOR output is an ark-tx
    output, not a tree leaf, so VerifyOORAncestryLineage binds the 
coin's
    real base inputs instead. It re-anchors each tree's batch output on
    the validated evidence (OOR metadata can be a durable replay), runs
    verifyAuthenticatedTree, and requires every base coin input to be an
    authenticated ancestry leaf. BaseCoinInputs derives those by walking
    the coin's authenticated checkpoint/ark chain (root
    FinalCheckpointPSBTs + AncestorPackages) down to the checkpoints 
whose
    prevouts are the committed leaves -- uniform for single- and
    multi-hop receives. The coin inputs are trustworthy: each checkpoint
    txid is committed by the ark txid, which equals the coin's own
    outpoint/sessionID (ValidateFinalizePackage), so an attacker cannot
    substitute them. This closes the genuine-but-unrelated (decoy)
    commitment variant. The bind never keys off the indexer-supplied
    chain depth (inflatable to skip). Injected via 
IncomingLineageVerifier
    so registration-logic tests keep mock trees; also wired defensively
    into the recovery handler so a future registrar cannot bypass it.

On failure both paths fail closed (drop ancestry / fail registration),
so no watch is armed on an unverified commitment; the VTXO still 
persists
for cooperative use and its exit material is restored by a later
backfill. OOR skips the check when evidence is absent (older indexer).
When a tracked tx's confirmation is reorged out and later reconfirms,
the terminal-notification idempotency key must change so a durable
subscriber does not drop the re-confirmation as a duplicate of the
pre-reorg one. terminalNotifyKey was stable across reorg epochs, so a
reorg-aware consumer that rolled its state back on TxReorged -- e.g. an
in-progress unilateral exit waiting to re-drive -- never received the
re-confirmation and stalled indefinitely (funds safe on-chain but the
exit never swept).

Fold a per-entry reorg epoch (incremented in handleConfirmationReorged)
and the last confirmation height into terminalNotifyKey. Same-epoch,
same-height retries still dedup; a reorg-and-reconfirmation now carries
a fresh key and is delivered. Adds a regression test that fails without
the epoch increment.
Reflow lines flagged by the ll linter across the reorg
lineage-producer files. Formatting only, no behavior change.
TestGetBatchReadsUnderLock swapped mgr.cfg.Store on the test goroutine
while the actor was still processing the spend observation, which reads
cfg.Store on its own goroutine -- a data race the -race unit job caught.
Add a synchronizing Ask so the FIFO mailbox drains before the swap.
Adapts #990's reorg lineage/OOR-gating code to the current lint gates:
drops a stray duplicated godoc line left by a merge on
LineageCommitmentTxIDs, and adds justified funlen/nestif directives
where main's growth plus the canonicality gate pushed the incoming-VTXO
dispatch and custom-input path past the thresholds. No functional
changes.
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 928eadb to 405a86f Compare August 17, 2026 15:52
@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

1 similar comment
@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

6 similar comments
@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

@bhandras

Copy link
Copy Markdown
Member

Closing this unmerged producer-wiring stack as superseded. The selected client confirmation-policy and retry behavior shipped in #1199. This closure does not claim independent local lineage authority or limbo-balance reporting. Remaining unroll recovery stays tracked in #207.

@bhandras bhandras closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants