Repository navigation
feat(platform-wallet-storage): persist DashPay backfill coverage and outbound-account markers - #5479
HashEngineering wants to merge 11 commits into
Conversation
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
✅ Final review complete — no blockers (commit 8e1ee2e) · triage: critical |
thepastaclaw
left a comment
There was a problem hiding this comment.
Final validation — Phase 1 + Phase 2
Verified the supplied findings against head b3a06b1. Two blocking persistence defects remain: restoration can advance an explicitly rewound cursor, and the backfill writer accepts records that its reader rejects; the new integration tests also expose a smaller public-API usability issue. This was a static review: current CI checks pass, including Kotlin build/tests, but do not independently confirm the reported Rust storage-suite results.
🔴 2 blocking | 🟡 1 suggestion(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 5: gemini-3.8-flash-high (agent: phase1-reviewer, role: general); reviewer 6: gemini-3.8-flash-high (agent: phase1-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The change intricately couples durable backfill coverage, cursor rewinds, buffered persistence and restart recovery while changing storage schema in migrations/V019__dashpay_backfill.rs and cursor semantics in core_state.rs::upsert_sync_state. - Phase 1 reviewers:
gemini-3.8-flash-high— general (completed, effort high); agentphase1-reviewer,gemini-3.8-flash-high— rust-quality (completed, effort high); agentphase1-reviewer - Phase 1 model:
gemini-3.8-flash-high— antigravity quota: weekly 71% left, 5h 100% left - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— rust-quality (completed, effort xhigh); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.
In `packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/rehydrate.rs:293-299: Preserve an explicit rewind below the wallet-birth seed
The claimed clamp does not exist in the pinned dependency: WalletInfoInterface::update_synced_height directly assigns its argument, and reconcile_dashpay_rescan passes the contact checkpoint without clamping it to wallet birth. For example, a wallet with birth_height = 501 can persist a backfill cursor of 100, but reopening seeds synced_height at 500 and this max advances the restored cursor to 500. Because this PR now restores the coverage record too, reconciliation treats the contact as covered and does not rewind again. When the shared scanner has history below this wallet's birth, such as alongside an older wallet, the unscanned range between the saved cursor and 500 is skipped for this wallet. Restore the explicit stored synced_height instead of treating the initialization seed as observed progress, and add a relaunch regression with a checkpoint below a nonzero birth height.
In `packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs`:
- [BLOCKING] packages/rs-platform-wallet-storage/src/sqlite/schema/dashpay_backfill.rs:37-42: Reject coverage blobs that the load path cannot read
The writer inserts record.covered_bytes() without enforcing the 16 MiB limit checked by load_for_wallet at line 75. DashPayBackfillRecord::from_entries does not cap the entry count, V019 only requires a nonempty multiple of 72 bytes, and the connection's SQLite limit is 32 MiB. A record containing 233,017 distinct account keys therefore produces a 16,777,224-byte blob that store can successfully commit, but the next default load fails with BlobTooLarge and aborts restoration; Recovery mode omits the affected wallet. Although this requires a large coverage set, it lets a successful persistence operation make the wallet unrestorable through the normal load path. Check the encoded length before allocating and writing the blob, and add a boundary regression proving rejection preserves both the previous record and its cursor.
In `packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs`:
- [SUGGESTION] packages/rs-platform-wallet-storage/tests/integration/sqlite_dashpay_backfill_relaunch.rs:469: Expose registration outcomes instead of asserting Debug strings
The new test compares the Debug strings "Built" and "AlreadyExisted" because ExternalAccountRegistration is public inside the private network::contacts module but is not re-exported. The public DashPayView method therefore returns an outcome whose variants external callers cannot name, and the integration test works around that API gap by depending on presentation text rather than typed outcomes. Re-export ExternalAccountRegistration and RegisterExternalError from the wallet's public identity API, then use typed assertions here and at the relaunch assertion. This is a non-blocking API-usability improvement, not a layer-ownership violation.
Out-of-scope follow-up suggestions (1)
These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.
- Track SQLite outbound-account rotation conflicts separately — Both the base and reviewed head reject an outbound-account rebuild when a genuinely rotated xpub supplies different typed pool keys at existing indices: core_pool::apply_pools returns TypedPoolKeyConflict. The rotation teardown removes only in-memory accounts, leaving the conflicting SQLite pool rows in place for subsequent retries. This predates the PR, and the registration transaction rolls back rather than persisting a falsely trusted marker.
- Follow-up: Track a separate SQLite rotation operation that atomically replaces the old outbound pool with its account registration, preserves immutable-key protections for other account types, and covers rotation plus reopen through the real manager.
|
Waiting for bot review — coderabbitai not yet · thepastaclaw requested changes — dismiss the review or push a fix, 2 threads unresolved — resolve them. Wait for the missing reviews, or a writer can post |
|
Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them. |
b3a06b1 to
88283da
Compare
|
Waiting for bot review — coderabbitai ✓ · thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
Reviewed the complete diff at 88283da and independently confirmed that all three prior findings are fixed, with no remaining in-scope defects identified. SQLite preserves atomic record/cursor writes, restores deliberate rewinds faithfully, and persists outbound-account markers while leaving wallet policy in rs-platform-wallet. Validation was static locally; current exact-head CI passes scoped Rust wallet tests and clippy, Swift schema/build/tests, and Kotlin build/tests, while the full Rust workspace suite was skipped.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 5: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 6: gpt-6-astra (agent: phase2-reviewer, role: architecture-layering); reviewer 7: gpt-6-astra (agent: phase2-reviewer, role: platform-versioning); reviewer 8: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 9: gemini-3.8-flash-high (agent: phase1-reviewer, role: general); reviewer 10: gemini-3.8-flash-high (agent: phase1-reviewer, role: rust-quality); reviewer 11: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 12: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 13: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 14: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The change combines a storage migration in migrations/V019__dashpay_backfill.rs with intricate persistence and recovery semantics spanning atomic backfill records, deliberate cursor rewinds, and restart rehydration. - Phase 1 reviewers:
gemini-3.8-flash-high— general (completed, effort high); agentphase1-reviewer,gemini-3.8-flash-high— rust-quality (completed, effort high); agentphase1-reviewer - Phase 1 model:
gemini-3.8-flash-high— antigravity quota: weekly 71% left, 5h 100% left - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— rust-quality (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— general (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— rust-quality (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— rust-quality (completed, effort xhigh); agentphase2-reviewer - Model comparison: every Phase-2 reviewer also ran on
gpt-6-astra; the verifier weighed both sets without knowing which model wrote which
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
Out-of-scope follow-up suggestions (1)
These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.
- Coordinate explicit below-birth backfills with the SPV scan floor — Both the pinned and current default rust-dashcore revisions derive earliest_required_height from the minimum loaded-wallet birth height, and dash-spv applies that floor during scan startup and restart. With only a wallet born at 501 loaded, correctly restoring its cursor to 100 therefore does not make the scanner traverse heights 101–500, although reconcile_dashpay_rescan can record coverage from that earlier checkpoint. This is a concrete pre-existing wallet/scanner contract limitation, not a defect introduced by SQLite persistence; restoring the stored cursor remains the correct adapter behavior.
- Follow-up: Track a separate coordinated platform-wallet/rust-dashcore fix that communicates explicit backfill requirements to the scanner and ensures recorded coverage reflects the reachable scan range. Include a real-scanner regression with a contact checkpoint below the only loaded wallet's birth height.
|
Bots are done — your move: post |
|
/self-reviewed |
|
Ready for review — |
|
/self-reviewed |
|
Ready for review — |
…outbound-account markers The SQLite backend loaded every wallet with an empty DashPay backfill record and kept a monotonic-max cursor, so on this backend the coreHeight backfill re-ran on every launch (#4302). Outbound-account markers were handed to it in Immediate mode (ACCOUNT_ADDRESS_POOLS is an alias of the ASSET_LOCK_FUNDING_INDICES it already declares) but silently dropped, since no column held them, so each cold start rebuilt every outbound account. - V019 adds `dashpay_backfill` (one row per wallet: floor, rewound_from, and the covered receival accounts as one `covered_bytes` blob — the layout the FFI hosts store) and a nullable `contacts.external_account_reference`. - The record is written in the store's transaction beside the cursor it vouches for; every write replaces it whole and an empty record clears it. Load hands it back through `ClientWalletStartState::dashpay_backfill`; a torn cover set reads as no record, the codec's safe direction. - `stores_dashpay_backfill()` is true; durable coverage still needs `FlushMode::Immediate` via `store_commits_inline`. - A cursor marked `synced_height_is_rewind` overwrites the stored `synced_height`; advances keep the monotonic max. Tests pin the order through the Manual-mode buffer, a failed-flush restore, and identity re-add segments. - New `DashPayBackfill` version domain; migration fingerprints bumped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…irth and refuse unreadable cover sets - Restore the stored `synced_height` as written instead of maxing it against the `birth_height - 1` seed. A DashPay backfill rewind can lower the cursor below the wallet's birth (`reconcile_dashpay_rescan` passes the contact checkpoint to `update_synced_height` unclamped), and the restored record vouches for that cursor, so the seed skipped the range between them. Matches the FFI restore path; replaces a comment that claimed a clamp the pinned key-wallet does not have. - Refuse, before writing, a backfill record whose encoded cover set exceeds the blob limit the loader reads under, so a successful store can never leave the wallet unloadable. The round fails whole; the record and cursor already on file stand. - Re-export `ExternalAccountRegistration` and `RegisterExternalError` from the identity API so callers (and the relaunch test) can match the outcome of `register_external_contact_account` by type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as plain columns Review asked for values rather than a blob, on the tables that already hold the facts. The record's extent moves to core_sync_state.dashpay_backfill_floor / dashpay_backfill_rewound_from, beside the cursor it lowered, and each covered entry becomes account_registrations.dashpay_covered_from on the receiving account it names. The dashpay_backfill table and its 72-byte cover-set blob are gone, and with them the blob size limit the writer had to enforce. A covered entry whose receiving account has no row is dropped (re-armed, the safe direction); when none remain the extent is cleared too. The account upsert only rewrites the xpub, so coverage survives a rotation. The migration is renumbered V022: V019 is taken on v5.1-dev and V020 / V021 are claimed by open PRs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
load() read each table in its own autocommit statement, so another process could commit between two of them. A rewound cursor plus its backfill coverage landing after the cursor read and before the coverage read handed back the old cursor beside coverage that suppresses the rescan below it. The whole load now runs in one deferred read transaction, which in WAL mode pins the snapshot every read sees. The regression test injects that commit from a second connection while load() is between the two reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on to V020 v5.1-dev ends at V019 and now requires migration versions to be unique and contiguous from V001 (#5553), so the migration takes the next free version instead of skipping ahead of open PRs. The two schema fingerprint pins follow the rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… own xpub and count the load snapshot A receiving account's xpub derives from the wallet's seed at index'/owner/contact, the row's key, so re-registering it carries the same xpub; the test no longer implies coverage survives a different key. The load statement-count bound allows the BEGIN and COMMIT of the read snapshot load() now runs in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… instead of zeroing it The coverage loader turned each identity column into an Identifier through a helper that fell back to the all-zero id on a wrong length. The width check before it made that unreachable, but a future caller skipping the check would have attributed coverage to the zero identity. Convert through the shared checked id32 instead, so a bad id is an error like every other identity column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…backend refuses writes A SqlitePersister opened under LoadPolicy::Recovery refuses every store, but publishing a wallet could still replace the host's DashPay backfill record: when the load dropped coverage for a receiving account it did not restore, when the backend cannot hold coverage durably (Manual), or when the same manager had published the wallet before. The refused write failed load_from_persistor instead of serving the degraded, read-only projection. PlatformWalletPersistence gains a defaulted refuses_writes(), which SqlitePersister answers under Recovery. settle_published_backfill skips the replacement round for such a backend and keeps its result in memory; the record on file waits for a writable load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed type and the sent-sweep gate Two base changes broke the SQLite relaunch tests after the rebase: - rust-dashcore past the FFI removal (#5729) returns the BIP-39 seed as `Zeroizing<[u8; 64]>`, so `seed_wallet` dereferences it for `Wallet::from_seed_bytes`. - #4740 defers a receiving account's rescan until a sent-request sweep has completed in this process. These tests seed contacts without a DashPay sync, so every pass returned `None` and the session-one rewind failed. After a relaunch the gate also made "a covered contact is not rewound again" pass whether or not the persisted record covered it. `ManagedIdentity::mark_sent_sweep_completed_for_test` (behind `test-utils`) lets the tests mark the sweep as each launch's sync would: after establishing a contact and in every relaunched process before its pass. The control that clears the record again rewinds, so the record is still what stops the second rewind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uses_writes `ensure_writable` repeated the `LoadPolicy::Recovery` comparison that `refuses_writes` reports to the manager. Delegating to it keeps the gate that refuses a write and the capability the manager reads before its own housekeeping writes on one predicate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on to V021 #5762 merged `V020__asset_lock_shielded_recipient.rs`, and `embedded_migration_versions_are_unique_and_contiguous` (#5553) requires one migration per version from V001 with no gaps, so the backfill migration moves to the next free version. Its SQL is unchanged. - SCHEMA.md cites V021 and drops two stale history rows left by the rebase. - The upgrade test migrates to V020, the last pre-backfill schema. - Both schema pins are recomputed: refinery checksums the name and version. - rusqlite 0.41 on the base returns a `Result` from `trace_v2` and needs the callback type when clearing it; the snapshot test now matches the base's pattern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2708f7d to
8e1ee2e
Compare
|
Rebased onto The rebase also needed two test-only adaptations to the new base:
@lklimek @llbartekll: since you wrote and merged #5762, note that the upgrade test now starts from V020, so it includes the |
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
At head 8e1ee2e, the additive V021 migration follows the base's V020 migration, and the rebase adaptations preserve the intended persistence behavior; no actionable in-scope defects remain. Current CI reports successful Rust wallet, Swift SDK, and Kotlin SDK checks, while the full Rust workspace suite was skipped and review/hygiene checks remain pending. This verification was static; no local builds or tests were run.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 6: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 7: gpt-6-astra (agent: phase2-reviewer, role: architecture-layering); reviewer 8: gpt-6-astra (agent: phase2-reviewer, role: platform-versioning); reviewer 9: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 10: muse-spark-1.3-contributor (agent: phase1-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
criticalbygpt-6.1-sol(effort low) — The change combines intricate transactional persistence, rewind handling, and restart reconstruction with a storage migration in packages/rs-platform-wallet-storage/migrations/V021__dashpay_backfill.rs, meeting both the complexity and critical-surface criteria. - Phase 1 reviewers:
muse-spark-1.3-contributor— general (completed, effort xhigh); agentphase1-reviewer,muse-spark-1.3-contributor— rust-quality (completed, effort xhigh); agentphase1-reviewer - Phase 1 model:
muse-spark-1.3-contributor— not quota-gated; passed overgemini-3.8-flash-high(antigravity quota lookup failed),glm-5.3-flash(not used above high effort; tier asks max) - Single stage: Phase 1 and Phase 2 reviewed this head side by side, with no blocker gate between them (triage tier)
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6.1-sol— rust-quality (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— general (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— architecture-layering (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— platform-versioning (completed, effort xhigh); agentphase2-reviewer,gpt-6-astra— rust-quality (completed, effort xhigh); agentphase2-reviewer - Model comparison: every Phase-2 reviewer also ran on
gpt-6-astra; the verifier weighed both sets without knowing which model wrote which
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
|
Bots are done — your move: post |
|
@HashEngineering note the #5150 based stack also has conflicting migrations, we need to handle that carefully. |
llbartekll
left a comment
There was a problem hiding this comment.
Approving the rs-platform-wallet part at 8e1ee2e2. That is the only part I read, because it is the only part that links into the iOS FFI.
Scope
- Read: the 7 files under
packages/rs-platform-wallet/src. - Not read:
rs-platform-wallet-storage(theV021migration, the SQLite persister, the schema code) and its tests. iOS stores through SwiftData and does not use that backend, so I leave it to the other reviewers.
The wallet-crate part
- No behaviour change for a host on
FFIPersister.refuses_writes()defaults tofalse, onlySqlitePersisteroverrides it (LoadPolicy::Recovery), andFFIPersisterdoes not. The new early return insettle_published_backfillis therefore never taken on iOS. - The rest is inert for a host. A doc comment on
synced_height_is_rewind, two re-exports of types that are already public incontacts.rs, and two test helpers.test_supportis behindcfg(any(test, feature = "test-utils"))andmark_sent_sweep_completed_for_testcarries the same gate, so neither reaches a release build.
iOS and the backfill record
You asked whether the re-run is an issue on iOS. I checked v5.1-dev at c8a284b6c0:
FFIPersister::stores_dashpay_backfill()is true only when the host registerson_persist_wallet_dashpay_backfill_fninPersistenceCallbacksExtension.- The Swift SDK's
makePersistenceCallbacksExtension()does not set that slot, and nothing underpackages/swift-sdk/Sourcesrefers to the record.
So on iOS the coverage is session-only, and by the reasoning in this PR's description the backfill runs again on a later launch. I have not measured it on a device. Two consequences:
- The description's line that #5026 gives this fix to "the Kotlin and Swift hosts" holds for Kotlin only.
- iOS needs its own follow-up in the Swift SDK: register the slot, keep the record in SwiftData and hand it back on load. This PR neither helps nor blocks that.
|
@lklimek thanks, agreed. Here's where the numbers stand on
#5781 and #5384 in the stack add none. Whichever side merges second renumbers. Which order do you prefer?
#5479 can go after the #5150 stack, so if that is easier for you, land the stack first and I'll renumber this one to V023. |
Basic explanation
What this does: Teaches the SQLite wallet store to save two facts that #5026 added to the wallet's persistence contract: how far the DashPay "backfill" rescan has already got (so a restart does not start it again), and which account version each DashPay contact's outbound payment account was built for (so a restart does not rebuild it).
Value: Wallets on the SQLite backend stop re-walking the same block range on every launch once a DashPay backfill has run — the same fix #5026 gives the Kotlin host. On the reference mainnet wallet that re-walk was ~377,000 filters per launch. Contacts' outbound accounts also stop being torn down and rebuilt on every cold start.
Risks: Low. No consensus impact — this is client-side wallet storage only. The migration only adds nullable columns, so an upgraded database behaves exactly as before until the first backfill is recorded (one more re-run on first launch). The one behaviour change to existing data is that a stored sync cursor can now go down when Rust marks it as a deliberate rewind; ordinary advances still keep the highest value.
Issue being fixed or feature implemented
Follows #5026 (fixes #4302 for FFI hosts that store the record). That PR made the DIP-15 §12.6 coreHeight backfill durable through a new
DashPayBackfillRecordthat hosts store and hand back on load, plus asynced_height_is_rewindflag the host must honour. Kotlin implements it; the Swift SDK and the SQLite backend did not. This PR covers the SQLite backend:synced_height, so on this backend the backfill re-ran on every launch — the Persist DashPay coreHeight backfill completion (rescan_triggered is in-memory only) #4302 behaviour.FlushMode::Immediateit already qualified for outbound-account markers (ACCOUNT_ADDRESS_POOLSis an alias of theASSET_LOCK_FUNDING_INDICESit declares), so markers were handed to it — and silently dropped, since no column held them.iOS is not covered here. The Swift SDK does not register
on_persist_wallet_dashpay_backfill_fn, soFFIPersister::stores_dashpay_backfill()is false there: coverage lasts one session and the backfill re-runs on a later launch. That needs its own Swift SDK follow-up (register the slot, keep the record in SwiftData, hand it back on load); this PR neither helps nor blocks it. Thanks to llbartekll for catching this.What was done?
Migration
V021__dashpay_backfill.rs(purely additive, nullable columns only — no new table, no blob)core_sync_state.dashpay_backfill_floor/dashpay_backfill_rewound_from: the record's extent, beside the cursor it lowered. CHECK: both set or both NULL, each au32.account_registrations.dashpay_covered_from: the height a covered DashPay receiving account is covered from. A covered entry names exactly one receiving account (owner, contact, index), which is that row's key. CHECK:u32, and only on'dashpay_receiving'rows.contacts.external_account_reference: the outbound-account marker, au32.v5.1-dev, which ends at V020 since feat(platform-wallet): persist the Orchard recipient of a shielded asset-lock funding #5762 mergedV020__asset_lock_shielded_recipient.rs(this PR used V020 until then). Migration versions there must be unique and contiguous from V001 (embedded_migration_versions_are_unique_and_contiguous, test(platform-wallet-storage): require unique, contiguous migration versions #5553), so this cannot skip ahead to dodge the open PRs that also add a migration (feat(platform-wallet): support DashPay shielded tips with dedicated accounts #5288, feat(platform-wallet)!: persist engine spent claims and restore them across backends #5306, fix(platform-wallet)!: correct wallet-level transaction history accounting #5150): whichever of them merges after this one renumbers to the next free version.Backfill record —
src/sqlite/schema/dashpay_backfill.rsapplyreplaces the wallet's record inside the store's transaction, after the round's account registrations: it clears everydashpay_covered_from, sets it on each covered account's row (addressed through the same typed-column helpers the account writer binds), then sets or clears the extent. An empty record only clears (the upstream invalidation).account_xpub_bytes, so coverage survives the receiving account being re-registered. A receiving account's xpub derives from the wallet's own seed atindex'/owner/contact, the row's key, so it never changes under a covered height.load_for_walletjoins the columns back throughDashPayBackfillRecord::from_entries(canonical order). Coverage without an extent reads as "no record". Wired intoload_one_wallet→ClientWalletStartState::dashpay_backfill.SqlitePersister::load()now reads inside one deferred read transaction, so the cursor and the coverage that suppresses the rescan below it come from one snapshot. Before, another process could commit a rewind plus its coverage between the two reads, and the load handed back the old cursor beside the new coverage.SqlitePersister::stores_dashpay_backfill()returnstrue. Durable coverage still requiresFlushMode::Immediatethrough the existingstore_commits_inlinegate inpersists_backfill_coverage; Manual mode writes and clears records but keeps coverage session-only.Domain::DashPayBackfillinschema/versions.rs(replaces the explicitdashpay_backfill: _ignore). Its version label is"dashpay_backfill", like"core"a domain rather than a table name.Rewind —
schema/core_state.rs::upsert_sync_staterehydrate.rs::apply_persisted_core_state) now assigns the storedsynced_heightas written instead of maxing it against thebirth_height - 1seed — a backfill rewind can sit below the wallet's birth, and the FFI restore path already assigns it directly.synced_heightmarkedsynced_height_is_rewindoverwrites the stored cursor; advances,last_processed_heightandchainlock_heightkeep the monotonic max. A rewind can only lower the tombstone finality boundary (min(chainlock, synced)), so it never collects more.Outbound-account marker —
schema/contacts.rsWalletPersister::storealready strips it on backends that cannot keep it, and the manager's load clears restored markers it cannot trust. A trusted marker relies onapply_registrationsoverwriting the account xpub on rotation, plus the load-side check that distrusts a marker when an external pool holds addresses its current xpub does not derive.Read-only backends — new defaulted
PlatformWalletPersistence::refuses_writes()(false);SqlitePersisteranswerstrueunderLoadPolicy::Recovery, where it already refuses every write.settle_published_backfillskips its replacement round for such a backend and keeps the result in memory, soload_from_persistor()serves the degraded projection instead of failing onReadOnlyRecoveryMode. The record on file is left for a writable load to replace.Identity API —
ExternalAccountRegistration/RegisterExternalErrorare re-exported fromplatform_wallet::wallet::identity, so callers can matchregister_external_contact_account's outcome by type.Docs —
SCHEMA.md(the new columns oncore_sync_state,account_registrationsandcontacts, the V021 log entry, the rewind exception oncore_sync_state); a stale comment inrs-platform-wallet/src/changeset/changeset.rsthat said SQLite ignores the rewind flag.How Has This Been Tested?
cargo test -p platform-wallet-storage --all-features(589 integration + 438 unit, all pass;cargo test -p platform-wallet --all-features --lib: 1778 pass) andcargo clippy -p platform-wallet-storage -p platform-wallet --all-features --all-targets -- -D warnings(clean).tests/integration/sqlite_dashpay_backfill_relaunch.rs(new, end-to-end): the realPlatformWalletManagerover a realSqlitePersister, with the file closed and reopened between sessions:DashpayExternalAccountpresent and the same xpub, so registering it again reportsAlreadyExisted.refuses_writesfails all three withReadOnlyRecoveryMode).test_support::encrypt_contact_xpub_for_test(test-utilsonly) so the storage tests can build a contact's encrypted xpub offline.tests/integration/sqlite_dashpay_backfill.rs(new):load()reads the cursor and its coverage from one snapshot: a second connection commits a rewind plus coverage between the two reads (mutation-checked).sqlite_contacts_keys_rehydration.rs: the outbound-account marker round-trips.sqlite_version_bump.rs,sqlite_schema_pinning.rs, the new domain and the migration fingerprints.Breaking Changes
None. The schema change is an additive migration; an older binary cannot open a V021 database, which is the existing forward-version gate for every migration.
Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull requestFor repository code-owners and collaborators only
🤖 Generated with Claude Code
PR Hygiene ·
8e1ee2e/self-reviewedwallet-storage(packages/rs-platform-wallet-storage/SCHEMA.md,packages/rs-platform-wallet-storage/migrations/V021__dashpay_backfill.rs,packages/rs-platform-wallet-storage/src/sqlite/persister.rsand 13 more) — lklimekrs-platform-wallet(packages/rs-platform-wallet/src/changeset/changeset.rs,packages/rs-platform-wallet/src/changeset/traits.rs,packages/rs-platform-wallet/src/manager/mod.rsand 4 more) — approved by llbartekllWhen every merge requirement is met, the
PR Hygienecheck passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.Summary by CodeRabbit