diff --git a/AGENTS.md b/AGENTS.md index fed2480b870..5b22542ff0e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -84,7 +84,7 @@ Platform uses data contracts to define application data schemas: Use LF, UTF‑8, and a final newline. - JS/TS: ESLint (Airbnb/TypeScript rules via package configs). Use camelCase for variables/functions, PascalCase for classes; prefer kebab-case filenames within JS packages. - Rust: Follow rustfmt defaults; keep code clippy-clean. Modules `snake_case`, types `PascalCase`, constants `SCREAMING_SNAKE_CASE`. -- Architecture rules for Rust and its consumers (FFI, WASM, Swift and Kotlin SDKs) live in The Dash Platform Book (`book/`). Read [book/src/contributing/coding-conventions.md](book/src/contributing/coding-conventions.md) before changing Rust, FFI, WASM, Swift or Kotlin code, and check the change against it before finishing; it states each rule, why it exists, and links to the chapter with the mechanics. Key rules: new behaviour is a new `vN` selected only by the unreleased protocol version's tables, and a shipped `vN` is edited in place only when the edit provably cannot modify consensus there, with the reason in the PR description's "In-place changes to shipped generations" section; numbers go in `SystemLimits`, fees in named `FEE_VERSION*` schedules; `platform_version` is the last parameter; no `unwrap`/`expect` on block-execution paths (a panic halts the chain); imports at the top, no inline `crate::` paths; latest-generation tests use `PlatformVersion::latest()`; FFI, WASM, Swift and Kotlin wrappers marshal rather than decide; validation lives once in its owner and is not duplicated outside it. +- Architecture rules for Rust and its consumers (FFI, WASM, Swift and Kotlin SDKs) live in The Dash Platform Book (`book/`). Read [book/src/contributing/coding-conventions.md](book/src/contributing/coding-conventions.md) before changing Rust, FFI, WASM, Swift or Kotlin code, and check the change against it before finishing; it states each rule, why it exists, and links to the chapter with the mechanics. Key rules: new behaviour is a new `vN` selected only by the unreleased protocol version's tables, and a shipped `vN` is edited in place only when the edit provably cannot modify consensus there, with the reason in the PR description's "In-place changes to shipped generations" section; numbers go in `SystemLimits`, fees in named `FEE_VERSION*` schedules; `platform_version` is the last parameter; no `unwrap`/`expect` on block-execution paths (a panic halts the chain); imports at the top, no inline `crate::` paths; no `#[non_exhaustive]` on the workspace's own error enums, and per-variant matches over them list every variant instead of `_` (a new variant must break the build); latest-generation tests use `PlatformVersion::latest()`; FFI, WASM, Swift and Kotlin wrappers marshal rather than decide; validation lives once in its owner and is not duplicated outside it. ## Testing Guidelines - Unit/integration tests live alongside each package (e.g., `packages//tests`). E2E lives in `packages/platform-test-suite`. diff --git a/book/src/contributing/coding-conventions.md b/book/src/contributing/coding-conventions.md index 3b79f66926c..58a68583b9b 100644 --- a/book/src/contributing/coding-conventions.md +++ b/book/src/contributing/coding-conventions.md @@ -362,6 +362,22 @@ or `ExecutionError` means the build is wrong, data is corrupt, or an API was misused. Never return a consensus error for an internal failure or an internal error for a user mistake; the block loop treats the two classes differently. +### Error enums stay exhaustive + +Do not mark the workspace's own error enums `#[non_exhaustive]`. A `match` +that decides something per variant (transient or not, an FFI error code, a +user-facing message) lists every variant instead of ending in `_`. A new +variant must break the build at each such `match`, so the author of the +variant sees every caller in the compiler output and handles it in the same +PR. With +`#[non_exhaustive]`, callers outside the defining crate need a wildcard arm, +and that arm silently absorbs every later variant: the code still compiles and +the new failure takes whatever path the wildcard picked, typically a generic +error or a test panic, without anyone being told. The workspace crates ship +together, so the compile break costs nothing for a consumer outside the crate. +A wildcard is correct only over enums from other repositories that are +`#[non_exhaustive]` upstream (for example `rusqlite::ErrorCode`). + ## Module layout and style - **One method, one directory.** `method_name/mod.rs` holds the public diff --git a/packages/rs-platform-wallet-storage/Cargo.toml b/packages/rs-platform-wallet-storage/Cargo.toml index 6877f8ba404..6e4f94059cd 100644 --- a/packages/rs-platform-wallet-storage/Cargo.toml +++ b/packages/rs-platform-wallet-storage/Cargo.toml @@ -161,6 +161,8 @@ apple-native-keyring-store = { version = "=1.0.0", features = ["keychain"], opti windows-native-keyring-store = { version = "=1.0.0", optional = true } [dev-dependencies] +# Safe SQLite fault injection for migration rollback tests. +rusqlite = { version = "0.41", features = ["functions"] } # `test-utils` reaches `provider_key_test_wallet`, shared with # `platform-wallet`'s own `rebuild_provider_key_account` tests — see its use # in `sqlite/provider_accounts.rs`'s test module. It also exposes diff --git a/packages/rs-platform-wallet-storage/SCHEMA.md b/packages/rs-platform-wallet-storage/SCHEMA.md index 94485778a6c..b6273525d6d 100644 --- a/packages/rs-platform-wallet-storage/SCHEMA.md +++ b/packages/rs-platform-wallet-storage/SCHEMA.md @@ -42,7 +42,7 @@ Any `meta_*` row whose parent object does not exist — because it was never cre A future garbage-collection pass is expected to reap orphan metadata — rows with no live parent object older than approximately one week — but no such GC is implemented yet. Callers should not rely on orphan metadata persisting forever, nor assume it will be cleaned up promptly. `meta_global` is intentionally parentless and always survives. -The tables are split into five domain diagrams below. `WALLETS` is the root anchor and appears in each diagram. The diagrams cover 21 of V001's 23 tables; `pending_contact_crypto` and `ignored_senders` appear in the [Tables](#tables) section but are not diagrammed. They show the V001 tables as amended in place by every later migration that changes one of them (the current `core_utxos`, `core_transactions`, `platform_addresses`, and `asset_locks` shapes). Nine tables added by later migrations are not yet diagrammed here: `core_address_pool`, `meta_data_versions`, and `meta_store_generation` (V009, plus its V010–V011 `core_address_pool` columns), `invitations` (V003), `shielded_viewing_keys` (V013), `dpns_name_states` (V005), `tracked_masternodes` (V006), and the `identity_scan_states` / `identity_scan_failed_indices` pair (V017) — see the [Migrations](#migrations) log for what each adds in the meantime. +The tables are split into five domain diagrams below. `WALLETS` is the root anchor and appears in each diagram. The diagrams cover 21 of V001's 23 tables; `pending_contact_crypto` and `ignored_senders` appear in the [Tables](#tables) section but are not diagrammed. They show the V001 tables as amended in place by every later migration that changes one of them (the current `core_utxos`, `core_transactions`, `platform_addresses`, and `asset_locks` shapes). Eleven tables added by later migrations are not yet diagrammed here: `core_address_pool`, `meta_data_versions`, and `meta_store_generation` (V009, plus its V010–V011 `core_address_pool` columns), `invitations` (V003), `shielded_viewing_keys` (V013), `dpns_name_states` (V005), `tracked_masternodes` (V006), the `identity_scan_states` / `identity_scan_failed_indices` pair (V017), and `core_transaction_inputs` / `core_transaction_record_originals` (V021) — see the [Migrations](#migrations) log for what each adds in the meantime. ## Diagram 1 — Core / L1 (Bitcoin/Dash layer) @@ -440,6 +440,36 @@ held spent and is permanently outside the collector's reach. the collector's per-round scan touches tombstones rather than the wallet's full spent history. +### `core_transaction_inputs` + +V021 indexes every raw input of each stored transaction, even when its ownership +is not yet known. New historical TXOs locate affected records through this index. +The shared wallet accounting rule updates input/output details, net amount and +direction; it never changes coins, spend claims or finality. Contact-only scripts +are excluded from wallet ownership. + +V021 also indexes and repairs existing mined and unmined records using retained +TXOs. Unknown historical ownership is filled when later observations supply it. +No rows are deleted and no sync checkpoint is rewound. The typed backfill uses +the live record decoder, with immutable V019 fixtures pinning codec compatibility. +Type, identifier-width and blob-size gates precede materialization. Strict failure +rolls back the entire migration; Recovery skips an unreadable record without +changing its bytes or partially indexing it. Each record's repair has a savepoint. + +- PK: `(wallet_id, txid, outpoint)`. +- FK: `(wallet_id, txid) → core_transactions(wallet_id, txid) ON DELETE CASCADE`. +- Index: `idx_core_transaction_inputs_outpoint(wallet_id, outpoint)`. + +### `core_transaction_record_originals` + +Append-only archive of the first pre-repair `core_transactions.record_blob`. +Migration and runtime repairs copy original bytes with `INSERT OR IGNORE` before +changing accounting. Later repairs cannot replace this recovery copy. + +- PK: `(wallet_id, txid)`. +- FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`. The original survives + deletion of its transaction row. + ### `core_instant_locks` Instant-lock blobs for transactions that are broadcast but not yet @@ -851,3 +881,4 @@ table-rebuild migration, as V004 does. | V018 | `V018__identity_hard_delete.rs` | Retires identity tombstoning. Adds `cascade_children_on_identity_delete` (brooms `identity_keys` / `contacts` / `ignored_senders` / `pending_contact_crypto` by the deleted identity id, covering the rows no live FK reaches) plus its access-path indexes `idx_contacts_owner`, `idx_ignored_senders_owner`, and `idx_pending_contact_crypto_owner`; purges every already-tombstoned identity and its dependents; drops `identities.tombstoned`. | | V019 | `V019__asset_lock_operation_id.rs` | Adds the nullable `asset_locks.operation_id BLOB` column (CHECK: NULL or exactly 16 bytes) that correlates a wallet-drain operation with its funding outpoint; the lifecycle blob is untouched. Adds the unique partial index `asset_locks_wallet_operation_id` on `(wallet_id, operation_id) WHERE operation_id IS NOT NULL` and the `asset_locks_operation_id_immutable` BEFORE UPDATE trigger, which aborts any change to a non-NULL `operation_id`. Purely additive; existing rows read back NULL. | | V020 | `V020__asset_lock_shielded_recipient.rs` | Adds the nullable `asset_locks.shielded_recipient BLOB` column (CHECK: NULL or exactly 43 bytes, the raw Orchard address) recording the recipient a shielded asset-lock funding was built to pay, so a resume after restart pays that recipient and refuses another; the lifecycle blob is untouched. Adds the `asset_locks_shielded_recipient_immutable` BEFORE UPDATE trigger, which aborts any change to a non-NULL `shielded_recipient`. Purely additive; existing rows read back NULL and keep accepting the resuming caller's recipient. | +| V021 | `V021__core_transaction_accounting.rs` | Adds the raw-input index and original-record archive. Backfills all decodable history and repairs accounting from retained TXOs, preserving coins, claims, checkpoints and unrelated wallet state. | diff --git a/packages/rs-platform-wallet-storage/migrations/V021__core_transaction_accounting.rs b/packages/rs-platform-wallet-storage/migrations/V021__core_transaction_accounting.rs new file mode 100644 index 00000000000..0a480e11058 --- /dev/null +++ b/packages/rs-platform-wallet-storage/migrations/V021__core_transaction_accounting.rs @@ -0,0 +1,21 @@ +//! Index historical inputs and preserve original records before accounting repair. + +pub fn migration() -> String { + "CREATE TABLE core_transaction_inputs ( + wallet_id BLOB NOT NULL, + txid BLOB NOT NULL, + outpoint BLOB NOT NULL, + PRIMARY KEY (wallet_id, txid, outpoint), + FOREIGN KEY (wallet_id, txid) REFERENCES core_transactions(wallet_id, txid) ON DELETE CASCADE + ); + CREATE INDEX idx_core_transaction_inputs_outpoint + ON core_transaction_inputs(wallet_id, outpoint); + CREATE TABLE core_transaction_record_originals ( + wallet_id BLOB NOT NULL, + txid BLOB NOT NULL, + record_blob BLOB NOT NULL, + PRIMARY KEY (wallet_id, txid), + FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE + );" + .to_owned() +} diff --git a/packages/rs-platform-wallet-storage/src/sqlite/error.rs b/packages/rs-platform-wallet-storage/src/sqlite/error.rs index 9f843edf7bb..42fee7af8c8 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/error.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/error.rs @@ -418,6 +418,38 @@ pub enum WalletStorageError { blob_height: Option, }, + /// An incoming transaction record reuses a stored txid with a body whose + /// txid-committed content differs (witness-only differences are not a + /// conflict); neither copy is trusted to replace the other. + #[error( + "transaction {txid} in wallet {} arrived with a body whose txid differs from the stored one", + hex::encode(wallet_id) + )] + TransactionBodyConflict { + wallet_id: [u8; 32], + txid: dashcore::Txid, + }, + + /// The `wallets.network` label is not one this build knows, so stored + /// scripts cannot be turned back into addresses. + #[error( + "wallet {} has unknown network label {label:?}", + hex::encode(wallet_id) + )] + UnknownWalletNetwork { wallet_id: [u8; 32], label: String }, + + /// A transaction's net amount (owned outputs minus owned inputs) does not + /// fit the `i64` the record stores. + #[error( + "net amount {value} of transaction {txid} in wallet {} does not fit i64", + hex::encode(wallet_id) + )] + NetAmountOverflow { + wallet_id: [u8; 32], + txid: dashcore::Txid, + value: i128, + }, + /// A blob exceeded the decode allocation cap (default 16 MiB). /// Separate from [`Self::BlobDecode`] so operators can distinguish an /// oversize blob from a structural decode failure. @@ -754,6 +786,9 @@ impl WalletStorageError { | Self::AssetLockEntryMismatch { .. } | Self::AssetLockStatusMismatch { .. } | Self::CoreTransactionEntryMismatch { .. } + | Self::TransactionBodyConflict { .. } + | Self::UnknownWalletNetwork { .. } + | Self::NetAmountOverflow { .. } | Self::BlobTooLarge { .. } | Self::IntegerOverflow { .. } | Self::RehydrationPoolMismatch { .. } @@ -799,6 +834,11 @@ impl WalletStorageError { // Typed re-mapping of an FK violation — same class as the raw // `ConstraintViolation` above, so it reports the same kind. Self::IdentityKeyWalletMismatch { .. } => PersistenceErrorKind::Constraint, + // History invariants checked in Rust on the write path: the incoming + // record contradicts stored history, so the data is wrong, not the engine. + Self::TransactionBodyConflict { .. } | Self::NetAmountOverflow { .. } => { + PersistenceErrorKind::Constraint + } // Refinery surfaces FK / constraint problems through rusqlite; // if that path leaks through here the typed variant lives in // `Self::Migration`, which we leave as `Fatal` since a @@ -858,6 +898,7 @@ impl WalletStorageError { | Self::AssetLockEntryMismatch { .. } | Self::AssetLockStatusMismatch { .. } | Self::CoreTransactionEntryMismatch { .. } + | Self::UnknownWalletNetwork { .. } | Self::BlobTooLarge { .. } | Self::IntegerOverflow { .. } | Self::RehydrationPoolMismatch { .. } @@ -939,6 +980,9 @@ impl WalletStorageError { Self::AssetLockEntryMismatch { .. } => "asset_lock_entry_mismatch", Self::AssetLockStatusMismatch { .. } => "asset_lock_status_mismatch", Self::CoreTransactionEntryMismatch { .. } => "core_transaction_entry_mismatch", + Self::TransactionBodyConflict { .. } => "transaction_body_conflict", + Self::UnknownWalletNetwork { .. } => "unknown_wallet_network", + Self::NetAmountOverflow { .. } => "net_amount_overflow", Self::BlobTooLarge { .. } => "blob_too_large", Self::IntegerOverflow { .. } => "integer_overflow", Self::RehydrationPoolMismatch { .. } => "rehydration_pool_mismatch", diff --git a/packages/rs-platform-wallet-storage/src/sqlite/migrations.rs b/packages/rs-platform-wallet-storage/src/sqlite/migrations.rs index 89a614282e5..ff762617532 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/migrations.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/migrations.rs @@ -6,27 +6,34 @@ use rusqlite::OptionalExtension; +use crate::sqlite::config::LoadPolicy; use crate::sqlite::error::WalletStorageError; use refinery_core::error::WrapMigrationError; mod legacy_v008; +mod v021; // Generates a `migrations` module with `runner()`; path is relative to // the crate root. refinery::embed_migrations!("./migrations"); -/// Apply every pending migration to `conn`. +/// Apply every pending migration to `conn` under [`LoadPolicy::Strict`]. pub fn run(conn: &mut rusqlite::Connection) -> Result { - run_with_runner(conn, migrations::runner()) + run_with_runner(conn, migrations::runner(), LoadPolicy::Strict) } /// Keep refinery's history validation, target selection and SQL/history writes /// inside the same writer exclusion as the typed legacy conversion. Its native /// rusqlite driver starts its own transactions, so this adapter implements the /// supported driver traits over our outer transaction instead. +/// +/// `policy` reaches the typed steps that decode stored rows: under +/// [`LoadPolicy::Strict`] a row they cannot read fails the migration, under +/// [`LoadPolicy::Recovery`] it is logged and skipped. fn run_with_runner( conn: &mut rusqlite::Connection, runner: refinery::Runner, + policy: LoadPolicy, ) -> Result { let tx = conn .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate) @@ -45,6 +52,8 @@ fn run_with_runner( tx, registration_sql: hook_sql(8), pool_sql: hook_sql(11), + accounting_sql: hook_sql(21), + policy, }; // Grouped reports never claim that rolled-back migrations were applied. let report = runner.set_grouped(true).run(&mut driver)?; @@ -59,6 +68,8 @@ struct MigrationTransaction<'conn> { tx: rusqlite::Transaction<'conn>, registration_sql: String, pool_sql: String, + accounting_sql: String, + policy: LoadPolicy, } impl refinery_core::traits::sync::Transaction for MigrationTransaction<'_> { @@ -75,6 +86,8 @@ impl refinery_core::traits::sync::Transaction for MigrationTransaction<'_> { legacy_v008::backfill_registrations(&self.tx)?; } else if query == self.pool_sql { legacy_v008::convert_pools(&self.tx)?; + } else if query == self.accounting_sql { + v021::backfill_accounting(&self.tx, self.policy)?; } count += 1; } @@ -122,8 +135,9 @@ impl refinery_core::traits::sync::Migrate for MigrationTransaction<'_> {} /// migration that needs to re-classify a refinery error. pub(crate) fn run_for_open( conn: &mut rusqlite::Connection, + policy: LoadPolicy, ) -> Result { - run(conn).map_err(WalletStorageError::Migration) + run_with_runner(conn, migrations::runner(), policy).map_err(WalletStorageError::Migration) } /// Return a fresh atomic migration runner seeded with the embedded list. @@ -146,7 +160,7 @@ impl MigrationRunner { /// Apply the selected migration set and typed conversions atomically. pub fn run(self, conn: &mut rusqlite::Connection) -> Result { - run_with_runner(conn, self.0) + run_with_runner(conn, self.0, LoadPolicy::Strict) } } @@ -420,6 +434,63 @@ pub fn embedded_migrations_sql() -> Vec { .collect() } +/// Undo V021 so the next [`run`] replays it over the current rows. +#[cfg(test)] +pub(crate) fn rewind_v021(conn: &rusqlite::Connection) { + conn.execute_batch( + "DROP TABLE core_transaction_inputs; DROP TABLE core_transaction_record_originals; \ + DELETE FROM refinery_schema_history WHERE version >= 21;", + ) + .unwrap(); +} + +/// A migrated in-memory database holding one testnet wallet born at 0. +#[cfg(test)] +pub(crate) fn wallet_db(wallet_id: &[u8; 32]) -> rusqlite::Connection { + let mut conn = rusqlite::Connection::open_in_memory().unwrap(); + run(&mut conn).unwrap(); + insert_test_wallet(&conn, wallet_id, 0); + conn +} + +/// A [`wallet_db`] whose one unspent row has a script over the connection's +/// length cap, so reading the column itself fails: only a length-only +/// pre-read reports it as oversize. Returns the row's outpoint. +#[cfg(test)] +pub(crate) fn oversize_script_db( + wallet_id: &[u8; 32], +) -> (rusqlite::Connection, dashcore::OutPoint) { + use crate::sqlite::conn::SQLITE_MAX_BLOB_BYTES; + + let conn = wallet_db(wallet_id); + let outpoint = dashcore::OutPoint::new(dashcore::Txid::from_byte_array([0x42; 32]), 0); + conn.execute( + "INSERT INTO core_utxos (wallet_id, outpoint, value, script, spent) \ + VALUES (?1, ?2, 0, ?3, 0)", + rusqlite::params![ + &wallet_id[..], + crate::sqlite::schema::blob::encode_outpoint(&outpoint).unwrap(), + vec![0u8; SQLITE_MAX_BLOB_BYTES as usize + 1] + ], + ) + .unwrap(); + conn.set_limit( + rusqlite::limits::Limit::SQLITE_LIMIT_LENGTH, + SQLITE_MAX_BLOB_BYTES, + ) + .unwrap(); + (conn, outpoint) +} + +#[cfg(test)] +pub(crate) fn insert_test_wallet(conn: &rusqlite::Connection, wallet_id: &[u8; 32], birth: i64) { + conn.execute( + "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', ?2)", + rusqlite::params![&wallet_id[..], birth], + ) + .unwrap(); +} + #[cfg(test)] mod tests { use super::*; diff --git a/packages/rs-platform-wallet-storage/src/sqlite/migrations/v021.rs b/packages/rs-platform-wallet-storage/src/sqlite/migrations/v021.rs new file mode 100644 index 00000000000..ebb682bb0b9 --- /dev/null +++ b/packages/rs-platform-wallet-storage/src/sqlite/migrations/v021.rs @@ -0,0 +1,417 @@ +//! Backfill all historical input indexes and repair accounting without changing spend state. + +use rusqlite::Transaction; + +use crate::sqlite::config::LoadPolicy; +use crate::sqlite::error::WalletStorageError; +use crate::sqlite::schema::{blob, core_history, core_state, id32}; +use crate::sqlite::util::safe_cast::u64_to_i64; +use platform_wallet::changeset::CoreChangeSet; + +/// Index and repair every decodable historical record in the migration transaction. +pub(super) fn backfill_accounting( + tx: &Transaction<'_>, + policy: LoadPolicy, +) -> Result<(), WalletStorageError> { + // CASE is lazy in SQLite: rejected columns never enter the result row. + let mut stmt = tx.prepare( + "SELECT rowid, + CASE WHEN typeof(wallet_id) = 'blob' THEN length(wallet_id) END, + CASE WHEN typeof(wallet_id) = 'blob' AND length(wallet_id) = 32 THEN wallet_id END, + CASE WHEN typeof(txid) = 'blob' THEN length(txid) END, + CASE WHEN typeof(txid) = 'blob' AND length(txid) = 32 THEN txid END, + CASE WHEN typeof(record_blob) = 'blob' THEN length(record_blob) END, + CASE WHEN typeof(record_blob) = 'blob' AND length(record_blob) <= ?1 THEN record_blob END, + typeof(wallet_id) = 'blob' AND typeof(txid) = 'blob' AND typeof(record_blob) = 'blob' + FROM core_transactions WHERE rowid >= ?2 AND record_blob IS NOT NULL ORDER BY rowid LIMIT 1", + )?; + let limit = u64_to_i64("record_blob.limit", blob::BLOB_SIZE_LIMIT_BYTES as u64)?; + let mut next_rowid = i64::MIN; + loop { + // Close each cursor before repairing its row; no active SELECT observes writes. + let (rowid, record) = { + let mut rows = stmt.query((limit, next_rowid))?; + let Some(row) = rows.next()? else { break }; + let rowid: i64 = row.get(0)?; + let record = (|| { + if !row.get::<_, bool>(7)? { + return Err(WalletStorageError::blob_decode( + "historical transaction columns must be BLOBs", + )); + } + blob::check_fixed_width(row.get(1)?, 32, "core_transactions.wallet_id")?; + let wallet_id = id32("core_transactions.wallet_id", &row.get::<_, Vec>(2)?)?; + blob::check_fixed_width(row.get(3)?, 32, "core_transactions.txid")?; + let txid: Vec = row.get(4)?; + blob::check_size(row.get(5)?)?; + let record = core_state::decode_transaction_record(&row.get::<_, Vec>(6)?)?; + if record.txid.as_byte_array().as_slice() != txid.as_slice() { + return Err(WalletStorageError::blob_decode( + "historical transaction record is stored under another txid", + )); + } + Ok((wallet_id, record)) + })(); + (rowid, record) + }; + tx.execute_batch("SAVEPOINT accounting_record")?; + let repaired = record.and_then(|(wallet_id, record)| { + core_history::index_record(tx, &wallet_id, &record)?; + core_history::apply( + tx, + &wallet_id, + &CoreChangeSet { + records: vec![record], + ..Default::default() + }, + ) + }); + match (repaired, policy) { + (Ok(()), _) => tx.execute_batch("RELEASE accounting_record")?, + (Err(error), _) if error.is_transient() => return Err(error), + (Err(error), LoadPolicy::Strict) => return Err(error), + (Err(error), LoadPolicy::Recovery) => { + tx.execute_batch("ROLLBACK TO accounting_record; RELEASE accounting_record")?; + tracing::warn!(rowid, %error, + "V021 skipped an unreadable historical transaction; its stored bytes are unchanged"); + } + } + let Some(next) = rowid.checked_add(1) else { + break; + }; + next_rowid = next; + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use key_wallet::account::{AccountType, StandardAccountType}; + use key_wallet::managed_account::transaction_record::{OutputRole, TransactionDirection}; + use key_wallet::transaction_checking::{TransactionContext, TransactionType}; + use rusqlite::{limits::Limit, params}; + + use super::*; + use crate::sqlite::conn::SQLITE_MAX_BLOB_BYTES; + use crate::sqlite::migrations::{self, rewind_v021, wallet_db}; + + #[test] + fn should_roll_back_v021_on_transient_repair_failure_even_in_recovery() { + let wallet = [0x73; 32]; + let mut conn = wallet_db(&wallet); + let fields: Vec<_> = include_str!("../../../tests/fixtures/v019_kept_records.txt") + .lines() + .next() + .unwrap() + .split_whitespace() + .collect(); + let mut record = + core_state::decode_transaction_record(&hex::decode(fields[3]).unwrap()).unwrap(); + record.net_amount = 777; + let original = blob::encode(&record).unwrap(); + conn.execute( + "INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, ?3)", + params![ + wallet.as_slice(), + record.txid.as_byte_array().as_slice(), + &original + ], + ) + .unwrap(); + rewind_v021(&conn); + conn.create_scalar_function( + "migration_busy", + 0, + rusqlite::functions::FunctionFlags::SQLITE_UTF8, + |_| -> rusqlite::Result { + Err(rusqlite::Error::SqliteFailure( + rusqlite::ffi::Error::new(rusqlite::ffi::SQLITE_BUSY), + None, + )) + }, + ) + .unwrap(); + let error = conn + .query_row("SELECT migration_busy()", [], |_| Ok(())) + .unwrap_err(); + assert!(WalletStorageError::from(error).is_transient()); + conn.execute_batch("CREATE TEMP TRIGGER fail_accounting_repair BEFORE UPDATE OF record_blob ON core_transactions BEGIN SELECT migration_busy(); END;").unwrap(); + let result = migrations::run_for_open(&mut conn, LoadPolicy::Recovery); + assert!( + result.is_err(), + "Recovery must not mark a transiently incomplete backfill successful" + ); + let version: i64 = conn + .query_row( + "SELECT max(version) FROM refinery_schema_history", + [], + |r| r.get(0), + ) + .unwrap(); + assert_eq!(version, 20); + let tables: i64 = conn.query_row("SELECT count(*) FROM sqlite_master WHERE name IN ('core_transaction_inputs', 'core_transaction_record_originals')", [], |r| r.get(0)).unwrap(); + assert_eq!(tables, 0, "all V021 DDL and row writes roll back"); + let bytes: Vec = conn + .query_row("SELECT record_blob FROM core_transactions", [], |r| { + r.get(0) + }) + .unwrap(); + assert_eq!(bytes, original); + conn.execute_batch("DROP TRIGGER fail_accounting_repair") + .unwrap(); + migrations::run_for_open(&mut conn, LoadPolicy::Recovery).unwrap(); + let count: i64 = conn + .query_row("SELECT count(*) FROM core_transaction_inputs", [], |r| { + r.get(0) + }) + .unwrap(); + assert_eq!(count, record.transaction.input.len() as i64); + let repaired = core_state::get_tx_record( + &conn, + &wallet, + &record.txid, + &crate::sqlite::load_ctx::LoadCtx::strict(), + ) + .unwrap() + .unwrap(); + assert_eq!(repaired.net_amount, -1); + } + + #[test] + fn should_decode_all_immutable_v019_transaction_fixtures_with_live_codec() { + let mut count = 0; + for line in include_str!("../../../tests/fixtures/core_transaction_rows.txt").lines() { + let (name, bytes) = line.split_once(' ').unwrap(); + if name == "asset_lock_lifecycle" { + continue; + } + let bytes = hex::decode(bytes).unwrap(); + let record = core_state::decode_transaction_record(&bytes) + .unwrap_or_else(|error| panic!("{name}: {error}")); + assert_eq!(record.txid, record.transaction.txid(), "{name}"); + assert_eq!(blob::encode(&record).unwrap(), bytes, "{name}"); + assert!( + matches!(record.context, TransactionContext::InChainLockedBlock(_)), + "{name}" + ); + assert_eq!(record.block_info().unwrap().height(), 200, "{name}"); + let wallet = [0xC1; 32]; + let mut conn = wallet_db(&wallet); + conn.execute("INSERT INTO core_transactions (wallet_id, txid, height, finalized, record_blob) VALUES (?1, ?2, 200, 1, ?3)", params![wallet.as_slice(), record.txid.as_byte_array().as_slice(), bytes]).unwrap(); + rewind_v021(&conn); + migrations::run_for_open(&mut conn, LoadPolicy::Strict).unwrap(); + let remaining: (i64, i64) = conn.query_row("SELECT (SELECT count(*) FROM core_transactions), (SELECT count(*) FROM core_transaction_inputs)", [], |row| Ok((row.get(0)?, row.get(1)?))).unwrap(); + assert_eq!(remaining.0, 1, "{name}: mined history must remain"); + assert_eq!(remaining.1, record.transaction.input.len() as i64); + count += 1; + } + assert_eq!(count, 18); + } + + #[test] + fn should_decode_immutable_unmined_contexts_and_leaf_values_with_live_codec() { + for line in include_str!("../../../tests/fixtures/v019_kept_records.txt").lines() { + let fields: Vec<_> = line.split_whitespace().collect(); + let bytes = hex::decode(fields[3]).unwrap(); + let record = core_state::decode_transaction_record(&bytes).unwrap(); + assert!( + record.block_info().is_none(), + "height NULL must match the context" + ); + match (&record.context, fields[0]) { + (TransactionContext::Mempool, "mempool") => {} + (TransactionContext::InstantSend(lock), "instant_send") => { + assert_eq!(lock.txid, record.txid); + assert_eq!( + lock.inputs, + vec![record.transaction.input[0].previous_output] + ); + } + _ => panic!("wrong fixture context"), + } + assert_eq!(record.transaction.input.len(), 1); + assert_eq!( + record.transaction.input[0] + .previous_output + .txid + .to_byte_array(), + [0x42; 32] + ); + assert_eq!(record.transaction.input[0].previous_output.vout, 7); + assert_eq!( + record.input_details[0].address.to_string(), + "yMsgnH1xKGa85n4bq2imrZbG2KgrmGttAV" + ); + assert_eq!(record.input_details[0].value, 1000); + assert_eq!(record.transaction.input[0].script_sig.as_bytes(), &[0x51]); + assert_eq!( + record.transaction.input[0].witness.to_vec(), + vec![vec![1, 2, 3]] + ); + assert_eq!(record.transaction.output[0].value, 999); + assert_eq!(record.output_details[0].role, OutputRole::Change); + assert_eq!( + record.output_details[0].address.as_ref(), + Some(&record.input_details[0].address) + ); + assert_eq!( + record.account_type, + AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account + } + ); + assert_eq!(record.transaction_type, TransactionType::Standard); + assert_eq!(record.direction, TransactionDirection::Internal); + assert_eq!(record.net_amount, -1); + assert_eq!(record.fee, Some(1)); + assert_eq!(record.label, format!("V019 {} fixture", fields[0])); + assert_eq!(blob::encode(&record).unwrap(), bytes); + assert_eq!(hex::encode(record.txid.to_byte_array()), fields[1]); + assert_eq!( + hex::encode( + blob::encode_outpoint(&record.transaction.input[0].previous_output).unwrap() + ), + fields[2] + ); + } + } + + /// Both contexts retained by V021, with literal txids and non-null input keys. + #[test] + fn should_index_immutable_v019_record_bytes() { + for line in include_str!("../../../tests/fixtures/v019_kept_records.txt").lines() { + let fields: Vec<_> = line.split_whitespace().collect(); + let (name, txid, outpoint, bytes) = ( + fields[0], + hex::decode(fields[1]).unwrap(), + hex::decode(fields[2]).unwrap(), + hex::decode(fields[3]).unwrap(), + ); + let wallet = [0xB7; 32]; + let mut conn = wallet_db(&wallet); + conn.execute("INSERT INTO core_transactions (wallet_id, txid, height, finalized, record_blob) VALUES (?1, ?2, NULL, 0, ?3)", params![wallet.as_slice(), txid, bytes]).unwrap(); + rewind_v021(&conn); + migrations::run_for_open(&mut conn, LoadPolicy::Strict).unwrap(); + let indexed: (Vec, Vec) = conn + .query_row( + "SELECT txid, outpoint FROM core_transaction_inputs", + [], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .unwrap(); + assert_eq!(indexed, (txid, outpoint), "{name}"); + } + } + + #[test] + fn should_skip_oversized_v019_bodies_before_sqlite_materializes_them() { + for size in [ + blob::BLOB_SIZE_LIMIT_BYTES + 1, + SQLITE_MAX_BLOB_BYTES as usize + 1, + ] { + let wallet_id = [0xB1; 32]; + let mut conn = wallet_db(&wallet_id); + conn.execute( + "INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, zeroblob(?3))", + params![wallet_id.as_slice(), [0xB2u8; 32].as_slice(), size as i64], + ).unwrap(); + conn.set_limit(Limit::SQLITE_LIMIT_LENGTH, SQLITE_MAX_BLOB_BYTES) + .unwrap(); + { + let tx = conn.transaction().unwrap(); + let error = backfill_accounting(&tx, LoadPolicy::Strict).unwrap_err(); + assert!( + matches!(error, WalletStorageError::BlobTooLarge { .. }), + "{error:?}" + ); + backfill_accounting(&tx, LoadPolicy::Recovery) + .expect("Recovery skips oversized bodies"); + } + rewind_v021(&conn); + migrations::run_for_open(&mut conn, LoadPolicy::Recovery) + .expect("Recovery completes V021"); + let count: i64 = conn + .query_row("SELECT count(*) FROM core_transaction_inputs", [], |r| { + r.get(0) + }) + .unwrap(); + assert_eq!(count, 0); + } + } + + #[test] + fn should_classify_oversized_v019_identifiers_before_materializing_them() { + for column in ["wallet_id", "txid"] { + let wallet_id = [0xB3; 32]; + let mut conn = wallet_db(&wallet_id); + conn.execute( + "INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, X'00')", + params![wallet_id.as_slice(), [0xB4u8; 32].as_slice()], + ).unwrap(); + conn.execute_batch("PRAGMA foreign_keys = OFF").unwrap(); + conn.execute( + &format!("UPDATE core_transactions SET {column} = zeroblob(?1)"), + [SQLITE_MAX_BLOB_BYTES + 1], + ) + .unwrap(); + conn.set_limit(Limit::SQLITE_LIMIT_LENGTH, SQLITE_MAX_BLOB_BYTES) + .unwrap(); + let tx = conn.transaction().unwrap(); + let error = backfill_accounting(&tx, LoadPolicy::Strict).unwrap_err(); + assert!( + matches!(error, WalletStorageError::BlobTooLarge { .. }), + "{column}: {error:?}" + ); + backfill_accounting(&tx, LoadPolicy::Recovery) + .expect("Recovery skips invalid identifier rows"); + } + } + + #[test] + fn should_skip_wrong_width_v019_identifiers_only_under_recovery() { + for column in ["wallet_id", "txid"] { + for width in [0, 31, 33] { + let mut conn = wallet_db(&[0xBF; 32]); + conn.execute("INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, X'00')", params![[0xBFu8; 32].as_slice(), [0xC0u8; 32].as_slice()]).unwrap(); + conn.execute_batch("PRAGMA foreign_keys = OFF").unwrap(); + conn.execute( + &format!("UPDATE core_transactions SET {column} = zeroblob(?1)"), + [width], + ) + .unwrap(); + let tx = conn.transaction().unwrap(); + assert!(matches!( + backfill_accounting(&tx, LoadPolicy::Strict), + Err(WalletStorageError::BlobDecode { .. }) + )); + backfill_accounting(&tx, LoadPolicy::Recovery).unwrap(); + } + } + } + + #[test] + fn should_reject_text_v019_values_before_materializing_them() { + for column in ["wallet_id", "txid", "record_blob"] { + let mut conn = wallet_db(&[0xBD; 32]); + conn.execute("INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, X'00')", params![[0xBDu8; 32].as_slice(), [0xBEu8; 32].as_slice()]).unwrap(); + conn.execute_batch("PRAGMA foreign_keys = OFF").unwrap(); + // Under the character cap, over SQLite's byte cap. + let text = "🦀".repeat(SQLITE_MAX_BLOB_BYTES as usize / 4 + 1); + conn.execute( + &format!("UPDATE core_transactions SET {column} = ?1"), + [text], + ) + .unwrap(); + conn.set_limit(Limit::SQLITE_LIMIT_LENGTH, SQLITE_MAX_BLOB_BYTES) + .unwrap(); + let tx = conn.transaction().unwrap(); + let error = backfill_accounting(&tx, LoadPolicy::Strict).unwrap_err(); + assert!( + matches!(error, WalletStorageError::BlobDecode { .. }), + "{column}: {error:?}" + ); + backfill_accounting(&tx, LoadPolicy::Recovery).unwrap(); + } + } +} diff --git a/packages/rs-platform-wallet-storage/src/sqlite/persister.rs b/packages/rs-platform-wallet-storage/src/sqlite/persister.rs index fb4080cc63c..5e4d971170d 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/persister.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/persister.rs @@ -394,7 +394,7 @@ impl SqlitePersister { )?; } - let _report = crate::sqlite::migrations::run_for_open(&mut conn)?; + let _report = crate::sqlite::migrations::run_for_open(&mut conn, config.load_policy)?; // The open succeeded, so the claim passes to the persister, whose // `Drop` releases it. @@ -1686,6 +1686,21 @@ fn load_one_wallet( conn: &Connection, wallet_id: WalletId, ctx: &LoadCtx, +) -> Result { + let tx = rusqlite::Transaction::new_unchecked(conn, rusqlite::TransactionBehavior::Deferred) + .map_err(WalletStorageError::from) + .map_err(PersistenceError::from)?; + let state = load_wallet_snapshot(&tx, wallet_id, ctx)?; + tx.commit() + .map_err(WalletStorageError::from) + .map_err(PersistenceError::from)?; + Ok(state) +} + +fn load_wallet_snapshot( + conn: &Connection, + wallet_id: WalletId, + ctx: &LoadCtx, ) -> Result { let (network_str, birth_height) = schema::wallets::fetch(conn, &wallet_id) .map_err(PersistenceError::from)? @@ -2409,18 +2424,21 @@ mod tests { // Not rehydrated by `load()`, but read on demand by a production // entry point, so the state is reachable rather than abandoned. const READ_BY_A_DEDICATED_API: &[&str] = &[ - "dpns_name_states", // get_dpns_name_state - "meta_contact", // the kv object store - "meta_data_versions", // schema::versions - "meta_global", // the kv object store - "meta_identity", // the kv object store - "meta_platform_address", // the kv object store - "meta_store_generation", // schema::versions - "meta_token", // the kv object store - "meta_wallet", // the kv object store - "tracked_masternodes", // load_tracked_masternodes + "core_transaction_inputs", // core_history::apply + "dpns_name_states", // get_dpns_name_state + "meta_contact", // the kv object store + "meta_data_versions", // schema::versions + "meta_global", // the kv object store + "meta_identity", // the kv object store + "meta_platform_address", // the kv object store + "meta_store_generation", // schema::versions + "meta_token", // the kv object store + "meta_wallet", // the kv object store + "tracked_masternodes", // load_tracked_masternodes ]; const INFRASTRUCTURE: &[&str] = &["refinery_schema_history"]; + // First pre-repair bytes, retained for recovery. + const RETAINED_FOR_RECOVERY: &[&str] = &["core_transaction_record_originals"]; // `load()` rehydrates these only with the `shielded` feature on, so // the classification follows the build rather than claiming one. #[cfg(feature = "shielded")] @@ -2462,6 +2480,7 @@ mod tests { && !READ_BY_A_DEDICATED_API.contains(&table.as_str()) && !LOAD_UNIMPLEMENTED_TABLES.contains(&table.as_str()) && !INFRASTRUCTURE.contains(&table.as_str()) + && !RETAINED_FOR_RECOVERY.contains(&table.as_str()) && !FEATURE_GATED.contains(&table.as_str()) && !NOT_REHYDRATED_WITHOUT_FEATURE.contains(&table.as_str()) }) diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs index 7213558f3b4..d7ab594edcd 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/accounts.rs @@ -653,6 +653,10 @@ pub(crate) const ACCOUNT_TYPE_LABELS: &[&str] = &[ "platform_payment", ]; +/// Database label of `AccountType::DashpayExternalAccount`; SQL that +/// singles out contact watch-only rows binds this instead of a literal. +pub(crate) const DASHPAY_EXTERNAL_LABEL: &str = "dashpay_external"; + /// Stable database label for an `AccountType` variant (the `Debug` impl is not /// a stable format; this match is the contract). An added upstream variant /// fails this match's exhaustiveness check at compile time. @@ -715,7 +719,7 @@ pub(crate) fn account_type_db_label(at: &key_wallet::account::AccountType) -> &' AccountType::ProviderOperatorKeys => "provider_operator", AccountType::ProviderPlatformKeys => "provider_platform", AccountType::DashpayReceivingFunds { .. } => "dashpay_receiving", - AccountType::DashpayExternalAccount { .. } => "dashpay_external", + AccountType::DashpayExternalAccount { .. } => DASHPAY_EXTERNAL_LABEL, AccountType::PlatformPayment { .. } => "platform_payment", } } diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/core_history.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_history.rs new file mode 100644 index 00000000000..d3279b72e33 --- /dev/null +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_history.rs @@ -0,0 +1,714 @@ +//! Wallet accounting repaired from historical owned outputs, independent of live UTXOs. + +use std::collections::{BTreeMap, HashSet}; + +use dashcore::{Address, OutPoint, ScriptBuf, Txid}; +use key_wallet::managed_account::transaction_record::{ + InputDetail, OutputDetail, OutputRole, TransactionRecord, +}; +use key_wallet::transaction_checking::TransactionContext; +use platform_wallet::changeset::{is_owned, wallet_accounting, CoreChangeSet}; +use platform_wallet::wallet::platform_wallet::WalletId; +use rusqlite::{params, Connection, OptionalExtension, Transaction}; + +use super::accounts::DASHPAY_EXTERNAL_LABEL; +use super::{blob, core_state, wallets}; +use crate::sqlite::error::WalletStorageError; +use crate::sqlite::load_ctx::LoadCtx; +use crate::sqlite::util::safe_cast::i64_to_u64; + +/// Preserve proven ownership when a partial account snapshot replaces the wallet record. +pub(super) fn preserve_known_details( + tx: &Transaction<'_>, + wallet_id: &WalletId, + incoming: &TransactionRecord, +) -> Result { + let mut merged = incoming.clone(); + let Some(previous) = prior_record(tx, wallet_id, &incoming.txid)? else { + return Ok(merged); + }; + // Compare only txid-committed content: a peer may attach BIP144 witnesses + // to a known txid, and that must neither conflict nor replace the stored body. + if previous.transaction.txid() != incoming.transaction.txid() { + return Err(WalletStorageError::TransactionBodyConflict { + wallet_id: *wallet_id, + txid: incoming.txid, + }); + } + merged.transaction = previous.transaction; + let mut inputs: BTreeMap<_, _> = previous + .input_details + .into_iter() + .map(|d| (d.index, d)) + .collect(); + for detail in &incoming.input_details { + inputs.insert(detail.index, detail.clone()); + } + let mut outputs: BTreeMap<_, _> = previous + .output_details + .into_iter() + .map(|d| (d.index, d)) + .collect(); + for detail in &incoming.output_details { + let keep_previous = outputs + .get(&detail.index) + .is_some_and(|old| is_owned(old.role)) + && !is_owned(detail.role); + if !keep_previous { + outputs.insert(detail.index, detail.clone()); + } + } + merged.input_details = inputs.into_values().collect(); + merged.output_details = outputs.into_values().collect(); + Ok(merged) +} + +/// Read a stored record strictly: corrupt history is an error, never skipped. +fn prior_record( + conn: &Connection, + wallet_id: &WalletId, + txid: &Txid, +) -> Result, WalletStorageError> { + core_state::get_tx_record(conn, wallet_id, txid, &LoadCtx::strict()) +} + +/// Index raw inputs independently of when their ownership becomes known. +pub(crate) fn index_record( + tx: &Transaction<'_>, + wallet_id: &WalletId, + record: &TransactionRecord, +) -> Result<(), WalletStorageError> { + let mut stmt = tx.prepare_cached( + "INSERT OR IGNORE INTO core_transaction_inputs (wallet_id, txid, outpoint) \ + VALUES (?1, ?2, ?3)", + )?; + for input in &record.transaction.input { + stmt.execute(params![ + wallet_id.as_slice(), + record.txid.as_byte_array().as_slice(), + blob::encode_outpoint(&input.previous_output)?, + ])?; + } + Ok(()) +} + +/// Repair changed records and consumers of newly materialized historical outputs. +pub(crate) fn apply( + tx: &Transaction<'_>, + wallet_id: &WalletId, + cs: &CoreChangeSet, +) -> Result<(), WalletStorageError> { + let mut affected: HashSet = cs.records.iter().map(|r| r.txid).collect(); + let mut consumers = tx.prepare_cached( + "SELECT txid FROM core_transaction_inputs WHERE wallet_id = ?1 AND outpoint = ?2", + )?; + for utxo in cs.new_utxos.iter().chain(&cs.spent_utxos) { + affected.insert(utxo.outpoint.txid); + let mut rows = consumers.query(params![ + wallet_id.as_slice(), + blob::encode_outpoint(&utxo.outpoint)? + ])?; + while let Some(row) = rows.next()? { + let bytes: Vec = row.get(0)?; + affected.insert(Txid::from_slice(&bytes)?); + } + } + if affected.is_empty() { + return Ok(()); + } + let network = network(tx, wallet_id)?; + for txid in affected { + repair_record(tx, wallet_id, &txid, network)?; + } + Ok(()) +} + +fn network( + tx: &Transaction<'_>, + wallet_id: &WalletId, +) -> Result { + let label: String = tx.query_row( + "SELECT network FROM wallets WHERE wallet_id = ?1", + params![wallet_id.as_slice()], + |r| r.get(0), + )?; + wallets::parse_network(&label).ok_or_else(|| WalletStorageError::UnknownWalletNetwork { + wallet_id: *wallet_id, + label, + }) +} + +fn owned_output( + tx: &Transaction<'_>, + wallet_id: &WalletId, + outpoint: &OutPoint, + network: dashcore::Network, +) -> Result, WalletStorageError> { + let encoded = blob::encode_outpoint(outpoint)?; + let key = params![wallet_id.as_slice(), encoded]; + // Gate the script length in its own statement: SQLite evaluates every + // result column before a row is returned. + let Some((value, len)) = tx + .prepare_cached( + "SELECT value, length(script) FROM core_utxos \ + WHERE wallet_id = ?1 AND outpoint = ?2 AND is_sweep_placeholder = 0", + )? + .query_row(key, |row| { + Ok((row.get::<_, i64>(0)?, row.get::<_, i64>(1)?)) + }) + .optional()? + else { + return Ok(None); + }; + let value = i64_to_u64("core_utxos.value", value)?; + blob::check_size(len)?; + let script: Vec = tx + .prepare_cached( + "SELECT script FROM core_utxos \ + WHERE wallet_id = ?1 AND outpoint = ?2 AND is_sweep_placeholder = 0", + )? + .query_row(key, |row| row.get(0))?; + if contact_only_script(tx, wallet_id, &script)? { + return Ok(None); + } + let address = Address::from_script(&ScriptBuf::from_bytes(script), network)?; + Ok(Some((value, address))) +} + +/// Whether `script` is tracked only by a contact's watch-only (DashPay external) chain. +pub(crate) fn contact_only_script( + conn: &Connection, + wallet_id: &WalletId, + script: &[u8], +) -> Result { + let mut probe = conn.prepare_cached( + "SELECT EXISTS(SELECT 1 FROM core_address_pool WHERE wallet_id = ?1 AND script = ?2) \ + AND NOT EXISTS(SELECT 1 FROM core_address_pool \ + WHERE wallet_id = ?1 AND script = ?2 AND account_type != ?3)", + )?; + let key = params![wallet_id.as_slice(), script, DASHPAY_EXTERNAL_LABEL]; + Ok(probe.query_row(key, |r| r.get(0))?) +} + +fn repair_record( + tx: &Transaction<'_>, + wallet_id: &WalletId, + txid: &Txid, + network: dashcore::Network, +) -> Result<(), WalletStorageError> { + let Some(mut record) = prior_record(tx, wallet_id, txid)? else { + return Ok(()); + }; + let original = blob::encode(&record)?; + let mut inputs = BTreeMap::new(); + for detail in record.input_details.drain(..) { + if !contact_only_script(tx, wallet_id, detail.address.script_pubkey().as_bytes())? { + inputs.insert(detail.index, detail); + } + } + for (index, input) in record.transaction.input.iter().enumerate() { + if let Some((value, address)) = + owned_output(tx, wallet_id, &input.previous_output, network)? + { + inputs.insert( + index as u32, + InputDetail { + index: index as u32, + value, + address, + }, + ); + } + } + let mut outputs = BTreeMap::new(); + for mut detail in record.output_details.drain(..) { + if let Some(address) = &detail.address { + if contact_only_script(tx, wallet_id, address.script_pubkey().as_bytes())? { + detail.role = OutputRole::Sent; + } + } + outputs.insert(detail.index, detail); + } + for (index, output) in record.transaction.output.iter().enumerate() { + let index = index as u32; + if let Some((_, address)) = owned_output( + tx, + wallet_id, + &OutPoint { + txid: *txid, + vout: index, + }, + network, + )? { + let role = outputs.get(&index).map_or(OutputRole::Received, |d| { + if d.role == OutputRole::Change { + OutputRole::Change + } else { + OutputRole::Received + } + }); + outputs.insert( + index, + OutputDetail { + index, + role, + address: Some(address), + value: output.value, + }, + ); + } + } + // Empty metadata is not accounting evidence (e.g. confirmation-only placeholders). + if inputs.is_empty() && outputs.is_empty() { + return Ok(()); + } + record.input_details = inputs.into_values().collect(); + record.output_details = outputs.into_values().collect(); + // The live projection computes the same accounting, so repair never + // flips a row it just wrote; only the overflow policy differs. + let (net, direction) = wallet_accounting(&record); + record.net_amount = match i64::try_from(net) { + Ok(net) => net, + // Nothing authenticates a mempool transaction: this is peer-supplied data, + // not corruption. Keep its stored accounting until representable evidence arrives. + Err(_) if matches!(record.context, TransactionContext::Mempool) => return Ok(()), + Err(_) => { + return Err(WalletStorageError::NetAmountOverflow { + wallet_id: *wallet_id, + txid: *txid, + value: net, + }) + } + }; + record.direction = direction; + let repaired = blob::encode(&record)?; + if repaired != original { + // Append-only: the first pre-repair blob is kept verbatim and never + // replaced, so a wrong repair can always be undone. + tx.execute( + "INSERT OR IGNORE INTO core_transaction_record_originals (wallet_id, txid, record_blob) \ + SELECT wallet_id, txid, record_blob FROM core_transactions \ + WHERE wallet_id = ?1 AND txid = ?2", + params![wallet_id.as_slice(), txid.as_byte_array().as_slice()], + )?; + tx.execute( + "UPDATE core_transactions SET record_blob = ?1 WHERE wallet_id = ?2 AND txid = ?3", + params![ + repaired, + wallet_id.as_slice(), + txid.as_byte_array().as_slice() + ], + )?; + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use dashcore::TxOut; + use key_wallet::account::{AccountType, StandardAccountType}; + use key_wallet::managed_account::transaction_record::TransactionDirection; + use key_wallet::transaction_checking::{BlockInfo, TransactionType}; + use platform_wallet::changeset::wallet_direction; + use platform_wallet::test_support::core_fixtures::{ + foreign_address as address, outside, pay, spend_of, + }; + use platform_wallet::test_support::fold_wallet_records; + + use super::*; + use crate::sqlite::migrations::oversize_script_db; + + const WALLET_ID: WalletId = [0x5Au8; 32]; + + #[test] + fn should_reject_oversize_owned_output_script_before_reading_it() { + const WALLET: WalletId = WALLET_ID; + let (mut conn, outpoint) = oversize_script_db(&WALLET); + let tx = conn.transaction().unwrap(); + + let err = owned_output(&tx, &WALLET, &outpoint, dashcore::Network::Testnet).unwrap_err(); + + assert!( + matches!(err, WalletStorageError::BlobTooLarge { .. }), + "got {err:?}" + ); + } + + fn wallet_db(network: &str) -> Connection { + let mut conn = Connection::open_in_memory().unwrap(); + crate::sqlite::migrations::run(&mut conn).unwrap(); + // The schema CHECK rejects unknown labels; a corrupt or newer file may still carry one. + conn.pragma_update(None, "ignore_check_constraints", true) + .unwrap(); + conn.execute( + "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, ?2, 0)", + params![&WALLET_ID[..], network], + ) + .unwrap(); + conn + } + + fn record(output_values: &[u64], received: &[u64]) -> TransactionRecord { + let owner = address(7); + let body = spend_of(&[], output_values.iter().map(|&v| pay(v, &owner)).collect()); + let details = received + .iter() + .enumerate() + .map(|(index, &value)| OutputDetail { + index: index as u32, + role: OutputRole::Received, + address: None, + value, + }) + .collect(); + TransactionRecord::new( + body, + AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + }, + TransactionContext::Mempool, + TransactionType::Standard, + TransactionDirection::Incoming, + Vec::new(), + details, + 0, + ) + } + + fn store(conn: &Connection, record: &TransactionRecord) { + conn.execute( + "INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) \ + VALUES (?1, ?2, 0, ?3)", + params![ + &WALLET_ID[..], + record.txid.as_byte_array().as_slice(), + blob::encode(record).unwrap() + ], + ) + .unwrap(); + } + + #[test] + fn should_report_a_body_conflict_for_the_same_txid_with_another_body() { + let mut conn = wallet_db("testnet"); + let stored = record(&[1_000], &[]); + store(&conn, &stored); + let mut incoming = record(&[2_000], &[]); + incoming.txid = stored.txid; + + let tx = conn.transaction().unwrap(); + let err = preserve_known_details(&tx, &WALLET_ID, &incoming).unwrap_err(); + + assert!( + matches!( + err, + WalletStorageError::TransactionBodyConflict { wallet_id, txid } + if wallet_id == WALLET_ID && txid == stored.txid + ), + "got {err:?}" + ); + } + + /// A peer can serve a known txid with BIP144 witnesses attached; the txid + /// does not commit to them, so the stored body stands and the flush goes on. + #[test] + fn should_keep_the_stored_body_when_a_same_txid_body_differs_only_in_witness() { + let mut conn = wallet_db("testnet"); + let mut stored = record(&[1_000], &[]); + stored.transaction.input.push(dashcore::TxIn::default()); + stored.txid = stored.transaction.txid(); + store(&conn, &stored); + let mut incoming = stored.clone(); + incoming.transaction.input[0].witness = dashcore::Witness::from_slice(&[[0xAB]]); + assert_eq!(incoming.transaction.txid(), stored.txid); + assert_ne!(incoming.transaction, stored.transaction); + + let tx = conn.transaction().unwrap(); + let merged = preserve_known_details(&tx, &WALLET_ID, &incoming).unwrap(); + + assert_eq!(merged.transaction, stored.transaction); + } + + #[test] + fn should_report_an_unknown_wallet_network_label() { + let mut conn = wallet_db("moonnet"); + let tx = conn.transaction().unwrap(); + + let err = network(&tx, &WALLET_ID).unwrap_err(); + + assert!( + matches!( + &err, + WalletStorageError::UnknownWalletNetwork { wallet_id, label } + if *wallet_id == WALLET_ID && label == "moonnet" + ), + "got {err:?}" + ); + } + + #[test] + fn should_store_mempool_record_with_unrepresentable_net_without_failing_the_round() { + let mut conn = wallet_db("testnet"); + let value = (1 << 62) + 1; + let mut live = record(&[value, value], &[value, value]); + // The live projection saturates a net it cannot represent. + live.net_amount = i64::MAX; + let tx = conn.transaction().unwrap(); + + core_state::apply( + &tx, + &WALLET_ID, + &CoreChangeSet { + records: vec![live.clone()], + ..Default::default() + }, + ) + .expect("a peer-supplied mempool amount must not fail the round"); + + let stored: Vec = tx + .query_row( + "SELECT record_blob FROM core_transactions WHERE wallet_id = ?1 AND txid = ?2", + params![&WALLET_ID[..], live.txid.as_byte_array().as_slice()], + |r| r.get(0), + ) + .unwrap(); + assert_eq!(stored, blob::encode(&live).unwrap()); + } + + #[test] + fn should_report_a_confirmed_net_amount_that_does_not_fit_i64() { + let mut conn = wallet_db("testnet"); + let mut stored = record(&[u64::MAX, u64::MAX], &[u64::MAX, u64::MAX]); + stored.context = + TransactionContext::InBlock(BlockInfo::new(101, dashcore::BlockHash::all_zeros(), 7)); + store(&conn, &stored); + conn.execute("UPDATE core_transactions SET height = 101", []) + .unwrap(); + let tx = conn.transaction().unwrap(); + + let err = + repair_record(&tx, &WALLET_ID, &stored.txid, dashcore::Network::Testnet).unwrap_err(); + + assert!( + matches!( + err, + WalletStorageError::NetAmountOverflow { wallet_id, txid, value } + if wallet_id == WALLET_ID + && txid == stored.txid + && value == 2 * i128::from(u64::MAX) + ), + "got {err:?}" + ); + } + + /// Shared with the Swift SDK's `TransactionAccountingTests` direction + /// table; pins the rule repair takes from `platform_wallet`. + #[test] + fn should_classify_repaired_direction_like_the_swift_sdk() { + use TransactionDirection::{CoinJoin, Incoming, Internal, Outgoing}; + use TransactionType::{AssetLock, Standard}; + // (type, spends ours, has owned output, has external output, expected) + let cases = [ + (Standard, true, true, false, Internal), + (Standard, true, true, true, Outgoing), + (Standard, true, false, false, Outgoing), + (AssetLock, true, false, false, Internal), + (AssetLock, true, true, false, Internal), + (AssetLock, true, false, true, Outgoing), + (Standard, false, true, false, Incoming), + (TransactionType::CoinJoin, true, true, false, CoinJoin), + ]; + for (kind, spends_ours, has_ours, has_external, expected) in cases { + assert_eq!( + wallet_direction(kind, spends_ours, has_ours, has_external), + expected, + "{kind:?} spends_ours={spends_ours} has_ours={has_ours} has_external={has_external}" + ); + } + } + + const FUNDING: u64 = 100_000_000; + const LOCK_CREDIT: u64 = 60_000_000; + const LOCK_FEE: u64 = 1_000; + + fn bip44() -> AccountType { + AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + } + } + + /// The funding account's record upstream emits for a spend of one + /// wallet coin worth [`FUNDING`], classified the way upstream + /// `record_transaction` does. `outputs` are `(script, value, role)`. + fn funding_slice( + kind: TransactionType, + outputs: &[(ScriptBuf, u64, OutputRole)], + ) -> TransactionRecord { + let outputs_of = outputs.iter().map(|(script, value, _)| TxOut { + value: *value, + script_pubkey: script.clone(), + }); + let mut body = spend_of(&[outside(0x11)], outputs_of.collect()); + body.version = 3; + let details: Vec = outputs + .iter() + .enumerate() + .map(|(index, (script, value, role))| OutputDetail { + index: index as u32, + role: *role, + address: Address::from_script(script, dashcore::Network::Testnet).ok(), + value: *value, + }) + .collect(); + let owned: u64 = details + .iter() + .filter(|d| is_owned(d.role)) + .map(|d| d.value) + .sum(); + let has_sent = details.iter().any(|d| d.role == OutputRole::Sent); + let direction = if !has_sent && owned > 0 { + TransactionDirection::Internal + } else { + TransactionDirection::Outgoing + }; + TransactionRecord::new( + body, + bip44(), + TransactionContext::Mempool, + kind, + direction, + vec![InputDetail { + index: 0, + value: FUNDING, + address: address(1), + }], + details, + owned as i64 - FUNDING as i64, + ) + } + + /// The keys account's thin marker for an asset lock: `Internal`, no + /// details, net `+credit` (the OP_RETURN output's value). + fn keys_marker(funding: &TransactionRecord) -> TransactionRecord { + let credit = funding.transaction.output[0].value; + TransactionRecord::new( + funding.transaction.clone(), + AccountType::AssetLockAddressTopUp, + TransactionContext::Mempool, + TransactionType::AssetLock, + TransactionDirection::Internal, + Vec::new(), + Vec::new(), + credit as i64, + ) + } + + /// The live projection and the SQLite repair must agree on a row's + /// net and direction, or the row flips each time storage repairs + /// what the live path just wrote. Covers the asset-lock shapes (no + /// change, change, paying an external output) and a plain + /// cross-account transfer. + #[test] + fn should_repair_live_folded_records_without_changing_their_accounting() { + let burn = || ScriptBuf::new_op_return(&[]); + let change = FUNDING - LOCK_CREDIT - LOCK_FEE; + // With no change, everything but the fee is burned into credits. + let lock_no_change = funding_slice( + TransactionType::AssetLock, + &[(burn(), FUNDING - LOCK_FEE, OutputRole::Unspendable)], + ); + let lock_change = funding_slice( + TransactionType::AssetLock, + &[ + (burn(), LOCK_CREDIT, OutputRole::Unspendable), + (address(2).script_pubkey(), change, OutputRole::Change), + ], + ); + let lock_external = funding_slice( + TransactionType::AssetLock, + &[ + (burn(), LOCK_CREDIT, OutputRole::Unspendable), + ( + address(2).script_pubkey(), + change - 5_000, + OutputRole::Change, + ), + (address(9).script_pubkey(), 5_000, OutputRole::Sent), + ], + ); + // Output 0 lands on a second account of the same wallet, which the + // funding account's local view can only call `Sent`. + let transfer = funding_slice( + TransactionType::Standard, + &[( + address(3).script_pubkey(), + FUNDING - LOCK_FEE, + OutputRole::Sent, + )], + ); + let mut transfer_receiver = transfer.clone(); + transfer_receiver.account_type = AccountType::Standard { + index: 1, + standard_account_type: StandardAccountType::BIP44Account, + }; + transfer_receiver.direction = TransactionDirection::Incoming; + transfer_receiver.input_details.clear(); + transfer_receiver.output_details[0].role = OutputRole::Received; + transfer_receiver.net_amount = (FUNDING - LOCK_FEE) as i64; + + let lock_net = -((LOCK_CREDIT + LOCK_FEE) as i64); + let cases = [ + ( + "asset lock, no change", + vec![lock_no_change.clone(), keys_marker(&lock_no_change)], + TransactionDirection::Internal, + -(FUNDING as i64), + ), + ( + "asset lock, change", + vec![lock_change.clone(), keys_marker(&lock_change)], + TransactionDirection::Internal, + lock_net, + ), + ( + "asset lock, no change, funding slice alone", + vec![lock_no_change.clone()], + TransactionDirection::Internal, + -(FUNDING as i64), + ), + ( + "asset lock paying an external output", + vec![lock_external.clone(), keys_marker(&lock_external)], + TransactionDirection::Outgoing, + lock_net - 5_000, + ), + ( + "cross-account transfer", + vec![transfer, transfer_receiver], + TransactionDirection::Internal, + -(LOCK_FEE as i64), + ), + ]; + for (name, mut records, direction, net) in cases { + fold_wallet_records(&mut records); + assert_eq!(records.len(), 1, "{name}"); + let live = &records[0]; + assert_eq!(live.direction, direction, "{name}: live direction"); + assert_eq!(live.net_amount, net, "{name}: live net"); + + let mut conn = wallet_db("testnet"); + store(&conn, live); + let tx = conn.transaction().unwrap(); + repair_record(&tx, &WALLET_ID, &live.txid, dashcore::Network::Testnet).unwrap(); + let repaired = prior_record(&tx, &WALLET_ID, &live.txid).unwrap().unwrap(); + + assert_eq!( + repaired.direction, live.direction, + "{name}: repaired direction" + ); + assert_eq!(repaired.net_amount, live.net_amount, "{name}: repaired net"); + } + } +} diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs index cc103ad5570..63ce0e420af 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs @@ -7,6 +7,7 @@ use std::collections::{HashMap, HashSet}; use rusqlite::{params, Connection, OptionalExtension, Transaction}; use dashcore::ephemerealdata::chain_lock::ChainLock; +use dashcore::transaction::special_transaction::TransactionPayload; use key_wallet::managed_account::transaction_record::TransactionRecord; use key_wallet::transaction_checking::TransactionContext; use key_wallet::Utxo; @@ -15,9 +16,9 @@ use platform_wallet::wallet::platform_wallet::WalletId; use crate::sqlite::error::WalletStorageError; use crate::sqlite::load_ctx::{LoadCtx, LoadSite}; -use crate::sqlite::schema::blob; use crate::sqlite::schema::blob::impl_persistable_blob; use crate::sqlite::schema::core_pool::{owning_account_for_script, OwningAccount}; +use crate::sqlite::schema::{blob, core_history}; // PUBLIC material only: core-chain state reaching `record_blob` / // `islock_blob` (transaction records + InstantLocks are public chain data). @@ -92,13 +93,15 @@ pub fn apply( finalized = excluded.finalized, \ record_blob = excluded.record_blob", )?; - for record in &cs.records { + for incoming in &cs.records { + ensure_current_transaction_identity(incoming)?; + let record = core_history::preserve_known_details(tx, wallet_id, incoming)?; let block_info = record.block_info(); let height = block_info.map(|b| i64::from(b.height())); let block_hash = block_info.map(|b| AsRef::<[u8]>::as_ref(&b.block_hash()).to_vec()); let block_time = block_info.map(|b| i64::from(b.timestamp())); let finalized = block_info.is_some(); - let payload = blob::encode(record)?; + let payload = blob::encode(&record)?; stmt.execute(params![ wallet_id.as_slice(), AsRef::<[u8]>::as_ref(&record.txid), @@ -108,6 +111,7 @@ pub fn apply( finalized, payload, ])?; + core_history::index_record(tx, wallet_id, &record)?; } } // `addresses_derived` is intentionally NOT persisted here — the pool @@ -227,6 +231,7 @@ pub fn apply( if heights_advanced { collect_finalized_tombstones(tx, wallet_id)?; } + core_history::apply(tx, wallet_id, cs)?; return Ok(()); } @@ -390,6 +395,7 @@ pub fn apply( if heights_advanced { collect_finalized_tombstones(tx, wallet_id)?; } + core_history::apply(tx, wallet_id, cs)?; Ok(()) } @@ -1253,7 +1259,7 @@ pub fn get_tx_record( }; blob::check_size(record_blob_len)?; let payload: Vec = row.get(2)?; - let record = blob::decode(&payload)?; + let record = decode_transaction_record(&payload)?; drop(rows); drop(stmt); if let Err(mismatch) = ensure_transaction_record_matches_columns(txid, height, &record) { @@ -1271,6 +1277,39 @@ pub fn get_tx_record( Ok(Some(record)) } +/// Decode historical records without changing their stored identifiers or index keys. +pub(crate) fn decode_transaction_record( + payload: &[u8], +) -> Result { + let record: TransactionRecord = blob::decode(payload)?; + if let Err(error) = ensure_current_transaction_identity(&record) { + // Before stable AssetUnlock txids, the writer committed the full quorum + // information. Only that historical payload shape may use an instance hash. + let historical_unlock = (record.transaction.version as i16) >= 3 + && matches!( + &record.transaction.special_transaction_payload, + Some(TransactionPayload::AssetUnlockPayloadType(payload)) if payload.base.version >= 2 + ) + && record.txid == record.transaction.instance_hash(); + if !historical_unlock { + return Err(error); + } + } + Ok(record) +} + +/// New writes use the current canonical identifier; historical aliases are read-only. +fn ensure_current_transaction_identity( + record: &TransactionRecord, +) -> Result<(), WalletStorageError> { + if record.txid != record.transaction.txid() { + return Err(WalletStorageError::blob_decode( + "transaction record txid disagrees with its transaction body hash", + )); + } + Ok(()) +} + /// Diagnose typed txid/height columns that disagree with the authoritative blob. fn ensure_transaction_record_matches_columns( typed_txid: &dashcore::Txid, @@ -1339,6 +1378,8 @@ pub fn list_unspent_utxos( #[cfg(test)] mod tests { use super::*; + use crate::sqlite::config::LoadPolicy; + use crate::sqlite::migrations::{self, insert_test_wallet, rewind_v021, wallet_db}; use dashcore::address::Payload; use dashcore::{BlockHash, OutPoint, PubkeyHash, Transaction, TxOut, Txid}; use key_wallet::account::{AccountType, StandardAccountType}; @@ -1347,8 +1388,175 @@ mod tests { }; use key_wallet::transaction_checking::{BlockInfo, TransactionContext, TransactionType}; - fn transaction_record(txid: Txid, context: TransactionContext) -> TransactionRecord { - let mut record = TransactionRecord::new( + // Encoded by TransactionRecord::new at rust-dashcore d631aaca3632cc713143476e363c9a9aca038d2e, + // using grovedb-bincode 2.1.0 serde standard config. Public mempool records, not finality proofs. + // Unlock: version 2, index 71, fee 1000, request height 200, quorum hash [0x31;32], sig [0x42;96]. + // The second record spends its historical outpoint. Never regenerate these with the live writer. + fn historical_asset_unlock_rows() -> Vec<(Txid, Vec, TransactionRecord)> { + include_str!("../../../tests/fixtures/v019_asset_unlock_v2.txt") + .lines() + .map(|line| { + let fields: Vec<_> = line.split_whitespace().collect(); + let txid = Txid::from_slice(&hex::decode(fields[1]).unwrap()).unwrap(); + let bytes = hex::decode(fields[2]).unwrap(); + let record: TransactionRecord = blob::decode(&bytes).unwrap(); + assert_eq!(record.txid, txid); + assert_eq!(blob::encode(&record).unwrap(), bytes); + (txid, bytes, record) + }) + .collect() + } + + #[test] + fn should_migrate_and_read_historical_asset_unlock_without_rekeying() { + let wallet = [0x74; 32]; + let mut conn = wallet_db(&wallet); + let rows = historical_asset_unlock_rows(); + let (legacy, _, withdrawal) = &rows[0]; + let canonical = withdrawal.transaction.txid(); + assert_eq!( + hex::encode(canonical.as_byte_array()), + "8df73be32e0ac0dea8619c57f212076f88e528563d1505e8316a2757abe32333" + ); + assert_ne!(*legacy, canonical); + assert_eq!(*legacy, withdrawal.transaction.instance_hash()); + for (txid, bytes, _) in &rows { + conn.execute("INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, ?3)", + params![wallet.as_slice(), txid.as_byte_array().as_slice(), bytes]).unwrap(); + } + rewind_v021(&conn); + migrations::run_for_open(&mut conn, LoadPolicy::Strict).unwrap(); + for (txid, bytes, original) in &rows { + let loaded = get_tx_record(&conn, &wallet, txid, &LoadCtx::strict()) + .unwrap() + .unwrap(); + assert_eq!(loaded.txid, *txid); + assert_eq!(loaded.transaction, original.transaction); + let stored: Vec = conn + .query_row( + "SELECT record_blob FROM core_transactions WHERE wallet_id = ?1 AND txid = ?2", + params![wallet.as_slice(), txid.as_byte_array().as_slice()], + |r| r.get(0), + ) + .unwrap(); + assert_eq!( + &stored, bytes, + "unchanged accounting keeps the exact historical bytes" + ); + } + assert!( + get_tx_record(&conn, &wallet, &canonical, &LoadCtx::strict()) + .unwrap() + .is_none() + ); + let (state, _) = load_state( + &conn, + &wallet, + dashcore::Network::Testnet, + &LoadCtx::strict(), + ) + .unwrap(); + assert!(state.records.iter().any(|r| r.txid == *legacy)); + let indexed: (Vec, Vec) = conn + .query_row( + "SELECT txid, outpoint FROM core_transaction_inputs", + [], + |r| Ok((r.get(0)?, r.get(1)?)), + ) + .unwrap(); + assert_eq!(indexed.0, rows[1].0.as_byte_array()); + assert_eq!( + blob::decode_outpoint(&indexed.1).unwrap(), + OutPoint::new(*legacy, 0) + ); + } + + #[test] + fn should_reject_tampering_with_historical_asset_unlock_identity() { + let original = historical_asset_unlock_rows().remove(0).2; + for mutation in [ + "output", + "quorum", + "version", + "type", + "record id", + "row key", + ] { + let wallet = [0x75; 32]; + let mut conn = wallet_db(&wallet); + let mut record = original.clone(); + let mut key = original.txid; + match mutation { + "output" => record.transaction.output[0].value += 1, + "quorum" => { + let Some(TransactionPayload::AssetUnlockPayloadType(payload)) = + &mut record.transaction.special_transaction_payload + else { + panic!("fixture payload") + }; + payload.quorum_sig = [0x43; 96].into(); + } + "version" => { + let Some(TransactionPayload::AssetUnlockPayloadType(payload)) = + &mut record.transaction.special_transaction_payload + else { + panic!("fixture payload") + }; + payload.base.version = 1; + } + "type" => record.transaction.special_transaction_payload = None, + "record id" => record.txid = Txid::from_byte_array([0x76; 32]), + "row key" => key = Txid::from_byte_array([0x76; 32]), + _ => unreachable!(), + } + conn.execute("INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) VALUES (?1, ?2, 0, ?3)", + params![wallet.as_slice(), key.as_byte_array().as_slice(), blob::encode(&record).unwrap()]).unwrap(); + assert!( + get_tx_record(&conn, &wallet, &key, &LoadCtx::strict()).is_err(), + "{mutation}" + ); + rewind_v021(&conn); + assert!( + migrations::run_for_open(&mut conn, LoadPolicy::Strict).is_err(), + "{mutation}" + ); + let version: i64 = conn + .query_row( + "SELECT max(version) FROM refinery_schema_history", + [], + |r| r.get(0), + ) + .unwrap(); + assert_eq!(version, 20, "{mutation}: failed migration is atomic"); + } + } + + #[test] + fn should_require_canonical_asset_unlock_identity_for_current_writes() { + let wallet = [0x77; 32]; + let mut conn = wallet_db(&wallet); + let historical = historical_asset_unlock_rows().remove(0).2; + { + let tx = conn.transaction().unwrap(); + assert!( + apply(&tx, &wallet, &round(vec![historical.clone()], vec![])).is_err(), + "legacy aliases are accepted only on historical reads" + ); + } + let mut canonical = historical; + canonical.txid = canonical.transaction.txid(); + let tx = conn.transaction().unwrap(); + apply(&tx, &wallet, &round(vec![canonical.clone()], vec![])).unwrap(); + tx.commit().unwrap(); + let loaded = get_tx_record(&conn, &wallet, &canonical.txid, &LoadCtx::strict()) + .unwrap() + .unwrap(); + assert_eq!(loaded.txid, canonical.txid); + assert_eq!(loaded.transaction, canonical.transaction); + } + + fn transaction_record(context: TransactionContext) -> TransactionRecord { + TransactionRecord::new( Transaction { version: 3, lock_time: 0, @@ -1366,9 +1574,7 @@ mod tests { Vec::new(), Vec::new(), 100, - ); - record.txid = txid; - record + ) } fn sample_utxo(txid: Txid, height: u32, is_confirmed: bool) -> Utxo { @@ -1392,6 +1598,453 @@ mod tests { } } + /// A record spending `coins` in `context`, with no details and a zero net. + fn spend(coins: &[OutPoint], context: TransactionContext) -> TransactionRecord { + let mut record = transaction_record(context); + record.transaction.input = coins + .iter() + .map(|&previous_output| dashcore::TxIn { + previous_output, + ..Default::default() + }) + .collect(); + record.txid = record.transaction.txid(); + record.net_amount = 0; + record + } + + fn round(records: Vec, new_utxos: Vec) -> CoreChangeSet { + CoreChangeSet { + records, + new_utxos, + ..Default::default() + } + } + + fn stored_record(conn: &Connection, wallet_id: &[u8; 32], txid: &Txid) -> TransactionRecord { + get_tx_record(conn, wallet_id, txid, &LoadCtx::strict()) + .unwrap() + .unwrap() + } + + fn original_blob(conn: &Connection, wallet_id: &[u8; 32], txid: &Txid) -> Vec { + conn.query_row( + "SELECT record_blob FROM core_transaction_record_originals \ + WHERE wallet_id = ?1 AND txid = ?2", + params![&wallet_id[..], AsRef::<[u8]>::as_ref(txid)], + |row| row.get(0), + ) + .unwrap() + } + + #[test] + fn should_repair_accounting_without_changing_spend_claims() { + let wallet_id = [0xAC; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x32; 32]), 100, true); + let context = TransactionContext::InBlock(BlockInfo::new(101, BlockHash::all_zeros(), 123)); + let spending = spend(&[funding.outpoint], context); + let tx = conn.transaction().unwrap(); + apply( + &tx, + &wallet_id, + &round(vec![spending.clone()], vec![funding.clone()]), + ) + .unwrap(); + tx.execute("UPDATE core_utxos SET spent = 0, spent_in_txid = NULL", []) + .unwrap(); + + core_history::apply(&tx, &wallet_id, &round(vec![spending.clone()], vec![])).unwrap(); + + assert_eq!( + stored_record(&tx, &wallet_id, &spending.txid).net_amount, + -150_000 + ); + let claim: (bool, Option>) = tx.query_row( + "SELECT spent, spent_in_txid FROM core_utxos WHERE wallet_id = ?1 AND outpoint = ?2", + params![wallet_id.as_slice(), blob::encode_outpoint(&funding.outpoint).unwrap()], + |row| Ok((row.get(0)?, row.get(1)?)), + ).unwrap(); + assert_eq!( + claim, + (false, None), + "accounting cannot authorize or reserve a spend" + ); + } + + #[test] + fn should_backfill_v019_accounting_without_discarding_history_or_spend_state() { + for context in [ + TransactionContext::Mempool, + TransactionContext::InBlock(BlockInfo::new(101, BlockHash::all_zeros(), 123)), + ] { + let wallet_id = [0xAD; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x33; 32]), 100, true); + let spending = spend(&[funding.outpoint], context); + let original = blob::encode(&spending).unwrap(); + let tx = conn.transaction().unwrap(); + apply( + &tx, + &wallet_id, + &round(vec![spending.clone()], vec![funding.clone()]), + ) + .unwrap(); + tx.execute( + "UPDATE core_transactions SET record_blob = ?1 WHERE txid = ?2", + params![original, spending.txid.as_byte_array().as_slice()], + ) + .unwrap(); + tx.execute( + "UPDATE core_utxos SET spent = 1, spent_in_txid = ?1", + params![spending.txid.as_byte_array().as_slice()], + ) + .unwrap(); + tx.execute("INSERT INTO core_sync_state (wallet_id, synced_height, last_processed_height) VALUES (?1, 200, 200)", params![wallet_id.as_slice()]).unwrap(); + tx.commit().unwrap(); + let snapshot = |conn: &Connection| { + let coin: (i64, Option>, i64, Vec) = conn.query_row( + "SELECT spent, spent_in_txid, value, script FROM core_utxos WHERE wallet_id = ?1", + params![wallet_id.as_slice()], |row| Ok((row.get(0)?, row.get(1)?, row.get(2)?, row.get(3)?)), + ).unwrap(); + let sync: (i64, i64) = conn.query_row( + "SELECT synced_height, last_processed_height FROM core_sync_state WHERE wallet_id = ?1", + params![wallet_id.as_slice()], |row| Ok((row.get(0)?, row.get(1)?)), + ).unwrap(); + (coin, sync) + }; + let before = snapshot(&conn); + rewind_v021(&conn); + + migrations::run_for_open(&mut conn, LoadPolicy::Strict).unwrap(); + + let repaired = stored_record(&conn, &wallet_id, &spending.txid); + assert_eq!(repaired.net_amount, -150_000); + assert_eq!(repaired.input_details.len(), 1); + assert_eq!(repaired.context, spending.context); + assert_eq!(original_blob(&conn, &wallet_id, &spending.txid), original); + assert_eq!(snapshot(&conn), before); + let indexed: i64 = conn + .query_row( + "SELECT count(*) FROM core_transaction_inputs WHERE txid = ?1", + params![spending.txid.as_byte_array().as_slice()], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(indexed, 1); + } + } + + #[test] + fn should_skip_failed_accounting_repair_without_partial_index_or_archive() { + let wallet_id = [0xAE; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x34; 32]), 100, true); + let bad = spend(&[funding.outpoint], TransactionContext::Mempool); + let mut good = bad.clone(); + good.transaction.version += 1; + good.txid = good.transaction.txid(); + let tx = conn.transaction().unwrap(); + apply( + &tx, + &wallet_id, + &round(vec![bad.clone(), good.clone()], vec![funding]), + ) + .unwrap(); + for record in [&bad, &good] { + tx.execute( + "UPDATE core_transactions SET record_blob = ?1 WHERE txid = ?2", + params![ + blob::encode(record).unwrap(), + record.txid.as_byte_array().as_slice() + ], + ) + .unwrap(); + } + tx.execute( + "UPDATE core_transactions SET height = 999 WHERE txid = ?1", + params![bad.txid.as_byte_array().as_slice()], + ) + .unwrap(); + tx.commit().unwrap(); + rewind_v021(&conn); + + migrations::run_for_open(&mut conn, LoadPolicy::Recovery).unwrap(); + + let unchanged: Vec = conn + .query_row( + "SELECT record_blob FROM core_transactions WHERE txid = ?1", + params![bad.txid.as_byte_array().as_slice()], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(unchanged, blob::encode(&bad).unwrap()); + for table in [ + "core_transaction_inputs", + "core_transaction_record_originals", + ] { + let count: i64 = conn + .query_row( + &format!("SELECT count(*) FROM {table} WHERE txid = ?1"), + params![bad.txid.as_byte_array().as_slice()], + |row| row.get(0), + ) + .unwrap(); + assert_eq!(count, 0, "failed repair leaves no partial {table} row"); + } + assert_eq!( + stored_record(&conn, &wallet_id, &good.txid).net_amount, + -150_000 + ); + assert_eq!( + original_blob(&conn, &wallet_id, &good.txid), + blob::encode(&good).unwrap() + ); + } + + #[test] + fn should_repair_indexed_legacy_history_when_spent_funding_arrives_after_migration() { + let wallet_id = [0xAF; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x35; 32]), 100, true); + let spending = spend(&[funding.outpoint], TransactionContext::Mempool); + let tx = conn.transaction().unwrap(); + apply(&tx, &wallet_id, &round(vec![spending.clone()], vec![])).unwrap(); + tx.commit().unwrap(); + rewind_v021(&conn); + migrations::run_for_open(&mut conn, LoadPolicy::Strict).unwrap(); + + let tx = conn.transaction().unwrap(); + apply( + &tx, + &wallet_id, + &CoreChangeSet { + spent_utxos: vec![funding], + ..Default::default() + }, + ) + .unwrap(); + + assert_eq!( + stored_record(&tx, &wallet_id, &spending.txid).net_amount, + -150_000 + ); + assert_eq!( + original_blob(&tx, &wallet_id, &spending.txid), + blob::encode(&spending).unwrap() + ); + } + + #[test] + fn should_repair_history_when_spent_funding_arrives_late() { + let wallet_id = [0xAB; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x31; 32]), 100, true); + let block = BlockInfo::new(101, BlockHash::all_zeros(), 123); + let mut spending = spend(&[funding.outpoint], TransactionContext::InBlock(block)); + spending.transaction.output.push(TxOut { + value: 90_000, + script_pubkey: funding.txout.script_pubkey.clone(), + }); + spending.txid = spending.transaction.txid(); + spending.net_amount = 90_000; + let mut change = sample_utxo(spending.txid, 101, true); + change.txout.value = 90_000; + let tx = conn.transaction().unwrap(); + apply( + &tx, + &wallet_id, + &round(vec![spending.clone()], vec![change]), + ) + .unwrap(); + let other_wallet = [0xCD; 32]; + insert_test_wallet(&tx, &other_wallet, 0); + apply(&tx, &other_wallet, &round(vec![], vec![funding.clone()])).unwrap(); + assert_eq!( + stored_record(&tx, &wallet_id, &spending.txid).net_amount, + 90_000, + "another wallet's funding must not affect this history" + ); + apply( + &tx, + &wallet_id, + &CoreChangeSet { + spent_utxos: vec![funding.clone()], + ..Default::default() + }, + ) + .unwrap(); + let repaired = stored_record(&tx, &wallet_id, &spending.txid); + assert_eq!(repaired.net_amount, -60_000); + assert_eq!(repaired.direction, TransactionDirection::Internal); + assert_eq!(repaired.input_details.len(), 1); + assert_eq!(repaired.block_info().unwrap().height(), 101); + let original = original_blob(&tx, &wallet_id, &spending.txid); + assert_eq!( + blob::decode::(&original) + .unwrap() + .net_amount, + 90_000, + "the pre-repair record is kept verbatim" + ); + apply( + &tx, + &wallet_id, + &round(vec![spending.clone()], vec![funding]), + ) + .unwrap(); + let replayed = stored_record(&tx, &wallet_id, &spending.txid); + assert_eq!(replayed.net_amount, -60_000); + assert_eq!(replayed.input_details.len(), 1); + } + + /// A store at V019 holding one kept pending record whose body no longer + /// decodes. + fn v019_store_with_undecodable_kept_record(wallet_id: &[u8; 32]) -> rusqlite::Connection { + let mut conn = wallet_db(wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x81; 32]), 150, true); + let pending = spend(&[funding.outpoint], TransactionContext::Mempool); + let coins = vec![funding]; + let tx = conn.transaction().unwrap(); + apply( + &tx, + wallet_id, + &CoreChangeSet { + spent_utxos: coins.clone(), + ..round(vec![pending.clone()], coins) + }, + ) + .unwrap(); + tx.commit().unwrap(); + rewind_v021(&conn); + conn.execute( + "UPDATE core_transactions SET record_blob = x'00' WHERE txid = ?1", + [pending.txid.as_byte_array().to_vec()], + ) + .unwrap(); + conn + } + + fn schema_version(conn: &rusqlite::Connection) -> i64 { + conn.query_row( + "SELECT max(version) FROM refinery_schema_history", + [], + |row| row.get(0), + ) + .unwrap() + } + + /// Under `Strict`, a kept pending record V021 cannot decode fails the + /// upgrade instead of being skipped: a skipped body leaves its inputs + /// unindexed with no signal, so a record-codec regression would only + /// show up as coins stuck later. The failed upgrade rolls back to V019. + #[test] + fn should_fail_v021_under_strict_on_a_kept_record_it_cannot_decode() { + let mut conn = v019_store_with_undecodable_kept_record(&[0xB8u8; 32]); + + migrations::run_for_open(&mut conn, LoadPolicy::Strict) + .expect_err("an undecodable kept record must fail V021 under Strict"); + + assert_eq!( + schema_version(&conn), + 20, + "the failed upgrade must roll back" + ); + } + + /// Under `Recovery`, V021 skips a kept record it cannot read and the + /// upgrade completes; that wallet degrades at load instead. + #[test] + fn should_skip_an_undecodable_kept_record_in_v021_under_recovery() { + let mut conn = v019_store_with_undecodable_kept_record(&[0xB9u8; 32]); + + migrations::run_for_open(&mut conn, LoadPolicy::Recovery) + .expect("Recovery must complete V021 past an undecodable kept record"); + + assert_eq!(schema_version(&conn), 21); + } + + #[test] + fn should_reject_corrupt_prior_record_when_storing_the_transaction() { + let wallet_id = [0xA9u8; 32]; + let mut conn = wallet_db(&wallet_id); + let record = transaction_record(TransactionContext::Mempool); + conn.execute( + "INSERT INTO core_transactions (wallet_id, txid, finalized, record_blob) \ + VALUES (?1, ?2, 0, ?3)", + params![ + &wallet_id[..], + AsRef::<[u8]>::as_ref(&record.txid), + &[0xffu8][..] + ], + ) + .unwrap(); + let tx = conn.transaction().unwrap(); + assert!( + apply(&tx, &wallet_id, &round(vec![record], vec![])).is_err(), + "normal operation treats corrupt stored history as an error" + ); + } + + #[test] + fn should_exclude_historical_contact_outputs_from_accounting() { + use key_wallet::managed_account::transaction_record::{OutputDetail, OutputRole}; + let wallet_id = [0xAFu8; 32]; + let mut conn = wallet_db(&wallet_id); + let mut record = transaction_record(TransactionContext::Mempool); + let mut output = sample_utxo(record.txid, 0, false); + record.transaction.output = vec![output.txout.clone()]; + record.txid = record.transaction.txid(); + record.output_details = vec![OutputDetail { + index: 0, + role: OutputRole::Received, + address: Some(output.address.clone()), + value: output.value(), + }]; + record.net_amount = output.value() as i64; + output.outpoint.txid = record.txid; + insert_contact_script(&conn, &wallet_id, &output.txout.script_pubkey); + let tx = conn.transaction().unwrap(); + apply(&tx, &wallet_id, &round(vec![record.clone()], vec![output])).unwrap(); + let repaired = stored_record(&tx, &wallet_id, &record.txid); + assert_eq!(repaired.net_amount, 0); + assert_eq!(repaired.output_details[0].role, OutputRole::Sent); + } + + /// Tracks `script` only on a contact's watch-only chain. + fn insert_contact_script(conn: &Connection, wallet_id: &[u8; 32], script: &dashcore::Script) { + conn.execute( + "INSERT INTO core_address_pool \ + (wallet_id, account_type, account_index, pool_type, address_index, script) \ + VALUES (?1, 'dashpay_external', 0, 0, 0, ?2)", + params![&wallet_id[..], script.as_bytes()], + ) + .unwrap(); + } + + #[test] + fn should_preserve_known_inputs_when_replay_has_no_historical_txo() { + use key_wallet::managed_account::transaction_record::InputDetail; + let wallet_id = [0xB0u8; 32]; + let mut conn = wallet_db(&wallet_id); + let funding = sample_utxo(Txid::from_byte_array([0x42; 32]), 100, true); + let mut record = spend(&[funding.outpoint], TransactionContext::Mempool); + record.input_details = vec![InputDetail { + index: 0, + value: funding.value(), + address: funding.address, + }]; + record.net_amount = -150_000; + let tx = conn.transaction().unwrap(); + apply(&tx, &wallet_id, &round(vec![record.clone()], vec![])).unwrap(); + record.input_details.clear(); + record.net_amount = 0; + apply(&tx, &wallet_id, &round(vec![record.clone()], vec![])).unwrap(); + let repaired = stored_record(&tx, &wallet_id, &record.txid); + assert_eq!(repaired.net_amount, -150_000); + assert_eq!(repaired.input_details.len(), 1); + } + fn sample_chain_lock(height: u32) -> ChainLock { ChainLock { block_height: height, @@ -1407,7 +2060,7 @@ mod tests { #[test] fn load_state_rejects_oversize_instant_lock_txid() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let w = [0xABu8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1435,7 +2088,7 @@ mod tests { #[test] fn load_state_reconciles_utxo_height_from_confirmed_transaction_record() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x42u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1443,7 +2096,7 @@ mod tests { ) .unwrap(); - let txid = Txid::from_byte_array([0x7Eu8; 32]); + let txid = transaction_record(TransactionContext::Mempool).txid; let utxo = sample_utxo(txid, 123, true); let outpoint = utxo.outpoint; @@ -1454,7 +2107,7 @@ mod tests { &wallet_id, &CoreChangeSet { new_utxos: vec![utxo], - records: vec![transaction_record(txid, TransactionContext::Mempool)], + records: vec![transaction_record(TransactionContext::Mempool)], ..Default::default() }, ) @@ -1469,14 +2122,13 @@ mod tests { &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record( - txid, - TransactionContext::InChainLockedBlock(BlockInfo::new( + records: vec![transaction_record(TransactionContext::InChainLockedBlock( + BlockInfo::new( confirmed_height, BlockHash::from_byte_array([0x34u8; 32]), 1_735_689_600, - )), - )], + ), + ))], ..Default::default() }, ) @@ -1503,7 +2155,7 @@ mod tests { #[test] fn load_state_restores_confirmed_recordless_utxo_height() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x44u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1546,7 +2198,7 @@ mod tests { #[test] fn height_only_placeholder_does_not_regress() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x49u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1596,7 +2248,7 @@ mod tests { #[test] fn load_state_treats_height_zero_as_confirmed() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x4Au8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1631,7 +2283,7 @@ mod tests { #[test] fn transaction_record_always_overrides_height_only_placeholder() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x45u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1639,7 +2291,7 @@ mod tests { ) .unwrap(); - let txid = Txid::from_byte_array([0x81u8; 32]); + let txid = transaction_record(TransactionContext::Mempool).txid; let mut utxo = sample_utxo(txid, 456, true); { let tx = conn.transaction().unwrap(); @@ -1661,7 +2313,7 @@ mod tests { &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record(txid, TransactionContext::Mempool)], + records: vec![transaction_record(TransactionContext::Mempool)], ..Default::default() }, ) @@ -1702,14 +2354,13 @@ mod tests { &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record( - txid, - TransactionContext::InChainLockedBlock(BlockInfo::new( + records: vec![transaction_record(TransactionContext::InChainLockedBlock( + BlockInfo::new( confirmed_height, BlockHash::from_byte_array([0x35u8; 32]), 1_735_689_700, - )), - )], + ), + ))], ..Default::default() }, ) @@ -1731,7 +2382,7 @@ mod tests { #[test] fn load_state_defaults_utxo_without_transaction_record_to_unconfirmed() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x43u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1789,7 +2440,7 @@ mod tests { #[test] fn load_state_tolerates_transaction_blob_txid_drift_in_recovery_without_repairing() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x46u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1797,7 +2448,7 @@ mod tests { ) .unwrap(); - let blob_txid = Txid::from_byte_array([0x82u8; 32]); + let blob_txid = transaction_record(TransactionContext::Mempool).txid; let typed_txid = Txid::from_byte_array([0x83u8; 32]); { let tx = conn.transaction().unwrap(); @@ -1805,7 +2456,7 @@ mod tests { &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record(blob_txid, TransactionContext::Mempool)], + records: vec![transaction_record(TransactionContext::Mempool)], ..Default::default() }, ) @@ -1842,9 +2493,9 @@ mod tests { #[test] fn load_state_blob_height_wins_over_drifted_typed_column_in_either_scan_order() { - for (case, typed_byte) in [0x10, 0xF0].into_iter().enumerate() { + for (case, typed_byte) in [0x00, 0xFF].into_iter().enumerate() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x50 + case as u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) \ @@ -1853,7 +2504,7 @@ mod tests { ) .unwrap(); - let blob_txid = Txid::from_byte_array([0x80; 32]); + let blob_txid = transaction_record(TransactionContext::Mempool).txid; let typed_txid = Txid::from_byte_array([typed_byte; 32]); let confirmed_utxo = sample_utxo(blob_txid, 500, true); let tx = conn.transaction().unwrap(); @@ -1862,14 +2513,9 @@ mod tests { &wallet_id, &CoreChangeSet { new_utxos: vec![confirmed_utxo.clone()], - records: vec![transaction_record( - blob_txid, - TransactionContext::InChainLockedBlock(BlockInfo::new( - 500, - BlockHash::from_byte_array([0x38; 32]), - 1_735_690_000, - )), - )], + records: vec![transaction_record(TransactionContext::InChainLockedBlock( + BlockInfo::new(500, BlockHash::from_byte_array([0x38; 32]), 1_735_690_000), + ))], ..Default::default() }, ) @@ -1911,7 +2557,7 @@ mod tests { #[test] fn load_state_tolerates_transaction_blob_height_drift_in_recovery_without_repairing() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x47u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1919,21 +2565,19 @@ mod tests { ) .unwrap(); - let txid = Txid::from_byte_array([0x84u8; 32]); { let tx = conn.transaction().unwrap(); apply( &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record( - txid, - TransactionContext::InChainLockedBlock(BlockInfo::new( + records: vec![transaction_record(TransactionContext::InChainLockedBlock( + BlockInfo::new( 500, BlockHash::from_byte_array([0x36u8; 32]), 1_735_689_800, - )), - )], + ), + ))], ..Default::default() }, ) @@ -1975,7 +2619,7 @@ mod tests { #[test] fn get_tx_record_declines_a_txid_drifted_row_in_recovery_without_repairing() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x4Bu8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -1983,14 +2627,14 @@ mod tests { ) .unwrap(); - let blob_txid = Txid::from_byte_array([0x88u8; 32]); + let blob_txid = transaction_record(TransactionContext::Mempool).txid; let typed_txid = Txid::from_byte_array([0x89u8; 32]); let tx = conn.transaction().unwrap(); apply( &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record(blob_txid, TransactionContext::Mempool)], + records: vec![transaction_record(TransactionContext::Mempool)], ..Default::default() }, ) @@ -2031,7 +2675,7 @@ mod tests { #[test] fn get_tx_record_tolerates_blob_height_drift_in_recovery_without_repairing() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x4Cu8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -2039,20 +2683,15 @@ mod tests { ) .unwrap(); - let txid = Txid::from_byte_array([0x8Au8; 32]); + let txid = transaction_record(TransactionContext::Mempool).txid; let tx = conn.transaction().unwrap(); apply( &tx, &wallet_id, &CoreChangeSet { - records: vec![transaction_record( - txid, - TransactionContext::InChainLockedBlock(BlockInfo::new( - 600, - BlockHash::from_byte_array([0x37u8; 32]), - 1_735_689_900, - )), - )], + records: vec![transaction_record(TransactionContext::InChainLockedBlock( + BlockInfo::new(600, BlockHash::from_byte_array([0x37u8; 32]), 1_735_689_900), + ))], ..Default::default() }, ) @@ -2089,7 +2728,7 @@ mod tests { #[test] fn load_used_addresses_wraps_address_error_as_address_decode() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let w = [0x99u8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -2124,7 +2763,7 @@ mod tests { #[test] fn apply_refuses_an_empty_script_on_a_new_utxo() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x5Bu8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -2159,7 +2798,7 @@ mod tests { #[test] fn apply_refuses_an_empty_script_on_a_synthetic_spent_row() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x5Cu8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", @@ -2191,7 +2830,7 @@ mod tests { #[test] fn apply_still_marks_an_existing_utxo_spent() { let mut conn = rusqlite::Connection::open_in_memory().unwrap(); - crate::sqlite::migrations::run(&mut conn).unwrap(); + migrations::run(&mut conn).unwrap(); let wallet_id = [0x5Du8; 32]; conn.execute( "INSERT INTO wallets (wallet_id, network, birth_height) VALUES (?1, 'testnet', 0)", diff --git a/packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs b/packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs index 88bff87e377..ad4bdbb387b 100644 --- a/packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs +++ b/packages/rs-platform-wallet-storage/src/sqlite/schema/mod.rs @@ -10,6 +10,7 @@ pub mod accounts; pub mod asset_locks; pub mod blob; pub mod contacts; +pub(crate) mod core_history; pub mod core_pool; pub mod core_state; pub mod dashpay; diff --git a/packages/rs-platform-wallet-storage/tests/fixtures/v019_asset_unlock_v2.txt b/packages/rs-platform-wallet-storage/tests/fixtures/v019_asset_unlock_v2.txt new file mode 100644 index 00000000000..886a1e01b9b --- /dev/null +++ b/packages/rs-platform-wallet-storage/tests/fixtures/v019_asset_unlock_v2.txt @@ -0,0 +1,2 @@ +asset_unlock_v2 083e1e2f5eb1617c72a8090ea9adf5c0b12852ff4b17f6a3c060f7a0bd3bbbc3 03000001fcf04902001976a914232323232323232323232323232323232323232388ac01080247fbe803c82031313131313131313131313131313131313131313131313131313131313131316042424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424242424220083e1e2f5eb1617c72a8090ea9adf5c0b12852ff4b17f6a3c060f7a0bd3bbbc300000000070000010000012279505845624a394b4851386a344d4533527677344176614b766944774747786f6252fcf0490200fce09304000000 +asset_unlock_v2_spend 1d0f034993273b3c7f8f866bce25fc1ea6e3f6723493fb9fe37c20b6513bc591 02000120083e1e2f5eb1617c72a8090ea9adf5c0b12852ff4b17f6a3c060f7a0bd3bbbc30000fcffffffff0001fc084602000000201d0f034993273b3c7f8f866bce25fc1ea6e3f6723493fb9fe37c20b6513bc5910000000000010100fcf04902002279505845624a394b4851386a344d4533527677344176614b766944774747786f625200fcdf9304000000 diff --git a/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.md b/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.md new file mode 100644 index 00000000000..0deae805746 --- /dev/null +++ b/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.md @@ -0,0 +1,31 @@ +# Immutable V019 kept records + +`v019_kept_records.txt` contains `name txid outpoint record_blob`, with binary +columns in hex. Do not regenerate it when runtime serializers change. + +Shipped 5.0 stores (schema V018) were written with rust-dashcore/key-wallet +revision `d631aaca`. These fixture bytes were encoded once at +`aca020263990a86a15cc1d97ae2cbf965d488db5` with grovedb-bincode `2.1.0`, using bincode's standard little-endian variable-integer +encoding. The serde types used by these rows are compatible with `d631aaca`: +TransactionRecord gained methods without changing its fields; TransactionContext, +AccountType and InstantLock stayed the same; TransactionPayload appended variants; +Address serde stayed unchanged. These are generated compatibility fixtures for +the pending record format that reaches V021. + +The writer uses `blob::encode(TransactionRecord)`. Both rows belong under SQL +`height = NULL`. Live-codec tests decode these exact bytes and assert their +values and unchanged re-encoding; the V021 test asserts literal indexed keys. +An incompatible dependency update must supply a historical decoder at that point. + +- `mempool`: `TransactionContext::Mempool`, transaction lock time 17. +- `instant_send`: `TransactionContext::InstantSend`, lock time 18. Its lock + names the transaction and the same input. The quorum signature is a zero + test value; migration decoding does not verify quorum signatures. + +Both are version-2 transactions spending output 7 of txid `[0x42; 32]`, with +scriptSig `51`, sequence `u32::MAX - 1`, and witness `[010203]`. The output +pays 999 duffs to testnet P2PKH hash `[0x11; 20]`. Each record uses standard +BIP44 account 0, standard transaction type, internal direction, one input +detail (1000 duffs), one change-output detail (999 duffs), net -1, fee 1, +and label `V019 fixture`. The input and output details carry that +address, exercising the historical address codec as well as scripts/witnesses. diff --git a/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.txt b/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.txt new file mode 100644 index 00000000000..3a56214c275 --- /dev/null +++ b/packages/rs-platform-wallet-storage/tests/fixtures/v019_kept_records.txt @@ -0,0 +1,2 @@ +mempool 26d0b7bb98569904b645f54b44bc2776a71d4510366f70f7769198f944053117 20424242424242424242424242424242424242424242424242424242424242424207 021101204242424242424242424242424242424242424242424242424242424242424242070151fcfeffffff010301020301fbe7031976a914111111111111111111111111111111111111111188ac002026d0b7bb98569904b645f54b44bc2776a71d4510366f70f7769198f9440531170000000000020100fbe80322794d73676e4831784b476138356e34627132696d725a6247324b67726d47747441560100010122794d73676e4831784b476138356e34627132696d725a6247324b67726d4774744156fbe7030101011456303139206d656d706f6f6c2066697874757265 +instant_send bc8dc33c345616d55e6e3876e2c941ccfbfb10760a0c7a888c6f34dbd04b2aa4 20424242424242424242424242424242424242424242424242424242424242424207 021201204242424242424242424242424242424242424242424242424242424242424242070151fcfeffffff010301020301fbe7031976a914111111111111111111111111111111111111111188ac0020bc8dc33c345616d55e6e3876e2c941ccfbfb10760a0c7a888c6f34dbd04b2aa40000000101012042424242424242424242424242424242424242424242424242424242424242420720bc8dc33c345616d55e6e3876e2c941ccfbfb10760a0c7a888c6f34dbd04b2aa42000000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020100fbe80322794d73676e4831784b476138356e34627132696d725a6247324b67726d47747441560100010122794d73676e4831784b476138356e34627132696d725a6247324b67726d4774744156fbe703010101195630313920696e7374616e745f73656e642066697874757265 diff --git a/packages/rs-platform-wallet-storage/tests/integration/persistence_error_kind_mapping.rs b/packages/rs-platform-wallet-storage/tests/integration/persistence_error_kind_mapping.rs index 98cfc8d0561..3d2ffe9d04a 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/persistence_error_kind_mapping.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/persistence_error_kind_mapping.rs @@ -117,11 +117,14 @@ fn tc_code_004_b_constraint_variants_map_to_constraint_kind() { } } -/// Identity-slot uniqueness is enforced in Rust rather than by a SQL -/// constraint, so its variants have to claim the `Constraint` kind -/// explicitly — they are caller-data violations like any FK breach. +/// Identity-slot uniqueness and history invariants are enforced in Rust +/// rather than by a SQL constraint, so their variants have to claim the +/// `Constraint` kind explicitly — they are caller-data violations like any +/// FK breach: an incoming record that contradicts stored history is a data +/// fault, not a retryable or engine failure. #[test] -fn tc_code_004_b_identity_index_variants_map_to_constraint_kind() { +fn tc_code_004_b_rust_enforced_variants_map_to_constraint_kind() { + let txid = dashcore::Txid::from_byte_array([0x44; 32]); let cases: Vec<(&str, WalletStorageError)> = vec![ ( "IdentityIndexConflict", @@ -139,6 +142,21 @@ fn tc_code_004_b_identity_index_variants_map_to_constraint_kind() { identity_index: 2, }, ), + ( + "TransactionBodyConflict", + WalletStorageError::TransactionBodyConflict { + wallet_id: [0xAA; 32], + txid, + }, + ), + ( + "NetAmountOverflow", + WalletStorageError::NetAmountOverflow { + wallet_id: [0xAA; 32], + txid, + value: i128::from(i64::MIN) - 1, + }, + ), ]; for (label, err) in cases { assert!(!err.is_transient(), "{label}: must not be transient"); @@ -232,6 +250,13 @@ fn tc_code_004_b_fatal_variants_map_to_fatal_kind() { "BlobDecode", WalletStorageError::BlobDecode { reason: "len" }, ), + ( + "UnknownWalletNetwork", + WalletStorageError::UnknownWalletNetwork { + wallet_id: [0xAA; 32], + label: "moonnet".into(), + }, + ), ( "ForeignKeysNotEnforced", WalletStorageError::ForeignKeysNotEnforced, diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_buffer_semantics.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_buffer_semantics.rs index 72c89318136..a61f7911aca 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_buffer_semantics.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_buffer_semantics.rs @@ -193,7 +193,6 @@ fn tc016_buffer_merge_oracle_smoke() { #[test] fn tc001_get_core_tx_record_roundtrip() { use dashcore::blockdata::transaction::Transaction; - use dashcore::Txid; use key_wallet::managed_account::transaction_record::{ TransactionDirection, TransactionRecord, }; @@ -201,7 +200,6 @@ fn tc001_get_core_tx_record_roundtrip() { let (persister, _tmp, _path) = fresh_persister(); let w = wid(0x50); ensure_wallet_meta(&persister, &w); - let txid = Txid::from_byte_array([9u8; 32]); let dummy_tx = Transaction { version: 3, lock_time: 0, @@ -209,7 +207,7 @@ fn tc001_get_core_tx_record_roundtrip() { output: vec![], special_transaction_payload: None, }; - let mut record = TransactionRecord::new( + let record = TransactionRecord::new( dummy_tx, key_wallet::account::AccountType::Standard { index: 0, @@ -226,7 +224,7 @@ fn tc001_get_core_tx_record_roundtrip() { Vec::new(), 100, ); - record.txid = txid; + let txid = record.txid; let mut cs = PlatformWalletChangeSet::default(); cs.core = Some(CoreChangeSet { records: vec![record], diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_error_classification.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_error_classification.rs index a67f27e8080..289aecb600f 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_error_classification.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_error_classification.rs @@ -213,6 +213,19 @@ fn samples() -> Vec { typed_height: Some(100), blob_height: Some(101), }, + WalletStorageError::TransactionBodyConflict { + wallet_id: [0x33; 32], + txid: dashcore::Txid::from_byte_array([0x44; 32]), + }, + WalletStorageError::UnknownWalletNetwork { + wallet_id: [0x33; 32], + label: "moonnet".into(), + }, + WalletStorageError::NetAmountOverflow { + wallet_id: [0x33; 32], + txid: dashcore::Txid::from_byte_array([0x44; 32]), + value: i128::from(u64::MAX) * 2, + }, WalletStorageError::BlobTooLarge { len_bytes: 32 * 1024 * 1024, limit_bytes: 16 * 1024 * 1024, @@ -436,6 +449,11 @@ fn tc_p2_005_is_transient_table() { WalletStorageError::CoreTransactionEntryMismatch { .. } => { (false, "core_transaction_entry_mismatch") } + WalletStorageError::TransactionBodyConflict { .. } => { + (false, "transaction_body_conflict") + } + WalletStorageError::UnknownWalletNetwork { .. } => (false, "unknown_wallet_network"), + WalletStorageError::NetAmountOverflow { .. } => (false, "net_amount_overflow"), WalletStorageError::BlobTooLarge { .. } => (false, "blob_too_large"), WalletStorageError::ForeignKeysNotEnforced => (false, "foreign_keys_not_enforced"), WalletStorageError::JournalModeNotApplied { .. } => (false, "journal_mode_not_applied"), diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_load_reconstruction.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_load_reconstruction.rs index 9f45ae82a58..f85936cabf4 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_load_reconstruction.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_load_reconstruction.rs @@ -1828,3 +1828,59 @@ fn tc_p4_020_unignore_survives_restart_table_is_authoritative() { "an ignore recorded in the table restores even when the blob snapshot predates it" ); } + +#[test] +fn should_load_wallet_from_one_snapshot_during_an_external_commit() { + use key_wallet::account::{AccountType, StandardAccountType}; + use platform_wallet::changeset::{AccountRegistrationEntry, CoreChangeSet}; + use rusqlite::hooks::{AuthAction, AuthContext, Authorization}; + use std::sync::atomic::{AtomicBool, Ordering}; + use std::sync::Arc; + + let (persister, _tmp, path) = fresh_persister(); + let wallet_id = wid(0xC4); + ensure_wallet_meta(&persister, &wallet_id); + persister + .store( + wallet_id, + PlatformWalletChangeSet { + account_registrations: vec![AccountRegistrationEntry { + account_type: AccountType::Standard { + index: 0, + standard_account_type: StandardAccountType::BIP44Account, + }, + account_xpub: test_xpub(), + }], + core: Some(CoreChangeSet { + synced_height: Some(100), + ..Default::default() + }), + ..Default::default() + }, + ) + .unwrap(); + let writer = rusqlite::Connection::open(path).unwrap(); + let changed = Arc::new(AtomicBool::new(false)); + let changed_hook = Arc::clone(&changed); + persister.lock_conn_for_test().authorizer(Some(move |ctx: AuthContext<'_>| { + if matches!(ctx.action, AuthAction::Read { table_name: "core_sync_state", .. }) + && !changed_hook.swap(true, Ordering::SeqCst) + { + writer.execute_batch("BEGIN IMMEDIATE; UPDATE wallets SET birth_height = 10; UPDATE core_sync_state SET synced_height = 200; COMMIT;").unwrap(); + } + Authorization::Allow + })).unwrap(); + + let state = persister.load().unwrap(); + + assert!( + changed.load(Ordering::SeqCst), + "external commit must occur during load" + ); + let metadata = &state.wallets[&wallet_id].wallet_info.metadata; + assert_eq!(metadata.birth_height, 0); + assert_eq!( + metadata.synced_height, 100, + "wallet metadata and checkpoint share one snapshot" + ); +} diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_migration_execution.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_migration_execution.rs index 2a4f6d242ee..b0cfc1ee715 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_migration_execution.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_migration_execution.rs @@ -65,9 +65,56 @@ fn copy_fixture(dir: &Path) -> PathBuf { fn copy_fixture_as(dir: &Path, file_name: &str) -> PathBuf { let dst = dir.join(file_name); std::fs::copy(fixture_src(), &dst).expect("copy fixture"); + correct_fixture_transaction_id(&dst); dst } +/// The historical generator assigned a synthetic txid to an empty transaction. +/// Preserve the committed schema fixture; make only this test copy self-consistent. +fn correct_fixture_transaction_id(path: &Path) { + use key_wallet::managed_account::transaction_record::TransactionRecord; + use platform_wallet_storage::sqlite::schema::blob; + + let mut conn = Connection::open(path).unwrap(); + let tx = conn.transaction().unwrap(); + let (_, bytes) = transaction_height_and_blob(&tx, &wid(FULL_WALLET)); + let mut record: TransactionRecord = blob::decode(&bytes).unwrap(); + let previous = record.txid; + record.txid = record.transaction.txid(); + tx.execute("UPDATE core_transactions SET txid = ?1, record_blob = ?2 WHERE wallet_id = ?3 AND txid = ?4", + rusqlite::params![record.txid.as_byte_array().as_slice(), blob::encode(&record).unwrap(), wid(FULL_WALLET).as_slice(), previous.as_byte_array().as_slice()]).unwrap(); + tx.execute( + "UPDATE core_utxos SET outpoint = ?1 WHERE wallet_id = ?2 AND outpoint = ?3", + rusqlite::params![ + blob::encode_outpoint(&dashcore::OutPoint::new(record.txid, 0)).unwrap(), + wid(FULL_WALLET).as_slice(), + blob::encode_outpoint(&dashcore::OutPoint::new(previous, 0)).unwrap() + ], + ) + .unwrap(); + tx.commit().unwrap(); +} + +#[test] +fn should_reject_legacy_fixture_with_a_synthetic_transaction_id() { + let tmp = common::secure_tempdir().unwrap(); + let path = tmp.path().join("invalid-record.db"); + std::fs::copy(fixture_src(), &path).unwrap(); + let before = transaction_height_and_blob(&ro_conn(&path), &wid(FULL_WALLET)); + + assert!(matches!( + SqlitePersister::open(SqlitePersisterConfig::new(&path)), + Err(WalletStorageError::Migration(_)) + )); + + let conn = ro_conn(&path); + assert_eq!(schema_version(&conn), 6); + assert_eq!( + transaction_height_and_blob(&conn, &wid(FULL_WALLET)), + before + ); +} + /// Names of the `pre-migration-*` backups sitting in `dir`. fn pre_migration_backup_names(dir: &Path) -> Vec { let mut names: Vec = std::fs::read_dir(dir) @@ -634,3 +681,35 @@ fn reopen_of_migrated_store_is_idempotent() { ); assert_eq!(first, second, "reopen is a byte-stable no-op"); } + +#[test] +fn should_refuse_a_database_that_skipped_an_earlier_migration() { + let tmp = common::secure_tempdir().unwrap(); + let path = tmp.path().join("wallet.db"); + drop(SqlitePersister::open(SqlitePersisterConfig::new(&path)).unwrap()); + let history = |conn: &Connection| -> Vec { + let mut stmt = conn + .prepare("SELECT version FROM refinery_schema_history ORDER BY version") + .unwrap(); + stmt.query_map([], |r| r.get(0)) + .unwrap() + .map(Result::unwrap) + .collect() + }; + let before = { + let conn = Connection::open(&path).unwrap(); + conn.execute("DELETE FROM refinery_schema_history WHERE version = 18", []) + .unwrap(); + history(&conn) + }; + + match SqlitePersister::open(SqlitePersisterConfig::new(&path)) { + Err(WalletStorageError::Migration(error)) => assert!( + matches!(error.kind(), refinery::error::Kind::MissingVersion(m) if m.version() == 18), + "expected MissingVersion(18), got {error:?}" + ), + Err(other) => panic!("expected a migration error, got {other:?}"), + Ok(_) => panic!("a skipped migration must not be applied out of order or ignored"), + } + assert_eq!(history(&Connection::open(&path).unwrap()), before); +} diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_migrations.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_migrations.rs index f2c835cf119..ce850eb949c 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_migrations.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_migrations.rs @@ -608,7 +608,11 @@ fn tc047_v013_drops_orphans_instead_of_aborting_the_rebuild() { .expect("re-enable foreign keys"); // 4. The rebuild must complete rather than abort on the orphans. - mig::run(&mut conn).expect("migrate to latest despite the orphan rows"); + // This fixture deliberately uses opaque record bytes; V014 preserves them. + mig::runner() + .set_target(refinery::Target::Version(14)) + .run(&mut conn) + .expect("V014 must migrate despite the orphan rows"); // 5. Both orphans are gone — the same outcome the declared cascade would // have produced had enforcement been on when the wallet was deleted. diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_recovery_mode.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_recovery_mode.rs index b5b136ff238..3548c0df3df 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_recovery_mode.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_recovery_mode.rs @@ -54,6 +54,17 @@ fn assert_only_site(persister: &SqlitePersister, site: LoadSite, expected: u32) assert_eq!(degradation.total, expected); } +/// The writer's encoding of an outpoint key: load decodes every spent row's +/// outpoint to restore its spend claim, so a planted row needs a real one. +fn spent_outpoint_key(marker: u8, vout: u32) -> Vec { + use platform_wallet_storage::sqlite::schema::blob; + blob::encode_outpoint(&dashcore::OutPoint::new( + Txid::from_byte_array([marker; 32]), + vout, + )) + .expect("encode outpoint") +} + /// Seed a `core_sync_state` row, then overwrite its chain lock with bytes /// that are not a `ChainLock`. fn seed_corrupt_chain_lock(persister: &SqlitePersister, wallet: &WalletId) { @@ -94,7 +105,7 @@ fn seed_drifted_transaction(persister: &SqlitePersister, wallet: &WalletId) { } fn drifted_txid() -> Txid { - Txid::from_byte_array([0x7Au8; 32]) + confirmed_record().txid } /// A record whose blob says height 300, so a drifted typed column is @@ -106,7 +117,7 @@ fn confirmed_record() -> key_wallet::managed_account::transaction_record::Transa TransactionDirection, TransactionRecord, }; use key_wallet::transaction_checking::{BlockInfo, TransactionContext, TransactionType}; - let mut record = TransactionRecord::new( + TransactionRecord::new( Transaction { version: 3, lock_time: 0, @@ -128,9 +139,7 @@ fn confirmed_record() -> key_wallet::managed_account::transaction_record::Transa Vec::new(), Vec::new(), 100, - ); - record.txid = drifted_txid(); - record + ) } /// Register a deterministic keyless wallet and return its BIP44 external @@ -516,7 +525,11 @@ fn unresolved_and_undecodable_addresses_are_counted_separately() { conn.execute( "INSERT INTO core_utxos (wallet_id, outpoint, value, script, spent) \ VALUES (?1, ?2, 0, ?3, 1)", - params![wallet.as_slice(), &[0xFF_u8; 36], bad_script.as_slice()], + params![ + wallet.as_slice(), + spent_outpoint_key(0xFF, 0), + bad_script.as_slice() + ], ) .expect("plant undecodable UTXO script"); @@ -527,15 +540,12 @@ fn unresolved_and_undecodable_addresses_are_counted_separately() { dashcore::Network::Testnet, dashcore::address::Payload::PubkeyHash(dashcore::PubkeyHash::from_byte_array(hash)), ); - let mut outpoint = [0_u8; 36]; - outpoint[..4].copy_from_slice(&index.to_le_bytes()); - outpoint[4] = 1; conn.execute( "INSERT INTO core_utxos (wallet_id, outpoint, value, script, spent) \ VALUES (?1, ?2, 0, ?3, 1)", params![ wallet.as_slice(), - outpoint, + spent_outpoint_key(0x01, index), address.script_pubkey().as_bytes() ], ) @@ -833,7 +843,11 @@ fn seed_conflicting_used_address_owner(persister: &SqlitePersister, wallet: &Wal conn.execute( "INSERT INTO core_utxos (wallet_id, outpoint, value, script, spent) \ VALUES (?1, ?2, 1000, ?3, 1)", - params![wallet.as_slice(), &[0x11u8; 36][..], script.as_slice()], + params![ + wallet.as_slice(), + spent_outpoint_key(0x11, 0), + script.as_slice() + ], ) .expect("seed spent utxo carrying the same script"); } diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs index f1c5755c448..f09aa4265b8 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_schema_pinning.rs @@ -15,13 +15,13 @@ use platform_wallet_storage::sqlite::{migrations as mig, schema::versions::Domai /// Golden `(version, name)` fingerprint of the frozen migration set. Bump /// deliberately only when adding/removing/renaming a migration file. const EXPECTED_ID_FINGERPRINT: &str = - "b1e10a6494cc31d24fabe4852477b3b93a2a98a7f310681b132ca8d9adef7311"; + "ff3bc3ff849170b50f05b94a87e7d3f2a62af6eaf507815e12e65377dd32846a"; /// Golden content-level fingerprint over every migration's rendered SQL. /// Bump it only when ADDING a migration file; a body change on an already /// applied migration is a defect, not a golden to refresh. const EXPECTED_SQL_FINGERPRINT: &str = - "4490b15ba9de27bf5b30a47e462dc3b71c061a47f738a5240cdc89b3abc4ff43"; + "fd8c85c3fd92bd6905715d221b665fe8d471d037f554b45d37c4a2742934dff7"; /// The migrations merged `v4.2-dev` already ships. Refinery keys /// `refinery_schema_history` by version and validates an applied migration's @@ -38,6 +38,11 @@ const MERGED_MIGRATION_VERSIONS: &[(i32, &str)] = &[ (7, "utxo_sweep_winner_height"), ]; +/// Versions another release line publishes and this one does not ship yet. +/// They stay free here until that line is merged forward, and then carry +/// exactly these names; anything else is a version clash with shipped data. +const RESERVED_FOR_OTHER_LINES: &[(i32, &str)] = &[(19, "asset_lock_operation_id")]; + /// Historical table names: V008 renames wallet metadata, and its typed /// conversion retains the pool tables through V010 before retiring them. /// Only the SQL history through V008 and the versioned conversion module @@ -96,6 +101,25 @@ fn merged_migration_versions_keep_their_shipped_names() { } } +/// A version another line ships is either still free here or holds that +/// line's migration. +/// +/// IF THIS FAILS: a migration on this branch took a version the 5.0 line +/// already uses. Renumber it past the reserved range. +#[test] +fn reserved_migration_versions_hold_only_their_owners_migrations() { + let embedded = mig::embedded_migrations(); + for (version, name) in RESERVED_FOR_OTHER_LINES { + if let Some((_, found)) = embedded.iter().find(|(v, _)| v == version) { + assert_eq!( + found.as_str(), + *name, + "version {version} belongs to `{name}` on another release line" + ); + } + } +} + /// The migration set's identity is pinned. #[test] fn identity_fingerprint_pinned() { diff --git a/packages/rs-platform-wallet-storage/tests/integration/sqlite_transaction_sweeps.rs b/packages/rs-platform-wallet-storage/tests/integration/sqlite_transaction_sweeps.rs index 53e4fa65026..43200970b2a 100644 --- a/packages/rs-platform-wallet-storage/tests/integration/sqlite_transaction_sweeps.rs +++ b/packages/rs-platform-wallet-storage/tests/integration/sqlite_transaction_sweeps.rs @@ -62,11 +62,12 @@ fn derive_address(conn: &rusqlite::Connection, w: &WalletId, account_index: u32, /// Build a `TransactionRecord` whose `transaction.input`/`.output` are the /// real, decodable fields `apply_sweep` reads back for its outpoint math — /// as opposed to `input_details`/`output_details`, which only cover the -/// wallet-relevant subset and are left empty here on purpose. -fn tx_record(txid: Txid, inputs: Vec, outputs: Vec) -> TransactionRecord { +/// wallet-relevant subset and are left empty here on purpose. `nonce` makes +/// otherwise identical fixture bodies distinct without overriding their hashes. +fn tx_record(nonce: u32, inputs: Vec, outputs: Vec) -> TransactionRecord { let inner = Transaction { version: 3, - lock_time: 0, + lock_time: nonce, input: inputs .into_iter() .map(|previous_output| TxIn { @@ -77,7 +78,7 @@ fn tx_record(txid: Txid, inputs: Vec, outputs: Vec) -> Transact output: outputs, special_transaction_payload: None, }; - let mut record = TransactionRecord::new( + TransactionRecord::new( inner, AccountType::Standard { index: 0, @@ -89,9 +90,7 @@ fn tx_record(txid: Txid, inputs: Vec, outputs: Vec) -> Transact Vec::new(), Vec::new(), 0, - ); - record.txid = txid; - record + ) } fn unspent(conn: &rusqlite::Connection, w: &WalletId) -> std::collections::BTreeSet { @@ -130,15 +129,15 @@ fn sweep_only_changeset_deletes_loser_row_and_its_outputs() { ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x01); - let loser_txid = Txid::from_byte_array([0x10; 32]); let loser = tx_record( - loser_txid, + 0x10, vec![], vec![TxOut { value: 5_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; let loser_output = OutPoint::new(loser_txid, 0); { @@ -233,26 +232,25 @@ fn the_released_set_frees_exactly_the_inputs_it_names() { let shared_input = OutPoint::new(funding_txid, 0); let exclusive_input = OutPoint::new(funding_txid, 1); - let loser_txid = Txid::from_byte_array([0x31; 32]); - let winner_txid = Txid::from_byte_array([0x32; 32]); - let loser = tx_record( - loser_txid, + 0x31, vec![shared_input, exclusive_input], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; // The winner only claimed the shared input. let winner = tx_record( - winner_txid, + 0x32, vec![shared_input], vec![TxOut { value: 900, script_pubkey: addr.script_pubkey(), }], ); + let winner_txid = winner.txid; let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -342,17 +340,16 @@ fn an_absent_winner_still_keeps_its_own_input_spent() { let taken_by_winner = OutPoint::new(funding_txid, 0); let loser_exclusive = OutPoint::new(funding_txid, 1); - let loser_txid = Txid::from_byte_array([0x41; 32]); - let unrecorded_winner_txid = Txid::from_byte_array([0x42; 32]); - let loser = tx_record( - loser_txid, + 0x41, vec![taken_by_winner, loser_exclusive], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; + let unrecorded_winner_txid = Txid::from_byte_array([0x42; 32]); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -429,20 +426,18 @@ fn a_released_coin_a_surviving_record_reclaims_stays_spent() { let funding_txid = Txid::from_byte_array([0x50; 32]); let freed_coin = OutPoint::new(funding_txid, 1); - let loser_txid = Txid::from_byte_array([0x51; 32]); - let winner_txid = Txid::from_byte_array([0x52; 32]); - let reclaimer_txid = Txid::from_byte_array([0x53; 32]); - let loser = tx_record( - loser_txid, + 0x51, vec![OutPoint::new(funding_txid, 0), freed_coin], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0x52; 32]); let reclaimer = tx_record( - reclaimer_txid, + 0x53, vec![freed_coin], vec![TxOut { value: 400, @@ -531,36 +526,34 @@ fn a_release_naming_a_coin_a_stored_finalized_record_claims_is_refused() { // Attacker-owned input shared by L and W — never ours, no row. let attacker_input = OutPoint::new(Txid::from_byte_array([0x9A; 32]), 0); - let finalized_txid = Txid::from_byte_array([0x91; 32]); - let loser_txid = Txid::from_byte_array([0x92; 32]); - let winner_txid = Txid::from_byte_array([0x93; 32]); - // F: chainlocked spender of `settled_coin`. Upstream keeps only its // txid from here on; this row keeps everything. let mut finalized = tx_record( - finalized_txid, + 0x91, vec![settled_coin], vec![TxOut { value: 400, script_pubkey: addr.script_pubkey(), }], ); - finalized.context = TransactionContext::InChainLockedBlock(BlockInfo::new( - 42, - dashcore::BlockHash::from_byte_array([0x9B; 32]), - 1_735_689_600, - )); - // L: arrives after F's pruning, pays this wallet, reuses F's input and // the attacker's. let loser = tx_record( - loser_txid, + 0x92, vec![settled_coin, attacker_input, losers_own_coin], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0x93; 32]); + + finalized.context = TransactionContext::InChainLockedBlock(BlockInfo::new( + 42, + dashcore::BlockHash::from_byte_array([0x9B; 32]), + 1_735_689_600, + )); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -662,21 +655,30 @@ fn a_stale_mempool_claimant_does_not_veto_an_authoritative_release() { let funding_txid = Txid::from_byte_array([0xA0; 32]); let coin = OutPoint::new(funding_txid, 0); - let stale_txid = Txid::from_byte_array([0xA1; 32]); - let loser_txid = Txid::from_byte_array([0xA2; 32]); - let winner_txid = Txid::from_byte_array([0xA3; 32]); - // M: a mempool spend of the coin, marked spent when recorded. It is // then evicted from the network's mempool without the wallet ever // hearing — its row simply goes stale. let stale = tx_record( - stale_txid, + 0xA1, vec![coin], vec![TxOut { value: 400, script_pubkey: addr.script_pubkey(), }], ); + // A fresh loser claims the same coin, and an authoritative sweep later + // frees it — upstream's word, computed from the wallet it actually + // holds. + let loser = tx_record( + 0xA2, + vec![coin], + vec![TxOut { + value: 300, + script_pubkey: addr.script_pubkey(), + }], + ); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0xA3; 32]); { let mut conn = persister.lock_conn_for_test(); @@ -705,17 +707,6 @@ fn a_stale_mempool_claimant_does_not_veto_an_authoritative_release() { let persister = SqlitePersister::open(SqlitePersisterConfig::new(&path)).unwrap(); let mut conn = persister.lock_conn_for_test(); - // A fresh loser claims the same coin, and an authoritative sweep later - // frees it — upstream's word, computed from the wallet it actually - // holds. - let loser = tx_record( - loser_txid, - vec![coin], - vec![TxOut { - value: 300, - script_pubkey: addr.script_pubkey(), - }], - ); { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { @@ -760,17 +751,16 @@ fn a_corrupt_stored_claimant_fails_the_sweep_round_closed() { let addr = p2pkh(0x0B); let funding_txid = Txid::from_byte_array([0xB0; 32]); let coin = OutPoint::new(funding_txid, 0); - let loser_txid = Txid::from_byte_array([0xB1; 32]); - let winner_txid = Txid::from_byte_array([0xB2; 32]); - let loser = tx_record( - loser_txid, + 0xB1, vec![coin], vec![TxOut { value: 300, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0xB2; 32]); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -787,7 +777,7 @@ fn a_corrupt_stored_claimant_fails_the_sweep_round_closed() { // A claimant row whose typed key is not 32 bytes. The blob is a real, // decodable record so the failure is attributable to the key alone. - let honest = tx_record(Txid::from_byte_array([0xB3; 32]), vec![coin], Vec::new()); + let honest = tx_record(0xB3, vec![coin], Vec::new()); let honest_blob = blob::encode(&honest).unwrap(); conn.execute( "INSERT INTO core_transactions \ @@ -875,19 +865,18 @@ fn a_swept_key_disagreeing_with_its_stored_record_fails_the_round_closed() { // F's coin — consumed on chain by a settled record, must never come back. let settled_coin = OutPoint::new(funding_txid, 0); - let finalized_txid = Txid::from_byte_array([0xC5; 32]); - let loser_txid = Txid::from_byte_array([0xC1; 32]); - let winner_txid = Txid::from_byte_array([0xC2; 32]); - // F: chainlocked spender of `settled_coin`. let mut finalized = tx_record( - finalized_txid, + 0xC5, vec![settled_coin], vec![TxOut { value: 300, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = Txid::from_byte_array([0xC1; 32]); + let winner_txid = Txid::from_byte_array([0xC2; 32]); + finalized.context = TransactionContext::InChainLockedBlock(BlockInfo::new( 42, dashcore::BlockHash::from_byte_array([0xCB; 32]), @@ -996,15 +985,15 @@ fn sweeping_a_transaction_deletes_its_instant_lock() { ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x05); - let loser_txid = Txid::from_byte_array([0x60; 32]); let loser = tx_record( - loser_txid, + 0x60, vec![], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -1081,27 +1070,26 @@ fn a_later_sweep_keeping_a_coin_spent_overrides_an_earlier_release() { let funding_txid = Txid::from_byte_array([0x70; 32]); let contested = OutPoint::new(funding_txid, 0); - let first_loser = Txid::from_byte_array([0x71; 32]); - let second_loser = Txid::from_byte_array([0x72; 32]); - let first = tx_record( - first_loser, + 0x71, vec![contested], vec![TxOut { value: 400, script_pubkey: addr.script_pubkey(), }], ); + let first_loser = first.txid; // The transaction that took the freed coin, and that the second sweep // removes. It is a loser too, so it is not a surviving claim. let second = tx_record( - second_loser, + 0x72, vec![contested], vec![TxOut { value: 300, script_pubkey: addr.script_pubkey(), }], ); + let second_loser = second.txid; let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -1174,17 +1162,16 @@ fn a_held_input_with_no_utxo_row_survives_restart_and_stays_spent_when_funded() let funding_txid = Txid::from_byte_array([0x80; 32]); let unfunded_input = OutPoint::new(funding_txid, 0); - let loser_txid = Txid::from_byte_array([0x81; 32]); - let winner_txid = Txid::from_byte_array([0x82; 32]); - let loser = tx_record( - loser_txid, + 0x81, vec![unfunded_input], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0x82; 32]); { let mut conn = persister.lock_conn_for_test(); @@ -1272,26 +1259,25 @@ fn a_chained_sweep_before_funding_still_frees_an_earlier_tombstone_on_release() let unfunded_input = OutPoint::new(funding_txid, 0); let funded_input = OutPoint::new(funding_txid, 1); - let first_loser = Txid::from_byte_array([0x91; 32]); // L - let second_loser = Txid::from_byte_array([0x92; 32]); // W - let final_winner = Txid::from_byte_array([0x93; 32]); // X - let l = tx_record( - first_loser, + 0x91, vec![unfunded_input], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let first_loser = l.txid; let w_record = tx_record( - second_loser, + 0x92, vec![unfunded_input, funded_input], vec![TxOut { value: 900, script_pubkey: addr.script_pubkey(), }], ); + let second_loser = w_record.txid; + let final_winner = Txid::from_byte_array([0x93; 32]); // X let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); @@ -1412,26 +1398,25 @@ fn a_chained_sweep_before_funding_repoints_an_earlier_tombstone_to_the_new_winne let funding_txid = Txid::from_byte_array([0xA0; 32]); let unfunded_input = OutPoint::new(funding_txid, 0); - let first_loser = Txid::from_byte_array([0xA1; 32]); // L - let second_loser = Txid::from_byte_array([0xA2; 32]); // W - let final_winner = Txid::from_byte_array([0xA3; 32]); // X - let l = tx_record( - first_loser, + 0xA1, vec![unfunded_input], vec![TxOut { value: 1_000, script_pubkey: addr.script_pubkey(), }], ); + let first_loser = l.txid; let w_record = tx_record( - second_loser, + 0xA2, vec![unfunded_input], vec![TxOut { value: 900, script_pubkey: addr.script_pubkey(), }], ); + let second_loser = w_record.txid; + let final_winner = Txid::from_byte_array([0xA3; 32]); // X { let mut conn = persister.lock_conn_for_test(); @@ -1556,8 +1541,10 @@ fn a_multi_wallet_chained_sweep_before_funding_reconciles_each_wallets_own_tombs // observed in either wallet until the very end. let p1 = OutPoint::new(funding_txid, 0); let p2 = OutPoint::new(funding_txid, 1); - let shared_loser = Txid::from_byte_array([0x53; 32]); // L - let shared_winner = Txid::from_byte_array([0x54; 32]); // W + let shared_loser_record = tx_record(0x53, vec![p1, p2], vec![]); + let shared_loser = shared_loser_record.txid; + let shared_winner_record = tx_record(0x54, vec![p1, p2], vec![]); + let shared_winner = shared_winner_record.txid; let final_winner = Txid::from_byte_array([0x55; 32]); // X // The raw transactions are the same for both wallets — a record is the @@ -1568,7 +1555,7 @@ fn a_multi_wallet_chained_sweep_before_funding_reconciles_each_wallets_own_tombs derive_address(&conn, w, 0, if w == &w1 { &addr1 } else { &addr2 }); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(shared_loser, vec![p1, p2], vec![])], + records: vec![shared_loser_record.clone()], ..Default::default() }; core_state::apply(&tx, w, &cs).unwrap(); @@ -1599,7 +1586,7 @@ fn a_multi_wallet_chained_sweep_before_funding_reconciles_each_wallets_own_tombs let mut conn = persister.lock_conn_for_test(); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(shared_winner, vec![p1, p2], vec![])], + records: vec![shared_winner_record.clone()], ..Default::default() }; core_state::apply(&tx, w, &cs).unwrap(); @@ -1732,29 +1719,33 @@ fn sweep_of_a_shared_loser_txid_is_independent_per_wallet() { let addr1 = p2pkh(0x31); let addr2 = p2pkh(0x32); - let funding_txid = Txid::from_byte_array([0x30; 32]); + let funding = tx_record( + 0x30, + vec![], + [&addr1, &addr2] + .into_iter() + .map(|addr| TxOut { + value: 100_000, + script_pubkey: addr.script_pubkey(), + }) + .collect(), + ); + let funding_txid = funding.txid; // Same txid recorded independently in both wallets' storage — as two // wallets sharing one on-chain transaction each would. - let loser_txid = Txid::from_byte_array([0x33; 32]); - let winner_txid = Txid::from_byte_array([0x34; 32]); let coin = OutPoint::new(funding_txid, 0); + let second_coin = OutPoint::new(funding_txid, 1); + let loser = tx_record(0x33, vec![coin, second_coin], vec![]); + let loser_txid = loser.txid; + let winner_txid = Txid::from_byte_array([0x34; 32]); - for (w, addr) in [(&w1, &addr1), (&w2, &addr2)] { + for (vout, (w, addr)) in [(&w1, &addr1), (&w2, &addr2)].into_iter().enumerate() { let mut conn = persister.lock_conn_for_test(); derive_address(&conn, w, 0, addr); let tx = conn.transaction().unwrap(); - let funding = tx_record( - funding_txid, - vec![], - vec![TxOut { - value: 100_000, - script_pubkey: addr.script_pubkey(), - }], - ); - let loser = tx_record(loser_txid, vec![coin], vec![]); let cs = CoreChangeSet { - records: vec![funding, loser], - new_utxos: vec![make_utxo(addr, funding_txid, 0, 100_000)], + records: vec![funding.clone(), loser.clone()], + new_utxos: vec![make_utxo(addr, funding_txid, vout as u32, 100_000)], ..Default::default() }; core_state::apply(&tx, w, &cs).unwrap(); @@ -1803,7 +1794,7 @@ fn sweep_of_a_shared_loser_txid_is_independent_per_wallet() { untouched by wallet 1's sweep" ); assert!( - row_exists(&conn, &w2, &coin), + row_exists(&conn, &w2, &second_coin), "wallet 2's coin is unaffected — it has not swept yet" ); } @@ -1840,13 +1831,16 @@ fn sweep_of_a_shared_loser_txid_is_independent_per_wallet() { ); assert!( - row_exists(&conn, &w2, &coin), + row_exists(&conn, &w2, &second_coin), "wallet 2 released nothing, so its coin stays held with a row of its own" ); let (spent, spent_in_txid): (i64, Option>) = conn .query_row( "SELECT spent, spent_in_txid FROM core_utxos WHERE wallet_id = ?1 AND outpoint = ?2", - params![w2.as_slice(), &blob::encode_outpoint(&coin).unwrap()[..]], + params![ + w2.as_slice(), + &blob::encode_outpoint(&second_coin).unwrap()[..] + ], |row| Ok((row.get(0)?, row.get(1)?)), ) .unwrap(); @@ -1886,23 +1880,26 @@ fn sweep_deletion_is_durable_even_when_the_other_wallets_callback_never_arrives( let addr2 = p2pkh(0x42); // Same loser txid recorded independently by both wallets, each with an // output of its own — the "phantom money" the blocking finding is about. - let loser_txid = Txid::from_byte_array([0x43; 32]); + let loser = tx_record( + 0x43, + vec![], + [&addr1, &addr2] + .into_iter() + .map(|addr| TxOut { + value: 60_000, + script_pubkey: addr.script_pubkey(), + }) + .collect(), + ); + let loser_txid = loser.txid; let winner_txid = Txid::from_byte_array([0x44; 32]); for (w, addr) in [(&w1, &addr1), (&w2, &addr2)] { let mut conn = persister.lock_conn_for_test(); derive_address(&conn, w, 0, addr); let tx = conn.transaction().unwrap(); - let loser = tx_record( - loser_txid, - vec![], - vec![TxOut { - value: 60_000, - script_pubkey: addr.script_pubkey(), - }], - ); let cs = CoreChangeSet { - records: vec![loser], + records: vec![loser.clone()], ..Default::default() }; core_state::apply(&tx, w, &cs).unwrap(); @@ -1998,7 +1995,15 @@ fn a_record_reinstating_a_swept_txid_in_a_later_round_is_accepted_and_durable() ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x51); - let txid = Txid::from_byte_array([0x53; 32]); + let txid_record = tx_record( + 0x53, + vec![], + vec![TxOut { + value: 45_000, + script_pubkey: addr.script_pubkey(), + }], + ); + let txid = txid_record.txid; let winner_txid = Txid::from_byte_array([0x54; 32]); let output = OutPoint::new(txid, 0); @@ -2007,14 +2012,7 @@ fn a_record_reinstating_a_swept_txid_in_a_later_round_is_accepted_and_durable() let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); - let record = tx_record( - txid, - vec![], - vec![TxOut { - value: 45_000, - script_pubkey: addr.script_pubkey(), - }], - ); + let record = txid_record.clone(); let cs = CoreChangeSet { records: vec![record], new_utxos: vec![make_utxo(&addr, txid, 0, 45_000)], @@ -2068,14 +2066,7 @@ fn a_record_reinstating_a_swept_txid_in_a_later_round_is_accepted_and_durable() { let mut conn = persister.lock_conn_for_test(); let tx = conn.transaction().unwrap(); - let record = tx_record( - txid, - vec![], - vec![TxOut { - value: 45_000, - script_pubkey: addr.script_pubkey(), - }], - ); + let record = txid_record.clone(); let cs = CoreChangeSet { records: vec![record], new_utxos: vec![make_utxo(&addr, txid, 0, 45_000)], @@ -2133,7 +2124,8 @@ fn a_release_applies_even_when_the_swept_txid_has_no_row() { let addr = p2pkh(0x31); let funding_txid = Txid::from_byte_array([0x30; 32]); let p = OutPoint::new(funding_txid, 0); - let loser_txid = Txid::from_byte_array([0x31; 32]); // L + let loser_txid_record = tx_record(0x31, vec![p], vec![]); + let loser_txid = loser_txid_record.txid; let winner_txid = Txid::from_byte_array([0x32; 32]); // W — never recorded let final_winner = Txid::from_byte_array([0x33; 32]); // X @@ -2144,7 +2136,7 @@ fn a_release_applies_even_when_the_swept_txid_has_no_row() { derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(loser_txid, vec![p], vec![])], + records: vec![loser_txid_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -2235,10 +2227,19 @@ fn a_batch_sweeping_parent_and_child_leaves_no_placeholder_for_the_parents_outpu ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x41); - let parent_txid = Txid::from_byte_array([0x40; 32]); // L - let child_txid = Txid::from_byte_array([0x41; 32]); // C - let winner_txid = Txid::from_byte_array([0x42; 32]); // W + let parent_txid_record = tx_record( + 0x40, + vec![], + vec![TxOut { + value: 5_000, + script_pubkey: addr.script_pubkey(), + }], + ); + let parent_txid = parent_txid_record.txid; let parent_output = OutPoint::new(parent_txid, 0); + let child_txid_record = tx_record(0x41, vec![parent_output], vec![]); + let child_txid = child_txid_record.txid; + let winner_txid = Txid::from_byte_array([0x42; 32]); // W // L pays us and is funded; C spends L's output. { @@ -2246,17 +2247,7 @@ fn a_batch_sweeping_parent_and_child_leaves_no_placeholder_for_the_parents_outpu derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![ - tx_record( - parent_txid, - vec![], - vec![TxOut { - value: 5_000, - script_pubkey: addr.script_pubkey(), - }], - ), - tx_record(child_txid, vec![parent_output], vec![]), - ], + records: vec![parent_txid_record.clone(), child_txid_record.clone()], new_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2294,14 +2285,7 @@ fn a_batch_sweeping_parent_and_child_leaves_no_placeholder_for_the_parents_outpu let mut conn = persister.lock_conn_for_test(); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record( - parent_txid, - vec![], - vec![TxOut { - value: 5_000, - script_pubkey: addr.script_pubkey(), - }], - )], + records: vec![parent_txid_record.clone()], new_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2332,10 +2316,19 @@ fn a_co_swept_parent_with_no_row_still_has_its_output_removed() { ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x51); - let parent_txid = Txid::from_byte_array([0x50; 32]); // P — record lost - let child_txid = Txid::from_byte_array([0x51; 32]); // C - let winner_txid = Txid::from_byte_array([0x52; 32]); // W + let parent_txid_record = tx_record( + 0x50, + vec![], + vec![TxOut { + value: 5_000, + script_pubkey: addr.script_pubkey(), + }], + ); + let parent_txid = parent_txid_record.txid; let parent_output = OutPoint::new(parent_txid, 0); + let child_txid_record = tx_record(0x51, vec![parent_output], vec![]); + let child_txid = child_txid_record.txid; + let winner_txid = Txid::from_byte_array([0x52; 32]); // W // P's record round was wiped, but its funded output row and C's record // both persisted. @@ -2344,7 +2337,7 @@ fn a_co_swept_parent_with_no_row_still_has_its_output_removed() { derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(child_txid, vec![parent_output], vec![])], + records: vec![child_txid_record.clone()], new_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2397,14 +2390,7 @@ fn a_co_swept_parent_with_no_row_still_has_its_output_removed() { let mut conn = persister.lock_conn_for_test(); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record( - parent_txid, - vec![], - vec![TxOut { - value: 5_000, - script_pubkey: addr.script_pubkey(), - }], - )], + records: vec![parent_txid_record.clone()], new_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2435,10 +2421,19 @@ fn a_co_swept_parent_known_only_through_the_childs_spend_is_still_removed() { ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x61); - let parent_txid = Txid::from_byte_array([0x60; 32]); // P — never recorded - let child_txid = Txid::from_byte_array([0x61; 32]); // C - let winner_txid = Txid::from_byte_array([0x62; 32]); // W + let parent_txid_record = tx_record( + 0x60, + vec![], + vec![TxOut { + value: 5_000, + script_pubkey: addr.script_pubkey(), + }], + ); + let parent_txid = parent_txid_record.txid; let parent_output = OutPoint::new(parent_txid, 0); + let child_txid_record = tx_record(0x61, vec![parent_output], vec![]); + let child_txid = child_txid_record.txid; + let winner_txid = Txid::from_byte_array([0x62; 32]); // W // C arrives in order, spending P:0 — the spent-utxos apply writes the // synthetic spent-only row because no funded row exists. @@ -2447,7 +2442,7 @@ fn a_co_swept_parent_known_only_through_the_childs_spend_is_still_removed() { derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(child_txid, vec![parent_output], vec![])], + records: vec![child_txid_record.clone()], spent_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2494,14 +2489,7 @@ fn a_co_swept_parent_known_only_through_the_childs_spend_is_still_removed() { let mut conn = persister.lock_conn_for_test(); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record( - parent_txid, - vec![], - vec![TxOut { - value: 5_000, - script_pubkey: addr.script_pubkey(), - }], - )], + records: vec![parent_txid_record.clone()], new_utxos: vec![make_utxo(&addr, parent_txid, 0, 5_000)], ..Default::default() }; @@ -2577,14 +2565,16 @@ fn seed_tombstone( conn: &mut rusqlite::Connection, w: &WalletId, input: OutPoint, - loser: Txid, + loser_nonce: u32, winner: Txid, winner_mined_height: Option, ) { + let record = tx_record(loser_nonce, vec![input], vec![]); + let loser = record.txid; { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(loser, vec![input], vec![])], + records: vec![record], ..Default::default() }; core_state::apply(&tx, w, &cs).unwrap(); @@ -2618,12 +2608,12 @@ fn a_never_materialised_tombstone_is_collected_at_finality_and_not_before() { ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x50; 32]), 0); - let loser = Txid::from_byte_array([0x51; 32]); + let loser_nonce = 0x51; let winner = Txid::from_byte_array([0x52; 32]); let mut conn = persister.lock_conn_for_test(); apply_heights(&mut conn, &w, 100); - seed_tombstone(&mut conn, &w, p, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, Some(WINNER_HEIGHT)); assert_eq!( utxo_row_state(&conn, &w, &p), @@ -2662,11 +2652,11 @@ fn a_block_context_tombstone_outlives_unrelated_advancement_below_its_winners_he ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x62; 32]), 0); - let loser = Txid::from_byte_array([0x63; 32]); + let loser_nonce = 0x63; let winner = Txid::from_byte_array([0x64; 32]); let mut conn = persister.lock_conn_for_test(); - seed_tombstone(&mut conn, &w, p, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, Some(WINNER_HEIGHT)); // Chainlocks race ahead by thousands of blocks; the filter scan has // only reached one block short of the winner. @@ -2726,9 +2716,9 @@ fn a_mempool_context_sweep_preserves_an_unstamped_tombstone() { // held-but-unfunded input. for i in 0u8..3 { let p = OutPoint::new(Txid::from_byte_array([0x70 + i; 32]), 0); - let loser = Txid::from_byte_array([0x80 + i; 32]); + let loser_nonce = u32::from(0x80 + i); let winner = Txid::from_byte_array([0x90 + i; 32]); - seed_tombstone(&mut conn, &w, p, loser, winner, None); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, None); assert_eq!( utxo_row_state(&conn, &w, &p), Some((true, true, None)), @@ -2769,7 +2759,7 @@ fn a_mempool_context_sweep_still_spend_marks_a_materialised_coin() { let addr = p2pkh(0x65); let funding_txid = Txid::from_byte_array([0x66; 32]); let p = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0x67; 32]); + let loser_nonce = 0x67; let winner = Txid::from_byte_array([0x68; 32]); let mut conn = persister.lock_conn_for_test(); @@ -2783,7 +2773,7 @@ fn a_mempool_context_sweep_still_spend_marks_a_materialised_coin() { core_state::apply(&tx, &w, &cs).unwrap(); tx.commit().unwrap(); } - seed_tombstone(&mut conn, &w, p, loser, winner, None); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, None); assert_eq!( utxo_row_state(&conn, &w, &p), Some((true, false, None)), @@ -2814,13 +2804,13 @@ fn a_funding_output_arriving_after_a_mempool_sweep_and_restart_lands_spent() { let addr = p2pkh(0x69); let funding_txid = Txid::from_byte_array([0x6A; 32]); let p = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0x6B; 32]); + let loser_nonce = 0x6B; let winner = Txid::from_byte_array([0x6C; 32]); { let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); - seed_tombstone(&mut conn, &w, p, loser, winner, None); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, None); } // Restart. drop(persister); @@ -2873,7 +2863,7 @@ fn a_tombstone_is_never_collected_without_a_persisted_chainlock() { ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x53; 32]), 0); - let loser = Txid::from_byte_array([0x54; 32]); + let loser_nonce = 0x54; let winner = Txid::from_byte_array([0x55; 32]); let mut conn = persister.lock_conn_for_test(); @@ -2887,7 +2877,7 @@ fn a_tombstone_is_never_collected_without_a_persisted_chainlock() { core_state::apply(&tx, &w, &cs).unwrap(); tx.commit().unwrap(); } - seed_tombstone(&mut conn, &w, p, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, Some(WINNER_HEIGHT)); { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { @@ -2926,13 +2916,13 @@ fn a_materialised_claim_is_never_collected() { let addr = p2pkh(0x61); let funding_txid = Txid::from_byte_array([0x56; 32]); let p = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0x57; 32]); + let loser_nonce = 0x57; let winner = Txid::from_byte_array([0x58; 32]); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); apply_heights(&mut conn, &w, 100); - seed_tombstone(&mut conn, &w, p, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, Some(WINNER_HEIGHT)); assert_eq!( utxo_row_state(&conn, &w, &p), Some((true, true, Some(i64::from(WINNER_HEIGHT)))), @@ -2989,14 +2979,14 @@ fn a_release_frees_a_materialised_claim_in_place() { let addr = p2pkh(0x62); let funding_txid = Txid::from_byte_array([0x71; 32]); let p = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0x72; 32]); + let loser_nonce = 0x72; let winner = Txid::from_byte_array([0x73; 32]); let final_winner = Txid::from_byte_array([0x74; 32]); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); apply_heights(&mut conn, &w, 100); - seed_tombstone(&mut conn, &w, p, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, Some(WINNER_HEIGHT)); // The funding output classifies: real data, stamp cleared, still spent. { @@ -3067,13 +3057,13 @@ fn a_tombstone_without_a_winner_height_is_never_collected() { ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x59; 32]), 0); - let loser = Txid::from_byte_array([0x5A; 32]); + let loser_nonce = 0x5A; let winner = Txid::from_byte_array([0x5B; 32]); let mut conn = persister.lock_conn_for_test(); // The real writer: an IS-context sweep of a loser whose funding row // never arrived. - seed_tombstone(&mut conn, &w, p, loser, winner, None); + seed_tombstone(&mut conn, &w, p, loser_nonce, winner, None); // Two rounds, not one: a back-filling collector (the rejected design) // would stamp the row on the first round and collect it on the second. @@ -3097,8 +3087,9 @@ fn a_repointed_tombstone_is_restamped_to_the_later_winners_height() { ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x5C; 32]), 0); - let first_loser = Txid::from_byte_array([0x5D; 32]); - let second_loser = Txid::from_byte_array([0x5E; 32]); + let first_loser_nonce = 0x5D; + let second_loser_record = tx_record(0x5E, vec![p], vec![]); + let second_loser = second_loser_record.txid; let final_winner = Txid::from_byte_array([0x5F; 32]); let mut conn = persister.lock_conn_for_test(); @@ -3106,7 +3097,7 @@ fn a_repointed_tombstone_is_restamped_to_the_later_winners_height() { &mut conn, &w, p, - first_loser, + first_loser_nonce, second_loser, Some(WINNER_HEIGHT), ); @@ -3121,7 +3112,7 @@ fn a_repointed_tombstone_is_restamped_to_the_later_winners_height() { { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(second_loser, vec![p], vec![])], + records: vec![second_loser_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -3161,8 +3152,9 @@ fn a_mempool_repointed_tombstone_keeps_its_block_context_stamp() { ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x6D; 32]), 0); - let first_loser = Txid::from_byte_array([0x6E; 32]); - let second_loser = Txid::from_byte_array([0x6F; 32]); + let first_loser_nonce = 0x6E; + let second_loser_record = tx_record(0x6F, vec![p], vec![]); + let second_loser = second_loser_record.txid; let final_winner = Txid::from_byte_array([0x71; 32]); let mut conn = persister.lock_conn_for_test(); @@ -3170,7 +3162,7 @@ fn a_mempool_repointed_tombstone_keeps_its_block_context_stamp() { &mut conn, &w, p, - first_loser, + first_loser_nonce, second_loser, Some(WINNER_HEIGHT), ); @@ -3180,7 +3172,7 @@ fn a_mempool_repointed_tombstone_keeps_its_block_context_stamp() { { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(second_loser, vec![p], vec![])], + records: vec![second_loser_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -3229,13 +3221,14 @@ fn an_unstamped_tombstone_restamped_by_a_block_context_sweep_becomes_collectible ensure_wallet_meta(&persister, &w); let p = OutPoint::new(Txid::from_byte_array([0x72; 32]), 0); - let first_loser = Txid::from_byte_array([0x73; 32]); - let second_loser = Txid::from_byte_array([0x74; 32]); + let first_loser_nonce = 0x73; + let second_loser_record = tx_record(0x74, vec![p], vec![]); + let second_loser = second_loser_record.txid; let final_winner = Txid::from_byte_array([0x75; 32]); let mut conn = persister.lock_conn_for_test(); // IS-context sweep: the hold lands unstamped. - seed_tombstone(&mut conn, &w, p, first_loser, second_loser, None); + seed_tombstone(&mut conn, &w, p, first_loser_nonce, second_loser, None); assert_eq!( utxo_row_state(&conn, &w, &p), Some((true, true, None)), @@ -3247,7 +3240,7 @@ fn an_unstamped_tombstone_restamped_by_a_block_context_sweep_becomes_collectible { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(second_loser, vec![p], vec![])], + records: vec![second_loser_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -3297,7 +3290,8 @@ fn a_materialised_coin_the_wallet_re_delivers_unspent_is_released_from_its_hold( let addr = p2pkh(0x0A); let funding_txid = Txid::from_byte_array([0xA0; 32]); let x = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0xA1; 32]); + let loser_record = tx_record(0xA1, vec![x], vec![]); + let loser = loser_record.txid; let winner = Txid::from_byte_array([0xA2; 32]); let mut conn = persister.lock_conn_for_test(); @@ -3306,7 +3300,7 @@ fn a_materialised_coin_the_wallet_re_delivers_unspent_is_released_from_its_hold( { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record(loser, vec![x], vec![])], + records: vec![loser_record.clone()], new_utxos: vec![make_utxo(&addr, funding_txid, 0, 1_000)], ..Default::default() }; @@ -3379,8 +3373,10 @@ fn a_placeholder_stays_held_after_the_trigger_nulls_its_link() { let funding_txid = Txid::from_byte_array([0xB0; 32]); let x = OutPoint::new(funding_txid, 0); // unfunded, held let y = OutPoint::new(Txid::from_byte_array([0xB1; 32]), 0); // W and W2 conflict here - let loser = Txid::from_byte_array([0xB2; 32]); // L spends X - let winner = Txid::from_byte_array([0xB3; 32]); // W spends Y, sweeps L + let loser_record = tx_record(0xB2, vec![x], vec![]); + let loser = loser_record.txid; + let winner_record = tx_record(0xB3, vec![y], vec![]); + let winner = winner_record.txid; let second_winner = Txid::from_byte_array([0xB4; 32]); // W2 spends Y, sweeps W let mut conn = persister.lock_conn_for_test(); @@ -3389,10 +3385,7 @@ fn a_placeholder_stays_held_after_the_trigger_nulls_its_link() { { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![ - tx_record(loser, vec![x], vec![]), - tx_record(winner, vec![y], vec![]), - ], + records: vec![loser_record.clone(), winner_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -3481,18 +3474,17 @@ fn a_placeholder_with_a_nulled_link_is_still_collected_at_its_stamp() { let x = OutPoint::new(Txid::from_byte_array([0xC0; 32]), 0); let y = OutPoint::new(Txid::from_byte_array([0xC1; 32]), 0); - let loser = Txid::from_byte_array([0xC2; 32]); - let winner = Txid::from_byte_array([0xC3; 32]); + let loser_record = tx_record(0xC2, vec![x], vec![]); + let loser = loser_record.txid; + let winner_record = tx_record(0xC3, vec![y], vec![]); + let winner = winner_record.txid; let second_winner = Txid::from_byte_array([0xC4; 32]); let mut conn = persister.lock_conn_for_test(); { let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![ - tx_record(loser, vec![x], vec![]), - tx_record(winner, vec![y], vec![]), - ], + records: vec![loser_record.clone(), winner_record.clone()], ..Default::default() }; core_state::apply(&tx, &w, &cs).unwrap(); @@ -3536,12 +3528,12 @@ fn a_spent_delivery_materialises_a_held_placeholder_out_of_the_collectors_reach( let addr = p2pkh(0x0D); let funding_txid = Txid::from_byte_array([0xD0; 32]); let x = OutPoint::new(funding_txid, 0); - let loser = Txid::from_byte_array([0xD1; 32]); + let loser_nonce = 0xD1; let winner = Txid::from_byte_array([0xD2; 32]); let mut conn = persister.lock_conn_for_test(); derive_address(&conn, &w, 0, &addr); - seed_tombstone(&mut conn, &w, x, loser, winner, Some(WINNER_HEIGHT)); + seed_tombstone(&mut conn, &w, x, loser_nonce, winner, Some(WINNER_HEIGHT)); assert_eq!( utxo_row_state(&conn, &w, &x), Some((true, true, Some(i64::from(WINNER_HEIGHT)))), @@ -3661,7 +3653,15 @@ fn a_record_and_its_sweep_in_one_round_end_with_the_transaction_gone() { ensure_wallet_meta(&persister, &w); let addr = p2pkh(0x0F); - let loser = Txid::from_byte_array([0xF0; 32]); + let loser_record = tx_record( + 0xF0, + vec![], + vec![TxOut { + value: 1_000, + script_pubkey: addr.script_pubkey(), + }], + ); + let loser = loser_record.txid; let winner = Txid::from_byte_array([0xF1; 32]); let loser_output = OutPoint::new(loser, 0); @@ -3669,14 +3669,7 @@ fn a_record_and_its_sweep_in_one_round_end_with_the_transaction_gone() { derive_address(&conn, &w, 0, &addr); let tx = conn.transaction().unwrap(); let cs = CoreChangeSet { - records: vec![tx_record( - loser, - vec![], - vec![TxOut { - value: 1_000, - script_pubkey: addr.script_pubkey(), - }], - )], + records: vec![loser_record.clone()], new_utxos: vec![make_utxo(&addr, loser, 0, 1_000)], sweeps: vec![SweepBatch { txids: vec![loser], @@ -3717,7 +3710,7 @@ fn should_restore_used_addresses_without_decoding_sweep_placeholders() { &mut conn, &w, input, - Txid::from_byte_array([0x92; 32]), + 0x92, Txid::from_byte_array([0x93; 32]), None, ); diff --git a/packages/rs-platform-wallet/src/changeset/changeset.rs b/packages/rs-platform-wallet/src/changeset/changeset.rs index 0611158faf1..dc3b871742a 100644 --- a/packages/rs-platform-wallet/src/changeset/changeset.rs +++ b/packages/rs-platform-wallet/src/changeset/changeset.rs @@ -36,6 +36,7 @@ use key_wallet::{AddressInfo, Network, PlatformP2PKHAddress, Utxo}; use crate::changeset::dashpay_backfill::DashPayBackfillRecord; use crate::changeset::identity_scan_state::IdentityScanStateEntry; +use crate::changeset::wallet_accounting::apply_wallet_accounting; use crate::wallet::platform_wallet::WalletId; use dpp::balances::credits::Credits; @@ -444,9 +445,6 @@ impl HighestUsedIndexes { /// −0.005 stored for a −2.61920199 spend). /// /// The fold, per txid group of 2+ records: -/// - `net_amount` — the SUM of the slices: each account's -/// `received − spent` over disjoint detail sets, so the sum is the -/// wallet's `Σreceived − Σspent` by construction. /// - `input_details` / `output_details` — the union (deduped by input /// index / output index): the slices are disjoint per account, and the /// union is exactly the wallet-relevant view downstream consumers @@ -454,15 +452,7 @@ impl HighestUsedIndexes { /// - `fee` — the first `Some` (only the funding account's record carries /// one, and disjoint accounts cannot disagree); left `None` when no /// record knew it. -/// - `direction` — recomputed over the MERGED details with the same rule -/// upstream applies per account (`record_transaction`): `CoinJoin` -/// transaction type wins outright; otherwise no `Sent` output + our -/// inputs + our outputs → `Internal` (a cross-account move whose -/// account-local slices said `Outgoing`/`Incoming` is, wallet-level, a -/// self-transfer); otherwise our inputs → `Outgoing`, else `Incoming`. -/// Deriving from the net's sign instead erased `Internal` and -/// `CoinJoin`: an internal transfer nets −fee and would relabel -/// `Outgoing`. +/// - `net_amount` / `direction` — see the wallet-level pass below. /// - `context` — the most advanced in the group (`Mempool` < /// `InstantSend` < `InBlock` < `InChainLockedBlock`), so a group mixing /// a stale mempool observation with a confirmed one keeps the @@ -472,6 +462,22 @@ impl HighestUsedIndexes { /// input details) so the row's account attribution names the spender, /// else the first record. /// +/// Then EVERY record — folded or single — gets its wallet-level +/// `net_amount` and `direction` from its details via +/// [`apply_wallet_accounting`], the rule the SQLite repair also applies, +/// so a row never changes accounting when storage repairs it: +/// - `net_amount` is `Σ owned outputs − Σ owned inputs`. Over +/// detail-bearing slices that equals the sum of their nets, but a +/// keys-account marker's `+credit` (Platform credits, not Core funds) +/// stays out: an asset lock nets `−(credit + fee)`, not `−fee`. +/// - `direction` is [`wallet_direction`](super::wallet_direction): account-local slices that said +/// `Outgoing`/`Incoming` for a cross-account move become `Internal`, and +/// an asset lock with no change is `Internal` like one with change. +/// Deriving direction from the net's sign instead would erase +/// `Internal` and `CoinJoin`. +/// - A group or record with no details at all (keys markers only) keeps +/// upstream's net and direction. +/// /// Order-preserving for untouched records; a fold lands at the group's /// FIRST position (`group[0]`) regardless of which record supplied the /// funding metadata, so unrelated records between two slices never move @@ -479,8 +485,16 @@ impl HighestUsedIndexes { /// reach here (filtered at projection — see /// `core_bridge::is_contact_watch_only`). pub(crate) fn fold_same_txid_records(records: &mut Vec) { - use key_wallet::managed_account::transaction_record::{OutputRole, TransactionDirection}; - use key_wallet::transaction_checking::transaction_router::TransactionType; + fold_groups(records); + for record in records.iter_mut() { + apply_wallet_accounting(record); + } +} + +/// The per-txid merge of [`fold_same_txid_records`], before the +/// wallet-level accounting pass. +fn fold_groups(records: &mut Vec) { + use key_wallet::managed_account::transaction_record::OutputRole; if records.len() < 2 { return; @@ -568,30 +582,10 @@ pub(crate) fn fold_same_txid_records(records: &mut Vec) { drop_idx.insert(base_pos); let first_pos = group[0]; drop_idx.remove(&first_pos); + // The slice sum stands only for a detail-less group (keys + // markers alone); `apply_wallet_accounting` recomputes net and + // direction from the merged details otherwise. merged.net_amount = net; - // Wallet-level direction over the merged details — same rule - // upstream applies per account (see the doc comment). The sign - // of the net cannot express `Internal` or `CoinJoin`. - merged.direction = if merged.transaction_type == TransactionType::CoinJoin { - TransactionDirection::CoinJoin - } else { - let has_inputs = !merged.input_details.is_empty(); - let has_sent = merged - .output_details - .iter() - .any(|d| d.role == OutputRole::Sent); - let has_our_outputs = merged - .output_details - .iter() - .any(|d| matches!(d.role, OutputRole::Received | OutputRole::Change)); - if !has_sent && has_inputs && has_our_outputs { - TransactionDirection::Internal - } else if has_inputs { - TransactionDirection::Outgoing - } else { - TransactionDirection::Incoming - } - }; folded.insert(first_pos, merged); } @@ -646,6 +640,33 @@ fn coalesce_newest_wins( } } +/// Keep one record per `(txid, account_type)` before folding account +/// contributions. The record with the higher [`context_rank`] wins +/// wholesale; on equal rank the later record wins. Records are never +/// spliced together: a record's amounts, inputs and outputs are only +/// coherent with the context they were observed under, so mixing a later +/// body with an earlier, higher-ranked context would fabricate a record +/// whose finality no single observation ever reported. +// TODO(unify-record-coalescing): `coalesce_newest_wins` (the `Merge` path) +// can regress a confirmed record to an older mempool slice; this helper +// keeps the higher-ranked record. Unify them once `Merge` semantics are +// reviewed. +pub(crate) fn coalesce_account_records(records: &mut Vec) { + let mut positions = BTreeMap::new(); + for record in std::mem::take(records) { + let key = (record.txid, record.account_type); + if let Some(&position) = positions.get(&key) { + let previous: &TransactionRecord = &records[position]; + if context_rank(&record.context) >= context_rank(&previous.context) { + records[position] = record; + } + } else { + positions.insert(key, records.len()); + records.push(record); + } + } +} + /// Rank a [`TransactionContext`](key_wallet::transaction_checking::TransactionContext) /// by how far along the confirmation lifecycle the observation is. /// Used by [`fold_same_txid_records`] so a fold never regresses a @@ -4027,3 +4048,140 @@ mod utxo_credit_verdict_merge_tests { assert!(older.utxo_credit_verdicts.is_empty()); } } + +#[cfg(test)] +mod coalesce_account_records_tests { + use super::*; + use dashcore::{Address, BlockHash, Network, PubkeyHash, TxOut}; + use key_wallet::managed_account::transaction_record::{ + InputDetail, OutputDetail, OutputRole, TransactionDirection, + }; + use key_wallet::transaction_checking::{BlockInfo, TransactionContext}; + + fn record(net_amount: i64, context: TransactionContext) -> TransactionRecord { + let addresses = [5, 9].map(|tag| { + Address::new( + Network::Testnet, + dashcore::address::Payload::PubkeyHash(PubkeyHash::from_byte_array([tag; 20])), + ) + }); + let detail_index = usize::from(net_amount == 900); + let address = addresses[detail_index].clone(); + let mut tx = crate::test_support::core_fixtures::spend_of( + &[ + OutPoint::new(Txid::from_byte_array([5; 32]), 1), + OutPoint::new(Txid::from_byte_array([9; 32]), 1), + ], + addresses + .iter() + .zip([500, 900]) + .map(|(address, value)| TxOut { + value, + script_pubkey: address.script_pubkey(), + }) + .collect(), + ); + tx.version = 2; + tx.lock_time = 1; + let mut record = TransactionRecord::new( + tx, + key_wallet::account::AccountType::Standard { + index: 0, + standard_account_type: key_wallet::account::StandardAccountType::BIP44Account, + }, + context, + key_wallet::transaction_checking::transaction_router::TransactionType::Standard, + if net_amount == 500 { + TransactionDirection::Incoming + } else { + TransactionDirection::Internal + }, + vec![InputDetail { + index: detail_index as u32, + value: net_amount as u64 + 100, + address: address.clone(), + }], + vec![OutputDetail { + index: detail_index as u32, + role: if net_amount == 500 { + OutputRole::Received + } else { + OutputRole::Change + }, + address: Some(address), + value: net_amount as u64, + }], + net_amount, + ); + record.fee = (net_amount == 900).then_some(200); + record.label = format!("snapshot {net_amount}"); + record + } + + fn in_block() -> TransactionContext { + TransactionContext::InBlock(BlockInfo::new(100, BlockHash::all_zeros(), 0)) + } + + /// A later record of equal or higher rank replaces an earlier one; a + /// stale mempool record never overwrites a confirmed one. + #[test] + fn should_coalesce_to_the_latest_record_of_the_highest_rank() { + let mempool = || TransactionContext::Mempool; + // (case, earlier context, later context, (kept net, kept confirmed)) + for (case, earlier, later, kept) in [ + ( + "stale mempool after confirmed", + in_block(), + mempool(), + (500, true), + ), + ("equal rank", mempool(), mempool(), (900, false)), + ("higher rank", mempool(), in_block(), (900, true)), + ] { + let earlier = record(500, earlier); + let later = record(900, later); + assert_eq!(earlier.transaction, later.transaction, "{case}: same body"); + assert_eq!(earlier.txid, earlier.transaction.txid()); + assert_eq!(later.txid, earlier.txid, "{case}: same computed identity"); + let expected = if kept.0 == 500 { + earlier.clone() + } else { + later.clone() + }; + let mut records = vec![earlier, later]; + coalesce_account_records(&mut records); + assert_eq!(records.len(), 1, "{case}"); + let confirmed = matches!(records[0].context, TransactionContext::InBlock(_)); + assert_eq!((records[0].net_amount, confirmed), kept, "{case}"); + let actual = &records[0]; + assert_eq!( + actual.transaction, expected.transaction, + "{case}: entire body" + ); + assert_eq!(actual.context, expected.context, "{case}"); + assert_eq!( + (actual.fee, &actual.label, actual.direction), + (expected.fee, &expected.label, expected.direction), + "{case}: metadata" + ); + let inputs = |r: &TransactionRecord| { + r.input_details + .iter() + .map(|d| (d.index, d.value, d.address.clone())) + .collect::>() + }; + let outputs = |r: &TransactionRecord| { + r.output_details + .iter() + .map(|d| (d.index, d.role, d.value, d.address.clone())) + .collect::>() + }; + assert_eq!(inputs(actual), inputs(&expected), "{case}: input details"); + assert_eq!( + outputs(actual), + outputs(&expected), + "{case}: output details" + ); + } + } +} diff --git a/packages/rs-platform-wallet/src/changeset/core_bridge.rs b/packages/rs-platform-wallet/src/changeset/core_bridge.rs index 9706a17a14d..ac34d13b601 100644 --- a/packages/rs-platform-wallet/src/changeset/core_bridge.rs +++ b/packages/rs-platform-wallet/src/changeset/core_bridge.rs @@ -40,7 +40,7 @@ //! manager's lifetime; on shutdown, fire the [`CancellationToken`] to //! make the task exit cleanly. -use std::collections::{BTreeMap, HashMap, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::{Arc, Mutex, Weak}; @@ -61,8 +61,9 @@ use tokio::task::JoinHandle; use tokio_util::sync::CancellationToken; use crate::changeset::changeset::{ - fold_same_txid_records, merge_payment_overlays, AssetLockChangeSet, CoreChangeSet, - HighestUsedIndexes, PaymentOverlay, PlatformWalletChangeSet, SweepBatch, UtxoCreditVerdict, + coalesce_account_records, fold_same_txid_records, merge_payment_overlays, AssetLockChangeSet, + CoreChangeSet, HighestUsedIndexes, PaymentOverlay, PlatformWalletChangeSet, SweepBatch, + UtxoCreditVerdict, }; use crate::changeset::merge::Merge; use crate::changeset::persistence_capabilities::PersistenceCapabilities; @@ -1772,7 +1773,16 @@ async fn build_core_changeset( .filter(|r| !is_contact_watch_only(r)) .cloned(), ); - cs.records = cs.account_records.clone(); + coalesce_account_records(&mut cs.account_records); + // The wallet-level row folds every slice the manager holds for + // the transaction, not only the ones this block re-emitted (see + // `with_held_sibling_slices_and_verdicts`). + (cs.records, cs.utxo_credit_verdicts) = with_held_sibling_slices_and_verdicts( + wallet_manager, + wallet_id, + &cs.account_records, + ) + .await; fold_same_txid_records(&mut cs.records); cs.last_processed_height = Some(*height); // Pool extensions triggered by any record in this block. @@ -1793,17 +1803,6 @@ async fn build_core_changeset( collect_usage_deltas(wallet_manager, wallet_id, records).await; cs.addresses_marked_used = addresses_marked_used; cs.account_highest_used = account_highest_used; - // The engine's verdict on the outputs the persister is about to - // materialise from these records — see - // `CoreChangeSet::utxo_credit_verdicts`. Over the owned slices - // only (`account_records` is already filtered): a contact's - // watch-only chain never defines the wallet's TXOs. - cs.utxo_credit_verdicts = utxo_credit_verdicts( - wallet_manager, - wallet_id, - &cs.account_records.iter().collect::>(), - ) - .await; cs } WalletEvent::TransactionsSwept { @@ -2207,36 +2206,7 @@ async fn collect_usage_deltas( collect_usage_deltas_from_accounts(&info.core_wallet.accounts, &records) } -/// The engine's credit verdict for every `Received` / `Change` output of -/// `records` that the owning account does NOT hold — see -/// [`CoreChangeSet::utxo_credit_verdicts`] for what a persister does with -/// it. Empty when every output is credited, when the wallet is unknown -/// (raced a removal — the next round re-emits), or when `records` is empty. -/// -/// One read of the wallet lock per event, like [`collect_usage_deltas`]; -/// the walk itself is [`utxo_credit_verdicts_from_wallet`], factored so -/// tests can drive it against a bare `ManagedWalletInfo`. `BlockProcessed` -/// uses this over the event's own records (block records carry their -/// block context and cannot read `Doomed`); `TransactionDetected` reads its -/// slices and verdicts under one guard through -/// [`wallet_slices_and_verdicts_for_txid`], and `TransactionInstantLocked` -/// under its own guard through [`slices_and_verdicts_for_txid`]. -async fn utxo_credit_verdicts( - wallet_manager: &Arc>>, - wallet_id: &WalletId, - records: &[&TransactionRecord], -) -> BTreeMap { - if records.is_empty() { - return BTreeMap::new(); - } - let guard = wallet_manager.read().await; - let Some(info) = guard.get_wallet_info(wallet_id) else { - return BTreeMap::new(); - }; - utxo_credit_verdicts_from_wallet(&info.core_wallet, records) -} - -/// Synchronous core of [`utxo_credit_verdicts`]. +/// Judge owned outputs against the same wallet snapshot as their account slices. /// /// For each record (a contact's watch-only slice excluded — its outputs /// are the contact's coins and never become this wallet's TXOs) and each @@ -2494,6 +2464,62 @@ async fn wallet_slices_and_verdicts_for_txid( Some(slices_and_verdicts_for_txid(info, txid)) } +/// Complete slices and their credit verdicts from one wallet snapshot. +/// `slices` plus every slice the manager still holds, for the same +/// transactions, under an account `slices` does not carry. +/// +/// A block re-emits only the slices key-wallet's re-check still matches. A +/// no-change spend (an asset lock draining its funding account, the burn as +/// its only output) leaves the funding account nothing to match: the mempool +/// check already removed the spent UTXO and no output of the account remains. +/// The block then carries only the keys account's detail-less marker, and +/// folded alone that marker becomes the wallet-level row, replacing the +/// funding accounting the host already stored. The funding slice is still in +/// the manager (at its earlier context), so folding it back in keeps the +/// row's details and accounting while the fold takes the block's context +/// from the marker. +/// +/// The event's own slices stay authoritative for their accounts; only +/// accounts it lacks are filled in. Contact watch-only slices stay out, as +/// everywhere else in the projection. An unknown wallet (removed mid-flight, +/// or a bare test manager) leaves `slices` as they are. +async fn with_held_sibling_slices_and_verdicts( + wallet_manager: &Arc>>, + wallet_id: &WalletId, + slices: &[TransactionRecord], +) -> ( + Vec, + BTreeMap, +) { + let mut complete = slices.to_vec(); + if slices.is_empty() { + return (complete, BTreeMap::new()); + } + let guard = wallet_manager.read().await; + let Some(info) = guard.get_wallet_info(wallet_id) else { + return (complete, BTreeMap::new()); + }; + let carried: HashSet<(Txid, AccountType)> = slices + .iter() + .map(|record| (record.txid, record.account_type)) + .collect(); + let txids: BTreeSet = slices.iter().map(|record| record.txid).collect(); + for account in info.core_wallet.accounts.all_accounts() { + for txid in &txids { + if let Some(record) = account.transactions().get(txid) { + if !is_contact_watch_only(record) + && !carried.contains(&(record.txid, record.account_type)) + { + complete.push(record.clone()); + } + } + } + } + let verdicts = + utxo_credit_verdicts_from_wallet(&info.core_wallet, &complete.iter().collect::>()); + (complete, verdicts) +} + /// The slices `info` holds of `txid` and the verdicts of their owned outputs. /// The caller holds the wallet read guard, so the records a verdict is judged /// on, the wallet state it is judged against, and anything else the caller @@ -4034,6 +4060,26 @@ mod contact_watch_only_projection_tests { assert_eq!(cs.records[0].direction, TransactionDirection::Outgoing); } + #[tokio::test] + async fn should_coalesce_repeated_account_corrections_within_one_block() { + let (_, corrected, _) = contact_payment_records(); + let mut stale = corrected.clone(); + stale.input_details.clear(); + stale.net_amount = CHANGE as i64; + stale.direction = TransactionDirection::Incoming; + let mut event = block_processed(vec![stale]); + if let WalletEvent::BlockProcessed { + height, updated, .. + } = &mut event + { + (*height, *updated) = (1_001, vec![corrected.clone()]); + } + let cs = build_core_changeset(&test_manager(), &event).await; + assert_eq!(cs.records.len(), 1); + assert_eq!(cs.records[0].net_amount, corrected.net_amount); + assert_eq!(cs.account_records.len(), 1); + } + /// A cross-account spend (CoinJoin-funded send with BIP44 change) /// emits one record per matched account, and the two slices DISAGREE /// on the change output's role: the funding account's slice carries it @@ -4267,6 +4313,457 @@ mod contact_watch_only_projection_tests { ); } + /// Platform credits an asset lock moves into the wallet's own + /// keys account (`AssetLockPayload.credit_outputs`, mirrored by the + /// OP_RETURN output's value). + const LOCK_CREDIT: u64 = 60_000_000; + const LOCK_FEE: u64 = 1_000; + + /// The credit an asset lock spending [`our_input`] burns: everything + /// but the fee when there is no change, else [`LOCK_CREDIT`]. + fn lock_credit(change: bool) -> u64 { + if change { + LOCK_CREDIT + } else { + FUNDING - LOCK_FEE + } + } + + /// An asset lock spending [`our_input`]: output 0 burns + /// [`lock_credit`] into the OP_RETURN, output 1 (when `change`) is + /// our change, and the rest is the fee. + fn asset_lock_tx(change: bool) -> Transaction { + let mut tx = tx_with(&[]); + tx.version = 3; + tx.output.push(TxOut { + value: lock_credit(change), + script_pubkey: ScriptBuf::new_op_return(&[]), + }); + if change { + tx.output.push(TxOut { + value: FUNDING - LOCK_CREDIT - LOCK_FEE, + script_pubkey: our_change_address().script_pubkey(), + }); + } + tx + } + + /// The two per-account records upstream emits for an asset lock: + /// the funding account's slice (classified by upstream + /// `record_transaction`: `Unspendable` OP_RETURN, `Change` output, + /// `Outgoing` when nothing of ours comes back) and the keys + /// account's thin marker (`Internal`, no details, `+credit` net). + fn asset_lock_slices(change: bool) -> (TransactionRecord, TransactionRecord) { + let tx = asset_lock_tx(change); + let mut outputs = vec![OutputDetail { + index: 0, + role: OutputRole::Unspendable, + address: None, + value: lock_credit(change), + }]; + let (direction, change_value) = if change { + let value = FUNDING - LOCK_CREDIT - LOCK_FEE; + outputs.push(output(1, OutputRole::Change, &our_change_address(), value)); + (TransactionDirection::Internal, value) + } else { + (TransactionDirection::Outgoing, 0) + }; + let funding = record_with( + &tx, + bip44_account_0(), + in_block(1_000), + TransactionType::AssetLock, + direction, + vec![our_input()], + outputs, + change_value as i64 - FUNDING as i64, + ); + let keys = record_with( + &tx, + AccountType::AssetLockAddressTopUp, + in_block(1_000), + TransactionType::AssetLock, + TransactionDirection::Internal, + Vec::new(), + Vec::new(), + lock_credit(change) as i64, + ); + (funding, keys) + } + + /// The wallet's Core balance drops by the locked credit plus the fee + /// — the credit leaves Core for the wallet's own Platform keys, so + /// the move is `Internal` and the net is `-(credit + fee)`, with or + /// without change. The keys account's `+credit` marker net counts + /// Platform credits, not Core funds, and must not be summed in (it + /// made the live row read `-fee` while every repair path wrote + /// `-(credit + fee)`). A funding slice that never meets the keys + /// marker (the keys account did not match, or its slice lands in + /// another round) is projected with the same wallet rule — not left + /// on upstream's account-local `Outgoing`. The keys account's marker + /// alone carries no Core accounting evidence, so the projection leaves + /// upstream's verdict alone. An asset lock that also pays someone else + /// is a real outgoing payment; the asset-lock exception covers only the burn. + #[tokio::test] + async fn should_project_asset_lock_accounting_from_the_wallet_side() { + use TransactionDirection::{Internal, Outgoing}; + let paid = 5_000_000; + let (mut paying, mut paying_keys) = asset_lock_slices(true); + let mut tx = paying.transaction.clone(); + tx.output[1].value -= paid; + tx.output.push(TxOut { + value: paid, + script_pubkey: contact_address().script_pubkey(), + }); + for slice in [&mut paying, &mut paying_keys] { + slice.transaction = tx.clone(); + slice.txid = tx.txid(); + } + paying.output_details[1].value -= paid; + paying.net_amount -= paid as i64; + let sent = output(2, OutputRole::Sent, &contact_address(), paid); + paying.output_details.push(sent); + let lock_net = |change| -((lock_credit(change) + LOCK_FEE) as i64); + let both = |change| { + let (funding, keys) = asset_lock_slices(change); + vec![funding, keys] + }; + let funding_alone = |change| vec![asset_lock_slices(change).0]; + let (bip44, keys_account) = (bip44_account_0(), AccountType::AssetLockAddressTopUp); + let keys_net = lock_credit(false) as i64; + let paying_net = -((LOCK_CREDIT + LOCK_FEE + paid) as i64); + for (case, records, expected) in [ + ("no change", both(false), (Internal, lock_net(false), bip44)), + ("change", both(true), (Internal, lock_net(true), bip44)), + ( + "lone funding slice", + funding_alone(false), + (Internal, lock_net(false), bip44), + ), + ( + "lone funding slice, change", + funding_alone(true), + (Internal, lock_net(true), bip44), + ), + ( + "lone keys marker", + vec![asset_lock_slices(false).1], + (Internal, keys_net, keys_account), + ), + ( + "paying an external output", + vec![paying, paying_keys], + (Outgoing, paying_net, bip44), + ), + ] { + let cs = build_core_changeset(&test_manager(), &block_processed(records)).await; + + assert_eq!(cs.records.len(), 1, "{case}"); + let row = &cs.records[0]; + assert_eq!( + (row.direction, row.net_amount, row.account_type), + expected, + "{case}" + ); + } + } + + /// The wallet-level accounting pass runs on every projected record, + /// singles included. Outside asset locks it must reproduce what the + /// projection emitted before the pass existed: a single record kept + /// upstream's own `net_amount`/`direction`, and a group summed its + /// slices' nets and re-derived direction over the merged details. + /// Each case feeds upstream-shaped records (upstream's own net and + /// direction for its account slice) through the live projection and + /// pins that prior output (verified against the pre-pass fold). + /// + /// One intentional difference: a group made only of keys-account + /// markers (no details) used to re-derive `Incoming` from its empty + /// details. It now keeps upstream's `Internal`, like a single marker + /// always did and like the SQLite repair, which ignores detail-less + /// records. + #[tokio::test] + async fn should_keep_live_fold_accounting_for_non_asset_lock_transactions() { + const PAID: u64 = 60_000_000; + const FEE: u64 = 1_000; + type Case = (Vec, TransactionDirection, i64); + use TransactionDirection::{CoinJoin, Incoming, Internal, Outgoing}; + use TransactionType::{ + AssetUnlock, CoinJoin as CoinJoinTx, Coinbase, ProviderRegistration, Standard, + }; + let external = contact_address; + let (ours, our_change) = (our_receive_address(), our_change_address()); + let second_account = || AccountType::Standard { + index: 1, + standard_account_type: StandardAccountType::BIP44Account, + }; + let slice = |tx: &Transaction, + account: AccountType, + kind: TransactionType, + direction: TransactionDirection, + inputs: Vec, + outputs: Vec, + net: i64| { + record_with( + tx, + account, + in_block(1_000), + kind, + direction, + inputs, + outputs, + net, + ) + }; + // A lone slice, whose own net and direction the projection keeps. + let single = |record: TransactionRecord| -> Case { + let (direction, net) = (record.direction, record.net_amount); + (vec![record], direction, net) + }; + let received = |value| vec![output(0, OutputRole::Received, &ours, value)]; + // A receipt of PAID into `account`, classified `kind`. + let incoming = |tx: &Transaction, account, kind| { + single(slice( + tx, + account, + kind, + Incoming, + vec![], + received(PAID), + PAID as i64, + )) + }; + // A standard spend of our coin from the first BIP44 account. + let spend = |tx: &Transaction, direction, outputs, net| { + let record = slice( + tx, + bip44_account_0(), + Standard, + direction, + vec![our_input()], + outputs, + net, + ); + single(record) + }; + // A slice of a keys account: no details, upstream's own net. + let marker = |tx: &Transaction, account, net| { + slice( + tx, + account, + TransactionType::AssetLock, + Internal, + vec![], + vec![], + net, + ) + }; + + let receive = tx_with(&[(&ours, PAID)]); + let send_change = tx_with(&[(&external(), PAID), (&our_change, CHANGE)]); + let send_all = tx_with(&[(&external(), FUNDING - FEE)]); + let self_send = tx_with(&[(&ours, FUNDING - FEE)]); + let cross = tx_with(&[(&ours, PAID), (&our_change, CHANGE)]); + let mix = tx_with(&[(&ours, FUNDING - FEE)]); + let provider = tx_with(&[(&ours, PAID), (&our_change, FUNDING - PAID - FEE)]); + let provider_ext = tx_with(&[(&external(), PAID), (&our_change, FUNDING - PAID - FEE)]); + let unlock = tx_with(&[(&ours, PAID)]); + let mut coinbase = tx_with(&[(&ours, PAID)]); + coinbase.lock_time = 9; + let keys_only = tx_with(&[]); + let (_, contact_funding, contact_watch_only) = contact_payment_records(); + let from_contact = tx_with(&[(&ours, PAID)]); + let dashpay_receiving = AccountType::DashpayReceivingFunds { + index: 0, + user_identity_id: [2u8; 32], + friend_identity_id: [1u8; 32], + }; + let provider_change = output(1, OutputRole::Change, &our_change, FUNDING - PAID - FEE); + let owner_marker = |tx: &Transaction| { + slice( + tx, + AccountType::ProviderOwnerKeys, + ProviderRegistration, + Internal, + vec![], + vec![], + 0, + ) + }; + + let cases: Vec<(&str, Case)> = vec![ + ( + "plain receive", + incoming(&receive, bip44_account_0(), Standard), + ), + ( + "send with change", + spend( + &send_change, + Outgoing, + vec![ + output(0, OutputRole::Sent, &external(), PAID), + output(1, OutputRole::Change, &our_change, CHANGE), + ], + CHANGE as i64 - FUNDING as i64, + ), + ), + ( + "send without change", + spend( + &send_all, + Outgoing, + vec![output(0, OutputRole::Sent, &external(), FUNDING - FEE)], + -(FUNDING as i64), + ), + ), + ( + "self-transfer within one account", + spend(&self_send, Internal, received(FUNDING - FEE), -(FEE as i64)), + ), + ( + "self-transfer between own accounts", + ( + vec![ + slice( + &cross, + bip44_account_0(), + Standard, + Outgoing, + vec![our_input()], + vec![ + output(0, OutputRole::Sent, &ours, PAID), + output(1, OutputRole::Change, &our_change, CHANGE), + ], + CHANGE as i64 - FUNDING as i64, + ), + slice( + &cross, + second_account(), + Standard, + Incoming, + vec![], + received(PAID), + PAID as i64, + ), + ], + Internal, + (PAID + CHANGE) as i64 - FUNDING as i64, + ), + ), + ( + "coinjoin", + single(slice( + &mix, + AccountType::CoinJoin { index: 0 }, + CoinJoinTx, + CoinJoin, + vec![our_input()], + received(FUNDING - FEE), + -(FEE as i64), + )), + ), + ( + "provider registration with owner-key marker", + ( + vec![ + slice( + &provider, + bip44_account_0(), + ProviderRegistration, + Internal, + vec![our_input()], + vec![ + output(0, OutputRole::Received, &ours, PAID), + provider_change.clone(), + ], + -(FEE as i64), + ), + owner_marker(&provider), + ], + Internal, + -(FEE as i64), + ), + ), + ( + "provider registration paying external collateral", + ( + vec![ + slice( + &provider_ext, + bip44_account_0(), + ProviderRegistration, + Outgoing, + vec![our_input()], + vec![ + output(0, OutputRole::Sent, &external(), PAID), + provider_change, + ], + -((PAID + FEE) as i64), + ), + owner_marker(&provider_ext), + ], + Outgoing, + -((PAID + FEE) as i64), + ), + ), + ( + "asset unlock", + incoming(&unlock, bip44_account_0(), AssetUnlock), + ), + ("coinbase", incoming(&coinbase, bip44_account_0(), Coinbase)), + ( + "keys-only single marker", + single(marker( + &keys_only, + AccountType::IdentityRegistration, + PAID as i64, + )), + ), + ( + "payment to a contact (watch-only slice present)", + ( + vec![contact_funding, contact_watch_only], + Outgoing, + CHANGE as i64 - FUNDING as i64, + ), + ), + ( + "payment from a contact into DashPay receiving funds", + incoming(&from_contact, dashpay_receiving, Standard), + ), + ( + "keys-only group", + ( + vec![ + marker(&keys_only, AccountType::IdentityRegistration, PAID as i64), + marker( + &keys_only, + AccountType::AssetLockAddressTopUp, + CHANGE as i64, + ), + ], + // Intentional change (see the doc comment): was `Incoming`. + Internal, + (PAID + CHANGE) as i64, + ), + ), + ]; + let mut failures = Vec::new(); + for (name, (records, direction, net)) in cases { + let cs = build_core_changeset(&test_manager(), &block_processed(records)).await; + assert_eq!(cs.records.len(), 1, "{name}"); + let got = (cs.records[0].direction, cs.records[0].net_amount); + if got != (direction, net) { + failures.push(format!( + "{name}: expected {:?}, got {got:?}", + (direction, net) + )); + } + } + assert!(failures.is_empty(), "{failures:#?}"); + } + /// A fold lands at the group's FIRST position even when the funding /// record (the metadata source) appears later — unrelated records /// between the slices must not move ahead of the folded transaction. @@ -9078,6 +9575,120 @@ mod utxo_credit_verdict_tests { assert!(unknown.utxo_credit_verdicts.is_empty()); } + #[tokio::test] + async fn should_preserve_held_sibling_verdict_through_buffered_block_projection() { + use crate::wallet::core::WalletGeneration; + use crate::wallet::identity::IdentityManager; + + for credited in [false, true] { + let mut ctx = TestWalletContext::new_random(); + let funding = funding_of(ctx.receive_address.script_pubkey(), 71); + let coin = OutPoint::new(funding.txid(), 0); + let burn = ctx + .check_transaction(&collateral_burn(coin), in_block(100_001)) + .await; + assert!( + !burn.is_relevant, + "there is no spender record to repair a lost verdict" + ); + let result = ctx.check_transaction(&funding, in_block(100_000)).await; + let sibling = result.new_records[0].clone(); + let mut marker = sibling.clone(); + marker.account_type = AccountType::IdentityRegistration; + marker.input_details.clear(); + marker.output_details.clear(); + marker.net_amount = 0; + marker.context = in_block(100_002); + let info = PlatformWalletInfo { + core_wallet: ctx.managed_wallet, + generation: Arc::new(WalletGeneration::new()), + identity_manager: IdentityManager::new(), + tracked_asset_locks: BTreeMap::new(), + dpns_name_states: BTreeMap::new(), + dashpay_backfill: Default::default(), + rewind_barrier: Default::default(), + observed_input_conflicts: Default::default(), + }; + let mut wm = WalletManager::::new(dashcore::Network::Testnet); + let wallet_id = wm.insert_wallet(ctx.wallet, info).unwrap(); + let manager = Arc::new(RwLock::new(wm)); + let block = |record| WalletEvent::BlockProcessed { + wallet_id, + height: 100_002, + chain_lock: None, + inserted: vec![], + updated: vec![record], + matured: vec![], + balance: WalletCoreBalance::default(), + account_balances: BTreeMap::new(), + addresses_derived: vec![], + }; + let durable = DurableCursors::default(); + let mut batch = BTreeMap::new(); + project_event_into_batch(&manager, &durable, &block(sibling.clone()), &mut batch).await; + assert_eq!( + batch[&wallet_id].core.utxo_credit_verdicts.get(&coin), + Some(&UtxoCreditVerdict::ObservedSpent { height: 100_001 }) + ); + if credited { + manager + .write() + .await + .get_wallet_info_mut(&wallet_id) + .unwrap() + .core_wallet + .accounts + .standard_bip44_accounts + .get_mut(&0) + .unwrap() + .utxos + .insert( + coin, + key_wallet::Utxo::new( + coin, + funding.output[0].clone(), + ctx.receive_address.clone(), + 100_000, + false, + ), + ); + } + project_event_into_batch(&manager, &durable, &block(marker), &mut batch).await; + let merged = &batch[&wallet_id].core; + assert_eq!(merged.records.len(), 1); + assert_eq!(merged.records[0].net_amount, sibling.net_amount); + let outputs = |record: &TransactionRecord| { + record + .output_details + .iter() + .map(|detail| { + ( + detail.index, + detail.role, + detail.address.clone(), + detail.value, + ) + }) + .collect::>() + }; + assert_eq!(outputs(&merged.records[0]), outputs(&sibling)); + assert_eq!(merged.records[0].context, in_block(100_002)); + assert!( + merged + .account_records + .iter() + .any(|r| r.account_type == sibling.account_type + && outputs(r) == outputs(&sibling)), + "the callback still carries the owned sibling output" + ); + assert_eq!( + merged.utxo_credit_verdicts.get(&coin), + (!credited).then_some(&UtxoCreditVerdict::ObservedSpent { height: 100_001 }), + "only genuine current credit clears the buffered withholding verdict" + ); + } + } + /// `TransactionDetected` judges the outputs of the records it READS, /// under the guard it reads them with — never the event's own record, /// which is a clone taken at emit time and can be stale by drain time. @@ -9166,3 +9777,316 @@ mod utxo_credit_verdict_tests { ); } } + +#[cfg(test)] +mod no_change_asset_lock_block_projection_tests { + //! A no-change asset lock (the wallet drain's shape: every input of the + //! funding account spent, only the OP_RETURN burn as an output) leaves + //! the funding account nothing key-wallet can match on a re-check: the + //! mempool check already removed the spent UTXO, and no output of the + //! account remains. So the block's `updated` list carries only the keys + //! account's detail-less marker, and the funding slice keeps its mempool + //! context in memory. Drives a real `ManagedWalletInfo` through + //! `check_core_transaction` and projects the block event exactly as the + //! event adapter does. + + use super::*; + use crate::wallet::core::WalletGeneration; + use crate::wallet::identity::IdentityManager; + use dashcore::blockdata::transaction::special_transaction::asset_lock::AssetLockPayload; + use dashcore::blockdata::transaction::special_transaction::TransactionPayload; + use dashcore::{BlockHash, OutPoint, ScriptBuf, Transaction, TxIn, TxOut, Txid, Witness}; + use key_wallet::managed_account::managed_account_trait::ManagedAccountTrait; + use key_wallet::managed_account::transaction_record::{InputDetail, TransactionDirection}; + use key_wallet::test_utils::TestWalletContext; + use key_wallet::transaction_checking::{BlockInfo, WalletTransactionChecker}; + use key_wallet::wallet::managed_wallet_info::ManagedWalletInfo; + use key_wallet::{Wallet, WalletCoreBalance}; + + const FUNDING: u64 = 1_750_000; + const FEE: u64 = 229; + const BLOCK_HEIGHT: u32 = 100_001; + + fn in_block(height: u32) -> TransactionContext { + TransactionContext::InBlock(BlockInfo::new( + height, + BlockHash::from_slice(&[1u8; 32]).expect("valid block hash"), + 1_234_567_890, + )) + } + + fn spend_of(previous_output: OutPoint) -> TxIn { + TxIn { + previous_output, + script_sig: ScriptBuf::new(), + sequence: 0xffffffff, + witness: Witness::new(), + } + } + + /// A no-change asset lock after its mempool and block checks. + struct NoChangeLock { + managed_wallet: ManagedWalletInfo, + wallet: Wallet, + txid: Txid, + /// What the block check re-emitted (its `updated` records). + block_updated: Vec, + } + + /// Fund BIP44 with one UTXO, then lock all of it: one input, the burn as + /// the only output, the credit paid to the shielded top-up keys account. + async fn no_change_lock() -> NoChangeLock { + let TestWalletContext { + mut managed_wallet, + mut wallet, + receive_address, + .. + } = TestWalletContext::new_random(); + let fund_tx = Transaction { + version: 2, + lock_time: 0, + input: vec![spend_of(OutPoint { + txid: Txid::from_slice(&[2u8; 32]).expect("valid txid"), + vout: 0, + })], + output: vec![TxOut { + value: FUNDING, + script_pubkey: receive_address.script_pubkey(), + }], + special_transaction_payload: None, + }; + assert!( + managed_wallet + .check_core_transaction(&fund_tx, in_block(100_000), &mut wallet, true, true) + .await + .is_relevant + ); + + let xpub = wallet + .accounts + .asset_lock_shielded_address_topup + .as_ref() + .map(|account| account.account_xpub) + .expect("shielded top-up account xpub"); + let credit_address = managed_wallet + .accounts + .asset_lock_shielded_address_topup + .as_mut() + .expect("shielded top-up account") + .next_address(Some(&xpub), true) + .expect("credit address"); + let credit = FUNDING - FEE; + let lock_tx = Transaction { + version: 3, + lock_time: 0, + input: vec![spend_of(OutPoint { + txid: fund_tx.txid(), + vout: 0, + })], + output: vec![TxOut { + value: credit, + script_pubkey: ScriptBuf::new_op_return(&[]), + }], + special_transaction_payload: Some(TransactionPayload::AssetLockPayloadType( + AssetLockPayload::new(vec![TxOut { + value: credit, + script_pubkey: credit_address.script_pubkey(), + }]), + )), + }; + assert!( + managed_wallet + .check_core_transaction( + &lock_tx, + TransactionContext::Mempool, + &mut wallet, + true, + true + ) + .await + .is_relevant + ); + let block = managed_wallet + .check_core_transaction(&lock_tx, in_block(BLOCK_HEIGHT), &mut wallet, true, true) + .await; + // Precondition, the shape observed live (AL-002): only the keys + // marker is re-emitted for the block. + assert!( + block.new_records.is_empty() + && block.updated_records.len() == 1 + && block.updated_records[0].input_details.is_empty() + && block.updated_records[0].output_details.is_empty(), + "precondition: the block re-emits only the detail-less keys marker, got {:?}", + block + .updated_records + .iter() + .map(|r| r.account_type) + .collect::>() + ); + NoChangeLock { + managed_wallet, + wallet, + txid: lock_tx.txid(), + block_updated: block.updated_records, + } + } + + /// The held BIP44 slice of `txid`. + fn held_funding_slice(managed_wallet: &ManagedWalletInfo, txid: &Txid) -> TransactionRecord { + managed_wallet + .first_bip44_managed_account() + .expect("bip44 account") + .transactions() + .get(txid) + .cloned() + .expect("the funding account still holds the lock's slice") + } + + /// An input detail no real slice of the lock carries: if a fold picks up + /// a slice holding it, the row shows two inputs. + fn foreign_input(template: &TransactionRecord) -> InputDetail { + InputDetail { + index: 1, + ..template.input_details[0].clone() + } + } + + /// Project the block event over a manager holding `managed_wallet` and + /// return the lock's wallet-level row. + async fn project_block( + managed_wallet: ManagedWalletInfo, + wallet: Wallet, + txid: &Txid, + updated: Vec, + ) -> TransactionRecord { + let info = PlatformWalletInfo { + core_wallet: managed_wallet, + generation: Arc::new(WalletGeneration::new()), + identity_manager: IdentityManager::new(), + tracked_asset_locks: BTreeMap::new(), + dpns_name_states: BTreeMap::new(), + dashpay_backfill: Default::default(), + rewind_barrier: Default::default(), + observed_input_conflicts: Default::default(), + }; + let mut wm = WalletManager::::new(dashcore::Network::Testnet); + let wallet_id = wm.insert_wallet(wallet, info).expect("insert wallet"); + let manager = Arc::new(RwLock::new(wm)); + let cs = build_core_changeset( + &manager, + &WalletEvent::BlockProcessed { + wallet_id, + height: BLOCK_HEIGHT, + chain_lock: None, + inserted: vec![], + updated, + matured: vec![], + balance: WalletCoreBalance::default(), + account_balances: BTreeMap::new(), + addresses_derived: vec![], + }, + ) + .await; + let rows: Vec<&TransactionRecord> = cs.records.iter().filter(|r| r.txid == *txid).collect(); + assert_eq!(rows.len(), 1, "one wallet-level row per transaction"); + rows[0].clone() + } + + #[tokio::test] + async fn should_keep_the_funding_accounting_when_a_block_updates_only_the_keys_marker() { + let lock = no_change_lock().await; + let row = project_block( + lock.managed_wallet, + lock.wallet, + &lock.txid, + lock.block_updated, + ) + .await; + + assert_eq!( + row.input_details.len(), + 1, + "the wallet-level row must keep the funding input, not degrade to the keys \ + marker (got {:?}, net {})", + row.account_type, + row.net_amount + ); + assert_eq!(row.direction, TransactionDirection::Internal); + assert_eq!( + row.net_amount, + -(FUNDING as i64), + "a no-change lock costs the whole funding: credit + fee" + ); + assert!( + row.context.block_info().is_some(), + "the row carries the block confirmation" + ); + } + + /// The event stays authoritative for the accounts it carries: a held + /// slice of an account the event already re-emitted is not folded in. + #[tokio::test] + async fn should_prefer_the_events_slice_over_a_held_slice_of_the_same_account() { + let mut lock = no_change_lock().await; + let mut from_event = held_funding_slice(&lock.managed_wallet, &lock.txid); + from_event.context = in_block(BLOCK_HEIGHT); + // Mark the held copy so folding it in would show. + let foreign = foreign_input(&from_event); + lock.managed_wallet + .first_bip44_managed_account_mut() + .expect("bip44 account") + .transactions_mut() + .get_mut(&lock.txid) + .expect("held funding slice") + .input_details + .push(foreign); + + let mut updated = lock.block_updated; + updated.push(from_event); + let row = project_block(lock.managed_wallet, lock.wallet, &lock.txid, updated).await; + + assert_eq!( + row.input_details.len(), + 1, + "the held slice of an account the event carries must not be folded in" + ); + assert_eq!(row.net_amount, -(FUNDING as i64)); + assert!(row.context.block_info().is_some()); + } + + /// A contact's watch-only slice never defines the wallet's row, so a + /// held one is not folded in either. + #[tokio::test] + async fn should_not_fold_a_held_contact_watch_only_slice() { + let mut lock = no_change_lock().await; + let mut watch_only = held_funding_slice(&lock.managed_wallet, &lock.txid); + watch_only.account_type = AccountType::DashpayExternalAccount { + index: 0, + user_identity_id: [7u8; 32], + friend_identity_id: [8u8; 32], + }; + let foreign = foreign_input(&watch_only); + watch_only.input_details.push(foreign); + assert!(is_contact_watch_only(&watch_only)); + lock.managed_wallet + .first_coinjoin_managed_account_mut() + .expect("default wallet has CoinJoin account 0") + .transactions_mut() + .insert(lock.txid, watch_only); + + let row = project_block( + lock.managed_wallet, + lock.wallet, + &lock.txid, + lock.block_updated, + ) + .await; + + assert_eq!( + row.input_details.len(), + 1, + "a held contact watch-only slice must stay out of the row" + ); + assert_eq!(row.net_amount, -(FUNDING as i64)); + } +} diff --git a/packages/rs-platform-wallet/src/changeset/mod.rs b/packages/rs-platform-wallet/src/changeset/mod.rs index 612d81bccaf..c4e5429ffc7 100644 --- a/packages/rs-platform-wallet/src/changeset/mod.rs +++ b/packages/rs-platform-wallet/src/changeset/mod.rs @@ -30,6 +30,7 @@ pub mod shielded_changeset_disabled; #[cfg(feature = "shielded")] pub mod shielded_sync_start_state; pub mod traits; +mod wallet_accounting; pub(crate) use changeset::account_address_pool_entries; pub use changeset::{ @@ -69,3 +70,4 @@ pub use traits::{ persists_outbound_account_markers, ListedCoreTxid, PersistenceError, PersistenceErrorKind, PlatformWalletPersistence, }; +pub use wallet_accounting::{is_owned, wallet_accounting, wallet_direction}; diff --git a/packages/rs-platform-wallet/src/changeset/wallet_accounting.rs b/packages/rs-platform-wallet/src/changeset/wallet_accounting.rs new file mode 100644 index 00000000000..48ae44a914a --- /dev/null +++ b/packages/rs-platform-wallet/src/changeset/wallet_accounting.rs @@ -0,0 +1,117 @@ +//! Wallet-level accounting of a Core transaction record: the ownership +//! predicate ([`is_owned`]), the direction rule ([`wallet_direction`]) and +//! the net formula ([`wallet_accounting`]). The live projection +//! (`fold_same_txid_records`) and the SQLite repair +//! (`platform-wallet-storage`'s `core_history`) both call them, so a row +//! cannot change accounting when storage repairs it; they differ only on +//! an `i64` overflow (live saturates, repair errors). +//! +//! Upstream `key-wallet` classifies each matched account on its own, and +//! its account-local views disagree for an asset lock: the funding +//! account reads a lock with no change as `Outgoing` (the OP_RETURN burn +//! is not an owned output), while the keys account holding the credit +//! keys reads it as `Internal`. From the wallet's side an asset lock +//! moves Core duffs into its own Platform credits, so only the fee +//! leaves the wallet. + +use std::collections::BTreeSet; + +use key_wallet::managed_account::transaction_record::{ + OutputRole, TransactionDirection, TransactionRecord, +}; +use key_wallet::transaction_checking::transaction_router::TransactionType; + +/// Classify a transaction from the wallet's point of view. +/// +/// - `spends_ours`: at least one input spends a wallet-owned output. +/// - `has_ours`: at least one output is wallet-owned (`Received`/`Change`). +/// - `has_external`: at least one output is neither wallet-owned nor an +/// OP_RETURN burn, so value leaves the wallet. +/// +/// An asset lock is `Internal` whenever nothing leaves the wallet, even +/// with no change output: its OP_RETURN value becomes the wallet's own +/// Platform credits. The Swift SDK's +/// `PersistentTransaction.reconciledAccounting` applies the same rule; +/// keep them in step. The case table +/// shared with Swift lives in `platform-wallet-storage` +/// (`should_classify_repaired_direction_like_the_swift_sdk`). +pub fn wallet_direction( + transaction_type: TransactionType, + spends_ours: bool, + has_ours: bool, + has_external: bool, +) -> TransactionDirection { + if transaction_type == TransactionType::CoinJoin { + TransactionDirection::CoinJoin + } else if !spends_ours { + TransactionDirection::Incoming + } else if !has_external && (has_ours || transaction_type == TransactionType::AssetLock) { + TransactionDirection::Internal + } else { + TransactionDirection::Outgoing + } +} + +/// Recompute a record's wallet-level `net_amount` and `direction` from +/// its input and output details, saturating a net that does not fit `i64`. +/// +/// A record with no details carries no accounting evidence (a keys-account +/// marker that met no funding slice), so it is left as upstream emitted +/// it. The SQLite repair skips such records for the same reason. +pub(crate) fn apply_wallet_accounting(record: &mut TransactionRecord) { + if record.input_details.is_empty() && record.output_details.is_empty() { + return; + } + let (net, direction) = wallet_accounting(record); + // Unreachable for real amounts (the whole supply fits i64 many times + // over); saturate rather than panic on a corrupt record. + record.net_amount = i64::try_from(net).unwrap_or(if net < 0 { i64::MIN } else { i64::MAX }); + record.direction = direction; +} + +/// Wallet-level net amount and direction of `record`, from its details. +/// +/// The net is `Σ owned outputs − Σ owned inputs`: the change in the +/// wallet's Core balance. A keys-account marker's `+credit` (Platform +/// credits, not Core funds) is therefore not part of it, so an asset lock +/// nets `−(credit + fee)`. The net is returned as `i128` so each caller +/// picks its own policy for a value outside `i64`. +pub fn wallet_accounting(record: &TransactionRecord) -> (i128, TransactionDirection) { + let owned: i128 = record + .output_details + .iter() + .filter(|d| is_owned(d.role)) + .map(|d| i128::from(d.value)) + .sum(); + let spent: i128 = record + .input_details + .iter() + .map(|d| i128::from(d.value)) + .sum(); + let has_ours = record.output_details.iter().any(|d| is_owned(d.role)); + // Indexed once so the per-output check below stays linear. + let accounted: BTreeSet = record + .output_details + .iter() + .filter(|d| is_owned(d.role) || d.role == OutputRole::Unspendable) + .map(|d| d.index as usize) + .collect(); + let has_external = record + .transaction + .output + .iter() + .enumerate() + .any(|(index, output)| !output.script_pubkey.is_op_return() && !accounted.contains(&index)); + let direction = wallet_direction( + record.transaction_type, + !record.input_details.is_empty(), + has_ours, + has_external, + ); + (owned - spent, direction) +} + +/// Whether an output with this role belongs to the wallet. +pub fn is_owned(role: OutputRole) -> bool { + matches!(role, OutputRole::Received | OutputRole::Change) +} diff --git a/packages/rs-platform-wallet/src/test_support.rs b/packages/rs-platform-wallet/src/test_support.rs index 9dd50f7e669..5c43f14ecdc 100644 --- a/packages/rs-platform-wallet/src/test_support.rs +++ b/packages/rs-platform-wallet/src/test_support.rs @@ -20,6 +20,7 @@ use dashcore::Txid; use dashcore::{Address, Network, ScriptBuf, Transaction, TxOut}; use key_wallet::account::account_type::StandardAccountType; use key_wallet::bip32::ExtendedPubKey; +use key_wallet::managed_account::transaction_record::TransactionRecord; // Only the `#[cfg(test)]` CoinJoin fixture needs the trait (for // `next_address_with_info` on a non-standard account); gate it to match so a // `test-utils`-only build does not flag it unused. @@ -43,6 +44,7 @@ use tokio::sync::Notify; use crate::broadcaster::SpvBroadcaster; #[cfg(test)] use crate::broadcaster::{BroadcastError, TransactionBroadcaster}; +use crate::changeset::changeset::fold_same_txid_records; #[cfg(test)] use crate::changeset::{DurableCursors, PlatformWalletPersistence}; #[cfg(test)] @@ -192,6 +194,112 @@ impl ExtendedPubKeySigner for WalletSigner { } } +/// Runs the live projection's per-txid fold and wallet-level accounting +/// over upstream per-account records, so downstream crates can pin their +/// own accounting (the SQLite repair) against the live path. +pub fn fold_wallet_records(records: &mut Vec) { + fold_same_txid_records(records); +} + +/// Core transaction and wallet fixtures shared by the history tests of this +/// crate and of `platform-wallet-storage`. +pub mod core_fixtures { + use dashcore::address::Payload; + use dashcore::{ + Address, BlockHash, Network, OutPoint, PubkeyHash, Transaction, TxIn, TxOut, Txid, + }; + use key_wallet::managed_account::transaction_record::TransactionRecord; + use key_wallet::transaction_checking::{ + BlockInfo, TransactionCheckResult, TransactionContext, WalletTransactionChecker, + }; + use key_wallet::wallet::initialization::WalletAccountCreationOptions; + use key_wallet::wallet::ManagedWalletInfo; + use key_wallet::{Utxo, Wallet}; + + /// A version-1 transaction spending `inputs` into `outputs`. + pub fn spend_of(inputs: &[OutPoint], outputs: Vec) -> Transaction { + let input = inputs.iter().map(|&previous_output| TxIn { + previous_output, + ..Default::default() + }); + Transaction { + version: 1, + lock_time: 0, + input: input.collect(), + output: outputs, + special_transaction_payload: None, + } + } + + /// An output paying `value` to `address`. + pub fn pay(value: u64, address: &Address) -> TxOut { + TxOut { + value, + script_pubkey: address.script_pubkey(), + } + } + + /// Output 0 of a transaction outside the wallet, named by `byte`. + pub fn outside(byte: u8) -> OutPoint { + OutPoint::new(Txid::from_byte_array([byte; 32]), 0) + } + + /// A testnet P2PKH address no test wallet derives. + pub fn foreign_address(marker: u8) -> Address { + let hash = PubkeyHash::from_byte_array([marker; 20]); + Address::new(Network::Testnet, Payload::PubkeyHash(hash)) + } + + /// A block at `height`, hashed `[height as u8; 32]`. + pub fn block(height: u32) -> TransactionContext { + let hash = BlockHash::from_byte_array([height as u8; 32]); + TransactionContext::InBlock(BlockInfo::new(height, hash, height)) + } + + /// Output `vout` of `tx` as a sync stores it: confirmed at height 100. + pub fn stored_utxo(tx: &Transaction, vout: u32, address: Address) -> Utxo { + let (outpoint, output) = ( + OutPoint::new(tx.txid(), vout), + tx.output[vout as usize].clone(), + ); + let mut utxo = Utxo::new(outpoint, output, address, 100, false); + utxo.is_confirmed = true; + utxo + } + + /// A fresh random testnet wallet and its first BIP44 receive address. + pub fn wallet_with_receive_address() -> (Wallet, ManagedWalletInfo, Address) { + let wallet = + Wallet::new_random(Network::Testnet, WalletAccountCreationOptions::Default).unwrap(); + let mut info = ManagedWalletInfo::from_wallet(&wallet, 0); + let xpub = wallet.accounts.standard_bip44_accounts[&0].account_xpub; + let account = info.accounts.standard_bip44_accounts.get_mut(&0).unwrap(); + let address = account.next_receive_address(Some(&xpub), true).unwrap(); + (wallet, info, address) + } + + /// What the live checker reports for `tx` in `context`, state updated. + pub async fn check( + info: &mut ManagedWalletInfo, + wallet: &mut Wallet, + tx: &Transaction, + context: TransactionContext, + ) -> TransactionCheckResult { + info.check_core_transaction(tx, context, wallet, true, true) + .await + } + + /// The records the live checker produces for `tx` in `context`. + pub async fn records_of( + info: &mut ManagedWalletInfo, + wallet: &mut Wallet, + tx: &Transaction, + context: TransactionContext, + ) -> Vec { + check(info, wallet, tx, context).await.new_records + } +} + /// Builds a testnet wallet manager whose `account_type`/index-0 account /// holds a single spendable UTXO (10_000_000 duffs) — the whole balance /// rides on that one input, so a leaked reservation strands it. Returns @@ -291,9 +399,7 @@ pub(crate) fn observed_spend_event( tx: &Transaction, ) -> key_wallet_manager::WalletEvent { use dashcore::Address as DashAddress; - use key_wallet::managed_account::transaction_record::{ - InputDetail, TransactionDirection, TransactionRecord, - }; + use key_wallet::managed_account::transaction_record::{InputDetail, TransactionDirection}; use key_wallet::transaction_checking::transaction_router::TransactionType; let record = TransactionRecord::new(