diff --git a/docs/kv-keys.md b/docs/kv-keys.md index aba15fac9..884b48d1b 100644 --- a/docs/kv-keys.md +++ b/docs/kv-keys.md @@ -85,9 +85,11 @@ The identity blob and top-up history are **identity-scoped** (`DetScope::Identit | Key | Scope | Store | Value type | Notes | |-----|-------|-------|------------|-------| | `det:identity:v1` | `DetScope::Identity(&id)` | `det-.sqlite` | `StoredQualifiedIdentity` | Fields: `qi_bytes` (inner bincode, redacted in `Debug`), `status: u8`, `identity_type: String`, `wallet_hash: Option<[u8;32]>`, `wallet_index: Option` | -| `det:identity_index:v1` | `None` | `det-.sqlite` | `Vec<[u8;32]>` | Complete enumeration index of stored identity ids | +| `det:identity_index:v1` | `None` | `det-.sqlite` | `Vec<[u8;32]>` | Complete enumeration index of stored identity ids. Rewritten wholesale on every add/remove, so all read-modify-write access is serialized by one process-wide lock — absence from this roster authorizes the vault-cleanup sweep to delete an identity's private keys, and a lost update would forge that evidence | | `det:identity_order:v1` | `None` | `det-.sqlite` | `Vec<[u8;32]>` | User-chosen display ordering of identity ID raw bytes | | `det:top_ups:v1` | `DetScope::Identity(&id)` | `det-.sqlite` | `BTreeMap` | Top-up history: account index → credits | +| `det:vault_cleanup_pending:v1:` | `None` | `det-.sqlite` | `Vec<(StoredPrivateKeyTarget, KeyID)>` | Durable manifest of the vault-key placements a `delete_local_qualified_identity` call must still clear; persisted before that call's first mutation, cleared once every listed key is deleted (a clear that fails leaves a stale manifest, not a failed removal — the sweep re-runs the idempotent deletes and clears it). Global-scoped (not `DetScope::Identity`) so it survives the index removal that unlists `id` — the exact step it exists to protect against. Resumed by the boot-time `AppContext::resume_pending_vault_cleanups` sweep. | +| `det:identity_unloaded:v1:` | `None` | `det-.sqlite` | `u64` | Presence-only marker that the user unloaded this identity from this device; the stored value is the unload's unix timestamp, kept for diagnostics only and never read for a decision. Written by `delete_local_qualified_identity` **before** `index_remove_identity` delists the identity, and by the devnet wipe for every identity it clears, so no window exists in which the identity is gone and the marker is not yet on file. Consulted by `AppContext::store_discovered_identity` — the single guarded store used by the discovery passes and by the `finish_unwire` migration import — which refuses to store an identity carrying one. That refusal is what makes an unload survive the automatic passes (boot sweep, post-unlock, wallet import, resumed migration), all of which re-derive or re-read the same identity from material that outlives the removal. The absent-record branch of `write_local_qualified_identity_locked` declines the same way, as a backstop covering every update path that could otherwise re-create a record for an identity a removal had already taken away. Global-scoped for the same reason as the manifest above: it must outlive the identity's own scope. Retired only by `insert_local_qualified_identity`, i.e. by the user deliberately loading the identity again. Never expired on a timer — an expiring tombstone is a resurrection with a delay — and never reaped: it grows by one entry per identity the user has actually unloaded on this network, a bound set by user action rather than by anything automatic, which any change letting an automatic path write these would break. | **Exception to the cascade above**: an identity stored without a wallet association (`wallet_hash: None`) is *also* mirrored into the upstream `identities` table under the **unowned scope** — the all-zero `WalletId`, which upstream stores as a NULL `wallet_id`. Masternode/evonode nodes are the expected case, but any wallet-less identity DET stores takes this path (e.g. a `User` identity looked up by id with no owning wallet). The scope is load-bearing, not incidental: a NULL `wallet_id` activates no foreign key, so no wallet's `ON DELETE CASCADE` reaches the row, and the `cascade_meta_on_identity_delete` trigger — which would delete the identity's `meta_identity` rows, i.e. the `det:identity:v1` record above — never fires for it. That is the whole point: filing the same identity under a real wallet's scope would make removing that unrelated wallet destroy the node's DET record. These rows are also kept out of every wallet's `IdentityManager`, because the `identities` upsert promotes an unowned row to the first wallet that flushes it. @@ -216,8 +218,8 @@ Source: `src/wallet_backend/single_key.rs` (`SINGLE_KEY_PRIV_LABEL_PREFIX`, `SIN | Store | Key count | |-------|-----------| | `det-app.sqlite` | 4 (settings, wallet-meta sidecar, single-key-meta sidecar, migration sentinel) | -| `det-.sqlite` | 21 (across 8 domains) | +| `det-.sqlite` | 23 (across 8 domains) | | `SecretStore` | 2 label patterns (seed envelopes, imported-key private bytes) | -| **Total** | **27** | +| **Total** | **29** | Prefixed/templated keys (e.g. `det:identity:`) are counted once per prefix, not per instance. `SecretStore` entries are counted as label-pattern families, not per-wallet instances. diff --git a/docs/user-stories.md b/docs/user-stories.md index fa384ef83..90c82d3e0 100644 --- a/docs/user-stories.md +++ b/docs/user-stories.md @@ -700,6 +700,25 @@ As a user whose identity was already in the app before the upgrade — a mastern - The previous version's data is never modified, so restoring can be repeated safely; repeating it reports that there was nothing left to restore and changes nothing. - Restoring keeps the app usable while it waits for the identity password: other identities can still be removed, and anything else saved for this identity while the password prompt is open survives the restore. +### IDN-021: Unload an identity from this device [Implemented] +**Persona:** Alex, Priya + +As a user, I want to remove an identity from this device from the identity's own Settings tab, so that I can clear out an identity I no longer use without hunting for a separate screen. + +- The identity's Settings tab has an "Unload this identity from this device" action in its Danger zone, and it is usable rather than announced as a coming feature. +- The confirmation names the identity it is about to unload and states plainly that this permanently deletes that identity's private keys stored on this device, and that using it here again needs the user's own backup — the identity record itself stays on Dash Platform, but "stays on Platform" is not "keys survive". +- While the confirmation is open the controls behind it, including the identity switcher, cannot be used, so the answer is always given for the identity the confirmation names. +- Confirming removes the identity, its top-up history, and its scheduled DPNS votes from this device. +- The removal applies to the identity that was on screen when the action was chosen, even if the selected identity changes while the confirmation is open. +- Choosing "Keep", or dismissing the confirmation, changes nothing. +- While a storage update is running the removal is refused with an explanation and can be retried once it finishes. +- Once the identity is gone, where the app lands depends on what is left: with no identities remaining it shows first-run onboarding, with exactly one it opens that identity, and with two or more it shows the identity picker so the user chooses which one to continue with rather than being dropped onto an arbitrary one. +- The outcome is reported: a plain success, a success noting a voter identity tied to it stayed behind, or — when the identity is delisted but a later step of the removal did not finish — a success noting its private keys may still be on this device, that the app will keep trying to clear them automatically, and that until then the device should be treated as still holding them. The report claims no more than that: not that keys are definitely present (the unfinished step may have had no keys to clear), and not that any particular launch clears them (the automatic attempt can return without doing anything, or can fail again). It names no launch at all, because this message does not survive a restart and the launch it named might make no attempt. Keys that were deleted are reported as deleted, whatever internal bookkeeping failed afterwards. +- A removal that cannot be completed at all leaves the identity's private keys intact, so a retry still has everything it needs; no identity is ever left listed without its keys, and no identity is ever reported as still removable once it is already gone from every list. +- The unload sticks. The wallet that derives the identity stays loaded, and the app re-derives and looks up that wallet's identities on its own — at start-up, after a wallet is unlocked, and when a wallet is imported — but none of those automatic passes put back an identity the user unloaded, whether the pass starts before, during, or after the removal. The same holds for the developer-only devnet "clear all identities" sweep. +- Adding a key to an identity that is unloaded while the key is being registered on the network does not leave that key's private half on this device. The key exists on Dash Platform and the app says so plainly, naming what to do — load the identity again, then add the key again — rather than reporting a save that did not happen. +- Loading the identity again is always available and always works: choosing it explicitly, or searching for it by wallet index, loads it back and ends the exclusion, so a later automatic pass keeps it up to date again like any other identity. + --- ## DPNS (DPN) diff --git a/src/backend_task/error.rs b/src/backend_task/error.rs index 07bb411af..18766dfcb 100644 --- a/src/backend_task/error.rs +++ b/src/backend_task/error.rs @@ -621,6 +621,20 @@ pub enum TaskError { source: Box, }, + /// The identity was removed from this device between the add-key broadcast + /// and the local write. The key is on Platform, but nothing about it is + /// saved here — deliberately: the user asked for this identity's keys to be + /// destroyed, and the new key is one of them. Sealing it anyway would leave + /// private key material on a device that reports it holds none, referenced + /// by no record and reachable by no cleanup. + /// + /// No loss beyond the on-chain slot: the private key was supplied by the + /// user on the add-key screen, so they still hold it. + #[error( + "The new key was added to your identity on the network, but this identity was removed from this device before the key could be saved here. Load the identity again, then add the key again." + )] + IdentityKeyAddedButIdentityUnloaded, + /// Fail-closed guard at the opt-in protect boundary: the task found /// keys still resident as plaintext on disk after the eager load-path vault /// migration, so the identity cannot be reported as fully protected. The diff --git a/src/backend_task/identity/add_key_to_identity.rs b/src/backend_task/identity/add_key_to_identity.rs index b8b4babc8..fcbdcb3e7 100644 --- a/src/backend_task/identity/add_key_to_identity.rs +++ b/src/backend_task/identity/add_key_to_identity.rs @@ -9,6 +9,7 @@ use crate::wallet_backend::secret_prompt::SecretScope; use crate::wallet_backend::{SecretAccess, VerifiedIdentityPassword}; use dash_sdk::Error as SdkError; use dash_sdk::Sdk; +use dash_sdk::dpp::identity::KeyID; use dash_sdk::dpp::identity::accessors::{IdentityGettersV0, IdentitySettersV0}; use dash_sdk::dpp::identity::identity_public_key::accessors::v0::{ IdentityPublicKeyGettersV0, IdentityPublicKeySettersV0, @@ -161,14 +162,78 @@ impl AppContext { PrivateKeyOnMainIdentity, public_key_to_add.identity_public_key.id(), ); + self.persist_added_identity_key( + &mut qualified_identity, + new_key, + &private_key, + verified_password, + )?; + Ok(BackendTaskSuccessResult::AddedKeyToIdentity(fee_result)) + } + + /// Seal the new key and store the updated record, both under this + /// identity's record guard. + /// + /// The guard spans the seal because the seal writes private key material + /// and the write is what makes anything point at it. Without it, an unload + /// completing during the broadcast leaves the removal deleting only the + /// placements the stored blob named — not this one, which is not in it yet + /// — and clearing its cleanup manifest; the seal then lands afterwards and + /// the record write is declined. The key would sit in the vault for an + /// identity with no record, no roster entry and no manifest: unreachable + /// through the identity model, so no sweep can ever collect it, on a device + /// that told the user it had destroyed that identity's keys. + /// + /// So the roster is rechecked under the guard first, and a delisted + /// identity ends the task before anything is sealed. Nothing is lost that + /// the user does not already have: the private key came from the add-key + /// screen, typed in by them. + fn persist_added_identity_key( + &self, + qualified_identity: &mut QualifiedIdentity, + new_key: (crate::model::qualified_identity::PrivateKeyTarget, KeyID), + private_key: &[u8; 32], + verified_password: Option, + ) -> Result<(), TaskError> { + let identity_id = qualified_identity.identity.id(); + let lock = self.identity_record_lock(identity_id); + let _record_guard = lock + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + + if !self.is_identity_listed(&identity_id)? { + tracing::warn!( + target = "backend_task::identity", + identity_id = %identity_id, + "Identity was removed from this device while its new key was being added; the key is on the network and was not sealed here", + ); + return Err(TaskError::IdentityKeyAddedButIdentityUnloaded); + } + + // A password-protected identity must never acquire a keyless + // key. The object password was already verified up front (before the + // broadcast above), so here we just seal the newly-added key Tier-2 + // under that SAME password and mark it `InVault` BEFORE saving, so the + // at-rest encode writes no plaintext for it. The encode-path guard + // (`encode_identity_blob_vault_first` → `IdentityKeyProtectionDowngrade`) + // still fails closed if this seal is ever skipped. + // + // This seal is the one fallible disk write between the broadcast above + // and the persist below, and on-chain + local cannot be made atomic. If + // it fails (I/O error, corrupt keystore), the key is already on-chain but + // not saved here: fail with the typed, actionable + // `IdentityKeyAddedButNotSaved` (the key is on the network; retry after + // freeing disk space) instead of a silent loss or a misleading storage + // error — and NEVER fall back to a keyless write (that would strip the + // protection this branch exists to preserve). if let Some(password) = verified_password { self.wallet_backend()? .secret_access() .seal_new_identity_key_with_password( - qualified_identity.identity.id().to_buffer(), + identity_id.to_buffer(), &new_key.0, new_key.1, - &private_key, + private_key, &password, ) .map_err(key_added_but_not_saved)?; @@ -186,8 +251,7 @@ impl AppContext { } } - self.update_local_qualified_identity(&qualified_identity)?; - Ok(BackendTaskSuccessResult::AddedKeyToIdentity(fee_result)) + self.write_local_qualified_identity_locked(qualified_identity) } } @@ -284,6 +348,89 @@ mod tests { } } + /// R1: an unload that completes during the broadcast must not leave the new + /// key's private half in the vault. + /// + /// The removal deletes the placements the *stored blob* names. The key + /// being added is not in that blob yet, so it is not in the removal's + /// delete set and not in the cleanup manifest it clears. A seal that lands + /// afterwards writes private key material for an identity with no record, + /// no roster entry and no manifest — unreachable through the identity + /// model, so no sweep can ever collect it, on a device that has just told + /// the user it destroyed that identity's keys. + /// + /// The record guard now spans the recheck, the seal and the write, so a + /// delisted identity ends the task before anything is sealed. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn an_unload_during_the_broadcast_leaves_no_key_in_the_vault() { + use crate::context::test_staging::stage_identity_with_vaulted_keys_using_prompt; + use crate::model::secret::Secret; + + const PW: &str = "identity-object-passwordpw"; + const NEW_KEY_ID: u32 = 9; + let prompt = Arc::new(TestPrompt::new([ScriptedAnswer::once(PW)])); + let staged = stage_identity_with_vaulted_keys_using_prompt( + Network::Testnet, + Some(prompt), + [0xAA; 32], + [0xBB; 32], + ) + .await; + let ctx = &staged.ctx; + + // A password-protected identity, which is the shape that seals. + ctx.protect_identity_keys(staged.id, Secret::new(PW), None) + .expect("seal the identity Tier-2"); + let mut qualified_identity = ctx + .get_local_qualified_identity(&staged.id) + .expect("read the protected identity") + .expect("identity present"); + + // The precondition the real flow satisfies before broadcasting. + let verify_scope = ctx + .protected_identity_verify_scope(&qualified_identity) + .expect("read the verify scope") + .expect("a protected identity has one"); + let password = ctx + .wallet_backend() + .expect("backend wired") + .secret_access() + .verify_identity_object_password(&verify_scope) + .await + .expect("the scripted password verifies"); + + // The user unloads the identity while the broadcast is in flight. + ctx.delete_local_qualified_identity(&staged.id) + .expect("unload the identity"); + + let new_key = (PrivateKeyTarget::PrivateKeyOnMainIdentity, NEW_KEY_ID); + let error = ctx + .persist_added_identity_key( + &mut qualified_identity, + (new_key.0.clone(), new_key.1), + &[0xCD; 32], + Some(password), + ) + .expect_err("an identity that is gone cannot take a new key"); + assert!( + matches!(error, TaskError::IdentityKeyAddedButIdentityUnloaded), + "expected IdentityKeyAddedButIdentityUnloaded, got {error:?}", + ); + + assert!( + crate::wallet_backend::IdentityKeyView::new(&staged.store, staged.id.to_buffer()) + .get(&new_key.0, new_key.1) + .expect("read the vault") + .is_none(), + "the new key must not be sealed for an identity whose keys the removal \ + just destroyed — nothing would ever point at it or collect it", + ); + assert!( + !ctx.is_identity_listed(&staged.id).expect("read the roster"), + "and the identity must not be put back on the roster", + ); + } + /// O-2 fail-closed: a HEADLESS add-key precondition for a PROTECTED identity /// returns `SecretPromptUnavailable`. `add_key_to_identity` propagates this /// with `?` BEFORE it builds or broadcasts the AddKeys state transition, so diff --git a/src/backend_task/identity/discover_identities.rs b/src/backend_task/identity/discover_identities.rs index 30a831aef..dc04b5500 100644 --- a/src/backend_task/identity/discover_identities.rs +++ b/src/backend_task/identity/discover_identities.rs @@ -3,7 +3,8 @@ use crate::backend_task::BackendTaskSuccessResult; use crate::backend_task::error::TaskError; use crate::context::AppContext; use crate::model::identity_discovery::{ - DiscoverySummary, IDENTITY_GAP_LIMIT, IDENTITY_SCAN_HARD_CAP, should_continue_scan, + DiscoveryIntent, DiscoverySummary, IDENTITY_GAP_LIMIT, IDENTITY_SCAN_HARD_CAP, + should_continue_scan, }; use crate::model::qualified_identity::DPNSNameInfo; use crate::model::wallet::Wallet; @@ -15,6 +16,21 @@ use std::sync::{Arc, RwLock}; /// concluding no identity is registered there. const AUTH_KEY_LOOKUP_WINDOW: u32 = 12; +/// The settings one gap-limited scan holds constant across every index it +/// probes, so a per-index call takes one argument instead of four. +struct ScanSettings<'a> { + sdk: &'a dash_sdk::Sdk, + /// The network the scan started on. A store is skipped if the active + /// network changed since — defense-in-depth against an in-flight pass + /// writing under the wrong network scope. + scan_network: dash_sdk::dpp::dashcore::Network, + /// Whether a cold auth-key cache miss may prompt for the passphrase. + allow_prompt: bool, + /// Whether the user asked for these identities, which decides whether the + /// scan may store one they unloaded from this device. + intent: DiscoveryIntent, +} + impl AppContext { /// Discover and load identities derived from a wallet by checking the /// network, with a rolling gap-limited lookahead. @@ -32,6 +48,11 @@ impl AppContext { /// `false` (the background sweep) a locked, protected wallet is skipped /// instead of prompting. /// + /// `intent` is a separate question from `allow_prompt` — whether the user + /// asked for these identities, which decides whether a pass may store one + /// they previously unloaded from this device. The post-unlock pass prompts + /// for nothing and is still automatic, so the two are not interchangeable. + /// /// When `progress` is `Some`, a [`BackendTaskSuccessResult::Progress`] event /// is sent before each probed index. pub(crate) async fn discover_identities_gap_limited( @@ -39,13 +60,19 @@ impl AppContext { wallet: &Arc>, seed_from_index: u32, allow_prompt: bool, + intent: DiscoveryIntent, progress: Option<&SenderAsync>, ) -> Result { use dash_sdk::platform::Fetch; use dash_sdk::platform::types::identity::NonUniquePublicKeyHashQuery; let sdk = self.sdk.load().as_ref().clone(); - let scan_network = self.network; + let settings = ScanSettings { + sdk: &sdk, + scan_network: self.network, + allow_prompt, + intent, + }; let seed_hash = wallet.read()?.seed_hash(); // Seed the rolling window from the explicit seed index and from any @@ -65,6 +92,7 @@ impl AppContext { seed = %hex::encode(seed_hash), seed_window = ?seed_window, allow_prompt, + ?intent, "Starting gap-limited identity discovery for wallet" ); @@ -163,17 +191,17 @@ impl AppContext { highest_found = Some(highest_found.map_or(current_index, |h| h.max(current_index))); match self - .upsert_discovered_identity( - &sdk, - identity, - wallet, - scan_network, - allow_prompt, - current_index, - ) + .upsert_discovered_identity(&settings, identity, wallet, current_index) .await { - Ok(()) => summary.stored = summary.stored.saturating_add(1), + Ok(true) => summary.stored = summary.stored.saturating_add(1), + // Found but deliberately not stored: an identity the user + // unloaded, or a network switch mid-scan. `found` still + // counts it, so the two numbers differ and the log says so. + Ok(false) => tracing::debug!( + identity_id = %identity_id, + "Discovered identity left unstored" + ), Err(e) => tracing::warn!( identity_id = %identity_id, error = %e, @@ -205,36 +233,41 @@ impl AppContext { wallet: &Arc>, max_identity_index: u32, ) -> Result<(), TaskError> { - self.discover_identities_gap_limited(wallet, max_identity_index, true, None) - .await?; + self.discover_identities_gap_limited( + wallet, + max_identity_index, + true, + DiscoveryIntent::Automatic, + None, + ) + .await?; Ok(()) } - /// Fetch, build, and store one discovered identity, preserving DET-only - /// metadata (the user alias) from any existing stored record. + /// Fetch, build, and store one discovered identity. Returns whether it was + /// stored. /// - /// The freshly-built [`QualifiedIdentity`](crate::model::qualified_identity::QualifiedIdentity) - /// carries `alias: None`; before storing, the existing record's alias is - /// copied onto it so a re-discovery refreshes keys and DPNS names without - /// wiping a user-assigned alias. Wallet association and top-up history are - /// preserved by [`AppContext::update_local_qualified_identity`] / - /// the separate top-up KV key, respectively. + /// The store itself is + /// [`AppContext::store_discovered_identity`], which decides between + /// refreshing an existing record and inserting a new one, carries the + /// user's alias onto the freshly built identity (which never has one), and + /// refuses an identity the user unloaded from this device unless `intent` + /// says the user asked for it. Top-up history lives under a separate k/v + /// key that no write here touches. /// /// `scan_network` is the network the scan started on. If the active network /// changed mid-scan, the store is skipped — defense-in-depth so an in-flight /// pass never writes a discovered identity under the wrong network scope. async fn upsert_discovered_identity( self: &Arc, - sdk: &dash_sdk::Sdk, + settings: &ScanSettings<'_>, identity: dash_sdk::platform::Identity, wallet: &Arc>, - scan_network: dash_sdk::dpp::dashcore::Network, - allow_prompt: bool, identity_index: u32, - ) -> Result<(), TaskError> { - if self.network != scan_network { + ) -> Result { + if self.network != settings.scan_network { tracing::debug!("Network changed mid-scan; skipping store of discovered identity"); - return Ok(()); + return Ok(false); } let identity_id = identity.id(); @@ -242,41 +275,33 @@ impl AppContext { let mut qualified_identity = self .build_qualified_identity_from_wallet( - sdk, + settings.sdk, identity, wallet, - allow_prompt, + settings.allow_prompt, identity_index, ) .await?; - match self.get_identity_by_id(&identity_id)? { - Some(existing) => { - // Carry DET-only metadata onto the refreshed identity, then - // update in place — `update_local_qualified_identity` keeps the - // stored wallet association, and top-ups live under a separate - // KV key untouched by this write. - qualified_identity.alias = existing.alias; - self.update_local_qualified_identity(&qualified_identity)?; - } - None => { - self.insert_local_qualified_identity( - &qualified_identity, - &Some((seed_hash, identity_index)), - )?; - } - } - - if let Ok(mut wallet_guard) = wallet.write() { - wallet_guard - .identities - .insert(identity_index, qualified_identity.identity.clone()); + // One guarded call for both the refresh and the insert: the choice + // between them, the unload check and the alias carry-over all happen + // under a single hold of the identity's record lock, so a removal + // cannot land between this pass's read and its write. + let stored = self.store_discovered_identity( + &mut qualified_identity, + &Some((seed_hash, identity_index)), + settings.intent, + )?; + + adopt_discovered_identity(wallet, identity_index, &qualified_identity.identity, stored); + + if stored { + tracing::info!( + identity_id = %identity_id, + "Successfully loaded discovered identity" + ); } - tracing::info!( - identity_id = %identity_id, - "Successfully loaded discovered identity" - ); - Ok(()) + Ok(stored) } /// Build a QualifiedIdentity from a fetched Identity with wallet key derivation paths. @@ -439,3 +464,80 @@ impl AppContext { }) } } + +/// Adopt a discovered identity into the wallet's in-memory identity map, but +/// only when the store actually kept it. +/// +/// The map is a second place a refused identity comes back. The wallet views +/// read it directly, so an identity the user unloaded would reappear there for +/// the rest of the session even with storage correctly refusing it — a gate +/// whose answer the caller ignores is a check-then-act with extra steps. The +/// outcome is a parameter rather than a caller-side `if` so the coupling is +/// enforced in one place and can be tested without a network. +fn adopt_discovered_identity( + wallet: &Arc>, + identity_index: u32, + identity: &dash_sdk::platform::Identity, + stored: bool, +) { + if !stored { + return; + } + if let Ok(mut wallet_guard) = wallet.write() { + wallet_guard + .identities + .insert(identity_index, identity.clone()); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::model::wallet::test_support::open_wallet; + use dash_sdk::dpp::identity::Identity; + use dash_sdk::dpp::version::PlatformVersion; + use dash_sdk::platform::Identifier; + + fn identity(byte: u8) -> Identity { + Identity::create_basic_identity(Identifier::from([byte; 32]), PlatformVersion::latest()) + .expect("basic identity") + } + + /// A refusal the caller ignores is not a refusal. Storage declining an + /// unloaded identity is undone if the wallet's in-memory map adopts it + /// anyway: the wallet views read that map directly, so the identity the + /// user unloaded is back on screen until the app restarts. + #[test] + fn a_refused_discovery_leaves_the_wallets_identity_map_untouched() { + let wallet = Arc::new(RwLock::new(open_wallet())); + + adopt_discovered_identity(&wallet, 3, &identity(7), false); + + assert!( + wallet + .read() + .expect("read the wallet") + .identities + .is_empty(), + "an identity the store refused must not enter the wallet's identity map", + ); + } + + #[test] + fn a_stored_discovery_is_adopted_at_its_derivation_index() { + let wallet = Arc::new(RwLock::new(open_wallet())); + + adopt_discovered_identity(&wallet, 3, &identity(7), true); + + assert_eq!( + wallet + .read() + .expect("read the wallet") + .identities + .get(&3) + .map(|identity| identity.id()), + Some(Identifier::from([7u8; 32])), + "a stored identity must be adopted under the index it was found at", + ); + } +} diff --git a/src/backend_task/identity/load_identity_from_wallet.rs b/src/backend_task/identity/load_identity_from_wallet.rs index 7af2df26a..d165b5ddc 100644 --- a/src/backend_task/identity/load_identity_from_wallet.rs +++ b/src/backend_task/identity/load_identity_from_wallet.rs @@ -284,6 +284,10 @@ impl AppContext { &wallet_arc_ref.wallet, seed_identity_index, true, + // The user typed an index and pressed Search, so this pass is a + // request for what it finds — including an identity they + // unloaded earlier and are now asking for back. + crate::model::identity_discovery::DiscoveryIntent::UserRequested, Some(&sender), ) .await?; diff --git a/src/backend_task/identity/recover_legacy_keys.rs b/src/backend_task/identity/recover_legacy_keys.rs index 8dd6a5269..c42cec71c 100644 --- a/src/backend_task/identity/recover_legacy_keys.rs +++ b/src/backend_task/identity/recover_legacy_keys.rs @@ -1314,6 +1314,76 @@ mod tests { /// write section re-reads it, and the write must refuse rather than seal /// the merged keys under a password state nothing ever verified. /// + /// The unload-marker guard in `write_local_qualified_identity_locked` + /// cannot make this task's success report a lie, and this pins why: the + /// record is re-read *under the same guard the write takes*, so an identity + /// unloaded before the recovery started is refused at that read, and one + /// unloaded during it cannot land between the read and the write at all. + /// + /// Which also means no key material is re-created for an unloaded identity: + /// the refusal happens before any merged key is sealed into the vault. + /// + /// A refactor that moves the read out from under the guard, or that + /// tolerates an absent record here, would reach the write with nothing + /// stored — declined, while the task still reported keys restored. This + /// fails first if that happens. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn recovery_refuses_an_unloaded_identity_before_it_seals_anything() { + let offline = Offline::new(Some(Arc::new(TestPrompt::never()))).await; + let ctx = &offline.ctx; + + let owner = test_key(1, Purpose::OWNER, 0x1A); + let stranded = test_key(2, Purpose::TRANSFER, 0x2B); + let modern = identity_with_keys( + 0xB2, + IdentityType::Masternode, + &[&stranded], + vec![(M, &owner, owner.clear())], + ); + let identity_id = modern.identity.id(); + ctx.insert_local_qualified_identity(&modern, &None) + .expect("insert the keyless modern record"); + let legacy = identity_with_keys( + 0xB2, + IdentityType::Masternode, + &[], + vec![(M, &stranded, stranded.clear())], + ); + offline.stage_legacy(&legacy); + + let approved = vec![RecoveryItem::Key { + target: M, + key_id: 2, + }]; + + // The user unloads the identity, which destroys its keys and records + // the unload. + ctx.delete_local_qualified_identity(&identity_id) + .expect("unload the identity"); + + let error = ctx + .persist_legacy_recovery(identity_id, &approved, None) + .expect_err("recovery must refuse an identity that is no longer on this device"); + assert!( + matches!(error, TaskError::IdentityNotFoundLocally), + "expected IdentityNotFoundLocally, got {error:?}", + ); + assert!( + ctx.stored_identity_blob(&identity_id) + .expect("read stored blob") + .is_none(), + "recovery must not put the unloaded identity's record back", + ); + assert!( + offline + .with_keys(identity_id, |view| view.get(&M, 2).expect("read the vault")) + .is_none(), + "and must not seal a legacy key into the vault for it", + ); + + offline.shutdown().await; + } + /// Drives `persist_legacy_recovery` directly with the `None` the dry run /// produced, the way B5 drives `verify_recovery_password`: a Tier-1 restore /// never prompts, so there is no await point between the two for a diff --git a/src/backend_task/identity/remove_identity.rs b/src/backend_task/identity/remove_identity.rs index 6a43a5678..1395411f5 100644 --- a/src/backend_task/identity/remove_identity.rs +++ b/src/backend_task/identity/remove_identity.rs @@ -3,6 +3,24 @@ use crate::context::AppContext; use dash_sdk::dpp::identity::accessors::IdentityGettersV0; use dash_sdk::platform::Identifier; +/// Fail-closed roster check for the post-delisting-failure distinction +/// below: an unreadable index cannot prove `id` is gone, so treat it as +/// still listed (the safer of the two wrong answers — a real failure stays +/// a real failure) rather than silently swallowing the read error. +fn still_listed_or_assume_so(app_context: &AppContext, id: &Identifier) -> bool { + match app_context.is_identity_listed(id) { + Ok(listed) => listed, + Err(error) => { + tracing::warn!( + ?error, + identity_id = %id, + "Could not read the identity index to check removal outcome; assuming still listed" + ); + true + } + } +} + impl AppContext { pub(super) fn remove_identity( &self, @@ -16,7 +34,34 @@ impl AppContext { .map(|(voter_identity, _)| voter_identity.id()) }); - self.delete_local_qualified_identity(&identity_id)?; + // `delete_local_qualified_identity` removes an identity from the + // Global index before its irreversible vault delete, so a failure + // can land strictly after that identity is already gone from every + // screen — a durable vault-cleanup manifest (see that method's doc + // comment on its `Err` postcondition) guarantees the next boot's + // sweep finishes the job regardless. `is_identity_listed` tells that + // case apart from a real failure: reporting a post-delisting failure + // as an outright removal failure would tell the user their + // already-gone identity is still there and safe to retry, when + // neither is true and no UI control remains to reach it with anyway. + // Applied to both identities this call can touch — the primary and, + // below, its associated voter twin — since the same failure window + // exists on both `delete_local_qualified_identity` calls. + let mut cleanup_deferred = false; + match self.delete_local_qualified_identity(&identity_id) { + Ok(()) => {} + Err(error) => { + if still_listed_or_assume_so(self, &identity_id) { + return Err(error); + } + cleanup_deferred = true; + tracing::warn!( + ?error, + %identity_id, + "Identity removed but its vault cleanup is still pending; the next boot's sweep will finish it" + ); + } + } let mut removed_identity_ids = vec![identity_id]; let mut associated_cleanup_failed = false; @@ -24,12 +69,22 @@ impl AppContext { match self.delete_local_qualified_identity(&voter_id) { Ok(()) => removed_identity_ids.push(voter_id), Err(error) => { - associated_cleanup_failed = true; - tracing::warn!( - ?error, - voter_identity_id = %voter_id, - "Associated voter identity cleanup failed" - ); + if still_listed_or_assume_so(self, &voter_id) { + associated_cleanup_failed = true; + tracing::warn!( + ?error, + voter_identity_id = %voter_id, + "Associated voter identity cleanup failed" + ); + } else { + removed_identity_ids.push(voter_id); + cleanup_deferred = true; + tracing::warn!( + ?error, + voter_identity_id = %voter_id, + "Associated voter identity removed but its vault cleanup is still pending; the next boot's sweep will finish it" + ); + } } } } @@ -37,6 +92,187 @@ impl AppContext { Ok(BackendTaskSuccessResult::RemovedIdentities { identity_ids: removed_identity_ids, associated_cleanup_failed, + cleanup_deferred, }) } } + +#[cfg(test)] +mod tests { + use super::*; + use crate::backend_task::BackendTaskSuccessResult; + use crate::context::test_staging::{ + StagedIdentity, fail_removal_before_delisting, fail_removals_after_delisting, + stage_identity_with_vaulted_keys, stage_identity_with_voter_twin, + }; + use crate::model::qualified_identity::PrivateKeyTarget; + use crate::wallet_backend::IdentityKeyView; + + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0xA1; 32]; + const MEDIUM: [u8; 32] = [0xB2; 32]; + + /// Unpack the only success shape this task produces. + fn removal_outcome(result: BackendTaskSuccessResult) -> (Vec, bool, bool) { + match result { + BackendTaskSuccessResult::RemovedIdentities { + identity_ids, + associated_cleanup_failed, + cleanup_deferred, + } => (identity_ids, associated_cleanup_failed, cleanup_deferred), + other => panic!("removing an identity must report a removal, got {other:?}"), + } + } + + /// Whether any of the identity's keys are still in the vault. + fn keys_remain(staged: &StagedIdentity, id: Identifier) -> bool { + let view = IdentityKeyView::new(&staged.store, id.to_buffer()); + [1, 2] + .iter() + .any(|key_id| view.get(&MAIN, *key_id).unwrap().is_some()) + } + + /// A removal that fails before the identity is delisted did not happen: + /// the identity is still on every screen, its keys are still on the + /// device, and the button that started this still works. Reporting it as a + /// success would strand a user who can plainly see the identity is still + /// there. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_removal_that_fails_before_delisting_is_reported_as_a_failure() { + let staged = stage_identity_with_vaulted_keys(HIGH, MEDIUM).await; + fail_removal_before_delisting(&staged.ctx, &staged.id); + + staged + .ctx + .remove_identity(staged.id) + .expect_err("a removal that never delisted the identity is a failure"); + + assert!( + staged + .ctx + .is_identity_listed(&staged.id) + .expect("read listed state"), + "the identity must still be listed, so the user's retry is reachable" + ); + assert!( + keys_remain(&staged, staged.id), + "nothing was deleted, so every key must still be in the vault" + ); + } + + /// The distinction this classifier exists for. Once `index_remove_identity` + /// has run the identity is off every screen and no UI control can reach it + /// again, so a later failure is not something the user can retry — it is a + /// completed removal whose vault cleanup the next boot's sweep finishes. + /// Reporting it as an outright failure would tell them an identity they can + /// no longer see is still there. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_removal_that_fails_after_delisting_is_a_removal_with_a_deferred_cleanup() { + let staged = stage_identity_with_vaulted_keys(HIGH, MEDIUM).await; + fail_removals_after_delisting(&staged.ctx); + + let (identity_ids, associated_cleanup_failed, cleanup_deferred) = removal_outcome( + staged + .ctx + .remove_identity(staged.id) + .expect("a delisted identity is removed, whatever failed afterwards"), + ); + + assert_eq!(identity_ids, vec![staged.id]); + assert!( + cleanup_deferred, + "the vault cleanup is outstanding and must be reported as such" + ); + assert!( + !associated_cleanup_failed, + "there is no associated identity here, so nothing may be blamed on one" + ); + assert!( + !staged + .ctx + .is_identity_listed(&staged.id) + .expect("read listed state"), + "the identity really is delisted; that is what makes this not a failure" + ); + assert!( + keys_remain(&staged, staged.id), + "the deferral must be real: the keys the sweep will collect are still there" + ); + } + + /// A voter twin that fails *after* its own delisting is in the same + /// position as the primary: gone from storage, cleanup pending. It belongs + /// in the removed set — counting it as an associated-cleanup failure would + /// invite the user to retry an identity that no longer exists. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_voter_twin_that_fails_after_delisting_still_counts_as_removed() { + let staged = stage_identity_with_voter_twin(HIGH, MEDIUM).await; + let voter_id = staged.voter_id.expect("the fixture stages a voter twin"); + fail_removals_after_delisting(&staged.ctx); + + let (identity_ids, associated_cleanup_failed, cleanup_deferred) = removal_outcome( + staged + .ctx + .remove_identity(staged.id) + .expect("both identities are delisted, whatever failed afterwards"), + ); + + assert_eq!( + identity_ids, + vec![staged.id, voter_id], + "both identities were delisted, so both count as removed" + ); + assert!( + !associated_cleanup_failed, + "the voter identity was removed; only its vault cleanup is pending" + ); + assert!(cleanup_deferred, "both cleanups are outstanding"); + assert!( + !staged + .ctx + .is_identity_listed(&voter_id) + .expect("read listed state"), + "the voter identity really is delisted" + ); + } + + /// The two outcomes are independent and can land in the same removal: the + /// primary delisted with its cleanup pending, the voter twin never removed + /// at all. Neither may mask the other — the user has one identity to retry + /// and one they will never see again, and needs to be told both. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_failed_voter_cleanup_and_a_deferred_primary_cleanup_are_reported_together() { + let staged = stage_identity_with_voter_twin(HIGH, MEDIUM).await; + let voter_id = staged.voter_id.expect("the fixture stages a voter twin"); + fail_removals_after_delisting(&staged.ctx); + fail_removal_before_delisting(&staged.ctx, &voter_id); + + let (identity_ids, associated_cleanup_failed, cleanup_deferred) = removal_outcome( + staged + .ctx + .remove_identity(staged.id) + .expect("the primary is delisted, so this is still a removal"), + ); + + assert_eq!( + identity_ids, + vec![staged.id], + "the voter identity was never removed, so it must not be listed as removed" + ); + assert!( + associated_cleanup_failed, + "the voter identity is still on file and its failure must be named" + ); + assert!( + cleanup_deferred, + "the primary's pending vault cleanup must survive the voter's failure" + ); + assert!( + staged + .ctx + .is_identity_listed(&voter_id) + .expect("read listed state"), + "the voter identity must still be listed, so its retry is reachable" + ); + } +} diff --git a/src/backend_task/migration/finish_unwire.rs b/src/backend_task/migration/finish_unwire.rs index 6856156be..a74722e12 100644 --- a/src/backend_task/migration/finish_unwire.rs +++ b/src/backend_task/migration/finish_unwire.rs @@ -1708,7 +1708,16 @@ fn migrate_identities( "Importing an identity whose wallet is not present; the link is kept so it re-attaches when that wallet is restored", ); } - app_context.insert_local_qualified_identity(qi, wallet) + // Gated exactly as a discovery pass is, and through the same + // critical section: an import is the machine finding an identity, + // never the user asking for one back, so it consults the unload + // marker and never retires it. + let mut qi = qi.clone(); + app_context.store_discovered_identity( + &mut qi, + wallet, + crate::model::identity_discovery::DiscoveryIntent::Automatic, + ) }, )?; @@ -1716,6 +1725,7 @@ fn migrate_identities( target = "migration::finish_unwire", imported = outcome.imported, skipped_existing = outcome.skipped_existing, + skipped_unloaded = outcome.skipped_unloaded, unreadable = outcome.unreadable, network = ?network, "Identity migration pass complete", @@ -1759,7 +1769,7 @@ fn migrate_identities_from_conn( where R: FnMut(&BTreeSet<[u8; 32]>) -> Result<(), MigrationError>, H: FnMut([u8; 32]) -> Result, - I: FnMut(&QualifiedIdentity, &Option<(WalletSeedHash, u32)>) -> Result<(), TaskError>, + I: FnMut(&QualifiedIdentity, &Option<(WalletSeedHash, u32)>) -> Result, { let import_failed = |source: TaskError| MigrationError::IdentityImportFailed { source: Box::new(source), @@ -1805,7 +1815,16 @@ where // The wallet link travels verbatim, never nulled — even when that wallet // did not come across: it is what re-attaches the identity when the // wallet is restored or unlocked later. - insert(&row.qi, &row.wallet).map_err(import_failed)?; + if !insert(&row.qi, &row.wallet).map_err(import_failed)? { + // Counted apart from `skipped_existing`, which would state + // something untrue about why this row was left out: nothing is in + // the store, and the row was declined rather than found redundant. + // Not recorded as processed either — a later pass re-offers it, and + // is declined again unless the user has loaded the identity back by + // then, which is the only thing that should change the answer. + outcome.skipped_unloaded = outcome.skipped_unloaded.saturating_add(1); + continue; + } processed.insert(row.id); record_processed(processed)?; outcome.imported = outcome.imported.saturating_add(1); @@ -1821,6 +1840,11 @@ struct IdentityMigrationOutcome { imported: u32, /// Identities already in the store and therefore left untouched. skipped_existing: u32, + /// Identities the user unloaded from this device, which the import must not + /// hand back. Distinct from `skipped_existing`: nothing is in the store for + /// these, and saying otherwise would make the migration report misstate its + /// own reason for skipping them. + skipped_unloaded: u32, /// Legacy rows that could not be decoded. Recorded as a durable /// [`UnreadableIdentitiesWarning`] so the user still hears about them, but /// never fails the pass — the identities that *did* decode must not be held @@ -3572,7 +3596,7 @@ mod tests { .imported .borrow_mut() .push(qi.identity.id().to_buffer()); - Ok(()) + Ok(true) }, ) .expect("an undecodable row must not fail the pass"); @@ -3617,7 +3641,7 @@ mod tests { .imported .borrow_mut() .push(qi.identity.id().to_buffer()); - Ok(()) + Ok(true) }, ) .expect("import"); @@ -3655,7 +3679,7 @@ mod tests { .imported .borrow_mut() .push(qi.identity.id().to_buffer()); - Ok(()) + Ok(true) }, ) .expect("presence check"); @@ -3694,7 +3718,7 @@ mod tests { if id == failing { Err(TaskError::WalletNotFound) } else { - Ok(()) + Ok(true) } }, ); @@ -3712,7 +3736,7 @@ mod tests { |_| Ok(false), |qi, _| { attempts.borrow_mut().push(qi.identity.id().to_buffer()); - Ok(()) + Ok(true) }, ) .expect("retry"); @@ -3749,7 +3773,7 @@ mod tests { .imported .borrow_mut() .push(qi.identity.id().to_buffer()); - Ok(()) + Ok(true) }, ) .expect("import"); @@ -3789,7 +3813,7 @@ mod tests { |_| Ok(false), |_, wallet| { links.borrow_mut().push(*wallet); - Ok(()) + Ok(true) }, ) .expect("import"); @@ -5291,6 +5315,76 @@ mod tests { ); } + /// The import is gated on the unload marker itself, not only on the + /// progress set. That matters because the progress set does not cover every + /// unload: `record_identity_deletion` is a no-op once migration reports + /// success, and the devnet wipe clears identities without touching it at + /// all — so an import that trusted progress alone would hand those back. + /// + /// The pass here is handed an empty progress set, which is what a lost or + /// never-written progress key looks like. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn an_unloaded_identity_is_declined_by_the_import_and_counted_apart() { + let tmp = tempfile::tempdir().expect("tempdir"); + let ctx = fresh_app_context(tmp.path()); + wire_backend(&ctx).await; + let backend = ctx.wallet_backend().expect("backend wired"); + let unloaded = [0x66; 32]; + ctx.delete_local_qualified_identity(&Identifier::from(unloaded)) + .expect("unload through the normal path"); + + let conn = Connection::open_in_memory().expect("in-memory db"); + identities::create_identity_table(&conn); + identities::insert_identity( + &conn, + unloaded, + Some(identities::identity_blob(unloaded)), + true, + ); + + let inserted = std::cell::Cell::new(false); + let outcome = migrate_identities_from_conn( + &conn, + ctx.network, + &mut BTreeSet::new(), + |_| Ok(()), + |_| Ok(false), + |qi, wallet| { + inserted.set(true); + let mut qi = qi.clone(); + ctx.store_discovered_identity( + &mut qi, + wallet, + crate::model::identity_discovery::DiscoveryIntent::Automatic, + ) + .map_err(|_| TaskError::WalletNotFound) + }, + ) + .expect("migration pass"); + + assert!( + inserted.get(), + "precondition: the row must reach the import, or this proves nothing about the gate" + ); + assert_eq!( + outcome.skipped_unloaded, 1, + "an unloaded identity must be counted as declined" + ); + assert_eq!( + outcome.skipped_existing, 0, + "and never as already present, which would state something untrue about why \ + it was left out" + ); + assert_eq!(outcome.imported, 0, "nothing was imported"); + assert!( + !ctx.has_local_qualified_identity(&Identifier::from(unloaded)) + .expect("read identity store"), + "the unloaded identity must stay off this device" + ); + + backend.shutdown().await; + } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn deletion_first_keeps_a_pending_legacy_identity_absent() { let tmp = tempfile::tempdir().expect("tempdir"); @@ -5322,7 +5416,7 @@ mod tests { |_| Ok(false), |_, _| { inserted.set(true); - Ok(()) + Ok(true) }, ) .expect("migration pass"); diff --git a/src/backend_task/mod.rs b/src/backend_task/mod.rs index e75273dd1..e648d05bd 100644 --- a/src/backend_task/mod.rs +++ b/src/backend_task/mod.rs @@ -777,6 +777,23 @@ pub enum BackendTaskSuccessResult { RemovedIdentities { identity_ids: Vec, associated_cleanup_failed: bool, + /// Set when the primary identity's, the associated voter identity's, + /// or both ones' cleanup failed strictly after they were already + /// delisted (index removal succeeded), so removal itself is done and + /// irreversible; only a k/v drain — which includes, but is not limited + /// to, the vault-key delete — is unfinished. + /// + /// `true` means every identity named in `identity_ids` is already gone + /// from every screen and there is no "try again" affordance left for + /// the user to reach one with. It does NOT establish that key material + /// is still present: the step that failed may be the scope purge for a + /// keyless identity, whose cleanup manifest names no placements. Nor + /// does it establish that the residue will be gone by the next launch: + /// the boot sweep is best-effort, skipped while a storage update runs, + /// and retains the manifest whenever the purge or vault delete fails + /// again. Callers may promise another automatic attempt; they may not + /// promise presence or completion. + cleanup_deferred: bool, }, RefreshedIdentity(QualifiedIdentity), LoadedIdentity(QualifiedIdentity), diff --git a/src/context/contract_token_db.rs b/src/context/contract_token_db.rs index 4631a24a2..e7a15df00 100644 --- a/src/context/contract_token_db.rs +++ b/src/context/contract_token_db.rs @@ -821,7 +821,7 @@ where #[cfg(test)] mod tests { use super::*; - use crate::wallet_backend::kv_test_support::InMemoryKv; + use crate::wallet_backend::kv_test_support::{InMemoryKv, StallingReadKv}; use platform_wallet_storage::{KvError, KvStore, ObjectId}; use std::sync::{Arc, Mutex}; @@ -1061,40 +1061,6 @@ mod tests { // token reappearing on the next refresh. // ---------------------------------------------------------------- - /// Delegates to [`InMemoryKv`], stalling *after* each read has taken its - /// snapshot. Two concurrent mutations therefore both observe the - /// pre-mutation state and write back late: a mutation that reads before it - /// writes loses its peer's update, while a mutation that only writes never - /// reads, never stalls, and cannot be clobbered. - #[derive(Default)] - struct StallingReadKv { - inner: InMemoryKv, - } - - impl KvStore for StallingReadKv { - fn get(&self, scope: &ObjectId, key: &str) -> Result>, KvError> { - let value = self.inner.get(scope, key); - std::thread::sleep(std::time::Duration::from_millis(200)); - value - } - - fn put(&self, scope: &ObjectId, key: &str, value: &[u8]) -> Result<(), KvError> { - self.inner.put(scope, key, value) - } - - fn delete(&self, scope: &ObjectId, key: &str) -> Result<(), KvError> { - self.inner.delete(scope, key) - } - - fn list_keys( - &self, - scope: &ObjectId, - prefix: Option<&str>, - ) -> Result, KvError> { - self.inner.list_keys(scope, prefix) - } - } - fn stalling_kv() -> DetKv { DetKv::from_store(Arc::new(StallingReadKv::default())) } diff --git a/src/context/identity_db.rs b/src/context/identity_db.rs index 4edf0ac81..fa0d797aa 100644 --- a/src/context/identity_db.rs +++ b/src/context/identity_db.rs @@ -1,6 +1,7 @@ use super::AppContext; use crate::backend_task::contested_names::ScheduledDPNSVote; use crate::backend_task::error::TaskError; +use crate::model::identity_discovery::DiscoveryIntent; use crate::model::qualified_identity::{ DPNSNameInfo, IdentityStatus, IdentityType, PrivateKeyTarget, QualifiedIdentity, }; @@ -11,11 +12,12 @@ use crate::wallet_backend::{DetKv, DetScope, KvAdapterError}; use dash_sdk::dpp::dashcore::Network; use dash_sdk::dpp::identity::KeyID; use dash_sdk::dpp::identity::accessors::IdentityGettersV0; +use dash_sdk::dpp::platform_value::string_encoding::Encoding; use dash_sdk::dpp::voting::vote_choices::resource_vote_choice::ResourceVoteChoice; use dash_sdk::platform::Identifier; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; -use std::sync::{Arc, RwLock}; +use std::sync::{Arc, Mutex, MutexGuard, RwLock}; /// Identity blob slot, scoped to [`DetScope::Identity`]. One entry per /// identity; the identity id is carried by the scope, so the key is a @@ -53,6 +55,88 @@ const SCHEDULED_VOTE_VOTERS_KEY: &str = "det:scheduled_vote_voters:v1"; /// identity; the identity id is carried by the scope. const TOP_UPS_KEY: &str = "det:top_ups:v1"; +/// Durable manifest of vault-key placements still pending deletion for one +/// identity, keyed by that identity's id in the key itself. +/// [`DetScope::Global`] is deliberate, not incidental: `purge_identity_scope` +/// only ever mutates [`DetScope::Identity`], so a manifest filed there would +/// share fate with the very state a partial `purge_identity_scope` failure +/// can destroy — the one case this manifest exists to survive. Key shape: +/// `det:vault_cleanup_pending:v1:`. +const VAULT_CLEANUP_PENDING_PREFIX: &str = "det:vault_cleanup_pending:v1:"; + +/// Per-identity marker recording that the user deliberately unloaded an +/// identity from this device. Global-scoped for the same reason the +/// vault-cleanup manifest is: an `Identity`-scoped key is reaped by +/// [`purge_identity_scope`] and the upstream cascade at exactly the moment +/// this record needs to start existing. +const IDENTITY_UNLOADED_PREFIX: &str = "det:identity_unloaded:v1:"; + +fn identity_unloaded_key(id: &[u8; 32]) -> String { + format!( + "{IDENTITY_UNLOADED_PREFIX}{}", + Identifier::from(*id).to_string(Encoding::Base58) + ) +} + +fn vault_cleanup_pending_key(id: &[u8; 32]) -> String { + format!( + "{VAULT_CLEANUP_PENDING_PREFIX}{}", + Identifier::from(*id).to_string(Encoding::Base58) + ) +} + +/// Recover the identity id named by a vault-cleanup manifest key, for the +/// boot-time sweep that enumerates every manifest without already knowing +/// which identities they belong to. `None` for anything that is not a +/// well-formed manifest key — a corrupt or foreign entry is skipped rather +/// than treated as fatal, so it never blocks the sweep from resuming every +/// other pending cleanup. +fn parse_vault_cleanup_pending_key(key: &str) -> Option<[u8; 32]> { + let suffix = key.strip_prefix(VAULT_CLEANUP_PENDING_PREFIX)?; + Identifier::from_string(suffix, Encoding::Base58) + .ok() + .map(|id| id.to_buffer()) +} + +/// Serializable mirror of [`PrivateKeyTarget`] for the vault-cleanup +/// manifest. [`PrivateKeyTarget`] itself derives `bincode::{Encode, Decode}` +/// for the vault blob's own wire format, not `serde` — the k/v sidecar is +/// serde-based, so the manifest needs its own small serializable shape. +/// +/// `DetKv` encodes values with `bincode::serde::encode_to_vec`, which is +/// positional (see `wallet_backend/kv.rs`'s `SCHEMA_VERSION` doc comment): +/// this enum's wire representation is its variants' *declaration order*, +/// not their names. Adding a variant is fine (compiler-caught at every +/// match site); reordering or removing an existing one silently re-labels +/// every already-persisted manifest entry to the wrong key target — +/// nothing decoding it would notice. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)] +enum StoredPrivateKeyTarget { + Main, + Voter, + Operator, +} + +impl From for StoredPrivateKeyTarget { + fn from(target: PrivateKeyTarget) -> Self { + match target { + PrivateKeyTarget::PrivateKeyOnMainIdentity => Self::Main, + PrivateKeyTarget::PrivateKeyOnVoterIdentity => Self::Voter, + PrivateKeyTarget::PrivateKeyOnOperatorIdentity => Self::Operator, + } + } +} + +impl From for PrivateKeyTarget { + fn from(target: StoredPrivateKeyTarget) -> Self { + match target { + StoredPrivateKeyTarget::Main => Self::PrivateKeyOnMainIdentity, + StoredPrivateKeyTarget::Voter => Self::PrivateKeyOnVoterIdentity, + StoredPrivateKeyTarget::Operator => Self::PrivateKeyOnOperatorIdentity, + } + } +} + fn scheduled_vote_key(contested_name: &str) -> String { format!("{SCHEDULED_VOTE_KEY_PREFIX}{contested_name}") } @@ -237,8 +321,58 @@ impl From for ScheduledDPNSVote { } } +/// Serializes every read-modify-write of [`IDENTITY_INDEX_KEY`]. +/// +/// The roster is a single blob rewritten wholesale, so two identities being +/// listed or delisted at the same time race on it: each pass reads the whole +/// index, edits its own entry, and writes the result back over its peer's. +/// The per-identity `identity_record_lock` cannot close that window — it +/// serializes writers of the *same* identity, while this race is between +/// writers of *different* ones. +/// +/// A lost entry is not cosmetic. [`AppContext::resume_pending_vault_cleanups`] +/// treats absence from this roster as proof an identity was removed and +/// deletes its vault keys, so an import whose write is clobbered hands a live +/// identity's private keys to the next boot's sweep. One process-wide lock +/// rather than a per-context field because the mutators are free functions +/// reached from contexts and bare [`DetKv`] handles alike; the writes are +/// user-paced (import, removal), so serializing them globally costs nothing. +/// +/// # Lock order +/// +/// An identity's `identity_record_lock` is OUTER, this lock is INNER — +/// [`AppContext::insert_local_qualified_identity`] and +/// [`AppContext::delete_local_qualified_identity`] both already hold the +/// record lock when their index write takes this one, so that is the order +/// every path must keep. Never acquire a record lock while holding this one, +/// and never hold it across an `await` or any call that can reach the index +/// again — `std::sync::Mutex` is not reentrant, so a second acquisition on +/// this thread self-deadlocks. Only +/// [`AppContext::delete_all_local_qualified_identities_in_devnet`] holds it +/// across other work, and every call it makes there (`clear_identity_vault_keys`, +/// `purge_identity_scope`) touches the vault and Identity-scoped keys only, +/// never the roster and never a record lock. +static IDENTITY_INDEX_LOCK: Mutex<()> = Mutex::new(()); + +/// Acquire [`IDENTITY_INDEX_LOCK`]. A poisoned lock guards no invariant of its +/// own — the k/v store holds the state — so the guard is taken regardless. +fn lock_identity_index() -> MutexGuard<'static, ()> { + // Counted only for the probe the calling thread attached itself to, so a + // fixture waiting on "my peer is parked here" cannot be answered by an + // unrelated test's lock traffic. Dropped once the lock is acquired. + #[cfg(test)] + let _waiting = + crate::context::lock_probe::enter_wait(crate::context::lock_probe::LockSite::RosterWait); + IDENTITY_INDEX_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) +} + /// Read the Global identity-id enumeration index. Returns an empty /// vector when the index has never been written. +/// +/// Callers that go on to write the index back, or that act irreversibly on +/// the answer, must hold [`lock_identity_index`] across both halves. fn load_identity_index(kv: &DetKv) -> std::result::Result, TaskError> { Ok(kv .get::>(DetScope::Global, IDENTITY_INDEX_KEY) @@ -246,9 +380,20 @@ fn load_identity_index(kv: &DetKv) -> std::result::Result, TaskErr .unwrap_or_default()) } +/// Whether `identity_id` is on the Global enumeration index. +/// +/// Read under [`lock_identity_index`] so the answer is never a half-written +/// roster. Callers hold the identity's record guard, keeping the documented +/// order — record OUTER, index INNER. +fn identity_is_listed(kv: &DetKv, identity_id: &[u8; 32]) -> std::result::Result { + let _index_guard = lock_identity_index(); + Ok(load_identity_index(kv)?.contains(identity_id)) +} + /// Add `identity_id` to the Global enumeration index if absent. No-op /// when the id is already tracked, so repeated inserts stay idempotent. fn index_add_identity(kv: &DetKv, identity_id: &[u8; 32]) -> std::result::Result<(), TaskError> { + let _index_guard = lock_identity_index(); let mut index = load_identity_index(kv)?; if index.contains(identity_id) { return Ok(()); @@ -261,6 +406,7 @@ fn index_add_identity(kv: &DetKv, identity_id: &[u8; 32]) -> std::result::Result /// Remove `identity_id` from the Global enumeration index. No-op when /// the id is not present. fn index_remove_identity(kv: &DetKv, identity_id: &[u8; 32]) -> std::result::Result<(), TaskError> { + let _index_guard = lock_identity_index(); let mut index = load_identity_index(kv)?; let before = index.len(); index.retain(|id| id != identity_id); @@ -271,6 +417,39 @@ fn index_remove_identity(kv: &DetKv, identity_id: &[u8; 32]) -> std::result::Res .map_err(identity_err) } +/// Run the irreversible tail of an identity removal: delete `vault_keys` +/// from the vault, then drop the manifest that recorded them. +/// +/// Returns `Err` only while key material may still be on the device. A vault +/// delete that fails is propagated, never swallowed — leaving keys on a device +/// the user asked to clear is the one part of a removal they must not be told +/// succeeded. A manifest clear that fails afterwards is not that: the keys are +/// already gone and only Global bookkeeping is stale, so it is logged and the +/// removal counts as complete. Reporting it as a failure would tell the user +/// their private keys are still here — the opposite of what happened — and +/// would surface a cleanup-pending warning for nothing. The stale manifest is +/// harmless: the next boot's sweep re-runs the same idempotent deletes and +/// clears it. +fn finish_vault_cleanup( + kv: &DetKv, + secret_store: &Arc, + id: &[u8; 32], + vault_keys: impl IntoIterator, +) -> std::result::Result<(), TaskError> { + crate::wallet_backend::IdentityKeyView::new(secret_store, *id).delete_all(vault_keys)?; + if let Err(error) = kv + .delete(DetScope::Global, &vault_cleanup_pending_key(id)) + .map_err(identity_err) + { + tracing::warn!( + identity = %Identifier::from(*id), + %error, + "Identity keys deleted but their cleanup manifest could not be cleared; the next boot re-runs the deletes and clears it" + ); + } + Ok(()) +} + /// Delete every Identity-scoped child of `id` (blob, top-up history, all /// scheduled votes) and prune the scheduled-vote voter index. Does not /// touch the Global identity index — callers decide whether to drop the @@ -621,6 +800,26 @@ impl AppContext { let _guard = lock .lock() .unwrap_or_else(std::sync::PoisonError::into_inner); + // Deliberate: asking for an identity is the one act that means the user + // changed their mind about unloading it, so this is the only place the + // unload marker is retired. Under the same guard as the write, so a + // discovery pass cannot observe a retired marker and a missing record. + self.clear_identity_unloaded( + &self.det_kv()?, + &qualified_identity.identity.id().to_buffer(), + )?; + self.insert_local_qualified_identity_locked(qualified_identity, wallet_and_identity_id_info) + } + + /// The body of [`Self::insert_local_qualified_identity`], for a caller that + /// already holds this identity's + /// [`identity_record_lock`](Self::identity_record_lock) — the record lock is + /// not reentrant, so a gated caller cannot go through the public method. + fn insert_local_qualified_identity_locked( + &self, + qualified_identity: &QualifiedIdentity, + wallet_and_identity_id_info: &Option<(WalletSeedHash, u32)>, + ) -> std::result::Result<(), TaskError> { let kv = self.det_kv()?; let mut wallet_link = match wallet_and_identity_id_info { Some((seed, idx)) => Some((*seed, *idx)), @@ -876,6 +1075,47 @@ impl AppContext { /// [`identity_record_lock`](Self::identity_record_lock) across a wider /// read-modify-write span. Calling it without that guard reopens the /// lost-update race the guard exists to close. + /// + /// # Declining an unloaded identity + /// + /// This method creates a record when none exists — an "update" of an + /// absent identity writes the blob and re-lists it on the roster. That is + /// deliberate and useful for a caller repairing a missing roster entry, but + /// it also means every update path can resurrect an identity the user + /// unloaded, if the removal lands between that caller's read and its write. + /// So a write for an identity that is **off the roster** and carries an + /// unload marker is declined and reported `Ok(())`. + /// + /// Roster membership, not blob presence, is what decides that. They differ + /// exactly where it matters: `purge_identity_scope` runs *after* + /// `index_remove_identity` and is three independent writes, so a removal + /// that stops in between leaves the identity delisted with its blob intact. + /// A blob-presence test passes such a write straight through, and + /// `index_add_identity` below then puts the identity back on the roster + /// after its removal was reported complete — where the boot sweep reads + /// that entry as proof the identity is live and spares the keys the removal + /// existed to destroy. Roster absence is the authority that sweep already + /// acts on; this guard asks the same question of the same source. + /// + /// It also settles the opposite case correctly. A marker written by a + /// removal that failed *before* delisting belongs to an identity still on + /// the roster, still on every screen, whose removal never happened — so + /// the write proceeds, which is what a user retrying that identity's + /// ordinary work expects. + /// + /// That is a completion, not a swallowed failure: the request was to bring + /// the stored record up to date, there is no stored record, and a marker + /// says the absence is what the user asked for. The requested end state — + /// no unwanted record — is exactly what the caller gets. + /// + /// # Invariant this rests on + /// + /// **A path that should create a record retires the marker first.** Today + /// that is [`Self::insert_local_qualified_identity`], which clears it under + /// the same record guard as the write, and + /// [`Self::store_discovered_identity`] for a user-requested pass. A new + /// deliberate-insert path that skips that step will find its writes + /// silently declined for any identity the user once unloaded. pub(crate) fn write_local_qualified_identity_locked( &self, qualified_identity: &QualifiedIdentity, @@ -885,6 +1125,13 @@ impl AppContext { let scope = DetScope::Identity(&id); let existing: Option = kv.get(scope, IDENTITY_KEY).map_err(identity_err)?; + if !identity_is_listed(&kv, &id)? && self.is_identity_unloaded(&kv, &id)? { + tracing::debug!( + identity_id = %qualified_identity.identity.id(), + "Declined to store an identity that is off this device's roster and was unloaded from it" + ); + return Ok(()); + } let (wallet_hash, wallet_index) = existing .as_ref() .map(|s| (s.wallet_hash, s.wallet_index)) @@ -1048,6 +1295,20 @@ impl AppContext { .collect()) } + /// Whether `id` is currently on the Global identity index — the roster + /// every screen reads. Distinct from [`Self::get_local_qualified_identity`], + /// which reads the per-identity blob: a [`Self::delete_local_qualified_identity`] + /// failure can leave the blob already gone (an early `purge_identity_scope` + /// step) while the index removal that actually delists the identity ran + /// even earlier, so only the index membership answers "is this identity + /// still reachable from the UI". + pub(crate) fn is_identity_listed( + &self, + id: &Identifier, + ) -> std::result::Result { + identity_is_listed(&self.det_kv()?, &id.to_buffer()) + } + /// Read one stored qualified identity by id, hydrated like the list loads /// (status, wallet index, network, wallets, secret access). `None` when no /// identity with `id` is stored. Backs the load-path existence check @@ -1381,6 +1642,18 @@ impl AppContext { /// here is re-issued by the next boot's /// `AppContext::reconcile_unowned_identities`, which withdraws every /// unowned registration whose sidecar record is gone. + /// + /// **`Err` does not mean "nothing happened".** The Global index removal + /// runs before the irreversible vault-key delete, so a failure can land + /// strictly after `identifier` is already gone from every screen — a + /// durable vault-cleanup manifest survives that and the next boot's + /// [`Self::resume_pending_vault_cleanups`] finishes the job regardless. + /// A caller that turns this `Err` straight into a user-facing "removal + /// failed, please retry" message is wrong in exactly that case: the + /// identity cannot be retried (it is unlisted) and nothing is actually + /// still broken. Call [`Self::is_identity_listed`] first to tell the two + /// outcomes apart — see `AppContext::remove_identity` (in + /// `backend_task/identity/remove_identity.rs`) for the reference pattern. pub fn delete_local_qualified_identity( &self, identifier: &Identifier, @@ -1405,9 +1678,29 @@ impl AppContext { source: Arc::new(source), }, )?; - self.clear_identity_vault_keys(&kv, &id)?; - purge_identity_scope(&kv, &id)?; + // Ordering is a safety property. The vault delete is the only step + // nothing can undo — Platform can re-supply the identity, but no one + // can re-supply its keys — so it runs last, once the identity is + // already unlisted and drained. Its delete set is read up front, + // because the blob `purge_identity_scope` drops is where that set is + // recorded. + // + // `purge_identity_scope` is itself not atomic (three independent k/v + // writes), so a failure inside it — after its own first write has + // already dropped the blob — would leave a retry with nothing to + // re-derive the delete set from. The manifest below is what survives + // that: persisted before any mutation runs, retained across every + // error, and cleared only once every listed key is confirmed absent. + let vault_keys = self.pending_vault_key_placements(&kv, &id)?; + self.persist_vault_cleanup_manifest(&kv, &id, &vault_keys)?; + // Before delisting, so a failure between the two leaves a marker for a + // still-listed identity — inert, since discovery only consults it when + // storage says the identity is absent. The reverse order would leave a + // delisted identity with no marker, which is the defect itself. + self.mark_identity_unloaded(&kv, &id)?; index_remove_identity(&kv, &id)?; + purge_identity_scope(&kv, &id)?; + finish_vault_cleanup(&kv, &self.secret_store, &id, vault_keys)?; // Mirror the removal into the wallet store's unowned scope, so a // deleted node does not linger there. Best-effort, and a no-op for a // wallet-owned identity, which is never registered unowned. @@ -1423,6 +1716,157 @@ impl AppContext { Ok(()) } + /// Boot-time sweep for vault-cleanup manifests left behind by a + /// [`Self::delete_local_qualified_identity`] call that failed after + /// `index_remove_identity` had already run. Once an identity leaves the + /// Global index it renders on no screen, so nothing in the UI can ever + /// call that method again for it — the manifest, and this sweep, are the + /// only surviving path back to the orphaned vault keys. + /// + /// Resumes a manifest only while its identity is absent from that index, + /// re-checked fresh under that identity's record lock (see below) rather + /// than from one snapshot read before the loop — a snapshot would miss a + /// concurrent re-import that lists the identity again mid-sweep. A + /// manifest whose identity is listed belongs to a removal that never + /// reached the irreversible step, or to an identity re-imported since: + /// either way the identity is live and the user keeps a working retry, so + /// deleting its keys here would strand exactly the identity this ordering + /// exists to protect. That absence is only trustworthy because every + /// mutation of the roster is serialized ([`lock_identity_index`]) and + /// every mutation *of this identity* is serialized against this sweep by + /// its record lock: without the first, another identity's concurrent + /// import could silently drop this one's entry and fake the very evidence + /// the delete below acts on. + /// + /// Also re-runs `purge_identity_scope` before deleting vault keys: the + /// manifest is persisted one step *before* that purge, so a crash between + /// the two leaves it incomplete, and every one of its steps is a delete- + /// if-present or list-then-conditional-prune, so re-running it against an + /// already-purged scope is a safe no-op. + /// + /// Best-effort and idempotent, like every other boot reconcile + /// ([`super::wallet_lifecycle::bootstrap`]'s unowned-identity pass): a + /// failure on one manifest is logged and retried next boot, and never + /// blocks the sweep from resuming every other one. + /// + /// Guarded by the same `migration_run` lock and in-progress check as + /// [`Self::delete_local_qualified_identity`]: a storage migration can be + /// mid-rewrite of this same Identity scope around the same boot window + /// this sweep runs in, and the sweep's purge/vault-delete pair is not + /// safe to interleave with that. + pub(crate) fn resume_pending_vault_cleanups(&self) { + let Ok(_migration_guard) = self.migration_run.try_lock() else { + tracing::debug!( + "Pending vault-cleanup sweep skipped; a storage migration is running, will retry at next boot" + ); + return; + }; + if self.migration_status().state().is_in_progress() { + tracing::debug!( + "Pending vault-cleanup sweep skipped; a storage migration is in progress, will retry at next boot" + ); + return; + } + let kv = match self.det_kv() { + Ok(kv) => kv, + Err(error) => { + tracing::debug!( + %error, + "Pending vault-cleanup sweep skipped; k/v store not ready, will retry at next boot" + ); + return; + } + }; + let keys = match kv.list(DetScope::Global, Some(VAULT_CLEANUP_PENDING_PREFIX)) { + Ok(keys) => keys, + Err(error) => { + tracing::warn!( + %error, + "Pending vault-cleanup sweep skipped; listing manifests failed, will retry at next boot" + ); + return; + } + }; + let mut resumed = 0usize; + for key in keys { + let Some(id) = parse_vault_cleanup_pending_key(&key) else { + tracing::warn!(%key, "Skipping an unparsable vault-cleanup manifest key"); + continue; + }; + // Held through the roster re-check, the purge, the vault delete, + // and the manifest clear — the same lock `insert_local_qualified_identity` + // takes before ever touching this identity's k/v, so a concurrent + // re-import cannot land between the "still listed?" check below + // and this sweep's own writes. + let lock = self.identity_record_lock(Identifier::from(id)); + let _record_guard = lock + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + // The manifest is persisted one step *before* `index_remove_identity`, + // so its presence alone does not mean the removal ever happened. An + // unreadable index cannot prove any identity is gone, so resume nothing + // rather than guess. Read fresh, under the lock, on every iteration — + // never hoisted above the loop — so a re-import that lands between + // manifests is always seen. + let listed = match load_identity_index(&kv) { + Ok(listed) => listed, + Err(error) => { + tracing::warn!( + %error, + "Pending vault-cleanup sweep skipped; the identity index is unreadable, will retry at next boot" + ); + return; + } + }; + if listed.contains(&id) { + tracing::debug!( + identity = %Identifier::from(id), + "Pending vault-cleanup left alone; this identity is still listed and still usable, so removing it stays the user's call" + ); + continue; + } + let placements: Vec<(StoredPrivateKeyTarget, KeyID)> = + match kv.get(DetScope::Global, &key) { + Ok(Some(placements)) => placements, + // Raced with another clear of the same manifest; nothing left to do. + Ok(None) => continue, + Err(error) => { + tracing::warn!( + identity = %Identifier::from(id), + %error, + "Pending vault-cleanup manifest unreadable, will retry at next boot" + ); + continue; + } + }; + if let Err(error) = purge_identity_scope(&kv, &id) { + tracing::warn!( + identity = %Identifier::from(id), + %error, + "Pending vault-cleanup deferred; scope purge incomplete, will retry at next boot" + ); + continue; + } + let vault_keys = placements + .into_iter() + .map(|(target, key_id)| (target.into(), key_id)); + match finish_vault_cleanup(&kv, &self.secret_store, &id, vault_keys) { + Ok(()) => resumed += 1, + Err(error) => tracing::warn!( + identity = %Identifier::from(id), + %error, + "Pending vault-cleanup deferred; will retry at next boot" + ), + } + } + if resumed > 0 { + tracing::info!( + resumed, + "Resumed vault-key cleanups left behind by an interrupted identity removal" + ); + } + } + /// Test-only: remove `identifier` from the Global enumeration index /// without touching the upstream unowned scope or any other /// Identity-scoped data. Simulates a sidecar delete whose upstream @@ -1498,22 +1942,212 @@ impl AppContext { kv.put(scope, IDENTITY_KEY, &stored).map_err(identity_err) } - /// Delete every identity-key raw secret for `id` from the vault. - /// Idempotent when the identity or an individual vault label is absent. - fn clear_identity_vault_keys( + /// The vault placements holding `id`'s identity-key secrets, as recorded in + /// its stored blob. Empty when the identity is not stored. + /// + /// Read this before anything drops the identity blob — the blob is the + /// only record of which vault labels belong to `id`, so a delete set + /// derived after it is gone is silently empty and strands every secret + /// in the vault. + fn identity_vault_key_placements( &self, kv: &DetKv, id: &[u8; 32], - ) -> std::result::Result<(), TaskError> { + ) -> std::result::Result, TaskError> { let Some(stored) = kv .get::(DetScope::Identity(id), IDENTITY_KEY) .map_err(identity_err)? else { - return Ok(()); + return Ok(std::collections::BTreeSet::new()); }; - let qi = decode_stored_identity(&stored.qi_bytes, self.network)?; - let view = crate::wallet_backend::IdentityKeyView::new(&self.secret_store, *id); - view.delete_all(qi.private_keys.keys_set()) + Ok(decode_stored_identity(&stored.qi_bytes, self.network)? + .private_keys + .keys_set()) + } + + /// The full vault-key delete set for `id`: freshly-derived placements + /// from the still-live blob (empty once it is gone), unioned with any + /// manifest left behind by an earlier failed delete. The union — never a + /// choice of one source over the other — means a placement discovered + /// by either source is never dropped, whether this is a first attempt + /// (manifest empty, blob live) or a retry after the blob was already + /// purged (blob empty, manifest live). + fn pending_vault_key_placements( + &self, + kv: &DetKv, + id: &[u8; 32], + ) -> std::result::Result, TaskError> { + let mut keys = self.identity_vault_key_placements(kv, id)?; + let manifest: Vec<(StoredPrivateKeyTarget, KeyID)> = kv + .get(DetScope::Global, &vault_cleanup_pending_key(id)) + .map_err(identity_err)? + .unwrap_or_default(); + keys.extend(manifest.into_iter().map(|(t, k)| (t.into(), k))); + Ok(keys) + } + + /// Record that the user deliberately unloaded `id` from this device, so + /// automatic discovery does not silently put it back. + /// + /// Removal destroys the identity's private keys and tells the user a backup + /// is now the only way in. Nothing else on disk distinguishes "the user + /// removed this" from "this was never here": the owning wallet stays + /// loaded, and every automatic discovery pass re-derives the same identity + /// and finds it absent. Presence of this key is the whole semantic; the + /// stored unix timestamp is for diagnostics only. + /// + /// The set is never reaped except by a deliberate re-load, and that is + /// intentional — an expiring tombstone is a resurrection with a delay. It + /// grows by one 8-byte entry per identity the user has actually unloaded on + /// this network, so it is bounded by user action rather than by anything + /// automatic. A change that lets some automatic path write these would + /// break that bound and needs to be weighed here. + /// + /// # Why the migration import consults this too + /// + /// The legacy `data.db` rows are never deleted, so the material to + /// re-import an unloaded identity is permanent: the door does not close + /// when the upgrade succeeds, it needs only one failed attempt at any point + /// afterwards. `record_identity_deletion` covers most of that — a removal + /// records its id in the migration progress set, and the retry skips it — + /// but not all of it. That recording is a no-op once migration reports + /// success, and the devnet wipe clears identities without touching the + /// progress set at all, so a wipe followed by a resumed pass would hand + /// every cleared identity back. This marker is consulted independently of + /// migration state and of the progress key, so the guarantee does not rest + /// on a condition the user cannot see. + fn mark_identity_unloaded( + &self, + kv: &DetKv, + id: &[u8; 32], + ) -> std::result::Result<(), TaskError> { + let unloaded_at = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0, |since_epoch| since_epoch.as_secs()); + kv.put(DetScope::Global, &identity_unloaded_key(id), &unloaded_at) + .map_err(identity_err) + } + + /// Whether `id` was deliberately unloaded and not deliberately re-loaded + /// since. + fn is_identity_unloaded( + &self, + kv: &DetKv, + id: &[u8; 32], + ) -> std::result::Result { + Ok(kv + .get::(DetScope::Global, &identity_unloaded_key(id)) + .map_err(identity_err)? + .is_some()) + } + + /// Retire `id`'s unload marker. Called only from the deliberate insert + /// path: asking for an identity back is the one thing that means the user + /// changed their mind. + fn clear_identity_unloaded( + &self, + kv: &DetKv, + id: &[u8; 32], + ) -> std::result::Result<(), TaskError> { + kv.delete(DetScope::Global, &identity_unloaded_key(id)) + .map_err(identity_err) + } + + /// Store an identity a discovery pass produced, refreshing the existing + /// record when there is one. + /// + /// Returns `true` when the identity was stored, `false` when a + /// [`DiscoveryIntent::Automatic`] pass was refused because the user + /// unloaded the identity from this device. A + /// [`DiscoveryIntent::UserRequested`] pass is the user asking for it back, + /// so it retires the marker and stores. + /// + /// The whole decision — the marker read, the existing-record read whose + /// alias is carried onto `qualified_identity`, and the write — happens + /// under one acquisition of this identity's record lock. That is what makes + /// the refusal hold against a discovery already in flight, whichever way it + /// interleaves: a pass that read the record before the removal cannot write + /// it back afterwards, because its decision is re-taken here rather than + /// carried in from the caller. + /// + /// The alias carry-over lives here for the same reason. Reading it in the + /// caller and writing it here spans an unguarded gap, so a concurrent alias + /// edit would be written away. + pub(crate) fn store_discovered_identity( + &self, + qualified_identity: &mut QualifiedIdentity, + wallet: &Option<(WalletSeedHash, u32)>, + intent: DiscoveryIntent, + ) -> std::result::Result { + let identity_id = qualified_identity.identity.id(); + let lock = self.identity_record_lock(identity_id); + #[cfg(test)] + let waiting = crate::context::lock_probe::enter_wait( + crate::context::lock_probe::LockSite::DiscoveryRecordWait, + ); + let _record_guard = lock + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + #[cfg(test)] + drop(waiting); + + let kv = self.det_kv()?; + let id = identity_id.to_buffer(); + if self.is_identity_unloaded(&kv, &id)? { + if !intent.may_restore_unloaded() { + tracing::debug!( + identity_id = %identity_id, + "Discovery left out an identity that was unloaded from this device" + ); + return Ok(false); + } + tracing::info!( + identity_id = %identity_id, + "Loading back an identity that was unloaded from this device, because the user asked for it" + ); + self.clear_identity_unloaded(&kv, &id)?; + } + + let existing: Option = kv + .get(DetScope::Identity(&id), IDENTITY_KEY) + .map_err(identity_err)?; + match existing { + // A record on file: refresh it, keeping the user's own alias, which + // a freshly built identity never carries. + Some(stored) => { + qualified_identity.alias = + decode_stored_identity(&stored.qi_bytes, self.network)?.alias; + self.write_local_qualified_identity_locked(qualified_identity)?; + } + None => self.insert_local_qualified_identity_locked(qualified_identity, wallet)?, + } + Ok(true) + } + + /// Persist `keys` as the durable vault-cleanup manifest for `id`, so a + /// failure anywhere between this call and the manifest clear in + /// [`finish_vault_cleanup`] leaves a record a retry can recover from. + fn persist_vault_cleanup_manifest( + &self, + kv: &DetKv, + id: &[u8; 32], + keys: &std::collections::BTreeSet<(PrivateKeyTarget, KeyID)>, + ) -> std::result::Result<(), TaskError> { + let stored: Vec<(StoredPrivateKeyTarget, KeyID)> = + keys.iter().cloned().map(|(t, k)| (t.into(), k)).collect(); + kv.put(DetScope::Global, &vault_cleanup_pending_key(id), &stored) + .map_err(identity_err) + } + + /// Delete every identity-key raw secret for `id` from the vault. + /// Idempotent when the identity or an individual vault label is absent. + fn clear_identity_vault_keys( + &self, + kv: &DetKv, + id: &[u8; 32], + ) -> std::result::Result<(), TaskError> { + let placements = self.identity_vault_key_placements(kv, id)?; + crate::wallet_backend::IdentityKeyView::new(&self.secret_store, *id).delete_all(placements) } /// Delete the vault secrets filed at `placements` for `identity_id`, leaving @@ -1540,6 +2174,12 @@ impl AppContext { /// current network. Matches the pre-C7 /// `delete_all_local_qualified_identities_in_devnet` guard — no-op on /// non-devnet networks. + /// + /// Holds [`lock_identity_index`] across the whole wipe, the read and the + /// index delete alike: an import that landed in between would have its + /// roster entry dropped by the delete while its blob and vault keys + /// survived — the unlisted-but-live shape the cleanup sweep destroys keys + /// over. pub fn delete_all_local_qualified_identities_in_devnet( &self, ) -> std::result::Result<(), TaskError> { @@ -1547,10 +2187,15 @@ impl AppContext { return Ok(()); } let kv = self.det_kv()?; + let _index_guard = lock_identity_index(); let ids = load_identity_index(&kv)?; for id in &ids { self.clear_identity_vault_keys(&kv, id)?; purge_identity_scope(&kv, id)?; + // The same unload the per-identity removal records: this destroys + // the same keys, and the wallets stay loaded, so without it the + // next discovery pass restores everything the wipe just cleared. + self.mark_identity_unloaded(&kv, id)?; } kv.delete(DetScope::Global, IDENTITY_INDEX_KEY) .map_err(identity_err) @@ -1744,33 +2389,428 @@ impl AppContext { } } +/// Test-only staging fixtures for the identity-removal paths. +/// +/// Lives outside `mod tests` so `backend_task::identity::remove_identity` can +/// stage the same on-disk shape this module's own tests do — the classifier it +/// owns reads exactly the state written here, and had no way to reach it while +/// these fixtures were private to this file's test module. #[cfg(test)] -mod tests { +pub(crate) mod test_staging { use super::*; - use crate::wallet_backend::kv_test_support::InMemoryKv; - use DetKv; - use std::sync::Arc; + use crate::model::qualified_identity::encrypted_key_storage::{ + KeyStorage, PrivateKeyData, WalletDerivationPath, + }; + use crate::model::qualified_identity::qualified_identity_public_key::QualifiedIdentityPublicKey; + use crate::model::qualified_identity::{IdentityStatus, IdentityType, PrivateKeyTarget}; + use crate::wallet_backend::IdentityKeyView; + use dash_sdk::dpp::identity::Identity; + use dash_sdk::dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dash_sdk::dpp::key_wallet::bip32::DerivationPath; + use dash_sdk::dpp::version::PlatformVersion; + use dash_sdk::platform::IdentityPublicKey; - fn empty_kv() -> DetKv { - DetKv::from_store(Arc::new(InMemoryKv::default())) + /// An offline `AppContext` over a throwaway data dir, plus the very vault it + /// was built on so a test can probe what the context wrote. + pub(crate) async fn ctx_with_vault() -> ( + Arc, + Arc, + tempfile::TempDir, + ) { + ctx_with_vault_on(Network::Testnet).await } - fn id(b: u8) -> [u8; 32] { - [b; 32] + /// [`ctx_with_vault`] on a caller-chosen network, for the devnet-only + /// sweep, which no-ops everywhere else. + pub(crate) async fn ctx_with_vault_on( + network: Network, + ) -> ( + Arc, + Arc, + tempfile::TempDir, + ) { + use crate::app_dir::ensure_env_file; + use crate::context::connection_status::ConnectionStatus; + use crate::database::test_helpers::create_database_at_path; + use crate::utils::tasks::TaskManager; + + let temp_dir = tempfile::tempdir().expect("tempdir"); + let data_dir = temp_dir.path().to_path_buf(); + ensure_env_file(&data_dir); + let db = Arc::new(create_database_at_path(&data_dir.join("data.db")).expect("db")); + let app_kv = AppContext::open_app_kv(&data_dir).expect("app kv"); + let secret_store = AppContext::open_secret_store(&data_dir).expect("secret store"); + let ctx = AppContext::new( + data_dir, + network, + db, + Arc::new(TaskManager::new()), + Arc::new(ConnectionStatus::new()), + egui::Context::default(), + app_kv, + Arc::clone(&secret_store), + crate::model::user_role::UserRoleCell::default(), + ) + .expect("offline AppContext::new"); + (ctx, secret_store, temp_dir) } - /// Minimal stored-identity blob carrying a synthetic `qi_bytes` - /// payload and the chosen type label. Avoids constructing a full - /// `QualifiedIdentity` (which needs an SDK identity) for storage-layer - /// tests. - fn stored(identity_type: &str) -> StoredQualifiedIdentity { - StoredQualifiedIdentity { - qi_bytes: vec![0xAB; 16], - status: 0, - identity_type: identity_type.to_string(), - wallet_hash: None, - wallet_index: None, - } + /// A `QualifiedIdentity` carrying one `Clear` (HIGH), one `AlwaysClear` + /// (MEDIUM), and one `AtWalletDerivationPath` key, under the default + /// (all-zero) identity id. + pub(crate) fn qi_with_plaintext_and_derived( + secret_high: [u8; 32], + secret_medium: [u8; 32], + ) -> QualifiedIdentity { + qi_with_plaintext_and_derived_at(Identifier::default(), secret_high, secret_medium) + } + + /// [`qi_with_plaintext_and_derived`] under a caller-chosen identity id, for + /// the fixtures that stage two related identities at once. + pub(crate) fn qi_with_plaintext_and_derived_at( + identity_id: Identifier, + secret_high: [u8; 32], + secret_medium: [u8; 32], + ) -> QualifiedIdentity { + let pv = PlatformVersion::latest(); + let mut ks = KeyStorage::default(); + let high = IdentityPublicKey::random_key(1, Some(1), pv); + ks.insert_at( + (PrivateKeyTarget::PrivateKeyOnMainIdentity, high.id()), + ( + QualifiedIdentityPublicKey::from(high), + PrivateKeyData::Clear(secret_high), + ), + ); + let medium = IdentityPublicKey::random_key(2, Some(2), pv); + ks.insert_at( + (PrivateKeyTarget::PrivateKeyOnMainIdentity, medium.id()), + ( + QualifiedIdentityPublicKey::from(medium), + PrivateKeyData::AlwaysClear(secret_medium), + ), + ); + let derived = IdentityPublicKey::random_key(3, Some(3), pv); + ks.insert_at( + (PrivateKeyTarget::PrivateKeyOnMainIdentity, derived.id()), + ( + QualifiedIdentityPublicKey::from(derived), + PrivateKeyData::AtWalletDerivationPath(WalletDerivationPath { + wallet_seed_hash: [0x07; 32], + derivation_path: DerivationPath::from(vec![]), + }), + ), + ); + let identity = Identity::create_basic_identity(identity_id, pv).expect("basic identity"); + QualifiedIdentity { + identity, + associated_voter_identity: None, + associated_operator_identity: None, + associated_owner_key_id: None, + identity_type: IdentityType::User, + alias: None, + private_keys: ks, + dpns_names: vec![], + associated_wallets: BTreeMap::new(), + secret_access: None, + wallet_index: None, + top_ups: BTreeMap::new(), + status: IdentityStatus::Active, + network: Network::Testnet, + } + } + + /// A stored identity whose plaintext keys the read path has already moved + /// into the vault — the state a real delete runs against. Holds the temp dir + /// and the event receiver so neither is dropped while the test runs. + pub(crate) struct StagedIdentity { + pub(crate) ctx: Arc, + pub(crate) store: Arc, + pub(crate) id: Identifier, + /// The associated voter identity, staged alongside `id` and named by + /// its record. `None` unless the fixture staged a voter twin. + pub(crate) voter_id: Option, + _dir: tempfile::TempDir, + _events: tokio::sync::mpsc::Receiver, + } + + /// The two vault placements [`qi_with_plaintext_and_derived_at`] leaves + /// behind once the read path has migrated its plaintext keys. + const STAGED_PLACEMENTS: [(PrivateKeyTarget, dash_sdk::dpp::identity::KeyID); 2] = [ + (PrivateKeyTarget::PrivateKeyOnMainIdentity, 1), + (PrivateKeyTarget::PrivateKeyOnMainIdentity, 2), + ]; + + /// An offline context with its wallet backend wired — `det_kv()` is only + /// reachable once it is. + async fn staged_context() -> ( + Arc, + Arc, + tempfile::TempDir, + tokio::sync::mpsc::Receiver, + ) { + staged_context_on(Network::Testnet).await + } + + async fn staged_context_on( + network: Network, + ) -> ( + Arc, + Arc, + tempfile::TempDir, + tokio::sync::mpsc::Receiver, + ) { + staged_context_with_prompt(network, None).await + } + + /// [`staged_context_on`] with a secret-prompt host installed *before* the + /// wallet backend is built, which is the only point at which the prompt is + /// read into the `SecretAccess` chokepoint — so a test that needs a + /// password-protected identity has to pass it here rather than install it + /// afterwards. + async fn staged_context_with_prompt( + network: Network, + prompt: Option>, + ) -> ( + Arc, + Arc, + tempfile::TempDir, + tokio::sync::mpsc::Receiver, + ) { + let (ctx, store, dir) = ctx_with_vault_on(network).await; + if let Some(prompt) = prompt { + ctx.install_secret_prompt(prompt); + } + let (tx, events) = tokio::sync::mpsc::channel::(32); + ctx.ensure_wallet_backend(crate::utils::egui_mpsc::SenderAsync::new( + tx, + ctx.egui_ctx().clone(), + )) + .await + .expect("wire the wallet backend offline"); + (ctx, store, dir, events) + } + + /// Write `qi`'s blob, list it on the roster, and read it back once so the + /// load path moves its plaintext keys into the vault. Asserts both keys + /// landed there, since every fixture below is only meaningful if they did. + fn stage_identity_record( + ctx: &Arc, + store: &Arc, + qi: &QualifiedIdentity, + ) { + let id = qi.identity.id(); + let id_buf = id.to_buffer(); + let kv = ctx.det_kv().expect("identity kv"); + kv.put( + DetScope::Identity(&id_buf), + IDENTITY_KEY, + &StoredQualifiedIdentity { + qi_bytes: qi.to_bytes(), + status: qi.status.as_u8(), + identity_type: qi.identity_type.as_tag().to_string(), + wallet_hash: None, + wallet_index: None, + }, + ) + .expect("stage the identity blob"); + index_add_identity(&kv, &id_buf).expect("index the identity"); + ctx.get_local_qualified_identity(&id) + .expect("hydrate the staged identity") + .expect("identity present"); + + let view = IdentityKeyView::new(store, id_buf); + for (target, key_id) in STAGED_PLACEMENTS { + assert!( + view.get(&target, key_id).unwrap().is_some(), + "precondition: key {key_id} must be in the vault before the delete" + ); + } + } + + /// One staged identity with no voter twin. + pub(crate) async fn stage_identity_with_vaulted_keys( + high: [u8; 32], + medium: [u8; 32], + ) -> StagedIdentity { + stage_identity_with_vaulted_keys_on(Network::Testnet, high, medium).await + } + + /// [`stage_identity_with_vaulted_keys`] on a caller-chosen network. + pub(crate) async fn stage_identity_with_vaulted_keys_on( + network: Network, + high: [u8; 32], + medium: [u8; 32], + ) -> StagedIdentity { + stage_identity_with_vaulted_keys_using_prompt(network, None, high, medium).await + } + + /// [`stage_identity_with_vaulted_keys_on`] with a secret-prompt host, for a + /// test that has to seal or unseal a password-protected identity key. + pub(crate) async fn stage_identity_with_vaulted_keys_using_prompt( + network: Network, + prompt: Option>, + high: [u8; 32], + medium: [u8; 32], + ) -> StagedIdentity { + let (ctx, store, dir, events) = staged_context_with_prompt(network, prompt).await; + let qi = qi_with_plaintext_and_derived(high, medium); + stage_identity_record(&ctx, &store, &qi); + StagedIdentity { + ctx, + store, + id: qi.identity.id(), + voter_id: None, + _dir: dir, + _events: events, + } + } + + /// Break the Global scheduled-vote voter index, so the next + /// `delete_local_qualified_identity` fails at the last step of + /// `purge_identity_scope` — strictly *after* `index_remove_identity` has + /// already delisted the identity. The reachable shape of a removal that + /// failed past its point of no return: the identity is gone from every + /// screen, and only its vault cleanup is outstanding. + pub(crate) fn fail_removals_after_delisting(ctx: &Arc) { + ctx.det_kv() + .expect("identity kv") + .put( + DetScope::Global, + SCHEDULED_VOTE_VOTERS_KEY, + &"not a voter index".to_string(), + ) + .expect("corrupt the scheduled-vote voter index"); + } + + /// Break `identity_id`'s vault-cleanup manifest slot, so its next + /// `delete_local_qualified_identity` fails while reading the delete set — + /// before any mutation, and before the identity is delisted. The reachable + /// shape of a removal that never happened and can still be retried. + pub(crate) fn fail_removal_before_delisting(ctx: &Arc, identity_id: &Identifier) { + ctx.det_kv() + .expect("identity kv") + .put( + DetScope::Global, + &vault_cleanup_pending_key(&identity_id.to_buffer()), + &"not a cleanup manifest".to_string(), + ) + .expect("corrupt the vault-cleanup manifest"); + } + + /// An evonode identity whose record names a separate voter identity, with + /// both staged and both holding vaulted keys — the shape + /// `AppContext::remove_identity` walks when one removal has to delete two + /// identities. + pub(crate) async fn stage_identity_with_voter_twin( + high: [u8; 32], + medium: [u8; 32], + ) -> StagedIdentity { + let (ctx, store, dir, events) = staged_context().await; + + let voter = qi_with_plaintext_and_derived_at(Identifier::from([0x2A; 32]), high, medium); + stage_identity_record(&ctx, &store, &voter); + + let mut primary = + qi_with_plaintext_and_derived_at(Identifier::from([0x1B; 32]), high, medium); + primary.identity_type = IdentityType::Evonode; + primary.associated_voter_identity = Some(( + voter.identity.clone(), + IdentityPublicKey::random_key(4, Some(4), PlatformVersion::latest()), + )); + stage_identity_record(&ctx, &store, &primary); + + StagedIdentity { + ctx, + store, + id: primary.identity.id(), + voter_id: Some(voter.identity.id()), + _dir: dir, + _events: events, + } + } +} + +#[cfg(test)] +mod tests { + use super::test_staging::*; + use super::*; + use crate::context::lock_probe::{LockProbe, LockSite}; + use crate::wallet_backend::kv_test_support::{FailingKv, InMemoryKv, RendezvousKv}; + use DetKv; + use std::sync::Arc; + + /// Hammer the roster lock from a thread attached to no probe, for as long + /// as the returned handle lives. + /// + /// The busy neighbour, made explicit rather than left to whatever else the + /// suite happens to be running: the roster lock is process-wide and shared + /// with every other test, so a fixture that counts *all* of its waiters + /// can be released by traffic its own test never created. Running that + /// traffic on purpose turns "does the probe ignore strangers" from a hope + /// about scheduling into a condition of the test. + struct NoisyNeighbour { + stop: Arc, + handle: Option>, + } + + impl NoisyNeighbour { + fn spawn() -> Self { + let stop = Arc::new(std::sync::atomic::AtomicBool::new(false)); + let stop_flag = Arc::clone(&stop); + // Its own store, so it contends on the roster lock without ever + // entering the rendezvous the test under way is using. + let handle = std::thread::spawn(move || { + let kv = empty_kv(); + let mut n = 0u8; + while !stop_flag.load(std::sync::atomic::Ordering::SeqCst) { + let _ = index_add_identity(&kv, &id(n)); + let _ = index_remove_identity(&kv, &id(n)); + n = n.wrapping_add(1); + } + }); + Self { + stop, + handle: Some(handle), + } + } + } + + impl Drop for NoisyNeighbour { + fn drop(&mut self) { + self.stop.store(true, std::sync::atomic::Ordering::SeqCst); + if let Some(handle) = self.handle.take() { + let _ = handle.join(); + } + } + } + + fn rendezvous_kv() -> (Arc, DetKv) { + let store = Arc::new(RendezvousKv::default()); + let kv = DetKv::from_store(store.clone()); + (store, kv) + } + + fn empty_kv() -> DetKv { + DetKv::from_store(Arc::new(InMemoryKv::default())) + } + + fn id(b: u8) -> [u8; 32] { + [b; 32] + } + + /// Minimal stored-identity blob carrying a synthetic `qi_bytes` + /// payload and the chosen type label. Avoids constructing a full + /// `QualifiedIdentity` (which needs an SDK identity) for storage-layer + /// tests. + fn stored(identity_type: &str) -> StoredQualifiedIdentity { + StoredQualifiedIdentity { + qi_bytes: vec![0xAB; 16], + status: 0, + identity_type: identity_type.to_string(), + wallet_hash: None, + wallet_index: None, + } } fn put_identity(kv: &DetKv, id: &[u8; 32], identity_type: &str) { @@ -1870,6 +2910,119 @@ mod tests { assert_eq!(load_identity_index(&kv).unwrap(), vec![id(2)]); } + /// A roster whose entries can be lost is a roster that authorizes an + /// irreversible delete on false evidence: `resume_pending_vault_cleanups` + /// reads "absent from the index" as proof an identity was removed and + /// destroys its vault keys. The index is a single blob rewritten + /// wholesale, so listing two identities at once is a read-modify-write + /// race — and the entry that loses it belongs to an identity that is + /// live, listed everywhere else, and about to lose its private keys on + /// the next boot. Every mutation of the key must be serialized. + /// + /// The rendezvous store makes that failure certain rather than likely: + /// both adds are released only once both hold the pre-add roster, so + /// unserialized code loses an entry on every run, on every runner. + #[test] + fn concurrently_listing_two_identities_keeps_both_on_the_roster() { + let _neighbour = NoisyNeighbour::spawn(); + let (store, kv) = rendezvous_kv(); + let probe = LockProbe::new(); + let watched = probe.clone(); + store.arm(2, move || watched.count(LockSite::RosterWait)); + + std::thread::scope(|scope| { + scope.spawn(|| { + let _attached = probe.attach(); + index_add_identity(&kv, &id(1)).expect("list the re-imported identity") + }); + scope.spawn(|| { + let _attached = probe.attach(); + index_add_identity(&kv, &id(2)).expect("list the imported identity") + }); + }); + + let mut listed = load_identity_index(&kv).unwrap(); + listed.sort_unstable(); + assert_eq!( + listed, + vec![id(1), id(2)], + "neither identity may lose its roster entry to the other's write" + ); + } + + /// The same race in its mixed form, which corrupts the roster in both + /// directions at once: the removal can be undone (a delisted identity + /// reappears on screen with its keys already deleted) or the addition can + /// be dropped (a live identity is handed to the cleanup sweep). Only one + /// of the two writes survives unless they are serialized. + #[test] + fn a_concurrent_listing_and_delisting_both_take_effect() { + let (store, kv) = rendezvous_kv(); + index_add_identity(&kv, &id(1)).unwrap(); + index_add_identity(&kv, &id(2)).unwrap(); + // Armed only now: the two seeding writes above are sequential, and an + // armed read waits for a peer that would never come. + let probe = LockProbe::new(); + let watched = probe.clone(); + store.arm(2, move || watched.count(LockSite::RosterWait)); + let _neighbour = NoisyNeighbour::spawn(); + + std::thread::scope(|scope| { + scope.spawn(|| { + let _attached = probe.attach(); + index_remove_identity(&kv, &id(1)).expect("delist the removed identity") + }); + scope.spawn(|| { + let _attached = probe.attach(); + index_add_identity(&kv, &id(3)).expect("list the imported identity") + }); + }); + + let mut listed = load_identity_index(&kv).unwrap(); + listed.sort_unstable(); + assert_eq!( + listed, + vec![id(2), id(3)], + "the removal must not resurrect the delisted identity, and the \ + import must not vanish from the roster" + ); + } + + // --------------------------------------------------------------- + // StoredPrivateKeyTarget: wire encoding is positional, not named. + // --------------------------------------------------------------- + + /// `DetKv` encodes with `bincode::serde::encode_to_vec`, which is + /// positional — a variant's wire identity is its declaration order, not + /// its name. This pins today's order so a future reorder or removal + /// fails loudly here instead of silently re-labelling every + /// already-persisted vault-cleanup manifest entry to the wrong key + /// target. A new variant appended at the end is fine and needs no + /// update; an insertion, removal, or reordering must update this test + /// deliberately, with a migration plan for existing manifests. + #[test] + fn stored_private_key_target_wire_order_is_pinned() { + for (variant, expected_index) in [ + (StoredPrivateKeyTarget::Main, 0u32), + (StoredPrivateKeyTarget::Voter, 1u32), + (StoredPrivateKeyTarget::Operator, 2u32), + ] { + let encoded = + bincode::serde::encode_to_vec(variant, bincode::config::standard()).unwrap(); + // bincode's derive/serde-adapter encodes a unit-only enum's + // discriminant as a `u32` varint prefix, with no payload after + // it for a fieldless variant — decode that prefix directly + // rather than asserting on raw bytes, which would be brittle + // to bincode's own varint format. + let (decoded_index, _): (u32, usize) = + bincode::serde::decode_from_slice(&encoded, bincode::config::standard()).unwrap(); + assert_eq!( + decoded_index, expected_index, + "{variant:?} must encode at wire position {expected_index}" + ); + } + } + // --------------------------------------------------------------- // Identity-type tag: writer and filter share one stable mapping. // --------------------------------------------------------------- @@ -2195,15 +3348,12 @@ mod tests { // Identity-key vault migration + deletion (funds-safety). // --------------------------------------------------------------- - use crate::model::qualified_identity::encrypted_key_storage::{ - KeyStorage, PrivateKeyData, WalletDerivationPath, - }; + use crate::model::qualified_identity::encrypted_key_storage::{KeyStorage, PrivateKeyData}; use crate::model::qualified_identity::qualified_identity_public_key::QualifiedIdentityPublicKey; use crate::model::qualified_identity::{IdentityType, PrivateKeyTarget}; use crate::wallet_backend::IdentityKeyView; use dash_sdk::dpp::identity::Identity; use dash_sdk::dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; - use dash_sdk::dpp::key_wallet::bip32::DerivationPath; use dash_sdk::dpp::version::PlatformVersion; use dash_sdk::platform::{Identifier, IdentityPublicKey}; @@ -2212,62 +3362,6 @@ mod tests { Arc::new(crate::wallet_backend::single_key::open_secret_store(&path).expect("open vault")) } - /// A `QualifiedIdentity` carrying one `Clear` (HIGH), one `AlwaysClear` - /// (MEDIUM), and one `AtWalletDerivationPath` key. Returns the QI plus the - /// `(target, key_id)` of each plaintext key for assertions. - fn qi_with_plaintext_and_derived( - secret_high: [u8; 32], - secret_medium: [u8; 32], - ) -> QualifiedIdentity { - let pv = PlatformVersion::latest(); - let mut ks = KeyStorage::default(); - let high = IdentityPublicKey::random_key(1, Some(1), pv); - ks.insert_at( - (PrivateKeyTarget::PrivateKeyOnMainIdentity, high.id()), - ( - QualifiedIdentityPublicKey::from(high), - PrivateKeyData::Clear(secret_high), - ), - ); - let medium = IdentityPublicKey::random_key(2, Some(2), pv); - ks.insert_at( - (PrivateKeyTarget::PrivateKeyOnMainIdentity, medium.id()), - ( - QualifiedIdentityPublicKey::from(medium), - PrivateKeyData::AlwaysClear(secret_medium), - ), - ); - let derived = IdentityPublicKey::random_key(3, Some(3), pv); - ks.insert_at( - (PrivateKeyTarget::PrivateKeyOnMainIdentity, derived.id()), - ( - QualifiedIdentityPublicKey::from(derived), - PrivateKeyData::AtWalletDerivationPath(WalletDerivationPath { - wallet_seed_hash: [0x07; 32], - derivation_path: DerivationPath::from(vec![]), - }), - ), - ); - let identity = - Identity::create_basic_identity(Identifier::default(), pv).expect("basic identity"); - QualifiedIdentity { - identity, - associated_voter_identity: None, - associated_operator_identity: None, - associated_owner_key_id: None, - identity_type: IdentityType::User, - alias: None, - private_keys: ks, - dpns_names: vec![], - associated_wallets: BTreeMap::new(), - secret_access: None, - wallet_index: None, - top_ups: BTreeMap::new(), - status: IdentityStatus::Active, - network: Network::Testnet, - } - } - /// Load-path migration — `migrate_keystore_to_vault` content-detects Clear/AlwaysClear, /// stores them in the vault FIRST, then rewrites the blob to InVault. /// Asserts: vault-first (the raw bytes are present), the wallet-derived key @@ -2745,39 +3839,6 @@ mod tests { ); } - /// An offline `AppContext` over a throwaway data dir, plus the very vault it - /// was built on so a test can probe what the context wrote. - async fn ctx_with_vault() -> ( - Arc, - Arc, - tempfile::TempDir, - ) { - use crate::app_dir::ensure_env_file; - use crate::context::connection_status::ConnectionStatus; - use crate::database::test_helpers::create_database_at_path; - use crate::utils::tasks::TaskManager; - - let temp_dir = tempfile::tempdir().expect("tempdir"); - let data_dir = temp_dir.path().to_path_buf(); - ensure_env_file(&data_dir); - let db = Arc::new(create_database_at_path(&data_dir.join("data.db")).expect("db")); - let app_kv = AppContext::open_app_kv(&data_dir).expect("app kv"); - let secret_store = AppContext::open_secret_store(&data_dir).expect("secret store"); - let ctx = AppContext::new( - data_dir, - Network::Testnet, - db, - Arc::new(TaskManager::new()), - Arc::new(ConnectionStatus::new()), - egui::Context::default(), - app_kv, - Arc::clone(&secret_store), - crate::model::user_role::UserRoleCell::default(), - ) - .expect("offline testnet AppContext::new"); - (ctx, secret_store, temp_dir) - } - /// Per-key vault deletion drops the placements it is given and nothing else. /// That is what separates it from `clear_identity_vault_keys`, which empties /// the identity: dropping one key must leave the identity's remaining keys — @@ -2819,4 +3880,1039 @@ mod tests { ctx.delete_identity_key_secrets(&victim, [(MAIN, 0)]) .expect("deleting an already-gone placement is not an error"); } + + /// Removal ordering is a safety property, not a style choice: the vault-key + /// delete is the one irreversible step (Platform can re-supply the identity, + /// nothing can re-supply the keys), so it must be unreachable until the + /// identity is already gone from local storage. Failing a step in the middle + /// of the delete must therefore leave every key where it was, rather than + /// leaving a "zombie" — an identity still on file whose keys are gone. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_failed_identity_delete_never_destroys_the_vault_keys() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0xAA; 32]; + const MEDIUM: [u8; 32] = [0xBB; 32]; + + let staged = stage_identity_with_vaulted_keys(HIGH, MEDIUM).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + + // Break the Global enumeration index so the delete fails partway: the + // index read no longer decodes, which is the same shape as a store that + // goes away mid-operation. + kv.put( + DetScope::Global, + IDENTITY_INDEX_KEY, + &"not an identity index".to_string(), + ) + .expect("corrupt the enumeration index"); + + let error = staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect_err("an unreadable index must fail the delete"); + assert!( + !matches!(error, TaskError::WalletStorageNotReady), + "the delete must reach the index step, not stop at the migration guard: {error:?}" + ); + + let view = IdentityKeyView::new(&staged.store, staged.id.to_buffer()); + assert_eq!( + *view + .get(&MAIN, 1) + .unwrap() + .expect("the HIGH key must survive a failed delete"), + HIGH, + ); + assert_eq!( + *view + .get(&MAIN, 2) + .unwrap() + .expect("the MEDIUM key must survive a failed delete"), + MEDIUM, + ); + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_some(), + "the identity record must survive a failed delete, so a retry still has something to delete" + ); + } + + /// `is_identity_listed` is the primitive `remove_identity` uses to tell a + /// benign "removed, cleanup still pending" failure apart from a real + /// "never removed" one: it must track the Global index, not the blob — + /// `purge_identity_scope`'s first step drops the blob before the index + /// removal that actually delists the identity has even run for the case + /// this distinction exists to catch (a failure strictly after delisting). + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn is_identity_listed_tracks_the_index_not_the_blob() { + let staged = stage_identity_with_vaulted_keys([0x77; 32], [0x88; 32]).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + + assert!( + staged + .ctx + .is_identity_listed(&staged.id) + .expect("read listed state"), + "a freshly staged identity must be listed" + ); + + // Drop the blob directly, leaving the index untouched — the inverse + // of the failure window this helper exists to distinguish. + kv.delete(DetScope::Identity(&staged.id.to_buffer()), IDENTITY_KEY) + .expect("drop the blob"); + assert!( + staged + .ctx + .is_identity_listed(&staged.id) + .expect("read listed state"), + "the index, not the blob, is authoritative: a blob-only removal \ + must still read as listed" + ); + + index_remove_identity(&kv, &staged.id.to_buffer()).expect("remove from the index"); + assert!( + !staged + .ctx + .is_identity_listed(&staged.id) + .expect("read listed state"), + "once the index entry is gone, the identity must read as unlisted" + ); + } + + /// The other half of the ordering contract. Deferring the vault delete must + /// not quietly turn it into a no-op: the delete set is read from the blob, + /// so reading it after `purge_identity_scope` drops that blob would strand + /// every private key in the vault. A completed removal leaves nothing. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_successful_identity_delete_leaves_no_orphaned_vault_key() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + + let staged = stage_identity_with_vaulted_keys([0xCC; 32], [0xDD; 32]).await; + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("delete the staged identity"); + + let view = IdentityKeyView::new(&staged.store, staged.id.to_buffer()); + for key_id in [1, 2] { + assert!( + view.get(&MAIN, key_id).unwrap().is_none(), + "key {key_id} must not be stranded in the vault after a completed delete" + ); + } + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "the identity record must be gone after a completed delete" + ); + assert!( + !load_identity_index(&staged.ctx.det_kv().expect("identity kv")) + .expect("read the index") + .contains(&staged.id.to_buffer()), + "the identity must be unlisted after a completed delete" + ); + } + + /// `purge_identity_scope` is not atomic: it can fail on its own last step + /// (pruning the scheduled-vote voter index) after its first step has + /// already deleted `IDENTITY_KEY` — the blob `identity_vault_key_placements` + /// needs to re-derive a delete set. Without a durable manifest, a retry + /// after such a failure would read an empty placement set from the (now + /// gone) blob and report success while every vault key stays orphaned. + /// The manifest persisted before the first mutation must survive that + /// failure and let a retry still find and delete every key. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_retry_after_purge_partially_fails_still_recovers_every_key() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0xEE; 32]; + const LOW: [u8; 32] = [0xFF; 32]; + + let staged = stage_identity_with_vaulted_keys(HIGH, LOW).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + + // Break the Global scheduled-vote voter index so `purge_identity_scope` + // fails at its LAST step (`delete_scheduled_votes_for_voter` -> + // `remove_vote_voter_from_index`) — strictly after `IDENTITY_KEY` (the + // blob) and `TOP_UPS_KEY` are already gone. + kv.put( + DetScope::Global, + SCHEDULED_VOTE_VOTERS_KEY, + &"not a voter index".to_string(), + ) + .expect("corrupt the scheduled-vote voter index"); + + let error = staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect_err("a corrupt voter index must fail the delete"); + assert!( + !matches!(error, TaskError::WalletStorageNotReady), + "the delete must reach purge_identity_scope, not stop at the migration guard: {error:?}" + ); + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "purge_identity_scope's first delete must already have removed the blob \ + before its own later step failed" + ); + + // Repair the index so the retry can actually complete. + kv.delete(DetScope::Global, SCHEDULED_VOTE_VOTERS_KEY) + .expect("repair the voter index"); + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("the retry must complete now that the index is repaired"); + + let view = IdentityKeyView::new(&staged.store, staged.id.to_buffer()); + for key_id in [1, 2] { + assert!( + view.get(&MAIN, key_id).unwrap().is_none(), + "key {key_id} must not be stranded: the manifest from the failed \ + attempt must have survived to inform the retry" + ); + } + assert!( + kv.get::>( + DetScope::Global, + &vault_cleanup_pending_key(&staged.id.to_buffer()) + ) + .expect("read the manifest slot") + .is_none(), + "the manifest must be cleared once every key is confirmed deleted" + ); + } + + /// The reachable form of the retry above: once `index_remove_identity` + /// has run, the identity is gone from the Global index and therefore off + /// every roster-backed screen — nothing in the UI can call + /// `delete_local_qualified_identity` again for it. The manifest must be + /// recoverable without that call: the boot-time sweep is the only + /// reachable path back to keys orphaned this way. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn resume_pending_vault_cleanups_recovers_a_manifest_no_ui_can_reach() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0x11; 32]; + const LOW: [u8; 32] = [0x22; 32]; + + let staged = stage_identity_with_vaulted_keys(HIGH, LOW).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + let id_buf = staged.id.to_buffer(); + + // Simulate a crash between the manifest write and `purge_identity_scope` + // ever starting — the manifest is durable and the identity is already + // off the roster, but its blob, top-up history, and a scheduled vote + // are all still sitting untouched in Identity scope. Built directly + // (not via a failing `delete_local_qualified_identity` call) because + // `purge_identity_scope`'s first two steps are unconditional deletes + // that cannot be made to fail independently of its last step — going + // through the real call would always leave the blob and top-ups + // already gone, proving nothing about whether the sweep re-purges. + let vault_keys = staged + .ctx + .identity_vault_key_placements(&kv, &id_buf) + .expect("read the live placements before removing from the index"); + staged + .ctx + .persist_vault_cleanup_manifest(&kv, &id_buf, &vault_keys) + .expect("persist the manifest"); + index_remove_identity(&kv, &id_buf).expect("remove from the roster"); + kv.put( + DetScope::Identity(&id_buf), + TOP_UPS_KEY, + &std::collections::BTreeMap::from([(0u32, 5u64)]), + ) + .expect("stage a top-up entry purge_identity_scope never reached"); + kv.put( + DetScope::Identity(&id_buf), + &scheduled_vote_key("alice"), + &StoredScheduledVote { + voter_id: id_buf, + contested_name: "alice".to_string(), + choice: StoredVoteChoice::Lock, + unix_timestamp: 0, + executed_successfully: false, + }, + ) + .expect("stage a scheduled vote purge_identity_scope never reached"); + index_add_vote_voter(&kv, &id_buf).expect("add to the voter index"); + + staged.ctx.resume_pending_vault_cleanups(); + + let view = IdentityKeyView::new(&staged.store, id_buf); + for key_id in [1, 2] { + assert!( + view.get(&MAIN, key_id).unwrap().is_none(), + "key {key_id} must be recovered by the sweep even with no UI path left to reach it" + ); + } + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "the sweep's purge_identity_scope re-run must leave the blob gone" + ); + assert!( + kv.get::>( + DetScope::Identity(&id_buf), + TOP_UPS_KEY + ) + .expect("read top-ups") + .is_none(), + "the sweep must drain the top-up history left behind by the interrupted purge" + ); + assert!( + kv.list(DetScope::Identity(&id_buf), Some(SCHEDULED_VOTE_KEY_PREFIX)) + .expect("list scheduled votes") + .is_empty(), + "the sweep must drain any scheduled votes left behind by the interrupted purge" + ); + assert!( + load_scheduled_vote_voters(&kv) + .expect("read the voter index") + .is_empty(), + "the sweep must prune this voter from the Global scheduled-vote index" + ); + assert!( + kv.get::>( + DetScope::Global, + &vault_cleanup_pending_key(&id_buf) + ) + .expect("read the manifest slot") + .is_none(), + "the manifest must be cleared once the sweep confirms every key deleted" + ); + } + + /// The sweep's mirror-image hazard. `index_remove_identity` runs *after* + /// the manifest is persisted, so a failure in that write leaves a manifest + /// behind for an identity that is still on the roster, still holding a live + /// blob, and still fully usable. Resuming such a manifest would delete the + /// keys of an identity the user can still see and was told was *not* + /// removed — the exact zombie + /// `a_failed_identity_delete_never_destroys_the_vault_keys` exists to + /// forbid, arriving one boot later. The sweep must therefore resume only + /// what the roster confirms is already gone. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn resume_pending_vault_cleanups_spares_an_identity_still_on_the_roster() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0x33; 32]; + const LOW: [u8; 32] = [0x44; 32]; + + let staged = stage_identity_with_vaulted_keys(HIGH, LOW).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + let id_buf = staged.id.to_buffer(); + + // Fail the delete *at* `index_remove_identity`, after the manifest is + // already durable: an unreadable index makes its own load step error. + kv.put( + DetScope::Global, + IDENTITY_INDEX_KEY, + &"not an identity index".to_string(), + ) + .expect("corrupt the identity index"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect_err("index_remove_identity must fail on an unreadable index"); + assert!( + kv.get::>( + DetScope::Global, + &vault_cleanup_pending_key(&id_buf) + ) + .expect("read the manifest slot") + .is_some(), + "precondition: the failed delete must have left a manifest behind" + ); + + // The index write never landed, so on the next boot the identity is + // still listed exactly as it was — the removal never happened. + kv.put(DetScope::Global, IDENTITY_INDEX_KEY, &vec![id_buf]) + .expect("restore the untouched index"); + + staged.ctx.resume_pending_vault_cleanups(); + + let view = IdentityKeyView::new(&staged.store, id_buf); + for key_id in [1, 2] { + assert!( + view.get(&MAIN, key_id).unwrap().is_some(), + "key {key_id} belongs to an identity still on the roster; the sweep \ + must not delete it" + ); + } + assert!( + load_identity_index(&kv) + .expect("read the index") + .contains(&id_buf), + "the identity must still be listed, making the UI retry path reachable" + ); + assert!( + kv.get::>( + DetScope::Global, + &vault_cleanup_pending_key(&id_buf) + ) + .expect("read the manifest slot") + .is_some(), + "the manifest must be retained: its keys are not confirmed absent, so \ + clearing it here would discard the delete set a retry still needs" + ); + } + + /// The sweep's "still listed?" check and a concurrent re-import's write + /// both touch this identity's roster entry and vault keys; without a + /// shared lock the two could interleave so the sweep deletes keys a + /// re-import just wrote for a live identity. `identity_record_lock` — the + /// same lock [`AppContext::insert_local_qualified_identity`] takes before + /// touching this identity's k/v — must serialize the two: whichever one + /// the sweep observes after acquiring it is the ground truth, so a + /// re-import that lands first must be honored, never overwritten. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn resume_pending_vault_cleanups_is_serialized_against_a_concurrent_reimport() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + const HIGH: [u8; 32] = [0x55; 32]; + const LOW: [u8; 32] = [0x66; 32]; + + let staged = stage_identity_with_vaulted_keys(HIGH, LOW).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + let id_buf = staged.id.to_buffer(); + + // Leave a manifest behind for an unlisted identity — the sweep's + // normal entry condition — via the same crash simulation as above: + // the index write landed, but `purge_identity_scope` never got the + // chance to run. + let vault_keys = staged + .ctx + .identity_vault_key_placements(&kv, &id_buf) + .expect("read the live placements before removing from the index"); + staged + .ctx + .persist_vault_cleanup_manifest(&kv, &id_buf, &vault_keys) + .expect("persist the manifest"); + index_remove_identity(&kv, &id_buf).expect("remove from the roster"); + + // Hold this identity's own record lock ourselves, simulating a + // concurrent `insert_local_qualified_identity` re-import already in + // flight when the boot sweep starts. + let lock = staged.ctx.identity_record_lock(staged.id); + let guard = lock + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + + // Only the sweep thread reports to this probe, so its count rises for + // the sweep and for nothing else — no baseline to subtract, and no + // other test's record-lock traffic can stand in for the sweep's. + let probe = LockProbe::new(); + let sweep_probe = probe.clone(); + + let (done_tx, done_rx) = std::sync::mpsc::channel::<()>(); + let sweep_ctx = Arc::clone(&staged.ctx); + let sweep = std::thread::spawn(move || { + let _attached = sweep_probe.attach(); + sweep_ctx.resume_pending_vault_cleanups(); + let _ = done_tx.send(()); + }); + + // Wait — without a deadline — for the sweep to reach the point just + // before it blocks on this identity's record lock. A timeout here + // would prove nothing: a sweep that has not started yet and a sweep + // parked on the lock are indistinguishable by elapsed time, so a slow + // worker would let the assertion below pass against a sweep that never + // took the lock at all. + while probe.count(LockSite::RecordRequest) == 0 { + std::thread::yield_now(); + } + + // It has asked for the lock and we hold it, so it can only be blocked + // acquiring it. Anything else — in particular finishing, having + // deleted the keys — is the bug. + assert!( + matches!( + done_rx.try_recv(), + Err(std::sync::mpsc::TryRecvError::Empty) + ), + "the sweep must block on this identity's record lock while a \ + re-import holds it, not race ahead and delete its keys" + ); + + // The re-import completes: re-list the identity, then release the + // lock — mirroring `insert_local_qualified_identity`'s own order of + // operations under the same guard. + index_add_identity(&kv, &id_buf).expect("re-list the identity"); + drop(guard); + + done_rx + .recv_timeout(std::time::Duration::from_secs(5)) + .expect("the sweep must proceed and finish once the lock is released"); + sweep.join().expect("sweep thread must not panic"); + + let view = IdentityKeyView::new(&staged.store, id_buf); + for key_id in [1, 2] { + assert!( + view.get(&MAIN, key_id).unwrap().is_some(), + "key {key_id} belongs to an identity the lock-holder re-listed \ + before releasing; the sweep must see that fresh state — read \ + after it acquires the lock, not before — and skip deleting it" + ); + } + assert!( + load_identity_index(&kv) + .expect("read the index") + .contains(&id_buf), + "the re-import must have won: the identity is listed again" + ); + } + + /// A removal is reported by what happened to the keys, not by what happened + /// to the bookkeeping. Once the vault delete lands the keys are gone, and a + /// manifest clear that fails afterwards leaves only a stale Global record + /// the next boot's sweep drops on its own. Propagating that failure would + /// tell the user their private keys are still on this device — the opposite + /// of the truth — and would raise a cleanup-pending warning for an identity + /// with nothing left to clean up. The other half of the distinction, a + /// cleanup that really is outstanding because the keys are still there, is + /// asserted by `remove_identity`'s deferred-cleanup tests. + #[test] + fn a_manifest_clear_failure_after_the_keys_are_deleted_still_completes_the_removal() { + const MAIN: PrivateKeyTarget = PrivateKeyTarget::PrivateKeyOnMainIdentity; + + let dir = tempfile::tempdir().unwrap(); + let store = fresh_vault(dir.path()); + let failing = Arc::new(FailingKv::default()); + let kv = DetKv::from_store(failing.clone()); + let id_buf = id(1); + let manifest_key = vault_cleanup_pending_key(&id_buf); + + let view = IdentityKeyView::new(&store, id_buf); + view.store(&MAIN, 1, &[0x11; 32]).expect("vault the key"); + kv.put( + DetScope::Global, + &manifest_key, + &vec![(StoredPrivateKeyTarget::from(MAIN), 1 as KeyID)], + ) + .expect("stage the manifest the delete is meant to clear"); + + failing.fail_deletes(true); + finish_vault_cleanup(&kv, &store, &id_buf, [(MAIN, 1)]) + .expect("the keys are gone, so only bookkeeping failed: not a failed removal"); + + assert!( + view.get(&MAIN, 1).unwrap().is_none(), + "the vault delete must have landed before the manifest clear was even attempted" + ); + failing.fail_deletes(false); + assert!( + kv.get::>(DetScope::Global, &manifest_key) + .expect("read the manifest slot") + .is_some(), + "the manifest must survive its failed clear, so the next boot's sweep still \ + finds it and re-runs the idempotent deletes" + ); + } + + // --------------------------------------------------------------- + // Unloading is a per-device decision: automatic discovery must not + // undo it, and only a deliberate load may. + // --------------------------------------------------------------- + + /// The user is told removal destroys the keys and a backup is the only way + /// back in. Nothing else on disk distinguishes a removed identity from one + /// that was never here, so without this marker the next discovery pass + /// re-derives it and puts it straight back. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn removing_an_identity_records_that_it_was_unloaded() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let kv = staged.ctx.det_kv().expect("identity kv"); + let id = staged.id.to_buffer(); + + assert!( + !staged + .ctx + .is_identity_unloaded(&kv, &id) + .expect("read the marker"), + "a stored identity must not carry an unload marker", + ); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + assert!( + staged + .ctx + .is_identity_unloaded(&kv, &id) + .expect("read the marker"), + "removal must record the unload, or discovery has nothing to consult", + ); + } + + /// The removal is only durable if the automatic passes honour it: the boot + /// sweep, the post-unlock pass and a wallet import all re-derive the same + /// identity from the still-loaded wallet. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn automatic_discovery_does_not_restore_an_unloaded_identity() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let mut rediscovered = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + let stored = staged + .ctx + .store_discovered_identity(&mut rediscovered, &None, DiscoveryIntent::Automatic) + .expect("a refusal is not an error"); + + assert!( + !stored, + "discovery must refuse an identity the user unloaded" + ); + assert!( + staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read back") + .is_none(), + "the refused identity must stay off disk", + ); + assert!( + !staged + .ctx + .local_identity_ids() + .expect("read the roster") + .contains(&staged.id), + "the refused identity must stay off the roster, or it reappears on every screen", + ); + } + + /// A discovery pass that read the record *before* the removal takes the + /// refresh branch, not the insert branch, and its write lands after the + /// removal finished. Sequential here because that ordering is the whole + /// content of the race: read before, write after. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_discovery_that_read_the_record_first_cannot_write_it_back() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + // The in-flight pass's own snapshot, taken while the identity is still + // stored — which is what sends it down the refresh branch. + let mut in_flight = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + let stored = staged + .ctx + .store_discovered_identity(&mut in_flight, &None, DiscoveryIntent::Automatic) + .expect("a refusal is not an error"); + + assert!( + !stored, + "a snapshot taken before the removal must not outlive it", + ); + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "the removed identity's blob must stay gone", + ); + } + + /// Proves the marker check is *inside* the record lock rather than before + /// it: the discovery thread is parked on the lock — observably, by count, + /// not by elapsed time — while the removal it must honour is recorded. + /// Move the check above the acquisition and the parked thread has already + /// decided "not unloaded" before this test writes the marker. + #[tokio::test(flavor = "multi_thread", worker_threads = 4)] + async fn a_discovery_parked_on_the_record_lock_still_sees_the_removal() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let mut in_flight = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + let lock = staged.ctx.identity_record_lock(staged.id); + let guard = lock + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + + let probe = LockProbe::new(); + let discovery_probe = probe.clone(); + let ctx = Arc::clone(&staged.ctx); + let discovery = std::thread::spawn(move || { + let _attached = discovery_probe.attach(); + ctx.store_discovered_identity(&mut in_flight, &None, DiscoveryIntent::Automatic) + }); + + // Unbounded on purpose: the discovery thread cannot acquire a lock this + // test holds, so the count it is waited on only ever settles one way. + // A bound here would be a deadline, and a deadline would let the thread + // be counted as parked when it had merely not started. The count is + // this test's own probe, so no other test's record-lock traffic can + // end the wait either. + while probe.count(LockSite::DiscoveryRecordWait) == 0 { + std::thread::yield_now(); + } + + let kv = staged.ctx.det_kv().expect("identity kv"); + let id = staged.id.to_buffer(); + staged + .ctx + .mark_identity_unloaded(&kv, &id) + .expect("record the unload"); + kv.delete(DetScope::Identity(&id), IDENTITY_KEY) + .expect("drop the record"); + index_remove_identity(&kv, &id).expect("delist the identity"); + drop(guard); + + let stored = discovery + .join() + .expect("the discovery thread must not panic") + .expect("a refusal is not an error"); + + assert!( + !stored, + "a discovery blocked across a removal must re-read the marker, not its own \ + earlier answer", + ); + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "the removed identity's blob must stay gone", + ); + } + + /// The one thing that means the user changed their mind: asking for the + /// identity back. Loading it must both store it and retire the marker, so + /// later automatic passes keep it refreshed rather than refusing it. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn deliberately_loading_an_identity_retires_its_unload_marker() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let mut reloaded = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + staged + .ctx + .insert_local_qualified_identity(&reloaded, &None) + .expect("load the identity back"); + + let kv = staged.ctx.det_kv().expect("identity kv"); + assert!( + !staged + .ctx + .is_identity_unloaded(&kv, &staged.id.to_buffer()) + .expect("read the marker"), + "a deliberate load must retire the marker", + ); + assert!( + staged + .ctx + .store_discovered_identity(&mut reloaded, &None, DiscoveryIntent::Automatic) + .expect("store"), + "with the marker retired, discovery must keep the identity refreshed again", + ); + } + + /// The devnet wipe destroys the same keys the per-identity removal does, + /// and leaves the same wallets loaded, so it needs the same marker — or the + /// next discovery pass hands the developer back everything they just + /// cleared. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn the_devnet_wipe_records_every_identity_it_clears_as_unloaded() { + let staged = + stage_identity_with_vaulted_keys_on(Network::Devnet, [0xAA; 32], [0xBB; 32]).await; + let mut rediscovered = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_all_local_qualified_identities_in_devnet() + .expect("wipe the devnet identities"); + + let kv = staged.ctx.det_kv().expect("identity kv"); + assert!( + staged + .ctx + .is_identity_unloaded(&kv, &staged.id.to_buffer()) + .expect("read the marker"), + "the wipe must record each identity it cleared as unloaded", + ); + assert!( + !staged + .ctx + .store_discovered_identity(&mut rediscovered, &None, DiscoveryIntent::Automatic) + .expect("a refusal is not an error"), + "discovery must not restore what the wipe cleared", + ); + } + + /// The By-Wallet search is the user typing an index and pressing Search, so + /// it is a request for those identities — including one they unloaded + /// earlier. Refusing it would leave the user with a search that silently + /// finds nothing and no way to undo an unload. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_user_requested_search_loads_back_an_unloaded_identity() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let mut found = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + assert!( + staged + .ctx + .store_discovered_identity(&mut found, &None, DiscoveryIntent::UserRequested) + .expect("store"), + "a search the user started must load the identity back", + ); + + let kv = staged.ctx.det_kv().expect("identity kv"); + assert!( + !staged + .ctx + .is_identity_unloaded(&kv, &staged.id.to_buffer()) + .expect("read the marker"), + "loading it back must retire the marker, or the next automatic pass \ + stops refreshing an identity the user has again", + ); + } + + /// The chokepoint guard, reached through the ordinary update path rather + /// than through discovery: every fund-moving task (top-up, transfer, + /// withdrawal, add-key, DPNS registration) holds a `QualifiedIdentity` it + /// read earlier and writes it back when the chain work finishes. A removal + /// landing in that window would otherwise re-create the record — a zombie + /// identity, listed on every screen, whose private keys are already gone. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn an_update_cannot_resurrect_an_identity_the_user_unloaded() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + // The snapshot a task in flight is holding. + let stale = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + staged + .ctx + .delete_local_qualified_identity(&staged.id) + .expect("remove the identity"); + + staged + .ctx + .update_local_qualified_identity(&stale) + .expect("declining is a completion, not an error"); + + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_none(), + "an update must not re-create the record of an unloaded identity", + ); + assert!( + !staged + .ctx + .local_identity_ids() + .expect("read the roster") + .contains(&staged.id), + "and must not put it back on the roster, which is what authorizes the \ + vault-cleanup sweep to destroy its keys", + ); + } + + /// The guard is narrow on purpose. Creating a record for an absent identity + /// is a real service to a caller repairing a lost roster entry, and only an + /// unload marker withdraws it. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn an_update_still_creates_a_record_for_an_identity_never_unloaded() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let identity = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + // Gone from storage without ever being unloaded — the shape a lost + // write or an interrupted migration leaves behind. + let kv = staged.ctx.det_kv().expect("identity kv"); + let id = staged.id.to_buffer(); + kv.delete(DetScope::Identity(&id), IDENTITY_KEY) + .expect("drop the record"); + index_remove_identity(&kv, &id).expect("delist the identity"); + + staged + .ctx + .update_local_qualified_identity(&identity) + .expect("store the identity"); + + assert!( + staged + .ctx + .local_identity_ids() + .expect("read the roster") + .contains(&staged.id), + "with no unload marker, an update must still restore the record", + ); + } + + /// R2: the chokepoint guard must ask the roster, not the blob. + /// + /// `purge_identity_scope` is documented as three independent k/v writes, + /// and it runs *after* `index_remove_identity`. So a removal that stops + /// between them — its first delete failing, or the process ending — leaves + /// the identity delisted with its blob still on disk. A stale snapshot + /// written back then finds `existing` present, sails past a guard keyed on + /// blob presence, and calls `index_add_identity`: the identity is back on + /// the roster after its removal was reported complete, and the boot sweep + /// reads that roster entry as authority to spare its keys. + /// + /// Roster membership is the authority this PR already established for + /// exactly this question. The guard has to use it. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_stale_update_cannot_relist_an_identity_whose_purge_did_not_finish() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + // The snapshot an in-flight task is holding across its chain round-trip. + let stale = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + // The removal reaches its point of no return and stops there: marker + // written, identity delisted, blob not yet purged. Built by calling the + // same steps in the same order `delete_local_qualified_identity` does, + // because no failure has to be injected for this state to be reachable + // — the three purge writes are not atomic and the code says so. + let kv = staged.ctx.det_kv().expect("identity kv"); + let id = staged.id.to_buffer(); + staged + .ctx + .mark_identity_unloaded(&kv, &id) + .expect("record the unload"); + index_remove_identity(&kv, &id).expect("delist the identity"); + + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_some(), + "precondition: the blob outlives the delisting, which is what makes a \ + blob-presence check answer the wrong question" + ); + assert!( + !staged + .ctx + .is_identity_listed(&staged.id) + .expect("read the roster"), + "precondition: the identity is already delisted" + ); + + staged + .ctx + .update_local_qualified_identity(&stale) + .expect("declining is a completion, not an error"); + + assert!( + !staged + .ctx + .is_identity_listed(&staged.id) + .expect("read the roster"), + "a stale write must not put a removed identity back on the roster — the \ + boot sweep reads that entry as proof the identity is live and spares \ + the keys the removal was meant to destroy" + ); + } + + /// The other side of the same predicate. A removal that fails *before* + /// delisting leaves a marker on an identity that is still on the roster and + /// still on every screen — its removal never happened. That identity's + /// ordinary work must keep working: a balance refresh after a failed unload + /// attempt is not a resurrection. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_marker_on_a_still_listed_identity_does_not_block_its_ordinary_writes() { + let staged = stage_identity_with_vaulted_keys([0xAA; 32], [0xBB; 32]).await; + let identity = staged + .ctx + .get_local_qualified_identity(&staged.id) + .expect("read the staged identity") + .expect("identity present"); + + // The shape a removal that failed at its delisting step leaves behind: + // marked, but never delisted. + let kv = staged.ctx.det_kv().expect("identity kv"); + staged + .ctx + .mark_identity_unloaded(&kv, &staged.id.to_buffer()) + .expect("record the unload"); + + staged + .ctx + .update_local_qualified_identity(&identity) + .expect("store the identity"); + + assert!( + staged + .ctx + .is_identity_listed(&staged.id) + .expect("read the roster"), + "an identity whose removal never got as far as delisting must keep \ + working, marker or not" + ); + assert!( + staged + .ctx + .stored_identity_blob(&staged.id) + .expect("read the blob") + .is_some(), + "and its record must still be written" + ); + } } diff --git a/src/context/lock_probe.rs b/src/context/lock_probe.rs new file mode 100644 index 000000000..0fdd69198 --- /dev/null +++ b/src/context/lock_probe.rs @@ -0,0 +1,207 @@ +//! Test-only lock instrumentation, scoped to the test that installs it. +//! +//! Concurrency fixtures need one fact the code cannot otherwise report: *this +//! thread has reached that lock and cannot proceed*. Measuring it by elapsed +//! time is unsound — a thread that has not started yet and a thread parked on a +//! lock are indistinguishable by a clock — so the lock sites count waiters +//! instead. +//! +//! A process-global counter would be equally unsound, for a subtler reason. The +//! test binary runs tests in parallel, and every one of them shares the same +//! `identity_record_lock` and roster lock. A global waiter count therefore +//! reports the *suite's* lock traffic, not the test's: an unrelated test parked +//! on the same lock satisfies a fixture's release condition, freeing a reader +//! whose peer has not arrived, and the test that exists to catch a lost update +//! passes without one. That is the timeout bug again with a different clock — +//! worse, in fact, because it depends on what else happens to be running and so +//! decays silently and unreproducibly. +//! +//! A [`LockProbe`] is owned by one test. Only threads that [`LockProbe::attach`] +//! themselves to it report to it, so a probe counts exactly the traffic its own +//! test created and a busy neighbour is invisible to it. + +use std::cell::RefCell; +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; + +/// Which lock a thread is waiting on, or asking for. +#[derive(Debug, Clone, Copy)] +pub(crate) enum LockSite { + /// Blocked acquiring the process-wide roster lock. + RosterWait, + /// Blocked acquiring an identity's record lock in + /// [`AppContext::store_discovered_identity`](crate::context::AppContext::store_discovered_identity). + DiscoveryRecordWait, + /// Asked for an identity's record lock. Counts requests, not waits, and is + /// never decremented — a test waits for it to rise, having taken the lock + /// itself, which makes "asked for it" and "blocked on it" the same fact. + RecordRequest, +} + +#[derive(Default)] +struct Counts { + roster_waiters: AtomicUsize, + discovery_record_waiters: AtomicUsize, + record_requests: AtomicUsize, +} + +impl Counts { + fn slot(&self, site: LockSite) -> &AtomicUsize { + match site { + LockSite::RosterWait => &self.roster_waiters, + LockSite::DiscoveryRecordWait => &self.discovery_record_waiters, + LockSite::RecordRequest => &self.record_requests, + } + } +} + +thread_local! { + static ATTACHED: RefCell>> = const { RefCell::new(None) }; +} + +/// One test's private view of lock activity on the threads it owns. +#[derive(Clone, Default)] +pub(crate) struct LockProbe { + counts: Arc, +} + +impl LockProbe { + pub(crate) fn new() -> Self { + Self::default() + } + + /// Report the calling thread's lock activity to this probe until the + /// returned guard is dropped. + /// + /// Hold the guard for the thread's whole run. Dropping it matters on a + /// pooled thread — a tokio worker outlives the test that used it, and an + /// attachment left behind would point a later test's lock waits at a probe + /// nobody reads, so the later test's own probe would stay at zero and its + /// wait would never finish. + pub(crate) fn attach(&self) -> Attachment { + ATTACHED.with(|attached| { + *attached.borrow_mut() = Some(Arc::clone(&self.counts)); + }); + Attachment + } + + /// Threads of this test currently blocked acquiring `site`'s lock, or — + /// for [`LockSite::RecordRequest`] — that have asked for one. + pub(crate) fn count(&self, site: LockSite) -> usize { + self.counts.slot(site).load(Ordering::SeqCst) + } +} + +/// Detaches its thread from a [`LockProbe`] when dropped. +#[must_use = "the thread stays attached only while this guard is alive"] +pub(crate) struct Attachment; + +impl Drop for Attachment { + fn drop(&mut self) { + ATTACHED.with(|attached| { + *attached.borrow_mut() = None; + }); + } +} + +/// Count the calling thread as waiting at `site` until the returned guard is +/// dropped. A thread attached to no probe counts nowhere. +/// +/// Bind the guard and drop it once the lock is *acquired*: the fact under +/// measurement is "waiting", not "holding". +pub(crate) fn enter_wait(site: LockSite) -> WaitGuard { + let counts = ATTACHED.with(|attached| attached.borrow().clone()); + if let Some(counts) = &counts { + counts.slot(site).fetch_add(1, Ordering::SeqCst); + } + WaitGuard { counts, site } +} + +/// Record that the calling thread asked for `site`'s lock. Not paired with a +/// decrement — see [`LockSite::RecordRequest`]. +pub(crate) fn note_request(site: LockSite) { + ATTACHED.with(|attached| { + if let Some(counts) = attached.borrow().as_ref() { + counts.slot(site).fetch_add(1, Ordering::SeqCst); + } + }); +} + +pub(crate) struct WaitGuard { + counts: Option>, + site: LockSite, +} + +impl Drop for WaitGuard { + fn drop(&mut self) { + if let Some(counts) = &self.counts { + counts.slot(self.site).fetch_sub(1, Ordering::SeqCst); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The whole point of the type: one test's probe must not see another + /// test's threads. An unattached thread standing in for the busy + /// neighbour must leave every count at zero. + #[test] + fn a_probe_counts_only_the_threads_attached_to_it() { + let mine = LockProbe::new(); + let theirs = LockProbe::new(); + + std::thread::scope(|scope| { + scope.spawn(|| { + let _attached = mine.attach(); + let _waiting = enter_wait(LockSite::RosterWait); + while mine.count(LockSite::RosterWait) == 0 { + std::thread::yield_now(); + } + }); + scope.spawn(|| { + let _attached = theirs.attach(); + let _waiting = enter_wait(LockSite::RosterWait); + while theirs.count(LockSite::RosterWait) == 0 { + std::thread::yield_now(); + } + }); + // The neighbour: same lock site, no probe. + scope.spawn(|| { + let _waiting = enter_wait(LockSite::RosterWait); + std::thread::yield_now(); + }); + }); + + assert_eq!( + mine.count(LockSite::RosterWait), + 0, + "every waiter this probe saw must have released", + ); + assert_eq!( + theirs.count(LockSite::RosterWait), + 0, + "one probe's waiter must never be counted by another", + ); + } + + /// A pooled thread runs later tests. An attachment that outlived its test + /// would point their lock waits at a probe nobody reads, and their own + /// probe would never leave zero. + #[test] + fn dropping_the_attachment_detaches_the_thread() { + let probe = LockProbe::new(); + { + let _attached = probe.attach(); + let _waiting = enter_wait(LockSite::RecordRequest); + assert_eq!(probe.count(LockSite::RecordRequest), 1); + } + note_request(LockSite::RecordRequest); + assert_eq!( + probe.count(LockSite::RecordRequest), + 0, + "activity after the attachment is dropped belongs to no probe", + ); + } +} diff --git a/src/context/mod.rs b/src/context/mod.rs index 8828108fd..163129987 100644 --- a/src/context/mod.rs +++ b/src/context/mod.rs @@ -3,6 +3,10 @@ mod contested_names_db; mod contract_token_db; pub mod feature_gate; mod identity_db; +#[cfg(test)] +pub(crate) mod lock_probe; +#[cfg(test)] +pub(crate) use identity_db::test_staging; pub(crate) mod identity_load_registry; pub mod migration_status; mod settings_db; @@ -317,6 +321,13 @@ impl AppContext { /// holder is a key-protection tier change, whose per-key derivation runs in /// the low hundreds of milliseconds. Different identities never contend. pub(crate) fn identity_record_lock(&self, identity_id: Identifier) -> Arc> { + // Counted before the caller's blocking acquire, so a test holding the + // lock learns a background worker reached it rather than merely being + // slow to start — which elapsed time cannot distinguish. Reported only + // to the probe the calling thread attached itself to, so a test never + // sees the suite's other lock traffic. + #[cfg(test)] + lock_probe::note_request(lock_probe::LockSite::RecordRequest); self.identity_record_locks .lock() .unwrap_or_else(std::sync::PoisonError::into_inner) diff --git a/src/context/wallet_lifecycle/bootstrap.rs b/src/context/wallet_lifecycle/bootstrap.rs index bdcea1ecd..f016e754b 100644 --- a/src/context/wallet_lifecycle/bootstrap.rs +++ b/src/context/wallet_lifecycle/bootstrap.rs @@ -2,6 +2,7 @@ //! managed identities, warming auth-key caches, and queuing identity discovery. use super::*; +use crate::model::identity_discovery::DiscoveryIntent; impl AppContext { /// Whether `wallet` still needs its bootstrap address set derived. @@ -606,7 +607,13 @@ impl AppContext { .subtasks .spawn_sync("all_wallets_identity_discovery", async move { if let Err(error) = ctx - .discover_identities_gap_limited(&wallet, 0, false, None) + .discover_identities_gap_limited( + &wallet, + 0, + false, + DiscoveryIntent::Automatic, + None, + ) .await { tracing::warn!( @@ -656,7 +663,7 @@ impl AppContext { } if let Err(error) = self - .discover_identities_gap_limited(wallet, 0, true, None) + .discover_identities_gap_limited(wallet, 0, true, DiscoveryIntent::Automatic, None) .await { tracing::warn!( @@ -695,6 +702,10 @@ impl AppContext { // identities exist on installs with no wallet at all. if let Ok(backend) = self.wallet_backend() { self.reconcile_unowned_identities(&backend); + // Resumes vault-key deletes an earlier removal couldn't finish, + // but only for identities already off the roster — the ones no + // UI retry can still reach. + self.resume_pending_vault_cleanups(); } let wallets: Vec<_> = { diff --git a/src/model/identity_discovery.rs b/src/model/identity_discovery.rs index 4828f8b2b..74c0f287c 100644 --- a/src/model/identity_discovery.rs +++ b/src/model/identity_discovery.rs @@ -74,6 +74,30 @@ pub fn should_continue_scan(current_index: u32, highest_found: Option) -> b } } +/// Why a discovery pass is running, which decides whether it may re-add an +/// identity the user unloaded from this device. +/// +/// Unloading is a per-device decision, so a pass the user did not ask for must +/// not undo it — automatic passes are refused by the unload marker, while a +/// pass the user explicitly started is itself the request to have the identity +/// back and retires the marker. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum DiscoveryIntent { + /// Background or side-effect discovery: the boot sweep, an unlock, a wallet + /// import. Never re-adds an unloaded identity. + Automatic, + /// The user asked for identities to be loaded, so an unloaded one is being + /// asked for back. + UserRequested, +} + +impl DiscoveryIntent { + /// Whether a pass with this intent may store an identity the user unloaded. + pub fn may_restore_unloaded(self) -> bool { + matches!(self, Self::UserRequested) + } +} + /// Outcome counts of one gap-limited discovery pass over a single wallet. /// /// `found` is the number of indices that resolved to an on-chain identity; diff --git a/src/ui/components/confirmation_dialog.rs b/src/ui/components/confirmation_dialog.rs index faa0fdfc9..e910b526a 100644 --- a/src/ui/components/confirmation_dialog.rs +++ b/src/ui/components/confirmation_dialog.rs @@ -158,6 +158,21 @@ impl ConfirmationDialog { self } + /// The dialog's message as plain text. Test-only: lets a caller assert + /// what a destructive prompt actually says without rendering a frame. + #[cfg(test)] + pub(crate) fn message_text(&self) -> &str { + self.message.text() + } + + /// Whether the dialog blocks input to the controls behind it. Test-only: + /// lets a caller assert that a destructive prompt cannot be answered while + /// the user has navigated elsewhere. + #[cfg(test)] + pub(crate) fn is_input_blocking(&self) -> bool { + self.blocks_input + } + fn confirmation_text_matches(&self) -> bool { self.required_confirmation_text .as_ref() diff --git a/src/ui/identities/identities_screen.rs b/src/ui/identities/identities_screen.rs index 3dce69f79..e620462ac 100644 --- a/src/ui/identities/identities_screen.rs +++ b/src/ui/identities/identities_screen.rs @@ -12,6 +12,7 @@ use crate::ui::components::styled::{ConfirmationDialog, ConfirmationStatus, isla use crate::ui::components::top_panel::{add_top_panel_with_global_nav, subdued_everyday_spec}; use crate::ui::components::{BannerHandle, MessageBanner, OptionBannerExt}; use crate::ui::helpers::{ModalOpeningGuard, clicked_outside_window_after_open}; +use crate::ui::identities::IDENTITY_REMOVAL_BLOCKED_BY_STORAGE_UPDATE; use crate::ui::identities::keys::add_key_screen::AddKeyScreen; use crate::ui::identities::keys::key_info_screen::KeyInfoScreen; use crate::ui::identities::register_dpns_name_screen::{ @@ -950,7 +951,7 @@ impl IdentitiesScreen { if self.app_context.migration_status().state().is_in_progress() { MessageBanner::set_global( self.app_context.egui_ctx(), - "The storage update is still running. Wait for it to finish before removing an identity.", + IDENTITY_REMOVAL_BLOCKED_BY_STORAGE_UPDATE, MessageType::Warning, ); return AppAction::None; @@ -1135,25 +1136,21 @@ impl ScreenLike for IdentitiesScreen { crate::ui::BackendTaskSuccessResult::RemovedIdentities { identity_ids, associated_cleanup_failed, + cleanup_deferred, } => { let mut identities = self.identities.lock_recover(); for identity_id in identity_ids { identities.shift_remove(&identity_id); } drop(identities); - if associated_cleanup_failed { - MessageBanner::set_global( - self.app_context.egui_ctx(), - "The identity was removed, but its associated voter identity could not be removed. Retry after restarting the app.", - MessageType::Warning, - ) - .disable_auto_dismiss(); - } else { - MessageBanner::set_global( - self.app_context.egui_ctx(), - "The identity was removed from this device.", - MessageType::Success, - ); + let (message, message_type) = crate::ui::identities::removed_identities_banner( + associated_cleanup_failed, + cleanup_deferred, + ); + let handle = + MessageBanner::set_global(self.app_context.egui_ctx(), message, message_type); + if message_type == MessageType::Warning { + handle.disable_auto_dismiss(); } } _ => {} diff --git a/src/ui/identities/mod.rs b/src/ui/identities/mod.rs index 5e8f89da5..09a12583c 100644 --- a/src/ui/identities/mod.rs +++ b/src/ui/identities/mod.rs @@ -19,6 +19,75 @@ pub mod top_up_identity_screen; pub mod transfer_screen; pub mod withdraw_screen; +/// Confirmation shown once an identity is removed from this device. +pub const IDENTITY_REMOVED: &str = "The identity was removed from this device."; + +/// Shown when the identity's removal succeeded — it is already gone from +/// every list — but a step that runs after delisting, including the one that +/// clears its private keys from this device's vault, did not finish. +/// +/// Framed as a completed removal, since there is no retry control left for the +/// user to reach the identity with, and hedged on both things the +/// `cleanup_deferred` flag does not actually establish. It does not establish +/// that keys are present: the failing step may be a scope purge for a keyless +/// identity, whose manifest names no placements at all. It does not establish +/// that any particular launch fixes it: the boot sweep returns before doing +/// anything while a storage update holds the migration lock or is in progress, +/// and also when the k/v store or the manifest list cannot be read. Since this +/// banner does not survive a restart, naming "the next time you open it" would +/// spend the user's only warning on a launch that may make no attempt at all. +/// +/// So the message promises a continuing automatic effort — which is true, the +/// manifest is retained until every listed key is confirmed gone — and neither +/// a schedule nor completion. It still names the one precaution the user can +/// act on now, because a warning that only hedges gives them nothing to do. +pub const IDENTITY_REMOVED_CLEANUP_PENDING: &str = "The identity was removed from this device, but its private keys may still be stored here. The app will keep trying to clear them automatically. Until then, treat this device as if it still holds them."; + +/// Shown when the identity was removed but the voter identity tied to it was +/// not. Naming the leftover matters: the user sees one entry disappear and one +/// stay, and this is what tells them the remaining entry is not a mistake. +pub const IDENTITY_REMOVED_VOTER_LEFT: &str = "The identity was removed, but its associated voter identity could not be removed. Retry after restarting the app."; + +/// Shown when both leftover outcomes above apply at once: the associated +/// voter identity failed to remove *and* a post-delisting step for at least +/// one of the two identities this call touched did not finish. A +/// single-outcome banner would silently drop one of the two — the voter +/// identity looking like a clean failure with nothing else wrong, or the +/// possible key residue going unmentioned entirely — so this names both, under +/// the same hedge as [`IDENTITY_REMOVED_CLEANUP_PENDING`]. +/// +/// It carries that constant's precaution too. This is the worse of the two +/// outcomes, so it must not be the one that tells the user less: the same +/// uncertainty about key material applies, and the same thing can be done +/// about it now. +pub const IDENTITY_REMOVED_VOTER_LEFT_AND_CLEANUP_PENDING: &str = "The identity was removed, but its associated voter identity could not be removed — retry after restarting the app. Private keys for one or both of them may still be stored on this device. The app will keep trying to clear them automatically. Until then, treat this device as if it still holds them."; + +/// Shown when a removal is refused because the storage update is still running. +pub const IDENTITY_REMOVAL_BLOCKED_BY_STORAGE_UPDATE: &str = + "The storage update is still running. Wait for it to finish before removing an identity."; + +/// Which banner to show for a `BackendTaskSuccessResult::RemovedIdentities` +/// result, and how urgently. Shared by the Identity Hub and the legacy +/// identities screen so the two, which handle the same result type, don't +/// carry two independently-maintained copies of this 4-way decision — the +/// combined-flags case in particular is easy to get wrong by handling each +/// flag in isolation (see the `both` test case below). +pub fn removed_identities_banner( + associated_cleanup_failed: bool, + cleanup_deferred: bool, +) -> (&'static str, crate::ui::MessageType) { + use crate::ui::MessageType; + match (associated_cleanup_failed, cleanup_deferred) { + (true, true) => ( + IDENTITY_REMOVED_VOTER_LEFT_AND_CLEANUP_PENDING, + MessageType::Warning, + ), + (true, false) => (IDENTITY_REMOVED_VOTER_LEFT, MessageType::Warning), + (false, true) => (IDENTITY_REMOVED_CLEANUP_PENDING, MessageType::Warning), + (false, false) => (IDENTITY_REMOVED, MessageType::Success), + } +} + /// Retrieves the appropriate wallet (if any) associated with the given identity. /// /// # Description @@ -96,6 +165,109 @@ pub fn get_selected_wallet( mod tests { use std::collections::BTreeMap; + use super::*; + use crate::ui::MessageType; + + /// The one case a naive if/else-if chain gets wrong: when the associated + /// voter identity's cleanup failed *and* the primary or voter's own + /// vault-key delete is separately still pending, both must surface — + /// picking only the voter-failure message silently drops the still-live + /// private key residue, and picking only the cleanup-pending message + /// silently drops the unretryable voter failure. + #[test] + fn removed_identities_banner_names_both_outcomes_when_both_apply() { + let (message, message_type) = removed_identities_banner(true, true); + assert_eq!(message, IDENTITY_REMOVED_VOTER_LEFT_AND_CLEANUP_PENDING); + // Case-insensitive: which clause opens a sentence is a copy decision, + // and this assertion is about both outcomes being named, not casing. + let message_lowercase = message.to_lowercase(); + assert!( + message_lowercase.contains("voter identity") + && message_lowercase.contains("private keys"), + "the combined message must name both the voter failure and the key residue" + ); + assert_eq!(message_type, MessageType::Warning); + } + + /// `cleanup_deferred` records that a step failed *after* delisting. It does + /// not establish that key material is present — `purge_identity_scope` can + /// fail for a keyless identity whose manifest holds no placements — and it + /// cannot promise that any particular launch fixes it, because the boot + /// sweep returns before doing anything while a storage update runs or when + /// the store cannot be read, and retains the manifest if the purge or vault + /// delete fails again. A banner that asserts either reads as a guarantee, + /// and the one thing worse than warning a user about key residue is telling + /// them it has been handled when it has not. + #[test] + fn cleanup_pending_banners_claim_no_more_than_the_flag_establishes() { + for message in [ + IDENTITY_REMOVED_CLEANUP_PENDING, + IDENTITY_REMOVED_VOTER_LEFT_AND_CLEANUP_PENDING, + ] { + assert!( + !message.contains("are still stored"), + "presence is not established, so the message must not assert it: {message}" + ); + assert!( + message.contains("may still be stored"), + "the message must say the keys may be present, not that they are: {message}" + ); + assert!( + !message.contains("will be cleared"), + "completion is not guaranteed, so the message must not promise it: {message}" + ); + assert!( + message.contains("try"), + "the message must promise another attempt in place of the completion it \ + cannot promise: {message}" + ); + assert!( + !message.contains("next time"), + "the sweep can return without attempting anything, and this banner does not \ + survive a restart — tying the attempt to one launch spends the user's only \ + warning on a launch that may do nothing: {message}" + ); + } + } + + /// Honest uncertainty still has to leave the user something to do. A + /// message that only hedges is its own failure — it reports a risk and + /// hands over no way to act on it. + /// + /// With no launch to point at, the precaution *is* that action: it is the + /// only part of these messages the user can act on immediately, and it does + /// not depend on anything the app manages to do later. + #[test] + fn cleanup_pending_banners_still_give_the_user_something_to_do() { + for message in [ + IDENTITY_REMOVED_CLEANUP_PENDING, + IDENTITY_REMOVED_VOTER_LEFT_AND_CLEANUP_PENDING, + ] { + assert!( + message.contains("treat this device as if it still holds"), + "the safe assumption under uncertainty is the precaution the user can take \ + now, and the combined outcome is the worse one — it must not be the banner \ + that says less: {message}" + ); + } + } + + #[test] + fn removed_identities_banner_covers_every_single_flag_combination() { + assert_eq!( + removed_identities_banner(true, false), + (IDENTITY_REMOVED_VOTER_LEFT, MessageType::Warning) + ); + assert_eq!( + removed_identities_banner(false, true), + (IDENTITY_REMOVED_CLEANUP_PENDING, MessageType::Warning) + ); + assert_eq!( + removed_identities_banner(false, false), + (IDENTITY_REMOVED, MessageType::Success) + ); + } + use dash_sdk::dpp::dashcore::Network; use dash_sdk::dpp::identity::Identity; use dash_sdk::dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; @@ -103,7 +275,6 @@ mod tests { use dash_sdk::dpp::version::PlatformVersion; use dash_sdk::platform::Identifier; - use super::*; use crate::model::qualified_identity::encrypted_key_storage::{ KeyStorage, PrivateKeyData, WalletDerivationPath, }; diff --git a/src/ui/identity/hub_screen.rs b/src/ui/identity/hub_screen.rs index 0a1e12728..2cf45e542 100644 --- a/src/ui/identity/hub_screen.rs +++ b/src/ui/identity/hub_screen.rs @@ -11,6 +11,7 @@ use super::identity_hub_tab_bar::IdentityHubTabBar; use crate::app::AppAction; use crate::backend_task::dashpay::DashPayTask; use crate::backend_task::error::TaskError; +use crate::backend_task::identity::IdentityTask; use crate::backend_task::{BackendTask, BackendTaskContext, BackendTaskSuccessResult}; use crate::context::AppContext; use crate::model::dashpay::UnreadableContactInfoPolicy; @@ -25,7 +26,7 @@ use crate::ui::{MessageType, RootScreenType, ScreenLike, ScreenType}; use dash_sdk::dpp::identity::accessors::IdentityGettersV0; use dash_sdk::platform::Identifier; use eframe::egui::{self, Context}; -use std::collections::{HashMap, VecDeque}; +use std::collections::{BTreeSet, HashMap, VecDeque}; use std::sync::Arc; use super::home::HomeState; @@ -57,6 +58,29 @@ pub struct IdentityHubScreen { /// Settings-tab state. Held on the hub so edit fields, unsaved drafts, /// and modal state persist across frames. settings_tab: SettingsTab, + /// Identities this screen dispatched an unload for, each retained until its + /// own result lands. Neither selection pointer survives that wait: a frame + /// can render before the result arrives and reconcile both of them onto the + /// identity that replaced this one. + /// + /// A set rather than one slot, because unloads overlap. The confirmation + /// blocks input only while it is open; it closes on confirm and the task + /// runs async, so the user can confirm A, switch to B and confirm B before + /// A's result lands. A single slot would then hold B, and A's caches — its + /// contact rows, load guard, pending tasks and cached profile — would never + /// be reset, which is the original bug with an extra step. + /// + /// An entry whose removal failed is never consumed — a failure before + /// delisting sends no result, and the screen's failure signal names no + /// identity, so it cannot know which of several in-flight unloads to drop. + /// The set therefore grows by one `Identifier` per failed attempt and never + /// shrinks for one, for the life of the screen. Bounded by user action and + /// gone when the screen is, so the size is not the concern; that a stale + /// entry cannot misfire is, and it cannot: the check is keyed on the + /// identities a *result* removed, never on this set, so a stale entry can + /// only ever match a later result removing that same identity — precisely + /// when the reset is wanted anyway. + pending_unloads: BTreeSet, /// Contacts-tab state. Owned here to debounce /// [`crate::backend_task::dashpay::DashPayTask::LoadContacts`] to a /// single dispatch per tab entry, instead of firing every frame. @@ -94,6 +118,7 @@ impl IdentityHubScreen { last_good_landing: HubLanding::Onboarding, home_state: HomeState::default(), settings_tab: SettingsTab::new(), + pending_unloads: BTreeSet::new(), contacts_state: super::contacts::ContactsState::default(), profile_cache: super::profile_cache::ProfileCache::default(), selection: HubSelection::default(), @@ -115,6 +140,23 @@ impl IdentityHubScreen { } } + /// Remember the identity an unload was dispatched for. + /// + /// The removal is answered frames later, and `SettingsTab::ensure_selected` + /// reconciles its retained identity to the effective one on every frame in + /// between — so by the time the result lands, both the explicit selection + /// and the tab's identity can name the replacement, and nothing left in the + /// screen would say which identity's caches are now stale. This is that + /// record, and the reconciler cannot move it. + fn capture_unload_dispatch(&mut self, action: &AppAction) { + if let AppAction::BackendTask(BackendTask::IdentityTask(IdentityTask::RemoveIdentity { + identity_id, + })) = action + { + self.pending_unloads.insert(*identity_id); + } + } + fn reset_contacts_for_identity_change(&mut self) { release_request_guards_for_abandoned_confirmations( &mut self.contacts_state, @@ -460,6 +502,7 @@ impl ScreenLike for IdentityHubScreen { } }); self.capture_contact_info_update(&action); + self.capture_unload_dispatch(&action); // A picker card click sets the active identity and routes to Home. if let Some(id_str) = picked_identity @@ -575,6 +618,55 @@ impl ScreenLike for IdentityHubScreen { ); } } + // The Settings tab's unload lands here. The tab re-resolves its own + // selection next frame, so this only has to report the outcome — + // including the case where the identity went but its voter twin + // stayed, which the user would otherwise read as a failed removal. + BackendTaskSuccessResult::RemovedIdentities { + identity_ids, + associated_cleanup_failed, + cleanup_deferred, + } => { + // Two separate questions with two separate answers. The + // app-wide selection is dropped only when it is the thing that + // was removed, so the hub falls back deliberately instead of + // resolving around a pointer to a deleted identity. + let explicit_selection = self.app_context.selected_identity_id(); + if explicit_selection.is_some_and(|active| identity_ids.contains(&active)) { + self.app_context.set_selected_identity(None); + } + // The caches follow the *effective* identity, which is a + // fallback whenever no selection is stored, so they are reset + // on the wider condition — contacts, pending contact + // confirmations, and the profile cache — matching every other + // identity-switch path, so a resolve to the next identity next + // frame does not inherit state that still belongs to the one + // just removed. + if removal_invalidates_identity_caches( + explicit_selection, + self.settings_tab + .selected_identity() + .map(|identity| identity.identity.id()), + &self.pending_unloads, + identity_ids, + ) { + self.reset_contacts_for_identity_change(); + self.profile_cache.reset(); + } + // Only the identities this result answers for; another unload + // still in flight keeps its own record. + self.pending_unloads + .retain(|pending| !identity_ids.contains(pending)); + let (message, message_type) = crate::ui::identities::removed_identities_banner( + *associated_cleanup_failed, + *cleanup_deferred, + ); + let handle = + MessageBanner::set_global(self.app_context.egui_ctx(), message, message_type); + if message_type == MessageType::Warning { + handle.disable_auto_dismiss(); + } + } _ => {} } } @@ -775,6 +867,40 @@ fn applies_to_selected_identity( selected.as_ref() == Some(result_identity) } +/// Whether a completed removal leaves the Hub's identity-scoped caches holding +/// data for an identity that no longer exists. +/// +/// The caches follow the *effective* identity — the explicit selection when +/// there is one, a fallback otherwise — so the explicit pointer alone does not +/// answer this: a single-identity account never stores one, and its unload +/// would reset nothing. Re-resolving the fallback cannot answer it either, +/// because by the time this result arrives the removed identity is gone from +/// storage and the resolver names whichever identity replaced it. +/// +/// Nor is the Settings tab's retained identity enough on its own, which is +/// what an earlier version of this got wrong: it is itself reconciled to the +/// effective identity on every frame, and a frame can render between the +/// dispatch and the result, leaving all three of storage, the selection and +/// the tab naming the replacement. Only the id captured when the unload was +/// dispatched survives that, so it is checked alongside the two live pointers +/// — any of the three naming a removed id invalidates the caches. +/// +/// `dispatched_unloads` is a set because unloads overlap: a result answers for +/// one of them while the others are still in flight, so matching against a +/// single remembered id would miss every removal but the most recent. +fn removal_invalidates_identity_caches( + explicit_selection: Option, + settings_identity: Option, + dispatched_unloads: &BTreeSet, + removed: &[Identifier], +) -> bool { + removed.iter().any(|identity| { + explicit_selection == Some(*identity) + || settings_identity == Some(*identity) + || dispatched_unloads.contains(identity) + }) +} + fn handle_profile_updated( settings: &mut SettingsTab, profiles: &mut super::profile_cache::ProfileCache, @@ -930,6 +1056,294 @@ mod tests { ); } + /// The Hub's identity-scoped caches follow the *effective* identity, which + /// is a fallback whenever no explicit selection is stored — the ordinary + /// state for a single-identity account, where the user never touches the + /// picker. Unloading that identity must therefore reset the caches even + /// though `selected_identity_id()` names nothing, or Contacts keeps the + /// removed identity's rows and its spent one-shot load guard while every + /// action resolves under whichever identity became the fallback. + /// + /// Re-resolving the fallback here cannot answer the question: by the time + /// this result lands the removed identity is gone from storage, so the + /// resolver names its replacement. The Settings tab's retained identity is + /// the surviving record of which identity the unload was started for. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn unloading_a_fallback_selected_identity_resets_the_identity_caches() { + let (_temp_dir, context) = wired_test_context().await; + let fallback = seed_user_identity(&context, 1); + let fallback_identity = context + .get_local_qualified_identity(&fallback) + .expect("read the seeded identity") + .expect("identity present"); + let mut screen = IdentityHubScreen::new(&context); + screen + .settings_tab + .select_identity_for_test(fallback_identity.clone()); + + assert_eq!( + context.selected_identity_id(), + None, + "precondition: the identity is effective by fallback, not by an explicit selection" + ); + assert!( + screen.contacts_state.claim_load(), + "precondition: the one-shot load guard starts unclaimed" + ); + screen.profile_cache.record_saved( + fallback, + crate::ui::identity::profile_cache::ProfileFields { + display_name: "Alicia".into(), + bio: String::new(), + avatar_url: String::new(), + }, + ); + + screen.display_task_result(BackendTaskSuccessResult::RemovedIdentities { + identity_ids: vec![fallback], + associated_cleanup_failed: false, + cleanup_deferred: false, + }); + + assert!( + screen.contacts_state.claim_load(), + "the load guard must be re-armed, or Contacts renders the removed identity's \ + rows and never re-fetches for the identity that replaced it" + ); + assert!( + screen + .profile_cache + .get_or_request(&fallback_identity) + .is_none(), + "the removed identity's cached profile must be dropped" + ); + } + + /// The frame ordering the previous fix missed. The unload is answered + /// frames after dispatch, and `SettingsTab::ensure_selected` reconciles the + /// tab's retained identity to the effective one on every frame in between — + /// so a single render before the result lands leaves the explicit selection, + /// the tab, and storage all naming the identity that replaced the removed + /// one. Checking live pointers alone then finds nothing to reset and leaves + /// contact rows, the one-shot load guard, pending tasks and confirmations, + /// and the profile cache bound to an identity that no longer exists. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn an_unload_answered_after_a_reconciling_frame_still_resets_the_caches() { + let (_temp_dir, context) = wired_test_context().await; + let unloaded = seed_user_identity(&context, 1); + let replacement = seed_user_identity(&context, 2); + let replacement_identity = context + .get_local_qualified_identity(&replacement) + .expect("read the seeded identity") + .expect("identity present"); + let mut screen = IdentityHubScreen::new(&context); + + // The user confirms the unload: the screen dispatches it for `unloaded`. + screen.capture_unload_dispatch(&AppAction::BackendTask(BackendTask::IdentityTask( + IdentityTask::RemoveIdentity { + identity_id: unloaded, + }, + ))); + + // A frame renders before the result arrives, and the tab reconciles + // onto the identity that is now effective. Both live pointers now name + // the replacement, or nothing. + screen + .settings_tab + .select_identity_for_test(replacement_identity); + assert_eq!( + context.selected_identity_id(), + None, + "precondition: no explicit selection, so only the dispatch record names the target" + ); + + assert!(screen.contacts_state.claim_load()); + screen.profile_cache.record_saved( + unloaded, + crate::ui::identity::profile_cache::ProfileFields { + display_name: "Alicia".into(), + bio: String::new(), + avatar_url: String::new(), + }, + ); + + screen.display_task_result(BackendTaskSuccessResult::RemovedIdentities { + identity_ids: vec![unloaded], + associated_cleanup_failed: false, + cleanup_deferred: false, + }); + + assert!( + screen.contacts_state.claim_load(), + "the load guard must be re-armed even though no live pointer still names the \ + removed identity" + ); + let unloaded_identity = QualifiedIdentity { + identity: Identity::create_basic_identity(unloaded, PlatformVersion::latest()) + .expect("identity"), + associated_voter_identity: None, + associated_operator_identity: None, + associated_owner_key_id: None, + identity_type: IdentityType::User, + alias: None, + private_keys: KeyStorage::default(), + dpns_names: vec![], + associated_wallets: BTreeMap::new(), + secret_access: None, + wallet_index: None, + top_ups: BTreeMap::new(), + status: IdentityStatus::PendingCreation, + network: Network::Testnet, + }; + assert!( + screen + .profile_cache + .get_or_request(&unloaded_identity) + .is_none(), + "the removed identity's cached profile must be dropped" + ); + } + + /// Two unloads in flight at once. The confirmation blocks input only while + /// it is open, so the user can confirm A, switch to B and confirm B before + /// A's result lands — and A's result then arrives second, when nothing live + /// names A any more. A single remembered id would have been overwritten by + /// B, leaving A's caches bound to an identity that no longer exists. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_second_unload_dispatched_first_does_not_erase_the_first_ones_record() { + let (_temp_dir, context) = wired_test_context().await; + let first = seed_user_identity(&context, 1); + let second = seed_user_identity(&context, 2); + let survivor = seed_user_identity(&context, 3); + let survivor_identity = context + .get_local_qualified_identity(&survivor) + .expect("read the seeded identity") + .expect("identity present"); + let mut screen = IdentityHubScreen::new(&context); + + // Both confirmed before either result arrives. + for identity_id in [first, second] { + screen.capture_unload_dispatch(&AppAction::BackendTask(BackendTask::IdentityTask( + IdentityTask::RemoveIdentity { identity_id }, + ))); + } + + // Frames render in between, so every live pointer names the survivor. + screen + .settings_tab + .select_identity_for_test(survivor_identity); + + // The second unload is answered first, and must not take the first + // one's record with it. + screen.display_task_result(BackendTaskSuccessResult::RemovedIdentities { + identity_ids: vec![second], + associated_cleanup_failed: false, + cleanup_deferred: false, + }); + assert!( + screen.pending_unloads.contains(&first), + "answering one unload must not discard the record of another still in flight" + ); + + // Now the first one's result lands, with nothing live naming it. + assert!(screen.contacts_state.claim_load()); + screen.profile_cache.record_saved( + first, + crate::ui::identity::profile_cache::ProfileFields { + display_name: "Alicia".into(), + bio: String::new(), + avatar_url: String::new(), + }, + ); + + screen.display_task_result(BackendTaskSuccessResult::RemovedIdentities { + identity_ids: vec![first], + associated_cleanup_failed: false, + cleanup_deferred: false, + }); + + assert!( + screen.contacts_state.claim_load(), + "the first unload's caches must be reset when its own result arrives" + ); + let first_identity = QualifiedIdentity { + identity: Identity::create_basic_identity(first, PlatformVersion::latest()) + .expect("identity"), + associated_voter_identity: None, + associated_operator_identity: None, + associated_owner_key_id: None, + identity_type: IdentityType::User, + alias: None, + private_keys: KeyStorage::default(), + dpns_names: vec![], + associated_wallets: BTreeMap::new(), + secret_access: None, + wallet_index: None, + top_ups: BTreeMap::new(), + status: IdentityStatus::PendingCreation, + network: Network::Testnet, + }; + assert!( + screen + .profile_cache + .get_or_request(&first_identity) + .is_none(), + "the first unloaded identity's cached profile must be dropped" + ); + assert!( + screen.pending_unloads.is_empty(), + "each answered unload must retire its own record" + ); + } + + /// A removal that fails before delisting sends no `RemovedIdentities`, so + /// its record is never consumed. That entry is inert rather than harmless + /// by assumption: the check is keyed on the identities a *result* removed, + /// never on the pending set, so a stale entry can only ever match a later + /// result removing that same identity — which is when the reset is wanted. + /// It cannot cause a spurious reset for any other identity. + /// + /// Asserted rather than argued because the alternative — clearing on the + /// error path — is not available: `display_message` carries a string and a + /// severity, no identity, so the screen cannot know which of several + /// in-flight unloads failed. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_failed_unloads_stale_record_never_resets_another_identitys_caches() { + let (_temp_dir, context) = wired_test_context().await; + let failed = seed_user_identity(&context, 1); + let elsewhere = seed_user_identity(&context, 2); + let mut screen = IdentityHubScreen::new(&context); + + // Dispatched, then the removal fails before delisting: no result ever + // arrives for it, so its record stays. + screen.capture_unload_dispatch(&AppAction::BackendTask(BackendTask::IdentityTask( + IdentityTask::RemoveIdentity { + identity_id: failed, + }, + ))); + + // Caches are warm and belong to the identity still in use. + assert!(screen.contacts_state.claim_load()); + + // An unrelated identity is removed — by another device, or by a screen + // that is not this one. Nothing live names it, and this screen never + // dispatched it. + screen.display_task_result(BackendTaskSuccessResult::RemovedIdentities { + identity_ids: vec![elsewhere], + associated_cleanup_failed: false, + cleanup_deferred: false, + }); + + assert!( + !screen.contacts_state.claim_load(), + "a removal naming no identity this screen cares about must not reset its caches" + ); + assert!( + screen.pending_unloads.contains(&failed), + "and must not consume the failed unload's record, which belongs to another identity" + ); + } + #[test] fn a_result_for_the_selected_identity_applies() { assert!(applies_to_selected_identity(Some(id(1)), &id(1))); diff --git a/src/ui/identity/settings.rs b/src/ui/identity/settings.rs index 0497cc6ed..e134e8835 100644 --- a/src/ui/identity/settings.rs +++ b/src/ui/identity/settings.rs @@ -15,14 +15,10 @@ //! - **Delete social profile** — no `DashPayTask::DeleteProfile` variant. //! - **Add / remove alias** and **Make primary** — no `IdentityTask::AddAlias` //! / `RemoveAlias` / `MakePrimaryAlias` variants. -//! - **Unload this identity from this device** — no identity-unload task; the -//! existing `wallet_lifecycle` unload path is wallet-scoped, not identity- -//! scoped, and wiring it here would bypass the dashpay / DPNS state cleanup -//! the operation implies. //! //! These appear as `Gated(missing_task)` non-interactive rows with the copy -//! from design-spec §D (tooltip catalog entries #49 and #59). A TODO comment -//! marks each one so the backend follow-up can search for the flag. +//! from design-spec §D (tooltip catalog entry #49). A TODO comment marks each +//! one so the backend follow-up can search for the flag. use crate::app::AppAction; use crate::backend_task::BackendTask; @@ -35,6 +31,7 @@ use crate::ui::components::component_trait::Component; use crate::ui::components::confirmation_dialog::{ConfirmationDialog, ConfirmationStatus}; use crate::ui::components::message_banner::MessageBanner; use crate::ui::components::pill; +use crate::ui::identities::IDENTITY_REMOVAL_BLOCKED_BY_STORAGE_UPDATE; use crate::ui::identities::register_dpns_name_screen::RegisterDpnsNameSource; use crate::ui::identity::identity_hero_card::HeroIdentityKind; use crate::ui::theme::{ComponentStyles, DashColors, ResponseExt, Spacing, Typography}; @@ -86,8 +83,9 @@ const TIP_ADD_KEY: &str = const TIP_MANAGE_KEYS: &str = "View this identity's keys and their security settings."; const TIP_VIEW_USERNAMES: &str = "Open the complete list of your registered usernames."; const TIP_REFRESH: &str = "Fetch the latest state of this identity from the network."; -const TIP_UNLOAD: &str = "Remove this identity from this device. It remains on Dash Platform — you can load it \ - again later."; +const TIP_UNLOAD: &str = "Remove this identity from this device and permanently delete the private keys \ + stored here. It remains on Dash Platform, but you will need your own backup to use it on this \ + device again."; const TIP_SAVE_ALIAS: &str = "Save this name on this device."; const TIP_ID_COPY: &str = "Copy the full identity ID to your clipboard."; @@ -114,6 +112,48 @@ use crate::model::dashpay::{ MAX_DISPLAY_NAME_CHARS as MAX_DISPLAY_NAME, ProfileFieldError, validate_profile_fields, }; +// --------------------------------------------------------------------------- +// Unload flow +// --------------------------------------------------------------------------- + +/// An open unload confirmation, bound to the identity that was on screen when +/// the button was clicked. +/// +/// Binding the two together is the whole point: the confirmation is answered +/// frames later, and the hub's selection can move in between. Re-reading the +/// selected identity at confirm time would unload whichever identity happens to +/// be selected by then, not the one the user was looking at. +struct PendingIdentityUnload { + dialog: ConfirmationDialog, + target: Identifier, +} + +/// The unload prompt's body, naming the identity it is about to unload. +/// +/// The identity is named because the answer permanently deletes its private +/// keys: a prompt that says only "this identity" gives the user nothing to +/// check it against. +fn unload_confirmation_message(identity_id: &Identifier) -> String { + format!( + "This removes the identity {identity_id} from this device and permanently deletes \ + the private keys stored here. It remains on Dash Platform, but using it again on \ + this device will require your own backup — such as your wallet's recovery phrase \ + or the key you imported.", + identity_id = identity_id.to_string(Encoding::Base58), + ) +} + +/// What a confirmed unload resolves to. +#[derive(Debug, PartialEq, Eq)] +enum UnloadDecision { + /// Remove the identity snapshotted when the button was clicked. + Remove(Identifier), + /// Refuse: the storage update is still running. + StorageBusy, + /// No unload was pending — nothing to do. + Nothing, +} + // --------------------------------------------------------------------------- // Stateful tab component // --------------------------------------------------------------------------- @@ -154,8 +194,8 @@ pub struct SettingsTab { advanced_open: bool, /// Confirmation dialog for the (gated) "Delete social profile" action. confirm_delete_profile: Option, - /// Confirmation dialog for the (gated) "Unload this identity" action. - confirm_unload: Option, + /// Open "Unload this identity" confirmation, bound to its target. + confirm_unload: Option, /// Track whether we have loaded the cached profile for the current /// identity. Reset on identity change. profile_loaded: bool, @@ -232,7 +272,7 @@ impl SettingsTab { }); // Dialogs on top. - action |= self.show_gated_dialogs(ui); + action |= self.show_confirmation_dialogs(ui, app_context); action } @@ -746,26 +786,15 @@ impl SettingsTab { .color(DashColors::text_secondary(dark_mode)), ); ui.add_space(6.0); - // TODO(identity-hub): wire once an identity-scoped unload task - // exists. Wallet-scoped unload (wallet_lifecycle) is too broad - // — it would silently drop sibling identities on the same wallet. let unload = ui - .add_enabled( - false, - ComponentStyles::danger_button("Unload this identity from this device"), - ) - .disabled_tooltip(format!("{TIP_UNLOAD} {GATED_COMING_SOON}")); + .add(ComponentStyles::danger_button( + "Unload this identity from this device", + )) + .clickable_tooltip(TIP_UNLOAD); if unload.clicked() { - self.confirm_unload = Some( - ConfirmationDialog::new( - "Unload this identity", - "This removes the identity from this device. It remains on Dash \ - Platform — you can load it again later.", - ) - .confirm_text(Some("Unload")) - .cancel_text(Some("Keep")) - .danger_mode(true), - ); + // Snapshot the identity being rendered, not the selected + // one: by the time the dialog is answered they can differ. + self.open_unload_confirmation(identity.identity.id()); } }); @@ -776,7 +805,11 @@ impl SettingsTab { // Dialog handling // ----------------------------------------------------------------- - fn show_gated_dialogs(&mut self, ui: &mut Ui) -> AppAction { + fn show_confirmation_dialogs( + &mut self, + ui: &mut Ui, + app_context: &Arc, + ) -> AppAction { if let Some(dialog) = self.confirm_delete_profile.as_mut() { match dialog.show(ui).inner.dialog_response { Some(ConfirmationStatus::Confirmed) | Some(ConfirmationStatus::Canceled) => { @@ -786,18 +819,73 @@ impl SettingsTab { } } - if let Some(dialog) = self.confirm_unload.as_mut() { - match dialog.show(ui).inner.dialog_response { - Some(ConfirmationStatus::Confirmed) | Some(ConfirmationStatus::Canceled) => { - self.confirm_unload = None; + let Some(pending) = self.confirm_unload.as_mut() else { + return AppAction::None; + }; + // Read the response out before touching `self` again — the dialog is + // borrowed from the very field the handlers below consume. + let response = pending.dialog.show(ui).inner.dialog_response; + match response { + Some(ConfirmationStatus::Confirmed) => { + let in_progress = app_context.migration_status().state().is_in_progress(); + match self.take_confirmed_unload(in_progress) { + UnloadDecision::Remove(identity_id) => { + return AppAction::BackendTask(BackendTask::IdentityTask( + IdentityTask::RemoveIdentity { identity_id }, + )); + } + UnloadDecision::StorageBusy => { + MessageBanner::set_global( + ui.ctx(), + IDENTITY_REMOVAL_BLOCKED_BY_STORAGE_UPDATE, + MessageType::Warning, + ); + } + UnloadDecision::Nothing => {} } - None => {} } + Some(ConfirmationStatus::Canceled) => { + self.confirm_unload = None; + } + None => {} } AppAction::None } + /// Open the unload confirmation for `identity_id`, capturing it now so the + /// answer cannot be applied to a different identity later. + /// + /// Blocks input: the answer destroys private keys, and the identity + /// switcher behind an unblocking modal stays clickable, so the user could + /// otherwise confirm for one identity while looking at another's page. + fn open_unload_confirmation(&mut self, identity_id: Identifier) { + self.confirm_unload = Some(PendingIdentityUnload { + dialog: ConfirmationDialog::new( + "Unload this identity", + unload_confirmation_message(&identity_id), + ) + .confirm_text(Some("Unload")) + .cancel_text(Some("Keep")) + .danger_mode(true) + .blocks_input(true), + target: identity_id, + }); + } + + /// Resolve a confirmed unload, closing the dialog either way. Returns the + /// snapshotted target, never the currently selected identity. + fn take_confirmed_unload(&mut self, migration_in_progress: bool) -> UnloadDecision { + let Some(pending) = self.confirm_unload.take() else { + return UnloadDecision::Nothing; + }; + if migration_in_progress { + UnloadDecision::StorageBusy + } else { + UnloadDecision::Remove(pending.target) + } + } + // ----------------------------------------------------------------- // Helpers // ----------------------------------------------------------------- @@ -955,6 +1043,15 @@ impl SettingsTab { // Test helpers (pub(crate)) // ----------------------------------------------------------------- + /// Test helper: adopt `identity` as the tab's current selection — the state + /// `ensure_selected` settles into after a frame in which that identity was + /// the effective one. Lets a caller stage the retained-identity half of an + /// unload without driving a render. + #[cfg(test)] + pub(crate) fn select_identity_for_test(&mut self, identity: QualifiedIdentity) { + self.selected_identity = Some(identity); + } + /// Test helper: force the advanced expander open so the kittest frame sees /// the interior widgets without a click event. Not used by production code /// but kept on the struct for future populated-render tests. @@ -1064,8 +1161,12 @@ mod tests { use std::collections::BTreeMap; fn qualified_identity() -> QualifiedIdentity { + qualified_identity_with_id(7) + } + + fn qualified_identity_with_id(byte: u8) -> QualifiedIdentity { let identity = Identity::create_basic_identity( - Identifier::from_bytes(&[7; 32]).expect("32-byte identifier"), + Identifier::from_bytes(&[byte; 32]).expect("32-byte identifier"), PlatformVersion::latest(), ) .expect("basic identity"); @@ -1401,4 +1502,82 @@ mod tests { // Snapshot consumed → a second call is a no-op. assert!(tab.on_profile_saved().is_none()); } + + /// The prompt is the last thing standing between a click and a permanent + /// key deletion, so it must name what it is about to destroy and hold the + /// screen still while it asks. Naming: "this identity" is unanswerable — + /// the user cannot tell which identity the dialog means. Blocking: the + /// identity switcher behind an unblocking modal stays clickable, so a + /// prompt opened for one identity can be confirmed while the user is + /// looking at another one's page, and the keys that die are not the ones + /// on screen. + #[test] + fn the_unload_prompt_names_its_target_and_freezes_the_screen_behind_it() { + let target = Identifier::from([7; 32]); + let mut tab = SettingsTab::new(); + tab.open_unload_confirmation(target); + + let pending = tab.confirm_unload.as_ref().expect("the prompt is open"); + assert!( + pending + .dialog + .message_text() + .contains(&target.to_string(Encoding::Base58)), + "the prompt must name the identity whose keys it is about to delete" + ); + assert!( + pending.dialog.is_input_blocking(), + "the prompt must block the controls behind it, so the answer cannot \ + be given for one identity while another is on screen" + ); + } + + /// The TOCTOU guard, still needed with the prompt blocking input: the + /// selection can also move without a click (a background refresh, a + /// network switch), and the removal must target the identity the user was + /// looking at when they clicked, not whatever is selected when they + /// confirm. + #[test] + fn confirming_an_unload_targets_the_identity_snapshotted_at_click_time() { + let clicked = Identifier::from([7; 32]); + let mut tab = SettingsTab::new(); + tab.open_unload_confirmation(clicked); + + // The selection moves on while the dialog is open. + tab.selected_identity = Some(qualified_identity_with_id(9)); + + assert_eq!( + tab.take_confirmed_unload(false), + UnloadDecision::Remove(clicked), + ); + } + + /// Removing an identity while the storage update runs is refused by + /// `delete_local_qualified_identity` itself, so the UI must say so up front + /// rather than dispatching a task that can only come back as an error. + #[test] + fn confirming_an_unload_is_refused_while_the_storage_update_runs() { + let mut tab = SettingsTab::new(); + tab.open_unload_confirmation(Identifier::from([7; 32])); + + assert_eq!(tab.take_confirmed_unload(true), UnloadDecision::StorageBusy); + assert!( + tab.confirm_unload.is_none(), + "a refused unload still closes the dialog, so the user can retry deliberately" + ); + } + + /// Resolving a confirmation consumes it: a second resolve must not re-issue + /// the removal for an identity that is already gone. + #[test] + fn an_unload_confirmation_resolves_only_once() { + let mut tab = SettingsTab::new(); + tab.open_unload_confirmation(Identifier::from([7; 32])); + + assert!(matches!( + tab.take_confirmed_unload(false), + UnloadDecision::Remove(_) + )); + assert_eq!(tab.take_confirmed_unload(false), UnloadDecision::Nothing); + } } diff --git a/src/wallet_backend/kv_test_support.rs b/src/wallet_backend/kv_test_support.rs index dbc62f6c0..a33c573a5 100644 --- a/src/wallet_backend/kv_test_support.rs +++ b/src/wallet_backend/kv_test_support.rs @@ -6,8 +6,8 @@ //! `backend_task/migration/`) with byte-identical `get`/`put`/`delete` //! bodies. Consolidated here following the `leak_test_support` pattern. -use std::sync::Mutex; use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::{Condvar, Mutex, PoisonError}; use platform_wallet_storage::{KvError, KvStore, ObjectId}; @@ -85,6 +85,7 @@ impl KvStore for InMemoryKv { pub(crate) struct FailingKv { inner: InMemoryKv, fail_reads: AtomicBool, + fail_deletes: AtomicBool, puts: AtomicUsize, } @@ -96,6 +97,13 @@ impl FailingKv { self.fail_reads.store(fail, Ordering::Relaxed); } + /// Make every subsequent `delete` fail with [`KvError::LockPoisoned`] + /// (`true`), or restore normal deletes (`false`). The stored value is left + /// in place, which is what a real failed delete leaves behind. + pub(crate) fn fail_deletes(&self, fail: bool) { + self.fail_deletes.store(fail, Ordering::Relaxed); + } + /// How many `put` calls have reached the store. pub(crate) fn put_count(&self) -> usize { self.puts.load(Ordering::Relaxed) @@ -117,6 +125,169 @@ impl KvStore for FailingKv { self.inner.put(scope, key, value) } + fn delete(&self, scope: &ObjectId, key: &str) -> Result<(), KvError> { + if self.fail_deletes.load(Ordering::Relaxed) { + return Err(KvError::LockPoisoned); + } + self.inner.delete(scope, key) + } + + fn list_keys(&self, scope: &ObjectId, prefix: Option<&str>) -> Result, KvError> { + self.inner.list_keys(scope, prefix) + } +} + +/// An [`InMemoryKv`] that stalls *after* each read has taken its snapshot. +/// +/// Widens the window in which two concurrent read-modify-write mutations of +/// one key both observe the pre-mutation state, so an unserialized mutation +/// *usually* loses its peer's update. +/// +/// Only usually: elapsed time establishes no happens-before between threads. +/// A delayed thread can wake and write before its peer has even reached the +/// read, in which case the peer observes the completed write and nothing is +/// lost — so a test built on this fake can pass against code whose +/// serialization was removed. Use it to make a race *likely* (a scheduling +/// probe); use [`RendezvousKv`] when a test has to be the standing guard for +/// an invariant, since only that one makes the interleaving certain. +#[derive(Default)] +pub(crate) struct StallingReadKv { + inner: InMemoryKv, +} + +impl KvStore for StallingReadKv { + fn get(&self, scope: &ObjectId, key: &str) -> Result>, KvError> { + let value = self.inner.get(scope, key); + std::thread::sleep(std::time::Duration::from_millis(200)); + value + } + + fn put(&self, scope: &ObjectId, key: &str, value: &[u8]) -> Result<(), KvError> { + self.inner.put(scope, key, value) + } + + fn delete(&self, scope: &ObjectId, key: &str) -> Result<(), KvError> { + self.inner.delete(scope, key) + } + + fn list_keys(&self, scope: &ObjectId, prefix: Option<&str>) -> Result, KvError> { + self.inner.list_keys(scope, prefix) + } +} + +/// How often a waiting [`RendezvousKv`] reader re-checks its release condition. +/// +/// A polling interval, not a deadline: the wait below has no give-up branch, so +/// this changes how promptly a released reader notices and nothing else. It +/// cannot decide an outcome — an earlier version of this fixture used a timeout +/// here that could, which is the bug this construction exists to remove. +const RENDEZVOUS_POLL: std::time::Duration = std::time::Duration::from_millis(1); + +/// An [`InMemoryKv`] that holds every read until its peers can no longer take a +/// pre-mutation snapshot of their own. +/// +/// The standing guard for lost-update invariants. A reader is released when +/// either of two things is true, and the distinction is the whole design: +/// +/// 1. every armed reader has taken its snapshot — the unserialized case, where +/// all of them hold the same pre-mutation value and whoever writes last +/// destroys its peers' updates; or +/// 2. the readers that have not arrived are *provably unable to arrive*, +/// because they are blocked acquiring the serialization the code under test +/// holds — the serialized case, reported by the `peers_blocked` predicate +/// the test supplies. +/// +/// Nothing here is timed. The wait has no deadline and no give-up branch, so a +/// peer that is merely slow cannot release its partner early: the partner waits +/// for it, however long it takes, and the lost update happens. That is the +/// property the previous sleep- and timeout-based versions of this fixture both +/// lacked — each let a delayed peer arrive *after* its partner had already +/// written, read the completed write, and pass a test that should have failed. +/// +/// The cost of having no deadline is that a peer which dies or never runs hangs +/// the test instead of failing it. That is the right trade: a hang is a loud +/// failure that gets investigated, and the alternative is a green that means +/// nothing. +/// +/// Reads are unrestricted until [`Self::arm`] is called, so a test can seed +/// state through the same store before the concurrent phase begins. +#[derive(Default)] +pub(crate) struct RendezvousKv { + inner: InMemoryKv, + state: Mutex, + released: Condvar, +} + +#[derive(Default)] +struct RendezvousState { + /// Readers that must arrive before any is released. `None` = unarmed. + expected: Option, + arrived: usize, + /// How many peers are currently blocked acquiring the serialization under + /// test, and so can never reach the read. Supplied by the test, because + /// only the test knows what the code under test serializes on. + peers_blocked: Option usize + Send + Sync>>, +} + +impl RendezvousKv { + /// Hold the next reads until all `readers` have snapshotted, or until the + /// absentees are accounted for by `peers_blocked` — the count of peers + /// stuck acquiring the lock the code under test holds while reading. + /// + /// A test whose code under test has no such lock passes a predicate that + /// always answers zero, which is exactly right: nothing accounts for the + /// absentees, so the arrived reader waits for them indefinitely. + pub(crate) fn arm( + &self, + readers: usize, + peers_blocked: impl Fn() -> usize + Send + Sync + 'static, + ) { + let mut state = self.state.lock().unwrap_or_else(PoisonError::into_inner); + state.expected = Some(readers); + state.arrived = 0; + state.peers_blocked = Some(Box::new(peers_blocked)); + } + + /// Block until every armed reader has snapshotted or the missing ones are + /// provably blocked. Never gives up: see the type's documentation. + fn rendezvous(&self) { + let mut state = self.state.lock().unwrap_or_else(PoisonError::into_inner); + let Some(expected) = state.expected else { + return; + }; + state.arrived += 1; + if state.arrived >= expected { + self.released.notify_all(); + return; + } + loop { + let blocked = state + .peers_blocked + .as_ref() + .map_or(0, |peers_blocked| peers_blocked()); + if state.arrived >= expected || state.arrived + blocked >= expected { + return; + } + state = self + .released + .wait_timeout(state, RENDEZVOUS_POLL) + .unwrap_or_else(PoisonError::into_inner) + .0; + } + } +} + +impl KvStore for RendezvousKv { + fn get(&self, scope: &ObjectId, key: &str) -> Result>, KvError> { + let value = self.inner.get(scope, key); + self.rendezvous(); + value + } + + fn put(&self, scope: &ObjectId, key: &str, value: &[u8]) -> Result<(), KvError> { + self.inner.put(scope, key, value) + } + fn delete(&self, scope: &ObjectId, key: &str) -> Result<(), KvError> { self.inner.delete(scope, key) }