diff --git a/.claude/skills/release-review/SKILL.md b/.claude/skills/release-review/SKILL.md new file mode 100644 index 000000000..b7c78f7eb --- /dev/null +++ b/.claude/skills/release-review/SKILL.md @@ -0,0 +1,487 @@ +--- +name: release-review +description: "This skill should be used when preparing to QA a dash-evo-tool nightly/weekly build before it ships — comparing the last published pre-release against the current development-branch HEAD via a verified changelog, a scoped code review, and a true A/B GUI regression pass with a visual changelog." +--- + +# Release Review + +End-to-end nightly-release QA campaign for dash-evo-tool: find what actually changed since the +last shipped pre-release, run a scoped code review, and prove the GUI isn't worse than before in +the happy flow. Produces a triageable findings report plus a visual (screenshot) changelog a human +can eyeball in minutes. + +This procedure assumes nothing about the machine it runs on beyond a working Rust toolchain, `git`, +and the `gh` CLI, and nothing about which Claude Code capabilities are installed beyond the +built-in ones (Bash, file tools, and — if available — the ability to spawn sub-agents for +independent work). Anywhere a specialized skill/plugin would make a step faster or more rigorous, +it's named as an optional example in parentheses, never as a hard requirement — read "if you have +access to X" as "otherwise do the equivalent by hand." + +**Pick a scratch root before starting** and reuse it throughout — a directory outside the repo's +own working tree that this run can write to freely (a fresh `mktemp -d`, or a fixed directory you +control). Every path below written as `/...` means "somewhere under that root," not a +literal path — don't hardcode one run's actual path into anything durable (docs, other agents' +instructions, the final report). + +## Rerunning after a fix lands (or a pause) + +The trigger is usually "a fix/PR landed, redo the campaign against the new HEAD" or resuming after +a § Handling a Fixture/Infra Blocker pause. Don't rebuild from scratch: + +1. **Re-verify the baseline is still current** (`gh release list --repo dashpay/dash-evo-tool` — a + newer pre-release may have shipped since the prior run). A matching tag *name* is not enough: + re-resolve the canonical peeled tag commit + (`git ls-remote "refs/tags/^{}"`) and compare it against the baseline SHA + recorded in this run's `qa-run-state.md`. Only when both the tag name and that SHA are unchanged + is the prior run's worktree/binary for it still valid — check for an existing `git worktree` and a + previously-built binary under your scratch root before rebuilding anything. Fail closed on a + mismatch, and equally when no baseline SHA was recorded: re-establish the baseline per Phase 1 and + rebuild its binary rather than reusing an artifact you can't prove. +2. **Only rebuild what moved.** If the baseline is unchanged *by that SHA comparison* and only HEAD + advanced, reuse the baseline binary as-is; `git fetch` + `git checkout ` the HEAD + worktree in place and rebuild only that binary. This turns a two-cold-build Phase 2b into one + incremental build. +3. **Archive, don't overwrite, the prior run's output.** Before writing fresh Phase 1-5 artifacts, + move the previous run's files into a subfolder (e.g. `run-vs-/`) under the + same artifacts location. Both runs stay comparable/auditable, and nothing silently clobbers a + finding a human might still want (e.g. a confirmed-fixed crash repro). +4. **Reuse scenario files and prior findings as a starting point, not from scratch.** A changelog + pass can diff the prior run's verified commit list against the new range instead of re-deriving + the whole thing; a GUI pass can decide per-scenario whether the flow is unchanged enough to reuse + the prior run's OLD-side (baseline) screenshots rather than recapturing them — a baseline binary + unchanged by step 1's peeled-SHA comparison (not merely by tag name) still has valid screenshots. + Always capture fresh NEW-side screenshots (HEAD moved), and log explicitly which screenshots were + reused vs freshly captured so it's auditable rather than assumed. +5. If any part of this run is delegated (to a sub-agent or a fresh session), give it the prior run's + key facts (SHAs, decisions already made, artifact paths) explicitly — it has no memory of a prior + or paused run and will otherwise re-derive, and burn time on, things already verified. + +## Phase 0 — Fixture preflight (do this FIRST, cheaply) + +Before committing to a real GUI run, verify the end-to-end test fixtures are actually usable *now* +— testnet masternodes deregister and test wallets drain between runs. + +**These fixture variable names are a scenario-doc naming convention, not app config the GUI reads +automatically.** `.env.example` does not define them, and the GUI binary itself doesn't consume any +of them directly — confirm this hasn't changed before relying on it (`grep -rn "E2E_" src/` from a +repo checkout; as of this writing it returns nothing). They exist purely so scenario files and +operators refer to the same fixture by the same name, matching the naming already used by +`tests/backend-e2e/` (which *does* read `E2E_WALLET_MNEMONIC` for its own harness, but that's a +separate test binary, not this GUI). As of this writing, the scenarios under +`docs/gui-testing/scenarios/` use: + +- `E2E_MN_PROTX_HASH`: confirm the value you intend to use is a currently-registered + masternode/evonode before relying on it in a scenario — a stale hash surfaces as a typed + `MasternodeNotFound` error the first time something touches it in the GUI. Check via an SDK/CLI + lookup or a quick platform-explorer query first, since nothing enforces this automatically. +- `E2E_WALLET_MNEMONIC`: confirm the wallet is actually funded on testnet *right now* (a live balance + check, not "it was funded last time"). If empty, get a faucet drop (a testnet faucet — web, CLI, + or whatever this project's e2e docs point at) or a fresh funded fixture BEFORE starting Phase 4 — + don't discover this after a long SPV sync. +- `E2E_IDENTITY_ID`: the identity the top-up check in + `docs/gui-testing/scenarios/dapi-budget-resilience-after-resync.md` funds. Preflight it with a live + identity lookup (does the ID still resolve, and can it receive a top-up?), or decide explicitly and + up front that provisioning will create the identity ad hoc — that costs credits and GUI time, so + it's a decision to make before Phase 4, not to discover mid-scenario. + +If any of these fixtures is broken and can't be fixed quickly: stop here, log it (see § Handling a +Fixture/Infra Blocker below), and don't burn hours running non-fund-dependent checks only — check +with the user whether a partial (navigation-only) pass is worth doing now or the whole campaign +should wait. + +**A fresh data directory does not start in a usable state for these scenarios — provisioning it is a +real, non-optional GUI setup sequence, not a formality.** Confirmed: a fresh install's default +settings select **Mainnet**, not Testnet, and a fresh data directory has no imported wallet, no +identities, and no masternode linkage. Before running any scenario against a freshly-seeded data +directory, drive the GUI itself through: selecting Testnet in the network selector, importing/ +restoring the wallet from the mnemonic fixture, discovering or importing the identities the scenario +needs, and — if the scenario needs it — linking a masternode or seeding legacy-migration state. Do +this explicitly and record it as setup, not as part of the scenario's own measured procedure. + +**Confirmed false-alarm trap**: precisely because the network selector has no reliable default, +double-check it's actually on Testnet before concluding a fixture is dead — a prior run spent real +time diagnosing a "dead masternode" and "unfunded wallet" that were both fine; the app was just +sitting on Mainnet the whole time. Check the network selector FIRST, before treating a zero balance +or a `MasternodeNotFound` as a real fixture problem. + +## Phase 1 — Baseline identification + verified changelog + +1. **Resolve the canonical remote before running any remote-qualified git command.** In a fork-based + checkout, `origin` commonly points at a contributor's fork, not the canonical repo — using it + unconditionally can silently sync tags/branches from the wrong place with no error (confirmed: in + one checkout, `origin` resolved the development branch to a materially different commit than the + canonical repo did). Find the remote whose *fetch URL* is exactly the canonical repo: read the URL + itself (`git remote get-url ` per remote, or the `(fetch)` line of `git remote -v`) and + require an exact match on `github.com/dashpay/dash-evo-tool` — HTTPS or SSH form, with or without + the `.git` suffix. A loose `git remote -v | grep dashpay/dash-evo-tool` is not that check: it also + matches a push-only line and any fork whose URL merely contains the string. If no remote matches, + add a **named** one (`git remote add upstream https://github.com/dashpay/dash-evo-tool.git`) and + fetch through that name — never fetch directly by bare URL, since step 4 resolves + `/` and a URL leaves no remote-tracking ref to rev-parse. + Call this `` below and use it consistently for every step in this skill — never + assume `origin` is it, and don't re-derive it more than once per run. +2. **Find the baseline release**: `gh release list --repo dashpay/dash-evo-tool` filtered to + `draft:false && prerelease:true`, most recent. Automated builds are commonly cut as drafts first + and can be deleted before undrafting (check `.github/workflows/` for the exact release-cutting + workflow and tag-naming scheme currently in use) — a newer *local* tag with no matching GitHub + release is not a valid baseline. +3. **Force-sync the tag from `` before trusting it**: + `git fetch "+refs/tags/:refs/tags/"`, then + `git rev-parse ^{commit}`. **Confirmed gotcha**: a local tag ref can silently diverge from the + remote (stale from an earlier fetch/session) and point at an *ancestor* commit, producing a diff + range that's wrong (larger than reality) without any error. Always re-verify against the remote + before computing any diff range from a tag — and compare peeled commit to peeled commit: + `git ls-remote "refs/tags/^{}"` returns the commit a tag points at, whereas + `git ls-remote --tags ` returns the *tag object* SHA for an annotated tag. + This project's release tags are annotated, so comparing the local peeled commit against the + unpeeled remote line mismatches on every healthy tag — that mismatch is an artifact of the wrong + comparison, not evidence of drift. (If the remote prints no `^{}` line the tag is lightweight and + its single SHA is already the commit. To check both forms explicitly, compare tag object against + `git rev-parse ` and peeled commit against `git rev-parse ^{commit}` as two separate + assertions.) +4. **Determine the active development branch and resolve it to a commit SHA — record that SHA, don't + carry a branch name or bare `HEAD` forward.** Check the repo's own contribution docs (e.g. + `CLAUDE.md`, `CONTRIBUTING.md`) rather than assuming `main`/`master`; this project may use a + dedicated long-lived dev branch instead. Fetch that branch before resolving it — Phase 1's other + fetch syncs tags only, and a stale remote-tracking ref freezes a wrong development SHA into every + later phase with no error (confirmed: a checkout's `/` sat behind + the canonical head that `git ls-remote` reported). Resolve it once: + `git fetch "+refs/heads/:refs/remotes//"` + then `git rev-parse /`, and treat that output as + **the development SHA** for the rest of this run — every later phase (the diff range, Phase 2b's + second worktree, screenshots, the final report) uses this exact SHA. Never substitute a bare + `HEAD` for it (that's the invoking checkout's HEAD, not necessarily the development branch's — + wrong whenever this procedure is run from a feature/PR checkout, which is common) and never + re-resolve the branch ref later in the run, since it can move while you're still working. Compute + the range: + `git log .. --oneline`. +5. **Don't trust `CHANGELOG.md` alone.** Its unreleased section can contain stale/premature entries + or miss real changes. For every commit in range, resolve its PR number and read the actual PR body + (`gh pr view --repo dashpay/dash-evo-tool`) — not just the commit subject, which can undersell + what a PR really changed (confirmed: one PR's CHANGELOG entry omitted two of its own later + review-round fixes). Cross-check both directions: CHANGELOG claims with no backing PR, and merged + PRs with no CHANGELOG entry. +6. **Confirm things are actually in-range, not pre-existing**: `git merge-base --is-ancestor + ` for anything you suspect might already be shipped — don't regression-test + pre-existing behavior as if it were new. This one check is necessary but not sufficient on its + own — see the fuller treatment in the blocker-classification section of Phase 4, which this same + caveat also applies to. +7. Produce a verified changelog grouped by theme, each entry citing commit SHA + PR number, tagged + user-visible (GUI-relevant) vs internal-only (tests/CI/deps), plus a regression risk map by + functional area. Save under your artifacts location for this run (see § Artifact Conventions). + +## Phase 2 — Scoped code review + +Review the diff `...` (three-dot range, using the exact SHA resolved +in Phase 1) thoroughly: security, +project/structural consistency, code quality, dependency risk, documentation accuracy. Run it from +an isolated worktree, never the invoking session's active checkout — a plain `git diff`/`git log` +between two refs doesn't need a checkout of either at all, but reviewing from a clean worktree +avoids any risk of touching the human's working tree. (If a multi-agent code-review skill/plugin is +available — e.g. one that fans reviewers out by dimension and file group — use it; otherwise cover +the same dimensions yourself, splitting the work across multiple passes for a large diff rather than +one shot.) Archive the resulting findings report to your artifacts location immediately if it was +produced somewhere scratch/ephemeral. + +**Scale for the actual diff size.** A genuinely large range (many dozens of files, tens of thousands +of changed lines) deserves a proportionally wider review — more reviewers/passes across more file +groups, not the default handful. If findings are emitted with a file:line `location`, make sure it's +**repo-relative, not an absolute worktree path** — an absolute path like +`/some/scratch/worktree/src/foo.rs:12` breaks any downstream permalink generation and is hard to +read out of context. Also check the diff for review-comment IDs (from a *previous* review round) +that leaked into committed code or test comments — a real, recurring defect class: a prior review's +transient ID (e.g. referencing an old finding number) ending up quoted in a doc comment as if it were +permanent documentation. + +**Severity scores are not negotiable by request.** If you ask a reviewer (human or agent) to +reclassify or re-score a finding, that request is not itself evidence. A reviewer that declines to +inflate a severity score to match your expectation and instead re-derives it honestly — even landing +lower than you expected — is doing its job correctly. Don't push back without a substantive +argument for the new score. + +## Phase 2b — Build both binaries for A/B comparison + +Create two isolated `git worktree`s under your scratch root — one at the baseline tag's commit, one +at **the development SHA resolved in Phase 1** (never a bare `HEAD` — if this procedure is running +from a feature/PR checkout, `HEAD` there is that checkout, not the development branch you diffed and +wrote the changelog against) — never inside the repo's own tracked working tree, and never let a +spawned sub-agent create worktrees on its own initiative if you're coordinating multiple agents +(create them yourself first, hand the paths out). Build each with: + +```shell +CARGO_TARGET_DIR=/-target cargo build --bin dash-evo-tool --manifest-path /Cargo.toml +``` + +Distinct `CARGO_TARGET_DIR`s are mandatory — both binaries must coexist afterward for the A/B run; +sharing a target dir means whichever built last silently overwrites the other's binary at the same +output path. Always pass `--manifest-path` explicitly (worktree auto-discovery can resolve to the +wrong checkout). Build sequentially if the machine is RAM-constrained (two full dependency-graph +cold builds concurrently risks OOM/thrashing) — check available memory / existing load first. + +**Confirmed gotcha, background builds only, both Linux and macOS**: a `cargo build` backgrounded +(`... &`, `nohup`, or similar) from an automated shell can fail with a "cargo: No such file or +directory"-style error even though `cargo` works fine in a normal foreground call — a backgrounded +subshell doesn't always inherit the interactive profile that puts `cargo` on `PATH` (common when +installed via `rustup` under `~/.cargo/bin`). Capture the resolved path with `command -v cargo` (or +`rustup which cargo`) *while it's confirmed working*, and use that absolute path for any backgrounded +build invocation rather than the bare `cargo` name. + +## Phase 3 — Author GUI regression scenarios + +Informed by Phase 1's risk map, write **version-agnostic** scenario files under +`docs/gui-testing/scenarios/` (from that directory's `TEMPLATE.md`) — the same procedure runs +against both binaries, so never phrase a step as "confirm the fix works." Bake the blocker rule (§ +below) directly into each scenario's "Expected outcome" section. Update the "Scenario index" table +in `docs/gui-testing/README.md`. + +Also produce a risk-prioritized `docs/user-stories.md` checklist, two tiers: +- **Deep tier**: full acceptance-criteria comparison (old vs new) for every story plausibly touched + by the diff. +- **Smoke tier**: everything else — screen opens, no crash, no error in logs — toured in one + continuous session per binary rather than relaunching per story ID. + +Save the checklist under your artifacts location (a QA-campaign planning doc, not permanent product +documentation — don't fold it into the tracked repo). + +If a story text is found stale against shipped behavior while cross-referencing (confirmed: found +one describing a pre-redesign button layout weeks after the redesign shipped), flag it as a doc +defect for the final report — don't silently fix it mid-campaign, that's scope creep. + +## Phase 4 — GUI A/B run + +Drive the actual compiled binary through a real rendered window, twice (once per build) — how you +reach a real window differs by OS, so confirm before assuming: + +- **Any headless/server Linux environment**: check whether `$DISPLAY` (or `$WAYLAND_DISPLAY`) is + already set to something — if the session was reached over SSH with X11 forwarding, that ambient + value points at the *human's own screen*, not a safe throwaway one. **Verify this before every GUI + launch** — launching against the wrong display pops the app window on someone's real desktop. If + no safe display exists, stand up a virtual one (e.g. `Xvfb`) and point `DISPLAY` at it explicitly + for every launch — don't rely on an inherited/ambient value. +- **macOS**: normally has a real display attached already; if running headless/remote (SSH, CI + runner), GUI automation typically needs Screen Recording/Accessibility permission granted to the + driving process, or a remote-desktop/VNC session to attach to. There is no X-server equivalent to + spin up standalone the way `Xvfb` works on Linux. +- Whatever the mechanism, **isolate a scratch data directory per launch** (`mktemp -d`, portable on + both Linux and macOS) and point the app's data-dir env var at it — never reuse or mutate a real + user's data directory. + +For each scenario/story check: confirm no conflicting instance of the app is already running +(process list, e.g. `pgrep -f dash-evo-tool` or `ps aux | grep dash-evo-tool` if `pgrep` isn't +available) before launching. Launch **old**, perform the flow, screenshot, fully close (confirm +process exit); launch **new**, identical flow, screenshot, close. Never run both simultaneously — +one real window/display target at a time, and it keeps screenshot provenance unambiguous. Check the +app's log output for both runs (with an isolated data directory, verify where logs actually landed +for this run rather than assuming a fixed default path). + +**Isolated data directories give the two builds separate local state, but not separate live-network +state — plan for that separately.** A scenario that spends a UTXO, consumes a deposit, registers a +DPNS name, or moves credits mutates shared on-chain/live-network fixture state; running the baseline +binary through it and then the development binary through the identical flow means the second run +starts from state the first run already changed (an already-consumed deposit, a spent UTXO set, a +name that's no longer available), which can produce a false regression or a false pass. For any +scenario step that mutates fund-moving or name-registering state, give each build its own +independently-funded/equivalent fixture — a separate wallet, identity, or name per side, or a fixture +restored to an equivalent starting state between the two runs — rather than sharing one. Recording +each side's starting balances/UTXOs/deposits/identities/DPNS names and "accounting for the +difference" afterwards is not an alternative: you cannot account your way past a DPNS name that no +longer exists or a deposit the first run already consumed, because the second build can't execute the +flow at all. Steps that only read state (navigation, display checks) aren't affected — reserve this +for anything that actually spends or registers something. + +**Blocker rule** (confirm with the user for each run — the default below is a starting point, not a +universal constant): +- **Blocking**: new version worse than old from the user's perspective **in the happy flow**, or a + **data-loss** scenario. +- **Not blocking**: timing noise with no user-visible effect; anything reproduced identically on + **both** builds (pre-existing — note it, don't flag as a regression). Read that exemption + narrowly — a *new* intermittent race stays blocking whenever it breaks the happy flow, crashes the + app, or loses data. "It's a race" never downgrades those; intermittency only means you reproduce it + more times before confirming. +- Reproduce anything about to be marked blocking at least twice before confirming. +- If a repro attempt genuinely can't be reproduced after several tries, don't leave it open + indefinitely — write up the negative evidence, downgrade to backlog, and say explicitly that repro + was attempted and failed, rather than silently dropping it or mandating another attempt on every + future rerun. + +**Blocking classification must say pre-existing vs diff-introduced, not just "blocking."** A defect +can be blocking-worthy on its own severity (real data loss) while NOT being a regression this diff +caused. But don't let a single check decide this either way: `git log .. +-- ` returning zero commits proves only that the file wasn't *directly* edited — a +changed caller, a shared model type, a persisted-data format change, a feature gate, or a bumped +dependency can still cause a regression that surfaces in an otherwise-untouched file. Treat old-vs-new +reproduction as the primary evidence (does the exact same trigger actually behave differently on the +two builds?), and use the git-log check as one corroborating signal, not proof by itself — if the +file is untouched but you suspect the behavior still changed, trace the transitive callers/ +dependencies of the code path before concluding "pre-existing." Frame the finding as "we are choosing +to hold/flag a pre-existing defect," not "this release broke X" — the second is a factual claim that +can be wrong and, if wrong, undermines the whole report's credibility. + +Emit confirmed findings in whatever findings format the rest of this campaign's report uses (see +Phase 2/5) — at minimum: title, severity, a location reference (a file:line if one applies, otherwise +a synthetic description like "GUI: `` > ``"), description, recommendation, and +whether it's blocking per the rule above. + +**If findings get consolidated/renumbered by any downstream tooling, treat IDs as provisional and +don't embed them anywhere durable.** A findings pipeline may reassign IDs when merging sections +(confirmed: GUI-section IDs were renumbered twice across two merge passes in one run). Never +reference a provisional or even a "final" ID in a scenario file, the visual changelog, or an +inter-agent message meant to outlive one exchange — reference findings by **title** or **scenario +name** instead, which survive renumbering. Warn whoever's writing durable artifacts about this +explicitly before they start, not after — catching and fixing already-embedded IDs later is possible +but wasteful. + +## Escalating a claim that can't be settled from the repo alone + +A review pass occasionally surfaces something it explicitly flags as unresolved from static analysis +— e.g. "does the live network actually behave the way I think the code implies?" Don't ship that as +a confirmed blocking finding, and don't dismiss it either: investigate it directly and narrowly +rather than waiting for a human to resolve it, if it's the kind of thing investigable from evidence +(read the actual pinned dependency's source, trace the real call order, and — if it's the only way to +get ground truth — make a strictly read-only live check, e.g. a version/status query against the +live network, never a transaction). Confirmed valuable: a claim about a possible mainnet outage was +fully resolved via a few read-only queries against live mainnet nodes in well under an hour — cheap +compared to either shipping a false alarm or silently dropping a real risk. When the verdict lands, +fold the correction into the report explicitly — state that an earlier pass got something wrong and +why, rather than quietly replacing it. A report that visibly corrects itself is more trustworthy than +one that never admits an error. + +## Provenance: unrelated concurrent work on a shared environment + +If this campaign runs on a machine or account that may have other unrelated work in flight (other +worktrees, other sessions, other automation touching the same repo), a QA pass may stumble on +evidence of a *different*, unaffiliated effort — a report file, a branch, a running build — that +looks relevant to something this campaign is investigating. Before treating it as this campaign's +own finding: +1. **Verify it's actually external**, don't assume: check for a still-live process tied to it, a + moving `git log` HEAD in its worktree, a different session/transcript identity, or simply that + nothing in this campaign's own roster claims authorship. +2. **If external, exclude it entirely from the formal report** — don't cite, reference, or fold in + findings/fixes from work with no chain of custody to this campaign, even if independently verified + as genuine. It may be unstable (still evolving), unauthorized for this release, or owned by + someone who hasn't consented to having their in-flight work published in your report. +3. **Do surface its existence to the user as a side note** when reporting results — it's relevant + context for a ship decision even though it's not your finding. +4. **Leak-scan the final artifacts** before declaring done: search the produced report/HTML for any + branch name, commit hash, worktree path, or distinctive identifier from the external work, to + confirm the exclusion actually held rather than assuming it did. + +If you're confident this run is the only thing touching the repo/environment, this section doesn't +apply — but check rather than assume on a shared machine. + +## Security incidents during GUI testing + +**Prevent first — every control below is cheaper than the incident response after it.** When a run +handles a real secret (a testnet mnemonic, a password fixture): +- Enter it through the GUI's own masked/protected input wherever one exists, rather than any path + that echoes the value back on screen. +- **Never pass a real mnemonic, password, or private key as a bare command-line argument** — it lands + in process listings, shell history, and this session's transcript, all of which outlive the run. + Pass it via an environment variable the app reads, or a file with restrictive permissions. +- Don't dump the environment (`env`, `printenv`, `set`) into a transcript, and redact fixture secrets + out of any log output before quoting it; check what a raised log level actually prints before + capturing it wholesale. +- **Check before saving or embedding, not after**: eyeball each screenshot for visible secret text, + and grep transcripts, logs, and generated artifacts for the fixture's own patterns (the mnemonic's + words, the password string, `xprv`/`tprv`/WIF prefixes) before they reach the artifacts location. + Repeat that check immediately before Phase 5's base64-inline step — inlining bakes the image into a + self-contained, trivially forwardable HTML file that is far harder to un-publish than a stray PNG. +- Restrict filesystem permissions where secrets and captures land (`chmod 700` the scratch/artifacts + directories, `chmod 600` any file holding a secret) — on the machine they actually land on. + +If GUI testing handles real secrets (a testnet mnemonic, a password fixture) and something goes +wrong — a screenshot with the secret visible, a secret typed into a visible log/transcript — the +self-report is not the end of it: +1. **Independently verify remediation** — don't just trust "I deleted it." Check the files are + actually gone, check adjacent files/screenshots for spillover, confirm no other copy exists. +2. **Write an incident note** into this run's artifacts (what happened, exposure window, what was + verified, actual risk given the context — testnet vs mainnet, network-restricted vs public, + closed window vs ongoing). +3. **Don't unilaterally rotate a shared fixture.** A fixture used beyond just this campaign (e.g. by + other test suites in the repo) shouldn't be silently burned and replaced — queue the + rotate-or-accept decision for the user. +4. **Never screenshot a populated secret-entry field going forward** — capture before typing or + after the field is cleared/masked, for the rest of the run. +5. Log the incident somewhere that survives this session ending (an issue tracker entry, a durable + notes/memory mechanism if one is available, or at minimum a clearly-named file in this run's + artifacts) so it isn't lost if the user doesn't act on it immediately. + +## Phase 5 — Visual changelog + consolidated report + +Build a self-contained HTML page (`visual-changelog.html`) presenting old/new screenshot pairs side +by side per scenario/story, with a verdict badge and a one-line rationale each — inline the images +(e.g. base64) so the page has no external file dependencies. Run the pre-embedding secret check from +§ Security incidents during GUI testing over every image and caption first: once inlined, a leaked +secret travels with every copy of the page. Save it to your artifacts location. +Merge Phase 4's findings into Phase 2's report as an additional section, re-validate the combined +report against whatever schema/format it uses, and re-render any human-readable version. The +combined report is then ready to hand to the user for triage — code-review and GUI-regression +findings reviewed together in one pass. (If an interactive triage tool is available, use it; +otherwise a written summary the user can respond to works fine.) + +## Handling a fixture/infra blocker + +If Phase 4 hits a real environment blocker (dead fixture, unreachable network, broken CI runner — +not an app bug), don't push through with silent scope cuts: +1. Report the exact blocker (the actual error, what was verified, what's still needed) to the user + and ask how to proceed — a scope trim is a real thoroughness-vs-speed tradeoff, not something to + decide alone. +2. If the user says to pause: cleanly close any running app instances, save interim findings under + this run's artifacts location, and log what's blocked, what already completed and is reusable on + a rerun, and what a fresh run needs from scratch (baseline/HEAD will likely have moved on by + then) — durably enough to survive the current session ending. +3. Updating this skill file with anything newly learned doesn't depend on Phase 4/5 completing — do + it whenever asked, even mid-block. + +## Artifact conventions + +Pick one consistent location outside the tracked repo for everything durable in a given run (a dated +directory under your scratch root, or wherever your own setup keeps generated reports) and use it +for: the verified changelog, the code-review report (JSON/Markdown or whatever format is +in use), GUI findings, the GUI test log, `visual-changelog.html`, screenshots, and the user-stories +checklist. If your environment has a way to share generated files with the user (a hosting mechanism, +an upload/artifact tool, or simply pointing at local paths they can open), use whatever's actually +available — don't assume a specific serving mechanism exists. Note that "git-ignored" is not a safety +property here: `.gitignore` only governs paths inside the repository, so a directory outside it needs +its own access controls (restrictive permissions, and not sitting under anything published or synced) +rather than an ignore rule. + +Keep a small `qa-run-state.md` at the top of that location as a durable scratchpad: verified SHAs, +worktree/binary paths, a phase checklist, and an explicit "open items for the user" list. That's what +lets a rerun (or a resumed session after a context loss) pick up cleanly instead of re-deriving +everything. On a same-day rerun, archive the prior run's files into a subfolder first (see § +Rerunning above) rather than overwriting them. + +Don't bake one run's actual findings, SHAs, or open items into *this* skill file — they belong in +that run's own artifacts. When updating this file after a run, fold in newly-learned *procedure* +(a gotcha, a missing step, a better ordering) and leave the specific bug numbers/dates/commits out. + +## Running this unattended (user stepped away) + +This campaign is long enough that the user often steps away mid-run. If so: +- **Corroborate status against direct evidence, not just self-reports.** A status message saying + "still working" is not itself proof of progress — check actual file timestamps in the scratch + area and running processes before concluding anything is stuck, or conversely before assuming + silence means done. +- **Queue decisions instead of fabricating approval.** Anything that's normally the user's call — + merging/pushing anything, rotating a shared fixture, a scope tradeoff on a fixture blocker — gets + written up and queued, never assumed. Keep working on whatever doesn't need that decision rather + than stalling the whole campaign on one open question. +- **Notify once, at the end**, with the full queue: ship read, report location, and every item that + needs a decision — not a running commentary as each phase completes. Use whatever notification + mechanism your environment provides (a desktop/mobile notification tool, a chat message); if none + is available, just deliver the full summary in your final response. + +## Skills used (optional accelerators, not requirements) + +Every step above works performed directly with standard tools (`git`, `gh`, `cargo`, a real GUI +session, plain file writes). If your environment happens to have compatible skills/plugins +installed, they can speed things up — for example: a multi-agent orchestration capability for +running independent phases in parallel and tracking a long multi-phase run durably; a multi-agent +code-review skill (e.g. something like `grumpy-review`) instead of a single-pass manual review; a +structured findings-report format/schema and severity-scoring convention instead of a plain +Markdown table; an interactive browser-based triage tool instead of a written summary; a GUI-driving +skill that already handles the headless-display setup for you; a testnet-faucet helper instead of a +manual funding step; and an end-of-run notification skill for alerting a user who's stepped away. +None of these are assumed to exist — treat every mention above as "if available," never as a +dependency this procedure requires. diff --git a/docs/gui-testing/README.md b/docs/gui-testing/README.md index 7da93a2b9..f66a7b5bd 100644 --- a/docs/gui-testing/README.md +++ b/docs/gui-testing/README.md @@ -251,10 +251,59 @@ one scenario silently invalidating another: run spanning hours, set `CARGO_TARGET_DIR` to a private path before building and hash-verify (`sha256sum`) before each relaunch. +## A/B build comparison contract + +Every scenario in this library is run twice with the identical procedure: once +against the **baseline** binary and once against the **development** binary +selected for the current campaign. Both are roles, not specific versions — +record the two exact commit SHAs in that campaign's own artifacts, never in a +scenario file, so the library stays valid for the next campaign too. + +**Judge by observation, not by expectation.** Record what each build actually +does at every step before classifying anything. Never assume which build is +"correct", which behavior a given change was meant to introduce, or which +direction a difference is supposed to point. + +**Blocker rule** (a campaign may narrow this, never silently widen it): + +- **Blocking**: the development build is worse than the baseline build from the + user's perspective in the happy flow, or a **data-loss** outcome on either + build. +- **Not blocking**: a purely cosmetic difference, or behavior reproduced + identically on **both** builds (pre-existing — note it, don't flag it as a + regression). Intermittency is not an exemption: a race that reproduces on one + build and not the other is still a regression, and timing sensitivity only + changes how many attempts a repro needs. Reproduce anything about to be + marked blocking at least twice before confirming it. + +**Equivalent fixture state per build.** Isolated data directories give the two +builds separate *local* state, but not separate *live-network* state. Any step +that spends a UTXO, consumes a deposit, registers a DPNS name, or moves credits +mutates shared on-chain state, so the second build's run would otherwise start +from whatever the first run left behind — an already-consumed deposit, a spent +UTXO set, a name that is no longer available — producing a false regression or +a false pass. Before any such step, give each build either its own +independently-funded, equivalent fixture (a separate wallet/identity per side) +or a restored snapshot of the same starting state, and record each side's +starting balances/deposits/identities/names first. Recording the difference and +accounting for it afterwards is **not** sufficient. Steps that only read state +(navigation, display checks) are unaffected. + ## Scenario index +These six scenarios are written **build-neutral**: the same procedure runs +unmodified against either binary of an A/B campaign, with the blocker rule and +the fixture requirement from the [A/B build comparison +contract](#ab-build-comparison-contract) above applied in each file's own +"Expected outcome" section. + | Scenario | What it verifies | |---|---| -| _(none yet — first one lands once the masternode-withdrawal-without-a-wallet run is verified)_ | | +| [`identity-key-recovery-migration.md`](scenarios/identity-key-recovery-migration.md) | Key-placement resolution and legacy-key recovery (#941, #945, #946, #948): keys found/signed/restored/removed correctly regardless of internal filing convention; Keys screen reachable from every interface level; same-numbered-key collisions never cross-contaminate | +| [`wallet-max-send-asset-lock.md`](scenarios/wallet-max-send-asset-lock.md) | "Max" agrees with what the wallet can build/send across Shield/Fund-Platform-Address/Create-Identity/Top-Up/Transfer/Withdraw (#937, #927), including dust-wallet and rapid-UTXO-churn variants | +| [`dpns-registration-flow.md`](scenarios/dpns-registration-flow.md) | Contested-vs-registered DPNS messaging, pending-registration indicator/tooltip, onboarding checklist, and DashPay social-profile save feedback (#918) | +| [`error-banners-identity-home-actions.md`](scenarios/error-banners-identity-home-actions.md) | Error-banner wording, validation consistency, identity-removal responsiveness, Identity Home's action row, already-consumed-deposit messaging, deposit-verification crash fix (#927, #934) | +| [`platform-shielded-availability-after-sync.md`](scenarios/platform-shielded-availability-after-sync.md) | Shielded features correctly activate right after SPV/platform sync instead of silently staying disabled (#936, #938) | +| [`dapi-budget-resilience-after-resync.md`](scenarios/dapi-budget-resilience-after-resync.md) | Repeated SPV `Syncing`↔`Synced` transitions no longer exhaust the shared DAPI request budget and break unrelated actions like identity top-up (#950) | 🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/dapi-budget-resilience-after-resync.md b/docs/gui-testing/scenarios/dapi-budget-resilience-after-resync.md new file mode 100644 index 000000000..bb9ff6880 --- /dev/null +++ b/docs/gui-testing/scenarios/dapi-budget-resilience-after-resync.md @@ -0,0 +1,134 @@ +# Scenario: Connection budget survives repeated resync cycles + +**Verifies:** Cycling the SPV connection state between `Syncing` and `Synced` +repeatedly does not exhaust the app's shared DAPI request budget, and does not +cause an unrelated Platform action (identity top-up) to fail with a generic +"servers are temporarily unreachable" banner. (Risk area: #950, building on +#936/#938.) + +**Tier justification:** The failure this scenario targets is a real, +live-network request-budget exhaustion that only manifests after several +actual `Syncing`→`Synced` transitions against a real DAPI connection — it +cannot be simulated in `kittest` (no network) or isolated to a single +`backend_task` call (the bug is about a background task's cumulative effect +across transitions, not any single call's return value). + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here). + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet + - `E2E_IDENTITY_ID` (or an identity created ad hoc) — an identity to top up +- Verify exact banner/error wording during execution rather than assuming the + text below is literal + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/dapi-budget-resilience.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Initial Platform Info check.** Open Platform Info (Developer/Advanced + view). Record the exact "Current Epoch Information" text, in particular + whether the fee-multiplier line reads "unavailable" or states it is "a + fixed value, not read from the network" (or neither, if the build under + test predates that line entirely). +2. **Force repeated `Syncing`↔`Synced` transitions.** With the app running + and initial sync complete, force several resync cycles in the same + session — e.g. toggle network connectivity briefly (airplane-mode-style + interruption, or a short `iptables`/firewall block on the DAPI/Core + ports if available) 4–6 times over a few minutes, or use "Clear cached + SPV data to force a resync" (NET-020) repeatedly. Each cycle should visibly + pass back through a `Syncing` state before returning to `Synced`. +3. **Attempt identity top-up immediately after the resync cycles.** Top up + an identity's credits (any small amount) right after step 2's last + transition back to `Synced`. Record whether it succeeds, and if it fails, + capture the exact banner text. +4. **Repeat the top-up attempt once more** a short time later (a minute or + two) without additional resync cycling, to distinguish "budget recovers + on its own" from "budget stays exhausted." +5. **Check the logs.** Confirm the log targets exist first — `ls -l "$LOG"` + and `ls -l "$DATADIR"`, which is where the app writes `det.log` / + `det-stderr.log` — then search them: + + ```bash + grep -R 'DapiAllAddressesExhausted' "$LOG" "$DATADIR" + ``` + + Look for that or similar rate-limit/address-pool exhaustion markers + appearing near the resync cycles from step 2, independent of whether the UI + showed an error. Record "not reproduced" only after confirming the files + exist and are non-empty — a missing log file means the check never ran, not + that the marker is absent. +6. **Re-check Platform Info.** Reopen Platform Info after the resync cycling + and top-up attempts; record whether the epoch/fee-multiplier text is + still consistent with step 1's observation (no crash, no stuck "loading" + state). + +## Safety constraints specific to this scenario + +- Step 3's top-up moves real (small) testnet funds — keep amounts minimal, + per the standing fund-movement cap. +- If simulating a network interruption via firewall rules, restore + connectivity immediately after each toggle and confirm via `pgrep`/process + liveness that the app itself never needed restarting. + +## Expected outcome / pass criteria + +Record the observed behavior for each build at every step, then apply this +rule: + +- **BLOCKING** only if HEAD is worse than baseline in the happy flow — e.g. + the identity top-up in step 3 succeeds on baseline after repeated resync + cycling but fails with a generic "servers are temporarily unreachable" + banner on HEAD, or `DapiAllAddressesExhausted` (or equivalent) appears in + HEAD's logs where it did not on baseline — or a **data-loss** outcome on + either build. +- **NOT blocking**: the reverse direction — the baseline build's top-up + failing after repeated resync cycling while the development build's + succeeds. Record it factually as an observed difference between the two + builds; do not assume it was the intended outcome, and do not treat it as a + baseline "failure" to flag. A wording-only difference in the Platform Info + fee-multiplier line (`unavailable` vs `a fixed value, not read from the + network`) is not itself a defect on either build. +- If neither build reproduces an exhaustion-triggered top-up failure within + the attempted resync-cycle budget, record that plainly as "not reproduced + this run" rather than asserting the fix is confirmed — a live-network + race like this can be timing-sensitive. + +## Known gotchas + +- Simulating repeated `Syncing`→`Synced` transitions without real network + interruption tooling may require several minutes of patience — SPV + reconnect timing is not instantaneous. Budget real wall-clock time for + this scenario rather than expecting each cycle to complete in seconds. +- The DAPI request-budget exhaustion this scenario targets is a *shared, + per-client* resource — if another tool/process is also hitting the same + SDK client concurrently (e.g. an `det-cli` smoke test running in + parallel), the observed exhaustion timing may differ from a clean, + single-consumer session. Run this scenario in isolation from other DAPI + traffic against the same data directory. +- This is the only scenario in the library that deliberately drives + multiple resync cycles in one session — if a *different*, unrelated + defect surfaces only under repeated resync (not specific to #950), note + it here rather than assuming it belongs to another scenario file. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/dpns-registration-flow.md b/docs/gui-testing/scenarios/dpns-registration-flow.md new file mode 100644 index 000000000..f386fccc2 --- /dev/null +++ b/docs/gui-testing/scenarios/dpns-registration-flow.md @@ -0,0 +1,134 @@ +# Scenario: DPNS registration messaging and social-profile save feedback + +**Verifies:** DPNS registration correctly distinguishes "registered outright" +from "submitted for a community vote" (contested name), a pending +registration shows a clear indicator/tooltip instead of looking unrequested, +the onboarding checklist counts a pending request toward "Pick a username," +and — from the same underlying PR — the DashPay social-profile save flow +gives real progress/success/error feedback without a stuck progress banner +or a stale result misattributed after switching identities mid-save. (Risk +area: #918.) + +**Tier justification:** Needs a real contested-name registration against a +live testnet name-contest window, real onboarding-checklist state derived +from actual identity data, and real async save timing (a save in flight +when the user switches identity) — none of which a no-display harness can +drive with a live contest clock and real background-task scheduling. + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here). + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet +- A DPNS name likely to be contested (short, generic, or currently popular) + to register during an active contest window +- At least two identities on the same wallet (for the identity-switch- + mid-save step) +- Verify exact tooltip/button text during execution rather than assuming + the wording below is literal + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/dpns-registration.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Register a contested name.** On an identity with no username yet, + register a name likely to be contested. Record the exact completion + message shown — does it say "registered" or does it indicate a pending + community vote? +2. **Identities list indicator.** Record whether/how the Identities list + shows this identity's pending request differently from "no username." +3. **Identity Home indicator.** Record what the Identity Home hero card + shows for this identity — the requested name, a status indicator, or the + generic "pick a username" prompt. +4. **Indicator tooltip.** If a status indicator/pill is present, open its + tooltip and record what it says (who decides, any estimated timing). +5. **Onboarding checklist.** Record whether the onboarding checklist treats + the submitted request as satisfying "Pick a username," and what wording + it uses. +6. **Uncontested name, for comparison.** On a second identity, register a + name unlikely to be contested and record the completion messaging — + confirm it's still distinguishable from the contested case. +7. **Contest resolves.** If the contest window is short enough to wait out, + revisit the identity after resolution and record what the indicator does + (disappears / name displays normally on a win; reverts to "pick a + username" on a loss). +8. **Social profile save, normal case.** Open Contacts → set up/edit social + profile, change the display name or bio, save. Record what feedback + appears during and after the save (progress indicator, then a + success/error banner) and whether the progress indicator clears when the + outcome appears. +9. **Social profile save, forced failure.** If you can force a failure + (briefly disconnect network mid-save), record the same as step 8 for the + failure path. +10. **Switch identity mid-save.** Start a profile save on identity A, and + before it completes, switch to identity B. Once the save's result + arrives in the background, record which identity (if any) shows a + result banner, and whether it's ever attributed to the wrong + (now-active) identity. +11. **Contacts setup CTA.** On an identity with no DashPay profile, open + the Contacts tab and record the setup card's CTA wording and whether any + "Why?"/explanation control actually opens an explanation. + +## Safety constraints specific to this scenario + +- None beyond the standing rules — DPNS registration fees are small and + testnet-only here. + +## Expected outcome / pass criteria + +Record the observed behavior for each build at every step, then apply this +rule: + +- **BLOCKING** only if HEAD is worse than baseline in the happy flow (e.g. + a contested registration falsely claims "Registered!" on HEAD when + baseline correctly said "pending"; a save's progress banner gets + permanently stuck on HEAD where baseline cleared it; a save result gets + attributed to the wrong identity on HEAD where baseline didn't), or a + **data-loss** outcome (a profile edit silently lost) on either build. +- **NOT blocking**: wording differences with no functional consequence; an + issue reproducing identically on both builds; a timing-sensitive race that + needs several attempts to reproduce and only sometimes shows on **both** + builds. +- **Precedence: intermittency never downgrades a wrong-identity result.** If a + save result is attributed to the wrong identity (step 10) on one build and + never on the other, it stays BLOCKING however rarely it reproduces — that + race is exactly what this scenario exists to catch, so being hard to land + inside the race window is a reason to attempt it more times, not a reason to + waive it under the timing-sensitive clause above. +- If the pending-registration indicator exists on only one of the two selected + builds, record that as a feature-presence difference, not a step failure on + the build that lacks it. + +## Known gotchas + +- Contest resolution timing depends on the live masternode voting schedule + on testnet — step 7 may need to be deferred to a later session rather than + blocking the rest of this scenario. +- The identity-switch race (step 10) is timing-sensitive; several attempts + with the save action and the identity switch performed in quick succession + may be needed to land inside the race window on either build. That + difficulty does not soften the result: per the precedence rule in "Expected + outcome", a wrong-identity attribution seen on one build only is blocking + even if it reproduced once in many tries. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/error-banners-identity-home-actions.md b/docs/gui-testing/scenarios/error-banners-identity-home-actions.md new file mode 100644 index 000000000..389fa93e4 --- /dev/null +++ b/docs/gui-testing/scenarios/error-banners-identity-home-actions.md @@ -0,0 +1,134 @@ +# Scenario: Error-banner wording, validation consistency, and Identity Home actions + +**Verifies:** Error banners show plain, actionable text rather than raw +upstream error strings; format-validation rules (DPNS name/memo/keyword/ +label length) behave the same everywhere they're triggered; identity removal +doesn't block the UI thread; Identity Home shows one non-redundant row of +action buttons instead of duplicate destinations; `Send to wallet` is +disabled with an explanation when no usable withdrawal key is held; an +already-consumed deposit says so plainly instead of suggesting a retry; and +the app doesn't crash during deposit verification. (Risk area: #927, #934.) + +**Tier justification:** Needs a real deposit broadcast/verification cycle to +exercise the worker-thread stack-size fix, and real visual confirmation of +button count/labels/layout that a no-display harness can assert on logic but +not on what actually renders. + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here). + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet +- One identity with a locally-held TRANSFER/OWNER key, and ideally a second + identity with only an on-chain authentication key and no locally-held + signing key (for the `Send to wallet` disabled-state comparison) +- Verify exact button labels/wording during execution rather than assuming + the text below is literal + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/identity-home-actions.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Identity Home button row.** Open Identity Home. Record how many action + buttons are present, their labels, and whether any two open the same + destination screen. +2. **`Send to wallet` gating, enabled case.** On the identity with a + locally-held signing key, record whether "Send to wallet" (or whatever + the withdrawal-initiating control is called) is enabled and whether it + opens a working withdrawal flow. +3. **`Send to wallet` gating, disabled case.** On the identity without a + locally-held signing key, record whether the same control is disabled, + and if so, what explanation (if any) is shown. +4. **Funding wording consistency.** Walk through the identity funding + wizard (breadcrumb, heading, sub-heading, CTA buttons) and record whether + the terminology is consistent throughout (vs. mixing two different terms + for the same action). +5. **Already-consumed deposit.** Attempt to fund using a deposit that has + already been consumed by another operation (re-use a completed one). + Record the exact message shown — does it say the deposit cannot be + reused and to choose/start a different one, or does it suggest retrying? +6. **Deposit verification stability.** Perform several deposit verifications + in a row (fund an identity via deposit two or three times). Record whether + the app crashes, then check the logs for a panic marker even if the UI + appeared fine. Confirm the log targets exist first — `ls -l "$LOG"` and + `ls -l "$DATADIR"`, which is where the app writes `det.log` / + `det-stderr.log` — then search them: + + ```bash + grep -R -e 'panicked' -e 'location=' "$LOG" "$DATADIR" + ``` + + Record "no panic" only after confirming the files exist and are non-empty — + a missing log file means the check never ran, not that the app was clean. +7. **Format-validation consistency.** DPNS names, contract keywords, and + DashPay account labels each have their own documented length limit (they + are different fields with different rules by design — do not expect them + to reject at the same length). Instead, pick ONE of these fields and + trigger its format-validation error through two different entry points + that both accept it (e.g. the same overlong DPNS name attempted through + registration and, if another screen also accepts a DPNS name, through + that screen too). Record whether both entry points reject at the same + limit with the same message, for that one field. +8. **Error banner wording.** Trigger a database/parsing-style error if + possible (e.g. an interrupted operation). Record whether the banner text + is plain and actionable, or contains raw error strings/stack + traces/internals. +9. **Identity removal responsiveness.** Remove a local identity and record + whether the UI freezes/stutters during the removal, and how the result is + reported. + +## Safety constraints specific to this scenario + +- Steps 5–6 move real (small) testnet funds — keep amounts minimal. + +## Expected outcome / pass criteria + +Record the observed behavior for each build at every step, then apply this +rule: + +- **BLOCKING** only if HEAD is worse than baseline in the happy flow (e.g. + HEAD crashes during deposit verification where baseline didn't; HEAD's + already-consumed-deposit message is more misleading than baseline's; a + button that worked correctly on baseline is now broken/duplicated on + HEAD), or a **data-loss** outcome on either build. +- **NOT blocking**: a UI freeze/stutter that's cosmetic and self-resolves; + wording differences with no functional consequence; an issue reproducing + identically on both builds (e.g. if baseline also shows 6 duplicate + buttons and HEAD also does — note it, don't block). +- If the two builds' button rows differ in count or labels, record both + observations plainly rather than treating either as automatically wrong — + apply the blocker rule based on functional correctness (do all buttons + work, are destinations distinguishable), not on matching layouts. + +## Known gotchas + +- If `Send to wallet` behaves inconsistently, check whether you're looking + at Identity Home, the identities-list popup, or the Withdrawal screen + directly — these three surfaces historically used different gating + criteria; note which surface any discrepancy is on. +- The change range this scenario covers touches many files across validation, + fee estimation, error banners, and identity removal — if a defect surfaces, + compare directly against the baseline build's behavior for the same step + before concluding it's new. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/identity-key-recovery-migration.md b/docs/gui-testing/scenarios/identity-key-recovery-migration.md new file mode 100644 index 000000000..39c2246ac --- /dev/null +++ b/docs/gui-testing/scenarios/identity-key-recovery-migration.md @@ -0,0 +1,147 @@ +# Scenario: Identity key recovery, migration, and key-placement consistency + +**Verifies:** A private key is found, signed with, restored, and removed +consistently regardless of which internal convention filed it; a legacy +(pre-1.0) install's stranded keys can be recovered without re-entering +secrets; the Key Info / "Manage keys" screen is reachable from the standard +identity view and from a masternode's own detail page, and the two never +disagree about a key's role or held-state. (Risk area: #941, #945, #946, +#948 — key-placement-resolution rework, issue #889.) + +**Tier justification:** Needs a real, previously-migrated v0.9.3-shaped +identity/masternode fixture, real password-prompt cancel timing, and +cross-screen navigation (masternode detail page ↔ Key Info ↔ identities keys +list) that a no-display harness can't drive end-to-end. + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here) — do not skip or alter +steps between runs. Record what actually happens on each build; do not +presuppose which behaviors are "the fix." + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet + - `E2E_MN_PROTX_HASH` — (optional) a testnet masternode ProTxHash this + wallet controls, for the masternode-side half of this scenario +- A password-protected standalone identity (create one via the identity's + key-protection settings if none exists) — needed for the password-cancel + step +- Ideally, an identity/masternode with data left over from a pre-1.0 + install (or one seeded to resemble that state) — if unavailable, note the + gap rather than fabricating one +- Verify exact widget names during execution — the labels below (e.g. + "Manage keys", "Restore") describe intent, not a confirmed literal string + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/identity-key-recovery.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Keys screen reachability.** Open an identity in the default (least + advanced) interface level. Look for a way to reach a screen listing the + identity's keys (e.g. under Settings) without switching to a more + advanced level and without starting a payment. Record whether it's + reachable, and from where. +2. **Masternode key screen.** Open a masternode's detail page and look for + an equivalent "manage keys" entry point. Record what it opens. +3. **Role-label consistency.** Pick a key visible on both the identity's own + keys list and (if applicable) the masternode detail page. Record the + role name shown in each place — are they the same wording? +4. **Held-but-unpublished key.** If a key is saved on this device but not + on any on-chain key list for the identity, look for where the app + surfaces it. Record whether it's visible/reachable anywhere, and what + options are offered (view, remove). +5. **Restore stranded keys.** On an identity/masternode with legacy data + holding keys not yet in the current store, look for a "restore" affordance + on both the masternode detail page and the identity's own keys screen. + Trigger it, restore one key, and record: does it now show as held? Does + the offer narrow or disappear once nothing is left? Run it again and + record whether it's a safe no-op. +6. **Password-protected restore, cancel path.** On the password-protected + identity, start a restore and cancel/dismiss the password prompt. + Record the state of the Restore control afterward — does it look ready + to try again, or does it look stuck/already-started? +7. **Password prompt necessity.** For a key held in the clear in one place + and password-protected in another (if your fixture has this shape), + use/sign with it and record whether a password prompt appears. +8. **Same-numbered-key collision.** If reachable in your fixture (a + masternode's own record and its linked voting identity sharing a key + number), enter/paste a private key for one of the two same-numbered keys. + Record what happens to *both* keys afterward — check each one's own page, + not just the confirmation banner. +9. **Key removal.** Remove a saved private key. Record: does the key's page + immediately reflect the removal, or does it require leaving and + returning? Can the removed key still be revealed/signed with afterward? +10. **Live update while a key's page is open.** With a key's page open, + trigger a restore or edit for the same identity from elsewhere (a + second window on the same data dir, or via Back/forward navigation), + then return to the open key's page and make an edit. Record whether the + other change survives. + +## Safety constraints specific to this scenario + +- Use a real WIF/private key only for a throwaway testnet + identity/masternode fixture — never a mainnet key, never a value pasted + into a scenario file, report, or commit. +- Step 8 can destroy a key's private half if something goes wrong — rehearse + first with a fixture that has a recovery path (mnemonic-derived), not a + hand-imported unique WIF with no backup. + +## Expected outcome / pass criteria + +Record the observed behavior for each build (baseline, then HEAD) at every +step above, then apply this rule: + +- **BLOCKING** only if HEAD behaves *worse than baseline* in the happy path + described above (e.g. a key that was findable/usable on baseline is no + longer findable/usable on HEAD; a restore that worked cleanly on baseline + now corrupts or loses a key on HEAD), or if any step reveals a **data-loss** + outcome (a key's private half becoming permanently unrecoverable) on + either build. +- **NOT blocking**: a timing-dependent race *with no data-loss consequence* + that only sometimes reproduces; a UI glitch with no functional consequence; + or an issue that reproduces identically on both builds (pre-existing — note + it, don't block on it). A race that strands or destroys a key's private half + stays blocking under the rule above, however rarely it reproduces. +- If a step's affordance exists on only one of the two selected builds (e.g. + the Keys screen or the restore offer is absent on one side), that is a + feature-presence difference, not a step failure on the build that lacks it + — record which build has it and compare the behavior only where both do. + +## Known gotchas + +- Verifying a restored voting key against its actual on-chain address is an + explicitly deferred follow-up (issue #942) on both builds where the + feature exists at all — a voting key on an unlinked separate voting + identity correctly being listed as "cannot be restored automatically" is + expected, not a defect. +- A voting key stored directly on the identity's own record (rather than a + separate voting identity) can, when saved/removed by hand, affect a + same-numbered voting key on a linked voting identity. Never treat that as + expected upfront on either build: if it leaves a key's private half + unrecoverable it is the data-loss outcome step 8 exists to catch, and stays + **BLOCKING** regardless of which build shows it. Compare the observed + behavior across the two selected build SHAs, and record any confirmed + exception — with the evidence for it — in that campaign's own artifacts. In + every case, re-open the keys list after such a save/removal and check every + key still reads as expected. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/platform-shielded-availability-after-sync.md b/docs/gui-testing/scenarios/platform-shielded-availability-after-sync.md new file mode 100644 index 000000000..795d474e8 --- /dev/null +++ b/docs/gui-testing/scenarios/platform-shielded-availability-after-sync.md @@ -0,0 +1,99 @@ +# Scenario: Shielded feature availability right after platform sync + +**Verifies:** The app detects the connected network's protocol version +successfully after SPV/platform sync completes, so shielded send/receive/ +shield/unshield become available as soon as the network supports them, +without a spurious generic error banner as a side effect of that detection. +(Risk area: #936, #938.) + +**Tier justification:** This is specifically about a real live-network +proof-verification call succeeding or failing against the actual connected +testnet — it cannot be simulated without a real SPV sync and a real +protocol-version-detection round trip. + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here). + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet +- Verify exact banner/error wording during execution rather than assuming + the text below is literal + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/shielded-availability.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Cold start.** Launch the app fresh (isolated data dir, no prior sync + state) and wait for SPV sync to complete. Record whether any generic + error banner (e.g. "An unexpected error occurred") appears purely as a + side effect of sync completing. +2. **Shielded action availability.** Once sync completes, record whether + Shield, Unshield, Send Privately (shielded), and shielded Receive are + reachable/enabled. +3. **Send-fee estimate.** Open a Send flow and record whether a fee + estimate is shown, and whether it appears to update or stays fixed at a + default value. +4. **Network interruption during detection.** If you can simulate a brief + network outage right at startup, record whether the app keeps retrying + detection (shielded stays unavailable but the app doesn't misbehave) or + assumes a version without confirmation. +5. **Restart after successful detection.** Restart the app once shielded + features were confirmed available, and record whether they remain + available immediately on the next launch. + +## Safety constraints specific to this scenario + +- None beyond the standing rules. + +## Expected outcome / pass criteria + +Record the observed behavior for each build at every step, then apply this +rule: + +- **BLOCKING** only if HEAD is worse than baseline in the happy flow (e.g. + shielded features activate correctly after sync on baseline but stay + silently disabled on HEAD, or vice versa in a way that matters for this + release; a spurious error banner appears on HEAD where baseline was + clean), or a **data-loss** outcome (unlikely for this scenario, but note + if found) on either build. +- **NOT blocking**: a difference in exactly which internal call detects the + protocol version, as long as the user-visible outcome (shielded features + available when the network supports them, generic error banner absent) is + the same or better on HEAD; an issue reproducing identically on both + builds. +- If shielded features never activate after sync on one of the two selected + builds but do on the other, record that factually as an observed difference + between the builds — do not assume either direction was intended, and apply + the blocker rule to the user-visible outcome rather than to a presumed fix. + +## Known gotchas + +- The fee estimate may legitimately keep using a "last known rate" rather + than a freshly fetched one on HEAD — this is a documented temporary + limitation of the fix, not itself a defect, as long as it's not worse than + baseline's own fee-estimate behavior. +- If testing via MCP/CLI tools rather than the GUI, the same underlying + protocol-version-caching code path applies — a caching bug here previously + affected `mcp::resolve`-backed tools too. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent diff --git a/docs/gui-testing/scenarios/wallet-max-send-asset-lock.md b/docs/gui-testing/scenarios/wallet-max-send-asset-lock.md new file mode 100644 index 000000000..91a10e1d6 --- /dev/null +++ b/docs/gui-testing/scenarios/wallet-max-send-asset-lock.md @@ -0,0 +1,132 @@ +# Scenario: "Max" across asset-lock funding and Transfer/Withdraw flows + +**Verifies:** "Max" (and its dispatch-time validation) reflects what the +wallet can actually build/send, across every flow that offers a Max button: +Shield DASH, Fund Platform Address (Simple builder-driven form), Send +directly to an identity, Create Identity / Top Up Identity from wallet +balance or a received deposit, and identity Transfer/Withdraw. Must include +a near-empty ("dust") wallet variant and a rapid-UTXO-composition-change +variant specifically — these are real edge cases in this diff and are not +reliably described by the CHANGELOG text. (Risk area: #937, #927.) + +**Tier justification:** Needs a live, synced SPV wallet with real UTXOs +(confirmed, unconfirmed, dust-only) and a real broadcast to prove Max +actually sends on the first try — the ceiling query and the fee-reserve +estimator both depend on genuine wallet/network state a no-display harness +can't fake. + +**Run against BOTH builds with this identical procedure** — the baseline and +development binaries selected for the current campaign (record their exact +SHAs in that campaign's own artifacts, not here). Describe what you observe +on each build without assuming which one is "correct." + +Before any step that spends funds, consumes a deposit, or registers a +name, each build needs its own independently-funded equivalent fixture (or +a restored snapshot of the same starting state) — see [A/B build comparison +contract](../README.md#ab-build-comparison-contract). + +## Prerequisites + +- Network: testnet +- Environment variables (names only): + - `E2E_WALLET_MNEMONIC` — funded testnet wallet +- A wallet with an unusually large number of small UTXOs, if available + (otherwise send several small amounts to the test wallet first) +- A second, separate near-empty wallet (drained close to the network dust + threshold) for the dust-wallet variant +- A registered, funded identity (for the Transfer/Withdraw half) +- Verify exact widget/button labels during execution before relying on the + names used below + +## Setup + +```bash +DATADIR=$(mktemp -d) +cp .env.example "$DATADIR/.env" +pgrep -af dash-evo-tool + +BIN= +test -x "$BIN" +LOG="$DATADIR/wallet-max-send.log" +DASH_EVO_DATA_DIR="$DATADIR" nohup "$BIN" >"$LOG" 2>&1 & +``` + +## Procedure + +1. **Checking → Max → send, each asset-lock flow.** For each of: Shield + DASH, Fund Platform Address (Simple form), Send to identity, Create + Identity (fund from wallet balance), Top Up Identity (fund from wallet + balance) — open the flow, note whether/how the amount field indicates a + check is in progress, press Max, then send. Record whether the send + succeeds on the first attempt. +2. **Received-deposit ceiling.** On Create Identity or Top Up Identity, use + "fund from a received deposit." Record what ceiling Max offers relative + to what actually arrived at that specific deposit address versus the + wallet's total balance elsewhere. +3. **Advanced manual-input path.** On the Advanced manual-input + Platform-address flow, record what Max/validation is governed by (the + Core inputs selected, or something else). +4. **Stale-quote behavior.** Start a Max check on any flow above, then — + before sending — change the wallet's spendable funds from elsewhere + (send some of the same wallet's funds out, or receive a new deposit). + Record what the amount field does, and what the eventual send validates + against. +5. **Check-failure UX.** If you can force the availability check to fail + (e.g. briefly interrupt network during the check), record what the + amount field shows and whether a retry path is offered, and whether you + can still switch to a different funding method. +6. **Dust wallet.** Using the near-empty/dust wallet, open any flow above + and press Max. Record the result and whether it settles or stays + perpetually "checking." +7. **Rapid UTXO churn.** With a wallet whose balance is actively changing + (mid-sync, or receiving several small deposits in quick succession), open + a funding flow and watch the amount field for at least 30 seconds. Record + how many times it re-checks automatically and whether it settles or + keeps re-dispatching. +8. **Identity Transfer, Max.** Open Transfer credits between identities, + press Max, confirm and send. Record success/failure and the reserved fee + amount shown. +9. **Identity Withdraw, Max.** Open Withdraw to Core address, press Max, + confirm and send. Same recording. +10. **Reserve accuracy.** After steps 8–9, compare the amount actually + deducted (visible in the transaction/activity detail) to what Max + reserved. Record whether they're a close, sane match or wildly + different. + +## Safety constraints specific to this scenario + +- Cap every real send to the smallest amount that proves the behavior. +- Step 7 needs to run long enough (≥30s) to distinguish "one legitimate + automatic retry" from "runs away re-dispatching" — don't conclude early. + +## Expected outcome / pass criteria + +Record the observed behavior for each build at every step, then apply this +rule: + +- **BLOCKING** only if HEAD is worse than baseline in the happy flow (e.g. + Max sends successfully on baseline but fails/rejects on HEAD for the same + wallet state; HEAD's dust-wallet or rapid-churn case gets permanently + stuck where baseline at least settled; HEAD's fee reserve is wildly less + accurate than baseline's), or if a **data-loss** outcome appears on either + build. +- **NOT blocking**: a check that legitimately runs longer on one build than + the other without ever getting stuck; a UI wording difference with no + functional consequence; an issue reproducing identically on both builds. +- If baseline doesn't have a Max button on a flow at all where HEAD does (or + vice versa), record that as a feature-presence difference, not a + pass/fail on that step. + +## Known gotchas + +- The probe can hold the wallet-manager write lock for up to ~5 seconds per + call in a pathological case on HEAD — a real send attempted at the exact + same moment may appear to briefly stall rather than fail; wait a few + seconds before treating that as a hang. +- Issue #909 / `rust-dashcore#911` (Send Core→Core Max fails with + `InsufficientFunds` when change would be zero) is a distinct, real, + already-tracked, **still-open** upstream bug unrelated to this scenario's + asset-lock/Transfer/Withdraw Max — if a plain Core-to-Core "Max send" + reproduces it on both builds, that's pre-existing, not new. + +🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent