Repository navigation
feat(platform)!: delete and erase lifecycle for keep-history documents - #4657
Merged
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Basic explanation
What this does: At protocol 15, a permitted delete removes a document from ordinary reads while retaining its history. A separately permitted erase removes that retained history in bounded chunks.
Value: Applications can keep revision history without making every document permanent. Historical queries distinguish active, deleted, erasing and absent documents.
Risks: This changes consensus validation, storage, proofs and fee accounting at protocol activation. Focused runtime, compatibility and build checks pass locally; full CI and validator-hardware capacity checks are separate verification layers.
Issue being fixed or feature implemented
Keep-history documents need an explicit lifecycle: removal from current reads must preserve revisions, while permanent removal of those revisions requires separate authorization. The lifecycle activates at protocol 15; protocols through 14 retain their existing behavior.
What was done?
Lifecycle and authorization
canBeEraseddefaults to false, requires both history and deletion, and cannot change on a contract update. New registration still refuses a keep-history type with a contested index; non-validating reads preserve legacy contracts. History types cannot be consumed through a document reference.[1]tree records the deletion. Multiple deletes share one lazy lifecycle-container insertion within a PV15 Drive batch.Reads and proofs
For a single-document lookup at time
T:T, with the existing history-time clamp; absent before the first revision.deleted_at; absent at or after it. The comparison uses the original, unclampedT, so a same-block delete wins.T, from the first erase chunk onward.T; no terminal receipt is kept.The PV15 temporal query authenticates the lifecycle record or its absence together with history. Whole-proof and subset verification share structural checks; typed verification also checks the returned document's id and revision. The raw serialized verifier keeps its public signature. Generation 0 remains available for cross-version execution proofs.
DriveDocumentQuerywith a block time remains refused for keep-history types.GetDocumentHistoryv1 reportsACTIVE,ABSENT,DELETEDandERASING, withremaining_revisions,deleted_at_ms,erasing_started_at_ms,erasing_from_time_msanderasing_from_revision. Pages continue to expose surviving revisions during erasure. Proved responses use one merged GroveDB proof; lifecycle fields are derived from authenticated state. Released history v0 remains unchanged.The Rust, WASM and JavaScript SDK surfaces expose erase and a separate current-lifecycle read. Erase confirmation authenticates affected state, rather than proving that this particular transition executed.
Fees and storage ownership
Protocol 15 version slots
The lifecycle selectors are documented in
v15.rs, note 41:try_from_schema4 /validate_update2add_document_for_contract_operations2 /delete_document_for_contract_operations1Some(0)Some(0)Some(0..=0)/Some(100)The PV15 tables are
CONTRACT_VERSIONS_V7,DPP_VALIDATION_VERSIONS_V6,STATE_TRANSITION_SERIALIZATION_VERSIONS_V4, document/state-transition method tables V5, verify table V4, ABCI validation V11 andSYSTEM_LIMITS_V5. Erase handlers, bounds and limits are absent in released tables.DocumentTypeV2gains the erasability flag; schema version 3 and compiled writer generation 0 are retained.StoredDeletionvariants 0/1 preserve the previous false/true bytes; variant 2 represents erasability and requires history.How Has This Been Tested?
Local macOS verification at
a9c0e1ae82:should_end_a_poll_of_the_most_contenders_a_contest_accepts); it was not executed. This includes Drive lifecycle 49, delete 28, client costs 23, temporal/single-document verification 33, history verification 14, history decoder 19, migration 10 and contested 83; ABCI erase 19, references 8, activation 3, genesis 5 and commit-reveal 59; DPP contested 105 and lifecycle parser 20; and all 3 committed-byte compatibility tests.cargo fmt --check --all.cargo clippy --workspace --all-targets --all-features --locked -- --no-deps -D warnings.cargo check -p wasm-sdk --target wasm32-unknown-unknown --locked; wasm-dpp was also built for wasm32 and used by the JavaScript tests.Follow-up verification at
d6347691f2: the compatibility-v1 suite passes all 18 cases; structure lint/conformance/export tests pass all 12 cases; export freshness, workspace formatting and DPP/Drive clippy pass. The generated structure adds only the lifecycle container and record nodes, preserving all 281 existing nodes and 34 layer shapes. The requested changed-crate non-shielded runs completed 17,968 selected cases using the CI filter and--no-fail-fast: 17,967 passed initially and one reference-registration case aborted with the local stack setting unset. That case passes on both the frozen base and this branch with CI’sRUST_MIN_STACK=4194304; no test or production stack-size code was changed. Exact group counts, skips and passing process-leak warnings are recorded in the follow-up report.Actual total delete-fee checks include:
These assert total estimates cover applied fees. Processing has wide slack; storage margins are tight.
Verification at
70dd05914a5c: 2,047 focused Rust cases passed, including all 45 new upstream Rust tests for schema patterns, contact requests, proposal descriptions, contest poll identity and wallet addresses. The compatibility keyword inventory, signed lifecycle reference matrix, lifecycle/delete/history and proposal/contest groups pass with CI’sRUST_MIN_STACK=4194304; 16,224 cases were skipped by the focused selection. One passing process-leak warning was recorded and the case passed cleanly in isolation. All 46 decision-proposal JavaScript tests, workspace formatting and exact workspace clippy pass. The JavaScript run uses a WASM dependency rebuilt from this source. Swift wrapper tests remain CI-only because the required SDK framework is absent locally.Local verification at
6edc4c051dc8:dpp,drive,drive-abci,dash-sdk,drive-proof-verifier,platform-versionand nativewasm-dpp2pass 16,521 non-shielded cases with the CI filter and--no-fail-fast(1,214 skipped). The complete CI shielded shared-process phase passes 1,029 cases, including all 418 ABCI cases; three existing fixtures are ignored. Both useRUST_MIN_STACK=4194304. All six converter/refund-dispatch regressions pass, and workspace fmt, exact workspace clippy and all 38 platform-version cases pass. The shielded characterization keeps all eighteen original hashes: exact base/head captures differ only by two inactive default-false erasure fields in contract-type Debug output, which the non-history fixture excludes. Actions, errors, charges, lowered operations and settled state retain their original pins.Final comment-only documentation commit
84ac1e73775calso passes workspace fmt and all 38 platform-version tests.CI-only verification: the full-workspace non-shielded and full-chain phases,
cargo test -p wasm-sdk --lib --locked -j 2, and workspace doctests. Those remaining phases are not established by these local runs. The network functional lifecycle test, representative validator-hardware activation rehearsal and maximum-block erase benchmarking are not established by these local checks. Client erase-refund modelling remains a separate follow-up.In-place changes to shipped generations
Erase is appended to existing document/batch enums with existing discriminants and batch wire formats preserved. Released protocols publish no Erase bounds and refuse it in basic structure before executable handlers. Existing-input branches remain unchanged. This follows the same per-kind admission pattern as index-only deletion and shielded-payment branches.
The shipped versioned code gaining exhaustive Erase handling is:
methods/v0,resolvers/v0,v0/v0_methods,v1/v0_methods, andvalidation/validate_basic_structure/v0.advanced_structure/v0,advanced_structure/v1,state/v0(includingaccepted_token_pool_nullifiers) andtransformer/v0(includingproperty_constraint_aggregates).verify_state_transition_was_executed_with_proof/v0: Erase is explicitly classified as affected state, not execution-proved.Unversioned exhaustive enum/accessor/dispatch updates accompany those arms. The complete appended-arm inventory is below.
Files with appended-kind handling
packages/rs-dpp/src/data_contract/document_type/action_fees/mod.rspackages/rs-dpp/src/state_transition/mod.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition_action_type.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/resolvers.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/token_transition.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/mod.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/v0/mod.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/resolvers/v0/mod.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v0/v0_methods.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v1/v0_methods.rspackages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/validation/validate_basic_structure/v0/mod.rspackages/rs-drive-abci/src/execution/validation/state_transition/common/validate_document_not_expired.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v0/mod.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v1/mod.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/accepted_token_pool_nullifiers.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/mod.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/mod.rspackages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/property_constraint_aggregates.rspackages/rs-drive/src/query/filter.rspackages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_transition.rspackages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_transition_action_type.rspackages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/mod.rspackages/rs-drive/src/verify/state_transition/verify_state_transition_was_executed_with_proof/v0/mod.rspackages/wasm-dpp/src/document/state_transition/batch_transition/document_transition/mod.rspackages/wasm-dpp2/src/enums/batch/batch_enum.rspackages/wasm-dpp2/src/state_transitions/batch/document_transition.rspackages/rs-platform-wallet/src/wallet/identity/network/state_transition_summary.rsOther shared-generation edits preserve released behavior:
canBeErasedflag. Released grammars refuse that flag; comparisons without it, including contract definitions, keep the original compatibility rules.DocumentTypeV2and compiled format v0 gain the flag using byte-compatible stored deletion variants; existing fixture bytes are pinned.update_keywords/v0carry fullBlockInfoand optional deleter context into version dispatch. Delete conversion 0 supplies no deleter, and generation 0 continues using the same timestamp and existing operations. Sponsorship sets the new deleter field, which only delete generation 1 reads.apply_drive_operations0/1 andconvert_drive_operations_to_grove_operations0/1 pass no accumulated sibling operations; their PV15 generation 2 passes them for lifecycle-container deduplication. Shared force-delete lowering forwards siblings only for history types, which released force-delete handlers refuse; non-history lowering remains unchanged.Breaking Changes
DocumentTypeV2gains an erasability field; exhaustive matches and direct struct construction may require updates. NoDocumentTypeV3or new batch wire format is introduced.DocumentOperationType::DeleteDocumentgainsdeleter_id, andEraseDocumentis added.DELETED,ERASINGand the lifecycle fields. Generated gRPC clients are updated. Erase confirmation remains an affected-state observation.Checklist:
structure.rs, regeneratedgrovedb-structure.json, and checked the structure viewer link posted on this pull requestFor repository code-owners and collaborators only
🤖 Generated with Claude Code
PR Hygiene ·
84ac1e7/self-reviewedjs-wasm-sdk— you own itdpp— you own itrs-drive-abci— you own itrs-drive— you own itrs-platform-wallet(packages/rs-platform-wallet/src/wallet/identity/network/state_transition_summary.rs) — HashEngineering or ZocoLini or llbartekll or romchornyirust-sdk— you own itWhen every merge requirement is met, the
PR Hygienecheck passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.