Skip to content

refactor(platform): read a consumed document through generation 0's lookup and test the PV14 consume path - #5874

Merged
shumkov merged 3 commits into
v5.1-devfrom
shumkov/v51-consume-lookup-review-fixes
Nov 12, 2026
Merged

shumkov merged 3 commits into
v5.1-devfrom
shumkov/v51-consume-lookup-review-fixes

Conversation

@shumkov

@shumkov shumkov commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Follow-up to #5360 (delete a consumed document from what its lookup read, PV15), fixing findings from its review.

  1. The storage flags a consuming lookup reads are never used. At protocol version 15 a refersTo lookup declaring consume read the document it found through query_documents_with_flags, in a near-copy of generation 0's lookup helper, only to carry ConsumedDocumentRead.storage_flags to the delete. No operation the delete builds carries those flags: when it is not estimating, the flags only reach size inputs that are ignored, and GroveDB refunds a removed element by the flags stored on it. So the flags plumbing was dead consensus surface, and the comments saying the delete "refunds by" them were wrong.
  2. Generation 0's consume path lost its PV14 coverage. Document reference validation 0 and create conversion 0 are still what protocol version 14 runs. perf(platform)!: delete a consumed document from what its lookup read (PV15) #5360 moved the deletion, refund and contested-create assertions to latest() (PV15, generation 1), so no test ran generation 0's contested arm (ForceDeleteDocument for each consumed document). A broken generation-0 delete loop would have kept CI green.
  3. Comments and docs. The notes copied into generation 1 said it was "generation 0, which every table selects" or "reached from protocol version 14 only". Generation 0 still said "every table selects", although PV15 now selects generation 1. The book and the v15 notes overstated the saving.

What was done?

  • Generation 1 reads through generation 0's lookup. validate_document_references_v1 calls fetch_document_through_lookup (query_documents) for every lookup and keeps only the Document it found: ConsumedDocument.read is now Option<Document>.
    • Removed: ConsumedDocumentRead, document_reference_validation/v1/lookup.rs (the 177-line copy), and the storage_flags parameter of Drive::force_delete_read_document_for_contract_operations. The read delete passes the document with no flags.
    • The dispatcher's doc comment gains its # Parameters and # Returns sections.
    • Comments that said the delete "refunds by" the flags are corrected.
    • The by-id read stays saved: conversion 1 still deletes from the document the lookup read.
  • PV14 coverage restored. These tests now run at protocol versions 14 and 15 (CONSUME_PROTOCOL_VERSIONS):
    • should_create_a_domain_revealing_its_writers_old_enough_commitment_and_consume_it
    • should_consume_a_commitment_its_owner_can_not_delete
    • should_consume_a_commitment_its_owner_can_not_delete_through_a_contested_create
    • should_create_a_ticket_that_consumes_an_earlier_ticket_of_its_writer (both tickets share an index bucket, so this covers generation 0's add-then-delete order)
  • New fee pin: should_pin_the_fees_of_a_consume_through_a_plain_and_a_contested_create, at 14 and 15.
    • The dpns_v3 tests stay at latest(). DPNS v3 exists from protocol version 15 only, and PV14 keeps DPNS v2 and its create trigger, so that flow cannot run at 14.
  • Text only:
    • The in-place notes copied into generation 1 are removed. Generation 1 is selected only by PV15, which has not shipped, so its code needs no in-place argument.
    • In generation 0 (reference validation 0, create conversion 0), "which every table selects" is dropped.
    • The book (refers-to-lookup.md) and the v15 note now say "one read fewer". A chooseFirst scan may read the commitment again, so a consume does not always pay for exactly one read.
    • The v15 note gives 10,660 credits as what perf(platform)!: delete a consumed document from what its lookup read (PV15) #5360's DPNS v3 pin dropped by, not as a constant, since the saving depends on where the commitment sits in its tree.
    • The validation v11 note no longer describes the flags read.
    • The PV14/15 parity tests' doc comments no longer describe the removed flags path.

How Has This Been Tested?

No fee or state change. A temporary test (not committed) ran seven consume flows and printed the fee result and the committed GroveDB root hash after each consume. I ran it twice on the base commit, which gave the same output both times, and then once after the change. The output was identical before and after:

Flow PV Processing Storage Refund to owner
plain create consumes a commitment 14 1,777,440 9,720,000 15,181,239
15 1,766,860 9,720,000 15,181,239
contested create consumes a commitment 14 3,489,420 75,411,000 15,181,239
15 3,478,880 75,411,000 15,181,239
ttl preorder, chooseFirst 14 3,574,720 25,299,000 0
15 3,562,700 25,299,000 0
ticket consumes a ticket in its own index bucket 14 2,147,980 27,135,000 22,488,973
15 2,137,540 27,135,000 22,488,973
voucher found by a plain lookup 14 1,670,640 9,153,000 15,181,239
15 1,660,060 9,153,000 15,181,239
DPNS v3 name, plain 15 3,739,640 42,687,000 0
DPNS v3 name, contested 15 4,564,780 81,729,000 0
  • The root hash after every row is also identical before and after.
  • The 14 → 15 drop is perf(platform)!: delete a consumed document from what its lookup read (PV15) #5360's saved read, which this PR leaves as it was. DPNS v3 exists from protocol version 15 only.
  • Three committed pins pass unchanged:
    • should_pin_the_fees_of_a_consume_through_a_plain_and_a_contested_create (new; plain and contested, at 14 and 15)
    • should_pin_the_fees_of_a_reveal_choosing_the_first_of_its_preorders (ttl preorder)
    • should_charge_a_name_at_protocol_versions_14_and_15 (DPNS v3)
  • In Drive, should_delete_a_read_document_as_a_force_delete_does_less_its_read now compares the by-id delete, which carries the stored element's flags, with a read delete given none. Their operations are equal.

Restored PV14 coverage, red → green. I removed the delete loop of generation 0's contested arm (document_create_transition.rs, conversion 0) on purpose:

  • ✖ should_consume_a_commitment_its_owner_can_not_delete_through_a_contested_create failed with "expected the commitment consumed at 14".
  • ✖ The new fee pin failed on its 14 row: refund 0 and processing 2,600,980, where the pin holds 15,181,239 and 3,489,420.
  • The 15 row did not change, which is why the latest()-only tests could not catch this.
  • ✔ Both pass again with the loop restored.

Local gate:

  • cargo test -p drive --lib: 4406 passed, 0 failed, 6 ignored.
  • RUST_MIN_STACK=4194304 cargo test -p drive-abci --lib: 4628 passed, 0 failed, 17 ignored.
  • cargo clippy -p drive -p drive-abci -p platform-version --all-features --all-targets -- -D warnings: clean.
  • cargo fmt --all -- --check: clean.

Breaking Changes

None. PV15 fees and state are unchanged (see above), and PV14 and earlier run no changed code. PV15 has not been released or activated on any network.

In-place changes to shipped generations

Tests and comments only:

  • Document reference validation 0 (DRIVE_ABCI_VALIDATION_VERSIONS_V1–V10, protocol versions ≤ 14): comment text only ("which every table selects" dropped). No code changed.
  • Create conversion 0 (DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V1–V4, protocol versions ≤ 14): comment text only. No code changed.
  • Drive::force_delete_read_document_for_contract_operations is called only by create conversion 1, which only PV15 selects. Its storage_flags parameter is removed, and generation 0 of every method is untouched.

Checklist:

  • I have performed a self-review of my own code
  • My Rust, FFI, WASM, Swift or Kotlin changes follow the coding conventions
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed
  • If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request
  • If I added or changed a system contract's document schemas, I checked the contract viewer link posted on this pull request

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · 64fe5df

  • Bots — thepastaclaw ✓
  • Self-review — post /self-reviewed
  • Build green
  • Approvals — you own every area touched; none needed

When every merge requirement is met, the PR Hygiene check passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.

Summary by CodeRabbit

  • New Features

    • From protocol version 15, consuming a document reference no longer requires an additional document read during deletion. This reduces the processing cost by one document read, with the resulting fee reduction depending on where the commitment is stored.
  • Documentation

    • Clarified that storage fees and refunds are unchanged, and that protocol versions 14 and earlier retain their existing behavior.

shumkov and others added 3 commits November 12, 2026 00:32
…5 and pin consume fees

Document reference validation 0 and the create conversion 0 are still
what protocol version 14 runs. #5360 moved the consume tests to
PlatformVersion::latest(), which now selects generation 1, so no test
ran generation 0's contested arm (a ForceDeleteDocument for each
consumed document). Breaking that arm would have left CI green.

The plain, onlyWhenConsumed, contested and shared-bucket (ticket)
consume tests now run at protocol versions 14 and 15.
should_pin_the_fees_of_a_consume_through_a_plain_and_a_contested_create
pins the processing fee, the storage fee and the commitment owner's
refund at both versions. The DPNS v3 consume tests stay at latest():
DPNS v3 exists only from protocol version 15.

Test would have caught a broken generation-0 contested arm. With the
arm's delete loop skipped:
- ✖ should_consume_a_commitment_its_owner_can_not_delete_through_a_contested_create
  fails with "expected the commitment consumed at 14".
- ✖ the fee pin fails on its 14 row (refund 0, processing 2_600_980
  where 3_489_420 is pinned).
- The 15 row is unchanged, which is why the latest()-only tests missed
  the break.
✔ Both pass once the arm is restored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ookup (PV15)

At protocol version 15, a refersTo lookup declaring consume read the
document it found through query_documents_with_flags, in a copy of
generation 0's lookup helper. The only reason was to carry
ConsumedDocumentRead.storage_flags to the delete, and no delete
operation carries those flags: GroveDB refunds a removed element by the
flags stored on it. The flags plumbing was dead consensus surface, and
the comments saying the delete refunds by those flags were wrong.

Document reference validation 1 now reads every lookup through
generation 0's fetch_document_through_lookup and keeps only the
document:
- ConsumedDocument.read is an Option<Document>.
- ConsumedDocumentRead and the copied v1/lookup.rs are removed.
- force_delete_read_document_for_contract_operations loses its
  storage_flags parameter, and its doc gains # Parameters and
  # Returns.
- The read that #5360 saved stays saved: conversion 1 still deletes
  from the document the lookup read, without reading it again by id.

No fee or state change at protocol version 15 or 14. Fees (processing,
storage, refund) and the GroveDB root hash after seven consume flows
were identical before and after this change, at both versions where a
flow runs:
- a plain create and a contested create
- a ttl preorder chosen first among its competitors
- a ticket sharing its index bucket with the one it consumes
- a voucher found by a plain lookup
- a DPNS v3 name, plain and contested (protocol version 15 only)
The fee pins pass unchanged:
- should_pin_the_fees_of_a_consume_through_a_plain_and_a_contested_create
- should_pin_the_fees_of_a_reveal_choosing_the_first_of_its_preorders
- should_charge_a_name_at_protocol_versions_14_and_15
should_delete_a_read_document_as_a_force_delete_does_less_its_read now
compares the by-id delete, which carries the stored element's flags,
with a read delete given none, and their operations are equal.

The v15 note now says a consume pays for one read fewer, a saving that
differs from create to create, rather than a fixed 10,660 credits.

Refs #5360

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Text only, no code change.

- Document reference validation 1 carried the replay-safety notes of
  the generation it was copied from: "in place in generation 0, which
  every table selects", "reached from protocol version 14 only".
  Generation 1 is selected only by protocol version 15, which has not
  shipped, so it needs no in-place argument. Those notes are removed.
- Generation 0 of reference validation and of the create conversion
  said "which every table selects", which stopped being true when
  protocol version 15 selected generation 1 of both. That phrase is
  dropped.
- The book said a consuming create pays for one read of the
  commitment. A chooseFirst scan can read it again, so the book now
  says one read fewer than before.

Refs #5360

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shumkov shumkov added this to the v5.1.0 milestone Nov 11, 2026
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Protocol version 15 reference validation retains the document found by a consuming lookup. Create conversion deletes that document without reading it again by ID. The consumed-document representation and deletion API no longer pass storage flags. Tests cover versions 14 and 15.

Changes: Consumed lookup deletion

Layer / File(s) Summary
Retain consumed lookup documents
packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v1/*, packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs
Validation fetches lookup results without storage flags and stores a consumed document directly. Comments clarify when generation-specific validation paths are reached.
Pass consumed documents to deletion
packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs, packages/rs-drive/src/util/batch/drive_op_batch/document.rs, packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/*, packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_create_transition.rs
ConsumedDocument.read holds a Document. The operation batch passes it to the already-read deletion path, which no longer accepts storage flags.
Verify version behavior and describe billing
packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/commit_reveal_lookup.rs, book/src/contract-keywords/refers-to-lookup.md, packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v11.rs, packages/rs-platform-version/src/version/v15.rs
Commit/reveal tests cover consumption, refunds, and fees at protocol versions 14 and 15. Documentation states that version 15 reuses the lookup read for deletion and reduces the read cost.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant ReferenceValidationV1
  participant ConsumedDocument
  participant CreateConversion
  participant DriveDeletion
  ReferenceValidationV1->>ConsumedDocument: Store the fetched Document
  ConsumedDocument->>CreateConversion: Provide the consumed Document
  CreateConversion->>DriveDeletion: Delete the already-read Document
Loading

Possibly related issues

  • dashpay/platform#4696: The tests compare fees and behavior across protocol versions 14 and 15, but the issue’s broader release-evidence requirements are not covered in full.

Possibly related PRs

  • dashpay/platform#5360: This PR revises the consumed-document read path introduced there by retaining only the Document and removing storage-flags plumbing.
  • dashpay/platform#5337: Both PRs change the handling of consumed refersTo lookup results in the DPNS commit-and-reveal flow.
  • dashpay/platform#4706: This PR updates already-read deletion to omit storage flags while retaining the stated refund behavior for removed elements.

Suggested reviewers: quantumexplorer, thepastaclaw

Merge Risk: Minimal · up to 64fe5

This change removes a redundant document read when a consume lookup feeds a create at protocol version 15, and it drops the unused storage-flags plumbing. Storage fees and refunds stay the same. Tests cover protocol versions 14 and 15. No merge-blocking risk was found.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly summarizes the main refactor: generation 0 lookup for consumed documents and PV14 consume-path testing.
Docstring Coverage Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 21 functions across 10 files. (1 skipped: 1 unsupported.)

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 7089480 and 64fe5df.

Files selected for processing: 12

  • book/src/contract-keywords/refers-to-lookup.md
  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v0/mod.rs
  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v1/lookup.rs
  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v1/mod.rs
  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/commit_reveal_lookup.rs
  • packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/mod.rs
  • packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/v0/mod.rs
  • packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_create_transition.rs
  • packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs
  • packages/rs-drive/src/util/batch/drive_op_batch/document.rs
  • packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v11.rs
  • packages/rs-platform-version/src/version/v15.rs
Files with no reviewable changes: 1

  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_reference_validation/v1/lookup.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Nov 11, 2026
@thepastaclaw

thepastaclaw commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Final review complete — no blockers (commit 64fe5df) · triage: normal · Phase 2 only (queue backlog)

@github-actions

Copy link
Copy Markdown
Contributor

📖 Book Preview built successfully.

Download the preview from the workflow artifacts.
To view locally: download the artifact, unzip, and open index.html.

Updated at 2026-11-11T17:44:40.885Z

@shumkov

shumkov commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (queue backlog)

Static verification of 7089480..64fe5df found no actionable in-scope issues. The shared lookup preserves validation and billing, stored-element flags remain authoritative for refunds, and the expanded tests cover PV14/PV15 consumption and fee behavior without changing historical execution. No builds or tests were run locally; current CI reports all completed code checks passing, including Rust workspace, wasm32, JavaScript, Swift, and Kotlin checks, with only review/hygiene checks pending.

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); reviewer 5: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 6: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 7: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 8: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: normal by gpt-6.1-sol (effort low) — The diff primarily simplifies the PV15 consume lookup and deletion plumbing and restores PV14 test coverage; despite its size and consensus-adjacent location, it does not clearly introduce a large or intricate change to consensus rules or storage behavior.
  • Phase 1 reviewers: not run (skipped for throughput: 13 PRs queued, above the 10 limit)
  • Second Phase-2 pass: on this first full review every Phase-2 reviewer also ran a second, independent time, and the verifier weighed both sets
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — general (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort high); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort high); agent phase2-reviewer

@github-actions

Copy link
Copy Markdown
Contributor

Bots are done — your move: post /self-reviewed.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed and removed waiting-bots Waiting for the review bots to report on this head labels Nov 11, 2026
@shumkov
shumkov merged commit c2029bc into v5.1-dev Nov 12, 2026
33 of 34 checks passed
@shumkov
shumkov deleted the shumkov/v51-consume-lookup-review-fixes branch November 12, 2026 03:44
QuantumExplorer added a commit that referenced this pull request Nov 12, 2026
…the decision proposals contract

`should_pin_the_fees_of_a_consume_through_a_plain_and_a_contested_create` came
in with v5.1-dev (#5874) and pins the fee of a contested create at protocol
version 15. On this branch the decision proposals contract, registered at
genesis from 15, has a contested index of its own (`choice`), so the tree of
contracts holding contests has one entry more and a write below it pays for
the larger tree: 3_528_980 rather than 3_478_880. The plain create, the
storage fee, the refund and every protocol version 14 figure are unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-self-review Waiting for the author to post /self-reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants