Skip to content

fix(drive): bump grovedb so a ranking over an unwritten pinned prefix proves an empty page - #4753

Merged
QuantumExplorer merged 1 commit into
v4.2-devfrom
claude/query-storage-path-error-14c408
Sep 14, 2026
Merged

QuantumExplorer merged 1 commit into
v4.2-devfrom
claude/query-storage-path-error-14c408

Conversation

@QuantumExplorer

@QuantumExplorer QuantumExplorer commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

A ranked (top-K) or having-range document query pinned to a prefix value tree that no document has written yet failed on both the unproved read and the prover, and reached clients as an internal error. The reported case is the Yappr windowed leaderboard on moutai:

{ "documentTypeName": "like", "groupBy": "postId", "aggregate": { "type": "count" },
  "direction": "desc", "limit": 20,
  "timeRange": [{ "field": "$createdAt", "selector": "newest", "grid": { "range": 86400, "step": 86400 } }] }
grpc error: code: 'Internal error', message: "query: storage: grovedb: invalid path: a single-path axis read must produce exactly one axis descent at the queried path, but the walk produced 0 - the path does not name an indexed tree carrying that axis"

IN_TIME_RANGE newest on a range == step grid resolves to the current bucket's start, so the query descends .../like/$createdAt#86400#86400/<bucket start>/postId. That bucket's value tree only comes into existence with the first document written under it (contract registration creates the level tree only, and TTL drops buckets again), so until the day's first like there was no indexed tree at the path. The same failure hit any == pin on a never-written value (a hashtag nobody has used) on both the ranked and the having-range surface.

The refusal was GroveDB's. Key selection and the branched axis shape already answer an absent path with an authenticated empty result; the single-path axis read was the one provable shape that did not (its prover refused to generate, its reader failed with a storage error). That is fixed by dashpay/grovedb#965 (merged), and this PR takes it.

What was done?

  • GroveDB pin bumped from 6816457d to a579d52f, the merge of fix: answer a single-path axis read over an absent path with the empty result grovedb#965 on develop. The delta also carries GroveDB's 6.0.1 crate version bump and CI chores; nothing else.
  • The GroveDB git pins now live once in the root [workspace.dependencies], like the rust-dashcore ones, and the six consuming crates take them with workspace = true while keeping their own optional and features. default-features = false for grovedb moves to the workspace declaration, where Cargo actually honours it (a crate-level setting on a workspace dependency is ignored). The next bump is a one-line change; the lock file resolves identically.
  • With that pin the reader, the prover and the verifier all answer a single-path axis read over an absent path with the traversal's empty page (skipped: 0), the absence authenticated by the layers the proof walk emits. Drive's query shapes are unchanged, so the OFFSET > 0 page over an unwritten prefix is covered too (the single-path shape keeps its skip attestation). A path that exists but does not lead to an indexed tree is still an error.
  • Executor docs (execute_top_k.rs, execute_range.rs) and the book (document-ranked-trees.md, time-range-ttl.md) describe the new absence rule.

An earlier revision of this PR worked around the refusal on the Drive side by serving single-branch pinned reads through GroveDB's branched envelope with one key; that is gone in favour of the GroveDB fix.

How Has This Been Tested?

New tests:

  • ranked pinned_prefix::an_absent_equality_pin_reads_empty_and_proves_empty: a never-written == pin reads empty, proves empty and verifies against the live root hash; a written prefix still returns its entry with no in_key; an offset page over the unwritten prefix reads and verifies empty at skip 0.
  • having pinned_prefix::unknown_prefix_value_reads_empty_and_proves_empty (flipped from the test that pinned the old "error rather than empty page" rule).
  • time_range_index_e2e_tests::an_unwritten_window_ranks_empty_and_proves_it: a ranked windowed index whose current window holds no documents ranks empty with a verified proof, reads under a caller transaction, and serves the leaderboard once a document lands in the window.
  • drive-abci an_unwritten_window_ranks_empty_through_the_entry_point: a BY_START window no post falls into verifies as an empty page at rank 0 through the SDK entry point.

Ran against the merged pin: cargo test -p drive --lib -- drive_document_ranked_query drive_document_having_query time_range_index_e2e_tests (135 passed), cargo test -p drive-abci --lib -- ranked having time_range (57 passed), cargo clippy -p drive --all-targets -- -D warnings, cargo check -p drive --no-default-features --features verify, cargo check of the five other crates that consume GroveDB through the workspace (Cargo.lock unchanged by the move, grovedb default features confirmed off via cargo tree -e features), cargo fmt --check --all. With the previous pin the three new Drive tests fail with the reported errors.

Breaking Changes

None.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 215bddd9-7a36-4f70-9d4c-22c5f36623b4

📥 Commits

Reviewing files that changed from the base of the PR and between 4bb2b3e and 9a0a406.

📒 Files selected for processing (14)
  • book/src/drive/document-ranked-trees.md
  • book/src/drive/time-range-ttl.md
  • packages/rs-drive-abci/src/query/document_query/v1/tests.rs
  • packages/rs-drive/src/drive/document/insert/add_document_for_contract/time_range_index_e2e_tests.rs
  • packages/rs-drive/src/query/drive_document_having_query/execute_range.rs
  • packages/rs-drive/src/query/drive_document_having_query/mod.rs
  • packages/rs-drive/src/query/drive_document_having_query/tests.rs
  • packages/rs-drive/src/query/drive_document_ranked_query/branches.rs
  • packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs
  • packages/rs-drive/src/query/drive_document_ranked_query/mod.rs
  • packages/rs-drive/src/query/drive_document_ranked_query/path.rs
  • packages/rs-drive/src/query/drive_document_ranked_query/tests.rs
  • packages/rs-drive/src/verify/document_having/verify_having_range_proof/v0/mod.rs
  • packages/rs-drive/src/verify/document_ranked/verify_ranked_top_k_proof/v0/mod.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The query system now treats never-written pinned prefixes as authenticated empty pages at OFFSET 0. Ranked and having-range reads share branched envelopes, proof verification uses a common helper, and tests cover unwritten windows, transactions, later writes, and entry-point verification.

Changes

Authenticated empty pinned pages

Layer / File(s) Summary
Branched envelope construction
packages/rs-drive/src/query/drive_document_ranked_query/mod.rs, packages/rs-drive/src/query/drive_document_having_query/mod.rs, packages/rs-drive/src/query/drive_document_ranked_query/path.rs
Ranked and having queries now create branched envelopes for multi-branch requests and zero-offset single pinned requests. Nonzero offsets retain single-path handling.
Branched reads and proof helper
packages/rs-drive/src/query/drive_document_ranked_query/branches.rs
Branched reads consume prebuilt envelopes. Single-branch reads can use caller transactions. verify_branched_union verifies branch sets, page limits, entries, and the root hash.
Query execution and verification
packages/rs-drive/src/query/drive_document_ranked_query/execute_top_k.rs, packages/rs-drive/src/query/drive_document_having_query/execute_range.rs, packages/rs-drive/src/verify/...
Ranked and having-range execution routes pinned requests through the envelope. Ranked and having proof verification delegates to the shared branch verifier. Missing pinned branches produce empty pages.
Regression coverage and documentation
packages/rs-drive/.../tests.rs, packages/rs-drive-abci/.../tests.rs, book/src/drive/*.md
Tests cover unwritten prefixes, proof verification, caller transactions, later inserts, and entry-point behavior. Documentation describes empty drained windows and offset rules.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant QueryExecutor
  participant branched_envelope
  participant read_branched_union
  participant GroveDb
  participant verify_branched_union
  QueryExecutor->>branched_envelope: Build pinned branch envelope
  QueryExecutor->>read_branched_union: Execute branched PathQuery
  read_branched_union->>GroveDb: Read branch entries
  GroveDb-->>read_branched_union: Return entries or authenticated absence
  QueryExecutor->>verify_branched_union: Verify proof envelope
  verify_branched_union->>GroveDb: Verify PathQuery
  GroveDb-->>verify_branched_union: Return verified branches and root hash
Loading

Merge Risk: ⚪ Minimal · up to 9a0a4

The alleged multi-null query failure is not supported by GroveDB’s path semantics, so no concrete merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 97.14% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 35 functions across 12 files. (2 skipped: 2…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: updating Drive/GroveDB behavior so ranked queries over never-written pinned prefixes prove an empty page. It is specific and related to the pull request o…
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/query-storage-path-error-14c408

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

🕓 Queued for automated review — 24th in line, estimated start in ~2.7 h (commit ab3caab)
Estimated review time once started: ~15 min (two-phase automated review; median of recent runs).

  • Request priority review — tick this box and the review moves to the front of the queue.

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

📖 Book Preview built successfully.

Download the preview from the workflow artifacts.
To view locally: download the artifact, unzip, and open index.html.

Updated at 2026-09-14T22:50:27.519Z

@github-actions github-actions Bot added this to the v4.2.0 milestone Sep 14, 2026
@codecov

codecov Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.64%. Comparing base (4bb2b3e) to head (ab3caab).
⚠️ Report is 1 commits behind head on v4.2-dev.

Files with missing lines Patch % Lines
...query/drive_document_having_query/execute_range.rs 0.00% 3 Missing ⚠️
...query/drive_document_ranked_query/execute_top_k.rs 0.00% 3 Missing ⚠️

❌ Your patch status has failed because the patch coverage (0.00%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@             Coverage Diff              @@
##           v4.2-dev    #4753      +/-   ##
============================================
- Coverage     87.32%   79.64%   -7.69%     
============================================
  Files          2827     2827              
  Lines        371564   400800   +29236     
============================================
- Hits         324476   319222    -5254     
- Misses        47088    81578   +34490     
Components Coverage Δ
dpp 77.45% <ø> (-11.44%) ⬇️
drive 80.60% <0.00%> (-5.88%) ⬇️
drive-abci 81.54% <ø> (-7.00%) ⬇️
sdk ∅ <ø> (∅)
dapi-client ∅ <ø> (∅)
platform-version ∅ <ø> (∅)
platform-value 86.79% <ø> (-6.13%) ⬇️
platform-wallet ∅ <ø> (∅)
drive-proof-verifier 29.41% <ø> (-11.32%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@QuantumExplorer
QuantumExplorer force-pushed the claude/query-storage-path-error-14c408 branch from 9a0a406 to 32f68b1 Compare September 14, 2026 22:34
@QuantumExplorer QuantumExplorer changed the title fix(drive): rank an unwritten pinned prefix as a proved empty page instead of failing fix(drive): bump grovedb so a ranking over an unwritten pinned prefix proves an empty page Sep 14, 2026

@QuantumExplorer QuantumExplorer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed

… proves an empty page

A ranked (top-K) or having-range query pinned to a prefix value tree no
document has written yet failed on both the unproved read and the prover.
The reported case is a windowed leaderboard: `IN_TIME_RANGE newest` on a
`range == step` grid resolves to the current bucket's start, and that
bucket's value tree only comes into existence with the first document
written under it (contract registration creates only the level trees, and
TTL drops buckets again), so until the day's first like the single-path
axis read had no indexed tree to descend into. GroveDB refused to generate
the proof ("a single-path axis read must produce exactly one axis descent
... the path does not name an indexed tree carrying that axis") and every
such request surfaced to clients as an internal error. The same failure
hit any `==` pin on a never-written value (a hashtag nobody has used) on
both surfaces.

The refusal was GroveDB's: key selection and the branched axis shape
already answered an absent path with an authenticated empty result, and
the single-path axis read was the one provable shape that did not.
dashpay/grovedb#965 fixes it there, so this bumps the pin to its merge on
develop (6816457d -> a579d52f, which also carries the 6.0.1 crate
versions): the reader, the prover and the verifier now all answer an
absent path with the traversal's empty page, absence authenticated by the
layers the walk emits. Drive's query shapes are unchanged; only the
executor docs and the book describe the new semantics. Because the
single-path shape keeps its skip attestation, this also covers
`OFFSET > 0` over an unwritten prefix.

The GroveDB git pins now live once in `[workspace.dependencies]`, like the
rust-dashcore ones, and the six consuming crates take them with
`workspace = true` (keeping their own `optional` and `features`;
`default-features = false` for `grovedb` moves to the workspace
declaration, where Cargo actually honours it). The next bump is one line.

Tests: ranked and having pinned-prefix tests for a never-written `==` pin
(read, prove, verify against the live root hash, including an offset page
that verifies empty at skip 0); a time-range e2e test for an unwritten
window that later fills; and an ABCI entry-point test verifying an empty
window through the SDK. The having test that pinned the old "error rather
than empty page" rule is flipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@QuantumExplorer
QuantumExplorer force-pushed the claude/query-storage-path-error-14c408 branch from 32f68b1 to ab3caab Compare September 14, 2026 22:50

@QuantumExplorer QuantumExplorer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed

@QuantumExplorer
QuantumExplorer merged commit 585db3e into v4.2-dev Sep 14, 2026
17 of 18 checks passed
@QuantumExplorer
QuantumExplorer deleted the claude/query-storage-path-error-14c408 branch September 14, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants