Skip to content

fix(batch)!: commit same-batch sum-item references and replacements to the bytes stored (GROVE_V4) - #1021

Merged
QuantumExplorer merged 4 commits into
developfrom
fix/grove-v4-flags-update-fixes
Oct 13, 2026
Merged

QuantumExplorer merged 4 commits into
developfrom
fix/grove-v4-flags-update-fixes

Conversation

@QuantumExplorer

@QuantumExplorer QuantumExplorer commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Basic explanation

What this does: It fixes two existing bugs in how a batch accounts for a write over a value that is already stored. Both fixes apply from GROVE_V4 only; GROVE_V1 to GROVE_V3 behave exactly as before.

  1. A reference to a sum item could commit to bytes that are never stored. When one batch updates an item and also writes or refreshes a reference to it, the batch predicts the item's new bytes so the reference can commit to their hash. That prediction was a hand-written copy of Merk's update, and it assumed a sum item (SumItem, ItemWithSumItem) keeps its stored flags. The apply actually stores the flags the flags callback merges. So whenever the merge changes them, the reference commits to a hash of bytes that never land, and verify_grovedb reports it.
  2. A replacement whose flags callback answers "unchanged" could fail with a cost mismatch. Merk measures a replacement with the new value carrying the old value's flags, and kept that measurement when the callback answered "unchanged". When the new value's flags have a different length, the commit failed with storage_cost cost mismatch.

Value: Dash Platform writes the first batch shape for every document of a summable type (protocol version 14). When such a document is updated in a later epoch and changes size, its index reference is refreshed in the same batch as its primary-storage ItemWithSumItem. Without the fix, that reference commits to the wrong hash, so proofs of reads through that index entry fail to verify. Landing the fix in GROVE_V4, before Platform's protocol version 14 activates, keeps such entries out of state from the start.

Risks: Low to medium. Only GROVE_V4 changes behaviour. The flags callback still answers bool, and BatchApplyOptions is unchanged. Two edits touch code that GROVE_V1 to GROVE_V3 also run; both are refactors with identical output (see "In-place changes to shipped generations"). Some Merk put functions now take a &GroveVersion (see "Breaking Changes").

These fixes were found during review of #1001 and are split out of it so they can ship without the opt-in settlement of storage owner changes, which is now planned for a later Platform release. #1001 will be rebased on this PR.

Issue being fixed or feature implemented

1. Stale same-batch reference to a sum item

A batch that updates an item and also writes or refreshes a reference to it must commit the reference to the item's final bytes, flags update included. The old prediction got this right for plain items but assumed a sum item keeps its stored flags. Two cases make the merged flags differ:

  • an owned item updated in a later epoch, when the update changes its size (the flags become multi-epoch);
  • an unowned item gaining an owner.
Item at [doc]/id: ItemWithSumItem(20 value bytes, sum 5), flags SingleEpochOwned(0, A).
Index reference at [index]/.../0 -> [doc]/id.
In epoch 2, one batch writes the item with 25 value bytes and refreshes the reference.

Before (GROVE_V4 today): the item stores MultiEpochOwned(0, {2: 5}, A); the reference commits to
                         the hash of the item carrying SingleEpochOwned(0, A). verify_grovedb
                         reports the reference; a proof through it does not verify.
After:                   the reference commits to the hash of the bytes the item stores.

The old prediction also refused a sum item replaced by an item (or the reverse) under a same-batch reference, a write that applies on its own. From GROVE_V4 it is accepted.

2. Cost mismatch on an "unchanged" flags answer

Item with flags F1 (n bytes) replaced in place with flags F2 of another length; the callback
answers "unchanged" (keep the value as written).

Before (GROVE_V4 today): the cost is measured with the new value carrying F1, the node stores F2,
                         and the commit fails with "storage_cost cost mismatch".
After:                   the replacement is measured again from the bytes it stores and charged
                         exactly for them.

What was done?

  • New version slots, 0 in GROVE_V1 to GROVE_V3 and 1 in GROVE_V4:
    • apply_batch.same_batch_reference_target_prediction (GroveDBApplyBatchVersions);
    • merk_versions.tree.just_in_time_value_update (MerkTreeVersions).
  • Same-batch reference prediction (grovedb/src/batch/just_in_time_reference_update/): a mod.rs dispatcher, v0.rs with the previous body unchanged, and v1.rs. Version 1:
    • hashes the stored bytes when the pass has already written the target;
    • otherwise runs Merk's own just-in-time value update with the put the apply performs (TreeNode::predict_put_final_value), once per target;
    • takes the bytes as given when the apply's write will not run that update: a node a partial batch's initial segment inserted is still held by the Merk with no value to replace (Merk::replacement_runs_value_update, which reads only the nodes in memory);
    • drops a target's prediction once the target is written, after checking that the write stored exactly the predicted bytes. A batch whose write stores anything else is refused (JustInTimeElementFlagsClientError), as it already is for a backward-references item. The check reads the node the pass just wrote and is not charged, so it changes no cost.
  • Merk just-in-time value update (merk/src/tree/just_in_time_value_update/): a mod.rs dispatcher, v0.rs with the previous body unchanged, and v1.rs. Version 1:
    • re-measures a replacement from the bytes it stores when the callback answers "unchanged" and the size differs from the last measurement;
    • restores the put's value-defined cost whenever it restores the put's value;
    • measures the stored value's own cost once rather than on every round, which changes no figure.
  • Merk put functions that run the update take a &GroveVersion to select it.

How Has This Been Tested?

New grovedb/src/batch/same_batch_reference_prediction_tests.rs:

  • a_reference_rewritten_in_the_same_batch_commits_to_the_stored_bytes: the batch Platform writes. It covers an ItemWithSumItem grown by 1, 3 and 25 bytes in a later epoch, a sum item gaining an owner, and plain items, under seven ways of writing or refreshing the reference, and checks that verify_grovedb is clean.
  • grove_v3_keeps_the_legacy_sum_item_prediction: pins the old stale references on GROVE_V3.
  • a_sum_item_and_an_item_replace_each_other_under_a_same_batch_reference: accepted on V4, refused on V3.
  • a_reference_to_a_target_written_earlier_in_the_batch_commits_to_its_stored_bytes
  • an_add_on_reference_commits_to_the_bytes_the_initial_segment_stored
  • an_add_on_rewriting_an_initial_segment_target_predicts_its_new_bytes
  • a_drive_shaped_reference_commits_to_the_stored_bytes: a flagged ReferenceWithSumItem in a sum tree, refreshed (trusted or not) or rewritten, to an ItemWithSumItem under a SumTree, CountSumTree, ProvableSumTree or ProvableCountProvableSumTree, grown in a later epoch or holding two epochs and shrunk in a later one, plus a NonCounted target in a CountSumTree.
  • a_proof_through_a_same_batch_reference_verifies: a proof read through the refreshed reference verifies against the root hash on V4 and does not on V3.
  • an_add_on_rewriting_an_item_the_initial_segment_inserted_commits_to_its_bytes: a partial batch whose add-on rewrites an item its initial segment inserted, with a deeper reference to it.
  • a_reference_predicted_under_merk_put_value_0_commits_to_the_stored_bytes: a hand-built version pairing prediction v1 with Merk put_value 0.
  • a_replacement_keeping_flags_of_another_length_is_charged_for_its_bytes: V4, exact charge.
  • grove_v3_refuses_a_replacement_keeping_flags_of_another_length: V3 cost mismatch, pinned.

Merk (merk/src/tree/just_in_time_value_update/mod.rs):

  • predicted_provided_value_hash_put_matches_the_apply (the existing test, renamed)
  • a_changed_round_then_unchanged_measures_the_value_it_stores

Locally: cargo fmt --all -- --check clean; cargo check -p grovedb-version and cargo check -p grovedb-merk --tests pass with no warnings. cargo test -p grovedb-merk --lib: 802 passed. cargo test -p grovedb --lib filtered to same_batch_reference_prediction, backward_references, just_in_time and reference: 432 passed; after the last review fix, the 13 tests in same_batch_reference_prediction_tests.rs pass, and the reference, backward-references and partial-batch tests (525) pass. cargo clippy -p grovedb -p grovedb-merk --all-targets is clean. CI covers the full suite.

In-place changes to shipped generations

Two edits touch code that GROVE_V1 to GROVE_V3 also run. Neither changes what those versions store, charge or return:

  • merk/src/tree/put_value/v0.rs: the line that installs an ordinary value moves into install_ordinary_value_v0 (the same kv.put_value_no_update_of_hashes(value) call), and the function passes the new grove_version argument through to the just-in-time value update dispatcher, which selects the unchanged v0 body on those versions.
  • merk/src/element/insert.rs: the inline "sum item takes its specialized cost, anything else an Op::Put" choice, repeated in two places, becomes the shared specialized_put_cost, with the same condition, cost and error.
  • grovedb/src/batch/mod.rs, the arm of follow_reference_get_value_hash for a reference to a pending flagged item write: it now reads the target's stored bytes first and serializes the new element after, in the version-0 arm of process_old_element_flags, which also decodes the stored element there. Successful batches are unchanged, since serialization is not metered. Only a batch whose new element fails to serialize, which a valid element cannot, would now report the cost of the read with the error.

Breaking Changes

  • GroveDBApplyBatchVersions gains same_batch_reference_target_prediction and MerkTreeVersions gains just_in_time_value_update. Code that builds either struct by hand must set them.
  • TreeNode::put_value_with_fixed_cost, TreeNode::put_value_with_reference_value_hash_and_value_cost, TreeNode::put_value_with_two_reference_value_hashes_and_value_cost and their Walker counterparts take a trailing &GroveVersion.
  • TreeNode::provided_value_hash_put_final_value is renamed TreeNode::predict_put_final_value. It also takes the stored node's value-defined cost, the put it predicts (new grovedb_merk::tree::PredictedPut) and a &GroveVersion.
  • GROVE_V4 behaviour: a same-batch reference to a sum item now commits to the stored bytes, and the replacements above are now accepted. Dash Platform uses none of the renamed or re-signed functions, so adopting this needs only a rev bump; the fixes then apply from its protocol version 14.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · 160c61d

  • Bots — coderabbitai not yet · thepastaclaw not yet — /skip-bots proceeds without the ones not yet reported
  • Self-review — post /self-reviewed
  • Build running
  • Approvals — you own every area touched; none needed

When every merge requirement is met, the PR Hygiene check passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.

Summary by CodeRabbit

  • Bug Fixes
    • Same-batch references now use the target’s final stored value, including when that target has already been written or replaced.
    • Replacement storage costs are now calculated using the bytes ultimately stored, including when flags remain unchanged.
  • Compatibility
    • The TreeNode and Walker value-update APIs now require version information, and the final-value prediction method has been renamed. Earlier Grove versions retain their previous behavior.

…o the bytes stored (GROVE_V4)

Two existing bugs in how a batch accounts for a write over a stored value,
both behind new GROVE_V4 version slots (0 in V1..V3, 1 in V4):

- apply_batch.same_batch_reference_target_prediction: a reference written
  or refreshed in a batch that also updates its target item predicts the
  bytes the update stores. Version 0 (the previous body, frozen) assumed a
  sum item keeps its stored flags, while the apply stores the flags the
  flags update merges, so an owned item updated in a later epoch or an
  unowned one gaining an owner left the reference committed to bytes that
  are never stored. This is the batch Drive writes for a document of a
  summable type. Version 1 hashes the bytes stored for a target the pass
  has already written, and otherwise runs Merk's own just-in-time value
  update with the put the apply performs, once per target. The sum item
  to item refusal goes with it.
- merk_versions.tree.just_in_time_value_update: a replacement whose flags
  update leaves the flags unchanged kept the measurement taken with the old
  value's flags, so new flags of another length failed the commit with a
  storage cost mismatch. Version 1 measures it again from the bytes it
  stores and restores the put's value-defined cost whenever it restores
  the put's value. Version 0 is the previous body, frozen.

Merk's put functions that run the update take a &GroveVersion, and
TreeNode::provided_value_hash_put_final_value becomes
predict_put_final_value, which predicts any put kind.

BREAKING CHANGE: GroveDBApplyBatchVersions and MerkTreeVersions gain public
fields; TreeNode::put_value_with_fixed_cost, the two layered put functions
and their Walker counterparts take a &GroveVersion;
provided_value_hash_put_final_value is renamed predict_put_final_value with
new arguments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 27 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ac85a501-0c43-4912-9a95-d970cc8e40c8
📥 Commits

Reviewing files that changed from the base of the PR and between 33bf989 and 160c61d.

📒 Files selected for processing (8)
  • CHANGELOG.md
  • grovedb/src/batch/just_in_time_reference_update/mod.rs
  • grovedb/src/batch/just_in_time_reference_update/v1.rs
  • grovedb/src/batch/just_in_time_value_update.rs
  • grovedb/src/batch/mod.rs
  • grovedb/src/batch/same_batch_reference_prediction_tests.rs
  • merk/src/merk/get.rs
  • merk/src/tree/put_value/mod.rs

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a542423c-8216-4e41-a8b7-f31acd6fdef1
📥 Commits

Reviewing files that changed from the base of the PR and between 5cb0962 and 33bf989.

📒 Files selected for processing (25)
  • CHANGELOG.md
  • grovedb-version/src/version/grovedb_versions.rs
  • grovedb-version/src/version/merk_versions.rs
  • grovedb-version/src/version/v1.rs
  • grovedb-version/src/version/v2.rs
  • grovedb-version/src/version/v3.rs
  • grovedb-version/src/version/v4.rs
  • grovedb/src/batch/just_in_time_reference_update.rs
  • grovedb/src/batch/just_in_time_reference_update/mod.rs
  • grovedb/src/batch/just_in_time_reference_update/v0.rs
  • grovedb/src/batch/just_in_time_reference_update/v1.rs
  • grovedb/src/batch/just_in_time_value_update.rs
  • grovedb/src/batch/mod.rs
  • grovedb/src/batch/same_batch_reference_prediction_tests.rs
  • merk/src/element/insert.rs
  • merk/src/tree/just_in_time_value_update.rs
  • merk/src/tree/just_in_time_value_update/mod.rs
  • merk/src/tree/just_in_time_value_update/v0.rs
  • merk/src/tree/just_in_time_value_update/v1.rs
  • merk/src/tree/mod.rs
  • merk/src/tree/ops.rs
  • merk/src/tree/put_value/mod.rs
  • merk/src/tree/put_value/v0.rs
  • merk/src/tree/put_value/v1.rs
  • merk/src/tree/walk/mod.rs
💤 Files with no reviewable changes (2)
  • merk/src/tree/just_in_time_value_update.rs
  • grovedb/src/batch/just_in_time_reference_update.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

GROVE_V4 enables versioned Merk value updates and same-batch reference prediction. Merk predicts final stored bytes for multiple put types. Batch reference resolution uses stored or predicted target bytes, while GROVE_V1–V3 retain version 0 behavior.

Changes: Versioned batch updates

Layer / File(s) Summary
Version gates and compatibility
grovedb-version/src/version/*
The version configuration adds gates for same-batch reference prediction and just-in-time value updates. GROVE_V1–V3 select version 0; GROVE_V4 selects version 1.
Merk value prediction and storage costs
merk/src/tree/just_in_time_value_update/*, merk/src/tree/put_value/*, merk/src/tree/mod.rs, merk/src/tree/walk/mod.rs, merk/src/tree/ops.rs, merk/src/element/insert.rs, grovedb/src/batch/just_in_time_value_update.rs
Merk adds versioned value-update logic and a shared predictor for ordinary, provided-value-hash, and specialized-cost puts. Value-put methods now receive GroveVersion. The V1 update path remeasures stored bytes when required.
Same-batch reference prediction
grovedb/src/batch/just_in_time_reference_update/*, grovedb/src/batch/mod.rs, grovedb/src/batch/same_batch_reference_prediction_tests.rs
Batch processing tracks eligible targets and caches predictions until each target is written. Reference resolution hashes stored or predicted bytes. Tests cover reference ordering, partial batches, and replacement storage costs.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant Batch as Batch apply
  participant Fetch as get_referenced_element_bytes
  participant Predict as process_old_element_flags_v1
  participant Helper as predict_put_final_bytes
  participant Merk as TreeNode.predict_put_final_value
  Batch->>Fetch: Fetch target bytes and tree type
  Batch->>Predict: Pass target path and written state
  alt Target already written
    Predict->>Predict: Hash supplied stored bytes
  else Target still pending
    Predict->>Helper: Request final stored bytes
    Helper->>Merk: Predict final put value
    Merk-->>Helper: Return predicted bytes
    Helper-->>Predict: Return predicted bytes
    Predict->>Predict: Hash and cache predicted bytes
  end
  Batch->>Batch: After target writes, mark it written and clear its prediction
Loading
Merge Risk: ⚪ Minimal · up to 33bf9

The change adds new GROVE_V4 behavior for same-batch reference prediction and replacement storage-cost accounting, while earlier Grove versions keep their existing behavior. No concrete defect was identified, so the change appears ready to merge once the full CI suite passes.

Pre-merge checks · Passed 5
Passed Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage Passed Docstring coverage is 82.89% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 76 functions across 22 files. (1 skipped: 1…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the main GROVE_V4 fixes: same-batch sum-item references and replacements now commit to the bytes stored.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Waiting for bot review — coderabbitai ✓ · thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Oct 13, 2026
@thepastaclaw

thepastaclaw commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

⏹️ Not reviewed — PR is closed (merged) (commit 160c61d)

@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.48986% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.21%. Comparing base (5cb0962) to head (160c61d).

Files with missing lines Patch % Lines
...vedb/src/batch/just_in_time_reference_update/v0.rs 82.24% 19 Missing ⚠️
...edb/src/batch/just_in_time_reference_update/mod.rs 88.50% 10 Missing ⚠️
merk/src/tree/just_in_time_value_update/mod.rs 97.46% 7 Missing ⚠️
merk/src/tree/just_in_time_value_update/v0.rs 90.90% 7 Missing ⚠️
merk/src/tree/just_in_time_value_update/v1.rs 91.95% 7 Missing ⚠️
merk/src/tree/put_value/mod.rs 65.00% 7 Missing ⚠️
grovedb/src/batch/mod.rs 98.00% 2 Missing ⚠️
merk/src/merk/get.rs 83.33% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##           develop    #1021    +/-   ##
=========================================
  Coverage    93.20%   93.21%            
=========================================
  Files          352      356     +4     
  Lines       108428   108953   +525     
=========================================
+ Hits        101062   101558   +496     
- Misses        7366     7395    +29     
Components Coverage Δ
grovedb-core 91.59% <92.15%> (+0.02%) ⬆️
merk 94.18% <94.46%> (-0.03%) ⬇️
storage 91.68% <ø> (ø)
commitment-tree 95.62% <ø> (ø)
mmr 95.43% <ø> (ø)
bulk-append-tree 93.72% <ø> (ø)
element 97.39% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (no Phase 1 for this repository)

The two supplied blocking findings describe one confirmed mismatch between prediction and application for newly inserted targets retained across partial-batch segments. The V4 predictor can commit a reference to callback-modified flags while the live target stores the supplied flags without running that callback. Verification was static at the exact head; the supplied CI snapshot shows all Ubuntu test shards, formatting, linting, and security audit passing, with PR Hygiene pending.

🔴 1 blocking

Review provenance

Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: architecture-layering); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: platform-versioning); reviewer 4: gpt-6.1-sol (agent: phase2-reviewer, role: rust-quality); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)

  • Triage: critical by gpt-6.1-sol (effort low) — The intricate cross-layer changes directly alter authenticated-state commitments in grovedb/src/batch/just_in_time_reference_update/v1.rs::process_old_element_flags_v1 by changing which serialized target bytes are hashed into same-batch references.
  • Phase 1 reviewers: not run (disabled for this repository)
  • Fresh verifier: gpt-6.1-sol — final-verifier; agent sol-verifier
  • Phase 2 reviewers: gpt-6.1-sol — general (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — architecture-layering (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — platform-versioning (completed, effort xhigh); agent phase2-reviewer, gpt-6.1-sol — rust-quality (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `grovedb/src/batch/just_in_time_reference_update/v1.rs`:
- [BLOCKING] grovedb/src/batch/just_in_time_reference_update/v1.rs:73-85: Preserve the live node's replacement eligibility when predicting
  A flagged target first inserted by a partial batch's initial segment remains in the retained Merk with `old_value: None`: the insertion uses `TreeNode::new`, and neither `TreeNode::commit` nor `MerkCommitter::write` advances that field. Rewriting this item in the add-on is allowed by the cross-segment gate. When the add-on also writes a deeper reference, `continue_partial_apply_body` clears the target's written marker, so this call predicts the pending rewrite from its cached bytes.

  The detached node in `TreeNode::predict_put_final_value` unconditionally sets `old_value: Some(old_value)` and runs the flags updater. The actual `put_value_with_fixed_cost` on the retained node skips that updater because its `old_value` is still `None`. A deterministic callback that changes flag contents without changing their length therefore makes prediction hash modified flags while application stores the supplied flags. Ordinary same-batch references have no post-apply comparison to reject this disagreement, leaving the reference committed to bytes its target does not hold.

  This is a V4 regression for sum targets when the insertion and add-on supply identical flags: the legacy sum-item shortcut hashes those supplied flags and matches application. Carry the live node's updater eligibility into prediction, or synchronize its mutation-history state under the V4 gate. Extend the continuation regression to cover a target first inserted by the initial segment; the existing rewrite test seeds an already-stored predecessor through `grove_with`.

@github-actions

Copy link
Copy Markdown

Your move: thepastaclaw requested changes on this head; dismiss the review or push a fix; thepastaclaw left review threads unresolved; resolve them.
Full checklist in the description.

@github-actions github-actions Bot added waiting-self-review Waiting for the author to post /self-reviewed and removed waiting-bots Waiting for the review bots to report on this head labels Oct 13, 2026
…rite

Review fixes for the GROVE_V4 same-batch reference prediction:

- After a pass writes a target that a same-batch reference was committed
  to by prediction, the batch checks that the write stored exactly the
  predicted bytes and is refused otherwise, as it already is for a
  backward-references item. The check reads the node just written and is
  not charged.
- The reference arm decides whether to predict with the same predicate
  that records written targets, instead of a second copy tied by a
  debug_assert.
- The prediction always loads the stored node's value-defined cost for
  an ordinary put and lets the versioned install decide whether to keep
  it; TreeNode::ordinary_put_keeps_value_defined_cost is gone, so the
  put_value semantics live in the install helpers only.
- Tests: Drive's shapes (a flagged ReferenceWithSumItem in a sum tree
  under SumTree, CountSumTree, ProvableSumTree and
  ProvableCountProvableSumTree primaries, a NonCounted target, a
  two-epoch target shrunk in a later epoch), a proof through the
  reference (verifies on V4, not on V3), prediction v1 with Merk
  put_value 0, and the refusal of a write that stores other bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Waiting for bot review — coderabbitai not yet · thepastaclaw 1 thread unresolved — resolve it. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@github-actions github-actions Bot added waiting-bots Waiting for the review bots to report on this head and removed waiting-self-review Waiting for the author to post /self-reviewed labels Oct 13, 2026
QuantumExplorer and others added 2 commits October 13, 2026 16:09
…s given

A partial batch's initial segment can insert a flagged item at a new key
and its add-on rewrite it alongside a deeper reference to it. The Merk
still holds the inserted node with no value to replace, so the add-on's
write stores its bytes as given without running the flags update, while
the version 1 prediction ran Merk's just-in-time value update over the
inserted bytes. A flags update that changes the flags' contents (the
merging one keeping epoch-0 flags over same-size epoch-2 ones) left the
reference committed to bytes the item does not hold, which the
post-apply check now refuses.

Merk::replacement_runs_value_update says, from the nodes already in
memory, whether a write over a key runs the update; the prediction hashes
the supplied bytes when it does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Waiting for bot review — coderabbitai not yet · thepastaclaw not yet. Wait for the missing reviews, or a writer can post /skip-bots to proceed without them; blocking findings still need addressing.
Full checklist in the description.

@QuantumExplorer
QuantumExplorer merged commit 7ea60ee into develop Oct 13, 2026
15 of 16 checks passed
@QuantumExplorer
QuantumExplorer deleted the fix/grove-v4-flags-update-fixes branch October 13, 2026 13:34
QuantumExplorer added a commit that referenced this pull request Nov 8, 2026
…tinuation-dd8070

develop's #1021 landed this branch's two GROVE_V4 fixes (same-batch
reference prediction and the Unchanged re-measure) with later review
fixes of its own, so those now exist once, in develop's form. What this
branch adds over develop is the settlement mode:

- the prediction keeps develop's runs_value_update and post-apply
  check; a settling batch uses version 1 on every grove version
- CHANGELOG: the fixes stay under develop's entries in 6.1.0, and the
  settlement entries move to Unreleased
- same_batch_reference_prediction_tests is develop's file with the
  settling modes added, its helpers shared with settle_test_support
- the option-off estimate pins are retaken from develop at 0ea327d:
  18 of the 66 latest-version rows moved with #1018..#1020 and none of
  the 66 grove v3 rows did
- a settled tree's lookup comment says why it is still needed after #1018

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-bots Waiting for the review bots to report on this head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants