Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 10 additions & 16 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,22 +52,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
deletion sections it, every byte of the new element, key included, counts as
added, and the element keeps the flags the callback leaves it with. GroveDB
does not look at owners: `StorageFlags::update_element_flags_replacing_on_owner_change`
answers it when the new flags name a different owner. A batch accepts the
answer only with the new `BatchApplyOptions::allow_replacement_accounting`,
and refuses it on a write whose flags are byte-equal to the stored ones
(which covers an untrusted `RefreshReference` and a tree's root update), with
`Error::InvalidBatchOperation`. With the option, average-case and worst-case
estimates charge every `InsertOrReplace`, `Replace`, `Patch` and trusted
`RefreshReference` of an element with flags, and the write of a flagged tree
the batch also writes under, at least the bytes its node adds when inserted.
The option needs `apply_batch.replacement_accounting` (0 in V1..V4, 1 in V5),
and Merk accepts the answer from `merk_versions.tree.just_in_time_value_update`
version 2 (V5);
earlier versions refuse a batch or estimate that sets it. Off, every cost,
estimate, stored element and hash is what it was. **BREAKING** for code
building `BatchApplyOptions` or `GroveDBApplyBatchVersions` field by field,
and for callers of Merk's just-in-time apply, whose update callback now
answers `(ElementFlagsUpdate, Option<ValueDefinedCostType>)`.
answers it when the new flags name a different owner. A batch refuses it on
a write whose flags are byte-equal to the stored ones (which covers an
untrusted `RefreshReference` and a tree's root update), with
`Error::InvalidBatchOperation`. Merk accepts the answer from
`merk_versions.tree.just_in_time_value_update` version 2 (V5); earlier
versions refuse it. Estimates do not model it, because an estimator cannot
ask the callback: estimate such a write as an insertion. A callback that never
answers it sees every cost, stored element and hash as before. **BREAKING**
for callers of Merk's just-in-time apply, whose update callback now answers
`(ElementFlagsUpdate, Option<ValueDefinedCostType>)`.
### Changed
- **BREAKING**: `Query` adds the public `offset: Option<u16>` field for
branch-local committed-count-unit pagination. Exhaustive struct literals
Expand Down
2 changes: 1 addition & 1 deletion docs/crates/auxiliary.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ These work on raw `ElementFlags` and match the callback signatures GroveDB expec

It always merges with `MergingOwnersStrategy::UseTheirs`. When `old_flags` is `Some`, both the old and the new bytes must decode to flags, or it returns `RemovingFlagsError`.
- `StorageFlags::update_element_flags_with_written_owner(cost, old_flags, new_flags) -> Result<bool, StorageFlagsError>` is `update_element_flags`, except that the owner `new_flags` names always owns the element afterwards, including where `update_element_flags` keeps the old owner: an update that keeps the size, or one that shrinks an element holding bytes of one epoch when the new flags' epoch is later than that one. To leave the element with its owner while someone else pays for the write, write flags naming the current owner. To hand it over, write flags naming the new owner. New flags that name no owner get what `update_element_flags` gives. Bytes the update frees are still refunded to the old owner, and bytes that stay stored move with the element.
- `StorageFlags::update_element_flags_replacing_on_owner_change(cost, old_flags, new_flags) -> Result<ElementFlagsUpdate, StorageFlagsError>` is `update_element_flags_with_written_owner`, except that when the new flags name a different owner than the old ones it leaves them as written and answers `ElementFlagsUpdate::AccountAsReplacement`: the old owner is refunded for the whole old element and the new element's every byte is added. The batch must set `BatchApplyOptions::allow_replacement_accounting` (GROVE_V5+).
- `StorageFlags::update_element_flags_replacing_on_owner_change(cost, old_flags, new_flags) -> Result<ElementFlagsUpdate, StorageFlagsError>` is `update_element_flags_with_written_owner`, except that when the new flags name a different owner than the old ones it leaves them as written and answers `ElementFlagsUpdate::AccountAsReplacement`: the old owner is refunded for the whole old element and the new element's every byte is added. GroveDB accepts the answer from GROVE_V5.
- `StorageFlags::split_removal_bytes(flags: &mut ElementFlags, removed_key_bytes, removed_value_bytes)` decodes the flags and calls `split_storage_removed_bytes`. It returns `BasicStorageRemoval` for both parts if the element has no flags.

Callers hand these helpers `new_flags` of one epoch. What the helpers return may hold several, and that is safe: the batch puts the written flags back before every retry, so each retry starts from the same input and the size settles. Written flags that already hold several epochs can make the size flip between retries until the batch stops with its cyclic error, whether or not the owner changes. So can stored multi-epoch flags with no owner under written flags that name one, on `update_element_flags`. Either way the batch is refused and nothing is written. This is known and left as it is: shipped protocol versions depend on this merge, and a write of one-epoch flags with an owner over stored flags with an owner does not reach it.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,7 @@ impl StorageFlags {
/// Every other update (same owner, or no owner on one side) is answered by
/// [`Self::update_element_flags_with_written_owner`].
///
/// The batch accepts the answer only when its options set
/// `allow_replacement_accounting`.
/// GroveDB accepts the answer from grove version 5.
///
/// Only an element written with the new owner's flags is handed over, under
/// this helper and [`Self::update_element_flags_with_written_owner`] alike.
Expand Down
1 change: 0 additions & 1 deletion grovedb-version/src/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ fn v5_matches_v4_apart_from_its_gates() {
path_query_methods.merge = GROVE_V4.grovedb_versions.path_query_methods.merge;
path_query_methods.per_instance_query_offsets = 0;
v5_as_v4.grovedb_versions.apply_batch.keyless_append_repeats = 0;
v5_as_v4.grovedb_versions.apply_batch.replacement_accounting = 0;
v5_as_v4.merk_versions.tree.just_in_time_value_update =
GROVE_V4.merk_versions.tree.just_in_time_value_update;
assert_eq!(format!("{v5_as_v4:?}"), format!("{GROVE_V4:?}"));
Expand Down
7 changes: 0 additions & 7 deletions grovedb-version/src/version/grovedb_versions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -307,13 +307,6 @@ pub struct GroveDBApplyBatchVersions {
/// slot gates an accepted/rejected outcome: a batch V1..V4 refuse is
/// accepted on V5+.
pub keyless_append_repeats: FeatureVersion,
/// Whether a batch may set `BatchApplyOptions::allow_replacement_accounting`.
///
/// - `0` (grove v1..v4): a batch or estimate that sets the option is
/// refused.
/// - `1`: the flags update may answer `AccountAsReplacement`, and an
/// estimate covers it.
pub replacement_accounting: FeatureVersion,
}

#[derive(Clone, Debug, Default)]
Expand Down
1 change: 0 additions & 1 deletion grovedb-version/src/version/v1.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ pub const GROVE_V1: GroveVersion = GroveVersion {
reference_through_conditional_insert: 0,
same_batch_reference_target_prediction: 0,
keyless_append_repeats: 0,
replacement_accounting: 0,
},
element: GroveDBElementMethodVersions {
delete: 0,
Expand Down
1 change: 0 additions & 1 deletion grovedb-version/src/version/v2.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ pub const GROVE_V2: GroveVersion = GroveVersion {
reference_through_conditional_insert: 0,
same_batch_reference_target_prediction: 0,
keyless_append_repeats: 0,
replacement_accounting: 0,
},
element: GroveDBElementMethodVersions {
delete: 0,
Expand Down
1 change: 0 additions & 1 deletion grovedb-version/src/version/v3.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ pub const GROVE_V3: GroveVersion = GroveVersion {
reference_through_conditional_insert: 0,
same_batch_reference_target_prediction: 0,
keyless_append_repeats: 0,
replacement_accounting: 0,
},
element: GroveDBElementMethodVersions {
delete: 0,
Expand Down
1 change: 0 additions & 1 deletion grovedb-version/src/version/v4.rs
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,6 @@ pub const GROVE_V4: GroveVersion = GroveVersion {
same_batch_reference_target_prediction: 1,
// v0: every operation is counted for repeats, keyless appends included.
keyless_append_repeats: 0,
replacement_accounting: 0,
},
element: GroveDBElementMethodVersions {
delete: 0,
Expand Down
10 changes: 2 additions & 8 deletions grovedb-version/src/version/v5.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,16 +42,11 @@
//! batch, as 6.1.1 did. Gated because it flips an accepted/rejected outcome
//! (issue #1031).
//!
//! - `apply_batch.replacement_accounting: 1`: a batch may set
//! `allow_replacement_accounting`, so a flags update can have an update
//! accounted as the removal of the old element plus the insertion of the
//! new one. V4 refuses the option. Gated because the accounting and the
//! stored flags are consensus-relevant once a caller uses it.
//!
//! - `merk_versions.tree.just_in_time_value_update: 2`: Merk accepts that
//! answer and records the removal plus the insertion. Versions 0 and 1
//! refuse it, so grove v1..v4 cannot account an update as a replacement at
//! any layer. Gated with the option above, for the same reason.
//! any layer. Gated because the accounting and the stored flags are
//! consensus-relevant once a caller uses it.
//!
//! It exists so that changes made after GroveDB 6.1.1 have somewhere to
//! land. GROVE_V4 as 6.1.1 shipped it is what Dash Platform 5.0 (protocol
Expand Down Expand Up @@ -86,7 +81,6 @@ pub const GROVE_V5: GroveVersion = GroveVersion {
grovedb_versions: GroveDBVersions {
apply_batch: GroveDBApplyBatchVersions {
keyless_append_repeats: 1, // changed: a batch may repeat a keyless append
replacement_accounting: 1, // changed: a batch may account an update as a replacement
..GROVE_V4.grovedb_versions.apply_batch
},
element: GroveDBElementMethodVersions {
Expand Down
Loading
Loading