Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions book/src/contract-keywords/aggregates.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,10 @@ A range total costs more on each write than a per-value total, since every node
- Each `range*` flag has its prerequisite, as listed above.
- The meta-schema refuses a malformed value (`JsonSchemaError`, 10101).

## Writes past a sum

A sum holds a signed 64-bit integer, and each node of a sum tree keeps the total of its own value and its children's, so a write can take a sum out of range even when the type's values are each in range. A create, replace or transfer that would do so is refused when it is applied, as nothing checks the sums before. From protocol version 15 it is a paid error, `DocumentSumOverflowError` (40154): nothing of the write is stored, the identity contract nonce is bumped, and the signer pays the processing of the refused write as fee validation estimates it, with no storage fee. Bounding the summed property (`minimum` and `maximum`) to what the application needs keeps the sums out of reach.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Document the full set of writes that can overflow sums

The implementation prepares a refusal for any event containing a DocumentOperation, but the documentation names only creates, replacements, and transfers. Deletes, erases, index-only deletes, purchases, price updates, and contested document operations also route through this path; removing a negative value can increase an aggregate past i64::MAX, and other writes can underflow toward i64::MIN. Broaden this section and the tests to describe all document writes covered by the implementation, or narrow the implementation if the paid error is intentionally limited to the documented operations.

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality)


## Choosing what to set

| The application needs | Set |
Expand Down
2 changes: 1 addition & 1 deletion book/src/error-handling/error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ The fee category currently has a single code. The 30000 range is reserved for fu
| Range | Category | Examples |
|-------|----------|----------|
| 40000-40009 | Data Contract | `DataContractAlreadyPresentError` (40000), `DataContractIsReadonlyError` (40001), `DataContractNotFoundError` (40008) |
| 40100-40199 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139), `DocumentExpiredError` (40140), `DocumentContestMaximumContendersReachedError` (40141), `ReferencedDocumentRequirementNotMetError` (40142), `ReferencedDocumentTypeNotModeratedError` (40143), `ReferencedDocumentTypeModeratedError` (40144), `ReferencedDocumentRemovedError` (40145), `ReferencedDocumentTypeIndexOnlyError` (40146), `DocumentDeleteConstraintViolatedError` (40147), `ReferencedDocumentRuleReadInvalidError` (40148), `ReferencedDocumentPrecededError` (40149), `ReferencedDocumentTypePrivateError` (40150), `ReferencedDocumentTypeRequirementNotMetError` (40153) |
| 40100-40199 | Documents | `DocumentAlreadyPresentError` (40100), `DocumentNotFoundError` (40101), `DuplicateUniqueIndexError` (40105), `DocumentActionFeeAgreementNotSetError` (40132), `DocumentActionFeeAgreementMismatchError` (40133), `DocumentActionFeeMultiplierNotToleratedError` (40134), `DocumentActionFeeModeratorsShareMismatchError` (40139), `DocumentExpiredError` (40140), `DocumentContestMaximumContendersReachedError` (40141), `ReferencedDocumentRequirementNotMetError` (40142), `ReferencedDocumentTypeNotModeratedError` (40143), `ReferencedDocumentTypeModeratedError` (40144), `ReferencedDocumentRemovedError` (40145), `ReferencedDocumentTypeIndexOnlyError` (40146), `DocumentDeleteConstraintViolatedError` (40147), `ReferencedDocumentRuleReadInvalidError` (40148), `ReferencedDocumentPrecededError` (40149), `ReferencedDocumentTypePrivateError` (40150), `ReferencedDocumentTypeRequirementNotMetError` (40153), `DocumentSumOverflowError` (40154) |
| 40200-40217 | Identity | `IdentityAlreadyExistsError` (40200), `InvalidIdentityRevisionError` (40203), `IdentityInsufficientBalanceError` (40210) |
| 40300-40308 | Voting | `MasternodeNotFoundError` (40300), `MasternodeVoteAlreadyPresentError` (40304), `VoteChoiceNotAllowedForVotePollError` (40307), `YesNoVotePollNotAvailableForVotingError` (40308) |
| 40400-40401 | Prefunded Balances | `PrefundedSpecializedBalanceInsufficientError` (40400) |
Expand Down
1 change: 1 addition & 0 deletions packages/rs-dpp/src/errors/consensus/codes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,7 @@ impl ErrorWithCode for StateError {
Self::DocumentTypeNotPrivateError(_) => 40151,
Self::PrivateDocumentEntrySizeMismatchError(_) => 40152,
Self::ReferencedDocumentTypeRequirementNotMetError(_) => 40153,
Self::DocumentSumOverflowError(_) => 40154,

// Identity Errors: 40200-40299
Self::IdentityAlreadyExistsError(_) => 40200,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
use crate::consensus::state::state_error::StateError;
use crate::consensus::ConsensusError;
use crate::ProtocolError;
use bincode::{Decode, DecodeUntrusted, Encode};
use platform_serialization_derive::{
PlatformDeserializeTrusted, PlatformDeserializeUntrusted, PlatformSerialize,
};
use platform_value::Identifier;
use thiserror::Error;

/// A document write to a data contract would take one of the sums its document types keep (an
/// index `summable` by a property, a `rangeSummable` index, or `documentsSummable`) past what a
/// sum holds, a signed 64 bit integer. The write is refused as a paid error from protocol
/// version 15: the identity contract nonce is bumped and nothing else of the write is stored.
#[derive(
Error,
Debug,
Clone,
PartialEq,
Eq,
Encode,
Decode,
PlatformSerialize,
PlatformDeserializeTrusted,
PlatformDeserializeUntrusted,
DecodeUntrusted,
)]
#[error("a document write to contract {contract_id} would take a document sum past the largest value a sum holds")]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💬 Nitpick: Describe underflow in the consensus error message

The same aggregate failure is raised when a signed sum falls below i64::MIN, but the display text says only that the write passes the largest value a sum holds. This gives an inaccurate diagnostic for writes that remove a negative value or otherwise cause a lower-bound underflow. Use range-neutral wording such as “outside the signed 64-bit range.”

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality); gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

#[platform_serialize(unversioned)]
pub struct DocumentSumOverflowError {
/*

DO NOT CHANGE ORDER OF FIELDS WITHOUT INTRODUCING OF NEW VERSION

*/
contract_id: Identifier,
}

impl DocumentSumOverflowError {
pub fn new(contract_id: Identifier) -> Self {
Self { contract_id }
}

pub fn contract_id(&self) -> &Identifier {
&self.contract_id
}
}

impl From<DocumentSumOverflowError> for ConsensusError {
fn from(err: DocumentSumOverflowError) -> Self {
Self::StateError(StateError::DocumentSumOverflowError(err))
}
}
1 change: 1 addition & 0 deletions packages/rs-dpp/src/errors/consensus/state/document/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ pub mod document_incorrect_purchase_price_error;
pub mod document_not_for_sale_error;
pub mod document_not_found_error;
pub mod document_owner_id_mismatch_error;
pub mod document_sum_overflow_error;
pub mod document_timestamp_window_violation_error;
pub mod document_timestamps_are_equal_error;
pub mod document_timestamps_mismatch_error;
Expand Down
14 changes: 14 additions & 0 deletions packages/rs-dpp/src/errors/consensus/state/state_error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ use crate::consensus::state::document::referenced_document_type_index_only_error
use crate::consensus::state::document::referenced_document_type_private_error::ReferencedDocumentTypePrivateError;
use crate::consensus::state::document::referenced_document_type_requirement_not_met_error::ReferencedDocumentTypeRequirementNotMetError;
use crate::consensus::state::document::document_type_not_private_error::DocumentTypeNotPrivateError;
use crate::consensus::state::document::document_sum_overflow_error::DocumentSumOverflowError;
use crate::consensus::state::document::private_document_entry_size_mismatch_error::PrivateDocumentEntrySizeMismatchError;
use crate::consensus::state::document::document_delete_constraint_violated_error::DocumentDeleteConstraintViolatedError;
use crate::consensus::state::document::referenced_document_rule_read_invalid_error::ReferencedDocumentRuleReadInvalidError;
Expand Down Expand Up @@ -777,6 +778,11 @@ pub enum StateError {
// document type that does not meet its `documentTypeRequirements` (protocol version 15).
#[error(transparent)]
ReferencedDocumentTypeRequirementNotMetError(ReferencedDocumentTypeRequirementNotMetError),

// A document write would take one of its contract's document sums past what a sum holds
// (protocol version 15).
#[error(transparent)]
DocumentSumOverflowError(DocumentSumOverflowError),
}

impl From<StateError> for ConsensusError {
Expand Down Expand Up @@ -1768,5 +1774,13 @@ mod tests {
)),
181
);
// A document write past one of its contract's document sums (protocol version 15): the
// tail of the enum.
assert_eq!(
discriminant_of(StateError::DocumentSumOverflowError(
DocumentSumOverflowError::new(identity_id)
)),
182
);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,14 @@ use crate::rpc::core::CoreRPCLike;
use dpp::address_funds::PlatformAddress;
use dpp::balances::credits::CreditOperation;
use dpp::block::block_info::BlockInfo;
use dpp::consensus::state::document::document_sum_overflow_error::DocumentSumOverflowError;
use dpp::consensus::ConsensusError;
use dpp::fee::Credits;
use dpp::platform_value::Identifier;
use dpp::version::PlatformVersion;
use drive::grovedb::Error as GroveError;
use drive::grovedb::Transaction;
use drive::util::batch::{ContractModerationOperationType, DriveOperation, IdentityOperationType};
use std::collections::BTreeMap;

impl<C> Platform<C>
Expand All @@ -18,6 +22,22 @@ where
{
/// Generation 1, except that a gas sponsor who pays is named as the owner of the storage
/// the batch's documents write, so the refunds of that storage go back to them.
///
/// A document write whose operations would take a document sum past what a sum tree holds
/// (GroveDB refuses to apply them) is refused as a paid consensus error,
/// [`DocumentSumOverflowError`]: everything the attempt wrote is rolled back, and the event
/// is executed again as the nonce bump a refused state validation of the batch leaves
/// ([`SumOverflowRefusal`]). The signer pays for the bump, the work validation billed, and
/// the processing of the refused write as fee validation estimates it; no storage. Whether a
/// write overflows is a function of the state it is applied to, so every node refuses the
/// same writes.
///
/// The rollback needs a savepoint on the block transaction, which stays on the transaction
/// when the write applies (RocksDB pops a savepoint only by rolling back to it). The genesis
/// height rewinds each re-proposal of its block to the one savepoint `init_chain` set, so no
/// savepoint is set there and a write overflowing a sum at the genesis height stays the
/// internal error it was. At every other height a savepoint left on the transaction is inert:
/// the block's transaction is committed or dropped as a whole.
#[inline(always)]
#[allow(clippy::too_many_arguments)]
pub(super) fn execute_event_v2(
Expand All @@ -26,19 +46,171 @@ where
consensus_errors: Vec<ConsensusError>,
block_info: &BlockInfo,
transaction: &Transaction,
address_balances_in_update: Option<&mut BTreeMap<PlatformAddress, CreditOperation>>,
mut address_balances_in_update: Option<&mut BTreeMap<PlatformAddress, CreditOperation>>,
block_credit_mints: &mut Credits,
platform_version: &PlatformVersion,
) -> Result<EventExecutionResult, Error> {
self.execute_event_v1_body(
// An event that already carries consensus errors is the nonce bump of a refused batch,
// which writes no document.
let refusal = if consensus_errors.is_empty()
&& block_info.height != self.config.abci.genesis_height
{
SumOverflowRefusal::of(&event)
} else {
None
Comment on lines +53 to +60

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Blocking: Partially invalid batches bypass paid overflow handling

consensus_errors.is_empty() does not mean that the event contains no document write. Batch state validation can replace an invalid subtransition with a nonce-bump action while retaining valid document actions, producing an event with both consensus errors and DocumentOperations. For such an event this guard does not construct SumOverflowRefusal; the retained document write is executed directly, and a sum overflow propagates as an internal error rather than the paid 40154 result. The v15 path must distinguish a bump-only event from an event that still contains document operations, preserve the existing errors and bump operations, and add a mixed valid-overflowing/invalid batch regression test.

source: gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

};
let Some(refusal) = refusal else {
return self.execute_event_v1_body(
event,
consensus_errors,
block_info,
transaction,
address_balances_in_update,
block_credit_mints,
true,
platform_version,
);
};

transaction.set_savepoint();
match self.execute_event_v1_body(
event,
consensus_errors,
block_info,
transaction,
address_balances_in_update,
address_balances_in_update.as_deref_mut(),
block_credit_mints,
true,
platform_version,
)
) {
Err(Error::Drive(error)) if error.is_sum_tree_overflow() => {
// Applying a write may take several GroveDB batches and direct writes (the
// clean-up of expired time range buckets before them), and only the refused
// batch is undone by GroveDB. Rolling back to the savepoint undoes the rest, so
// the state is as if the write had never been attempted. The fee and the
// balance changes are written only after the operations applied.
transaction
.rollback_to_savepoint()
Comment on lines +86 to +93

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Blocking: Overflow detection is not scoped to the document write

The guard converts any Error::Drive returned by the entire execute_event_v1_body call into the document-sum refusal path. That call includes fee estimation, application of all event operations, identity fee-balance writes, and other bookkeeping, so an aggregate overflow from a non-document operation can cause a successful document attempt to be rolled back and reported as DocumentSumOverflowError. In addition, drive::Error::is_sum_tree_overflow matches every MerkError::Overflow(_); the pinned Merk source uses that variant for signed sums, big sums, count aggregates, and several estimation paths. The fallback must retain operation/tree provenance and only classify the specific document sum-aggregate failure, or it must fail closed for unrelated overflow messages.

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality); gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

.map_err(|error| Error::Drive(GroveError::StorageError(error).into()))?;
let SumOverflowRefusal {
event: refusal_event,
attempted_operations,
contract_id,
} = refusal;
let attempted_processing_fee = self
.drive
.apply_drive_operations(
attempted_operations,
false,
block_info,
Some(transaction),
platform_version,
)
.map_err(Error::Drive)?
.processing_fee;
self.execute_event_v1_body(
refusal_event.with_additional_fixed_fee_cost(attempted_processing_fee),
vec![DocumentSumOverflowError::new(contract_id).into()],
Comment on lines +100 to +113

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Blocking: The overflow retry misclassifies processing fees as fixed costs

attempted_processing_fee is added to additional_fixed_fee_cost on the synthetic refusal event. This creates two correctness problems. First, settle_fees_of_event_v1 includes additional_fixed_fee_cost in the required budget for a limited signing key, even though the key-budget rule deliberately excludes the base metered processing fee; a key whose fixed costs and user-selected increase fit its remaining budget can therefore receive IdentityPublicKeyBudgetExceededError instead of the promised DocumentSumOverflowError. Second, execute_event_v1_body applies user_fee_increase before adding additional_fixed_fee_cost, so the refused write's processing is charged at its base amount rather than the multiplier selected by the signer. Charge this amount through the processing-fee path, or otherwise preserve both the key-budget exemption and the user-fee multiplier, and add regressions for a limited key and a nonzero fee increase.

source: gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

block_info,
transaction,
address_balances_in_update,
block_credit_mints,
true,
platform_version,
)
}
result => result,
}
}
}

/// What an identity-paid document write is executed as when its operations would take a
/// document sum past what a sum tree holds.
struct SumOverflowRefusal<'a> {
/// The write's event with only the operations a refused state validation of the batch
/// leaves: the identity contract nonce bumps and the removal of a suspension that lapsed.
/// Nothing else of the write runs, so it moves no balance, owes no action fee and is never
/// sponsored, like any batch that failed.
event: ExecutionEvent<'a>,
/// The operations of the refused write, which price the processing it cost.
attempted_operations: Vec<DriveOperation<'a>>,
/// The contract the write is to.
contract_id: Identifier,
}

impl<'a> SumOverflowRefusal<'a> {
/// The refusal of `event`, when it is an identity-paid write of documents to a contract.
fn of(event: &ExecutionEvent<'a>) -> Option<Self> {
let ExecutionEvent::Paid {
identity,
operations,
execution_operations,
user_fee_increase,
signing_key_limits,
..
} = event
else {
return None;
};
if !operations
.iter()
.any(|operation| matches!(operation, DriveOperation::DocumentOperation(_)))
{
return None;
}
let contract_id = operations.iter().find_map(|operation| match operation {
DriveOperation::IdentityOperation(
IdentityOperationType::UpdateIdentityContractNonce { contract_id, .. },
) => Some(Identifier::new(*contract_id)),
_ => None,
})?;
Comment on lines +161 to +166

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: The overflow error can name the wrong contract

SumOverflowRefusal::of selects the first UpdateIdentityContractNonce operation as contract_id. Batch transformation groups document transitions by contract and can emit several nonce operations in deterministic order. When a later contract's document operation causes the generic Merk overflow, the returned 40154 error carries the earlier contract's identifier, so its only identifying field and message point clients to a contract whose sum did not overflow. Preserve the failing operation's contract provenance or avoid claiming a single contract when the storage error cannot identify one; add a mixed-contract batch regression test.

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality); gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

let refused_operations = operations
.iter()
.filter(|operation| {
matches!(
operation,
DriveOperation::IdentityOperation(
IdentityOperationType::UpdateIdentityContractNonce { .. }
) | DriveOperation::ContractModerationOperation(
ContractModerationOperationType::RemoveSuspension { .. }
)
)
})
.cloned()
.collect();
Some(Self {
event: ExecutionEvent::Paid {
identity: identity.clone(),
removed_balance: None,
added_to_balance_outputs: None,
operations: refused_operations,
execution_operations: execution_operations.clone(),
additional_fixed_fee_cost: None,
user_fee_increase: *user_fee_increase,
signing_key_limits: *signing_key_limits,
gas_sponsor: None,
action_fees: vec![],
},
Comment on lines +182 to +193

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Blocking: Sponsored overflow writes can become unpaid internal errors

SumOverflowRefusal unconditionally removes gas_sponsor. The original sponsored event can pass check_tx with an unfunded signer because the sponsor covers the estimated gas, but the retry is settled as identity-paid and includes the refused-write surcharge. If the signer cannot cover that amount, fee settlement returns an insufficient-balance or key-budget error; process_validation_result_v1 converts the resulting UnpaidConsensusExecutionError into an internal error without charging the nonce or fee. This permits repeated admitted sponsored overflow writes to consume proposer validation, rollback, and fee-estimation work without payment and can cause proposals containing them to be rejected. The refusal must preserve a valid payment path or reject/charge the sponsored failure deterministically, with coverage for a zero-balance signer and a sponsor-funded document type.

source: gpt-6.1-sol (phase2-reviewer: security-auditor)

attempted_operations: operations.clone(),
Comment on lines +143 to +194

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Avoid cloning the complete event on every successful document write

SumOverflowRefusal::of runs before every eligible identity-paid document event and eagerly clones the identity, execution operations, filtered refusal operations, and the full operations vector, including converted document payloads. The resulting owned fallback is discarded on the normal success path, so the new rare-error handling adds unconditional allocation and cloning to every successful document batch. Defer ownership until the first classified overflow, or restructure the fallible execution stage to return the original event on failure.

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality); gpt-6.1-sol (phase2-reviewer: general, architecture-layering, platform-versioning, rust-quality, security-auditor)

Comment on lines +167 to +194

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: Share or verify the refused-batch operation construction

SumOverflowRefusal::of independently reconstructs the operations and fee metadata that a failed batch validation is expected to leave: it filters nonce bumps and suspension removals, strips balances, sponsors, and action fees, and resets the fixed fee. The canonical batch validation and action-conversion paths already encode those failed-batch semantics. A future change to the set of operations surviving a failed validation can therefore make this new refusal path diverge in state writes or fees without a compiler failure. Reuse the canonical construction where possible, or add a parity test comparing both refusal paths for the same batch, including lapsed suspensions and fee-related fields.

source: muse-spark-1.3-contributor (phase1-reviewer: general, rust-quality)

contract_id,
})
}
}

impl ExecutionEvent<'_> {
/// The event with `fee` added to the processing fee it charges up front.
fn with_additional_fixed_fee_cost(mut self, fee: Credits) -> Self {
if let ExecutionEvent::Paid {
additional_fixed_fee_cost,
..
} = &mut self
{
*additional_fixed_fee_cost = Some(
additional_fixed_fee_cost
.unwrap_or_default()
.saturating_add(fee),
);
}
self
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ mod repeated_properties_update;
mod replacement;
mod required_since;
mod stored_non_document_reference;
mod sum_overflow;
mod system_agreement;
mod token_cost_policy;
mod token_market_orders;
Expand Down
Loading
Loading