diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index d4c1a6f368a..f26dba5affe 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -148,6 +148,7 @@ - [Builder Pattern](sdk/builder-pattern.md) - [Fetch Traits](sdk/fetch-traits.md) +- [Protocol Versions in the SDK](sdk/protocol-versions.md) - [Features from Protocol Version 14](sdk/protocol-version-gate.md) - [Put Operations](sdk/put-operations.md) - [Identity Keys Deep Dive](sdk/identity-keys.md) diff --git a/book/src/sdk/fetch-traits.md b/book/src/sdk/fetch-traits.md index 9d33449f107..0876448d5d4 100644 --- a/book/src/sdk/fetch-traits.md +++ b/book/src/sdk/fetch-traits.md @@ -190,10 +190,21 @@ where a contract was updated but the local cache still has the old version. ```rust pub trait Query: Send + Debug + Clone { fn query(&self, settings: &QuerySettings<'_>) -> Result; + + /// The protocol version every node serves the request from; see step 2 below. + fn protocol_requirement( + &self, + settings: &QuerySettings<'_>, + ) -> Result, Error> + where + T: TransportRequest, + { /* reads TransportRequest::needs_protocol_version */ } } ``` -Wire requests opt into `WireQuery` and serve as queries for themselves: +Wire requests opt into `WireQuery` and serve as queries for themselves. +`GetDocumentsRequest` does not: a document request is fetched with a `DocumentQuery`, whose +features decide which networks serve it. ```rust impl Query for T @@ -290,10 +301,11 @@ When you call `Identity::fetch(&sdk, id)`, here is what happens: enabled. 2. **Protocol version check**: a request that only newer nodes serve - (`TransportRequest::needs_protocol_version`) is refused with - `Error::ProtocolVersionNotActive` until a verified answer shows the network runs that - version, asking the network for it first when it may have moved. Nothing is sent and - no node is banned. See [Features from Protocol Version 14](protocol-version-gate.md). + (`Query::protocol_requirement`: what the request declares, or what a document query's + features need) is refused with `Error::ProtocolVersionNotActive` until a verified answer + shows the network runs that version, asking the network for it first when it may have + moved. Nothing is sent and no node is banned. See [Protocol Versions in the + SDK](protocol-versions.md). Asking may raise the SDK's version, so the request is then encoded again with it. `FromProof::check_request` then checks the request at the version it is encoded with, and refuses one that no node serves or that the output type cannot be verified diff --git a/book/src/sdk/protocol-version-gate.md b/book/src/sdk/protocol-version-gate.md index 72ffb60b478..9aef9b2d34f 100644 --- a/book/src/sdk/protocol-version-gate.md +++ b/book/src/sdk/protocol-version-gate.md @@ -1,112 +1,82 @@ # Features from Protocol Version 14 -Protocol version 14 adds query methods and request fields that nodes of the previous -release do not know. Until a network activates protocol 14, it runs both releases: 4.1 -nodes and 5.0 nodes, all at protocol 13. A 4.1 node answers a method it does not know -with `UNIMPLEMENTED`, and decodes a request with the 4.1 protobuf types, dropping the -fields it does not know and answering the request that is left. Either answer gets an -honest node banned: the SDK retries `UNIMPLEMENTED` elsewhere and bans the node, and it -cannot verify the answer to a different question. - -So the SDK does not send such a request until it knows every node serves it. A request -that needs it is refused locally, with a dedicated error, and no node is banned. +Protocol version 14 adds query methods, request fields and document query features that nodes +of the previous release do not serve. Until a network activates protocol 14, it runs both +releases: 4.1 nodes and 5.0 nodes, all at protocol 13. A 4.1 node answers a method it does not +know with `UNIMPLEMENTED`, and decodes a request with the 4.1 protobuf types, dropping the +fields it does not know and answering the request that is left. Either answer gets an honest +node banned: the SDK retries `UNIMPLEMENTED` elsewhere and bans the node, and it cannot verify +the answer to a different question. Some document features are refused by every node at 13, +and one fails there in a way that bans the node. + +So the SDK does not send such a request until a verified answer shows the network runs +protocol 14. A request that needs it is refused locally, with a dedicated error, and no node is +banned. [Protocol Versions in the SDK](protocol-versions.md) describes how the SDK decides, how +long a probe of the network's version holds, and the errors. ## What needs protocol version 14 | Request | Needs 14 when | |---|---| | `getIdentityKeysRemainingBudgets`, `getDataContractsLatestVersions`, `getDataContractsByRange`, `getContractGroupInfo`, `getContractGroupMembers`, `getContractGroupsForContract`, `getContractModerationStatus`, `getContractModerationEntries`, `getContractDocumentRemovals`, `getContractTeamActions`, `getContractTeamActionSigners`, `getContractModerationActionCounts`, `getContractFeePots` | always | -| `getDocuments` | a composite query (`sub_queries`) or a chained query (`chained`) | +| `getDocuments` | the query uses any of: a composite query (sub-queries), a chained query, `contains` or `containsAny`, a time-range or integer-range selection, an aggregate ordered by the selected aggregate (ranked), a `having` clause, several `in` clauses in a documents query, or a start cursor on a document type whose documents are summed (`documentsSummable`) | | `getShieldedPoolState`, `getShieldedNotesCount`, `getShieldedAnchors`, `getMostRecentShieldedAnchor`, `getShieldedEncryptedNotes`, `getShieldedNullifiers` | a token's pool is asked for (`token_id`); requests for the credit pool are the ones 4.1 SDKs send, and are always sent | -Other protocol 14 additions are refused alike by every node before protocol 14 (for -example the `contains` and time-range where operators, which nodes at 13 answer with -`INVALID_ARGUMENT`), or are selected by the SDK only once protocol 14 is verified (the -non-unique key lookup's request version 1). Neither bans a node. - -Each request declares what it needs through -`TransportRequest::needs_protocol_version` in `rs-dapi-client`. A test in -`rs-dapi-client/tests/request_inventory.rs` compares every request field, oneof arm, -enum value and method with what 4.1 nodes understand, and fails on any addition without -a declaration. - -## How the SDK decides - -The SDK sends such a request when both hold: - -- a response whose proof and quorum signature it verified was signed at protocol 14 or - later: proof that every node answering now runs 5.0 or newer, since no 4.1 node can - sign protocol 14; and -- its own protocol version is 14 or later. A version seeded at build time, persisted from - an earlier session or pinned with `SdkBuilder::with_version` says nothing about the - network on its own. - -This is the same rule that selects the non-unique key lookup's request version 1; both use -`QuerySettings::network_protocol_basis`. An unpinned SDK on a network proven at a version -newer than it knows speaks the newest version it knows, so that the newer network serves -its protocol 14 features. +The non-unique key lookup's request version 1 is selected only once protocol 14 is verified, +and is otherwise sent as version 0, which every node serves. -When the network may have reached protocol 14 since the SDK last heard from it (nothing -verified yet, or the last verified answer was signed below 14), the SDK first asks for the -genesis epoch, with a proof whatever the SDK's proof setting, under the caller's request -settings (timeouts, retries). Once an answer signed at 14 has been verified, the SDK never -asks again. It does not ask when no answer can change the outcome: +How each request declares what it needs: -- an SDK pinned below 14 speaks its pinned version, whatever the network runs; -- an SDK without a context provider cannot verify the answer, and an answer it cannot - verify would ban the node that sent it. - -An SDK that does not ask for proofs still asks for this one, and needs a context provider -that can give quorum keys for it. When its provider cannot, the question fails on every -node; that says nothing about the nodes, so none is banned for it, and the request is -refused with that failure as its cause. Before this check, such an SDK sent these requests -unverified. - -Mock SDKs do not enforce the rule: they answer what a test registered, and have no nodes -to ban. +- A method or request field declares it through `TransportRequest::needs_protocol_version` in + `rs-dapi-client`. A test in `rs-dapi-client/tests/request_inventory.rs` compares every request + field, oneof arm, enum value and method with what 4.1 nodes understand, and fails on any + addition without a verdict. +- A document query declares it through `DocumentQuery::required_protocol_version` (and + `ChainedDocumentQuery::required_protocol_version`) in `dash-platform-queries`, from the + features it uses. Each feature reads a version table slot: a slot that is `None` before the + protocol version that adds a field older nodes drop (`document_sub_queries`, + `chained_document_query`), or the slot the server switches on for a route + (`document_type_schema`, `compute_aggregate_mode_and_check_limit`, `where_clause_grouping`, + `start_at_document`). The refusal names the feature. ## The refusal | Surface | Error | |---|---| -| Rust (`dash-sdk`) | `Error::ProtocolVersionNotActive { method, required, network, pinned, cause }` | +| Rust (`dash-sdk`) | `Error::ProtocolVersionNotActive { method, required, feature, network, pinned, cause }` | | JS (`wasm-sdk`, `evo-sdk`) | `WasmSdkError` with kind `ProtocolVersionNotActive` (`error.name === 'ProtocolVersionNotActive'`) | | FFI (Swift, Kotlin) | `DashSDKErrorCode::ProtocolError`, with a message that starts with ` needs Platform protocol version`; `NetworkError` when asking for the network's version failed for a reason that can pass | -`method` names the gRPC method (for example `get_contract_fee_pots`), `required` the -version it needs, and `network` the version the SDK could rely on: the lower of its own -version and the newest version a verified answer was signed at, 0 before the first one. -`pinned` says the SDK is pinned below `required`, so that no answer can lift the refusal. -When asking for the network's version failed, `cause` carries that error, and the JS and -FFI messages include it. - -Another node cannot help, so the SDK never retries the request elsewhere and bans no node -for it. An app that offers a protocol 14 feature can catch the error and hide the feature -until the network activates protocol 14; the same call succeeds once it has. When asking -for the network's version failed for a reason that can pass (nodes or a quorum source not -answering), the network may serve the request already: JS reports the error as retriable -(`isRetriable`), and FFI as a network error, so the app can try again later instead of -hiding the feature. A failure the SDK's own setup causes, no context provider or one -configured so that it cannot give quorum keys (a non-trusted EvoSDK), is not retriable: -Rust apps can tell the two apart with `Error::is_transient_protocol_version_refusal`. - -## At the activation - -When the network activates protocol 14, a node that has not caught up yet still answers -at protocol 13. An answer to one of these requests signed below 14 is treated as coming -from a stale node (`StaleNodeError::BehindProtocolUpgrade`): it is discarded, the node is -banned, and the SDK asks another one. On FFI this is a network error. - -The question the SDK asks for the network's version can reach such a node too. Its fresh -answer, signed at 13, then decides that call: the request is refused as if the network -had not activated protocol 14. The refusal is not retriable, but it lasts only that call: -the next one asks again, usually another node. Right after an activation, an app that -hides a feature on a refusal should look again soon. +Another node cannot help, so the SDK never retries the request elsewhere and bans no node for +it. An app that offers a protocol 14 feature can catch the error and hide the feature until the +network activates protocol 14; the same call succeeds once it has. A refusal of an SDK pinned +below 14 (`pinned`; the message says the SDK is pinned to an earlier protocol version) does not +lift when the network activates it: the app has to raise the pin, or let the SDK follow the +network. When asking for the +network's version failed for a reason that can pass (nodes or a quorum source not answering), +the network may serve the request already: JS reports the error as retriable (`isRetriable`), +and FFI as a network error, so the app can try again later instead of hiding the feature. A +failure the SDK's own setup causes, no context provider or one configured so that it cannot give +quorum keys (a non-trusted EvoSDK), is not retriable: Rust apps can tell the two apart with +`Error::is_transient_protocol_version_refusal`. + +Right after an activation, a node that has not caught up can still answer the SDK's question +for the network's version at 13. The refusal it causes lasts until that node's epoch can have +ended, which it already has or is about to; an app that hides a feature on a refusal should +look again soon. ## Adding a request A method or request field that only a newer release serves declares that release's first -protocol version in `impl_transport_request_grpc!` (`needs_protocol_version: ...`), and -gets a line in the inventory test's `ADDITIONS`. A feature a later release adds declares -that release's protocol version, not 14. A caller that executes a request directly -(`DapiRequestExecutor::execute`) skips the SDK's check and owns it. +protocol version in `impl_transport_request_grpc!` (`needs_protocol_version: ...`), and gets a +line in the inventory test's `ADDITIONS`. A feature a later release adds declares that release's +protocol version, not 14. + +A `getDocuments` field or operator a later release adds gets an `OptionalFeatureVersion` slot in +the version tables instead, `None` in every table before the protocol version that adds it +(backfilled into the shipped tables), and `DocumentQuery::required_protocol_version` reads it. +A new route the server opens is read from the slot the server switches on. Its inventory line +is `NoGate`, since the wire request declares nothing. + +A caller that executes a request directly (`DapiRequestExecutor::execute`) skips the SDK's check +and owns it. diff --git a/book/src/sdk/protocol-versions.md b/book/src/sdk/protocol-versions.md new file mode 100644 index 00000000000..affb45654de --- /dev/null +++ b/book/src/sdk/protocol-versions.md @@ -0,0 +1,185 @@ +# Protocol Versions in the SDK + +A Dash Platform network runs one protocol version at a time, and moves to the next only when +an epoch starts. Its nodes do not all run the same release: during an upgrade, nodes of the +previous release and of the new one answer side by side, and a node that has fallen behind +still answers at the version it last saw. The SDK has to send requests every node it may reach +serves, and read each answer with the rules of the version that produced it. This chapter +describes how it keeps track of versions and what it does with them. + +## The versions the SDK keeps + +| Version | What it is | Where it comes from | +|---|---|---| +| Stored | The version whose rules the SDK builds requests and transitions with (`Sdk::version`) | Seeded, initial or pinned at build time; raised by verified answers | +| Verified | The newest version a verified answer was signed at (`ProtocolVersionInfo::verified_version`) | Verified answers only; none before the first (`None`, and 0 in the network basis) | +| Network basis | The version the SDK may rely on both the network and itself serving (`QuerySettings::network_protocol_basis`) | The lower of the two above | + +The stored version starts at one of three values: + +- **Seeded.** `SdkBuilder` picks a per-network default (`min_protocol_version`), a version + every node of that network serves; it may be older than the version the network runs. +- **Initial.** `SdkBuilder::with_initial_version` starts it elsewhere, for example at a version + an app persisted from an earlier session. The SDK can tell the app each time the version + rises (`SdkBuilder::with_protocol_version_observer`). +- **Pinned.** `SdkBuilder::with_version` fixes it: no answer moves it. + +An unpinned stored version only ever rises. It rises to the version a verified answer was +signed at, when this build knows that version. An unpinned SDK on a network proven to run a +version newer than this build knows speaks the newest version it knows. + +None of the three values says what the network runs. A seed may be older than the network, a +persisted version may come from another network, and a pin may be ahead of it. Only the +verified version is evidence, which is why the network basis takes the lower of the two. + +## Learning from verified answers + +Every proved answer carries metadata: the block height and time, and the protocol version +that signed it. The quorum signature covers these fields together with the proof. Once the +proof and the signature verify, the SDK checks that the answer is fresh: its time within the +time tolerance of the local clock, and its height not too far below the highest height seen. +Only then does it learn from it, raising the verified version and the stored one. + +Nothing is learned from metadata that was not verified: an unproved answer, an answer whose +proof fails, or metadata a caller hands to `Sdk::verify_response_metadata`. +`Sdk::refresh_protocol_version` asks the network for a proved answer to learn from on purpose; +on an SDK that does not request proofs it does nothing, and only the probe described below +learns for it. + +## Request and response versions + +Most requests are versioned envelopes, `V0`, `V1` and so on, and so are their responses. A +version table (`drive_abci.query.`) lists which envelopes the nodes of each protocol +version accept. The SDK encodes a request in the envelope its stored version selects, so a +request built at a version the network does not run yet may not be understood. + +Where a newer envelope is refused by older nodes, the SDK picks it only from the network basis: +it sends the older envelope until a verified answer shows the network accepts the newer one. +Document requests are the exception: every network the SDK is seeded for serves their `V1` +envelope (protocol 12 and later), so the SDK sends `V1` from its stored version and judges each +document feature separately, as the next section describes. + +A request whose answer has a new shape gets a new envelope version rather than a new field in +an existing one. A node that does not know the new version refuses it, instead of answering +in the old shape. + +## Requests only newer nodes serve + +A protocol upgrade can add a method, a request field or a query feature. Nodes below the +version that adds it do not serve it: + +- a method they do not know is answered `UNIMPLEMENTED`; +- a field they do not know is dropped, and the node proves the request without it, which is + another question; +- a feature they do not route is refused, or served as another question. + +Several of these answers get an honest node banned, and none answers the request. So before +sending a request the SDK asks what it needs (`Query::protocol_requirement`): + +- for most requests, what the request declares (`TransportRequest::needs_protocol_version`); +- for a document query, what its features need (`DocumentQuery::required_protocol_version`), + read from the version tables: a feature slot for a field older nodes drop, or the slot the + server itself switches on for a route. + +The SDK then compares the requirement with the network basis: + +| The SDK | When | +|---|---| +| Sends | the network basis is at least the requirement, or the SDK is a mock, which has no nodes to ban | +| Refuses | it is pinned below the requirement, or has no context provider to verify an answer with, or a recent probe proved the network below it (see below) | +| Probes, then decides again | otherwise | + +The probe asks for the genesis epoch, with a proof whatever the SDK's proof setting, under the +caller's request settings. Every node serves it at every version, and its verified answer +raises the verified version like any other. An SDK that does not request proofs still asks for +this one, and bans no node for its answers: its context provider may not be able to give +quorum keys, and then the probe fails on every node, which says nothing about the nodes. + +Calls that need a probe while one runs, from the SDK or its clones, wait for its answer and +decide from it as the call that probed does, so one probe serves them all. They wait as long as +that probe takes under its own call's request settings. When it fails, or its call is dropped, +each of them asks under its own request settings at once. + +A refused request is not sent and bans no node. [Features from Protocol Version +14](protocol-version-gate.md) lists the requests and features this applies to. + +### How long a probe holds + +The protocol version changes only when an epoch starts, and epochs follow a fixed grid: epoch +`n` runs from `n` epoch lengths after the genesis block to `n + 1`, and starts with its first +block. When a probe shows the network below the requirement, the SDK also proves the current +epoch: it asks for the epoch the answer names as current and the epoch after it, and the proof +shows that the first has started and the second has not, and when the first started. + +The genesis epoch's start, which the probe proves, and the current epoch's start bound the +epoch length from above, so they tell when the current epoch has ended at the latest. Until +then the SDK refuses requests that need more than the version it saw, without asking again. +It measures the time on the local clock from when the answer arrived, and never past that +bound by block time, so neither a local clock behind block time nor an older answer lengthens +the refusal. The hold saves probes rather than promising one per epoch: one answer cannot tell +a local clock running ahead of block time from an answer that is old, so a clock ahead ends the +hold early by the difference, and the next request that needs more asks again. A hold too short +costs a query; one too long would refuse what the network serves. An SDK and its clones share +this. Any verified answer that shows the network at the requirement, or at a newer version than +the probe saw, lifts the refusal at once, whichever request it answered. + +The bound is late by about how long after its start on the grid the current epoch's first +block came, which is seconds while blocks are made, and longer after a halt across the epoch's +start. Then a request may be refused after the network has moved to a newer version, until the +next verified answer or the end of the hold. + +Nothing is held when the epoch cannot be proven: when the answer comes from the genesis epoch, +or when the epoch query fails or proves another epoch current. The next request that needs more +asks again. + +### At the activation + +When the network moves to a new version, a node that has not caught up still answers at the +old one. An answer to a request that needs the new version, signed below it, is treated as +coming from a stale node (`StaleNodeError::BehindProtocolUpgrade`): the request moves to +another node, and this one is banned. + +A probe can reach such a node too. Its answer, fresh and signed at the old version, can decide +that call: unless the epoch query that follows reaches an up-to-date node, the request is +refused as if the network had not moved. The epoch that node proves +has already ended or is about to, so the next request asks again, usually of another node. + +## Document answers + +Document queries (documents, counts, sums, averages and their split forms, ranked, having, +composite and chained queries) are verified at the version that signed them. An answer signed at a +newer version than the SDK's is verified with that version's rules when this build knows it, +and with the newest rules it knows otherwise. + +An answer signed at an older version than the SDK's stored version is not verified. What the +SDK does with it depends on what verified answers have shown: + +- **The network is proven at the SDK's version.** The node is behind its network. The answer + fails as a proof error: the request moves to another node, and this one is banned. +- **Nothing has shown it yet.** The SDK may be pinned, seeded or persisted ahead of its + network, where every node answers this way. The request moves to another node, and none is + banned (`StaleNodeError::ProtocolVersion`). A request whose every attempt ends this way fails + with `Error::Config`. An answer outside the height or time tolerance still fails as stale, and + its node is banned. + +Before a document query is sent, the result type checks that it can be verified from an answer +to it (`FromProof::check_request`), at the version the request is encoded with. A query no node +serves, or none of whose answers can be verified as that type, is refused without being sent. + +## Errors + +| Situation | Rust (`dash-sdk`) | JS (`wasm-sdk`, `evo-sdk`) | FFI (Swift, Kotlin) | Retried elsewhere / node banned | +|---|---|---|---|---| +| The network is not proven to serve what the request needs | `Error::ProtocolVersionNotActive { method, required, feature, network, pinned, cause }` | kind `ProtocolVersionNotActive` | `ProtocolError`, or `NetworkError` when learning the network's version failed for a reason that can pass | no / no | +| No node can answer the request as the requested result type: refused before sending, or a fresh verified answer cannot be returned as that type (several documents for a single-document fetch, a total that overflows) | `Error::InvalidRequest` | kind `InvalidArgument` | `InvalidParameter` | no / no | +| An answer to a request that needs a version, signed below it | `StaleNodeError::BehindProtocolUpgrade` | kind `StaleNode` | `NetworkError` | yes / yes | +| A document answer signed below the SDK's version, before the network is proven at it | `StaleNodeError::ProtocolVersion` per attempt; the request fails with `Error::Config` once no attempt is left | kind `Config` | `ProtocolError` | yes / no | +| A document answer signed below the SDK's version, once the network is proven at it | `Error::Proof` | kind `Proof` | `ProtocolError` (its message names both versions) | yes / yes | + +`ProtocolVersionNotActive` names the method and, for a document query, the feature that needs +the version, for example `get_documents needs Platform protocol version 14 for time-range +selections`. `network` is the network basis, `pinned` says no answer can lift the refusal, and +`cause` holds the error when learning the network's version failed. A refusal is transient only +when that failure can pass (`Error::is_transient_protocol_version_refusal`; JS `isRetriable`): +then the network may serve the request already. An app that offers a newer feature can catch +the refusal and hide the feature until the network serves it. diff --git a/packages/dash-platform-queries/src/documents/document_query.rs b/packages/dash-platform-queries/src/documents/document_query.rs index 3db536af9bb..616794b576a 100644 --- a/packages/dash-platform-queries/src/documents/document_query.rs +++ b/packages/dash-platform-queries/src/documents/document_query.rs @@ -565,6 +565,9 @@ impl DocumentQuery { /// specific [`PlatformVersion`] to pick V0 vs V1. The dispatch /// boundary is the document_query feature-version on the /// platform_version: `0` → V0, `1` → V1. + /// + /// A caller that sends the request itself owns checking + /// [`Self::required_protocol_version`] against the network first. pub fn try_into_request_for_version( self, platform_version: &PlatformVersion, @@ -825,8 +828,11 @@ pub(super) fn resolve_time_range_clauses_with_metadata_time( /// V0 lacks `selects` / `group_by` / `having` / `offset` and the /// optional-limit semantics — callers that set those features get /// `Error::Config` with a clear "requires Platform v3.1+" message -/// rather than a silently-truncated request. Time-range clauses are -/// additionally gated on the v14 contract grammar — see the `1 =>` arm. +/// rather than a silently-truncated request. +/// +/// The encoder decides nothing about which networks serve the query: +/// callers check [`DocumentQuery::required_protocol_version`] against +/// the network first. impl TryFromPlatformVersioned for GetDocumentsRequest { type Error = Error; @@ -866,38 +872,20 @@ impl TryFromPlatformVersioned for GetDocumentsRequest { "encoding GetDocumentsRequest" ); - // The elements of a typed array are first indexed at protocol version - // 14, the first whose contract grammar (document meta-schema - // generation 3) admits an index on a typed array. A server before it - // knows neither operator, in the page or in any sub-query, which - // rides the same wire mapping. - let reads_array_elements = where_clauses - .iter() - .chain( - sub_queries - .iter() - .flat_map(|sub_query| sub_query.where_clauses.iter()), - ) - .any(|clause| clause.operator.reads_array_elements()); - if reads_array_elements - && platform_version - .dpp - .contract_versions - .document_type_versions - .schema - .document_type_schema - < 3 - { - return Err(Error::Config(format!( - "contains and containsAny queries require protocol version 14+, the first \ - version whose contract grammar indexes the elements of a typed array; this \ - network runs protocol version {}", - platform_version.protocol_version - ))); - } - match feature_version { 0 => { + // The v0 CBOR where clauses have no operator that reads the elements of a + // typed array. + if where_clauses + .iter() + .any(|clause| clause.operator.reads_array_elements()) + { + return Err(Error::Config( + "contains and containsAny queries require the V1 documents wire; the \ + v0 getDocuments wire has no element operator" + .to_string(), + )); + } if !sub_queries.is_empty() { return Err(Error::Config( "composite document queries require the V1 documents wire (Platform v3.1+)" @@ -931,52 +919,21 @@ impl TryFromPlatformVersioned for GetDocumentsRequest { &having, ) } - 1 => { - // The v1 wire predates time-range indexes: protocol - // versions 12 and 13 also serve it, but their contract - // grammar (document meta-schema generations 1 and 2) - // cannot host a `timeRange` index. Gate on the grammar - // generation — the same table the server's parser reads — - // rather than emitting an operator a pre-v14 server - // rejects as an unknown discriminant. - let grammar_generation = platform_version - .dpp - .contract_versions - .document_type_versions - .schema - .document_type_schema; - if !time_range_clauses.is_empty() && grammar_generation < 3 { - return Err(Error::Config(format!( - "time range (IN_TIME_RANGE) queries require protocol version 14+ — the \ - first version whose contract grammar hosts `timeRange` indexes; this \ - network runs protocol version {}", - platform_version.protocol_version - ))); - } - if !integer_range_clauses.is_empty() && grammar_generation < 3 { - return Err(Error::Config(format!( - "integer range (IN_INTEGER_RANGE) queries require protocol version 14+ \ - — the first version whose contract grammar hosts `integerRange` \ - indexes; this network runs protocol version {}", - platform_version.protocol_version - ))); - } - encode_v1( - data_contract.id().to_vec(), - document_type_name, - where_clauses, - time_range_clauses, - integer_range_clauses, - order_by_clauses, - limit, - offset, - start, - select, - group_by, - having, - sub_queries_to_proto(sub_queries)?, - ) - } + 1 => encode_v1( + data_contract.id().to_vec(), + document_type_name, + where_clauses, + time_range_clauses, + integer_range_clauses, + order_by_clauses, + limit, + offset, + start, + select, + group_by, + having, + sub_queries_to_proto(sub_queries)?, + ), n => Err(Error::Config(format!( "GetDocumentsRequest wire encoder does not support feature_version={n} \ (drive_abci.query.document_query) on PlatformVersion v{}", @@ -1659,18 +1616,17 @@ fn value_to_proto_at_depth(value: Value, depth: u8) -> Result Arc { @@ -1698,22 +1654,36 @@ mod encode_version_gate_tests { .with_time_range("$createdAt", TimeRangeSelector::Newest) } + /// The protocol versions whose documents wire is `version`. + fn versions_with_wire(version: u16) -> impl Iterator { + PLATFORM_VERSIONS.iter().filter(move |platform_version| { + platform_version + .drive_abci + .query + .document_query + .default_current_version + == version + }) + } + #[test] - fn a_time_range_query_refuses_to_encode_for_protocol_version_13() { - let platform_version = PlatformVersion::get(13).expect("protocol version 13 exists"); - let error = GetDocumentsRequest::try_from_platform_versioned( - newest_time_range_query(), - platform_version, - ) - .expect_err("protocol version 13's contract grammar has no timeRange indexes"); - let message = match error { - Error::Config(message) => message, - other => panic!("expected Error::Config, got {other:?}"), - }; - assert!( - message.contains("protocol version 14"), - "the refusal must name the real version floor, got: {message}" - ); + fn should_encode_a_time_range_query_on_the_v1_wire_whatever_the_version() { + for platform_version in versions_with_wire(1) { + let request = GetDocumentsRequest::try_from_platform_versioned( + newest_time_range_query(), + platform_version, + ) + .expect("the V1 wire carries time-range selections"); + let Some(V1(v1)) = request.version else { + panic!("protocol {}: V1", platform_version.protocol_version) + }; + assert_eq!( + v1.where_clauses[0].operator, + ProtoWhereOperator::InTimeRange as i32, + "protocol {}", + platform_version.protocol_version + ); + } } #[test] @@ -1754,35 +1724,18 @@ mod encode_version_gate_tests { }) } + /// Element operators ride the V1 wire at every version with it, in the page and in a + /// sub-query alike; the V0 wire's CBOR clauses have no element operator. #[test] - fn should_refuse_to_encode_an_element_query_for_protocol_version_13() { - let platform_version = PlatformVersion::get(13).expect("protocol version 13 exists"); - let error = - GetDocumentsRequest::try_from_platform_versioned(contains_query(), platform_version) - .expect_err("protocol version 13's contract grammar indexes no typed array"); - let message = match error { - Error::Config(message) => message, - other => panic!("expected Error::Config, got {other:?}"), - }; - assert!( - message.contains("protocol version 14"), - "the refusal must name the real version floor, got: {message}" - ); - } - - /// A sub-query's clauses ride the same wire mapping as the page's, so an - /// element operator only there is refused before protocol version 14 too. - #[test] - fn should_refuse_to_encode_an_element_sub_query_for_protocol_version_13() { + fn should_encode_element_queries_on_the_v1_wire_and_refuse_them_on_v0() { use super::super::composite_document_query::CompositeSubQueryKind; - let platform_version = PlatformVersion::get(13).expect("protocol version 13 exists"); - for operator in [WhereOperator::Contains, WhereOperator::ContainsAny] { + let in_sub_query = |operator: WhereOperator| { let value = match operator { WhereOperator::ContainsAny => platform_value!(["dash"]), _ => platform_value!("dash"), }; - let query = DocumentQuery::new(post_contract(), "post") + DocumentQuery::new(post_contract(), "post") .expect("the fixture has this document type") .with_limit(10) .with_sub_query(CompositeSubQuery { @@ -1797,17 +1750,50 @@ mod encode_version_gate_tests { order_by_clauses: vec![], limit: Some(10), binding: None, - }); - let error = GetDocumentsRequest::try_from_platform_versioned(query, platform_version) - .expect_err("protocol version 13's contract grammar indexes no typed array"); - let message = match error { - Error::Config(message) => message, - other => panic!("expected Error::Config, got {other:?}"), + }) + }; + for platform_version in versions_with_wire(1) { + let page = GetDocumentsRequest::try_from_platform_versioned( + contains_query(), + platform_version, + ) + .expect("the V1 wire carries element operators"); + let Some(V1(page)) = page.version else { + panic!("V1") }; + assert_eq!( + page.where_clauses[0].operator, + ProtoWhereOperator::Contains as i32 + ); + for (operator, expected) in [ + (WhereOperator::Contains, ProtoWhereOperator::Contains), + (WhereOperator::ContainsAny, ProtoWhereOperator::ContainsAny), + ] { + let request = GetDocumentsRequest::try_from_platform_versioned( + in_sub_query(operator), + platform_version, + ) + .expect("the V1 wire carries element operators"); + let Some(V1(v1)) = request.version else { + panic!("V1") + }; + assert_eq!( + v1.sub_queries[0].where_clauses[0].operator, expected as i32, + "protocol {}", + platform_version.protocol_version + ); + } + } + for platform_version in versions_with_wire(0) { + let error = GetDocumentsRequest::try_from_platform_versioned( + contains_query(), + platform_version, + ) + .expect_err("the V0 wire has no element operator"); assert!( - message.contains("contains and containsAny queries require protocol version 14"), - "{operator:?} in a sub-query must be refused with the version floor, got: \ - {message}" + matches!(&error, Error::Config(message) if message.contains("V1 documents wire")), + "protocol {}: {error:?}", + platform_version.protocol_version ); } } @@ -1881,6 +1867,6 @@ mod encode_version_gate_tests { let query = DocumentQuery::new(post_contract(), "post") .expect("the fixture has this document type"); GetDocumentsRequest::try_from_platform_versioned(query, platform_version) - .expect("the gate only refuses queries that carry a time-range selection"); + .expect("a plain query encodes at every version"); } } diff --git a/packages/dash-platform-queries/src/documents/mod.rs b/packages/dash-platform-queries/src/documents/mod.rs index 9edd4cb2316..560799913ec 100644 --- a/packages/dash-platform-queries/src/documents/mod.rs +++ b/packages/dash-platform-queries/src/documents/mod.rs @@ -17,4 +17,5 @@ pub mod document_split_sums; pub mod document_sum; pub(crate) mod having_proof_helpers; pub(crate) mod ranked_proof_helpers; +pub mod requirement; pub(crate) mod sum_proof_helpers; diff --git a/packages/dash-platform-queries/src/documents/requirement.rs b/packages/dash-platform-queries/src/documents/requirement.rs new file mode 100644 index 00000000000..e63a668bcb9 --- /dev/null +++ b/packages/dash-platform-queries/src/documents/requirement.rs @@ -0,0 +1,621 @@ +//! The protocol version from which every node serves what a document request uses. +//! +//! A protocol upgrade adds document features in two ways: a request field or operator that +//! nodes of an older release do not know, or a route the server opens behind a version table +//! slot. Below the version that adds it, a node drops the field and proves another question, +//! refuses the request, or serves it as another one. A caller compares +//! [`DocumentQuery::required_protocol_version`] with the protocol version the network is +//! proven to run before it sends the request. +//! +//! Each feature reads the version table slot that decides it, never a literal: the slot the +//! server switches on, or, for a field nodes of an older release drop, the feature slot that +//! is `None` before the protocol version that adds the field. + +use crate::documents::chained_document_query::ChainedDocumentQuery; +use crate::documents::document_query::DocumentQuery; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV2Getters; +use dpp::version::{PlatformVersion, ProtocolVersion, PLATFORM_VERSIONS}; +use drive::query::drive_document_ranked_query::mode_detection::ranked_order_key; +use drive::query::{SelectFunction, WhereOperator}; +use std::collections::BTreeSet; + +/// The protocol version from which every node serves a document request, and the feature of +/// the request that needs it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct RequiredProtocolVersion { + /// The first protocol version at which every node serves the request. + pub protocol_version: ProtocolVersion, + /// The feature that needs it, named for the caller, for example "time-range selections". + pub feature: &'static str, +} + +/// Declares [`Feature`] and `Feature::ALL` from one list, so that every feature is in `ALL` +/// and the tests over it cover each. +macro_rules! features { + ($($feature:ident,)*) => { + /// A document request feature that nodes below some protocol version do not serve. + #[derive(Debug, Clone, Copy, PartialEq, Eq)] + enum Feature { + $($feature,)* + } + + impl Feature { + #[cfg(test)] + const ALL: &'static [Self] = &[$(Self::$feature,)*]; + } + }; +} + +features! { + SubQueries, + Chained, + ArrayElements, + TimeRanges, + IntegerRanges, + Ranked, + HavingRange, + SeveralIns, + CursorOnSummedDocuments, +} + +impl Feature { + fn name(self) -> &'static str { + match self { + Self::SubQueries => "composite sub-queries", + Self::Chained => "chained queries", + Self::ArrayElements => "contains and containsAny clauses", + Self::TimeRanges => "time-range selections", + Self::IntegerRanges => "integer-range selections", + Self::Ranked => "aggregates ordered by the selected aggregate", + Self::HavingRange => "having clauses", + Self::SeveralIns => "several in clauses", + Self::CursorOnSummedDocuments => "start cursors on a type whose documents are summed", + } + } + + /// Whether every node that signs `platform_version` serves the feature. + fn served_at(self, platform_version: &PlatformVersion) -> bool { + let query = &platform_version.drive_abci.query; + let drive_query = &platform_version.drive.methods.document.query; + match self { + // Nodes of a release without the field drop it. + Self::SubQueries => query.document_sub_queries.is_some(), + Self::Chained => query.chained_document_query.is_some(), + // No index keys the elements of a typed array, or a time or integer window, before + // contract grammar generation 3, and nodes refuse the operators until then. + Self::ArrayElements | Self::TimeRanges | Self::IntegerRanges => { + platform_version + .dpp + .contract_versions + .document_type_versions + .schema + .document_type_schema + >= 3 + } + // The server's own routing: below these helper versions it serves a ranked + // request as a plain grouped aggregate or refuses it, and refuses every having. + Self::Ranked => { + query + .document_query_helpers + .compute_aggregate_mode_and_check_limit + >= 1 + } + Self::HavingRange => { + query + .document_query_helpers + .compute_aggregate_mode_and_check_limit + >= 2 + } + // Grouping v0 refuses more than one `In` clause. + Self::SeveralIns => drive_query.where_clause_grouping >= 1, + // Cursor reading v0 refuses the sum items such documents are stored as, with an + // error clients retry on every other node. + Self::CursorOnSummedDocuments => drive_query.start_at_document >= 1, + } + } + + /// The first protocol version from which every node serves the feature, or + /// [`ProtocolVersion::MAX`] when no protocol version this build knows serves it. + fn first_protocol_version(self) -> ProtocolVersion { + PLATFORM_VERSIONS + .iter() + .find(|platform_version| self.served_at(platform_version)) + .map_or(ProtocolVersion::MAX, |platform_version| { + platform_version.protocol_version + }) + } +} + +/// The most demanding of `features`, or `None` when there are none. +fn requirement(features: impl IntoIterator) -> Option { + features + .into_iter() + .map(|feature| RequiredProtocolVersion { + protocol_version: feature.first_protocol_version(), + feature: feature.name(), + }) + .max_by_key(|requirement| requirement.protocol_version) +} + +impl DocumentQuery { + /// The protocol version from which every node serves the features this query uses, or + /// `None` when every node serves it. + /// + /// It does not depend on the protocol version the query is encoded with: it says which + /// networks can answer the query, not how to encode it. + pub fn required_protocol_version(&self) -> Option { + requirement(self.features()) + } + + fn features(&self) -> Vec { + let mut features = Vec::new(); + if !self.sub_queries.is_empty() { + features.push(Feature::SubQueries); + } + if self + .where_clauses + .iter() + .chain( + self.sub_queries + .iter() + .flat_map(|sub_query| sub_query.where_clauses.iter()), + ) + .any(|clause| clause.operator.reads_array_elements()) + { + features.push(Feature::ArrayElements); + } + if !self.time_range_clauses.is_empty() { + features.push(Feature::TimeRanges); + } + if !self.integer_range_clauses.is_empty() { + features.push(Feature::IntegerRanges); + } + if self.select.function == SelectFunction::Documents { + // A documents select is served `In` clauses on several fields from grouping v1; a + // second `In` on one field is refused at every version. No version serves several + // `In` clauses in a count, sum or average, and the ranked and having routes that + // serve some need 14 for their own feature. + let fields_with_ins: BTreeSet<&str> = self + .where_clauses + .iter() + .filter(|clause| clause.operator == WhereOperator::In && !clause.is_identifier()) + .map(|clause| clause.field.as_str()) + .collect(); + if fields_with_ins.len() > 1 { + features.push(Feature::SeveralIns); + } + if self.start.is_some() && self.documents_are_summed() { + features.push(Feature::CursorOnSummedDocuments); + } + } else if !self.group_by.is_empty() { + if let [only] = self.order_by_clauses.as_slice() { + if only.field == ranked_order_key(&self.select) { + features.push(Feature::Ranked); + } + } + if !self.having.is_empty() { + features.push(Feature::HavingRange); + } + } + features + } + + /// Whether the queried document type sums its documents, which stores them as sum items. + fn documents_are_summed(&self) -> bool { + self.data_contract + .document_type_for_name(&self.document_type_name) + .is_ok_and(|document_type| document_type.documents_summable().is_some()) + } +} + +impl ChainedDocumentQuery { + /// The protocol version from which every node serves the join and the features its inner + /// query uses. + pub fn required_protocol_version(&self) -> Option { + requirement(std::iter::once(Feature::Chained).chain(self.inner.features())) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::documents::composite_document_query::CompositeSubQuery; + use crate::documents::document_query::RankingDirection; + use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v0::Start; + use dpp::data_contract::{DataContract, DataContractFactory}; + use dpp::platform_value::{platform_value, Identifier, Value}; + use dpp::tests::fixtures::get_data_contract_fixture; + use drive::query::{ + DriveDocumentQuery, HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, + HavingRightOperand, OrderClause, SelectProjection, TimeRangeSelector, WhereClause, + }; + use drive_proof_verifier::{ + DocumentAverage, DocumentCount, DocumentSplitCounts, DocumentSum, FromProof, + }; + use std::sync::Arc; + + fn contract() -> Arc { + Arc::new( + get_data_contract_fixture(None, 0, PlatformVersion::latest().protocol_version) + .data_contract_owned(), + ) + } + + fn query() -> DocumentQuery { + DocumentQuery::new(contract(), "niceDocument").expect("the fixture has this document type") + } + + /// A `tip` summed by `amount`. + fn tips() -> DocumentQuery { + let schemas = platform_value!({"tip": { + "type": "object", + "documentsSummable": "amount", + "properties": { + "recipient": {"type": "string", "minLength": 1, "maxLength": 63, "position": 0}, + "amount": {"type": "integer", "minimum": 0, "maximum": 4294967295u64, "position": 1} + }, + "required": ["recipient", "amount"], + "indices": [ + {"name": "byRecipient", "properties": [{"recipient": "asc"}], "summable": "amount"} + ], + "additionalProperties": false + }}); + let contract = DataContractFactory::new(PlatformVersion::latest().protocol_version) + .expect("factory") + .create_with_value_config(Identifier::new([8u8; 32]), 0, schemas, None, None) + .expect("the tip contract is well-formed") + .data_contract_owned(); + DocumentQuery::new(Arc::new(contract), "tip").expect("the contract has this type") + } + + fn clause(field: &str, operator: WhereOperator, value: Value) -> WhereClause { + WhereClause { + field: field.to_string(), + operator, + value, + } + } + + fn having() -> Vec { + vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(1)), + }] + } + + fn sub_query() -> CompositeSubQuery { + CompositeSubQuery::documents(contract(), "niceDocument") + .expect("the fixture has this document type") + .with_limit(10) + } + + /// Whether a protocol version's tables serve a feature. + type Served = fn(&PlatformVersion) -> bool; + + /// The first protocol version at which `served` holds, read from the tables. + fn first_where(served: impl Fn(&PlatformVersion) -> bool) -> ProtocolVersion { + PLATFORM_VERSIONS + .iter() + .find(|platform_version| served(platform_version)) + .expect("some known version serves it") + .protocol_version + } + + fn required(query: &DocumentQuery) -> Option<(ProtocolVersion, &'static str)> { + query + .required_protocol_version() + .map(|requirement| (requirement.protocol_version, requirement.feature)) + } + + /// Queries built only from what every node serves need nothing. + #[test] + fn should_require_nothing_of_queries_every_node_serves() { + let ins = |field: &str| clause(field, WhereOperator::In, platform_value!(["a", "b"])); + for query in [ + query(), + query().with_where(ins("$ownerId")), + query().with_where(ins("$id")).with_where(ins("$ownerId")), + query().with_select(SelectProjection::count_star()), + query() + .with_select(SelectProjection::count_star()) + .with_where(ins("restaurantId")) + .with_group_by("restaurantId"), + // Ordered by the grouped property, not by the aggregate: a grouped count. + query() + .with_select(SelectProjection::count_star()) + .with_group_by("restaurantId") + .with_order_by(OrderClause { + field: "restaurantId".to_string(), + ascending: true, + }), + tips(), + tips().with_select(SelectProjection::sum("amount")), + ] { + assert_eq!(required(&query), None, "{query:?}"); + } + } + + /// Several `In` clauses are served only on distinct fields. A second `In` on one field is + /// refused at every version: grouping v0 refuses any second `In`, and v1 one on a field an + /// `In` already constrains, as the verifier's own conversion does. So it is no feature a + /// version serves, and needs none. + #[test] + fn should_require_no_version_of_several_in_clauses_on_one_field() { + let one_field = query() + .with_where(clause( + "restaurantId", + WhereOperator::In, + platform_value!(["a", "b"]), + )) + .with_where(clause( + "restaurantId", + WhereOperator::In, + platform_value!(["c", "d"]), + )); + assert_eq!(required(&one_field), None); + let error = DriveDocumentQuery::try_from(&one_field) + .expect_err("the latest grammar refuses a second in clause on one field"); + assert!( + error.to_string().contains("in clause has same field"), + "{error}" + ); + } + + /// Several `In` clauses are a feature of a documents select. Drive refuses them in a count, + /// sum or average at every protocol version: the grouping refuses them before 14, and the + /// aggregate surfaces still do from 14. No version serves such a query, so it needs none, + /// and the view's check refuses this shape of it before sending at any version instead. + #[test] + fn should_require_no_version_of_an_aggregate_with_several_ins_and_refuse_it_at_any() { + type Check = + fn(&DocumentQuery, &PlatformVersion) -> Result<(), drive_proof_verifier::Error>; + let ins = |query: DocumentQuery, first: &str, second: &str| { + query + .with_where(clause( + first, + WhereOperator::In, + platform_value!(["a", "b"]), + )) + .with_where(clause(second, WhereOperator::In, platform_value!([1, 2]))) + }; + let views: [(&str, Check, DocumentQuery); 4] = [ + ( + "DocumentCount", + >::check_request, + ins( + query().with_select(SelectProjection::count_star()), + "restaurantId", + "grade", + ), + ), + ( + "DocumentSplitCounts", + >::check_request, + ins( + query() + .with_select(SelectProjection::count_star()) + .with_group_by("restaurantId"), + "restaurantId", + "grade", + ), + ), + ( + "DocumentSum", + >::check_request, + ins( + tips().with_select(SelectProjection::sum("amount")), + "recipient", + "amount", + ), + ), + ( + "DocumentAverage", + >::check_request, + ins( + tips().with_select(SelectProjection::avg("amount")), + "recipient", + "amount", + ), + ), + ]; + for (view, check, query) in views { + assert_eq!(required(&query), None, "{view}"); + for platform_version in PLATFORM_VERSIONS { + match check(&query, platform_version) { + Err(drive_proof_verifier::Error::RequestError { error }) + if error.contains("multiple in clauses") => {} + other => panic!( + "{view} at {}: expected a refusal of several in clauses, got {other:?}", + platform_version.protocol_version + ), + } + } + } + } + + /// Each feature is detected and needs the first protocol version whose table serves it, + /// whichever slot that is. + #[test] + fn should_require_the_first_version_whose_table_serves_each_feature() { + let ranked = query() + .with_select(SelectProjection::count_star()) + .with_group_by("restaurantId") + .order_by_selected_aggregate(RankingDirection::Descending) + .with_limit(5); + let start = |mut query: DocumentQuery| { + query.start = Some(Start::StartAfter(vec![1; 32])); + query + }; + let cases: Vec<(&str, DocumentQuery, Served)> = vec![ + ( + "composite sub-queries", + query().with_limit(10).with_sub_query(sub_query()), + |v| v.drive_abci.query.document_sub_queries.is_some(), + ), + ( + "contains and containsAny clauses", + query().with_where(clause( + "tags", + WhereOperator::Contains, + Value::Text("dash".into()), + )), + |v| { + v.dpp + .contract_versions + .document_type_versions + .schema + .document_type_schema + >= 3 + }, + ), + ( + "contains and containsAny clauses", + query() + .with_limit(10) + .with_sub_query(sub_query().with_where(clause( + "tags", + WhereOperator::ContainsAny, + platform_value!(["dash"]), + ))), + |v| { + v.dpp + .contract_versions + .document_type_versions + .schema + .document_type_schema + >= 3 + }, + ), + ( + "time-range selections", + query().with_time_range("$createdAt", TimeRangeSelector::Newest), + |v| { + v.dpp + .contract_versions + .document_type_versions + .schema + .document_type_schema + >= 3 + }, + ), + ( + "integer-range selections", + query().with_integer_range("price", Value::U64(600)), + |v| { + v.dpp + .contract_versions + .document_type_versions + .schema + .document_type_schema + >= 3 + }, + ), + ( + "aggregates ordered by the selected aggregate", + ranked, + |v| { + v.drive_abci + .query + .document_query_helpers + .compute_aggregate_mode_and_check_limit + >= 1 + }, + ), + ( + "having clauses", + query() + .with_select(SelectProjection::count_star()) + .with_group_by("hashtag") + .with_having(having()), + |v| { + v.drive_abci + .query + .document_query_helpers + .compute_aggregate_mode_and_check_limit + >= 2 + }, + ), + ( + "several in clauses", + query() + .with_where(clause( + "restaurantId", + WhereOperator::In, + platform_value!(["a"]), + )) + .with_where(clause("grade", WhereOperator::In, platform_value!([1, 2]))), + |v| v.drive.methods.document.query.where_clause_grouping >= 1, + ), + ( + "start cursors on a type whose documents are summed", + start(tips()), + |v| v.drive.methods.document.query.start_at_document >= 1, + ), + ]; + for (feature, query, served) in cases { + assert_eq!( + required(&query), + Some((first_where(served), feature)), + "{query:?}" + ); + } + // A cursor on a type whose documents are not summed is read by every node. + assert_eq!(required(&start(query())), None); + } + + /// The join needs its own version, and the query needs the most demanding of its + /// features, in the page and in the chained inner query alike. + #[test] + fn should_require_the_most_demanding_feature_of_a_query() { + let chained = ChainedDocumentQuery::new(query().with_limit(10), "ownerId", "niceDocument"); + assert_eq!( + chained + .required_protocol_version() + .map(|requirement| (requirement.protocol_version, requirement.feature)), + Some(( + first_where(|v| v.drive_abci.query.chained_document_query.is_some()), + "chained queries" + )) + ); + let several = query() + .with_limit(10) + .with_sub_query(sub_query()) + .with_time_range("$createdAt", TimeRangeSelector::Newest); + let most_demanding = [Feature::SubQueries, Feature::TimeRanges] + .into_iter() + .map(Feature::first_protocol_version) + .max(); + assert_eq!( + several + .required_protocol_version() + .map(|requirement| requirement.protocol_version), + most_demanding + ); + } + + /// Once a protocol version serves a feature, every later one does: the requirement is + /// the first version, so a feature withdrawn later would be sent to nodes that refuse it. + #[test] + fn should_serve_each_feature_at_every_version_from_its_first() { + for &feature in Feature::ALL { + let first = feature.first_protocol_version(); + assert!( + first <= PlatformVersion::latest().protocol_version, + "{feature:?} is served by no known version" + ); + for platform_version in PLATFORM_VERSIONS { + assert_eq!( + feature.served_at(platform_version), + platform_version.protocol_version >= first, + "{feature:?} at {}", + platform_version.protocol_version + ); + } + } + } +} diff --git a/packages/js-evo-sdk/README.md b/packages/js-evo-sdk/README.md index 661b7e236e5..1b189a03df8 100644 --- a/packages/js-evo-sdk/README.md +++ b/packages/js-evo-sdk/README.md @@ -14,6 +14,7 @@ Evo SDK provides a high-level, strongly-typed interface for interacting with [Da - [Install](#install) - [Usage](#usage) - [Facades](#facades) +- [Document queries that need protocol version 14](#document-queries-that-need-protocol-version-14) - [Ranked queries](#ranked-queries) - [Querying the elements of a list (`contains`, `containsAny`)](#querying-the-elements-of-a-list-contains-containsany) - [Document references (`refersTo`)](#document-references-refersto) @@ -115,6 +116,21 @@ The SDK organises its API into domain-specific facades, each accessible as a pro A `wallet` namespace is also exported with utilities for BIP39 mnemonic generation and validation, BIP44/DIP9/DIP13 key derivation (path helpers included), extended-key conversion (`xprvToXpub`, `deriveChildPublicKey`), key-pair generation and import (`generateKeyPair`, `keyPairFromWif`, `keyPairFromHex`), public-key-to-address conversion, address validation, message signing, and Dashpay contact-key derivation. See [`src/wallet/functions.ts`](src/wallet/functions.ts) for the full list. +## Document queries that need protocol version 14 + +Ranked and `having` queries, `contains` and `containsAny`, `timeRange` and `integerRange` selections, chained and composite queries, several `in` clauses in one documents query, and a `startAt` / `startAfter` cursor on a document type whose documents are summed are served from protocol version 14. Until a verified answer shows the network running it, the SDK refuses such a query without sending it, with a `ProtocolVersionNotActive` error naming the feature, and no node is banned: + +```ts +try { + return await sdk.documents.ranked(query); +} catch (e) { + if (e.name !== 'ProtocolVersionNotActive' || e.isRetriable) throw e; + // The network does not serve ranked queries yet: hide the leaderboard. +} +``` + +A fresh SDK asks the network for its version once before the first such query, so it does not have to be seeded at the network's version. An SDK pinned below 14 keeps refusing these queries after the network activates 14 (the message says it is pinned): raise the pin. A query that cannot be answered as asked, for example `groupBy` on `sdk.documents.query`, which returns the documents themselves, is refused before sending with an `InvalidArgument` error. See [Protocol Versions in the SDK](../../book/src/sdk/protocol-versions.md). + ## Ranked queries From protocol version 14, a contract index can declare `rankedCountable`, `rankedSummable` or `rankedAverageable`. Against such an index the SDK can answer "which groups score highest?" with a proof, in `O(log n + k)`, without walking every group: diff --git a/packages/rs-dapi-client/src/transport.rs b/packages/rs-dapi-client/src/transport.rs index 0e8cb7507ef..2f01af52e50 100644 --- a/packages/rs-dapi-client/src/transport.rs +++ b/packages/rs-dapi-client/src/transport.rs @@ -95,6 +95,10 @@ pub trait TransportRequest: Clone + Send + Sync + Debug + Mockable { /// /// The SDK's fetch paths refuse such a request until the network has proven that /// version; a caller that executes the request directly owns the check. + /// + /// A `GetDocumentsRequest` declares nothing: which networks serve it depends on the + /// features of the query it was encoded from, which `DocumentQuery::required_protocol_version` + /// (in `dash-platform-queries`) reads from the version tables before the query is encoded. fn needs_protocol_version(&self) -> Option { None } diff --git a/packages/rs-dapi-client/src/transport/grpc.rs b/packages/rs-dapi-client/src/transport/grpc.rs index 73a66302046..23b2b9fe042 100644 --- a/packages/rs-dapi-client/src/transport/grpc.rs +++ b/packages/rs-dapi-client/src/transport/grpc.rs @@ -325,21 +325,6 @@ const PROTOCOL_VERSION_OF_5_0: u32 = 14; /// or newer, and serves the methods and request fields 5.1 added. const PROTOCOL_VERSION_OF_5_1: u32 = 15; -/// A 4.x node drops a composite query's `sub_queries` and a chained query's `chained`, and -/// proves the plain page or the inner query instead. -fn documents_request_needs_protocol_version( - request: &platform_proto::GetDocumentsRequest, -) -> Option { - match &request.version { - Some(platform_proto::get_documents_request::Version::V1(v1)) - if v1.chained.is_some() || !v1.sub_queries.is_empty() => - { - Some(PROTOCOL_VERSION_OF_5_0) - } - _ => None, - } -} - /// A shielded pool request names a token's pool with `token_id`; a 4.x node drops it and /// answers for the credit pool instead. fn token_pool(token_id: &Option>) -> Option { @@ -366,8 +351,10 @@ impl_transport_request_grpc!( get_identity ); +// Which networks serve a document request depends on the query it was encoded from, so the +// SDK decides it before encoding (`DocumentQuery::required_protocol_version` in +// `dash-platform-queries`), and the wire request declares nothing. impl_transport_request_grpc!( - needs_protocol_version: documents_request_needs_protocol_version; platform_proto::GetDocumentsRequest, platform_proto::GetDocumentsResponse, PlatformGrpcClient, diff --git a/packages/rs-dapi-client/tests/request_inventory.rs b/packages/rs-dapi-client/tests/request_inventory.rs index f860a504619..e03f948ec89 100644 --- a/packages/rs-dapi-client/tests/request_inventory.rs +++ b/packages/rs-dapi-client/tests/request_inventory.rs @@ -4,7 +4,10 @@ //! different question, and refuses a method, oneof arm or enum value it does not know with //! UNIMPLEMENTED or UNKNOWN. Either answer gets an honest node banned, so a request using //! such a feature must declare the protocol version from which every node serves it -//! (`TransportRequest::needs_protocol_version`). +//! (`TransportRequest::needs_protocol_version`). A document request is the exception: the +//! SDK decides it from the query it encodes the request from +//! (`DocumentQuery::required_protocol_version` in `dash-platform-queries`, from the version +//! tables), so its additions are `NoGate` here and must get a feature slot there. //! //! `request_inventory/v4.1.1.txt` lists what 4.1 nodes understand: every method and every //! field, oneof arm and enum value reachable from a request in v4.1.1's `platform.proto`. @@ -50,14 +53,16 @@ const ADDITIONS: &[(&str, Verdict)] = &[ ("rpc getYesNoVotePollState GetYesNoVotePollStateRequest", Verdict::Gate(15)), ("rpc getMasternodeYesNoVote GetMasternodeYesNoVoteRequest", Verdict::Gate(15)), ("rpc getPrivateDocumentEntries GetPrivateDocumentEntriesRequest", Verdict::Gate(15)), - // A 4.1 node drops both and proves the plain page or the inner query instead. + // A 4.1 node drops both and proves the plain page or the inner query instead. The SDK + // refuses them before protocol 14 from the query they are encoded from, which reads its + // version tables; the wire request declares nothing. ( "field GetDocumentsRequest.GetDocumentsRequestV1.chained = 13 GetDocumentsRequest.GetDocumentsRequestV1.ChainedJoin", - Verdict::Gate(14), + Verdict::NoGate("decided by DocumentQuery::required_protocol_version before encoding"), ), ( "field GetDocumentsRequest.GetDocumentsRequestV1.sub_queries = 14 repeated GetDocumentsRequest.GetDocumentsRequestV1.SubQuery", - Verdict::Gate(14), + Verdict::NoGate("decided by DocumentQuery::required_protocol_version before encoding"), ), // A 4.1 node drops `token_id` and proves the credit pool instead. ("field GetShieldedEncryptedNotesRequest.GetShieldedEncryptedNotesRequestV0.token_id = 4 bytes", Verdict::Gate(14)), @@ -68,27 +73,27 @@ const ADDITIONS: &[(&str, Verdict)] = &[ ("field GetShieldedNullifiersRequest.GetShieldedNullifiersRequestV0.token_id = 3 bytes", Verdict::Gate(14)), ( "field GetDocumentsRequest.WhereClause.time_range = 4 GetDocumentsRequest.TimeRangeSelection", - Verdict::NoGate("sent only with the IN_TIME_RANGE operator, which every node before protocol 14 refuses with INVALID_ARGUMENT"), + Verdict::NoGate("sent only with the IN_TIME_RANGE operator, which every node before protocol 14 refuses with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "field GetDocumentsRequest.WhereClause.integer_range = 5 GetDocumentsRequest.IntegerRangeSelection", - Verdict::NoGate("sent only with the IN_INTEGER_RANGE operator, which every node before protocol 14 refuses with INVALID_ARGUMENT"), + Verdict::NoGate("sent only with the IN_INTEGER_RANGE operator, which every node before protocol 14 refuses with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "enum GetDocumentsRequest.WhereOperator.IN_TIME_RANGE = 11", - Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT"), + Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "enum GetDocumentsRequest.WhereOperator.IN_INTEGER_RANGE = 12", - Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT"), + Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "enum GetDocumentsRequest.WhereOperator.CONTAINS = 13", - Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT, and the SDK checks it against its own version"), + Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "enum GetDocumentsRequest.WhereOperator.CONTAINS_ANY = 14", - Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT, and the SDK checks it against its own version"), + Verdict::NoGate("every node before protocol 14 refuses it with INVALID_ARGUMENT; the SDK refuses it before then from the query"), ), ( "field GetIdentityByNonUniquePublicKeyHashRequest.v1 = 2 GetIdentityByNonUniquePublicKeyHashRequest.GetIdentityByNonUniquePublicKeyHashRequestV1", @@ -982,9 +987,10 @@ fn hex(bytes: &[u8]) -> String { bytes.iter().map(|byte| format!("{byte:02x}")).collect() } -/// The shapes a 4.1 node answers differently, spelled out. +/// The shapes a 4.1 node answers differently, spelled out. A document request declares +/// nothing whatever it carries: the SDK decides from the query it encodes it from. #[test] -fn should_need_protocol_14_only_for_composite_chained_and_token_pool_requests() { +fn should_need_protocol_14_only_for_token_pool_requests() { use proto::get_documents_request::get_documents_request_v1::{ChainedJoin, SubQuery}; use proto::get_documents_request::{GetDocumentsRequestV0, GetDocumentsRequestV1, Version}; let documents = |v1: GetDocumentsRequestV1| proto::GetDocumentsRequest { @@ -1010,7 +1016,7 @@ fn should_need_protocol_14_only_for_composite_chained_and_token_pool_requests() ..plain.clone() }) .needs_protocol_version(), - Some(14) + None ); assert_eq!( documents(GetDocumentsRequestV1 { @@ -1018,7 +1024,7 @@ fn should_need_protocol_14_only_for_composite_chained_and_token_pool_requests() ..plain }) .needs_protocol_version(), - Some(14) + None ); macro_rules! token_pool { diff --git a/packages/rs-drive-abci/tests/older_node_answers.rs b/packages/rs-drive-abci/tests/older_node_answers.rs index 4c779181c20..36970310251 100644 --- a/packages/rs-drive-abci/tests/older_node_answers.rs +++ b/packages/rs-drive-abci/tests/older_node_answers.rs @@ -7,9 +7,10 @@ //! here by this Drive at protocol 13 answering the request without those fields, signed by //! a test quorum over the genuine protocol 13 root. The SDK verifies the answer against //! the request it sent and refuses it, at protocol 13 rules and at 14, and the node that -//! sent it is banned. So every such request declares that it needs protocol 14 -//! (`TransportRequest::needs_protocol_version`), which keeps the SDK from sending it until a -//! verified answer shows the network runs protocol 14. +//! sent it is banned. So every such request needs protocol 14, which keeps the SDK from +//! sending it until a verified answer shows the network runs protocol 14: a composite or +//! chained query says so from its version tables (`DocumentQuery::required_protocol_version`), +//! a token's pool request on the wire (`TransportRequest::needs_protocol_version`). use dapi_grpc::platform::v0::get_documents_request::Version as RequestVersion; use dapi_grpc::platform::v0::get_documents_response::{ @@ -64,6 +65,16 @@ const QUORUM_HASH: [u8; 32] = [2; 32]; const YAPPR_CONTRACT_PATH: &str = "../rs-drive/tests/supporting_files/contract/yappr-likes/yappr-likes-contract.json"; +/// The first protocol version that no node of a 4.1 release can sign: the first whose table +/// records composite document queries, a field 4.1 nodes drop. +fn first_version_without_4_1_nodes() -> u32 { + dpp::version::PLATFORM_VERSIONS + .iter() + .find(|version| version.drive_abci.query.document_sub_queries.is_some()) + .expect("a known version has composite queries") + .protocol_version +} + fn v13() -> &'static PlatformVersion { PlatformVersion::get(13).expect("protocol 13") } @@ -414,8 +425,18 @@ fn should_refuse_an_older_node_answer_to_a_composite_query() { rules.protocol_version ); } - assert_eq!(request.needs_protocol_version(), Some(14)); + assert_eq!(request.needs_protocol_version(), None); } + assert_eq!( + query + .required_protocol_version() + .map(|requirement| requirement.protocol_version), + Some(first_version_without_4_1_nodes()) + ); + assert!( + first_version_without_4_1_nodes() > v13().protocol_version, + "the version the 4.1 emulation signs at is refused" + ); } /// The answer of a 4.1 node to a chained documents query is its inner query alone, which @@ -491,8 +512,18 @@ fn should_refuse_an_older_node_answer_to_a_chained_query() { rules.protocol_version ); } - assert_eq!(request.needs_protocol_version(), Some(14)); + assert_eq!(request.needs_protocol_version(), None); } + assert_eq!( + query + .required_protocol_version() + .map(|requirement| requirement.protocol_version), + Some(first_version_without_4_1_nodes()) + ); + assert!( + first_version_without_4_1_nodes() > v13().protocol_version, + "the version the 4.1 emulation signs at is refused" + ); } /// The answer of a 4.1 node to a token's shielded pool request is the credit pool's, which @@ -557,7 +588,12 @@ fn should_refuse_an_older_node_answer_to_a_token_pool_request() { rules.protocol_version ); } - assert_eq!(request(token_id.clone()).needs_protocol_version(), Some(14)); + // rs-dapi-client cannot read the version tables, so its literal is pinned to them + // here: the first protocol version no 4.1 node signs. + assert_eq!( + request(token_id.clone()).needs_protocol_version(), + Some(first_version_without_4_1_nodes()) + ); assert_eq!(request(None).needs_protocol_version(), None); }}; } diff --git a/packages/rs-drive-abci/tests/protocol_gate.rs b/packages/rs-drive-abci/tests/protocol_gate.rs index a39710c8129..61c4740ba8d 100644 --- a/packages/rs-drive-abci/tests/protocol_gate.rs +++ b/packages/rs-drive-abci/tests/protocol_gate.rs @@ -23,13 +23,19 @@ use dapi_grpc::platform::v0::{ GetDataContractsResponse, GetEpochsInfoRequest, GetEpochsInfoResponse, ResponseMetadata, }; use dapi_grpc::Message; +use dpp::block::epoch::Epoch; +use dpp::block::extended_epoch_info::v0::ExtendedEpochInfoV0Getters; use dpp::block::extended_epoch_info::ExtendedEpochInfo; use dpp::dashcore::Network; use dpp::version::PlatformVersion; +use drive::drive::credit_pools::epochs::operations_factory::EpochOperations; use drive::drive::Drive; +use drive::util::batch::grovedb_op_batch::GroveDbOpBatchV0Methods; +use drive::util::batch::GroveDbOpBatch; use drive::util::test_helpers::setup::setup_drive_with_initial_state_structure; use drive_proof_verifier::types::data_contracts_by_range::DataContractsByRange; use drive_proof_verifier::types::data_contracts_latest_versions::DataContractsLatestVersions; +use drive_proof_verifier::types::ExtendedEpochInfos; use drive_proof_verifier::{ContextProvider, FromProof}; use signed_response::{signed, Provider, NEWER_PROTOCOL_VERSION}; use std::path::PathBuf; @@ -37,6 +43,29 @@ use std::path::PathBuf; /// The contract whose latest version the recorded answer proves absent. const CONTRACT_ID: [u8; 32] = [0x42; 32]; +/// The epoch the epoch-2 states are in. +const CURRENT_EPOCH: u16 = 2; + +/// The epoch length of the epoch-2 states. +const EPOCH_LENGTH_MS: u64 = 3_600_000; + +/// The genesis time of an epoch-2 state far enough ahead that the hold an SDK derives from it +/// has not ended while its tests run. +const GENESIS_AHEAD_MS: u64 = 4_000_000_000_000; + +/// The genesis time of an epoch-2 state whose epoch 2 ended before its answers' block time. +const GENESIS_ENDED_MS: u64 = 1_700_000_000_000 - 10 * EPOCH_LENGTH_MS; + +/// When epochs 0 to 2 of the epoch-2 state with genesis at `genesis_ms` started: each first +/// block a second after its epoch's start on the grid. +fn epoch_starts_ms(genesis_ms: u64) -> [(u16, u64); 3] { + [ + (0, genesis_ms), + (1, genesis_ms + EPOCH_LENGTH_MS + 1000), + (2, genesis_ms + 2 * EPOCH_LENGTH_MS + 1000), + ] +} + fn directory() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")) .join("../rs-drive-proof-verifier/tests/vectors/protocol-gate") @@ -57,6 +86,61 @@ impl Node { } } + /// A node's state at `protocol_version` in epoch [`CURRENT_EPOCH`], its epochs started + /// at [`epoch_starts_ms`] of `genesis_ms`. + fn in_epoch_2(protocol_version: u32, genesis_ms: u64) -> Self { + let node = Self::new(protocol_version); + let mut batch = GroveDbOpBatch::new(); + for (index, start_ms) in epoch_starts_ms(genesis_ms) { + Epoch::new(index).unwrap().add_init_current_operations( + 1000, + u64::from(index) * 100, + u32::from(index) + 1, + start_ms, + protocol_version, + &mut batch, + ); + } + node.drive + .grove_apply_batch(batch, false, None, &node.version.drive) + .unwrap(); + node + } + + /// The genesis epoch this state proves, with the current-epoch hint in the metadata. + fn epoch_probe_in_epoch_2(&self) -> GetEpochsInfoResponse { + let mut answer = self.epochs_answer(0, 1); + set_epoch_hint(&mut answer); + answer + } + + /// Epochs 2 and 3 of this state: the proof shows 2 is the newest that started. + fn current_epoch(&self) -> GetEpochsInfoResponse { + let mut answer = self.epochs_answer(CURRENT_EPOCH, 2); + set_epoch_hint(&mut answer); + answer + } + + fn epochs_answer(&self, start: u16, count: u16) -> GetEpochsInfoResponse { + let proof = self + .drive + .prove_epochs_infos(start, count, true, None, self.version) + .unwrap(); + let metadata = self.metadata(); + GetEpochsInfoResponse { + version: Some(get_epochs_info_response::Version::V0( + GetEpochsInfoResponseV0 { + result: Some(get_epochs_info_response_v0::Result::Proof(signed( + proof, + &metadata, + self.root(), + ))), + metadata: Some(metadata), + }, + )), + } + } + fn root(&self) -> [u8; 32] { self.drive .grove @@ -149,6 +233,29 @@ impl Node { } } +/// The epoch index in response metadata is not covered by the quorum signature, so it is set +/// after signing, as a node reports it. +fn set_epoch_hint(answer: &mut GetEpochsInfoResponse) { + let Some(get_epochs_info_response::Version::V0(v0)) = answer.version.as_mut() else { + panic!("v0 answer") + }; + v0.metadata.as_mut().unwrap().epoch = u32::from(CURRENT_EPOCH); +} + +/// The request that proves the current epoch of an epoch-2 state. +fn current_epoch_request() -> GetEpochsInfoRequest { + GetEpochsInfoRequest { + version: Some(get_epochs_info_request::Version::V0( + GetEpochsInfoRequestV0 { + start_epoch: Some(u32::from(CURRENT_EPOCH)), + count: 2, + ascending: true, + prove: true, + }, + )), + } +} + /// The request the SDK sends to learn the network's protocol version: the genesis epoch. fn epoch_probe_request() -> GetEpochsInfoRequest { GetEpochsInfoRequest { @@ -206,6 +313,25 @@ fn verified_epoch_probe(response: GetEpochsInfoResponse, rules_version: u32) -> metadata.protocol_version } +/// The started epochs, with their start times, an answer to [`current_epoch_request`] +/// proves. +fn verified_epoch_starts(response: GetEpochsInfoResponse) -> Vec<(u16, u64)> { + let (epochs, _, _) = + >::maybe_from_proof_with_metadata( + current_epoch_request(), + response, + Network::Testnet, + rules(13), + &Provider, + ) + .unwrap(); + epochs + .unwrap() + .into_iter() + .filter_map(|(index, epoch)| epoch.map(|epoch| (index, epoch.first_block_time()))) + .collect() +} + /// The genesis epoch answer verifies at every protocol version an SDK may hold, so any /// SDK can learn the version the answering node signed it at. #[test] @@ -291,6 +417,26 @@ fn should_export_protocol_gate_sdk_fixtures() { newer.encode_to_vec(), ) .unwrap(); + for (suffix, genesis_ms) in [("", GENESIS_AHEAD_MS), ("-ended", GENESIS_ENDED_MS)] { + let in_epoch_2 = Node::in_epoch_2(13, genesis_ms); + let probe = in_epoch_2.epoch_probe_in_epoch_2(); + assert_eq!(verified_epoch_probe(probe.clone(), 13), 13); + std::fs::write( + directory.join(format!("epoch-probe-pv13-epoch-2{suffix}.bin")), + probe.encode_to_vec(), + ) + .unwrap(); + let current = in_epoch_2.current_epoch(); + assert_eq!( + verified_epoch_starts(current.clone()), + epoch_starts_ms(genesis_ms)[2..].to_vec() + ); + std::fs::write( + directory.join(format!("current-epoch-pv13-epoch-2{suffix}.bin")), + current.encode_to_vec(), + ) + .unwrap(); + } let node = Node::new(13); std::fs::write( directory.join("contracts-by-range-pv13.bin"), @@ -319,6 +465,21 @@ fn should_keep_the_recorded_protocol_gate_vectors_verifiable() { } let newer = GetEpochsInfoResponse::decode(read("epoch-probe-newer.bin").as_slice()).unwrap(); assert_eq!(verified_epoch_probe(newer, 14), NEWER_PROTOCOL_VERSION); + for (suffix, genesis_ms) in [("", GENESIS_AHEAD_MS), ("-ended", GENESIS_ENDED_MS)] { + let probe = GetEpochsInfoResponse::decode( + read(&format!("epoch-probe-pv13-epoch-2{suffix}.bin")).as_slice(), + ) + .unwrap(); + assert_eq!(verified_epoch_probe(probe, 13), 13); + let current = GetEpochsInfoResponse::decode( + read(&format!("current-epoch-pv13-epoch-2{suffix}.bin")).as_slice(), + ) + .unwrap(); + assert_eq!( + verified_epoch_starts(current), + epoch_starts_ms(genesis_ms)[2..].to_vec() + ); + } let by_range = GetDataContractsResponse::decode(read("contracts-by-range-pv13.bin").as_slice()).unwrap(); >::maybe_from_proof( diff --git a/packages/rs-drive-abci/tests/query_request_errors.rs b/packages/rs-drive-abci/tests/query_request_errors.rs index a37cbf016d9..6b5b1766aae 100644 --- a/packages/rs-drive-abci/tests/query_request_errors.rs +++ b/packages/rs-drive-abci/tests/query_request_errors.rs @@ -82,6 +82,8 @@ use tower::ServiceExt; #[path = "query_request_errors/composite_caps.rs"] mod composite_caps; +#[path = "query_request_errors/document_features.rs"] +mod document_features; #[path = "query_request_errors/features_not_active.rs"] mod features_not_active; #[path = "query_request_errors/offset_controls.rs"] diff --git a/packages/rs-drive-abci/tests/query_request_errors/document_features.rs b/packages/rs-drive-abci/tests/query_request_errors/document_features.rs new file mode 100644 index 00000000000..5f2387eaa8d --- /dev/null +++ b/packages/rs-drive-abci/tests/query_request_errors/document_features.rs @@ -0,0 +1,408 @@ +//! The document features the SDK refuses below their protocol version +//! (`DocumentQuery::required_protocol_version`), sent to a node one version below it. +//! +//! Most of them are refused with a status clients do not retry, so sending them would only +//! fail the call. A start cursor on a type whose documents are summed fails with a status +//! clients retry on every other node, banning each one, so it must never be sent there; from +//! its requirement on, it is served. + +use super::*; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v0::Start; +use dash_platform_queries::documents::document_query::DocumentQuery; +use dpp::data_contract::document_type::random_document::CreateRandomDocument; +use dpp::document::DocumentV0Getters; +use dpp::platform_value::{platform_value, Value}; +use dpp::prelude::DataContract; +use dpp::tests::json_document::json_document_to_contract; +use dpp::version::TryFromPlatformVersioned; +use drive::query::{ + HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, + SelectProjection, TimeRangeSelector, WhereClause, WhereOperator, +}; +use drive::util::object_size_info::{ + DocumentAndContractInfo, DocumentInfo::DocumentRefInfo, OwnedDocumentInfo, +}; +use rs_dapi_client::CanRetry; + +/// Its `tip` documents are summed by `amount`. +const TIP_JAR_CONTRACT_PATH: &str = + "../rs-drive/tests/supporting_files/contract/tip-jar/tip-jar-contract.json"; + +/// The fixture contract at `version`, stored in `fixture`. +fn fixture_contract(fixture: &QueryFixture, version: &PlatformVersion) -> Arc { + let contract = + get_data_contract_fixture(Some(Identifier::new([9; 32])), 0, version.protocol_version) + .data_contract_owned(); + fixture + .platform + .drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("store the fixture contract"); + Arc::new(contract) +} + +/// The tip jar contract at `version`, stored in `fixture` with one tip, and that tip's id. +fn tip_jar(fixture: &QueryFixture, version: &PlatformVersion) -> (Arc, [u8; 32]) { + let contract = json_document_to_contract(TIP_JAR_CONTRACT_PATH, false, version) + .expect("the tip jar contract"); + let drive = &fixture.platform.drive; + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("store the tip jar contract"); + let document_type = contract.document_type_for_name("tip").expect("tip"); + let tip = document_type + .random_document(Some(7), version) + .expect("a random tip"); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo((&tip, None)), + owner_id: Some(tip.owner_id().to_buffer()), + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default(), + true, + None, + version, + None, + ) + .expect("store a tip"); + (Arc::new(contract), tip.id().to_buffer()) +} + +/// Builds a query over the fixture contract it is given. +type BuildQuery = Box) -> DocumentQuery>; + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn should_refuse_each_document_feature_below_its_protocol_version_without_a_ban() { + let fixture_at = |query: &DocumentQuery| { + let requirement = query + .required_protocol_version() + .expect("the query uses a feature"); + PlatformVersion::get(requirement.protocol_version - 1).expect("the version below") + }; + let count = SelectProjection::count_star; + let cases: Vec<(&str, BuildQuery)> = vec![ + ( + "contains", + Box::new(|contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_where(WhereClause { + field: "name".to_string(), + operator: WhereOperator::Contains, + value: Value::Text("dash".into()), + }) + }), + ), + ( + "time range", + Box::new(move |contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_select(count()) + .with_time_range("$createdAt", TimeRangeSelector::Newest) + }), + ), + ( + "integer range", + Box::new(move |contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_select(count()) + .with_integer_range("$revision", Value::U64(1)) + }), + ), + ( + "ranked", + Box::new(move |contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_select(count()) + .with_group_by("$ownerId") + .order_by_selected_aggregate( + dash_platform_queries::documents::document_query::RankingDirection::Descending, + ) + .with_limit(5) + }), + ), + ( + "having", + Box::new(move |contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_select(count()) + .with_group_by("$ownerId") + .with_having(vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(1)), + }]) + .with_limit(5) + }), + ), + ( + "several in clauses", + Box::new(|contract| { + DocumentQuery::new(contract, "niceDocument") + .unwrap() + .with_where(WhereClause { + field: "$ownerId".to_string(), + operator: WhereOperator::In, + value: Value::Array(vec![ + Value::Identifier([1; 32]), + Value::Identifier([2; 32]), + ]), + }) + .with_where(WhereClause { + field: "name".to_string(), + operator: WhereOperator::In, + value: platform_value!(["a", "b"]), + }) + }), + ), + ]; + let mut failures = Vec::new(); + for (feature, build) in cases { + let probe = build(Arc::new( + get_data_contract_fixture(None, 0, PlatformVersion::latest().protocol_version) + .data_contract_owned(), + )); + let below = fixture_at(&probe); + let fixture = QueryFixture::new_with_version(below); + let query = build(fixture_contract(&fixture, below)); + let request = wire::GetDocumentsRequest::try_from_platform_versioned(query, below) + .expect("the query encodes"); + let (mut client, server) = fixture.client(); + match client.get_documents(request).await { + Err(status) if status.code() == Code::InvalidArgument && !status.can_retry() => {} + other => failures.push(format!( + "{feature} at protocol {}: {:?}", + below.protocol_version, + other.map(|_| ()) + )), + } + server.abort(); + } + assert!(failures.is_empty(), "{failures:#?}"); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn should_fail_a_cursor_on_summed_documents_with_a_retried_status_only_below_its_protocol_version( +) { + let probe = { + let fixture = QueryFixture::new(); + let (contract, _) = tip_jar(&fixture, PlatformVersion::latest()); + let mut query = DocumentQuery::new(contract, "tip").unwrap().with_limit(10); + query.start = Some(Start::StartAfter(vec![1; 32])); + query + .required_protocol_version() + .expect("a cursor on summed documents needs a version") + }; + for (version, served) in [ + (probe.protocol_version - 1, false), + (probe.protocol_version, true), + ] { + let version = PlatformVersion::get(version).expect("a known protocol version"); + let fixture = QueryFixture::new_with_version(version); + let (contract, tip) = tip_jar(&fixture, version); + let mut query = DocumentQuery::new(contract, "tip").unwrap().with_limit(10); + query.start = Some(Start::StartAt(tip.to_vec())); + let request = wire::GetDocumentsRequest::try_from_platform_versioned(query, version) + .expect("the query encodes"); + let (mut client, server) = fixture.client(); + let result = client.get_documents(request).await; + if served { + result.unwrap_or_else(|status| { + panic!("protocol {}: served: {status:?}", version.protocol_version) + }); + } else { + let status = result.expect_err("refused below its version"); + assert!( + status.can_retry(), + "protocol {}: {status:?} would ban no node; the SDK's refusal guards a ban", + version.protocol_version + ); + } + server.abort(); + } +} + +/// Its `post` indexes the elements of `hashtags`, and ranks them (`byHashtag`). +const ARRAY_INDEX_CONTRACT_PATH: &str = + "../rs-drive/tests/supporting_files/contract/array-index/array-index-contract.json"; + +/// An `item` whose compound index takes an `In` on each of its two properties. +fn two_in_contract(version: &PlatformVersion) -> DataContract { + let schemas = platform_value!({"item": { + "type": "object", + "properties": { + "a": {"type": "string", "minLength": 1, "maxLength": 32, "position": 0}, + "b": {"type": "string", "minLength": 1, "maxLength": 32, "position": 1} + }, + "required": ["a", "b"], + "indices": [{"name": "byAB", "properties": [{"a": "asc"}, {"b": "asc"}]}], + "additionalProperties": false + }}); + dpp::data_contract::DataContractFactory::new(version.protocol_version) + .expect("factory") + .create_with_value_config(Identifier::new([10; 32]), 0, schemas, None, None) + .expect("the item contract is well-formed") + .data_contract_owned() +} + +fn two_ins(contract: Arc) -> DocumentQuery { + DocumentQuery::new(contract, "item") + .unwrap() + .with_where(WhereClause { + field: "a".to_string(), + operator: WhereOperator::In, + value: platform_value!(["x", "y"]), + }) + .with_where(WhereClause { + field: "b".to_string(), + operator: WhereOperator::In, + value: platform_value!(["p", "q"]), + }) + .with_order_by(drive::query::OrderClause { + field: "a".to_string(), + ascending: true, + }) + .with_order_by(drive::query::OrderClause { + field: "b".to_string(), + ascending: true, + }) + .with_limit(10) +} + +/// A contract a feature's query is served against, and the query. +struct Served { + feature: &'static str, + contract: fn(&PlatformVersion) -> Option, + query: fn(Arc) -> DocumentQuery, +} + +fn array_index_contract(version: &PlatformVersion) -> Option { + json_document_to_contract(ARRAY_INDEX_CONTRACT_PATH, false, version).ok() +} + +fn served_cases() -> Vec { + vec![ + Served { + feature: "contains", + contract: array_index_contract, + query: |contract| { + DocumentQuery::new(contract, "post") + .unwrap() + .with_where(WhereClause { + field: "hashtags".to_string(), + operator: WhereOperator::Contains, + value: Value::Text("dash".into()), + }) + .with_limit(10) + }, + }, + Served { + feature: "ranked", + contract: array_index_contract, + query: |contract| { + DocumentQuery::new(contract, "post") + .unwrap() + .with_select(SelectProjection::count_star()) + .with_group_by("hashtags") + .order_by_selected_aggregate( + dash_platform_queries::documents::document_query::RankingDirection::Descending, + ) + .with_limit(5) + }, + }, + Served { + feature: "having", + contract: array_index_contract, + query: |contract| { + DocumentQuery::new(contract, "post") + .unwrap() + .with_select(SelectProjection::count_star()) + .with_group_by("hashtags") + .with_having(vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(0)), + }]) + .with_limit(5) + }, + }, + Served { + feature: "several in clauses", + contract: |version| Some(two_in_contract(version)), + query: two_ins, + }, + ] +} + +/// From its protocol version on, a node serves each feature on a contract that indexes it; +/// one version below, the same query on the same contract is refused without a ban, when the +/// contract can be registered there at all. So the requirement is neither later nor earlier +/// than the version that serves the feature. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn should_serve_each_document_feature_from_its_protocol_version_and_not_before() { + let mut failures = Vec::new(); + for case in served_cases() { + let latest = Arc::new((case.contract)(PlatformVersion::latest()).expect("the contract")); + let required = (case.query)(latest) + .required_protocol_version() + .expect("the query uses a feature") + .protocol_version; + for (version, served) in [(required - 1, false), (required, true)] { + let version = PlatformVersion::get(version).expect("a known protocol version"); + let Some(contract) = (case.contract)(version) else { + // The contract's indexes do not exist at this version, so neither can the + // query it serves. + assert!( + !served, + "{}: the contract exists from its version", + case.feature + ); + continue; + }; + let fixture = QueryFixture::new_with_version(version); + fixture + .platform + .drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("store the contract"); + let request = wire::GetDocumentsRequest::try_from_platform_versioned( + (case.query)(Arc::new(contract)), + version, + ) + .expect("the query encodes"); + let (mut client, server) = fixture.client(); + let result = client.get_documents(request).await.map(|_| ()); + let outcome_ok = match (&result, served) { + (Ok(()), true) => true, + (Err(status), false) => { + status.code() == Code::InvalidArgument && !status.can_retry() + } + _ => false, + }; + if !outcome_ok { + failures.push(format!( + "{} at protocol {} (served expected: {served}): {result:?}", + case.feature, version.protocol_version + )); + } + server.abort(); + } + } + assert!(failures.is_empty(), "{failures:#?}"); +} diff --git a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/README.md b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/README.md index 25535338533..47937ca43d2 100644 --- a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/README.md +++ b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/README.md @@ -23,8 +23,18 @@ through the normal proof and signature verifier: a request every node serves from protocol 14. - `latest-versions-pv13.bin`: a protocol 13 node's answer (the multi-contract proof) to `GetDataContractsLatestVersionsRequestV0 { ids: [[0x42; 32]], include_contracts: false, prove: true }`. +- `epoch-probe-pv13-epoch-2.bin`, `current-epoch-pv13-epoch-2.bin`: a protocol 13 node in epoch + 2, with one-hour epochs, each epoch's first block a second after its start on the grid. The + first answers the genesis epoch query above; the second answers + `GetEpochsInfoRequestV0 { start_epoch: Some(2), count: 2, ascending: true, prove: true }`, + proving epoch 2 current. Their metadata names epoch 2 (the field is not signed). Genesis is in + 2096, so the probe hold the SDK derives from them has not ended while the tests run. +- `epoch-probe-pv13-epoch-2-ended.bin`, `current-epoch-pv13-epoch-2-ended.bin`: the same, with + genesis ten epochs before the answers' block time, so epoch 2 has ended when they are + signed. -Every answer carries metadata at height 10, Core chain-locked height 1, epoch 0, time +Every answer carries metadata at height 10, Core chain-locked height 1, epoch 0 (2 for the +epoch-2 answers), time `1_700_000_000_000` and chain id `dash-testnet-51`. The fixed test signing key is `[1; 32]`; `quorum-key.bin` is its public key (quorum type 106, quorum hash `[2; 32]`). These are synthetic fixtures, never network trust anchors. The generator verifies every diff --git a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2-ended.bin b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2-ended.bin new file mode 100644 index 00000000000..a29669f12db Binary files /dev/null and b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2-ended.bin differ diff --git a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2.bin b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2.bin new file mode 100644 index 00000000000..9258668cfbb Binary files /dev/null and b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/current-epoch-pv13-epoch-2.bin differ diff --git a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2-ended.bin b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2-ended.bin new file mode 100644 index 00000000000..f9a51a1764b Binary files /dev/null and b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2-ended.bin differ diff --git a/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2.bin b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2.bin new file mode 100644 index 00000000000..b2e4755d48f Binary files /dev/null and b/packages/rs-drive-proof-verifier/tests/vectors/protocol-gate/epoch-probe-pv13-epoch-2.bin differ diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs index 5da69b458a7..f0e2ec445bf 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs @@ -133,6 +133,12 @@ pub struct DocumentTypeMethodVersions { #[derive(Clone, Debug, Default)] pub struct DocumentTypeSchemaVersions { + /// The contract grammar (document meta-schema generation). From generation 3 an index + /// can key the elements of a typed array and time and integer windows. + /// + /// The SDK reads it too, to know from which protocol version every node serves + /// `contains` / `containsAny` clauses and window selections + /// (`DocumentQuery::required_protocol_version` in `dash-platform-queries`). pub document_type_schema: FeatureVersion, /// How full validation checks a document type's schema at registration. 0: against the /// document meta-schema `document_type_schema` names, through the `jsonschema` crate, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs index d172478060d..ab0f493c2a0 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs @@ -2,7 +2,7 @@ pub mod v0; pub mod v1; pub mod v2; -use versioned_feature_core::{FeatureVersion, FeatureVersionBounds}; +use versioned_feature_core::{FeatureVersion, FeatureVersionBounds, OptionalFeatureVersion}; #[derive(Clone, Debug, Default)] pub struct DriveAbciQueryVersions { @@ -10,6 +10,21 @@ pub struct DriveAbciQueryVersions { pub response_metadata: FeatureVersion, pub proofs_query: FeatureVersion, pub document_query: FeatureVersionBounds, + /// Composite document queries: the `sub_queries` of a `getDocuments` V1 request. `None` + /// at the protocol versions that nodes of a release without the field can sign: such a + /// node drops it and proves the plain page instead. + /// + /// The SDK reads it to know from which protocol version every node serves composite + /// queries (`DocumentQuery::required_protocol_version` in `dash-platform-queries`). Nodes + /// do not read it: a node whose release has the field serves it at every protocol version. + /// Only whether it is set is read; the number selects nothing. + pub document_sub_queries: OptionalFeatureVersion, + /// Chained document queries: the `chained` join of a `getDocuments` V1 request. `None` at + /// the protocol versions that nodes of a release without the field can sign: such a node + /// drops it and proves the inner query instead. + /// + /// Read by the SDK and not by nodes, as [`Self::document_sub_queries`] is. + pub chained_document_query: OptionalFeatureVersion, pub document_history: FeatureVersionBounds, /// `getPrivateDocumentEntries`: a page of the entries of a private document type's store. /// Served from protocol version 15, where Drive's private document entry methods are @@ -48,6 +63,10 @@ pub struct DriveAbciDocumentQueryHelperVersions { /// mode for `SELECT COUNT / SUM / AVG` and enforces the /// per-mode `accepts_limit()` contract. See /// `query::document_query::v1::compute_aggregate_mode_and_check_limit`. + /// + /// The SDK reads it too, to know from which protocol version every node routes a ranked + /// aggregate (from 1) and a having-range aggregate (from 2) + /// (`DocumentQuery::required_protocol_version` in `dash-platform-queries`). pub compute_aggregate_mode_and_check_limit: FeatureVersion, } diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v0.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v0.rs index f0fff878523..8b5e02620b6 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v0.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v0.rs @@ -24,6 +24,8 @@ pub const DRIVE_ABCI_QUERY_VERSIONS_V0: DriveAbciQueryVersions = DriveAbciQueryV max_version: 0, default_current_version: 0, }, + document_sub_queries: None, + chained_document_query: None, document_history: FeatureVersionBounds { min_version: 0, max_version: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v1.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v1.rs index b5ad73ddc20..917d966d1f9 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v1.rs @@ -26,6 +26,8 @@ pub const DRIVE_ABCI_QUERY_VERSIONS_V1: DriveAbciQueryVersions = DriveAbciQueryV max_version: 1, default_current_version: 1, }, + document_sub_queries: None, + chained_document_query: None, document_history: FeatureVersionBounds { min_version: 0, max_version: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v2.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v2.rs index a12a7eb28c0..58ece9051e8 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v2.rs @@ -7,7 +7,7 @@ use versioned_feature_core::FeatureVersionBounds; /// Version 2 of the Drive ABCI query versions, selected by protocol version 14. /// -/// Differs from v1 in three slots. +/// Differs from v1 in five slots. /// /// `document_query_helpers.compute_aggregate_mode_and_check_limit` is 2 /// rather than 0. That opens two routes on the v1 document-query handler: @@ -21,6 +21,10 @@ use versioned_feature_core::FeatureVersionBounds; /// select with no ranked axis, is rejected as before ("HAVING clause is not /// yet implemented"). /// +/// `document_sub_queries` and `chained_document_query` are `Some(0)` rather than +/// `None`: protocol version 14 is the first that no node of a release without composite +/// and chained document queries can sign, so every node at it serves both. +/// /// `data_contract_query_helpers.latest_versions_read` is 1 rather than 0: /// from protocol version 14 every contract carries a four-byte version item /// beside it (`[64, id, 2] / 64`, backfilled on the first block of the version), @@ -58,6 +62,8 @@ pub const DRIVE_ABCI_QUERY_VERSIONS_V2: DriveAbciQueryVersions = DriveAbciQueryV }, ..DRIVE_ABCI_QUERY_VERSIONS_V1.identity_based_queries }, + document_sub_queries: Some(0), + chained_document_query: Some(0), document_query_helpers: DriveAbciDocumentQueryHelperVersions { compute_aggregate_mode_and_check_limit: 2, }, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs index c4fbf1f7354..35deb9762cc 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs @@ -121,6 +121,10 @@ pub struct DriveDocumentQueryMethodVersions { /// checks; v1 (protocol version 14) groups multiple `In` clauses /// structurally and reports duplicate-field and overlap errors /// instead. Flips together with `non_primary_key_path_query`. + /// + /// The SDK reads it too, to know from which protocol version every node serves a + /// documents query with several `In` clauses + /// (`DocumentQuery::required_protocol_version` in `dash-platform-queries`). pub where_clause_grouping: FeatureVersion, /// Reading a query's `startAt`/`startAfter` cursor document from the /// type's primary-key tree. v0 decodes only a plain `Item` and refuses @@ -128,6 +132,10 @@ pub struct DriveDocumentQueryMethodVersions { /// `documentsSummable` type, whose documents are stored as /// `ItemWithSumItem`, fails every query; v1 (protocol version 14) /// decodes any item element, as the proof verifier always has. + /// + /// The SDK reads it too, to know from which protocol version every node serves a cursor + /// on such a type (`DocumentQuery::required_protocol_version` in + /// `dash-platform-queries`). pub start_at_document: FeatureVersion, } diff --git a/packages/rs-platform-version/src/version/mocks/v2_test.rs b/packages/rs-platform-version/src/version/mocks/v2_test.rs index 3c2923431f7..368dda1c658 100644 --- a/packages/rs-platform-version/src/version/mocks/v2_test.rs +++ b/packages/rs-platform-version/src/version/mocks/v2_test.rs @@ -181,6 +181,8 @@ pub const TEST_PLATFORM_V2: PlatformVersion = PlatformVersion { max_version: 0, default_current_version: 0, }, + document_sub_queries: None, + chained_document_query: None, document_history: FeatureVersionBounds { min_version: 0, max_version: 0, diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 1135ef2097f..c069e00b1e4 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -187,7 +187,10 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// LIMIT n`), the latter served as a value-bounded range /// read of the covering ranked index's axis secondary. v13 and earlier /// keep the v1 table and therefore keep rejecting both shapes, so -/// mixed-version networks agree across the upgrade. The same table accepts +/// mixed-version networks agree across the upgrade. It records composite +/// and chained document queries (`document_sub_queries`, +/// `chained_document_query`: `Some(0)`, `None` in earlier tables), which +/// the SDK reads and nodes do not. The same table accepts /// `getIdentityByNonUniquePublicKeyHash` request V1, answered with one proof /// of the index selection and the identity; V0 keeps its separate proofs and /// stays accepted, and v13 and earlier refuse V1 as 4.1 nodes do. @@ -2785,7 +2788,7 @@ pub const PLATFORM_V14: PlatformVersion = PlatformVersion { methods: DRIVE_ABCI_METHOD_VERSIONS_V10, // changed: records the per-block total credits history for the daily withdrawal limit; record_token_shielded_pool_anchors records and prunes the anchors of the token pools a block touched; decode_raw_state_transitions, execute_event, validate_fees_of_event and add_distribute_storage_fee_to_epochs_operations each move to 1 — the table's own per-slot comments carry the full list validation_and_processing: DRIVE_ABCI_VALIDATION_VERSIONS_V10, // changed: contested-index cross-check + refersTo document reference validation; the ContractUserModeration gates and the batch transformer's contract_moderation_gate; a contest accepts at most max_contenders_per_contest contenders and maximum_contenders_to_consider rises to 10,000; a contender's fund doubles past 250 contenders and for every 50 more; the three shielded-fee token pool transitions gain basic structure validation and document_base_transition_state_validation 2 admits a document token cost paid from a token pool and enforces pause and the issuer's frozen-recipient policy on transparent payments; the ShieldFromAssetLock transform_into_action 1 checks its bundle against the bound preimage; identity create and identity create from addresses advanced_structure 1 refuse unpaid a failure in a field the funding signature does not cover withdrawal_constants: DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V3, // changed: prune bound for the total credits history - query: DRIVE_ABCI_QUERY_VERSIONS_V2, // changed: ranked + boolean-HAVING routing gate; the v1 handler also resolves IN_TIME_RANGE from committed block time; getIdentityByNonUniquePublicKeyHash accepts request V1 (one proof) + query: DRIVE_ABCI_QUERY_VERSIONS_V2, // changed: ranked + boolean-HAVING routing gate; the v1 handler also resolves IN_TIME_RANGE from committed block time; getIdentityByNonUniquePublicKeyHash accepts request V1 (one proof); records composite and chained document queries checkpoints: DRIVE_ABCI_CHECKPOINT_PARAMETERS_V1, }, dpp: DPPVersion { diff --git a/packages/rs-sdk-ffi/src/error.rs b/packages/rs-sdk-ffi/src/error.rs index 718b36b2b37..cd16db11c0f 100644 --- a/packages/rs-sdk-ffi/src/error.rs +++ b/packages/rs-sdk-ffi/src/error.rs @@ -544,6 +544,7 @@ mod tests { let err = dash_sdk::Error::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network: 13, pinned, cause: cause.map(Box::new), diff --git a/packages/rs-sdk/Cargo.toml b/packages/rs-sdk/Cargo.toml index 3d016298c7d..128e5c1a043 100644 --- a/packages/rs-sdk/Cargo.toml +++ b/packages/rs-sdk/Cargo.toml @@ -30,7 +30,7 @@ platform-encryption = { path = "../rs-platform-encryption" } http = { version = "1.1" } ciborium = { version = "0.2.2" } thiserror = "2.0.17" -tokio = { version = "1.40", features = ["macros", "time"] } +tokio = { version = "1.40", features = ["macros", "sync", "time"] } tokio-util = { version = "0.7.12" } async-trait = { version = "0.1.83" } serde = { version = "1.0.219", default-features = false, features = [ diff --git a/packages/rs-sdk/README.md b/packages/rs-sdk/README.md index 49da2cdab6b..8dc3eecd1eb 100644 --- a/packages/rs-sdk/README.md +++ b/packages/rs-sdk/README.md @@ -158,7 +158,9 @@ Checklist: in `packages/rs-dapi-client/src/transport/grpc.rs`. If nodes of an earlier release do not know the method or a new request field, declare the first protocol version that every node serves it from with `needs_protocol_version:`, and add it to `ADDITIONS` in `packages/rs-dapi-client/tests/request_inventory.rs`. The SDK refuses the request until the -network has proven that version (see `book/src/sdk/protocol-version-gate.md`). +network has proven that version (see `book/src/sdk/protocol-versions.md`). A new `getDocuments` field or operator +gets a feature slot in the version tables instead, read by `DocumentQuery::required_protocol_version` +(see `book/src/sdk/protocol-version-gate.md`). 4. [ ] If needed, implement new type in `packages/rs-drive-proof-verifier/src/types.rs` to hide complexity of data structures used internally. diff --git a/packages/rs-sdk/src/error.rs b/packages/rs-sdk/src/error.rs index 88647d34f39..d8e8e91ad57 100644 --- a/packages/rs-sdk/src/error.rs +++ b/packages/rs-sdk/src/error.rs @@ -154,9 +154,10 @@ pub enum Error { /// /// Retrying another node cannot help. The same request succeeds once a verified answer /// shows the network runs `required`, and this SDK's own version is at least - /// `required`. See [`TransportRequest::needs_protocol_version`](rs_dapi_client::transport::TransportRequest::needs_protocol_version). + /// `required`. See [`Query::protocol_requirement`](crate::platform::Query::protocol_requirement). #[error( - "{method} needs Platform protocol version {required}; {}", + "{method} needs Platform protocol version {required}{}; {}", + describe_feature(*.feature), describe_proven_protocol_version(*.network, *.pinned, .cause.is_some()) )] ProtocolVersionNotActive { @@ -164,6 +165,9 @@ pub enum Error { method: &'static str, /// Protocol version from which every node serves the request. required: u32, + /// The feature of the request that needs `required`, when the method alone does not, + /// for example "time-range selections" of a `get_documents` request. + feature: Option<&'static str>, /// Protocol version the SDK could rely on: the lower of its own version and the /// newest version a verified answer was signed with; 0 before any verified answer. network: u32, @@ -178,6 +182,13 @@ pub enum Error { }, } +/// The feature [`Error::ProtocolVersionNotActive`]'s message names, if any. +fn describe_feature(feature: Option<&'static str>) -> String { + feature + .map(|feature| format!(" for {feature}")) + .unwrap_or_default() +} + /// The end of [`Error::ProtocolVersionNotActive`]'s message. fn describe_proven_protocol_version( network: u32, @@ -665,6 +676,7 @@ mod tests { let refused = |network, pinned, cause: Option| Error::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network, pinned, cause: cause.map(Box::new), @@ -739,6 +751,7 @@ mod tests { let refused = |cause: Option| Error::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network: 0, pinned: false, cause: cause.map(Box::new), diff --git a/packages/rs-sdk/src/platform.rs b/packages/rs-sdk/src/platform.rs index 9c43294648b..0f3b7a88d57 100644 --- a/packages/rs-sdk/src/platform.rs +++ b/packages/rs-sdk/src/platform.rs @@ -64,9 +64,9 @@ pub use { fetch_many::FetchMany, fetch_unproved::FetchUnproved, query::{ - IdentityKeysQuery, LimitQuery, ProposerBlockCountByIdsQuery, Query, QueryStartInfo, - RecentAddressBalanceChangesQuery, RecentCompactedAddressBalanceChangesQuery, WireQuery, - DEFAULT_EPOCH_QUERY_LIMIT, + IdentityKeysQuery, LimitQuery, ProposerBlockCountByIdsQuery, ProtocolRequirement, Query, + QueryStartInfo, RecentAddressBalanceChangesQuery, + RecentCompactedAddressBalanceChangesQuery, WireQuery, DEFAULT_EPOCH_QUERY_LIMIT, }, query_settings::QuerySettings, }; diff --git a/packages/rs-sdk/src/platform/documents/chained_document_query_sdk.rs b/packages/rs-sdk/src/platform/documents/chained_document_query_sdk.rs index eebf6fa70f4..d95a44569de 100644 --- a/packages/rs-sdk/src/platform/documents/chained_document_query_sdk.rs +++ b/packages/rs-sdk/src/platform/documents/chained_document_query_sdk.rs @@ -7,6 +7,8 @@ use dapi_grpc::platform::v0::GetDocumentsRequest; use dash_platform_queries::documents::chained_document_query::ChainedDocumentQuery; use dpp::version::TryFromPlatformVersioned; +use crate::platform::documents::document_query_sdk::document_requirement; +use crate::platform::ProtocolRequirement; use crate::Error; /// Encode a [`ChainedDocumentQuery`] onto the wire. @@ -24,6 +26,15 @@ impl crate::platform::Query for ChainedDocu GetDocumentsRequest::try_from_platform_versioned(self.clone(), settings.protocol_version) .map_err(Error::from) } + + /// What the join and its inner query's features need + /// ([`ChainedDocumentQuery::required_protocol_version`]). + fn protocol_requirement( + &self, + _settings: &crate::platform::QuerySettings<'_>, + ) -> Result, Error> { + Ok(document_requirement(self.required_protocol_version())) + } } // `ChainedDocumentQuery` does not implement `TransportRequest` (the wire diff --git a/packages/rs-sdk/src/platform/documents/document_query_sdk.rs b/packages/rs-sdk/src/platform/documents/document_query_sdk.rs index 4eb62837ea1..5af38515b36 100644 --- a/packages/rs-sdk/src/platform/documents/document_query_sdk.rs +++ b/packages/rs-sdk/src/platform/documents/document_query_sdk.rs @@ -6,12 +6,14 @@ //! [`Query`](crate::platform::Query) encoding step. use crate::platform::documents::document_query::DocumentQuery; -use crate::platform::Fetch; +use crate::platform::{Fetch, ProtocolRequirement}; use crate::{error::Error, sdk::Sdk}; use dapi_grpc::platform::v0 as platform_proto; use dapi_grpc::platform::v0::GetDocumentsRequest; +use dash_platform_queries::documents::requirement::RequiredProtocolVersion; use dpp::prelude::{DataContract, Identifier}; use dpp::version::TryFromPlatformVersioned; +use rs_dapi_client::transport::TransportRequest; /// Sdk-bound extension methods for [`DocumentQuery`]. /// @@ -66,4 +68,24 @@ impl crate::platform::Query for DocumentQue GetDocumentsRequest::try_from_platform_versioned(self.clone(), settings.protocol_version) .map_err(Error::from) } + + /// What the query's features need ([`DocumentQuery::required_protocol_version`]), + /// whatever version it is encoded with. + fn protocol_requirement( + &self, + _settings: &crate::platform::QuerySettings<'_>, + ) -> Result, Error> { + Ok(document_requirement(self.required_protocol_version())) + } +} + +/// A document query's requirement, on the method every document query is sent with. +pub(crate) fn document_requirement( + requirement: Option, +) -> Option { + requirement.map(|requirement| ProtocolRequirement { + protocol_version: requirement.protocol_version, + method: GetDocumentsRequest::default().method_name(), + feature: Some(requirement.feature), + }) } diff --git a/packages/rs-sdk/src/platform/documents/mod.rs b/packages/rs-sdk/src/platform/documents/mod.rs index 4b8d63789c9..9f43abbaf99 100644 --- a/packages/rs-sdk/src/platform/documents/mod.rs +++ b/packages/rs-sdk/src/platform/documents/mod.rs @@ -8,7 +8,7 @@ pub use dash_platform_queries::documents::{ chained_document_query, composite_document_query, document_average, document_count, document_having_entries, document_history_query, document_query, document_ranked_entries, - document_split_averages, document_split_counts, document_split_sums, document_sum, + document_split_averages, document_split_counts, document_split_sums, document_sum, requirement, }; pub mod chained_document_query_sdk; diff --git a/packages/rs-sdk/src/platform/fetch.rs b/packages/rs-sdk/src/platform/fetch.rs index d6ec1144ee0..03455d612c7 100644 --- a/packages/rs-sdk/src/platform/fetch.rs +++ b/packages/rs-sdk/src/platform/fetch.rs @@ -187,21 +187,17 @@ where // this path). Specializing via a `fn encode_request_owned()` default method on // `Fetch` would eliminate the clone — deferred as future-perf work; trait shape // is intentionally uniform for now. - let mut owned_wire: ::Request = owned_rich.query(&settings)?; - let required_protocol_version = owned_wire.needs_protocol_version(); - if required_protocol_version.is_some() { - sdk.ensure_network_serves( - required_protocol_version, - owned_wire.method_name(), - request_settings, - ) - .await?; + let requirement = owned_rich.protocol_requirement(&settings)?; + if requirement.is_some() { + sdk.ensure_network_serves(requirement, request_settings) + .await?; // Asking the network for its version may have raised the version requests are // encoded with: encode the request again at it. settings = sdk.query_settings(); owned_rich = query.query(&settings)?; - owned_wire = owned_rich.query(&settings)?; } + let required_protocol_version = requirement.map(|requirement| requirement.protocol_version); + let owned_wire: ::Request = owned_rich.query(&settings)?; check_before_sending::<::Query, Self>( &owned_rich, settings.protocol_version, diff --git a/packages/rs-sdk/src/platform/fetch_many.rs b/packages/rs-sdk/src/platform/fetch_many.rs index 2f7bc4acee0..9d6fd7b1284 100644 --- a/packages/rs-sdk/src/platform/fetch_many.rs +++ b/packages/rs-sdk/src/platform/fetch_many.rs @@ -235,21 +235,17 @@ where let mut settings = sdk.query_settings(); let mut owned_rich: >::Query = Self::prepare_query(query.query(&settings)?); - let mut owned_wire: >::Request = owned_rich.query(&settings)?; - let required_protocol_version = owned_wire.needs_protocol_version(); - if required_protocol_version.is_some() { - sdk.ensure_network_serves( - required_protocol_version, - owned_wire.method_name(), - request_settings, - ) - .await?; + let requirement = owned_rich.protocol_requirement(&settings)?; + if requirement.is_some() { + sdk.ensure_network_serves(requirement, request_settings) + .await?; // Asking the network for its version may have raised the version requests are // encoded with: encode the request again at it. settings = sdk.query_settings(); owned_rich = Self::prepare_query(query.query(&settings)?); - owned_wire = owned_rich.query(&settings)?; } + let required_protocol_version = requirement.map(|requirement| requirement.protocol_version); + let owned_wire: >::Request = owned_rich.query(&settings)?; check_before_sending::<>::Query, O>( &owned_rich, settings.protocol_version, diff --git a/packages/rs-sdk/src/platform/fetch_unproved.rs b/packages/rs-sdk/src/platform/fetch_unproved.rs index 5cd07c46d93..62e3ce9f777 100644 --- a/packages/rs-sdk/src/platform/fetch_unproved.rs +++ b/packages/rs-sdk/src/platform/fetch_unproved.rs @@ -71,24 +71,14 @@ where >, { // Default implementation + let requirement = query.protocol_requirement(&sdk.query_settings().without_proofs())?; + let required_protocol_version = requirement.map(|requirement| requirement.protocol_version); + sdk.ensure_network_serves(requirement, Some(request_settings)) + .await?; + // The request is encoded, and its answer read, with the version the SDK speaks once + // the check above has run: asking the network for its version may have raised it. let settings = sdk.query_settings().without_proofs(); - let request: ::Request = query.query(&settings)?; - let required_protocol_version = request.needs_protocol_version(); - sdk.ensure_network_serves( - required_protocol_version, - request.method_name(), - Some(request_settings), - ) - .await?; - // Asking the network for its version may have raised the version the SDK speaks: - // encode the request again at it, as `Fetch` does, and read the answer with it. - let settings = sdk.query_settings().without_proofs(); - let request = if required_protocol_version.is_some() { - query.query(&settings)? - } else { - request - }; - let request = &request; + let request: &::Request = &query.query(&settings)?; let platform_version = settings.protocol_version; let closure = move |local_settings: RequestSettings| async move { // Execute the request using the Sdk instance diff --git a/packages/rs-sdk/src/platform/query.rs b/packages/rs-sdk/src/platform/query.rs index ee94f9fd34b..1622f701a95 100644 --- a/packages/rs-sdk/src/platform/query.rs +++ b/packages/rs-sdk/src/platform/query.rs @@ -109,6 +109,62 @@ pub trait Query: Send + Debug + Clone { /// On success, this method yields an instance of the `TransportRequest` type (`T`). /// On failure, it yields an [`Error`]. fn query(&self, settings: &crate::platform::QuerySettings<'_>) -> Result; + + /// The protocol version from which every node serves the request this query encodes + /// to, or `None` when every node serves it. + /// + /// The fetch paths refuse a request the network is not proven to serve, without sending + /// it ([`Error::ProtocolVersionNotActive`]): nodes below the version would answer a + /// different question or refuse it in a way that gets them banned. The default encodes + /// the query with `settings` and reads [`TransportRequest::needs_protocol_version`]; a + /// query whose features decide it, such as [`DocumentQuery`], overrides it. + /// + /// A document request declares nothing on the wire, so the default refuses a query that + /// encodes one: such a query overrides this, for example from + /// [`DocumentQuery::required_protocol_version`]. + fn protocol_requirement( + &self, + settings: &crate::platform::QuerySettings<'_>, + ) -> Result, Error> + where + T: TransportRequest, + { + let request = self.query(settings)?; + if request.method_name() == proto::GetDocumentsRequest::default().method_name() { + return Err(Error::Config( + "a query that encodes a get_documents request must say which protocol version \ + its features need (Query::protocol_requirement); the wire request does not" + .to_string(), + )); + } + Ok(ProtocolRequirement::of_request(&request)) + } +} + +/// A protocol version a request needs: not every node of a network below it serves the +/// request. See [`Query::protocol_requirement`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ProtocolRequirement { + /// The first protocol version at which every node serves the request. + pub protocol_version: u32, + /// The gRPC method of the request, for example `get_documents`. + pub method: &'static str, + /// The feature of the request that needs the version, when the method alone does not, + /// for example "time-range selections". + pub feature: Option<&'static str>, +} + +impl ProtocolRequirement { + /// What `request` declares it needs ([`TransportRequest::needs_protocol_version`]). + pub fn of_request(request: &R) -> Option { + request + .needs_protocol_version() + .map(|protocol_version| Self { + protocol_version, + method: request.method_name(), + feature: None, + }) + } } /// Marker for wire proto request types that serve as their own [`Query`] @@ -129,6 +185,22 @@ pub trait Query: Send + Debug + Clone { /// `impl_wire_query!` list below; a missing entry fails to compile at the /// fetch call site with a `WireQuery is not satisfied` error. /// +/// `GetDocumentsRequest` is deliberately not in the list. Which networks +/// serve a document request depends on the features of the query it was +/// encoded from ([`DocumentQuery::required_protocol_version`]), which the +/// wire request does not show; fetch documents with a [`DocumentQuery`] or +/// a [`ChainedDocumentQuery`](crate::platform::ChainedDocumentQuery). +/// +/// ``` +/// fn wire_query() {} +/// wire_query::(); +/// ``` +/// +/// ```compile_fail +/// fn wire_query() {} +/// wire_query::(); +/// ``` +/// /// # Breaking change for downstream crates /// /// Before this bound existed, *every* [`TransportRequest`] got `Query for @@ -174,7 +246,6 @@ impl_wire_query!( proto::GetDataContractsByRangeRequest, proto::GetDataContractsRequest, proto::GetDocumentHistoryRequest, - proto::GetDocumentsRequest, proto::GetEpochsInfoRequest, proto::GetEvonodesProposedEpochBlocksByIdsRequest, proto::GetEvonodesProposedEpochBlocksByRangeRequest, @@ -237,6 +308,13 @@ where } Ok(self.clone()) } + + fn protocol_requirement( + &self, + _settings: &crate::platform::QuerySettings<'_>, + ) -> Result, Error> { + Ok(ProtocolRequirement::of_request(self)) + } } impl Query for Identifier { diff --git a/packages/rs-sdk/src/platform/types/epoch.rs b/packages/rs-sdk/src/platform/types/epoch.rs index a7b18082819..b74180f49ed 100644 --- a/packages/rs-sdk/src/platform/types/epoch.rs +++ b/packages/rs-sdk/src/platform/types/epoch.rs @@ -164,7 +164,7 @@ pub(crate) fn current_epoch_probe_query() -> LimitQuery { /// Second step of [`ExtendedEpochInfo::fetch_current`]: fetch `candidate` and the /// epoch above it, so the proof shows whether `candidate` is the newest started /// epoch. -fn current_epoch_confirmation_query(candidate: EpochIndex) -> LimitQuery { +pub(crate) fn current_epoch_confirmation_query(candidate: EpochIndex) -> LimitQuery { LimitQuery { query: EpochQuery::ascending_from(candidate), limit: Some(2), @@ -174,7 +174,7 @@ fn current_epoch_confirmation_query(candidate: EpochIndex) -> LimitQuery EpochIndex { +pub(crate) fn epoch_hint_from_metadata(metadata: &ResponseMetadata) -> EpochIndex { metadata.epoch.min(MAX_EPOCH as u32) as EpochIndex } diff --git a/packages/rs-sdk/src/sdk.rs b/packages/rs-sdk/src/sdk.rs index ef06880eaa9..6626880c326 100644 --- a/packages/rs-sdk/src/sdk.rs +++ b/packages/rs-sdk/src/sdk.rs @@ -328,6 +328,11 @@ pub struct Sdk { /// host can persist the learned version and seed the next SDK with it. protocol_version_observer: Option, + /// What the last probe of the network's protocol version proved about the epoch it ran + /// in, so that the gate does not ask again before that epoch can end, and the probe + /// running, so that calls made meanwhile wait for its answer. Shared between clones. + probe_hold: Arc, + /// Enforce the protocol version gate on a mock SDK, as on a network one. #[cfg(all(test, feature = "mocks"))] enforce_protocol_gate_on_mock: bool, @@ -369,6 +374,7 @@ impl Clone for Sdk { version_pinned: self.version_pinned, verified_protocol_version: Arc::clone(&self.verified_protocol_version), protocol_version_observer: self.protocol_version_observer.clone(), + probe_hold: Arc::clone(&self.probe_hold), #[cfg(all(test, feature = "mocks"))] enforce_protocol_gate_on_mock: self.enforce_protocol_gate_on_mock, metadata_last_seen_height: Arc::clone(&self.metadata_last_seen_height), @@ -1616,6 +1622,7 @@ impl SdkBuilder { protocol_version: Arc::new(atomic::AtomicU32::new(initial_version.protocol_version)), version_pinned: self.version_pinned, verified_protocol_version: Arc::new(atomic::AtomicU32::new(0)), + probe_hold: Arc::default(), protocol_version_observer: self.protocol_version_observer.clone(), #[cfg(all(test, feature = "mocks"))] enforce_protocol_gate_on_mock: false, @@ -1689,6 +1696,7 @@ impl SdkBuilder { protocol_version: Arc::new(atomic::AtomicU32::new(initial_version.protocol_version)), version_pinned: self.version_pinned, verified_protocol_version: Arc::new(atomic::AtomicU32::new(0)), + probe_hold: Arc::default(), protocol_version_observer: self.protocol_version_observer.clone(), #[cfg(all(test, feature = "mocks"))] enforce_protocol_gate_on_mock: false, diff --git a/packages/rs-sdk/src/sdk/document_query_shape_tests.rs b/packages/rs-sdk/src/sdk/document_query_shape_tests.rs index fd99b8d1bce..6ebb918a297 100644 --- a/packages/rs-sdk/src/sdk/document_query_shape_tests.rs +++ b/packages/rs-sdk/src/sdk/document_query_shape_tests.rs @@ -310,6 +310,26 @@ impl FetchMany for CheckedAt { type Request = GetDocumentsRequest; } +/// An unproved view of documents, as a downstream crate may define one. Only its gating is +/// tested: it reads no answer. +impl drive_proof_verifier::unproved::FromUnproved for CheckedAt { + type Request = GetDocumentsRequest; + type Response = GetDocumentsResponse; + + fn maybe_from_unproved_with_metadata, O: Into>( + _: I, + _: O, + _: Network, + _: &PlatformVersion, + ) -> Result<(Option, ResponseMetadata), drive_proof_verifier::Error> { + unreachable!("refused before it is sent") + } +} + +impl crate::platform::FetchUnproved for CheckedAt { + type Request = GetDocumentsRequest; +} + /// A query whose building raises the SDK's protocol version, as a verified /// answer to another request can while this one is being built. #[derive(Clone, Debug)] diff --git a/packages/rs-sdk/src/sdk/protocol_gate.rs b/packages/rs-sdk/src/sdk/protocol_gate.rs index 83e4f9ed161..7be2d4f8e46 100644 --- a/packages/rs-sdk/src/sdk/protocol_gate.rs +++ b/packages/rs-sdk/src/sdk/protocol_gate.rs @@ -1,23 +1,46 @@ //! Refusing requests that not every node of the network serves yet. //! -//! A protocol upgrade can add a method or a request field. Until the network has activated -//! it, nodes of the previous release still answer such a request: with UNIMPLEMENTED, or by -//! dropping the field they do not know and proving a different question. Either answer gets -//! an honest node banned. So before sending a request whose -//! [`TransportRequest::needs_protocol_version`] is set, the fetch paths ask -//! [`Sdk::ensure_network_serves`], which sends it only once a verified answer shows the -//! network runs that version, and they check its answer with [`Sdk::ensure_answer_serves`]. +//! A protocol upgrade can add a method, a request field or a query feature. Until the network +//! has activated it, nodes of the previous release still answer such a request: with +//! UNIMPLEMENTED, by dropping the field they do not know and proving a different question, +//! or by refusing or reinterpreting the feature. Several of these answers get an honest node +//! banned, and none serves the request. So before sending a request whose +//! [`Query::protocol_requirement`] is set (a request's +//! [`TransportRequest::needs_protocol_version`], or the features of a document query), the +//! fetch paths ask [`Sdk::ensure_network_serves`], which sends it only once a verified answer +//! shows the network runs that version, and they check its answer with +//! [`Sdk::ensure_answer_serves`]. +//! +//! The protocol version changes only when an epoch starts. A probe that proves the network +//! below a requirement also proves the current epoch and when it started ([`ProbeHold`]), and +//! the gate refuses requests that need more without asking again until the current epoch can +//! have ended. Any verified answer that shows the network at the requirement, or at any version +//! newer than the probe saw, lifts the refusal at once. +//! +//! The hold saves probes; it does not promise one per epoch. One answer cannot tell a local +//! clock running ahead of block time from an answer that is old, and the hold takes the +//! shorter reading, so a clock ahead ends it early by the difference and the next request asks +//! again: a hold too short costs a query, one too long refuses what the network serves. //! //! [`TransportRequest::needs_protocol_version`]: rs_dapi_client::transport::TransportRequest::needs_protocol_version use super::{Sdk, SdkInstance}; use crate::error::{Error, StaleNodeError}; -use crate::platform::types::epoch::current_epoch_probe_query; -use crate::platform::{Fetch, Query, QuerySettings}; +use crate::platform::types::epoch::{ + current_epoch_confirmation_query, current_epoch_probe_query, epoch_hint_from_metadata, +}; +use crate::platform::{Fetch, FetchMany, ProtocolRequirement, Query, QuerySettings}; use dapi_grpc::platform::v0::ResponseMetadata; +use dpp::block::epoch::EpochIndex; +use dpp::block::extended_epoch_info::v0::ExtendedEpochInfoV0Getters; use dpp::block::extended_epoch_info::ExtendedEpochInfo; +use dpp::fee::epoch::GENESIS_EPOCH_INDEX; use dpp::version::PlatformVersion; use rs_dapi_client::RequestSettings; +use std::cmp::Ordering as Order; +use std::collections::BTreeMap; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::Mutex; /// What to do with a request that needs a protocol version. #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -42,6 +65,109 @@ pub(super) struct GateState<'a> { /// Whether the SDK enforces the gate. Mock SDKs answer what a test registered and have /// no nodes to ban, and their expectations are never verified answers. pub(super) enforced: bool, + /// What the last probe proved about the epoch it ran in, if anything. + pub(super) hold: Option, + /// The local clock, in milliseconds since the Unix epoch. + pub(super) now_ms: u64, +} + +/// What a probe proved: the network ran protocol version `network` in an epoch that may not +/// end before `until_ms`, on the local clock. The protocol version changes only when an epoch +/// starts, so until then no answer can show the network serving a newer version. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(super) struct ProbeHold { + /// The newest protocol version a verified answer was signed at when the probe ran. + pub(super) network: u32, + /// The probed epoch. + pub(super) epoch: EpochIndex, + /// When the probed epoch has ended at the latest, on the local clock, in milliseconds + /// since the Unix epoch. + pub(super) until_ms: u64, +} + +impl ProbeHold { + /// The hold from verified start times: of the genesis epoch (`genesis_ms`) and of the + /// probed epoch, `current`, which started at `current_ms`. `block_time_ms` is the verified + /// time of the answer that proved them, and `local_now_ms` the local clock when it arrived. + /// + /// Epochs follow a fixed grid: epoch `n` runs from `n` epoch lengths after genesis to + /// `n + 1`, and starts with its first block. Epoch `current` started no earlier than + /// `current` lengths after genesis, so the length is at most + /// `(current_ms - genesis_ms) / current`, and the epoch ends no later than that much after + /// it started. The hold outlasts the epoch by about how late the epoch's first block came + /// after the epoch's grid start (for example after a halt). + /// + /// The hold is measured on the local clock from when the answer arrived, so a local clock + /// running behind block time does not lengthen it, and it never runs past the bound by + /// block time, so an answer older than the local clock does not lengthen it either. `None` + /// for the genesis epoch, or when the times are not in order. + pub(super) fn from_epoch_starts( + network: u32, + genesis_ms: u64, + current: EpochIndex, + current_ms: u64, + block_time_ms: u64, + local_now_ms: u64, + ) -> Option { + let longest_epoch = current_ms + .checked_sub(genesis_ms)? + .checked_div(u64::from(current)) + .filter(|length| *length > 0)?; + let ends_by_block_time = current_ms.checked_add(longest_epoch)?; + let ends_by_local_time = + local_now_ms.checked_add(ends_by_block_time.saturating_sub(block_time_ms))?; + Some(Self { + network, + epoch: current, + until_ms: ends_by_local_time.min(ends_by_block_time), + }) + } + + /// Whether the hold still shows, at `now_ms`, that the network does not serve `required`. + /// A verified answer newer than the probe's (`verified`) shows the epoch changed since. + fn covers(&self, required: u32, verified: u32, now_ms: u64) -> bool { + self.network < required && verified <= self.network && now_ms < self.until_ms + } +} + +/// The [`ProbeHold`] an SDK and its clones share, and the probe one of them is running. +#[derive(Debug, Default)] +pub(super) struct SharedProbeHold { + hold: Mutex>, + /// Locked while a probe runs, so that calls made meanwhile wait for its answer instead + /// of asking too. + probing: tokio::sync::Mutex<()>, + /// How many probes the network answered, so that a call that waited for one can tell + /// that it was answered. + answered: AtomicU64, +} + +impl SharedProbeHold { + pub(super) fn get(&self) -> Option { + self.hold.lock().map(|hold| *hold).unwrap_or(None) + } + + /// Keep the hold of the newest epoch proved: of the newer version, then of the later + /// epoch, whichever probe finished last. Two holds of one epoch both bound when that + /// epoch has ended, so the earlier bound is kept. + pub(super) fn set(&self, hold: ProbeHold) { + if let Ok(mut current) = self.hold.lock() { + let newer = current.is_none_or(|held| { + match (hold.network, hold.epoch).cmp(&(held.network, held.epoch)) { + Order::Greater => true, + Order::Equal => hold.until_ms < held.until_ms, + Order::Less => false, + } + }); + if newer { + *current = Some(hold); + } + } + } + + fn answered(&self) -> u64 { + self.answered.load(Ordering::Acquire) + } } /// Whether a request that needs `required` is sent, refused, or sent only after asking the @@ -53,8 +179,17 @@ pub(super) fn decide(required: u32, state: &GateState<'_>) -> Gate { // Asking raises the verified version, and the version of an unpinned SDK; a pinned SDK // below `required` stays below it whatever the answer. Without a context provider the // answer cannot be verified, and an answer the SDK cannot verify bans the node that - // sent it. - if pinned_below(required, state) || !state.has_context_provider { + // sent it. While a probe's epoch has not ended, asking again would show the same version. + if pinned_below(required, state) + || !state.has_context_provider + || state.hold.is_some_and(|hold| { + hold.covers( + required, + state.settings.verified_protocol_version, + state.now_ms, + ) + }) + { Gate::Refuse } else { Gate::Probe @@ -89,6 +224,8 @@ impl Sdk { pinned: self.version_pinned, has_context_provider: self.context_provider().is_some(), enforced: self.enforces_protocol_gate(), + hold: self.probe_hold.get(), + now_ms: chrono::Utc::now().timestamp_millis() as u64, } } @@ -117,19 +254,21 @@ impl Sdk { self } - /// Refuse `method`'s request, which needs protocol version `required` (see - /// [`TransportRequest::needs_protocol_version`]), unless a verified answer shows the - /// network runs it and this SDK speaks it. When the network may have reached it since, - /// ask it first with one proved query, under the caller's `settings`. - /// - /// [`TransportRequest::needs_protocol_version`]: rs_dapi_client::transport::TransportRequest::needs_protocol_version + /// Refuse a request that needs `requirement` (see [`Query::protocol_requirement`]), + /// unless a verified answer shows the network runs its protocol version and this SDK + /// speaks it. When the network may have reached it since, ask it first with one proved + /// query, under the caller's `settings`. pub(crate) async fn ensure_network_serves( &self, - required: Option, - method: &'static str, + requirement: Option, settings: Option, ) -> Result<(), Error> { - let Some(required) = required else { + let Some(ProtocolRequirement { + protocol_version: required, + method, + feature, + }) = requirement + else { return Ok(()); }; let state = self.gate_state(); @@ -137,11 +276,7 @@ impl Sdk { let cause = match decide(required, &state) { Gate::Send => return Ok(()), Gate::Refuse => None, - Gate::Probe => self - .probe_network_protocol_version(settings) - .await - .err() - .map(Box::new), + Gate::Probe => self.probe_unless_probing(required, settings).await, }; // Another call may have verified the version meanwhile, whatever this probe did. let network = self.query_settings().network_protocol_basis(); @@ -151,17 +286,55 @@ impl Sdk { Err(Error::ProtocolVersionNotActive { method, required, + feature, network, pinned, cause, }) } + /// Probe for `required` ([`Self::probe_network_protocol_version`]) and return why the + /// probe failed, if it did, unless another call is probing: then wait for its answer and + /// decide from it, as the call that probed does. + /// + /// A call waits for one probe at most, which runs under the settings of the call that + /// started it. When that probe failed, or its call was dropped, the call asks under its + /// own settings at once, without waiting for the others that ask too. + async fn probe_unless_probing( + &self, + required: u32, + settings: Option, + ) -> Option> { + let answered = self.probe_hold.answered(); + let probing = self.probe_hold.probing.try_lock(); + if probing.is_err() { + drop(self.probe_hold.probing.lock().await); + } + // A probe answered since this call decided to ask, or another answer decided it. + if self.probe_hold.answered() != answered + || decide(required, &self.gate_state()) != Gate::Probe + { + return None; + } + let probed = self + .probe_network_protocol_version(required, settings) + .await; + if probed.is_ok() { + self.probe_hold.answered.fetch_add(1, Ordering::AcqRel); + } + drop(probing); + probed.err().map(Box::new) + } + /// Learn the network's protocol version from the genesis epoch, which every node serves /// at every version: the first query of `ExtendedEpochInfo::fetch_current`. It asks for /// a proof whatever the SDK's proof setting, since only a verified answer counts. + /// + /// When the answer shows the network below `required`, also prove the current epoch, + /// to hold further probes until it can end ([`Self::hold_probes`]). async fn probe_network_protocol_version( &self, + required: u32, settings: Option, ) -> Result<(), Error> { let request = current_epoch_probe_query().query(&QuerySettings { @@ -175,10 +348,79 @@ impl Sdk { // about the nodes, so it bans none of them. settings.ban_failed_address = Some(false); } - ExtendedEpochInfo::fetch_with_metadata(self, request, Some(settings)).await?; + let (genesis, metadata) = + ExtendedEpochInfo::fetch_with_metadata(self, request, Some(settings)).await?; + if let (Some(genesis), true) = ( + genesis.filter(|genesis| genesis.index() == GENESIS_EPOCH_INDEX), + self.query_settings().network_protocol_basis() < required, + ) { + self.hold_probes( + genesis.first_block_time(), + epoch_hint_from_metadata(&metadata), + settings, + ) + .await; + } Ok(()) } + /// Prove that `hinted` is the current epoch and when it started, and hold probes until it + /// can have ended ([`ProbeHold`]), from the start of the genesis epoch, `genesis_ms`, which + /// the probe proved. + /// + /// The epoch index in response metadata is not covered by the quorum signature, so the + /// hint only shapes this proved query; the hold is built from what its proof shows. When + /// it shows something else, the query fails, or `hinted` is the genesis epoch, nothing + /// is held and the next request that needs more asks again. + async fn hold_probes(&self, genesis_ms: u64, hinted: EpochIndex, settings: RequestSettings) { + if hinted == GENESIS_EPOCH_INDEX { + return; + } + let request = match current_epoch_confirmation_query(hinted).query(&QuerySettings { + prove: true, + ..self.query_settings() + }) { + Ok(request) => request, + Err(error) => { + tracing::debug!(%error, "cannot ask for the current epoch; probes are not held"); + return; + } + }; + let (epochs, metadata) = match ExtendedEpochInfo::fetch_many_with_metadata( + self, + request, + Some(settings), + ) + .await + { + Ok(answer) => answer, + Err(error) => { + tracing::debug!(%error, "the current epoch is not proven; probes are not held"); + return; + } + }; + let local_now_ms = chrono::Utc::now().timestamp_millis() as u64; + // The query reads `hinted` and the epoch above: `hinted` is current when it is the + // only one that started. + let started: BTreeMap = epochs + .into_iter() + .filter_map(|(index, epoch)| epoch.map(|epoch| (index, epoch))) + .collect(); + let (Some(current), 1) = (started.get(&hinted), started.len()) else { + return; + }; + if let Some(hold) = ProbeHold::from_epoch_starts( + metadata.protocol_version, + genesis_ms, + hinted, + current.first_block_time(), + metadata.time_ms, + local_now_ms, + ) { + self.probe_hold.set(hold); + } + } + /// Refuse an answer to a request that needs protocol version `required` when it is /// signed at an older one: only a node behind the network's activation of `required` /// signs it, and it answered under rules it does not follow. diff --git a/packages/rs-sdk/src/sdk/protocol_gate_tests.rs b/packages/rs-sdk/src/sdk/protocol_gate_tests.rs index f00147f2a87..4661e946ba9 100644 --- a/packages/rs-sdk/src/sdk/protocol_gate_tests.rs +++ b/packages/rs-sdk/src/sdk/protocol_gate_tests.rs @@ -4,7 +4,9 @@ //! Network answers are signed vectors generated by drive-abci's `protocol_gate` test //! target, replayed through the normal proof and signature verifier. use super::document_query_shape_tests::CheckedAt; -use super::protocol_gate::{decide, version_for_requests, Gate, GateState}; +use super::protocol_gate::{ + decide, version_for_requests, Gate, GateState, ProbeHold, SharedProbeHold, +}; use super::{Sdk, SdkInstance}; use crate::error::{Error, StaleNodeError}; use crate::platform::contract_fee_pots::ContractFeePots; @@ -19,6 +21,7 @@ use crate::platform::data_contracts_by_range::DataContractsByRange; use crate::platform::data_contracts_latest_versions::{ DataContractLatestVersion, DataContractsLatestVersions, DataContractsLatestVersionsQuery, }; +use crate::platform::documents::document_query::RankingDirection; use crate::platform::identity_keys_remaining_budgets::IdentityKeysRemainingBudgets; use crate::platform::private_document_entries::PrivateDocumentEntries; use crate::platform::types::epoch::current_epoch_probe_query; @@ -78,13 +81,21 @@ use dpp::version::{PlatformVersion, PLATFORM_VERSIONS}; use dpp::voting::vote_polls::yes_no_vote_poll::{ YesNoMinimumVotingPower, YesNoVotePoll, YesNoVotePollPurpose, }; -use drive::query::{WhereClause as DriveWhereClause, WhereOperator}; +use drive::query::{ + HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, + SelectProjection, TimeRangeSelector, WhereClause as DriveWhereClause, WhereOperator, +}; use drive_proof_verifier::types::{ Documents, MostRecentShieldedAnchor, ShieldedAnchors, ShieldedEncryptedNotes, ShieldedNotesCount, ShieldedNullifierStatuses, ShieldedPoolState, }; +use drive_proof_verifier::{ + DocumentAverage, DocumentCount, DocumentHavingEntries, DocumentRankedEntries, + DocumentSplitAverages, DocumentSplitCounts, DocumentSplitSums, DocumentSum, +}; use rs_dapi_client::mock::MockDapiClient; use rs_dapi_client::transport::{TransportError, TransportRequest}; +use rs_dapi_client::CanRetry; use rs_dapi_client::{ Address, DapiClientError, ExecutionError, ExecutionResponse, RequestSettings, }; @@ -298,6 +309,7 @@ fn assert_refused( Err(Error::ProtocolVersionNotActive { method: refused, required, + feature: _, network: proven, pinned: refused_for_pin, cause: None, @@ -377,6 +389,8 @@ fn should_decide_from_the_versions_the_network_and_this_sdk_are_proven_to_serve( pinned, has_context_provider, enforced, + hold: None, + now_ms: 0, }, ) }; @@ -454,6 +468,76 @@ fn should_decide_from_the_versions_the_network_and_this_sdk_are_proven_to_serve( assert_eq!(decision(V13, NEWER, true, true, true), Gate::Refuse); } +/// A probe that proved the network below a requirement in an epoch holds further probes +/// for that requirement, or a later one, until the epoch can have ended; the hold neither +/// refuses what the network is proven to serve nor outlives the epoch. +#[test] +fn should_hold_probes_until_the_probed_epoch_can_end() { + let request_settings = RequestSettings::default(); + let hour = 3_600_000; + // Epoch 2 of a network with one-hour epochs, whose first block came a minute late. Its + // bound by block time is when epoch 2 has lasted as long as an epoch can have. + let started = 2 * hour + 60_000; + let ends_by_block_time = started + started / 2; + assert!(ends_by_block_time >= 3 * hour, "not before the grid's end"); + let answered = 2 * hour + hour / 2; + let hold_from = |local_now_ms| { + ProbeHold::from_epoch_starts(V13, 0, 2, started, answered, local_now_ms) + .expect("times in order") + }; + assert_eq!( + hold_from(answered - hour).until_ms, + ends_by_block_time - hour, + "a local clock behind block time does not lengthen it" + ); + assert_eq!( + hold_from(answered + hour).until_ms, + ends_by_block_time, + "an answer older than the local clock does not lengthen it; read as a local clock \ + ahead of block time, the hold ends early by the difference and the next request asks" + ); + assert_eq!( + ProbeHold::from_epoch_starts(V13, 0, 0, 0, hour, 10 * hour), + None, + "the genesis epoch has no length to read" + ); + assert_eq!( + ProbeHold::from_epoch_starts(V13, 3 * hour, 2, started, 3 * hour, 10 * hour), + None, + "an epoch that starts before genesis" + ); + let hold = hold_from(answered); + let until = hold.until_ms; + let decide_at = |required: u32, verified: u32, now_ms: u64| { + decide( + required, + &GateState { + settings: QuerySettings { + request_settings: &request_settings, + protocol_version: PlatformVersion::get(V13).unwrap(), + verified_protocol_version: verified, + prove: true, + }, + pinned: false, + has_context_provider: true, + enforced: true, + hold: Some(hold), + now_ms, + }, + ) + }; + for required in [V14, V14 + 1] { + assert_eq!(decide_at(required, V13, answered + 1), Gate::Refuse); + assert_eq!(decide_at(required, V13, until - 1), Gate::Refuse); + assert_eq!(decide_at(required, V13, until), Gate::Probe); + } + // A verified answer newer than the probe's shows the epoch changed since: the hold no + // longer holds what the newer version does not serve either. + assert_eq!(decide_at(V14 + 1, V14, answered + 1), Gate::Probe); + // A hold at 13 says nothing against what 13 serves. + assert_eq!(decide_at(V13, V13, answered + 1), Gate::Send); +} + /// A fresh SDK has verified nothing. Its first gated call asks the network for its version /// once, then sends; later calls do not ask again. #[tokio::test] @@ -580,6 +664,7 @@ async fn should_refuse_with_the_cause_when_the_network_version_cannot_be_learned Err(Error::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: V14, + feature: None, network: 0, pinned: false, cause: Some(_), @@ -1286,13 +1371,12 @@ async fn should_speak_the_newest_known_version_on_a_network_proven_newer() { else { panic!("non-unique lookup request") }; - let contains: Result = plain_query() - .with_where(DriveWhereClause { - field: "records.identity".to_string(), - operator: WhereOperator::Contains, - value: dpp::platform_value::Value::Identifier([1; 32]), - }) - .query(&settings); + let contains_query = plain_query().with_where(DriveWhereClause { + field: "records.identity".to_string(), + operator: WhereOperator::Contains, + value: dpp::platform_value::Value::Identifier([1; 32]), + }); + let contains: Result = contains_query.query(&settings); let speaks = settings.protocol_version.protocol_version; @@ -1305,10 +1389,21 @@ async fn should_speak_the_newest_known_version_on_a_network_proven_newer() { lookup.version, Some(get_identity_by_non_unique_public_key_hash_request::Version::V0(_)) )); + // A pin at 13 keeps the 13 rules, whatever the network runs: the SDK refuses + // what the network serves but it does not, without asking again. + let fetched = Document::fetch_many(&sdk, contains_query).await; assert!( - matches!(contains, Err(Error::Config(_))), - "a pin at 13 keeps the 13 grammar: {contains:?}" + matches!( + &fetched, + Err(Error::ProtocolVersionNotActive { + feature: Some("contains and containsAny clauses"), + .. + }) + ), + "{fetched:?}" ); + assert_refused(fetched, "get_documents", V13, true, "contains pinned at 13"); + assert_eq!(probes(&sdk).await, 1, "no probe for a pin"); } else { result.expect("sent to a network proven newer"); assert_eq!(speaks, PlatformVersion::latest().protocol_version); @@ -1349,6 +1444,651 @@ fn should_learn_nothing_from_metadata_it_did_not_verify() { } } +/// `result` is the gate's refusal of a `get_documents` request for `feature`, with the +/// network proven at `network`. +fn assert_refused_for( + result: Result, + feature: &str, + network: u32, + context: &str, +) { + match result { + Err(Error::ProtocolVersionNotActive { + method: "get_documents", + required: V14, + feature: Some(refused), + network: proven, + pinned: false, + cause: None, + }) => assert_eq!((refused, proven), (feature, network), "{context}"), + other => panic!("{context}: expected a refusal of {feature}, got {other:?}"), + } +} + +/// A `domain` query that reads the elements of `records.identity`. +fn contains_query() -> DocumentQuery { + plain_query().with_where(DriveWhereClause { + field: "records.identity".to_string(), + operator: WhereOperator::Contains, + value: dpp::platform_value::Value::Identifier([1; 32]), + }) +} + +/// A `tip` contract whose documents are summed by `amount`. +fn tips() -> Arc { + let schemas = dpp::platform_value::platform_value!({"tip": { + "type": "object", + "documentsSummable": "amount", + "properties": { + "recipient": {"type": "string", "minLength": 1, "maxLength": 63, "position": 0}, + "amount": {"type": "integer", "minimum": 0, "maximum": 4294967295u64, "position": 1} + }, + "required": ["recipient", "amount"], + "indices": [ + {"name": "byRecipient", "properties": [{"recipient": "asc"}], "summable": "amount"} + ], + "additionalProperties": false + }}); + Arc::new( + dpp::data_contract::DataContractFactory::new(PlatformVersion::latest().protocol_version) + .unwrap() + .create_with_value_config(Identifier::new([8; 32]), 0, schemas, None, None) + .unwrap() + .data_contract_owned(), + ) +} + +/// A fresh SDK seeded at 13 on a network at 14 asks the network for its version once, then +/// sends a query that uses a protocol 14 feature. Refusing it at the seed would keep the +/// feature from every SDK that starts below the network's version. +#[tokio::test] +async fn should_ask_then_send_a_protocol_14_document_query_from_an_sdk_seeded_at_13() { + let composite_contains = || { + composite_query().with_where(DriveWhereClause { + field: "records.identity".to_string(), + operator: WhereOperator::Contains, + value: dpp::platform_value::Value::Identifier([1; 32]), + }) + }; + let mut sdk = mock_sdk(V13, false, 0); + answer_probe(&sdk, "pv14").await; + sdk.mock() + .expect_fetch_many(contains_query(), None::) + .await + .expect("a seeded SDK encodes the query"); + sdk.mock() + .expect_fetch(composite_contains(), None::) + .await + .expect("a seeded SDK encodes the query"); + Document::fetch_many(&sdk, contains_query()) + .await + .expect("sent once the network is proven at 14"); + CompositeDocuments::fetch(&sdk, composite_contains()) + .await + .expect("sent once the network is proven at 14"); + assert_eq!(probes(&sdk).await, 1, "one question for both"); + assert_eq!(sent(&sdk, &encoded(&sdk, &contains_query())).await, 1); + assert_eq!(sent(&sdk, &encoded(&sdk, &composite_contains())).await, 1); + assert!(!sdk.address_list().is_banned(&node())); +} + +/// On a network proven at 13 every protocol 14 document feature is refused alike, naming the +/// feature, and nothing is sent: nodes at 13 would refuse it, serve it as another question, +/// or fail in a way that gets them banned. +#[tokio::test] +async fn should_refuse_every_protocol_14_document_feature_on_a_network_proven_at_13() { + let sdk = mock_sdk(V13, false, V13); + answer_probe(&sdk, "pv13").await; + let ranked = plain_query() + .with_select(SelectProjection::count_star()) + .with_group_by("normalizedParentDomainName") + .order_by_selected_aggregate(RankingDirection::Descending) + .with_limit(5); + let having = plain_query() + .with_select(SelectProjection::count_star()) + .with_group_by("normalizedParentDomainName") + .with_having(vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(dpp::platform_value::Value::U64(1)), + }]) + .with_limit(5); + let several_ins = plain_query() + .with_where(DriveWhereClause { + field: "normalizedParentDomainName".to_string(), + operator: WhereOperator::In, + value: dpp::platform_value::platform_value!(["dash", "test"]), + }) + .with_where(DriveWhereClause { + field: "normalizedLabel".to_string(), + operator: WhereOperator::In, + value: dpp::platform_value::platform_value!(["a", "b"]), + }); + let mut cursor_on_tips = DocumentQuery::new(tips(), "tip").unwrap().with_limit(10); + cursor_on_tips.start = + Some(get_documents_request::get_documents_request_v0::Start::StartAfter(vec![1; 32])); + let time_range = DocumentQuery::new(dpns(), "domain") + .unwrap() + .with_select(SelectProjection::count_star()) + .with_time_range("$createdAt", TimeRangeSelector::Newest); + let integer_range = DocumentQuery::new(dpns(), "domain") + .unwrap() + .with_select(SelectProjection::count_star()) + .with_integer_range("$revision", dpp::platform_value::Value::U64(1)); + + let cases: Vec<(&str, DocumentQuery, Result<(), Error>)> = vec![ + ( + "contains and containsAny clauses", + contains_query(), + Document::fetch_many(&sdk, contains_query()).await.map(drop), + ), + ( + "time-range selections", + time_range.clone(), + DocumentCount::fetch(&sdk, time_range).await.map(drop), + ), + ( + "integer-range selections", + integer_range.clone(), + DocumentCount::fetch(&sdk, integer_range).await.map(drop), + ), + ( + "aggregates ordered by the selected aggregate", + ranked.clone(), + DocumentRankedEntries::fetch(&sdk, ranked).await.map(drop), + ), + ( + "having clauses", + having.clone(), + DocumentHavingEntries::fetch(&sdk, having).await.map(drop), + ), + ( + "several in clauses", + several_ins.clone(), + Document::fetch_many(&sdk, several_ins).await.map(drop), + ), + ( + "start cursors on a type whose documents are summed", + cursor_on_tips.clone(), + Document::fetch_many(&sdk, cursor_on_tips).await.map(drop), + ), + ]; + for (feature, query, result) in cases { + assert_refused_for(result, feature, V13, feature); + assert_eq!(sent(&sdk, &encoded(&sdk, &query)).await, 0, "{feature}"); + } + assert!(!sdk.address_list().is_banned(&node())); +} + +/// The requirement belongs to the query, not to a result view, so every fetch of a document +/// query is gated: the SDK's own views, both fetch paths, and a view a downstream crate +/// defines over the same query. +#[tokio::test] +async fn should_gate_every_fetch_of_a_document_query() { + let sdk = mock_sdk(V13, false, V13); + answer_probe(&sdk, "pv13").await; + let composite = encoded(&sdk, &composite_query()); + let q = composite_query; + let results: Vec<(&str, Result<(), Error>)> = vec![ + ("Document", Document::fetch(&sdk, q()).await.map(drop)), + ( + "Document many", + Document::fetch_many(&sdk, q()).await.map(drop), + ), + ( + "DocumentCount", + DocumentCount::fetch(&sdk, q()).await.map(drop), + ), + ( + "DocumentSplitCounts", + DocumentSplitCounts::fetch(&sdk, q()).await.map(drop), + ), + ("DocumentSum", DocumentSum::fetch(&sdk, q()).await.map(drop)), + ( + "DocumentSplitSums", + DocumentSplitSums::fetch(&sdk, q()).await.map(drop), + ), + ( + "DocumentAverage", + DocumentAverage::fetch(&sdk, q()).await.map(drop), + ), + ( + "DocumentSplitAverages", + DocumentSplitAverages::fetch(&sdk, q()).await.map(drop), + ), + ( + "DocumentRankedEntries", + DocumentRankedEntries::fetch(&sdk, q()).await.map(drop), + ), + ( + "DocumentHavingEntries", + DocumentHavingEntries::fetch(&sdk, q()).await.map(drop), + ), + ( + "CompositeDocuments", + CompositeDocuments::fetch(&sdk, q()).await.map(drop), + ), + ( + "a downstream view", + CheckedAt::fetch(&sdk, q()).await.map(drop), + ), + ( + "a downstream view, many", + CheckedAt::fetch_many(&sdk, q()).await.map(drop), + ), + ]; + for (view, result) in results { + assert_refused_for(result, "composite sub-queries", V13, view); + } + assert_refused_for( + ChainedDocuments::fetch(&sdk, chained_query()).await, + "chained queries", + V13, + "ChainedDocuments", + ); + assert_eq!(sent(&sdk, &composite).await, 0); + assert_eq!(sent(&sdk, &encoded(&sdk, &chained_query())).await, 0); +} + +/// An unproved fetch of a document query is gated as a proved one is. +#[tokio::test] +async fn should_gate_an_unproved_fetch_of_a_document_query() { + let sdk = mock_sdk(V13, false, V13); + answer_probe(&sdk, "pv13").await; + assert_refused_for( + CheckedAt::fetch_unproved_with_settings(&sdk, composite_query(), once()) + .await + .map(drop), + "composite sub-queries", + V13, + "unproved", + ); + assert_eq!(sent(&sdk, &encoded(&sdk, &composite_query())).await, 0); +} + +/// A query that encodes a document request without saying what its features need is refused +/// rather than sent ungated: the wire request declares nothing. +#[test] +fn should_refuse_a_document_query_that_does_not_state_its_requirement() { + #[derive(Debug, Clone)] + struct RawDocuments(GetDocumentsRequest); + impl Query for RawDocuments { + fn query(&self, _: &QuerySettings<'_>) -> Result { + Ok(self.0.clone()) + } + } + let request_settings = RequestSettings::default(); + let settings = QuerySettings { + request_settings: &request_settings, + protocol_version: PlatformVersion::latest(), + verified_protocol_version: 0, + prove: true, + }; + let raw = RawDocuments(plain_query().query(&settings).unwrap()); + assert!(matches!( + raw.protocol_requirement(&settings), + Err(Error::Config(_)) + )); +} + +/// A mock SDK does not enforce the gate: it sends a query that uses a protocol 14 feature +/// whatever version it starts at, as a test registered it. +#[tokio::test] +async fn should_send_protocol_14_document_queries_from_a_mock_that_does_not_enforce_the_gate() { + let mut sdk = SdkBuilder::new_mock() + .with_initial_version(PlatformVersion::get(V13).unwrap()) + .build() + .unwrap(); + sdk.mock() + .expect_fetch_many(contains_query(), None::) + .await + .expect("a mock at 13 encodes the query"); + Document::fetch_many(&sdk, contains_query()) + .await + .expect("a mock sends what a test registered"); +} + +/// The confirmation the SDK sends when the probe answer, signed below the requirement, names +/// epoch 2: epochs 2 and 3. +fn current_epoch_request() -> GetEpochsInfoRequest { + GetEpochsInfoRequest { + version: Some(get_epochs_info_request::Version::V0( + get_epochs_info_request::GetEpochsInfoRequestV0 { + start_epoch: Some(2), + count: 2, + ascending: true, + prove: true, + }, + )), + } +} + +/// The protocol version changes only when an epoch starts. Once a probe proves the network at +/// 13 in an epoch, the SDK and its clones refuse what needs 14 without asking again until the +/// epoch can end; a verified answer that shows 14 lifts the refusal at once. +#[tokio::test] +async fn should_not_ask_again_before_the_probed_epoch_can_end() { + let mut sdk = mock_sdk(V13, false, 0); + answer( + &sdk, + &probe_request(), + Ok(vector("epoch-probe-pv13-epoch-2.bin")), + ) + .await; + answer( + &sdk, + ¤t_epoch_request(), + Ok(vector("current-epoch-pv13-epoch-2.bin")), + ) + .await; + expect_fee_pots(&mut sdk).await; + for call in 0..2 { + assert_refused( + ContractFeePots::fetch(&sdk, fee_pots()).await, + "get_contract_fee_pots", + V13, + false, + &format!("call {call}"), + ); + } + assert_refused_for( + CompositeDocuments::fetch(&sdk.clone(), composite_query()).await, + "composite sub-queries", + V13, + "a clone, another request", + ); + assert_eq!(probes(&sdk).await, 1, "one probe for the epoch"); + assert_eq!(sent(&sdk, ¤t_epoch_request()).await, 1); + + sdk.learn_verified_protocol_version(V14); + ContractFeePots::fetch(&sdk, fee_pots()) + .await + .expect("sent once a verified answer shows 14"); + assert_eq!(probes(&sdk).await, 1); +} + +/// A probe whose epoch, by its proven start, has already ended holds nothing: the network may +/// have moved since, so the next request that needs 14 asks again. +#[tokio::test] +async fn should_ask_again_once_the_probed_epoch_can_have_ended() { + let mut sdk = mock_sdk(V13, false, 0); + answer( + &sdk, + &probe_request(), + Ok(vector("epoch-probe-pv13-epoch-2-ended.bin")), + ) + .await; + answer( + &sdk, + ¤t_epoch_request(), + Ok(vector("current-epoch-pv13-epoch-2-ended.bin")), + ) + .await; + expect_fee_pots(&mut sdk).await; + for call in 0..2 { + assert_refused( + ContractFeePots::fetch(&sdk, fee_pots()).await, + "get_contract_fee_pots", + V13, + false, + &format!("call {call}"), + ); + } + assert_eq!(probes(&sdk).await, 2); + assert_eq!(sent(&sdk, ¤t_epoch_request()).await, 2); +} + +/// Once the network is proven at 14, a node still at 13 that answers a composite query is +/// behind its network: its answer is not verified, and the node is banned. +#[tokio::test] +async fn should_ban_a_node_that_answers_a_document_query_below_its_requirement() { + let sdk = mock_sdk(V14, false, V14); + let request = encoded(&sdk, &composite_query()); + let behind = dapi_grpc::platform::v0::GetDocumentsResponse { + version: Some(dapi_grpc::platform::v0::get_documents_response::Version::V0( + dapi_grpc::platform::v0::get_documents_response::GetDocumentsResponseV0 { + result: Some( + dapi_grpc::platform::v0::get_documents_response::get_documents_response_v0::Result::Proof( + dapi_grpc::platform::v0::Proof { + grovedb_proof: vec![1; 32], + ..Default::default() + }, + ), + ), + metadata: Some(ResponseMetadata { + height: 10, + protocol_version: V13, + ..Default::default() + }), + }, + )), + }; + answer(&sdk, &request, Ok(behind)).await; + let error = CompositeDocuments::fetch_with_settings(&sdk, composite_query(), once()) + .await + .expect_err("an answer signed below the network's version"); + assert!(error.can_retry(), "{error:?}"); + assert!(sdk.address_list().is_banned(&node())); +} + +/// Without proof of the epoch the probe ran in, nothing is held: an answer from the genesis +/// epoch has no epoch before it, and a confirmation that fails proves nothing. The next +/// request that needs 14 asks again. +#[tokio::test] +async fn should_ask_again_when_the_probed_epoch_is_not_proven() { + let mut sdk = mock_sdk(V13, false, 0); + answer_probe(&sdk, "pv13").await; + expect_fee_pots(&mut sdk).await; + for _ in 0..2 { + assert_refused( + ContractFeePots::fetch(&sdk, fee_pots()).await, + "get_contract_fee_pots", + V13, + false, + "genesis epoch", + ); + } + assert_eq!(probes(&sdk).await, 2); + + let mut sdk = mock_sdk(V13, false, 0); + answer( + &sdk, + &probe_request(), + Ok(vector("epoch-probe-pv13-epoch-2.bin")), + ) + .await; + answer( + &sdk, + ¤t_epoch_request(), + Err(Status::unavailable("no epochs today")), + ) + .await; + expect_fee_pots(&mut sdk).await; + for _ in 0..2 { + assert_refused( + ContractFeePots::fetch_with_settings(&sdk, fee_pots(), once()).await, + "get_contract_fee_pots", + V13, + false, + "unconfirmed epoch", + ); + } + assert_eq!(probes(&sdk).await, 2); +} + +/// Two calls for fee pots made together, each with one attempt: the node answers only once +/// both wait for it. +async fn fetch_fee_pots_together(sdk: &Sdk) -> [Result, Error>; 2] { + let node = dapi(sdk).lock().await; + let answer_both = async move { + tokio::task::yield_now().await; + drop(node); + }; + let (first, second, ()) = tokio::join!( + ContractFeePots::fetch_with_settings(sdk, fee_pots(), once()), + ContractFeePots::fetch_with_settings(sdk, fee_pots(), once()), + answer_both, + ); + [first, second] +} + +/// Calls made while another call asks the network for its version wait for that answer and +/// decide from it, instead of each asking again: one probe serves them, whether it shows the +/// network serves the version or below it, with a hold or without one. +#[tokio::test] +async fn should_ask_once_for_calls_made_while_the_network_is_being_asked() { + let mut sdk = mock_sdk(V13, false, 0); + answer_probe(&sdk, "pv14").await; + expect_fee_pots(&mut sdk).await; + for (call, result) in fetch_fee_pots_together(&sdk).await.into_iter().enumerate() { + result.unwrap_or_else(|error| panic!("call {call}: {error}")); + } + assert_eq!(probes(&sdk).await, 1, "network at 14"); + assert_eq!(sent(&sdk, &fee_pots()).await, 2); + + let mut sdk = mock_sdk(V13, false, 0); + answer( + &sdk, + &probe_request(), + Ok(vector("epoch-probe-pv13-epoch-2.bin")), + ) + .await; + answer( + &sdk, + ¤t_epoch_request(), + Ok(vector("current-epoch-pv13-epoch-2.bin")), + ) + .await; + expect_fee_pots(&mut sdk).await; + for (call, result) in fetch_fee_pots_together(&sdk).await.into_iter().enumerate() { + assert_refused( + result, + "get_contract_fee_pots", + V13, + false, + &format!("held, call {call}"), + ); + } + assert_eq!(probes(&sdk).await, 1, "network at 13, held"); + assert_eq!(sent(&sdk, ¤t_epoch_request()).await, 1); + assert_eq!(sent(&sdk, &fee_pots()).await, 0); + + // The genesis epoch has no epoch before it, so the probe holds nothing; its answer still + // decides the calls that waited for it. + let mut sdk = mock_sdk(V13, false, 0); + answer_probe(&sdk, "pv13").await; + expect_fee_pots(&mut sdk).await; + for (call, result) in fetch_fee_pots_together(&sdk).await.into_iter().enumerate() { + assert_refused( + result, + "get_contract_fee_pots", + V13, + false, + &format!("not held, call {call}"), + ); + } + assert_eq!(probes(&sdk).await, 1, "network at 13, not held"); + assert_eq!(sent(&sdk, &fee_pots()).await, 0); +} + +/// A call that waited for a probe that failed learned nothing from it, so it asks under its +/// own settings, and its refusal carries its own cause. +#[tokio::test] +async fn should_ask_again_for_a_call_that_waited_for_a_probe_that_failed() { + let mut sdk = mock_sdk(V13, false, 0); + expect_fee_pots(&mut sdk).await; + for (call, result) in fetch_fee_pots_together(&sdk).await.into_iter().enumerate() { + match result { + Err(Error::ProtocolVersionNotActive { + network: 0, + cause: Some(_), + .. + }) => {} + other => panic!("call {call}: expected a refusal with its cause, got {other:?}"), + } + } + assert_eq!(probes(&sdk).await, 2); + assert_eq!(sent(&sdk, &fee_pots()).await, 0); +} + +/// A call dropped while it asks the network does not hold up the calls waiting for it: they +/// ask themselves. +#[tokio::test] +async fn should_let_waiting_calls_ask_when_the_asking_call_is_dropped() { + let mut sdk = mock_sdk(V13, false, 0); + answer_probe(&sdk, "pv14").await; + expect_fee_pots(&mut sdk).await; + let node = dapi(&sdk).lock().await; + let mut first = Box::pin(ContractFeePots::fetch_with_settings( + &sdk, + fee_pots(), + once(), + )); + assert!( + futures::poll!(&mut first).is_pending(), + "the first call asks, and waits for the node" + ); + let drop_first = async move { + tokio::task::yield_now().await; + drop(first); + drop(node); + }; + let (second, ()) = tokio::join!( + ContractFeePots::fetch_with_settings(&sdk, fee_pots(), once()), + drop_first, + ); + second.expect("sent once its own probe shows 14"); + assert_eq!( + probes(&sdk).await, + 1, + "only the second call's probe reached the node" + ); + assert_eq!(sent(&sdk, &fee_pots()).await, 1); +} + +/// Probes that overlap can finish in any order. The hold kept is the one of the newest epoch +/// proved, not the last to arrive: a slower probe of an earlier epoch does not shorten it. Two +/// holds of one epoch both bound when that epoch has ended, so the earlier bound is kept. +#[test] +fn should_keep_the_hold_of_the_newest_epoch_when_probes_finish_out_of_order() { + let hold = |network, epoch, until_ms| ProbeHold { + network, + epoch, + until_ms, + }; + let shared = SharedProbeHold::default(); + shared.set(hold(V13, 3, 2_000)); + shared.set(hold(V13, 2, 1_000)); + assert_eq!( + shared.get(), + Some(hold(V13, 3, 2_000)), + "a slower probe of an earlier epoch" + ); + shared.set(hold(V13, 3, 1_900)); + assert_eq!( + shared.get(), + Some(hold(V13, 3, 1_900)), + "a closer bound of the same epoch" + ); + shared.set(hold(V13, 3, 2_100)); + assert_eq!( + shared.get(), + Some(hold(V13, 3, 1_900)), + "a looser bound of the same epoch" + ); + shared.set(hold(V14, 4, 3_000)); + assert_eq!(shared.get(), Some(hold(V14, 4, 3_000)), "a newer version"); + shared.set(hold(V13, 3, 1_900)); + assert_eq!( + shared.get(), + Some(hold(V14, 4, 3_000)), + "a slower probe of an older version" + ); +} + /// Each method protocol 15 added (the yes/no vote poll queries and the entries of a private /// document type's store), fetched proved and then /// unproved from a node that answers UNIMPLEMENTED as a 5.0 node does; per method, both @@ -1433,6 +2173,7 @@ async fn should_send_methods_protocol_15_added_only_once_it_is_verified() { Err(Error::ProtocolVersionNotActive { method: refused, required, + feature: None, network, pinned, cause: None, diff --git a/packages/wasm-sdk/src/error.rs b/packages/wasm-sdk/src/error.rs index 5ab2a8d9625..a93fd6e9a1c 100644 --- a/packages/wasm-sdk/src/error.rs +++ b/packages/wasm-sdk/src/error.rs @@ -663,6 +663,7 @@ mod tests { let refused = SdkError::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network: 13, pinned, cause: None, @@ -678,6 +679,7 @@ mod tests { let unconfirmed = WasmSdkError::from(SdkError::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network: 0, pinned: false, cause: Some(Box::new(SdkError::Generic("all nodes refused".to_string()))), @@ -697,6 +699,7 @@ mod tests { WasmSdkError::from(SdkError::ProtocolVersionNotActive { method: "get_contract_fee_pots", required: 14, + feature: None, network: 0, pinned: false, cause: Some(Box::new(SdkError::Proof( diff --git a/packages/wasm-sdk/src/queries/document.rs b/packages/wasm-sdk/src/queries/document.rs index bf5c56c73bd..6b540d4bbd9 100644 --- a/packages/wasm-sdk/src/queries/document.rs +++ b/packages/wasm-sdk/src/queries/document.rs @@ -107,8 +107,9 @@ export interface DocumentsQuery { /** * Aggregate-query option: SQL-shaped `GROUP BY` field list. Mirrors - * the v1 wire's `group_by: repeated string` directly. Ignored by - * the regular document-fetch path. + * the v1 wire's `group_by: repeated string` directly. The regular + * document fetch (`getDocuments`) refuses it: it returns the documents + * themselves, which cannot be grouped. * * - `[]` or omitted → one ungrouped result entry, keyed by the * empty string. Its value is the total count for @@ -138,7 +139,9 @@ export interface DocumentsQuery { * signed response metadata; `"byStart"` names the bucket absolutely, so * both sides read it straight from the query. Provable either way. * Requires protocol version 14+ (the first version whose contract - * grammar hosts `timeRange` indexes). + * grammar hosts `timeRange` indexes): on a network not yet proven to run + * it, the SDK refuses the query without sending it + * (`ProtocolVersionNotActive`). * * - `selector: "oldest"` → the oldest still-active range (a near-full * trailing window of ~`range`; best for "trending over the last window"). @@ -171,7 +174,7 @@ export interface DocumentsQuery { * window). An off-grid start is rejected rather than snapped, and an * empty window is a provable empty answer. The server and the proof * verifier resolve the same window from the query alone. Requires - * protocol version 14+. + * protocol version 14+, as `timeRange` does. * * `start` is a number, or a decimal string for values beyond * `Number.MAX_SAFE_INTEGER`. `grid` names one of the field's grids @@ -251,7 +254,7 @@ pub(super) struct DocumentsQueryInput { pub(super) start_at: Option, /// Aggregate-query option: SQL-shaped `GROUP BY` field list, /// mirroring the v1 wire `group_by: repeated string` field - /// one-to-one. Ignored by the regular document-fetch path. + /// one-to-one. The regular document fetch refuses it. /// See the TypeScript declaration for the supported shapes. /// Default empty (one ungrouped aggregate result entry). #[serde(rename = "groupBy", default)] @@ -304,9 +307,8 @@ pub(super) async fn build_documents_query( sdk: &WasmSdk, input: DocumentsQueryInput, ) -> Result { - // `group_by` on the shared input struct is an aggregate-query-only - // option; the regular document-fetch path destructured here just - // drops it. + // `group_by` on the shared input struct is applied by the callers + // that read it, after the shared clauses. let DocumentsQueryInput { data_contract_id, document_type_name, @@ -381,7 +383,12 @@ async fn parse_documents_query( let input: DocumentsQueryInput = deserialize_required_query(query, "Query object is required", "documents query")?; - build_documents_query(sdk, input).await + // A documents select cannot be grouped. The grouping is kept so that the SDK refuses + // it before sending, instead of the fetch quietly answering an ungrouped page. + let group_by = input.group_by.clone().unwrap_or_default(); + Ok(build_documents_query(sdk, input) + .await? + .with_group_by_fields(group_by)) } /// Parse a JS query object into a [`DocumentQuery`] configured diff --git a/packages/wasm-sdk/src/queries/document_ranked.rs b/packages/wasm-sdk/src/queries/document_ranked.rs index a49a4c172b7..59cc3f0602f 100644 --- a/packages/wasm-sdk/src/queries/document_ranked.rs +++ b/packages/wasm-sdk/src/queries/document_ranked.rs @@ -12,13 +12,12 @@ //! "Which groups' aggregate falls in this range?" Value is the answer, //! so there is no rank and no offset. //! -//! The grammar for both is not restated here. [`detect_ranked_mode`] and -//! [`detect_having_mode`] are rs-drive's own versioned classifiers — the -//! same functions the server's query table and the proof verifier call — -//! and they are `pub` under the `verify` feature this crate already -//! enables. Calling them client-side means a malformed query fails -//! locally with rs-drive's own message and cannot drift from what the -//! network enforces. +//! The grammar for both is not restated here: this module only translates +//! the JS input into a [`DocumentQuery`]. The Rust SDK checks the query +//! before sending it, once the network is proven to serve ranked and +//! having-range queries, with rs-drive's own versioned classifiers, the +//! same functions the server and the proof verifier call; a malformed +//! query fails locally with their message, as an invalid argument. use crate::error::WasmSdkError; use crate::queries::document::{ @@ -28,20 +27,18 @@ use crate::queries::document::{ use crate::queries::utils::deserialize_required_query; use crate::queries::ProofMetadataResponseWasm; use crate::sdk::WasmSdk; -use dash_sdk::dash_platform_queries::documents::array_element_clauses::key_element_clauses; use dash_sdk::dpp::data_contract::accessors::v0::DataContractV0Getters; use dash_sdk::dpp::data_contract::document_type::methods::DocumentTypeV0Methods; use dash_sdk::dpp::data_contract::document_type::DocumentTypeRef; use dash_sdk::dpp::platform_value::Value; use dash_sdk::dpp::version::PlatformVersion; use dash_sdk::platform::documents::document_query::{DocumentQuery, RankingDirection}; +use dash_sdk::platform::proto::{Proof, ResponseMetadata}; use dash_sdk::platform::{DataContract, Fetch}; -use drive::query::drive_document_having_query::mode_detection::detect_having_mode; -use drive::query::drive_document_ranked_query::mode_detection::detect_ranked_mode; +use dash_sdk::Sdk; use drive::query::{ HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, - RankedAxis, RankedEntry, RankedEntryValue, RankedPaginationInputs, SelectFunction, - SelectProjection, RANKED_AVG_SCALE, + RankedAxis, RankedEntry, RankedEntryValue, SelectFunction, SelectProjection, RANKED_AVG_SCALE, }; use drive_proof_verifier::{DocumentHavingEntries, DocumentRankedEntries}; use js_sys::{Array, Object, Reflect}; @@ -637,14 +634,13 @@ fn ranking_direction( /// Having operators, spelled the way `DocumentWhereOperator` spells the /// same comparisons so a caller who learned `'>='` for `where` writes -/// `'>='` here too. -/// -/// `!=` and `in` are absent on purpose rather than mapped and then -/// rejected downstream: neither describes one contiguous range, and a -/// having-range read is exactly one contiguous slice of one axis. +/// `'>='` here too. Which of them a node serves is the SDK's check to +/// decide, not this mapping's. fn having_operator_from_str(operator: &str) -> Result { match operator { "==" | "=" => Ok(HavingOperator::Equal), + "!=" => Ok(HavingOperator::NotEqual), + "in" | "In" => Ok(HavingOperator::In), ">" => Ok(HavingOperator::GreaterThan), ">=" => Ok(HavingOperator::GreaterThanOrEquals), "<" => Ok(HavingOperator::LessThan), @@ -654,26 +650,13 @@ fn having_operator_from_str(operator: &str) -> Result Ok(HavingOperator::BetweenExcludeLeft), "BetweenExcludeRight" => Ok(HavingOperator::BetweenExcludeRight), other => Err(WasmSdkError::invalid_argument(format!( - "unsupported having operator `{other}`; expected one of '==', '>', '>=', '<', '<=', \ - 'Between', 'BetweenExcludeBounds', 'BetweenExcludeLeft', 'BetweenExcludeRight'. \ - '!=' and 'in' describe non-contiguous ranges and cannot be served as a having-range \ - read." + "unsupported having operator `{other}`; a having-range query takes one of '==', \ + '>', '>=', '<', '<=', 'Between', 'BetweenExcludeBounds', 'BetweenExcludeLeft', \ + 'BetweenExcludeRight'" ))), } } -/// True for the operators whose right-hand operand is a `[lower, upper]` -/// pair rather than a scalar. -fn is_between_operator(operator: HavingOperator) -> bool { - matches!( - operator, - HavingOperator::Between - | HavingOperator::BetweenExcludeBounds - | HavingOperator::BetweenExcludeLeft - | HavingOperator::BetweenExcludeRight - ) -} - /// Build the one having clause from the JS bound. /// /// The clause's aggregate is *derived* from the select, never supplied @@ -696,21 +679,6 @@ fn having_clause_from_input( let operator = having_operator_from_str(&bound.operator)?; - // rs-drive rejects a wrong operand shape too, but naming the JS - // field is a better message than a bounds-translation failure. - if is_between_operator(operator) - && !bound - .value - .as_array() - .map(|operands| operands.len() == 2) - .unwrap_or(false) - { - return Err(WasmSdkError::invalid_argument(format!( - "having operator `{}` needs `value: [lower, upper]` (exactly two operands)", - bound.operator - ))); - } - Ok(HavingClause { aggregate: HavingAggregate { function, @@ -755,90 +723,6 @@ fn value_scale(axis: RankedAxis) -> i128 { } } -/// The pagination triple both classifiers take, read off a built query. -fn pagination_inputs(query: &DocumentQuery) -> RankedPaginationInputs { - RankedPaginationInputs { - // `DocumentQuery::limit` uses `0` as the "unset" sentinel. - limit: (query.limit != 0).then_some(query.limit), - offset: query.offset, - has_start_at: query.start.is_some(), - } -} - -/// Re-run rs-drive's own versioned ranked grammar client-side. -/// -/// This is the same function the server's query table and the proof -/// verifier call — not a copy — so it cannot drift from either, and it -/// is versioned by `platform_version`, so an SDK built against one -/// protocol version cannot quietly accept a shape that version rejects. -fn assert_ranked_shape( - query: &DocumentQuery, - platform_version: &PlatformVersion, -) -> Result<(), WasmSdkError> { - // The server keys a typed array's `contains` pin as an equality first. - let where_clauses = key_element_clauses( - &query.data_contract, - &query.document_type_name, - query.where_clauses.clone(), - false, - platform_version, - ) - .map_err(|e| WasmSdkError::invalid_argument(format!("not a well-formed ranked query: {e}")))?; - detect_ranked_mode( - &query.select, - &query.group_by, - &query.having, - &query.order_by_clauses, - &where_clauses, - pagination_inputs(query), - platform_version, - ) - .map(|_| ()) - .map_err(|e| { - WasmSdkError::invalid_argument(format!( - "not a well-formed ranked query: {e}. A ranked query is \ - {{ groupBy, aggregate, limit }} plus optional {{ direction, offset, where }}; \ - `where` entries pin the covering compound index's leading properties with \ - `==` (`contains` on a typed array), or one bounded `in`." - )) - }) -} - -/// Having-range counterpart of [`assert_ranked_shape`]. -fn assert_having_shape( - query: &DocumentQuery, - platform_version: &PlatformVersion, -) -> Result<(), WasmSdkError> { - // The server keys a typed array's `contains` pin as an equality first. - let where_clauses = key_element_clauses( - &query.data_contract, - &query.document_type_name, - query.where_clauses.clone(), - false, - platform_version, - ) - .map_err(|e| { - WasmSdkError::invalid_argument(format!("not a well-formed having-range query: {e}")) - })?; - detect_having_mode( - &query.select, - &query.group_by, - &query.having, - &query.order_by_clauses, - &where_clauses, - pagination_inputs(query), - platform_version, - ) - .map(|_| ()) - .map_err(|e| { - WasmSdkError::invalid_argument(format!( - "not a well-formed having-range query: {e}. A having-range query is \ - {{ groupBy, aggregate, having, limit }} plus optional {{ direction, where }}; \ - the bound must describe one contiguous range over the selected aggregate." - )) - }) -} - /// Attach the (at most one) time-range selection to an already-built /// query. The selection stays pending — the server resolves it from /// committed block time, the proof verifier from the signed metadata @@ -878,15 +762,8 @@ fn apply_index_pins( fn apply_ranked_shape( base: DocumentQuery, input: &DocumentsRankedQueryInput, - platform_version: &PlatformVersion, ) -> Result { let group_by = input.group_by.as_str(); - if group_by.is_empty() { - return Err(WasmSdkError::invalid_argument( - "groupBy must name the covering ranked index's trailing property", - )); - } - let select = select_from_input(&input.aggregate)?; let direction = ranking_direction(input.direction.as_deref(), RankingDirection::Descending)?; @@ -909,10 +786,7 @@ fn apply_ranked_shape( let query = apply_index_pins(query, input.where_clauses.as_deref())?; let query = apply_time_range_selections(query, input.time_range.as_deref())?; - let query = apply_integer_range_selections(query, input.integer_range.as_deref())?; - - assert_ranked_shape(&query, platform_version)?; - Ok(query) + apply_integer_range_selections(query, input.integer_range.as_deref()) } /// Having-range counterpart of [`apply_ranked_shape`]. @@ -924,15 +798,8 @@ fn apply_ranked_shape( fn apply_having_shape( base: DocumentQuery, input: &DocumentsHavingQueryInput, - platform_version: &PlatformVersion, ) -> Result { let group_by = input.group_by.as_str(); - if group_by.is_empty() { - return Err(WasmSdkError::invalid_argument( - "groupBy must name the covering ranked index's trailing property", - )); - } - let select = select_from_input(&input.aggregate)?; let having = having_clause_from_input(&select, &input.having)?; @@ -949,10 +816,7 @@ fn apply_having_shape( let query = apply_index_pins(query, input.where_clauses.as_deref())?; let query = apply_time_range_selections(query, input.time_range.as_deref())?; - let query = apply_integer_range_selections(query, input.integer_range.as_deref())?; - - assert_having_shape(&query, platform_version)?; - Ok(query) + apply_integer_range_selections(query, input.integer_range.as_deref()) } async fn parse_documents_ranked_query( @@ -967,7 +831,7 @@ async fn parse_documents_ranked_query( .await?; let base = DocumentQuery::new(contract, &input.document_type_name)?; - apply_ranked_shape(base, &input, sdk.inner_sdk().version()) + apply_ranked_shape(base, &input) } async fn parse_documents_having_query( @@ -985,7 +849,7 @@ async fn parse_documents_having_query( .await?; let base = DocumentQuery::new(contract, &input.document_type_name)?; - apply_having_shape(base, &input, sdk.inner_sdk().version()) + apply_having_shape(base, &input) } /// One entry decoded as far as it can be without touching a JS type, so @@ -1249,7 +1113,7 @@ impl ResultContext { fn from_query(query: &DocumentQuery) -> Result { Ok(Self { axis: axis_from_select(&query.select)?, - // Shape assertion already established there is exactly one. + // Read after the fetch: the SDK's check before sending allows exactly one. group_by: query .group_by .first() @@ -1279,6 +1143,26 @@ impl ResultContext { } } +/// A page of `T` for `query`, with the block metadata and proof it was verified against, and +/// the context to decode its entries with. +/// +/// The context is read from the query only after the fetch. The SDK gates the query on the +/// network's protocol version and refuses, with its own error, a query the context cannot be +/// read from (no grouping, or a projection that is not an aggregate); the decoder decides +/// nothing about the query's shape. +async fn fetch_grouped_page( + sdk: &Sdk, + query: DocumentQuery, +) -> Result<(T, ResponseMetadata, Proof, ResultContext), WasmSdkError> +where + T: Fetch + Default + Send, +{ + let decoding = query.clone(); + let (page, metadata, proof) = T::fetch_with_metadata_and_proof(sdk, query, None).await?; + let context = ResultContext::from_query(&decoding)?; + Ok((page.unwrap_or_default(), metadata, proof, context)) +} + #[wasm_bindgen] impl WasmSdk { /// The fixed-point divisor the `avg` axis sorts by. @@ -1314,9 +1198,10 @@ impl WasmSdk { /// `SELECT GROUP BY ORDER BY /// LIMIT n [OFFSET m]`, served from protocol version 14 against a /// contract index that declares the matching `rankedCountable` / - /// `rankedSummable` / `rankedAverageable` keyword. A node on an - /// earlier protocol version, or a contract whose index does not opt - /// in, rejects the query and names what is missing. + /// `rankedSummable` / `rankedAverageable` keyword. On a network not yet + /// proven to run protocol version 14 the SDK refuses the query without + /// sending it (`ProtocolVersionNotActive`); a contract whose index does + /// not opt in gets the query rejected, naming what is missing. /// /// Entries come back in ranking order and must not be re-sorted; /// `startingRank` plus the entry's position is its absolute rank. @@ -1329,13 +1214,10 @@ impl WasmSdk { query: DocumentsRankedQueryJs, ) -> Result { let query = parse_documents_ranked_query(self, query).await?; - let context = ResultContext::from_query(&query)?; - // An empty ranking proves, so `None` is an empty page rather // than an absent answer. - let page = DocumentRankedEntries::fetch(self.as_ref(), query) - .await? - .unwrap_or_default(); + let (page, _, _, context) = + fetch_grouped_page::(self.as_ref(), query).await?; let parts = context.decode(&page.entries, self.inner_sdk().version())?; ranked_result_to_js(&parts, page.starting_rank, context.axis, &context.group_by) @@ -1352,12 +1234,8 @@ impl WasmSdk { query: DocumentsRankedQueryJs, ) -> Result { let query = parse_documents_ranked_query(self, query).await?; - let context = ResultContext::from_query(&query)?; - - let (page, metadata, proof) = - DocumentRankedEntries::fetch_with_metadata_and_proof(self.as_ref(), query, None) - .await?; - let page = page.unwrap_or_default(); + let (page, metadata, proof, context) = + fetch_grouped_page::(self.as_ref(), query).await?; let parts = context.decode(&page.entries, self.inner_sdk().version())?; let result = @@ -1388,11 +1266,8 @@ impl WasmSdk { query: DocumentsHavingQueryJs, ) -> Result { let query = parse_documents_having_query(self, query).await?; - let context = ResultContext::from_query(&query)?; - - let page = DocumentHavingEntries::fetch(self.as_ref(), query) - .await? - .unwrap_or_default(); + let (page, _, _, context) = + fetch_grouped_page::(self.as_ref(), query).await?; let parts = context.decode(&page.entries, self.inner_sdk().version())?; having_result_to_js(&parts, context.axis, &context.group_by) @@ -1409,12 +1284,8 @@ impl WasmSdk { query: DocumentsHavingQueryJs, ) -> Result { let query = parse_documents_having_query(self, query).await?; - let context = ResultContext::from_query(&query)?; - - let (page, metadata, proof) = - DocumentHavingEntries::fetch_with_metadata_and_proof(self.as_ref(), query, None) - .await?; - let page = page.unwrap_or_default(); + let (page, metadata, proof, context) = + fetch_grouped_page::(self.as_ref(), query).await?; let parts = context.decode(&page.entries, self.inner_sdk().version())?; let result = having_result_to_js(&parts, context.axis, &context.group_by)?; @@ -1566,6 +1437,7 @@ mod tests { use dash_sdk::dpp::system_data_contracts::{load_system_data_contract, SystemDataContract}; use drive::query::drive_document_ranked_query::MAX_PREFIX_IN_BRANCHES; use drive::query::{WhereOperator, MAX_HAVING_LIMIT, MAX_RANKED_LIMIT, RANKED_COUNT_ORDER_KEY}; + use drive_proof_verifier::FromProof; use serde_json::json; /// DPNS stands in for any contract here. Every function under test is @@ -1657,16 +1529,30 @@ mod tests { } } - fn build_ranked(input: DocumentsRankedQueryInput) -> Result { - apply_ranked_shape(base_query(), &input, platform_version()) + /// The query the JS input builds, checked as the SDK checks it before sending. + fn build_ranked(input: DocumentsRankedQueryInput) -> Result { + let query = apply_ranked_shape(base_query(), &input).map_err(|e| e.to_string())?; + >::check_request( + &query, + platform_version(), + ) + .map_err(|e| e.to_string())?; + Ok(query) } - fn build_having(input: DocumentsHavingQueryInput) -> Result { - apply_having_shape(base_query(), &input, platform_version()) + /// The query the JS input builds, checked as the SDK checks it before sending. + fn build_having(input: DocumentsHavingQueryInput) -> Result { + let query = apply_having_shape(base_query(), &input).map_err(|e| e.to_string())?; + >::check_request( + &query, + platform_version(), + ) + .map_err(|e| e.to_string())?; + Ok(query) } /// A ranked query with everything optional left unset. - fn ranked(aggregate: AggregateSelectInput, limit: u32) -> Result { + fn ranked(aggregate: AggregateSelectInput, limit: u32) -> Result { build_ranked(ranked_input(GROUP_BY, aggregate, limit, None, None, None)) } @@ -1675,7 +1561,7 @@ mod tests { aggregate: AggregateSelectInput, bound: HavingBoundInput, limit: u32, - ) -> Result { + ) -> Result { build_having(having_input(aggregate, bound, limit, None)) } @@ -1806,7 +1692,7 @@ mod tests { let error = build_ranked(ranked_input("", count(), 3, None, None, None)) .expect_err("a ranking without a grouping ranks nothing"); - assert!(error.to_string().contains("groupBy")); + assert!(error.contains("not a well-formed ranked query"), "{error}"); } // ---- ranked index pins --------------------------------------------- @@ -2053,21 +1939,24 @@ mod tests { ); } - /// Neither describes one contiguous range, and a having-range read is - /// exactly one contiguous slice of one axis secondary. + /// `!=` and `in` map to their operators, and the SDK's check refuses them: neither + /// describes one contiguous range, and a having-range read is exactly one contiguous + /// slice of one axis secondary. A spelling with no operator is refused while parsing. #[test] fn having_rejects_non_contiguous_operators() { - for operator in ["!=", "in", "startsWith"] { + for operator in ["!=", "in"] { let error = match having(count(), bound(operator, json!(1)), 10) { Ok(_) => panic!("`{operator}` should have been rejected"), - Err(error) => error.to_string(), + Err(error) => error, }; - assert!( - error.contains("unsupported having operator"), - "`{operator}` should name the supported set; got: {error}" + error.contains("not a well-formed having-range query"), + "`{operator}`: {error}" ); } + let error = having(count(), bound("startsWith", json!(1)), 10) + .expect_err("no having operator is spelled this way"); + assert!(error.contains("unsupported having operator"), "{error}"); } #[test] @@ -2447,4 +2336,64 @@ mod tests { "the docs state one ceiling for both surfaces; split the prose if these diverge" ); } + + /// The SDK reads a ranked or having-range query first: it gates it on the network's + /// protocol version, and refuses one it cannot serve with its own error. The context its + /// entries are decoded with is read only after that, so a query without a grouping gets the + /// SDK's refusal. + #[tokio::test] + async fn should_leave_a_grouped_query_without_a_grouping_to_the_sdk() { + let sdk = Sdk::new_mock(); + let ranked = base_query() + .with_select(SelectProjection::count_star()) + .order_by_selected_aggregate(RankingDirection::Descending) + .with_limit(5); + let having = base_query() + .with_select(SelectProjection::count_star()) + .with_having(vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(1)), + }]) + .with_limit(5); + let sdk_refusal = |refusal: Result<(), drive_proof_verifier::Error>| match refusal { + Err(drive_proof_verifier::Error::RequestError { error }) => error, + other => panic!("the SDK refuses a query without a grouping, got {other:?}"), + }; + + let expected = sdk_refusal( + >::check_request( + &ranked, + sdk.version(), + ), + ); + let error = fetch_grouped_page::(&sdk, ranked) + .await + .err() + .expect("a ranked query without a grouping is refused"); + assert!( + error.message().contains(&expected), + "ranked: {}", + error.message() + ); + + let expected = sdk_refusal( + >::check_request( + &having, + sdk.version(), + ), + ); + let error = fetch_grouped_page::(&sdk, having) + .await + .err() + .expect("a having query without a grouping is refused"); + assert!( + error.message().contains(&expected), + "having: {}", + error.message() + ); + } }