docs: sync results-cache invalidation, Arrow on_conflict, SQLite null keys, query nesting limit, and v2.4 per-dataset cdc_prefetch_buffer

website/docs/features/caching/index.md

@@ -81,8 +81,10 @@ Initialized search results cache; max size: 128.00 MiB, item ttl: 1s, shards: 16
 
 A table invalidation walks every shard to evict the entries that read the table, so its cost is
 proportional to the number of cached entries. The walk runs on a blocking thread, not on the query
-runtime. SQL results are invalidated only by client writes, dataset reload, and dataset unload; an
-accelerated refresh leaves them to expire by age. See
+runtime. SQL results are invalidated only by client writes, dataset reload, dataset unload, and
+the view, catalog, and function changes listed in
+[Cached Results and Definition Changes](#cached-results-and-definition-changes); an accelerated
+refresh leaves them to expire by age. See
 [Cached Results and Dataset Refresh](#cached-results-and-dataset-refresh). Search results are also
 invalidated by refreshes, and search result invalidations always evict entries.
 
@@ -122,7 +124,7 @@ The hashing algorithm determines how cache keys are hashed before being stored,
 - **`ahash`:** Uses the [AHash](https://github.com/tkaitchuck/ahash) algorithm for hashing the cache keys. The AHash algorithm is a [high quality](https://github.com/tkaitchuck/aHash/blob/master/compare/readme#Quality) hashing algorithm, and has claimed resistance against hashing DoS attacks. AHash has higher performance than SipHash1-3, especially when used with `cache_key_type: plan`.
 - **`blake3`:** Uses the [BLAKE3](https://github.com/BLAKE3-team/BLAKE3) cryptographic hash function. BLAKE3 is a fast, parallelizable hash function that provides cryptographic security while maintaining high performance. It is suitable for scenarios requiring both speed and cryptographic guarantees.
 - **`xxh64`, `xxh128`:** Variants of the XXH hashing algorithm with different output sizes. These algorithms offer a balance between speed and collision resistance, with larger hash sizes providing better collision resistance at the cost of performance.
-- **`xxh32`:** Deprecated, and will be removed in a future release. A runtime that sets it logs a warning at startup. Use `xxh3` instead.
+- **`xxh32`:** Deprecated, and will be removed in a future release. A runtime that sets it logs a warning at startup. Use `xxh3` instead. A cache key is a 64-bit hash, and a 32-bit hash would leave half of it unused, so `xxh32` builds its keys with XXH64, the same as `xxh64`.
 
 Use `xxh3` (the default) for its superior speed in most scenarios. Use `ahash`, `xxh64` or `xxh128` for reduced collision probability when caching a large number of queries. Use `blake3` when cryptographic security is required. Use `siphash` when protection against hash flooding attacks is a priority.
 
@@ -403,6 +405,18 @@ Removing a dataset, for example by deleting it from a Spicepod the runtime watch
 The cache removes an entry from memory `item_ttl + stale_while_revalidate_ttl` after it was stored, while servability counts from when its query started. An entry can therefore stay in memory, unserved, for about as long as its query ran. A lookup that finds such an entry counts as a miss.
 :::
 
+### Cached Results and Definition Changes
+
+A change to a definition that cached results were computed from clears every cached SQL result, together with the [logical plan cache](#logical-plan-cache). The following changes clear both caches:
+
+- a [view](../../reference/spicepod/views.md) is updated or removed;
+- a Spicepod hot reload replaces a [`catalog`](../../reference/spicepod/catalogs.md), either one whose declaration changed or one whose name a provider is already registered under;
+- a Spicepod hot reload changes the set of registered [`functions`](../../reference/spicepod/functions.md).
+
+A cached result records the tables a view reads, not the view itself, so the cache cannot find the entries that read through a changed view and clears all of them. A query that started before the clear does not store its result, even when it reads no table. After a view is updated, a repeat of an earlier query returns the result of the new definition. After a view is removed, the repeat fails because the view no longer exists.
+
+In earlier releases, including v2.4.x, updating or removing a view and replacing a catalog cleared only the logical plan cache. A query on an updated view could return the previous definition's result, and with `cache_key_type: sql` a removed view could still answer, until the cached result aged past `item_ttl`, or past `item_ttl + stale_while_revalidate_ttl` when `stale_while_revalidate_ttl` is set.
+
 ### HTTP/Flight API
 
 The following endpoints support the standard HTTP [`Cache-Control` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control):

website/docs/features/data-acceleration/constraints.md

@@ -56,6 +56,8 @@ Primary key column 'region' has null values. Every primary key column must be no
 
 Either populate the column in the source data, or choose a `primary_key` made only of columns that are always present.
 
+The [DuckDB](../../components/data-accelerators/duckdb) and [SQLite](../../components/data-accelerators/sqlite) accelerators also reject a write that carries a null in a primary key column, with an error that names the column(s), before the write reaches the engine. On SQLite, a `NaN` in a floating-point primary key column is rejected as a null, because SQLite stores a `NaN` as `NULL`. In earlier releases, including v2.4.x, SQLite stored a row with a null primary key.
+
 ## One row per primary key on Spice Cayenne
 
 On the [Spice Cayenne](../../components/data-accelerators/cayenne/index.md) accelerator, a `primary_key` alone decides which version of a key is kept, with nothing else to configure. Cayenne is the [default engine](../../components/data-accelerators/index.md#default-engine) on Linux and macOS, so this also applies to an acceleration that sets no `engine` there. A refresh keeps:

website/docs/reference/spicepod/datasets.md

@@ -953,7 +953,7 @@ For detailed information about metrics available for specific components, see th
 
 ## `acceleration.indexes`
 
-Optional. Specify which indexes should be applied to the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify which indexes should be applied to the locally accelerated table. On the in-memory Arrow acceleration engine, indexes are built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `indexes` field is a map where the key is the column reference and the value is the index type.
 
@@ -975,7 +975,7 @@ datasets:
 
 ## `acceleration.primary_key`
 
-Optional. Specify the primary key constraint on the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify the primary key constraint on the locally accelerated table. On the in-memory Arrow acceleration engine, the primary key is built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `primary_key` field is a string that represents the column reference that should be used as the primary key. The column reference can be a single column name or a multicolumn key. A multicolumn key is a comma-separated list of column names, and the enclosing parentheses are optional. A column name may be double-quoted the way SQL writes it, and a column whose name contains `,`, `;`, `:`, `(`, `)` or `"` cannot be referenced — see [Column names](../../features/data-acceleration/constraints#column-names).
 
@@ -999,7 +999,7 @@ datasets:
 `on_conflict` is deprecated and will be removed in Spice 3.0. Spice Cayenne does not use it to resolve keys and keeps [one row per primary key](../../features/data-acceleration/constraints.md#one-row-per-primary-key-on-spice-cayenne) without it. Other accelerators keep the behavior below. A dataset that sets `on_conflict` logs a deprecation warning at load. PostgreSQL, MySQL, and MongoDB change streams on engines other than Spice Cayenne and Arrow still require an `on_conflict` upsert on the primary key, so keep the entry on those datasets. Cayenne durable write-back does not require it.
 :::
 
-Optional. Specify what should happen when a constraint is violated. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify what should happen when a constraint is violated. On the in-memory Arrow acceleration engine, the `on_conflict` target must be the `primary_key`, and every strategy below is supported. On Arrow, `upsert_dedup` and `upsert_dedup_by_row_id` cannot be used on a table that has a column named `__row_num` or `__max_row_num`: a write that has duplicate keys to resolve fails with a planning error.
 
 The `on_conflict` field is a map where the key is the column reference and the value is the conflict resolution strategy.
 

website/docs/reference/spicepod/views.md

@@ -259,7 +259,7 @@ Optional. The maximum amount of jitter to add to the refresh interval. The jitte
 
 ## `acceleration.indexes`
 
-Optional. Specify which indexes should be applied to the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify which indexes should be applied to the locally accelerated table. On the in-memory Arrow acceleration engine, indexes are built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `indexes` field is a map where the key is the column reference and the value is the index type.
 
@@ -281,7 +281,7 @@ views:
 
 ## `acceleration.primary_key`
 
-Optional. Specify the primary key constraint on the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify the primary key constraint on the locally accelerated table. On the in-memory Arrow acceleration engine, the primary key is built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `primary_key` field is a string that represents the column reference that should be used as the primary key. The column reference can be a single column name or a multicolumn key. A multicolumn key is a comma-separated list of column names, and the enclosing parentheses are optional. A column name may be double-quoted the way SQL writes it, and a column whose name contains `,`, `;`, `:`, `(`, `)` or `"` cannot be referenced — see [Column names](../../features/data-acceleration/constraints#column-names).
 
@@ -303,7 +303,7 @@ views:
 `on_conflict` is deprecated and will be removed in Spice 3.0. Spice Cayenne does not use it to resolve keys and keeps [one row per primary key](../../features/data-acceleration/constraints.md#one-row-per-primary-key-on-spice-cayenne) without it. Other accelerators keep the behavior below.
 :::
 
-Optional. Specify what should happen when a constraint is violated. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify what should happen when a constraint is violated. On the in-memory Arrow acceleration engine, the `on_conflict` target must be the `primary_key`, and every strategy below is supported. On Arrow, `upsert_dedup` and `upsert_dedup_by_row_id` cannot be used on a table that has a column named `__row_num` or `__max_row_num`: a write that has duplicate keys to resolve fails with a planning error.
 
 The `on_conflict` field is a map where the key is the column reference and the value is the conflict resolution strategy.
 

website/docs/reference/sql/operators.md

@@ -362,6 +362,8 @@ A selected, grouped, or ordered expression, or a `VALUES` cell, has the same 512
 Query rejected: a selected, grouped, ordered, or VALUES expression is nested more than 512 levels deep. Simplify the expression (for example, compute part of a long chain of +, || or AND in a subquery).
 ```
 
+Subqueries and common table expressions that read one another have a separate limit of 64 levels. See [Nesting depth limit](./subqueries.md#nesting-depth-limit).
+
 ### `AND`
 
 Logical And

website/docs/reference/sql/select.md

@@ -70,6 +70,8 @@ WITH x AS (SELECT a, MAX(b) AS b FROM t GROUP BY a)
 SELECT a, b FROM x;
 ```
 
+Each named subquery is a common table expression (CTE). A CTE that reads an earlier CTE adds a level of query nesting, and Spice rejects a statement nested more than 64 levels deep. See [Nesting depth limit](./subqueries.md#nesting-depth-limit).
+
 ### SELECT clause
 
 The `SELECT` clause is used to select data from a database by defining the columns it returns. Each `select_expr` in the

website/docs/reference/sql/subqueries.md

@@ -348,6 +348,20 @@ A correlated `[NOT] EXISTS` can be pushed down to a federated source. Spice reje
 The error identifies the correlation. Distinct relation aliases allow the outer reference to
 remain unambiguous inside the subquery.
 
+## Nesting depth limit
+
+Spice rejects a SQL statement whose queries are nested more than 64 levels deep, before it plans the query. A subquery or derived table adds one level to the query that contains it. A common table expression (CTE) that reads an earlier CTE is planned on top of it, so in a `WITH` chain where each CTE reads the one before it, each CTE adds one level. CTEs that do not read each other are wide rather than deep and do not add to the depth. The limit is fixed and cannot be configured, and it also applies to `EXPLAIN` of such a statement.
+
+For example, a `WITH` chain of 63 CTEs, each reading the one before it, followed by a `SELECT` from the last CTE, is 64 levels deep and is accepted. A chain of 64 CTEs is rejected.
+
+A rejected query fails with HTTP `400` on `/v1/sql` and the gRPC status `InvalidArgument` over Arrow Flight SQL, and the error contains:
+
+```text
+Query rejected: queries are nested more than 64 levels deep, counting subqueries and common table expressions that read one another. Simplify the query (for example, combine steps of a long WITH chain).
+```
+
+To bring a query under the limit, combine consecutive steps of a long `WITH` chain into one CTE. Expressions have a separate limit of 512 levels; see [Logical Operators](./operators.md#logical-operators).
+
 ## Subquery categories
 
 Subqueries can be categorized as one or more of the following based on the

website/versioned_docs/version-2.4.x/features/cdc/index.md

@@ -66,9 +66,12 @@ datasets:
       refresh_mode: changes
       params:
         cdc_max_coalesce_age_ms: 1000 # this dataset lingers longer than the global 250ms
-        cdc_prefetch_buffer: 1024 # ...and buffers more aggressively
 ```
 
+:::note
+In v2.4.0, a per-dataset `cdc_prefetch_buffer` is accepted but has no effect: the dataset's change queue is sized from the instance-wide value (`runtime.params.cdc_prefetch_buffer`, then `SPICE_CDC_PREFETCH_BUFFER`, then the default of `128`). The `dataset_acceleration_cdc_prefetch_buffer_capacity` metric reports the size in use. To buffer more change events, set `cdc_prefetch_buffer` under `runtime.params`, which applies to every CDC-accelerated dataset. The other four parameters take effect per dataset as described above.
+:::
+
 ## Supported Data Connectors
 
 Enabling CDC by setting `refresh_mode: changes` in the acceleration settings requires support from the data connector to provide a stream of row-level changes.

website/versioned_docs/version-2.4.x/reference/spicepod/datasets.md

@@ -945,7 +945,7 @@ For detailed information about metrics available for specific components, see th
 
 ## `acceleration.indexes`
 
-Optional. Specify which indexes should be applied to the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify which indexes should be applied to the locally accelerated table. On the in-memory Arrow acceleration engine, indexes are built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `indexes` field is a map where the key is the column reference and the value is the index type.
 
@@ -967,7 +967,7 @@ datasets:
 
 ## `acceleration.primary_key`
 
-Optional. Specify the primary key constraint on the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify the primary key constraint on the locally accelerated table. On the in-memory Arrow acceleration engine, the primary key is built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `primary_key` field is a string that represents the column reference that should be used as the primary key. The column reference can be a single column name or a multicolumn key. A multicolumn key is a comma-separated list of column names, and the enclosing parentheses are optional. A column name may be double-quoted the way SQL writes it, and a column whose name contains `,`, `;`, `:`, `(`, `)` or `"` cannot be referenced — see [Column names](../../features/data-acceleration/constraints#column-names).
 
@@ -991,7 +991,7 @@ datasets:
 `on_conflict` is deprecated and will be removed in Spice 3.0. Spice Cayenne does not use it to resolve keys and keeps [one row per primary key](../../features/data-acceleration/constraints.md#one-row-per-primary-key-on-spice-cayenne) without it. Other accelerators keep the behavior below. A dataset that sets `on_conflict` logs a deprecation warning at load. PostgreSQL, MySQL, and MongoDB change streams on engines other than Spice Cayenne and Arrow still require an `on_conflict` upsert on the primary key, so keep the entry on those datasets. Cayenne durable write-back does not require it.
 :::
 
-Optional. Specify what should happen when a constraint is violated. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify what should happen when a constraint is violated. On the in-memory Arrow acceleration engine, only `upsert` on the `primary_key` takes effect. Arrow ignores `drop` and logs a warning, and it applies `upsert_dedup` and `upsert_dedup_by_row_id` as plain `upsert`, so a write that repeats a primary key value fails with `Primary key values must be unique`.
 
 The `on_conflict` field is a map where the key is the column reference and the value is the conflict resolution strategy.
 

website/versioned_docs/version-2.4.x/reference/spicepod/views.md

@@ -259,7 +259,7 @@ Optional. The maximum amount of jitter to add to the refresh interval. The jitte
 
 ## `acceleration.indexes`
 
-Optional. Specify which indexes should be applied to the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify which indexes should be applied to the locally accelerated table. On the in-memory Arrow acceleration engine, indexes are built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `indexes` field is a map where the key is the column reference and the value is the index type.
 
@@ -281,7 +281,7 @@ views:
 
 ## `acceleration.primary_key`
 
-Optional. Specify the primary key constraint on the locally accelerated table. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify the primary key constraint on the locally accelerated table. On the in-memory Arrow acceleration engine, the primary key is built as a [hash index](../../features/data-acceleration/hash-index).
 
 The `primary_key` field is a string that represents the column reference that should be used as the primary key. The column reference can be a single column name or a multicolumn key. A multicolumn key is a comma-separated list of column names, and the enclosing parentheses are optional. A column name may be double-quoted the way SQL writes it, and a column whose name contains `,`, `;`, `:`, `(`, `)` or `"` cannot be referenced — see [Column names](../../features/data-acceleration/constraints#column-names).
 
@@ -303,7 +303,7 @@ views:
 `on_conflict` is deprecated and will be removed in Spice 3.0. Spice Cayenne does not use it to resolve keys and keeps [one row per primary key](../../features/data-acceleration/constraints.md#one-row-per-primary-key-on-spice-cayenne) without it. Other accelerators keep the behavior below.
 :::
 
-Optional. Specify what should happen when a constraint is violated. Not supported for in-memory Arrow acceleration engine.
+Optional. Specify what should happen when a constraint is violated. On the in-memory Arrow acceleration engine, only `upsert` on the `primary_key` takes effect. Arrow ignores `drop` and logs a warning, and it applies `upsert_dedup` and `upsert_dedup_by_row_id` as plain `upsert`, so a write that repeats a primary key value fails with `Primary key values must be unique`.
 
 The `on_conflict` field is a map where the key is the column reference and the value is the conflict resolution strategy.