docs(snapshots): per-dataset snapshot settings; deprecate top-level snapshots and runtime.scheduler state settings

website/docs/cli/reference/query.md

@@ -84,5 +84,5 @@ Time: 5.10000000 seconds. 1 rows.
 Query IDs can be abbreviated if they uniquely identify a query within the tracked session.
 
 :::note
-The `spice query` command requires the runtime to be running in [cluster mode](/docs/features/distributed-query) with `--role scheduler` and a scheduler state location configured (`runtime.scheduler.state_location`, or [`runtime.state.location`](../../reference/spicepod/runtime#runtimestate)).
+The `spice query` command requires the runtime to be running in [cluster mode](/docs/features/distributed-query) with `--role scheduler` and a scheduler state location configured ([`runtime.state.location`](../../reference/spicepod/runtime#runtimestate), or the deprecated `runtime.scheduler.state_location`).
 :::

website/docs/features/data-acceleration/data-refresh.md

@@ -146,12 +146,6 @@ See [Caching Mode](./refresh-modes/caching) for detailed documentation and examp
 The `snapshot` refresh mode creates a read-only acceleration that reloads exclusively from the [snapshot store](./snapshots). The federated data source is never queried for refreshes — instead, the runtime polls the snapshot store on a configurable interval and atomically swaps in newer snapshots when available.
 
 ```yaml
-snapshots:
-  enabled: true
-  location: s3://my-bucket/snapshots/
-  params:
-    s3_auth: iam_role
-
 datasets:
   - from: postgres:public.my_table
     name: my_table
@@ -162,6 +156,7 @@ datasets:
       refresh_mode: snapshot
       refresh_check_interval: 30s  # Poll interval; defaults to 1m
       snapshots: enabled
+      snapshots_location: s3://my-bucket/spice/snapshots/ # The writer's snapshot folder
       params:
         duckdb_file: /nvme/my_table.db
 ```

website/docs/features/data-acceleration/refresh-modes/snapshot.md

@@ -20,12 +20,6 @@ To read snapshots published to S3 without naming the original federated source,
 ## Configuration
 
 ```yaml
-snapshots:
-  enabled: true
-  location: s3://my-bucket/snapshots/
-  params:
-    s3_auth: iam_role
-
 datasets:
   - from: postgres:public.my_table
     name: my_table
@@ -36,10 +30,13 @@ datasets:
       refresh_mode: snapshot
       refresh_check_interval: 30s # Poll interval; defaults to 1m
       snapshots: enabled
+      snapshots_location: s3://my-bucket/spice/snapshots/ # The writer's snapshot folder
       params:
         duckdb_file: /nvme/my_table.db
 ```
 
+This reader sets [`acceleration.snapshots_location`](../snapshots#override-the-snapshot-location-per-dataset) to the folder the writer publishes to, so it needs only read access to the snapshots. It could instead set [`runtime.state`](../../../reference/spicepod/runtime#runtimestate) to the writer's state location, whose `snapshots/` folder holds the snapshots. An instance with `runtime.state` checks the location at startup, which needs write access under `_spice/`, and does not report ready until the [check](../../../reference/spicepod/runtime#startup-check-of-the-state-location) passes, even though snapshot reads do not wait for it.
+
 ## Requirements
 
 - `acceleration.snapshots` must be `enabled` or `bootstrap_only`. Snapshot mode is a snapshot *consumer* only, so the two behave identically here: creation is skipped for the mode entirely and the dataset never publishes new snapshots — a separate writer must produce them.
@@ -64,16 +61,9 @@ Use `refresh_mode: snapshot` for read-only replicas that should not access the f
 
 ## Reload on S3 event notifications
 
-A reader that only polls loads a new snapshot up to one `refresh_check_interval` after a writer publishes it. When the snapshot location is on Amazon S3, set `snapshots.params.s3_queue_url` to an Amazon SQS queue that receives the location's [S3 event notifications](https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html), and `refresh_mode: snapshot` datasets reload as soon as a new snapshot is published:
+A reader that only polls loads a new snapshot up to one `refresh_check_interval` after a writer publishes it. When the snapshot location is on Amazon S3, set `s3_queue_url` in the location's params (`runtime.state.params`, or a dataset's [`snapshots_params`](../snapshots#override-the-snapshot-location-per-dataset)) to an Amazon SQS queue that receives the location's [S3 event notifications](https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html), and `refresh_mode: snapshot` datasets reload as soon as a new snapshot is published:
 
 ```yaml
-snapshots:
-  enabled: true
-  location: s3://my-bucket/snapshots/
-  params:
-    s3_auth: iam_role
-    s3_queue_url: https://sqs.us-east-1.amazonaws.com/123456789012/spice-reader-1
-
 datasets:
   - from: postgres:public.my_table
     name: my_table
@@ -84,19 +74,22 @@ datasets:
       refresh_mode: snapshot
       refresh_check_interval: 10m # Fallback check for missed notifications
       snapshots: bootstrap_only
+      snapshots_location: s3://my-bucket/spice/snapshots/
+      snapshots_params:
+        s3_queue_url: https://sqs.us-east-1.amazonaws.com/123456789012/spice-reader-1
 ```
 
 A writer makes a snapshot current by rewriting the location's `metadata.json`, so an `ObjectCreated` notification for that object triggers the reload. Spice reads `metadata.json` once for each batch of such notifications it receives and reloads only the datasets whose current snapshot changed, with the same snapshot id, schema, and checksum checks as a poll. Notifications for snapshot files and for removed objects are deleted from the queue without a reload. A reader that is still waiting for its first snapshot restores it when the notification arrives.
 
 Set up the queue as follows:
 
-- **One queue per reader process.** SQS delivers each message to one receiver, so readers that share a queue each see only some notifications. To serve several readers, publish the bucket's notifications to an Amazon SNS topic and subscribe one queue per reader process. All `refresh_mode: snapshot` datasets in one process share that process's queue.
-- **Only this location's notifications.** Filter the bucket notification by the location's prefix. A notification for an object outside the location is logged as an error and left on the queue, which delivers it again after each visibility timeout until it expires.
+- **One queue per reader process.** SQS delivers each message to one receiver, so readers that share a queue each see only some notifications. To serve several readers, publish the bucket's notifications to an Amazon SNS topic and subscribe one queue per reader process. All `refresh_mode: snapshot` datasets in one process share that process's queue. Snapshot locations in one process that name the same queue share one consumer, which reads the queue with the credentials of the first location. A location that names the queue with other credentials logs a warning that names both locations.
+- **Only the notifications of the locations on the queue.** Filter the bucket notification by the prefix of each snapshot location that names the queue. A notification for an object outside every one of those locations is logged as an error and left on the queue, which delivers it again after each visibility timeout until it expires. When the location is the `snapshots/` folder of `runtime.state.location`, the notification can cover the whole `runtime.state` prefix: notifications for the other [runtime state folders](../../../reference/spicepod/runtime#runtime-state-layout), such as the scheduler's `cluster/` folder, are deleted from the queue without a log.
 - **Permissions.** Spice reads the queue with the snapshot location's credentials: `s3_key` and `s3_secret` when both are set, otherwise the default AWS credential chain. The region comes from the queue URL. Grant `sqs:ReceiveMessage` and `sqs:DeleteMessage` on the queue.
 
 Datasets keep checking the location every `refresh_check_interval`, which covers notifications that never arrive. When Spice cannot reach SQS, it logs a warning when the outage starts, an error if it lasts five minutes, and an info message when it recovers. Datasets reload on `refresh_check_interval` in the meantime.
 
-A queue URL that is empty, is an ARN, or is not an SQS queue URL, and a snapshot location that is not `s3://`, fail the dataset at load with an error that names `s3_queue_url`. When `s3_queue_url` is set but no dataset uses `refresh_mode: snapshot`, Spice logs a warning and does not read the queue. A dataset that reads snapshots with [`file_format: snapshot`](../snapshots#serve-a-dataset-from-published-snapshots) does not accept `s3_queue_url` and checks for new snapshots every `refresh_check_interval`.
+A queue URL that is empty, is an ARN, or is not an SQS queue URL, and a snapshot location that is not `s3://`, fail the dataset at load with an error that names `s3_queue_url`. When `s3_queue_url` is set but no dataset uses `refresh_mode: snapshot`, Spice logs a warning that names the setting that holds the queue, such as `runtime.state.params.s3_queue_url`, and does not read the queue. A cluster scheduler, which loads no accelerations, does not log this warning. A dataset that reads snapshots with [`file_format: snapshot`](../snapshots#serve-a-dataset-from-published-snapshots) does not accept `s3_queue_url` and checks for new snapshots every `refresh_check_interval`.
 
 ## Related Topics