Merge remote-tracking branch 'origin/main' into claude/issue-22642-flow-cel-record-binding

Author: claude

.changeset/22470-upload-scope-vocabulary.md

@@ -0,0 +1,40 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-storage': patch
+'@objectstack/client': minor
+---
+
+feat(spec,service-storage,client)!: one upload-scope list — the upload requests' `scope` and the SDK's `storage.upload` close to the new `UploadScope` enum, the `sys_file` scope select is built from it, and the upload doors answer any other scope with `400` naming the allowed values instead of `500 INTERNAL` (#22470)
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: registered upload-request-scope-closed -->
+
+**BREAKING** — an accept-set narrowing on a published request contract and on the SDK method that sends it, shipped as `minor` under the launch-window convention for accept-set narrowings, plus one new export.
+
+The presigned and chunked upload requests declared `scope` an open string, while the stored file record (`sys_file.scope`, a closed select) only ever took `user`, `tenant`, `private`, `temp` and `attachments`. Any other value reached the `sys_file` insert, the data engine refused it as an invalid option, and the upload door answered that caller error as `500 INTERNAL`, with a message telling the operator to restore the data engine.
+
+### What changes
+
+- **`UploadScopeSchema` / `UploadScope`** (`@objectstack/spec`, new, exported from `@objectstack/spec/api`): the upload-scope vocabulary, declared once — `user`, `tenant`, `private`, `temp`, `attachments`. It is a different list from `StorageScopeSchema`, which classifies a storage configuration and is not read by any upload.
+- **`GetPresignedUrlRequestSchema.scope` and `InitiateChunkedUploadRequestSchema.scope`** read it. The default stays `user`. A literal outside the list fails `tsc`, and a parse refuses it on the `scope` key.
+- **`client.storage.upload(file, scope)`** (`@objectstack/client`): the `scope` parameter is typed `UploadScope` instead of `string`, default `user` unchanged, so a scope outside the list fails `tsc` at the SDK call. `client.storage.getPresignedUrl` and `client.storage.initChunkedUpload` take the request types above and narrow with them.
+- **The `sys_file` scope select** (`@objectstack/service-storage`) takes its options from the enum, in its order, with the same labels. The stored values do not change.
+- **The presigned upload door and the chunked upload door** answer a scope outside the list — any string, a case variant, `null`, a number — with `400 INVALID_REQUEST`, naming the allowed values, before a file record, a session record, an upload URL or a backend upload exists. `public` is refused by the same gate and keeps its remedy (`acl: 'public_read'` on the stored file record). An omitted scope is the default `user`, as before. A real data-engine fault still answers `500`.
+
+### FROM → TO
+
+| before | what to write instead |
+| --- | --- |
+| an upload naming a scope outside the list (a key prefix such as `avatars`, a folder, a record path) | one of `user`, `tenant`, `private`, `temp`, `attachments`, or no `scope` for the default `user` |
+| `client.storage.upload(file, scope)` called with a `scope` typed `string` (`@objectstack/client`) | pass one of the five, or type the value `UploadScope` (`import type { UploadScope } from '@objectstack/spec/api'`) |
+| a caller passing a scope typed `string` into either upload request | type it `UploadScope` |
+
+**The one-line fix: send one of the five upload scopes, or none.** `attachments` is for a file whose referrers are record attachment rows (it is what orphan tombstoning reads); `temp` for a scratch file; otherwise `user` or `tenant`.
+
+**No stored file record moves**: no upload naming another scope ever succeeded, so no stored record carries one.
+
+### The kit
+
+- **D3 entry `upload-request-scope-closed`** carries the judgement no rewrite can make: which scope a file that was sent under a free name really belongs under. No D2 conversion: no metadata type carries the upload request, so `os migrate meta` has no authored source to rewrite.
+- **Liveness.** No ledger row: the liveness ledger walks metadata types, and no metadata type carries the upload request.

content/docs/references/api/storage.mdx

@@ -19,8 +19,8 @@ rather than proxying bytes through the API server.
 ## TypeScript Usage
 
 ```typescript
-import { CompleteChunkedUploadRequestSchema, CompleteChunkedUploadResponseSchema, CompleteUploadRequestSchema, FileDownloadUrlResponseSchema, FileTypeValidationSchema, FileUploadResponseSchema, GetPresignedUrlRequestSchema, InitiateChunkedUploadRequestSchema, InitiateChunkedUploadResponseSchema, PresignedUrlResponseSchema, RawUploadResponseSchema, UploadChunkRequestSchema, UploadChunkResponseSchema, UploadProgressSchema } from '@objectstack/spec/api';
-import type { CompleteChunkedUploadRequest, CompleteChunkedUploadResponse, CompleteUploadRequest, FileDownloadUrlResponse, FileTypeValidation, FileUploadResponse, GetPresignedUrlRequest, InitiateChunkedUploadRequest, InitiateChunkedUploadResponse, PresignedUrlResponse, RawUploadResponse, UploadChunkRequest, UploadChunkResponse, UploadProgress } from '@objectstack/spec/api';
+import { CompleteChunkedUploadRequestSchema, CompleteChunkedUploadResponseSchema, CompleteUploadRequestSchema, FileDownloadUrlResponseSchema, FileTypeValidationSchema, FileUploadResponseSchema, GetPresignedUrlRequestSchema, InitiateChunkedUploadRequestSchema, InitiateChunkedUploadResponseSchema, PresignedUrlResponseSchema, RawUploadResponseSchema, UploadChunkRequestSchema, UploadChunkResponseSchema, UploadProgressSchema, UploadScopeSchema } from '@objectstack/spec/api';
+import type { CompleteChunkedUploadRequest, CompleteChunkedUploadResponse, CompleteUploadRequest, FileDownloadUrlResponse, FileTypeValidation, FileUploadResponse, GetPresignedUrlRequest, InitiateChunkedUploadRequest, InitiateChunkedUploadResponse, PresignedUrlResponse, RawUploadResponse, UploadChunkRequest, UploadChunkResponse, UploadProgress, UploadScope } from '@objectstack/spec/api';
 
 // Validate data
 const result = CompleteChunkedUploadRequestSchema.parse(data);
@@ -224,7 +224,7 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 | **filename** | `string` | ✅ | Original filename |
 | **mimeType** | `string` | ✅ | File MIME type |
 | **size** | `number` | ✅ | File size in bytes |
-| **scope** | `string` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
+| **scope** | `Enum<'user' \| 'tenant' \| 'private' \| 'temp' \| 'attachments'>` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
 | **bucket** | `string` | optional | Specific bucket override (admin only) |
 
 
@@ -240,7 +240,7 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 | **mimeType** | `string` | ✅ | File MIME type |
 | **totalSize** | `integer` | ✅ | Total file size in bytes |
 | **chunkSize** | `integer` | optional (default: `5242880`) | Size of each chunk in bytes (minimum 5MB per S3 spec) |
-| **scope** | `string` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
+| **scope** | `Enum<'user' \| 'tenant' \| 'private' \| 'temp' \| 'attachments'>` | optional (default: `"user"`) | Storage scope the new file is filed under (default user; attachments for the record attachments surface). Not an access setting: the upload doors refuse public, and a file is served without sign-in only when its stored file record carries acl 'public_read' (ADR-0104). The upload request carries no acl; every upload is stored private. |
 | **bucket** | `string` | optional | Specific bucket override (admin only) |
 | **metadata** | `Record<string, string>` | optional | Custom metadata key-value pairs |
 
@@ -497,3 +497,18 @@ const result = CompleteChunkedUploadRequestSchema.parse(data);
 
 ---
 
+## UploadScope
+
+Storage scope an uploaded file is filed under
+
+### Allowed Values
+
+* `user`
+* `tenant`
+* `private`
+* `temp`
+* `attachments`
+
+
+---
+

content/docs/references/index.mdx

@@ -1,7 +1,7 @@
 ---
 title: Protocol reference — every schema by module
 navTitle: Protocol Reference
-description: Every schema published by @objectstack/spec — 1515 schemas across 14 protocol modules
+description: Every schema published by @objectstack/spec — 1516 schemas across 14 protocol modules
 ---
 
 {/* ⚠️  AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
@@ -21,7 +21,7 @@ counts are sums of the rows they head. Regenerate with
 | Module | Pages | Schemas | Description |
 | :--- | ---: | ---: | :--- |
 | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. |
-| [API Protocol](/docs/references/api) | 32 | 431 | REST contracts, endpoints, routing, realtime, batch, discovery. |
+| [API Protocol](/docs/references/api) | 32 | 432 | REST contracts, endpoints, routing, realtime, batch, discovery. |
 | [Automation Protocol](/docs/references/automation) | 13 | 70 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. |
 | [Data Protocol](/docs/references/data) | 29 | 175 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
 | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. |
@@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with
 | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
 | [System Protocol](/docs/references/system) | 34 | 275 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
 | [UI Protocol](/docs/references/ui) | 16 | 166 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
-| **Total** | **195** | **1515** | 14 protocol modules |
+| **Total** | **195** | **1516** | 14 protocol modules |
 
 ---
 
@@ -63,7 +63,7 @@ Agents, tools, skills, RAG and knowledge sources, model registry, conversations.
 
 ## API Protocol
 
-**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **32 pages, 431 schemas**
+**Source:** `packages/spec/src/api/` · **Import:** `@objectstack/spec/api` · **32 pages, 432 schemas**
 
 REST contracts, endpoints, routing, realtime, batch, discovery.
 
@@ -98,7 +98,7 @@ REST contracts, endpoints, routing, realtime, batch, discovery.
 | [`rest-server.zod.ts`](/docs/references/api/rest-server) | `BatchEndpointsConfig`, `CrudEndpointsConfig`, `CrudOperation`, `EndpointRegistry`, `GeneratedEndpoint`, `MetadataEndpointsConfig`, `RestApiConfig`, `RestServerConfig`, `RouteGenerationConfig` |
 | [`router.zod.ts`](/docs/references/api/router) | `ConflictResolutionStrategy`, `HttpMethod`, `RouteCategory`, `RouteDefinition`, `RouterConfig` |
 | [`sortability.zod.ts`](/docs/references/api/sortability) | `FieldSortability`, `ObjectSortability` |
-| [`storage.zod.ts`](/docs/references/api/storage) | `CompleteChunkedUploadRequest`, `CompleteChunkedUploadResponse`, `CompleteUploadRequest`, `FileDownloadUrlResponse`, `FileTypeValidation`, `FileUploadResponse`, `GetPresignedUrlRequest`, `InitiateChunkedUploadRequest`, `InitiateChunkedUploadResponse`, `PresignedUrlResponse`, `RawUploadResponse`, `UploadChunkRequest`, `UploadChunkResponse`, `UploadProgress` |
+| [`storage.zod.ts`](/docs/references/api/storage) | `CompleteChunkedUploadRequest`, `CompleteChunkedUploadResponse`, `CompleteUploadRequest`, `FileDownloadUrlResponse`, `FileTypeValidation`, `FileUploadResponse`, `GetPresignedUrlRequest`, `InitiateChunkedUploadRequest`, `InitiateChunkedUploadResponse`, `PresignedUrlResponse`, `RawUploadResponse`, `UploadChunkRequest`, `UploadChunkResponse`, `UploadProgress`, `UploadScope` |
 | [`versioning.zod.ts`](/docs/references/api/versioning) | `VersionDefinition`, `VersionNegotiationResponse`, `VersionStatus`, `VersioningConfig`, `VersioningStrategy` |
 | [`websocket.zod.ts`](/docs/references/api/websocket) | `AckMessage`, `CursorMessage`, `CursorPosition`, `DocumentState`, `EditMessage`, `EditOperation`, `EditOperationType`, `ErrorMessage`, `EventMessage`, `EventPattern`, `EventSubscription`, `PingMessage`, `PongMessage`, `PresenceMessage`, `PresenceState`, `PresenceUpdate`, `SimpleCursorPosition`, `SimplePresenceState`, `SubscribeMessage`, `UnsubscribeMessage`, `UnsubscribeRequest`, `WebSocketConfig`, `WebSocketEvent`, `WebSocketMessage`, `WebSocketMessageType`, `WebSocketPresenceStatus`, `WebSocketServerConfig` |
 

packages/client/src/index.ts

@@ -37,6 +37,7 @@ import {
   CompleteChunkedUploadRequest,
   CompleteChunkedUploadResponse,
   UploadProgress,
+  UploadScope,
   ListNotificationsResponse,
   MarkNotificationsReadResponse,
   MarkAllNotificationsReadResponse,
@@ -5189,7 +5190,7 @@ export class ObjectStackClient {
    * Storage Services
    */
   storage = {
-    upload: async (file: any, scope: string = 'user'): Promise<FileUploadResponse> => {
+    upload: async (file: any, scope: UploadScope = 'user'): Promise<FileUploadResponse> => {
         // 1. Get Presigned URL
         const presignedReq: GetPresignedUrlRequest = {
             filename: file.name,

packages/services/service-storage/src/objects/system-file.object.ts

@@ -1,6 +1,28 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
 import { ObjectSchema, Field } from '@objectstack/spec/data';
+import { UploadScopeSchema, type UploadScope } from '@objectstack/spec/api';
+
+/**
+ * Display labels of the `scope` select, one per upload scope.
+ *
+ * The VALUES are not listed here: the select reads them off
+ * `UploadScopeSchema`, the one upload-scope list `@objectstack/spec` declares
+ * for the upload requests and the upload doors too (#22470), so the store, the
+ * request and the doors cannot drift apart. `Record<UploadScope, …>` makes a
+ * member added to that enum without a label here a compile error.
+ */
+const UPLOAD_SCOPE_LABELS: Record<UploadScope, string> = {
+  user: 'User',
+  tenant: 'Tenant',
+  private: 'Private',
+  temp: 'Temp',
+  // Files uploaded through the generic Attachments surface (#2727).
+  // Their only legitimate referrers are sys_attachment join rows, so
+  // this scope is the discriminator for orphan tombstoning (#2755) —
+  // field-attachment scopes above are never tombstoned.
+  attachments: 'Attachments',
+};
 
 /**
  * System File Object
@@ -62,19 +84,12 @@ export const SystemFile = ObjectSchema.create({
     // are rewritten to `user` by `backfill-sys-file-public-scope.ts`, the
     // operator step that must run before a copy of such a row can succeed.
     // Anonymous download is `acl: 'public_read'` and nothing else.
+    //
+    // The options are `UploadScopeSchema`'s values, in its order, each with
+    // its label above — the list the upload requests and doors read (#22470).
     scope: Field.select({
       label: 'Scope',
-      options: [
-        { label: 'User', value: 'user' },
-        { label: 'Tenant', value: 'tenant' },
-        { label: 'Private', value: 'private' },
-        { label: 'Temp', value: 'temp' },
-        // Files uploaded through the generic Attachments surface (#2727).
-        // Their only legitimate referrers are sys_attachment join rows, so
-        // this scope is the discriminator for orphan tombstoning (#2755) —
-        // field-attachment scopes above are never tombstoned.
-        { label: 'Attachments', value: 'attachments' },
-      ],
+      options: UploadScopeSchema.options.map((value) => ({ label: UPLOAD_SCOPE_LABELS[value], value })),
     }),
 
     bucket: Field.text({