Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .changeset/27549-chunked-init-declared-shape.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
'@objectstack/service-storage': patch
---

fix(service-storage): the chunked-upload init door parses its body with `InitiateChunkedUploadRequestSchema`, so a `totalSize` that is not a positive whole number and a `chunkSize` below the declared minimum are refused instead of minting a session that can never complete

Clause-②: no (narrowing)
Metadata-protocol: none

<!-- adr-0087: not-required (no-migration-prescription) A REST request body is refused at the chunked-upload init door when it does not parse with the spec schema it already declares (`InitiateChunkedUploadRequestSchema`, unchanged): no key of any metadata schema is removed, renamed or re-shaped, no export, type or error code is removed or renamed, and the refusal reuses the existing 400 INVALID_REQUEST envelope. There is nothing for `objectstack migrate meta` to rewrite and no tombstone. -->

**BREAKING** (an accept-set narrowing), shipped as `patch`: nothing a metadata author writes changes (`Metadata-protocol: none`), so the level follows the kind, and the banner and the ADR-0087 disposition stay.

**FROM.** `POST /api/v1/storage/upload/chunked` checked `totalSize` for truthiness only and read the rest of the body as sent:

- A negative `totalSize` such as `-5` answered `200` and minted a session with `totalChunks` of `-0`, so every chunk was then refused and the upload could never complete. A numeric string such as `"100"` was stored on the pending `sys_file` row as sent, and a fraction was accepted.
- A `chunkSize` below 5242880 (5 MiB, the declared minimum) was silently clamped up to 5242880; `null` was read as omitted, and a non-numeric one produced a `NaN` chunk count.
- `bucket` and `metadata` were stored whatever their type.

**TO.** The body is parsed with `InitiateChunkedUploadRequestSchema` before the `sys_file` row, the backend multipart and the upload-session row:

- `totalSize` must be a whole number of at least 1, sent as a JSON number.
- `chunkSize`, when sent, must be a whole number of at least 5242880, sent as a JSON number; a smaller one is refused, not clamped. An omitted `chunkSize` keeps the declared default of 5242880, so `totalChunks` for a valid body is unchanged.
- `bucket`, when sent, must be a string, and `metadata`, when sent, an object of string values.
- Anything else answers `400 INVALID_REQUEST` naming each failing field, and neither row is written. A missing `filename`, `mimeType` or `totalSize` and an off-vocabulary `scope` keep their existing `400` answers.

**Fix.** A client that sent a smaller `chunkSize` sends 5242880 or more, or omits it. A client that sent `totalSize` as a string sends the JSON number. The SDK's `storage.initChunkedUpload` takes the schema's own input type, so a typed caller already sends an in-schema body; no Console path calls this door.
12 changes: 9 additions & 3 deletions docs/qa/platform-checklist/areas/attachments-storage.json
Original file line number Diff line number Diff line change
Expand Up @@ -808,7 +808,7 @@
"title": "Chunked uploads are resumable; abandoned sessions are reaped and their backend multipart uploads ABORTED before the row (the only pointer) is deleted",
"since": "v15.1",
"status": "active",
"revision": 6,
"revision": 7,
"priority": "P2",
"surface": "api",
"personas": ["seeded admin (admin@objectos.ai)"],
Expand All @@ -823,7 +823,7 @@
]
},
"steps": [
"POST /api/v1/storage/upload/chunked { filename, mimeType, totalSize, chunkSize } and capture { uploadId, fileId } — the init route's required trio is filename/mimeType/`totalSize` (a body spelling `size` is refused 400 INVALID_REQUEST), and `chunkSize` is floored at 5 MiB, so a smaller requested value silently yields a different totalChunks than the caller computed (storage-routes.ts)",
"POST /api/v1/storage/upload/chunked { filename, mimeType, totalSize, chunkSize } and capture { uploadId, fileId } — the init route's required trio is filename/mimeType/`totalSize` (a body spelling `size` is refused 400 INVALID_REQUEST), and the body is parsed with InitiateChunkedUploadRequestSchema: a `totalSize` that is not a whole JSON number of at least 1, or a `chunkSize` below 5 MiB (5242880), is refused 400 INVALID_REQUEST naming the field, never clamped; an omitted `chunkSize` takes the declared 5242880 default (storage-routes.ts)",
"PUT one chunk, then GET /upload/chunked/<uploadId>/progress and record uploadedChunks/percentComplete/status ('in_progress') — progress is the first step of the SDK's resumeUpload",
"complete a full session on a second upload and verify its sys_upload_session row reaches status 'completed'",
"abandon the first session mid-flight; backdate its expires_at past the 1d TTL (system write) and trigger the lifecycle sweep",
Expand Down Expand Up @@ -882,7 +882,7 @@
"packages/services/service-storage/src/attachment-lifecycle.ts#createUploadSessionReapGuard (createUploadSessionReapGuard — abort-before-reap contract, completed/no-backend confirm branches, veto-on-failure)",
"packages/services/service-storage/src/objects/system-upload-session.object.ts#expires_at (status enum = the variants list; ttl expires_at+1d, retention 7d terminal statuses)",
"packages/services/service-storage/src/storage-route-ledger.ts (upload-chunked family)",
"packages/services/service-storage/src/storage-routes.ts#mimeType (POST /upload/chunked init contract: filename/mimeType/totalSize required, else 400 INVALID_REQUEST; chunkSize = Math.max(reqChunkSize ?? 5242880, 5242880), i.e. a 5 MiB floor)",
"packages/services/service-storage/src/storage-routes.ts#mimeType (POST /upload/chunked init contract: filename/mimeType/totalSize required, else 400 INVALID_REQUEST; the body parsed with InitiateChunkedUploadRequestSchema, so a chunkSize below 5242880 is refused 400 INVALID_REQUEST, not floored, and an omitted one takes the 5242880 default)",
"packages/services/service-storage/src/storage-routes.ts#expireIfPastDeadline (expireIfPastDeadline / markSessionFailed — the #7667 producers for 'expired' and 'failed'; chunk + complete refuse 410 UPLOAD_SESSION_EXPIRED, progress reports the status)",
"packages/spec/src/api/storage.zod.ts#UploadProgressSchema (UploadProgressSchema.status — the client-facing declaration the enum must stay in step with)",
"docs/plans/release-15.1-test-plan.md §C4 (#2970 item 4)"
Expand All @@ -903,6 +903,12 @@
"date": "2026-10-28",
"change": "automated.ref names two more storage-routes cases and the chunked dogfood file. Checklist-accuracy finding of the 18.2.0 wave 2 (QA run #26468, Tier 1 half A, at 756220c4; wave anchor #25807, slice B): storage-routes.test.ts carries two more terminal-status cases (expired on progress, a completed session left alone), and storage-chunked-resume-integrity.dogfood.test.ts drives the chunked doors over HTTP without reading the session row, so acceptance[0] is still unpinned. Re-read at base fd246c22 (storage-routes.test.ts, the two cases named in the ref). No clause changed",
"ref": "#25807"
},
{
"revision": 7,
"date": "2026-10-29",
"change": "steps[0] and the storage-routes.ts source line described the 5 MiB chunkSize floor (a smaller requested chunkSize silently clamped up). The init door now parses its body with InitiateChunkedUploadRequestSchema, so a chunkSize below 5242880 and a totalSize that is not a whole number of at least 1 are refused 400 INVALID_REQUEST naming the field; an omitted chunkSize keeps the 5242880 default. Both strings now describe the refusal. No clause, oracle or variant changed",
"ref": "#27549"
}
]
},
Expand Down
50 changes: 50 additions & 0 deletions packages/services/service-storage/src/storage-routes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,56 @@ describe('Storage REST Routes', () => {
expect(progressRes._json.data.uploadedChunks).toBe(0);
expect(progressRes._json.data.status).toBe('in_progress');
});

// [#27549] The init body is parsed with `InitiateChunkedUploadRequestSchema`:
// declared is enforced. `totalSize` was checked for truthiness only, so a
// negative or string total minted a session no chunk could ever complete,
// and a `chunkSize` under the declared minimum was clamped up instead of
// refused. Each refusal names its field and writes neither row.
it.each<[string, Record<string, unknown>, string]>([
['a negative totalSize', { totalSize: -5 }, 'totalSize'],
['a numeric-string totalSize', { totalSize: '100' }, 'totalSize'],
['a zero totalSize', { totalSize: 0 }, 'totalSize'],
['a fractional totalSize', { totalSize: 1.5 }, 'totalSize'],
['a chunkSize below the declared minimum', { totalSize: 100, chunkSize: 1024 }, 'chunkSize'],
['a numeric-string chunkSize', { totalSize: 100, chunkSize: '5242880' }, 'chunkSize'],
['a metadata value that is not a string', { totalSize: 100, metadata: { tag: 1 } }, 'metadata'],
])('REFUSED 400 INVALID_REQUEST: %s, with no sys_file or upload-session row written', async (_label, fields, named) => {
const initHandler = httpServer._getHandler('POST', '/api/v1/storage/upload/chunked')!;
const createFile = vi.spyOn(store, 'createFile');
const createSession = vi.spyOn(store, 'createSession');
const res = createMockRes();
await initHandler(createMockReq({ body: { filename: 'b.bin', mimeType: 'application/octet-stream', ...fields } }), res);

expect(res._status, JSON.stringify(res._json)).toBe(400);
expect(res._json.success).toBe(false);
expect(res._json.error.code).toBe('INVALID_REQUEST');
expect(res._json.error.message).toContain(named);
expect(createFile, 'no pending sys_file row').not.toHaveBeenCalled();
expect(createSession, 'no upload-session row').not.toHaveBeenCalled();
});

it.each<[string, number | undefined, number, number]>([
['an omitted chunkSize takes the declared default', undefined, 5242880, 3],
['a chunkSize at the declared minimum', 5242880, 5242880, 3],
['a chunkSize above the declared minimum', 10485760, 10485760, 2],
])('CONTROL: %s and mints a session with the same total_chunks', async (_label, chunkSize, servedChunkSize, totalChunks) => {
const initHandler = httpServer._getHandler('POST', '/api/v1/storage/upload/chunked')!;
const totalSize = 12_000_000;
const res = createMockRes();
await initHandler(
createMockReq({ body: { filename: 'b.bin', mimeType: 'application/octet-stream', totalSize, chunkSize } }),
res,
);

expect(res._status, JSON.stringify(res._json)).toBe(200);
expect(res._json.data.chunkSize).toBe(servedChunkSize);
expect(res._json.data.totalChunks).toBe(totalChunks);
const session = await store.getSession(res._json.data.uploadId);
expect(session!.total_chunks).toBe(totalChunks);
expect(session!.chunk_size).toBe(servedChunkSize);
expect((await store.getFile(res._json.data.fileId))!.size).toBe(totalSize);
});
});

// ── terminal session statuses (#7667) ──────────────────────────────────
Expand Down
38 changes: 31 additions & 7 deletions packages/services/service-storage/src/storage-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ import type { IHttpServer, IHttpRequest, IHttpResponse, IStorageService, RouteHa
import type { ErrorCode, RegisteredErrorCode, StandardErrorCode } from '@objectstack/spec/api';
// [#22470] The ONE upload-scope list — the upload requests' `scope` and the
// `sys_file.scope` select read it too. A value import: the doors ask it.
import { UploadScopeSchema } from '@objectstack/spec/api';
// [#27549] The chunked-upload init door parses its whole body with the
// declared request schema, so declared is enforced.
import { InitiateChunkedUploadRequestSchema, UploadScopeSchema } from '@objectstack/spec/api';
// The declared envelope is written in ONE place for the whole platform (#3973).
// [#23174] A catch-all 5xx answers through `sendServerFault`: the route's
// generic text on the wire, the caught error with its stack in the fault log.
Expand Down Expand Up @@ -174,6 +176,18 @@ function isDeclaredByteCount(size: unknown): size is number {
const PRESIGNED_SIZE_REFUSAL_MESSAGE =
'size must be the file size in bytes: a non-negative number, sent as a JSON number';

/**
* [#27549] The chunked-upload init door's refusal for a body that does not
* parse with `InitiateChunkedUploadRequestSchema`: every failing field named
* with the schema's own words. `totalSize` is a whole number of bytes, at
* least 1, and `chunkSize` (when sent) a whole number at or above the
* declared 5 MiB minimum — a smaller one is refused, never clamped up.
*/
function chunkedInitRefusalMessage(issues: ReadonlyArray<{ path: ReadonlyArray<PropertyKey>; message: string }>): string {
const named = issues.map((issue) => `${issue.path.map(String).join('.') || '(body)'}: ${issue.message}`);
return `The chunked upload request does not match its declared shape. ${named.join('; ')}`;
}

/**
* [#27412] The chunk door's `chunkIndex` path segment, read strictly: decimal
* digits only, so `-1`, `0.5`, `1e2` and `0x1` are not indexes. `null` ⇒ not
Expand Down Expand Up @@ -1100,21 +1114,31 @@ export function registerStorageFamily(
try {
const session = await requireUploadSession(req, res);
if (session === false) return;
const { filename, mimeType, totalSize, chunkSize: reqChunkSize, scope, bucket, metadata } = req.body ?? {};
if (!filename || !mimeType || !totalSize) {
const body = req.body ?? {};
if (!body.filename || !body.mimeType || body.totalSize == null) {
sendError(res, 400, 'INVALID_REQUEST', 'filename, mimeType, and totalSize are required');
return;
}
// [#22470] Same scope gate as the presigned door — before the file row,
// the backend multipart and the session row.
if (!requireAcceptedUploadScope(scope, res)) return;
// the backend multipart and the session row. Asked before the parse
// below so an off-vocabulary scope keeps its own remedy.
if (!requireAcceptedUploadScope(body.scope, res)) return;
// [#27549] Declared is enforced: the body must parse with the declared
// request schema, before anything is stored or minted. A total that is
// not a positive whole number used to mint a session no chunk could
// complete, and a `chunkSize` under the minimum was clamped up.
const parsed = InitiateChunkedUploadRequestSchema.safeParse(body);
if (!parsed.success) {
sendError(res, 400, 'INVALID_REQUEST', chunkedInitRefusalMessage(parsed.error.issues));
return;
}
const { filename, mimeType, totalSize, chunkSize, scope, bucket, metadata } = parsed.data;
// [#22283] The declared total against the saved limit — before the file
// row, the backend multipart and the session row. The chunk door judges
// the bytes as they arrive.
const { sessionTtl, maxUploadBytes } = currentLimits();
if (!requireWithinUploadLimit(Number(totalSize), maxUploadBytes, (n) => `The declared total size is ${n} bytes`, res)) return;
if (!requireWithinUploadLimit(totalSize, maxUploadBytes, (n) => `The declared total size is ${n} bytes`, res)) return;

const chunkSize = Math.max(reqChunkSize ?? 5242880, 5242880);
const totalChunks = Math.ceil(totalSize / chunkSize);

const fileId = randomUUID();
Expand Down
Loading