fix(gridfs): close remaining upload races and stream append safely

config/package-budget.json

@@ -1,8 +1,8 @@
 {
-  "comment": "The 3.5.3 bundle ceilings were 881096 CJS and 878096 ESM. The GridFS cross-operation ownership, durable session CAS/abort fencing, delete protection and idle-TTL migration add ~4.4/4.7 KiB over that ceiling (CI build 2026-11-12: 885536 CJS / 882819 ESM). Raise to 890000 / 887000, keeping ~4 KiB reviewed headroom; no dependency change. Enforce these ceilings in release preflight.",
+  "comment": "The previous 890000 / 887000 ceilings were exceeded by the GridFS remaining-issue fix (local 2026-11-12 build: 892639 CJS / 889922 ESM). The added bytes are the incremental Readable append consumer, beginResumableUpload post-insert files re-check, finalize pending rollback, and rename release-owner alignment — no dependency change. Raise to 896000 / 893000, keeping ~3 KiB reviewed headroom. Enforce these ceilings in release preflight.",
   "artifacts": {
-    "dist/cjs/index.cjs": 890000,
-    "dist/esm/index.mjs": 887000,
+    "dist/cjs/index.cjs": 896000,
+    "dist/esm/index.mjs": 893000,
     "dist/cjs/cli/data-task.cjs": 50000
   }
 }

docs/en/gridfs-recovery.md

@@ -6,7 +6,14 @@
 
 The ownership registry is the per-bucket `<bucket>.uploads` collection. A reservation excludes ordinary upload, uploadStream, resumable append/finalize/abort, delete, rename and orphan cleanup for the same ObjectId. A missing file document **does not** prove its chunks are orphaned: an ordinary upload may still be writing, or GridFS driver delete may have removed the files document before chunks. Recovering while an old writer can resume can destroy new data, even with a token-CAS. The maintenance process cannot prove cluster/process shutdown on its own.
 
-The script `scripts/maintenance/gridfs-recover.cjs` uses a default read-only preview, exact state/token/timestamp checks, a persisted `recovering` ownership claim, verified preservation of committed visible chunks, and an append-only-by-intent audit collection `<bucket>.ownership_recovery_audit`. It never performs a time-based lock steal.
+The script `scripts/maintenance/gridfs-recover.cjs` uses a default read-only preview, exact state/token/timestamp checks, a persisted `recovering` ownership claim, verified preservation of committed visible chunks, and an append-only-by-intent audit collection `<bucket>.ownership_recovery_audit`. It never performs a time-based lock steal. Dry-run plans may use a pre-claim visibility snapshot; **apply always re-reads `files` after the exclusive claim**. If visibility changed, or `verifyVisibleFile` cannot confirm a visible file that the plan required, the tool fails closed and leaves the `recovering` owner in place.
+
+To survey unknown-id leftover chunks without deleting anything, use the inspect-only reporter `scripts/maintenance/gridfs-inspect-orphans.cjs`. It walks `<bucket>.chunks` in index order with range pagination, re-checks `files` and `uploads` for each `files_id` at classification time, and reports `visible_file`, `owned`, or `candidate_orphan`. It has no apply mode, does not delete, and does not use document age as a deletion criterion. It is not part of the public `GridFSBucket` API. The CLI emits one NDJSON finding per line plus a final summary object. `--max-findings` (default 1000, cap 100000) bounds memory; a truncated scan returns `resumeAfter.filesId` for the next `--after-files-id`. Summary counters accumulate incrementally.
+
+```sh
+export MONSQLIZE_RECOVERY_MONGODB_URI='<MongoDB URI from the secrets manager>'
+node scripts/maintenance/gridfs-inspect-orphans.cjs --db=my_database --bucket=fs --page-size=200 --max-findings=1000
+```
 
 ## Deployment and rollback gates (required)
 

docs/en/gridfs.md

@@ -46,7 +46,7 @@ Calling either method before `connect()` throws `NOT_CONNECTED`.
 `upload` defaults to `ifExists: 'error'`.
 
 - When `sha256` is set, the conflict key is `metadata.sha256`. The digest must be 64 lowercase hex characters. `Buffer` and `Uint8Array` payloads are hashed before any GridFS write. A mismatch throws `GRIDFS_INTEGRITY_MISMATCH` and leaves no visible file.
-- A `Readable` passed with `sha256` is buffered completely, then hashed, before any write. Large files should use the resumable API instead.
+- A `Readable` passed with `sha256` is buffered completely, then hashed, before any write. That is required to honor the contract that a failed verification leaves no visible file. `upload()` without `sha256` can stream, but the whole payload is still a single commit. For large files, use the resumable API with `appendChunks` and a `Readable` whose `highWaterMark` stays modest; do not pass a multi-gigabyte `Buffer` or a fully-buffered `upload({ sha256 })`.
 - `skip` with `sha256` returns the newest visible file with that digest. `skip` without `sha256` returns the newest visible file with the same filename. `duplicate` always creates another file.
 - A `metadata.sha256` value supplied inside `metadata` is caller data and is not verified. To request byte verification with `upload()`, use the top-level `sha256` option.
 - `uploadStream` stores its top-level `sha256` option in metadata only. It does not check the bytes and it does not accept `ifExists`.
@@ -75,13 +75,13 @@ const info = await bucket.finalizeResumableUpload(session.uploadId);
 await bucket.abortResumableUpload(session.uploadId);
 ```
 
-`beginResumableUpload` requires `length` and `sha256`. An id that already exists in `files` or `uploads` throws `GRIDFS_FILE_EXISTS`.
+`beginResumableUpload` requires `length` and `sha256`. An id that already exists in `files` or `uploads` throws `GRIDFS_FILE_EXISTS`. Those prechecks are not a lock: the session insert is the atomic id claim. After the insert, `files` is checked again. If a file became visible in that window, the new pending session is dropped (best effort), chunks of the visible file are never deleted, and the call throws `GRIDFS_FILE_EXISTS`.
 
 Chunk bytes must follow the session size:
 
 - Every chunk except the last has length `chunkSizeBytes`.
 - The last chunk has length `length % chunkSizeBytes`, or `chunkSizeBytes` when the length divides evenly.
-- A payload that does not split into those sizes throws `INVALID_ARGUMENT`. The whole payload is checked before any chunk is inserted.
+- A payload that does not split into those sizes throws `INVALID_ARGUMENT`. For a `Buffer` or `Uint8Array`, the whole payload is planned and checked before any chunk is inserted. For a `Readable`, complete chunks are written as they arrive; a trailing partial or oversize remainder is rejected during consumption, after any complete prefix chunks have been stored. A failed append throws and does not return `nextChunk`. Resume from the last successfully acknowledged position; if the stored prefix has moved, `GRIDFS_RESUME_MISMATCH` includes `expectedNextChunk`. Peak memory is the residual (shorter than the next expected chunk) plus the current incoming stream buffer, not the full payload. Use a modest `highWaterMark` for large files.
 
 `appendChunks` is addressed by `fromChunk`:
 
@@ -94,7 +94,9 @@ Chunk bytes must follow the session size:
 
 `finalizeResumableUpload` requires a contiguous prefix, a total length equal to the session length, and a sha256 equal to the session digest. It then inserts the files document, sets `metadata.sha256`, and deletes the session. Missing chunk indexes throw `GRIDFS_UPLOAD_INCOMPLETE` with `details[0].missingChunks`. A bad digest or a chunk of the wrong size throws `GRIDFS_INTEGRITY_MISMATCH`. The files document is not inserted in those cases.
 
-Retrying `finalizeResumableUpload` is idempotent while the session still exists: if a file with the same id is already visible and its `metadata.sha256` and length match the session, the call returns that file and deletes the session. A file with a different sha256 or length still throws `GRIDFS_FILE_EXISTS`. Once the session is gone (a finalize that fully succeeded, an abort, or the TTL), there is no digest to compare, so a retry throws `GRIDFS_FILE_NOT_FOUND`; use `getFile(uploadId)` to check the result.
+Retrying `finalizeResumableUpload` is idempotent while the session still exists: if a file with the same id is already visible, the library re-hashes the actual stored chunks and returns the file only when that digest and the file length match the session. A file with a different sha256 or length still throws `GRIDFS_FILE_EXISTS`. Once the session is gone (a finalize that fully succeeded, an abort, or the TTL), there is no digest to compare, so a retry throws `GRIDFS_FILE_NOT_FOUND`. Callers must persist `uploadId`, `length`, and `sha256` themselves. After a vanished session, use `getFile(uploadId)` to see whether a file is visible, then re-hash the **actual** bytes (for example `download(uploadId)` or a walk of `<bucket>.chunks`) and compare them to the persisted length and digest. Never treat `metadata.sha256` alone as proof: `uploadStream()` and a raw `metadata.sha256` can store an unverified claim. This package does not add a separate public verification API.
+
+A `finalizeResumableUpload` that fails with `GRIDFS_FILE_EXISTS` or `GRIDFS_INTEGRITY_MISMATCH` after it has claimed `finalizing` rolls the session back to `pending` so `abortResumableUpload` (and then a delete of a conflicting visible file) can proceed through the public API. Ambiguous driver errors during the files insert still leave `finalizing` for explicit recovery.
 
 `abortResumableUpload` atomically claims a pending session before deleting chunks. It fails with `GRIDFS_UPLOAD_BUSY` while append or finalize owns the session. It deletes the session and, when no files document is visible, that upload's chunks. A missing or already aborted session resolves, and any chunks left under that id without a visible file are still deleted. Aborting an id that already has a visible file does not delete that file's chunks.
 

docs/en/support-matrix.md

@@ -18,7 +18,7 @@
 | MongoDB Driver | 6.x / 7.x | `npm run test:server-matrix` |
 | MongoDB Server | 7.x / 8.x | strict `mongodb-memory-server` 7.0.37 / 8.0.26 single + replica set matrix |
 
-GridFS is a client convention on `<bucket>.files` and `<bucket>.chunks`. Its integration tests run on `mongodb-memory-server` 7.0.37. That does not add MongoDB 4.x, or any server outside 7.x / 8.x, to this matrix.
+GridFS is a client convention on `<bucket>.files` and `<bucket>.chunks`. Its integration tests run on `mongodb-memory-server` 7.0.37 and 8.0.26 in the current GridFS safety / release-gate jobs. That does not add MongoDB 4.x, or any server outside 7.x / 8.x, to this matrix. Search and Vector Search against Atlas, atlas-local, or mongot remain UNVERIFIED in this checkout; those deployments are not implied by the memory-server matrix.
 
 ## Default verification method
 

docs/zh/gridfs-recovery.md

@@ -8,6 +8,16 @@
 
 任何强制恢复必须先从集群所有节点停止并排空旧版和新版 GridFS 写入进程、队列消费者、定时任务与自动重启，确认没有仍可恢复的旧写入。恢复脚本无法在 MongoDB 层直接阻止旧进程已发出的 chunks 写入；只有外部进程停机隔离后才安全。不得混用忽略 ownership 的旧版与新版本。
 
+dry-run 可以使用领取所有权之前的可见性快照；**apply 必须在独占 claim 之后重新读取 `files`**。若可见性已经变化，或在需要可见文件时 `verifyVisibleFile` 无法确认，脚本失败并保留 `recovering` 所有权。
+
+排查未知 ID 的残留 chunks 时，使用只读巡检脚本 `scripts/maintenance/gridfs-inspect-orphans.cjs`。它按已有 `{ files_id, n }` 索引分页扫描 `<bucket>.chunks`，在分类时重新核对 `files` 和 `uploads`，报告 `visible_file`、`owned` 或 `candidate_orphan`。没有 apply 模式，不会删除，也不会按文档年龄删除。它不是公开的 `GridFSBucket` API。CLI 按行输出 NDJSON 结果及一条摘要。`--max-findings`（默认 1000，上限 100000）限制单次内存；截断时用 `resumeAfter.filesId` 作为下次的 `--after-files-id`。摘要计数在扫描过程中增量累计。
+
+```sh
+export MONSQLIZE_RECOVERY_MONGODB_URI='<从密钥管理系统读取的 MongoDB URI>'
+node scripts/maintenance/gridfs-inspect-orphans.cjs --db=database --bucket=fs --page-size=200 --max-findings=1000
+```
+
+
 ## 上线和回滚步骤
 
 1. 冻结所有 GridFS 写入，禁止新请求和自动重启，核查所有旧版 Writer 已排空；保留相关运行实例、进程、队列、版本和时间证据。

docs/zh/gridfs.md

@@ -46,7 +46,7 @@ const modelBucket = model.gridfs(bucketName?: string, opts?: {
 `upload` 的 `ifExists` 默认为 `'error'`。
 
 - 提供 `sha256` 时，冲突键是 `metadata.sha256`。摘要必须是 64 位小写十六进制。`Buffer` 与 `Uint8Array` 会在任何 GridFS 写入之前完成哈希。不一致时抛出 `GRIDFS_INTEGRITY_MISMATCH`，并且不会留下可见文件。
-- 带 `sha256` 的 `Readable` 会先完整缓冲再哈希，然后才写入。大文件应改用可恢复上传。
+- 带 `sha256` 的 `Readable` 会先完整缓冲再哈希，然后才写入。这是为了遵守“校验失败不会留下可见文件”的契约。不带 `sha256` 的 `upload()` 可以边读边写，但整段 payload 仍是一次提交。大文件应使用可恢复上传，用 `appendChunks` 配合 `highWaterMark` 较小的 `Readable`，不要传入数 GB 的 `Buffer`，也不要走会全量缓冲的 `upload({ sha256 })`。
 - 带 `sha256` 的 `skip` 返回该摘要最新的可见文件。不带 `sha256` 的 `skip` 返回同名最新可见文件。`duplicate` 总是再创建一份。
 - 通过 `metadata` 对象传入的 `metadata.sha256` 是调用方数据，不会被验证。`upload()` 如需校验字节，必须使用顶层 `sha256` 选项。
 - `uploadStream` 的顶层 `sha256` 只写入 metadata，不校验字节，也不接受 `ifExists`。
@@ -75,13 +75,13 @@ const info = await bucket.finalizeResumableUpload(session.uploadId);
 await bucket.abortResumableUpload(session.uploadId);
 ```
 
-`beginResumableUpload` 必须提供 `length` 和 `sha256`。`files` 或 `uploads` 中已经存在该 id 时抛出 `GRIDFS_FILE_EXISTS`。
+`beginResumableUpload` 必须提供 `length` 和 `sha256`。`files` 或 `uploads` 中已经存在该 id 时抛出 `GRIDFS_FILE_EXISTS`。预检查不是锁，会话插入才是原子占位。插入之后会再次检查 `files`。若在这个窗口里文件已经可见，会尽量删除刚插入的 pending 会话，**绝不会**删除可见文件的 chunks，并抛出 `GRIDFS_FILE_EXISTS`。
 
 分片长度必须符合会话：
 
 - 除最后一片外，每片长度都是 `chunkSizeBytes`。
 - 最后一片长度是 `length % chunkSizeBytes`；能够整除时仍是 `chunkSizeBytes`。
-- 无法按这些长度切开的 payload 抛出 `INVALID_ARGUMENT`。整段 payload 会在插入任何分片之前完成检查。
+- 无法按这些长度切开的 payload 抛出 `INVALID_ARGUMENT`。对 `Buffer` / `Uint8Array`，整段 payload 会在插入任何分片之前完成规划与检查。对 `Readable`，完整分片会随读随写；末尾不足或超出的残余会在消费过程中被拒绝，此前已经写入的完整前缀分片会保留。失败的 append 会抛错，不会返回 `nextChunk`。应从上一次已成功确认的位置继续；若已写入前缀已前进，以 `GRIDFS_RESUME_MISMATCH.expectedNextChunk` 为准。峰值内存是“下一片所需长度以内的残余 + 当前流入的 stream 缓冲”，不会随整个 payload 线性增长。大文件应使用较小的 `highWaterMark`。
 
 `appendChunks` 以 `fromChunk` 定位：
 
@@ -94,7 +94,9 @@ await bucket.abortResumableUpload(session.uploadId);
 
 `finalizeResumableUpload` 要求分片从 0 连续、总长度等于会话长度，且 sha256 等于会话摘要。然后插入 files 文档、写入 `metadata.sha256`，并删除会话。缺少的分片序号抛出 `GRIDFS_UPLOAD_INCOMPLETE`，`details[0].missingChunks` 列出缺口。摘要不符或分片长度不对抛出 `GRIDFS_INTEGRITY_MISMATCH`。这些失败都不会插入 files 文档。
 
-会话仍在时，重试 `finalizeResumableUpload` 是幂等的：若同 id 的文件已可见，且其 `metadata.sha256` 与长度都和会话一致，调用直接返回该文件并删除会话。sha256 或长度不同的文件仍抛 `GRIDFS_FILE_EXISTS`。会话已不存在时（上一次 finalize 已完整成功、已 abort 或 TTL 过期），没有可比对的摘要，重试抛 `GRIDFS_FILE_NOT_FOUND`，可用 `getFile(uploadId)` 确认结果。
+会话仍在时，重试 `finalizeResumableUpload` 是幂等的：若同 id 的文件已可见，库会重新哈希**实际存储的 chunks**，只有摘要和长度都与会话一致时才返回该文件并删除会话。sha256 或长度不同的文件仍抛 `GRIDFS_FILE_EXISTS`。会话已不存在时（上一次 finalize 已完整成功、已 abort 或 TTL 过期），没有可比对的摘要，重试抛 `GRIDFS_FILE_NOT_FOUND`。调用方必须自己持久化 `uploadId`、`length` 和 `sha256`。会话消失后，先用 `getFile(uploadId)` 看文件是否可见，再对**实际字节**重新哈希（例如 `download(uploadId)` 或遍历 `<bucket>.chunks`），与自己保存的长度和摘要比对。不要只信任 `metadata.sha256`：`uploadStream()` 和调用方写入的 `metadata.sha256` 可以是未校验的声明。本包不为此再提供新的公开校验 API。
+
+`finalizeResumableUpload` 在已经占有 `finalizing` 之后，若因 `GRIDFS_FILE_EXISTS` 或 `GRIDFS_INTEGRITY_MISMATCH` 失败，会把会话 CAS 回 `pending`，以便通过公开 API 执行 `abortResumableUpload`（然后再删除冲突的可见文件）。files 插入过程中含义不明的驱动错误仍会保留 `finalizing`，需显式恢复。
 
 `abortResumableUpload` 先原子取得 pending 会话的所有权；append 或 finalize 仍在执行时返回 `GRIDFS_UPLOAD_BUSY`，绝不删除其 chunks。取得所有权后删除会话；如果还没有可见的 files 文档，同时删除该上传的 chunks。会话不存在或已经中止时直接成功，并且仍会删除该 id 下没有可见文件的残留 chunks。对已经可见的文件调用 abort 不会删除该文件的 chunks。
 

docs/zh/support-matrix.md

@@ -18,7 +18,7 @@
 | MongoDB Driver | 6.x / 7.x | `npm run test:server-matrix` |
 | MongoDB Server | 7.x / 8.x | `mongodb-memory-server` 7.0.37 / 8.0.26 single + replica set 严格矩阵 |
 
-GridFS 是 `<bucket>.files` 与 `<bucket>.chunks` 上的客户端约定。它的集成测试跑在 `mongodb-memory-server` 7.0.37 上。这不会把 MongoDB 4.x，或 7.x / 8.x 以外的服务器，写进本矩阵。
+GridFS 是 `<bucket>.files` 与 `<bucket>.chunks` 上的客户端约定。它的集成测试跑在当前 GridFS safety / release-gate 任务中的 `mongodb-memory-server` 7.0.37 和 8.0.26 上。这不会把 MongoDB 4.x，或 7.x / 8.x 以外的服务器，写进本矩阵。本检出内针对 Atlas、atlas-local 或 mongot 的 Search / Vector Search 仍为 UNVERIFIED；memory-server 矩阵不能外推到这些部署。
 
 ## 默认验证方式