Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/storage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
- `parseEventLogContent` and `validateEventRecords` require each record to match `EventRecordSchema`, the first `seq` to be 0, and each next `seq` to be the previous one plus 1. Blank lines are skipped. [eventLogCodec.ts:parseEventLogContent] [eventLogCodec.ts:validateEventRecords]
- Read and write paths in `manifests.ts` must be non-empty and absolute. Invalid JSON and schema failures throw `MANIFEST_VALIDATION_ERROR`. Other read errors throw `STORAGE_READ_ERROR`. Write failures throw `STORAGE_WRITE_ERROR`, except that `writeManifest` and `writeDestroyMarker` throw `SESSION_NOT_FOUND` when the session directory is gone. `writeDestroyMarker` writes an empty file through a temp file and a rename, like `writeManifest`, so it replaces a symlink at that path instead of writing through it, and it creates no folder. [manifests.ts:readValidatedJsonFile] [manifests.ts:writeTextFileAtomic] [manifests.ts:writeManifest] [manifests.ts:writeDestroyMarker]
- `ArtifactEntrySchema` and `ArtifactManifestSchema` are `.strict()`. `kind` is `screenshot`, `snapshot`, `recording` or `video`. `capturedAtSeq` is an integer of at least -1 (-1 for a capture before the first event). The manifest `version` is the literal `1`. The manifest and entry `sessionId` must equal the session directory name. [artifactManifest.ts:ArtifactEntrySchema] [artifactManifest.ts:ArtifactManifestSchema]
- `appendArtifactWithRollback` serializes appends per session directory with an in-process `KeyedSerializer` (from `src/util/keyedSerializer.ts`). `publishArtifactWithRollback` runs its publish step in the same queue, so a failed append cannot delete a file that another capture reused in between. Inside that queue, `withArtifactManifestLock` holds `<sessionDir>/artifacts/manifest.json.lock` around the publish step and the manifest read and write, so the host and CLI processes do not drop each other's entries. The lock file is created from a temp file with `createWithoutReplacing` (a `link()`, or an exclusive copy on a filesystem without hard links), with the temp file's timestamps set just before each attempt, and holds `{ version: 1, token, pid, process }`, where `process` is the `captureHost` record or `null`. A writer waits up to 10 s and then throws a retryable `STORAGE_WRITE_ERROR` with `lockPath` and `holderPid`. Other file errors while taking the lock throw `STORAGE_WRITE_ERROR` too (`SESSION_NOT_FOUND` when the folder is gone), and `appendArtifactWithRollback` then removes `rollbackArtifactPath` unless the manifest already lists a file of that name. That check runs without the lock, so a concurrent writer that lists the same file name between the check and the removal is left with a missing file. A lock path that is not a regular file (a FIFO or a symlink) reads as an `unknown` owner, so a writer never blocks on it. It breaks the lock when `judgeHost` says the owner is dead, or when the owner reads `unknown` (a live pid on macOS, another pid namespace, an owner without a process record on Linux, or lock contents that do not parse) and the lock's modification time is more than 60 s away from now, in either direction. To break, a writer first takes `manifest.json.lock.break` the same exclusive way, re-reads the lock, removes it only when its bytes still equal the ones it judged, and then removes its break file. A break file older than 60 s is removed by age alone. Release removes the lock only when it holds this writer's token. It tries 3 times. When all fail, the process remembers the lock, and its next acquisition on that session breaks it at once. Until then, other processes wait for it as for a live owner. A writer that takes the lock also removes `manifest.json.lock.tmp-*` and break files that are older than 60 s. Limits: a writer with an `unknown` owner that stops for over 60 s inside the section loses the lock while it still runs, so two writers can overlap. A breaker stopped for over 60 s while it holds the break file loses it the same way. [artifactManifest.ts:withArtifactManifestLock] [artifactManifest.ts:appendArtifactWithRollback] [artifactManifest.ts:publishArtifactWithRollback]
- `appendArtifactWithRollback` serializes appends per session directory with an in-process `KeyedSerializer` (from `src/util/keyedSerializer.ts`). `publishArtifactWithRollback` runs its publish step in the same queue, so a failed append cannot delete a file that another capture reused in between. Inside that queue, `withArtifactManifestLock` holds `<sessionDir>/artifacts/manifest.json.lock` around the publish step and the manifest read and write, so the host and CLI processes do not drop each other's entries. The lock file is created from a temp file with `createWithoutReplacing` (a `link()`, or an exclusive copy on a filesystem without hard links), with the temp file's timestamps set just before each attempt, and holds `{ version: 1, token, pid, process }`, where `process` is the `captureHost` record or `null`. A writer waits up to 10 s and then throws a retryable `STORAGE_WRITE_ERROR` with `lockPath` and `holderPid`. Other file errors while taking the lock throw `STORAGE_WRITE_ERROR` too (`SESSION_NOT_FOUND` when the folder is gone), and `appendArtifactWithRollback` then removes `rollbackArtifactPath` unless the manifest already lists a file of that name. That check runs without the lock, so a concurrent writer that lists the same file name between the check and the removal is left with a missing file. A lock path that is not a regular file (a FIFO or a symlink) reads as an `unknown` owner, so a writer never blocks on it. It breaks the lock when `judgeHost` says the owner is dead, or when the owner reads `unknown` (a live pid on macOS, another pid namespace, an owner without a process record on Linux, or lock contents that do not parse) and the lock's modification time is more than 60 s away from now, in either direction. To break, a writer first announces itself with `manifest.json.lock.breakers/<id>`, a link of its temp file in a folder that holds only announcements, and lists that folder. So the listing costs the same however many artifacts the session holds. It creates the folder without `recursive`, and the last breaker removes it with `rmdir`, which removes only an empty folder. When another breaker's announcement is there and not stale (by the break-file rule below), it removes its own and waits. It removes stale announcements, whose names no other writer uses. So breakers act one at a time: of two that announce, the one that reads later sees the other. The acting breaker takes `manifest.json.lock.break` the same exclusive way as the lock, which also keeps out older agent-tty versions, re-reads the lock, removes it only when its bytes still equal the ones it judged, and then removes its break file and its announcement. A breaker that saw another one waits a random 0.5 to 1.5 times a backoff of 5 ms that doubles with each such meeting, up to 200 ms. Breakers that keep meeting so spread their attempts until one of them is alone. This makes a meeting less likely each time. It does not rule one out. The break file and the announcement hold the breaker's record. Another writer removes them when `judgeHost` says the breaker is dead, or when the file is more than 60 s old, also while the breaker lives. Release removes the lock only when it holds this writer's token. It tries 3 times. When all fail, the process remembers the lock, and its next acquisition on that session breaks it at once. Until then, other processes wait for it as for a live owner. A writer that found the lock taken, once it holds the lock, removes each break file and announcement that it could remove as above, and each `manifest.json.lock.tmp-*` file whose owner is dead, or reads `unknown` and the file is more than 60 s old. A writer that finds the lock free does not read the artifacts folder, so the temp file of a waiter that died stays until a later writer finds the lock taken. A waiting writer whose temp file another writer removed writes it again before its next attempt. Limits: a writer with an `unknown` owner that stops for over 60 s inside the section loses the lock while it still runs, so two writers can overlap. A breaker stopped for over 60 s while it holds its announcement or the break file loses them the same way. Writers of agent-tty versions without announcements take only the break file, and removing a stale break file reads it and removes it by path in two steps. So an older writer and a newer one that both remove a dead breaker's break file at the same time can still overlap, as two older writers can. A writer with an `unknown` owner that stops for over 60 s between setting its temp file's timestamps and the `link` fails with `STORAGE_WRITE_ERROR` when another writer removed the temp file in that time. [artifactManifest.ts:withArtifactManifestLock] [artifactManifest.ts:appendArtifactWithRollback] [artifactManifest.ts:publishArtifactWithRollback]
- `computeArtifactHealth` maps only `MANIFEST_VALIDATION_ERROR` to health `manifest-invalid` and rethrows other errors. It validates its result with `ArtifactHealthSummarySchema`. [artifactHealth.ts:computeArtifactHealth]
- The registry read returns `[]` on a read error, invalid JSON or a schema failure. The read checks each `homes` entry on its own and drops an invalid entry, so the valid entries survive. `resolveHomeRegistryPath` ignores an empty or relative `XDG_STATE_HOME`. [homeRegistry.ts:createHomeRegistry] [homeRegistry.ts:resolveHomeRegistryPath]
- `upsert` reads and then writes the registry with no lock. A code comment states that concurrent creates can race and drop an entry, which registers again on the next create. [homeRegistry.ts:createHomeRegistry]
Expand Down
Loading
Loading