diff --git a/.github/ISSUE_SPECS/128-preserve-structured-storage-failures-instead-of-classifying-them-as-verified-corruption.md b/.github/ISSUE_SPECS/128-preserve-structured-storage-failures-instead-of-classifying-them-as-verified-corruption.md new file mode 100644 index 0000000..c4cb072 --- /dev/null +++ b/.github/ISSUE_SPECS/128-preserve-structured-storage-failures-instead-of-classifying-them-as-verified-corruption.md @@ -0,0 +1,119 @@ +# Preserve structured storage failures instead of classifying them as verified corruption + +## Problem + +The app investigation in `misofm/app#294` found that adapter 0.5.13 maps sparse +storage-open failures to `stem.corrupt`, before any download. UnknownError, +InvalidStateError, NotAllowedError, and NoModificationAllowedError injected at +root/open must not be reported as verified audio corruption. The original +device exception was not captured; injection does not establish that device's +root cause. + +Base: upstream main `602b7c7b77d6228a73a0b9932a8809a8933ba692`, package +`@misofm/engine-web-adapter@0.5.13`, Engine `0.4.3`, Effect `4.0.0-rc.112`. + +## Bounded T2 implementation + +This issue owns only the upstream adapter's public storage-error contract and +the sparse-store/worker boundaries that implement it. It does not change app +code, the sparse format, shutdown ownership, or close retry policy. + +- Introduce the documented public `storage.io` code for storage failures with + additive operation/resource/granular-phase details. +- Preserve unknown browser exception names and causes without guessing from + message text. +- Keep `stem.invalid_declaration` and `session.declaration_mismatch` for + declaration/metadata failures, and reserve `stem.corrupt` for actual digest + or byte-length verification evidence (`kind`, `expected`, `observed`, and + `algorithm` where applicable). +- Preserve typed failures, Effect interruptions, defects, and every finalizer + reason through the public edge and relevant Worker transports. +- Document supported close/cancellation semantics, add regression tests and an + unreleased changelog/evidence entry, and qualify public exports on a packed + artifact. + +## Acceptance + +- Inject root/open/read/write/close failures, including UnknownError, + InvalidStateError, NotAllowedError, and NoModificationAllowedError; none is + reported as verified integrity. +- Root-open refusal happens before resolver/network work and produces zero stem + fetches. +- Genuine wrong digest and wrong length remain `stem.corrupt` with structured + verification evidence. +- Invalid metadata/declaration remains distinct from storage and integrity + failures. +- Worker round trips preserve nested causes, interruption/defect distinctions, + and finalizer failures rather than flattening to the first reason. +- Public exports and package tests pass against a fresh packed artifact. + +## Workflow and boundaries + +Implementation is Luna reasoning=max followed by a fresh independent Astra +reasoning=medium verification. No spawning, push, PR creation, merge, or +publication is authorized here; coordinator owns reviewed integration. + +The release metadata remains at version 0.5.13. Do not copy app sources, +patch `node_modules`, rewrite the sparse format, force shutdown, or invent a +retry of a failed physical close. Record exact commands, results, candidate +SHA, package provenance, fixture shape, close/cancellation semantics, and +browser capability limitations below. + +## Implementation evidence + +Reviewed implementation: `27795844c2ebaed2061eb0b6cdde0d15a2550973` +(parent `93cf2f790ea2ecba5428e8a32ac3bbc40c05b605`). +Fresh independent Astra medium review approved this exact clean candidate; +this subsequent evidence-only commit changes no source or test code. + +Verification completed: + +- `npm exec -- tsc -p tsconfig.test.json` — test compilation passed. +- `node --test .test-dist/tests/error-contract.test.js .test-dist/tests/sparse-store.test.js .test-dist/tests/opfs-storage.test.js` — 104/104 focused tests passed. +- Recursive reservation coverage — 1,026 nested pad cases (cause and causes, every pad length 0..512) plus the bounded Unicode/escape depth-width table passed serialization, repeat serialization, and revival; every serialized result stayed at or below 16,384 UTF-8 bytes and retained cause summaries were nonempty and valid. +- `npm run check` — passed, including format, wasm verification, lint, decoder checks, the full 396/396 test suite, and package policy (`248 files, 507013 bytes`). +- `CHROME_EXECUTABLE=/home/bl/.cache/ms-playwright/chromium-1234/chrome-linux64/chrome npm run test:browser` — passed against the fresh packed consumer on Chromium 151.0.7922.34, 15 assets, no request failures or console errors. Artifact: `/tmp/engine-web-adapter-browser-TvwzzD`. + +The real storage-root `installSource` boundary keeps foreign/nested +`stem.corrupt` payloads and invalid proof shapes under primary `storage.io` +evidence; only adapter-performed verification with a meaningful differing +canonical digest/byte-length proof can promote integrity corruption. The +public legacy constructor/API remains compatible. Native interruption, +physical write/body settlement, and cleanup failures retain explicit +`interruption`/`body`/`finalizer` roles; physical settlement and ownership stay +authoritative, and cached-close policy is unchanged. + +Publication and app adoption remain outside this adapter change. The local +Linux WebKit runner lacks `FileSystemFileHandle`; that capability limitation +was not misreported as an OPFS pass. Subsequent existing macOS CI on exact +head `512029f387d3edb27608cdc9c57195219ba6889d` passed real OPFS and packed +indexed-sparse qualification in both Chromium 151.0.7922.34 and WebKit 26.5. +Runner provenance: macOS 15.7.9 arm64, Node 22.23.2, npm 10.9.8. +Run: https://github.com/misofm/engine-web-adapter/actions/runs/35694712900 +Artifact: `opfs-35694712900-1`, including checkout/runner/tool provenance, +actual OPFS cold/warm/marker-failure/late-cancellation evidence, and packed +three-warm-worker qualification for both browsers. This following commit +only records that evidence; it changes no implementation or tests. + +The upstream candidate's WebKit environment gap is closed by actual macOS +results. The final combined app and real iPhone/deadline qualification remain +open release gates; these upstream checks do not prove app recovery behavior +or identify the original device trigger. The fixed SDK is not published and +app adoption awaits the normal reviewed release workflow; neither merge nor +publication is authorized by this work. + +The committed public fixture `tests/fixtures/storage-error-contract.json` +records `storage.io`, coarse phase `store`, details +`{ operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io" }`, +and the retained `UnknownError` cause. Independent review of this exact candidate approved the recursive global +reservation invariant and retained all prior taxonomy, cancellation, physical +settlement, and integrity-proof closures. The reviewer independently ran +104 focused tests, the full 396-test check, and a fresh packed Chromium +consumer (artifact `/tmp/engine-web-adapter-browser-MHXdK5`, 15 assets, no +request failures or console errors). Generated verification covered 1,835 +cases / 14,680 serialization checks with a maximum of 16,337 UTF-8 bytes; +full padding sweeps, deep/wide/mixed trees, Unicode, JSON escapes, six +reserializations, and revival retained useful mandatory summaries. +The coordinator synchronizes this evidence to the matching issue before PR +creation. The macOS CI qualification above records only observed results; +no real-device or final combined app result is claimed here. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..61a79d3 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,8 @@ +# Changelog + +## Unreleased + +- Preserve browser storage failures as `storage.io` with operation/resource/ + phase context and bounded nested causes; reserve `stem.corrupt` for verified + digest or byte-length evidence and keep metadata/declaration failures + distinct. diff --git a/README.md b/README.md index 77eadd4..e5244d3 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,33 @@ identical operation is `transient`, frozen `details`, and the underlying `cause`. No Engine host object, Worker message or Effect value reaches a caller through it. +Storage access failures use the distinct `storage.io` code. Its coarse public +`.phase` remains `store`; storage details add `operation`, `resource`, and a +granular `phase`, while the original browser exception name and cause are +retained without classifying from message text. `stem.invalid_declaration` +covers invalid metadata or declarations. `stem.corrupt` is reserved for a +verified digest or byte-length mismatch and carries +`details.verification = { kind, expected, observed, algorithm? }`; a storage or +metadata refusal never receives that code. + +For example, an OPFS root refusal is app-consumable as: + +```json +{ + "name": "EngineWebAdapterError", + "message": "Sparse root access failed", + "code": "storage.io", + "phase": "store", + "details": { + "operation": "open", + "resource": "sparse-root", + "phase": "root-open", + "failureKind": "io" + }, + "cause": { "name": "UnknownError", "message": "Storage access was refused" } +} +``` + Console submission returns the SDK `CommandReport` unchanged. A semantic refusal has `ok: false`, `admitted: 0`, and its original reason, rejected index, and application sample; transport failures reject. Request identifiers are @@ -235,9 +262,15 @@ asset fields without discarding the other common fields; the low-level - Canonical PCM is headerless interleaved little-endian PCM16 or PCM24 at 44.1, 48, 88.2, or 96 kHz; there is no implicit sample-rate conversion. -`close()` is idempotent. The `./stems` entry exports what a caller can supply -or replace: the canonical-PCM resolver seam, the verified store and its storage -backends, the FLAC resolver, the pump, and the ring control words a `createPump` +`close()` is idempotent. Store close aborts active logical operations, waits for +their physical reads/writes and writer aborts to settle, then disposes shared +verification workers. Cancellation has the same settlement rule: late browser +work cannot publish a source or trigger an automatic retry. A failed physical +writer close is surfaced with its original cause and is not retried or replaced +by a forced shutdown; staging cleanup remains separately observable. The +session-owned OPFS backend is released by session teardown. The `./stems` entry +exports what a caller can supply or replace: the canonical-PCM resolver seam, +the verified store and its storage backends, the FLAC resolver, the pump, and the ring control words a `createPump` override reads. The existing ring control exports are SDK re-exports. Digests, admission width, the decoder pool and adapter Worker protocols remain internal; PCM ring arithmetic is owned by the SDK. @@ -420,7 +453,7 @@ ingest locks. Each lease also holds its historical folder-qualified `:pin:` lifetime lock so existing app recovery recognizes it. Recovery preserves ambiguous session pins and offline pins; it leaves the historical `staging/` directory alone. Explicit unsupported index versions -refuse with `stem.corrupt` before recovery changes any file. +refuse with `stem.invalid_declaration` before recovery changes any file. When a known storage estimate cannot fit a cold source, the store reclaims unpinned indexed entries by oldest `lastUsedAt`, breaking ties by identity, diff --git a/package.json b/package.json index 4771b3c..0bfad98 100644 --- a/package.json +++ b/package.json @@ -13,6 +13,7 @@ ], "files": [ "dist", + "CHANGELOG.md", "LICENSE", "NOTICE", "README.md" diff --git a/scripts/browser-packed.mjs b/scripts/browser-packed.mjs index bd93f7b..7b81c59 100644 --- a/scripts/browser-packed.mjs +++ b/scripts/browser-packed.mjs @@ -240,8 +240,14 @@ const peak: TrackMeter | undefined = update.tracks.get("track-000"); declare const failure: EngineWebAdapterError; const remedy: string = failure.remedy; const transient: boolean = failure.transient; +const storageFailure = new EngineWebAdapterError("storage.io", "packed storage fixture", { + operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io", +}); +const storageCode: EngineWebAdapterError["code"] = storageFailure.code; +const storagePhase: EngineWebAdapterError["phase"] = storageFailure.phase; void [openEngineWebSession, openSparseEngineWebSession, createFlacStemResolver, PcmPumpWorkerClient, ADAPTER_ASSETS, packageJson, - minimal, sparseMinimal, live, meters, telemetry, sdk, directMeters, directTelemetry, directResponse, directSpectrum, peak, remedy, transient]; + minimal, sparseMinimal, live, meters, telemetry, sdk, directMeters, directTelemetry, directResponse, directSpectrum, peak, remedy, transient, + storageFailure, storageCode, storagePhase]; `); await writeFile(join(consumer, "tsconfig.json"), JSON.stringify({ compilerOptions: { strict: true, noEmit: true, target: "ES2022", module: "NodeNext", moduleResolution: "NodeNext", diff --git a/scripts/check-package.mjs b/scripts/check-package.mjs index f4df6bb..7ccea02 100644 --- a/scripts/check-package.mjs +++ b/scripts/check-package.mjs @@ -24,6 +24,13 @@ assert.equal( ); const { ADAPTER_PROVENANCE } = await import("../dist/provenance.js"); assert.equal(ADAPTER_PROVENANCE.engine.package, `@misofm/engine@${packageJson.dependencies["@misofm/engine"]}`, "built provenance must identify the installed Engine dependency"); +const { EngineWebAdapterError } = await import("../dist/index.js"); +const packedSurfaceFailure = new EngineWebAdapterError("storage.io", "packed storage fixture", { + operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io", +}, new Error("UnknownError fixture")); +assert.equal(packedSurfaceFailure.code, "storage.io"); +assert.equal(packedSurfaceFailure.phase, "store"); +assert.equal(packedSurfaceFailure.details.operation, "open"); const cache = await mkdtemp(join(tmpdir(), "engine-web-adapter-npm-cache-")); const packed = spawnSync("npm", ["pack", "--dry-run", "--json"], { encoding: "utf8", env: { ...process.env, npm_config_cache: cache }, diff --git a/src/errors.ts b/src/errors.ts index 76a562e..9a888b0 100644 --- a/src/errors.ts +++ b/src/errors.ts @@ -16,6 +16,7 @@ export type EngineWebAdapterErrorCode = | "session.input_path" | "session.open" | "session.playback" + | "storage.io" | "stem.cancelled" | "stem.corrupt" | "stem.decode.asset" @@ -52,6 +53,44 @@ export type EngineWebAdapterErrorPhase = | "console" | "lifecycle"; +/** + * Additive context for an operation which crossed a browser storage boundary. + * `phase` is deliberately nested in `details`; the public `phase` property + * remains the coarse adapter phase used by existing consumers. + */ +export interface EngineWebAdapterErrorContext { + readonly operation: string; + readonly resource: string; + readonly phase: string; +} + +/** + * Evidence for a verified mismatch. A corruption code without this shape is + * not a valid integrity result. + */ +export interface EngineWebAdapterVerification { + readonly kind: "digest" | "length"; + readonly expected: string | number; + readonly observed: string | number; + readonly algorithm?: string; +} + +/** + * Bounded, structured form used when an Error crosses a Worker boundary. + * `causes` preserves all members of an AggregateError instead of selecting + * one reason. It is intentionally data-only so callers can log it safely. + */ +export interface EngineWebAdapterErrorCause { + readonly name: string; + readonly message: string; + readonly role?: "interruption" | "body" | "finalizer"; + readonly code?: string; + readonly details?: Readonly>; + readonly cause?: EngineWebAdapterErrorCause; + readonly causes?: readonly EngineWebAdapterErrorCause[]; + readonly truncated?: true; +} + interface CodeRow { readonly phase: EngineWebAdapterErrorPhase; /** What the caller can actually do about it. Never empty. */ @@ -74,6 +113,11 @@ const CODES: Record = { remedy: "Open a new session after the playback worker fails; inspect the retained cause.", transient: false, }, + "storage.io": { + phase: "store", + remedy: "Inspect details.operation, details.resource, details.phase and the retained cause; retry only after the caller has resolved the named storage condition.", + transient: false, + }, "session.busy": { phase: "lifecycle", remedy: "Await seekFrames() before calling play() from a user gesture.", @@ -251,6 +295,735 @@ const CODES: Record = { }, }; +const MAX_SERIALIZED_CAUSE_DEPTH = 4; +const MAX_SERIALIZED_CAUSES = 8; +const MAX_SERIALIZED_STRING = 512; +const MAX_SERIALIZED_DETAIL_KEYS = 24; +const MAX_SERIALIZED_PAYLOAD = 16 * 1024; +const SERIALIZED_TRUNCATION_RESERVE = 64; +const SERIALIZED_CAUSE_SUMMARY_RESERVE = 512; +const SERIALIZED_TRUNCATION_MARKER = "[truncated]"; + +/** Runtime guard used by Worker clients before reviving a public code. */ +export function isEngineWebAdapterErrorCode(value: unknown): value is EngineWebAdapterErrorCode { + return typeof value === "string" && Object.hasOwn(CODES, value); +} + +/** + * Convert an arbitrary Error-like value to bounded plain data. This preserves + * browser exception names and nested causes without making classification + * decisions from their message text. + */ +export function serializeEngineWebAdapterCause(value: unknown): EngineWebAdapterErrorCause { + const budget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const planned = planCause(value, 0, new Set()); + const admitted = admitCauseTree(planned, budget); + const serialized = serializeCause(admitted, new Set(), budget); + const marked = budget.truncated ? { ...serialized, truncated: true as const } : serialized; + return jsonLength(marked) <= MAX_SERIALIZED_PAYLOAD ? marked : compactCause(marked); +} + +/** Revive the data-only representation used by the Worker wire protocols. */ +export function reviveEngineWebAdapterCause(value: EngineWebAdapterErrorCause): unknown { + return reviveCause(value, 0); +} + +function reviveCause(value: EngineWebAdapterErrorCause, depth: number): Error { + if (depth >= MAX_SERIALIZED_CAUSE_DEPTH) { + const bounded = new Error("Nested cause depth exceeded the adapter bound"); + bounded.name = "CauseDepthLimit"; + return bounded; + } + const nestedCause = value.cause === undefined ? undefined : reviveCause(value.cause, depth + 1); + const nestedCauses = Array.isArray(value.causes) + ? value.causes.slice(0, MAX_SERIALIZED_CAUSES).map((cause) => reviveCause(cause, depth + 1)) + : []; + const revived = value.causes === undefined + ? new Error(value.message, nestedCause === undefined ? undefined : { cause: nestedCause }) + : new AggregateError(nestedCauses, value.message, nestedCause === undefined ? undefined : { cause: nestedCause }); + revived.name = value.name; + if (value.role !== undefined) Object.defineProperty(revived, "role", { value: value.role, enumerable: true, configurable: true }); + if (value.truncated === true) Object.defineProperty(revived, "truncated", { value: true, enumerable: true, configurable: true }); + if (value.code !== undefined && !(value.code === "stem.corrupt" && !serializedVerification(value.details?.verification))) { + Object.defineProperty(revived, "code", { value: value.code, enumerable: true, configurable: true }); + } + if (value.details !== undefined) Object.defineProperty(revived, "details", { value: value.details, enumerable: true, configurable: true }); + return revived; +} + +interface SerializationBudget { used: number; protected: number; truncated: boolean } +interface MutableSerializedCause { + [key: string]: unknown; + name: string; + message: string; + role?: "interruption" | "body" | "finalizer"; + code?: string; + details?: Readonly>; + cause?: EngineWebAdapterErrorCause; + causes?: readonly EngineWebAdapterErrorCause[]; + truncated?: true; +} +interface ContainerState { count: number } +interface PropertyRead { found: boolean; value?: unknown; threw?: boolean } +interface CauseSummary { readonly name: string; readonly message: string } +interface CausePlan { + readonly value: unknown; + readonly depth: number; + readonly summary: CauseSummary; + readonly synthetic: boolean; + readonly summaryTruncated: boolean; + readonly aggregatePresent: boolean; + readonly aggregateTruncated: boolean; + readonly aggregateCandidates: readonly CausePlan[]; + readonly causePresent: boolean; + readonly causeCandidate?: CausePlan; + readonly inputTruncated: boolean; + causesIncluded: boolean; + admittedCauses: readonly CausePlan[]; + causeIncluded: boolean; + admittedCause?: CausePlan; +} + +function planCause(value: unknown, depth: number, seen: Set): CausePlan { + if (depth >= MAX_SERIALIZED_CAUSE_DEPTH) { + return syntheticCausePlan(depth, "CauseDepthLimit", "Nested cause depth exceeded the adapter bound"); + } + if (isObjectLike(value)) { + if (seen.has(value)) return syntheticCausePlan(depth, "CircularCause", "Circular cause was omitted at the adapter boundary"); + seen.add(value); + } + const nameRead = readProperty(value, "name"); + const messageRead = readProperty(value, "message"); + const name = typeof nameRead.value === "string" ? nameRead.value : causeName(value); + const message = typeof messageRead.value === "string" ? messageRead.value : safeString(value); + const summaryBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const summary = boundedSummary(name, message, SERIALIZED_CAUSE_SUMMARY_RESERVE, summaryBudget); + const aggregate = aggregateValues(value); + const aggregateCandidates = aggregate.values === undefined + ? [] + : aggregate.values.map((child) => planCause(child, depth + 1, seen)); + const nested = readCause(value); + const causeCandidate = nested.found + ? planCause(nested.threw + ? { name: "CauseAccessError", message: "Cause getter threw while crossing the adapter boundary" } + : nested.value, depth + 1, seen) + : undefined; + return { + value, + depth, + summary, + synthetic: false, + summaryTruncated: summaryBudget.truncated, + aggregatePresent: aggregate.values !== undefined, + aggregateTruncated: aggregate.threw === true || aggregate.truncated === true, + aggregateCandidates, + causePresent: nested.found, + ...(causeCandidate === undefined ? {} : { causeCandidate }), + inputTruncated: readProperty(value, "truncated").value === true, + causesIncluded: false, + admittedCauses: [], + causeIncluded: false, + }; +} + +function syntheticCausePlan(depth: number, name: string, message: string): CausePlan { + return { + value: undefined, + depth, + summary: { name, message }, + synthetic: true, + summaryTruncated: false, + aggregatePresent: false, + aggregateTruncated: false, + aggregateCandidates: [], + causePresent: false, + inputTruncated: true, + causesIncluded: false, + admittedCauses: [], + causeIncluded: false, + }; +} + +function admitCauseTree(root: CausePlan, budget: SerializationBudget): CausePlan { + if (!reserve(budget, summaryLength(root.summary.name, root.summary.message))) { + budget.truncated = true; + return root; + } + const queue: CausePlan[] = [root]; + for (let index = 0; index < queue.length; index += 1) { + const plan = queue[index]!; + if (plan.summaryTruncated || plan.aggregateTruncated || plan.inputTruncated || plan.synthetic) budget.truncated = true; + + if (plan.aggregatePresent) { + const admitted: CausePlan[] = []; + for (const candidate of plan.aggregateCandidates) { + const amount = (admitted.length === 0 + ? fieldPrefixLength("causes") + jsonLength([]) + : 1) + summaryLength(candidate.summary.name, candidate.summary.message); + if (!reserve(budget, amount)) { + budget.truncated = true; + break; + } + admitted.push(candidate); + queue.push(candidate); + } + if (admitted.length > 0) { + plan.causesIncluded = true; + plan.admittedCauses = admitted; + } else if (plan.aggregateCandidates.length === 0) { + if (reserve(budget, fieldPrefixLength("causes") + jsonLength([]))) { + plan.causesIncluded = true; + plan.admittedCauses = []; + } + } else { + budget.truncated = true; + } + if (plan.aggregateCandidates.length > admitted.length) budget.truncated = true; + } + + if (plan.causePresent && plan.causeCandidate !== undefined) { + const amount = fieldPrefixLength("cause") + summaryLength( + plan.causeCandidate.summary.name, + plan.causeCandidate.summary.message, + ); + if (reserve(budget, amount)) { + plan.causeIncluded = true; + plan.admittedCause = plan.causeCandidate; + queue.push(plan.causeCandidate); + } else { + budget.truncated = true; + } + } + } + return root; +} + +function serializeCause( + plan: CausePlan, + seen: Set, + budget: SerializationBudget, +): EngineWebAdapterErrorCause { + const output = {} as MutableSerializedCause; + const state: ContainerState = { count: 0 }; + const summaryBudget: SerializationBudget = { used: 2, protected: 0, truncated: false }; + if (!addSummary(output, state, plan.summary, summaryBudget)) { + budget.truncated = true; + return { name: "CausePayloadLimit", message: "Cause payload exceeded the adapter bound" }; + } + if (plan.summaryTruncated || plan.aggregateTruncated || plan.inputTruncated || plan.synthetic) budget.truncated = true; + if (plan.synthetic) return output; + + if (isObjectLike(plan.value)) seen.add(plan.value); + const value = plan.value; + const roleRead = readProperty(value, "role"); + if (roleRead.value === "interruption" || roleRead.value === "body" || roleRead.value === "finalizer") { + if (addValue(output, state, "role", roleRead.value, budget)) output.role = roleRead.value; + } + + const detailsRead = readProperty(value, "details"); + const codeRead = readProperty(value, "code"); + const rawCode = typeof codeRead.value === "string" ? codeRead.value : undefined; + const boundedCode = rawCode === undefined ? undefined : boundedText(rawCode, budget); + let serializedDetails: Readonly> | undefined; + let detailsIncluded = false; + let serializedDetailsSeen: Set | undefined; + let serializedDetailsBudget: SerializationBudget | undefined; + if (detailsRead.threw) { + budget.truncated = true; + serializedDetails = { serialization: "details-unreadable" }; + } else if (isPlainRecord(detailsRead.value)) { + const detailsBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const detailsSeen = new Set(seen); + serializedDetailsSeen = detailsSeen; + serializedDetailsBudget = detailsBudget; + serializedDetails = serializeDetails(detailsRead.value, plan.depth + 1, detailsSeen, detailsBudget, + rawCode === "stem.corrupt" && !serializedVerification(readProperty(detailsRead.value, "verification").value) + ? "integrity-evidence-insufficient" : undefined); + } + if (serializedDetails !== undefined) { + if (addValue(output, state, "details", serializedDetails, budget)) { + output.details = serializedDetails; + detailsIncluded = true; + if (serializedDetailsSeen !== undefined) for (const entry of serializedDetailsSeen) seen.add(entry); + if (serializedDetailsBudget?.truncated === true) budget.truncated = true; + } + } + const verified = detailsIncluded ? serializedVerification(output.details?.verification) : undefined; + if (rawCode !== undefined && (rawCode !== "stem.corrupt" || verified !== undefined)) { + addValue(output, state, "code", boundedCode ?? rawCode, budget); + } else if (rawCode === "stem.corrupt") { + budget.truncated = true; + if (output.details === undefined) { + addValue(output, state, "details", { serialization: "integrity-evidence-insufficient" }, budget); + } else if (isPlainRecord(output.details) && output.details.serialization === undefined) { + budget.truncated = true; + } + } + + if (plan.causesIncluded) { + state.count += 1; + output.causes = serializeCauseArray(plan.admittedCauses, seen, budget); + } + if (plan.causeIncluded && plan.admittedCause !== undefined) { + state.count += 1; + output.cause = serializeCause(plan.admittedCause, seen, budget); + } + return output; +} + +function readCause(value: unknown): PropertyRead { + return readProperty(value, "cause"); +} + +function aggregateValues(value: unknown): { readonly values?: readonly unknown[]; readonly threw?: boolean; readonly truncated?: boolean } { + const errors = readProperty(value, "errors"); + const causes = readProperty(value, "causes"); + if (errors.threw || causes.threw) return { threw: true }; + const arrays: unknown[][] = []; + let truncated = false; + if (isAggregate(value) || isArray(errors.value)) { + if (isArray(errors.value)) { + arrays.push([...boundedArrayValues(errors.value)]); + truncated ||= safeArrayLength(errors.value) > MAX_SERIALIZED_CAUSES; + } + } + if (isArray(causes.value)) { + arrays.push([...boundedArrayValues(causes.value)]); + truncated ||= safeArrayLength(causes.value) > MAX_SERIALIZED_CAUSES; + } + if (arrays.length === 0) return {}; + const combined = arrays.flat(); + const values = combined.slice(0, MAX_SERIALIZED_CAUSES); + return { values, ...(truncated || combined.length > MAX_SERIALIZED_CAUSES ? { truncated: true } : {}) }; +} + +function serializeCauseArray(plans: readonly CausePlan[], seen: Set, budget: SerializationBudget): readonly EngineWebAdapterErrorCause[] { + return plans.map((plan) => serializeCause(plan, seen, budget)); +} + +function boundedSummary(name: string, message: string, limit: number, budget: SerializationBudget): CauseSummary { + const boundedName = boundedText(name, budget); + const boundedMessage = boundedText(message, budget); + if (summaryLength(boundedName, boundedMessage) <= limit) { + return { name: boundedName, message: boundedMessage }; + } + + const emptyValueLength = jsonLength(""); + const availableValueBytes = Math.max(0, limit - summaryLength("", "")); + const nameBytes = Math.max(0, jsonLength(boundedName) - emptyValueLength); + const messageBytes = Math.max(0, jsonLength(boundedMessage) - emptyValueLength); + let nameBudget = Math.min(nameBytes, Math.floor(availableValueBytes / 2)); + let messageBudget = Math.min(messageBytes, availableValueBytes - nameBudget); + let remaining = availableValueBytes - nameBudget - messageBudget; + if (remaining > 0 && nameBudget < nameBytes) { + const extra = Math.min(remaining, nameBytes - nameBudget); + nameBudget += extra; + remaining -= extra; + } + if (remaining > 0 && messageBudget < messageBytes) messageBudget += Math.min(remaining, messageBytes - messageBudget); + + const fittedName = fitSummaryText(boundedName, emptyValueLength + nameBudget); + const fittedMessage = fitSummaryText(boundedMessage, emptyValueLength + messageBudget); + if (fittedName !== boundedName || fittedMessage !== boundedMessage) budget.truncated = true; + return { name: fittedName, message: fittedMessage }; +} + +function summaryLength(name: string, message: string): number { + return 2 + + jsonLength("name") + 1 + jsonLength(name) + 1 + + jsonLength("message") + 1 + jsonLength(message); +} + +function fitSummaryText(value: string, maxJsonBytes: number): string { + if (jsonLength(value) <= maxJsonBytes) return value; + if (jsonLength(SERIALIZED_TRUNCATION_MARKER) > maxJsonBytes) return ""; + const characters = Array.from(value); + let low = 0; + let high = characters.length; + while (low < high) { + const middle = Math.ceil((low + high) / 2); + const candidate = `${characters.slice(0, middle).join("")}${SERIALIZED_TRUNCATION_MARKER}`; + if (jsonLength(candidate) <= maxJsonBytes) low = middle; + else high = middle - 1; + } + return `${characters.slice(0, low).join("")}${SERIALIZED_TRUNCATION_MARKER}`; +} + +function serializeDetails( + value: Record, + depth: number, + seen: Set, + budget: SerializationBudget, + marker?: string, +): Readonly> { + const result: Record = {}; + if (seen.has(value)) { + budget.truncated = true; + return { serialization: "circular detail" }; + } + seen.add(value); + const state = beginContainer(budget); + if (state === undefined) return result; + if (depth >= MAX_SERIALIZED_CAUSE_DEPTH) { + budget.truncated = true; + const verification = readProperty(value, "verification"); + const proof = verification.threw ? undefined : serializedVerification(verification.value); + if (proof !== undefined) addValue(result, state, "verification", proof, budget); + else addValue(result, state, "serialization", "detail depth limit", budget); + return result; + } + if (marker !== undefined) addValue(result, state, "serialization", marker, budget); + const verification = readProperty(value, "verification"); + if (verification.threw) { + budget.truncated = true; + addValue(result, state, "verification", "Unable to serialize verification evidence", budget); + } else { + const proof = serializedVerification(verification.value); + if (proof !== undefined) addValue(result, state, "verification", proof, budget); + else if (verification.found) { + budget.truncated = true; + addValue(result, state, "serialization", "integrity-evidence-insufficient", budget); + } + } + try { + const keys = enumerableKeys(value); + for (const key of keys.values) { + if (key === "verification") continue; + const property = readProperty(value, key); + const outputKey = boundedText(key, budget); + if (property.threw) { + addValue(result, state, outputKey, "Unable to serialize detail", budget); + continue; + } + addBoundedValue(result, state, outputKey, property.value, depth, seen, budget); + } + if (keys.truncated) budget.truncated = true; + } catch { + budget.truncated = true; + } + return result; +} + +function addBoundedValue( + target: Record, + state: ContainerState, + key: string, + value: unknown, + depth: number, + seen: Set, + budget: SerializationBudget, +): void { + if (depth >= MAX_SERIALIZED_CAUSE_DEPTH) { + budget.truncated = true; + addValue(target, state, key, "[detail depth limit]", budget); + return; + } + if (isArray(value)) { + const childBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const childSeen = new Set(seen); + const bounded = boundedArray(value, depth + 1, childSeen, childBudget); + if (!reserve(budget, jsonLength(key) + 1 + (state.count === 0 ? 0 : 1) + jsonLength(bounded))) return; + state.count += 1; + target[key] = bounded; + if (childBudget.truncated) budget.truncated = true; + for (const entry of childSeen) seen.add(entry); + return; + } + if (isPlainRecord(value)) { + const childBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const childSeen = new Set(seen); + const serialized = serializeDetails(value, depth + 1, childSeen, childBudget); + if (!reserve(budget, jsonLength(key) + 1 + (state.count === 0 ? 0 : 1) + jsonLength(serialized))) return; + state.count += 1; + target[key] = serialized; + if (childBudget.truncated) budget.truncated = true; + for (const entry of childSeen) seen.add(entry); + return; + } + addValue(target, state, key, boundedScalar(value, budget), budget); +} + +function boundedArray(value: unknown[], depth: number, seen: Set, budget: SerializationBudget): readonly unknown[] { + if (seen.has(value)) { + budget.truncated = true; + return ["[circular detail]"]; + } + seen.add(value); + const state = beginArray(budget); + if (state === undefined) return []; + if (depth >= MAX_SERIALIZED_CAUSE_DEPTH) { + budget.truncated = true; + return ["[detail depth limit]"]; + } + const output: unknown[] = []; + const length = safeArrayLength(value); + for (let index = 0; index < Math.min(length, MAX_SERIALIZED_CAUSES); index += 1) { + let item: unknown; + try { item = value[index]; } + catch { item = "Unable to serialize detail"; budget.truncated = true; } + if (isArray(item)) { + const childBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const childSeen = new Set(seen); + const bounded = boundedArray(item, depth + 1, childSeen, childBudget); + if (!reserve(budget, jsonLength(bounded) + (state.count === 0 ? 0 : 1))) break; + state.count += 1; + output.push(bounded); + if (childBudget.truncated) budget.truncated = true; + for (const entry of childSeen) seen.add(entry); + } else if (isPlainRecord(item)) { + const childBudget: SerializationBudget = { used: 0, protected: 0, truncated: false }; + const childSeen = new Set(seen); + const serialized = serializeDetails(item, depth + 1, childSeen, childBudget); + if (!reserve(budget, jsonLength(serialized) + (state.count === 0 ? 0 : 1))) break; + state.count += 1; + output.push(serialized); + if (childBudget.truncated) budget.truncated = true; + for (const entry of childSeen) seen.add(entry); + } else { + const scalar = boundedScalar(item, budget); + if (!reserveArrayValue(state, scalar, budget)) break; + output.push(scalar); + } + } + if (length > MAX_SERIALIZED_CAUSES) budget.truncated = true; + return output; +} + +function boundedScalar(value: unknown, budget: SerializationBudget): unknown { + if (value === null || typeof value === "number" || typeof value === "boolean") return value; + if (typeof value === "string") return boundedText(value, budget); + if (value === undefined) return "undefined"; + if (typeof value === "bigint") return boundedText(`${value}n`, budget); + return boundedText(safeString(value), budget); +} + +function serializedVerification(value: unknown): EngineWebAdapterVerification | undefined { + if (!isEngineWebAdapterVerification(value)) return undefined; + const kind = readProperty(value, "kind").value as EngineWebAdapterVerification["kind"]; + const expected = readProperty(value, "expected").value as string | number; + const observed = readProperty(value, "observed").value as string | number; + const algorithm = readProperty(value, "algorithm").value as string | undefined; + return { + kind, + expected, + observed, + ...(algorithm === undefined ? {} : { algorithm }), + }; +} + +/** + * Integrity evidence is meaningful only when it could have come from one of + * the adapter's canonical verification boundaries. This is also the guard + * used when reviving evidence from a Worker or a storage exception. + */ +export function isEngineWebAdapterVerification(value: unknown): value is EngineWebAdapterVerification { + if (!isObjectLike(value)) return false; + const kind = readProperty(value, "kind"); + const expected = readProperty(value, "expected"); + const observed = readProperty(value, "observed"); + const algorithm = readProperty(value, "algorithm"); + const truncated = readProperty(value, "truncated"); + if (kind.threw || expected.threw || observed.threw || algorithm.threw || truncated.threw || truncated.value === true) return false; + if (kind.value === "length") { + return algorithm.value === undefined && + typeof expected.value === "number" && Number.isSafeInteger(expected.value) && expected.value >= 0 && + typeof observed.value === "number" && Number.isSafeInteger(observed.value) && observed.value >= 0 && + expected.value !== observed.value; + } + if (kind.value !== "digest" || + typeof expected.value !== "string" || !/^[0-9a-f]{64}$/u.test(expected.value) || + typeof observed.value !== "string" || !/^[0-9a-f]{64}$/u.test(observed.value) || + expected.value === observed.value) return false; + return algorithm.value === undefined || algorithm.value === "blake3-256" || algorithm.value === "sha256"; +} + +function isPlainRecord(value: unknown): value is Record { + if (!isObjectLike(value) || isArray(value)) return false; + try { + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; + } catch { + return false; + } +} + +function isObjectLike(value: unknown): value is object { + return (typeof value === "object" && value !== null) || typeof value === "function"; +} + +function isArray(value: unknown): value is unknown[] { + try { return Array.isArray(value); } + catch { return false; } +} + +function isAggregate(value: unknown): boolean { + try { return value instanceof AggregateError; } + catch { return false; } +} + +function readProperty(value: unknown, property: string): PropertyRead { + if (!isObjectLike(value)) return { found: false }; + try { + if (!(property in value)) return { found: false }; + return { found: true, value: (value as Record)[property] }; + } catch { + return { found: true, threw: true }; + } +} + +function enumerableKeys(value: Record): { readonly values: readonly string[]; readonly truncated: boolean } { + const keys: string[] = []; + let truncated = false; + for (const key in value) { + try { + if (!Object.prototype.propertyIsEnumerable.call(value, key)) continue; + if (keys.length >= MAX_SERIALIZED_DETAIL_KEYS) { + truncated = true; + break; + } + keys.push(key); + } catch { + truncated = true; + break; + } + } + return { values: keys, truncated }; +} + +function boundedArrayValues(value: unknown[]): readonly unknown[] { + const length = safeArrayLength(value); + const output: unknown[] = []; + for (let index = 0; index < Math.min(length, MAX_SERIALIZED_CAUSES); index += 1) { + try { output.push(value[index]); } + catch { output.push({ name: "CauseAccessError", message: "Aggregate member could not be read" }); } + } + return output; +} + +function safeArrayLength(value: unknown[]): number { + try { return Number.isSafeInteger(value.length) && value.length >= 0 ? value.length : 0; } + catch { return 0; } +} + +function beginContainer(budget: SerializationBudget): ContainerState | undefined { + return reserve(budget, 2) ? { count: 0 } : undefined; +} + +function beginArray(budget: SerializationBudget): ContainerState | undefined { + return beginContainer(budget); +} + +function fieldPrefixLength(key: string): number { + return jsonLength(key) + 2; +} + +function addSummary(target: MutableSerializedCause, state: ContainerState, summary: CauseSummary, budget: SerializationBudget): boolean { + const amount = jsonLength("name") + 1 + jsonLength(summary.name) + + 1 + + jsonLength("message") + 1 + jsonLength(summary.message); + if (!reserve(budget, amount)) return false; + state.count += 2; + target.name = summary.name; + target.message = summary.message; + return true; +} + +function addValue(target: Record, state: ContainerState, key: string, value: unknown, budget: SerializationBudget): boolean { + const amount = jsonLength(key) + 1 + (state.count === 0 ? 0 : 1) + jsonLength(value); + if (!reserve(budget, amount)) return false; + state.count += 1; + target[key] = value; + return true; +} + +function reserveArrayValue(state: ContainerState, value: unknown, budget: SerializationBudget): boolean { + const amount = jsonLength(value) + (state.count === 0 ? 0 : 1); + if (!reserve(budget, amount)) return false; + state.count += 1; + return true; +} + +function reserve(budget: SerializationBudget, amount: number): boolean { + if (!Number.isSafeInteger(amount) || amount < 0 || amount > MAX_SERIALIZED_PAYLOAD - SERIALIZED_TRUNCATION_RESERVE - budget.protected - budget.used) { + budget.truncated = true; + return false; + } + budget.used += amount; + return true; +} + +function jsonLength(value: unknown): number { + try { + const encoded = JSON.stringify(value); + return encoded === undefined ? 0 : new TextEncoder().encode(encoded).byteLength; + } catch { + return MAX_SERIALIZED_PAYLOAD; + } +} + +function compactCause(value: EngineWebAdapterErrorCause): EngineWebAdapterErrorCause { + const output: MutableSerializedCause = { + name: trimString(value.name), + message: trimString(value.message), + truncated: true, + }; + if (value.role !== undefined) output.role = value.role; + const verification = serializedVerification(value.details?.verification); + if (value.code !== undefined && (value.code !== "stem.corrupt" || verification !== undefined)) output.code = trimString(value.code); + if (verification !== undefined) output.details = { verification }; + else if (value.code === "stem.corrupt") output.details = { serialization: "integrity-evidence-insufficient" }; + if (value.causes !== undefined) output.causes = value.causes.slice(0, MAX_SERIALIZED_CAUSES).map(compactCause); + if (value.cause !== undefined) output.cause = compactCause(value.cause); + if (jsonLength(output) > MAX_SERIALIZED_PAYLOAD) { + delete output.cause; + if (output.causes !== undefined) { + output.causes = output.causes.map((cause) => ({ + name: compactSummary(cause.name), + message: compactSummary(cause.message), + ...(cause.role === undefined ? {} : { role: cause.role }), + truncated: true as const, + })); + } + } + while (jsonLength(output) > MAX_SERIALIZED_PAYLOAD && output.causes !== undefined && output.causes.length > 0) { + output.causes = output.causes.slice(0, -1); + } + if (jsonLength(output) > MAX_SERIALIZED_PAYLOAD) { + delete output.causes; + delete output.cause; + delete output.details; + delete output.code; + output.name = compactSummary(output.name); + output.message = compactSummary(output.message); + } + return output; +} + +function compactSummary(value: string): string { + const limit = 128; + return value.length > limit + ? `${value.slice(0, limit - SERIALIZED_TRUNCATION_MARKER.length)}${SERIALIZED_TRUNCATION_MARKER}` + : value; +} + +function causeName(value: unknown): string { + if (value === null) return "null"; + if (isObjectLike(value)) return "Object"; + return typeof value; +} + +function boundedText(value: string, budget: SerializationBudget): string { + const bounded = trimString(value); + if (bounded !== value) budget.truncated = true; + return bounded; +} +function trimString(value: string): string { + return value.length > MAX_SERIALIZED_STRING + ? `${value.slice(0, MAX_SERIALIZED_STRING - SERIALIZED_TRUNCATION_MARKER.length)}${SERIALIZED_TRUNCATION_MARKER}` + : value; +} +function safeString(value: unknown): string { try { return String(value); } catch { return "[unprintable cause]"; } } + /** * The single adapter-boundary failure. * diff --git a/src/index.ts b/src/index.ts index 8cd4232..df50828 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,7 +1,13 @@ export { ADAPTER_PROVENANCE } from "./provenance.js"; export { EngineWebAdapterError } from "./errors.js"; export { openEngineWebSession, openSparseEngineWebSession } from "./session.js"; -export type { EngineWebAdapterErrorCode, EngineWebAdapterErrorPhase } from "./errors.js"; +export type { + EngineWebAdapterErrorCause, + EngineWebAdapterErrorCode, + EngineWebAdapterErrorContext, + EngineWebAdapterErrorPhase, + EngineWebAdapterVerification, +} from "./errors.js"; export type { EngineAudioContext, EnginePump, diff --git a/src/internal/engine-web-flac-worker.ts b/src/internal/engine-web-flac-worker.ts index bf34493..22f112a 100644 --- a/src/internal/engine-web-flac-worker.ts +++ b/src/internal/engine-web-flac-worker.ts @@ -8,7 +8,7 @@ import { } from "@misofm/codec"; import { createIncrementalBlake3, type IncrementalBlake3 } from "../stems/blake3.js"; -import { EngineWebAdapterError } from "../errors.js"; +import { EngineWebAdapterError, serializeEngineWebAdapterCause } from "../errors.js"; import { FlacInputSlotConsumer, FLAC_INPUT_SLOT_BYTES, type FlacInputReadResult, type FlacInputSlotBuffers } from "../stems/flac-input-slot.js"; import { FlacOutputCredits } from "../stems/flac-output-credits.js"; import { loadFlacDecoderModule, mapFlacDecodeError } from "../stems/native-flac-decoder.js"; @@ -67,16 +67,17 @@ function clearJobState(): void { } function serialize(error: unknown): Extract['error'] { - if (error instanceof Error) { - const record = error as Error & { readonly code?: unknown; readonly details?: unknown }; - return { - name: error.name, - message: error.message, - ...(typeof record.code === "string" ? { code: record.code } : {}), - ...(typeof record.details === "object" && record.details !== null ? { details: record.details as Readonly> } : {}), - }; - } - return { name: "Error", message: String(error) }; + const serialized = serializeEngineWebAdapterCause(error); + return { + name: serialized.name, + message: serialized.message, + ...(serialized.role === undefined ? {} : { role: serialized.role }), + ...(serialized.truncated === undefined ? {} : { truncated: serialized.truncated }), + ...(serialized.code === undefined ? {} : { code: serialized.code }), + ...(serialized.details === undefined ? {} : { details: serialized.details }), + ...(serialized.cause === undefined ? {} : { cause: serialized.cause }), + ...(serialized.causes === undefined ? {} : { causes: serialized.causes }), + }; } function fail(error: unknown): void { diff --git a/src/internal/engine-web-opfs-worker.ts b/src/internal/engine-web-opfs-worker.ts index ddae7c0..48a9f40 100644 --- a/src/internal/engine-web-opfs-worker.ts +++ b/src/internal/engine-web-opfs-worker.ts @@ -1,4 +1,4 @@ -import { EngineWebAdapterError } from "../errors.js"; +import { EngineWebAdapterError, serializeEngineWebAdapterCause } from "../errors.js"; import { OPFS_WRITE_METHOD, OPFS_WRITE_REMEDY } from "../stems/opfs-worker-protocol.js"; import type { OpfsWorkerRequest, OpfsWorkerResponse } from "../stems/opfs-worker-protocol.js"; @@ -70,7 +70,11 @@ async function apply(request: OpfsWorkerRequest): Promise { try { handle.truncate(0); } catch (error) { - try { handle.close(); } catch { /* the truncate failure is the authoritative cause */ } + let closeFailed = false; + let closeError: unknown; + try { handle.close(); } + catch (cause) { closeFailed = true; closeError = cause; } + if (closeFailed) throw compoundPhysicalFailure("OPFS writer open failed during cleanup", error, closeError); throw error; } handles.set(request.writerId, { handle, offset: 0 }); @@ -80,9 +84,10 @@ async function apply(request: OpfsWorkerRequest): Promise { const entry = opened(request.writerId); const written = entry.handle.write(request.chunk, { at: entry.offset }); if (written !== request.chunk.byteLength) { - throw new EngineWebAdapterError("stem.corrupt", "Origin-private file storage wrote a short chunk", { + throw new EngineWebAdapterError("storage.io", "Origin-private file storage wrote a short chunk", { expectedBytes: request.chunk.byteLength, writtenBytes: written, + failureKind: "io", }); } entry.offset += written; @@ -91,7 +96,17 @@ async function apply(request: OpfsWorkerRequest): Promise { case "write-close": { const entry = opened(request.writerId); handles.delete(request.writerId); - try { entry.handle.flush(); } finally { entry.handle.close(); } + let flushFailed = false; + let flushError: unknown; + try { entry.handle.flush(); } + catch (cause) { flushFailed = true; flushError = cause; } + let closeFailed = false; + let closeError: unknown; + try { entry.handle.close(); } + catch (cause) { closeFailed = true; closeError = cause; } + if (flushFailed && closeFailed) throw compoundPhysicalFailure("OPFS writer close failed during flush and cleanup", flushError, closeError); + if (flushFailed) throw flushError; + if (closeFailed) throw closeError; return; } case "write-abort": { @@ -133,16 +148,29 @@ function opened(writerId: number): { readonly handle: SyncAccessHandleLike; offs } function serialize(error: unknown): Extract["error"] { - if (error instanceof Error) { - const record = error as Error & { readonly code?: unknown; readonly details?: unknown }; - return { - name: error.name, - message: error.message, - ...(typeof record.code === "string" ? { code: record.code } : {}), - ...(typeof record.details === "object" && record.details !== null - ? { details: record.details as Readonly> } - : {}), - }; - } - return { name: "Error", message: String(error) }; + const serialized = serializeEngineWebAdapterCause(error); + return { + name: serialized.name, + message: serialized.message, + ...(serialized.role === undefined ? {} : { role: serialized.role }), + ...(serialized.truncated === undefined ? {} : { truncated: serialized.truncated }), + ...(serialized.code === undefined ? {} : { code: serialized.code }), + ...(serialized.details === undefined ? {} : { details: serialized.details }), + ...(serialized.cause === undefined ? {} : { cause: serialized.cause }), + ...(serialized.causes === undefined ? {} : { causes: serialized.causes }), + }; +} + +function compoundPhysicalFailure(message: string, body: unknown, finalizer: unknown): AggregateError { + return new AggregateError([ + roleFailure("body", body), + roleFailure("finalizer", finalizer), + ], message); +} + +function roleFailure(role: "body" | "finalizer", error: unknown): Error { + const wrapped = new Error(error instanceof Error ? error.message : String(error), { cause: error }); + wrapped.name = error instanceof Error ? error.name : "Error"; + Object.defineProperty(wrapped, "role", { value: role, enumerable: true, configurable: true }); + return wrapped; } diff --git a/src/internal/engine-web-sparse-verify-worker.ts b/src/internal/engine-web-sparse-verify-worker.ts index 0521d90..557699f 100644 --- a/src/internal/engine-web-sparse-verify-worker.ts +++ b/src/internal/engine-web-sparse-verify-worker.ts @@ -1,6 +1,8 @@ import { Cause, Effect, Exit, Fiber } from "effect"; import { createIncrementalBlake3, type IncrementalBlake3 } from "../stems/blake3.js"; +import { serializeEngineWebAdapterCause } from "../errors.js"; +import type { EngineWebAdapterVerification } from "../errors.js"; import { verifiedSparseVerifyArrayBufferTransfer, type SparseVerifyArrayBufferTransfer } from "../stems/sparse-verify-buffer.js"; import { SPARSE_VERIFY_READ_BYTES, @@ -45,11 +47,13 @@ interface Counters { class WorkerVerificationError extends Error { readonly kind: SparseVerifyFailureKind; + readonly verification: EngineWebAdapterVerification | undefined; - constructor(kind: SparseVerifyFailureKind, message: string, cause?: unknown) { + constructor(kind: SparseVerifyFailureKind, message: string, cause?: unknown, verification?: EngineWebAdapterVerification) { super(message, cause === undefined ? undefined : { cause }); this.name = "WorkerVerificationError"; this.kind = kind; + this.verification = verification; } } @@ -94,16 +98,16 @@ function start(value: SparseVerifyStart): void { try { decoded = sparseVerifyWorkerStart(value); } catch (cause) { - postFailure(value, "corrupt", message(cause)); + postFailure(value, "corrupt", message(cause), cause); return; } if (active !== undefined) { - postFailure(value, "corrupt", "Sparse verification worker received overlapping jobs"); + postFailure(value, "corrupt", "Sparse verification worker received overlapping jobs", new Error("Sparse verification worker received overlapping jobs")); return; } const transfer = verifiedSparseVerifyArrayBufferTransfer(); if (transfer === undefined) { - postFailure(value, "boundary", "Sparse verification ArrayBuffer transfer capability is unavailable"); + postFailure(value, "boundary", "Sparse verification ArrayBuffer transfer capability is unavailable", new Error("Sparse verification ArrayBuffer transfer capability is unavailable")); return; } const job: JobState = { @@ -151,7 +155,9 @@ function verify(job: JobState): Effect.Effect request.activeBytes) throw new WorkerVerificationError("corrupt", "Sparse verification payload cursor exceeded its bound"); + if (payloadCursor > request.activeBytes) throw new WorkerVerificationError("corrupt", "Sparse verification payload cursor exceeded its bound", undefined, { + kind: "length", expected: request.activeBytes, observed: payloadCursor, + }); } frameCursor = addCount(startFrame, frames); } @@ -160,11 +166,23 @@ function verify(job: JobState): Effect.Effect hash.digest("hex"), - catch: (cause) => new WorkerVerificationError("corrupt", "Sparse verification digest failed", cause), + catch: (cause) => new WorkerVerificationError("io", "Sparse verification digest failed", cause), }); counters.hashMs += Math.max(0, now() - beforeDigest); - if (payloadCursor !== request.data.size || counters.readBytes !== request.activeBytes || counters.hashedBytes !== request.canonicalBytes) { - throw new WorkerVerificationError("corrupt", "Sparse verification byte counts are inconsistent"); + if (payloadCursor !== request.data.size) { + throw new WorkerVerificationError("corrupt", "Sparse verification payload length is inconsistent", undefined, { + kind: "length", expected: request.data.size, observed: payloadCursor, + }); + } + if (counters.readBytes !== request.activeBytes) { + throw new WorkerVerificationError("corrupt", "Sparse verification active byte count is inconsistent", undefined, { + kind: "length", expected: request.activeBytes, observed: counters.readBytes, + }); + } + if (counters.hashedBytes !== request.canonicalBytes) { + throw new WorkerVerificationError("corrupt", "Sparse verification canonical byte count is inconsistent", undefined, { + kind: "length", expected: request.canonicalBytes, observed: counters.hashedBytes, + }); } return { type: "complete", @@ -306,7 +324,9 @@ function consumeBuffer(job: JobState, buffer: ArrayBuffer, expectedLength: numbe let primary: WorkerVerificationError | undefined; try { readLength = owned.byteLength; - if (readLength !== expectedLength) throw new WorkerVerificationError("corrupt", "Sparse verification payload read was short"); + if (readLength !== expectedLength) throw new WorkerVerificationError("corrupt", "Sparse verification payload read was short", undefined, { + kind: "length", expected: expectedLength, observed: readLength, + }); checkCancelled(job); view = new Uint8Array(owned); hashUpdateSync(job, hash, view, counters); @@ -319,6 +339,12 @@ function consumeBuffer(job: JobState, buffer: ArrayBuffer, expectedLength: numbe try { void job.transfer(owned, 0); } catch (cause) { if (primary === undefined) primary = new WorkerVerificationError("io", "Sparse verification payload disposal failed", cause); + else primary = new WorkerVerificationError( + primary.kind, + primary.message, + new AggregateError([primary, cause], "Sparse verification payload disposal failed"), + primary.verification, + ); } } view = undefined; @@ -415,17 +441,18 @@ function finish(job: JobState, exit: Exit.Exit, kind: SparseVerifyFailureKind, text: string): void { +function postFailure(request: Partial, kind: SparseVerifyFailureKind, text: string, cause?: unknown, verification?: EngineWebAdapterVerification): void { if (typeof request.jobId !== "number" || typeof request.generation !== "number") return; try { scope.postMessage({ @@ -435,12 +462,37 @@ function postFailure(request: Partial, kind: SparseVerifyFail generation: request.generation, kind, message: text, + ...(verification === undefined ? {} : { verification }), + ...(cause === undefined ? {} : { cause: serializeEngineWebAdapterCause(cause) }), }); } catch { scope.close?.(); } } +function effectCause(cause: Cause.Cause): unknown { + const values = cause.reasons.map((reason) => { + if (Cause.isFailReason(reason)) return reason.error; + if (Cause.isDieReason(reason)) return reason.defect; + const interrupted = new Error("Sparse verification was interrupted"); + interrupted.name = "AbortError"; + return interrupted; + }); + return values.length === 1 ? values[0] : new AggregateError(values, "Sparse verification failed"); +} + +function findWorkerVerificationError(value: unknown, depth = 0): WorkerVerificationError | undefined { + if (depth > 4) return undefined; + if (value instanceof WorkerVerificationError) return value; + if (value instanceof AggregateError) { + for (const item of value.errors) { + const found = findWorkerVerificationError(item, depth + 1); + if (found !== undefined) return found; + } + } + return undefined; +} + function clear(job: JobState): void { if (job.cleared) return; job.cleared = true; diff --git a/src/stems/flac-resolver.ts b/src/stems/flac-resolver.ts index 7cf15dc..6959f71 100644 --- a/src/stems/flac-resolver.ts +++ b/src/stems/flac-resolver.ts @@ -1,7 +1,8 @@ import { Cause, Effect, Exit, Queue } from "effect"; import { beginIngestStage, ownRunnableWorker, configureProcessingDiagnostics, recordWorkerProcessing, deliveredRangeOwner, registerFlacResolver, releaseDecoded, retainDecoded, type IngestDiagnostics } from "./ingest-diagnostics.js"; import { MAXIMUM_CANONICAL_OUTPUT_BYTES } from "./native-flac-decoder.js"; -import { EngineWebAdapterError } from "../errors.js"; +import { EngineWebAdapterError, isEngineWebAdapterErrorCode, isEngineWebAdapterVerification, reviveEngineWebAdapterCause } from "../errors.js"; +import type { EngineWebAdapterVerification } from "../errors.js"; import { ADAPTER_ASSETS, type AdapterAssetOverrides } from "../assets.js"; import { BoundedStemAdmission, flacPipelineWidths, type FlacProcessingOptions } from "./flac-admission.js"; import { registerFlacResult } from "./flac-result.js"; @@ -50,10 +51,27 @@ let nextRequestId = 1; function workerError(message: Extract): EngineWebAdapterError { const code = message.error.code; - if (code !== undefined) { - return new EngineWebAdapterError(code as EngineWebAdapterError["code"], message.error.message, message.error.details ?? {}); + const cause = message.error.causes === undefined + ? message.error.cause === undefined ? undefined : reviveEngineWebAdapterCause(message.error.cause) + : reviveEngineWebAdapterCause({ + name: message.error.name, + message: message.error.message, + ...(message.error.role === undefined ? {} : { role: message.error.role }), + ...(message.error.truncated === undefined ? {} : { truncated: message.error.truncated }), + ...(message.error.cause === undefined ? {} : { cause: message.error.cause }), + causes: message.error.causes, + }); + if (code !== undefined && isEngineWebAdapterErrorCode(code) && + (code !== "stem.corrupt" || isEngineWebAdapterVerification(message.error.details?.verification))) { + return new EngineWebAdapterError(code, message.error.message, message.error.details ?? {}, cause); } - return new EngineWebAdapterError("stem.decode.worker", message.error.message); + if (code === "stem.corrupt") { + return new EngineWebAdapterError("storage.io", message.error.message, { + ...(message.error.details ?? {}), + failureKind: "adapter", + }, cause); + } + return new EngineWebAdapterError("stem.decode.worker", message.error.message, {}, cause); } function decoderSourceError(error: unknown): EngineWebAdapterError { @@ -482,9 +500,10 @@ function makeFlacStemResolver(options: FlacResolverOptions, diagnostics?: Ingest preparedMetadata.expectedFrames !== chunk.expected.frames || preparedMetadata.totalPcmBytes !== chunk.expected.canonicalBytes )) { - return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "FLAC chunk preparation does not match its declared shape", { + return yield* Effect.fail(new EngineWebAdapterError("stem.invalid_declaration", "FLAC chunk preparation does not match its declared shape", { identity, expectedFrames: chunk.expected.frames, expectedPcmBytes: chunk.expected.canonicalBytes, + metadata: "flac-chunk", })); } initialized = true; @@ -515,12 +534,40 @@ function makeFlacStemResolver(options: FlacResolverOptions, diagnostics?: Ingest } else { if (!initialized) return yield* new DecoderByteSourceError({ operation: "finish", message: "FLAC Worker completed before readiness" }); const message = command.message; - if (message.pcmBytes !== decodedBytes || message.frames !== decodedFrames || - (options.processing !== undefined && (decodedBytes !== preparedMetadata!.totalPcmBytes || decodedFrames !== preparedMetadata!.expectedFrames)) || - (chunk !== undefined && (decodedBytes !== chunk.expected.canonicalBytes || decodedFrames !== chunk.expected.frames || - hostPcmHash?.digestHex() !== chunk.pcmSha256)) || - (workerHashes && (message.digest !== identity.slice(7) || !/^[a-f0-9]{64}$/u.test(message.digest)))) { - return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "FLAC Worker completion does not verify canonical PCM", { identity })); + let verification: EngineWebAdapterVerification | undefined; + if (message.pcmBytes !== decodedBytes) { + verification = { kind: "length", expected: decodedBytes, observed: message.pcmBytes }; + } else if (message.frames !== decodedFrames) { + const frameBytes = preparedMetadata!.streamInfo.channels * (preparedMetadata!.streamInfo.bitDepth / 8); + verification = { kind: "length", expected: decodedFrames * frameBytes, observed: message.frames * frameBytes }; + } else if (options.processing !== undefined && decodedBytes !== preparedMetadata!.totalPcmBytes) { + verification = { kind: "length", expected: preparedMetadata!.totalPcmBytes, observed: decodedBytes }; + } else if (options.processing !== undefined && decodedFrames !== preparedMetadata!.expectedFrames) { + const frameBytes = preparedMetadata!.streamInfo.channels * (preparedMetadata!.streamInfo.bitDepth / 8); + verification = { kind: "length", expected: preparedMetadata!.expectedFrames * frameBytes, observed: decodedFrames * frameBytes }; + } else if (chunk !== undefined && decodedBytes !== chunk.expected.canonicalBytes) { + verification = { kind: "length", expected: chunk.expected.canonicalBytes, observed: decodedBytes }; + } else if (chunk !== undefined && decodedFrames !== chunk.expected.frames) { + const frameBytes = chunk.expected.channels * (chunk.expected.bitDepth / 8); + verification = { kind: "length", expected: chunk.expected.frames * frameBytes, observed: decodedFrames * frameBytes }; + } else if (chunk !== undefined) { + const observed = hostPcmHash?.digestHex(); + if (observed !== chunk.pcmSha256 && observed !== undefined) { + verification = { kind: "digest", expected: chunk.pcmSha256, observed, algorithm: "sha256" }; + } + } + if (verification === undefined && workerHashes && typeof message.digest === "string" && (message.digest !== identity.slice(7) || !/^[a-f0-9]{64}$/u.test(message.digest))) { + verification = { kind: "digest", expected: identity.slice(7), observed: message.digest, algorithm: "blake3-256" }; + } + if (verification !== undefined) { + return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "FLAC Worker completion does not verify canonical PCM", { + identity, operation: "verify", resource: "flac-worker", phase: "completion", verification, + })); + } + if (workerHashes && (typeof message.digest !== "string" || message.digest !== identity.slice(7))) { + return yield* Effect.fail(new EngineWebAdapterError("storage.io", "FLAC Worker did not return canonical digest evidence", { + identity, operation: "verify", resource: "flac-worker", phase: "completion", failureKind: "adapter", + })); } if (message.metrics !== undefined) recordWorkerProcessing(diagnostics, message.metrics); if (workerHashes) verifiedDigest = message.digest; diff --git a/src/stems/flac-worker-protocol.ts b/src/stems/flac-worker-protocol.ts index e68358a..c437265 100644 --- a/src/stems/flac-worker-protocol.ts +++ b/src/stems/flac-worker-protocol.ts @@ -1,4 +1,5 @@ import type { WorkerProcessingMetrics } from "./ingest-diagnostics.js"; +import type { EngineWebAdapterErrorCause } from "../errors.js"; import type { CanonicalPcmExpectation, StemIdentity } from "./types.js"; import type { FlacInputSlotBuffers } from "./flac-input-slot.js"; import type { NativeFlacStreamInfo } from "./native-flac-metadata.js"; @@ -59,8 +60,17 @@ export type FlacWorkerResponse = | { readonly type: "error"; readonly requestId: number; - readonly error: { readonly name: string; readonly message: string; readonly code?: string; readonly details?: Readonly> }; - }; + readonly error: { + readonly name: string; + readonly message: string; + readonly role?: "interruption" | "body" | "finalizer"; + readonly truncated?: true; + readonly code?: string; + readonly details?: Readonly>; + readonly cause?: EngineWebAdapterErrorCause; + readonly causes?: readonly EngineWebAdapterErrorCause[]; + }; + }; export interface FlacWorkerLike { postMessage(message: FlacWorkerRequest, transfer?: Transferable[]): void; diff --git a/src/stems/opfs-worker-client.ts b/src/stems/opfs-worker-client.ts index c3edf10..43712a0 100644 --- a/src/stems/opfs-worker-client.ts +++ b/src/stems/opfs-worker-client.ts @@ -1,7 +1,6 @@ import { createOpfsWorker } from "../assets.js"; import type { AdapterAssetOverrides } from "../assets.js"; -import { EngineWebAdapterError } from "../errors.js"; -import type { EngineWebAdapterErrorCode } from "../errors.js"; +import { EngineWebAdapterError, isEngineWebAdapterErrorCode, reviveEngineWebAdapterCause } from "../errors.js"; import { OPFS_WRITE_METHOD, OPFS_WRITE_REMEDY } from "./opfs-worker-protocol.js"; import type { OpfsWorkerLike, OpfsWorkerRequest, OpfsWorkerResponse } from "./opfs-worker-protocol.js"; import type { StemStorageWriter } from "./storage.js"; @@ -324,10 +323,25 @@ export class OpfsWriteWorkerClient { } function reviveWorkerError(error: Extract["error"]): unknown { - if (error.code !== undefined) { - return new EngineWebAdapterError(error.code as EngineWebAdapterErrorCode, error.message, error.details ?? {}); + const causeValue = { + name: error.name, + message: error.message, + ...(error.role === undefined ? {} : { role: error.role }), + ...(error.truncated === undefined ? {} : { truncated: error.truncated }), + ...(error.cause === undefined ? {} : { cause: error.cause }), + ...(error.causes === undefined ? {} : { causes: error.causes }), + }; + const cause = error.causes === undefined + ? error.cause === undefined ? undefined : reviveEngineWebAdapterCause(error.cause) + : reviveEngineWebAdapterCause(causeValue); + if (error.code !== undefined && isEngineWebAdapterErrorCode(error.code) && error.code !== "stem.corrupt") { + return new EngineWebAdapterError(error.code, error.message, error.details ?? {}, cause); } - const revived = new Error(error.message); - revived.name = error.name; - return revived; + if (error.code === "stem.corrupt") { + return new EngineWebAdapterError("storage.io", error.message, { + ...(error.details ?? {}), + failureKind: "adapter", + }, cause ?? reviveEngineWebAdapterCause(causeValue)); + } + return reviveEngineWebAdapterCause(causeValue); } diff --git a/src/stems/opfs-worker-protocol.ts b/src/stems/opfs-worker-protocol.ts index d56e98a..d436c9b 100644 --- a/src/stems/opfs-worker-protocol.ts +++ b/src/stems/opfs-worker-protocol.ts @@ -1,3 +1,5 @@ +import type { EngineWebAdapterErrorCause } from "../errors.js"; + /** * Wire contract for the OPFS write Worker. * @@ -33,8 +35,12 @@ export type OpfsWorkerResponse = readonly error: { readonly name: string; readonly message: string; + readonly role?: "interruption" | "body" | "finalizer"; + readonly truncated?: true; readonly code?: string; readonly details?: Readonly>; + readonly cause?: EngineWebAdapterErrorCause; + readonly causes?: readonly EngineWebAdapterErrorCause[]; }; }; diff --git a/src/stems/sparse-format.ts b/src/stems/sparse-format.ts index 5427a23..98378a1 100644 --- a/src/stems/sparse-format.ts +++ b/src/stems/sparse-format.ts @@ -70,7 +70,7 @@ export interface SparseSessionDeclaredSource { export type SparseSessionSourceExpectation = SparseSessionSourceShape | SparseSessionDeclaredSource; function corrupt(message: string, details: Readonly> = {}): EngineWebAdapterError { - return new EngineWebAdapterError("stem.corrupt", message, details); + return new EngineWebAdapterError("stem.invalid_declaration", message, { ...details, metadata: "sparse-stem" }); } function declaration(message: string, details: Readonly> = {}): EngineWebAdapterError { diff --git a/src/stems/sparse-pcm.ts b/src/stems/sparse-pcm.ts index 8bb55ab..7872385 100644 --- a/src/stems/sparse-pcm.ts +++ b/src/stems/sparse-pcm.ts @@ -44,7 +44,7 @@ export interface SparsePcmIndexDraft { } function corrupt(message: string, details: Readonly> = {}): EngineWebAdapterError { - return new EngineWebAdapterError("stem.corrupt", message, details); + return new EngineWebAdapterError("stem.invalid_declaration", message, { ...details, metadata: "sparse-pcm-index" }); } function integer(value: unknown, path: string, minimum: number): number { diff --git a/src/stems/sparse-resolver.ts b/src/stems/sparse-resolver.ts index 93494c4..ec75313 100644 --- a/src/stems/sparse-resolver.ts +++ b/src/stems/sparse-resolver.ts @@ -158,6 +158,20 @@ interface SparsePullState { activeOutput: ReadableStream | undefined; } +function sparseAdapterFailure( + message: string, + details: Readonly> = {}, + cause?: unknown, +): EngineWebAdapterError { + return new EngineWebAdapterError("storage.io", message, { + operation: "decode", + resource: "sparse-decoder", + phase: "worker-output", + failureKind: "adapter", + ...details, + }, cause); +} + function intervalEnd(interval: SparseStemInterval): number { return interval.packedFrameOffset + interval.frames; } @@ -169,23 +183,23 @@ function expectedPackedFrames(manifest: SparseStemManifest): number { function mapNextBlock(state: SparsePullState): SparsePcmSpan { const block = state.currentBlock; - if (block === undefined) throw new EngineWebAdapterError("stem.corrupt", "Sparse mapper has no decoder block", { identity: state.expected.identity }); + if (block === undefined) throw sparseAdapterFailure("Sparse mapper has no decoder block", { identity: state.expected.identity }); while (state.intervalIndex < state.manifest.intervals.length && state.packedFrame >= intervalEnd(state.manifest.intervals[state.intervalIndex]!)) state.intervalIndex += 1; const interval = state.manifest.intervals[state.intervalIndex]; if (interval === undefined || state.packedFrame < interval.packedFrameOffset || state.packedFrame >= intervalEnd(interval)) { - throw new EngineWebAdapterError("stem.corrupt", "Sparse decoder block falls outside admitted active intervals", { identity: state.expected.identity, packedFrame: state.packedFrame }); + throw sparseAdapterFailure("Sparse decoder block falls outside admitted active intervals", { identity: state.expected.identity, packedFrame: state.packedFrame }); } const maxFrames = Math.max(1, Math.floor(128 * 1024 / state.frameBytes)); const blockEnd = block.startPackedFrame + block.frames; const end = Math.min(blockEnd, intervalEnd(interval), state.packedFrame + maxFrames); - if (end <= state.packedFrame) throw new EngineWebAdapterError("stem.corrupt", "Sparse mapper made no frame progress", { identity: state.expected.identity }); + if (end <= state.packedFrame) throw sparseAdapterFailure("Sparse mapper made no frame progress", { identity: state.expected.identity }); const frames = end - state.packedFrame; const startInBlock = state.packedFrame - block.startPackedFrame; const startByte = startInBlock * state.frameBytes; const endByte = startByte + frames * state.frameBytes; const bytes = block.value.bytes.subarray(startByte, endByte); - if (bytes.byteLength !== frames * state.frameBytes) throw new EngineWebAdapterError("stem.corrupt", "Sparse decoder block view is not frame aligned", { identity: state.expected.identity }); + if (bytes.byteLength !== frames * state.frameBytes) throw sparseAdapterFailure("Sparse decoder block view is not frame aligned", { identity: state.expected.identity }); const span = { startFrame: interval.startFrame + (state.packedFrame - interval.packedFrameOffset), bytes, @@ -225,7 +239,7 @@ const nextSparseSpan = Effect.fn("SparseResolver.nextSpan")(function*(state: Spa if (bytes.byteLength < 1 || bytes.byteLength % state.frameBytes !== 0 || bytes.byteLength > 384 * 1024 || next.value.bytes.buffer.byteLength > 384 * 1024) { next.value.release(); - return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "Sparse decoder output has an unsafe PCM block shape", { identity: state.expected.identity })); + return yield* Effect.fail(sparseAdapterFailure("Sparse decoder output has an unsafe PCM block shape", { identity: state.expected.identity })); } const frames = bytes.byteLength / state.frameBytes; state.currentBlock = { value: next.value, startPackedFrame: state.packedFrame, frames, offsetFrames: 0 }; @@ -236,7 +250,7 @@ const nextSparseSpan = Effect.fn("SparseResolver.nextSpan")(function*(state: Spa state.activeOutput = undefined; const chunk = state.currentChunk; if (chunk === undefined || state.chunkFrames !== chunk.frames) { - return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "Sparse decoder output did not cover its declared chunk", { + return yield* Effect.fail(sparseAdapterFailure("Sparse decoder output did not cover its declared chunk", { identity: state.expected.identity, expectedFrames: chunk?.frames, actualFrames: state.chunkFrames, @@ -251,7 +265,7 @@ const nextSparseSpan = Effect.fn("SparseResolver.nextSpan")(function*(state: Spa const chunk = state.manifest.chunks[state.chunkIndex]; if (chunk === undefined) { - if (state.packedFrame !== expectedPackedFrames(state.manifest)) return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "Sparse packed frame endpoint changed during mapping", { identity: state.expected.identity })); + if (state.packedFrame !== expectedPackedFrames(state.manifest)) return yield* Effect.fail(sparseAdapterFailure("Sparse packed frame endpoint changed during mapping", { identity: state.expected.identity })); yield* state.cursor.assertEof; if (state.signal.aborted) return yield* Effect.fail(sparseStreamCancelled(state.expected.identity, state.signal.reason)); // Active PCM decoding can finish below the full canonical total when @@ -273,7 +287,7 @@ const nextSparseSpan = Effect.fn("SparseResolver.nextSpan")(function*(state: Spa return yield* Cause.done(); } const expectedPosition = state.payloadStart + chunk.offset; - if (state.cursor.position !== expectedPosition) return yield* Effect.fail(new EngineWebAdapterError("stem.corrupt", "Sparse finite source cursor is not at its admitted chunk", { identity: state.expected.identity, expectedPosition, actualPosition: state.cursor.position })); + if (state.cursor.position !== expectedPosition) return yield* Effect.fail(sparseAdapterFailure("Sparse finite source cursor is not at its admitted chunk", { identity: state.expected.identity, expectedPosition, actualPosition: state.cursor.position })); const onChunkProgress = (progress: StemProgress): void => { if (progress.stage === "decoding" && progress.byteKind === "pcm") { if (!Number.isSafeInteger(progress.bytes) || progress.bytes < state.chunkDecodedBytes || @@ -568,9 +582,9 @@ export function createSparseStemResolver( onProgress: reportResponse, }); const header = yield* cursor.readExact(SPARSE_STEM_HEADER_BYTES); - const headerAdmission = yield* Effect.try({ try: () => admitSparseStemHeader(header), catch: cause => cause instanceof EngineWebAdapterError ? cause : new EngineWebAdapterError("stem.corrupt", "Sparse header admission failed", { identity: expected.identity }, cause) }); + const headerAdmission = yield* Effect.try({ try: () => admitSparseStemHeader(header), catch: cause => cause instanceof EngineWebAdapterError ? cause : new EngineWebAdapterError("stem.invalid_declaration", "Sparse header admission failed", { identity: expected.identity, metadata: "sparse-stem-header" }, cause) }); const encodedManifest = yield* cursor.readExact(headerAdmission.manifestBytes); - const manifest = yield* Effect.try({ try: () => admitSparseStemManifest(encodedManifest), catch: cause => cause instanceof EngineWebAdapterError ? cause : new EngineWebAdapterError("stem.corrupt", "Sparse manifest admission failed", { identity: expected.identity }, cause) }); + const manifest = yield* Effect.try({ try: () => admitSparseStemManifest(encodedManifest), catch: cause => cause instanceof EngineWebAdapterError ? cause : new EngineWebAdapterError("stem.invalid_declaration", "Sparse manifest admission failed", { identity: expected.identity, metadata: "sparse-stem-manifest" }, cause) }); yield* Effect.try({ try: () => assertSparseStemSessionBinding(manifest, expected), catch: cause => cause instanceof EngineWebAdapterError ? cause : new EngineWebAdapterError("stem.invalid_declaration", "Sparse manifest does not match its session source", { identity: expected.identity }, cause) }); const payload = payloadBytes(manifest); responseTotal = headerAdmission.payloadStart + payload; diff --git a/src/stems/sparse-store.ts b/src/stems/sparse-store.ts index 9feaa8a..d563ea2 100644 --- a/src/stems/sparse-store.ts +++ b/src/stems/sparse-store.ts @@ -1,6 +1,7 @@ import { Cause, Context, Effect, Exit, Layer, ManagedRuntime, Option, Random, Ref, Schema, Scope, Stream } from "effect"; -import { EngineWebAdapterError } from "../errors.js"; +import { EngineWebAdapterError, isEngineWebAdapterVerification } from "../errors.js"; +import type { EngineWebAdapterErrorContext, EngineWebAdapterVerification } from "../errors.js"; import { createIncrementalBlake3, type IncrementalBlake3 } from "./blake3.js"; import { registerForcedProgressObserver, sparseProgressReporter, type ProgressObserver } from "./progress.js"; import { sparseResolverScheduling } from "./sparse-scheduling.js"; @@ -64,15 +65,23 @@ export type SparsePcmPresence = | { readonly status: "missing" } | { readonly status: "present"; readonly activeBytes: number }; -class SparseBoundaryError extends Schema.TaggedError()("SparseBoundaryError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseCorruptError extends Schema.TaggedError()("SparseCorruptError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseNotFoundError extends Schema.TaggedError()("SparseNotFoundError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseCancelledError extends Schema.TaggedError()("SparseCancelledError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseDeadlineError extends Schema.TaggedError()("SparseDeadlineError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseQuotaError extends Schema.TaggedError()("SparseQuotaError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseConflictError extends Schema.TaggedError()("SparseConflictError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -class SparseIoError extends Schema.TaggedError()("SparseIoError", { message: Schema.String, cause: Schema.optionalKey(Schema.Unknown) }) {} -type SparseFailure = SparseBoundaryError | SparseCorruptError | SparseNotFoundError | SparseCancelledError | SparseDeadlineError | SparseQuotaError | SparseConflictError | SparseIoError; +const SparseFailureFields = { + message: Schema.String, + cause: Schema.optionalKey(Schema.Unknown), + context: Schema.optionalKey(Schema.Unknown), + verification: Schema.optionalKey(Schema.Unknown), + failureKind: Schema.optionalKey(Schema.String), +} as const; +class SparseBoundaryError extends Schema.TaggedError()("SparseBoundaryError", SparseFailureFields) {} +class SparseMetadataError extends Schema.TaggedError()("SparseMetadataError", SparseFailureFields) {} +class SparseIntegrityError extends Schema.TaggedError()("SparseIntegrityError", SparseFailureFields) {} +class SparseNotFoundError extends Schema.TaggedError()("SparseNotFoundError", SparseFailureFields) {} +class SparseCancelledError extends Schema.TaggedError()("SparseCancelledError", SparseFailureFields) {} +class SparseDeadlineError extends Schema.TaggedError()("SparseDeadlineError", SparseFailureFields) {} +class SparseQuotaError extends Schema.TaggedError()("SparseQuotaError", SparseFailureFields) {} +class SparseConflictError extends Schema.TaggedError()("SparseConflictError", SparseFailureFields) {} +class SparseIoError extends Schema.TaggedError()("SparseIoError", SparseFailureFields) {} +type SparseFailure = SparseBoundaryError | SparseMetadataError | SparseIntegrityError | SparseNotFoundError | SparseCancelledError | SparseDeadlineError | SparseQuotaError | SparseConflictError | SparseIoError; const ExpectedSchema = Schema.Struct({ identity: Schema.String, @@ -122,7 +131,7 @@ interface BackendShape { readonly createWriter: (name: string, signal: AbortSignal) => Effect.Effect; readonly write: (writer: StemStorageWriter, chunk: Uint8Array | string) => Effect.Effect; readonly close: (writer: StemStorageWriter) => Effect.Effect; - readonly abort: (writer: StemStorageWriter, reason: unknown) => Effect.Effect; + readonly abort: (writer: StemStorageWriter, reason: unknown) => Effect.Effect; readonly remove: (name: string) => Effect.Effect; readonly estimate: Effect.Effect<{ readonly quota?: number; readonly usage?: number }, SparseFailure>; readonly lock: { readonly locks?: WebLockProvider; readonly shared: SharedLockState; readonly folderName: string }; @@ -131,24 +140,43 @@ interface BackendShape { readonly opfsOwnedDeadlines: boolean; } +interface SparseWriterState { + readonly signal: AbortSignal; + readonly context: EngineWebAdapterErrorContext; +} + +function storageContext(operation: string, resource: string, phase: string): EngineWebAdapterErrorContext { + return Object.freeze({ operation, resource, phase }); +} + +function resourceForName(name: string): string { + if (name.startsWith("sparse-pcm-v1-commit-")) return "sparse-marker"; + if (name.startsWith("sparse-pcm-v1-data-")) return "sparse-payload"; + return "sparse-storage"; +} + class SparseBackend extends Context.Service()("engine-web/SparseBackend") { static layer(backend: StemStorageBackend, options: { readonly lock: BackendShape["lock"]; readonly instanceId: string; readonly readDeadlineMs: number }): Layer.Layer { - const writerSignals = new WeakMap(); - const fail = (operation: string, cause: unknown): SparseFailure => { + const writerSignals = new WeakMap(); + const fail = (operation: string, cause: unknown, context: EngineWebAdapterErrorContext, failureKind = "io"): SparseFailure => { const name = cause && typeof cause === "object" && "name" in cause ? String((cause as { readonly name?: unknown }).name) : ""; - if (name === "AbortError") return new SparseCancelledError({ message: `${operation} was cancelled`, cause }); - if (name === "TimeoutError") return new SparseDeadlineError({ message: `${operation} exceeded its deadline`, cause }); - if (name === "QuotaExceededError") return new SparseQuotaError({ message: `${operation} exceeded quota`, cause }); - return new SparseIoError({ message: `${operation} failed`, cause }); + if (name === "AbortError") return new SparseCancelledError({ message: `${operation} was cancelled`, cause, context, failureKind: "interruption" }); + if (name === "TimeoutError") return new SparseDeadlineError({ message: `${operation} exceeded its deadline`, cause, context, failureKind: "deadline" }); + if (name === "QuotaExceededError") return new SparseQuotaError({ message: `${operation} exceeded quota`, cause, context, failureKind: "quota" }); + return new SparseIoError({ message: `${operation} failed`, cause, context, failureKind }); }; - const promise = (operation: string, run: (signal: AbortSignal) => PromiseLike): Effect.Effect => Effect.tryPromise({ try: run, catch: (cause) => fail(operation, cause) }); - const timed = (operation: string, run: (signal: AbortSignal) => PromiseLike): Effect.Effect => { - const base = promise(operation, run); - return base.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline` })))); + const promise = (operation: string, context: EngineWebAdapterErrorContext, run: (signal: AbortSignal) => PromiseLike): Effect.Effect => Effect.tryPromise({ try: run, catch: (cause) => fail(operation, cause, context) }); + const timed = (operation: string, context: EngineWebAdapterErrorContext, run: (signal: AbortSignal) => PromiseLike): Effect.Effect => { + const base = promise(operation, context, run); + return base.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline`, context, failureKind: "deadline" })))); }; - const physical = (operation: string, writer: StemStorageWriter | undefined, run: () => PromiseLike): Effect.Effect => { + const physical = (operation: string, writer: StemStorageWriter | undefined, context: EngineWebAdapterErrorContext, run: () => PromiseLike): Effect.Effect => { const effect = Effect.callback((resume, effectSignal) => { - const requestedSignal = writer === undefined ? undefined : writerSignals.get(writer); + const writerState = writer === undefined ? undefined : writerSignals.get(writer); + const requestedSignal = writerState?.signal; + const operationContext = writerState === undefined + ? context + : storageContext(operation, writerState.context.resource, context.phase); let settled = false; let mutationStarted = false; let abortRequested = false; @@ -158,7 +186,7 @@ class SparseBackend extends Context.Service()("engi const onAbort = () => { abortRequested = true; if (!mutationStarted) { - finish(Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled` }))); + finish(Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled`, context: operationContext, failureKind: "interruption" }))); return; } if (pending === undefined) return; @@ -169,12 +197,46 @@ class SparseBackend extends Context.Service()("engi } void Promise.allSettled([pending, abortPromise]).then((results) => { if (settled) return; + const physicalResult = results[0]; const abortResult = results[1]; - finish(abortResult?.status === "rejected" ? Effect.fail(fail(operation, abortResult.reason)) : Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled` }))); + if (physicalResult?.status === "rejected") { + const failures: Error[] = [ + roleFailure("interruption", requestedSignal?.reason ?? new DOMException("Physical operation interrupted", "AbortError")), + roleFailure("body", physicalResult.reason), + ]; + if (abortResult?.status === "rejected") failures.push(roleFailure("finalizer", abortResult.reason)); + finish(Effect.fail(new SparseIoError({ + message: abortResult?.status === "rejected" + ? `${operation} and its cancellation cleanup failed` + : `${operation} failed after cancellation`, + cause: new AggregateError(failures, `${operation} cancellation and physical settlement failed`), + context: operationContext, + failureKind: "io", + }))); + return; + } + if (abortResult?.status === "rejected") { + finish(Effect.fail(new SparseIoError({ + message: `${operation} cancellation cleanup failed`, + cause: new AggregateError([ + roleFailure("interruption", requestedSignal?.reason ?? new DOMException("Physical operation interrupted", "AbortError")), + roleFailure("finalizer", abortResult.reason), + ], `${operation} cancellation cleanup failed`), + context: operationContext, + failureKind: "io", + }))); + return; + } + finish(Effect.fail(new SparseCancelledError({ + message: `${operation} was cancelled`, + cause: requestedSignal?.reason, + context: operationContext, + failureKind: "interruption", + }))); }); }; if (requestedSignal?.aborted) { - finish(Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled`, cause: requestedSignal.reason }))); + finish(Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled`, cause: requestedSignal.reason, context: operationContext, failureKind: "interruption" }))); return Effect.void; } requestedSignal?.addEventListener("abort", onAbort, { once: true }); @@ -185,7 +247,9 @@ class SparseBackend extends Context.Service()("engi mutationStarted = true; try { pending = Promise.resolve(run()); } catch (cause) { - finish(requestedSignal?.aborted || abortRequested ? Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled`, cause })) : Effect.fail(fail(operation, cause))); + finish(requestedSignal?.aborted || abortRequested + ? Effect.fail(new SparseCancelledError({ message: `${operation} was cancelled`, cause, context: operationContext, failureKind: "interruption" })) + : Effect.fail(fail(operation, cause, operationContext))); return Effect.void; } if (requestedSignal?.aborted || abortRequested) onAbort(); @@ -196,7 +260,7 @@ class SparseBackend extends Context.Service()("engi }, (cause) => { if (requestedSignal?.aborted || abortRequested) { onAbort(); return; } - if (!effectSignal.aborted) finish(Effect.fail(fail(operation, cause))); + if (!effectSignal.aborted) finish(Effect.fail(fail(operation, cause, operationContext))); }, ); return Effect.promise(async () => { @@ -211,66 +275,77 @@ class SparseBackend extends Context.Service()("engi // physical mutation. Never remove or unlock while a late write can // still settle against the owned file. await Promise.allSettled([pending]); - if (abortFailure !== undefined) throw abortFailure; + if (abortFailure !== undefined) throw fail("abort", abortFailure, storageContext("abort", operationContext.resource, "writer-abort")); }); }); if (ownsOpfsWriteDeadlines(backend, options.readDeadlineMs)) return effect; - return effect.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline` })))); + return effect.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline`, context, failureKind: "deadline" })))); }; const createWriter = (name: string, requestedSignal: AbortSignal): Effect.Effect => Effect.callback((resume, effectSignal) => { const signal = requestedSignal; + const context = storageContext("create", resourceForName(name), "writer-open"); let pending: Promise; try { pending = Promise.resolve(backend.createWriter(name, signal)); } - catch (cause) { resume(Effect.fail(fail("writer create", cause))); return; } + catch (cause) { resume(Effect.fail(fail("create", cause, context))); return; } let settled = false; const complete = (result: Effect.Effect) => { if (!settled) { settled = true; signal.removeEventListener("abort", onAbort); resume(result); } }; const onAbort = () => { void pending.then(async (writer) => { try { await writer.abort(new DOMException("Physical writer acquisition interrupted", "AbortError")); - complete(Effect.fail(new SparseCancelledError({ message: "Sparse writer acquisition was cancelled" }))); + complete(Effect.fail(new SparseCancelledError({ message: "Sparse writer acquisition was cancelled", context, failureKind: "interruption" }))); } catch (cause) { - complete(Effect.fail(fail("writer abort", cause))); + complete(Effect.fail(fail("abort", cause, storageContext("abort", context.resource, "writer-abort")))); } - }, (cause) => complete(Effect.fail(isAbort(cause) ? new SparseCancelledError({ message: "Sparse writer acquisition was cancelled", cause }) : fail("writer create", cause)))); + }, (cause) => complete(Effect.fail(isAbort(cause) + ? new SparseCancelledError({ message: "Sparse writer acquisition was cancelled", cause, context, failureKind: "interruption" }) + : fail("create", cause, context)))); }; signal.addEventListener("abort", onAbort, { once: true }); pending.then( - (writer) => { if (!signal.aborted && !effectSignal.aborted) { writerSignals.set(writer, signal); complete(Effect.succeed(writer)); } else onAbort(); }, - (cause) => { if (!effectSignal.aborted) complete(Effect.fail(signal.aborted || isAbort(cause) ? new SparseCancelledError({ message: "Sparse writer acquisition was cancelled", cause }) : fail("writer create", cause))); }, + (writer) => { if (!signal.aborted && !effectSignal.aborted) { writerSignals.set(writer, { signal, context }); complete(Effect.succeed(writer)); } else onAbort(); }, + (cause) => { if (!effectSignal.aborted) complete(Effect.fail(signal.aborted || isAbort(cause) + ? new SparseCancelledError({ message: "Sparse writer acquisition was cancelled", cause, context, failureKind: "interruption" }) + : fail("create", cause, context))); }, ); return Effect.promise(async () => { signal.removeEventListener("abort", onAbort); let writer: StemStorageWriter; try { writer = await pending; } catch { return; } // a rejected create has no physical writer to abort - await writer.abort(new DOMException("Physical writer acquisition interrupted", "AbortError")); + try { await writer.abort(new DOMException("Physical writer acquisition interrupted", "AbortError")); } + catch (cause) { throw fail("abort", cause, storageContext("abort", context.resource, "writer-abort")); } }); }); const createTimed = (name: string, signal: AbortSignal): Effect.Effect => { const effect = createWriter(name, signal); - return ownsOpfsWriteDeadlines(backend, options.readDeadlineMs) ? effect : effect.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: "writer create exceeded its deadline" })))); + const context = storageContext("create", resourceForName(name), "writer-open"); + return ownsOpfsWriteDeadlines(backend, options.readDeadlineMs) ? effect : effect.pipe(Effect.timeout(options.readDeadlineMs), Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: "writer create exceeded its deadline", context, failureKind: "deadline" })))); }; const removePhysical = (name: string): Effect.Effect => Effect.callback((resume, effectSignal) => { + const context = storageContext("remove", resourceForName(name), "cleanup"); let pending: Promise; try { pending = Promise.resolve(backend.remove(name)); } - catch (cause) { resume(Effect.fail(fail("storage remove", cause))); return; } + catch (cause) { resume(Effect.fail(fail("remove", cause, context))); return; } pending.then( () => { if (!effectSignal.aborted) resume(Effect.succeed(undefined)); }, - (cause) => { if (!effectSignal.aborted) resume(Effect.fail(fail("storage remove", cause))); }, + (cause) => { if (!effectSignal.aborted) resume(Effect.fail(fail("remove", cause, context))); }, ); return Effect.promise(() => pending); }); return Layer.succeed(this, this.of({ - open: timed("storage open", () => backend.open()), - exists: (name) => timed("storage exists", () => backend.exists(name)), - read: (name) => timed("storage read", () => backend.read(name)), + open: timed("open", storageContext("open", "sparse-root", "root-open"), () => backend.open()), + exists: (name) => timed("exists", storageContext("exists", resourceForName(name), "metadata"), () => backend.exists(name)), + read: (name) => timed("read", storageContext("read", resourceForName(name), "metadata"), () => backend.read(name)), createWriter: (name, signal) => createTimed(name, signal), - write: (writer, chunk) => physical("writer write", writer, () => writer.write(chunk)).pipe(Effect.asVoid), - close: (writer) => physical("writer close", writer, () => writer.close()).pipe(Effect.asVoid), - abort: (writer, reason) => Effect.promise(() => writer.abort(reason)), + write: (writer, chunk) => physical("write", writer, storageContext("write", "sparse-storage", "payload-write"), () => writer.write(chunk)).pipe(Effect.asVoid), + close: (writer) => physical("close", writer, storageContext("close", "sparse-storage", "writer-close"), () => writer.close()).pipe(Effect.asVoid), + abort: (writer, reason) => { + const context = writerSignals.get(writer)?.context ?? storageContext("abort", "sparse-storage", "writer-abort"); + return promise("abort", storageContext("abort", context.resource, "writer-abort"), () => writer.abort(reason)); + }, remove: removePhysical, - estimate: timed("storage estimate", () => backend.estimate?.() ?? Promise.resolve({})), + estimate: timed("estimate", storageContext("estimate", "sparse-root", "quota"), () => backend.estimate?.() ?? Promise.resolve({})), lock: options.lock, instanceId: options.instanceId, readDeadlineMs: options.readDeadlineMs, @@ -969,10 +1044,14 @@ function resolveSource( const resolving = Effect.callback((resume, effectSignal) => { let pending: Promise; try { pending = Promise.resolve(resolve(child.signal, context)); } - catch (cause) { resume(Effect.fail(isAbort(cause) ? new SparseCancelledError({ message: "Sparse resolver was cancelled", cause }) : new SparseIoError({ message: "Sparse resolver failed", cause }))); return Effect.sync(() => { child.abort(cause); }); } + catch (cause) { resume(Effect.fail(isAbort(cause) + ? new SparseCancelledError({ message: "Sparse resolver was cancelled", cause, context: storageContext("resolve", "resolver", "resolution"), failureKind: "interruption" }) + : new SparseIoError({ message: "Sparse resolver failed", cause, context: storageContext("resolve", "resolver", "resolution"), failureKind: "io" }))); return Effect.sync(() => { child.abort(cause); }); } pending.then( (resolved) => { if (!effectSignal.aborted) resume(Effect.succeed(resolved)); }, - (cause) => { if (!effectSignal.aborted) resume(Effect.fail(isAbort(cause) ? new SparseCancelledError({ message: "Sparse resolver was cancelled", cause }) : new SparseIoError({ message: "Sparse resolver failed", cause }))); }, + (cause) => { if (!effectSignal.aborted) resume(Effect.fail(isAbort(cause) + ? new SparseCancelledError({ message: "Sparse resolver was cancelled", cause, context: storageContext("resolve", "resolver", "resolution"), failureKind: "interruption" }) + : new SparseIoError({ message: "Sparse resolver failed", cause, context: storageContext("resolve", "resolver", "resolution"), failureKind: "io" }))); }, ); return Effect.promise(async () => { child.abort(new DOMException("Sparse resolver interrupted", "AbortError")); @@ -980,13 +1059,13 @@ function resolveSource( }); }).pipe( Effect.timeout(deadlineMs), - Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: "Sparse resolver exceeded its deadline" }))), + Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: "Sparse resolver exceeded its deadline", context: storageContext("resolve", "resolver", "resolution"), failureKind: "deadline" }))), ); const value = yield* resolving; return yield* Effect.try({ try: () => { if (!isRecord(value) || typeof (value.spans as { readonly [Symbol.asyncIterator]?: unknown } | undefined)?.[Symbol.asyncIterator] !== "function") throw new Error("resolver spans must be an AsyncIterable"); return { ...value, sourceController: child } as ResolvedInput; - }, catch: (cause) => new SparseBoundaryError({ message: "Sparse resolver result is invalid", cause }) }); + }, catch: (cause) => new SparseBoundaryError({ message: "Sparse resolver result is invalid", cause, context: storageContext("resolve", "resolver", "resolution"), failureKind: "adapter" }) }); }); } @@ -1031,10 +1110,10 @@ function ingestAndCommit( // Reserve the unpublished name before asking the backend to create it. // A backend may create a file and then reject/cancel the create promise; // cleanup must still know that this transaction owns that name. - const dataWriter = yield* Effect.acquireRelease(backend.createWriter(dataName, operation.signal), (writer) => dataClosed ? Effect.void : backend.abort(writer, "data writer scope closed"), { interruptible: true }); + const dataWriter = yield* Effect.acquireRelease(backend.createWriter(dataName, operation.signal), (writer) => dataClosed ? Effect.void : backend.abort(writer, "data writer scope closed").pipe(Effect.orDie), { interruptible: true }); const hash = yield* Effect.tryPromise({ try: createIncrementalBlake3, - catch: (cause) => new SparseBoundaryError({ message: "BLAKE3 verification could not initialize", cause }), + catch: (cause) => new SparseBoundaryError({ message: "BLAKE3 verification could not initialize", cause, context: storageContext("verify", "sparse-payload", "warm-verification"), failureKind: "adapter" }), }); const intervals: SparsePcmInterval[] = []; let previousEnd = 0; @@ -1090,16 +1169,21 @@ function ingestAndCommit( yield* hashZeros(hash, (expected.frames - previousEnd) * frameBytes, operation.signal, (bytes) => onProgress?.({ stage: "ingesting", identity: expected.identity, bytes, totalBytes: expected.canonicalBytes, byteKind: "pcm", }), previousEnd * frameBytes); - const index = yield* Effect.try({ try: () => validateSparsePcmIndex({ format: SPARSE_PCM_FORMAT, identity: expected.identity, sampleRateHz: expected.sampleRateHz, channels: expected.channels, bitDepth: expected.bitDepth, frames: expected.frames, intervals, activeBytes, canonicalBytes: expected.canonicalBytes }, activeBytes), catch: (cause) => new SparseCorruptError({ message: "Derived sparse index is invalid", cause }) }); + const index = yield* Effect.try({ try: () => validateSparsePcmIndex({ format: SPARSE_PCM_FORMAT, identity: expected.identity, sampleRateHz: expected.sampleRateHz, channels: expected.channels, bitDepth: expected.bitDepth, frames: expected.frames, intervals, activeBytes, canonicalBytes: expected.canonicalBytes }, activeBytes), catch: (cause) => new SparseBoundaryError({ message: "Derived sparse index is invalid", cause, context: storageContext("validate", "sparse-marker", "metadata") }) }); if (asserted !== undefined) compareIndexes(asserted, index); - if (hash.digest("hex") !== identityHex(expected.identity)) return yield* new SparseCorruptError({ message: "Sparse active spans do not match canonical identity" }); - if (dataClosed) return yield* new SparseCorruptError({ message: "Sparse data writer closed unexpectedly" }); + const digest = hash.digest("hex"); + if (digest !== identityHex(expected.identity)) return yield* new SparseIntegrityError({ + message: "Sparse active spans do not match canonical identity", + context: storageContext("verify", "sparse-payload", "ingest-verification"), + verification: { kind: "digest", expected: identityHex(expected.identity), observed: digest, algorithm: "blake3-256" }, + }); + if (dataClosed) return yield* new SparseIoError({ message: "Sparse data writer closed unexpectedly", context: storageContext("close", "sparse-payload", "writer-state"), failureKind: "adapter" }); yield* backend.close(dataWriter); dataClosed = true; yield* Ref.set(operation.ref, { _tag: "data-closed" } as Lifecycle); const marker = makeMarker(expected, index, dataName, generation); const markerBytes = canonicalJsonBytes(marker); - if (markerBytes.byteLength > MAX_MARKER_BYTES) return yield* new SparseCorruptError({ message: "Sparse marker exceeds its bound" }); + if (markerBytes.byteLength > MAX_MARKER_BYTES) return yield* new SparseBoundaryError({ message: "Sparse marker exceeds its bound", context: storageContext("validate", "sparse-marker", "metadata") }); yield* quotaCheck(backend, markerBytes.byteLength); yield* Ref.set(operation.ref, { _tag: "marker-writing" } as Lifecycle); let markerClosed = false; @@ -1107,13 +1191,17 @@ function ingestAndCommit( // The marker is visible only after close, but its ordinary file entry may // exist as soon as create is attempted. Register ownership before that // attempt so a partial create is removed only after physical abort. - const markerWriter = yield* Effect.acquireRelease(backend.createWriter(markerName(expected.identity), operation.signal), (writer) => markerClosed ? Effect.void : backend.abort(writer, "marker writer scope closed"), { interruptible: true }); + const markerWriter = yield* Effect.acquireRelease(backend.createWriter(markerName(expected.identity), operation.signal), (writer) => markerClosed ? Effect.void : backend.abort(writer, "marker writer scope closed").pipe(Effect.orDie), { interruptible: true }); yield* backend.write(markerWriter, markerBytes); yield* backend.close(markerWriter); markerClosed = true; yield* checkSignal(operation.signal); const data = yield* backend.read(dataName); - if (data.size !== index.activeBytes) return yield* new SparseCorruptError({ message: "Sparse payload length changed before commit" }); + if (data.size !== index.activeBytes) return yield* new SparseIntegrityError({ + message: "Sparse payload length changed before commit", + context: storageContext("verify", "sparse-payload", "commit-verification"), + verification: { kind: "length", expected: index.activeBytes, observed: data.size }, + }); yield* checkSignal(operation.signal); markerCommitted = markerClosed; return { marker, data }; @@ -1266,7 +1354,12 @@ function makeWarmTaskYieldResource(signal: AbortSignal): WarmTaskYieldResource { if (failures.length === 0) return cause; return new AggregateError([cause, ...failures], "Warm task-yield resource cleanup failed"); }; - const boundaryFailure = (message: string, cause: unknown, failures: readonly unknown[] = []): SparseBoundaryError => new SparseBoundaryError({ message, cause: cleanupCause(cause, failures) }); + const boundaryFailure = (message: string, cause: unknown, failures: readonly unknown[] = []): SparseBoundaryError => new SparseBoundaryError({ + message, + cause: cleanupCause(cause, failures), + context: storageContext("verify", "sparse-payload", "warm-task"), + failureKind: "adapter", + }); const close = (): void => { if (closed) return; closed = true; @@ -1288,7 +1381,7 @@ function makeWarmTaskYieldResource(signal: AbortSignal): WarmTaskYieldResource { try { close(); } catch (cause) { cleanupFailure = cause; } return cleanupFailure === undefined ? cancellation() - : Effect.fail(new SparseBoundaryError({ message: "Warm task-yield cancellation cleanup failed", cause: cleanupFailure })); + : Effect.fail(new SparseBoundaryError({ message: "Warm task-yield cancellation cleanup failed", cause: cleanupFailure, context: storageContext("verify", "sparse-payload", "warm-task"), failureKind: "adapter" })); }; const closeRawPorts = (ports: readonly unknown[]): unknown[] => { const failures: unknown[] = []; @@ -1344,12 +1437,12 @@ function makeWarmTaskYieldResource(signal: AbortSignal): WarmTaskYieldResource { return Effect.void; } if (pending !== undefined) { - resume(Effect.fail(new SparseBoundaryError({ message: "Warm task-yield wait overlapped" }))); + resume(Effect.fail(new SparseBoundaryError({ message: "Warm task-yield wait overlapped", context: storageContext("verify", "sparse-payload", "warm-task"), failureKind: "adapter" }))); return Effect.void; } try { ensureChannel(); } catch (cause) { - resume(Effect.fail(cause instanceof SparseBoundaryError ? cause : new SparseBoundaryError({ message: "Warm task-yield channel failed", cause }))); + resume(Effect.fail(cause instanceof SparseBoundaryError ? cause : new SparseBoundaryError({ message: "Warm task-yield channel failed", cause, context: storageContext("verify", "sparse-payload", "warm-task"), failureKind: "adapter" }))); return Effect.void; } if (fallback) { @@ -1412,7 +1505,7 @@ function hashZeros( yieldEffect: Effect.Effect = Effect.yieldNow, ): Effect.Effect { return Effect.fn("SparseProgram.hashZeros")(function*() { - if (!Number.isSafeInteger(bytes) || bytes < 0 || !Number.isSafeInteger(baseBytes) || baseBytes < 0 || baseBytes + bytes > Number.MAX_SAFE_INTEGER) return yield* new SparseCorruptError({ message: "Sparse zero gap arithmetic is unsafe" }); + if (!Number.isSafeInteger(bytes) || bytes < 0 || !Number.isSafeInteger(baseBytes) || baseBytes < 0 || baseBytes + bytes > Number.MAX_SAFE_INTEGER) return yield* new SparseBoundaryError({ message: "Sparse zero gap arithmetic is unsafe", failureKind: "adapter" }); for (let offset = 0; offset < bytes; offset += ZERO_BLOCK.byteLength) { yield* checkSignal(signal); const count = Math.min(ZERO_BLOCK.byteLength, bytes - offset); @@ -1427,14 +1520,15 @@ function readMarker(backend: BackendShape, name: string, signal: AbortSignal): E return Effect.fn("SparseProgram.readMarker")(function*() { yield* checkSignal(signal); const blob = yield* backend.read(name); - if (!Number.isSafeInteger(blob.size) || blob.size < 1 || blob.size > MAX_MARKER_BYTES) return yield* new SparseCorruptError({ message: "Sparse marker is outside its bounded byte size" }); + const metadataContext = storageContext("read", "sparse-marker", "metadata"); + if (!Number.isSafeInteger(blob.size) || blob.size < 1 || blob.size > MAX_MARKER_BYTES) return yield* new SparseMetadataError({ message: "Sparse marker is outside its bounded byte size", context: metadataContext }); const bytes = yield* readBlobBytes(backend, blob, 0, blob.size, signal, "Sparse marker read"); - const value = yield* Effect.try({ try: () => JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)) as unknown, catch: (cause) => new SparseCorruptError({ message: "Sparse marker JSON is invalid", cause }) }); - const decoded = yield* Schema.decodeUnknownEffect(MarkerSchema, { onExcessProperty: "error" })(value).pipe(Effect.mapError((cause) => new SparseCorruptError({ message: "Sparse marker schema is invalid", cause }))); - const index = yield* Effect.try({ try: () => validateSparsePcmIndex(decoded.index, decoded.activeBytes), catch: (cause) => new SparseCorruptError({ message: "Sparse marker index is invalid", cause }) }); + const value = yield* Effect.try({ try: () => JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)) as unknown, catch: (cause) => new SparseMetadataError({ message: "Sparse marker JSON is invalid", cause, context: metadataContext }) }); + const decoded = yield* Schema.decodeUnknownEffect(MarkerSchema, { onExcessProperty: "error" })(value).pipe(Effect.mapError((cause) => new SparseMetadataError({ message: "Sparse marker schema is invalid", cause, context: metadataContext }))); + const index = yield* Effect.try({ try: () => validateSparsePcmIndex(decoded.index, decoded.activeBytes), catch: (cause) => new SparseMetadataError({ message: "Sparse marker index is invalid", cause, context: metadataContext }) }); const marker = Object.freeze({ ...decoded, index }) as Marker; const canonical = canonicalJsonBytes(marker); - if (!sameBytes(canonical, new Uint8Array(bytes))) return yield* new SparseCorruptError({ message: "Sparse marker is not canonical JSON" }); + if (!sameBytes(canonical, new Uint8Array(bytes))) return yield* new SparseMetadataError({ message: "Sparse marker is not canonical JSON", context: metadataContext }); return marker; })(); } @@ -1463,15 +1557,20 @@ function prospectiveMarkerBytes(expected: SparsePcmExpectation, generation: stri interface AdmittedSparseMetadata { readonly marker: Marker; readonly data: Blob } function admitMarkerMetadata(backend: BackendShape, marker: Marker, expected: SparsePcmExpectation, signal: AbortSignal): Effect.Effect { return Effect.fn("SparseProgram.admitMarkerMetadata")(function*() { - if (marker.format !== MARKER_TAG || marker.identity !== expected.identity || marker.sampleRateHz !== expected.sampleRateHz || marker.channels !== expected.channels || marker.bitDepth !== expected.bitDepth || marker.frames !== expected.frames || marker.canonicalBytes !== expected.canonicalBytes || marker.activeBytes !== marker.index.activeBytes || marker.payloadName !== payloadName(expected.identity, marker.generation)) return yield* new SparseCorruptError({ message: "Sparse marker conflicts with the expectation" }); + const metadataContext = storageContext("read", "sparse-marker", "metadata"); + if (marker.format !== MARKER_TAG || marker.identity !== expected.identity || marker.sampleRateHz !== expected.sampleRateHz || marker.channels !== expected.channels || marker.bitDepth !== expected.bitDepth || marker.frames !== expected.frames || marker.canonicalBytes !== expected.canonicalBytes || marker.activeBytes !== marker.index.activeBytes || marker.payloadName !== payloadName(expected.identity, marker.generation)) return yield* new SparseMetadataError({ message: "Sparse marker conflicts with the expectation", context: metadataContext }); compareIndexShape(marker.index, expected); - if (!(yield* backend.exists(marker.payloadName))) return yield* new SparseCorruptError({ message: "Sparse payload generation is missing" }); + if (!(yield* backend.exists(marker.payloadName))) return yield* new SparseMetadataError({ message: "Sparse payload generation is missing", context: storageContext("exists", "sparse-payload", "metadata") }); const data = yield* backend.read(marker.payloadName).pipe( Effect.catchTag("SparseIoError", (error): Effect.Effect => isNotFound(error.cause) - ? Effect.fail(new SparseCorruptError({ message: "Sparse payload generation is missing", cause: error.cause })) + ? Effect.fail(new SparseMetadataError({ message: "Sparse payload generation is missing", cause: error.cause, context: storageContext("read", "sparse-payload", "metadata") })) : Effect.fail(error)), ); - if (data.size !== marker.activeBytes) return yield* new SparseCorruptError({ message: "Sparse payload length conflicts with its index" }); + if (data.size !== marker.activeBytes) return yield* new SparseIntegrityError({ + message: "Sparse payload length conflicts with its index", + context: storageContext("verify", "sparse-payload", "metadata-verification"), + verification: { kind: "length", expected: marker.activeBytes, observed: data.size }, + }); yield* checkSignal(signal); return { marker, data }; })(); @@ -1507,12 +1606,20 @@ function verifyMarker( }).pipe(Effect.mapError(mapWarmWorkerFailure)); const expectedCounts = yield* Effect.try({ try: () => sparseVerifyExpectedCounts(admitted.marker.index, expected.frames, expected.channels * (expected.bitDepth / 8)), - catch: (cause) => new SparseCorruptError({ message: "Sparse Worker count expectation could not be derived", cause }), + catch: (cause) => new SparseBoundaryError({ message: "Sparse Worker count expectation could not be derived", cause, context: storageContext("verify", "sparse-payload", "warm-worker"), failureKind: "adapter" }), }); if (result.identity !== expected.identity || result.canonicalBytes !== expected.canonicalBytes || result.readBytes !== admitted.marker.activeBytes || result.readCalls !== expectedCounts.readCalls || result.hashUpdates !== expectedCounts.hashUpdates || result.zeroUpdates !== expectedCounts.zeroUpdates || result.hashedBytes !== expected.canonicalBytes || result.digest !== identityHex(expected.identity)) { - return yield* new SparseCorruptError({ message: "Sparse payload failed canonical Worker verification" }); + const verification: EngineWebAdapterVerification | undefined = result.identity !== expected.identity || result.digest !== identityHex(expected.identity) + ? { kind: "digest", expected: identityHex(expected.identity), observed: result.digest, algorithm: "blake3-256" } + : result.canonicalBytes !== expected.canonicalBytes || result.hashedBytes !== expected.canonicalBytes + ? { kind: "length", expected: expected.canonicalBytes, observed: result.hashedBytes } + : result.readBytes !== admitted.marker.activeBytes + ? { kind: "length", expected: admitted.marker.activeBytes, observed: result.readBytes } + : undefined; + if (verification !== undefined) return yield* new SparseIntegrityError({ message: "Sparse payload failed canonical Worker verification", verification, context: storageContext("verify", "sparse-payload", "warm-verification") }); + return yield* new SparseIoError({ message: "Sparse verification Worker accounting disagreed with the admitted descriptor", failureKind: "adapter", context: storageContext("verify", "sparse-payload", "warm-verification") }); } yield* checkSignal(signal); onProgress?.({ @@ -1544,7 +1651,7 @@ function verifyMarker( const data = admitted.data; const hash = yield* Effect.tryPromise({ try: createIncrementalBlake3, - catch: (cause) => new SparseBoundaryError({ message: "BLAKE3 verification could not initialize", cause }), + catch: (cause) => new SparseBoundaryError({ message: "BLAKE3 verification could not initialize", cause, context: storageContext("verify", "sparse-payload", "warm-verification"), failureKind: "adapter" }), }); const update = (chunk: Uint8Array): void => { const before = performance.now(); @@ -1567,7 +1674,11 @@ function verifyMarker( readWaitMs += performance.now() - beforeRead; readCalls += 1; readBytes += bytes.byteLength; - if (bytes.byteLength !== end - offset) return yield* new SparseCorruptError({ message: "Sparse payload read was short" }); + if (bytes.byteLength !== end - offset) return yield* new SparseIntegrityError({ + message: "Sparse payload read was short", + context: storageContext("verify", "sparse-payload", "warm-verification"), + verification: { kind: "length", expected: end - offset, observed: bytes.byteLength }, + }); update(new Uint8Array(bytes)); onProgress?.({ stage: "verifying", identity: expected.identity, @@ -1584,7 +1695,16 @@ function verifyMarker( const beforeDigest = performance.now(); const digest = hash.digest("hex"); hashMs += performance.now() - beforeDigest; - if (payloadCursor !== data.size || digest !== identityHex(expected.identity)) return yield* new SparseCorruptError({ message: "Sparse payload failed canonical verification" }); + if (digest !== identityHex(expected.identity)) return yield* new SparseIntegrityError({ + message: "Sparse payload failed canonical verification", + context: storageContext("verify", "sparse-payload", "warm-verification"), + verification: { kind: "digest", expected: identityHex(expected.identity), observed: digest, algorithm: "blake3-256" }, + }); + if (payloadCursor !== data.size) return yield* new SparseIntegrityError({ + message: "Sparse payload length changed during canonical verification", + context: storageContext("verify", "sparse-payload", "warm-verification"), + verification: { kind: "length", expected: data.size, observed: payloadCursor }, + }); yield* checkSignal(signal); onProgress?.({ stage: "verifying", identity: expected.identity, bytes: expected.canonicalBytes, @@ -1598,27 +1718,31 @@ function verifyMarker( } function mapWarmWorkerFailure(cause: unknown): SparseFailure { - if (!(cause instanceof SparseVerifyWorkerError)) return new SparseIoError({ message: "Sparse verification Worker failed", cause }); + if (!(cause instanceof SparseVerifyWorkerError)) return new SparseIoError({ message: "Sparse verification Worker failed", cause, failureKind: "adapter", context: storageContext("verify", "sparse-payload", "warm-worker") }); switch (cause.kind) { - case "cancelled": return new SparseCancelledError({ message: cause.message, cause }); - case "deadline": return new SparseDeadlineError({ message: cause.message, cause }); - case "boundary": return new SparseBoundaryError({ message: cause.message, cause }); - case "corrupt": return new SparseCorruptError({ message: cause.message, cause }); - case "io": return new SparseIoError({ message: cause.message, cause }); + case "cancelled": return new SparseCancelledError({ message: cause.message, cause, failureKind: "interruption", context: storageContext("verify", "sparse-payload", "warm-worker") }); + case "deadline": return new SparseDeadlineError({ message: cause.message, cause, failureKind: "deadline", context: storageContext("verify", "sparse-payload", "warm-worker") }); + case "boundary": return new SparseBoundaryError({ message: cause.message, cause, failureKind: "adapter", context: storageContext("verify", "sparse-payload", "warm-worker") }); + case "corrupt": return !isEngineWebAdapterVerification(cause.verification) + ? new SparseIoError({ message: cause.message, cause, failureKind: "adapter", context: storageContext("verify", "sparse-payload", "warm-worker") }) + : new SparseIntegrityError({ message: cause.message, cause, verification: cause.verification, context: storageContext("verify", "sparse-payload", "warm-worker") }); + case "io": return new SparseIoError({ message: cause.message, cause, failureKind: "adapter", context: storageContext("verify", "sparse-payload", "warm-worker") }); } } function readBlobBytes(backend: BackendShape, blob: Blob, start: number, end: number, signal: AbortSignal, operation: string): Effect.Effect { return Effect.gen(function*() { yield* checkSignal(signal); + const resource = operation.toLowerCase().includes("marker") ? "sparse-marker" : "sparse-payload"; + const context = storageContext("read", resource, "verification"); const bytes = yield* Effect.tryPromise({ try: () => blob.slice(start, end).arrayBuffer(), catch: (cause) => isAbort(cause) - ? new SparseCancelledError({ message: `${operation} was cancelled`, cause }) - : new SparseIoError({ message: `${operation} failed`, cause }), + ? new SparseCancelledError({ message: `${operation} was cancelled`, cause, failureKind: "interruption", context }) + : new SparseIoError({ message: `${operation} failed`, cause, failureKind: "io", context }), }).pipe( Effect.timeout(backend.readDeadlineMs), - Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline` }))), + Effect.catchTag("TimeoutError", () => Effect.fail(new SparseDeadlineError({ message: `${operation} exceeded its deadline`, failureKind: "deadline", context }))), ); yield* checkSignal(signal); return bytes; @@ -1626,26 +1750,34 @@ function readBlobBytes(backend: BackendShape, blob: Blob, start: number, end: nu } function compareIndexShape(index: SparsePcmIndex, expected: SparsePcmExpectation): void { if (index.identity !== expected.identity || index.sampleRateHz !== expected.sampleRateHz || index.channels !== expected.channels || index.bitDepth !== expected.bitDepth || index.frames !== expected.frames || index.canonicalBytes !== expected.canonicalBytes) throw new EngineWebAdapterError("stem.invalid_declaration", "Sparse PCM index shape conflicts with expectation"); } -function compareIndexes(a: SparsePcmIndex, b: SparsePcmIndex): void { if (a.activeBytes !== b.activeBytes || a.canonicalBytes !== b.canonicalBytes || a.intervals.length !== b.intervals.length || a.intervals.some((x, i) => x.startFrame !== b.intervals[i]!.startFrame || x.frames !== b.intervals[i]!.frames || x.byteOffset !== b.intervals[i]!.byteOffset)) throw new EngineWebAdapterError("stem.corrupt", "Sparse resolver index disagrees with derived spans"); } +function compareIndexes(a: SparsePcmIndex, b: SparsePcmIndex): void { if (a.activeBytes !== b.activeBytes || a.canonicalBytes !== b.canonicalBytes || a.intervals.length !== b.intervals.length || a.intervals.some((x, i) => x.startFrame !== b.intervals[i]!.startFrame || x.frames !== b.intervals[i]!.frames || x.byteOffset !== b.intervals[i]!.byteOffset)) throw new EngineWebAdapterError("stem.invalid_declaration", "Sparse resolver index disagrees with derived spans", { metadata: "index" }); } function sameBytes(a: Uint8Array, b: Uint8Array): boolean { return a.byteLength === b.byteLength && a.every((value, index) => value === b[index]); } function quotaCheck(backend: BackendShape, additional: number): Effect.Effect { return Effect.gen(function*() { const estimate = yield* backend.estimate; if (estimate.quota !== undefined && estimate.usage !== undefined && (!Number.isSafeInteger(additional) || additional < 0 || estimate.usage + additional > estimate.quota)) return yield* new SparseQuotaError({ message: "Sparse storage quota is insufficient" }); }); } +function roleFailure(role: "interruption" | "body" | "finalizer", error: unknown): Error { + const wrapped = new Error(error instanceof Error ? error.message : String(error), { cause: error }); + wrapped.name = error instanceof Error ? error.name : "Error"; + Object.defineProperty(wrapped, "role", { value: role, enumerable: true, configurable: true }); + return wrapped; +} function isAbort(error: unknown): boolean { return error instanceof DOMException && error.name === "AbortError" || error instanceof Error && error.name === "AbortError"; } function isTimeout(error: unknown): boolean { return error instanceof DOMException && error.name === "TimeoutError" || error instanceof Error && error.name === "TimeoutError"; } function mapPublicCause(cause: Cause.Cause): unknown { const reasons = cause.reasons; if (reasons.length === 0) { - return new EngineWebAdapterError("stem.corrupt", "Sparse operation failed", {}, new Error("Sparse operation failed without a cause")); + return new EngineWebAdapterError("storage.io", "Sparse operation failed without a cause", { failureKind: "defect" }, new Error("Sparse operation failed without a cause")); } const values = reasons.map((reason) => causeValue(reason)); - const primary = values[0]; - const mapped = mapPublicError(primary); + const mappedValues = values.map((value) => mapPublicError(value)); + // Preserve the Effect cause's primary ordering for the public code. Every + // sibling remains in the AggregateError below; selecting a later, more + // familiar code would flatten a real storage failure into its finalizer. + const mapped = mappedValues[0]; if (mapped instanceof EngineWebAdapterError) { if (values.length === 1) return mapped; return new EngineWebAdapterError(mapped.code, mapped.message, mapped.details, new AggregateError(values, "Sparse operation failed")); } const preserved = values.length === 1 && values[0] !== undefined ? values[0] : new AggregateError(values, "Sparse operation failed"); - const code = Cause.isInterruptReason(reasons[0]!) || isAbort(primary) ? "stem.cancelled" : "stem.corrupt"; - return new EngineWebAdapterError(code, code === "stem.cancelled" ? "Sparse operation was cancelled" : "Sparse operation failed", {}, preserved); + return new EngineWebAdapterError("storage.io", "Sparse operation failed", { failureKind: "defect" }, preserved); } function causeValue(reason: Cause.Reason): unknown { if (Cause.isFailReason(reason)) return reason.error; @@ -1655,26 +1787,73 @@ function causeValue(reason: Cause.Reason): unknown { return interrupted; } function mapPublicError(error: unknown): unknown { - if (error instanceof EngineWebAdapterError) return error; - const preserved = nestedAdapterError(error); - if (preserved !== undefined) return preserved; - if (error instanceof SparseCancelledError) return new EngineWebAdapterError("stem.cancelled", error.message, {}, error.cause); - if (error instanceof SparseDeadlineError) return new EngineWebAdapterError("stem.read_deadline", error.message, {}, error.cause); - if (error instanceof SparseQuotaError) return new EngineWebAdapterError("stem.quota", error.message, {}, error.cause); - if (error instanceof SparseBoundaryError) return new EngineWebAdapterError("stem.invalid_declaration", error.message, {}, error.cause); - if (error instanceof SparseNotFoundError) return new EngineWebAdapterError("stem.not_found", error.message, {}, error.cause); - if (error instanceof SparseConflictError || error instanceof SparseCorruptError) return new EngineWebAdapterError("stem.corrupt", error.message, {}, error.cause); - if (error instanceof SparseIoError) return new EngineWebAdapterError("stem.corrupt", error.message, {}, error.cause); - return isAbort(error) ? new EngineWebAdapterError("stem.cancelled", "Sparse operation was cancelled", {}, error) : error; + const inheritedDetails = sparseFailureDetails(error); + if (error instanceof EngineWebAdapterError) return mapTypedPublicError(error, inheritedDetails); + const preserved = directNestedAdapterError(error); + if (preserved !== undefined) return mapTypedPublicError(preserved, inheritedDetails); + if (error instanceof SparseCancelledError) return new EngineWebAdapterError("stem.cancelled", error.message, { ...inheritedDetails, failureKind: "interruption" }, error.cause); + if (error instanceof SparseDeadlineError) return new EngineWebAdapterError("stem.read_deadline", error.message, { ...inheritedDetails, failureKind: "deadline" }, error.cause); + if (error instanceof SparseQuotaError) return new EngineWebAdapterError("stem.quota", error.message, { ...inheritedDetails, failureKind: "quota" }, error.cause); + if (error instanceof SparseBoundaryError) { + return error.failureKind === "adapter" + ? new EngineWebAdapterError("storage.io", error.message, { ...inheritedDetails, failureKind: "adapter" }, error.cause) + : new EngineWebAdapterError("stem.invalid_declaration", error.message, inheritedDetails, error.cause); + } + if (error instanceof SparseMetadataError) return new EngineWebAdapterError("stem.invalid_declaration", error.message, { ...inheritedDetails, failureKind: "invalid_metadata" }, error.cause); + if (error instanceof SparseIntegrityError) { + return !isEngineWebAdapterVerification(error.verification) + ? new EngineWebAdapterError("storage.io", error.message, { ...inheritedDetails, failureKind: "adapter" }, error.cause) + : new EngineWebAdapterError("stem.corrupt", error.message, inheritedDetails, error.cause); + } + if (error instanceof SparseNotFoundError) return new EngineWebAdapterError("stem.not_found", error.message, inheritedDetails, error.cause); + if (error instanceof SparseConflictError) return new EngineWebAdapterError("storage.io", error.message, { ...inheritedDetails, failureKind: "contention" }, error.cause); + if (error instanceof SparseIoError) return new EngineWebAdapterError("storage.io", error.message, inheritedDetails, error.cause); + return isAbort(error) + ? new EngineWebAdapterError("stem.cancelled", "Sparse operation was cancelled", { failureKind: "interruption" }, error) + : new EngineWebAdapterError("storage.io", "Sparse operation failed", { failureKind: "defect" }, error); } -function nestedAdapterError(error: unknown): EngineWebAdapterError | undefined { - let current: unknown = error; - for (let depth = 0; depth < 4; depth += 1) { - if (current instanceof EngineWebAdapterError) return current; - if (typeof current !== "object" || current === null || !("cause" in current)) return undefined; - current = (current as { readonly cause?: unknown }).cause; + +function mapTypedPublicError(error: EngineWebAdapterError, inheritedDetails: Readonly>): EngineWebAdapterError { + if (error.code === "stem.corrupt" && !isEngineWebAdapterVerification(error.details.verification)) { + return new EngineWebAdapterError("storage.io", error.message, { + ...error.details, + ...inheritedDetails, + failureKind: "adapter", + }, error); } - return current instanceof EngineWebAdapterError ? current : undefined; + return mergePublicError(error, inheritedDetails); +} + +function mergePublicError(error: EngineWebAdapterError, details: Readonly>): EngineWebAdapterError { + if (Object.keys(details).length === 0) return error; + return new EngineWebAdapterError(error.code, error.message, { ...error.details, ...details }, error.cause); +} + +function sparseFailureDetails(error: unknown): Readonly> { + if (typeof error !== "object" || error === null) return {}; + const value = error as { readonly context?: unknown; readonly verification?: unknown; readonly failureKind?: unknown }; + const details: Record = {}; + if (isStorageContext(value.context)) { + details.operation = value.context.operation; + details.resource = value.context.resource; + details.phase = value.context.phase; + } + if (isEngineWebAdapterVerification(value.verification)) details.verification = value.verification; + if (typeof value.failureKind === "string") details.failureKind = value.failureKind; + return details; +} + +function isStorageContext(value: unknown): value is EngineWebAdapterErrorContext { + return typeof value === "object" && value !== null && + typeof (value as { readonly operation?: unknown }).operation === "string" && + typeof (value as { readonly resource?: unknown }).resource === "string" && + typeof (value as { readonly phase?: unknown }).phase === "string"; +} + +function directNestedAdapterError(error: unknown): EngineWebAdapterError | undefined { + if (!(error instanceof SparseIoError || error instanceof SparseBoundaryError)) return undefined; + const cause = error.cause; + return cause instanceof EngineWebAdapterError && cause.code !== "stem.corrupt" ? cause : undefined; } function isNotFound(error: unknown): boolean { return typeof error === "object" && error !== null && "name" in error && (error as { readonly name?: unknown }).name === "NotFoundError"; diff --git a/src/stems/sparse-verify-worker-pool.ts b/src/stems/sparse-verify-worker-pool.ts index b8b2b32..bb22d77 100644 --- a/src/stems/sparse-verify-worker-pool.ts +++ b/src/stems/sparse-verify-worker-pool.ts @@ -1,5 +1,6 @@ import { Effect } from "effect"; +import { reviveEngineWebAdapterCause } from "../errors.js"; import { createSparseVerifyWorker } from "./sparse-verify-worker-asset.js"; import { verifiedSparseVerifyArrayBufferTransfer, type SparseVerifyArrayBufferTransfer } from "./sparse-verify-buffer.js"; import { @@ -10,6 +11,7 @@ import { sparseVerifyWorkerResponse, type SparseVerifyComplete, type SparseVerifyFailureKind, + type SparseVerifyFailure, type SparseVerifyStart, type SparseVerifyWorkerResponse, } from "./sparse-verify-worker-protocol.js"; @@ -18,11 +20,13 @@ import type { StemIdentity } from "./types.js"; export class SparseVerifyWorkerError extends Error { readonly kind: SparseVerifyFailureKind; + readonly verification: SparseVerifyFailure["verification"]; - constructor(kind: SparseVerifyFailureKind, message: string, cause?: unknown) { + constructor(kind: SparseVerifyFailureKind, message: string, cause?: unknown, verification?: SparseVerifyFailure["verification"]) { super(message, cause === undefined ? undefined : { cause }); this.name = "SparseVerifyWorkerError"; this.kind = kind; + this.verification = verification; } } @@ -321,7 +325,12 @@ export class SparseVerifyWorkerPool { return; } if (response.type === "failure") { - this.failSlot(slot, generation, new SparseVerifyWorkerError(response.kind, response.message)); + this.failSlot(slot, generation, new SparseVerifyWorkerError( + response.kind, + response.message, + response.cause === undefined ? undefined : reviveEngineWebAdapterCause(response.cause), + response.verification, + )); return; } let expectedCounts: ReturnType; @@ -330,10 +339,23 @@ export class SparseVerifyWorkerPool { this.failSlot(slot, generation, new SparseVerifyWorkerError("corrupt", "Sparse verification count expectation was invalid", cause)); return; } - if (response.identity !== job.options.identity || response.progressBytes !== job.options.canonicalBytes || response.canonicalBytes !== job.options.canonicalBytes || - response.readBytes !== job.options.index.activeBytes || response.hashedBytes !== job.options.canonicalBytes || - response.readCalls !== expectedCounts.readCalls || response.hashUpdates !== expectedCounts.hashUpdates || response.zeroUpdates !== expectedCounts.zeroUpdates) { - this.failSlot(slot, generation, new SparseVerifyWorkerError("corrupt", "Sparse verification Worker counts or identity disagreed with the admitted descriptor")); + const expectedDigest = job.options.identity.slice("blake3:".length); + const verification = response.digest !== expectedDigest + ? { kind: "digest" as const, expected: expectedDigest, observed: response.digest, algorithm: "blake3-256" } + : response.progressBytes !== job.options.canonicalBytes || response.canonicalBytes !== job.options.canonicalBytes || response.hashedBytes !== job.options.canonicalBytes + ? { kind: "length" as const, expected: job.options.canonicalBytes, observed: response.hashedBytes } + : response.readBytes !== job.options.index.activeBytes + ? { kind: "length" as const, expected: job.options.index.activeBytes, observed: response.readBytes } + : undefined; + if (verification !== undefined || response.identity !== job.options.identity || response.readCalls !== expectedCounts.readCalls || response.hashUpdates !== expectedCounts.hashUpdates || response.zeroUpdates !== expectedCounts.zeroUpdates) { + this.failSlot(slot, generation, new SparseVerifyWorkerError( + "corrupt", + verification === undefined && response.identity === job.options.identity + ? "Sparse verification Worker accounting disagreed with the admitted descriptor" + : "Sparse verification Worker result did not match the admitted content", + undefined, + verification, + )); return; } this.resetWatchdog(slot, generation, job.options.readDeadlineMs); diff --git a/src/stems/sparse-verify-worker-protocol.ts b/src/stems/sparse-verify-worker-protocol.ts index 104dc4c..84647c8 100644 --- a/src/stems/sparse-verify-worker-protocol.ts +++ b/src/stems/sparse-verify-worker-protocol.ts @@ -1,3 +1,5 @@ +import { isEngineWebAdapterVerification } from "../errors.js"; +import type { EngineWebAdapterErrorCause, EngineWebAdapterVerification } from "../errors.js"; import type { SparsePcmIndex } from "./sparse-pcm.js"; import type { StemIdentity } from "./types.js"; @@ -80,6 +82,8 @@ export interface SparseVerifyFailure { readonly generation: number; readonly kind: SparseVerifyFailureKind; readonly message: string; + readonly verification?: EngineWebAdapterVerification; + readonly cause?: EngineWebAdapterErrorCause; } export type SparseVerifyWorkerRequest = SparseVerifyStart | SparseVerifyAck | SparseVerifyCancel; @@ -202,10 +206,27 @@ export function sparseVerifyWorkerResponse(value: unknown): SparseVerifyWorkerRe if (!["boundary", "corrupt", "deadline", "io", "cancelled"].includes(response.kind) || typeof response.message !== "string") { throw new Error("Sparse verification worker failure is invalid"); } + if (response.verification !== undefined) validateVerification(response.verification); + if (response.cause !== undefined) validateCause(response.cause); } return response; } +function validateVerification(value: unknown): asserts value is EngineWebAdapterVerification { + if (!isEngineWebAdapterVerification(value)) { + throw new Error("Sparse verification evidence is invalid"); + } +} + +function validateCause(value: unknown, depth = 0): asserts value is EngineWebAdapterErrorCause { + if (depth > 4 || !record(value) || typeof value.name !== "string" || typeof value.message !== "string") throw new Error("Sparse verification cause is invalid"); + if (value.cause !== undefined) validateCause(value.cause, depth + 1); + if (value.causes !== undefined) { + if (!Array.isArray(value.causes) || value.causes.length > 8) throw new Error("Sparse verification causes exceed their bound"); + for (const cause of value.causes) validateCause(cause, depth + 1); + } +} + function record(value: unknown): value is Record { return typeof value === "object" && value !== null && !Array.isArray(value); } diff --git a/src/stems/store.ts b/src/stems/store.ts index 39fda4c..39ba669 100644 --- a/src/stems/store.ts +++ b/src/stems/store.ts @@ -107,7 +107,9 @@ export class VerifiedStemStore implements StemStore { await this.#mutateIndex((index) => { const row = index.stems[stem.identity]; if (row === undefined || row.bytes !== stem.bytes) { - throw new EngineWebAdapterError("stem.corrupt", "Verified stem disappeared before leasing", { identity: stem.identity }); + throw new EngineWebAdapterError("storage.io", "Verified stem disappeared before leasing", { + identity: stem.identity, operation: "read", resource: "dense-stem", phase: "lease", failureKind: "io", + }); } row.pins.push(pin); row.lastUsedAt = this.#now(); @@ -180,7 +182,9 @@ export class VerifiedStemStore implements StemStore { return await deadline(this.#backend.read(finalName(identity)), this.#readDeadlineMs); } catch (error) { await this.#demote(identity).catch(() => undefined); - throw classify(error, "stem.corrupt", `Stored stem cannot be read: ${identity}`, { identity }); + throw classify(error, "storage.io", `Stored stem cannot be read: ${identity}`, { + identity, operation: "read", resource: "dense-stem", phase: "storage-read", + }); } } @@ -372,7 +376,7 @@ export class VerifiedStemStore implements StemStore { } const observed = hash?.digest("hex") ?? packageResult?.digest?.(); if (bytes !== stem.bytes || observed !== digest(stem.identity)) { - throw integrity(stem, bytes, observed !== digest(stem.identity) ? "BLAKE3 mismatch" : "truncated stream"); + throw integrity(stem, bytes, observed !== digest(stem.identity) ? "BLAKE3 mismatch" : "truncated stream", observed); } signal.throwIfAborted(); await writeOperation(() => writer!.close()); @@ -466,7 +470,7 @@ export class VerifiedStemStore implements StemStore { } } if (typeof parsed === "object" && parsed !== null && "version" in parsed && parsed.version !== INDEX_VERSION) { - throw new EngineWebAdapterError("stem.corrupt", "Unsupported stem index version", { version: parsed.version }); + throw new EngineWebAdapterError("stem.invalid_declaration", "Unsupported stem index version", { version: parsed.version, metadata: "index" }); } if (validIndex(parsed)) return parsed; return this.#rebuildIndex(liveLocks ?? await this.#liveLockNames()); @@ -671,9 +675,22 @@ async function runBounded( await Promise.all(Array.from({ length: Math.min(width, values.length) }, next)); if (failed) throw firstError; } -function integrity(stem: StemRequirement, observedBytes: number, reason: string): EngineWebAdapterError { +function integrity(stem: StemRequirement, observedBytes: number, reason: string, observedDigest?: string): EngineWebAdapterError { + const expectedDigest = digest(stem.identity); + if (reason === "BLAKE3 mismatch" && observedDigest === undefined) { + return new EngineWebAdapterError("storage.io", `Resolved stem could not expose verification evidence: ${reason}`, { + identity: stem.identity, operation: "verify", resource: "dense-stem", phase: "verification", failureKind: "adapter", + }); + } + const verification = reason === "BLAKE3 mismatch" && observedDigest !== undefined + ? { kind: "digest" as const, expected: expectedDigest, observed: observedDigest, algorithm: "blake3-256" } + : { kind: "length" as const, expected: stem.bytes, observed: observedBytes }; return new EngineWebAdapterError("stem.corrupt", `Resolved stem failed verification: ${reason}`, { - identity: stem.identity, expectedBytes: stem.bytes, observedBytes, reason, + identity: stem.identity, + expectedBytes: stem.bytes, + observedBytes, + reason, + verification, }); } function emptyIndex(): StoreIndex { return { version: INDEX_VERSION, stems: {} }; } diff --git a/tests/console.test.ts b/tests/console.test.ts index 6b9f165..a777b21 100644 --- a/tests/console.test.ts +++ b/tests/console.test.ts @@ -648,7 +648,7 @@ function allCodes(): readonly EngineWebAdapterError["code"][] { "console.lease_refused": true, "console.not_attached": true, "console.refused": true, "session.closed": true, "session.busy": true, "session.seek": true, "session.declaration_mismatch": true, "session.input_path": true, - "session.open": true, "session.playback": true, "stem.cancelled": true, "stem.corrupt": true, "stem.decode.asset": true, + "session.open": true, "session.playback": true, "storage.io": true, "stem.cancelled": true, "stem.corrupt": true, "stem.decode.asset": true, "stem.decode.flac": true, "stem.decode.output": true, "stem.decode.stall": true, "stem.decode.worker": true, "stem.delivery.address": true, "stem.delivery.http": true, "stem.delivery.range": true, "stem.delivery.retry_exhausted": true, "stem.delivery.stall": true, diff --git a/tests/error-contract.test.ts b/tests/error-contract.test.ts new file mode 100644 index 0000000..35bcae4 --- /dev/null +++ b/tests/error-contract.test.ts @@ -0,0 +1,316 @@ +import assert from "node:assert/strict"; +import { readFile } from "node:fs/promises"; +import test from "node:test"; + +import { + EngineWebAdapterError, + type EngineWebAdapterErrorCode, + type EngineWebAdapterErrorCause, +} from "../src/index.js"; +import { reviveEngineWebAdapterCause, serializeEngineWebAdapterCause } from "../src/errors.js"; + +const fixture = JSON.parse(await readFile("tests/fixtures/storage-error-contract.json", "utf8")) as { + readonly name: string; + readonly message: string; + readonly code: EngineWebAdapterErrorCode; + readonly phase: string; + readonly details: Readonly>; + readonly cause: EngineWebAdapterErrorCause; +}; + +test("public storage failure fixture keeps coarse and granular context separate", () => { + const browserCause = new Error(fixture.cause.message); + browserCause.name = fixture.cause.name; + const error = new EngineWebAdapterError(fixture.code, fixture.message, fixture.details, browserCause); + assert.equal(error.name, fixture.name); + assert.equal(error.code, "storage.io"); + assert.equal(error.phase, fixture.phase); + assert.equal(error.details.operation, "open"); + assert.equal(error.details.resource, "sparse-root"); + assert.equal(error.details.phase, "root-open"); + assert.equal(error.details.verification, undefined); + assert.equal((error.cause as Error).name, "UnknownError"); +}); + +test("cause transport bounds strings, keys, total payload, and serialized aggregates", () => { + const corrupt = new EngineWebAdapterError("stem.corrupt", "verified mismatch", { + verification: { kind: "digest", expected: "a".repeat(64), observed: "b".repeat(64), algorithm: "blake3-256" }, + detail: "x".repeat(1_000_000), + }); + const aggregate = new AggregateError([ + new Error("BODY_IO"), + new Error("wrapper", { cause: corrupt }), + new Error("sibling"), + ], "triple aggregate"); + const serialized = serializeEngineWebAdapterCause(aggregate); + assert.ok(JSON.stringify(serialized).length < 16_384); + assert.equal(serialized.causes?.length, 3); + const wrapped = serialized.causes?.[1]; + const nested = wrapped?.cause; + assert.equal(nested?.code, "stem.corrupt"); + assert.deepEqual(nested?.details?.verification, corrupt.details.verification); + + const plainSerialized = { + name: "AggregateError", + message: "serialized aggregate", + causes: [ + { name: "BODY_IO", message: "body" }, + serialized.causes![1]!, + { name: "FINALIZER", message: "cleanup" }, + ], + } satisfies EngineWebAdapterErrorCause; + const reserialized = serializeEngineWebAdapterCause(plainSerialized); + assert.equal(reserialized.causes?.length, 3); + assert.equal(reserialized.causes?.[1]?.cause?.code, "stem.corrupt"); + assert.deepEqual(reserialized.causes?.[1]?.cause?.details?.verification, corrupt.details.verification); + const revived = reviveEngineWebAdapterCause(reserialized) as AggregateError; + assert.equal(revived.errors.length, 3); +}); + +test("cause transport uses UTF-8 byte bounds and keeps bounded sibling summaries", () => { + const details = Object.fromEntries(Array.from({ length: 24 }, (_, index) => [`detail-${index}`, "漢".repeat(512)])); + const bounded = serializeEngineWebAdapterCause(new EngineWebAdapterError("storage.io", "wide details", details)); + assert.ok(new TextEncoder().encode(JSON.stringify(bounded)).byteLength <= 16 * 1024); + assert.equal(bounded.truncated, true); + + const siblings = Array.from({ length: 8 }, (_, index) => new EngineWebAdapterError("storage.io", `sibling-${index}`, Object.fromEntries( + Array.from({ length: 6 }, (_, detail) => [`detail-${detail}`, "x".repeat(483)]), + ))); + const first = serializeEngineWebAdapterCause(new AggregateError(siblings, "many siblings")); + const second = serializeEngineWebAdapterCause(first); + assert.deepEqual(first.causes?.map((cause) => cause.message), siblings.map((error) => error.message)); + assert.deepEqual(second.causes?.map((cause) => cause.message), siblings.map((error) => error.message)); + assert.ok(new TextEncoder().encode(JSON.stringify(second)).byteLength <= 16 * 1024); + const revived = reviveEngineWebAdapterCause(second) as AggregateError; + const revivedSerialized = serializeEngineWebAdapterCause(revived); + assert.deepEqual(revivedSerialized.causes?.map((cause) => cause.message), siblings.map((error) => error.message)); + + const plainErrors = serializeEngineWebAdapterCause({ + name: "AggregateError", + message: "plain errors", + errors: [{ name: "BODY", message: "body" }, { name: "FINALIZER", message: "cleanup" }], + }); + assert.deepEqual(plainErrors.causes?.map((cause) => cause.name), ["BODY", "FINALIZER"]); + + const clipped = serializeEngineWebAdapterCause(new Error("m".repeat(2_000), { + cause: new EngineWebAdapterError("storage.io", "key clipping", { ["k".repeat(2_000)]: "v".repeat(2_000) }), + })); + assert.equal(clipped.truncated, true); + assert.match(clipped.message, /\[truncated\]$/u); + const clippedDetails = Object.entries(clipped.cause?.details ?? {}); + assert.ok(clippedDetails.some(([key, value]) => key.endsWith("[truncated]") && value === "v".repeat(501) + "[truncated]")); + + const clippedProof = serializeEngineWebAdapterCause(new EngineWebAdapterError("stem.corrupt", "clipped proof", { + verification: { kind: "digest", expected: "漢".repeat(512), observed: "異".repeat(512), algorithm: "blake3-256" }, + })); + assert.equal(clippedProof.code, undefined); + assert.equal(clippedProof.details?.serialization, "integrity-evidence-insufficient"); +}); + +test("cause transport marks depth/claim truncation and survives throwing exotic access", () => { + const leaf = new EngineWebAdapterError("stem.corrupt", "verified mismatch", { + verification: { kind: "length", expected: 4, observed: 3 }, + }); + let supported: unknown = leaf; + for (let index = 0; index < 3; index += 1) supported = new Error(`supported-${index}`, { cause: supported }); + const supportedCause = serializeEngineWebAdapterCause(supported); + const supportedLeaf = supportedCause.cause?.cause?.cause; + assert.equal(supportedLeaf?.code, "stem.corrupt"); + assert.deepEqual(supportedLeaf?.details?.verification, leaf.details.verification); + + let value: unknown = leaf; + for (let index = 0; index < 5; index += 1) value = new Error(`wrapper-${index}`, { cause: value }); + const deep = serializeEngineWebAdapterCause(value); + assert.equal(deep.truncated, true); + const exotic = new Proxy({}, { + get() { throw new Error("getter exploded"); }, + has() { throw new Error("has exploded"); }, + ownKeys() { throw new Error("keys exploded"); }, + }); + assert.doesNotThrow(() => serializeEngineWebAdapterCause(exotic)); + + const codeOnly = serializeEngineWebAdapterCause(new EngineWebAdapterError("stem.corrupt", "no proof")); + assert.equal(codeOnly.code, undefined); + assert.equal(codeOnly.truncated, true); + assert.equal(codeOnly.details?.serialization, "integrity-evidence-insufficient"); + const equalEvidence = serializeEngineWebAdapterCause(new EngineWebAdapterError("stem.corrupt", "no mismatch", { + verification: { kind: "length", expected: 4, observed: 4 }, + })); + assert.equal(equalEvidence.code, undefined); +}); + +test("cause transport reserves every retained sibling summary before detail pressure", () => { + const detailPressure = Object.fromEntries(Array.from({ length: 24 }, (_, index) => [`d${index}`, "x".repeat(483)])); + const identity = (cause: EngineWebAdapterErrorCause) => cause.causes?.map((sibling) => [sibling.name, sibling.message]); + const assertRoundTrips = (serialized: EngineWebAdapterErrorCause, expected: readonly (readonly [string, string])[]) => { + assert.equal(serialized.causes?.length, expected.length); + assert.deepEqual(identity(serialized), expected); + assert.ok(serialized.causes?.every((sibling) => sibling.name.length > 0 && sibling.message.length > 0)); + assert.ok(new TextEncoder().encode(JSON.stringify(serialized)).byteLength <= 16 * 1024); + + const repeated = serializeEngineWebAdapterCause(serialized); + assert.deepEqual(identity(repeated), expected); + const revived = reviveEngineWebAdapterCause(repeated) as AggregateError; + assert.deepEqual(identity(serializeEngineWebAdapterCause(revived)), expected); + }; + + const wide = serializeEngineWebAdapterCause({ + name: "AggregateError", + message: "root", + errors: Array.from({ length: 3 }, (_, index) => ({ + name: `N${index}${"漢".repeat(512)}`, + message: `S${index}${"漢".repeat(512)}`, + details: detailPressure, + })), + }); + const wideExpected = wide.causes!.map((sibling) => [sibling.name, sibling.message] as const); + assert.ok(wideExpected.every(([name, message], index) => + name.startsWith(`N${index}`) && name.endsWith("[truncated]") && + message.startsWith(`S${index}`) && message.endsWith("[truncated]"))); + assertRoundTrips(wide, wideExpected); + assert.equal(wide.truncated, true); + + const ordinary = serializeEngineWebAdapterCause({ + name: "AggregateError", + message: "root", + errors: Array.from({ length: 3 }, (_, index) => ({ + name: "Error", + message: `S${index}${"漢".repeat(512)}`, + details: detailPressure, + })), + }); + const ordinaryExpected = ordinary.causes!.map((sibling) => [sibling.name, sibling.message] as const); + assert.ok(ordinaryExpected.every(([name, message], index) => + name === "Error" && message.startsWith(`S${index}`) && message.endsWith("[truncated]"))); + assertRoundTrips(ordinary, ordinaryExpected); + + const cases = [ + { count: 1, glyph: "x", repeats: 32, detailKeys: 0, detailLength: 0 }, + { count: 2, glyph: "漢", repeats: 256, detailKeys: 8, detailLength: 192 }, + { count: 4, glyph: "𐍈", repeats: 128, detailKeys: 24, detailLength: 320 }, + ] as const; + for (const { count, glyph, repeats, detailKeys, detailLength } of cases) { + const details = Object.fromEntries(Array.from({ length: detailKeys }, (_, index) => [`d${index}`, "v".repeat(detailLength)])); + const serialized = serializeEngineWebAdapterCause(new AggregateError( + Array.from({ length: count }, (_, index) => ({ + name: `N${index}${glyph.repeat(repeats)}`, + message: `S${index}${glyph.repeat(repeats)}`, + details, + })), + `table-${count}`, + )); + const expected = serialized.causes!.map((sibling) => [sibling.name, sibling.message] as const); + assert.ok(expected.every(([name, message], index) => + name.startsWith(`N${index}`) && message.startsWith(`S${index}`))); + assertRoundTrips(serialized, expected); + } +}); + +const serializedCauseBytes = (cause: EngineWebAdapterErrorCause): number => + new TextEncoder().encode(JSON.stringify(cause)).byteLength; + +function assertUsefulCauseSummaries(cause: EngineWebAdapterErrorCause, path = "root"): void { + assert.equal(typeof cause.name, "string", `${path} name must be present`); + assert.equal(typeof cause.message, "string", `${path} message must be present`); + assert.ok(cause.name.length > 0, `${path} name must be useful`); + assert.ok(cause.message.length > 0, `${path} message must be useful`); + for (const [index, child] of (cause.causes ?? []).entries()) { + assertUsefulCauseSummaries(child, `${path}.causes[${index}]`); + } + if (cause.cause !== undefined) assertUsefulCauseSummaries(cause.cause, `${path}.cause`); +} + +function causeIdentity(cause: EngineWebAdapterErrorCause): unknown { + return { + name: cause.name, + message: cause.message, + causes: cause.causes?.map(causeIdentity), + cause: cause.cause === undefined ? undefined : causeIdentity(cause.cause), + }; +} + +test("cause transport globally reserves nested direct and aggregate identities across the pad boundary", () => { + const detailPressure = Object.fromEntries(Array.from({ length: 10 }, (_, index) => [`d${index}`, "漢".repeat(512)])); + const build = (edge: "cause" | "causes", pad: number): Record => { + const child = { name: "Child", message: "child" }; + const parent: Record = { + name: `N${"漢".repeat(512)}`, + message: `S${"漢".repeat(512)}`, + details: { ...detailPressure, pad: "x".repeat(pad) }, + }; + if (edge === "cause") parent.cause = child; + else parent.causes = [child]; + return { name: "AggregateError", message: "root", errors: [parent] }; + }; + + for (const edge of ["cause", "causes"] as const) { + for (let pad = 0; pad <= 512; pad += 1) { + const first = serializeEngineWebAdapterCause(build(edge, pad)); + assert.ok(serializedCauseBytes(first) <= 16 * 1024, `${edge} pad ${pad} exceeded the byte bound`); + assertUsefulCauseSummaries(first); + const parent = first.causes?.[0]; + assert.ok(parent, `${edge} pad ${pad} lost its retained parent`); + const retainedChild = edge === "cause" ? parent.cause : parent.causes?.[0]; + assert.deepEqual(retainedChild && { name: retainedChild.name, message: retainedChild.message }, { + name: "Child", + message: "child", + }, `${edge} pad ${pad} lost its child identity`); + + const repeated = serializeEngineWebAdapterCause(first); + assertUsefulCauseSummaries(repeated); + assert.deepEqual(causeIdentity(repeated), causeIdentity(first), `${edge} pad ${pad} changed on repeat`); + const revived = serializeEngineWebAdapterCause(reviveEngineWebAdapterCause(repeated)); + assertUsefulCauseSummaries(revived); + assert.deepEqual(causeIdentity(revived), causeIdentity(first), `${edge} pad ${pad} changed after revival`); + if (pad === 252 || pad === 270) { + assert.equal(retainedChild?.name, "Child"); + assert.equal(retainedChild?.message, "child"); + } + } + } +}); + +test("cause transport keeps a bounded generated tree valid for depth, width, Unicode, and JSON escapes", () => { + const tokens = ["plain", "漢", "𐍈", "\"\\\n\t\u0000"]; + const detailFor = (token: string, depth: number): Readonly> => Object.fromEntries( + Array.from({ length: 10 }, (_, index) => [ + `k${index}-${token}`, + `${token}${"漢".repeat(256)}-${depth}-${"\\\"\n".repeat(12)}`, + ]), + ); + const build = (depth: number, width: number, token: string, edge: "cause" | "causes"): Record => { + const node: Record = { + name: `N${depth}-${token}`, + message: `M${depth}-${token}`, + details: { ...detailFor(token, depth), pad: "x".repeat(512) }, + }; + if (depth > 0) { + const children = Array.from({ length: width }, () => build(depth - 1, width, token, edge)); + if (edge === "cause") node.cause = children[0]; + else node.causes = children; + } + return node; + }; + + const cases = tokens.flatMap((token, index) => [ + { depth: index + 3, width: 1, edge: index % 2 === 0 ? ("cause" as const) : ("causes" as const) }, + { depth: 2, width: 8, edge: "causes" as const }, + ]); + for (const { depth, width, edge } of cases) { + const first = serializeEngineWebAdapterCause({ + name: "AggregateError", + message: `root-${tokens[depth % tokens.length]}`, + errors: [build(depth, width, tokens[depth % tokens.length]!, edge)], + }); + assert.ok(serializedCauseBytes(first) <= 16 * 1024, `generated depth ${depth} width ${width} exceeded the byte bound`); + assertUsefulCauseSummaries(first); + const repeated = serializeEngineWebAdapterCause(first); + assert.ok(serializedCauseBytes(repeated) <= 16 * 1024); + assertUsefulCauseSummaries(repeated); + assert.deepEqual(causeIdentity(repeated), causeIdentity(first)); + const revived = serializeEngineWebAdapterCause(reviveEngineWebAdapterCause(repeated)); + assert.ok(serializedCauseBytes(revived) <= 16 * 1024); + assertUsefulCauseSummaries(revived); + assert.deepEqual(causeIdentity(revived), causeIdentity(first)); + } +}); diff --git a/tests/fixtures/storage-error-contract.json b/tests/fixtures/storage-error-contract.json new file mode 100644 index 0000000..98295e5 --- /dev/null +++ b/tests/fixtures/storage-error-contract.json @@ -0,0 +1,16 @@ +{ + "name": "EngineWebAdapterError", + "message": "Sparse root access failed", + "code": "storage.io", + "phase": "store", + "details": { + "operation": "open", + "resource": "sparse-root", + "phase": "root-open", + "failureKind": "io" + }, + "cause": { + "name": "UnknownError", + "message": "Storage access was refused" + } +} diff --git a/tests/flac-delivery-worker.test.ts b/tests/flac-delivery-worker.test.ts index 12ec276..577b5fd 100644 --- a/tests/flac-delivery-worker.test.ts +++ b/tests/flac-delivery-worker.test.ts @@ -658,7 +658,8 @@ test("private completion checks host PCM hash and strict final counts", async () first.value!.release(); const requestId = worker.posted.find((message): message is Extract => message.type === "start")!.requestId; worker.emit({ type: "complete", requestId, pcmBytes: 4, frames: problem === "count" ? 1 : 2 }); - await assert.rejects(reader.read(), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(reader.read(), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + (error.details.verification as { readonly kind?: unknown } | undefined)?.kind === (problem === "hash" ? "digest" : "length")); assert.equal(worker.terminated, true); } }); diff --git a/tests/opfs-storage.test.ts b/tests/opfs-storage.test.ts index 3d7585e..8ddce5a 100644 --- a/tests/opfs-storage.test.ts +++ b/tests/opfs-storage.test.ts @@ -6,7 +6,7 @@ import { getEventListeners } from "node:events"; import { EngineWebAdapterError } from "../src/errors.js"; import { OpfsStorageBackend } from "../src/stems/storage.js"; import { OpfsWriteWorkerClient } from "../src/stems/opfs-worker-client.js"; -import { VerifiedStemStore } from "../src/stems/store.js"; +import { VerifiedSparsePcmStore, VerifiedStemStore } from "../src/stems/index.js"; import type { OpfsWorkerLike, OpfsWorkerRequest, OpfsWorkerResponse } from "../src/stems/opfs-worker-protocol.js"; import type { StemIdentity } from "../src/stems/types.js"; @@ -21,18 +21,33 @@ const identityFor = (bytes: Uint8Array) => `blake3:${identityHasher.init().updat interface FakeEntry { bytes: Uint8Array } +interface FakeFaults { + directoryOpen?: Error; + writerOpen?: Error; + truncate?: Error; + read?: Error; + write?: Error; + shortWrite?: number; + flush?: Error; + close?: Error; +} + class FakeDirectory { readonly kind = "directory" as const; readonly files = new Map(); readonly directories = new Map(); readonly locked = new Set(); + readonly faults: FakeFaults; handlesWithoutSyncAccess = false; + constructor(faults: FakeFaults = {}) { this.faults = faults; } + async getDirectoryHandle(name: string, options: { create?: boolean } = {}): Promise { + if (this.faults.directoryOpen !== undefined) throw this.faults.directoryOpen; const existing = this.directories.get(name); if (existing !== undefined) return existing; if (options.create !== true) throw named(`${name} was not found`, "NotFoundError"); - const created = new FakeDirectory(); + const created = new FakeDirectory(this.faults); created.handlesWithoutSyncAccess = this.handlesWithoutSyncAccess; this.directories.set(name, created); return created; @@ -66,9 +81,11 @@ class FakeDirectory { class FakeFileHandle { constructor(private readonly directory: FakeDirectory, private readonly name: string) {} - async getFile(): Promise<{ stream(): ReadableStream; text(): Promise }> { + async getFile(): Promise<{ readonly size: number; stream(): ReadableStream; text(): Promise }> { + if (this.directory.faults.read !== undefined) throw this.directory.faults.read; const bytes = this.#entry().bytes.slice(); return { + size: bytes.byteLength, stream: () => new ReadableStream({ start(controller) { controller.enqueue(bytes); controller.close(); }, }), @@ -82,6 +99,7 @@ class FakeFileHandle { flush(): void; close(): void; }> { + if (this.directory.faults.writerOpen !== undefined) throw this.directory.faults.writerOpen; if (this.directory.locked.has(this.name)) throw named(`${this.name} is locked`, "NoModificationAllowedError"); this.directory.locked.add(this.name); const entry = this.#entry(); @@ -90,6 +108,7 @@ class FakeFileHandle { let closed = false; return { write: (data, options = {}) => { + if (directory.faults.write !== undefined) throw directory.faults.write; if (closed) throw named("handle is closed", "InvalidStateError"); const at = options.at ?? 0; const size = Math.max(entry.bytes.byteLength, at + data.byteLength); @@ -97,14 +116,22 @@ class FakeFileHandle { grown.set(entry.bytes, 0); grown.set(data, at); entry.bytes = grown; - return data.byteLength; + return directory.faults.shortWrite ?? data.byteLength; }, truncate: (size) => { + if (directory.faults.truncate !== undefined) throw directory.faults.truncate; if (closed) throw named("handle is closed", "InvalidStateError"); entry.bytes = entry.bytes.slice(0, size); }, - flush: () => { if (closed) throw named("handle is closed", "InvalidStateError"); }, - close: () => { closed = true; directory.locked.delete(name); }, + flush: () => { + if (directory.faults.flush !== undefined) throw directory.faults.flush; + if (closed) throw named("handle is closed", "InvalidStateError"); + }, + close: () => { + if (directory.faults.close !== undefined) throw directory.faults.close; + closed = true; + directory.locked.delete(name); + }, }; } @@ -737,6 +764,187 @@ test("a file handle without createSyncAccessHandle fails typed, never as a bare && error.details["missing"] === "FileSystemFileHandle.createSyncAccessHandle"); }); +test("OPFS Worker errors retain browser names and nested finalizer causes", async () => { + const harness = controlledWorkerHarness(); + const client = new OpfsWriteWorkerClient({ idleGraceMs: 0, createWorker: () => harness.worker }); + const opening = client.createWriter("folder", "nested-cause"); + harness.emit({ type: "worker-ready", writeSupport: true }); + await new Promise((resolve) => setImmediate(resolve)); + const request = harness.messages.find((message) => message.type === "write-open"); + assert.ok(request); + harness.emit({ + type: "opfs-error", + requestId: request.requestId, + error: { + name: "UnknownError", + message: "storage write refused", + causes: [ + { name: "InvalidStateError", message: "handle state" }, + { name: "NoModificationAllowedError", message: "cleanup lock" }, + ], + cause: { name: "NotAllowedError", message: "browser policy" }, + }, + }); + await assert.rejects(opening, (error: unknown) => { + if (!(error instanceof Error) || error.name !== "UnknownError") return false; + const aggregate = error as Error & { readonly errors?: readonly Error[]; readonly cause?: Error }; + return aggregate.cause?.name === "NotAllowedError" && + aggregate.errors?.map((item) => item.name).join(",") === "InvalidStateError,NoModificationAllowedError"; + }); + client.close(); +}); + +test("OPFS storage exceptions stay structured I/O failures across root, open, read, write, and close", async () => { + const bytes = new Uint8Array([1, 2, 3, 4]); + const expected = { + identity: identityFor(bytes), sampleRateHz: 48_000, channels: 1 as const, bitDepth: 16 as const, + frames: 2, canonicalBytes: bytes.byteLength, + }; + const cases = [ + { + label: "root", name: "UnknownError", operation: "open", resource: "sparse-root", phase: "root-open", + make: () => { + const root = new FakeDirectory(); + const backend = new OpfsStorageBackend({ + folderName: "opfs-fault-root-v1", storage: { getDirectory: async () => { throw named("root denied", "UnknownError"); } } as never, + createWorker: () => fakeOpfsWorker(root), readDeadlineMs: 5_000, + }); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-root" }) }; + }, + }, + { + label: "open", name: "InvalidStateError", operation: "open", resource: "sparse-root", phase: "root-open", + make: () => { + const root = new FakeDirectory({ directoryOpen: named("root handle is invalid", "InvalidStateError") }); + const backend = backendFor(root, "opfs-fault-open-v1"); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-open" }) }; + }, + }, + { + label: "writer-open", name: "NotAllowedError", operation: "create", resource: "sparse-payload", phase: "writer-open", + make: () => { + const root = new FakeDirectory({ writerOpen: named("writer creation denied", "NotAllowedError") }); + const backend = backendFor(root, "opfs-fault-writer-open-v1"); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-writer-open" }) }; + }, + }, + { + label: "read", name: "NotAllowedError", operation: "read", resource: "sparse-marker", phase: "metadata", + make: () => { + const root = new FakeDirectory(); + const backend = backendFor(root, "opfs-fault-read-v1"); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-read" }), root }; + }, + }, + { + label: "write", name: "NoModificationAllowedError", operation: "write", resource: "sparse-payload", phase: "payload-write", + make: () => { + const root = new FakeDirectory({ write: named("writer is locked", "NoModificationAllowedError") }); + const backend = backendFor(root, "opfs-fault-write-v1"); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-write" }) }; + }, + }, + { + label: "close", name: "UnknownError", operation: "close", resource: "sparse-payload", phase: "writer-close", + make: () => { + const root = new FakeDirectory({ flush: named("flush failed", "UnknownError") }); + const backend = backendFor(root, "opfs-fault-close-v1"); + return { backend, store: new VerifiedSparsePcmStore({ backend, instanceId: "fault-close" }) }; + }, + }, + ] as const; + + for (const fixture of cases) { + const resources = fixture.make(); + const { backend, store } = resources; + let resolverCalls = 0; + try { + if (fixture.label === "read") { + await store.installSource(expected, { resolve: async () => ({ spans: (async function* () { yield { startFrame: 0, bytes }; })() }) }); + const root = (resources as unknown as { readonly root: FakeDirectory }).root; + root.faults.read = named("marker read denied", "NotAllowedError"); + await assert.rejects(store.openSource(expected), (error: unknown) => + error instanceof EngineWebAdapterError && error.code === "storage.io" && + error.details.operation === fixture.operation && error.details.resource === fixture.resource && error.details.phase === fixture.phase && + (error.cause as Error | undefined)?.name === fixture.name); + } else { + await assert.rejects(store.installSource(expected, { + resolve: async () => { resolverCalls += 1; return { spans: (async function* () { yield { startFrame: 0, bytes }; })() }; }, + }), (error: unknown) => + error instanceof EngineWebAdapterError && error.code === "storage.io" && + error.details.operation === fixture.operation && error.details.resource === fixture.resource && error.details.phase === fixture.phase && + (error.cause as Error | undefined)?.name === fixture.name); + if (fixture.label === "root" || fixture.label === "open") assert.equal(resolverCalls, 0, `${fixture.label} refuses before resolver work`); + } + } finally { + await store.close(); + backend.close(); + } + } +}); + +test("a real OPFS Worker short write is storage I/O with operational byte evidence", async () => { + const root = new FakeDirectory({ shortWrite: 3 }); + const backend = backendFor(root, "opfs-short-write-v1"); + await backend.open(); + const writer = await backend.createWriter("short-write"); + await assert.rejects(writer.write(new Uint8Array([1, 2, 3, 4])), (error: unknown) => + error instanceof EngineWebAdapterError && + error.code === "storage.io" && + error.details.expectedBytes === 4 && + error.details.writtenBytes === 3 && + error.details.verification === undefined); + await writer.abort(new Error("short write cleanup")); + backend.close(); +}); + +test("a real OPFS Worker preserves truncate/cleanup-close and flush/close compounds", async () => { + const truncateRoot = new FakeDirectory({ + truncate: named("truncate failed", "InvalidStateError"), + close: named("cleanup close failed", "NoModificationAllowedError"), + }); + const truncateBackend = backendFor(truncateRoot, "opfs-truncate-close-v1"); + await truncateBackend.open(); + await assert.rejects(truncateBackend.createWriter("truncate-close"), (error: unknown) => { + if (!(error instanceof AggregateError)) return false; + return error.errors.map((item) => (item as Error & { readonly role?: unknown }).role).join(",") === "body,finalizer" && + (error.errors[0] as Error).name === "InvalidStateError" && + (error.errors[1] as Error).name === "NoModificationAllowedError"; + }); + truncateBackend.close(); + + for (const [label, faults, expectedName] of [ + ["truncate-body", { truncate: named("truncate failed", "InvalidStateError") }, "InvalidStateError"], + ] as const) { + const root = new FakeDirectory(faults); + const backend = backendFor(root, `opfs-${label}-v1`); + await backend.open(); + await assert.rejects(backend.createWriter(label), (error: unknown) => error instanceof Error && !(error instanceof AggregateError) && error.name === expectedName); + backend.close(); + } + + const flushRoot = new FakeDirectory(); + const flushBackend = backendFor(flushRoot, "opfs-flush-close-v1"); + await flushBackend.open(); + const writer = await flushBackend.createWriter("flush-close"); + flushRoot.faults.flush = named("flush failed", "UnknownError"); + flushRoot.faults.close = named("close failed", "NoModificationAllowedError"); + await assert.rejects(writer.close(), (error: unknown) => { + if (!(error instanceof AggregateError)) return false; + return error.errors.map((item) => (item as Error & { readonly role?: unknown }).role).join(",") === "body,finalizer" && + (error.errors[0] as Error).name === "UnknownError" && + (error.errors[1] as Error).name === "NoModificationAllowedError"; + }); + flushBackend.close(); + + const finalizerRoot = new FakeDirectory({ close: named("close failed", "NoModificationAllowedError") }); + const finalizerBackend = backendFor(finalizerRoot, "opfs-flush-finalizer-v1"); + await finalizerBackend.open(); + const finalizerWriter = await finalizerBackend.createWriter("flush-finalizer"); + await assert.rejects(finalizerWriter.close(), (error: unknown) => error instanceof Error && !(error instanceof AggregateError) && error.name === "NoModificationAllowedError"); + finalizerBackend.close(); +}); + test("the verified store ingests, promotes, and rereads canonical PCM on the OPFS backend", async () => { const root = new FakeDirectory(); const backend = backendFor(root, "opfs-store-v1"); diff --git a/tests/sparse-store.test.ts b/tests/sparse-store.test.ts index e215eff..75dae35 100644 --- a/tests/sparse-store.test.ts +++ b/tests/sparse-store.test.ts @@ -492,7 +492,9 @@ describe("VerifiedSparsePcmStore", () => { changed[0] = (changed[0] ?? 0) ^ 1; backend.files.set(payload, changed); await assert.rejects(store.openSource(expected), (error: unknown) => - error instanceof Error && "code" in error && error.code === "stem.corrupt"); + error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + error.details.verification !== undefined && + (error.details.verification as { readonly kind?: unknown }).kind === "digest"); }); it("rejects malformed spans and an upfront index that disagrees before publication", async () => { @@ -788,7 +790,7 @@ describe("VerifiedSparsePcmStore", () => { const backend = new CollisionBackend({ files: { [orphanName]: orphan } }); const store = new VerifiedSparsePcmStore({ backend, instanceId: "collision" }); const bytes = new Uint8Array([1, 2, 3, 4]); - await assert.rejects(store.installSource(expectation(bytes, 2), { resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(store.installSource(expectation(bytes, 2), { resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "storage.io"); assert.deepEqual(backend.files.get(orphanName), orphan); await store.close(); }); @@ -827,6 +829,80 @@ describe("VerifiedSparsePcmStore", () => { await store.close(); }); + it("keeps an outer open I/O aggregate authoritative over nested declaration evidence", async () => { + const declaration = new EngineWebAdapterError("stem.invalid_declaration", "nested declaration"); + class AggregateOpenBackend extends MemoryStemStorageBackend { + override async open(): Promise { + throw new AggregateError([ + new Error("BODY_IO"), + new Error("wrapped declaration", { cause: declaration }), + ], "open body and declaration failed"); + } + } + const bytes = new Uint8Array([1, 2]); + const store = new VerifiedSparsePcmStore({ backend: new AggregateOpenBackend(), instanceId: "aggregate-open" }); + await assert.rejects(store.installSource(expectation(bytes, 1), { resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }) }), (error: unknown) => { + if (!(error instanceof EngineWebAdapterError) || error.code !== "storage.io") return false; + if (!(error.cause instanceof AggregateError)) return false; + const nested = error.cause.errors[1] as Error & { readonly cause?: unknown }; + return error.details.operation === "open" && + error.cause.errors[0] instanceof Error && + nested.cause === declaration; + }); + await store.close(); + }); + + it("does not trust a code-only nested corruption claim", async () => { + const corrupt = new EngineWebAdapterError("stem.corrupt", "code without proof"); + class CodeOnlyBackend extends MemoryStemStorageBackend { + override async open(): Promise { throw corrupt; } + } + const bytes = new Uint8Array([1, 2]); + const store = new VerifiedSparsePcmStore({ backend: new CodeOnlyBackend(), instanceId: "code-only-corrupt" }); + await assert.rejects(store.installSource(expectation(bytes, 1), { resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }) }), (error: unknown) => + error instanceof EngineWebAdapterError && error.code === "storage.io" && error.details.verification === undefined && + error.cause === corrupt); + await store.close(); + }); + + it("keeps foreign root corruption evidence as storage I/O, including invalid proofs", async () => { + const bytes = new Uint8Array([1, 2]); + const expected = expectation(bytes, 1); + const invalidProofs: readonly unknown[] = [ + { kind: "length", expected: -1, observed: 2 }, + { kind: "length", expected: 1.5, observed: 2 }, + { kind: "length", expected: Number.NaN, observed: 2 }, + { kind: "digest", expected: 1, observed: 2 }, + { kind: "digest", expected: "", observed: "a" }, + { kind: "digest", expected: "not-a-digest", observed: "also-not-a-digest" }, + { kind: "digest", expected: "a".repeat(64), observed: "b".repeat(64), algorithm: "sha512" }, + { kind: "digest", expected: "a".repeat(64), observed: 2 }, + ]; + for (const [index, verification] of [ + ...invalidProofs.map((proof) => proof), + { kind: "digest", expected: "a".repeat(64), observed: "b".repeat(64), algorithm: "blake3-256" }, + ].entries()) { + const borrowed = new EngineWebAdapterError("stem.corrupt", "foreign corruption claim", { verification }); + class ForeignRootBackend extends MemoryStemStorageBackend { + override async open(): Promise { throw borrowed; } + } + const backend = new ForeignRootBackend(); + const store = new VerifiedSparsePcmStore({ backend, instanceId: `foreign-root-${index}` }); + let resolverCalls = 0; + await assert.rejects(store.installSource(expected, { + resolve: async () => { resolverCalls += 1; return { spans: spans({ startFrame: 0, bytes }) }; }, + }), (error: unknown) => error instanceof EngineWebAdapterError && + error.code === "storage.io" && + error.details.operation === "open" && + error.details.resource === "sparse-root" && + error.details.phase === "root-open" && + error.details.verification === undefined && + error.cause === borrowed); + assert.equal(resolverCalls, 0); + await store.close(); + } + }); + it("cancels a resolver before publication and waits for its physical release", async () => { const bytes = new Uint8Array([1, 2, 3, 4]); const backend = new MemoryStemStorageBackend(); @@ -996,6 +1072,46 @@ describe("VerifiedSparsePcmStore", () => { assert.ok(firstRemove > events.indexOf("write-settled"), events.join(",")); }); + it("preserves a late physical write rejection observed after cancellation", async () => { + const bytes = new Uint8Array([1, 2]); + const controller = new AbortController(); + const started = deferred(); + const settled = deferred(); + const userCancel = new Error("USER_CANCEL_SENTINEL"); + const physicalFailure = new Error("PHYSICAL_REJECT_SENTINEL"); + const cleanupFailure = new Error("CLEANUP_REJECT_SENTINEL"); + class LateRejectBackend extends MemoryStemStorageBackend { + override async createWriter(name: string, signal?: AbortSignal) { + const writer = await super.createWriter(name, signal); + if (!name.startsWith("sparse-pcm-v1-data-")) return writer; + return { + write: async () => { started.resolve(); await settled.promise; throw physicalFailure; }, + close: () => writer.close(), + abort: async () => { if (abortCalls++ === 0) throw cleanupFailure; }, + }; + } + } + let abortCalls = 0; + const backend = new LateRejectBackend(); + const store = new VerifiedSparsePcmStore({ backend, instanceId: "late-write-rejection" }); + const installing = store.installSource(expectation(bytes, 1), { + signal: controller.signal, + resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }), + }); + await started.promise; + controller.abort(userCancel); + await new Promise((resolve) => setImmediate(resolve)); + settled.resolve(); + await assert.rejects(installing, (error: unknown) => { + if (!(error instanceof EngineWebAdapterError) || error.code !== "storage.io" || error.details.operation !== "write") return false; + if (!(error.cause instanceof AggregateError)) return false; + const roles = error.cause.errors as readonly (Error & { readonly role?: unknown; readonly cause?: unknown })[]; + return roles.map((cause) => cause.role).join(",") === "interruption,body,finalizer" && + roles[0]?.cause === userCancel && roles[1]?.cause === physicalFailure && roles[2]?.cause === cleanupFailure; + }); + await store.close(); + }); + it("preserves async and sync iterator cleanup failures beside public primary errors", async () => { const bytes = new Uint8Array([1, 2]); for (const phase of ["span", "write", "early"] as const) { @@ -1102,7 +1218,11 @@ describe("VerifiedSparsePcmStore", () => { const installing = store.installSource(expectation(bytes, 1), { signal: controller.signal, resolve: async () => ({ spans: spans({ startFrame: 0, bytes }) }) }); await writeStarted.promise; gate.resolve(); - await assert.rejects(installing, (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.cancelled"); + await assert.rejects(installing, (error: unknown) => + error instanceof EngineWebAdapterError && error.code === "storage.io" && + error.details.operation === "write" && error.cause instanceof AggregateError && + (error.cause.errors as readonly (Error & { readonly role?: unknown; readonly cause?: unknown })[]).map((cause) => cause.role).join(",") === "interruption,body" && + (error.cause.errors[1] as Error & { readonly cause?: Error }).cause?.message === "writer is closed"); assert.equal(events[0], "write-start"); assert.ok(events.includes("abort"), "physical abort runs after synchronous cancellation"); assert.ok(events.indexOf("write-settled") > events.indexOf("abort"), "physical abort precedes write settlement"); @@ -1193,7 +1313,7 @@ describe("VerifiedSparsePcmStore", () => { assert.equal(aggregate.errors.some((value) => value === undefined), true); } else if (phase === "getter") { assert.ok(error instanceof EngineWebAdapterError); - assert.equal(error.code, "stem.corrupt"); + assert.equal(error.code, "storage.io"); assert.ok(error.cause instanceof AggregateError); assert.equal(error.cause.errors.includes(primary), true); assert.equal(error.cause.errors.includes(cleanup), true); @@ -1319,7 +1439,9 @@ describe("VerifiedSparsePcmStore", () => { tampered[0] = (tampered[0] ?? 0) ^ 1; backend.files.set(payload, tampered); assert.deepEqual(await store.inspectSourcePresence(expected), { status: "present", activeBytes: bytes.byteLength }); - await assert.rejects(store.openSource(expected), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(store.openSource(expected), (error: unknown) => + error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + (error.details.verification as { readonly kind?: unknown } | undefined)?.kind === "digest"); assert.equal(payloadArrayBufferReads > 0, true, "full open must perform canonical PCM verification"); await store.close(); }); @@ -1338,7 +1460,7 @@ describe("VerifiedSparsePcmStore", () => { if (mode === "malformed-marker") backend.files.set(marker, new Uint8Array([123, 125])); if (mode === "missing-generation") backend.files.delete(payload); if (mode === "wrong-extent") backend.files.set(payload, backend.files.get(payload)!.slice(0, 2)); - await assert.rejects(store.inspectSourcePresence(expected), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(store.inspectSourcePresence(expected), (error: unknown) => error instanceof EngineWebAdapterError && error.code === (mode === "wrong-extent" ? "stem.corrupt" : "stem.invalid_declaration")); await store.close(); } }); @@ -1511,7 +1633,8 @@ describe("VerifiedSparsePcmStore", () => { sources: [{ ...tampered.expected, sourceId: "tampered" }], resolve: resolver, onProgress: (event) => tamperedEvents.push(event), - }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + (error.details.verification as { readonly kind?: unknown } | undefined)?.kind === "digest"); assert.equal(tamperedBackend.dataGets, 1); assert.equal(tamperedEvents.some((event) => event.stage === "source-ready" || event.stage === "ready"), false); await tampered.store.close(); @@ -1578,7 +1701,12 @@ describe("VerifiedSparsePcmStore", () => { assert.notEqual(outcome, "timeout", `${nextMode} must settle within its boundary`); assert.equal(outcome === "success", false, `${nextMode} must fail closed`); const error = outcome as unknown; - assert.equal(error instanceof EngineWebAdapterError && error.code, nextMode === "silent" ? "stem.read_deadline" : "stem.corrupt", nextMode); + const expectedCode = nextMode === "silent" + ? "stem.read_deadline" + : nextMode === "digest" + ? "stem.corrupt" + : "storage.io"; + assert.equal(error instanceof EngineWebAdapterError && error.code, expectedCode, nextMode); assert.equal(events.some((event) => event.stage === "source-ready" || event.stage === "ready"), false, `${nextMode} cannot publish readiness`); await store.close(); } @@ -1802,7 +1930,7 @@ describe("VerifiedSparsePcmStore", () => { if (outcome.ok) return; assert.equal(mode === "cancel" ? outcome.error instanceof EngineWebAdapterError && outcome.error.code === "stem.cancelled" - : outcome.error instanceof EngineWebAdapterError && outcome.error.code === "stem.corrupt", true); + : outcome.error instanceof EngineWebAdapterError && outcome.error.code === "storage.io", true, inspect(outcome.error, { depth: 8 })); const sourceReady = events.filter((event) => event.stage === "source-ready"); assert.equal(events.filter((event) => event.stage === "ready").length, 0, `${mode} cannot publish aggregate readiness`); if (mode === "cancel") { @@ -2026,7 +2154,7 @@ describe("VerifiedSparsePcmStore", () => { globalThis.MessageChannel = ThrowingWarmMessageChannel as unknown as typeof globalThis.MessageChannel; const constructionStore = new VerifiedSparsePcmStore({ backend: new MemoryStemStorageBackend(), instanceId: "warm-task-construction" }); await constructionStore.installSource(fixture.expected, { resolve: async () => ({ spans: spans(...fixture.spans) }) }); - await assert.rejects(constructionStore.openSource(fixture.expected), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.invalid_declaration"); + await assert.rejects(constructionStore.openSource(fixture.expected), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "storage.io"); assert.equal(probe.constructed, 1); await constructionStore.close(); @@ -2036,7 +2164,7 @@ describe("VerifiedSparsePcmStore", () => { const postStore = new VerifiedSparsePcmStore({ backend: new MemoryStemStorageBackend(), instanceId: "warm-task-post" }); await postStore.installSource(fixture.expected, { resolve: async () => ({ spans: spans(...fixture.spans) }) }); const postEvents: import("../src/stems/types.js").StemProgress[] = []; - await assert.rejects(postStore.openSource(fixture.expected, { onProgress: (event) => postEvents.push(event) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.invalid_declaration"); + await assert.rejects(postStore.openSource(fixture.expected, { onProgress: (event) => postEvents.push(event) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "storage.io"); assert.equal(postProbe.constructed, 1); assert.equal(postProbe.posts, 1); assert.equal(postProbe.closes, 2); @@ -2050,7 +2178,7 @@ describe("VerifiedSparsePcmStore", () => { globalThis.MessageChannel = ProbeWarmMessageChannel as unknown as typeof globalThis.MessageChannel; await closeStore.installSource(fixture.expected, { resolve: async () => ({ spans: spans(...fixture.spans) }) }); const closeEvents: import("../src/stems/types.js").StemProgress[] = []; - await assert.rejects(closeStore.openSource(fixture.expected, { onProgress: (event) => closeEvents.push(event) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(closeStore.openSource(fixture.expected, { onProgress: (event) => closeEvents.push(event) }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "storage.io"); assert.equal(closeProbe.constructed, 1); assert.equal(closeProbe.posts, 4); assert.equal(closeProbe.closes, 2, "a fault in port1 close cannot skip port2 close"); @@ -2120,7 +2248,8 @@ describe("VerifiedSparsePcmStore", () => { returnedDescriptor = descriptor; throw new Error("short warm payload read unexpectedly succeeded"); }); - await assert.rejects(opening, (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + await assert.rejects(opening, (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + (error.details.verification as { readonly kind?: unknown } | undefined)?.kind === "length"); const target = backend.payloadReads.find((read) => faultKind === "full" ? read.end - read.start === warmReadBytes && read.returned < warmReadBytes : read.end - read.start < warmReadBytes); assert.ok(target); assert.equal(target.returned, target.end - target.start - 1); @@ -2142,7 +2271,8 @@ describe("VerifiedSparsePcmStore", () => { const tamperedEvents: import("../src/stems/types.js").StemProgress[] = []; await assert.rejects(tampered.store.openSource(tampered.expected, { onProgress: (event) => tamperedEvents.push(event), - }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt" && + (error.details.verification as { readonly kind?: unknown } | undefined)?.kind === "digest"); assert.ok(tamperedBackend.payloadReads.length >= 2); assert.equal(tamperedEvents.some((event) => event.stage === "source-ready"), false); assert.equal(tamperedEvents.some((event) => event.verificationTiming !== undefined), false); @@ -2277,7 +2407,7 @@ describe("VerifiedSparsePcmStore", () => { sources: [{ ...expected, sourceId: "source" }], resolve: resolver, onProgress: (event) => events.push(event), - }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "stem.corrupt"); + }), (error: unknown) => error instanceof EngineWebAdapterError && error.code === "storage.io"); const sourceReady = events.findIndex((event) => event.stage === "source-ready"); const physicalRelease = lifecycle.indexOf("physical-release"); assert.ok(sourceReady >= 0, "a committed source remains observable"); diff --git a/tests/sparse-verify-worker.test.ts b/tests/sparse-verify-worker.test.ts index d7c31bb..83cdb5d 100644 --- a/tests/sparse-verify-worker.test.ts +++ b/tests/sparse-verify-worker.test.ts @@ -3,6 +3,7 @@ import { afterEach, describe, it } from "node:test"; import { createBLAKE3 } from "hash-wasm"; import { SparseVerifyWorkerPool } from "../src/stems/sparse-verify-worker-pool.js"; +import type { EngineWebAdapterErrorCause, EngineWebAdapterVerification } from "../src/errors.js"; import { MemoryStemStorageBackend, VerifiedSparsePcmStore, createSparseStemResolver } from "../src/stems/index.js"; import { SPARSE_VERIFY_PROTOCOL_VERSION, @@ -454,9 +455,48 @@ describe("Sparse verification Worker transport", () => { readDeadlineMs: item.mode === "deadline" ? 15 : 250, }); assert.equal(result.failure?.kind, item.kind, item.mode); + if (item.mode === "short") { + assert.deepEqual(result.failure?.verification, { kind: "length", expected: 4, observed: 3 }); + } } }); + it("round trips sparse worker nested causes and verification evidence", async () => { + class FailureWorker extends FakeVerifyWorker { + override postMessage(message: any): void { + if (message.type !== "start") return super.postMessage(message); + queueMicrotask(() => this.emitForTest({ data: { + type: "failure", version: SPARSE_VERIFY_PROTOCOL_VERSION, + jobId: message.jobId, generation: message.generation, kind: "io", + message: "Sparse verification read failed", + verification: { kind: "length", expected: 4, observed: 3 }, + cause: { + name: "AggregateError", message: "read and cleanup failed", + cause: { name: "UnknownError", message: "browser read failed" }, + causes: [ + { name: "InvalidStateError", message: "reader state" }, + { name: "NoModificationAllowedError", message: "cleanup state" }, + ], + } satisfies EngineWebAdapterErrorCause, + } })); + } + } + (globalThis as unknown as { Worker: new () => ListenerWorker }).Worker = FailureWorker; + const pool = new SparseVerifyWorkerPool({ width: 1 }); + const result = await pool.run(runOptions(smallIndex(), new AbortController().signal)).catch((error: unknown) => error); + assert.equal(result instanceof Error && "kind" in result && result.kind, "io"); + assert.deepEqual((result as Error & { readonly verification?: EngineWebAdapterVerification }).verification, { + kind: "length", expected: 4, observed: 3, + }); + const cause = (result as Error).cause; + assert.equal(cause instanceof AggregateError, true); + const aggregate = cause as AggregateError & { readonly cause?: unknown }; + const nestedCause = aggregate.cause; + assert.equal(nestedCause instanceof Error && nestedCause.name, "UnknownError"); + assert.deepEqual(aggregate.errors.map((value) => (value as Error).name), ["InvalidStateError", "NoModificationAllowedError"]); + await pool.close(); + }); + it("coalesces held progress, cancels the actual loop, and leaves no terminal delivery", async () => { const payload = new Uint8Array([1, 2, 3, 4]); const frames = 100_003; @@ -581,13 +621,19 @@ function runOptions(index: ReturnType, signal: AbortSignal) { function completeMessage(identity: `blake3:${string}`, jobId: number, generation: number): Record { return { type: "complete", version: SPARSE_VERIFY_PROTOCOL_VERSION, jobId, generation, - identity, digest: "0".repeat(64), progressBytes: 8, canonicalBytes: 8, + identity, digest: identity.slice(7), progressBytes: 8, canonicalBytes: 8, readCalls: 1, readBytes: 4, hashedBytes: 8, hashUpdates: 2, zeroUpdates: 1, elapsedMs: 1, readWaitMs: 1, hashMs: 1, }; } -type ActualWorkerFailure = { readonly type: "failure"; readonly kind: string; readonly message: string }; +type ActualWorkerFailure = { + readonly type: "failure"; + readonly kind: string; + readonly message: string; + readonly verification?: EngineWebAdapterVerification; + readonly cause?: EngineWebAdapterErrorCause; +}; type ActualWorkerScope = { onmessage: ((event: MessageEvent) => void) | null }; interface ActualWorkerLoopOptions { @@ -778,7 +824,7 @@ class FakeVerifyWorker implements ListenerWorker { if (this.hold) return; queueMicrotask(() => this.emit({ data: { type: "complete", version: SPARSE_VERIFY_PROTOCOL_VERSION, jobId: message.jobId, generation: message.generation, - identity: this.identity, digest: "0".repeat(64), progressBytes: 8, canonicalBytes: 8, readCalls: 1, readBytes: 4, hashedBytes: 8, + identity: this.identity, digest: this.identity.slice(7), progressBytes: 8, canonicalBytes: 8, readCalls: 1, readBytes: 4, hashedBytes: 8, hashUpdates: 2, zeroUpdates: 1, elapsedMs: 1, readWaitMs: 1, hashMs: 1, } })); diff --git a/tests/store.test.ts b/tests/store.test.ts index e929780..9b9fbb8 100644 --- a/tests/store.test.ts +++ b/tests/store.test.ts @@ -614,7 +614,7 @@ test("unsupported index version refuses before touching historical files", async backend.files.set("staging-old", new Uint8Array([5])); backend.files.set("staging/old", new Uint8Array([6])); const before = [...backend.files].map(([name, bytes]) => [name, [...bytes]]); - await assert.rejects(new VerifiedStemStore({ backend, locks: new TestLocks() }).open(), { code: "stem.corrupt" }); + await assert.rejects(new VerifiedStemStore({ backend, locks: new TestLocks() }).open(), { code: "stem.invalid_declaration" }); assert.deepEqual([...backend.files].map(([name, bytes]) => [name, [...bytes]]), before); });