Preserve storage I/O and structured failure evidence

.github/ISSUE_SPECS/128-preserve-structured-storage-failures-instead-of-classifying-them-as-verified-corruption.md

@@ -0,0 +1,119 @@
+# Preserve structured storage failures instead of classifying them as verified corruption
+
+## Problem
+
+The app investigation in `misofm/app#294` found that adapter 0.5.13 maps sparse
+storage-open failures to `stem.corrupt`, before any download. UnknownError,
+InvalidStateError, NotAllowedError, and NoModificationAllowedError injected at
+root/open must not be reported as verified audio corruption. The original
+device exception was not captured; injection does not establish that device's
+root cause.
+
+Base: upstream main `602b7c7b77d6228a73a0b9932a8809a8933ba692`, package
+`@misofm/engine-web-adapter@0.5.13`, Engine `0.4.3`, Effect `4.0.0-rc.112`.
+
+## Bounded T2 implementation
+
+This issue owns only the upstream adapter's public storage-error contract and
+the sparse-store/worker boundaries that implement it. It does not change app
+code, the sparse format, shutdown ownership, or close retry policy.
+
+- Introduce the documented public `storage.io` code for storage failures with
+  additive operation/resource/granular-phase details.
+- Preserve unknown browser exception names and causes without guessing from
+  message text.
+- Keep `stem.invalid_declaration` and `session.declaration_mismatch` for
+  declaration/metadata failures, and reserve `stem.corrupt` for actual digest
+  or byte-length verification evidence (`kind`, `expected`, `observed`, and
+  `algorithm` where applicable).
+- Preserve typed failures, Effect interruptions, defects, and every finalizer
+  reason through the public edge and relevant Worker transports.
+- Document supported close/cancellation semantics, add regression tests and an
+  unreleased changelog/evidence entry, and qualify public exports on a packed
+  artifact.
+
+## Acceptance
+
+- Inject root/open/read/write/close failures, including UnknownError,
+  InvalidStateError, NotAllowedError, and NoModificationAllowedError; none is
+  reported as verified integrity.
+- Root-open refusal happens before resolver/network work and produces zero stem
+  fetches.
+- Genuine wrong digest and wrong length remain `stem.corrupt` with structured
+  verification evidence.
+- Invalid metadata/declaration remains distinct from storage and integrity
+  failures.
+- Worker round trips preserve nested causes, interruption/defect distinctions,
+  and finalizer failures rather than flattening to the first reason.
+- Public exports and package tests pass against a fresh packed artifact.
+
+## Workflow and boundaries
+
+Implementation is Luna reasoning=max followed by a fresh independent Astra
+reasoning=medium verification. No spawning, push, PR creation, merge, or
+publication is authorized here; coordinator owns reviewed integration.
+
+The release metadata remains at version 0.5.13. Do not copy app sources,
+patch `node_modules`, rewrite the sparse format, force shutdown, or invent a
+retry of a failed physical close. Record exact commands, results, candidate
+SHA, package provenance, fixture shape, close/cancellation semantics, and
+browser capability limitations below.
+
+## Implementation evidence
+
+Reviewed implementation: `27795844c2ebaed2061eb0b6cdde0d15a2550973`
+(parent `93cf2f790ea2ecba5428e8a32ac3bbc40c05b605`).
+Fresh independent Astra medium review approved this exact clean candidate;
+this subsequent evidence-only commit changes no source or test code.
+
+Verification completed:
+
+- `npm exec -- tsc -p tsconfig.test.json` — test compilation passed.
+- `node --test .test-dist/tests/error-contract.test.js .test-dist/tests/sparse-store.test.js .test-dist/tests/opfs-storage.test.js` — 104/104 focused tests passed.
+- Recursive reservation coverage — 1,026 nested pad cases (cause and causes, every pad length 0..512) plus the bounded Unicode/escape depth-width table passed serialization, repeat serialization, and revival; every serialized result stayed at or below 16,384 UTF-8 bytes and retained cause summaries were nonempty and valid.
+- `npm run check` — passed, including format, wasm verification, lint, decoder checks, the full 396/396 test suite, and package policy (`248 files, 507013 bytes`).
+- `CHROME_EXECUTABLE=/home/bl/.cache/ms-playwright/chromium-1234/chrome-linux64/chrome npm run test:browser` — passed against the fresh packed consumer on Chromium 151.0.7922.34, 15 assets, no request failures or console errors. Artifact: `/tmp/engine-web-adapter-browser-TvwzzD`.
+
+The real storage-root `installSource` boundary keeps foreign/nested
+`stem.corrupt` payloads and invalid proof shapes under primary `storage.io`
+evidence; only adapter-performed verification with a meaningful differing
+canonical digest/byte-length proof can promote integrity corruption. The
+public legacy constructor/API remains compatible. Native interruption,
+physical write/body settlement, and cleanup failures retain explicit
+`interruption`/`body`/`finalizer` roles; physical settlement and ownership stay
+authoritative, and cached-close policy is unchanged.
+
+Publication and app adoption remain outside this adapter change. The local
+Linux WebKit runner lacks `FileSystemFileHandle`; that capability limitation
+was not misreported as an OPFS pass. Subsequent existing macOS CI on exact
+head `512029f387d3edb27608cdc9c57195219ba6889d` passed real OPFS and packed
+indexed-sparse qualification in both Chromium 151.0.7922.34 and WebKit 26.5.
+Runner provenance: macOS 15.7.9 arm64, Node 22.23.2, npm 10.9.8.
+Run: https://github.com/misofm/engine-web-adapter/actions/runs/35694712900
+Artifact: `opfs-35694712900-1`, including checkout/runner/tool provenance,
+actual OPFS cold/warm/marker-failure/late-cancellation evidence, and packed
+three-warm-worker qualification for both browsers. This following commit
+only records that evidence; it changes no implementation or tests.
+
+The upstream candidate's WebKit environment gap is closed by actual macOS
+results. The final combined app and real iPhone/deadline qualification remain
+open release gates; these upstream checks do not prove app recovery behavior
+or identify the original device trigger. The fixed SDK is not published and
+app adoption awaits the normal reviewed release workflow; neither merge nor
+publication is authorized by this work.
+
+The committed public fixture `tests/fixtures/storage-error-contract.json`
+records `storage.io`, coarse phase `store`, details
+`{ operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io" }`,
+and the retained `UnknownError` cause. Independent review of this exact candidate approved the recursive global
+reservation invariant and retained all prior taxonomy, cancellation, physical
+settlement, and integrity-proof closures. The reviewer independently ran
+104 focused tests, the full 396-test check, and a fresh packed Chromium
+consumer (artifact `/tmp/engine-web-adapter-browser-MHXdK5`, 15 assets, no
+request failures or console errors). Generated verification covered 1,835
+cases / 14,680 serialization checks with a maximum of 16,337 UTF-8 bytes;
+full padding sweeps, deep/wide/mixed trees, Unicode, JSON escapes, six
+reserializations, and revival retained useful mandatory summaries.
+The coordinator synchronizes this evidence to the matching issue before PR
+creation. The macOS CI qualification above records only observed results;
+no real-device or final combined app result is claimed here.

CHANGELOG.md

@@ -0,0 +1,8 @@
+# Changelog
+
+## Unreleased
+
+- Preserve browser storage failures as `storage.io` with operation/resource/
+  phase context and bounded nested causes; reserve `stem.corrupt` for verified
+  digest or byte-length evidence and keep metadata/declaration failures
+  distinct.

README.md

@@ -114,6 +114,33 @@ identical operation is `transient`, frozen `details`, and the underlying
 `cause`. No Engine host object, Worker message or Effect value reaches a caller
 through it.
 
+Storage access failures use the distinct `storage.io` code. Its coarse public
+`.phase` remains `store`; storage details add `operation`, `resource`, and a
+granular `phase`, while the original browser exception name and cause are
+retained without classifying from message text. `stem.invalid_declaration`
+covers invalid metadata or declarations. `stem.corrupt` is reserved for a
+verified digest or byte-length mismatch and carries
+`details.verification = { kind, expected, observed, algorithm? }`; a storage or
+metadata refusal never receives that code.
+
+For example, an OPFS root refusal is app-consumable as:
+
+```json
+{
+  "name": "EngineWebAdapterError",
+  "message": "Sparse root access failed",
+  "code": "storage.io",
+  "phase": "store",
+  "details": {
+    "operation": "open",
+    "resource": "sparse-root",
+    "phase": "root-open",
+    "failureKind": "io"
+  },
+  "cause": { "name": "UnknownError", "message": "Storage access was refused" }
+}
+```
+
 Console submission returns the SDK `CommandReport` unchanged. A semantic
 refusal has `ok: false`, `admitted: 0`, and its original reason, rejected index,
 and application sample; transport failures reject. Request identifiers are
@@ -235,9 +262,15 @@ asset fields without discarding the other common fields; the low-level
 - Canonical PCM is headerless interleaved little-endian PCM16 or PCM24 at
   44.1, 48, 88.2, or 96 kHz; there is no implicit sample-rate conversion.
 
-`close()` is idempotent. The `./stems` entry exports what a caller can supply
-or replace: the canonical-PCM resolver seam, the verified store and its storage
-backends, the FLAC resolver, the pump, and the ring control words a `createPump`
+`close()` is idempotent. Store close aborts active logical operations, waits for
+their physical reads/writes and writer aborts to settle, then disposes shared
+verification workers. Cancellation has the same settlement rule: late browser
+work cannot publish a source or trigger an automatic retry. A failed physical
+writer close is surfaced with its original cause and is not retried or replaced
+by a forced shutdown; staging cleanup remains separately observable. The
+session-owned OPFS backend is released by session teardown. The `./stems` entry
+exports what a caller can supply or replace: the canonical-PCM resolver seam,
+the verified store and its storage backends, the FLAC resolver, the pump, and the ring control words a `createPump`
 override reads. The existing ring control exports are SDK re-exports. Digests,
 admission width, the decoder pool and adapter Worker protocols remain internal;
 PCM ring arithmetic is owned by the SDK.
@@ -420,7 +453,7 @@ ingest locks. Each lease also holds its historical folder-qualified
 `:pin:<session-pin>` lifetime lock so existing app recovery recognizes it.
 Recovery preserves ambiguous session pins and offline pins; it leaves the
 historical `staging/` directory alone. Explicit unsupported index versions
-refuse with `stem.corrupt` before recovery changes any file.
+refuse with `stem.invalid_declaration` before recovery changes any file.
 
 When a known storage estimate cannot fit a cold source, the store reclaims
 unpinned indexed entries by oldest `lastUsedAt`, breaking ties by identity,

package.json

@@ -13,6 +13,7 @@
   ],
   "files": [
     "dist",
+    "CHANGELOG.md",
     "LICENSE",
     "NOTICE",
     "README.md"

scripts/browser-packed.mjs

@@ -240,8 +240,14 @@ const peak: TrackMeter | undefined = update.tracks.get("track-000");
 declare const failure: EngineWebAdapterError;
 const remedy: string = failure.remedy;
 const transient: boolean = failure.transient;
+const storageFailure = new EngineWebAdapterError("storage.io", "packed storage fixture", {
+  operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io",
+});
+const storageCode: EngineWebAdapterError["code"] = storageFailure.code;
+const storagePhase: EngineWebAdapterError["phase"] = storageFailure.phase;
 void [openEngineWebSession, openSparseEngineWebSession, createFlacStemResolver, PcmPumpWorkerClient, ADAPTER_ASSETS, packageJson,
-  minimal, sparseMinimal, live, meters, telemetry, sdk, directMeters, directTelemetry, directResponse, directSpectrum, peak, remedy, transient];
+  minimal, sparseMinimal, live, meters, telemetry, sdk, directMeters, directTelemetry, directResponse, directSpectrum, peak, remedy, transient,
+  storageFailure, storageCode, storagePhase];
 `);
 await writeFile(join(consumer, "tsconfig.json"), JSON.stringify({ compilerOptions: {
   strict: true, noEmit: true, target: "ES2022", module: "NodeNext", moduleResolution: "NodeNext",

scripts/check-package.mjs

@@ -24,6 +24,13 @@ assert.equal(
 );
 const { ADAPTER_PROVENANCE } = await import("../dist/provenance.js");
 assert.equal(ADAPTER_PROVENANCE.engine.package, `@misofm/engine@${packageJson.dependencies["@misofm/engine"]}`, "built provenance must identify the installed Engine dependency");
+const { EngineWebAdapterError } = await import("../dist/index.js");
+const packedSurfaceFailure = new EngineWebAdapterError("storage.io", "packed storage fixture", {
+  operation: "open", resource: "sparse-root", phase: "root-open", failureKind: "io",
+}, new Error("UnknownError fixture"));
+assert.equal(packedSurfaceFailure.code, "storage.io");
+assert.equal(packedSurfaceFailure.phase, "store");
+assert.equal(packedSurfaceFailure.details.operation, "open");
 const cache = await mkdtemp(join(tmpdir(), "engine-web-adapter-npm-cache-"));
 const packed = spawnSync("npm", ["pack", "--dry-run", "--json"], {
   encoding: "utf8", env: { ...process.env, npm_config_cache: cache },