Skip to content

feat(assets): high-quality assets go through the cook — embedded WebP (PRD-485) - #425

Closed
jonit-dev wants to merge 15 commits into
developfrom
feat/prd-485-hq-assets-cook
Closed

jonit-dev wants to merge 15 commits into
developfrom
feat/prd-485-hq-assets-cook

Conversation

@jonit-dev

Copy link
Copy Markdown
Contributor

TL;DR — runbook row B2 (PRD-485): Fab/Megascans-quality assets go through the default cook instead of around it. Phase 1 box 1 is in: an embedded EXT_texture_webp image now cooks like PNG — to KTX2 on a decoding target, to a capped PNG on the decoder-free path — and the extension leaves the output. That removes the WebP reason lumen-hall gave for models: "none".

PRD: docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md.

Done

  • decodeImageBytes decodes WebP with sharp (already declared in packages/assets, previously unused; loads in the pass worker, stays external in dist, no new binary for games — @gltf-transform/cli already brings it). EXT_texture_webp disposed once no WebP image remains. Spec red → green 3/3.

Missing

  • lumen-hall web build without models: "none" + its asset scenario (sandbox repo, GPU run).
  • Phase 2 (Fab Hornbeam default cook, AutoLOD chain ≤10%), Phase 3 (Android V8 after VQ-01, metahuman-lab, docs).
  • 🎨👁 pnpm visuals:ab --raters 3 once a template-visible change lands.

Evidence

  • packages/assets 474 passed / 2 skipped; pnpm typecheck, pnpm lint, assets build + publint green.

decodeImageBytes reads WebP through sharp — already declared in this package's
dependencies and unused until now, so no dependency and no new native binary
reaches a game (every template already ships the same one via @gltf-transform/cli).
An embedded WebP now cooks like a PNG: KTX2 on a target that decodes it, a capped
PNG on the decoder-free path, and the pass disposes EXT_texture_webp once no image
is left in it because the glTF writer emits every attached extension.

The pass's "never grow a native dependency" rule changes with it: this is a
Node-only build-time package and sharp is the only native thing in it.

Gates: red before (TN_ASSETS_TEXTURE_CONTAINER: 'character.glb#checker' is not a
PNG or JPEG), green after — 26/26 model-texture-pass, 474 passed / 2 skipped across
packages/assets/__tests__, pnpm typecheck, pnpm lint, assets build + publint.
@jonit-dev jonit-dev added the prd:25% PRD phase 1 verified label Oct 3, 2026
…85 phase 1)

lumen-hall's lady.glb and sanctuary.glb cook with `compression skipped:
not-smaller` three times each, so the shipped GLBs kept `images:
image/webp` and required `EXT_texture_webp` — a container the native
desktop/Android/iOS decoder-free paths cannot read.

Retention is now conditioned on the container as well as on size: only
`UNIVERSAL_IMAGE_CONTAINERS` (PNG, JPEG) may keep authored bytes, so a
WebP always converts — to KTX2 on a decoding target even when that is
larger, or to the capped PNG on the decoder-free path. `EXT_texture_webp`
then always leaves the output, and the size increase is reported by the
existing `bytes before -> after` row of the build report (+621% on
lady.glb), with no new marker. PNG/JPEG behaviour is unchanged.

`keepSmallerSource` moves from `true` to `"universal-containers"` in the
model pass's cache key and in the shared-image store settings: a warm cook
keyed on the old rule would otherwise keep shipping the WebP-requiring
models this replaces.

Verified: red (519 -> 519 bytes retained, extension still required) then
green in the `embedded EXT_texture_webp` block of
`packages/assets/__tests__/model-texture-pass.spec.ts`, whose new PNG leg
pins that a PNG whose KTX2 would be larger is still kept; file 28/28,
`packages/assets` 476 passed, `pnpm typecheck` and `pnpm lint` clean.
three clones through `new this.constructor().copy(source)`, so `ClusteredMesh`
died in its own constructor's table check before a game drew anything: the
default cook puts a cluster DAG in every model, and lumen-hall's authored
statues stood a second copy of one.

`clone()` now builds from the source's own arguments, and `copy()` completes
the job the three way: the baked table and the vertex attributes are shared —
one copy of the vertices on the GPU, since three keys attributes on the
attribute — while the cut is the copy's own, because the cut lives in the
geometry's index buffer and two meshes on one geometry would overwrite each
other's every frame. `ClusteredBatchRoot.clone()` keeps its batch, so a copied
root is still cut instead of freezing on the cut it was cloned with.

Red: 5 failing specs on `packages/core/__tests__/clustered*.spec.ts`, the
game's TypeError. Green: 49 in those four files, 2322 across packages/core,
plus typecheck, lint and build.
@jonit-dev jonit-dev removed the prd:25% PRD phase 1 verified label Oct 4, 2026
@jonit-dev jonit-dev added the prd:25% PRD phase 1 verified label Oct 4, 2026
The Hornbeam needed no engine fix: on the tarballs packed from this branch the
default cook exits 0 in 163 s (127 MB -> 27 MB, 6 embedded PNGs to KTX2, 0
uncooked bytes) and its trunk primitive gets a 3-rung TN_discrete_lod chain with
a 1,500-triangle far rung. Box 3 (browser + native scenarios) stays open, and
the PRD now records that the 745k-triangle MASK foliage took TN_virtual_geometry
with stalled cluster levels rather than a chain.
…D (PRD-485)

The Hornbeam's two MASK foliage primitives (225,310 + 520,018 = 745,328 of
755,677 triangles) shipped with no coarse representation at all. The cluster
builder simplifies every group with `LockBorder` (dag.ts:259) and every edge of a
disconnected alpha-cutout card is a border, so the first fold simplifies nothing
(dag.ts:461), dag.ts:319 records `stalled`, and every level reports the source
count. The primitive was then `virtual-owned` (eligibility.ts:182), so AutoLOD
declined it and the asset's only chain was the 10,349-triangle trunk.

`bakeVirtualGeometry` now declines a primitive whose DAG stopped `stalled` while
still holding more than 80% of its triangles: no extension is attached, so the
primitive is unowned and the discrete ladder's card + terminal levels — the path
built for exactly this shape (generate.ts:851) — take it. Measured, not assumed: a
DAG that stalls after real folds has shed its triangles and keeps the primitive.

Red: on a 16,384-card MASK fixture the cook emitted `TN_virtual_geometry`,
`lod.reasons ["virtual-owned"]` and no chain; the report named the bake as "no
primitive was dense enough". Green: no extension, `lod.reasons []`, a 3-level card
chain whose far rung is 1,638 triangles = 5.0% of LOD0 with `cellCoverage 1` on
every level, and the alpha cutoff the asset shipped unchanged.

Sandbox proof (fab-import-proof, repacked assets tarball): build exit 0, `lod
...: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 0 skipped`,
`TN_virtual_geometry` gone from extensionsUsed, far total 38,767 = 5.13%, cooked
GLB 26,990,484 -> 26,428,684 B.

Gates: packages/assets 480 passed, pnpm typecheck 0, pnpm lint 0 (1032 warnings,
unchanged), pnpm check:docs 0.
@jonit-dev jonit-dev added prd:50% PRD half the phases verified and removed prd:25% PRD phase 1 verified labels Oct 4, 2026
@jonit-dev jonit-dev removed the prd:50% PRD half the phases verified label Oct 4, 2026
jonit-dev and others added 4 commits October 3, 2026 18:45
Phase 3 box 3. `finding-assets.md` already said the cook picks imported
GLBs up with no configuration; it now says what the cook does with them and
why `models: "none"` / `textures: "none"` is the wrong repair: a converted
Fab pack's embedded PNG/WebP maps become KTX2, its attributes are quantized
and Meshopt-compressed, and a `MASK` primitive gets a discrete LOD chain.
Turning a pass off ships every source byte verbatim and counts it all
against `assets.budget.uncooked`, which is what raises
TN_ASSETS_BUDGET_EXCEEDED. Measured on the Hornbeam: 126,934,280 ->
26,428,684 B, six embedded PNGs 45.4 MB -> 16.4 MB, 755,677 triangles to a
5.1% far rung.

Also names the three cook outputs a content-addressed glob cannot match
(`public/basis/`, `public/shared/`, `public/bake.receipt.json`), found by
building metahuman-lab's 301 MB tree through the cook.

The Phase 3 box 2 build leg is recorded beside its box, unticked: the cook
is exit 0 with 0 uncooked bytes, and the playtest is for the browser lane.

Proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` 7/7,
`pnpm check:docs` 2,229 links, plus check-doc-links/sync-agent-docs/
evidence-citations 30/30. No template file changed, so no scaffold hash moved.
…ok (PRD-485)

`packages/assets/src/passes/model.ts:925` called the library's `reorder` on the
whole document, which renumbers the vertices of every primitive it finds. That is
the cheapest transmission-size win in the cook and it is invisible until
something outside the file holds a vertex index: the metahuman-lab's
`brows.skin.json` names the head vertex each brow strand root rides and reads its
position back every frame, and after the cook it indexed a different vertex
(`brow root 0 indexes head vertex 11498, which is not where it was prepared`).
Measured on the 26.3 MB Ada specimen, 87 of 24408 head-primitive slots survived
the reorder.

The cook cannot know who holds an index, so the rule is the broader one: **any
primitive whose vertex indices can be addressed from outside is undeclarable**,
and deforming — skinned, morph-targeted, or joint-bound — is the measurable proxy,
because it is the one case detectable from the file alone and a rig is where
index-addressed data comes from. `deforming()` in `lod/eligibility.ts` is now the
single definition of that word, shared with the discrete-LOD and joined-rung
rules that already refused the same geometry.

`reorder-static.ts` runs the remap per shared index buffer and skips any group
holding a deforming primitive. Compression is untouched:
`EXT_meshopt_compression` encodes whatever order it is handed, and `quantize`
renumbers nothing — `specimen.glb` cooks 16,104,464 -> 16,114,804 B (+0.06%), the
price of leaving the head's 24408 vertices where the sidecar expects them.
`MODEL_REORDER_VERSION` joins the pass configuration, or a warm compile entry
would keep serving the renumbered files this replaces.

Red/green in `packages/assets/__tests__/model-pass.spec.ts`: a skinned primitive
and a morph-targeted one come out of the cook with position k exactly where it
started and the same vertex count, while a static mesh in the same document is
still reordered (and still only reordered). `packages/assets/__tests__` 481
passed, `pnpm typecheck` and `pnpm lint` clean.
…through the cook (PRD-485)

`packages/assets/src/passes/model.ts:964` handed the whole document to the library's
`quantize`, which rewrites POSITION into normalized int16 snapped to a grid whose step is
the mesh's own bounding volume and then folds that shift into the node holding the mesh, the
skin's inverse bind matrices, GPU instancing batches and any volumetric material. Exact for a
static mesh; for a rigged one it breaks the invariant this repo holds, one metre is one metre.
The metahuman-lab's `brows.skin.json` names the head vertex each brow strand root rides and
reads its position back every frame, and the sidecar was prepared against the source file to
1e-5 m — the 16-bit grid over a head-sized volume is micrometres away from every vertex it
recorded, and the shift that would undo it lives in a node transform the sidecar never sees.

So the same `deforming()` rule that stopped the cook renumbering a rigged vertex now stops it
quantizing one: a deforming mesh keeps POSITION and its morph-target POSITION deltas as the
float32 values the author wrote, at every requested depth. `quantize-static.ts` holds those
meshes across the stage and spends the window with each mesh detached from its nodes, so the
library's node, skin, instancing and thickness compensation is never applied and there is
nothing to undo; normals, UVs, colours and every static primitive still quantize, and
`EXT_meshopt_compression` still encodes whatever the accessors hold (its `QUANTIZE` encoder
method only names the pre-processing, which is this stage). The unit of source space is the
mesh, because the compensation is per mesh — one deforming primitive holds the whole thing.
An index buffer the stage replaced fails the cook by name rather than shipping a mesh whose
vertices have quietly moved, and `snapPositions` skips rigged meshes for the same reason it
exists to snap them. `MODEL_QUANTIZE_VERSION` joins the pass configuration so a warm compile
entry cannot keep serving the quantized files this replaces.

Red/green in `packages/assets/__tests__/model-pass.spec.ts`: a skinned primitive that also
carries a morph target comes out of the cook with float32 POSITION equal to the source within
1e-6, float32 deltas, and its skin's inverse bind matrices untouched, while a static mesh in
the same document still quantizes to normalized int16 under KHR_mesh_quantization. The 4-bit
floor test now states what is true of it: that request used to land on the fixture's skinned
head and trip the self-verify at 0.167% drift; it no longer reaches deforming geometry at all,
and the fail-closed comparison is covered directly by the drift-kind test. `packages/assets/
__tests__` 482 passed | 2 skipped, `pnpm typecheck` clean, biome clean on the four files.
@jonit-dev jonit-dev added the prd:75% PRD phases in, acceptance open label Oct 4, 2026
@jonit-dev

Copy link
Copy Markdown
Contributor Author

Consolidated into #396 at 2d9da5e71e73ba2578ea4425cd948c9d855cd74b to share one native-assets/default-cook CI board. GitHub compare verifies exact published head 7876e4b7bc0caf853650965ed3b0513deb877506 is preserved as an ancestor. The original PR body/checklist is retained verbatim in #396; incomplete PRD-485 game/Android acceptance and VQ-01 GPU lifecycle criteria remain open. Branch feat/prd-485-hq-assets-cook is retained; owner checkout untouched.

@jonit-dev jonit-dev closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

prd:75% PRD phases in, acceptance open

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant