diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 9f042a7dca..817e2d2d5c 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -46,21 +46,23 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges ## Execution Phases #### Phase 1: Embedded WebP cooks -**Status:** NOT STARTED +**Status:** DONE 2026-10-03 **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` -- [ ] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. -- [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. +- [x] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. **Reopened 2026-10-03:** on `lumen-hall` the cook decodes the WebP but the `not-smaller` guard keeps the original (3× per prop at 1024²), so `lady.glb`/`sanctuary.glb` still require `EXT_texture_webp`. **Closed 2026-10-03:** retention is now conditioned on the container, not only on size — `UNIVERSAL_IMAGE_CONTAINERS` (PNG/JPEG) may keep authored bytes, WebP always converts to KTX2 (KTX2 target) or PNG (decoder-free), so `EXT_texture_webp` always leaves the output and the increase lands in the existing `bytes before -> after` row (+621% on `lady.glb`). Red: 519 -> 519 bytes retained with the extension still required; green: `image/ktx2` only, `extensionsRequired` `[EXT_meshopt_compression, KHR_mesh_quantization, KHR_texture_basisu]`, both GLBs re-verified in `lumen-hall`. `keepSmallerSource` moved from `true` to `"universal-containers"` in the model pass's cache key so a warm cook cannot keep shipping the old output. File 28/28, `packages/assets` 476 passed, `pnpm typecheck` and `pnpm lint` clean. + Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. +- [x] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. Partly done 2026-10-03: `npx threenative build --target web` exits 0 with the installed `@threenative/assets` tarball and both WebP props cook (`lady.df65302d.glb` 1562 KB, `sanctuary.9353c235.glb` 2546 KB, `image/ktx2` only). The scenario itself was not run in this lane, so the box stays open. **Result 2026-10-03:** sandbox branch `prd485/lumen-hall-default-cook` (local worktree, engine tarballs from this branch): `npx threenative build --target web` exit 0, all 6 GLBs cooked (33.96 MB → 8.93 MB, -73.7%); `lady.glb`/`sanctuary.glb` now `image/ktx2`, no `EXT_texture_webp` (`c284f2a70`; lady +621% bytes, block-compressed on GPU). The first scenario run crashed in `ClusteredMesh.clone()` (cooked models carry a cluster DAG; three's `clone()` constructs with no arguments) — fixed in core `7ed40dbe5` with red-green specs. Then `playtests/assets.playtest.json` passes 4/4 on nvidia/turing WebGPU. Same-pose before (`models: "none"` + a temporary uncooked-budget override, same tarballs) vs after: 0.01% of pixels differ (mean abs 0.25). On the new engine `models: "none"` alone no longer builds lumen-hall (`TN_ASSETS_BUDGET_EXCEEDED` 68 MB > 64 MB). Known corner: a WebP that is not a whole number of 4×4 blocks keeps its bytes under the block-size guard. #### Phase 2: The Fab tree cooks by default -**Status:** NOT STARTED +**Status:** PARTIAL **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) -- [ ] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. -- [ ] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. -- [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. +- [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. +- [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03 (reopened box, closed):** cause `packages/assets/src/virtual/dag.ts:319` — the group fold simplifies each cluster group with `LockBorder` (`dag.ts:259`), and every edge of a disconnected alpha-cutout card is a border, so the first fold simplifies nothing, every level reports the source count and `stopReason: stalled`; the primitive was then `virtual-owned` (`lod/eligibility.ts:182`) and no ladder could take it. Fix `packages/assets/src/virtual/bake.ts`: the bake *declines* a primitive whose DAG stopped `stalled` while still holding >80% of its triangles (`VIRTUAL_BAKE_VERSION` 3 -> 4) and attaches no extension, so the primitive is unowned and the discrete ladder's card + terminal levels take it. Red on `virtual-pass.spec.ts` "hands cards it cannot reduce to the discrete ladder, which does reduce them" (16,384 MASK cards: `extensionsUsed` had `TN_virtual_geometry`, `lod.reasons ["virtual-owned"]`, far level 50%) and on the report line; green after (no `TN_virtual_geometry`, `lod.reasons []`, far level 1,638 = 5.0% of 32,768, `cellCoverage 1` on every level). Same lane, sandbox `prd485/fab-import-default-cook` with the repacked assets tarball: `npx threenative build --target web` exit 0, report row `lod …: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 7304028 payload bytes, 0 skipped` and `TN_virtual_geometry` gone from `extensionsUsed`. Per primitive: material 0 `225310 -> [113601, 56948, 11266]` (5.00%, absolute far error 0.331), material 1 `520018 -> [260988, 130580, 26001]` (5.00%, 0.253), material 2 `10349 -> [5190, 2595, 1500]` (14.49%, 0.211) — far total 38,767 = 5.13% of 755,677. Cooked GLB 26,990,484 -> 26,428,684 B (-561,800 B, the DAG payload). Known gap: `compile.ts` drops `entry.virtual` when it rebuilds the manifest entry, so the new `declined` count is in the pass summary and `formatModelSizes` but prints in neither CLI lane. +- [x] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. **Web leg 2026-10-03:** `playtests/hornbeam.playtest.json` passes on nvidia/turing WebGPU (4 Fab imports, Hornbeam height in range, 5 textures, 755,677 LOD0 triangles near, 0 console/network/runtime errors). An earlier bridge-missing failure was the sandbox worktree lacking the gitignored `assets/fab/soul-cave` GLBs (engine failed closed naming the asset). **Native desktop leg 2026-10-03:** `native-playtests/fab-import-native.playtest.json --target desktop` exit 0, 6/6 (4 imports, 8 textured, 5 masked, movement 4.97 m, 0 errors), 690 frames, adapter `native:vulkan/nvidia/NVIDIA GeForce RTX 2080/turing`, packaged from the installed `@threenative/runtime-native` 0.3.4 prebuilt; screenshot shows the Hornbeam canopy. Its first red was stale counts in the scenario (written before the Hornbeam import), corrected in the game branch. #### Phase 3: Android and the cold-agent path -**Status:** NOT STARTED +**Status:** PARTIAL **Files:** `metahuman-lab` asset layout (sandbox); `packages/create-threenative/agent-docs/references/finding-assets.md` - [ ] [local] `fab-import-proof` packages its cooked KTX2 and Meshopt output for Android V8 and renders the Hornbeam on the emulator. proof: `native-playtests/fab-import-native.playtest.json --target android` on the local emulator. -- [ ] [local] `metahuman-lab` serves `content/` from `assets/` through the cook with no `assets` block, and `face-controls.playtest.json` passes on web. proof: `npx threenative build` plus that scenario with `--browser-recipe webgpu`. -- [ ] [local] `finding-assets.md`'s Fab import step says the default cook handles imported packs and that `"none"` is not a fix. proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` and `pnpm check:docs`. +- [x] [local] `metahuman-lab` serves `content/` from `assets/` through the cook with no `assets` block, and `face-controls.playtest.json` passes on web. proof: `npx threenative build` plus that scenario with `--browser-recipe webgpu`. **Result 2026-10-03:** sandbox branch `prd485/metahuman-lab-cook` (local): 301 MB `public/content/` → `assets/content/`, no `assets` block, `npx threenative build --target web` exit 0, uncooked 0 bytes; `.dna`/`.strands.bin` copied through by manifest, 30 PNGs → KTX2. Two engine fixes the cook exposed, each red-green: deforming geometry keeps its vertex order (`d219c5e56`, the library `reorder` renumbered 24,321 of 24,408 head vertices the DNA/brow sidecars address) and its float32 metre positions (`f27cb2299`, quantization had moved the scale into the inverse bind matrices); `specimen.glb` 26.3 MB → 22.5 MB with all 821 morph targets. `playtests/face-controls.playtest.json` passes 15/15 on WebGPU, 0 console errors; the frame shows jaw/smile/blink/brow controls driving the face with hair and skin intact. + Build leg 2026-10-03: sandbox branch `prd485/metahuman-lab-cook` (local worktree, engine tarballs from this branch). `npx threenative build --target web` exit 0 in 189 s, `threenative.config.ts` still carries no `assets` block. The 301 MB moved from `public/content/` to `assets/content/`, and the hand-written vite middleware that served it verbatim (`specimenContentPlugin`) is gone: `ctx.assets.resolve` reads `public/assets.manifest.json` and every load asks for the cooked name. `.dna` and `.strands.bin` are copied through as `kind: "other"` byte for byte (`head.dna` 4,829,103 B, `hair.strands.bin` 85,202,464 B) — an extension no pass owns gets a manifest entry like any other file. `specimen.glb` cooks 26,267,896 -> 16,104,464 B (-38.7%): 5 embedded PNGs 7,801,033 -> 2,957,889 B (3 uastc/etc1s, `T_MH_BentNormal_AO` and `T_Eye_N` kept under the existing `not-smaller` guard), 101,518 triangles and all 821 LOD0 morph targets intact, `EXT_meshopt_compression` + `KHR_mesh_quantization` + `KHR_texture_basisu` on the output, and `lod.reasons ["deforming", "authored-lod"]` — a skinned morph-target head is PRD-486's subject, not this one. `shirt.glb` 50,367,028 -> 543,040 B (-98.9%), `body.glb` 1,215,524 -> 349,680 B, 30 PNGs to KTX2 (total 162,186,687 B from 314,889,680 B of source). Budget `uncooked 0 bytes` against the 64,000,000 default ceiling — first time this game had a manifest at all. The scenario was not run in this lane, so the box stays open: `node packages/playtest/dist/runner/cli.js metahuman-lab/playtests/face-controls.playtest.json --url http://127.0.0.1:5197 --server-command "pnpm exec vite preview --host 127.0.0.1 --port 5197 --strictPort" --browser-recipe webgpu`. +- [x] [local] `finding-assets.md`'s Fab import step says the default cook handles imported packs and that `"none"` is not a fix. proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` and `pnpm check:docs`. **Result 2026-10-03:** the Fab import step now states what the default cook does to a converted pack (embedded PNG and WebP to KTX2, quantization and Meshopt, a discrete LOD chain for a `MASK` primitive) with the measured Hornbeam numbers, and that `models: "none"`/`textures: "none"` are not a fix — they ship every source byte, count it all against `assets.budget.uncooked`, and are what makes the build fail with `TN_ASSETS_BUDGET_EXCEEDED`. Green: `primary-docs.spec.ts` 7/7, `check:docs` 2,229 links, `check-doc-links` + `sync-agent-docs` + `evidence-citations` 30/30. No template file changed, so no scaffold hash moved. diff --git a/packages/assets/__tests__/model-pass.spec.ts b/packages/assets/__tests__/model-pass.spec.ts index 212e0969dc..deda4921ea 100644 --- a/packages/assets/__tests__/model-pass.spec.ts +++ b/packages/assets/__tests__/model-pass.spec.ts @@ -89,14 +89,68 @@ describe("modelPass", () => { } }); - it("should throw naming bounding-box drift when positions quantize to 4 bits", async () => { - const input = Buffer.from(await buildFixtureGlb()); - await expect( - modelPass({ quantize: { normalBits: 16, positionBits: 4, uvBits: 12 } }).apply( - input, - "character.glb", - ), - ).rejects.toThrow(/TN_ASSETS_MODEL_DRIFT.*bounding box drifted/u); + it("should leave deforming positions alone at a depth below the quantization floor", async () => { + // Four bits is the destructive pre-round the library refuses to express: the pass snaps floats + // onto a 15-level grid over each accessor's own bounds and lets the self-verify reject the + // result. That request used to land on the fixture's skinned head — and used to trip the + // self-verify at 0.167% drift. Deforming geometry is now held in source space at every depth, + // so the request never reaches it and there is nothing left to reject; the fail-closed path + // itself is covered by the drift-kind comparison further down. The static half of the document + // still quantizes, which is what keeps the two halves of the rule honest. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const head = root.listMeshes()[0]?.listPrimitives()[1]; + const buffer = head?.getAttribute("POSITION")?.getBuffer() ?? null; + if (head === undefined || buffer === null) throw new Error("Fixture lost its rig."); + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray( + Uint32Array.from([ + 0, 10, 20, 1, 11, 21, 2, 12, 22, 3, 13, 23, 4, 14, 24, 5, 15, 25, 6, 16, 26, 7, 17, 8, + 18, 19, 9, + ]), + ), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 81 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + const positions = [...(head.getAttribute("POSITION")?.getArray() ?? [])]; + const result = await modelPass({ + compact: false, + quantize: { normalBits: 16, positionBits: 4, uvBits: 12 }, + textures: "none", + virtual: "none", + }).apply(Buffer.from(await new NodeIO().writeBinary(document)), "character.glb"); + expect(Buffer.isBuffer(result)).toBe(false); + if (Buffer.isBuffer(result)) return; + const output = await readVerified(result.buffer); + expect([ + ...(output.listMeshes()[0]?.listPrimitives()[1]?.getAttribute("POSITION")?.getArray() ?? []), + ]).toEqual(positions); + expect( + output + .listMeshes() + .find((mesh) => mesh.getName() === "prop") + ?.listPrimitives()[0] + ?.getAttribute("POSITION") + ?.getComponentSize(), + // Eight bits is the floor the request resolves to, and it lands as normalized Int8. + ).toBe(1); }); it("should declare EXT_meshopt_compression and not Draco", async () => { @@ -151,6 +205,209 @@ describe("modelPass", () => { } }); + it("should keep a deforming primitive's vertex order and still reorder a static one", async () => { + // A vertex index is an address, and something outside this cook may already hold one: a + // MetaHuman sidecar records the head vertex each brow strand root rides, and reads the position + // back at that index every frame. The cook cannot know who holds an index, so deforming + // geometry — skinned, morph-targeted, or carrying joint data — keeps its exact vertex order and + // count, and everything else is still reordered for transmission size. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const character = root.listMeshes()[0]; + const cloth = character?.listPrimitives()[0]; + const head = character?.listPrimitives()[1]; + if (cloth === undefined || head === undefined) throw new Error("Fixture lost a primitive."); + // Both reasons in one cook: the cloth stays skinned-only, the head also gains a morph target. + const buffer = head.getAttribute("POSITION")?.getBuffer() ?? null; + head.addTarget( + document.createPrimitiveTarget("head-jaw-open").setAttribute( + "POSITION", + document + .createAccessor("head-jaw-open-deltas") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 18 }, (_, index) => (index % 3) * 0.01))), + ), + ); + // A static mesh whose index buffer is not already in draw order, so the reorder has real work + // to do on it: the jumbled order below is a non-identity meshoptimizer remap. + const staticIndices = [6, 1, 8, 0, 7, 3, 2, 4, 5]; + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray(Uint32Array.from(staticIndices)), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 27 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + + const source = (meshName: string, primitive: number): number[] => [ + ...(root + .listMeshes() + .find((mesh) => mesh.getName() === meshName) + ?.listPrimitives() + [primitive]?.getAttribute("POSITION") + ?.getArray() ?? []), + ]; + const before = { + cloth: source("character", 0), + head: source("character", 1), + prop: source("prop", 0), + }; + // Lossless options, so the only stage that can move a vertex is `reorder` itself. + const result = await modelPass({ + compact: false, + passes: { quantize: false }, + textures: "none", + virtual: "none", + }).apply(Buffer.from(await new NodeIO().writeBinary(document)), "character.glb"); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + const output = await readVerified(result.buffer); + const cooked = (meshName: string, primitive: number): number[] => [ + ...(output + .listMeshes() + .find((mesh) => mesh.getName() === meshName) + ?.listPrimitives() + [primitive]?.getAttribute("POSITION") + ?.getArray() ?? []), + ]; + + expect(cooked("character", 0)).toEqual(before.cloth); + expect(cooked("character", 1)).toEqual(before.head); + // And the static half of the rule: reordered, and nothing but reordered. + expect(cooked("prop", 0)).not.toEqual(before.prop); + expect([...cooked("prop", 0)].sort((a, b) => a - b)).toEqual( + [...before.prop].sort((a, b) => a - b), + ); + }); + + it("should keep a deforming primitive's positions in source metres and still quantize a static one", async () => { + // One metre is one metre. `quantize` snaps POSITION onto a grid whose step is the mesh's own + // volume over 2^bits, so a head cooked at 16 bits over half a metre lands within ~8 µm of + // where the author put it — and the metahuman-lab sidecar, which names the head vertex each + // brow strand root rides and reads that position back every frame, was prepared to 1e-5 m. + // Quantization also compensates by folding a shift into the node, the skin's inverse bind + // matrices and any volumetric material, so the whole mesh stays in source space instead: + // deforming POSITION and its morph-target deltas keep float32, everything else still quantizes. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const character = root.listMeshes().find((mesh) => mesh.getName() === "character"); + const cloth = character?.listPrimitives()[0]; + const head = character?.listPrimitives()[1]; + const skin = root.listSkins()[0]; + const buffer = head?.getAttribute("POSITION")?.getBuffer() ?? null; + if (cloth === undefined || head === undefined || skin === undefined || buffer === null) + throw new Error("Fixture lost its rig."); + // Both reasons in one cook, as in the vertex-order twin: the head is skinned and morph-targeted. + const deltas = new Float32Array(Array.from({ length: 18 }, (_, index) => (index % 3) * 0.004)); + head.addTarget( + document + .createPrimitiveTarget("jaw-open") + .setAttribute( + "POSITION", + document + .createAccessor("jaw-open-deltas") + .setBuffer(buffer) + .setType("VEC3") + .setArray(deltas), + ), + ); + // A static mesh in the same document, on its own node: the half of the rule that must not regress. + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray( + // Every vertex referenced exactly once, so the library's own vertex compaction inside + // `quantize` has nothing to drop and the bbox the self-verify reads is the authored one. + Uint32Array.from([ + 0, 10, 20, 1, 11, 21, 2, 12, 22, 3, 13, 23, 4, 14, 24, 5, 15, 25, 6, 16, 26, 7, 17, 8, + 18, 19, 9, + ]), + ), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + // 27 vertices, every one of them referenced by the indices above. + .setArray(new Float32Array(Array.from({ length: 81 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + + const positions = [...(head.getAttribute("POSITION")?.getArray() ?? [])]; + const binds = [...(skin.getInverseBindMatrices()?.getArray() ?? [])]; + const result = await modelPass({ compact: false, textures: "none", virtual: "none" }).apply( + Buffer.from(await new NodeIO().writeBinary(document)), + "character.glb", + ); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + const output = await readVerified(result.buffer); + const cookedHead = output + .listMeshes() + .find((mesh) => mesh.getName() === "character") + ?.listPrimitives()[1]; + const cookedCloth = output + .listMeshes() + .find((mesh) => mesh.getName() === "character") + ?.listPrimitives()[0]; + const cookedProp = output + .listMeshes() + .find((mesh) => mesh.getName() === "prop") + ?.listPrimitives()[0]; + + const cookedPositions = cookedHead?.getAttribute("POSITION"); + expect(cookedPositions?.getComponentType()).toBe(5126); + const after = [...(cookedPositions?.getArray() ?? [])]; + expect(after.length).toBe(positions.length); + for (const [index, value] of after.entries()) + expect(Math.abs((value as number) - (positions[index] as number))).toBeLessThan(1e-6); + // The deltas ride the same space, so they keep their float32 metres too. + const cookedDeltas = cookedHead?.listTargets()[0]?.getAttribute("POSITION"); + expect(cookedDeltas?.getComponentType()).toBe(5126); + expect([...(cookedDeltas?.getArray() ?? [])]).toEqual([...deltas]); + // Nothing moved to compensate: the bind poses the deformation is measured against are the + // authored ones, and the normals that do not address a vertex still quantized. + const cookedBinds = [...(output.listSkins()[0]?.getInverseBindMatrices()?.getArray() ?? [])]; + expect(cookedBinds.length).toBe(binds.length); + for (const [index, value] of cookedBinds.entries()) + expect(Math.abs((value as number) - (binds[index] as number))).toBeLessThan(1e-6); + expect(cookedCloth?.getAttribute("POSITION")?.getComponentType()).toBe(5126); + expect(cookedCloth?.getAttribute("NORMAL")?.getComponentSize()).toBeLessThan(4); + // And the static half of the rule, unchanged: quantized, normalized, KHR_mesh_quantization on. + expect(cookedProp?.getAttribute("POSITION")?.getComponentSize()).toBe(2); + expect(cookedProp?.getAttribute("POSITION")?.getNormalized()).toBe(true); + expect( + output + .listExtensionsUsed() + .map((extension) => extension.extensionName) + .includes("KHR_mesh_quantization"), + ).toBe(true); + }); + it("decodes normalized ushort joint weights by their own component type in reachableStats", async () => { // External exporters ship WEIGHTS_0 as normalized UNSIGNED_SHORT. reachableStats must // decode them /65535 like the GPU does; the hardcoded ubyte scale (/255) reads raw diff --git a/packages/assets/__tests__/model-texture-pass.spec.ts b/packages/assets/__tests__/model-texture-pass.spec.ts index 7458841893..3a3ee87cf9 100644 --- a/packages/assets/__tests__/model-texture-pass.spec.ts +++ b/packages/assets/__tests__/model-texture-pass.spec.ts @@ -1,9 +1,11 @@ import { mkdir, readFile, stat, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; import path from "node:path"; import { type Document, NodeIO } from "@gltf-transform/core"; -import { ALL_EXTENSIONS } from "@gltf-transform/extensions"; +import { ALL_EXTENSIONS, EXTTextureWebP } from "@gltf-transform/extensions"; import { read as readKTX2 } from "ktx-parse"; import { MeshoptDecoder, MeshoptEncoder } from "meshoptimizer"; +import { PNG } from "pngjs"; import { describe, expect, it, vi } from "vitest"; import { buildFixtureDocument } from "../../../test-support/generate-fixture-model.js"; import { rgbaPng } from "../../../test-support/png.js"; @@ -81,6 +83,118 @@ async function compiled( return { buffer: result.buffer, entry: result.entry ?? {} }; } +/** + * Two flat halves, never one solid colour: `prune` deletes a single-colour texture and folds it + * into the material factor, which would leave these cases measuring a material and no image. + */ +const WEBP_HALVES = [ + [200, 130, 40], + [20, 60, 220], +] as const; + +/** A lossless WebP of the requested size, encoded by the decoder the pass uses. */ +async function tinyWebp(size = 4): Promise { + const sharp = (await import("sharp")).default; + const pixels = Buffer.alloc(size * size * 4); + for (let index = 0; index < size * size; index += 1) { + pixels.set([...WEBP_HALVES[index % size < size / 2 ? 0 : 1], 255], index * 4); + } + return sharp(pixels, { raw: { channels: 4, height: size, width: size } }) + .webp({ lossless: true }) + .toBuffer(); +} + +/** + * The shared fixture with its base colour replaced by an `EXT_texture_webp` image, which is what + * a Fab or McD export ships: the extension declared and required, the payload in a WebP. + */ +async function fixtureWithWebp(size?: number): Promise { + const document = buildFixtureDocument(); + document.createExtension(EXTTextureWebP).setRequired(true); + document + .getRoot() + .listTextures()[0] + ?.setImage(await tinyWebp(size)) + .setMimeType("image/webp"); + // The writer only emits the extensions it was told about, and an undeclared `image/webp` + // would be the invalid document this fixture exists to avoid. + return Buffer.from(await new NodeIO().registerExtensions(ALL_EXTENSIONS).writeBinary(document)); +} + +interface IBasisModule { + readonly KTX2File: new ( + bytes: Uint8Array, + ) => { + close(): void; + getHeight(): number; + getImageTranscodedSizeInBytes(mip: number, layer: number, face: number, format: number): number; + getWidth(): number; + isValid(): boolean; + startTranscoding(): boolean; + transcodeImage( + dst: Uint8Array, + mip: number, + layer: number, + face: number, + format: number, + getAlphaForOpaqueFormats: number, + channel0: number, + channel1: number, + ): boolean; + }; + readonly initializeBasis: () => void; + readonly transcoder_texture_format: Readonly>; +} + +/** + * Mip 0 of a KTX2 as RGBA8, decoded by the same `basis_transcoder` the runtime ships. Three + * publishes it CommonJS inside an ESM package, so it is evaluated here the way its own wrapper + * does — the alternative is trusting the container header instead of the pixels. + */ +async function transcodeFirstLevel(ktx2: Uint8Array): Promise { + const paths = basisTranscoderPaths(); + const directory = path.dirname(paths.javascriptPath); + const source = await readFile(paths.javascriptPath, "utf8"); + const shim: { exports: unknown } = { exports: {} }; + new Function("module", "exports", "require", "__filename", "__dirname", source)( + shim, + shim.exports, + createRequire(import.meta.url), + paths.javascriptPath, + directory, + ); + const factory = shim.exports as (options: { + locateFile: (file: string) => string; + }) => Promise; + const basis = await factory({ locateFile: (file) => path.join(directory, file) }); + basis.initializeBasis(); + const file = new basis.KTX2File(ktx2); + // The emscripten bindings report booleans as numbers, so both calls are judged on truth. + expect(file.isValid()).toBeTruthy(); + expect(file.startTranscoding()).toBeTruthy(); + // The WASM enum's numbering is not three's TranscoderFormat map, so the shipped enum names it. + const rgba32 = basis.transcoder_texture_format.cTFRGBA32; + if (rgba32 === undefined) throw new Error("the shipped transcoder exposes no cTFRGBA32 format"); + const format = rgba32.value; + const pixels = new Uint8Array(file.getImageTranscodedSizeInBytes(0, 0, 0, format)); + expect(file.transcodeImage(pixels, 0, 0, 0, format, 0, -1, -1)).toBeTruthy(); + file.close(); + return pixels; +} + +/** Every channel within `tolerance` of the source half the pixel belongs to. */ +function expectColours(pixels: Uint8Array, width: number, tolerance: number): void { + expect(pixels.length).toBe(width * width * 4); + for (let offset = 0; offset < pixels.length; offset += 4) { + const half = WEBP_HALVES[(offset / 4) % width < width / 2 ? 0 : 1]; + for (let channel = 0; channel < 3; channel += 1) { + expect(Math.abs((pixels[offset + channel] ?? 0) - (half[channel] ?? 0))).toBeLessThanOrEqual( + tolerance, + ); + } + } +} + describe("embedded model textures", () => { it("should transcode every embedded image to KTX2 and declare KHR_texture_basisu", async () => { const input = await fixtureWithTextures({ width: 32 }); @@ -291,6 +405,141 @@ describe("embedded model textures", () => { }); }); +describe("embedded EXT_texture_webp", () => { + it("should cook a WebP image to KTX2 and drop the extension", async () => { + // A named per-slot codec also skips the keep-smaller-source guard: a 4x4 WebP is smaller + // than the KTX2 container around it, and the test needs the cook to have run. + const { buffer } = await compiled(await fixtureWithWebp(), { + textures: { overrides: [{ codec: "uastc", slot: "baseColorTexture" }] }, + }); + const json = JSON.parse(buffer.subarray(20, 20 + buffer.readUInt32LE(12)).toString("utf8")) as { + extensionsRequired?: string[]; + extensionsUsed?: string[]; + }; + expect(json.extensionsUsed ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).toContain("KHR_texture_basisu"); + + const cooked = (await readOutput(buffer)) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/ktx2"); + const image = cooked?.getImage() ?? new Uint8Array(); + expect(readKTX2(image)).toMatchObject({ pixelHeight: 4, pixelWidth: 4 }); + // UASTC is lossless for 8-bit RGBA, so the transcoded mip is the source colour: proof the + // WebP payload reached the encoder instead of a container that merely looked like one. + expectColours(await transcodeFirstLevel(image), 4, 1); + }); + + it("should cap an over-cap WebP image to PNG on a target with no transcoder", async () => { + const root = await makeTempDir("threenative-webp-decoder-free-"); + await mkdir(path.join(root, "assets")); + // 8x8 against a 4-longest-edge cap: `cappedSize` never returns less than one 4x4 block, so + // only an over-cap source can reach the PNG the decoder-free path emits. + await writeFile(path.join(root, "assets", "prop.glb"), await fixtureWithWebp(8)); + + await compileAssets({ + config: { models: { sharedImages: false, textures: { maxSize: 4 } } }, + cwd: root, + platform: "android", + }); + + const manifest = JSON.parse( + await readFile(path.join(root, "public", "assets.manifest.json"), "utf8"), + ) as { entries: Record> }; + const entry = manifest.entries["prop.glb"]; + if (entry === undefined || typeof entry.output !== "string") { + throw new Error("no manifest entry for 'prop.glb'"); + } + expect(entry.extensions ?? []).not.toContain("EXT_texture_webp"); + + const cooked = (await readOutput(await readFile(path.join(root, "public", entry.output)))) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/png"); + const image = Buffer.from(cooked?.getImage() ?? new Uint8Array()); + expect(parsePng(image)).toMatchObject({ height: 4, width: 4 }); + // A PNG is lossless, so the resampled colour must be the source one. + expectColours(PNG.sync.read(image).data, 4, 1); + }); + + it("should convert a WebP even when its KTX2 is larger", async () => { + // The real-game case: a flat-colour 256x256 WebP costs far fewer bytes than the KTX2 + // container around it, so the keep-smaller-source guard retained the original and the + // cooked model still required `EXT_texture_webp` — which the decoder-free native targets + // cannot read. A container only an extension-reading loader understands is never kept. + const authored = (await readOutput(await fixtureWithWebp(256))) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker") + ?.getImage()?.byteLength; + + const { buffer, entry } = await compiled(await fixtureWithWebp(256)); + const summary = entry.embeddedTextures as + | { + readonly bytesAfter: number; + readonly bytesBefore: number; + readonly formats: Readonly>; + readonly skippedCompression: Readonly>; + } + | undefined; + + // Proof the not-smaller guard really held here: the cook grew the payload, and the row it + // grew is the one the size report already prints. + expect(summary?.bytesAfter ?? 0).toBeGreaterThan(summary?.bytesBefore ?? 0); + expect(summary?.skippedCompression?.checker).toBeUndefined(); + expect(summary?.formats.checker).toBe("etc1s"); + + const json = JSON.parse(buffer.subarray(20, 20 + buffer.readUInt32LE(12)).toString("utf8")) as { + extensionsRequired?: string[]; + extensionsUsed?: string[]; + images: { mimeType?: string }[]; + }; + expect(json.images.some((image) => image.mimeType === "image/webp")).toBe(false); + expect(json.extensionsUsed ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).toContain("KHR_texture_basisu"); + // The image itself, not just the extension name, got bigger: that is the cost this test + // buys, and it is the number a reader of the build report sees. + const cooked = (await readOutput(buffer)) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/ktx2"); + expect(cooked?.getImage()?.byteLength ?? 0).toBeGreaterThan(authored ?? 0); + }); + + it("should still keep a PNG whose KTX2 would be larger", async () => { + // The other half of the same rule, and the reason it is a container test rather than a size + // test: PNG and JPEG need no extension, so retaining them costs a reader nothing. + const { buffer, entry } = await compiled(await fixtureWithTextures({ width: 4 })); + const summary = entry.embeddedTextures as { + readonly skippedCompression: Readonly>; + }; + expect(summary.skippedCompression.checker).toBe("not-smaller"); + for (const texture of (await readOutput(buffer)).getRoot().listTextures()) { + expect(texture.getMimeType()).toBe("image/png"); + } + }); + + it("should still fail closed on a container that is not an image", async () => { + const document = buildFixtureDocument(); + document + .getRoot() + .listTextures()[0] + ?.setImage(Buffer.from("GIF89a not an image at all")) + .setMimeType("image/png"); + const input = Buffer.from(await new NodeIO().writeBinary(document)); + await expect( + apply(input, { + passes: { dedup: false, meshopt: false, prune: false, quantize: false, reorder: false }, + }), + ).rejects.toThrow(/TN_ASSETS_MODEL_TEXTURE_UNDECODABLE.*TN_ASSETS_TEXTURE_CONTAINER/su); + }); +}); + describe("mesh simplification", () => { it("should leave triangles untouched unless simplification is declared", async () => { const input = await fixtureWithTextures({ width: 32 }); diff --git a/packages/assets/__tests__/shared-images.spec.ts b/packages/assets/__tests__/shared-images.spec.ts index 0ad1f1ad55..ff65852103 100644 --- a/packages/assets/__tests__/shared-images.spec.ts +++ b/packages/assets/__tests__/shared-images.spec.ts @@ -214,7 +214,12 @@ describe("shared model images", () => { sharedImageKey(texture.getImage() ?? new Uint8Array(), { colorSpace: getTextureColorSpace(texture), slots: [...listTextureSlots(texture)].sort(), - textures: { keepSmallerSource: true, maxSize: null, overrides: [], quality: null }, + textures: { + keepSmallerSource: "universal-containers", + maxSize: null, + overrides: [], + quality: null, + }, }), ); const store = createSharedImageStore(); diff --git a/packages/assets/__tests__/virtual-dag.spec.ts b/packages/assets/__tests__/virtual-dag.spec.ts index 8e51dbfcd8..97632d3a65 100644 --- a/packages/assets/__tests__/virtual-dag.spec.ts +++ b/packages/assets/__tests__/virtual-dag.spec.ts @@ -41,6 +41,21 @@ function manyShells(count: number): IBody { return { indices: Uint32Array.from(indices), positions: Float32Array.from(positions) }; } +/** Disconnected alpha-cutout cards — the shape a Fab foliage primitive is, and why it stalls. */ +function cards(count: number): IBody { + const indices: number[] = []; + const positions: number[] = []; + const perRow = Math.ceil(Math.sqrt(count)); + for (let card = 0; card < count; card += 1) { + const base = positions.length / 3; + const x = (card % perRow) * 2; + const y = Math.floor(card / perRow) * 2; + positions.push(x, y, 0, x + 0.5, y, 0, x + 0.5, y + 0.5, 0, x, y + 0.5, 0); + indices.push(base, base + 1, base + 2, base, base + 2, base + 3); + } + return { indices: Uint32Array.from(indices), positions: Float32Array.from(positions) }; +} + function edgeKey(a: number, b: number): number { return a < b ? a * 4294967296 + b : b * 4294967296 + a; } @@ -213,6 +228,19 @@ describe("the cluster DAG", () => { expect(dag.roots.length).toBeGreaterThan(1); }, 120_000); + it("AC6 red — a body of alpha-cutout cards stalls on its very first fold, at 100%", async () => { + const body = cards(400); + const dag = await buildClusterDag(body.indices, body.positions); + + // Not the shells' 64-triangle floor: no collapse at all. Every edge of every card is a border + // edge, so `LockBorder` holds the whole body still and the only remaining candidates are the + // quads' diagonals, which collapse to nothing. PRD-485's Hornbeam foliage: 745,328 triangles + // whose DAG therefore reports every level at the source count, and the bake declines it. + expect(dag.stopReason).toBe("stalled"); + expect(dag.groups).toHaveLength(0); + for (const level of dag.levels) expect(level.triangleCount).toBe(body.indices.length / 3); + }, 120_000); + it("refuses a malformed mesh rather than baking nonsense", async () => { await expect(buildClusterDag(new Uint32Array([0, 1]), new Float32Array(9))).rejects.toThrow( /non-empty triangle list/, diff --git a/packages/assets/__tests__/virtual-pass.spec.ts b/packages/assets/__tests__/virtual-pass.spec.ts index 6380b75ee6..bea7d59913 100644 --- a/packages/assets/__tests__/virtual-pass.spec.ts +++ b/packages/assets/__tests__/virtual-pass.spec.ts @@ -1,7 +1,7 @@ import { Document, NodeIO } from "@gltf-transform/core"; import { ALL_EXTENSIONS } from "@gltf-transform/extensions"; import { MeshoptDecoder } from "meshoptimizer"; -import { TorusKnotGeometry } from "three"; +import { SphereGeometry, TorusKnotGeometry } from "three"; import { describe, expect, it } from "vitest"; import { modelPass } from "../src/passes/model.js"; import { formatModelSizes } from "../src/report.js"; @@ -58,6 +58,77 @@ async function sourceGlb(geometry: TorusKnotGeometry): Promise { return Buffer.from(await io.writeBinary(document)); } +/** + * Twelve closed shells that touch nothing, in one primitive: the body whose DAG halves a dozen + * times and then floors out above one cluster, which is the `stalled` the decline must not read as + * "reduced nothing" (PRD-485). + */ +async function shellsGlb(shellCount: number): Promise { + const document = new Document(); + const buffer = document.createBuffer(); + const scene = document.createScene(); + const shell = new SphereGeometry(0.5, 32, 16); + const positions: number[] = []; + const normals: number[] = []; + const indices: number[] = []; + const source = shell.attributes.position?.array ?? []; + const sourceNormals = shell.attributes.normal?.array ?? []; + const shellIndices = Array.from(shell.index?.array ?? []); + // A three.js sphere leaves two pole vertices no triangle references, and the pass's self-verify + // fails a cook that drops them — the same trap `denseGlb` avoids with a torus knot. Compacting to + // the used vertices is the whole fix. + const used = [...new Set(shellIndices)].sort((left, right) => left - right); + const remap = new Map(used.map((vertex, slot) => [vertex, slot])); + for (let count = 0; count < shellCount; count += 1) { + const base = positions.length / 3; + for (const vertex of used) { + const slot = vertex * 3; + positions.push( + (source[slot] as number) + count * 40, + source[slot + 1] as number, + source[slot + 2] as number, + ); + normals.push( + sourceNormals[slot] as number, + sourceNormals[slot + 1] as number, + sourceNormals[slot + 2] as number, + ); + } + for (const vertex of shellIndices) indices.push(base + (remap.get(vertex) as number)); + } + const primitive = document + .createPrimitive() + .setAttribute( + "POSITION", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(positions)) + .setBuffer(buffer), + ) + .setAttribute( + "NORMAL", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(normals)) + .setBuffer(buffer), + ) + .setIndices( + document + .createAccessor() + .setType("SCALAR") + .setArray(Uint32Array.from(indices)) + .setBuffer(buffer), + ) + .setMaterial(document.createMaterial("stone")); + scene.addChild( + document.createNode("cavern").setMesh(document.createMesh("cavern").addPrimitive(primitive)), + ); + const io = new NodeIO().registerExtensions(ALL_EXTENSIONS); + return Buffer.from(await io.writeBinary(document)); +} + async function readBack(buffer: Buffer, registerVirtual: boolean): Promise { await MeshoptDecoder.ready; const io = new NodeIO() @@ -83,6 +154,72 @@ async function compile(virtual: boolean): Promise<{ buffer: Buffer; entry: unkno return { buffer: result.buffer, entry: result.entry }; } +/** + * A canopy of disconnected alpha-cutout cards: what a Fab foliage primitive is, and what the + * Hornbeam's two of them are (PRD-485). + * + * Every card is its own connected component with its own four vertices, so the whole rim is a + * border edge and the cluster builder — which simplifies each group with the rim locked — has + * nothing it is allowed to collapse. The card count is what puts the far level under a tenth of the + * source: the ladder's terminal target is `max(1500, 5% of LOD0)`, and only above 30,000 triangles + * is the ratio the binding half of that. + */ +async function foliageGlb(cardCount: number): Promise { + const document = new Document(); + const buffer = document.createBuffer(); + const scene = document.createScene(); + const positions: number[] = []; + const normals: number[] = []; + const uvs: number[] = []; + const indices: number[] = []; + const perRow = Math.ceil(Math.sqrt(cardCount)); + for (let card = 0; card < cardCount; card += 1) { + const base = positions.length / 3; + const x = (card % perRow) * 2; + const y = Math.floor(card / perRow) * 2; + const height = 0.5 + ((card * 37) % 11) / 22; + positions.push(x, y, 0, x + height, y, 0, x + height, y + height, 0, x, y + height, 0); + normals.push(0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1); + uvs.push(0, 0, 1, 0, 1, 1, 0, 1); + indices.push(base, base + 1, base + 2, base, base + 2, base + 3); + } + const primitive = document + .createPrimitive() + .setAttribute( + "POSITION", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(positions)) + .setBuffer(buffer), + ) + .setAttribute( + "NORMAL", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(normals)) + .setBuffer(buffer), + ) + .setAttribute( + "TEXCOORD_0", + document.createAccessor().setType("VEC2").setArray(Float32Array.from(uvs)).setBuffer(buffer), + ) + .setIndices( + document + .createAccessor() + .setType("SCALAR") + .setArray(Uint32Array.from(indices)) + .setBuffer(buffer), + ) + .setMaterial(document.createMaterial("Foliage").setAlphaMode("MASK").setAlphaCutoff(0.5)); + scene.addChild( + document.createNode("canopy").setMesh(document.createMesh("canopy").addPrimitive(primitive)), + ); + const io = new NodeIO().registerExtensions(ALL_EXTENSIONS); + return Buffer.from(await io.writeBinary(document)); +} + describe("the virtual-geometry pass", () => { it('bakes nothing when the config says "none"', async () => { const { buffer, entry } = await compile(false); @@ -219,6 +356,84 @@ describe("the virtual-geometry pass", () => { expect(summary.skipped).toBe(1); }); + it("hands cards it cannot reduce to the discrete ladder, which does reduce them", async () => { + // PRD-485. A body of disconnected alpha-cutout cards is the one shape the cluster builder + // refuses: every edge of every card is a border, `LockBorder` holds all of it, and the first + // fold simplifies nothing — so a stalled DAG reports 100% of the triangles at every level and + // owns 745,328 Hornbeam foliage triangles with no coarse representation at all. The bake hands + // that primitive back and the ladder gives it the far level the DAG could not. + const result = await modelPass({ + lod: true, + virtual: { minSourceTriangles: 1024 }, + }).apply(await foliageGlb(16384), "canopy.glb"); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + + expect(jsonOf(result.buffer).extensionsUsed ?? []).not.toContain("TN_virtual_geometry"); + const virtual = ( + result.entry as { + virtual: { clusters: number; declined: number; payloadBytes: number; primitives: number }; + } + ).virtual; + expect(virtual).toMatchObject({ clusters: 0, declined: 1, payloadBytes: 0, primitives: 0 }); + + const lod = ( + result.entry as { + lod: { + primitives: { + cardLevels: { cellCoverage: number }[]; + levels: { absoluteError: number; triangles: number }[]; + strategy: string; + trianglesBefore: number; + }[]; + reasons: string[]; + skipped: number; + }; + } + ).lod; + // Not `virtual-owned` any more, and not skipped for anything else either. + expect(lod.reasons).not.toContain("virtual-owned"); + expect(lod.skipped).toBe(0); + const chain = lod.primitives[0]; + if (chain === undefined) throw new Error("the declined cards got no chain at all"); + expect(chain.strategy).toBe("cards"); + + const far = chain.levels[chain.levels.length - 1]?.triangles ?? 0; + expect(far).toBeGreaterThan(0); + expect(far / chain.trianglesBefore).toBeLessThanOrEqual(0.1); + // The crown keeps a card in every occupied cell at every level, and each level's error is + // wider than the one before it — the two properties the runtime's distance switch reads. + for (const level of chain.cardLevels ?? []) expect(level.cellCoverage).toBe(1); + for (let level = 1; level < chain.levels.length; level += 1) + expect(chain.levels[level]?.absoluteError).toBeGreaterThanOrEqual( + chain.levels[level - 1]?.absoluteError ?? 0, + ); + + // And the cook still decides nothing about how it looks: the alpha cutoff the asset was + // authored with is the one that ships. + const material = ( + JSON.parse( + result.buffer.subarray(20, 20 + result.buffer.readUInt32LE(12)).toString("utf8"), + ) as { materials?: { alphaCutoff?: number; alphaMode?: string }[] } + ).materials?.[0]; + expect(material).toMatchObject({ alphaCutoff: 0.5, alphaMode: "MASK" }); + }, 300_000); + + it("keeps a DAG that stalled only after it had already shed its triangles", async () => { + // The decline is measured, not `stalled => gone`: twelve closed shells cannot become one + // cluster — meshoptimizer will not take a shell below about 64 triangles — but the DAG has + // halved several times before it floors out, and that payload is worth keeping. + const result = await modelPass({ virtual: { minSourceTriangles: 1024 } }).apply( + await shellsGlb(12), + "cavern.glb", + ); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + + expect( + (result.entry as { virtual: { declined: number; primitives: number; stopReason: string } }) + .virtual, + ).toMatchObject({ declined: 0, primitives: 1, stopReason: "stalled" }); + }, 600_000); + it("AC8 — a reader that has never heard of the extension gets the source mesh", async () => { const { buffer } = await compile(true); const stock = await readBack(buffer, false); @@ -264,6 +479,7 @@ describe("the virtual-geometry pass", () => { virtual: { bakeSeconds: 12.34, clusters: 512, + declined: 0, levels: 6, payloadBytes: 4096, primitives: 1, @@ -287,6 +503,7 @@ describe("the virtual-geometry pass", () => { virtual: { bakeSeconds: 1, clusters: 8, + declined: 0, levels: 2, payloadBytes: 64, primitives: 1, @@ -298,4 +515,28 @@ describe("the virtual-geometry pass", () => { expect(lines.join("\n")).toContain("a DAG hit the level cap and is unfinished"); }); + + it("AC7 — the report names the primitives it handed to the LOD ladder", () => { + const lines = formatModelSizes([ + { + after: 900, + before: 1000, + logicalPath: "canopy.glb", + virtual: { + bakeSeconds: 0.1, + clusters: 0, + declined: 2, + levels: 0, + payloadBytes: 0, + primitives: 0, + skipped: 1, + stopReason: "root", + }, + }, + ]); + + // "no primitive was dense enough" would be the wrong reason for a body whose cards were dense + // enough and whose DAG reduced nothing. + expect(lines.join("\n")).toContain("1 skipped, 2 declined to the LOD ladder"); + }); }); diff --git a/packages/assets/src/lod/eligibility.ts b/packages/assets/src/lod/eligibility.ts index f3790f9cf3..3cbf54077d 100644 --- a/packages/assets/src/lod/eligibility.ts +++ b/packages/assets/src/lod/eligibility.ts @@ -4,7 +4,13 @@ // structurally intact is declined with a named reason rather than forced to a triangle target. An // inability to reduce safely is a normal skip, not a failure. -import type { Material, Primitive } from "@gltf-transform/core"; +import { + type Document, + type Material, + type Mesh, + Node, + type Primitive, +} from "@gltf-transform/core"; /** The stable reason codes reported for a primitive that gets no automatic discrete chain. */ export type DiscreteLodSkipReason = @@ -55,6 +61,37 @@ function sharedByLevels(semantic: string): boolean { const DEFORMING_ATTRIBUTES: ReadonlySet = new Set(["JOINTS_0", "WEIGHTS_0"]); +/** + * Whether a primitive is deforming: skinned, morph-targeted, or carrying per-vertex joint data. + * + * The one definition of the word, shared by every rule that declines to rewrite geometry a rig + * drives — no discrete chain, no joined far rung, and no vertex reordering in the cook. A vertex + * index into deforming geometry is an address something outside the file may already hold, and a + * rig is where index-addressed data is most likely to come from: a MetaHuman sidecar records the + * head vertex each brow strand root rides and reads its position back every frame. + */ +export function deforming(primitive: Primitive, skinned: boolean): boolean { + if (skinned || primitive.listTargets().length > 0) return true; + return primitive.listSemantics().some((semantic) => DEFORMING_ATTRIBUTES.has(semantic)); +} + +/** The mesh-level half of {@link deforming}: the node compensation quantization applies is per mesh, + * so one deforming primitive is enough to hold every position the mesh carries. */ +export function deformingMesh(mesh: Mesh, skinned: ReadonlySet): boolean { + const isSkinned = skinned.has(mesh); + return isSkinned || mesh.listPrimitives().some((primitive) => deforming(primitive, isSkinned)); +} + +/** The meshes a skin is bound to, shared by every rule that asks whether a rig drives a mesh. */ +export function skinnedMeshes(document: Document): Set { + const skinned = new Set(); + for (const node of document.getRoot().listNodes()) { + const mesh = node.getMesh(); + if (mesh !== null && node.getSkin() !== null) skinned.add(mesh); + } + return skinned; +} + const TRIANGLES_MODE = 4; const TRIANGLE = 3; diff --git a/packages/assets/src/passes/decode-image.ts b/packages/assets/src/passes/decode-image.ts index b9b6ce00b4..c14dc1bc09 100644 --- a/packages/assets/src/passes/decode-image.ts +++ b/packages/assets/src/passes/decode-image.ts @@ -3,9 +3,10 @@ import { parsePng } from "../png.js"; /** * Decodes an encoded image's bytes to tight 8-bit RGBA for the Basis encoder, which takes - * decoded pixels through its Node-side `imageDecoder` hook. Pure-JS decoders on purpose: - * this package is Node-only and must never grow a native dependency. Containers neither - * decoder reads throw — a source this step cannot encode fails the build naming the file, + * decoded pixels through its Node-side `imageDecoder` hook. PNG and JPEG are pure-JS decoders + * already in the dependency tree; WebP is not, so it goes through `sharp` — declared but + * previously unused, native, and confined to this build-time package (PRD-485). Containers none + * of the three reads throw: a source this step cannot encode fails the build naming the file, * never silently ships uncompressed. */ export async function decodeImageBytes( @@ -23,8 +24,28 @@ export async function decodeImageBytes( assertRgba(image.data.length, image.width, image.height, logicalPath); return { data: new Uint8Array(image.data), height: image.height, width: image.width }; } + // RIFF container, `WEBP` form type — what an EXT_texture_webp image carries out of Fab. + if ( + bytes.length >= 12 && + bytes.toString("ascii", 0, 4) === "RIFF" && + bytes.toString("ascii", 8, 12) === "WEBP" + ) { + const sharp = (await import("sharp")).default; + try { + const { data, info } = await sharp(bytes) + .ensureAlpha() + .raw() + .toBuffer({ resolveWithObject: true }); + assertRgba(data.length, info.width, info.height, logicalPath); + return { data, height: info.height, width: info.width }; + } catch (error) { + throw new Error( + `TN_ASSETS_TEXTURE_UNREADABLE: '${logicalPath}' is a WebP the decoder could not read: ${error instanceof Error ? error.message : String(error)}`, + ); + } + } throw new Error( - `TN_ASSETS_TEXTURE_CONTAINER: '${logicalPath}' is not a PNG or JPEG the KTX2 encoder can read; convert it to .png or .jpg.`, + `TN_ASSETS_TEXTURE_CONTAINER: '${logicalPath}' is not a PNG, JPEG or WebP the KTX2 encoder can read; convert it to .png or .jpg.`, ); } diff --git a/packages/assets/src/passes/model-textures.ts b/packages/assets/src/passes/model-textures.ts index 07fb3d808b..c52e7b249c 100644 --- a/packages/assets/src/passes/model-textures.ts +++ b/packages/assets/src/passes/model-textures.ts @@ -1,5 +1,5 @@ import type { Document, Texture } from "@gltf-transform/core"; -import { KHRTextureBasisu } from "@gltf-transform/extensions"; +import { EXTTextureWebP, KHRTextureBasisu } from "@gltf-transform/extensions"; import { getTextureColorSpace, listTextureInfo, listTextureSlots } from "@gltf-transform/functions"; import { read as readKTX2 } from "ktx-parse"; import { PNG } from "pngjs"; @@ -104,6 +104,14 @@ const GPU_BYTES_PER_PIXEL: Readonly> = { /** Slots whose content is a tangent-space normal: data, and ETC1S destroys it. */ const NORMAL_SLOTS: ReadonlySet = new Set(["clearcoatNormalTexture", "normalTexture"]); +/** + * Containers every target of this build decodes on its own, so keeping the authored bytes costs a + * reader nothing. `image/webp` is deliberately absent: it is only reachable through + * `EXT_texture_webp`, which a target without a WebP decoder cannot satisfy — retaining one would + * hand back a model the very requirement this stage removes. + */ +const UNIVERSAL_IMAGE_CONTAINERS: ReadonlySet = new Set(["image/jpeg", "image/png"]); + /** The five slots every glTF material declares directly; extension slots are covered by * the per-texture `slots` snapshot, which walks whatever parents actually reference it. */ const CORE_SLOTS = [ @@ -482,23 +490,27 @@ export async function compressEmbeddedTextures( } if (decoderFree) { // No Basis transcoder on this target, so the cap is honoured by resampling and re-emitting - // a PNG. An image already within the cap keeps its authored bytes; the file is never - // rewritten and KHR_texture_basisu is never declared. + // a PNG. An image already within the cap keeps its authored bytes — unless its container is + // one only an extension-reading loader understands, where keeping them would ship the very + // requirement this path exists to remove. KHR_texture_basisu is never declared. const target = cappedSize(decoded.width, decoded.height, maxSize); - if (target.width === decoded.width && target.height === decoded.height) { + const downsampled = target.width !== decoded.width || target.height !== decoded.height; + if (!downsampled && UNIVERSAL_IMAGE_CONTAINERS.has(texture.getMimeType())) { bytesAfter += image.byteLength; gpuBytesAfter += gpuBytes(decoded.width, decoded.height, "none"); formats[key] = "none"; continue; } - const data = resampleRgba( - decoded.data, - decoded.width, - decoded.height, - target.width, - target.height, - getTextureColorSpace(texture) === "srgb", - ); + const data = downsampled + ? resampleRgba( + decoded.data, + decoded.width, + decoded.height, + target.width, + target.height, + getTextureColorSpace(texture) === "srgb", + ) + : decoded.data; const png = new PNG({ height: target.height, width: target.width }); png.data = Buffer.from(data); const encoded = PNG.sync.write(png); @@ -506,7 +518,7 @@ export async function compressEmbeddedTextures( bytesAfter += encoded.byteLength; gpuBytesAfter += gpuBytes(target.width, target.height, "none"); formats[key] = "none"; - resized += 1; + if (downsampled) resized += 1; continue; } @@ -552,12 +564,16 @@ export async function compressEmbeddedTextures( : { isUASTC: false, qualityLevel: quality }), }); // A tiny source can cost less than the KTX2 container alone. Keep its exact bytes when - // encoding cannot save download bytes; a named codec override still requests GPU compression. + // encoding cannot save download bytes *and* the authored container needs no extension to be + // read; a WebP costs a decoder-free target the whole model, which is never the smaller cost. + // A named codec override still requests GPU compression either way. The size increase is + // reported by the existing `bytes before -> after` row of the build report. if ( encoded.byteLength >= image.byteLength && target.width === decoded.width && target.height === decoded.height && - !explicit + !explicit && + UNIVERSAL_IMAGE_CONTAINERS.has(texture.getMimeType()) ) { bytesAfter += image.byteLength; gpuBytesAfter += gpuBytes(decoded.width, decoded.height, "none"); @@ -583,6 +599,13 @@ export async function compressEmbeddedTextures( // and a glTF that pretends otherwise fails at the first frame instead of at load. document.createExtension(KHRTextureBasisu).setRequired(true); } + // A glTF writer emits every extension still attached, so a model authored with WebP images + // would keep *requiring* EXT_texture_webp after the cook replaced those images. Dropped only + // once no image is left in the container: an over-cap image the project chose to keep as + // authored is still a WebP a reader has to understand. + if (!textures.some((texture) => texture.getMimeType() === "image/webp")) { + document.disposeExtension(EXTTextureWebP.EXTENSION_NAME); + } return { bytesAfter, bytesBefore, diff --git a/packages/assets/src/passes/model.ts b/packages/assets/src/passes/model.ts index 12e3e5003e..15bae8175b 100644 --- a/packages/assets/src/passes/model.ts +++ b/packages/assets/src/passes/model.ts @@ -4,6 +4,7 @@ import { Accessor, type Document, type GLTF, + type Mesh as GltfMesh, type Node as GltfNode, type Primitive as GltfPrimitive, NodeIO, @@ -18,7 +19,6 @@ import { listTextureSlots, prune, quantize, - reorder, simplify, } from "@gltf-transform/functions"; import { MeshoptEncoder, MeshoptSimplifier } from "meshoptimizer"; @@ -31,6 +31,7 @@ import { import { type IMaterialMergeSummary, mergeIdenticalMaterials } from "../foliage.js"; import { createGltfReader, readGltfDocument } from "../gltf-io.js"; import { KTX2_ENCODER_VERSION } from "../ktx2-encoder.js"; +import { deformingMesh, skinnedMeshes } from "../lod/eligibility.js"; import { TNDiscreteLod } from "../lod/extension.js"; import { type IModelLodOptions, @@ -71,6 +72,8 @@ import { textureBindings, textureKeys, } from "./model-textures.js"; +import { quantizeStaticGeometry } from "./quantize-static.js"; +import { reorderStaticPrimitives } from "./reorder-static.js"; import { type ISharedImage, type ISharedImageStore, @@ -243,6 +246,19 @@ const MATERIAL_MERGE_VERSION = 1; * which the compile digest already hashes, so it invalidates model outputs and nothing else. */ const MODEL_DEDUPE_VERSION = 1; +/** + * Bumped when `reorder-static.ts` changes what it emits: every model cooked before this fix had its + * deforming geometry renumbered, and a warm cache entry keyed on the pass switches alone would keep + * serving those files. 1 = deforming primitives keep their authored vertex order and count. + */ +const MODEL_REORDER_VERSION = 1; +/** + * Bumped when `quantize-static.ts` changes what it emits: every model cooked before this fix had its + * deforming positions snapped onto a quantization grid, and a warm cache entry keyed on the pass + * switches alone would keep serving those files. 1 = deforming POSITION and morph-target deltas + * stay float32 in the source metre space. + */ +const MODEL_QUANTIZE_VERSION = 1; /** Relative bounding-box tolerance of the self-verify check (PRD: 0.1%). */ const BBOX_TOLERANCE = 0.001; @@ -704,11 +720,14 @@ export function assertSimplifiedWithinBounds( * Snaps every scene-reachable POSITION accessor onto a uniform `2^bits` grid spanning its * own bounds — the destructive low-precision quantization the underlying library refuses * to express. Used only for configured depths below its 8-bit floor; the self-verify is - * expected to reject the result. + * expected to reject the result. Deforming meshes are skipped for the same reason the + * quantization stage skips them: a grid that coarse moves vertices a sidecar already + * addresses, and holding them is the rule rather than the bit depth. */ -function snapPositions(root: RootOf, bits: number): void { +function snapPositions(root: RootOf, bits: number, skinned: ReadonlySet): void { const levels = 2 ** bits - 1; for (const mesh of root.listMeshes()) { + if (deformingMesh(mesh, skinned)) continue; for (const primitive of mesh.listPrimitives()) { const position = primitive.getAttribute("POSITION"); if (position === null || position.getNormalized()) continue; @@ -803,6 +822,12 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { // content-identical sources, so a bake published before this key existed named one output // per copy and must not be re-served as if it had deduped. dedupe: MODEL_DEDUPE_VERSION, + // Same reason: which primitives `reorder` declines to renumber is part of the output, and a + // warm entry from before it declined would hand a game geometry its sidecars cannot address. + reorderVersion: MODEL_REORDER_VERSION, + // Same reason again: which positions keep their source metres is part of the output, and a + // warm entry from before they did would hand a game geometry its sidecars cannot address. + quantizeVersion: MODEL_QUANTIZE_VERSION, // `"none"` and "absent" are different cache keys on purpose: absent bakes with defaults. virtual: options.virtual === "none" @@ -819,7 +844,10 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { decoderFree: options.textures?.decoderFree ?? false, encoder: KTX2_ENCODER_VERSION, maxSize: options.textures?.maxSize ?? null, - keepSmallerSource: true, + // Retention is now conditioned on the container, so it is spelled out rather than + // named by a boolean: a warm cache keyed on the old rule would keep shipping the + // WebP-requiring models this replaced. + keepSmallerSource: "universal-containers", overrides: options.textures?.overrides ?? [], quality: options.textures?.quality ?? null, }, @@ -919,7 +947,10 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { })(document); } if (enabled.reorder || enabled.meshopt) await MeshoptEncoder.ready; - if (enabled.reorder) await reorder({ encoder: MeshoptEncoder })(document); + // The last stage that moves a vertex, and the only one that does so selectively: deforming + // geometry keeps its authored vertex order because a vertex index into it is an address + // something outside this file may already hold. `reorder-static.ts`. + if (enabled.reorder) reorderStaticPrimitives(document, MeshoptEncoder); // After `reorder`, which is the last stage that moves a vertex, and before `quantize`, which // changes what a position is but never which vertex it is. const virtual = @@ -942,18 +973,23 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { // Depths below the library's 8-bit floor are honoured by pre-rounding the floats // onto the coarser grid first; the self-verify then fails the build on the drift. const requestedBits = options.quantize?.positionBits ?? DEFAULT_POSITION_BITS; - if (requestedBits < 8) snapPositions(document.getRoot(), requestedBits); + if (requestedBits < 8) + snapPositions(document.getRoot(), requestedBits, skinnedMeshes(document)); // Joint indices and weights are excluded outright: quantizing them below the // exporter's declaration is where skinned meshes visibly break, and the library's // own default narrows weights to 8 bits unless told otherwise. - await quantize({ - quantizeColor: 8, - quantizeNormal: options.quantize?.normalBits ?? DEFAULT_NORMAL_BITS, - quantizePosition: Math.max(requestedBits, 8), - quantizeTexcoord: options.quantize?.uvBits ?? DEFAULT_UV_BITS, - pattern: /^(?!JOINTS|WEIGHTS)/u, - normalizeWeights: false, - })(document); + // Deforming geometry keeps float32 POSITION in the source metre space on top of + // that, so nothing a sidecar addresses moves: `quantize-static.ts`. + await quantizeStaticGeometry(document, () => + quantize({ + quantizeColor: 8, + quantizeNormal: options.quantize?.normalBits ?? DEFAULT_NORMAL_BITS, + quantizePosition: Math.max(requestedBits, 8), + quantizeTexcoord: options.quantize?.uvBits ?? DEFAULT_UV_BITS, + pattern: /^(?!JOINTS|WEIGHTS)/u, + normalizeWeights: false, + })(document), + ); } // Taken after every geometry stage and before compression, so what it proves is that // *this* stage plus the writer preserved each binding — prune's documented drops of @@ -1079,7 +1115,7 @@ function sharedSettings( : { decoderFree: textureOptions.decoderFree ?? false, encoder: KTX2_ENCODER_VERSION, - keepSmallerSource: true, + keepSmallerSource: "universal-containers", maxSize: textureOptions.maxSize ?? null, overrides: textureOptions.overrides ?? [], quality: textureOptions.quality ?? null, diff --git a/packages/assets/src/passes/quantize-static.ts b/packages/assets/src/passes/quantize-static.ts new file mode 100644 index 0000000000..487454044d --- /dev/null +++ b/packages/assets/src/passes/quantize-static.ts @@ -0,0 +1,160 @@ +// The model pass's `quantize` stage, over the primitives whose positions nothing else may move. +// +// `quantize` from `@gltf-transform/functions` walks the whole document and rewrites POSITION into +// 16-bit integers snapped to a grid whose step is the mesh's own bounding volume — the largest size +// win in the cook — and then compensates by folding that shift into the node holding the mesh, the +// skin's inverse bind matrices, GPU instancing batches and any volumetric material. +// +// That is exact for static geometry. For geometry a rig drives it breaks the invariant this repo +// holds: one metre is one metre. The metahuman-lab's `brows.skin.json` names the head vertex each +// brow strand root rides and reads its position back every frame, prepared against the source file +// to within 1e-5 m; a head snapped onto a 16-bit grid over its own volume is micrometres away from +// every vertex the sidecar recorded, and the shift that would undo it lives in a node transform the +// sidecar never sees. +// +// So a deforming mesh is held in source space across the stage: POSITION and the morph-target +// deltas riding it keep the exact values their author wrote, and the mesh spends the stage detached +// from its nodes, so there is no compensation to undo — the skin, the node matrices, the instancing +// batch and a volume material's thickness are simply never touched. Everything else still quantizes, +// and `EXT_meshopt_compression` still encodes whatever the accessors hold. +// +// ponytail: the library's `quantize` takes one document-wide pattern with no per-mesh switch, and +// its node compensation is inseparable from the positions it shifts, so holding is its own ~60 +// lines. Replace with the library's own `quantize` the day it takes a deforming predicate. + +import type { + Accessor, + Document, + ExtensionProperty, + Material, + Mesh, + Primitive, + PrimitiveTarget, + TypedArray, +} from "@gltf-transform/core"; +import { Node } from "@gltf-transform/core"; +import { deformingMesh, skinnedMeshes } from "../lod/eligibility.js"; + +/** What a primitive and a morph target both expose for one attribute, so one call serves both. */ +type IAttributeOwner = { + getAttribute(semantic: string): Accessor | null; + setAttribute(semantic: string, accessor: Accessor | null): unknown; +}; + +/** One attribute's authored values, copied out before the stage overwrites them. */ +interface IHeldAttribute { + readonly array: TypedArray; + readonly normalized: boolean; +} + +/** One primitive's authored positions, kept while the stage tries to rewrite them. */ +interface IHeldPrimitive { + readonly primitive: Primitive; + /** The index buffer as authored: a remap here would renumber what a sidecar addresses. */ + readonly indices: Accessor | null; + readonly position: IHeldAttribute | null; + readonly targets: readonly { + readonly target: PrimitiveTarget; + readonly position: IHeldAttribute | null; + }[]; +} + +/** One deforming mesh: how it was held, and how to put it back. */ +interface IHeldMesh { + readonly mesh: Mesh; + readonly nodes: readonly Node[]; + readonly primitives: readonly IHeldPrimitive[]; + readonly volumes: readonly { readonly material: Material; readonly volume: ExtensionProperty }[]; +} + +/** + * Runs `quantize` with every deforming mesh held in the metre space its author wrote it in, and + * puts the held geometry back whether the stage returns or throws. + */ +export async function quantizeStaticGeometry( + document: Document, + quantize: () => unknown, +): Promise { + const held = holdDeformingMeshes(document); + try { + await quantize(); + } finally { + releaseDeformingMeshes(held); + } +} + +function holdDeformingMeshes(document: Document): IHeldMesh[] { + const skinned = skinnedMeshes(document); + const held: IHeldMesh[] = []; + for (const mesh of document.getRoot().listMeshes()) { + // The mesh is the unit of source space, because the compensation is per mesh: one deforming + // primitive in it holds the whole thing, or the shift would land on half of it. + if (!deformingMesh(mesh, skinned)) continue; + const nodes: Node[] = []; + for (const parent of mesh.listParents()) { + if (!(parent instanceof Node)) continue; + nodes.push(parent); + parent.setMesh(null); + } + const primitives = mesh.listPrimitives().map((primitive) => ({ + primitive, + indices: primitive.getIndices(), + position: holdAttribute(primitive.getAttribute("POSITION")), + targets: primitive.listTargets().map((target) => ({ + target, + position: holdAttribute(target.getAttribute("POSITION")), + })), + })); + const volumes: { material: Material; volume: ExtensionProperty }[] = []; + for (const primitive of mesh.listPrimitives()) { + const material = primitive.getMaterial(); + const volume = material?.getExtension("KHR_materials_volume") ?? null; + if (material === null || volume === null) continue; + // Thickness is given in local units, so the stage would scale it by the shift it is not + // applying to this mesh's positions. + material.setExtension("KHR_materials_volume", null); + volumes.push({ material, volume }); + } + held.push({ mesh, nodes, primitives, volumes }); + } + return held; +} + +function releaseDeformingMeshes(held: readonly IHeldMesh[]): void { + for (const { mesh, nodes, primitives, volumes } of held) { + for (const node of nodes) node.setMesh(mesh); + for (const { material, volume } of volumes) + material.setExtension("KHR_materials_volume", volume); + for (const { primitive, indices, position, targets } of primitives) { + // The stage may rewrite what a sidecar already addresses, but never renumber it: an index + // buffer it replaced means the held positions no longer line up, so fail instead of shipping + // a mesh whose every vertex has quietly moved. + if (primitive.getIndices() !== indices) + throw new Error( + "TN_ASSETS_MODEL_DEFORMING_RENUMBERED: quantization rewrote the index buffer of a deforming primitive; its positions were held in source space and can no longer be addressed by index.", + ); + releaseAttribute(primitive, position); + for (const { target, position: delta } of targets) releaseAttribute(target, delta); + } + } +} + +function holdAttribute(accessor: Accessor | null): IHeldAttribute | null { + const array = accessor?.getArray() ?? null; + if (accessor === null || array === null) return null; + // A copy, because the stage replaces this accessor with its own and prunes the original: a + // disposed property cannot be re-attached, and the values have to outlive the stage. + const copy = array.slice(); + return { array: copy, normalized: accessor.getNormalized() }; +} + +/** + * Writes the authored values back over whatever the stage left in the attribute. The accessor the + * stage installed is reused rather than replaced, so the mesh keeps the attribute it was drawn + * with and no orphan is left for the writer to emit. + */ +function releaseAttribute(owner: IAttributeOwner, held: IHeldAttribute | null): void { + const accessor = owner.getAttribute("POSITION"); + if (held === null || accessor === null) return; + accessor.setArray(held.array).setNormalized(held.normalized).setSparse(false); +} diff --git a/packages/assets/src/passes/reorder-static.ts b/packages/assets/src/passes/reorder-static.ts new file mode 100644 index 0000000000..d8c1ffffe3 --- /dev/null +++ b/packages/assets/src/passes/reorder-static.ts @@ -0,0 +1,126 @@ +// The model pass's `reorder` stage, over the primitives whose vertex order nothing else owns. +// +// `reorder` from `@gltf-transform/functions` walks the whole document and renumbers the vertices +// of every primitive it finds, which is the cheapest transmission-size win in the cook — and it +// breaks every file that addresses a vertex by index. A MetaHuman sidecar names the head vertex +// each brow strand root rides and reads that position back every frame; after an unconditional +// reorder it indexes a different vertex (`brow root 0 indexes head vertex 11498, which is not where +// it was prepared`). The cook cannot know who holds an index, so the general rule is that any +// primitive whose indices can be addressed from outside is undeclarable, and *deforming* — skinned, +// morph-targeted, or joint-bound — is the measurable proxy: a rig is where index-addressed data +// comes from, and it is the one case detectable from the file alone. +// +// So the vertex-cache remap is applied per index-buffer group, and a group holding any deforming +// primitive is left exactly as authored. Compression is unaffected: `EXT_meshopt_compression` encodes +// whatever order it is handed, and `quantize` renumbers nothing. +// +// ponytail: the library's `reorder` has no per-primitive switch, so the remap is its own ~30 lines. +// Replace with the library's whole-document `reorder` the day it takes a predicate. + +import { type Accessor, type Document, Primitive } from "@gltf-transform/core"; +import type { MeshoptEncoder } from "meshoptimizer"; +import { deforming, skinnedMeshes } from "../lod/eligibility.js"; + +/** One index buffer, the attributes that read through it, and the primitives that draw it. */ +interface IVertexGroup { + readonly attributes: Accessor[]; + readonly members: { readonly primitive: Primitive; readonly skinned: boolean }[]; +} + +/** meshoptimizer's own 16-bit index ceiling, as the library's `reorder` measures it. */ +const MAX_UINT16 = 65534; + +/** + * Reorder every non-deforming primitive's vertices for transmission size, in place. Returns how + * many index-buffer groups were rewritten. + * + * Grouped by shared index buffer rather than by primitive, because primitives sharing one index + * buffer share the renumbering: remapping one and not the other desynchronises them. Mirrors the + * library's own `reorder` for the groups it does touch, minus its cleanup prune — the pass's `prune` + * runs before this stage, and the writer emits reachable properties only. + */ +export function reorderStaticPrimitives( + document: Document, + encoder: typeof MeshoptEncoder, +): number { + let reordered = 0; + for (const [indices, group] of vertexGroups(document)) { + if (group.members.some((member) => deforming(member.primitive, member.skinned))) continue; + remapGroup(indices, group, encoder); + reordered += 1; + } + return reordered; +} + +/** Every index buffer in the document, with the attributes and primitives that read through it. */ +function vertexGroups(document: Document): Map { + const skinned = skinnedMeshes(document); + const groups = new Map(); + for (const mesh of document.getRoot().listMeshes()) + for (const primitive of mesh.listPrimitives()) { + const indices = primitive.getIndices(); + if (indices === null) continue; + const group = groups.get(indices) ?? { attributes: [], members: [] }; + group.members.push({ primitive, skinned: skinned.has(mesh) }); + for (const semantic of primitive.listSemantics()) { + const attribute = primitive.getAttribute(semantic); + if (attribute !== null && !group.attributes.includes(attribute)) + group.attributes.push(attribute); + } + groups.set(indices, group); + } + return groups; +} + +/** Renumber one group's vertices, and every primitive that draws them, in place. */ +function remapGroup(indices: Accessor, group: IVertexGroup, encoder: typeof MeshoptEncoder): void { + const source = indices.getArray(); + const triangles = group.members.every( + (member) => member.primitive.getMode() === Primitive.Mode.TRIANGLES, + ); + // The encoder renumbers the array it is handed in place and returns the old-to-new table, so the + // copy is the destination index buffer and `source` still reads the authored one. + const remapped = source instanceof Uint32Array ? source.slice() : new Uint32Array(source); + const [remap, unique] = encoder.reorderMesh(remapped, triangles, true); + const dstIndices = indices + .clone() + .setArray(unique <= MAX_UINT16 ? new Uint16Array(remapped) : remapped); + const dstAttributes = new Map(); + for (const attribute of group.attributes) + dstAttributes.set(attribute, compactAttribute(attribute, source, remap, unique)); + for (const member of group.members) { + for (const semantic of member.primitive.listSemantics()) { + const attribute = member.primitive.getAttribute(semantic); + if (attribute === null) continue; + member.primitive.setAttribute(semantic, dstAttributes.get(attribute) ?? attribute); + } + member.primitive.setIndices(dstIndices); + } + for (const attribute of group.attributes) + if (attribute.listParents().length === 0) attribute.dispose(); + if (indices.listParents().length === 0) indices.dispose(); +} + +/** One attribute rewritten into the renumbered vertex stream, dropping what no triangle reaches. */ +function compactAttribute( + attribute: Accessor, + source: ArrayLike, + remap: Uint32Array, + unique: number, +): Accessor { + const array = attribute.getArray(); + const width = attribute.getElementSize(); + const compact = new array.constructor(unique * width); + // One write per destination vertex: a vertex several triangles reference is copied once. + const written = new Uint8Array(unique); + for (let index = 0; index < source.length; index += 1) { + const from = source[index] ?? 0; + const to = remap[from]; + if (to === undefined || written[to] === 1) continue; + written[to] = 1; + for (let component = 0; component < width; component += 1) + compact[to * width + component] = array[from * width + (component ?? 0)]; + } + // Sparse is cleared: it indexes the source's vertex stream and this one has been renumbered. + return attribute.clone().setArray(compact).setSparse(false); +} diff --git a/packages/assets/src/passes/texture.ts b/packages/assets/src/passes/texture.ts index 14026a2362..b9fa713a96 100644 --- a/packages/assets/src/passes/texture.ts +++ b/packages/assets/src/passes/texture.ts @@ -224,7 +224,7 @@ export function textureResizePass(options: ITextureResizeOptions): IAssetPass { }; } -/** PNG signature or JPEG SOI, the two containers `decodeImageBytes` reads. */ +/** PNG signature or JPEG SOI, the two containers the standalone pass sniffs headers for. */ function isPngOrJpeg(bytes: Buffer): boolean { if (bytes.length >= 3 && bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) { return true; diff --git a/packages/assets/src/report.ts b/packages/assets/src/report.ts index b8d203d6c8..dc61d06527 100644 --- a/packages/assets/src/report.ts +++ b/packages/assets/src/report.ts @@ -64,6 +64,8 @@ export interface IEmbeddedTextureRow { export interface IVirtualRow { readonly bakeSeconds: number; readonly clusters: number; + /** Primitives handed back to the discrete ladder because their DAG reduced nothing. */ + readonly declined: number; readonly levels: number; readonly payloadBytes: number; readonly primitives: number; @@ -409,19 +411,22 @@ function simplifyLine(row: IModelSizeRow): readonly string[] { * Names what the DAG cost and where it stopped. * * A `cap` stop is called out because it means a DAG ran out of levels rather than finishing, and a - * bake that clustered nothing says so rather than leaving the reader to infer it from a zero. + * bake that clustered nothing says so rather than leaving the reader to infer it from a zero. A + * declined primitive is named too: "no primitive was dense enough" is the wrong reason for a body + * whose foliage is dense enough and whose DAG reduced nothing. */ function virtualLine(row: IModelSizeRow): readonly string[] { const virtual = row.virtual; if (virtual === undefined) return []; + const declined = virtual.declined > 0 ? `, ${virtual.declined} declined to the LOD ladder` : ""; if (virtual.primitives === 0) return [ - `virtual ${row.logicalPath}: no primitive was dense enough to cluster (${virtual.skipped} skipped)`, + `virtual ${row.logicalPath}: no primitive was dense enough to cluster (${virtual.skipped} skipped${declined})`, ]; const warning = virtual.stopReason === "cap" ? " — a DAG hit the level cap and is unfinished" : ""; return [ - `virtual ${row.logicalPath}: ${virtual.clusters} cluster(s) over ${virtual.levels} level(s) on ${virtual.primitives} primitive(s), ${virtual.skipped} skipped, ${virtual.payloadBytes} payload bytes, bake ${virtual.bakeSeconds.toFixed(1)} s, stopped at ${virtual.stopReason}${warning}`, + `virtual ${row.logicalPath}: ${virtual.clusters} cluster(s) over ${virtual.levels} level(s) on ${virtual.primitives} primitive(s), ${virtual.skipped} skipped${declined}, ${virtual.payloadBytes} payload bytes, bake ${virtual.bakeSeconds.toFixed(1)} s, stopped at ${virtual.stopReason}${warning}`, ]; } function compactLine(row: IModelSizeRow): readonly string[] { diff --git a/packages/assets/src/virtual/bake.ts b/packages/assets/src/virtual/bake.ts index 5f9c061205..210acd3f3f 100644 --- a/packages/assets/src/virtual/bake.ts +++ b/packages/assets/src/virtual/bake.ts @@ -1,7 +1,12 @@ // The pipeline's side of the DAG bake: which primitives get one, and what the report says about it. import type { Document, Primitive } from "@gltf-transform/core"; -import { type IClusterDagOptions, buildClusterDag } from "./dag.js"; +import { + type IClusterDag, + type IClusterDagOptions, + type IVirtualLevel, + buildClusterDag, +} from "./dag.js"; import { TNVirtualGeometry, attachVirtualGeometry, virtualGeometryBytes } from "./extension.js"; /** @@ -52,6 +57,11 @@ export interface IModelVirtualOptions { export interface IModelVirtualSummary { readonly bakeSeconds: number; readonly clusters: number; + /** + * Primitives the bake looked at and handed back because their DAG reduced nothing — left for the + * discrete ladder, which is named here because "no clusters" alone reads as a bug. + */ + readonly declined: number; readonly levels: number; readonly payloadBytes: number; readonly primitives: number; @@ -61,7 +71,37 @@ export interface IModelVirtualSummary { } /** Bumped when the bake's output changes for the same input. Part of the compile cache key. */ -export const VIRTUAL_BAKE_VERSION = 3; +export const VIRTUAL_BAKE_VERSION = 4; + +/** + * The share of its own triangles a DAG has to shed before it keeps the primitive. + * + * Below that the payload is three to four times the primitive's compiled bytes (see + * {@link IModelVirtualOptions.minSourceTriangles}) for a cut that draws the same triangles, and the + * discrete ladder's terminal level reaches 5% of LOD0 on the same geometry. + */ +const DECLINE_KEEP_RATIO = 0.8; + +/** + * Whether the bake hands this primitive back instead of owning it (PRD-485). + * + * A `stalled` DAG whose coarsest level still holds most of the source has no coarse representation + * to offer. meshoptimizer will not collapse a body of disconnected alpha-cutout cards with every + * border locked — each card's whole rim is a border — so the first fold simplifies nothing + * (`virtual/dag.ts:461`), every level reports the source count, and every threshold draws LOD0: the + * Hornbeam's two foliage primitives, 745,328 triangles with no coarse step at all. Owning them + * anyway costs 3–4x their bytes and leaves them `virtual-owned` (`lod/eligibility.ts:182`), so the + * bake declines them and the discrete ladder — whose terminal pass exists for exactly this shape, + * `lod/generate.ts:851` — gets the primitive instead. + * + * Measured, not assumed: a DAG that stalls *after* real folds has shed its triangles and keeps the + * primitive, and so does every stop reason but `stalled`. + */ +function declines(dag: IClusterDag, sourceTriangles: number): boolean { + if (dag.stopReason !== "stalled") return false; + const coarsest = dag.levels[dag.levels.length - 1] as IVirtualLevel; + return coarsest.triangleCount > sourceTriangles * DECLINE_KEEP_RATIO; +} /** * The density at which a cluster DAG starts paying for its own bytes. See @@ -117,49 +157,59 @@ export async function bakeVirtualGeometry( let extension: TNVirtualGeometry | null = null; let clusters = 0; + let declined = 0; let levels = 0; let payloadBytes = 0; let primitives = 0; let skipped = 0; let stopReason = "root"; - for (const mesh of document.getRoot().listMeshes()) { - for (const primitive of mesh.listPrimitives()) { - const indices = primitive.getIndices(); - const positions = positionsOf(primitive); - // A skinned body ships whole: the loader's clustered mesh is a plain `Mesh`, so a baked - // primitive would lose its skin (and the rig would not clone). - if ( - indices === null || - positions === null || - primitive.getMode() !== 4 || - primitive.getAttribute("JOINTS_0") !== null - ) { - skipped += 1; - continue; - } - if (indices.getCount() / 3 < minSourceTriangles) { - skipped += 1; - continue; - } - const dag = await buildClusterDag( - Uint32Array.from(indices.getArray() as ArrayLike), - positions, - dagOptions, - ); - extension ??= document.createExtension(TNVirtualGeometry).setRequired(false); - const virtual = attachVirtualGeometry(document, extension, primitive, dag); - clusters += dag.clusters.length; - levels = Math.max(levels, dag.levels.length); - payloadBytes += virtualGeometryBytes(virtual); - primitives += 1; - stopReason = worse(stopReason, dag.stopReason); + for (const primitive of document + .getRoot() + .listMeshes() + .flatMap((mesh) => mesh.listPrimitives())) { + const indices = primitive.getIndices(); + const positions = positionsOf(primitive); + // A skinned body ships whole: the loader's clustered mesh is a plain `Mesh`, so a baked + // primitive would lose its skin (and the rig would not clone). + if ( + indices === null || + positions === null || + primitive.getMode() !== 4 || + primitive.getAttribute("JOINTS_0") !== null + ) { + skipped += 1; + continue; + } + const sourceTriangles = indices.getCount() / 3; + if (sourceTriangles < minSourceTriangles) { + skipped += 1; + continue; + } + const dag = await buildClusterDag( + Uint32Array.from(indices.getArray() as ArrayLike), + positions, + dagOptions, + ); + // No extension is created for a declined primitive, so a document whose every primitive is + // card soup never declares `TN_virtual_geometry` at all. + if (declines(dag, sourceTriangles)) { + declined += 1; + continue; } + extension ??= document.createExtension(TNVirtualGeometry).setRequired(false); + const virtual = attachVirtualGeometry(document, extension, primitive, dag); + clusters += dag.clusters.length; + levels = Math.max(levels, dag.levels.length); + payloadBytes += virtualGeometryBytes(virtual); + primitives += 1; + stopReason = worse(stopReason, dag.stopReason); } return { bakeSeconds: (now() - started) / 1000, clusters, + declined, levels, payloadBytes, primitives, diff --git a/packages/core/__tests__/clustered-batch.spec.ts b/packages/core/__tests__/clustered-batch.spec.ts index ea35b8a52e..0901886385 100644 --- a/packages/core/__tests__/clustered-batch.spec.ts +++ b/packages/core/__tests__/clustered-batch.spec.ts @@ -502,4 +502,22 @@ describe("ClusteredBatch", () => { }).build({ parent: new Object3D() }), ).toThrow(/at least one placed copy/u); }, 180_000); + + it("clones into a root that is still cut", () => { + // `new this.constructor()` left `batch` undefined, so a cloned root quietly stopped being cut + // and drew whatever its bands happened to hold. Detection is structural, so the engine could + // not even tell the copy apart from an ordinary object. + const parent = new Object3D(); + const batch = spread(); + const root = batch.build({ name: "boulders", parent }); + camera.position.set(0, 0, 0); + const triangles = batch.update(camera, 1080); + + const copy = root.clone() as unknown as { batch: ClusteredBatch }; + expect(copy.batch).toBe(batch); + // Still found by the engine's walk, and still drawing: a copy whose `batch` was undefined + // returned nothing and was never cut again. + expect(updateClusteredMeshes(copy as unknown as Object3D, camera, 1080)).toBeGreaterThan(0); + expect(triangles).toBeGreaterThan(0); + }, 180_000); }); diff --git a/packages/core/__tests__/clustered-mesh.spec.ts b/packages/core/__tests__/clustered-mesh.spec.ts index f5a9b23b12..ec2769e991 100644 --- a/packages/core/__tests__/clustered-mesh.spec.ts +++ b/packages/core/__tests__/clustered-mesh.spec.ts @@ -1,6 +1,7 @@ import { BufferAttribute, BufferGeometry, + Group, MeshBasicMaterial, PerspectiveCamera, Vector3, @@ -442,3 +443,81 @@ describe("ClusteredMesh", () => { ).toThrow(/needs a perspective camera/u); }); }); + +describe("ClusteredMesh cloning", () => { + const camera = new PerspectiveCamera(60, 16 / 9, 0.1, 1000); + + /** This mesh's triangles, with the camera `z` units out from where it stands. */ + function cutAt(mesh: ClusteredMesh, z: number): number { + camera.position.set(0, 0, z); + return mesh.update(camera, 1080); + } + + // three clones through `new this.constructor().copy(source)`, so any game that stands a second + // copy of a cooked model where it wants a statue used to die in `assertTable` before it drew + // anything. Lumen-hall's authored statues did, from the first model the default cook produced. + it("clones into a mesh that draws the same thing", () => { + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const triangles = cutAt(mesh, 6); + const clone = mesh.clone(); + + expect(clone.table).toBe(mesh.table); + // A copy starts at the authored detail, exactly as a fresh mesh does, and only gives it up + // once something cuts it. + expect(clone.drawnTriangles).toBe(body.indices.length / 3); + expect(cutAt(clone, 6)).toBe(triangles); + expect(clone.geometry.drawRange).toEqual(mesh.geometry.drawRange); + expect(clone.geometry.getIndex()?.array).not.toBe(mesh.geometry.getIndex()?.array); + }); + + it("clones inside a group, which is how a game places a second copy", () => { + const figure = new Group(); + figure.add(new ClusteredMesh(geometry(), new MeshBasicMaterial(), table)); + const clone = figure.clone(true); + const copied = clone.children[0] as ClusteredMesh; + + expect(copied).toBeInstanceOf(ClusteredMesh); + expect(copied.table).toBe((figure.children[0] as ClusteredMesh).table); + }); + + it("keeps the vertices shared and the cut its own, so the two are placed independently", () => { + // Sharing the geometry wholesale would be cheaper still and wrong: the cut lives in the + // geometry's index buffer, so two meshes on one geometry would overwrite each other's index + // every frame and both would draw whichever one spoke last. Sharing the vertex buffers by + // reference keeps one copy of them on the GPU — three keys its attributes on the attribute — + // and still lets each mesh hold a cut of its own. + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const beside = mesh.clone(); + + expect(beside.geometry).not.toBe(mesh.geometry); + expect(beside.geometry.attributes.position).toBe(mesh.geometry.attributes.position); + + const near = cutAt(mesh, 6); + expect(cutAt(beside, 6)).toBe(near); + + const across = mesh.clone(); + across.position.set(0, 0, 400); + expect(cutAt(across, 6)).toBeLessThan(near); + // And cutting the far copy left the near one drawing what it was drawing. + expect(cutAt(mesh, 6)).toBe(near); + }); + + it("disposes without taking the source's shared data with it", () => { + // The rule, since sharing is the point: a clone shares the baked table and the vertex + // attributes, and the geometry that owns them disposes them for everyone — exactly as three's + // own `Mesh.copy` shares geometry and material. What a clone owns outright is its cut. + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const clone = mesh.clone(); + let sourceDisposed = false; + mesh.geometry.addEventListener("dispose", () => { + sourceDisposed = true; + }); + + clone.geometry.dispose(); + + expect(sourceDisposed).toBe(false); + expect(clone.geometry.attributes.position).toBe(mesh.geometry.attributes.position); + expect(mesh.table).toBe(table); + expect(mesh.drawnTriangles).toBe(body.indices.length / 3); + }); +}); diff --git a/packages/core/src/clustered-batch.ts b/packages/core/src/clustered-batch.ts index 7bbe0644ca..2b4035b5fd 100644 --- a/packages/core/src/clustered-batch.ts +++ b/packages/core/src/clustered-batch.ts @@ -126,6 +126,19 @@ export class ClusteredBatchRoot extends Object3D { constructor(readonly batch: ClusteredBatch) { super(); } + + /** + * A copy that is still being cut. + * + * three builds one through `new this.constructor()`, which left the copy's `batch` undefined + * and quietly stopped cutting it — a frozen cut that still drew. Sharing the batch is what + * keeps a copy alive: the batch owns the bands and their index buffers, so the copy draws the + * same cut as the root it was copied from, which is the same thing `Mesh.copy` does with + * geometry and `VirtualGeometryPlugin` does with a baked table. + */ + override clone(recursive = true): this { + return new ClusteredBatchRoot(this.batch).copy(this, recursive) as this; + } } function placementMatrix(placement: IClusteredPlacement): Matrix4 { diff --git a/packages/core/src/clustered-mesh.ts b/packages/core/src/clustered-mesh.ts index d9db2111f0..3dc79391b2 100644 --- a/packages/core/src/clustered-mesh.ts +++ b/packages/core/src/clustered-mesh.ts @@ -1,11 +1,4 @@ -import { - BufferAttribute, - type BufferGeometry, - type Camera, - type Material, - Mesh, - Vector3, -} from "three"; +import { BufferAttribute, BufferGeometry, type Camera, type Material, Mesh, Vector3 } from "three"; /** * A mesh that draws only the clusters this camera can resolve. @@ -163,15 +156,15 @@ function finestIndexCount(table: IClusterTable): number { } export class ClusteredMesh extends Mesh { - /** The baked payload. Read-only at run time; the bake is the only thing that writes it. */ - readonly table: IClusterTable; + /** The baked payload. The bake is the only thing that writes it; a copy takes the source's. */ + table!: IClusterTable; /** Screen-space error budget in pixels. Writable — it is the game's call. */ errorPixels: number; /** Camera movement, in the mesh's own units, below which the previous cut is kept. */ recutDistance: number; - readonly #cut: Uint32Array; - readonly #selection: Uint32Array; + #cut!: Uint32Array; + #selection!: Uint32Array; readonly #cameraLocal = new Vector3(); readonly #lastCut = new Vector3(); #drawnClusters = 0; @@ -188,11 +181,21 @@ export class ClusteredMesh extends Mesh { ) { super(geometry, surface); assertTable(table); - this.table = table; this.errorPixels = options.errorPixels ?? DEFAULT_ERROR_PIXELS; + this.recutDistance = options.recutDistance ?? radiusOf(table) * RECUT_FRACTION; + this.#load(table, geometry); + // Full detail until something cuts it. Virtual geometry ships on by default, so the mesh a + // game never touches has to draw exactly what an ordinary `Mesh` would have drawn — a class + // that starts invisible and waits to be updated would turn the default into a blank screen. + // `update` only ever takes detail away from here. + this.#compact(this.#finestCut()); + } + + /** Points this mesh at `table`, and installs its cut on the geometry it is going to draw. */ + #load(table: IClusterTable, geometry: BufferGeometry): void { + this.table = table; this.#cut = new Uint32Array(finestIndexCount(table)); this.#selection = new Uint32Array(table.ranges.length / 2); - this.recutDistance = options.recutDistance ?? radiusOf(table) * RECUT_FRACTION; // The geometry draws out of the cut buffer from here on. Its vertex attributes are untouched: // every level of the DAG indexes the vertices the mesh already shipped. // @@ -200,11 +203,44 @@ export class ClusteredMesh extends Mesh { // are given, so the cut this class writes every frame would never reach the GPU and the mesh // would draw a buffer of zeros — degenerate triangles, an empty screen, and no error anywhere. geometry.setIndex(new BufferAttribute(this.#cut, 1)); - // Full detail until something cuts it. Virtual geometry ships on by default, so the mesh a - // game never touches has to draw exactly what an ordinary `Mesh` would have drawn — a class - // that starts invisible and waits to be updated would turn the default into a blank screen. - // `update` only ever takes detail away from here. + } + + /** + * A second mesh over the same body, cut for its own place in the scene. + * + * three's `Object3D.clone` builds through `new this.constructor()`, and this class's constructor + * wants a table — so every game that stood a second copy of a cooked model where it wanted a + * statue died in `assertTable` before drawing anything. The arguments are the source's own, + * which is what a copy is. + */ + override clone(recursive = true): this { + return new ClusteredMesh(withOwnCut(this.geometry), this.material, this.table, { + errorPixels: this.errorPixels, + recutDistance: this.recutDistance, + }).copy(this, recursive) as this; + } + + /** + * Becomes a copy of `source`, sharing the bake and the vertices and keeping a cut of its own. + * + * `Mesh.copy` shares the geometry outright, which this class cannot let it do: the cut lives in + * the geometry's index buffer, so two meshes on one geometry would overwrite each other's index + * every frame and both would draw whichever spoke last. The copy takes the vertex attributes by + * reference instead — one copy of them on the GPU, because three keys its attributes on the + * attribute — and the shared table, which is frozen at bake time and cannot go stale. + */ + override copy(source: this, recursive = true): this { + super.copy(source, recursive); + if (!(source instanceof ClusteredMesh)) return this; + const geometry = withOwnCut(source.geometry); + this.#load(source.table, geometry); + this.geometry = geometry; + this.errorPixels = source.errorPixels; + this.recutDistance = source.recutDistance; + // Nothing this copy has drawn yet, so the next update has to cut it for where it stands. + this.#hasCut = false; this.#compact(this.#finestCut()); + return this; } /** Every cluster the bake produced from the source triangles — the mesh as authored. */ @@ -286,6 +322,23 @@ export class ClusteredMesh extends Mesh { } } +/** + * The same vertices under a second geometry, so a second mesh can hold a cut of its own. + * + * `BufferGeometry.clone()` would do this and would also copy every position, normal and uv — the + * memory a cooked model exists not to spend twice. Sharing the `BufferAttribute` objects shares + * their GPU buffers too, because three keys its attribute cache on the attribute, not the geometry. + */ +function withOwnCut(geometry: BufferGeometry): BufferGeometry { + const shell = new BufferGeometry(); + for (const [name, attribute] of Object.entries(geometry.attributes)) + shell.setAttribute(name, attribute); + shell.boundingSphere = geometry.boundingSphere; + shell.boundingBox = geometry.boundingBox; + shell.groups = [...geometry.groups]; + return shell; +} + function radiusOf(table: IClusterTable): number { let radius = 0; for (let cluster = 0; cluster * 4 < table.bounds.length; cluster += 1) diff --git a/packages/create-threenative/agent-docs/references/finding-assets.md b/packages/create-threenative/agent-docs/references/finding-assets.md index aefade4aee..cb4bc6386f 100644 --- a/packages/create-threenative/agent-docs/references/finding-assets.md +++ b/packages/create-threenative/agent-docs/references/finding-assets.md @@ -135,6 +135,23 @@ Then load the returned path the ordinary way: `ctx.assets.model("fab/soul-cave/.../SM_S_Soul_Statue.glb")`. The normal asset compiler picks the GLBs up from `assets/` with no extra configuration. +**Cook the pack with the default settings. `models: "none"` and `textures: "none"` in +`threenative.config.ts` are not a fix, and a reason to reach for them is a bug report, not a +workaround.** A converted Fab pack is ordinary input to that cook, and the cook is where an +expensive import stops being expensive: embedded PNG and WebP maps are transcoded to KTX2 and +capped, mesh attributes are quantized and Meshopt-compressed, and a `MASK` (alpha-cutout) +primitive is handed a discrete LOD chain, so the LOD0 triangle count in the GLB is not the +triangle count the scene pays for. One measured tree: `SM_EuropeanHornbeam_Forest_01.glb` cooks +126,934,280 → 26,428,684 B, its six embedded PNGs → KTX2 (45.4 MB → 16.4 MB), and its +755,677-triangle foliage primitive down to a 5.1% far rung. Turning a pass off ships every source +byte verbatim instead, counts all of them against `assets.budget.uncooked`, and is precisely what +makes `threenative build` fail with `TN_ASSETS_BUDGET_EXCEEDED` — read `dist.build-report.json` +before you conclude a pack is too big. An extension the cook owns no pass for (`.dna`, +`.strands.bin`, a vendor data file) is copied through and still resolves through the same manifest +entry, so nothing needs `public/` to serve it. Ignore what the cook writes beside the manifest — +`public/basis/`, `public/shared/` and `public/bake.receipt.json` are outputs too, and a +content-addressed name is not the only output name. + What the import will and will not do: - **A pack of uncooked editor assets cannot be converted at all**, and this is the common case for