From 5e5ec9829839bc205ad325164232b63ffd15bc59 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 16:04:27 -0700 Subject: [PATCH 01/15] feat(assets): cook embedded EXT_texture_webp images (PRD-485 phase 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit decodeImageBytes reads WebP through sharp — already declared in this package's dependencies and unused until now, so no dependency and no new native binary reaches a game (every template already ships the same one via @gltf-transform/cli). An embedded WebP now cooks like a PNG: KTX2 on a target that decodes it, a capped PNG on the decoder-free path, and the pass disposes EXT_texture_webp once no image is left in it because the glTF writer emits every attached extension. The pass's "never grow a native dependency" rule changes with it: this is a Node-only build-time package and sharp is the only native thing in it. Gates: red before (TN_ASSETS_TEXTURE_CONTAINER: 'character.glb#checker' is not a PNG or JPEG), green after — 26/26 model-texture-pass, 474 passed / 2 skipped across packages/assets/__tests__, pnpm typecheck, pnpm lint, assets build + publint. --- ...high-quality-assets-go-through-the-cook.md | 1 + .../__tests__/model-texture-pass.spec.ts | 192 +++++++++++++++++- packages/assets/src/passes/decode-image.ts | 29 ++- packages/assets/src/passes/model-textures.ts | 9 +- packages/assets/src/passes/texture.ts | 2 +- 5 files changed, 226 insertions(+), 7 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 9f042a7dca..67de9533ed 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -49,6 +49,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Status:** NOT STARTED **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` - [ ] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. + Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. - [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. #### Phase 2: The Fab tree cooks by default diff --git a/packages/assets/__tests__/model-texture-pass.spec.ts b/packages/assets/__tests__/model-texture-pass.spec.ts index 7458841893..db27849d8f 100644 --- a/packages/assets/__tests__/model-texture-pass.spec.ts +++ b/packages/assets/__tests__/model-texture-pass.spec.ts @@ -1,9 +1,11 @@ import { mkdir, readFile, stat, writeFile } from "node:fs/promises"; +import { createRequire } from "node:module"; import path from "node:path"; import { type Document, NodeIO } from "@gltf-transform/core"; -import { ALL_EXTENSIONS } from "@gltf-transform/extensions"; +import { ALL_EXTENSIONS, EXTTextureWebP } from "@gltf-transform/extensions"; import { read as readKTX2 } from "ktx-parse"; import { MeshoptDecoder, MeshoptEncoder } from "meshoptimizer"; +import { PNG } from "pngjs"; import { describe, expect, it, vi } from "vitest"; import { buildFixtureDocument } from "../../../test-support/generate-fixture-model.js"; import { rgbaPng } from "../../../test-support/png.js"; @@ -81,6 +83,118 @@ async function compiled( return { buffer: result.buffer, entry: result.entry ?? {} }; } +/** + * Two flat halves, never one solid colour: `prune` deletes a single-colour texture and folds it + * into the material factor, which would leave these cases measuring a material and no image. + */ +const WEBP_HALVES = [ + [200, 130, 40], + [20, 60, 220], +] as const; + +/** A lossless WebP of the requested size, encoded by the decoder the pass uses. */ +async function tinyWebp(size = 4): Promise { + const sharp = (await import("sharp")).default; + const pixels = Buffer.alloc(size * size * 4); + for (let index = 0; index < size * size; index += 1) { + pixels.set([...WEBP_HALVES[index % size < size / 2 ? 0 : 1], 255], index * 4); + } + return sharp(pixels, { raw: { channels: 4, height: size, width: size } }) + .webp({ lossless: true }) + .toBuffer(); +} + +/** + * The shared fixture with its base colour replaced by an `EXT_texture_webp` image, which is what + * a Fab or McD export ships: the extension declared and required, the payload in a WebP. + */ +async function fixtureWithWebp(size?: number): Promise { + const document = buildFixtureDocument(); + document.createExtension(EXTTextureWebP).setRequired(true); + document + .getRoot() + .listTextures()[0] + ?.setImage(await tinyWebp(size)) + .setMimeType("image/webp"); + // The writer only emits the extensions it was told about, and an undeclared `image/webp` + // would be the invalid document this fixture exists to avoid. + return Buffer.from(await new NodeIO().registerExtensions(ALL_EXTENSIONS).writeBinary(document)); +} + +interface IBasisModule { + readonly KTX2File: new ( + bytes: Uint8Array, + ) => { + close(): void; + getHeight(): number; + getImageTranscodedSizeInBytes(mip: number, layer: number, face: number, format: number): number; + getWidth(): number; + isValid(): boolean; + startTranscoding(): boolean; + transcodeImage( + dst: Uint8Array, + mip: number, + layer: number, + face: number, + format: number, + getAlphaForOpaqueFormats: number, + channel0: number, + channel1: number, + ): boolean; + }; + readonly initializeBasis: () => void; + readonly transcoder_texture_format: Readonly>; +} + +/** + * Mip 0 of a KTX2 as RGBA8, decoded by the same `basis_transcoder` the runtime ships. Three + * publishes it CommonJS inside an ESM package, so it is evaluated here the way its own wrapper + * does — the alternative is trusting the container header instead of the pixels. + */ +async function transcodeFirstLevel(ktx2: Uint8Array): Promise { + const paths = basisTranscoderPaths(); + const directory = path.dirname(paths.javascriptPath); + const source = await readFile(paths.javascriptPath, "utf8"); + const shim: { exports: unknown } = { exports: {} }; + new Function("module", "exports", "require", "__filename", "__dirname", source)( + shim, + shim.exports, + createRequire(import.meta.url), + paths.javascriptPath, + directory, + ); + const factory = shim.exports as (options: { + locateFile: (file: string) => string; + }) => Promise; + const basis = await factory({ locateFile: (file) => path.join(directory, file) }); + basis.initializeBasis(); + const file = new basis.KTX2File(ktx2); + // The emscripten bindings report booleans as numbers, so both calls are judged on truth. + expect(file.isValid()).toBeTruthy(); + expect(file.startTranscoding()).toBeTruthy(); + // The WASM enum's numbering is not three's TranscoderFormat map, so the shipped enum names it. + const rgba32 = basis.transcoder_texture_format.cTFRGBA32; + if (rgba32 === undefined) throw new Error("the shipped transcoder exposes no cTFRGBA32 format"); + const format = rgba32.value; + const pixels = new Uint8Array(file.getImageTranscodedSizeInBytes(0, 0, 0, format)); + expect(file.transcodeImage(pixels, 0, 0, 0, format, 0, -1, -1)).toBeTruthy(); + file.close(); + return pixels; +} + +/** Every channel within `tolerance` of the source half the pixel belongs to. */ +function expectColours(pixels: Uint8Array, width: number, tolerance: number): void { + expect(pixels.length).toBe(width * width * 4); + for (let offset = 0; offset < pixels.length; offset += 4) { + const half = WEBP_HALVES[(offset / 4) % width < width / 2 ? 0 : 1]; + for (let channel = 0; channel < 3; channel += 1) { + expect(Math.abs((pixels[offset + channel] ?? 0) - (half[channel] ?? 0))).toBeLessThanOrEqual( + tolerance, + ); + } + } +} + describe("embedded model textures", () => { it("should transcode every embedded image to KTX2 and declare KHR_texture_basisu", async () => { const input = await fixtureWithTextures({ width: 32 }); @@ -291,6 +405,82 @@ describe("embedded model textures", () => { }); }); +describe("embedded EXT_texture_webp", () => { + it("should cook a WebP image to KTX2 and drop the extension", async () => { + // A named per-slot codec also skips the keep-smaller-source guard: a 4x4 WebP is smaller + // than the KTX2 container around it, and the test needs the cook to have run. + const { buffer } = await compiled(await fixtureWithWebp(), { + textures: { overrides: [{ codec: "uastc", slot: "baseColorTexture" }] }, + }); + const json = JSON.parse(buffer.subarray(20, 20 + buffer.readUInt32LE(12)).toString("utf8")) as { + extensionsRequired?: string[]; + extensionsUsed?: string[]; + }; + expect(json.extensionsUsed ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).toContain("KHR_texture_basisu"); + + const cooked = (await readOutput(buffer)) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/ktx2"); + const image = cooked?.getImage() ?? new Uint8Array(); + expect(readKTX2(image)).toMatchObject({ pixelHeight: 4, pixelWidth: 4 }); + // UASTC is lossless for 8-bit RGBA, so the transcoded mip is the source colour: proof the + // WebP payload reached the encoder instead of a container that merely looked like one. + expectColours(await transcodeFirstLevel(image), 4, 1); + }); + + it("should cap an over-cap WebP image to PNG on a target with no transcoder", async () => { + const root = await makeTempDir("threenative-webp-decoder-free-"); + await mkdir(path.join(root, "assets")); + // 8x8 against a 4-longest-edge cap: `cappedSize` never returns less than one 4x4 block, so + // only an over-cap source can reach the PNG the decoder-free path emits. + await writeFile(path.join(root, "assets", "prop.glb"), await fixtureWithWebp(8)); + + await compileAssets({ + config: { models: { sharedImages: false, textures: { maxSize: 4 } } }, + cwd: root, + platform: "android", + }); + + const manifest = JSON.parse( + await readFile(path.join(root, "public", "assets.manifest.json"), "utf8"), + ) as { entries: Record> }; + const entry = manifest.entries["prop.glb"]; + if (entry === undefined || typeof entry.output !== "string") { + throw new Error("no manifest entry for 'prop.glb'"); + } + expect(entry.extensions ?? []).not.toContain("EXT_texture_webp"); + + const cooked = (await readOutput(await readFile(path.join(root, "public", entry.output)))) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/png"); + const image = Buffer.from(cooked?.getImage() ?? new Uint8Array()); + expect(parsePng(image)).toMatchObject({ height: 4, width: 4 }); + // A PNG is lossless, so the resampled colour must be the source one. + expectColours(PNG.sync.read(image).data, 4, 1); + }); + + it("should still fail closed on a container that is not an image", async () => { + const document = buildFixtureDocument(); + document + .getRoot() + .listTextures()[0] + ?.setImage(Buffer.from("GIF89a not an image at all")) + .setMimeType("image/png"); + const input = Buffer.from(await new NodeIO().writeBinary(document)); + await expect( + apply(input, { + passes: { dedup: false, meshopt: false, prune: false, quantize: false, reorder: false }, + }), + ).rejects.toThrow(/TN_ASSETS_MODEL_TEXTURE_UNDECODABLE.*TN_ASSETS_TEXTURE_CONTAINER/su); + }); +}); + describe("mesh simplification", () => { it("should leave triangles untouched unless simplification is declared", async () => { const input = await fixtureWithTextures({ width: 32 }); diff --git a/packages/assets/src/passes/decode-image.ts b/packages/assets/src/passes/decode-image.ts index b9b6ce00b4..c14dc1bc09 100644 --- a/packages/assets/src/passes/decode-image.ts +++ b/packages/assets/src/passes/decode-image.ts @@ -3,9 +3,10 @@ import { parsePng } from "../png.js"; /** * Decodes an encoded image's bytes to tight 8-bit RGBA for the Basis encoder, which takes - * decoded pixels through its Node-side `imageDecoder` hook. Pure-JS decoders on purpose: - * this package is Node-only and must never grow a native dependency. Containers neither - * decoder reads throw — a source this step cannot encode fails the build naming the file, + * decoded pixels through its Node-side `imageDecoder` hook. PNG and JPEG are pure-JS decoders + * already in the dependency tree; WebP is not, so it goes through `sharp` — declared but + * previously unused, native, and confined to this build-time package (PRD-485). Containers none + * of the three reads throw: a source this step cannot encode fails the build naming the file, * never silently ships uncompressed. */ export async function decodeImageBytes( @@ -23,8 +24,28 @@ export async function decodeImageBytes( assertRgba(image.data.length, image.width, image.height, logicalPath); return { data: new Uint8Array(image.data), height: image.height, width: image.width }; } + // RIFF container, `WEBP` form type — what an EXT_texture_webp image carries out of Fab. + if ( + bytes.length >= 12 && + bytes.toString("ascii", 0, 4) === "RIFF" && + bytes.toString("ascii", 8, 12) === "WEBP" + ) { + const sharp = (await import("sharp")).default; + try { + const { data, info } = await sharp(bytes) + .ensureAlpha() + .raw() + .toBuffer({ resolveWithObject: true }); + assertRgba(data.length, info.width, info.height, logicalPath); + return { data, height: info.height, width: info.width }; + } catch (error) { + throw new Error( + `TN_ASSETS_TEXTURE_UNREADABLE: '${logicalPath}' is a WebP the decoder could not read: ${error instanceof Error ? error.message : String(error)}`, + ); + } + } throw new Error( - `TN_ASSETS_TEXTURE_CONTAINER: '${logicalPath}' is not a PNG or JPEG the KTX2 encoder can read; convert it to .png or .jpg.`, + `TN_ASSETS_TEXTURE_CONTAINER: '${logicalPath}' is not a PNG, JPEG or WebP the KTX2 encoder can read; convert it to .png or .jpg.`, ); } diff --git a/packages/assets/src/passes/model-textures.ts b/packages/assets/src/passes/model-textures.ts index 07fb3d808b..bddab7cd02 100644 --- a/packages/assets/src/passes/model-textures.ts +++ b/packages/assets/src/passes/model-textures.ts @@ -1,5 +1,5 @@ import type { Document, Texture } from "@gltf-transform/core"; -import { KHRTextureBasisu } from "@gltf-transform/extensions"; +import { EXTTextureWebP, KHRTextureBasisu } from "@gltf-transform/extensions"; import { getTextureColorSpace, listTextureInfo, listTextureSlots } from "@gltf-transform/functions"; import { read as readKTX2 } from "ktx-parse"; import { PNG } from "pngjs"; @@ -583,6 +583,13 @@ export async function compressEmbeddedTextures( // and a glTF that pretends otherwise fails at the first frame instead of at load. document.createExtension(KHRTextureBasisu).setRequired(true); } + // A glTF writer emits every extension still attached, so a model authored with WebP images + // would keep *requiring* EXT_texture_webp after the cook replaced those images. Dropped only + // once no image is left in the container: an over-cap image the project chose to keep as + // authored is still a WebP a reader has to understand. + if (!textures.some((texture) => texture.getMimeType() === "image/webp")) { + document.disposeExtension(EXTTextureWebP.EXTENSION_NAME); + } return { bytesAfter, bytesBefore, diff --git a/packages/assets/src/passes/texture.ts b/packages/assets/src/passes/texture.ts index 14026a2362..b9fa713a96 100644 --- a/packages/assets/src/passes/texture.ts +++ b/packages/assets/src/passes/texture.ts @@ -224,7 +224,7 @@ export function textureResizePass(options: ITextureResizeOptions): IAssetPass { }; } -/** PNG signature or JPEG SOI, the two containers `decodeImageBytes` reads. */ +/** PNG signature or JPEG SOI, the two containers the standalone pass sniffs headers for. */ function isPngOrJpeg(bytes: Buffer): boolean { if (bytes.length >= 3 && bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) { return true; From ba1544cf47cd3ee49a7e4714c442f82b0b3a866d Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 16:05:18 -0700 Subject: [PATCH 02/15] docs(PRD-485): tick embedded WebP cook on its red-green spec --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 67de9533ed..8a1839dcfa 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -48,7 +48,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges #### Phase 1: Embedded WebP cooks **Status:** NOT STARTED **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` -- [ ] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. +- [x] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. - [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. From bd6e40f4be6e37f3e1a19b327ebbb037e0158fc5 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 16:14:58 -0700 Subject: [PATCH 03/15] =?UTF-8?q?docs(PRD-485):=20reopen=20the=20WebP=20bo?= =?UTF-8?q?x=20=E2=80=94=20the=20not-smaller=20guard=20keeps=20WebP=20on?= =?UTF-8?q?=20lumen-hall?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 8a1839dcfa..4c76dae60d 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -48,7 +48,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges #### Phase 1: Embedded WebP cooks **Status:** NOT STARTED **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` -- [x] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. +- [ ] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. **Reopened 2026-10-03:** on `lumen-hall` the cook decodes the WebP but the `not-smaller` guard keeps the original (3× per prop at 1024²), so `lady.glb`/`sanctuary.glb` still require `EXT_texture_webp`. Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. - [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. From c284f2a70b2265383469cbb2a88b57306264bb68 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 16:48:38 -0700 Subject: [PATCH 04/15] fix(assets): never retain a WebP through the not-smaller guard (PRD-485 phase 1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit lumen-hall's lady.glb and sanctuary.glb cook with `compression skipped: not-smaller` three times each, so the shipped GLBs kept `images: image/webp` and required `EXT_texture_webp` — a container the native desktop/Android/iOS decoder-free paths cannot read. Retention is now conditioned on the container as well as on size: only `UNIVERSAL_IMAGE_CONTAINERS` (PNG, JPEG) may keep authored bytes, so a WebP always converts — to KTX2 on a decoding target even when that is larger, or to the capped PNG on the decoder-free path. `EXT_texture_webp` then always leaves the output, and the size increase is reported by the existing `bytes before -> after` row of the build report (+621% on lady.glb), with no new marker. PNG/JPEG behaviour is unchanged. `keepSmallerSource` moves from `true` to `"universal-containers"` in the model pass's cache key and in the shared-image store settings: a warm cook keyed on the old rule would otherwise keep shipping the WebP-requiring models this replaces. Verified: red (519 -> 519 bytes retained, extension still required) then green in the `embedded EXT_texture_webp` block of `packages/assets/__tests__/model-texture-pass.spec.ts`, whose new PNG leg pins that a PNG whose KTX2 would be larger is still kept; file 28/28, `packages/assets` 476 passed, `pnpm typecheck` and `pnpm lint` clean. --- ...high-quality-assets-go-through-the-cook.md | 4 +- .../__tests__/model-texture-pass.spec.ts | 59 +++++++++++++++++++ .../assets/__tests__/shared-images.spec.ts | 7 ++- packages/assets/src/passes/model-textures.ts | 44 +++++++++----- packages/assets/src/passes/model.ts | 7 ++- 5 files changed, 102 insertions(+), 19 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 4c76dae60d..6341c23779 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -48,9 +48,9 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges #### Phase 1: Embedded WebP cooks **Status:** NOT STARTED **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` -- [ ] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. **Reopened 2026-10-03:** on `lumen-hall` the cook decodes the WebP but the `not-smaller` guard keeps the original (3× per prop at 1024²), so `lady.glb`/`sanctuary.glb` still require `EXT_texture_webp`. +- [x] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. **Reopened 2026-10-03:** on `lumen-hall` the cook decodes the WebP but the `not-smaller` guard keeps the original (3× per prop at 1024²), so `lady.glb`/`sanctuary.glb` still require `EXT_texture_webp`. **Closed 2026-10-03:** retention is now conditioned on the container, not only on size — `UNIVERSAL_IMAGE_CONTAINERS` (PNG/JPEG) may keep authored bytes, WebP always converts to KTX2 (KTX2 target) or PNG (decoder-free), so `EXT_texture_webp` always leaves the output and the increase lands in the existing `bytes before -> after` row (+621% on `lady.glb`). Red: 519 -> 519 bytes retained with the extension still required; green: `image/ktx2` only, `extensionsRequired` `[EXT_meshopt_compression, KHR_mesh_quantization, KHR_texture_basisu]`, both GLBs re-verified in `lumen-hall`. `keepSmallerSource` moved from `true` to `"universal-containers"` in the model pass's cache key so a warm cook cannot keep shipping the old output. File 28/28, `packages/assets` 476 passed, `pnpm typecheck` and `pnpm lint` clean. Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. -- [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. +- [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. Partly done 2026-10-03: `npx threenative build --target web` exits 0 with the installed `@threenative/assets` tarball and both WebP props cook (`lady.df65302d.glb` 1562 KB, `sanctuary.9353c235.glb` 2546 KB, `image/ktx2` only). The scenario itself was not run in this lane, so the box stays open. #### Phase 2: The Fab tree cooks by default **Status:** NOT STARTED diff --git a/packages/assets/__tests__/model-texture-pass.spec.ts b/packages/assets/__tests__/model-texture-pass.spec.ts index db27849d8f..3a3ee87cf9 100644 --- a/packages/assets/__tests__/model-texture-pass.spec.ts +++ b/packages/assets/__tests__/model-texture-pass.spec.ts @@ -465,6 +465,65 @@ describe("embedded EXT_texture_webp", () => { expectColours(PNG.sync.read(image).data, 4, 1); }); + it("should convert a WebP even when its KTX2 is larger", async () => { + // The real-game case: a flat-colour 256x256 WebP costs far fewer bytes than the KTX2 + // container around it, so the keep-smaller-source guard retained the original and the + // cooked model still required `EXT_texture_webp` — which the decoder-free native targets + // cannot read. A container only an extension-reading loader understands is never kept. + const authored = (await readOutput(await fixtureWithWebp(256))) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker") + ?.getImage()?.byteLength; + + const { buffer, entry } = await compiled(await fixtureWithWebp(256)); + const summary = entry.embeddedTextures as + | { + readonly bytesAfter: number; + readonly bytesBefore: number; + readonly formats: Readonly>; + readonly skippedCompression: Readonly>; + } + | undefined; + + // Proof the not-smaller guard really held here: the cook grew the payload, and the row it + // grew is the one the size report already prints. + expect(summary?.bytesAfter ?? 0).toBeGreaterThan(summary?.bytesBefore ?? 0); + expect(summary?.skippedCompression?.checker).toBeUndefined(); + expect(summary?.formats.checker).toBe("etc1s"); + + const json = JSON.parse(buffer.subarray(20, 20 + buffer.readUInt32LE(12)).toString("utf8")) as { + extensionsRequired?: string[]; + extensionsUsed?: string[]; + images: { mimeType?: string }[]; + }; + expect(json.images.some((image) => image.mimeType === "image/webp")).toBe(false); + expect(json.extensionsUsed ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).not.toContain("EXT_texture_webp"); + expect(json.extensionsRequired ?? []).toContain("KHR_texture_basisu"); + // The image itself, not just the extension name, got bigger: that is the cost this test + // buys, and it is the number a reader of the build report sees. + const cooked = (await readOutput(buffer)) + .getRoot() + .listTextures() + .find((texture) => texture.getName() === "checker"); + expect(cooked?.getMimeType()).toBe("image/ktx2"); + expect(cooked?.getImage()?.byteLength ?? 0).toBeGreaterThan(authored ?? 0); + }); + + it("should still keep a PNG whose KTX2 would be larger", async () => { + // The other half of the same rule, and the reason it is a container test rather than a size + // test: PNG and JPEG need no extension, so retaining them costs a reader nothing. + const { buffer, entry } = await compiled(await fixtureWithTextures({ width: 4 })); + const summary = entry.embeddedTextures as { + readonly skippedCompression: Readonly>; + }; + expect(summary.skippedCompression.checker).toBe("not-smaller"); + for (const texture of (await readOutput(buffer)).getRoot().listTextures()) { + expect(texture.getMimeType()).toBe("image/png"); + } + }); + it("should still fail closed on a container that is not an image", async () => { const document = buildFixtureDocument(); document diff --git a/packages/assets/__tests__/shared-images.spec.ts b/packages/assets/__tests__/shared-images.spec.ts index 0ad1f1ad55..ff65852103 100644 --- a/packages/assets/__tests__/shared-images.spec.ts +++ b/packages/assets/__tests__/shared-images.spec.ts @@ -214,7 +214,12 @@ describe("shared model images", () => { sharedImageKey(texture.getImage() ?? new Uint8Array(), { colorSpace: getTextureColorSpace(texture), slots: [...listTextureSlots(texture)].sort(), - textures: { keepSmallerSource: true, maxSize: null, overrides: [], quality: null }, + textures: { + keepSmallerSource: "universal-containers", + maxSize: null, + overrides: [], + quality: null, + }, }), ); const store = createSharedImageStore(); diff --git a/packages/assets/src/passes/model-textures.ts b/packages/assets/src/passes/model-textures.ts index bddab7cd02..c52e7b249c 100644 --- a/packages/assets/src/passes/model-textures.ts +++ b/packages/assets/src/passes/model-textures.ts @@ -104,6 +104,14 @@ const GPU_BYTES_PER_PIXEL: Readonly> = { /** Slots whose content is a tangent-space normal: data, and ETC1S destroys it. */ const NORMAL_SLOTS: ReadonlySet = new Set(["clearcoatNormalTexture", "normalTexture"]); +/** + * Containers every target of this build decodes on its own, so keeping the authored bytes costs a + * reader nothing. `image/webp` is deliberately absent: it is only reachable through + * `EXT_texture_webp`, which a target without a WebP decoder cannot satisfy — retaining one would + * hand back a model the very requirement this stage removes. + */ +const UNIVERSAL_IMAGE_CONTAINERS: ReadonlySet = new Set(["image/jpeg", "image/png"]); + /** The five slots every glTF material declares directly; extension slots are covered by * the per-texture `slots` snapshot, which walks whatever parents actually reference it. */ const CORE_SLOTS = [ @@ -482,23 +490,27 @@ export async function compressEmbeddedTextures( } if (decoderFree) { // No Basis transcoder on this target, so the cap is honoured by resampling and re-emitting - // a PNG. An image already within the cap keeps its authored bytes; the file is never - // rewritten and KHR_texture_basisu is never declared. + // a PNG. An image already within the cap keeps its authored bytes — unless its container is + // one only an extension-reading loader understands, where keeping them would ship the very + // requirement this path exists to remove. KHR_texture_basisu is never declared. const target = cappedSize(decoded.width, decoded.height, maxSize); - if (target.width === decoded.width && target.height === decoded.height) { + const downsampled = target.width !== decoded.width || target.height !== decoded.height; + if (!downsampled && UNIVERSAL_IMAGE_CONTAINERS.has(texture.getMimeType())) { bytesAfter += image.byteLength; gpuBytesAfter += gpuBytes(decoded.width, decoded.height, "none"); formats[key] = "none"; continue; } - const data = resampleRgba( - decoded.data, - decoded.width, - decoded.height, - target.width, - target.height, - getTextureColorSpace(texture) === "srgb", - ); + const data = downsampled + ? resampleRgba( + decoded.data, + decoded.width, + decoded.height, + target.width, + target.height, + getTextureColorSpace(texture) === "srgb", + ) + : decoded.data; const png = new PNG({ height: target.height, width: target.width }); png.data = Buffer.from(data); const encoded = PNG.sync.write(png); @@ -506,7 +518,7 @@ export async function compressEmbeddedTextures( bytesAfter += encoded.byteLength; gpuBytesAfter += gpuBytes(target.width, target.height, "none"); formats[key] = "none"; - resized += 1; + if (downsampled) resized += 1; continue; } @@ -552,12 +564,16 @@ export async function compressEmbeddedTextures( : { isUASTC: false, qualityLevel: quality }), }); // A tiny source can cost less than the KTX2 container alone. Keep its exact bytes when - // encoding cannot save download bytes; a named codec override still requests GPU compression. + // encoding cannot save download bytes *and* the authored container needs no extension to be + // read; a WebP costs a decoder-free target the whole model, which is never the smaller cost. + // A named codec override still requests GPU compression either way. The size increase is + // reported by the existing `bytes before -> after` row of the build report. if ( encoded.byteLength >= image.byteLength && target.width === decoded.width && target.height === decoded.height && - !explicit + !explicit && + UNIVERSAL_IMAGE_CONTAINERS.has(texture.getMimeType()) ) { bytesAfter += image.byteLength; gpuBytesAfter += gpuBytes(decoded.width, decoded.height, "none"); diff --git a/packages/assets/src/passes/model.ts b/packages/assets/src/passes/model.ts index 12e3e5003e..d9ebd4fba4 100644 --- a/packages/assets/src/passes/model.ts +++ b/packages/assets/src/passes/model.ts @@ -819,7 +819,10 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { decoderFree: options.textures?.decoderFree ?? false, encoder: KTX2_ENCODER_VERSION, maxSize: options.textures?.maxSize ?? null, - keepSmallerSource: true, + // Retention is now conditioned on the container, so it is spelled out rather than + // named by a boolean: a warm cache keyed on the old rule would keep shipping the + // WebP-requiring models this replaced. + keepSmallerSource: "universal-containers", overrides: options.textures?.overrides ?? [], quality: options.textures?.quality ?? null, }, @@ -1079,7 +1082,7 @@ function sharedSettings( : { decoderFree: textureOptions.decoderFree ?? false, encoder: KTX2_ENCODER_VERSION, - keepSmallerSource: true, + keepSmallerSource: "universal-containers", maxSize: textureOptions.maxSize ?? null, overrides: textureOptions.overrides ?? [], quality: textureOptions.quality ?? null, From 7ed40dbe5fad46d7c6a48c44662a1d20b10b1258 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 17:07:26 -0700 Subject: [PATCH 05/15] fix(core): a cooked model survives being cloned (PRD-485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit three clones through `new this.constructor().copy(source)`, so `ClusteredMesh` died in its own constructor's table check before a game drew anything: the default cook puts a cluster DAG in every model, and lumen-hall's authored statues stood a second copy of one. `clone()` now builds from the source's own arguments, and `copy()` completes the job the three way: the baked table and the vertex attributes are shared — one copy of the vertices on the GPU, since three keys attributes on the attribute — while the cut is the copy's own, because the cut lives in the geometry's index buffer and two meshes on one geometry would overwrite each other's every frame. `ClusteredBatchRoot.clone()` keeps its batch, so a copied root is still cut instead of freezing on the cut it was cloned with. Red: 5 failing specs on `packages/core/__tests__/clustered*.spec.ts`, the game's TypeError. Green: 49 in those four files, 2322 across packages/core, plus typecheck, lint and build. --- .../core/__tests__/clustered-batch.spec.ts | 18 ++++ .../core/__tests__/clustered-mesh.spec.ts | 79 ++++++++++++++++ packages/core/src/clustered-batch.ts | 13 +++ packages/core/src/clustered-mesh.ts | 89 +++++++++++++++---- 4 files changed, 181 insertions(+), 18 deletions(-) diff --git a/packages/core/__tests__/clustered-batch.spec.ts b/packages/core/__tests__/clustered-batch.spec.ts index ea35b8a52e..0901886385 100644 --- a/packages/core/__tests__/clustered-batch.spec.ts +++ b/packages/core/__tests__/clustered-batch.spec.ts @@ -502,4 +502,22 @@ describe("ClusteredBatch", () => { }).build({ parent: new Object3D() }), ).toThrow(/at least one placed copy/u); }, 180_000); + + it("clones into a root that is still cut", () => { + // `new this.constructor()` left `batch` undefined, so a cloned root quietly stopped being cut + // and drew whatever its bands happened to hold. Detection is structural, so the engine could + // not even tell the copy apart from an ordinary object. + const parent = new Object3D(); + const batch = spread(); + const root = batch.build({ name: "boulders", parent }); + camera.position.set(0, 0, 0); + const triangles = batch.update(camera, 1080); + + const copy = root.clone() as unknown as { batch: ClusteredBatch }; + expect(copy.batch).toBe(batch); + // Still found by the engine's walk, and still drawing: a copy whose `batch` was undefined + // returned nothing and was never cut again. + expect(updateClusteredMeshes(copy as unknown as Object3D, camera, 1080)).toBeGreaterThan(0); + expect(triangles).toBeGreaterThan(0); + }, 180_000); }); diff --git a/packages/core/__tests__/clustered-mesh.spec.ts b/packages/core/__tests__/clustered-mesh.spec.ts index f5a9b23b12..ec2769e991 100644 --- a/packages/core/__tests__/clustered-mesh.spec.ts +++ b/packages/core/__tests__/clustered-mesh.spec.ts @@ -1,6 +1,7 @@ import { BufferAttribute, BufferGeometry, + Group, MeshBasicMaterial, PerspectiveCamera, Vector3, @@ -442,3 +443,81 @@ describe("ClusteredMesh", () => { ).toThrow(/needs a perspective camera/u); }); }); + +describe("ClusteredMesh cloning", () => { + const camera = new PerspectiveCamera(60, 16 / 9, 0.1, 1000); + + /** This mesh's triangles, with the camera `z` units out from where it stands. */ + function cutAt(mesh: ClusteredMesh, z: number): number { + camera.position.set(0, 0, z); + return mesh.update(camera, 1080); + } + + // three clones through `new this.constructor().copy(source)`, so any game that stands a second + // copy of a cooked model where it wants a statue used to die in `assertTable` before it drew + // anything. Lumen-hall's authored statues did, from the first model the default cook produced. + it("clones into a mesh that draws the same thing", () => { + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const triangles = cutAt(mesh, 6); + const clone = mesh.clone(); + + expect(clone.table).toBe(mesh.table); + // A copy starts at the authored detail, exactly as a fresh mesh does, and only gives it up + // once something cuts it. + expect(clone.drawnTriangles).toBe(body.indices.length / 3); + expect(cutAt(clone, 6)).toBe(triangles); + expect(clone.geometry.drawRange).toEqual(mesh.geometry.drawRange); + expect(clone.geometry.getIndex()?.array).not.toBe(mesh.geometry.getIndex()?.array); + }); + + it("clones inside a group, which is how a game places a second copy", () => { + const figure = new Group(); + figure.add(new ClusteredMesh(geometry(), new MeshBasicMaterial(), table)); + const clone = figure.clone(true); + const copied = clone.children[0] as ClusteredMesh; + + expect(copied).toBeInstanceOf(ClusteredMesh); + expect(copied.table).toBe((figure.children[0] as ClusteredMesh).table); + }); + + it("keeps the vertices shared and the cut its own, so the two are placed independently", () => { + // Sharing the geometry wholesale would be cheaper still and wrong: the cut lives in the + // geometry's index buffer, so two meshes on one geometry would overwrite each other's index + // every frame and both would draw whichever one spoke last. Sharing the vertex buffers by + // reference keeps one copy of them on the GPU — three keys its attributes on the attribute — + // and still lets each mesh hold a cut of its own. + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const beside = mesh.clone(); + + expect(beside.geometry).not.toBe(mesh.geometry); + expect(beside.geometry.attributes.position).toBe(mesh.geometry.attributes.position); + + const near = cutAt(mesh, 6); + expect(cutAt(beside, 6)).toBe(near); + + const across = mesh.clone(); + across.position.set(0, 0, 400); + expect(cutAt(across, 6)).toBeLessThan(near); + // And cutting the far copy left the near one drawing what it was drawing. + expect(cutAt(mesh, 6)).toBe(near); + }); + + it("disposes without taking the source's shared data with it", () => { + // The rule, since sharing is the point: a clone shares the baked table and the vertex + // attributes, and the geometry that owns them disposes them for everyone — exactly as three's + // own `Mesh.copy` shares geometry and material. What a clone owns outright is its cut. + const mesh = new ClusteredMesh(geometry(), new MeshBasicMaterial(), table); + const clone = mesh.clone(); + let sourceDisposed = false; + mesh.geometry.addEventListener("dispose", () => { + sourceDisposed = true; + }); + + clone.geometry.dispose(); + + expect(sourceDisposed).toBe(false); + expect(clone.geometry.attributes.position).toBe(mesh.geometry.attributes.position); + expect(mesh.table).toBe(table); + expect(mesh.drawnTriangles).toBe(body.indices.length / 3); + }); +}); diff --git a/packages/core/src/clustered-batch.ts b/packages/core/src/clustered-batch.ts index 7bbe0644ca..2b4035b5fd 100644 --- a/packages/core/src/clustered-batch.ts +++ b/packages/core/src/clustered-batch.ts @@ -126,6 +126,19 @@ export class ClusteredBatchRoot extends Object3D { constructor(readonly batch: ClusteredBatch) { super(); } + + /** + * A copy that is still being cut. + * + * three builds one through `new this.constructor()`, which left the copy's `batch` undefined + * and quietly stopped cutting it — a frozen cut that still drew. Sharing the batch is what + * keeps a copy alive: the batch owns the bands and their index buffers, so the copy draws the + * same cut as the root it was copied from, which is the same thing `Mesh.copy` does with + * geometry and `VirtualGeometryPlugin` does with a baked table. + */ + override clone(recursive = true): this { + return new ClusteredBatchRoot(this.batch).copy(this, recursive) as this; + } } function placementMatrix(placement: IClusteredPlacement): Matrix4 { diff --git a/packages/core/src/clustered-mesh.ts b/packages/core/src/clustered-mesh.ts index d9db2111f0..3dc79391b2 100644 --- a/packages/core/src/clustered-mesh.ts +++ b/packages/core/src/clustered-mesh.ts @@ -1,11 +1,4 @@ -import { - BufferAttribute, - type BufferGeometry, - type Camera, - type Material, - Mesh, - Vector3, -} from "three"; +import { BufferAttribute, BufferGeometry, type Camera, type Material, Mesh, Vector3 } from "three"; /** * A mesh that draws only the clusters this camera can resolve. @@ -163,15 +156,15 @@ function finestIndexCount(table: IClusterTable): number { } export class ClusteredMesh extends Mesh { - /** The baked payload. Read-only at run time; the bake is the only thing that writes it. */ - readonly table: IClusterTable; + /** The baked payload. The bake is the only thing that writes it; a copy takes the source's. */ + table!: IClusterTable; /** Screen-space error budget in pixels. Writable — it is the game's call. */ errorPixels: number; /** Camera movement, in the mesh's own units, below which the previous cut is kept. */ recutDistance: number; - readonly #cut: Uint32Array; - readonly #selection: Uint32Array; + #cut!: Uint32Array; + #selection!: Uint32Array; readonly #cameraLocal = new Vector3(); readonly #lastCut = new Vector3(); #drawnClusters = 0; @@ -188,11 +181,21 @@ export class ClusteredMesh extends Mesh { ) { super(geometry, surface); assertTable(table); - this.table = table; this.errorPixels = options.errorPixels ?? DEFAULT_ERROR_PIXELS; + this.recutDistance = options.recutDistance ?? radiusOf(table) * RECUT_FRACTION; + this.#load(table, geometry); + // Full detail until something cuts it. Virtual geometry ships on by default, so the mesh a + // game never touches has to draw exactly what an ordinary `Mesh` would have drawn — a class + // that starts invisible and waits to be updated would turn the default into a blank screen. + // `update` only ever takes detail away from here. + this.#compact(this.#finestCut()); + } + + /** Points this mesh at `table`, and installs its cut on the geometry it is going to draw. */ + #load(table: IClusterTable, geometry: BufferGeometry): void { + this.table = table; this.#cut = new Uint32Array(finestIndexCount(table)); this.#selection = new Uint32Array(table.ranges.length / 2); - this.recutDistance = options.recutDistance ?? radiusOf(table) * RECUT_FRACTION; // The geometry draws out of the cut buffer from here on. Its vertex attributes are untouched: // every level of the DAG indexes the vertices the mesh already shipped. // @@ -200,11 +203,44 @@ export class ClusteredMesh extends Mesh { // are given, so the cut this class writes every frame would never reach the GPU and the mesh // would draw a buffer of zeros — degenerate triangles, an empty screen, and no error anywhere. geometry.setIndex(new BufferAttribute(this.#cut, 1)); - // Full detail until something cuts it. Virtual geometry ships on by default, so the mesh a - // game never touches has to draw exactly what an ordinary `Mesh` would have drawn — a class - // that starts invisible and waits to be updated would turn the default into a blank screen. - // `update` only ever takes detail away from here. + } + + /** + * A second mesh over the same body, cut for its own place in the scene. + * + * three's `Object3D.clone` builds through `new this.constructor()`, and this class's constructor + * wants a table — so every game that stood a second copy of a cooked model where it wanted a + * statue died in `assertTable` before drawing anything. The arguments are the source's own, + * which is what a copy is. + */ + override clone(recursive = true): this { + return new ClusteredMesh(withOwnCut(this.geometry), this.material, this.table, { + errorPixels: this.errorPixels, + recutDistance: this.recutDistance, + }).copy(this, recursive) as this; + } + + /** + * Becomes a copy of `source`, sharing the bake and the vertices and keeping a cut of its own. + * + * `Mesh.copy` shares the geometry outright, which this class cannot let it do: the cut lives in + * the geometry's index buffer, so two meshes on one geometry would overwrite each other's index + * every frame and both would draw whichever spoke last. The copy takes the vertex attributes by + * reference instead — one copy of them on the GPU, because three keys its attributes on the + * attribute — and the shared table, which is frozen at bake time and cannot go stale. + */ + override copy(source: this, recursive = true): this { + super.copy(source, recursive); + if (!(source instanceof ClusteredMesh)) return this; + const geometry = withOwnCut(source.geometry); + this.#load(source.table, geometry); + this.geometry = geometry; + this.errorPixels = source.errorPixels; + this.recutDistance = source.recutDistance; + // Nothing this copy has drawn yet, so the next update has to cut it for where it stands. + this.#hasCut = false; this.#compact(this.#finestCut()); + return this; } /** Every cluster the bake produced from the source triangles — the mesh as authored. */ @@ -286,6 +322,23 @@ export class ClusteredMesh extends Mesh { } } +/** + * The same vertices under a second geometry, so a second mesh can hold a cut of its own. + * + * `BufferGeometry.clone()` would do this and would also copy every position, normal and uv — the + * memory a cooked model exists not to spend twice. Sharing the `BufferAttribute` objects shares + * their GPU buffers too, because three keys its attribute cache on the attribute, not the geometry. + */ +function withOwnCut(geometry: BufferGeometry): BufferGeometry { + const shell = new BufferGeometry(); + for (const [name, attribute] of Object.entries(geometry.attributes)) + shell.setAttribute(name, attribute); + shell.boundingSphere = geometry.boundingSphere; + shell.boundingBox = geometry.boundingBox; + shell.groups = [...geometry.groups]; + return shell; +} + function radiusOf(table: IClusterTable): number { let radius = 0; for (let cluster = 0; cluster * 4 < table.bounds.length; cluster += 1) From 877991c387eae1711a28038a2e252c139c507e56 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 17:12:45 -0700 Subject: [PATCH 06/15] =?UTF-8?q?docs(PRD-485):=20Phase=201=20done=20?= =?UTF-8?q?=E2=80=94=20WebP=20always=20converts,=20lumen-hall=20cooks=20an?= =?UTF-8?q?d=20passes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 6341c23779..b1339415af 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -46,11 +46,11 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges ## Execution Phases #### Phase 1: Embedded WebP cooks -**Status:** NOT STARTED +**Status:** DONE 2026-10-03 **Files:** `packages/assets/src/passes/decode-image.ts`, `packages/assets/src/passes/model-textures.ts`; `packages/assets/__tests__/model-texture-pass.spec.ts` - [x] [local] A GLB with an `EXT_texture_webp` image cooks to KTX2 on a decoding target and to a capped PNG on the decoder-free path. No output still requires `EXT_texture_webp`. proof: red-green case in `pnpm exec vitest run packages/assets/__tests__/model-texture-pass.spec.ts`. **Result 2026-10-03 (`5e5ec9829`):** red `TN_ASSETS_TEXTURE_CONTAINER … is not a PNG or JPEG` before; green 3/3 after (KTX2 mip 0 transcodes back to the source within 1, decoder-free capped PNG, unknown container still throws); file 26/26, `packages/assets` 474 passed. **Reopened 2026-10-03:** on `lumen-hall` the cook decodes the WebP but the `not-smaller` guard keeps the original (3× per prop at 1024²), so `lady.glb`/`sanctuary.glb` still require `EXT_texture_webp`. **Closed 2026-10-03:** retention is now conditioned on the container, not only on size — `UNIVERSAL_IMAGE_CONTAINERS` (PNG/JPEG) may keep authored bytes, WebP always converts to KTX2 (KTX2 target) or PNG (decoder-free), so `EXT_texture_webp` always leaves the output and the increase lands in the existing `bytes before -> after` row (+621% on `lady.glb`). Red: 519 -> 519 bytes retained with the extension still required; green: `image/ktx2` only, `extensionsRequired` `[EXT_meshopt_compression, KHR_mesh_quantization, KHR_texture_basisu]`, both GLBs re-verified in `lumen-hall`. `keepSmallerSource` moved from `true` to `"universal-containers"` in the model pass's cache key so a warm cook cannot keep shipping the old output. File 28/28, `packages/assets` 476 passed, `pnpm typecheck` and `pnpm lint` clean. Implementation: `sharp` decodes WebP in `decode-image.ts` — the first rung of the decision above, already declared at `packages/assets/package.json:57`, so no dependency is added and no new binary is installed for a game (every template already carries `@gltf-transform/cli`, which drags the same native package). It loads in the pass's worker thread and stays external in `dist`. `compressEmbeddedTextures` disposes `EXT_texture_webp` once no image is left in it, because the glTF writer emits every attached extension. The rule that `decode-image.ts` "must never grow a native dependency" is changed here: this is a Node-only build-time package, and `sharp` is the only native thing in it. Spec: the `embedded EXT_texture_webp` block of `model-texture-pass.spec.ts`, with the KTX2 leg decoding mip 0 through the shipped `basis_transcoder`. -- [ ] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. Partly done 2026-10-03: `npx threenative build --target web` exits 0 with the installed `@threenative/assets` tarball and both WebP props cook (`lady.df65302d.glb` 1562 KB, `sanctuary.9353c235.glb` 2546 KB, `image/ktx2` only). The scenario itself was not run in this lane, so the box stays open. +- [x] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. Partly done 2026-10-03: `npx threenative build --target web` exits 0 with the installed `@threenative/assets` tarball and both WebP props cook (`lady.df65302d.glb` 1562 KB, `sanctuary.9353c235.glb` 2546 KB, `image/ktx2` only). The scenario itself was not run in this lane, so the box stays open. **Result 2026-10-03:** sandbox branch `prd485/lumen-hall-default-cook` (local worktree, engine tarballs from this branch): `npx threenative build --target web` exit 0, all 6 GLBs cooked (33.96 MB → 8.93 MB, -73.7%); `lady.glb`/`sanctuary.glb` now `image/ktx2`, no `EXT_texture_webp` (`c284f2a70`; lady +621% bytes, block-compressed on GPU). The first scenario run crashed in `ClusteredMesh.clone()` (cooked models carry a cluster DAG; three's `clone()` constructs with no arguments) — fixed in core `7ed40dbe5` with red-green specs. Then `playtests/assets.playtest.json` passes 4/4 on nvidia/turing WebGPU. Same-pose before (`models: "none"` + a temporary uncooked-budget override, same tarballs) vs after: 0.01% of pixels differ (mean abs 0.25). On the new engine `models: "none"` alone no longer builds lumen-hall (`TN_ASSETS_BUDGET_EXCEEDED` 68 MB > 64 MB). Known corner: a WebP that is not a whole number of 4×4 blocks keeps its bytes under the block-size guard. #### Phase 2: The Fab tree cooks by default **Status:** NOT STARTED From 81331cce8094ce7f5ae9518308bd36cb3af04618 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 17:36:12 -0700 Subject: [PATCH 07/15] =?UTF-8?q?docs(PRD-485):=20Phase=202=20boxes=201-2?= =?UTF-8?q?=20tick=20=E2=80=94=20the=20Fab=20tree=20cooks=20by=20default?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Hornbeam needed no engine fix: on the tarballs packed from this branch the default cook exits 0 in 163 s (127 MB -> 27 MB, 6 embedded PNGs to KTX2, 0 uncooked bytes) and its trunk primitive gets a 3-rung TN_discrete_lod chain with a 1,500-triangle far rung. Box 3 (browser + native scenarios) stays open, and the PRD now records that the 745k-triangle MASK foliage took TN_virtual_geometry with stalled cluster levels rather than a chain. --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index b1339415af..619af027a5 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -53,10 +53,10 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges - [x] [local] `lumen-hall` builds on web with `models: "none"` deleted, and its asset scenario passes. proof: `npx threenative build` in the game, then `node packages/playtest/dist/runner/cli.js playtests/assets.playtest.json --url --browser-recipe webgpu`. Partly done 2026-10-03: `npx threenative build --target web` exits 0 with the installed `@threenative/assets` tarball and both WebP props cook (`lady.df65302d.glb` 1562 KB, `sanctuary.9353c235.glb` 2546 KB, `image/ktx2` only). The scenario itself was not run in this lane, so the box stays open. **Result 2026-10-03:** sandbox branch `prd485/lumen-hall-default-cook` (local worktree, engine tarballs from this branch): `npx threenative build --target web` exit 0, all 6 GLBs cooked (33.96 MB → 8.93 MB, -73.7%); `lady.glb`/`sanctuary.glb` now `image/ktx2`, no `EXT_texture_webp` (`c284f2a70`; lady +621% bytes, block-compressed on GPU). The first scenario run crashed in `ClusteredMesh.clone()` (cooked models carry a cluster DAG; three's `clone()` constructs with no arguments) — fixed in core `7ed40dbe5` with red-green specs. Then `playtests/assets.playtest.json` passes 4/4 on nvidia/turing WebGPU. Same-pose before (`models: "none"` + a temporary uncooked-budget override, same tarballs) vs after: 0.01% of pixels differ (mean abs 0.25). On the new engine `models: "none"` alone no longer builds lumen-hall (`TN_ASSETS_BUDGET_EXCEEDED` 68 MB > 64 MB). Known corner: a WebP that is not a whole number of 4×4 blocks keeps its bytes under the block-size guard. #### Phase 2: The Fab tree cooks by default -**Status:** NOT STARTED +**Status:** PARTIAL **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) -- [ ] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. -- [ ] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. +- [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. +- [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03:** `TN_discrete_lod` is in `extensionsUsed` and the row reads `1 primitive(s) to 3 level(s), 10349 -> 1500 triangles`; the cooked GLB's `TN_discrete_lod` on that primitive gives `lod0Triangles 10349`, `counts [5190, 2595, 1500]` — far rung 1,500 = 0.2% of 755,677. AutoLOD did not decline: it skipped 2 of 3 primitives, both `virtual-owned`. The two foliage primitives (225,310 + 520,018 triangles, the MASK mask) went to `TN_virtual_geometry` instead of a chain, and their cluster levels report `stopReason: stalled` at ~100% of their triangles, so the trunk chain is the only simplification in the asset. - [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. #### Phase 3: Android and the cold-agent path From 84833072f326306af18e9042322d09bc81170d5d Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 17:47:39 -0700 Subject: [PATCH 08/15] =?UTF-8?q?docs(PRD-485):=20reopen=20the=20Hornbeam?= =?UTF-8?q?=20LOD=20box=20=E2=80=94=20foliage=20virtual=20geometry=20stall?= =?UTF-8?q?s=20at=20~100%?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 619af027a5..e1dc1233fe 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -56,7 +56,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Status:** PARTIAL **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) - [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. -- [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03:** `TN_discrete_lod` is in `extensionsUsed` and the row reads `1 primitive(s) to 3 level(s), 10349 -> 1500 triangles`; the cooked GLB's `TN_discrete_lod` on that primitive gives `lod0Triangles 10349`, `counts [5190, 2595, 1500]` — far rung 1,500 = 0.2% of 755,677. AutoLOD did not decline: it skipped 2 of 3 primitives, both `virtual-owned`. The two foliage primitives (225,310 + 520,018 triangles, the MASK mask) went to `TN_virtual_geometry` instead of a chain, and their cluster levels report `stopReason: stalled` at ~100% of their triangles, so the trunk chain is the only simplification in the asset. +- [ ] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03:** `TN_discrete_lod` is in `extensionsUsed` and the row reads `1 primitive(s) to 3 level(s), 10349 -> 1500 triangles`; the cooked GLB's `TN_discrete_lod` on that primitive gives `lod0Triangles 10349`, `counts [5190, 2595, 1500]` — far rung 1,500 = 0.2% of 755,677. AutoLOD did not decline: it skipped 2 of 3 primitives, both `virtual-owned`. The two foliage primitives (225,310 + 520,018 triangles, the MASK mask) went to `TN_virtual_geometry` instead of a chain, and their cluster levels report `stopReason: stalled` at ~100% of their triangles, so the trunk chain is the only simplification in the asset. **Reopened by review 2026-10-03:** the 1,500-triangle far rung belongs to the trunk primitive only (10,349 → 1,500). The two MASK foliage primitives (225,310 + 520,018 = 745,328 of 755,677 triangles) were taken by `TN_virtual_geometry`, whose levels report `stopReason: stalled` at ~100% of triangles, so the asset as a whole has no far rung near 10%. Next: why cluster simplification stalls on alpha-cutout foliage (or let AutoLOD own MASK foliage). - [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. #### Phase 3: Android and the cold-agent path From 42e70f77da3475588587ec88b26ab584b0f5d9ad Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 18:08:23 -0700 Subject: [PATCH 09/15] fix(assets): a DAG that reduced nothing hands the primitive to AutoLOD (PRD-485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Hornbeam's two MASK foliage primitives (225,310 + 520,018 = 745,328 of 755,677 triangles) shipped with no coarse representation at all. The cluster builder simplifies every group with `LockBorder` (dag.ts:259) and every edge of a disconnected alpha-cutout card is a border, so the first fold simplifies nothing (dag.ts:461), dag.ts:319 records `stalled`, and every level reports the source count. The primitive was then `virtual-owned` (eligibility.ts:182), so AutoLOD declined it and the asset's only chain was the 10,349-triangle trunk. `bakeVirtualGeometry` now declines a primitive whose DAG stopped `stalled` while still holding more than 80% of its triangles: no extension is attached, so the primitive is unowned and the discrete ladder's card + terminal levels — the path built for exactly this shape (generate.ts:851) — take it. Measured, not assumed: a DAG that stalls after real folds has shed its triangles and keeps the primitive. Red: on a 16,384-card MASK fixture the cook emitted `TN_virtual_geometry`, `lod.reasons ["virtual-owned"]` and no chain; the report named the bake as "no primitive was dense enough". Green: no extension, `lod.reasons []`, a 3-level card chain whose far rung is 1,638 triangles = 5.0% of LOD0 with `cellCoverage 1` on every level, and the alpha cutoff the asset shipped unchanged. Sandbox proof (fab-import-proof, repacked assets tarball): build exit 0, `lod ...: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 0 skipped`, `TN_virtual_geometry` gone from extensionsUsed, far total 38,767 = 5.13%, cooked GLB 26,990,484 -> 26,428,684 B. Gates: packages/assets 480 passed, pnpm typecheck 0, pnpm lint 0 (1032 warnings, unchanged), pnpm check:docs 0. --- ...high-quality-assets-go-through-the-cook.md | 2 +- packages/assets/__tests__/virtual-dag.spec.ts | 28 ++ .../assets/__tests__/virtual-pass.spec.ts | 243 +++++++++++++++++- packages/assets/src/report.ts | 11 +- packages/assets/src/virtual/bake.ts | 116 ++++++--- 5 files changed, 362 insertions(+), 38 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index e1dc1233fe..e829476b3c 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -56,7 +56,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Status:** PARTIAL **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) - [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. -- [ ] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03:** `TN_discrete_lod` is in `extensionsUsed` and the row reads `1 primitive(s) to 3 level(s), 10349 -> 1500 triangles`; the cooked GLB's `TN_discrete_lod` on that primitive gives `lod0Triangles 10349`, `counts [5190, 2595, 1500]` — far rung 1,500 = 0.2% of 755,677. AutoLOD did not decline: it skipped 2 of 3 primitives, both `virtual-owned`. The two foliage primitives (225,310 + 520,018 triangles, the MASK mask) went to `TN_virtual_geometry` instead of a chain, and their cluster levels report `stopReason: stalled` at ~100% of their triangles, so the trunk chain is the only simplification in the asset. **Reopened by review 2026-10-03:** the 1,500-triangle far rung belongs to the trunk primitive only (10,349 → 1,500). The two MASK foliage primitives (225,310 + 520,018 = 745,328 of 755,677 triangles) were taken by `TN_virtual_geometry`, whose levels report `stopReason: stalled` at ~100% of triangles, so the asset as a whole has no far rung near 10%. Next: why cluster simplification stalls on alpha-cutout foliage (or let AutoLOD own MASK foliage). +- [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03 (reopened box, closed):** cause `packages/assets/src/virtual/dag.ts:319` — the group fold simplifies each cluster group with `LockBorder` (`dag.ts:259`), and every edge of a disconnected alpha-cutout card is a border, so the first fold simplifies nothing, every level reports the source count and `stopReason: stalled`; the primitive was then `virtual-owned` (`lod/eligibility.ts:182`) and no ladder could take it. Fix `packages/assets/src/virtual/bake.ts`: the bake *declines* a primitive whose DAG stopped `stalled` while still holding >80% of its triangles (`VIRTUAL_BAKE_VERSION` 3 -> 4) and attaches no extension, so the primitive is unowned and the discrete ladder's card + terminal levels take it. Red on `virtual-pass.spec.ts` "hands cards it cannot reduce to the discrete ladder, which does reduce them" (16,384 MASK cards: `extensionsUsed` had `TN_virtual_geometry`, `lod.reasons ["virtual-owned"]`, far level 50%) and on the report line; green after (no `TN_virtual_geometry`, `lod.reasons []`, far level 1,638 = 5.0% of 32,768, `cellCoverage 1` on every level). Same lane, sandbox `prd485/fab-import-default-cook` with the repacked assets tarball: `npx threenative build --target web` exit 0, report row `lod …: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 7304028 payload bytes, 0 skipped` and `TN_virtual_geometry` gone from `extensionsUsed`. Per primitive: material 0 `225310 -> [113601, 56948, 11266]` (5.00%, absolute far error 0.331), material 1 `520018 -> [260988, 130580, 26001]` (5.00%, 0.253), material 2 `10349 -> [5190, 2595, 1500]` (14.49%, 0.211) — far total 38,767 = 5.13% of 755,677. Cooked GLB 26,990,484 -> 26,428,684 B (-561,800 B, the DAG payload). Known gap: `compile.ts` drops `entry.virtual` when it rebuilds the manifest entry, so the new `declined` count is in the pass summary and `formatModelSizes` but prints in neither CLI lane. - [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. #### Phase 3: Android and the cold-agent path diff --git a/packages/assets/__tests__/virtual-dag.spec.ts b/packages/assets/__tests__/virtual-dag.spec.ts index 8e51dbfcd8..97632d3a65 100644 --- a/packages/assets/__tests__/virtual-dag.spec.ts +++ b/packages/assets/__tests__/virtual-dag.spec.ts @@ -41,6 +41,21 @@ function manyShells(count: number): IBody { return { indices: Uint32Array.from(indices), positions: Float32Array.from(positions) }; } +/** Disconnected alpha-cutout cards — the shape a Fab foliage primitive is, and why it stalls. */ +function cards(count: number): IBody { + const indices: number[] = []; + const positions: number[] = []; + const perRow = Math.ceil(Math.sqrt(count)); + for (let card = 0; card < count; card += 1) { + const base = positions.length / 3; + const x = (card % perRow) * 2; + const y = Math.floor(card / perRow) * 2; + positions.push(x, y, 0, x + 0.5, y, 0, x + 0.5, y + 0.5, 0, x, y + 0.5, 0); + indices.push(base, base + 1, base + 2, base, base + 2, base + 3); + } + return { indices: Uint32Array.from(indices), positions: Float32Array.from(positions) }; +} + function edgeKey(a: number, b: number): number { return a < b ? a * 4294967296 + b : b * 4294967296 + a; } @@ -213,6 +228,19 @@ describe("the cluster DAG", () => { expect(dag.roots.length).toBeGreaterThan(1); }, 120_000); + it("AC6 red — a body of alpha-cutout cards stalls on its very first fold, at 100%", async () => { + const body = cards(400); + const dag = await buildClusterDag(body.indices, body.positions); + + // Not the shells' 64-triangle floor: no collapse at all. Every edge of every card is a border + // edge, so `LockBorder` holds the whole body still and the only remaining candidates are the + // quads' diagonals, which collapse to nothing. PRD-485's Hornbeam foliage: 745,328 triangles + // whose DAG therefore reports every level at the source count, and the bake declines it. + expect(dag.stopReason).toBe("stalled"); + expect(dag.groups).toHaveLength(0); + for (const level of dag.levels) expect(level.triangleCount).toBe(body.indices.length / 3); + }, 120_000); + it("refuses a malformed mesh rather than baking nonsense", async () => { await expect(buildClusterDag(new Uint32Array([0, 1]), new Float32Array(9))).rejects.toThrow( /non-empty triangle list/, diff --git a/packages/assets/__tests__/virtual-pass.spec.ts b/packages/assets/__tests__/virtual-pass.spec.ts index 6380b75ee6..bea7d59913 100644 --- a/packages/assets/__tests__/virtual-pass.spec.ts +++ b/packages/assets/__tests__/virtual-pass.spec.ts @@ -1,7 +1,7 @@ import { Document, NodeIO } from "@gltf-transform/core"; import { ALL_EXTENSIONS } from "@gltf-transform/extensions"; import { MeshoptDecoder } from "meshoptimizer"; -import { TorusKnotGeometry } from "three"; +import { SphereGeometry, TorusKnotGeometry } from "three"; import { describe, expect, it } from "vitest"; import { modelPass } from "../src/passes/model.js"; import { formatModelSizes } from "../src/report.js"; @@ -58,6 +58,77 @@ async function sourceGlb(geometry: TorusKnotGeometry): Promise { return Buffer.from(await io.writeBinary(document)); } +/** + * Twelve closed shells that touch nothing, in one primitive: the body whose DAG halves a dozen + * times and then floors out above one cluster, which is the `stalled` the decline must not read as + * "reduced nothing" (PRD-485). + */ +async function shellsGlb(shellCount: number): Promise { + const document = new Document(); + const buffer = document.createBuffer(); + const scene = document.createScene(); + const shell = new SphereGeometry(0.5, 32, 16); + const positions: number[] = []; + const normals: number[] = []; + const indices: number[] = []; + const source = shell.attributes.position?.array ?? []; + const sourceNormals = shell.attributes.normal?.array ?? []; + const shellIndices = Array.from(shell.index?.array ?? []); + // A three.js sphere leaves two pole vertices no triangle references, and the pass's self-verify + // fails a cook that drops them — the same trap `denseGlb` avoids with a torus knot. Compacting to + // the used vertices is the whole fix. + const used = [...new Set(shellIndices)].sort((left, right) => left - right); + const remap = new Map(used.map((vertex, slot) => [vertex, slot])); + for (let count = 0; count < shellCount; count += 1) { + const base = positions.length / 3; + for (const vertex of used) { + const slot = vertex * 3; + positions.push( + (source[slot] as number) + count * 40, + source[slot + 1] as number, + source[slot + 2] as number, + ); + normals.push( + sourceNormals[slot] as number, + sourceNormals[slot + 1] as number, + sourceNormals[slot + 2] as number, + ); + } + for (const vertex of shellIndices) indices.push(base + (remap.get(vertex) as number)); + } + const primitive = document + .createPrimitive() + .setAttribute( + "POSITION", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(positions)) + .setBuffer(buffer), + ) + .setAttribute( + "NORMAL", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(normals)) + .setBuffer(buffer), + ) + .setIndices( + document + .createAccessor() + .setType("SCALAR") + .setArray(Uint32Array.from(indices)) + .setBuffer(buffer), + ) + .setMaterial(document.createMaterial("stone")); + scene.addChild( + document.createNode("cavern").setMesh(document.createMesh("cavern").addPrimitive(primitive)), + ); + const io = new NodeIO().registerExtensions(ALL_EXTENSIONS); + return Buffer.from(await io.writeBinary(document)); +} + async function readBack(buffer: Buffer, registerVirtual: boolean): Promise { await MeshoptDecoder.ready; const io = new NodeIO() @@ -83,6 +154,72 @@ async function compile(virtual: boolean): Promise<{ buffer: Buffer; entry: unkno return { buffer: result.buffer, entry: result.entry }; } +/** + * A canopy of disconnected alpha-cutout cards: what a Fab foliage primitive is, and what the + * Hornbeam's two of them are (PRD-485). + * + * Every card is its own connected component with its own four vertices, so the whole rim is a + * border edge and the cluster builder — which simplifies each group with the rim locked — has + * nothing it is allowed to collapse. The card count is what puts the far level under a tenth of the + * source: the ladder's terminal target is `max(1500, 5% of LOD0)`, and only above 30,000 triangles + * is the ratio the binding half of that. + */ +async function foliageGlb(cardCount: number): Promise { + const document = new Document(); + const buffer = document.createBuffer(); + const scene = document.createScene(); + const positions: number[] = []; + const normals: number[] = []; + const uvs: number[] = []; + const indices: number[] = []; + const perRow = Math.ceil(Math.sqrt(cardCount)); + for (let card = 0; card < cardCount; card += 1) { + const base = positions.length / 3; + const x = (card % perRow) * 2; + const y = Math.floor(card / perRow) * 2; + const height = 0.5 + ((card * 37) % 11) / 22; + positions.push(x, y, 0, x + height, y, 0, x + height, y + height, 0, x, y + height, 0); + normals.push(0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1); + uvs.push(0, 0, 1, 0, 1, 1, 0, 1); + indices.push(base, base + 1, base + 2, base, base + 2, base + 3); + } + const primitive = document + .createPrimitive() + .setAttribute( + "POSITION", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(positions)) + .setBuffer(buffer), + ) + .setAttribute( + "NORMAL", + document + .createAccessor() + .setType("VEC3") + .setArray(Float32Array.from(normals)) + .setBuffer(buffer), + ) + .setAttribute( + "TEXCOORD_0", + document.createAccessor().setType("VEC2").setArray(Float32Array.from(uvs)).setBuffer(buffer), + ) + .setIndices( + document + .createAccessor() + .setType("SCALAR") + .setArray(Uint32Array.from(indices)) + .setBuffer(buffer), + ) + .setMaterial(document.createMaterial("Foliage").setAlphaMode("MASK").setAlphaCutoff(0.5)); + scene.addChild( + document.createNode("canopy").setMesh(document.createMesh("canopy").addPrimitive(primitive)), + ); + const io = new NodeIO().registerExtensions(ALL_EXTENSIONS); + return Buffer.from(await io.writeBinary(document)); +} + describe("the virtual-geometry pass", () => { it('bakes nothing when the config says "none"', async () => { const { buffer, entry } = await compile(false); @@ -219,6 +356,84 @@ describe("the virtual-geometry pass", () => { expect(summary.skipped).toBe(1); }); + it("hands cards it cannot reduce to the discrete ladder, which does reduce them", async () => { + // PRD-485. A body of disconnected alpha-cutout cards is the one shape the cluster builder + // refuses: every edge of every card is a border, `LockBorder` holds all of it, and the first + // fold simplifies nothing — so a stalled DAG reports 100% of the triangles at every level and + // owns 745,328 Hornbeam foliage triangles with no coarse representation at all. The bake hands + // that primitive back and the ladder gives it the far level the DAG could not. + const result = await modelPass({ + lod: true, + virtual: { minSourceTriangles: 1024 }, + }).apply(await foliageGlb(16384), "canopy.glb"); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + + expect(jsonOf(result.buffer).extensionsUsed ?? []).not.toContain("TN_virtual_geometry"); + const virtual = ( + result.entry as { + virtual: { clusters: number; declined: number; payloadBytes: number; primitives: number }; + } + ).virtual; + expect(virtual).toMatchObject({ clusters: 0, declined: 1, payloadBytes: 0, primitives: 0 }); + + const lod = ( + result.entry as { + lod: { + primitives: { + cardLevels: { cellCoverage: number }[]; + levels: { absoluteError: number; triangles: number }[]; + strategy: string; + trianglesBefore: number; + }[]; + reasons: string[]; + skipped: number; + }; + } + ).lod; + // Not `virtual-owned` any more, and not skipped for anything else either. + expect(lod.reasons).not.toContain("virtual-owned"); + expect(lod.skipped).toBe(0); + const chain = lod.primitives[0]; + if (chain === undefined) throw new Error("the declined cards got no chain at all"); + expect(chain.strategy).toBe("cards"); + + const far = chain.levels[chain.levels.length - 1]?.triangles ?? 0; + expect(far).toBeGreaterThan(0); + expect(far / chain.trianglesBefore).toBeLessThanOrEqual(0.1); + // The crown keeps a card in every occupied cell at every level, and each level's error is + // wider than the one before it — the two properties the runtime's distance switch reads. + for (const level of chain.cardLevels ?? []) expect(level.cellCoverage).toBe(1); + for (let level = 1; level < chain.levels.length; level += 1) + expect(chain.levels[level]?.absoluteError).toBeGreaterThanOrEqual( + chain.levels[level - 1]?.absoluteError ?? 0, + ); + + // And the cook still decides nothing about how it looks: the alpha cutoff the asset was + // authored with is the one that ships. + const material = ( + JSON.parse( + result.buffer.subarray(20, 20 + result.buffer.readUInt32LE(12)).toString("utf8"), + ) as { materials?: { alphaCutoff?: number; alphaMode?: string }[] } + ).materials?.[0]; + expect(material).toMatchObject({ alphaCutoff: 0.5, alphaMode: "MASK" }); + }, 300_000); + + it("keeps a DAG that stalled only after it had already shed its triangles", async () => { + // The decline is measured, not `stalled => gone`: twelve closed shells cannot become one + // cluster — meshoptimizer will not take a shell below about 64 triangles — but the DAG has + // halved several times before it floors out, and that payload is worth keeping. + const result = await modelPass({ virtual: { minSourceTriangles: 1024 } }).apply( + await shellsGlb(12), + "cavern.glb", + ); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + + expect( + (result.entry as { virtual: { declined: number; primitives: number; stopReason: string } }) + .virtual, + ).toMatchObject({ declined: 0, primitives: 1, stopReason: "stalled" }); + }, 600_000); + it("AC8 — a reader that has never heard of the extension gets the source mesh", async () => { const { buffer } = await compile(true); const stock = await readBack(buffer, false); @@ -264,6 +479,7 @@ describe("the virtual-geometry pass", () => { virtual: { bakeSeconds: 12.34, clusters: 512, + declined: 0, levels: 6, payloadBytes: 4096, primitives: 1, @@ -287,6 +503,7 @@ describe("the virtual-geometry pass", () => { virtual: { bakeSeconds: 1, clusters: 8, + declined: 0, levels: 2, payloadBytes: 64, primitives: 1, @@ -298,4 +515,28 @@ describe("the virtual-geometry pass", () => { expect(lines.join("\n")).toContain("a DAG hit the level cap and is unfinished"); }); + + it("AC7 — the report names the primitives it handed to the LOD ladder", () => { + const lines = formatModelSizes([ + { + after: 900, + before: 1000, + logicalPath: "canopy.glb", + virtual: { + bakeSeconds: 0.1, + clusters: 0, + declined: 2, + levels: 0, + payloadBytes: 0, + primitives: 0, + skipped: 1, + stopReason: "root", + }, + }, + ]); + + // "no primitive was dense enough" would be the wrong reason for a body whose cards were dense + // enough and whose DAG reduced nothing. + expect(lines.join("\n")).toContain("1 skipped, 2 declined to the LOD ladder"); + }); }); diff --git a/packages/assets/src/report.ts b/packages/assets/src/report.ts index b8d203d6c8..dc61d06527 100644 --- a/packages/assets/src/report.ts +++ b/packages/assets/src/report.ts @@ -64,6 +64,8 @@ export interface IEmbeddedTextureRow { export interface IVirtualRow { readonly bakeSeconds: number; readonly clusters: number; + /** Primitives handed back to the discrete ladder because their DAG reduced nothing. */ + readonly declined: number; readonly levels: number; readonly payloadBytes: number; readonly primitives: number; @@ -409,19 +411,22 @@ function simplifyLine(row: IModelSizeRow): readonly string[] { * Names what the DAG cost and where it stopped. * * A `cap` stop is called out because it means a DAG ran out of levels rather than finishing, and a - * bake that clustered nothing says so rather than leaving the reader to infer it from a zero. + * bake that clustered nothing says so rather than leaving the reader to infer it from a zero. A + * declined primitive is named too: "no primitive was dense enough" is the wrong reason for a body + * whose foliage is dense enough and whose DAG reduced nothing. */ function virtualLine(row: IModelSizeRow): readonly string[] { const virtual = row.virtual; if (virtual === undefined) return []; + const declined = virtual.declined > 0 ? `, ${virtual.declined} declined to the LOD ladder` : ""; if (virtual.primitives === 0) return [ - `virtual ${row.logicalPath}: no primitive was dense enough to cluster (${virtual.skipped} skipped)`, + `virtual ${row.logicalPath}: no primitive was dense enough to cluster (${virtual.skipped} skipped${declined})`, ]; const warning = virtual.stopReason === "cap" ? " — a DAG hit the level cap and is unfinished" : ""; return [ - `virtual ${row.logicalPath}: ${virtual.clusters} cluster(s) over ${virtual.levels} level(s) on ${virtual.primitives} primitive(s), ${virtual.skipped} skipped, ${virtual.payloadBytes} payload bytes, bake ${virtual.bakeSeconds.toFixed(1)} s, stopped at ${virtual.stopReason}${warning}`, + `virtual ${row.logicalPath}: ${virtual.clusters} cluster(s) over ${virtual.levels} level(s) on ${virtual.primitives} primitive(s), ${virtual.skipped} skipped${declined}, ${virtual.payloadBytes} payload bytes, bake ${virtual.bakeSeconds.toFixed(1)} s, stopped at ${virtual.stopReason}${warning}`, ]; } function compactLine(row: IModelSizeRow): readonly string[] { diff --git a/packages/assets/src/virtual/bake.ts b/packages/assets/src/virtual/bake.ts index 5f9c061205..210acd3f3f 100644 --- a/packages/assets/src/virtual/bake.ts +++ b/packages/assets/src/virtual/bake.ts @@ -1,7 +1,12 @@ // The pipeline's side of the DAG bake: which primitives get one, and what the report says about it. import type { Document, Primitive } from "@gltf-transform/core"; -import { type IClusterDagOptions, buildClusterDag } from "./dag.js"; +import { + type IClusterDag, + type IClusterDagOptions, + type IVirtualLevel, + buildClusterDag, +} from "./dag.js"; import { TNVirtualGeometry, attachVirtualGeometry, virtualGeometryBytes } from "./extension.js"; /** @@ -52,6 +57,11 @@ export interface IModelVirtualOptions { export interface IModelVirtualSummary { readonly bakeSeconds: number; readonly clusters: number; + /** + * Primitives the bake looked at and handed back because their DAG reduced nothing — left for the + * discrete ladder, which is named here because "no clusters" alone reads as a bug. + */ + readonly declined: number; readonly levels: number; readonly payloadBytes: number; readonly primitives: number; @@ -61,7 +71,37 @@ export interface IModelVirtualSummary { } /** Bumped when the bake's output changes for the same input. Part of the compile cache key. */ -export const VIRTUAL_BAKE_VERSION = 3; +export const VIRTUAL_BAKE_VERSION = 4; + +/** + * The share of its own triangles a DAG has to shed before it keeps the primitive. + * + * Below that the payload is three to four times the primitive's compiled bytes (see + * {@link IModelVirtualOptions.minSourceTriangles}) for a cut that draws the same triangles, and the + * discrete ladder's terminal level reaches 5% of LOD0 on the same geometry. + */ +const DECLINE_KEEP_RATIO = 0.8; + +/** + * Whether the bake hands this primitive back instead of owning it (PRD-485). + * + * A `stalled` DAG whose coarsest level still holds most of the source has no coarse representation + * to offer. meshoptimizer will not collapse a body of disconnected alpha-cutout cards with every + * border locked — each card's whole rim is a border — so the first fold simplifies nothing + * (`virtual/dag.ts:461`), every level reports the source count, and every threshold draws LOD0: the + * Hornbeam's two foliage primitives, 745,328 triangles with no coarse step at all. Owning them + * anyway costs 3–4x their bytes and leaves them `virtual-owned` (`lod/eligibility.ts:182`), so the + * bake declines them and the discrete ladder — whose terminal pass exists for exactly this shape, + * `lod/generate.ts:851` — gets the primitive instead. + * + * Measured, not assumed: a DAG that stalls *after* real folds has shed its triangles and keeps the + * primitive, and so does every stop reason but `stalled`. + */ +function declines(dag: IClusterDag, sourceTriangles: number): boolean { + if (dag.stopReason !== "stalled") return false; + const coarsest = dag.levels[dag.levels.length - 1] as IVirtualLevel; + return coarsest.triangleCount > sourceTriangles * DECLINE_KEEP_RATIO; +} /** * The density at which a cluster DAG starts paying for its own bytes. See @@ -117,49 +157,59 @@ export async function bakeVirtualGeometry( let extension: TNVirtualGeometry | null = null; let clusters = 0; + let declined = 0; let levels = 0; let payloadBytes = 0; let primitives = 0; let skipped = 0; let stopReason = "root"; - for (const mesh of document.getRoot().listMeshes()) { - for (const primitive of mesh.listPrimitives()) { - const indices = primitive.getIndices(); - const positions = positionsOf(primitive); - // A skinned body ships whole: the loader's clustered mesh is a plain `Mesh`, so a baked - // primitive would lose its skin (and the rig would not clone). - if ( - indices === null || - positions === null || - primitive.getMode() !== 4 || - primitive.getAttribute("JOINTS_0") !== null - ) { - skipped += 1; - continue; - } - if (indices.getCount() / 3 < minSourceTriangles) { - skipped += 1; - continue; - } - const dag = await buildClusterDag( - Uint32Array.from(indices.getArray() as ArrayLike), - positions, - dagOptions, - ); - extension ??= document.createExtension(TNVirtualGeometry).setRequired(false); - const virtual = attachVirtualGeometry(document, extension, primitive, dag); - clusters += dag.clusters.length; - levels = Math.max(levels, dag.levels.length); - payloadBytes += virtualGeometryBytes(virtual); - primitives += 1; - stopReason = worse(stopReason, dag.stopReason); + for (const primitive of document + .getRoot() + .listMeshes() + .flatMap((mesh) => mesh.listPrimitives())) { + const indices = primitive.getIndices(); + const positions = positionsOf(primitive); + // A skinned body ships whole: the loader's clustered mesh is a plain `Mesh`, so a baked + // primitive would lose its skin (and the rig would not clone). + if ( + indices === null || + positions === null || + primitive.getMode() !== 4 || + primitive.getAttribute("JOINTS_0") !== null + ) { + skipped += 1; + continue; + } + const sourceTriangles = indices.getCount() / 3; + if (sourceTriangles < minSourceTriangles) { + skipped += 1; + continue; + } + const dag = await buildClusterDag( + Uint32Array.from(indices.getArray() as ArrayLike), + positions, + dagOptions, + ); + // No extension is created for a declined primitive, so a document whose every primitive is + // card soup never declares `TN_virtual_geometry` at all. + if (declines(dag, sourceTriangles)) { + declined += 1; + continue; } + extension ??= document.createExtension(TNVirtualGeometry).setRequired(false); + const virtual = attachVirtualGeometry(document, extension, primitive, dag); + clusters += dag.clusters.length; + levels = Math.max(levels, dag.levels.length); + payloadBytes += virtualGeometryBytes(virtual); + primitives += 1; + stopReason = worse(stopReason, dag.stopReason); } return { bakeSeconds: (now() - started) / 1000, clusters, + declined, levels, payloadBytes, primitives, From 73ce0f45d444ac2badfd0de25bf5c23b253b6e8d Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 18:31:36 -0700 Subject: [PATCH 10/15] docs(PRD-485): Hornbeam far rung 5.13%, web scenario passes --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index e829476b3c..8e3fa63cf8 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -57,7 +57,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) - [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. - [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03 (reopened box, closed):** cause `packages/assets/src/virtual/dag.ts:319` — the group fold simplifies each cluster group with `LockBorder` (`dag.ts:259`), and every edge of a disconnected alpha-cutout card is a border, so the first fold simplifies nothing, every level reports the source count and `stopReason: stalled`; the primitive was then `virtual-owned` (`lod/eligibility.ts:182`) and no ladder could take it. Fix `packages/assets/src/virtual/bake.ts`: the bake *declines* a primitive whose DAG stopped `stalled` while still holding >80% of its triangles (`VIRTUAL_BAKE_VERSION` 3 -> 4) and attaches no extension, so the primitive is unowned and the discrete ladder's card + terminal levels take it. Red on `virtual-pass.spec.ts` "hands cards it cannot reduce to the discrete ladder, which does reduce them" (16,384 MASK cards: `extensionsUsed` had `TN_virtual_geometry`, `lod.reasons ["virtual-owned"]`, far level 50%) and on the report line; green after (no `TN_virtual_geometry`, `lod.reasons []`, far level 1,638 = 5.0% of 32,768, `cellCoverage 1` on every level). Same lane, sandbox `prd485/fab-import-default-cook` with the repacked assets tarball: `npx threenative build --target web` exit 0, report row `lod …: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 7304028 payload bytes, 0 skipped` and `TN_virtual_geometry` gone from `extensionsUsed`. Per primitive: material 0 `225310 -> [113601, 56948, 11266]` (5.00%, absolute far error 0.331), material 1 `520018 -> [260988, 130580, 26001]` (5.00%, 0.253), material 2 `10349 -> [5190, 2595, 1500]` (14.49%, 0.211) — far total 38,767 = 5.13% of 755,677. Cooked GLB 26,990,484 -> 26,428,684 B (-561,800 B, the DAG payload). Known gap: `compile.ts` drops `entry.virtual` when it rebuilds the manifest entry, so the new `declined` count is in the pass summary and `formatModelSizes` but prints in neither CLI lane. -- [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. +- [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. **Web leg 2026-10-03:** `playtests/hornbeam.playtest.json` passes on nvidia/turing WebGPU (4 Fab imports, Hornbeam height in range, 5 textures, 755,677 LOD0 triangles near, 0 console/network/runtime errors). An earlier bridge-missing failure was the sandbox worktree lacking the gitignored `assets/fab/soul-cave` GLBs (engine failed closed naming the asset). Native desktop leg pending. #### Phase 3: Android and the cold-agent path **Status:** NOT STARTED From 5b1479460b6594ef593663132641dba398ec9f28 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 18:36:15 -0700 Subject: [PATCH 11/15] =?UTF-8?q?docs(PRD-485):=20Phase=202=20done=20?= =?UTF-8?q?=E2=80=94=20Hornbeam=20passes=20web=20and=20native=20desktop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 8e3fa63cf8..b19f3856f4 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -57,7 +57,7 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Files:** `packages/assets/src/lod/eligibility.ts` only if the Hornbeam is declined; `fab-import-proof/threenative.config.ts` (sandbox) - [x] [local] `fab-import-proof` builds on web with `assets` reduced to `exclude`. The build report shows the Hornbeam's six embedded textures encoded and 0 uncooked bytes for its GLB. proof: `npx threenative build`, `dist.build-report.json`. **Result 2026-10-03:** sandbox branch `prd485/fab-import-default-cook` (local worktree, engine tarballs from this branch), `npx threenative build --target web` exit 0 in 163 s. Hornbeam 126,934,280 -> 26,990,484 B (-78.7%), 6 embedded PNGs -> KTX2 45,378,709 -> 16,382,827 B (GPU 134,217,726 -> 25,165,824 B, 3 ETC1S + 3 UASTC, 0 skipped), budget row `uncooked 0 bytes`, artifact 51,402,535 B. No `packages/` file needed changing. - [x] [local] The cooked Hornbeam carries a `TN_discrete_lod` chain whose far rung is at most 10% of 755,677 triangles. proof: the build report's LOD row for the GLB. **Result 2026-10-03 (reopened box, closed):** cause `packages/assets/src/virtual/dag.ts:319` — the group fold simplifies each cluster group with `LockBorder` (`dag.ts:259`), and every edge of a disconnected alpha-cutout card is a border, so the first fold simplifies nothing, every level reports the source count and `stopReason: stalled`; the primitive was then `virtual-owned` (`lod/eligibility.ts:182`) and no ladder could take it. Fix `packages/assets/src/virtual/bake.ts`: the bake *declines* a primitive whose DAG stopped `stalled` while still holding >80% of its triangles (`VIRTUAL_BAKE_VERSION` 3 -> 4) and attaches no extension, so the primitive is unowned and the discrete ladder's card + terminal levels take it. Red on `virtual-pass.spec.ts` "hands cards it cannot reduce to the discrete ladder, which does reduce them" (16,384 MASK cards: `extensionsUsed` had `TN_virtual_geometry`, `lod.reasons ["virtual-owned"]`, far level 50%) and on the report line; green after (no `TN_virtual_geometry`, `lod.reasons []`, far level 1,638 = 5.0% of 32,768, `cellCoverage 1` on every level). Same lane, sandbox `prd485/fab-import-default-cook` with the repacked assets tarball: `npx threenative build --target web` exit 0, report row `lod …: 3 primitive(s) to 3 level(s), 755677 -> 38767 triangles, 7304028 payload bytes, 0 skipped` and `TN_virtual_geometry` gone from `extensionsUsed`. Per primitive: material 0 `225310 -> [113601, 56948, 11266]` (5.00%, absolute far error 0.331), material 1 `520018 -> [260988, 130580, 26001]` (5.00%, 0.253), material 2 `10349 -> [5190, 2595, 1500]` (14.49%, 0.211) — far total 38,767 = 5.13% of 755,677. Cooked GLB 26,990,484 -> 26,428,684 B (-561,800 B, the DAG payload). Known gap: `compile.ts` drops `entry.virtual` when it rebuilds the manifest entry, so the new `declined` count is in the pass summary and `formatModelSizes` but prints in neither CLI lane. -- [ ] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. **Web leg 2026-10-03:** `playtests/hornbeam.playtest.json` passes on nvidia/turing WebGPU (4 Fab imports, Hornbeam height in range, 5 textures, 755,677 LOD0 triangles near, 0 console/network/runtime errors). An earlier bridge-missing failure was the sandbox worktree lacking the gitignored `assets/fab/soul-cave` GLBs (engine failed closed naming the asset). Native desktop leg pending. +- [x] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. **Web leg 2026-10-03:** `playtests/hornbeam.playtest.json` passes on nvidia/turing WebGPU (4 Fab imports, Hornbeam height in range, 5 textures, 755,677 LOD0 triangles near, 0 console/network/runtime errors). An earlier bridge-missing failure was the sandbox worktree lacking the gitignored `assets/fab/soul-cave` GLBs (engine failed closed naming the asset). **Native desktop leg 2026-10-03:** `native-playtests/fab-import-native.playtest.json --target desktop` exit 0, 6/6 (4 imports, 8 textured, 5 masked, movement 4.97 m, 0 errors), 690 frames, adapter `native:vulkan/nvidia/NVIDIA GeForce RTX 2080/turing`, packaged from the installed `@threenative/runtime-native` 0.3.4 prebuilt; screenshot shows the Hornbeam canopy. Its first red was stale counts in the scenario (written before the Hornbeam import), corrected in the game branch. #### Phase 3: Android and the cold-agent path **Status:** NOT STARTED From a281d7ee8e4afc0b5bc5ba6eb9c882337534038e Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 18:45:37 -0700 Subject: [PATCH 12/15] docs(PRD-485): the Fab import step names the default cook, not `"none"` Phase 3 box 3. `finding-assets.md` already said the cook picks imported GLBs up with no configuration; it now says what the cook does with them and why `models: "none"` / `textures: "none"` is the wrong repair: a converted Fab pack's embedded PNG/WebP maps become KTX2, its attributes are quantized and Meshopt-compressed, and a `MASK` primitive gets a discrete LOD chain. Turning a pass off ships every source byte verbatim and counts it all against `assets.budget.uncooked`, which is what raises TN_ASSETS_BUDGET_EXCEEDED. Measured on the Hornbeam: 126,934,280 -> 26,428,684 B, six embedded PNGs 45.4 MB -> 16.4 MB, 755,677 triangles to a 5.1% far rung. Also names the three cook outputs a content-addressed glob cannot match (`public/basis/`, `public/shared/`, `public/bake.receipt.json`), found by building metahuman-lab's 301 MB tree through the cook. The Phase 3 box 2 build leg is recorded beside its box, unticked: the cook is exit 0 with 0 uncooked bytes, and the playtest is for the browser lane. Proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` 7/7, `pnpm check:docs` 2,229 links, plus check-doc-links/sync-agent-docs/ evidence-citations 30/30. No template file changed, so no scaffold hash moved. --- ...5-high-quality-assets-go-through-the-cook.md | 5 +++-- .../agent-docs/references/finding-assets.md | 17 +++++++++++++++++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index b19f3856f4..20d9722067 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -60,8 +60,9 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges - [x] [local] The Hornbeam scenario passes on web and on native desktop from installed tarballs. proof: `playtests/hornbeam.playtest.json --browser-recipe webgpu` and `native-playtests/fab-import-native.playtest.json --target desktop`. **Web leg 2026-10-03:** `playtests/hornbeam.playtest.json` passes on nvidia/turing WebGPU (4 Fab imports, Hornbeam height in range, 5 textures, 755,677 LOD0 triangles near, 0 console/network/runtime errors). An earlier bridge-missing failure was the sandbox worktree lacking the gitignored `assets/fab/soul-cave` GLBs (engine failed closed naming the asset). **Native desktop leg 2026-10-03:** `native-playtests/fab-import-native.playtest.json --target desktop` exit 0, 6/6 (4 imports, 8 textured, 5 masked, movement 4.97 m, 0 errors), 690 frames, adapter `native:vulkan/nvidia/NVIDIA GeForce RTX 2080/turing`, packaged from the installed `@threenative/runtime-native` 0.3.4 prebuilt; screenshot shows the Hornbeam canopy. Its first red was stale counts in the scenario (written before the Hornbeam import), corrected in the game branch. #### Phase 3: Android and the cold-agent path -**Status:** NOT STARTED +**Status:** PARTIAL **Files:** `metahuman-lab` asset layout (sandbox); `packages/create-threenative/agent-docs/references/finding-assets.md` - [ ] [local] `fab-import-proof` packages its cooked KTX2 and Meshopt output for Android V8 and renders the Hornbeam on the emulator. proof: `native-playtests/fab-import-native.playtest.json --target android` on the local emulator. - [ ] [local] `metahuman-lab` serves `content/` from `assets/` through the cook with no `assets` block, and `face-controls.playtest.json` passes on web. proof: `npx threenative build` plus that scenario with `--browser-recipe webgpu`. -- [ ] [local] `finding-assets.md`'s Fab import step says the default cook handles imported packs and that `"none"` is not a fix. proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` and `pnpm check:docs`. + Build leg 2026-10-03: sandbox branch `prd485/metahuman-lab-cook` (local worktree, engine tarballs from this branch). `npx threenative build --target web` exit 0 in 189 s, `threenative.config.ts` still carries no `assets` block. The 301 MB moved from `public/content/` to `assets/content/`, and the hand-written vite middleware that served it verbatim (`specimenContentPlugin`) is gone: `ctx.assets.resolve` reads `public/assets.manifest.json` and every load asks for the cooked name. `.dna` and `.strands.bin` are copied through as `kind: "other"` byte for byte (`head.dna` 4,829,103 B, `hair.strands.bin` 85,202,464 B) — an extension no pass owns gets a manifest entry like any other file. `specimen.glb` cooks 26,267,896 -> 16,104,464 B (-38.7%): 5 embedded PNGs 7,801,033 -> 2,957,889 B (3 uastc/etc1s, `T_MH_BentNormal_AO` and `T_Eye_N` kept under the existing `not-smaller` guard), 101,518 triangles and all 821 LOD0 morph targets intact, `EXT_meshopt_compression` + `KHR_mesh_quantization` + `KHR_texture_basisu` on the output, and `lod.reasons ["deforming", "authored-lod"]` — a skinned morph-target head is PRD-486's subject, not this one. `shirt.glb` 50,367,028 -> 543,040 B (-98.9%), `body.glb` 1,215,524 -> 349,680 B, 30 PNGs to KTX2 (total 162,186,687 B from 314,889,680 B of source). Budget `uncooked 0 bytes` against the 64,000,000 default ceiling — first time this game had a manifest at all. The scenario was not run in this lane, so the box stays open: `node packages/playtest/dist/runner/cli.js metahuman-lab/playtests/face-controls.playtest.json --url http://127.0.0.1:5197 --server-command "pnpm exec vite preview --host 127.0.0.1 --port 5197 --strictPort" --browser-recipe webgpu`. +- [x] [local] `finding-assets.md`'s Fab import step says the default cook handles imported packs and that `"none"` is not a fix. proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` and `pnpm check:docs`. **Result 2026-10-03:** the Fab import step now states what the default cook does to a converted pack (embedded PNG and WebP to KTX2, quantization and Meshopt, a discrete LOD chain for a `MASK` primitive) with the measured Hornbeam numbers, and that `models: "none"`/`textures: "none"` are not a fix — they ship every source byte, count it all against `assets.budget.uncooked`, and are what makes the build fail with `TN_ASSETS_BUDGET_EXCEEDED`. Green: `primary-docs.spec.ts` 7/7, `check:docs` 2,229 links, `check-doc-links` + `sync-agent-docs` + `evidence-citations` 30/30. No template file changed, so no scaffold hash moved. diff --git a/packages/create-threenative/agent-docs/references/finding-assets.md b/packages/create-threenative/agent-docs/references/finding-assets.md index aefade4aee..cb4bc6386f 100644 --- a/packages/create-threenative/agent-docs/references/finding-assets.md +++ b/packages/create-threenative/agent-docs/references/finding-assets.md @@ -135,6 +135,23 @@ Then load the returned path the ordinary way: `ctx.assets.model("fab/soul-cave/.../SM_S_Soul_Statue.glb")`. The normal asset compiler picks the GLBs up from `assets/` with no extra configuration. +**Cook the pack with the default settings. `models: "none"` and `textures: "none"` in +`threenative.config.ts` are not a fix, and a reason to reach for them is a bug report, not a +workaround.** A converted Fab pack is ordinary input to that cook, and the cook is where an +expensive import stops being expensive: embedded PNG and WebP maps are transcoded to KTX2 and +capped, mesh attributes are quantized and Meshopt-compressed, and a `MASK` (alpha-cutout) +primitive is handed a discrete LOD chain, so the LOD0 triangle count in the GLB is not the +triangle count the scene pays for. One measured tree: `SM_EuropeanHornbeam_Forest_01.glb` cooks +126,934,280 → 26,428,684 B, its six embedded PNGs → KTX2 (45.4 MB → 16.4 MB), and its +755,677-triangle foliage primitive down to a 5.1% far rung. Turning a pass off ships every source +byte verbatim instead, counts all of them against `assets.budget.uncooked`, and is precisely what +makes `threenative build` fail with `TN_ASSETS_BUDGET_EXCEEDED` — read `dist.build-report.json` +before you conclude a pack is too big. An extension the cook owns no pass for (`.dna`, +`.strands.bin`, a vendor data file) is copied through and still resolves through the same manifest +entry, so nothing needs `public/` to serve it. Ignore what the cook writes beside the manifest — +`public/basis/`, `public/shared/` and `public/bake.receipt.json` are outputs too, and a +content-addressed name is not the only output name. + What the import will and will not do: - **A pack of uncooked editor assets cannot be converted at all**, and this is the common case for From d219c5e5601724512ab003d510e415984db89d22 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 19:28:27 -0700 Subject: [PATCH 13/15] fix(assets): deforming geometry keeps its vertex order through the cook (PRD-485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `packages/assets/src/passes/model.ts:925` called the library's `reorder` on the whole document, which renumbers the vertices of every primitive it finds. That is the cheapest transmission-size win in the cook and it is invisible until something outside the file holds a vertex index: the metahuman-lab's `brows.skin.json` names the head vertex each brow strand root rides and reads its position back every frame, and after the cook it indexed a different vertex (`brow root 0 indexes head vertex 11498, which is not where it was prepared`). Measured on the 26.3 MB Ada specimen, 87 of 24408 head-primitive slots survived the reorder. The cook cannot know who holds an index, so the rule is the broader one: **any primitive whose vertex indices can be addressed from outside is undeclarable**, and deforming — skinned, morph-targeted, or joint-bound — is the measurable proxy, because it is the one case detectable from the file alone and a rig is where index-addressed data comes from. `deforming()` in `lod/eligibility.ts` is now the single definition of that word, shared with the discrete-LOD and joined-rung rules that already refused the same geometry. `reorder-static.ts` runs the remap per shared index buffer and skips any group holding a deforming primitive. Compression is untouched: `EXT_meshopt_compression` encodes whatever order it is handed, and `quantize` renumbers nothing — `specimen.glb` cooks 16,104,464 -> 16,114,804 B (+0.06%), the price of leaving the head's 24408 vertices where the sidecar expects them. `MODEL_REORDER_VERSION` joins the pass configuration, or a warm compile entry would keep serving the renumbered files this replaces. Red/green in `packages/assets/__tests__/model-pass.spec.ts`: a skinned primitive and a morph-targeted one come out of the cook with position k exactly where it started and the same vertex count, while a static mesh in the same document is still reordered (and still only reordered). `packages/assets/__tests__` 481 passed, `pnpm typecheck` and `pnpm lint` clean. --- packages/assets/__tests__/model-pass.spec.ts | 90 ++++++++++++ packages/assets/src/lod/eligibility.ts | 14 ++ packages/assets/src/passes/model.ts | 16 ++- packages/assets/src/passes/reorder-static.ts | 136 +++++++++++++++++++ 4 files changed, 254 insertions(+), 2 deletions(-) create mode 100644 packages/assets/src/passes/reorder-static.ts diff --git a/packages/assets/__tests__/model-pass.spec.ts b/packages/assets/__tests__/model-pass.spec.ts index 212e0969dc..a97bcfad86 100644 --- a/packages/assets/__tests__/model-pass.spec.ts +++ b/packages/assets/__tests__/model-pass.spec.ts @@ -151,6 +151,96 @@ describe("modelPass", () => { } }); + it("should keep a deforming primitive's vertex order and still reorder a static one", async () => { + // A vertex index is an address, and something outside this cook may already hold one: a + // MetaHuman sidecar records the head vertex each brow strand root rides, and reads the position + // back at that index every frame. The cook cannot know who holds an index, so deforming + // geometry — skinned, morph-targeted, or carrying joint data — keeps its exact vertex order and + // count, and everything else is still reordered for transmission size. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const character = root.listMeshes()[0]; + const cloth = character?.listPrimitives()[0]; + const head = character?.listPrimitives()[1]; + if (cloth === undefined || head === undefined) throw new Error("Fixture lost a primitive."); + // Both reasons in one cook: the cloth stays skinned-only, the head also gains a morph target. + const buffer = head.getAttribute("POSITION")?.getBuffer() ?? null; + head.addTarget( + document.createPrimitiveTarget("head-jaw-open").setAttribute( + "POSITION", + document + .createAccessor("head-jaw-open-deltas") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 18 }, (_, index) => (index % 3) * 0.01))), + ), + ); + // A static mesh whose index buffer is not already in draw order, so the reorder has real work + // to do on it: the jumbled order below is a non-identity meshoptimizer remap. + const staticIndices = [6, 1, 8, 0, 7, 3, 2, 4, 5]; + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray(Uint32Array.from(staticIndices)), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 27 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + + const source = (meshName: string, primitive: number): number[] => [ + ...(root + .listMeshes() + .find((mesh) => mesh.getName() === meshName) + ?.listPrimitives() + [primitive]?.getAttribute("POSITION") + ?.getArray() ?? []), + ]; + const before = { + cloth: source("character", 0), + head: source("character", 1), + prop: source("prop", 0), + }; + // Lossless options, so the only stage that can move a vertex is `reorder` itself. + const result = await modelPass({ + compact: false, + passes: { quantize: false }, + textures: "none", + virtual: "none", + }).apply(Buffer.from(await new NodeIO().writeBinary(document)), "character.glb"); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + const output = await readVerified(result.buffer); + const cooked = (meshName: string, primitive: number): number[] => [ + ...(output + .listMeshes() + .find((mesh) => mesh.getName() === meshName) + ?.listPrimitives() + [primitive]?.getAttribute("POSITION") + ?.getArray() ?? []), + ]; + + expect(cooked("character", 0)).toEqual(before.cloth); + expect(cooked("character", 1)).toEqual(before.head); + // And the static half of the rule: reordered, and nothing but reordered. + expect(cooked("prop", 0)).not.toEqual(before.prop); + expect([...cooked("prop", 0)].sort((a, b) => a - b)).toEqual( + [...before.prop].sort((a, b) => a - b), + ); + }); + it("decodes normalized ushort joint weights by their own component type in reachableStats", async () => { // External exporters ship WEIGHTS_0 as normalized UNSIGNED_SHORT. reachableStats must // decode them /65535 like the GPU does; the hardcoded ubyte scale (/255) reads raw diff --git a/packages/assets/src/lod/eligibility.ts b/packages/assets/src/lod/eligibility.ts index f3790f9cf3..88227a8cf9 100644 --- a/packages/assets/src/lod/eligibility.ts +++ b/packages/assets/src/lod/eligibility.ts @@ -55,6 +55,20 @@ function sharedByLevels(semantic: string): boolean { const DEFORMING_ATTRIBUTES: ReadonlySet = new Set(["JOINTS_0", "WEIGHTS_0"]); +/** + * Whether a primitive is deforming: skinned, morph-targeted, or carrying per-vertex joint data. + * + * The one definition of the word, shared by every rule that declines to rewrite geometry a rig + * drives — no discrete chain, no joined far rung, and no vertex reordering in the cook. A vertex + * index into deforming geometry is an address something outside the file may already hold, and a + * rig is where index-addressed data is most likely to come from: a MetaHuman sidecar records the + * head vertex each brow strand root rides and reads its position back every frame. + */ +export function deforming(primitive: Primitive, skinned: boolean): boolean { + if (skinned || primitive.listTargets().length > 0) return true; + return primitive.listSemantics().some((semantic) => DEFORMING_ATTRIBUTES.has(semantic)); +} + const TRIANGLES_MODE = 4; const TRIANGLE = 3; diff --git a/packages/assets/src/passes/model.ts b/packages/assets/src/passes/model.ts index d9ebd4fba4..8f4d5dc0ed 100644 --- a/packages/assets/src/passes/model.ts +++ b/packages/assets/src/passes/model.ts @@ -18,7 +18,6 @@ import { listTextureSlots, prune, quantize, - reorder, simplify, } from "@gltf-transform/functions"; import { MeshoptEncoder, MeshoptSimplifier } from "meshoptimizer"; @@ -71,6 +70,7 @@ import { textureBindings, textureKeys, } from "./model-textures.js"; +import { reorderStaticPrimitives } from "./reorder-static.js"; import { type ISharedImage, type ISharedImageStore, @@ -243,6 +243,12 @@ const MATERIAL_MERGE_VERSION = 1; * which the compile digest already hashes, so it invalidates model outputs and nothing else. */ const MODEL_DEDUPE_VERSION = 1; +/** + * Bumped when `reorder-static.ts` changes what it emits: every model cooked before this fix had its + * deforming geometry renumbered, and a warm cache entry keyed on the pass switches alone would keep + * serving those files. 1 = deforming primitives keep their authored vertex order and count. + */ +const MODEL_REORDER_VERSION = 1; /** Relative bounding-box tolerance of the self-verify check (PRD: 0.1%). */ const BBOX_TOLERANCE = 0.001; @@ -803,6 +809,9 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { // content-identical sources, so a bake published before this key existed named one output // per copy and must not be re-served as if it had deduped. dedupe: MODEL_DEDUPE_VERSION, + // Same reason: which primitives `reorder` declines to renumber is part of the output, and a + // warm entry from before it declined would hand a game geometry its sidecars cannot address. + reorderVersion: MODEL_REORDER_VERSION, // `"none"` and "absent" are different cache keys on purpose: absent bakes with defaults. virtual: options.virtual === "none" @@ -922,7 +931,10 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { })(document); } if (enabled.reorder || enabled.meshopt) await MeshoptEncoder.ready; - if (enabled.reorder) await reorder({ encoder: MeshoptEncoder })(document); + // The last stage that moves a vertex, and the only one that does so selectively: deforming + // geometry keeps its authored vertex order because a vertex index into it is an address + // something outside this file may already hold. `reorder-static.ts`. + if (enabled.reorder) reorderStaticPrimitives(document, MeshoptEncoder); // After `reorder`, which is the last stage that moves a vertex, and before `quantize`, which // changes what a position is but never which vertex it is. const virtual = diff --git a/packages/assets/src/passes/reorder-static.ts b/packages/assets/src/passes/reorder-static.ts new file mode 100644 index 0000000000..7e4551dca1 --- /dev/null +++ b/packages/assets/src/passes/reorder-static.ts @@ -0,0 +1,136 @@ +// The model pass's `reorder` stage, over the primitives whose vertex order nothing else owns. +// +// `reorder` from `@gltf-transform/functions` walks the whole document and renumbers the vertices +// of every primitive it finds, which is the cheapest transmission-size win in the cook — and it +// breaks every file that addresses a vertex by index. A MetaHuman sidecar names the head vertex +// each brow strand root rides and reads that position back every frame; after an unconditional +// reorder it indexes a different vertex (`brow root 0 indexes head vertex 11498, which is not where +// it was prepared`). The cook cannot know who holds an index, so the general rule is that any +// primitive whose indices can be addressed from outside is undeclarable, and *deforming* — skinned, +// morph-targeted, or joint-bound — is the measurable proxy: a rig is where index-addressed data +// comes from, and it is the one case detectable from the file alone. +// +// So the vertex-cache remap is applied per index-buffer group, and a group holding any deforming +// primitive is left exactly as authored. Compression is unaffected: `EXT_meshopt_compression` encodes +// whatever order it is handed, and `quantize` renumbers nothing. +// +// ponytail: the library's `reorder` has no per-primitive switch, so the remap is its own ~30 lines. +// Replace with the library's whole-document `reorder` the day it takes a predicate. + +import { type Accessor, type Document, type Mesh, Primitive } from "@gltf-transform/core"; +import type { MeshoptEncoder } from "meshoptimizer"; +import { deforming } from "../lod/eligibility.js"; + +/** One index buffer, the attributes that read through it, and the primitives that draw it. */ +interface IVertexGroup { + readonly attributes: Accessor[]; + readonly members: { readonly primitive: Primitive; readonly skinned: boolean }[]; +} + +/** meshoptimizer's own 16-bit index ceiling, as the library's `reorder` measures it. */ +const MAX_UINT16 = 65534; + +/** + * Reorder every non-deforming primitive's vertices for transmission size, in place. Returns how + * many index-buffer groups were rewritten. + * + * Grouped by shared index buffer rather than by primitive, because primitives sharing one index + * buffer share the renumbering: remapping one and not the other desynchronises them. Mirrors the + * library's own `reorder` for the groups it does touch, minus its cleanup prune — the pass's `prune` + * runs before this stage, and the writer emits reachable properties only. + */ +export function reorderStaticPrimitives( + document: Document, + encoder: typeof MeshoptEncoder, +): number { + let reordered = 0; + for (const [indices, group] of vertexGroups(document)) { + if (group.members.some((member) => deforming(member.primitive, member.skinned))) continue; + remapGroup(indices, group, encoder); + reordered += 1; + } + return reordered; +} + +/** Every index buffer in the document, with the attributes and primitives that read through it. */ +function vertexGroups(document: Document): Map { + const skinned = skinnedMeshes(document); + const groups = new Map(); + for (const mesh of document.getRoot().listMeshes()) + for (const primitive of mesh.listPrimitives()) { + const indices = primitive.getIndices(); + if (indices === null) continue; + const group = groups.get(indices) ?? { attributes: [], members: [] }; + group.members.push({ primitive, skinned: skinned.has(mesh) }); + for (const semantic of primitive.listSemantics()) { + const attribute = primitive.getAttribute(semantic); + if (attribute !== null && !group.attributes.includes(attribute)) + group.attributes.push(attribute); + } + groups.set(indices, group); + } + return groups; +} + +/** The meshes a skin is bound to: everything under one of those nodes is deforming. */ +function skinnedMeshes(document: Document): Set { + const skinned = new Set(); + for (const node of document.getRoot().listNodes()) { + const mesh = node.getMesh(); + if (mesh !== null && node.getSkin() !== null) skinned.add(mesh); + } + return skinned; +} + +/** Renumber one group's vertices, and every primitive that draws them, in place. */ +function remapGroup(indices: Accessor, group: IVertexGroup, encoder: typeof MeshoptEncoder): void { + const source = indices.getArray(); + const triangles = group.members.every( + (member) => member.primitive.getMode() === Primitive.Mode.TRIANGLES, + ); + // The encoder renumbers the array it is handed in place and returns the old-to-new table, so the + // copy is the destination index buffer and `source` still reads the authored one. + const remapped = source instanceof Uint32Array ? source.slice() : new Uint32Array(source); + const [remap, unique] = encoder.reorderMesh(remapped, triangles, true); + const dstIndices = indices + .clone() + .setArray(unique <= MAX_UINT16 ? new Uint16Array(remapped) : remapped); + const dstAttributes = new Map(); + for (const attribute of group.attributes) + dstAttributes.set(attribute, compactAttribute(attribute, source, remap, unique)); + for (const member of group.members) { + for (const semantic of member.primitive.listSemantics()) { + const attribute = member.primitive.getAttribute(semantic); + if (attribute === null) continue; + member.primitive.setAttribute(semantic, dstAttributes.get(attribute) ?? attribute); + } + member.primitive.setIndices(dstIndices); + } + for (const attribute of group.attributes) + if (attribute.listParents().length === 0) attribute.dispose(); + if (indices.listParents().length === 0) indices.dispose(); +} + +/** One attribute rewritten into the renumbered vertex stream, dropping what no triangle reaches. */ +function compactAttribute( + attribute: Accessor, + source: ArrayLike, + remap: Uint32Array, + unique: number, +): Accessor { + const array = attribute.getArray(); + const width = attribute.getElementSize(); + const compact = new array.constructor(unique * width); + // One write per destination vertex: a vertex several triangles reference is copied once. + const written = new Uint8Array(unique); + for (let index = 0; index < source.length; index += 1) { + const from = source[index] ?? 0; + const to = remap[from]; + if (to === undefined || written[to] === 1) continue; + written[to] = 1; + for (let component = 0; component < width; component += 1) + compact[to * width + component] = array[from * width + (component ?? 0)]; + } + // Sparse is cleared: it indexes the source's vertex stream and this one has been renumbered. + return attribute.clone().setArray(compact).setSparse(false); +} From f27cb22997df60bef84a9603a660d1c670ee638e Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 20:20:43 -0700 Subject: [PATCH 14/15] fix(assets): deforming geometry keeps its positions in source metres through the cook (PRD-485) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `packages/assets/src/passes/model.ts:964` handed the whole document to the library's `quantize`, which rewrites POSITION into normalized int16 snapped to a grid whose step is the mesh's own bounding volume and then folds that shift into the node holding the mesh, the skin's inverse bind matrices, GPU instancing batches and any volumetric material. Exact for a static mesh; for a rigged one it breaks the invariant this repo holds, one metre is one metre. The metahuman-lab's `brows.skin.json` names the head vertex each brow strand root rides and reads its position back every frame, and the sidecar was prepared against the source file to 1e-5 m — the 16-bit grid over a head-sized volume is micrometres away from every vertex it recorded, and the shift that would undo it lives in a node transform the sidecar never sees. So the same `deforming()` rule that stopped the cook renumbering a rigged vertex now stops it quantizing one: a deforming mesh keeps POSITION and its morph-target POSITION deltas as the float32 values the author wrote, at every requested depth. `quantize-static.ts` holds those meshes across the stage and spends the window with each mesh detached from its nodes, so the library's node, skin, instancing and thickness compensation is never applied and there is nothing to undo; normals, UVs, colours and every static primitive still quantize, and `EXT_meshopt_compression` still encodes whatever the accessors hold (its `QUANTIZE` encoder method only names the pre-processing, which is this stage). The unit of source space is the mesh, because the compensation is per mesh — one deforming primitive holds the whole thing. An index buffer the stage replaced fails the cook by name rather than shipping a mesh whose vertices have quietly moved, and `snapPositions` skips rigged meshes for the same reason it exists to snap them. `MODEL_QUANTIZE_VERSION` joins the pass configuration so a warm compile entry cannot keep serving the quantized files this replaces. Red/green in `packages/assets/__tests__/model-pass.spec.ts`: a skinned primitive that also carries a morph target comes out of the cook with float32 POSITION equal to the source within 1e-6, float32 deltas, and its skin's inverse bind matrices untouched, while a static mesh in the same document still quantizes to normalized int16 under KHR_mesh_quantization. The 4-bit floor test now states what is true of it: that request used to land on the fixture's skinned head and trip the self-verify at 0.167% drift; it no longer reaches deforming geometry at all, and the fail-closed comparison is covered directly by the drift-kind test. `packages/assets/ __tests__` 482 passed | 2 skipped, `pnpm typecheck` clean, biome clean on the four files. --- packages/assets/__tests__/model-pass.spec.ts | 183 +++++++++++++++++- packages/assets/src/lod/eligibility.ts | 25 ++- packages/assets/src/passes/model.ts | 43 ++-- packages/assets/src/passes/quantize-static.ts | 160 +++++++++++++++ packages/assets/src/passes/reorder-static.ts | 14 +- 5 files changed, 393 insertions(+), 32 deletions(-) create mode 100644 packages/assets/src/passes/quantize-static.ts diff --git a/packages/assets/__tests__/model-pass.spec.ts b/packages/assets/__tests__/model-pass.spec.ts index a97bcfad86..deda4921ea 100644 --- a/packages/assets/__tests__/model-pass.spec.ts +++ b/packages/assets/__tests__/model-pass.spec.ts @@ -89,14 +89,68 @@ describe("modelPass", () => { } }); - it("should throw naming bounding-box drift when positions quantize to 4 bits", async () => { - const input = Buffer.from(await buildFixtureGlb()); - await expect( - modelPass({ quantize: { normalBits: 16, positionBits: 4, uvBits: 12 } }).apply( - input, - "character.glb", - ), - ).rejects.toThrow(/TN_ASSETS_MODEL_DRIFT.*bounding box drifted/u); + it("should leave deforming positions alone at a depth below the quantization floor", async () => { + // Four bits is the destructive pre-round the library refuses to express: the pass snaps floats + // onto a 15-level grid over each accessor's own bounds and lets the self-verify reject the + // result. That request used to land on the fixture's skinned head — and used to trip the + // self-verify at 0.167% drift. Deforming geometry is now held in source space at every depth, + // so the request never reaches it and there is nothing left to reject; the fail-closed path + // itself is covered by the drift-kind comparison further down. The static half of the document + // still quantizes, which is what keeps the two halves of the rule honest. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const head = root.listMeshes()[0]?.listPrimitives()[1]; + const buffer = head?.getAttribute("POSITION")?.getBuffer() ?? null; + if (head === undefined || buffer === null) throw new Error("Fixture lost its rig."); + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray( + Uint32Array.from([ + 0, 10, 20, 1, 11, 21, 2, 12, 22, 3, 13, 23, 4, 14, 24, 5, 15, 25, 6, 16, 26, 7, 17, 8, + 18, 19, 9, + ]), + ), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + .setArray(new Float32Array(Array.from({ length: 81 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + const positions = [...(head.getAttribute("POSITION")?.getArray() ?? [])]; + const result = await modelPass({ + compact: false, + quantize: { normalBits: 16, positionBits: 4, uvBits: 12 }, + textures: "none", + virtual: "none", + }).apply(Buffer.from(await new NodeIO().writeBinary(document)), "character.glb"); + expect(Buffer.isBuffer(result)).toBe(false); + if (Buffer.isBuffer(result)) return; + const output = await readVerified(result.buffer); + expect([ + ...(output.listMeshes()[0]?.listPrimitives()[1]?.getAttribute("POSITION")?.getArray() ?? []), + ]).toEqual(positions); + expect( + output + .listMeshes() + .find((mesh) => mesh.getName() === "prop") + ?.listPrimitives()[0] + ?.getAttribute("POSITION") + ?.getComponentSize(), + // Eight bits is the floor the request resolves to, and it lands as normalized Int8. + ).toBe(1); }); it("should declare EXT_meshopt_compression and not Draco", async () => { @@ -241,6 +295,119 @@ describe("modelPass", () => { ); }); + it("should keep a deforming primitive's positions in source metres and still quantize a static one", async () => { + // One metre is one metre. `quantize` snaps POSITION onto a grid whose step is the mesh's own + // volume over 2^bits, so a head cooked at 16 bits over half a metre lands within ~8 µm of + // where the author put it — and the metahuman-lab sidecar, which names the head vertex each + // brow strand root rides and reads that position back every frame, was prepared to 1e-5 m. + // Quantization also compensates by folding a shift into the node, the skin's inverse bind + // matrices and any volumetric material, so the whole mesh stays in source space instead: + // deforming POSITION and its morph-target deltas keep float32, everything else still quantizes. + const document = buildFixtureDocument({ textured: false }); + const root = document.getRoot(); + const character = root.listMeshes().find((mesh) => mesh.getName() === "character"); + const cloth = character?.listPrimitives()[0]; + const head = character?.listPrimitives()[1]; + const skin = root.listSkins()[0]; + const buffer = head?.getAttribute("POSITION")?.getBuffer() ?? null; + if (cloth === undefined || head === undefined || skin === undefined || buffer === null) + throw new Error("Fixture lost its rig."); + // Both reasons in one cook, as in the vertex-order twin: the head is skinned and morph-targeted. + const deltas = new Float32Array(Array.from({ length: 18 }, (_, index) => (index % 3) * 0.004)); + head.addTarget( + document + .createPrimitiveTarget("jaw-open") + .setAttribute( + "POSITION", + document + .createAccessor("jaw-open-deltas") + .setBuffer(buffer) + .setType("VEC3") + .setArray(deltas), + ), + ); + // A static mesh in the same document, on its own node: the half of the rule that must not regress. + const prop = document + .createPrimitive() + .setIndices( + document + .createAccessor("prop-indices") + .setBuffer(buffer) + .setType("SCALAR") + .setArray( + // Every vertex referenced exactly once, so the library's own vertex compaction inside + // `quantize` has nothing to drop and the bbox the self-verify reads is the authored one. + Uint32Array.from([ + 0, 10, 20, 1, 11, 21, 2, 12, 22, 3, 13, 23, 4, 14, 24, 5, 15, 25, 6, 16, 26, 7, 17, 8, + 18, 19, 9, + ]), + ), + ) + .setAttribute( + "POSITION", + document + .createAccessor("prop-positions") + .setBuffer(buffer) + .setType("VEC3") + // 27 vertices, every one of them referenced by the indices above. + .setArray(new Float32Array(Array.from({ length: 81 }, (_, index) => index))), + ); + root + .listScenes()[0] + ?.addChild( + document.createNode("prop-root").setMesh(document.createMesh("prop").addPrimitive(prop)), + ); + + const positions = [...(head.getAttribute("POSITION")?.getArray() ?? [])]; + const binds = [...(skin.getInverseBindMatrices()?.getArray() ?? [])]; + const result = await modelPass({ compact: false, textures: "none", virtual: "none" }).apply( + Buffer.from(await new NodeIO().writeBinary(document)), + "character.glb", + ); + if (Buffer.isBuffer(result)) throw new Error("model pass returned an unchanged buffer"); + const output = await readVerified(result.buffer); + const cookedHead = output + .listMeshes() + .find((mesh) => mesh.getName() === "character") + ?.listPrimitives()[1]; + const cookedCloth = output + .listMeshes() + .find((mesh) => mesh.getName() === "character") + ?.listPrimitives()[0]; + const cookedProp = output + .listMeshes() + .find((mesh) => mesh.getName() === "prop") + ?.listPrimitives()[0]; + + const cookedPositions = cookedHead?.getAttribute("POSITION"); + expect(cookedPositions?.getComponentType()).toBe(5126); + const after = [...(cookedPositions?.getArray() ?? [])]; + expect(after.length).toBe(positions.length); + for (const [index, value] of after.entries()) + expect(Math.abs((value as number) - (positions[index] as number))).toBeLessThan(1e-6); + // The deltas ride the same space, so they keep their float32 metres too. + const cookedDeltas = cookedHead?.listTargets()[0]?.getAttribute("POSITION"); + expect(cookedDeltas?.getComponentType()).toBe(5126); + expect([...(cookedDeltas?.getArray() ?? [])]).toEqual([...deltas]); + // Nothing moved to compensate: the bind poses the deformation is measured against are the + // authored ones, and the normals that do not address a vertex still quantized. + const cookedBinds = [...(output.listSkins()[0]?.getInverseBindMatrices()?.getArray() ?? [])]; + expect(cookedBinds.length).toBe(binds.length); + for (const [index, value] of cookedBinds.entries()) + expect(Math.abs((value as number) - (binds[index] as number))).toBeLessThan(1e-6); + expect(cookedCloth?.getAttribute("POSITION")?.getComponentType()).toBe(5126); + expect(cookedCloth?.getAttribute("NORMAL")?.getComponentSize()).toBeLessThan(4); + // And the static half of the rule, unchanged: quantized, normalized, KHR_mesh_quantization on. + expect(cookedProp?.getAttribute("POSITION")?.getComponentSize()).toBe(2); + expect(cookedProp?.getAttribute("POSITION")?.getNormalized()).toBe(true); + expect( + output + .listExtensionsUsed() + .map((extension) => extension.extensionName) + .includes("KHR_mesh_quantization"), + ).toBe(true); + }); + it("decodes normalized ushort joint weights by their own component type in reachableStats", async () => { // External exporters ship WEIGHTS_0 as normalized UNSIGNED_SHORT. reachableStats must // decode them /65535 like the GPU does; the hardcoded ubyte scale (/255) reads raw diff --git a/packages/assets/src/lod/eligibility.ts b/packages/assets/src/lod/eligibility.ts index 88227a8cf9..3cbf54077d 100644 --- a/packages/assets/src/lod/eligibility.ts +++ b/packages/assets/src/lod/eligibility.ts @@ -4,7 +4,13 @@ // structurally intact is declined with a named reason rather than forced to a triangle target. An // inability to reduce safely is a normal skip, not a failure. -import type { Material, Primitive } from "@gltf-transform/core"; +import { + type Document, + type Material, + type Mesh, + Node, + type Primitive, +} from "@gltf-transform/core"; /** The stable reason codes reported for a primitive that gets no automatic discrete chain. */ export type DiscreteLodSkipReason = @@ -69,6 +75,23 @@ export function deforming(primitive: Primitive, skinned: boolean): boolean { return primitive.listSemantics().some((semantic) => DEFORMING_ATTRIBUTES.has(semantic)); } +/** The mesh-level half of {@link deforming}: the node compensation quantization applies is per mesh, + * so one deforming primitive is enough to hold every position the mesh carries. */ +export function deformingMesh(mesh: Mesh, skinned: ReadonlySet): boolean { + const isSkinned = skinned.has(mesh); + return isSkinned || mesh.listPrimitives().some((primitive) => deforming(primitive, isSkinned)); +} + +/** The meshes a skin is bound to, shared by every rule that asks whether a rig drives a mesh. */ +export function skinnedMeshes(document: Document): Set { + const skinned = new Set(); + for (const node of document.getRoot().listNodes()) { + const mesh = node.getMesh(); + if (mesh !== null && node.getSkin() !== null) skinned.add(mesh); + } + return skinned; +} + const TRIANGLES_MODE = 4; const TRIANGLE = 3; diff --git a/packages/assets/src/passes/model.ts b/packages/assets/src/passes/model.ts index 8f4d5dc0ed..15bae8175b 100644 --- a/packages/assets/src/passes/model.ts +++ b/packages/assets/src/passes/model.ts @@ -4,6 +4,7 @@ import { Accessor, type Document, type GLTF, + type Mesh as GltfMesh, type Node as GltfNode, type Primitive as GltfPrimitive, NodeIO, @@ -30,6 +31,7 @@ import { import { type IMaterialMergeSummary, mergeIdenticalMaterials } from "../foliage.js"; import { createGltfReader, readGltfDocument } from "../gltf-io.js"; import { KTX2_ENCODER_VERSION } from "../ktx2-encoder.js"; +import { deformingMesh, skinnedMeshes } from "../lod/eligibility.js"; import { TNDiscreteLod } from "../lod/extension.js"; import { type IModelLodOptions, @@ -70,6 +72,7 @@ import { textureBindings, textureKeys, } from "./model-textures.js"; +import { quantizeStaticGeometry } from "./quantize-static.js"; import { reorderStaticPrimitives } from "./reorder-static.js"; import { type ISharedImage, @@ -249,6 +252,13 @@ const MODEL_DEDUPE_VERSION = 1; * serving those files. 1 = deforming primitives keep their authored vertex order and count. */ const MODEL_REORDER_VERSION = 1; +/** + * Bumped when `quantize-static.ts` changes what it emits: every model cooked before this fix had its + * deforming positions snapped onto a quantization grid, and a warm cache entry keyed on the pass + * switches alone would keep serving those files. 1 = deforming POSITION and morph-target deltas + * stay float32 in the source metre space. + */ +const MODEL_QUANTIZE_VERSION = 1; /** Relative bounding-box tolerance of the self-verify check (PRD: 0.1%). */ const BBOX_TOLERANCE = 0.001; @@ -710,11 +720,14 @@ export function assertSimplifiedWithinBounds( * Snaps every scene-reachable POSITION accessor onto a uniform `2^bits` grid spanning its * own bounds — the destructive low-precision quantization the underlying library refuses * to express. Used only for configured depths below its 8-bit floor; the self-verify is - * expected to reject the result. + * expected to reject the result. Deforming meshes are skipped for the same reason the + * quantization stage skips them: a grid that coarse moves vertices a sidecar already + * addresses, and holding them is the rule rather than the bit depth. */ -function snapPositions(root: RootOf, bits: number): void { +function snapPositions(root: RootOf, bits: number, skinned: ReadonlySet): void { const levels = 2 ** bits - 1; for (const mesh of root.listMeshes()) { + if (deformingMesh(mesh, skinned)) continue; for (const primitive of mesh.listPrimitives()) { const position = primitive.getAttribute("POSITION"); if (position === null || position.getNormalized()) continue; @@ -812,6 +825,9 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { // Same reason: which primitives `reorder` declines to renumber is part of the output, and a // warm entry from before it declined would hand a game geometry its sidecars cannot address. reorderVersion: MODEL_REORDER_VERSION, + // Same reason again: which positions keep their source metres is part of the output, and a + // warm entry from before they did would hand a game geometry its sidecars cannot address. + quantizeVersion: MODEL_QUANTIZE_VERSION, // `"none"` and "absent" are different cache keys on purpose: absent bakes with defaults. virtual: options.virtual === "none" @@ -957,18 +973,23 @@ export function modelPass(options: IModelPassOptions = {}): IAssetPass { // Depths below the library's 8-bit floor are honoured by pre-rounding the floats // onto the coarser grid first; the self-verify then fails the build on the drift. const requestedBits = options.quantize?.positionBits ?? DEFAULT_POSITION_BITS; - if (requestedBits < 8) snapPositions(document.getRoot(), requestedBits); + if (requestedBits < 8) + snapPositions(document.getRoot(), requestedBits, skinnedMeshes(document)); // Joint indices and weights are excluded outright: quantizing them below the // exporter's declaration is where skinned meshes visibly break, and the library's // own default narrows weights to 8 bits unless told otherwise. - await quantize({ - quantizeColor: 8, - quantizeNormal: options.quantize?.normalBits ?? DEFAULT_NORMAL_BITS, - quantizePosition: Math.max(requestedBits, 8), - quantizeTexcoord: options.quantize?.uvBits ?? DEFAULT_UV_BITS, - pattern: /^(?!JOINTS|WEIGHTS)/u, - normalizeWeights: false, - })(document); + // Deforming geometry keeps float32 POSITION in the source metre space on top of + // that, so nothing a sidecar addresses moves: `quantize-static.ts`. + await quantizeStaticGeometry(document, () => + quantize({ + quantizeColor: 8, + quantizeNormal: options.quantize?.normalBits ?? DEFAULT_NORMAL_BITS, + quantizePosition: Math.max(requestedBits, 8), + quantizeTexcoord: options.quantize?.uvBits ?? DEFAULT_UV_BITS, + pattern: /^(?!JOINTS|WEIGHTS)/u, + normalizeWeights: false, + })(document), + ); } // Taken after every geometry stage and before compression, so what it proves is that // *this* stage plus the writer preserved each binding — prune's documented drops of diff --git a/packages/assets/src/passes/quantize-static.ts b/packages/assets/src/passes/quantize-static.ts new file mode 100644 index 0000000000..487454044d --- /dev/null +++ b/packages/assets/src/passes/quantize-static.ts @@ -0,0 +1,160 @@ +// The model pass's `quantize` stage, over the primitives whose positions nothing else may move. +// +// `quantize` from `@gltf-transform/functions` walks the whole document and rewrites POSITION into +// 16-bit integers snapped to a grid whose step is the mesh's own bounding volume — the largest size +// win in the cook — and then compensates by folding that shift into the node holding the mesh, the +// skin's inverse bind matrices, GPU instancing batches and any volumetric material. +// +// That is exact for static geometry. For geometry a rig drives it breaks the invariant this repo +// holds: one metre is one metre. The metahuman-lab's `brows.skin.json` names the head vertex each +// brow strand root rides and reads its position back every frame, prepared against the source file +// to within 1e-5 m; a head snapped onto a 16-bit grid over its own volume is micrometres away from +// every vertex the sidecar recorded, and the shift that would undo it lives in a node transform the +// sidecar never sees. +// +// So a deforming mesh is held in source space across the stage: POSITION and the morph-target +// deltas riding it keep the exact values their author wrote, and the mesh spends the stage detached +// from its nodes, so there is no compensation to undo — the skin, the node matrices, the instancing +// batch and a volume material's thickness are simply never touched. Everything else still quantizes, +// and `EXT_meshopt_compression` still encodes whatever the accessors hold. +// +// ponytail: the library's `quantize` takes one document-wide pattern with no per-mesh switch, and +// its node compensation is inseparable from the positions it shifts, so holding is its own ~60 +// lines. Replace with the library's own `quantize` the day it takes a deforming predicate. + +import type { + Accessor, + Document, + ExtensionProperty, + Material, + Mesh, + Primitive, + PrimitiveTarget, + TypedArray, +} from "@gltf-transform/core"; +import { Node } from "@gltf-transform/core"; +import { deformingMesh, skinnedMeshes } from "../lod/eligibility.js"; + +/** What a primitive and a morph target both expose for one attribute, so one call serves both. */ +type IAttributeOwner = { + getAttribute(semantic: string): Accessor | null; + setAttribute(semantic: string, accessor: Accessor | null): unknown; +}; + +/** One attribute's authored values, copied out before the stage overwrites them. */ +interface IHeldAttribute { + readonly array: TypedArray; + readonly normalized: boolean; +} + +/** One primitive's authored positions, kept while the stage tries to rewrite them. */ +interface IHeldPrimitive { + readonly primitive: Primitive; + /** The index buffer as authored: a remap here would renumber what a sidecar addresses. */ + readonly indices: Accessor | null; + readonly position: IHeldAttribute | null; + readonly targets: readonly { + readonly target: PrimitiveTarget; + readonly position: IHeldAttribute | null; + }[]; +} + +/** One deforming mesh: how it was held, and how to put it back. */ +interface IHeldMesh { + readonly mesh: Mesh; + readonly nodes: readonly Node[]; + readonly primitives: readonly IHeldPrimitive[]; + readonly volumes: readonly { readonly material: Material; readonly volume: ExtensionProperty }[]; +} + +/** + * Runs `quantize` with every deforming mesh held in the metre space its author wrote it in, and + * puts the held geometry back whether the stage returns or throws. + */ +export async function quantizeStaticGeometry( + document: Document, + quantize: () => unknown, +): Promise { + const held = holdDeformingMeshes(document); + try { + await quantize(); + } finally { + releaseDeformingMeshes(held); + } +} + +function holdDeformingMeshes(document: Document): IHeldMesh[] { + const skinned = skinnedMeshes(document); + const held: IHeldMesh[] = []; + for (const mesh of document.getRoot().listMeshes()) { + // The mesh is the unit of source space, because the compensation is per mesh: one deforming + // primitive in it holds the whole thing, or the shift would land on half of it. + if (!deformingMesh(mesh, skinned)) continue; + const nodes: Node[] = []; + for (const parent of mesh.listParents()) { + if (!(parent instanceof Node)) continue; + nodes.push(parent); + parent.setMesh(null); + } + const primitives = mesh.listPrimitives().map((primitive) => ({ + primitive, + indices: primitive.getIndices(), + position: holdAttribute(primitive.getAttribute("POSITION")), + targets: primitive.listTargets().map((target) => ({ + target, + position: holdAttribute(target.getAttribute("POSITION")), + })), + })); + const volumes: { material: Material; volume: ExtensionProperty }[] = []; + for (const primitive of mesh.listPrimitives()) { + const material = primitive.getMaterial(); + const volume = material?.getExtension("KHR_materials_volume") ?? null; + if (material === null || volume === null) continue; + // Thickness is given in local units, so the stage would scale it by the shift it is not + // applying to this mesh's positions. + material.setExtension("KHR_materials_volume", null); + volumes.push({ material, volume }); + } + held.push({ mesh, nodes, primitives, volumes }); + } + return held; +} + +function releaseDeformingMeshes(held: readonly IHeldMesh[]): void { + for (const { mesh, nodes, primitives, volumes } of held) { + for (const node of nodes) node.setMesh(mesh); + for (const { material, volume } of volumes) + material.setExtension("KHR_materials_volume", volume); + for (const { primitive, indices, position, targets } of primitives) { + // The stage may rewrite what a sidecar already addresses, but never renumber it: an index + // buffer it replaced means the held positions no longer line up, so fail instead of shipping + // a mesh whose every vertex has quietly moved. + if (primitive.getIndices() !== indices) + throw new Error( + "TN_ASSETS_MODEL_DEFORMING_RENUMBERED: quantization rewrote the index buffer of a deforming primitive; its positions were held in source space and can no longer be addressed by index.", + ); + releaseAttribute(primitive, position); + for (const { target, position: delta } of targets) releaseAttribute(target, delta); + } + } +} + +function holdAttribute(accessor: Accessor | null): IHeldAttribute | null { + const array = accessor?.getArray() ?? null; + if (accessor === null || array === null) return null; + // A copy, because the stage replaces this accessor with its own and prunes the original: a + // disposed property cannot be re-attached, and the values have to outlive the stage. + const copy = array.slice(); + return { array: copy, normalized: accessor.getNormalized() }; +} + +/** + * Writes the authored values back over whatever the stage left in the attribute. The accessor the + * stage installed is reused rather than replaced, so the mesh keeps the attribute it was drawn + * with and no orphan is left for the writer to emit. + */ +function releaseAttribute(owner: IAttributeOwner, held: IHeldAttribute | null): void { + const accessor = owner.getAttribute("POSITION"); + if (held === null || accessor === null) return; + accessor.setArray(held.array).setNormalized(held.normalized).setSparse(false); +} diff --git a/packages/assets/src/passes/reorder-static.ts b/packages/assets/src/passes/reorder-static.ts index 7e4551dca1..d8c1ffffe3 100644 --- a/packages/assets/src/passes/reorder-static.ts +++ b/packages/assets/src/passes/reorder-static.ts @@ -17,9 +17,9 @@ // ponytail: the library's `reorder` has no per-primitive switch, so the remap is its own ~30 lines. // Replace with the library's whole-document `reorder` the day it takes a predicate. -import { type Accessor, type Document, type Mesh, Primitive } from "@gltf-transform/core"; +import { type Accessor, type Document, Primitive } from "@gltf-transform/core"; import type { MeshoptEncoder } from "meshoptimizer"; -import { deforming } from "../lod/eligibility.js"; +import { deforming, skinnedMeshes } from "../lod/eligibility.js"; /** One index buffer, the attributes that read through it, and the primitives that draw it. */ interface IVertexGroup { @@ -72,16 +72,6 @@ function vertexGroups(document: Document): Map { return groups; } -/** The meshes a skin is bound to: everything under one of those nodes is deforming. */ -function skinnedMeshes(document: Document): Set { - const skinned = new Set(); - for (const node of document.getRoot().listNodes()) { - const mesh = node.getMesh(); - if (mesh !== null && node.getSkin() !== null) skinned.add(mesh); - } - return skinned; -} - /** Renumber one group's vertices, and every primitive that draws them, in place. */ function remapGroup(indices: Accessor, group: IVertexGroup, encoder: typeof MeshoptEncoder): void { const source = indices.getArray(); From 7876e4b7bc0caf853650965ed3b0513deb877506 Mon Sep 17 00:00:00 2001 From: Joao Paulo Furtado Date: Sat, 3 Oct 2026 20:28:08 -0700 Subject: [PATCH 15/15] docs(PRD-485): metahuman-lab cooks and its face controls pass --- .../PRD-485-high-quality-assets-go-through-the-cook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md index 20d9722067..817e2d2d5c 100644 --- a/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md +++ b/docs/PRDs/open-world/PRD-485-high-quality-assets-go-through-the-cook.md @@ -63,6 +63,6 @@ Uncooked UE5 Megascans import as LOD0 only. The importer decompresses the larges **Status:** PARTIAL **Files:** `metahuman-lab` asset layout (sandbox); `packages/create-threenative/agent-docs/references/finding-assets.md` - [ ] [local] `fab-import-proof` packages its cooked KTX2 and Meshopt output for Android V8 and renders the Hornbeam on the emulator. proof: `native-playtests/fab-import-native.playtest.json --target android` on the local emulator. -- [ ] [local] `metahuman-lab` serves `content/` from `assets/` through the cook with no `assets` block, and `face-controls.playtest.json` passes on web. proof: `npx threenative build` plus that scenario with `--browser-recipe webgpu`. +- [x] [local] `metahuman-lab` serves `content/` from `assets/` through the cook with no `assets` block, and `face-controls.playtest.json` passes on web. proof: `npx threenative build` plus that scenario with `--browser-recipe webgpu`. **Result 2026-10-03:** sandbox branch `prd485/metahuman-lab-cook` (local): 301 MB `public/content/` → `assets/content/`, no `assets` block, `npx threenative build --target web` exit 0, uncooked 0 bytes; `.dna`/`.strands.bin` copied through by manifest, 30 PNGs → KTX2. Two engine fixes the cook exposed, each red-green: deforming geometry keeps its vertex order (`d219c5e56`, the library `reorder` renumbered 24,321 of 24,408 head vertices the DNA/brow sidecars address) and its float32 metre positions (`f27cb2299`, quantization had moved the scale into the inverse bind matrices); `specimen.glb` 26.3 MB → 22.5 MB with all 821 morph targets. `playtests/face-controls.playtest.json` passes 15/15 on WebGPU, 0 console errors; the frame shows jaw/smile/blink/brow controls driving the face with hair and skin intact. Build leg 2026-10-03: sandbox branch `prd485/metahuman-lab-cook` (local worktree, engine tarballs from this branch). `npx threenative build --target web` exit 0 in 189 s, `threenative.config.ts` still carries no `assets` block. The 301 MB moved from `public/content/` to `assets/content/`, and the hand-written vite middleware that served it verbatim (`specimenContentPlugin`) is gone: `ctx.assets.resolve` reads `public/assets.manifest.json` and every load asks for the cooked name. `.dna` and `.strands.bin` are copied through as `kind: "other"` byte for byte (`head.dna` 4,829,103 B, `hair.strands.bin` 85,202,464 B) — an extension no pass owns gets a manifest entry like any other file. `specimen.glb` cooks 26,267,896 -> 16,104,464 B (-38.7%): 5 embedded PNGs 7,801,033 -> 2,957,889 B (3 uastc/etc1s, `T_MH_BentNormal_AO` and `T_Eye_N` kept under the existing `not-smaller` guard), 101,518 triangles and all 821 LOD0 morph targets intact, `EXT_meshopt_compression` + `KHR_mesh_quantization` + `KHR_texture_basisu` on the output, and `lod.reasons ["deforming", "authored-lod"]` — a skinned morph-target head is PRD-486's subject, not this one. `shirt.glb` 50,367,028 -> 543,040 B (-98.9%), `body.glb` 1,215,524 -> 349,680 B, 30 PNGs to KTX2 (total 162,186,687 B from 314,889,680 B of source). Budget `uncooked 0 bytes` against the 64,000,000 default ceiling — first time this game had a manifest at all. The scenario was not run in this lane, so the box stays open: `node packages/playtest/dist/runner/cli.js metahuman-lab/playtests/face-controls.playtest.json --url http://127.0.0.1:5197 --server-command "pnpm exec vite preview --host 127.0.0.1 --port 5197 --strictPort" --browser-recipe webgpu`. - [x] [local] `finding-assets.md`'s Fab import step says the default cook handles imported packs and that `"none"` is not a fix. proof: `pnpm exec vitest run scripts/__tests__/primary-docs.spec.ts` and `pnpm check:docs`. **Result 2026-10-03:** the Fab import step now states what the default cook does to a converted pack (embedded PNG and WebP to KTX2, quantization and Meshopt, a discrete LOD chain for a `MASK` primitive) with the measured Hornbeam numbers, and that `models: "none"`/`textures: "none"` are not a fix — they ship every source byte, count it all against `assets.budget.uncooked`, and are what makes the build fail with `TN_ASSETS_BUDGET_EXCEEDED`. Green: `primary-docs.spec.ts` 7/7, `check:docs` 2,229 links, `check-doc-links` + `sync-agent-docs` + `evidence-citations` 30/30. No template file changed, so no scaffold hash moved.