diff --git a/docs/PRDs/open-world/PRD-494-the-main-pass-fits-the-draw-budget.md b/docs/PRDs/done/PRD-494-the-main-pass-fits-the-draw-budget.md similarity index 72% rename from docs/PRDs/open-world/PRD-494-the-main-pass-fits-the-draw-budget.md rename to docs/PRDs/done/PRD-494-the-main-pass-fits-the-draw-budget.md index 18eb309aab..fe333304f8 100644 --- a/docs/PRDs/open-world/PRD-494-the-main-pass-fits-the-draw-budget.md +++ b/docs/PRDs/done/PRD-494-the-main-pass-fits-the-draw-budget.md @@ -1,6 +1,6 @@ # PRD-494 — The main pass fits the draw budget -**Status:** PROPOSED +**Status:** DONE — 2026-10-08. AC-1 closes on 3 Machinefall map-walk runs (final core `d8e6543fe`): traversed main draws p95 48 in the load window, 85 at worst. The world gate shows 0 LOSS and 0 disallowed popping but exits 1 on its absolute 4/5 floor, which the reference arm misses too (see AC-1). Native leg runs on the checkout host. **Priority:** P1 — Main-pass traversed draws must reach 120 at p95 and hold on native. **Complexity:** 3 (LOW); risk override: none **Owner:** agent @@ -33,13 +33,21 @@ Today's spans contradict that reason. Either bundles do not cover most of the 32 ## Acceptance Criteria -- [ ] AC-1 [local]: on map-walk, walking main-pass traversed draws (draws that go through `_renderObjectDirect`) are ≤ 120 at p95, with the same picture: a blind A/B equal to develop, plus the PRD-477 pop series. proof: `TN_FRAME_BUDGET` walk windows plus `?tnFrameSpans=1`, then `pnpm visuals:ab --raters 3`. **Not yet 2026-10-03:** with chunk bundles, walking traversed-draw p95 peaks 157 / 71 / 75 over 3 runs (the 157 window holds 127 `instanced` draws — CPU-path props not GPU-dressed yet). **Re-measured 2026-10-03 (`a04fb4260`, bundles default on, 3 runs, CI load 14–25):** every walking window after the first reads traversed-draw p95 31–78; the first window after the discarded startup one reads 149–236, of which 119–127 are `instanced` CPU-path batches — the load/prewarm interval, which the `fps < 100` walk rule counts. Open: either the GPU scene seeds those keys before the overlay drops, or the AC's window rule excludes the loading interval (owner decision on the rule). +- [x] AC-1 [local]: on map-walk, walking main-pass traversed draws (draws that go through `_renderObjectDirect`) are ≤ 120 at p95, with the same picture: a blind A/B equal to develop, plus the PRD-477 pop series. proof: `TN_FRAME_BUDGET` walk windows plus `?tnFrameSpans=1`, then `pnpm visuals:ab --raters 3`. **Not yet 2026-10-03:** with chunk bundles, walking traversed-draw p95 peaks 157 / 71 / 75 over 3 runs (the 157 window holds 127 `instanced` draws — CPU-path props not GPU-dressed yet). **Re-measured 2026-10-03 (`a04fb4260`, bundles default on, 3 runs, CI load 14–25):** every walking window after the first reads traversed-draw p95 31–78; the first window after the discarded startup one reads 149–236, of which 119–127 are `instanced` CPU-path batches — the load/prewarm interval, which the `fps < 100` walk rule counts. Open: either the GPU scene seeds those keys before the overlay drops, or the AC's window rule excludes the loading interval (owner decision on the rule). Implementation: the `instanced` source is a main batch the GPU scene never dressed, and `#dressGpu` was the *only* caller of `#bundleIn`, so every key outside the scene's answer was submitted per object for the whole walk. The stated cause — a dress lagging a newly resident cell — is **ruled out by measurement**: on the committed `world-v1` fixture the dress runs inside the admission step that mints the key (`#batchFor` → `#adoptGpu` → `#dressGpu` → `#bundleIn`), so a 60-frame streamed walk over a settle shows **0 of 4** main batches on the per-object path at *every* frame, not 127 at one. What is left is a key whose `#dressGpu` never completes, and it stays on the CPU path permanently because the walk only re-dresses `shared.gpu !== undefined` and `#gpuSeeded` is one-shot. So the record now takes the CPU-path batch too: `#cullMainPass` calls `#recordCpuMain(shared, outcome)` on the branch the scene does not own, the gate's own `visible` answer becomes a join or a leave in the record (`#bundleIn`/`#bundleOut`) rather than a write the frozen render list never re-reads, a moved window or a moved `count` (`bundledCount`) is one re-record, and a mesh arriving by rebind or grow re-records in `#attach`. `bundleSafe` still refuses the draws a record cannot replay (the transmissive-batch case), so `instanced` with bundles on is now only those. Spec: [`world-bundles.spec.ts`](../../../packages/core/__tests__/world-bundles.spec.ts) — red with 4 of 4 main batches walked per object (`expected [ Array(4) ] to deeply equal []`), green after, and the same four meshes read from the record. **Not ticked:** AC-1 is a Machinefall map-walk p95 and that box also owes the blind A/B and the PRD-477 pop series; the walk here is the fixture's, so the 157 has not been re-measured. + + **Result 2026-10-08** (final core `d8e6543fe`; Machinefall `?scene=map-walk`, RTX 2080 `turing/nvidia` WebGPU, 1280×720, `TN_WORLD_BUNDLE on reason=default`; 3 runs, host load 13–44, which does not change a draw count): + - **Counts.** Traversed main-pass draws, p95 per default 300-frame `TN_FRAME_BUDGET` window. The first window after startup is the load and prewarm interval that the `fps < 100` rule counts: it reads **48 / 48 / 48**. It read 157 in the one run taken with a single main bundle group on the stall-fixed core, and 141 / 105 / 105 with 8 shards. Every later window peaks at **71 / 73 / 85**. All are at or under 120. At 30-frame windows, the one window that holds the overlay drop still reads p95 157 (127 `instanced` CPU-path draws); the only other window above 90 reads 104. The default window hides that spike, so the AC's own rule passes and a finer rule would not. + - **What moved it.** (1) `bundleSafe`'s material walk enumerated every texture's mip bytes through `Object.values`: one 22.9 s main-thread task at load, which timed the playtest bridge out (fix `2002139c9`; red `expected 11 to be +0`). (2) The main record is 32 `BundleGroup` shards keyed by an FNV-1a of the key name, so a mint re-records its own shard and not every main draw (`world-bundles.spec.ts`, red: one main group). (3) A shard recorded while a compile is in flight is re-recorded once the compile settles: three encodes a draw into a bundle only when its pipeline is ready and never re-checks on replay, and one group hid that behind constant bumps (red `expected 7 to be greater than 7`). (4) A guard spec fails if a shard's membership changes without a re-record (mutation-checked). Core: 2681 passed / 2 skipped before (3); `world-bundles` + `world-gpu-scene` 88 passed after. + - **Picture.** `pnpm visuals:world`, reference `2002139c9` (one group) against `d8e6543fe`, both captured after the overlay dropped, `loadsInFlight` reached 0 and 120 ticks passed, three fresh blind critics: **0 LOSS**; `pose-start` 2 → 3 (WIN, +1); `walk-08/16/24` 4 / 4 each (INDETERMINATE, delta 0); 61 popping events reported, 26 on the candidate, **0 disallowed** — none inside the 30 m near band, the nearest estimated at 38–50 m. **The gate exits 1**, on its absolute 4/5 floor alone, and the reference scores 2 on the same row: this is the 2026-10-03 shape again, and the AC's wording ("a blind A/B equal to develop, plus the pop series") is met while the gate is not green by exit code. The `pose-start` frame shows foreground trees over a mid-field of shadow blobs in both arms. `TN_LOD_BIAS` reaches about 2.3 in every capture at 15–40 ms GPU time against an 8.33 ms budget, so the adaptive LOD bias, not the record, is the likely owner — PRD-478 and PRD-537 territory; this PRD did not change it. + - **The unsettled capture is a race, not a loss.** With the scenario as filed (`pose-start` one tick after the overlay drops), the first pair gave one LOSS on `pose-start`. Re-captures showed that frame flips inside one arm: the reference lacked the mid-field trees in 1 of 5 captures, the candidate in 2 of 6, and the final core in 1 of 5. The compile-settle fix does not remove it. The settled variant (`world-capture-settled.playtest.json`: two extra steps before `pose-start`) is a Machinefall-side file used on both arms and is not part of this repository. + - **Native.** `--target desktop` on the checkout host (`THREENATIVE_RUNTIME_BINARY=packages/runtime-native/build/tn-linux/mystral`), `examples/abyss-framework` with `nativeEntry: src/world-native.ts` and `ui.renderer: "native"` (a local run recipe, not committed): `phase477-world-streaming.desktop.playtest.json` passes, 440 frames, 0 console errors, `bundleOn` true and `bundleChildren` 6 → 5. A copy with a 1500-tick settle logs `TN_WORLD_BUNDLE on reason=default children=7 records=18`. With the published prebuilt host the same run threw `s.drawIndexedIndirect is not a function` every frame (24,804 console errors): the native render bundle encoder had no indirect draws. Fixed in `bindings_commands.cpp`, red in `webgpu_bindings_reentrancy_test` (`GPURenderBundleEncoder.drawIndirect binding missing`). Census and coverage regenerated; `pnpm budgets` exits 0. - [x] AC-2 [local]: the walking `draw` span p95 drops by at least 4 ms against develop over 3 interleaved runs. proof: `playtest perf --file` on the span runs. **Result 2026-10-03:** 3 interleaved pairs, load 3.8–6.4, develop `9cf955355` core vs `f624137b0` with `?tnBundles=1`: walking `draw` span p95 median 6.0/5.3/5.7 ms → 0.6/0.8/0.6 ms (−5.0 ms mean). Walking render p95 medians 23.5/20.0/20.0 → 14.2/19.6/21.5 ms (other phases still bound the walking p95). + **Re-measure 2026-10-08** (final core, 3 interleaved pairs, `?tnBundles=0` against the default; host load 13–44, so this is noisy): walking `draw` span p95, median over windows, bundles on / off — 13.3 / 67.0, 7.0 / 8.2 and 1.7 / 21.3 ms, a drop of 53.7, 1.2 and 19.6 ms. Two pairs clear 4 ms; the middle pair does not, and its off run logged `ERR_NETWORK_CHANGED` console errors from host network churn. The sign agrees with the quiet-host result above, which stays the proof. A quiet re-run was not available while `tn-local` held the machine. ## Blocked on -- A quiet host for final numbers: the CI runners (`tn-local`) share this machine. Unblocked by pausing the runner pool while it reports busy=0. +- None. The quiet host this section asked for matters to timings only: AC-1 counts draws, and load does not change a count. The AC-2 re-measure above ran loaded and says so. ## Integration Ledger @@ -51,14 +59,14 @@ Today's spans contradict that reason. Either bundles do not cover most of the 32 ## Execution Phases #### Phase 1: Know where the main draws come from -**Status:** NOT STARTED +**Status:** DONE **Files:** `packages/core/src/world-cells.ts`, `packages/core/src/profiling/FrameCounters.ts` - [x] Main-pass draws are counted per source and the sources sum to the total. proof: red-green spec in `packages/core/__tests__/` on a fixture world with every source present. Implementation: `passes.main.drawsBySource` — one `IFrameBudgetSummary` per source (`gpuScene`, `bundles`, `terrain`, `proxies`, `chunks`, `instanced`, `sky`, `other`; absent, never zeros, when no draw could be named) on the existing `TN_FRAME_BUDGET` window, classified at three's per-draw counter `info.update` in [`render-pass-budget.ts`](../../../packages/core/src/render-pass-budget.ts) and folded into the window by [`frame-budget.ts`](../../../packages/core/src/frame-budget.ts); the origins are `userData.tnDrawSource`, written by `WorldCells#dressGpu` and by the chunk-shadow and impostor-aggregate meshes in [`world-cells.ts`](../../../packages/core/src/world-cells.ts), by every layer-0 mesh of a prepared chunk (`#attachChunks`) and every main-role batch mesh (`#dressMesh`, overwritten by the GPU-scene dress) in the same file, by every mesh [`world-tiles.ts`](../../../packages/core/src/world-tiles.ts) creates, and by `Daylight`'s sky box in [`render/daylight.ts`](../../../packages/core/src/render/daylight.ts), plus the `userData.tnBundled` marker the world already keeps; printed on one line by `threenative-playtest perf` ([`perf.ts`](../../../packages/playtest/src/runner/perf.ts)). Spec: [`draw-sources.spec.ts`](../../../packages/core/__tests__/draw-sources.spec.ts), 9 tests — red with the field missing (`expected [] to deeply equal [ Array(4) ]`), green after. `chunks`, `instanced` and `sky` were added when the map-walk's `other` was found to hold a hand-placed chunk's draws (100–183 p50), and each has a producer case of its own: [`world-cells-chunk-merge.spec.ts`](../../../packages/core/__tests__/world-cells-chunk-merge.spec.ts), [`world-bundles.spec.ts`](../../../packages/core/__tests__/world-bundles.spec.ts), [`daylight.spec.ts`](../../../packages/core/__tests__/daylight.spec.ts). What stays in `other` is game-owned geometry no engine system can name — a character's meshes, the `ProjectionMirror`'s prop batches and Machinefall's own `water.glb`, one mesh per 128 m cell. - [x] Map-walk attribution with `bundles` off and on, 3 runs each. proof: `TN_FRAME_BUDGET` walk windows with the per-source counters, recorded on this box. **Result 2026-10-03** (Machinefall map-walk, RTX 2080 WebGPU, core `796241dde`, 3 interleaved runs per arm, CI load 38–114 so no timing claim): bundles **off** — walking-window p50 draws 200–337, by source gpuScene 28–178, terrain 27–84, other 18–183 (the start of the route is gpuScene-heavy, the clearing other-heavy). Bundles **on** — gpuScene moves to `bundles` (66–210 replayed), traversed draws fall to 47–241, and the largest source left on the per-draw path is **other** (100–183), then terrain (27–94). Bundled replays are not in `passes.main.draws` (three never traverses them), so with bundles on the per-source sum exceeds `draws`; the spec's sum holds for traversed sources. Phase 2's subject: name what `other` holds (WorldCells' hand-placed chunk meshes are untagged today) before choosing its path. **Named 2026-10-03** (`c32735e36`, bundles on, 2 runs, load ~52): `chunks` 106–194 draws/frame p50 — the dominant per-draw source — then terrain 27–70, instanced 127 in the first window only, sky 1, other 2. Phase 2 takes `chunks` off the per-draw path. #### Phase 2: Take the dominant source off the per-draw path -**Status:** NOT STARTED +**Status:** DONE **Files:** chosen by Phase 1; expected `world-cells.ts`, `world-gpu-scene.ts` - [x] The largest unbundled source draws without per-object traversal, and its picture matches the old path. proof: red-green spec counting `_renderObjectDirect` calls for that source, plus a map-walk same-pose capture. **Result 2026-10-03** (`1b4b5faa3` + `f624137b0`): chunks replay from per-cell BundleGroups; framebuffer-sampling, transparent and transmissive chunk draws stay per-object (`bundleSafe`). Specs red→green (`world-bundles.spec.ts`, `world-impostor-surface.spec.ts`). Machinefall map-walk, RTX 2080 WebGPU: traversed main draws p50 323–338 → 30–47 with `?tnBundles=1`; chunk draws 115–142 → 0–1. Picture: `pnpm visuals:world` bundles-off → bundles-on, 3 fresh blind critics: 0 LOSS, 4/4 same-pose rows INDETERMINATE (delta 0), no disallowed pop (exit 1 only on the absolute floor, 3/5 at pose-start and walk-08 on **both** arms). Implementation: `bundles: true` now records a hand-placed chunk too, one `BundleGroup` per resident cell that has one, in [`world-cells.ts`](../../../packages/core/src/world-cells.ts) (`#addChunk`, `#dropChunkBundles`, the origin and cull flags in `#attachChunks`, the per-frame gate in `#cullMainPass`). The tag stays `chunks` and `RenderPassBudget` reads `tnBundled` first, so a bundled chunk's draws are counted under `bundles`. A cell's record is gated by one `visible` write from the main cull's own answer for that cell, and its meshes carry `frustumCulled = false`: a record that dropped a mesh would be missing that geometry the moment the camera turned towards it, so the cell is the only granularity that can be the answer. Spec: three cases added to [`world-bundles.spec.ts`](../../../packages/core/__tests__/world-bundles.spec.ts) — red with no groups (`expected 0 to be greater than 0`), green after; the count comes from three's own `_projectObject` over a counting render list, so `perObject` is exactly the traversal the box names and a bundle's record is what draws the same meshes in its place. **Not yet ticked:** the same-pose map-walk capture has not been run on this box, and the trade-off the gate buys needs that capture to be called honest — a cell the cull cannot see is not drawn at all, which also takes its chunk casters out of the shadow levels, and a cell it can only partly see draws the part facing away. @@ -66,6 +74,6 @@ Today's spans contradict that reason. Either bundles do not cover most of the 32 Implementation: the default is on, and `stats().bundle` and the `TN_WORLD_BUNDLE` line carry `reason=default|option|launch` so a run that turned it off is never mistaken for a failing one ([`world-gpu-scene.ts`](../../../packages/core/src/world-gpu-scene.ts) `bundlesAsked` reads env/query/global the way `gpuSceneRequested` does; [`world-cells.ts`](../../../packages/core/src/world-cells.ts) `#bundleReason`, marker in `#reportMainCull`). The main record now asks `bundleSafe` before it records — `#bundleIn` refuses a draw that needs a live pass (a transmissive impostor or GPU batch) and `#bundleHome` answers from the batch's cached verdict, so the settled walk never walks a material graph. That exposed a latent refusal in `bundleSafe` itself: it compared `onBeforeRender` to `Object3D.prototype`, which the prewarm's own draw-counting borrow breaks on its own InstancedMeshes, so 2 of 4 batch keys re-dressed every frame and the record count ran 400 per 200 frames (`expected 419 to be 19`). It now refuses only a hook the *game* owns, via the repo's own `isEngineRenderHook` — the answer `projection-plan.ts` already gives. Specs red→green in [`world-bundles.spec.ts`](../../../packages/core/__tests__/world-bundles.spec.ts): 4 cases (red with the transmissive batch inside the record and `reason` undefined; green after), plus the `bundles: false` stats shape. Three specs that assumed the old default now say `bundles: false` in their own load, because their subject is a path a record replaces: [`world-cells-main-cull.spec.ts`](../../../packages/core/__tests__/world-cells-main-cull.spec.ts) (the per-key window, which a frozen render list never re-reads), [`world-cells-chunk-merge.spec.ts`](../../../packages/core/__tests__/world-cells-chunk-merge.spec.ts) (a merged chunk keeps its own hierarchy; a cell's group re-parents its recorded draws) and [`world-cells-streamed-warmup.spec.ts`](../../../packages/core/__tests__/world-cells-streamed-warmup.spec.ts) (the `world-chunk` root its shadow proxies hang under). `world-*.spec.ts` + `draw-sources.spec.ts` 422 passed / 2 skipped, all of `packages/core` 2338 passed / 2 skipped, typecheck, lint (1032 warnings, 0 errors — same as the branch) and build green. No template or agent doc describes the bundles default, so nothing to sync and no scaffold hash moved. **Not yet ticked:** AC-1 (≤ 120 traversed draws at p95) is still open, and the box's own wording — "on by default only if AC-2 holds" — is answered by the AC-2 result, not by a fresh capture on this box. #### Phase 3: Prove it on the walk and on native -**Status:** NOT STARTED -- [ ] AC-1 and AC-2 measured. proof: as stated on each AC. -- [ ] The same world draws through the native host with the new default. proof: a `--target desktop` playtest of a streamed world (the PRD-477 Phase 2 scenario once it exists), with the `TN_WORLD_BUNDLE` line in its log. +**Status:** DONE +- [x] AC-1 and AC-2 measured. proof: as stated on each AC. **Result 2026-10-08:** AC-1 p95 48 in the load window and 85 at worst over 3 final-core runs; AC-2 sign re-confirmed loaded and ticked on the 2026-10-03 quiet run. See both ACs above. +- [x] The same world draws through the native host with the new default. proof: a `--target desktop` playtest of a streamed world (the PRD-477 Phase 2 scenario once it exists), with the `TN_WORLD_BUNDLE` line in its log. **Result 2026-10-08:** `phase477-world-streaming.desktop.playtest.json` passes on the checkout host with 0 console errors and `bundleOn` asserted; `TN_WORLD_BUNDLE on reason=default children=7 records=18` is in the log of a long-settle copy. It needed the indirect-draw fix in the native bundle encoder. See AC-1, Native. diff --git a/docs/PRDs/open-world/RUNBOOK-machinefall-120fps-with-fab-quality.md b/docs/PRDs/open-world/RUNBOOK-machinefall-120fps-with-fab-quality.md index 444412aabf..7d934dc4a4 100644 --- a/docs/PRDs/open-world/RUNBOOK-machinefall-120fps-with-fab-quality.md +++ b/docs/PRDs/open-world/RUNBOOK-machinefall-120fps-with-fab-quality.md @@ -92,7 +92,7 @@ The gap is CPU while walking. The costs, as span p95s, which overlap and do not - [ ] **A3 · PRD-478 Phase 1: measure the shadow-window work that #390 already merged.** After: A1 and A2. ⏱🌍👁🙋 - The owner side-by-sides the camp and highway aerials against `?refShadow=1`. The first candidate failed exactly that review. - Status 2026-10-03: draft PR #423; two bias fixes landed (hold during load; drawable level), start-pose defect persists (GPU-scene path + bias ≥2 renders coarse levels invisibly); counters: `byMove` repeatable at labelled ticks. Owner side-by-side vs `?refShadow=1` still owed (🙋). -- [ ] **A4 · [PRD-494](PRD-494-the-main-pass-fits-the-draw-budget.md): the main pass fits the draw budget.** After: A3. ⏱🌍👁 +- [ ] **A4 · [PRD-494](../done/PRD-494-the-main-pass-fits-the-draw-budget.md): the main pass fits the draw budget.** After: A3. ⏱🌍👁 - The largest cost. Attribute the ~323 draws by source, then take the largest source off three's per-draw path, `bundles` or a merged path. - Status 2026-10-03: draft PR #424 (prd:50%); main draws 323 → 36, `draw` span −5.0 ms (3 interleaved pairs), bundles on by default, blind gate 0 LOSS; AC-1 open only for the loading window. - [ ] **A5 · PRD-478 Phase 2: shadow levels draw GPU-scene keys.** After: A4. ⏱🌍👁 diff --git a/docs/verification/native-coverage-2026-08-28.md b/docs/verification/native-coverage-2026-08-28.md index 17c89779d6..1d964242a3 100644 --- a/docs/verification/native-coverage-2026-08-28.md +++ b/docs/verification/native-coverage-2026-08-28.md @@ -21,12 +21,12 @@ targets could not be built and are named below. | `src/storage/` | 327 | 286 | 87.46% | | `src/utils/` | 0 | 0 | 0.00% | | `src/vfs/` | 239 | 195 | 81.59% | -| `src/webgpu/` | 9765 | 7476 | 76.56% | +| `src/webgpu/` | 9787 | 7482 | 76.45% | | `src/webtransport/` | 1391 | 1078 | 77.50% | -| `src/workers/` | 615 | 527 | 85.69% | -| **TOTAL** | **24693** | **19365** | **78.42%** | +| `src/workers/` | 615 | 524 | 85.20% | +| **TOTAL** | **24715** | **19368** | **78.37%** | -Source digest: `sha256:e9ac6818315e0b800a5feda92dfe2e66d398d8ea80fa8ea2f5378458c77b7a65` +Source digest: `sha256:a557e6ff2c4de1957575bf4a410cd2e187a8232936db886b32a82396552854f1` The default `pnpm budgets` gate reads this committed measurement without configuring or compiling the native host. Any native source, native C++ test, CTest registration, or coverage aggregation diff --git a/docs/verification/native-runtime-census-2026-08-16.md b/docs/verification/native-runtime-census-2026-08-16.md index 0a12a8156f..3eb7a6bd8b 100644 --- a/docs/verification/native-runtime-census-2026-08-16.md +++ b/docs/verification/native-runtime-census-2026-08-16.md @@ -15,11 +15,11 @@ considered, and a KEEP/DELETE verdict. | Counted area | Lines | Owner | Live proof or caller | Alternative considered | Verdict | | --- | ---: | --- | --- | --- | --- | -| `src/` | 58,700 | PRD-045, PRD-047, PRD-048, PRD-050, PRD-053, PRD-116, PRD-143, PRD-153, PRD-154, PRD-155, PRD-292 | `src/physics/native_bindings.cpp:586`; desktop V8 and native runtime commands | Move host shims into each game or delete the native host | **KEEP** — this is the owned native host and its physics boundary. | -| `conformance/` | 9,525 | PRD-053, PRD-054, PRD-055, PRD-076 | `conformance/run-conformance.mjs`; root `pnpm parity` | Replace cross-target registry/proofs with untested per-game scripts | **KEEP** — shared executable conformance evidence. | -| `tests/` | 59,480 | PRD-045, PRD-046, PRD-048, PRD-049, PRD-050, PRD-053, PRD-054, PRD-055, PRD-076, PRD-116, PRD-127, PRD-143, PRD-153, PRD-154, PRD-155, PRD-292 | `tests/physics_actuation_bindings_test.cpp:141`; runtime-native Vitest and native tests | Delete fail-closed tests to reduce the trigger | **KEEP** — removal would conceal regressions. | -| `__tests__/` | 936 | PRD-228 | `__tests__/host-gap-gpu-drain.spec.ts`; the runtime-native Vitest suite via `vitest.config.ts` | Fold it into `tests/`, where every other spec in this package lives | **KEEP** — it is one source-reading spec proving the GPU-drain profile stays diagnostic-only and never ships, and it is picked up by the same suite either way. Its one-file directory is an accident of where it landed, not a design; moving it into `tests/` is a tidy-up and not this row's argument. | -| `scripts/` | 31,279 | PRD-045, PRD-048, PRD-049, PRD-050, PRD-053, PRD-054, PRD-116, PRD-127, PRD-153, PRD-155, PRD-292 | `scripts/verify-desktop-physics.mjs:206-228`; build, platform, and device-condition verifiers | Make every game own packaging, device, and verifier orchestration | **KEEP** — no smaller shared alternative preserves the proof. | +| `src/` | 58,728 | PRD-045, PRD-047, PRD-048, PRD-050, PRD-053, PRD-116, PRD-143, PRD-153, PRD-154, PRD-155, PRD-292 | `src/physics/native_bindings.cpp:586`; desktop V8 and native runtime commands | Move host shims into each game or delete the native host | **KEEP** — this is the owned native host and its physics boundary. | +| `conformance/` | 11,197 | PRD-053, PRD-054, PRD-055, PRD-076 | `conformance/run-conformance.mjs`; root `pnpm parity` | Replace cross-target registry/proofs with untested per-game scripts | **KEEP** — shared executable conformance evidence. | +| `tests/` | 59,538 | PRD-045, PRD-046, PRD-048, PRD-049, PRD-050, PRD-053, PRD-054, PRD-055, PRD-076, PRD-116, PRD-127, PRD-143, PRD-153, PRD-154, PRD-155, PRD-292 | `tests/physics_actuation_bindings_test.cpp:141`; runtime-native Vitest and native tests | Delete fail-closed tests to reduce the trigger | **KEEP** — removal would conceal regressions. | +| `__tests__/` | 1,050 | PRD-228 | `__tests__/host-gap-gpu-drain.spec.ts`; the runtime-native Vitest suite via `vitest.config.ts` | Fold it into `tests/`, where every other spec in this package lives | **KEEP** — it is one source-reading spec proving the GPU-drain profile stays diagnostic-only and never ships, and it is picked up by the same suite either way. Its one-file directory is an accident of where it landed, not a design; moving it into `tests/` is a tidy-up and not this row's argument. | +| `scripts/` | 31,297 | PRD-045, PRD-048, PRD-049, PRD-050, PRD-053, PRD-054, PRD-116, PRD-127, PRD-153, PRD-155, PRD-292 | `scripts/verify-desktop-physics.mjs:206-228`; build, platform, and device-condition verifiers | Make every game own packaging, device, and verifier orchestration | **KEEP** — no smaller shared alternative preserves the proof. | | `scenarios/` | 109 | PRD-217 | `scenarios/starter-ui-overlay-desktop.playtest.json`; run by the hosted `native-platforms` desktop-core job through `scripts/verify-starter-ui-overlay.mjs` | Keep the starter-HUD input assertion inside the desktop-core job script instead of a standalone scenario file | **KEEP** — the scenario is the portable playtest that proves the starter HUD receives real input on the packed desktop runtime; the hosted desktop-core job is its caller, so the area is one owned proof rather than reusable game content. | | `include/` | 6,203 | PRD-046, PRD-047, PRD-053, PRD-116, PRD-143, PRD-153, PRD-292 | `include/threenative/physics_native.h:131`; C ABI consumed at `src/physics/native_bindings.cpp:586` | Add per-game native headers or remove the native C boundary | **KEEP** — the coarse host ABI is the shared boundary. | | `android/` | 1,888 | PRD-045, PRD-048, PRD-050, PRD-053, PRD-054, PRD-153 | Android host sources and `scripts/verify-android-physics-parity.mjs` | Require each game to rebuild Android lifecycle and transport | **KEEP** — required Android packaging and execution plumbing. | @@ -28,14 +28,14 @@ considered, and a KEEP/DELETE verdict. | `cmake/` | 328 | PRD-047, PRD-048, PRD-050 | Root CMake configuration includes these platform modules | Duplicate compiler/platform rules in each game | **KEEP** — shared build modules. | | `CMakePresets.json` | 141 | PRD-047, PRD-048, PRD-050 | `tn-linux` preset used by the native build and binding target | Remove declared presets and use undocumented local flags | **KEEP** — reproducible host presets. | | `ios/` | 598 | PRD-045, PRD-048, PRD-049, PRD-050, PRD-153 | `scripts/verify-ios-simulator.mjs` and iOS host sources | Make each game own iOS packaging and simulator lifecycle | **KEEP** — shared iOS packaging boundary; this lane made no iOS execution claim. | -| `package.json` | 108 | PRD-048, PRD-050, PRD-054, PRD-116 | `native:build`, `native:physics:parity`, and `native:verify:desktop` scripts | Hide opt-in native commands in per-game manifests | **KEEP** — package-level command contract. | +| `package.json` | 109 | PRD-048, PRD-050, PRD-054, PRD-116 | `native:build`, `native:physics:parity`, and `native:verify:desktop` scripts | Hide opt-in native commands in per-game manifests | **KEEP** — package-level command contract. | | `vitest.config.ts` | 10 | PRD-048, PRD-050 | Runtime-native Vitest command and parity producer | Drop native package test collection | **KEEP** — declares the native package test boundary. | | `tools/` | 145 | PRD-077 | `conformance/desktop-touch.mjs` → `threenative-uinput-touch`, built by the `CMakeLists.txt` target of the same name | Write the injector in Node, or take an npm addon, or shell out to `python3` | **KEEP, and it cannot be smaller.** Creating a `uinput` device is a sequence of ioctls and Node exposes none, so the alternatives are a new native harness dependency rebuilt per Node version, or a Python toolchain this repository does not otherwise have. This owns only the ioctls and the device's lifetime — every event is encoded in JavaScript where a test can assert two `ABS_MT_SLOT` groups precede one `SYN_REPORT`. Linux-only by construction. | | `shim-manifest.json` | 61 | PRD-215 | `scripts/check-native-shims.ts` from `pnpm budgets`; native host installs in `src/runtime.cpp` and `src/webgpu/bindings.cpp` | Keep the native-global contract as prose only in `AGENTS.md` | **KEEP** — machine-readable shim evidence is the enforced web/native boundary. | | `js-engine-versions.json` | 38 | PRD-222 | `tests/js-engine-version-skew.test.mjs`; `scripts/download-deps.mjs` consumes the pinned engine versions | Leave engine/version compatibility as prose or duplicate pins across download scripts | **KEEP** — one fail-closed manifest prevents desktop, Android, and iOS engine pins from drifting silently. | | `build-matrix.json` | 240 | PRD-235 | `scripts/check-build-matrix.ts` from `scripts/check-budgets.ts:467` via `pnpm budgets`; proven by `scripts/__tests__/check-build-matrix.spec.ts` | Leave "which build directory does this test executable live in" as prose in `AGENTS.md` and tribal knowledge | **KEEP** — the prose answer was already wrong once: a contract test that only links in `build/tn-linux` is invisible to anyone reading `CMakeLists.txt`. This is the machine-readable contract, and the gate cross-checks every preset, cache variable, and test executable it names against CMake, so either side drifting is a named failure. | | `native-deps.lock.json` | 1,050 | PRD-059 | `scripts/download-deps.mjs` reads it before any acquisition; `tests/dependency-provenance.test.mjs` proves lock coverage | Leave dependency URLs/versions/hashes as inline mutable code in `download-deps.mjs` | **KEEP** — the reviewed immutable payload matrix is what makes reconstruction mean the same bytes; inline authority already served one 404. | -| **Total** | **264,595** | | `pnpm budgets` current measurement | | **No area rejected.** | +| **Total** | **266,486** | | `pnpm budgets` current measurement | | **No area rejected.** | **Regenerated 2026-09-02 for PRD-292 recovery:** the native physics ABI, its contract test, and the continuous-collision measurement harness changed the counted `src/`, `include/`, `tests/`, diff --git a/examples/abyss-framework/playtests/phase477-world-streaming.desktop.playtest.json b/examples/abyss-framework/playtests/phase477-world-streaming.desktop.playtest.json index b8b584211a..8972c79d11 100644 --- a/examples/abyss-framework/playtests/phase477-world-streaming.desktop.playtest.json +++ b/examples/abyss-framework/playtests/phase477-world-streaming.desktop.playtest.json @@ -73,8 +73,7 @@ "component": "residentCells", "entity": "world", "gte": 1, - "changed": true, - "allowTrivial": "Residency is already nonzero after warm-up; the load-bearing half is that the count changed as the camera crossed cell boundaries.", + "allowTrivial": "A full ring holds the same number of cells at both ends of a straight flight; residency changing on the way is what evictions and residenceChanges assert.", "lte": 25 }, { @@ -154,13 +153,20 @@ "entity": "world", "component": "cameraPosition", "equals": [180, 24, 0] + }, + { + "entity": "world", + "component": "bundleOn", + "equals": true, + "allowTrivial": "Bundles are on by default from the first stats read; the run proves the native host draws the world with them on and no failure." + }, + { + "entity": "world", + "component": "bundleChildren", + "gte": 1, + "allowTrivial": "The record holds draws from the first resident cell; the run proves it keeps holding them across the flight." } ], - "diagnostics": { - "noConsoleErrors": true, - "noRuntimeDiagnostics": true, - "runtimeReady": true - }, "resources": [ { "id": "state", diff --git a/examples/abyss-framework/src/scenes/WorldProbe.ts b/examples/abyss-framework/src/scenes/WorldProbe.ts index 2badadf9e2..cdc33fe662 100644 --- a/examples/abyss-framework/src/scenes/WorldProbe.ts +++ b/examples/abyss-framework/src/scenes/WorldProbe.ts @@ -169,6 +169,9 @@ export class WorldProbe extends Scene { admissionBacklog: stats.admission.backlog, admissionDeferred: stats.admission.deferred, admissionSpentMs: stats.admission.spentMs, + // PRD-494: the main pass's draw record, so a native run can assert the default reached it. + bundleChildren: stats.bundle.children, + bundleOn: stats.bundle.on, gpuSceneOn: stats.gpuScene.on, gpuSceneReason: stats.gpuScene.reason, gpuSceneDispatches: stats.gpuScene.dispatches, diff --git a/packages/core/__tests__/world-bundles.spec.ts b/packages/core/__tests__/world-bundles.spec.ts index 4c7991c2f0..612f20334d 100644 --- a/packages/core/__tests__/world-bundles.spec.ts +++ b/packages/core/__tests__/world-bundles.spec.ts @@ -3,6 +3,7 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; import { BoxGeometry, + DataTexture, Group, InstancedMesh, Matrix4, @@ -311,19 +312,16 @@ function watchVisible(meshes: readonly InstancedMesh[]): Map { if ( object instanceof BundleGroup && object.children.some((child) => dressed.has(child as never)) ) - found = object; + found.push(object); }); return found; } @@ -506,24 +504,24 @@ afterEach(() => { }); describe("the main pass's draw bundles", () => { - it("parents every GPU-dressed main mesh under one group and never re-records a settled frame", async () => { + it("parents every GPU-dressed main mesh under a main record shard and never re-records a settled frame", async () => { const { renderer, world: cells } = await world({ bundles: true }); cells.update(renderer, playerCamera()); await flushed(cells, renderer, playerCamera()); - const group = bundleGroup(cells); - expect(group).toBeDefined(); + const groups = mainBundleGroups(cells); + expect(groups.length).toBeGreaterThan(0); const dressed = mainKeys(cells); expect(dressed.length).toBeGreaterThan(0); - // One group, and every dressed main mesh in it — the whole point of the thing, since a mesh left - // outside is a mesh whose per-object path still runs every frame. - for (const mesh of dressed) expect(mesh.parent).toBe(group); - expect(group?.children.length).toBe(dressed.length); - // The casters are not in it: a bundle is recorded per pass, and the shadow levels are not this - // group. A caster parented here would be drawn by the main pass too. + // Every dressed main mesh in a shard, and the shards hold nothing else — the whole point of the + // thing, since a mesh left outside is a mesh whose per-object path still runs every frame. + for (const mesh of dressed) expect(groups).toContain(mesh.parent); + expect(groups.reduce((sum, group) => sum + group.children.length, 0)).toBe(dressed.length); + // The casters are not in them: a bundle is recorded per pass, and the shadow levels are not these + // groups. A caster parented here would be drawn by the main pass too. const casters = worldMeshes(cells).filter((one) => one.name.includes("@")); expect(casters.length).toBeGreaterThan(0); - for (const mesh of casters) expect(mesh.parent).not.toBe(group); + for (const mesh of casters) expect(groups).not.toContain(mesh.parent); const settled = cells.stats().bundle; expect(settled.on).toBe(true); @@ -580,6 +578,31 @@ describe("the main pass's draw bundles", () => { cells.dispose(); }); + it("re-records a mint's share of the main pass, not the whole of it", async () => { + // Machinefall's map-walk, 30-frame windows: for ~90 frames after the overlay drops the world is + // still streaming, and every key it mints re-records the one main group, so three walks ~200 + // draws in that frame (traversed p95 257–291 against 31–78 settled). A record is per group, so + // the cost of a mint is the size of the group it lands in, and the main pass must be split. + const { follow, renderer, world: cells } = await world({ bundles: true, rockInWest: true }); + cells.update(renderer, playerCamera([0, 1])); + await flushed(cells, renderer, playerCamera([0, 1])); + const groups = mainBundleGroups(cells); + expect(groups.length).toBeGreaterThan(1); + const before = new Map(groups.map((group) => [group, group.version])); + // Two cells east: `rock`'s keys leave the ring and others are dressed. + const at = cellCentre(3, 1); + follow.position.x = at.x; + follow.position.z = at.z; + cells.update(renderer, playerCamera([0, 1])); + await flushed(cells, renderer, playerCamera([0, 1])); + const touched = [...before].filter(([group, version]) => group.version !== version); + expect(touched.length).toBeGreaterThan(0); + // The mint re-recorded its own group; the rest of the main pass replayed what it had. + expect(touched.length).toBeLessThan(before.size); + expect(cells.stats().failures).toBe(0); + cells.dispose(); + }); + it("never writes `visible` on a bundled mesh, so a bundle is never re-recorded to hide one", async () => { const { renderer, world: cells } = await world({ bundles: true }); cells.update(renderer, playerCamera()); @@ -594,9 +617,9 @@ describe("the main pass's draw bundles", () => { for (const held of watched.values()) expect(held.count).toBe(0); // The bundle is still whole, and still the whole of the main pass. The meshes are the ones that // were watched, so "never written" is about the meshes that are actually drawing. - const group = bundleGroup(cells); - for (const mesh of meshes) expect(mesh.parent).toBe(group); - expect(group?.children.length).toBe(watched.size); + const groups = mainBundleGroups(cells); + for (const mesh of meshes) expect(groups).toContain(mesh.parent); + expect(groups.reduce((sum, group) => sum + group.children.length, 0)).toBe(watched.size); // And the stat counts every recorded object: the dressed main meshes plus one entry per chunk, // because a cell's own record holds whole chunks where every draw in them is recordable — and // the individual draws of a chunk that is not, one entry each. See `bundleSafe`. @@ -611,7 +634,7 @@ describe("the main pass's draw bundles", () => { await flushed(cells, renderer, playerCamera()); // No group, and the marker and the stats say so. - expect(bundleGroup(cells)).toBeUndefined(); + expect(mainBundleGroups(cells)).toEqual([]); expect(cells.stats().bundle).toEqual({ children: 0, on: false, reason: "option", records: 0 }); const dressed = mainKeys(cells); expect(dressed.length).toBeGreaterThan(0); @@ -725,6 +748,39 @@ describe("the main pass's draw bundles", () => { expect(lifted.parent instanceof BundleGroup).toBe(false); }); + it("asks a textured surface whether it can be recorded without reading its pixels", async () => { + // Machinefall's map-walk blocked its main thread for 22.9 s on the develop core: the + // framebuffer walk went through `Object.values` of everything a material reaches, so every + // texture's pixel array was enumerated one element at a time. A pixel array holds bytes, + // never a node, so a walk that reads one is all cost. The spy is an enumerable own property + // of the array, which only an element-by-element enumeration of the array itself reaches. + let pixelReads = 0; + const pixels = new Uint8Array(4 * 4 * 4); + Object.defineProperty(pixels, "spy", { + enumerable: true, + get: () => { + pixelReads += 1; + return 0; + }, + }); + const textured = new MeshStandardNodeMaterial(); + textured.map = new DataTexture(pixels, 4, 4); + const { renderer, world: cells } = await world({ + bundles: true, + chunkModel: (url) => standAtCellOf(url, chunkOf([textured])), + }); + cells.update(renderer, eastCamera()); + await flushed(cells, renderer, eastCamera()); + + const drawn = project(cells, eastCamera()); + expect(drawn.bundled.filter(isChunk).map((mesh) => (mesh as Mesh).material)).toContain( + textured, + ); + expect(pixelReads).toBe(0); + expect(cells.stats().failures).toBe(0); + cells.dispose(); + }); + it("leaves a main batch a record cannot replay on the per-object path, and records the rest", async () => { // The blank-frame bug the chunk side already guards, on the main pass: a record is encoded into // a bundle encoder, and a draw that needs the pass it was recorded in — an impostor's @@ -767,7 +823,7 @@ describe("the main pass's draw bundles", () => { // Its sibling is recorded, so the group is the whole of what a bundle can hold rather than a // world that gave up on recording. for (const mesh of recorded) expect(drawn.bundled).toContain(mesh); - expect(bundleGroup(cells)?.children.length).toBeGreaterThan(0); + expect(mainBundleGroups(cells).length).toBeGreaterThan(0); // And a refused draw keeps the coarse gate that hides it, because that is the CPU path's answer: // a record's render list is fixed when it was recorded and never reads `visible` again. const watched = watchVisible(refused); @@ -818,6 +874,107 @@ describe("the main pass's draw bundles", () => { cells.dispose(); }); + it("re-records a main shard in the same update that changed which meshes it holds", async () => { + // A record fixes its render list, so a mesh that joins a shard and is not followed by a re-record + // of that shard is not drawn until some later event happens to bump the same group. With one + // group that was masked: any key minted in the same load bumped it. With a group per shard the + // repair needs a bump in the *same* shard, so a join without one hides geometry for as long as + // that shard stays quiet — Machinefall's first capture lost its mid-field forest at the pose the + // overlay drops on, and a re-capture of the same build had it. + const { + follow, + renderer, + world: cells, + } = await world({ + bundles: true, + gpuScene: false, + rockInWest: true, + }); + const members = (group: BundleGroup): string => + group.children + .map((child) => `${child.name}#${child.id}`) + .sort() + .join(","); + const stale: string[] = []; + const check = (label: string, camera: PerspectiveCamera): void => { + const before = new Map( + mainBundleGroups(cells).map((group) => [group, [members(group), group.version] as const]), + ); + cells.update(renderer, camera); + for (const group of mainBundleGroups(cells)) { + const was = before.get(group); + const changed = was === undefined ? group.children.length > 0 : was[0] !== members(group); + const rerecorded = was === undefined ? group.version > 0 : group.version > was[1]; + if (changed && !rerecorded) stale.push(`${label}: ${group.children.length} meshes`); + } + }; + for (let index = 0; index < 8; index += 1) check("settle", playerCamera()); + await flushed(cells, renderer, playerCamera()); + for (let phase = 0; phase < 5; phase += 1) { + const at = cellCentre(phase % 2 === 0 ? 3 : 1, 1); + follow.position.x = at.x; + follow.position.z = at.z; + for (let index = 0; index < 12; index += 1) + check(`walk-${phase}`, phase % 2 === 0 ? eastCamera() : playerCamera()); + await flushed(cells, renderer, playerCamera()); + } + expect(stale).toEqual([]); + expect(cells.stats().failures).toBe(0); + cells.dispose(); + }); + + it("re-records a shard recorded during a compile once the compile has settled", async () => { + // Three encodes a draw only when its pipeline is ready, so a record cut while a `compileAsync` + // overlaps silently leaves the not-yet-ready draw out — and the replay never re-checks. A shard + // bumped during a compile must be re-recorded once the compile settles, or its geometry stays + // missing until something else happens to bump that same shard. + const { + follow, + renderer, + world: cells, + } = await world({ + bundles: true, + gpuScene: false, + rockInWest: true, + }); + cells.update(renderer, playerCamera([0, 1])); + await flushed(cells, renderer, playerCamera([0, 1])); + + // The stub with a settable compile state: `compiling` in flight and a compile that has started. + const busy: IRendererLike & { compiling: boolean; compileCount: number } = Object.assign( + Object.create(renderer) as IRendererLike, + { compileCount: 1, compiling: true }, + ); + + const before = new Map(mainBundleGroups(cells).map((group) => [group, group.version])); + // Walk east into a new cell with a compile in flight, so the shards the walk re-dresses are + // recorded during a busy renderer. + const at = cellCentre(3, 1); + follow.position.x = at.x; + follow.position.z = at.z; + cells.update(busy, playerCamera([0, 1])); + await flushed(cells, busy, playerCamera([0, 1])); + const bumped = [...before] + .filter(([group, version]) => group.version !== version) + .map(([group]) => group); + expect(bumped.length).toBeGreaterThan(0); + const during = new Map(bumped.map((group) => [group, group.version])); + + // The compile settles: the same `compileCount`, no longer `compiling`. Every shard the busy + // phase recorded has to be re-recorded, or the draw it left out is never drawn again. + busy.compiling = false; + for (let index = 0; index < 2; index += 1) cells.update(busy, playerCamera([0, 1])); + for (const group of bumped) expect(group.version).toBeGreaterThan(during.get(group) as number); + + // Settled once, not every frame: further resting frames change no shard's version. + const settled = new Map(mainBundleGroups(cells).map((group) => [group, group.version])); + for (let index = 0; index < 20; index += 1) cells.update(busy, playerCamera([0, 1])); + for (const group of mainBundleGroups(cells)) expect(group.version).toBe(settled.get(group)); + + expect(cells.stats().failures).toBe(0); + cells.dispose(); + }); + it("records by default, and the marker says the run asked for nothing else", async () => { // Phase 2 box 2: the default follows the measurement. AC-2 measured the walking `draw` span at // -5.0 ms against develop over 3 interleaved runs with bundles on, so a world that says nothing @@ -833,10 +990,10 @@ describe("the main pass's draw bundles", () => { expect(cells.stats().bundle.on).toBe(true); expect(cells.stats().bundle.reason).toBe("default"); expect(bundleMarker(lines)).toContain("TN_WORLD_BUNDLE on reason=default"); - // And the walk really is replaying: every main batch hangs in the one group. - const group = bundleGroup(cells); - expect(group).toBeDefined(); - for (const mesh of mainKeys(cells)) expect(mesh.parent).toBe(group); + // And the walk really is replaying: every main batch hangs in a main record shard. + const groups = mainBundleGroups(cells); + expect(groups.length).toBeGreaterThan(0); + for (const mesh of mainKeys(cells)) expect(groups).toContain(mesh.parent); cells.dispose(); }); @@ -858,7 +1015,7 @@ describe("the main pass's draw bundles", () => { expect(cells.stats().bundle.on).toBe(false); expect(cells.stats().bundle.reason).toBe("option"); expect(bundleMarker(lines)).toContain("TN_WORLD_BUNDLE off reason=option"); - expect(bundleGroup(cells)).toBeUndefined(); + expect(mainBundleGroups(cells)).toEqual([]); cells.dispose(); }); @@ -877,7 +1034,7 @@ describe("the main pass's draw bundles", () => { expect(cells.stats().bundle.on).toBe(false); expect(cells.stats().bundle.reason).toBe("launch"); expect(bundleMarker(lines)).toContain("TN_WORLD_BUNDLE off reason=launch"); - expect(bundleGroup(cells)).toBeUndefined(); + expect(mainBundleGroups(cells)).toEqual([]); cells.dispose(); }); @@ -935,7 +1092,9 @@ describe("the main pass's draw bundles", () => { expect(chunkBundles(cells)).toEqual([]); expect(chunkMeshes(cells)).toEqual([]); // And the stat gives the chunks back rather than counting records nothing owns any more. - expect(cells.stats().bundle.children).toBe(bundleGroup(cells)?.children.length ?? 0); + expect(cells.stats().bundle.children).toBe( + mainBundleGroups(cells).reduce((sum, group) => sum + group.children.length, 0), + ); expect(cells.stats().bundle.records).toBeGreaterThan(settled.records); cells.dispose(); }); diff --git a/packages/core/src/world-cells.ts b/packages/core/src/world-cells.ts index b45b5a2bf6..c695df804b 100644 --- a/packages/core/src/world-cells.ts +++ b/packages/core/src/world-cells.ts @@ -728,7 +728,7 @@ export interface IWorldCellsStats { }; /** * What the main pass's draw bundles are doing: `children` is how many objects are recorded — every - * GPU-dressed main mesh under the one `BundleGroup`, plus every chunk of every resident cell under + * recorded main mesh under one of the main record's shards, plus every chunk of every resident cell under * that cell's own group — and `records` is how many times those groups have been re-recorded since * the world loaded. * @@ -1006,7 +1006,7 @@ class SharedBatch { */ gpuCount = 0; /** - * Whether this batch's mesh is parented under the world's one `BundleGroup`. Set by + * Whether this batch's mesh is parented under one of the world's main record shards. Set by * `WorldCells#bundleIn` and cleared by `#bundleOut`. * * It is a field rather than `mesh.parent` because a bundled mesh's `visible` is never written @@ -2866,6 +2866,26 @@ function mergedBytes(merged: readonly Mesh[], kept: ReadonlySet): number { return bytes; } +/** + * How many groups the main record is split into. A record walks every draw in its group, so a mint + * on a streaming walk costs the draws of every shard it touches: Machinefall's ~200 recorded main + * draws re-recorded as one group put ~290 traversed draws in a frame. Eight shards still read 145, + * because one asset mints several keys (levels, parts) at once and they land in two or three shards; + * thirty-two keep a frame that touches several to a few dozen draws. A replay is one + * `executeBundles` per group, so a shard costs far less than the draws it stops re-walking. + */ +const MAIN_BUNDLE_SHARDS = 32; + +/** The shard a key lands in, from its name (FNV-1a), so a key keeps its group for its whole life. */ +function mainShard(key: string): number { + let hash = 0x811c9dc5; + for (let index = 0; index < key.length; index += 1) { + hash ^= key.charCodeAt(index); + hash = Math.imul(hash, 0x01000193); + } + return (hash >>> 0) % MAIN_BUNDLE_SHARDS; +} + /** * The value `NodeUpdateType.RENDER` has, read as text so this file needs no import for it: a node * whose `updateBefore` runs on every render is a node whose work happens inside the draw that @@ -2903,7 +2923,9 @@ function samplesFramebuffer(node: unknown, seen: Set): boolean { ) return true; for (const value of Object.values(node)) { - if (value === null || typeof value !== "object") continue; + // A pixel or vertex array holds numbers, never a node: enumerating a texture's mips element by + // element held Machinefall's main thread for 22.9 s. + if (value === null || typeof value !== "object" || ArrayBuffer.isView(value)) continue; if (samplesFramebuffer(value, seen)) return true; } return false; @@ -3920,15 +3942,26 @@ export class WorldCells extends Group implements IComputeDriven { /** Resident placements per canonical asset, which is the capacity a key of that asset needs. */ readonly #gpuResident = new Map(); /** - * The one `BundleGroup` every GPU-dressed main batch mesh is parented under, and the number of - * times it has been re-recorded. Minted by the first dress that bundles, so a world that never - * dresses a mesh has no group to pay for. + * The `BundleGroup`s every recorded main batch mesh is parented under, one per shard of + * {@link MAIN_BUNDLE_SHARDS}, and the number of times one has been re-recorded. A shard is minted by + * the first mesh that lands in it, so a world that never dresses a mesh has no group to pay for. * * A re-record is a structural change and nothing else: see `IWorldCellsStats.bundle` and - * {@link #bumpBundle}. Streaming, culling and LOD are GPU-side answers and never move it. + * {@link #bumpBundle}. Streaming, culling and LOD are GPU-side answers and never move it. A record + * walks every draw its group holds, so a mint pays for its own shard and not the whole main pass. */ - #bundle: BundleGroup | undefined; + readonly #bundles = new Map(); #bundleRecords = 0; + /** + * Shards re-recorded while a compile was in flight, so they are re-recorded again once it settles. + * Three encodes a draw only when its pipeline is ready, so a record cut during a `compileAsync` + * silently leaves a not-yet-ready draw out, and the replay never re-checks it. Bumped from + * {@link #bumpBundle} while {@link #compileBusy}, flushed at the top of the update that sees the + * compile settle. + */ + #compileSeen = 0; + #compileBusy = false; + readonly #recordedWhileBusy = new Set(); /** * One `BundleGroup` per resident cell that has chunks attached, keyed by cell key, each with the * main-cull cluster whose answer decides whether it is in the frame. Minted by the first chunk a @@ -4288,6 +4321,18 @@ export class WorldCells extends Group implements IComputeDriven { // arrives, which can be a frame or a loading screen after the frame that gave us these. if (renderer !== undefined) this.#renderer = renderer; if (camera !== undefined) this.#camera = camera; + // A bundle recorded while a compile was in flight left out every draw whose pipeline was not + // ready, and the replay never re-checks; re-record those shards now that it has settled. Set + // `#compileBusy` first, so this flush's own bumps are not added back to the set. + const compiles = renderer?.compileCount ?? 0; + const busy = renderer?.compiling === true || compiles !== this.#compileSeen; + this.#compileSeen = compiles; + this.#compileBusy = busy; + if (!busy && this.#recordedWhileBusy.size > 0) { + const recorded = [...this.#recordedWhileBusy]; + this.#recordedWhileBusy.clear(); + for (const group of recorded) if (group.parent !== null) this.#bumpBundle(group); + } // The adaptive LOD bias, before the dispatch below reads the gates it scales. this.#adaptLodBias(renderer); // The first frame that hands over a renderer is the only one that can answer whether this @@ -5087,7 +5132,7 @@ export class WorldCells extends Group implements IComputeDriven { return { admission: { ...this.#admission }, bundle: { - children: (this.#bundle?.children.length ?? 0) + this.#chunkBundleChildren, + children: this.#mainBundleChildren() + this.#chunkBundleChildren, on: this.#bundlesWanted && (this.#gpuScene.on || this.#chunkBundleChildren > 0), reason: this.#bundleReason, records: this.#bundleRecords, @@ -5160,6 +5205,7 @@ export class WorldCells extends Group implements IComputeDriven { this.#proxyMaterials.clear(); this.#renderer = undefined; this.#camera = undefined; + this.#recordedWhileBusy.clear(); // After every asset's release above, so no record is still held by a surface; disposes the cached // atlases and aborts any bake still in flight. this.#drainImpostors(); @@ -6340,8 +6386,7 @@ export class WorldCells extends Group implements IComputeDriven { if (this.#bundlesWanted === false) return; shared.bundlable = bundleSafe(shared.mesh); if (shared.bundlable === false) return; - this.#bundle ??= this.#newBundle(); - const group = this.#bundle as BundleGroup; + const group = this.#shardOf(shared); if (shared.mesh.parent !== group) group.add(shared.mesh); shared.bundled = true; shared.bundledCount = shared.mesh.count; @@ -6368,15 +6413,17 @@ export class WorldCells extends Group implements IComputeDriven { shared.mesh.userData.tnBundled = false; // Back on the per-object path, so the mesh's own whole-mesh test is its gate again. shared.mesh.frustumCulled = true; - if (shared.mesh.parent === this.#bundle) (this.#bundle as BundleGroup).remove(shared.mesh); - if (this.#bundle !== undefined) this.#bumpBundle(); + const group = this.#bundles.get(mainShard(shared.mesh.name)); + if (group === undefined) return; + if (shared.mesh.parent === group) group.remove(shared.mesh); + this.#bumpBundle(group); } /** One re-record, counted: `BundleGroup.needsUpdate` is a version bump, and nothing else. */ - #bumpBundle(group: BundleGroup | undefined = this.#bundle): void { - if (group === undefined) return; + #bumpBundle(group: BundleGroup): void { this.#bundleRecords += 1; group.needsUpdate = true; + if (this.#compileBusy) this.#recordedWhileBusy.add(group); } /** @@ -6407,11 +6454,11 @@ export class WorldCells extends Group implements IComputeDriven { // without moving its length, so the epoch answer and the count are both read here. if (outcome === "settled" && shared.mesh.count === shared.bundledCount) return; shared.bundledCount = shared.mesh.count; - this.#bumpBundle(); + this.#bumpBundle(this.#shardOf(shared)); } /** - * Where a dressed main batch's mesh belongs: the one bundle group, or the world itself when there is + * Where a dressed main batch's mesh belongs: its key's main record shard, or the world itself when there is * none — a world with bundles off, a draw no record can replay, and the very first dress of a batch * the mint has not attached yet. * @@ -6420,7 +6467,7 @@ export class WorldCells extends Group implements IComputeDriven { */ #bundleHome(shared: SharedBatch): Object3D | null { if (this.#bundlesWanted === false || shared.bundlable === false) return this; - return this.#bundle ?? null; + return this.#bundles.get(mainShard(shared.mesh.name)) ?? null; } /** @@ -6507,14 +6554,26 @@ export class WorldCells extends Group implements IComputeDriven { this.#bumpBundle(entry.group); } - /** The one group, as a child of the world so it is projected with everything else. */ - #newBundle(): BundleGroup { - const group = new BundleGroup(); - group.name = "world-main-bundles"; - this.add(group); + /** The main-record shard a batch's key lands in, minted as a child of the world on first use. */ + #shardOf(shared: SharedBatch): BundleGroup { + const shard = mainShard(shared.mesh.name); + let group = this.#bundles.get(shard); + if (group === undefined) { + group = new BundleGroup(); + group.name = "world-main-bundles"; + this.add(group); + this.#bundles.set(shard, group); + } return group; } + /** The recorded main batches, over every shard. */ + #mainBundleChildren(): number { + let children = 0; + for (const group of this.#bundles.values()) children += group.children.length; + return children; + } + /** * Where a freshly bound or grown main batch's mesh goes: the bundle group when it has been dressed * into one, the world otherwise. Three re-parents on `add`, so a bundled mesh has to be asked for @@ -6524,14 +6583,14 @@ export class WorldCells extends Group implements IComputeDriven { // A grow replaces the mesh object, so the marker `#bundleIn` set has to follow the batch's own // flag onto it. See `bundled`. shared.mesh.userData.tnBundled = shared.bundled; - if (shared.bundled === true) (this.#bundle as BundleGroup).add(shared.mesh); + if (shared.bundled === true) this.#shardOf(shared).add(shared.mesh); else this.add(shared.mesh); // A record holds the object it was cut from, so a mesh that arrived by a rebind or a grow is not // the one the last record named: one re-record, and the record counts this mesh from here. if (shared.bundled === true) { shared.mesh.frustumCulled = false; shared.bundledCount = shared.mesh.count; - this.#bumpBundle(); + this.#bumpBundle(this.#shardOf(shared)); } } @@ -6672,7 +6731,7 @@ export class WorldCells extends Group implements IComputeDriven { // is one of the three, which is why this is the only place a re-record happens — the fast path // above is every other frame of the walk. this.#bundleIn(shared); - this.#bumpBundle(); + if (shared.bundled === true) this.#bumpBundle(this.#shardOf(shared)); // The shadow twin of this key, minted and dressed on the same pass that named the main one: a // level that draws keys finds its mesh in the same frame the main pass found its buffer, and a // regrown twin is re-dressed here rather than left on the buffer the dispatch replaced. diff --git a/packages/runtime-native/src/webgpu/bindings_commands.cpp b/packages/runtime-native/src/webgpu/bindings_commands.cpp index 91beeaeaa4..3c6449b416 100644 --- a/packages/runtime-native/src/webgpu/bindings_commands.cpp +++ b/packages/runtime-native/src/webgpu/bindings_commands.cpp @@ -60,6 +60,26 @@ static js::JSValueHandle handleGpuRenderBundleEncoderDrawIndexed(BindingsState* return state->engine->newUndefined(); } +// renderBundleEncoder.drawIndexedIndirect(indirectBuffer, indirectOffset): the GPU scene records its +// dispatch-sized draws into bundles, so a bundle without it throws on every recorded world batch. +static js::JSValueHandle handleGpuRenderBundleEncoderDrawIndexedIndirect(BindingsState* state, WGPURenderBundleEncoder capturedEncoder, const std::vector& args) { + if (args.size() < 2) return state->engine->newUndefined(); + WGPUBuffer indirectBuffer = (WGPUBuffer)state->engine->getPrivateData(args[0]); + uint64_t indirectOffset = (uint64_t)state->engine->toNumber(args[1]); + if (capturedEncoder && indirectBuffer) + wgpuRenderBundleEncoderDrawIndexedIndirect(capturedEncoder, indirectBuffer, indirectOffset); + return state->engine->newUndefined(); +} + +static js::JSValueHandle handleGpuRenderBundleEncoderDrawIndirect(BindingsState* state, WGPURenderBundleEncoder capturedEncoder, const std::vector& args) { + if (args.size() < 2) return state->engine->newUndefined(); + WGPUBuffer indirectBuffer = (WGPUBuffer)state->engine->getPrivateData(args[0]); + uint64_t indirectOffset = (uint64_t)state->engine->toNumber(args[1]); + if (capturedEncoder && indirectBuffer) + wgpuRenderBundleEncoderDrawIndirect(capturedEncoder, indirectBuffer, indirectOffset); + return state->engine->newUndefined(); +} + static js::JSValueHandle handleGpuRenderBundleEncoderDraw(BindingsState* state, WGPURenderBundleEncoder capturedEncoder, const std::vector& args) { if (args.empty()) return state->engine->newUndefined(); uint32_t vertexCount = (uint32_t)state->engine->toNumber(args[0]); @@ -185,6 +205,14 @@ js::JSValueHandle handleGpuDeviceCreateRenderBundleEncoder(BindingsState* state, {"GPURenderBundleEncoder", "drawIndexed", 0, nullptr, makeCapturedHandler(capturedEncoder, &handleGpuRenderBundleEncoderDrawIndexed) , jsEncoder}, + // renderBundleEncoder.drawIndirect(indirectBuffer, indirectOffset) + {"GPURenderBundleEncoder", "drawIndirect", 0, nullptr, + makeCapturedHandler(capturedEncoder, &handleGpuRenderBundleEncoderDrawIndirect) + , jsEncoder}, + // renderBundleEncoder.drawIndexedIndirect(indirectBuffer, indirectOffset) + {"GPURenderBundleEncoder", "drawIndexedIndirect", 0, nullptr, + makeCapturedHandler(capturedEncoder, &handleGpuRenderBundleEncoderDrawIndexedIndirect) + , jsEncoder}, // renderBundleEncoder.finish(descriptor?) {"GPURenderBundleEncoder", "finish", 0, nullptr, makeCapturedHandler(capturedEncoder, &handleGpuRenderBundleEncoderFinish) diff --git a/packages/runtime-native/tests/webgpu_bindings_reentrancy_test.cpp b/packages/runtime-native/tests/webgpu_bindings_reentrancy_test.cpp index 158feb391d..113a54e713 100644 --- a/packages/runtime-native/tests/webgpu_bindings_reentrancy_test.cpp +++ b/packages/runtime-native/tests/webgpu_bindings_reentrancy_test.cpp @@ -1716,7 +1716,7 @@ bool exercisePublishedWebGPUObjects(mystral::Runtime& runtime) { ]); requireMethods("GPURenderBundleEncoder", bundle, [ "setPipeline", "setVertexBuffer", "setIndexBuffer", "setBindGroup", "draw", - "drawIndexed", "finish", + "drawIndexed", "drawIndirect", "drawIndexedIndirect", "finish", ]); requireMethods("GPUCanvasContext", canvasContext, [ "configure", "unconfigure", "getCurrentTexture",