feat(native-engine): the JS-free native engine batch (PRD-497…535)

.agents/skills/native-performance-loop/SKILL.md

@@ -114,6 +114,30 @@ Confirm apparent wins with a fresh replay/seed or held-out scene before final de
 the original baseline after three experiments to detect drift. Avoid retesting known losers unless
 new evidence changes the hypothesis.
 
+## Perry vs V8 workload
+
+`pnpm bench:perry --project <template> --scenario <playtest.json> --perry <linked-binary> --v8-bundle <dir> --open`
+runs one authored scenario twice on the shared native player (Perry then V8) and writes
+`artifacts/perf-loop/<iso>/report.html` plus `index.html`. Any metric slower than V8 is a MISS:
+
+| Metric | Target (Perry / V8) |
+| --- | ---: |
+| First frame | <= 1.0x |
+| Game JS p50 | <= 0.5x |
+| Total CPU frame p50 | <= 0.68x |
+| Launch to ready | <= 0.33x |
+
+Pass `--incumbent <last accepted run dir>` to gate the candidate: any metric whose Perry / V8 ratio
+worsens past its margin is REGRESSED and the bench exits 3 = REGRESSION: never commit the candidate, fix the cause and bench again.
+
+Runs above 0.75 x cores load (`--max-load` overrides) are refused after a 5 minute wait: serialise builds and benches.
+
+Never report fps on Xvfb. Measurements run in the parent session; Codex sandboxes cannot start
+Xvfb. A Perry relink takes about 17 minutes; append `--perf` for profiling, excluded from the verdict.
+The V8 player needs `TN_ENABLE_NATIVE_PHYSICS=ON`, and its `--v8-bundle` must pack the game's assets:
+build it with `bundle-native-engine.mjs --assets <dir>` whose `public/` holds the assets that
+`TN_ASSET_ROOT` serves to Perry, or V8 fails `TN_NATIVE_ASSET_MISSING`.
+
 ## Guardrails and stopping
 
 Do not buy speed by altering gameplay, appearance, time scale, measurement boundaries, validation,

.agents/skills/profiling/SKILL.md

@@ -26,6 +26,7 @@ is not presented frame time, and a parity pass is not a performance result.
 | V8 CPU profile | `TN_JS_CPU_PROFILE=1 <host>` (window via `TN_JS_CPU_PROFILE_START_FRAME`); `--cpu-prof <file>` on the host (`mystral run <script> --cpu-prof=<file>`) or a playtest (`node packages/playtest/dist/runner/cli.js <scenario> --cpu-prof <file>`) | Attribute JS self-time; open a DevTools profile | Host built with the profiler (desktop default); `--cpu-prof` is browser/desktop only (device refuses `TN_PLAYTEST_CPU_PROFILE_UNSUPPORTED`) | `TN_JS_CPU_PROFILE:` / `_TOTAL:` marker lines; a Chrome DevTools `.cpuprofile` | Perturbs the frame; diagnose only, never a timing verdict |
 | Profiled-host counters | Build host `-DTN_ANDROID_JS_PROFILE=ON`, run `node packages/runtime-native/scripts/measure-android-js-engine.mjs --device <serial>` or `measure-desktop-frame-pair.mjs --control <bin> --candidate <bin> --bundle <dir> --output <dir> --runs <n>` | Per-frame JS↔host binding/command cost | Profiled host; physical Android for acceptance | `TN_ANDROID_JS_NATIVE:` and `TN_BRIDGE_BY_NAME:` markers | Profiled build inflates absolutes; emulator proves plumbing only. Recipe: `packages/runtime-native/docs/G5-profiling.md` |
 | Linux OS `perf` (optional) | `perf stat -p <host-pid>`; `perf record -e cpu-clock:u --call-graph dwarf -p <host-pid>` then `perf report`; start the host with `packages/runtime-native/build/tn-linux/mystral run <script>` (or under `perf record -- …`) | Native C/C++ CPU attribution the V8 `.cpuprofile` cannot name | Linux; a kernel-matching `perf` from the OS package (`linux-tools`/`linux-perf`), an unstripped host for symbols; unprivileged sampling can be blocked by `perf_event_paranoid` (an OS/operator setting) | `perf stat` counters; `perf.data` + `perf report` call tree | Optional system executable; no bundled binary, wrapper, automatic install or sysctl change. Resolves native frames only — V8/JIT frames stay unnamed, so pair with `--cpu-prof`; no shipped flamegraph or JIT-symbol helper |
+| Wasm-engine page vs three.js | `sh scripts/xvfb.sh pnpm profile:wasm-page -- --url <wasm page> [--control <three.js page>] [--calls] [--gpu-calls] [--gpu-time] [--gpu-passes] [--cpu-work] [--save-profile <file>] [--seconds 5] [--json]` | A game is slower on the Wasm engine than on three.js; find where its frame goes | Served pages (`vite preview`), hardware WebGPU (refuses software); Wasm function names need the web host configured with `-DTN_WEB_PROFILING_FUNCS=ON`, JS names an unminified build | rAF p50/p95, CPU split into engine Wasm / JS / WebGPU calls with top inclusive functions, `--calls` = JS->Wasm engine calls per frame by class and member, `--gpu-calls` = WebGPU calls per frame, `--gpu-time` = the engine's GPU time per frame split into shadow, scene, post and output (timestamp-query), `--gpu-passes` = GPU time of every pass on either page by target size and format, `--cpu-work` = main-thread ms per frame less the getCurrentTexture/submit waits (what a faster display still pays), `--save-profile` = the raw profile for DevTools, top self and inclusive functions, subject/control ratio | A blank WebGPU page caps a private-Xvfb lane near 640 fps, so a frame-rate ratio past ~1.7x vs a 370 fps control is the lane's ceiling: compare `--cpu-work` and `--gpu-passes` there. Diagnosis only: shared-GPU noise moves absolutes, so read the ratio on one lane; hold the fix with a counter gate (`render_database_test steady_state`, wasm-engine-boot `steady*` probes), never a timing gate |
 | Chromium JS CPU profile | `pnpm profile:native-cpu` (`:fox` = `--visual-evidence fox-scale --allow-software`) | Isolate shared JS cost across scenarios in the browser | Playwright Chromium; `--output-dir` (default `artifacts/native-cpu-profile`) | Samples, stage/render advisor, optional PNGs | Chromium only; proves nothing about the native host |
 | Android cold start | `node packages/runtime-native/scripts/measure-cold-start.mjs` | Attribute launch time per segment | Physical Android (emulator exits `TN_COLD_START_EMULATOR_BLOCKED`) | `TN_COLD_START:` segment stamps, median/p95 | Emulator proof is plumbing only. Record: `packages/runtime-native/docs/G5-profiling.md` |
 

.github/workflows/ci.yml

@@ -384,6 +384,11 @@ jobs:
           node-version: 20
           cache: pnpm
       - run: pnpm install --frozen-lockfile
+      # The compatibility goldens the native drivers are judged against, rechecked against three.
+      # No render fixtures: their goldens hold one GPU's frame hash and adapter, so the full check
+      # stays a local gate. This run needs no GPU and reports how many renders it skipped.
+      - name: Check the non-render compatibility goldens against three
+        run: pnpm --workspace-root exec tsx packages/three-native/tests/compatibility/run-reference.ts --check --no-renders
       - uses: ./.github/actions/playwright-chromium
       # The runtime-native contract tests execute real binaries and refuse to skip when one is
       # missing — an unbuilt executable is unexecuted, and a silent skip would read as a pass. CI
@@ -506,6 +511,45 @@ jobs:
         if: always()
         shell: bash
         run: ccache --show-stats --verbose
+      # PRD-499 gate E. The engine targets configure with no scripting at all and run headless
+      # under ASan/UBSan; inside the V8 build their link graph must still reach no JS engine; and
+      # the driver binary is inspected for VM symbols, VM libraries and embedded scripts.
+      - name: Native engine gate E
+        working-directory: packages/runtime-native
+        run: |
+          set -euo pipefail
+          cmake --preset tn-linux -B build/tn-linux-engine-asan -DCMAKE_BUILD_TYPE=RelWithDebInfo \
+            -DTN_ENGINE_ONLY=ON -DTN_ENGINE_SANITIZE=ON \
+            -DMYSTRAL_USE_V8=OFF -DMYSTRAL_USE_QUICKJS=OFF -DMYSTRAL_USE_DAWN=ON
+          cmake --build build/tn-linux-engine-asan --target tn-native-engine-gate-e --target tn-native-engine-tests --parallel "$(nproc)"
+          ctest --test-dir build/tn-linux-engine-asan -L native-sanitizer --no-tests=error --output-on-failure
+          cmake --build build/tn-linux --target tn-native-engine-gate-e --target tn-native-engine-tests --parallel "$(nproc)"
+          ctest --test-dir build/tn-linux -L native-engine --output-on-failure
+          node scripts/inspect-js-free.mjs --binary build/tn-linux/tn-native-engine-gate-e \
+            --manifest ../../artifacts/native-engine/gate-e.json --target tn-native-engine-gate-e --capability gate-e
+      # PRD-499, owner decision 4: the portable engine core builds and passes its tests as Wasm
+      # under node, so it stays Wasm-safe from day one. emsdk is pinned by tag and SDK version.
+      - name: Native engine core as Wasm
+        working-directory: packages/runtime-native
+        shell: bash
+        run: |
+          set -euo pipefail
+          git clone --depth 1 --branch 6.0.11 https://github.com/emscripten-core/emsdk.git "$RUNNER_TEMP/emsdk"
+          "$RUNNER_TEMP/emsdk/emsdk" install 6.0.11
+          "$RUNNER_TEMP/emsdk/emsdk" activate 6.0.11
+          source "$RUNNER_TEMP/emsdk/emsdk_env.sh"
+          emcmake cmake -S . -B build/wasm-check -G Ninja -DCMAKE_BUILD_TYPE=Release
+          cmake --build build/wasm-check --target tn-native-engine-core-tests --parallel "$(nproc)"
+          ctest --test-dir build/wasm-check -L native-engine --output-on-failure
+          # PRD-533's web arms: the browser engine build and the pinned Perry toolchain are what
+          # web-engine-bench.spec.ts and perry-packed.spec.ts load, so they run here and nowhere
+          # else (TN_WEB_ENGINE_SPECS); wasm-dis comes from emsdk's binaryen.
+          cmake --preset wasm-browser
+          cmake --build build/wasm-browser --target tn-native-engine-wasm-browser --parallel "$(nproc)"
+          node ../../tools/native-typescript/provision.mjs
+          export PATH="$PATH:$EMSDK/upstream/bin"
+          (cd ../.. && TN_WEB_ENGINE_SPECS=1 pnpm exec vitest run \
+            scripts/__tests__/web-engine-bench.spec.ts scripts/__tests__/perry-packed.spec.ts)
       - name: Report the compiler cache hit rate
         if: always()
         shell: bash
@@ -531,7 +575,19 @@ jobs:
       # `native:physics:parity` ride along inside that script, which is the package's whole gate.
       - run: pnpm --filter @threenative/runtime-native test
         env:
+          TN_GATE_E_BINARY: ${{ github.workspace }}/packages/runtime-native/build/tn-linux/tn-native-engine-gate-e
           TN_REQUIRE_ANDROID_CANVAS_SKIA_FIXTURE: "1"
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: native-engine-gate-e-evidence
+          path: |
+            artifacts/native-engine/gate-e.json
+            packages/runtime-native/build/tn-linux/native-engine-target-graph.txt
+            packages/runtime-native/build/tn-linux/Testing/Temporary/LastTest.log
+            packages/runtime-native/build/tn-linux-engine-asan/native-engine-target-graph.txt
+            packages/runtime-native/build/tn-linux-engine-asan/Testing/Temporary/LastTest.log
+          if-no-files-found: error
       - name: Collect bounded native Linux performance evidence
         if: always()
         shell: bash

.github/workflows/native-release.yml

@@ -24,6 +24,11 @@ on:
     # to four files, one of them touched by ordinary runtime-native PRs.
     types: [opened, synchronize, reopened, labeled]
   workflow_dispatch:
+    inputs:
+      native_engine:
+        description: Qualify only the JS-free Linux, Windows and macOS native-engine players (no release)
+        type: boolean
+        default: false
 
 permissions:
   contents: read
@@ -126,6 +131,7 @@ jobs:
     # other PR. Every proof job below needs `gates`, so refusing here spends nothing.
     if: >-
       ${{ !cancelled() &&
+      !(github.event_name == 'workflow_dispatch' && github.event.inputs && github.event.inputs.native_engine == 'true') &&
       (github.event_name != 'pull_request' ||
       github.event.pull_request.base.ref == 'main' ||
       contains(github.event.pull_request.labels.*.name, 'release-proof')) &&
@@ -558,6 +564,164 @@ jobs:
           path: packages/runtime-native/artifacts/
           if-no-files-found: error
 
+  native-engine:
+    name: native-engine-${{ matrix.platform }}
+    # Independent of release gates: dispatch the selected branch, without a tag or main CI run.
+    if: github.event_name == 'workflow_dispatch' && inputs.native_engine
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - platform: linux
+            key: linux-x64
+            runner: ubuntu-24.04
+            preset: tn-linux
+            executable: tn-native-engine-player
+            dawn_dependency: dawn
+          - platform: macos
+            key: darwin-arm64
+            runner: macos-15
+            preset: tn-macos
+            executable: tn-native-engine-player
+            dawn_dependency: dawn
+          - platform: windows
+            key: win32-x64
+            runner: windows-2025
+            preset: tn-windows
+            executable: tn-native-engine-player.exe
+            dawn_dependency: skia-win-static
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 60
+    env:
+      TN_PLAYTEST_ALLOW_SOFTWARE: "1"
+      NATIVE_ENGINE_PRESET: ${{ matrix.preset }}
+      NATIVE_ENGINE_PLAYER: ${{ github.workspace }}/packages/runtime-native/build/${{ matrix.preset }}/${{ matrix.executable }}
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/cache@v4
+        with:
+          path: packages/runtime-native/third_party
+          key: native-third-party-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('packages/runtime-native/scripts/download-deps.mjs', 'packages/runtime-native/scripts/package-android.mjs') }}
+      - uses: pnpm/action-setup@v6
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 20
+          cache: pnpm
+      - uses: actions/setup-python@v7
+        with:
+          python-version: "3.12"
+      - run: pnpm install --frozen-lockfile
+      - name: Build the existing playtest runner
+        run: pnpm --filter @threenative/playtest build
+      - name: Install Linux JS-free desktop build and display dependencies
+        if: runner.os == 'Linux'
+        run: >-
+          sudo apt-get update && sudo apt-get install -y
+          cmake ninja-build binutils pkg-config
+          libasound2-dev libpulse-dev libudev-dev libx11-dev libxext-dev libxrandr-dev
+          libxcursor-dev libxi-dev libxfixes-dev libxss-dev libxkbcommon-dev
+          libvulkan1 mesa-vulkan-drivers vulkan-tools xvfb xauth xcompmgr
+      - name: Provision the host build's pinned SDL, stb and Dawn dependencies
+        shell: bash
+        env:
+          DAWN_DEPENDENCY: ${{ matrix.dawn_dependency }}
+        run: |
+          set -euo pipefail
+          cd packages/runtime-native
+          node scripts/download-deps.mjs --only sdl3
+          node scripts/download-deps.mjs --only stb
+          node scripts/download-deps.mjs --only "$DAWN_DEPENDENCY"
+      - name: Build the JS-free player with MSVC and inspect PE imports
+        if: runner.os == 'Windows'
+        shell: cmd
+        run: |
+          for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do set "VSINSTALL=%%i"
+          if not defined VSINSTALL exit /b 1
+          call "%VSINSTALL%\Common7\Tools\VsDevCmd.bat" -arch=x64 -host_arch=x64
+          if errorlevel 1 exit /b 1
+          set "CC=cl"
+          set "CXX=cl"
+          where cl
+          if errorlevel 1 exit /b 1
+          cd packages\runtime-native
+          cmake --preset %NATIVE_ENGINE_PRESET% -DTN_ENGINE_ONLY=ON -DMYSTRAL_USE_V8=OFF -DMYSTRAL_USE_QUICKJS=OFF
+          if errorlevel 1 exit /b 1
+          cmake --build --preset %NATIVE_ENGINE_PRESET% --target tn-native-engine-player --parallel
+          if errorlevel 1 exit /b 1
+          cd ..\..
+          node packages/runtime-native/scripts/inspect-js-free.mjs --binary "%NATIVE_ENGINE_PLAYER%" --manifest packages/runtime-native/artifacts/native-engine/js-free.json
+      - name: Build the JS-free player on Linux or macOS and inspect ELF or Mach-O
+        if: runner.os != 'Windows'
+        shell: bash
+        env:
+          CC: clang
+          CXX: clang++
+        run: |
+          set -euo pipefail
+          cd packages/runtime-native
+          cmake --preset "$NATIVE_ENGINE_PRESET" -DTN_ENGINE_ONLY=ON -DMYSTRAL_USE_V8=OFF -DMYSTRAL_USE_QUICKJS=OFF
+          cmake --build --preset "$NATIVE_ENGINE_PRESET" --target tn-native-engine-player --parallel
+          cd ../..
+          node packages/runtime-native/scripts/inspect-js-free.mjs --binary "$NATIVE_ENGINE_PLAYER" --manifest packages/runtime-native/artifacts/native-engine/js-free.json
+      - name: Run inspect, input and capture journeys on the built player
+        shell: bash
+        run: |
+          set -euo pipefail
+          for journey in inspect input capture; do
+            artifacts="packages/runtime-native/artifacts/native-engine/$journey"
+            mkdir -p "$artifacts"
+            sh scripts/xvfb.sh node packages/playtest/dist/runner/cli.js "packages/runtime-native/tests/native-engine/playtests/native-engine-$journey.playtest.json" \
+              --target desktop --executable "$NATIVE_ENGINE_PLAYER" --artifacts "$artifacts" \
+              > "$artifacts/runner.log" 2>&1 && status=0 || status=$?
+            cat "$artifacts/runner.log"
+            test "$status" -eq 0
+          done
+      - name: Record the Linux adapter (software is acceptable)
+        if: always() && runner.os == 'Linux'
+        shell: bash
+        run: |
+          set -euo pipefail
+          grep -hE '\[WebGPU\] (Adapter|Vendor|Backend):' packages/runtime-native/artifacts/native-engine/*/console.json \
+            | tee packages/runtime-native/artifacts/native-engine/linux-adapter.txt
+          grep -F '[WebGPU] Adapter:' packages/runtime-native/artifacts/native-engine/linux-adapter.txt
+          cat packages/runtime-native/artifacts/native-engine/linux-adapter.txt >> "$GITHUB_STEP_SUMMARY"
+      - name: Record the Windows adapter (D3D12 WARP is acceptable)
+        if: always() && runner.os == 'Windows'
+        shell: bash
+        run: |
+          set -euo pipefail
+          grep -hE '\[WebGPU\] (Adapter|Vendor|Backend):' packages/runtime-native/artifacts/native-engine/*/console.json \
+            | tee packages/runtime-native/artifacts/native-engine/windows-adapter.txt
+          grep -F '[WebGPU] Adapter:' packages/runtime-native/artifacts/native-engine/windows-adapter.txt
+          cat packages/runtime-native/artifacts/native-engine/windows-adapter.txt >> "$GITHUB_STEP_SUMMARY"
+      - name: Stage the inspected and verified player
+        shell: bash
+        env:
+          NATIVE_ENGINE_KEY: ${{ matrix.key }}
+        run: |
+          node --input-type=module <<'NODE'
+          import { copyFileSync, existsSync, mkdirSync, statSync } from "node:fs";
+          import { basename, join } from "node:path";
+          const source = process.env.NATIVE_ENGINE_PLAYER;
+          const output = join("release", `native-engine-${process.env.NATIVE_ENGINE_KEY}`);
+          mkdirSync(output, { recursive: true });
+          copyFileSync(source, join(output, basename(source)));
+          if (statSync(source).size === 0) throw new Error(`${source} is empty.`);
+          const map = source.replace(/\.exe$/i, ".map");
+          if (process.platform === "win32" && existsSync(map)) copyFileSync(map, join(output, basename(map)));
+          NODE
+      - uses: actions/upload-artifact@v7
+        with:
+          name: native-engine-player-${{ matrix.key }}
+          path: release/native-engine-${{ matrix.key }}/
+          if-no-files-found: error
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: evidence-native-engine-${{ matrix.key }}
+          path: packages/runtime-native/artifacts/native-engine/
+          if-no-files-found: error
+
   build-android:
     needs: [validate-tag, gates]
     if: >-