engine: correct LTM scoring and split wasm functions - #1063
Conversation
Freeze lagged co-inputs at their prior evaluation and use exact zero-delta guards so attribution remains consistent across time and unit scales. Require reported loops to reach half of their normalization mass before labeling a dominant period, including when the report cap omits loops. Add production-pipeline regressions and align the LTM references and always-on design constraints with the audited semantics.
Large LTM models can exceed V8's per-function byte limit even when the simulation is otherwise valid. Partition encoded phase programs at points where runtime evaluation state is no longer live, preserving instruction order, module parameters, and stable phase entry points. Check every final body against the host limit, including indivisible expressions and drivers. Exercise partitioning with small production-compiled fixtures across integration methods, modules, snapshots, arrays, and special stocks, plus an opt-in Node benchmark for C-LEARN with LTM enabled.
The project already maintains an incremental compiler database, but the WASM compilation endpoint rebuilt it on every request. Compile through the shared special-stock dispatch under the established lock order, then emit WASM from the immutable compiled snapshot after releasing the locks. Restore per-call discovery mode and latch LTM diagnostics like the VM path. Verify actual query reuse through the public FFI, edit invalidation, old artifact ownership, mode isolation on success and failure, and special-stock warnings after LTM requests.
Per-helper accuracy checks do not bound composed finite-difference ratios near cancellation. Describe the separate raw, normalized, sign, and nonfinite comparisons required for LTM, and align the power lowering comment with its negative-base and special-value handling.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1063 +/- ##
==========================================
+ Coverage 93.22% 93.23% +0.01%
==========================================
Files 253 254 +1
Lines 153270 153508 +238
==========================================
+ Hits 142889 143128 +239
+ Misses 10381 10380 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Reviewed the LTM scoring corrections, the wasm function-splitting infrastructure, and the incremental-compile reuse in No blocking correctness issues identified in the diff. Verdict: correct. |
The zero-change guard lived in three hand-written copies, and the switch from approximate `=` to exact `ABS(delta) <= 0` reached only two of them. The black-box module transfer kept `(delta) = 0`, so a loop through a pathway-less module whose deltas fall under f64::EPSILON scored 0 at every step. Every generator now wraps its score in one builder, `ltm_augment::link_score_guard`, so the spelling cannot drift again. The first-step test had the same defect on the time axis: `TIME = INITIAL_TIME` treated steps within f64::EPSILON of a zero start, or within 4 ULPs of a large one, as the first evaluation. The VM and the wasm backend start the clock at exactly the start time and only advance it, so the guard is `TIME <= INITIAL_TIME`. Both regressions run through the production pipeline; the black-box one also asserts which generator produced the score, because a composite scores the same values. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N838Szayoh8oz3snEC9na2
1 participant
LTM attribution must preserve the temporal meaning of co-inputs, survive changes of units, and justify a dominant period against its full normalization mass. Large LTM simulations also need to compile successfully in JavaScript: C-LEARN currently generates a 9,471,955-byte function, which Node 24/V8 rejects.
This PR contains three implementation changes and a numerical-accuracy documentation clarification:
PREVIOUS(...)co-input at its previous evaluated value, use exact zero-change score guards, and require the reported dominant set to reach 50% even when the display cap omits loops. Production-pipeline regressions and the LTM references describe these semantics.The measurements use system Node v24.18.0 / V8 13.6.233.17-node.50 on a Ryzen 9950X, pinned to CPUs 0–7, with shipping fat-LTO/wasm-opt artifacts. The public
model.simulate({}, {engine: 'wasm', enableLtm})call compiles an actual generated simulation module. Opening/parsing, library initialization, simulation execution, and extraction are outside the compilation timer. Fresh-project measurements reuse a warm Node process; component timings wrap the actual compiler export and generated Module/Instance construction, with uninstrumented controls.Median compilation times in milliseconds (seven paired samples after two warmups):
The edit column uses three paired samples of an actual public
applyPatchfollowed bysimulate, changing World3'sunit_populationor C-LEARN'spopulation_scenarioconstant and verifying the edited value by executing the new simulation. It includes patch validation/invalidation, excludes simulation/extraction/UI, and does not establish latency for every structural edit. Repeated compiles improve 15.8×/12.0× in the first two cases; fresh compilation remains similar.The original fresh World3 LTM call spends 25.82 ms in the library compiler export, 0.60 ms in generated V8 Module construction, 0.04 ms instantiating, and 0.31 ms in JS/layout/copies. C-LEARN off spends 72.29/1.01/0.34/1.02 ms respectively. After reuse, the third World3 compile is 1.23/0.04/0.04/0.27 ms; C-LEARN off is 4.73/0.14/0.12/0.91 ms. The new successful C-LEARN LTM fresh compile is 906.15/5.67/0.40/9.58 ms, and its third compile is 66.25/1.50/0.42/7.34 ms. Component medians need not sum to median totals; instrumented/control fresh totals differ by about 1% or less.
C-LEARN LTM grows from 49 to 200 functions, with the largest body shrinking from 9,471,955 to 1,132,923 bytes; total artifact growth is only 3,665 bytes (0.028%). World3 LTM and C-LEARN off warm execution remain about 4.05 and 20.37 ms. C-LEARN LTM execution remains expensive: one instance's seven measured runs after ten warmups have median 1,834.63 ms (1,829.59–1,836.91 ms). An additional five-run diagnostic gives approximately 1.02 s in the VM versus 1.84 s in generated WASM. Regrouping the same instructions at known-safe boundaries into 1 MiB functions yields about 1.80 s and a byte-identical 69,061,144-byte full result slab; it does not remove the execution cost. This PR retains the 64 KiB target. There is no runnable unsplit Node baseline for C-LEARN LTM.
The compiler-export bucket includes V8 work performed inside execution of the library WASM. The separate Module bucket measures generated-model construction; these measurements do not isolate every V8 optimization phase. Reusing compiler queries does not cache WASM emission, and overriding a differing discovery setting can invalidate queries.
Validation:
RUN_BENCH=1 BENCH_MODELS=clearn BENCH_LTM=on pnpm -C src/engine exec rstest run backend-benchreproduces the old V8 rejection and exercises acceptance plus public-series parity.BENCH_LTM=off|on|bothalso supports ordinary corpus comparisons.abs(a-b)/max(1,abs(a),abs(b))is 2.66e-8. A binary comparison reconstructs the original C-LEARN initials and flow bodies byte-for-byte by concatenating their 151 helper bodies; the other 47 original bodies and all non-function/code sections are identical. The existing approximate transcendental helpers and subtraction of nearly equal quantities can amplify trajectory differences in raw finite-difference ratios. No score tolerance was relaxed in production or existing tests.This establishes the tested engine and Node/V8 behavior, not a browser interaction journey or fidelity of every LTM extension. The always-on design document records remaining requirements for complete explanations, numerical normalization, saved evidence, and resumable analysis; it does not enable LTM by default.