From 747868a153bbee153b87c9d990a9142cce035627 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 11:20:59 -0700 Subject: [PATCH 01/43] the portable-tier plan: dasLLAMA without the JIT crowns in three stages (no-LLVM build on reference bodies, an AOT example target, WASM64), with the kernel test coverage audit as stage 0 and the settled cut - inert annotation shells in daslib, the framework whole in dasLLVM Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/portable_tier_plan.md | 264 +++++++++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 modules/dasLLAMA/portable_tier_plan.md diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md new file mode 100644 index 0000000000..46f8aa91b0 --- /dev/null +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -0,0 +1,264 @@ +# The portable tier - dasLLAMA without the JIT crowns: no-LLVM build, AOT, WASM64 - plan + +**Status: decisions settled (2026-09-04 discussion), pre-implementation.** The arc has one +product goal, reached in three stages that each prove the next one's premise: first a regular +`-jit` build with the `[tune]` framework short-circuited (the portable tier running on JIT-compiled +reference bodies), then an AOT compilation target under `examples/dasLLAMA/` on a very small +model, and only then WASM64. Main targets are ASR and TTS; a small LLM rides along, and the +model set may go smaller still. Vectorizing the default kernels is a separate, measured leg +between the stages (sec. "The vectorization leg"). The `-jit`-only rule in +`ARCHITECTURE_INVARIANTS.md` sec.3 is adjusted when the AOT stage reaches it, not before. + +Companion evidence: the four inventories the 2026-09-04 session produced (the LLVM dependency +census, the tune-framework coupling, the WASM build state, the require-guard grammar, and the +kernel test coverage) are summarized here with their file:line cites; nothing below is a guess. + +--- + +## The frame + +dasLLAMA is pure daslang (108 files under `dasllama/`, no C++ of its own). Its whole +dependency on LLVM is one file, `dasllama/dasllama_gemm_gen.das` (3756 lines), which builds +LLVM IR through the `llvm_boost` / `llvm_jit_code` / `llvm_jit_common` bindings; 40 generator +keys, permutations over `dot` (sdot, maddubs, vpdpbusd, vpdpbssd, smmla, amx_int8) and `width` +(128, 256, 512). It is reached only through path guards (`dasllama_common.das:22`, +`modules/dasLLVM/daslib/llvm_user_modules.das:8`). Everything else that spells `llvm/` is +packaging: 33 requires of `llvm/daslib/f16_cvt` and 4 of `llvm/daslib/aarch64_neon` (pure +das, zero requires of their own, recognized by the JIT by mangled name), and 27 of +`llvm/daslib/llvm_tune` (2759 lines, not one LLVM call). The `llvm/` path root exists only when +`DAS_LLVM_DISABLED=OFF` (`modules/dasLLVM/CMakeLists.txt:5`), and a source tree still resolves +it through the `.das_module` folder scan, which is why the module "works" today with LLVM +disabled by accident and fails in an installed SDK. + +`[tune]` never rewrites a body: it stamps `[llvm_code]`, and the tuned kernel IS the IR the +generator emits (`llvm_tune.das:1436-1448`, `llvm_jit.das:7961`). Every `[tune]` family keeps a +runnable das reference body; the `portable` backend registers unconditionally at priority 0 +(`dasllama_math_default.das:1613`); `arm64-gen`, `x64-gen` and `arm64-sdot` are +`jit_enabled()`-gated (`dasllama_math_gen.das:2775, 2807`, `dasllama_math_aarch64_neon.das:458`). +das `float4`/`int4` lower to 128-bit SIMD in every tier; 256/512-bit exists only as JIT +name-recognized intrinsics and the IR generators. There is no mechanism that freezes a chosen +permutation as das or C++; `policies.tune_frozen` means de-tuned. + +What the portable tier therefore is: the reference bodies, on whichever compiler the tier has. +Under the JIT they are auto-vectorized (the NEON file prefers the portable `dot_q8q8` over its +own scalar `sdot4` fallback for that reason, `dasllama_math_aarch64_neon.das:455-457`). Under +AOT they are C++ through clang or gcc. Under WASM they are that C++ through emcc with +`-msimd128`, or LLVM IR from the host emitter through the `-exe` cross path. + +The WASM side is further along than assumed: a wasm64 + pthreads runtime builds and ships +(`web/CMakeLists.txt` with `DAS_WASM_MEMORY64` and `DAS_WASM_PTHREADS`, deployed by +`pages.yml:260`); the cross-compile is `daslang -exe --jit-target=wasm64-unknown-emscripten` +with LLVM on the host only; wasm32 cross is unsupported by design (pointer-width layout bake, +`web/CMakeLists.txt:363-372`). + +--- + +## Settled decisions + +- **The cut.** Annotations that are 100% inert without the framework move to daslib; the + framework stays in dasLLVM whole. With no LLVM there is no tune, no sidecar, no exchange, + no tuner spawn - as if nothing happened. Not the read/mint split first proposed. +- **`f16_cvt` moves to `daslib/f16_cvt.das`.** The JIT recognizes it by mangled name from a + target-independent table (`llvm_jit_intrin.das:232-234`), so the move changes nothing for + the JIT; 33 require spellings change. +- **`aarch64_neon` and `x64_avx` stay in dasLLVM.** Nothing in the engine requires `x64_avx`. + The NEON tier file registers only under `jit_enabled()`, so its require becomes the path + guard `?llvm/daslib/aarch64_neon` from the umbrella; the harness, probe and benchmark that + require it are JIT-only by nature. +- **`require ?G target` names a guard module, never the target** (`ds2_parser.ypp:849-861`, + `parser_impl.cpp:1225-1279`). There is no module named `llvm` (dasLLVM is a pure-das dasbind + package), so `?llvm` and `builtin_module_exists(llvm)` are permanently false; the working + spelling is the path guard `?llvm/daslib/llvm_tune`. `get_platform_name()`, + `get_cross_platform_name()` and `get_architecture_name()` fold at compile time + (`module_builtin_runtime.cpp:2887-2907`), so a synthetic `wasm` module buys nothing for code + gating. `require !?X` is about five parser lines plus the hand-written pre-parse scanner + mirror (`ast_parse.cpp:229-268`); reach for it only if a module must not LOAD when X exists. +- **The routes.** AOT C++ through emcc is the LLVM-free WASM route. The `-exe` cross route + keeps tuned kernels and needs host LLVM; it is the later option, not the first. + +--- + +## Stage 0 - the coverage audit, and what it must add before any kernel moves + +Running the suite with the JIT off proves nothing by itself: every model-backed cell self-skips +on `if (!jit_enabled())`, and `guard_interp_inference` (`dasllama_math.das:29-33`) turns anything +that tries to serve into a panic. The portable tier's real safety net is the synthetic, +direct-call unit layer, and that layer is complete exactly where the K-quant work happened +(`tests/test_kquant.das`: 13 formats x dots, gemv rows, repack grp, tile, groupn against fp64 +dequant and bit-exact disk references; its `test_kq_tile` header at `:1673-1676` claims validity +on reference bodies and stamped kernels alike) and thin to absent elsewhere. The additions, in +priority order: + +**(a) A `[tuned]` elementwise unit.** `dot` (`dasllama_math.das:48`), `axpy` (`:66`), +`dot_bf16` (`:81`), `add_inplace` (`:97`), `add_scale_inplace` (`:111`), `mul_inplace` (`:123`), +`scale_inplace` (`:137`), `copy_floats` (`:151`). All `[tuned]`, the highest-traffic kernels in the +module (`copy_floats` 62 engine sites, `add_inplace` 35, `scale_inplace` 33, `axpy` 26), zero +coverage. `dot` is pinned only through three hand-computed `matmul` cases. The unit sweeps +lengths with tails against fp64 accumulation references. + +**(b) `test_q8q8_family.das`.** The crown family (`q8q8_tile_gen` `dasllama_math_gen.das:1222`, +`q8q8_gemv_gen` `:107`, the mx4 and s16 companions `:124, :145, :1153, :1170`) has its +tile-vs-gemv-vs-reference parity only in hand-run probes (`harness/gen_parity_probe.das:110,131`, +`harness/gen_slot_parity_probe.das`). Promote them into the suite with the ladder the 13 kq +families already have, and cover what has no gate at all: the whole s16 (`wscale_f16`) arm +(`dot_q8q8_f16s` `dasllama_math_default.das:64`, `q8q8_rows_kernel_s16` `:320`, +`q8q8_groupn_kernel_s16` `:354`, the s16 overloads `dasllama_math.das:2096-2138`), +`matmul_q8q8_group3` (`:2081`), `matmul_q8` / `dot_q8` (`:906, :881`), and +`matmul_mx4q8_batch` / `_batch_groupn` (`:2194-2226`). + +**(c) A CPU rope-application unit.** `rope_scaled`, `rope_scaled_neox`, `rope_scaled_tab`, +`rope_scaled_neox_tab`, the `_part` twins and `rope_apply` (`dasllama_math.das:2992-3160`) are +today only the oracle of the Apple-only Metal rope tests; `test_rope.das` covers the table +builders and the legacy `rope()`. An in-test angle reference makes them a subject. + +**(d) The activation and norm cells.** `softmax_sink` (`:2569`, six gpt-oss sites, no test), +`swiglu_oai` / `swiglu_oai4` (`:2749, :2820`), `softcap` / `softcap4` (`:2932, :2947`), +`sigmoid_gate` / `sigmoid_gate4` (`:2884, :2893`), `silu_mul4` (`:2911`), `silu4_batch` +(`:2869`), `l2_norm_rows` (`:2677`), `gemm_f32` / `gemm_f32_jo` / `gemm_f32_uk_4x16` +(`:834, :858, :828`; 23 engine sites, the tower and TTS float GEMM). Alongside the existing +`test_silu.das` / `test_softmax.das` property tests. + +**(e) Prefill and KV cells.** `matmul_kq_batch` (`:1944`, no test; portable has no slot, so +the serial-GEMV fallback is also ungated), `matmul_kq_batch_groupn` (`:1918`, whose only gate +`test_kquant.das:2136` self-skips off-JIT at `:1962-1966`), `kv_row_to_f32` and +`cvt_q8kv_to_f32` (`dasllama_kv_codec.das:394-409, :140`), `requant_rows_q8` / +`requant_rows_q8k_bs` (`dasllama_convert.das:1490, :1509`), `repack_kq_weight` +(`dasllama_math.das:1974`). + +**(f) Suite membership and honesty.** `test_kv_codec.das` and `test_vad.das` (a frozen +onnxruntime oracle with weights in-tree, `tests/test_vad.das:75`) are absent from +`tests/run.das:75-171`'s `model-free` list; add them. `test_mxfp4.das:168-170` returns silently +instead of `t |> skip` (against `tests/CLAUDE.md:40-42`). `test_groupn.das:12-15` takes its +expected values from the per-region GEMV under the same backend pin, so a bug shared by +`q8q8_kernel` and `q8q8_groupn_kernel` passes; give it an independent reference. + +**(g) A way to point the parity rails at the portable backend.** `tests/test_parity.das` +(`:177`) and `harness/parity.das` cannot select a backend from the command line; the +documented `DASLLAMA_PIN_BACKEND` knob (`ENVIRONMENT.md:25`, `dasllama_env.das:50`) is read by +three benchmarks and never by the library. Wire it into the parity harness (or add +`--backend`), so the same frozen ids replay against the tier this arc ships. + +**(h) Tower, ASR and TTS.** Encoder-tower GEMM wrappers `mm_blob_b` / `mm_bf16_b` / +`mm_plane_b` (`dasllama_tower.das:253-277`) and `layernorm_batch` (`:182`) have no CPU unit; +the ASR conv stems and mels (`dasllama_qwen3a.das:514, :406`, `dasllama_canary.das:786-841, +:670`, `dasllama_gemma4a.das:778, :744`, `dasllama_parakeet.das:905, :987, :805`, the whisper +stem `dasllama_whisper.das:987`) are `private` and covered only by jit-gated, model-gated +transcript oracles in `test_whisper.das`; `fft_pow2_run` (`dasllama_audio.das:157`) has no +direct test. TTS: `bilstm`, `sine_source`, `magnitude_phase`, `istft_envelope_divide` +(`dasllama_tts_blocks.das:1633, :1920, :1973, :2037`) reach only `_tts_parity.das` (model + +oracle-dir gated); `resize_linear` / `resize_linear_torch` (`:1738, :1776`) have none. Since ASR +and TTS are the main targets, the private stems get exposed to a synthetic unit or a +weights-in-tree oracle the way VAD already has one. + +Stage 0 also lands the four rule-document and gate fixes owed from the is_nan arc +(a `tests-cpp/REVIEW.das` gate that every `set_source_files_properties` path exists, the +`tests-cpp/REVIEW.md` rule 3 wording, the `include/daScript/simulate/REVIEW.md` hot-path rule +naming the shipped build as the judge, and the `writing_cpp_tests.md` line about CMake edits). + +**Exit:** every kernel family in the inventory has a synthetic gate that does not self-skip +without the JIT, or a written reason it cannot; the suite's `model-free` arm lists every such +file. + +--- + +## Stage 1 - the refactor, and the portable tier on a regular `-jit` build + +1. `daslib/f16_cvt.das`; 33 require spellings. +2. The inert annotation module in daslib carrying `tune`, `tune_perm`, `tune_companion`, + `tune_scope`, `tune_policy` and `llvm_code` as shells: `require ?llvm/daslib/llvm_tune + llvm/daslib/llvm_tune`, then `static_if (typeinfo builtin_module_exists(llvm_tune))` in each + `apply` delegating to the framework, else return true and do nothing. The shell's own + path-guarded require is the witness, so the order caveat on `builtin_module_exists` + (`ast_infer_type.cpp:2833-2848`) does not bite. Framework change: its annotation classes + stop self-registering under those names (two registrations of `[tune]` would resolve + ambiguously) and expose their `apply` as plain functions the shells forward to. The + `_variants()` registries are emitted only by the framework; the engine references none + outside the guarded generator tier. +3. `[tuned]` goes framework-optional: `dasllama_tune.das:14` takes the path guard, and + `resolve_perm` (`:330-346`) with no framework returns the declared `fallback=` or the default + perm and reports nothing - no sidecar opened, no `tune_status` line, no box profile. +4. `dasllama_exchange.das` becomes a guarded require from the umbrella and the server. +5. The NEON tier's require takes the path guard. +6. `tests/run.das` gets a no-tune arm (the framework's reference policy; `run.das:309` + hard-codes `-jit` with no knob today), and the suite plus the parity rails run under it on + the stage-0 gates. +7. Measure the portable floor on the reference small models (decode and prefill, ASR and + TTS cells), under the measurement discipline: it is the number every later tier is judged + against. + +**Exit:** a `-DDAS_LLVM_DISABLED=ON` tree compiles the engine and its suite; a `-jit` run with +the framework short-circuited is green on the stage-0 gates and parity; the floor is in +`PERF_LEDGER.md`. + +--- + +## Stage 2 - an AOT compilation target under `examples/dasLLAMA/` + +- One example program (a small-LLM chat, then ASR and TTS twins) registered as an AOT target, + the first exception to `modules/dasLLAMA/CMakeLists.txt:7` ("never AOT'd"), scoped to that + example. +- `guard_interp_inference` accepts `aot_enabled()` beside `jit_enabled()` and + `is_standalone_exe()`; `guard_interp_gguf_load` likewise. This is the rule adjustment the + invariants doc gets in the same change. +- The stamped-function hash question is moot under `-aot` (`tune_frozen`), and with the + stage-1 shells the AOT emitter sees ordinary functions. +- Expect heavy TUs and a long C++ compile: the engine closure is large. Known AOT traps from + earlier arcs to watch first: int64 pointer indexing, managed-vector for-in. +- Parity against the stage-1 JIT run on the same model is the pass criterion, ids exact. + +**Exit:** the example compiles and runs the small model with exact parity to the JIT run; its +decode/prefill numbers sit beside the stage-1 floor. + +--- + +## Stage 3 - WASM64 + +- Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + + pthreads), under `wasmtime -W memory64=y -W exceptions=y` first, the browser after. +- Repairs on the way, all small: the `wasm_cross` CI lane configures without + `-DDAS_WASM_MEMORY64=ON` (`wasm_build.yml:209`) so `web/CMakeLists.txt:374` registers no + examples and the lane is a green no-op; `get_architecture_name()` answers `wasm32` on a + memory64 build (`module_builtin_runtime.cpp:2064`) and the engine keys backends off that + string; `DAS_MAX_HW_JOBS` is 4 on wasm (`platform.h:629`); the two emsdk pins contradict + (`wasm_build.yml:188` vs `pages.yml:73`). +- Limits to design around: SIMD is 128-bit through the `-msse2` shim and that flag is + load-bearing (`web/CMakeLists.txt:34`; without it vecmath drops to scalar); a process-wide + malloc spinlock under shared memory (`src/hal/wasm_thread_malloc.cpp`); the main-thread join + is sliced at 500 ms; no dasHV on wasm, so no server surface. +- Model IO is the wall: `fmap` on MEMFS copies the whole file into the heap, there is no + OPFS or WASMFS anywhere, and no `-sMAXIMUM_MEMORY`. Small quantized models load; a + windowed OPFS reader is its own item if anything larger is ever wanted. + +**Exit:** the ASR and TTS examples run under wasmtime with parity; a small LLM decodes; the +browser demo is a stretch on the same artifact. + +--- + +## The vectorization leg (between stages 1 and 3, measured) + +Under the JIT the reference loops are auto-vectorized, so stage 1 is the SIMD-achievable floor +for the default kernels. AOT and WASM have no vectorizer we control: the emitted C++ goes +through clang, gcc or emcc, whose auto-vectorization of daslang loop shapes is hit and miss, +and the interpreter never vectorizes a loop. Explicit `float4` / `int4` bodies are how those +tiers reach the floor structurally. Do it per hot kernel by the stage-2 profile, not up +front: the q8 dot and GEMV/GEMM, the k-quant dequant-dot, f16 widen, attention dot and axpy, +the KV codec, the TTS conv and GEMM. The crux is the int8 dot: das has no int8 vector type, +so it either widens through `int4` lanes or gets a name-recognized lattice intrinsic with a +fallback body (the `x64_avx.das` pattern), which on wasm would want SIMD128's +`i32x4.dot_i16x8_s`. That decision is taken with stage-2 numbers in hand. + +--- + +## Models + +ASR and TTS first: whisper tiny, the silero VAD (weights in-tree), Kitten nano, Kokoro. For the +LLM: gemma-3-270m, SmolLM2-135M, Qwen3-0.6B are all registered arches; the two under 300M are +the honest wasm candidates, and smaller is on the table. + +## Open questions + +- Which small-LLM carrier gets the parity fixture for stages 1 and 2 (SmolLM2-135M already has + a cls_q8 parity cell, `test_parity.das:76`). +- Whether `test_groupn.das`'s independent reference is fp64 dequant (the kq ladder's choice) or + a second, scalar implementation. +- Whether the stage-2 example is its own daspkg app or a plain `examples/` program. From 19bc49a16117b64d94e47056c97c4581d1a5c900 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 13:33:55 -0700 Subject: [PATCH 02/43] the four rulings owed from the is_nan arc: tests-cpp/REVIEW.das gains the gate that every set_source_files_properties path exists (a moved per-flag TU otherwise builds the ordinary way and stays green), the tests-cpp checklist's lane rule covers a subject that compiles to an unexercised implementation on some lane, the simulate hot-path rule names the shipped build as the judge, and the C++ test skill stops claiming a small test never needs a CMake edit Co-Authored-By: Claude Fable 5.1 --- include/daScript/simulate/REVIEW.md | 5 ++++- skills/internal/writing_cpp_tests.md | 2 +- tests-cpp/REVIEW.das | 18 ++++++++++++++++++ tests-cpp/REVIEW.md | 7 ++++--- 4 files changed, 27 insertions(+), 5 deletions(-) diff --git a/include/daScript/simulate/REVIEW.md b/include/daScript/simulate/REVIEW.md index d408bf4545..5c80f23622 100644 --- a/include/daScript/simulate/REVIEW.md +++ b/include/daScript/simulate/REVIEW.md @@ -32,7 +32,10 @@ checklist on its own. folder that generated code executes per evaluated expression. Such a diff - including one an optimized build flattens to nothing - lands its entry under `ARCHITECTURE.md`'s sanctioned hot-path additions in the same diff: what was added, where, why correctness - required it, and the alternative that was rejected. + required it, and the alternative that was rejected. Cost is judged in the build the repo + ships: a change that costs more only under a relaxed-math or otherwise non-default compiler + flag states which flavor and how much in its PR description, and ledgers only when the + shipped build pays. - **A diff that changes the layout of a `debug_info.h` struct - a field added, removed, reordered, or retyped, or a base changed - states a per-consumer verdict (updated / no diff --git a/skills/internal/writing_cpp_tests.md b/skills/internal/writing_cpp_tests.md index 608485d7cb..160b4014c8 100644 --- a/skills/internal/writing_cpp_tests.md +++ b/skills/internal/writing_cpp_tests.md @@ -38,7 +38,7 @@ TEST_CASE("my feature does X") { Optionally drop `tests-cpp/small/test_my_feature.das` next to it; load via `getDasRoot() + "/tests-cpp/small/test_my_feature.das"`. -Rebuild - `cmake --build build`. The new test appears in `ctest -N` automatically (no CMake edit needed). +Rebuild - `cmake --build build`. The new test appears in `ctest -N` automatically; the glob needs no CMake edit. A test that must be compiled differently from the rest of the exe - a per-source flag such as the fast-math one on `test_isnan_fastmath.cpp` - adds its `set_source_files_properties` line to `tests-cpp/CMakeLists.txt`, and its TU `#error`s when built without the flag, so a moved or renamed file cannot silently build the ordinary way and stay green. **Don't include doctest with `DOCTEST_CONFIG_IMPLEMENT`** - that's owned exclusively by `doctest_main.cpp`. Including the header without that macro is the right pattern for every other TU. diff --git a/tests-cpp/REVIEW.das b/tests-cpp/REVIEW.das index 214d882f53..66f7b4296c 100644 --- a/tests-cpp/REVIEW.das +++ b/tests-cpp/REVIEW.das @@ -61,6 +61,23 @@ def private check_test_targets(f : string; text : string) { } } +// Every source a `set_source_files_properties` names exists beside its CMakeLists. The +// property applies to nothing when the file moves, so a test whose compile flag went +// missing builds the ordinary way and stays green. +def private check_source_property_paths(f : string; text : string) { + let folder = dir_name(f) + for (blk in cmake_command_blocks(text, "set_source_files_properties")) { + var inscope args <- cmake_args(blk) + for (a in args) { + break if (a == "PROPERTIES" || a == "DIRECTORY" || a == "TARGET_DIRECTORY") + continue if (a |> starts_with("$")) + if (!fexist(path_join(folder, a))) { + gate_finding(f, "set_source_files_properties names {a}, which does not exist beside {f} - the property applies to nothing and its test builds without the flag") + } + } + } +} + // Every lane a file labels tests into is a lane that file also builds for. def private check_lane_wiring { let top_cmake = "tests-cpp/CMakeLists.txt" @@ -79,6 +96,7 @@ def private check_lane_wiring { gate_finding(f, "labels a test \"{lab}\" but wires nothing into test-{lab} — wire it with add_dependencies(test-{lab} ), or the lane runs a binary it never built") } check_test_targets(f, text) + check_source_property_paths(f, text) } } diff --git a/tests-cpp/REVIEW.md b/tests-cpp/REVIEW.md index af4b8d8543..f0748cd3a9 100644 --- a/tests-cpp/REVIEW.md +++ b/tests-cpp/REVIEW.md @@ -9,6 +9,7 @@ checklist as well as this one.** **A test that owns its own `CMakeLists.txt`, wherever the diff puts it, answers to the `big/` subfolder's checklist as well as this one.** -**A diff adding or changing a test that skips or reports pass without running, under any -condition not every lane meets - a missing file or binary, an unset environment variable - -says in the PR which lane runs it for real, naming the command.** +**A diff adding or changing a test that cannot fail on some lane - it skips there, or its +subject compiles there to an implementation the test does not exercise (a per-source flag one +compiler honors and another ignores, a `#if` arm the lane never takes) - says in the PR which +lane runs it for real, naming the command.** From 3d71cb25fc2917b1df2f6c5c8ade2475e0c54f55 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 14:10:07 -0700 Subject: [PATCH 03/43] stage 0 of the portable tier: the kernel gates the portable path needs before any kernel moves - six new units (the tuned elementwise set, activations and fp32 GEMM, the q8q8 family promoted from the hand-run probes with the s16 arm and group3, rope application, the prefill and KV CPU kernels, the tower/audio/TTS kernels), an fp64 reference beside test_groupn's self-consistency compare, a loud skip where test_mxfp4's gen arm returned silently, the parity rails pinnable to one backend through DASLLAMA_PIN_BACKEND (restored on exit), the twelve suite-less test files registered in the model-free gate, and matmul_q8q8_batch's doc claim narrowed to the backend it holds on Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/dasllama/dasllama_math.das | 6 +- modules/dasLLAMA/harness/parity.das | 30 +- modules/dasLLAMA/portable_tier_plan.md | 36 + modules/dasLLAMA/tests/CLAUDE.md | 104 +- modules/dasLLAMA/tests/run.das | 7 + modules/dasLLAMA/tests/test_groupn.das | 46 +- .../dasLLAMA/tests/test_math_activations.das | 634 ++++++++ .../dasLLAMA/tests/test_math_elementwise.das | 395 +++++ modules/dasLLAMA/tests/test_mxfp4.das | 3 +- modules/dasLLAMA/tests/test_parity.das | 60 +- .../tests/test_prefill_cpu_kernels.das | 741 ++++++++++ modules/dasLLAMA/tests/test_q8q8_family.das | 885 ++++++++++++ modules/dasLLAMA/tests/test_rope_apply.das | 481 +++++++ .../dasLLAMA/tests/test_tower_asr_kernels.das | 1276 +++++++++++++++++ 14 files changed, 4682 insertions(+), 22 deletions(-) create mode 100644 modules/dasLLAMA/tests/test_math_activations.das create mode 100644 modules/dasLLAMA/tests/test_math_elementwise.das create mode 100644 modules/dasLLAMA/tests/test_prefill_cpu_kernels.das create mode 100644 modules/dasLLAMA/tests/test_q8q8_family.das create mode 100644 modules/dasLLAMA/tests/test_rope_apply.das create mode 100644 modules/dasLLAMA/tests/test_tower_asr_kernels.das diff --git a/modules/dasLLAMA/dasllama/dasllama_math.das b/modules/dasLLAMA/dasllama/dasllama_math.das index a6070c3e36..2d51af418b 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math.das +++ b/modules/dasLLAMA/dasllama/dasllama_math.das @@ -2046,9 +2046,9 @@ def matmul_q8q8(var y : array; wq : array; ws : array; woff } } -//! Batched Q8·Q8 GEMM for prefill: same weight-stationary nest as matmul_batch over the SDOT kernel. -//! Activations PRE-quantized per token into (xq [ntok x n], xs [ntok x n/32]). Y token-major -//! [ntok x d]. Bit-for-bit ntok× matmul_q8q8. Dispatches to the active backend. n % 32 == 0. +//! Batched Q8·Q8 GEMM for prefill over PRE-quantized activations (xq [ntok x n], xs [ntok x n/32]), +//! Y token-major [ntok x d], n % 32 == 0. Bit-for-bit ntok× matmul_q8q8 on the portable backend; +//! a generated backend's tile and token-tail perms fold differently (test_q8q8_family holds them within the dot's envelope). def matmul_q8q8_batch(var y : array; wq : array | #; ws : array | #; xq : array; xs : array; n, d, ntok : int64) { unsafe { invoke(g_mm_q8q8_batch, addr(y[0]), diff --git a/modules/dasLLAMA/harness/parity.das b/modules/dasLLAMA/harness/parity.das index 07d15166df..1f2a8d8b3e 100644 --- a/modules/dasLLAMA/harness/parity.das +++ b/modules/dasLLAMA/harness/parity.das @@ -49,11 +49,36 @@ struct Args { @clarg_doc = "GPU layers, llama-bench spelling: 0 = CPU, anything else = the whole graph on Metal, required (default: 0)" ngl : int = 0 + @clarg_default_doc = "DASLLAMA_PIN_BACKEND, else the measured auto-selection" + @clarg_doc = "Pin the matmul kernel backend by name (portable, arm64-sdot, ...); overrides DASLLAMA_PIN_BACKEND" + backend : string = "" + @clarg_short = "?" @clarg_doc = "Show this help and exit" help : bool } +// `--backend`, defaulting to DASLLAMA_PIN_BACKEND: resolve the name against the registry and pin +// it, or refuse. Called before load_gguf — select_matmul_backend_for_load_ reads the pin to pick +// the weight layout the loader repacks into — and after the --ngl block, so the flag outranks the +// portable pin metal mode applies. +def private apply_backend_pin(flag : string) : bool { + let name = empty(flag) ? g_env_engine.pin_backend : flag + if (empty(name)) { + return true + } + var names <- kernel_backend_names() + let known = find_index(names, name) >= 0 + if (!known) { + print("error: unknown backend '{name}' - registered: {join(names, ", ")}\n") + } + delete names + if (known) { + pin_kernel_backend(name) + } + return known +} + def private parse_ids(s : string) : array { var ids <- [for (part in split(s, ",")); to_int64(part, false); where !empty(part)] return <- ids @@ -87,6 +112,7 @@ def main : int { if (cfg.ngl != 0) { // before load_model — pins the portable backend, activates the metal overrides set_metal_mode(MetalMode.required) } + return 2 if (!apply_backend_pin(cfg.backend)) var t <- load_gguf(cfg.model, cfg.quant) t.tok <- load_tokenizer_auto(cfg.model) // load_gguf skips the tokenizer; GEN_TEXT needs it if (cfg.ngl != 0 && !convert_model_to_metal_blob(t)) { // the Metal drivers serve blob-form models only @@ -97,8 +123,8 @@ def main : int { if (t.config.seq_len > 2048l) { t.config.seq_len = 2048l } - let c = t.config - var s = make_run_state(c, cfg.kv, cfg.kv) + print("GEN_BACKEND: {active_kernel_backend()} (batch: {active_batch_backend()})\n") // the GEN_ block's header — a log names the backend its ids came off + var s = make_run_state(t.config, cfg.kv, cfg.kv) var rc = 0 with_job_que() { setup_dasllama_jobque_() diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 46f8aa91b0..e7f6fd6140 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -132,6 +132,17 @@ instead of `t |> skip` (against `tests/CLAUDE.md:40-42`). `test_groupn.das:12-15 expected values from the per-region GEMV under the same backend pin, so a bug shared by `q8q8_kernel` and `q8q8_groupn_kernel` passes; give it an independent reference. +**(f2) The same shared-bug class in the two mx4 groupn cells.** `test_mx4q8_groupn` takes its +expected values from the per-region `matmul_mx4q8` under the same backend pin, and +`test_groupn_bias_fold` takes its q8 and mx4 expectations from the unbiased groupn kernel; the +fp64 mx4 reference already exists as `ref_dot_f64` in `test_mxfp4.das:35` (private to that +file). Lift it into a shared `_kernel_ref.das` beside the two files and close both cells the +way (f) closed the q8 one. Lint candidate for a folder-local `tests/REVIEW.das` gate: a +`t |> run` arm that returns before asserting registers a skip (the guard whitelist is +`model_available`, `arm_on`, `family_on`, `model_missing`, `gguf_missing`, +`facade_data_missing`); the `test_mxfp4.das` silent return was the one raw-capability return +that did neither. + **(g) A way to point the parity rails at the portable backend.** `tests/test_parity.das` (`:177`) and `harness/parity.das` cannot select a backend from the command line; the documented `DASLLAMA_PIN_BACKEND` knob (`ENVIRONMENT.md:25`, `dasllama_env.das:50`) is read by @@ -150,6 +161,31 @@ oracle-dir gated); `resize_linear` / `resize_linear_torch` (`:1738, :1776`) have and TTS are the main targets, the private stems get exposed to a synthetic unit or a weights-in-tree oracle the way VAD already has one. +**(h) findings from writing the tower/ASR/TTS unit.** The four family mels and every TTS +kernel named above are already public; only the seven conv-stem helpers and `lstm_direction` +are private, and the whisper-class stem is not in the whisper file at all but lines 1300-1333 of +`audio_encode_blocks` (`dasllama_audio.das`), serving whisper, ultravox, voxtral, qwen2audio and +omni. Ranked exposure: (1) split that stem into one public kernel in `dasllama_audio.das` +(ARCHITECTURE_MEDIA sec.2.14 names the seam); (2)-(3) the four mels need no engine change, only +an fp64 STFT reference; (4) `bilstm` is public, gate `lstm_direction` through it and run one +shape twice for the scratch-global reset; (5) `magnitude_phase` / `istft_envelope_divide` are +pure and now covered; (6) drop `private` on the two parakeet convs; (7) `g4a_conv_stage` splits +into a public im2col and a LayerNorm+ReLU epilogue that has no twin anywhere; (8)-(10) the +qwen3a and canary stems converge on the parakeet shape (the three im2col walks are +near-identical copies); `cn_conv_pw` is a wrapper, not a kernel. `sine_source` cannot take an +fp64 reference (ARCHITECTURE_TTS sec.2.33 phase law): its gate is a golden vector. + +**(i) findings from writing the other units.** `test_kquant.das`'s synthetic plane builders are +private in a `[test]` root; lift them into a `_kq_fixtures.das` sibling (the +`_metal_kernel_common.das` precedent) so the kq gates stop hand-packing blocks. +`kq_batch_kernel_gen` / `kq_batch_groupn_gen` (`dasllama_math_gen.das:2185, :2233`) are private +unlike their groupn sibling, so their reference bodies are reachable only through a backend +pin; make them public. `KernelBackend` has no `kernel_backend_has_mx4_batch()` predicate, so a +sweep of `kernel_backend_names()` that touches `matmul_mx4q8_batch` panics on `arm64-sdot` +(`mx4q8_unset_batch`, `dasllama_math.das:1051, :1573`). An exact-equality float compare whose +fixture lattice no longer guarantees exactness has no lint; the two elementwise files carry +hand-derived lattice arguments, worth a fixture-intent annotation before any rule. + Stage 0 also lands the four rule-document and gate fixes owed from the is_nan arc (a `tests-cpp/REVIEW.das` gate that every `set_source_files_properties` path exists, the `tests-cpp/REVIEW.md` rule 3 wording, the `include/daScript/simulate/REVIEW.md` hot-path rule diff --git a/modules/dasLLAMA/tests/CLAUDE.md b/modules/dasLLAMA/tests/CLAUDE.md index 96ed59a7e9..3571497bee 100644 --- a/modules/dasLLAMA/tests/CLAUDE.md +++ b/modules/dasLLAMA/tests/CLAUDE.md @@ -338,7 +338,10 @@ generic loop drives every evidence-carrying spec of the model-set table through pinned arms (evidence is DATA on `ModelSpec.parity` - ids + arms, regenerated via `../harness/parity.sh`); hand-written arms remain only for the tied-cls bit-match and the gpt-oss shared-load double fixture. Every compare logs both decoded streams. Large carriers gate -on `DASLLAMA_PARITY_FULL=1` via `model_available`. +on `DASLLAMA_PARITY_FULL=1` via `model_available`. `DASLLAMA_PIN_BACKEND` points the compares +at one kernel backend: the generic loop and the gpt-oss arm pin it before their load, restore +it on every exit, and name the served backend in every assert label; a name the box has not +registered skips the cell naming it, never a silent run on the default backend. `test_parity_pregate.das` - stocked suite; model-gated: the board parity pregate (`lcpp_bench --parity`, via `parity_check` fed a controlled spec) on small carriers - the real fixture passes, a flipped id fails, an evidence-less spec is refused, the text-form prompt @@ -572,6 +575,105 @@ F0 phase drift the file header explains). `test_tower_helpers.das` - model-free: the shared encoder-tower helpers in `dasllama/dasllama_tower` (clamp, row norms, f16-table GEGLU-quick, im2col, two-axis rope, avg-pool, `attention_bidir`), each against an in-test reference. +`test_tower_asr_kernels.das` - model-free: the public tower / audio / TTS kernels with no CPU +unit of their own, each against an in-test fp64 reference over a procedural seeded-LCG fixture +and each bar carrying its own must-EXCEED poison arm - the padded-width GEMM wrappers +(`mm_blob_b` on an owned blob and on a borrowed plane, `mm_bf16_b` on bf16-exact operands, +`mm_plane_b`'s per-tensor routing, and the sec.2.13 claim that a zero-padded served width is +bit-identical to the unpadded GEMM), `layernorm` and both `layernorm_batch` overloads at a dim +off every lane width, `add_bias_rows` and `add_inplace_rows` against the exact scalar loop at +lengths off the float4 block, `gelu_erf_batch` against an fp64 erf series and against its own +whole-image form, `rope_neox_tab_rows` with a per-row-table-index leg, `attention_bidir_windows` +against an oracle that attends strictly inside each ragged window with per-window content so a +leak is material, `interpolate_grid_bilinear_aa` down and up at a non-zero plane offset, the +audio front end's `build_fft_plan` / `fft_pow2_run` / `build_dft_twiddles` against an fp64 +O(n^2) DFT of the same forward convention (plus the plan's non-power-of-2 refusal), the TTS +`resize_linear` / `resize_linear_torch` against an fp64 reference of each interpolation +convention, and the STFT pair `magnitude_phase` (with the zero-imaginary branch planted at both +zero signs - a bare `atan2` reads -pi where torch.angle reads +pi) and `istft_envelope_divide` +(four overlap-add geometries plus the zero-window leg that must leave the wave untouched through +the negligible-envelope guard). +`test_math_elementwise.das` - model-free: the `[tuned]` elementwise kernels (`dot`, `dot_bf16`, +`axpy`, `add_inplace`, `add_scale_inplace`, `mul_inplace`, `scale_inplace`, `copy_floats`) over +a 27-length sweep that reaches every width-8/unroll-2 tail (1..9, 15/16/17, 23/24/25, 31/32/33, +63/64/65, 100, 127/128/129, 1023/1024/1025). Inputs come from a seeded LCG tainted through a +`[sideeffects]` seam so no fixture folds at compile time, and every reference is accumulated in +`double` in-test. The two dots run twice: once on a dyadic lattice chosen so the whole fp32 +accumulation is provably exact (products are multiples of 3/64 bounded well inside 2^24, so any +vectorized or reassociated reduction still bit-matches), and once on unit-range inputs against +a length-only bar `1e-7*n^2 + 1e-6` - the textbook sequential-FMA bound at |a|,|b| <= 1 - each +length shipping its must-EXCEED control, the expected result offset by an added 0.5. `dot_bf16` +builds its operands AS bf16 and widens them for the reference, so the compare never measures +input rounding. The six in-place kernels are exact-equality gates: their operand lattices make +the fp64 reference land on a representable float and differ from the input at every element by +construction (`copy_floats` writes over a 1e9 sentinel, `add_scale_inplace` uses positive +operands with a negative scale so the result cannot be the input, `mul_inplace`'s multiplier is +a multiple of 0.375 so it can never be one), and every cell asserts that count. A closing cell +pins a zero count as a no-op on all eight. +`test_math_activations.das` - model-free: the activation, norm and fp32 GEMM kernels against +closed forms and in-test fp64 references over the same tail sweep. `softmax_sink` is gated per +element plus the closed form its name carries - the row's sum plus the sink mass is one - at a +negligible sink (-1000, where the -80 clamp engages) and a dominant one, and by shift +invariance (adding 5 to the scores and to the sink leaves the distribution alone). +`softcap`/`softcap4` are gated as `cap*tanh(x/cap)` at caps 30 and 50 over inputs out to 200, +with the band assert `|out| <= cap`; `sigmoid_gate`/`sigmoid_gate4`, `swiglu_oai`/`swiglu_oai4` +and `silu_mul4` (writing over a sentinel destination) each get the scalar and vec4 forms against +the same fp64 reference, so the twins are held to an oracle rather than to each other. +`swiglu_oai` additionally carries the trained-clamp gate: a gate of 20 must read bit-identically +to a gate of 7, an up value of +-20 to +-7, with a gate pair inside the limit as the must-differ +control. `silu4_batch` is gated for split invariance - bit-identical to `silu4` over the whole +buffer inline and under a `with_job_que` scope, with the dispatch counter (`get_disp_count`) as +the anti-vacuous witness that the threaded leg really split - plus the two fallback shapes (row +length off the float4 grouping, single row) which must not dispatch. `l2_norm_rows` is gated +per element and by the unit-norm property over eight (rows, row_size) pairs, plus the eps floor +on a zero row and a 1e-9 row. `gemm_f32` and `gemm_f32_jo` run twelve shapes covering both +remainder strips and the pure 4x16 tile, each against an fp64 GEMM that starts from the +pre-initialized C (so the accumulate contract is part of the claim) and against each other +bit-for-bit. Every tolerance bar in the file ships its control in the same cell: the expected +value offset by an added 0.01, which must land outside the bar. +`test_q8q8_family.das` - model-free: the q8q8 kernel family end to end, promoted from the +hand-run `harness/gen_parity_probe.das` / `gen_slot_parity_probe.das`. Six widths with tails +(64, 96, 512, 1024, 1056, 3072) across five cells, each judged by an in-test fp64 dequant +reference (int8 products summed exactly, both block scales applied in double) whose bar is the +per-block envelope times the block count, and each bar carrying a poison leg - 0.25 ADDED to +one expected element - that must EXCEED it. The stamped family repacks through +`repack_q8q8_grp` at the layout/wbias/kgroup companions and walks `q8q8_tile_gen` at the +tokstep companion's stride with a `q8q8_gemv_gen` token tail, bit-exact against per-token +GEMVs. The dispatch wrappers run on every backend `kernel_backend_names()` lists - repack +backends through their own `repack_q8q8_weight`, never on row-major data - with +`matmul_q8q8_batch` bit-exact against ntok x `matmul_q8q8`. The wscale_f16 arm covers +`dot_q8q8_f16s`, the `_s16` rows and groupN kernels, the stamped s16 tile/GEMV twins and the +`matmul_q8q8` / `_batch` / `_groupn` s16 overloads, bit-exact against their f32 twins over +f16-exact scales. `matmul_q8q8_group3` (f32 and s16) runs against three independent GEMVs on +unequal regions 32/40/44 (the row tail); `matmul_q8` / `dot_q8` cover the fp32-activation rail; +the mx4 cell drives `matmul_mx4q8_batch` and `matmul_mx4q8_batch_groupn` against ntok +independent `matmul_mx4q8` GEMVs and the `dot_mx4q8_scalar` leaf, pinned portable and swept +over the repack backends that carry those slots. Nothing here self-skips off-JIT - every cell +holds on the reference bodies. +`test_rope_apply.das` - model-free: the CPU rope APPLICATION leaves (`dasllama_math`'s +`rope_scaled`, `rope_scaled_neox`, the `_tab` and `_part` twins and `rope_apply`) against an +in-test fp64 angle reference over the same theta schedule, on head sizes 40/64/96/256 with +tails, positions 0/1/7/37/512, two thetas, fscale and mscale arms and the `rope_freqs` divisor +- the bar is one rotation's f32 accumulation plus the f32 angle's own rounding, each with an +added-value poison that must exceed it. The `_tab` forms also ride a tight twin bar against the +un-tabled forms (loose only by the cross-compilation-unit cos/sin ulp drift), the `_part` forms +are bit-exact against a full apply over the gathered rotated prefix with the un-rotated dims +proven to pass through, and `rope_apply` is bit-exact against the leaf its `neox` flag names, +including the `use_ff = false` p-RoPE arm. These kernels were previously only the Apple-only +Metal rope tests' oracle; this file makes them a subject on every platform. +`test_prefill_cpu_kernels.das` - model-free: the prefill and KV CPU kernels no suite gated +before, on q4_K / q6_K / q4_0 synthetic disk planes built in-file. `matmul_kq_batch` and +`matmul_kq_batch_groupn` - the per-position and per-expert GEMV routes a tier with no kq batch +slot runs, bit-matched against per-(token,row) and per-(region,token) disk dots, with no skip on +any tier; where a kq-carrying backend can be pinned (restored on exit) the native batched +kernels additionally ride bit-for-bit against `matmul_kq_active` and the rows-core GEMVs. +`cvt_q8kv_to_f32` / `kv_row_to_f32` (all four overloads, at a non-zero `kv_head_off` base) +against `quantize_q8kv_row`'s stored blocks plus a half-step round-trip assert. +`requant_rows_q8` vs `quantize_q8_0` per row and `requant_rows_q8k_bs` vs an in-test Q8_K +reference, on both `par` arms. `repack_kq_weight`'s round trip: the pure grp repack read back +through `kq_grp_row_dot` on every tier, then the dispatch seam - byte-identical to that layout +where the tier carries kq slots, the registered identity where it does not. Every compare ships +an added-value poison control. `test_attn_span.das` - stocked suite; the non-causal image span (`eval_embd_` with `non_causal = true`): mask direction by perturbation (causal row 0 blind to the last row, span row 0 sees it), classic/blocked/flash agreement, and the flag-reset bit-exactness; plus the FUSED mid-turn span diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index 75b3cb30f1..5f69f6aa74 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -99,6 +99,7 @@ def suite_files(name : string) : array { // nolint:STYLE038 - a flat s "modules/dasLLAMA/tests/test_bench_records_schema.das", "modules/dasLLAMA/tests/test_box_ident.das", "modules/dasLLAMA/tests/test_box_profile.das", + "modules/dasLLAMA/tests/test_chat.das", "modules/dasLLAMA/tests/test_mtp_snapshot.das", "modules/dasLLAMA/tests/test_mtp_sampled_walk.das", "modules/dasLLAMA/tests/test_dasllama_lint_contracts.das", @@ -117,6 +118,8 @@ def suite_files(name : string) : array { // nolint:STYLE038 - a flat s "modules/dasLLAMA/tests/test_kernel_backend.das", "modules/dasLLAMA/tests/test_kgroup_repack.das", "modules/dasLLAMA/tests/test_kqformat.das", + "modules/dasLLAMA/tests/test_math_activations.das", + "modules/dasLLAMA/tests/test_math_elementwise.das", "modules/dasLLAMA/tests/test_matmul.das", "modules/dasLLAMA/tests/test_matmul_batch.das", "modules/dasLLAMA/tests/test_metal_gemm.das", @@ -128,12 +131,15 @@ def suite_files(name : string) : array { // nolint:STYLE038 - a flat s "modules/dasLLAMA/tests/test_jobque_tripwire.das", "modules/dasLLAMA/tests/test_plane.das", "modules/dasLLAMA/tests/test_ple_check.das", + "modules/dasLLAMA/tests/test_prefill_cpu_kernels.das", "modules/dasLLAMA/tests/test_program_roots.das", + "modules/dasLLAMA/tests/test_q8q8_family.das", "modules/dasLLAMA/tests/test_quant.das", "modules/dasLLAMA/tests/test_repack.das", "modules/dasLLAMA/tests/test_repack_lane_context.das", "modules/dasLLAMA/tests/test_rmsnorm.das", "modules/dasLLAMA/tests/test_rope.das", + "modules/dasLLAMA/tests/test_rope_apply.das", "modules/dasLLAMA/tests/test_run_suites.das", "modules/dasLLAMA/tests/test_run_summary.das", "modules/dasLLAMA/tests/test_silu.das", @@ -145,6 +151,7 @@ def suite_files(name : string) : array { // nolint:STYLE038 - a flat s "modules/dasLLAMA/tests/test_tts_textnorm.das", "modules/dasLLAMA/tests/test_tts_blocks.das", "modules/dasLLAMA/tests/test_tool_formats.das", + "modules/dasLLAMA/tests/test_tower_asr_kernels.das", "modules/dasLLAMA/tests/test_tower_helpers.das", "modules/dasLLAMA/tests/test_tune_interrupt.das", "modules/dasLLAMA/tests/test_unicode.das", diff --git a/modules/dasLLAMA/tests/test_groupn.das b/modules/dasLLAMA/tests/test_groupn.das index d4a6a3888d..67c2e72e72 100644 --- a/modules/dasLLAMA/tests/test_groupn.das +++ b/modules/dasLLAMA/tests/test_groupn.das @@ -9,9 +9,11 @@ require dasllama/dasllama_convert // quantize_q8_0 (deterministic require math // Region-list (groupN) GEMVs — the MoE decode dispatch fuse: N same-shape [d x n] projections in -// one dispatch, offs packing (weight, activation) element-offset pairs. Expected values come from -// the per-region single GEMV under the SAME backend pin (identical dots, only scheduling differs), -// so equality is exact per backend. Regions 0/1 share activation row 0; region 2 reads row 1. +// one dispatch, offs packing (weight, activation) element-offset pairs. Every q8 row carries TWO +// compares: the per-region single GEMV under the SAME backend pin (identical dots, only scheduling +// differs, so equality is exact per backend — it catches groupn-vs-gemv divergence and nothing a +// shared kernel bug would move), and the in-test fp64 dequant dot below, which shares no code with +// any kernel. Regions 0/1 share activation row 0; region 2 reads row 1. def private close(t : T?; got, want : float; what : string) { t |> success(abs(got - want) <= 1e-6 * max(abs(want), 1.0), "{what}: got {got} want {want}") @@ -20,6 +22,8 @@ def private close(t : T?; got, want : float; what : string) { let N = 64l let D = 4l // one interleaved group per region (laneq-compatible), no tail let NREG = 3l +let REF_ULP = 2.0e-6 // per-32-block fp32 fold margin — the bar scales with the accumulation length +let REF_POISON = 0.05 // added to one expected element; the bar must not swallow it def private build_offs() : array { return <- [N * D * 0l, 0l, N * D * 1l, 0l, N * D * 2l, N] @@ -37,6 +41,25 @@ def private slice_f(src : array; off, n : int64) : array { return <- [for (i in range64(off, off + n)); src[i]] } +// The independent truth for one q8 row: an fp64 dot over the region's dequantized Q8_0 blocks — +// int8 products summed exactly in int, both block scales applied in double. `env` is that sum +// taken on block magnitudes: the envelope the kernels' fp32 per-block folds round inside, which is +// what the bar is derived from. Layout-free, so it judges repacked and row-major backends alike. +def private ref_dot_q8_f64(wq : array; ws : array; woff : int64; + xq : array; xs : array; xoff : int64; n : int64) : tuple { + var res : tuple + for (bi in range64(n / 32l)) { + var iacc = 0 + for (k in range64(32l)) { + iacc += int(wq[woff + bi * 32l + k]) * int(xq[xoff + bi * 32l + k]) + } + let term = double(iacc) * double(ws[woff / 32l + bi]) * double(xs[xoff / 32l + bi]) + res.dot += term + res.env += abs(term) + } + return res +} + [test] def test_q8q8_groupn(t : T?) { t |> run("q8 groupN == per-region GEMV under every backend (repack where needed)") @(t : T?) { @@ -54,6 +77,19 @@ def test_q8q8_groupn(t : T?) { let wq0 <- quantize_q8_0(wf, NREG * D * N) let xq <- quantize_q8_0(xf, 2l * N) var offs <- build_offs() + // the fp64 reference and its bar, built once off the row-major quants: both are + // backend-free, since a repack only permutes the same weights + var refy : array + var refbar : array + refy |> resize(int(NREG * D)) + refbar |> resize(int(NREG * D)) + for (r in range64(NREG)) { + for (row in range64(D)) { + let rf = ref_dot_q8_f64(wq0.q, wq0.scales, r * D * N + row * N, xq.q, xq.scales, offs[r * 2l + 1l], N) + refy[r * D + row] = float(rf.dot) + refbar[r * D + row] = REF_ULP * float(N / 32l) * max(float(rf.env), 1.0) + } + } let saved = active_kernel_backend() for (name in kernel_backend_names()) { pin_kernel_backend(name) @@ -86,7 +122,11 @@ def test_q8q8_groupn(t : T?) { matmul_q8q8_groupn(y, wq, ws, offs, NREG, xq.q, xq.scales, N, D) for (i in range(int(NREG * D))) { close(t, y[i], want[i], "q8 row {i} (backend {name})") + t |> success(abs(y[i] - refy[i]) <= refbar[i], + "q8 row {i} vs fp64 dequant dot (backend {name}): got {y[i]} want {refy[i]} bar {refbar[i]}") } + t |> success(abs(y[0] - (refy[0] + REF_POISON)) > refbar[0], + "poison control: row 0 expectation + {REF_POISON} must exceed bar {refbar[0]} (backend {name})") } clear_kernel_backend_pin() if (saved != "") { diff --git a/modules/dasLLAMA/tests/test_math_activations.das b/modules/dasLLAMA/tests/test_math_activations.das new file mode 100644 index 0000000000..9691b9580a --- /dev/null +++ b/modules/dasLLAMA/tests/test_math_activations.das @@ -0,0 +1,634 @@ +options gen2 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_math +require dasllama/dasllama_par +require daslib/jobque_boost +require math + +//! The activation, norm and fp32 GEMM kernels - softmax_sink, swiglu_oai(4), softcap(4), +//! sigmoid_gate(4), silu_mul4, silu4_batch, l2_norm_rows and gemm_f32 / _jo / _uk_4x16 - against +//! closed forms and in-test fp64 references over shapes that reach every tail. + +var g_taint = 0u + +[init] +def init_activation_taint() { + g_taint = 0u +} + +[sideeffects] +def private opaque_seed(s : uint) : uint { + return s ^ g_taint +} + +let ACT_SWEEP <- [ 1, 2, 3, 4, 5, 7, 8, 9, 15, 16, 17, 31, 32, 33, 63, 64, 65, + 100, 127, 128, 129, 1023, 1024, 1025 ] +let L2_SHAPES <- [ 1, 1, 1, 7, 3, 8, 4, 64, 2, 127, 5, 33, 3, 128, 7, 4 ] +let GEMM_SHAPES <- [ 1, 1, 1, 2, 3, 5, 3, 7, 16, 4, 5, 16, 4, 16, 16, 5, 9, 17, + 7, 4, 33, 8, 32, 48, 9, 13, 19, 4, 1, 16, 6, 2, 31, 16, 40, 64 ] + +let POISON = 0.01 +let XMAG = 4.0 +let ZMAG = 8.0 +let OAI_MAG = 56.0 +let SENTINEL = 1.0e9 +let L2_EPS = 1.0e-6 + +struct ActSweep { + bad : int + inert : int + cases : int + elems : int + worst : float + worst_bar : float + worst_n : int + poison_inside : int +} + +def private lcg_pm(n : int; seed : uint; lo, hi : float) : array { + var st = opaque_seed(seed) + var a : array + a |> resize(n) + for (v in a) { + st = st * 1664525u + 1013904223u + let u01 = float(st >> 8u) * (1.0 / 16777216.0) + let mag = lo + (hi - lo) * u01 + v = ((st >> 4u) & 1u) == 0u ? -mag : mag + } + return <- a +} + +def private filled(n : int; v : float) : array { + var a : array + a |> resize(n) + for (x in a) { + x = v + } + return <- a +} + +def private check(var r : ActSweep; n : int; bar : float; got, want : array) { + r.cases++ + for (g, w in got, want) { + r.elems++ + let diff = abs(g - w) + if (diff > bar) { + r.bad++ + } + if (abs(g - (w + POISON)) <= bar) { + r.poison_inside++ + } + if (diff >= r.worst) { + r.worst = diff + r.worst_bar = bar + r.worst_n = n + } + } +} + +def private count_inert(var r : ActSweep; got, before : array) { + for (g, b in got, before) { + if (g == b) { + r.inert++ + } + } +} + +def private report(t : T?; name : string; r : ActSweep) { + t |> success(r.cases > 0 && r.elems > 0, "{name}: the sweep must run at least one case") + t |> equal(r.bad, 0, "{name}: values past the bar, worst |diff| {r.worst} against bar {r.worst_bar} at n={r.worst_n}") + t |> equal(r.poison_inside, 0, "{name}: poison legs inside the bar - a bar nothing exceeds discriminates nothing") + feint("{name}: {r.elems} elements, worst |diff| {r.worst} against bar {r.worst_bar} at n={r.worst_n}\n") +} + +def private report_moved(t : T?; name : string; r : ActSweep) { + report(t, name, r) + t |> equal(r.inert, 0, "{name}: elements the kernel left exactly where it found them") +} + +def private softmax_bar(n : int) : float { + return 1.0e-6 + 4.0e-7 * float(n) +} + +def private softmax_sink_ref(x : array; sink : float) : tuple; float> { + var mx = double(sink) + for (v in x) { + mx = max(mx, double(v)) + } + var terms : array + terms |> resize(length(x)) + let sk = exp(max(double(sink) - mx, -80.0lf)) + var sum = sk + for (e, v in terms, x) { + e = exp(max(double(v) - mx, -80.0lf)) + sum += e + } + var out : array + out |> resize(length(x)) + for (o, e in out, terms) { + o = float(e / sum) + } + return out => float(sk / sum) +} + +def private sweep_softmax_sink(sink : float; seed : uint) : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let x0 <- lcg_pm(n, opaque_seed(seed) + uint(n), 0.25, 6.0) + let want <- softmax_sink_ref(x0, sink) + var x := x0 + softmax_sink(x, int64(n), sink) + check(r, n, softmax_bar(n), x, want._0) + var total = 0.0lf + for (v in x) { + total += double(v) + } + if (abs(total + double(want._1) - 1.0lf) > double(softmax_bar(n))) { + r.bad++ + } + } + return r +} + +def private sweep_softmax_sink_shift() : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let x0 <- lcg_pm(n, opaque_seed(0x71a5u) + uint(n), 0.25, 6.0) + let want <- softmax_sink_ref(x0, 1.5) + var plain := x0 + softmax_sink(plain, int64(n), 1.5) + check(r, n, softmax_bar(n), plain, want._0) + var shifted : array + shifted |> resize(n) + for (s, v in shifted, x0) { + s = v + 5.0 + } + softmax_sink(shifted, int64(n), 6.5) + check(r, n, softmax_bar(n), shifted, plain) + } + return r +} + +def private softcap_ref(x : array; cap : float) : array { + var out : array + out |> resize(length(x)) + for (o, v in out, x) { + o = float(double(cap) * tanh(double(v) / double(cap))) + } + return <- out +} + +def private softcap_bar(cap : float) : float { + return 1.0e-6 + 1.0e-6 * cap +} + +def private sweep_softcap(cap : float; vec : bool) : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let x0 <- lcg_pm(n, opaque_seed(vec ? 0x2c81u : 0x9b44u) + uint(n), 0.5, 200.0) + let want <- softcap_ref(x0, cap) + var x := x0 + if (vec) { + softcap4(x, int64(n), cap) + } else { + softcap(x, int64(n), cap) + } + check(r, n, softcap_bar(cap), x, want) + count_inert(r, x, x0) + for (v in x) { + if (abs(v) > cap) { + r.bad++ + } + } + } + return r +} + +def private sigmoid_gate_ref(x, g : array) : array { + var out : array + out |> resize(length(x)) + for (o, xv, gv in out, x, g) { + o = float(double(xv) / (1.0lf + exp(-double(gv)))) + } + return <- out +} + +def private sweep_sigmoid_gate(vec : bool) : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let x0 <- lcg_pm(n, opaque_seed(0x5ab3u) + uint(n), 0.25, XMAG) + let g <- lcg_pm(n, opaque_seed(0x9d10u) + uint(n), 0.25, ZMAG) + let want <- sigmoid_gate_ref(x0, g) + var x := x0 + if (vec) { + sigmoid_gate4(unsafe(addr(x[0])), unsafe(addr(g[0])), int64(n)) + } else { + sigmoid_gate(unsafe(addr(x[0])), unsafe(addr(g[0])), int64(n)) + } + check(r, n, 1.0e-6 + 1.0e-6 * XMAG, x, want) + count_inert(r, x, x0) + } + return r +} + +def private swiglu_oai_ref(g, u : array) : array { + var out : array + out |> resize(length(g)) + for (o, gv, uv in out, g, u) { + let xv = min(double(gv), 7.0lf) + let yv = clamp(double(uv), -7.0lf, 7.0lf) + o = float((xv / (1.0lf + exp(-1.702lf * xv))) * (yv + 1.0lf)) + } + return <- out +} + +def private sweep_swiglu_oai(vec : bool) : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let g0 <- lcg_pm(n, opaque_seed(0x4411u) + uint(n), 0.25, 12.0) + let u <- lcg_pm(n, opaque_seed(0x77c2u) + uint(n), 0.25, 12.0) + let want <- swiglu_oai_ref(g0, u) + var g := g0 + if (vec) { + swiglu_oai4(unsafe(addr(g[0])), unsafe(addr(u[0])), int64(n)) + } else { + swiglu_oai(unsafe(addr(g[0])), unsafe(addr(u[0])), int64(n)) + } + check(r, n, 1.0e-6 + 1.0e-6 * OAI_MAG, g, want) + } + return r +} + +struct ClampCase { + clamp_diffs : int + control_same : int +} + +def private oai_pair_diffs(ga, gb, ua, ub : array; vec : bool) : int { + var a := ga + var b := gb + let n = long_length(ga) + if (vec) { + swiglu_oai4(unsafe(addr(a[0])), unsafe(addr(ua[0])), n) + swiglu_oai4(unsafe(addr(b[0])), unsafe(addr(ub[0])), n) + } else { + swiglu_oai(unsafe(addr(a[0])), unsafe(addr(ua[0])), n) + swiglu_oai(unsafe(addr(b[0])), unsafe(addr(ub[0])), n) + } + var d = 0 + for (x, y in a, b) { + if (x != y) { + d++ + } + } + return d +} + +def private oai_clamp_case(vec : bool) : ClampCase { + let n = 18 + let u <- lcg_pm(n, opaque_seed(0x3f00u), 1.5, 5.0) + var res = ClampCase() + res.clamp_diffs += oai_pair_diffs(filled(n, 20.0), filled(n, 7.0), u, u, vec) + res.clamp_diffs += oai_pair_diffs(filled(n, 2.0), filled(n, 2.0), filled(n, 20.0), filled(n, 7.0), vec) + res.clamp_diffs += oai_pair_diffs(filled(n, 2.0), filled(n, 2.0), filled(n, -20.0), filled(n, -7.0), vec) + res.control_same = n - oai_pair_diffs(filled(n, 3.0), filled(n, 7.0), u, u, vec) + return res +} + +def private silu_mul_ref(a, z : array) : array { + var out : array + out |> resize(length(a)) + for (o, av, zv in out, a, z) { + let d = double(zv) + o = float(double(av) * (d / (1.0lf + exp(-d)))) + } + return <- out +} + +def private sweep_silu_mul4() : ActSweep { + var r = ActSweep() + for (n in ACT_SWEEP) { + let a <- lcg_pm(n, opaque_seed(0x6e21u) + uint(n), 0.25, XMAG) + let z <- lcg_pm(n, opaque_seed(0xf13cu) + uint(n), 0.25, ZMAG) + let want <- silu_mul_ref(a, z) + let before <- filled(n, SENTINEL) + var o := before + silu_mul4(unsafe(addr(o[0])), unsafe(addr(a[0])), unsafe(addr(z[0])), int64(n)) + check(r, n, 1.0e-6 + 1.0e-6 * XMAG * ZMAG, o, want) + count_inert(r, o, before) + } + return r +} + +def private silu_ref(x : array) : array { + var out : array + out |> resize(length(x)) + for (o, v in out, x) { + let d = double(v) + o = float(d / (1.0lf + exp(-d))) + } + return <- out +} + +struct BatchCase { + split_diffs : int + dispatches : int64 + elems : int +} + +def private silu4_batch_case(var r : ActSweep; d, npos : int) : BatchCase { + var res = BatchCase(elems = d * npos) + let x0 <- lcg_pm(d * npos, opaque_seed(0x84c1u) + uint(d * 131 + npos), 0.25, ZMAG) + var whole := x0 + silu4(whole, int64(d) * int64(npos)) + var batched := x0 + let before = get_disp_count() + silu4_batch(batched, int64(d), int64(npos)) + res.dispatches = get_disp_count() - before + let want <- silu_ref(x0) + check(r, d * npos, 1.0e-6 + 1.0e-6 * ZMAG, whole, want) + count_inert(r, batched, x0) + for (bv, wv in batched, whole) { + if (bv != wv) { + res.split_diffs++ + } + } + return res +} + +def private l2_ref(v : array; rows, row_size : int; eps : float) : array { + var out : array + out |> resize(rows * row_size) + for (r in range(rows)) { + let base = r * row_size + var sum = 0.0lf + for (i in range(row_size)) { + let x = v[base + i] + sum += double(x * x) + } + let scale = 1.0 / max(sqrt(float(sum)), eps) + for (i in range(row_size)) { + out[base + i] = v[base + i] * scale + } + } + return <- out +} + +def private l2_bar(row_size : int) : float { + return 1.0e-6 + 2.0e-7 * float(row_size) +} + +def private sweep_l2_norm_rows() : ActSweep { + var r = ActSweep() + for (s in range(length(L2_SHAPES) / 2)) { + let rows = L2_SHAPES[s * 2] + let row_size = L2_SHAPES[s * 2 + 1] + let v0 <- lcg_pm(rows * row_size, opaque_seed(0xa1b2u) + uint(s), 1.25, 3.0) + let want <- l2_ref(v0, rows, row_size, L2_EPS) + var v := v0 + l2_norm_rows(unsafe(addr(v[0])), int64(rows), int64(row_size), L2_EPS) + check(r, row_size, l2_bar(row_size), v, want) + count_inert(r, v, v0) + for (rw in range(rows)) { + var sum = 0.0lf + for (i in range(row_size)) { + sum += double(v[rw * row_size + i]) * double(v[rw * row_size + i]) + } + if (abs(sqrt(sum) - 1.0lf) > double(l2_bar(row_size))) { + r.bad++ + } + } + } + return r +} + +def private l2_eps_floor() : ActSweep { + var r = ActSweep() + let rows = 3 + let row_size = 8 + var v0 <- filled(rows * row_size, 0.0) + for (i in range(row_size)) { + v0[row_size + i] = 1.0e-9 + } + let want <- l2_ref(v0, rows, row_size, L2_EPS) + var v := v0 + l2_norm_rows(unsafe(addr(v[0])), int64(rows), int64(row_size), L2_EPS) + check(r, row_size, l2_bar(row_size), v, want) + return r +} + +struct GemmCase { + bad : int + jo_diffs : int + cases : int + elems : int + poison_inside : int + worst : float + worst_bar : float + worst_k : int +} + +def private gemm_ref(c0, a, b : array; M, K, N : int) : array { + var out : array + out |> resize(M * N) + for (i in range(M)) { + for (j in range(N)) { + var acc = double(c0[i * N + j]) + for (k in range(K)) { + acc += double(a[i * K + k]) * double(b[k * N + j]) + } + out[i * N + j] = float(acc) + } + } + return <- out +} + +def private gemm_bar(K : int) : float { + return 6.0e-8 * float(K) * (float(K) + 2.0) + 1.0e-6 +} + +def private sweep_gemm() : GemmCase { + var r = GemmCase() + for (s in range(length(GEMM_SHAPES) / 3)) { + let M = GEMM_SHAPES[s * 3] + let K = GEMM_SHAPES[s * 3 + 1] + let N = GEMM_SHAPES[s * 3 + 2] + let a <- lcg_pm(M * K, opaque_seed(0x1717u) + uint(s), 0.25, 1.0) + let b <- lcg_pm(K * N, opaque_seed(0x2828u) + uint(s), 0.25, 1.0) + let c0 <- lcg_pm(M * N, opaque_seed(0x3939u) + uint(s), 0.25, 1.0) + let want <- gemm_ref(c0, a, b, M, K, N) + var c1 := c0 + gemm_f32(unsafe(addr(c1[0])), unsafe(addr(a[0])), unsafe(addr(b[0])), int64(M), int64(K), int64(N)) + var c2 := c0 + gemm_f32_jo(unsafe(addr(c2[0])), unsafe(addr(a[0])), unsafe(addr(b[0])), int64(M), int64(K), int64(N)) + r.cases++ + tally_gemm(r, K, c1, c2, want) + } + return r +} + +def private tally_gemm(var r : GemmCase; K : int; c1, c2, want : array) { + let bar = gemm_bar(K) + for (g, j, w in c1, c2, want) { + r.elems++ + let diff = abs(g - w) + if (diff > bar || abs(j - w) > bar) { + r.bad++ + } + if (g != j) { + r.jo_diffs++ + } + if (abs(g - (w + POISON)) <= bar) { + r.poison_inside++ + } + if (diff >= r.worst) { + r.worst = diff + r.worst_bar = bar + r.worst_k = K + } + } +} + +def private zero_size_moves() : int { + let before <- lcg_pm(8, opaque_seed(0x0f01u), 0.25, 2.0) + let g <- lcg_pm(8, opaque_seed(0x0f02u), 0.25, 2.0) + var x := before + softmax_sink(x, 0l, 1.0) + softcap(x, 0l, 30.0) + softcap4(x, 0l, 30.0) + silu4_batch(x, 8l, 0l) + sigmoid_gate(unsafe(addr(x[0])), unsafe(addr(g[0])), 0l) + sigmoid_gate4(unsafe(addr(x[0])), unsafe(addr(g[0])), 0l) + swiglu_oai(unsafe(addr(x[0])), unsafe(addr(g[0])), 0l) + swiglu_oai4(unsafe(addr(x[0])), unsafe(addr(g[0])), 0l) + silu_mul4(unsafe(addr(x[0])), unsafe(addr(g[0])), unsafe(addr(g[0])), 0l) + l2_norm_rows(unsafe(addr(x[0])), 0l, 8l, L2_EPS) + var moved = 0 + for (v, b in x, before) { + if (v != b) { + moved++ + } + } + return moved +} + +[test] +def test_softmax_sink(t : T?) { + t |> run("softmax_sink against an fp64 reference, the row plus the sink mass summing to one") @(t : T?) { + report(t, "softmax_sink negligible sink", sweep_softmax_sink(-1000.0, 0x1001u)) + report(t, "softmax_sink dominant sink", sweep_softmax_sink(8.0, 0x1002u)) + } + t |> run("softmax_sink is invariant to a shift of the scores and the sink together") @(t : T?) { + report(t, "softmax_sink shift", sweep_softmax_sink_shift()) + } +} + +[test] +def test_softcap(t : T?) { + t |> run("softcap is cap*tanh(x/cap) and never leaves the band") @(t : T?) { + report_moved(t, "softcap cap=30", sweep_softcap(30.0, false)) + report_moved(t, "softcap cap=50", sweep_softcap(50.0, false)) + } + t |> run("softcap4 holds the same closed form through its tail") @(t : T?) { + report_moved(t, "softcap4 cap=30", sweep_softcap(30.0, true)) + report_moved(t, "softcap4 cap=50", sweep_softcap(50.0, true)) + } +} + +[test] +def test_sigmoid_gate(t : T?) { + t |> run("sigmoid_gate scales by the logistic of the gate row") @(t : T?) { + report_moved(t, "sigmoid_gate", sweep_sigmoid_gate(false)) + } + t |> run("sigmoid_gate4 holds the same form through its tail") @(t : T?) { + report_moved(t, "sigmoid_gate4", sweep_sigmoid_gate(true)) + } +} + +[test] +def test_swiglu_oai(t : T?) { + t |> run("swiglu_oai against an fp64 reference carrying the trained clamps") @(t : T?) { + report(t, "swiglu_oai", sweep_swiglu_oai(false)) + report(t, "swiglu_oai4", sweep_swiglu_oai(true)) + } + t |> run("the gate limit and the up-branch band are the activation, not an optimization") @(t : T?) { + let scalar = oai_clamp_case(false) + t |> equal(scalar.clamp_diffs, 0, "swiglu_oai: a value past the limit must read as the limit") + t |> equal(scalar.control_same, 0, "swiglu_oai: gates inside the limit must not collapse together") + let vec = oai_clamp_case(true) + t |> equal(vec.clamp_diffs, 0, "swiglu_oai4: a value past the limit must read as the limit") + t |> equal(vec.control_same, 0, "swiglu_oai4: gates inside the limit must not collapse together") + } +} + +[test] +def test_silu_mul4(t : T?) { + t |> run("silu_mul4 writes a*silu(z) over a sentinel destination") @(t : T?) { + report_moved(t, "silu_mul4", sweep_silu_mul4()) + } +} + +[test] +def test_silu4_batch(t : T?) { + t |> run("silu4_batch splits rows without moving a bit, threaded and inline") @(t : T?) { + var r = ActSweep() + let inline_wide = silu4_batch_case(r, 256, 512) + var threaded = BatchCase() + with_job_que() { + threaded = silu4_batch_case(r, 256, 512) + } + report_moved(t, "silu4_batch", r) + t |> equal(inline_wide.split_diffs, 0, "silu4_batch inline: elements differing from silu4 over the whole buffer") + t |> equal(threaded.split_diffs, 0, "silu4_batch threaded: elements differing from silu4 over the whole buffer") + t |> success(!is_job_que_available(), "the inline leg needs a que-less process to be one lane") + t |> equal(inline_wide.dispatches, 0l, "a que-less process must run silu4_batch inline") + t |> success(threaded.dispatches > 0l, "the threaded leg must dispatch, else both legs are the same one lane") + } + t |> run("silu4_batch falls back whole where the row length or the row count forbids a split") @(t : T?) { + var r = ActSweep() + with_job_que() { + let odd = silu4_batch_case(r, 250, 64) + let single = silu4_batch_case(r, 64, 1) + let tail = silu4_batch_case(r, 4, 1025) + t |> equal(odd.split_diffs, 0, "a row length off the float4 grouping must run silu4 whole") + t |> equal(single.split_diffs, 0, "a single row must run silu4 whole") + t |> equal(tail.split_diffs, 0, "a narrow many-row image must still land the same bits") + t |> equal(odd.dispatches, 0l, "a row length off the float4 grouping must not dispatch") + t |> equal(single.dispatches, 0l, "a single row must not dispatch") + t |> success(tail.dispatches > 0l, "a narrow many-row image must still split across lanes") + } + report_moved(t, "silu4_batch fallbacks", r) + } +} + +[test] +def test_l2_norm_rows(t : T?) { + t |> run("l2_norm_rows drives every row to unit length") @(t : T?) { + report_moved(t, "l2_norm_rows", sweep_l2_norm_rows()) + } + t |> run("l2_norm_rows floors the divisor at eps") @(t : T?) { + report(t, "l2_norm_rows eps floor", l2_eps_floor()) + } +} + +[test] +def test_gemm_f32(t : T?) { + t |> run("gemm_f32 and gemm_f32_jo accumulate the same fp64 product over tiles and both strips") @(t : T?) { + let r = sweep_gemm() + t |> equal(r.cases, length(GEMM_SHAPES) / 3, "every declared shape must run") + t |> success(r.elems > 0, "the sweep must compare at least one element") + t |> equal(r.bad, 0, "gemm_f32: elements past the bar, worst |diff| {r.worst} against bar {r.worst_bar} at K={r.worst_k}") + t |> equal(r.jo_diffs, 0, "gemm_f32_jo must be bit-identical to gemm_f32") + t |> equal(r.poison_inside, 0, "gemm_f32: poison legs inside the bar - a bar nothing exceeds discriminates nothing") + feint("gemm_f32: {r.elems} elements, worst |diff| {r.worst} against bar {r.worst_bar} at K={r.worst_k}\n") + } +} + +[test] +def test_zero_size(t : T?) { + t |> run("a zero count is a no-op on every activation and norm kernel") @(t : T?) { + t |> equal(zero_size_moves(), 0, "a zero count must leave the buffer untouched") + } +} diff --git a/modules/dasLLAMA/tests/test_math_elementwise.das b/modules/dasLLAMA/tests/test_math_elementwise.das new file mode 100644 index 0000000000..fc86ba7e5b --- /dev/null +++ b/modules/dasLLAMA/tests/test_math_elementwise.das @@ -0,0 +1,395 @@ +options gen2 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_math +require math + +//! The [tuned] elementwise kernels - dot, dot_bf16, axpy, add_inplace, add_scale_inplace, +//! mul_inplace, scale_inplace, copy_floats - swept over tail lengths against fp64 references. + +var g_taint = 0u + +[init] +def init_elementwise_taint() { + g_taint = 0u +} + +[sideeffects] +def private opaque_seed(s : uint) : uint { + return s ^ g_taint +} + +let SWEEP <- [ 1, 2, 3, 4, 5, 7, 8, 9, 15, 16, 17, 23, 24, 25, 31, 32, 33, + 63, 64, 65, 100, 127, 128, 129, 1023, 1024, 1025 ] + +let TEST_SCALE = -0.75 +let SENTINEL = 1.0e9 +let POISON = 0.5 +let ROUND_U = 1.0e-7 +let BAR_FLOOR = 1.0e-6 + +struct KernelSweep { + bad : int + inert : int + cases : int + worst : float + worst_bar : float + worst_n : int + poison_inside : int +} + +def private note(var r : KernelSweep; n : int; diff, bar : float) { + if (diff >= r.worst) { + r.worst = diff + r.worst_bar = bar + r.worst_n = n + } +} + +def private dot_bar(n : int) : float { + return ROUND_U * float(n) * float(n) + BAR_FLOOR +} + +def private lcg_grid(n : int; seed : uint; step : float; signed_values : bool) : array { + var st = opaque_seed(seed) + var a : array + a |> resize(n) + for (v in a) { + st = st * 1664525u + 1013904223u + let k = int((st >> 9u) & 31u) + 1 + let neg = signed_values && ((st >> 20u) & 1u) == 0u + v = float(neg ? -k : k) * step + } + return <- a +} + +def private lcg_unit(n : int; seed : uint) : array { + var st = opaque_seed(seed) + var a : array + a |> resize(n) + for (v in a) { + st = st * 1664525u + 1013904223u + v = float(st >> 8u) * (2.0 / 16777216.0) - 1.0 + } + return <- a +} + +def private to_bf16(src : array) : array { + var b : array + b |> resize(length(src)) + for (w, v in b, src) { + w = uint16(unsafe(reinterpret(v)) >> 16u) + } + return <- b +} + +def private bf16_value(w : uint16) : float { + return unsafe(reinterpret(uint(w) << 16u)) +} + +def private tally(var r : KernelSweep; n : int; got, want, before : array) { + r.cases++ + for (g, w, b in got, want, before) { + if (g != w) { + r.bad++ + note(r, n, abs(g - w), 0.0) + } + if (g == b) { + r.inert++ + } + } +} + +def private report_cases(t : T?; name : string; r : KernelSweep) { + t |> equal(r.cases, length(SWEEP), "{name}: every swept length must run") +} + +def private report_exact(t : T?; name : string; r : KernelSweep) { + report_cases(t, name, r) + t |> equal(r.bad, 0, "{name}: values off the in-test fp64 reference, worst |diff| {r.worst} at n={r.worst_n}") +} + +def private report_inplace(t : T?; name : string; r : KernelSweep) { + report_exact(t, name, r) + t |> equal(r.inert, 0, "{name}: elements the kernel left exactly where it found them") +} + +def private report_bar(t : T?; name : string; r : KernelSweep) { + report_cases(t, name, r) + t |> equal(r.bad, 0, "{name}: lengths past the bar, worst |diff| {r.worst} against bar {r.worst_bar} at n={r.worst_n}") + t |> equal(r.poison_inside, 0, "{name}: poison legs inside the bar - a bar nothing exceeds discriminates nothing") + feint("{name}: worst |diff| {r.worst} against bar {r.worst_bar} at n={r.worst_n}\n") +} + +def private sweep_dot_exact() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let a <- lcg_grid(n, opaque_seed(0x11a3u) + uint(n), 0.125, true) + let b <- lcg_grid(n, opaque_seed(0x9e37u) + uint(n), 0.375, true) + var want = 0.0lf + for (av, bv in a, b) { + want += double(av) * double(bv) + } + let got = dot(unsafe(addr(a[0])), unsafe(addr(b[0])), int64(n)) + r.cases++ + if (got != float(want)) { + r.bad++ + note(r, n, abs(got - float(want)), 0.0) + } + } + return r +} + +def private sweep_dot_bar() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let a <- lcg_unit(n, opaque_seed(0x4d2bu) + uint(n)) + let b <- lcg_unit(n, opaque_seed(0xa51fu) + uint(n)) + var want = 0.0lf + for (av, bv in a, b) { + want += double(av) * double(bv) + } + let got = dot(unsafe(addr(a[0])), unsafe(addr(b[0])), int64(n)) + let bar = dot_bar(n) + let diff = abs(got - float(want)) + r.cases++ + if (diff > bar) { + r.bad++ + } + if (abs(got - (float(want) + POISON)) <= bar) { + r.poison_inside++ + } + note(r, n, diff, bar) + } + return r +} + +def private sweep_dot_bf16_exact() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let a <- lcg_grid(n, opaque_seed(0x3c07u) + uint(n), 0.125, true) + let bf <- lcg_grid(n, opaque_seed(0x7b91u) + uint(n), 0.375, true) + let b <- to_bf16(bf) + var want = 0.0lf + for (av, bw in a, b) { + want += double(av) * double(bf16_value(bw)) + } + let got = dot_bf16(unsafe(addr(a[0])), unsafe(addr(b[0])), int64(n)) + r.cases++ + if (got != float(want)) { + r.bad++ + note(r, n, abs(got - float(want)), 0.0) + } + } + return r +} + +def private sweep_dot_bf16_bar() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let a <- lcg_unit(n, opaque_seed(0x2f6eu) + uint(n)) + let bf <- lcg_unit(n, opaque_seed(0xc0deu) + uint(n)) + let b <- to_bf16(bf) + var want = 0.0lf + for (av, bw in a, b) { + want += double(av) * double(bf16_value(bw)) + } + let got = dot_bf16(unsafe(addr(a[0])), unsafe(addr(b[0])), int64(n)) + let bar = dot_bar(n) + let diff = abs(got - float(want)) + r.cases++ + if (diff > bar) { + r.bad++ + } + if (abs(got - (float(want) + POISON)) <= bar) { + r.poison_inside++ + } + note(r, n, diff, bar) + } + return r +} + +def private sweep_copy_floats() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let s <- lcg_unit(n, opaque_seed(0x51c0u) + uint(n)) + var before : array + before |> resize(n) + for (v in before) { + v = SENTINEL + } + var d := before + copy_floats(unsafe(addr(d[0])), unsafe(addr(s[0])), int64(n)) + tally(r, n, d, s, before) + } + return r +} + +def private sweep_add_inplace() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let before <- lcg_grid(n, opaque_seed(0x6a12u) + uint(n), 0.125, true) + let s <- lcg_grid(n, opaque_seed(0xb33fu) + uint(n), 0.375, true) + var want : array + want |> resize(n) + for (w, b, sv in want, before, s) { + w = float(double(b) + double(sv)) + } + var d := before + add_inplace(unsafe(addr(d[0])), unsafe(addr(s[0])), int64(n)) + tally(r, n, d, want, before) + } + return r +} + +def private sweep_add_scale_inplace() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let before <- lcg_grid(n, opaque_seed(0x1de5u) + uint(n), 0.125, false) + let s <- lcg_grid(n, opaque_seed(0x8c44u) + uint(n), 0.375, false) + var want : array + want |> resize(n) + for (w, b, sv in want, before, s) { + w = float((double(b) + double(sv)) * double(TEST_SCALE)) + } + var d := before + add_scale_inplace(unsafe(addr(d[0])), unsafe(addr(s[0])), TEST_SCALE, int64(n)) + tally(r, n, d, want, before) + } + return r +} + +def private sweep_mul_inplace() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let before <- lcg_grid(n, opaque_seed(0x9077u) + uint(n), 0.125, true) + let s <- lcg_grid(n, opaque_seed(0x2bb8u) + uint(n), 0.375, true) + var want : array + want |> resize(n) + for (w, b, sv in want, before, s) { + w = float(double(b) * double(sv)) + } + var d := before + mul_inplace(unsafe(addr(d[0])), unsafe(addr(s[0])), int64(n)) + tally(r, n, d, want, before) + } + return r +} + +def private sweep_scale_inplace() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let before <- lcg_grid(n, opaque_seed(0x4e19u) + uint(n), 0.125, true) + var want : array + want |> resize(n) + for (w, b in want, before) { + w = float(double(b) * double(TEST_SCALE)) + } + var d := before + scale_inplace(unsafe(addr(d[0])), TEST_SCALE, int64(n)) + tally(r, n, d, want, before) + } + return r +} + +def private sweep_axpy() : KernelSweep { + var r = KernelSweep() + for (n in SWEEP) { + let before <- lcg_grid(n, opaque_seed(0x7231u) + uint(n), 0.125, true) + let s <- lcg_grid(n, opaque_seed(0xd4a0u) + uint(n), 0.375, true) + var want : array + want |> resize(n) + for (w, b, sv in want, before, s) { + w = float(double(b) + double(TEST_SCALE) * double(sv)) + } + var d := before + axpy(unsafe(addr(d[0])), unsafe(addr(s[0])), TEST_SCALE, int64(n)) + tally(r, n, d, want, before) + } + return r +} + +def private zero_length_moves() : int { + let before <- lcg_grid(4, opaque_seed(0x0e11u), 0.125, true) + let s <- lcg_grid(4, opaque_seed(0x0e12u), 0.375, true) + var d := before + unsafe { + copy_floats(addr(d[0]), addr(s[0]), 0l) + add_inplace(addr(d[0]), addr(s[0]), 0l) + add_scale_inplace(addr(d[0]), addr(s[0]), TEST_SCALE, 0l) + mul_inplace(addr(d[0]), addr(s[0]), 0l) + scale_inplace(addr(d[0]), TEST_SCALE, 0l) + axpy(addr(d[0]), addr(s[0]), TEST_SCALE, 0l) + } + var moved = 0 + for (x, b in d, before) { + if (x != b) { + moved++ + } + } + return moved +} + +def private zero_length_dots() : float { + let a <- lcg_grid(4, opaque_seed(0x0e13u), 0.125, true) + let bf <- lcg_grid(4, opaque_seed(0x0e14u), 0.375, true) + let b <- to_bf16(bf) + unsafe { + return abs(dot(addr(a[0]), addr(a[0]), 0l)) + abs(dot_bf16(addr(a[0]), addr(b[0]), 0l)) + } +} + +[test] +def test_dot(t : T?) { + t |> run("dot bit-matches an fp64 accumulation on inputs fp32 cannot round") @(t : T?) { + report_exact(t, "dot exact", sweep_dot_exact()) + } + t |> run("dot against an fp64 accumulation on unit-range inputs, with the bar's control") @(t : T?) { + report_bar(t, "dot bar", sweep_dot_bar()) + } +} + +[test] +def test_dot_bf16(t : T?) { + t |> run("dot_bf16 bit-matches an fp64 accumulation over exactly-bf16 operands") @(t : T?) { + report_exact(t, "dot_bf16 exact", sweep_dot_bf16_exact()) + } + t |> run("dot_bf16 against an fp64 accumulation over the widened operands, with the bar's control") @(t : T?) { + report_bar(t, "dot_bf16 bar", sweep_dot_bf16_bar()) + } +} + +[test] +def test_copy_and_add(t : T?) { + t |> run("copy_floats replaces every sentinel with the source bit for bit") @(t : T?) { + report_inplace(t, "copy_floats", sweep_copy_floats()) + } + t |> run("add_inplace is exact on dyadic operands") @(t : T?) { + report_inplace(t, "add_inplace", sweep_add_inplace()) + } + t |> run("add_scale_inplace is exact and its sign flip cannot return the input") @(t : T?) { + report_inplace(t, "add_scale_inplace", sweep_add_scale_inplace()) + } +} + +[test] +def test_mul_scale_axpy(t : T?) { + t |> run("mul_inplace is exact on operands no multiplier of which is one") @(t : T?) { + report_inplace(t, "mul_inplace", sweep_mul_inplace()) + } + t |> run("scale_inplace is exact at a dyadic scale") @(t : T?) { + report_inplace(t, "scale_inplace", sweep_scale_inplace()) + } + t |> run("axpy is exact on dyadic operands") @(t : T?) { + report_inplace(t, "axpy", sweep_axpy()) + } +} + +[test] +def test_zero_length(t : T?) { + t |> run("a zero count is a no-op on every elementwise kernel") @(t : T?) { + t |> equal(zero_length_moves(), 0, "a zero count must leave the destination untouched") + t |> equal(zero_length_dots(), 0.0, "a zero count must answer an empty sum") + } +} diff --git a/modules/dasLLAMA/tests/test_mxfp4.das b/modules/dasLLAMA/tests/test_mxfp4.das index 3853a29150..8de23c344b 100644 --- a/modules/dasLLAMA/tests/test_mxfp4.das +++ b/modules/dasLLAMA/tests/test_mxfp4.das @@ -165,7 +165,8 @@ def test_matmul_mx4q8(t : T?) { } } t |> run("gen tier: repack + interleaved GEMV matches the scalar leaf (incl. tail rows)") @(t : T?) { - if (!has_backend("arm64-gen")) { // registered on arm64 + JIT (with dasLLVM) only + if (!has_backend("arm64-gen")) { + t |> skip("backend 'arm64-gen' is not registered here (arm64 + JIT with dasLLVM only)") return } // d = 6 = one 4-row group + 2 row-major tail rows diff --git a/modules/dasLLAMA/tests/test_parity.das b/modules/dasLLAMA/tests/test_parity.das index 77fd6b7536..567c822d30 100644 --- a/modules/dasLLAMA/tests/test_parity.das +++ b/modules/dasLLAMA/tests/test_parity.das @@ -5,6 +5,9 @@ options _dasllama_internal = true require dastest/testing_boost public require dasllama/dasllama // the public facade — public-path tests exercise the API through it (engine + arch registrations + chat) require llvm/daslib/f16_cvt // f16_to_f32 — the tied-cls arm reads the live scale plane +require dasllama/dasllama_math // pin_kernel_backend / kernel_backend_names / active_kernel_backend — the facade wraps none of them +require dasllama/dasllama_env // g_env_engine.pin_backend +require daslib/defer // the backend pin is restored on every path out of a pinned cell require daslib/jobque_boost require daslib/fio require _model_tier // models_dir() + model_available() presence/size-tier gate (see the file's header) @@ -15,6 +18,33 @@ require ../performance/model_specs.das // the model-set table — parity evide // frozen under; regenerate with harness/parity.sh. The board's --parity pregate reads the same rows. // Models are gitignored multi-GB files: absent = clean skip (a real local regression, a no-op in CI). +// DASLLAMA_PIN_BACKEND points these gates at one kernel backend, the way the benches read it — +// the pin lands BEFORE the load, so select_matmul_backend_for_load_ picks the layout it repacks +// into. pin_kernel_backend is process-wide, so the entry backend is restored on every path out. +// Unknown name => the body never runs and the cell skips naming it, never a silent run on the +// default backend. +def private with_parity_backend(t : T?; blk : block<() : void>) { + let name = g_env_engine.pin_backend + if (empty(name)) { + invoke(blk) + return + } + var names <- kernel_backend_names() + let known = find_index(names, name) >= 0 + delete names + if (!known) { + t |> skip("DASLLAMA_PIN_BACKEND='{name}' is not a registered kernel backend on this box") + return + } + let prev = active_kernel_backend() + defer() { + clear_kernel_backend_pin() + select_kernel_backend(prev) // "" is a no-op: nothing had activated on entry + } + pin_kernel_backend(name) + invoke(blk) +} + def private same(a, b : array) : bool { if (length(a) != length(b)) { return false @@ -184,27 +214,28 @@ def private parity_spec_arms(t : T?; spec : ModelSpec; path : string) { if (tr.config.seq_len > 2048l) { tr.config.seq_len = 2048l } + let be = " [backend {active_kernel_backend()}]" // read AFTER the load: that is where the pin resolves var prompt <- (empty(spec.parity.prompt_text) ? clone_to_move(spec.parity.prompt) : encode(tr, spec.parity.prompt_text, true, false)) if (spec.parity.require_dn_kq) { t |> success(dn_kq_engaged(tr.dnqkv_fmt) > 0l && dn_kq_engaged(tr.dngate_fmt) > 0l && dn_kq_engaged(tr.dnout_fmt) > 0l, "{spec.file} dn planes kq-native (not demoted)") } - t |> success(parity_gen_ok(tr, prompt, spec.parity.gen, [eyeball = true]), "{spec.file} token-for-token") + t |> success(parity_gen_ok(tr, prompt, spec.parity.gen, [eyeball = true]), "{spec.file} token-for-token{be}") if (spec.parity.dn_chunked) { t |> success(parity_gen_ok(tr, prompt, spec.parity.gen, [dn_chunked = true, eyeball = true]), - "{spec.file} chunked-prefill token-for-token") + "{spec.file} chunked-prefill token-for-token{be}") } if (spec.parity.moe_reference) { set_moe_grouped_prefill(false) t |> success(parity_gen_ok(tr, prompt, spec.parity.gen, [eyeball = true]), - "{spec.file} reference-prefill token-for-token") + "{spec.file} reference-prefill token-for-token{be}") set_moe_grouped_prefill(true) } if (spec.parity.moe_per_expert) { set_moe_fused_expert_gemms(false) t |> success(parity_gen_ok(tr, prompt, spec.parity.gen, [eyeball = true]), - "{spec.file} per-expert-prefill token-for-token") + "{spec.file} per-expert-prefill token-for-token{be}") set_moe_fused_expert_gemms(true) } delete prompt @@ -232,7 +263,9 @@ def test_parity_specs(t : T?) { var inscope spec <- spec_by_file(file) let path = path_join(models_dir(), spec.file) if (!model_available(t, path)) return - parity_spec_arms(t, spec, path) + with_parity_backend(t) { + parity_spec_arms(t, spec, path) + } } } delete files @@ -263,13 +296,16 @@ def test_parity_gptoss_20b(t : T?) { var inscope spec <- spec_by_file("gpt-oss-20b-mxfp4.gguf") let path = path_join(models_dir(), spec.file) if (!model_available(t, path)) return - var tr <- load_model_(path, QuantMode.q8) - if (tr.config.seq_len > 2048l) { - tr.config.seq_len = 2048l + with_parity_backend(t) { + var tr <- load_model_(path, QuantMode.q8) + if (tr.config.seq_len > 2048l) { + tr.config.seq_len = 2048l + } + let be = " [backend {active_kernel_backend()}]" + t |> success(parity_gen_ok(tr, spec.parity.prompt, spec.parity.gen, [eyeball = true]), "gpt-oss-20b token-for-token{be}") + let long_prompt <- encode(tr, counting_prompt(150), true, false) + t |> success(parity_gen_ok(tr, long_prompt, GPTOSS_LONG_GEN, [eyeball = true]), "gpt-oss-20b window-engaged token-for-token{be}") + delete tr } - t |> success(parity_gen_ok(tr, spec.parity.prompt, spec.parity.gen, [eyeball = true]), "gpt-oss-20b token-for-token") - let long_prompt <- encode(tr, counting_prompt(150), true, false) - t |> success(parity_gen_ok(tr, long_prompt, GPTOSS_LONG_GEN, [eyeball = true]), "gpt-oss-20b window-engaged token-for-token") - delete tr } } diff --git a/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das b/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das new file mode 100644 index 0000000000..cef0bb4e03 --- /dev/null +++ b/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das @@ -0,0 +1,741 @@ +//! The prefill and KV CPU kernels no suite gated before - the batched K-quant GEMMs and the per-position GEMV route a tier without those slots runs instead, the q8_0 cache dequant seam, the batch activation requants, and repack_kq_weight's round trip: every cell asserts on whatever tier is active, and the native batched arms ride a restored kq-backend pin. +options gen2 +options stack = 524288 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_math +require dasllama/dasllama_math_default +require dasllama/dasllama_math_gen +require dasllama/dasllama_convert +require dasllama/dasllama_kv_codec +require dasllama/dasllama_repack +require dasllama/dasllama_gemm_schema +require llvm/daslib/f16_cvt +require math + +var g_taint = 0.0 + +[sideeffects] +def private opaque(x : float) : float => x + g_taint + +def private poison_of(v : float) : float => max(0.5, abs(v) * 1.0e-3) + +def private cmp_exact(t : T?; got : array; want : array; n : int64; tag : string) { + var bad = 0 + var seen = 0 + let pi = n / 2l + for (i in range64(n)) { + if (got[i] != want[i]) { + bad++ + } + if (got[i] != (i == pi ? want[i] + poison_of(want[i]) : want[i])) { + seen++ + } + } + t |> success(bad == 0, "{tag}: {bad} of {int(n)} elements differ") + t |> success(seen > 0, "{tag}: the poisoned expected element must be seen, and was not") +} + +def private put_f16(var b : array; off : int64; v : float) { + let bits = f32_to_f16(v) + b[off] = uint8(bits & 0xFFu) + b[off + 1l] = uint8(bits >> 8u) +} + +def private build_q4k_disk() : array { + var b : array + b |> resize(144l) + put_f16(b, 0l, 0.375) + put_f16(b, 2l, 0.125) + for (i in range64(12l)) { + b[4l + i] = uint8((i * 37l + 13l) % 256l) + } + for (i in range64(128l)) { + b[16l + i] = uint8((i * 29l + 7l) % 256l) + } + return <- b +} + +def private build_q6k_disk() : array { + var b : array + b |> resize(210l) + for (i in range64(192l)) { + b[i] = uint8((i * 23l + 5l) % 256l) + } + for (i in range64(16l)) { + b[192l + i] = uint8((i * 41l + 11l) % 256l) + } + put_f16(b, 208l, 0.25) + return <- b +} + +def private build_q40_disk() : array { + var b : array + b |> resize(144l) + for (blk in range64(8l)) { + put_f16(b, blk * 18l, 0.0625 * float(blk + 1l)) + for (j in range64(16l)) { + b[blk * 18l + 2l + j] = uint8(((blk * 16l + j) * 19l + 3l) % 256l) + } + } + return <- b +} + +def private fmt_tag(fmt : int) : string => fmt == 40 ? "q40" : "k{fmt}" + +def private build_kq_region(fmt : int; var kq : array; var ks : array; rows, n : int64; seed : uint) { + let nsb = n / 256l + let qsb = kq_qsb(fmt) + let ssb = kq_ssb(fmt) + let blk <- fmt == 4 ? build_q4k_disk() : (fmt == 40 ? build_q40_disk() : build_q6k_disk()) + kq |> resize(rows * nsb * qsb) + ks |> resize(rows * nsb * ssb) + for (r in range64(rows)) { + for (s in range64(nsb)) { + if (fmt == 4) { + transcode_q4k_superblock(blk, 0l, kq, (r * nsb + s) * qsb, ks, (r * nsb + s) * ssb) + } elif (fmt == 40) { + transcode_q40_superblock(blk, 0l, kq, (r * nsb + s) * qsb, ks, (r * nsb + s) * ssb) + } else { + transcode_q6k_superblock(blk, 0l, kq, (r * nsb + s) * qsb, ks, (r * nsb + s) * ssb) + } + } + var st = seed + uint(r) * 0x85EBCA6Bu + for (i in range64(nsb * qsb)) { + st = st * 1664525u + 1013904223u + kq[r * nsb * qsb + i] = uint8(uint(kq[r * nsb * qsb + i]) ^ (st >> 16u)) + } + } +} + +def private build_acts(var src : array; var xq : array; var xs : array; var xbs : array; ntok, n : int64) { + src |> resize(ntok * n) + for (i in range64(ntok * n)) { + src[i] = opaque(0.9 * sin(float(i) * 0.27) - 0.5 * cos(float(i) * 0.061)) + } + xq |> resize(ntok * n) + xs |> resize(ntok * n / 256l) + xbs |> resize(ntok * n / 16l) + unsafe { + for (p in range64(ntok)) { + quantize_q8_k_into_ptr(addr(src[p * n]), n, addr(xq[0]), addr(xs[0]), addr(xbs[0]), + p * n, p * (n / 256l), p * (n / 16l)) + } + } +} + +def private kq_row_dot(fmt : int; kq : array; ks : array; row, nsb : int64; + xq : array; xs : array; xbs : array; + xoff, soff, boff, n : int64) : float { + let qsb = kq_qsb(fmt) + let ssb = kq_ssb(fmt) + unsafe { + let kqr = addr(kq[row * nsb * qsb]) + let ksr = addr(ks[row * nsb * ssb]) + let xqp = addr(xq[xoff]) + let xsp = addr(xs[soff]) + let xbp = addr(xbs[boff]) + if (fmt == 4) { + return dot_k4q8(kqr, ksr, xqp, xsp, xbp, n) + } elif (fmt == 40) { + return dot_q40q8(kqr, ksr, xqp, xsp, xbp, n) + } + return dot_k6q8(kqr, ksr, xqp, xsp, xbp, n) + } +} + +def private gemv_gen_rows(fmt : int; var yp : float?; kqp : uint8 const?; ksp : uint8 const?; + xqp : int8 const?; xsp : float const?; xbsp : int const?; n, rb, re : int64) { + if (fmt == 4) { + k4q8_gemv_gen(yp, kqp, ksp, xqp, xsp, xbsp, n, rb, re) + } elif (fmt == 40) { + q40q8_gemv_gen(yp, kqp, ksp, xqp, xsp, xbsp, n, rb, re) + } else { + k6q8_gemv_gen(yp, kqp, ksp, xqp, xsp, xbsp, n, rb, re) + } +} + +def private grp_row_dot(fmt : int; kqg : uint8 const?; ksg : uint8 const?; r, mr : int64; + xqp : int8 const?; xsp : float const?; xbsp : int const?; n : int64) : float { + if (fmt == 40) { + return q40_grp_row_dot(kqg, ksg, r, mr, xqp, xsp, xbsp, n) + } + return kq_grp_row_dot(int64(fmt), kqg, ksg, r, mr, xqp, xsp, xbsp, n) +} + +def private repack_grp(fmt : int; var kq : array; var ks : array; n, d, mr : int64) { + unsafe { + if (fmt == 4) { + repack_k4_grp(addr(kq[0]), addr(ks[0]), n, d, mr) + } elif (fmt == 40) { + repack_q40_grp(addr(kq[0]), addr(ks[0]), n, d, mr) + } else { + repack_k6_grp(addr(kq[0]), addr(ks[0]), n, d, mr) + } + } +} + +def private gen_backend_name() : string => get_architecture_name() == "arm64" ? "arm64-gen" : "x64-gen" + +def private quant_one(src : array; poff, n : int64; var q1 : array; var s1 : array; var b1 : array) { + unsafe { + quantize_q8_k_into_ptr(addr(src[poff]), n, addr(q1[0]), addr(s1[0]), addr(b1[0]), 0l, 0l, 0l) + } +} + +def private alloc_row_scratch(var q1 : array; var s1 : array; var b1 : array; n : int64) { + q1 |> resize(n) + s1 |> resize(n / 256l) + b1 |> resize(n / 16l) +} + +def private kq_batch_gate(t : T?; fmt : int; n, d, ntok : int64) { + let nsb = n / 256l + var kq : array + var ks : array + build_kq_region(fmt, kq, ks, d, n, 0x51ED270Bu) + var src : array + var xq : array + var xs : array + var xbs : array + build_acts(src, xq, xs, xbs, ntok, n) + var want : array + want |> resize(ntok * d) + for (p in range64(ntok)) { + for (r in range64(d)) { + want[p * d + r] = kq_row_dot(fmt, kq, ks, r, nsb, xq, xs, xbs, p * n, p * nsb, p * (n / 16l), n) + } + } + var q1 : array + var s1 : array + var b1 : array + alloc_row_scratch(q1, s1, b1, n) + var y : array + y |> resize(ntok * d) + for (p in range64(ntok)) { + quant_one(src, p * n, n, q1, s1, b1) + matmul_kq(fmt, y, kq, ks, 0l, q1, s1, b1, n, d, p * d) + } + cmp_exact(t, y, want, ntok * d, + "{fmt_tag(fmt)} serial per-position GEMV (n={int(n)} d={int(d)} ntok={int(ntok)}) vs per-(token,row) disk dots") + let prev = active_kernel_backend() + pin_kernel_backend(gen_backend_name()) + if (kernel_backend_has_kq() && kernel_backend_has_kq_batch()) { + repack_kq_weight(fmt, kq, ks, 0l, n, d) + var ybatch : array + ybatch |> resize(ntok * d) + matmul_kq_batch(fmt, ybatch, kq, ks, 0l, xq, xs, xbs, n, d, ntok) + var yser : array + yser |> resize(ntok * d) + for (p in range64(ntok)) { + quant_one(src, p * n, n, q1, s1, b1) + matmul_kq_active(fmt, yser, kq, ks, 0l, q1, s1, b1, n, d, p * d) + } + cmp_exact(t, ybatch, yser, ntok * d, + "{fmt_tag(fmt)} native kq batch is bit-for-bit ntok x matmul_kq_active (ntok={int(ntok)})") + delete ybatch + delete yser + } + clear_kernel_backend_pin() + select_kernel_backend(prev) + delete kq + delete ks + delete src + delete xq + delete xs + delete xbs + delete want + delete q1 + delete s1 + delete b1 + delete y +} + +[test] +def test_kq_batch(t : T?) { + for (fmt in [4, 6, 40]) { + for (ntok in [1l, 5l, 13l]) { + t |> run("{fmt_tag(fmt)} batched kq GEMM at ntok={int(ntok)}: the tier's route matches the per-token disk dots") @(t : T?) { + kq_batch_gate(t, fmt, 512l, 32l, ntok) + } + } + t |> run("{fmt_tag(fmt)} batched kq GEMM at the multi-superblock shape (n=1024 d=64 ntok=6)") @(t : T?) { + kq_batch_gate(t, fmt, 1024l, 64l, 6l) + } + } +} + +def private groupn_fallback(t : T?; fmt : int; kq : array; ks : array; src : array; + cnts : array; n, d : int64; want : array; nk : int64) { + var q1 : array + var s1 : array + var b1 : array + alloc_row_scratch(q1, s1, b1, n) + var y : array + y |> resize(nk * d) + var row0 = 0l + for (r in range64(long_length(cnts))) { + for (tk in range64(row0, row0 + cnts[r])) { + quant_one(src, tk * n, n, q1, s1, b1) + matmul_kq(fmt, y, kq, ks, r * d * n, q1, s1, b1, n, d, tk * d) + } + row0 += cnts[r] + } + cmp_exact(t, y, want, nk * d, + "{fmt_tag(fmt)} per-expert per-token GEMV route (what a tier with no fused kq slot runs) vs per-(region,token) disk dots") + delete q1 + delete s1 + delete b1 + delete y +} + +def private groupn_fused(t : T?; fmt : int; var kq : array; var ks : array; + offs : array; cnts : array; xq : array; xs : array; + xbs : array; n, d, nk : int64) { + let nsb = n / 256l + let qsb = kq_qsb(fmt) + let ssb = kq_ssb(fmt) + let nreg = long_length(cnts) + for (r in range64(nreg)) { + repack_kq_weight(fmt, kq, ks, r * d * n, n, d) + } + var yb : array + yb |> resize(nk * d) + matmul_kq_batch_groupn(fmt, yb, kq, ks, offs, nreg, xq, xs, xbs, n, d) + var wantg : array + wantg |> resize(nk * d) + var row0 = 0l + unsafe { + for (r in range64(nreg)) { + let kqr = addr(kq[r * d * nsb * qsb]) + let ksr = addr(ks[r * d * nsb * ssb]) + for (tk in range64(row0, row0 + cnts[r])) { + gemv_gen_rows(fmt, addr(wantg[tk * d]), kqr, ksr, addr(xq[tk * n]), + addr(xs[tk * nsb]), addr(xbs[tk * (n / 16l)]), n, 0l, d) + } + row0 += cnts[r] + } + } + cmp_exact(t, yb, wantg, nk * d, + "{fmt_tag(fmt)} fused kq batch groupn bit-matches per-(region, token) rows-core calls") + delete yb + delete wantg +} + +def private kq_batch_groupn_gate(t : T?; fmt : int; n, d : int64) { + var cnts <- [1l, 4l, 6l] + let nreg = long_length(cnts) + var nk = 0l + for (c in cnts) { + nk += c + } + let nsb = n / 256l + var kq : array + var ks : array + build_kq_region(fmt, kq, ks, nreg * d, n, 0x7F4A7C15u) + var src : array + var xq : array + var xs : array + var xbs : array + build_acts(src, xq, xs, xbs, nk, n) + var offs : array + offs |> reserve(nreg * 3l) + var want : array + want |> resize(nk * d) + var row0 = 0l + for (r in range64(nreg)) { + offs |> push(r * d * n) + offs |> push(row0) + offs |> push(cnts[r]) + for (tk in range64(row0, row0 + cnts[r])) { + for (row in range64(d)) { + want[tk * d + row] = kq_row_dot(fmt, kq, ks, r * d + row, nsb, xq, xs, xbs, + tk * n, tk * nsb, tk * (n / 16l), n) + } + } + row0 += cnts[r] + } + groupn_fallback(t, fmt, kq, ks, src, cnts, n, d, want, nk) + let prev = active_kernel_backend() + pin_kernel_backend(gen_backend_name()) + if (kernel_backend_has_kq() && kernel_backend_has_kq_batch_groupn()) { + groupn_fused(t, fmt, kq, ks, offs, cnts, xq, xs, xbs, n, d, nk) + } + clear_kernel_backend_pin() + select_kernel_backend(prev) + delete cnts + delete kq + delete ks + delete src + delete xq + delete xs + delete xbs + delete offs + delete want +} + +[test] +def test_kq_batch_groupn_portable(t : T?) { + for (fmt in [4, 6, 40]) { + t |> run("{fmt_tag(fmt)} fused MoE-prefill region walk (runs 1/4/6) on whatever backend is active") @(t : T?) { + kq_batch_groupn_gate(t, fmt, 512l, 32l) + } + } + t |> run("k4 fused region walk at the expert-slice shape (n=1024 d=128)") @(t : T?) { + kq_batch_groupn_gate(t, 4, 1024l, 128l) + } +} + +def private grp_rows_after_repack(fmt : int; kq : array; ks : array; nsb, d, mr : int64; + xq : array; xs : array; xbs : array; n : int64; + var got : array) { + let qsb = kq_qsb(fmt) + let ssb = kq_ssb(fmt) + got |> resize(d) + unsafe { + for (r in range64(d)) { + let g = r / mr + got[r] = grp_row_dot(fmt, addr(kq[g * mr * nsb * qsb]), + addr(ks[g * mr * nsb * ssb]), r % mr, mr, + addr(xq[0]), addr(xs[0]), addr(xbs[0]), n) + } + } +} + +def private byte_diff(a : array; b : array) : int { + var n = 0 + for (x, y in a, b) { + if (x != y) { + n++ + } + } + return n +} + +def private repack_seam(t : T?; fmt : int; kq : array; ks : array; + kqg : array; ksg : array; want : array; + nsb, n, d : int64; xq : array; xs : array; xbs : array) { + var kqs : array + kqs := kq + var kss : array + kss := ks + let prev = active_kernel_backend() + pin_kernel_backend(gen_backend_name()) + let has_kq = kernel_backend_has_kq() + repack_kq_weight(fmt, kqs, kss, 0l, n, d) + if (has_kq) { + let moved = byte_diff(kqs, kqg) + byte_diff(kss, ksg) + t |> success(moved == 0, "repack_kq_weight lands on the grp<{int(kq_layout_of(fmt))}> layout the readers expect ({moved} bytes differ)") + var got : array + grp_rows_after_repack(fmt, kqs, kss, nsb, d, kq_layout_of(fmt), xq, xs, xbs, n, got) + cmp_exact(t, got, want, d, "{fmt_tag(fmt)} row dots through repack_kq_weight's own planes") + delete got + } else { + let moved = byte_diff(kqs, kq) + byte_diff(kss, ks) + t |> success(moved == 0, "with no kq slot on the tier repack_kq_weight is the registered identity ({moved} bytes moved)") + var got : array + got |> resize(d) + for (r in range64(d)) { + got[r] = kq_row_dot(fmt, kqs, kss, r, nsb, xq, xs, xbs, 0l, 0l, 0l, n) + } + cmp_exact(t, got, want, d, "{fmt_tag(fmt)} disk row dots survive the identity repack") + } + clear_kernel_backend_pin() + select_kernel_backend(prev) + delete kqs + delete kss +} + +def private repack_gate(t : T?; fmt : int; n, d : int64) { + let nsb = n / 256l + var kq : array + var ks : array + build_kq_region(fmt, kq, ks, d, n, 0x9E3779B9u) + var src : array + var xq : array + var xs : array + var xbs : array + build_acts(src, xq, xs, xbs, 1l, n) + var want : array + want |> resize(d) + for (r in range64(d)) { + want[r] = kq_row_dot(fmt, kq, ks, r, nsb, xq, xs, xbs, 0l, 0l, 0l, n) + } + let mr = kq_layout_of(fmt) + var kqg : array + kqg := kq + var ksg : array + ksg := ks + repack_grp(fmt, kqg, ksg, n, d, mr) + var got : array + grp_rows_after_repack(fmt, kqg, ksg, nsb, d, mr, xq, xs, xbs, n, got) + cmp_exact(t, got, want, d, "{fmt_tag(fmt)} grp<{int(mr)}> row dots after the repack (n={int(n)} d={int(d)})") + repack_seam(t, fmt, kq, ks, kqg, ksg, want, nsb, n, d, xq, xs, xbs) + delete kq + delete ks + delete kqg + delete ksg + delete src + delete xq + delete xs + delete xbs + delete want + delete got +} + +[test] +def test_repack_kq_weight(t : T?) { + for (fmt in [4, 6, 40]) { + t |> run("{fmt_tag(fmt)} the grp repack and repack_kq_weight preserve every row's dot") @(t : T?) { + repack_gate(t, fmt, 512l, 32l) + repack_gate(t, fmt, 1024l, 64l) + } + } +} + +def private q8kv_block_ref(blob : array; base, n : int64; var want : array) { + want |> resize(n) + for (bi in range64(n / 32l)) { + let bb = base + bi * 34l + let d = f16_to_f32(uint(blob[bb]) | (uint(blob[bb + 1l]) << 8u)) + for (k in range64(32l)) { + want[bi * 32l + k] = float(int8(blob[bb + 2l + k])) * d + } + } +} + +def private kv_q8_gate(t : T?; n : int64) { + let nb = n / 32l + var src : array + src |> resize(n) + for (i in range64(n)) { + src[i] = opaque(2.5 * sin(float(i) * 0.37 + 0.11)) + } + var blob : array + blob |> resize(2l * nb * 34l) + unsafe { + quantize_q8kv_row(addr(blob[0]), addr(src[0]), n) + quantize_q8kv_row(addr(blob[nb * 34l]), addr(src[0]), n) + } + var want : array + q8kv_block_ref(blob, 0l, n, want) + var got : array + got |> resize(n) + unsafe { + cvt_q8kv_to_f32(addr(got[0]), addr(blob[0]), n) + } + cmp_exact(t, got, want, n, "cvt_q8kv_to_f32 vs the per-block scalar reference") + var got2 : array + got2 |> resize(n) + unsafe { + let bp = addr(blob[0]) + kv_row_to_f32(addr(got2[0]), bp, kv_head_off(bp, n), n) + } + cmp_exact(t, got2, want, n, "kv_row_to_f32(q8_0) at the second row's byte base") + var rt_bad = 0 + for (bi in range64(nb)) { + let d = f16_to_f32(uint(blob[bi * 34l]) | (uint(blob[bi * 34l + 1l]) << 8u)) + for (k in range64(32l)) { + if (abs(src[bi * 32l + k] - want[bi * 32l + k]) > d * 0.57 + 1.0e-7) { + rt_bad++ + } + } + } + t |> success(rt_bad == 0, "every dequantized element within half a step of the source ({rt_bad} outside)") + delete src + delete blob + delete want + delete got + delete got2 +} + +def private kv_wide_gate(t : T?; n : int64) { + var src : array + src |> resize(n) + for (i in range64(n)) { + src[i] = opaque(1.75 * sin(float(i) * 0.21 + 0.4)) + } + var wide : array + wide |> resize(2l * n) + for (i in range64(n)) { + wide[n + i] = src[i] + } + var got : array + got |> resize(n) + unsafe { + let wp = addr(wide[0]) + kv_row_to_f32(addr(got[0]), wp, kv_head_off(wp, n), n) + } + cmp_exact(t, got, src, n, "kv_row_to_f32(f32) is the plain row copy") + var half : array + half |> resize(2l * n) + unsafe { + cvt_f32_to_f16(addr(half[n]), addr(src[0]), n) + } + var want : array + want |> resize(n) + for (i in range64(n)) { + want[i] = f16_to_f32(uint(half[n + i])) + } + var got2 : array + got2 |> resize(n) + unsafe { + let hp = addr(half[0]) + kv_row_to_f32(addr(got2[0]), hp, kv_head_off(hp, n), n) + } + cmp_exact(t, got2, want, n, "kv_row_to_f32(f16) widens the row it was pointed at") + delete src + delete wide + delete got + delete half + delete want + delete got2 +} + +[test] +def test_kv_q8_row_to_f32(t : T?) { + t |> run("cvt_q8kv_to_f32 and kv_row_to_f32 invert quantize_q8kv_row's stored blocks") @(t : T?) { + kv_q8_gate(t, 256l) + kv_q8_gate(t, 64l) + } + t |> run("kv_row_to_f32's f32 and f16 overloads carry a row at a non-zero base") @(t : T?) { + kv_wide_gate(t, 128l) + } +} + +def private ref_q8k_row(src : array; base, n : int64; var q : array; var sc : array; var bs : array) { + let nsb = n / 256l + q |> resize(n) + sc |> resize(nsb) + bs |> resize(n / 16l) + for (s in range64(nsb)) { + var amax = 0.0 + for (i in range64(256l)) { + amax = max(amax, abs(src[base + s * 256l + i])) + } + let d = amax / 127.0 + let id = d != 0.0 ? 1.0 / d : 0.0 + sc[s] = d + for (i in range64(256l)) { + q[s * 256l + i] = int8(round(src[base + s * 256l + i] * id)) + } + for (h in range64(16l)) { + var acc = 0 + for (i in range64(16l)) { + acc += int(q[s * 256l + h * 16l + i]) + } + bs[s * 16l + h] = acc + } + } +} + +def private fill_acts_flat(var x : array; npos, n : int64) { + x |> resize(npos * n) + for (i in range64(npos * n)) { + x[i] = opaque(1.4 * sin(float(i) * 0.19 + 0.3) - 0.6 * cos(float(i) * 0.083)) + } +} + +def private requant_q8_gate(t : T?; par : bool; npos, n : int64) { + var x : array + fill_acts_flat(x, npos, n) + let nb = n / 32l + var xqb : array + var xsb : array + xqb |> resize(npos * n) + xsb |> resize(npos * nb) + requant_rows_q8(x, n, npos, xqb, xsb, par) + var bad = 0 + var seen = 0 + for (p in range64(npos)) { + var row : array + row |> resize(n) + for (i in range64(n)) { + row[i] = x[p * n + i] + } + var qt <- quantize_q8_0(row, n) + for (i in range64(n)) { + if (xqb[p * n + i] != qt.q[i]) { + bad++ + } + if (p == 0l && i == n / 2l && int(xqb[p * n + i]) != int(qt.q[i]) + 7) { + seen++ + } + } + for (b in range64(nb)) { + if (xsb[p * nb + b] != qt.scales[b]) { + bad++ + } + } + delete qt + delete row + } + t |> success(bad == 0, "requant_rows_q8 (par={par}) bit-matches quantize_q8_0 per row ({bad} differ)") + t |> success(seen > 0, "the +7 poison on one expected quant must be seen (par={par})") + delete x + delete xqb + delete xsb +} + +def private requant_q8k_gate(t : T?; par : bool; npos, n : int64) { + var x : array + fill_acts_flat(x, npos, n) + let nsb = n / 256l + var xqb : array + var xsb : array + var xbsb : array + xqb |> resize(npos * n) + xsb |> resize(npos * nsb) + xbsb |> resize(npos * n / 16l) + requant_rows_q8k_bs(x, n, npos, xqb, xsb, xbsb, par) + var bad = 0 + var seen = 0 + for (p in range64(npos)) { + var q : array + var sc : array + var bs : array + ref_q8k_row(x, p * n, n, q, sc, bs) + for (i in range64(n)) { + if (xqb[p * n + i] != q[i]) { + bad++ + } + } + for (s in range64(nsb)) { + if (xsb[p * nsb + s] != sc[s]) { + bad++ + } + } + for (h in range64(n / 16l)) { + if (xbsb[p * (n / 16l) + h] != bs[h]) { + bad++ + } + if (p == 0l && h == 0l && xbsb[p * (n / 16l) + h] != bs[h] + 3) { + seen++ + } + } + delete q + delete sc + delete bs + } + t |> success(bad == 0, "requant_rows_q8k_bs (par={par}) matches the in-test Q8_K reference ({bad} differ)") + t |> success(seen > 0, "the +3 poison on one expected per-16 sum must be seen (par={par})") + delete x + delete xqb + delete xsb + delete xbsb +} + +[test] +def test_requant_rows(t : T?) { + for (par in [false, true]) { + t |> run("requant_rows_q8 par={par}: every row is quantize_q8_0 of that row") @(t : T?) { + requant_q8_gate(t, par, 5l, 512l) + requant_q8_gate(t, par, 1l, 1024l) + } + t |> run("requant_rows_q8k_bs par={par}: per-256 scales and per-16 sums against an in-test reference") @(t : T?) { + requant_q8k_gate(t, par, 5l, 512l) + requant_q8k_gate(t, par, 1l, 1024l) + } + } +} diff --git a/modules/dasLLAMA/tests/test_q8q8_family.das b/modules/dasLLAMA/tests/test_q8q8_family.das new file mode 100644 index 0000000000..da3410130e --- /dev/null +++ b/modules/dasLLAMA/tests/test_q8q8_family.das @@ -0,0 +1,885 @@ +options gen2 +options stack = 524288 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_math +require dasllama/dasllama_math_default +require dasllama/dasllama_math_aarch64_neon +require dasllama/dasllama_math_gen +require dasllama/dasllama_repack +require dasllama/dasllama_convert +require llvm/daslib/f16_cvt +require daslib/jobque_boost +require math + +//! The q8q8 kernel family - stamped tile vs GEMV, the mm/batch/group3/groupN slots on every registered backend, the wscale_f16 twins, matmul_q8/dot_q8 and the mx4 batch slots - each against an in-test fp64 dequant reference carrying its own poison control. + +let WIDTHS <- [64l, 96l, 512l, 1024l, 1056l, 3072l] +let REF_ULP = 1.0e-6 +let REF_POISON = 0.25 +let MX4_E2M1 <- [0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0] + +def private mix32(seed, idx : uint) : uint { + var s = seed + idx * 2654435761u + s = s ^ (s >> 15u) + s = s * 2246822519u + s = s ^ (s >> 13u) + s = s * 3266489917u + return s ^ (s >> 16u) +} + +def private unit_at(seed, idx : uint) : float { + return float(int(mix32(seed, idx) % 20001u) - 10000) * 1.0e-4 +} + +def private fill_unit(var a : array; cnt : int64; seed : uint) { + a |> resize(cnt) + for (i in range64(cnt)) { + a[i] = unit_at(seed, uint(int(i))) + } +} + +def private slice_i8(src : array; off, cnt : int64) : array { + return <- [for (i in range64(off, off + cnt)); src[i]] +} + +def private slice_f(src : array; off, cnt : int64) : array { + return <- [for (i in range64(off, off + cnt)); src[i]] +} + +def private pow2d(k : int) : double { + var v = 1.0lf + for (_i in range(abs(k))) { + v = k >= 0 ? v * 2.0lf : v * 0.5lf + } + return v +} + +def private ref_bar(env : double; nb : int64) : float { + return REF_ULP * float(nb) * max(float(env), 1.0) +} + +def private ref_dot_f64(wq : array; ws : array; woff : int64; + xq : array; xs : array; xoff, n : int64) : tuple { + var res : tuple + for (bi in range64(n / 32l)) { + var iacc = 0 + for (k in range64(32l)) { + iacc += int(wq[woff + bi * 32l + k]) * int(xq[xoff + bi * 32l + k]) + } + let term = double(iacc) * double(ws[woff / 32l + bi]) * double(xs[xoff / 32l + bi]) + res.dot += term + res.env += abs(term) + } + return res +} + +def private ref_dot_q8f_f64(wq : array; ws : array; woff : int64; + x : array; n : int64) : tuple { + var res : tuple + for (bi in range64(n / 32l)) { + var bacc : double + for (k in range64(32l)) { + bacc += double(int(wq[woff + bi * 32l + k])) * double(x[bi * 32l + k]) + } + let term = bacc * double(ws[woff / 32l + bi]) + res.dot += term + res.env += abs(term) + } + return res +} + +def private mx4_val(nib : int) : double { + let m = MX4_E2M1[nib & 7] + return double((nib & 8) != 0 ? -m : m) +} + +def private ref_dot_mx4_f64(wn : array; we : array; roff, eoff : int64; + xq : array; xs : array; xoff, n : int64) : tuple { + var res : tuple + for (bi in range64(n / 32l)) { + var bacc : double + for (j in range64(16l)) { + let b = int(wn[roff + bi * 16l + j]) + bacc += mx4_val(b & 15) * double(int(xq[xoff + bi * 32l + j])) + bacc += mx4_val(b >> 4) * double(int(xq[xoff + bi * 32l + j + 16l])) + } + let term = bacc * pow2d(int(we[eoff + bi]) - 127) * double(xs[xoff / 32l + bi]) + res.dot += term + res.env += abs(term) + } + return res +} + +def private identical(t : T?; got : array; want : array; what : string) { + var bad = 0 + for (gv, wv in got, want) { + if (gv != wv) { + bad++ + } + } + t |> success(bad == 0, "{what}: {bad} of {long_length(got)} elements differ") +} + +def private held(t : T?; got : array; ref : array; bar : array; what : string) { + var over = 0 + var worst = 0.0 + for (gv, rv, bv in got, ref, bar) { + let df = abs(gv - rv) + if (df > bv) { + over++ + } + worst = max(worst, df) + } + t |> success(over == 0, "{what}: {over} of {long_length(got)} past the fp64 bar, maxdiff {worst}, bar[0] {bar[0]}") +} + +def private poisoned(t : T?; got : array; ref : array; bar : array; what : string) { + let i = long_length(ref) / 2l + t |> success(abs(got[i] - (ref[i] + REF_POISON)) > bar[i], + "{what}: poison {REF_POISON} added at element {i} must exceed bar {bar[i]}") +} + +def private quantize_rows(src : array; ntok, n : int64; var xq : array; var xs : array) { + let nb = n / 32l + xq |> resize(ntok * n) + xs |> resize(ntok * nb) + var row : array + row |> resize(n) + for (p in range64(ntok)) { + for (i in range64(n)) { + row[i] = src[p * n + i] + } + quantize_q8_0_into(row, n, xq, xs, p * n, p * nb) + } +} + +def private s16_planes(src : array; cnt : int64; var s16 : array; var s32 : array) { + s16 |> resize(cnt) + s32 |> resize(cnt) + for (i in range64(cnt)) { + let h = f32_to_f16(src[i]) + s16[i] = uint16(h) + s32[i] = f16_to_f32(h) + } +} + +def private narrow_scales(src : array; cnt : int64; var s16 : array) { + s16 |> resize(cnt) + for (i in range64(cnt)) { + s16[i] = uint16(f32_to_f16(src[i])) + } +} + +def private activation_bsums(xq : array; ntok, n : int64; var xbs : array) { + let nb = n / 32l + xbs |> resize(ntok * nb) + for (p in range64(ntok)) { + for (bi in range64(nb)) { + var sm = 0 + for (j in range64(32l)) { + sm += int(xq[p * n + bi * 32l + j]) + } + xbs[p * nb + bi] = -128 * sm + } + } +} + +def private ref_image(wq : array; ws : array; xq : array; xs : array; + ntok, d, n : int64; var ref : array; var bar : array) { + let nb = n / 32l + ref |> resize(ntok * d) + bar |> resize(ntok * d) + for (p in range64(ntok)) { + for (r in range64(d)) { + let rf = ref_dot_f64(wq, ws, r * n, xq, xs, p * n, n) + ref[p * d + r] = float(rf.dot) + bar[p * d + r] = ref_bar(rf.env, nb) + } + } +} + +def private groupn_ref(wq : array; ws : array; xq : array; xs : array; + nreg, dreg, n : int64; var ref : array; var bar : array) { + let nb = n / 32l + ref |> resize(nreg * dreg) + bar |> resize(nreg * dreg) + for (r in range64(nreg)) { + for (row in range64(dreg)) { + let rf = ref_dot_f64(wq, ws, (r * dreg + row) * n, xq, xs, r * n, n) + ref[r * dreg + row] = float(rf.dot) + bar[r * dreg + row] = ref_bar(rf.env, nb) + } + } +} + +def private gemv_image_q8(var y : array; wq : array; ws : array; + xq : array; xs : array; ntok, d, n : int64) { + let nb = n / 32l + y |> resize(ntok * d) + var yrow : array + yrow |> resize(d) + for (p in range64(ntok)) { + let rxq <- slice_i8(xq, p * n, n) + let rxs <- slice_f(xs, p * nb, nb) + matmul_q8q8(yrow, wq, ws, rxq, rxs, n, d) + for (r in range64(d)) { + y[p * d + r] = yrow[r] + } + } +} + +def private gemv_image_s16(var y : array; wq : array; ws : array; + xq : array; xs : array; ntok, d, n : int64) { + let nb = n / 32l + y |> resize(ntok * d) + var yrow : array + yrow |> resize(d) + for (p in range64(ntok)) { + let rxq <- slice_i8(xq, p * n, n) + let rxs <- slice_f(xs, p * nb, nb) + matmul_q8q8(yrow, wq, ws, 0l, rxq, rxs, n, d) + for (r in range64(d)) { + y[p * d + r] = yrow[r] + } + } +} + +def private concat3(y0 : array; y1 : array; y2 : array; var fused : array) { + fused |> resize(long_length(y0) + long_length(y1) + long_length(y2)) + var w = 0l + for (v in y0) { + fused[w] = v + w++ + } + for (v in y1) { + fused[w] = v + w++ + } + for (v in y2) { + fused[w] = v + w++ + } +} + +def private restore_backend(saved : string) { + clear_kernel_backend_pin() + if (saved != "") { + select_kernel_backend(saved) + } +} + +def private q8q8_stamped_gate(t : T?; n : int64) { + let d = 64l + let nb = n / 32l + let mr = int64(q8q8_layout_gen()) + let ts = int64(q8q8_tokstep_gen()) + let gs = ts == 4l ? 1l : ts / 16l + let ng = d / mr + let ntok = 2l * ts + 3l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x51D0Au) + fill_unit(xf, ntok * n, 0x51D0Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, wq0.scales, xq, xs, ntok, d, n, ref, bar) + var xbs : array + activation_bsums(xq, ntok, n, xbs) + var q : array + var s : array + q := wq0.q + s := wq0.scales + unsafe { + repack_q8q8_grp(addr(q[0]), addr(s[0]), n, d, mr, int64(q8q8_wbias_gen()), int64(q8q8_kgroup_gen())) + } + var ytile : array + var ygemv : array + ytile |> resize(ntok * d) + ygemv |> resize(ntok * d) + unsafe { + let wp = addr(q[0]) + let sp = addr(s[0]) + let xqp = addr(xq[0]) + let xsp = addr(xs[0]) + let xbsp = addr(xbs[0]) + var ytp = addr(ytile[0]) + var ygp = addr(ygemv[0]) + q8q8_amx_cfg_gen() + for (gu in range64(ng / gs)) { + let gg = gu * gs + var tk = 0l + while (tk + ts <= ntok) { + q8q8_tile_gen(ytp, wp + gg * mr * n, sp + gg * mr * nb, xqp, xsp, xbsp, n, d, gg, tk) + tk += ts + } + while (tk < ntok) { + q8q8_gemv_gen(ytp + tk * d, wp, sp, xqp + tk * n, xsp + tk * nb, n, gg * mr, (gg + gs) * mr) + tk++ + } + } + for (p in range64(ntok)) { + q8q8_gemv_gen(ygp + p * d, wp, sp, xqp + p * n, xsp + p * nb, n, 0l, ng * mr) + } + } + identical(t, ytile, ygemv, "q8q8 tile (mr={mr} tokstep={ts}) bit-matches the per-token GEMV, n={n}") + held(t, ygemv, ref, bar, "q8q8 stamped GEMV, n={n}") + held(t, ytile, ref, bar, "q8q8 stamped tile, n={n}") + poisoned(t, ygemv, ref, bar, "q8q8 stamped GEMV, n={n}") +} + +def private q8q8_wrapper_gate(t : T?; n : int64) { + let d = 64l + let ntok = 2l * int64(q8q8_tokstep_gen()) + 3l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x51D1Au) + fill_unit(xf, ntok * n, 0x51D1Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, wq0.scales, xq, xs, ntok, d, n, ref, bar) + let saved = active_kernel_backend() + for (name in kernel_backend_names()) { + pin_kernel_backend(name) + var q : array + var s : array + q := wq0.q + s := wq0.scales + if (kernel_backend_needs_repack(name)) { + repack_q8q8_weight(q, s, 0l, n, d) + } + var ymm : array + gemv_image_q8(ymm, q, s, xq, xs, ntok, d, n) + var ybat : array + ybat |> resize(ntok * d) + matmul_q8q8_batch(ybat, q, s, xq, xs, n, d, ntok) + if (name == "portable") { + identical(t, ybat, ymm, "{name} matmul_q8q8_batch bit-matches ntok x matmul_q8q8, n={n}") + } else { + held(t, ybat, ymm, bar, "{name} matmul_q8q8_batch within the fp64 bar of ntok x matmul_q8q8, n={n}") + } + held(t, ymm, ref, bar, "{name} matmul_q8q8, n={n}") + held(t, ybat, ref, bar, "{name} matmul_q8q8_batch, n={n}") + poisoned(t, ymm, ref, bar, "{name} matmul_q8q8, n={n}") + } + restore_backend(saved) +} + +[test] +def test_q8q8_tile_gemv_reference(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + for (n in WIDTHS) { + t |> run("q8q8 stamped tile vs GEMV vs fp64 dequant, n={n}") @(t : T?) { + q8q8_stamped_gate(t, n) + } + t |> run("q8q8 mm/batch vs fp64 dequant on every backend, n={n}") @(t : T?) { + q8q8_wrapper_gate(t, n) + } + } + } +} + +def private s16_kernel_gate(t : T?; n : int64) { + let d = 64l + let dreg = d / 2l + let nb = n / 32l + let ntok = 2l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x5F16Au) + fill_unit(xf, ntok * n, 0x5F16Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var ws16 : array + var ws32 : array + s16_planes(wq0.scales, d * nb, ws16, ws32) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, ws32, xq, xs, 1l, d, n, ref, bar) + var gref : array + var gbar : array + groupn_ref(wq0.q, ws32, xq, xs, 2l, dreg, n, gref, gbar) + var ydot : array + var ydot16 : array + var yrows : array + var yrows16 : array + var ygn : array + var ygn16 : array + ydot |> resize(d) + ydot16 |> resize(d) + yrows |> resize(d) + yrows16 |> resize(d) + ygn |> resize(d) + ygn16 |> resize(d) + var offs <- [0l, 0l, dreg * n, n] + unsafe { + let wp = addr(wq0.q[0]) + let s32p = addr(ws32[0]) + let s16p = addr(ws16[0]) + let xqp = addr(xq[0]) + let xsp = addr(xs[0]) + let offp = addr(offs[0]) + for (r in range64(d)) { + ydot[r] = dot_q8q8(wp + r * n, s32p + r * nb, xqp, xsp, n) + ydot16[r] = dot_q8q8_f16s(wp + r * n, s16p + r * nb, xqp, xsp, n) + } + q8q8_rows_kernel(addr(yrows[0]), wp, s32p, xqp, xsp, n, 0l, d) + q8q8_rows_kernel_s16(addr(yrows16[0]), wp, s16p, xqp, xsp, n, 0l, d) + q8q8_groupn_kernel(addr(ygn[0]), wp, s32p, offp, 2l, xqp, xsp, null, null, n, dreg) + q8q8_groupn_kernel_s16(addr(ygn16[0]), wp, s16p, offp, 2l, xqp, xsp, null, null, n, dreg) + } + identical(t, ydot16, ydot, "dot_q8q8_f16s bit-matches dot_q8q8 over the widened plane, n={n}") + identical(t, yrows16, yrows, "q8q8_rows_kernel_s16 bit-matches its f32 twin, n={n}") + identical(t, yrows16, ydot16, "q8q8_rows_kernel_s16 bit-matches the dot it wraps, n={n}") + identical(t, ygn16, ygn, "q8q8_groupn_kernel_s16 bit-matches its f32 twin, n={n}") + held(t, ydot16, ref, bar, "dot_q8q8_f16s, n={n}") + held(t, yrows16, ref, bar, "q8q8_rows_kernel_s16, n={n}") + held(t, ygn16, gref, gbar, "q8q8_groupn_kernel_s16, n={n}") + poisoned(t, ydot16, ref, bar, "dot_q8q8_f16s, n={n}") + poisoned(t, ygn16, gref, gbar, "q8q8_groupn_kernel_s16, n={n}") +} + +def private s16_stamped_gate(t : T?; n : int64) { + let d = 64l + let nb = n / 32l + let mr = int64(q8q8_layout_gen()) + let ts = int64(q8q8_tokstep_gen()) + let gs = ts == 4l ? 1l : ts / 16l + let ng = d / mr + let ntok = 2l * ts + 3l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x5F17Au) + fill_unit(xf, ntok * n, 0x5F17Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var ws16m : array + var ws32m : array + s16_planes(wq0.scales, d * nb, ws16m, ws32m) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, ws32m, xq, xs, ntok, d, n, ref, bar) + var xbs : array + activation_bsums(xq, ntok, n, xbs) + var q : array + var s : array + q := wq0.q + s := ws32m + unsafe { + repack_q8q8_grp(addr(q[0]), addr(s[0]), n, d, mr, int64(q8q8_wbias_gen()), int64(q8q8_kgroup_gen())) + } + var s16 : array + narrow_scales(s, d * nb, s16) + var ytile : array + var ygemv : array + ytile |> resize(ntok * d) + ygemv |> resize(ntok * d) + unsafe { + let wp = addr(q[0]) + let sp = addr(s16[0]) + let xqp = addr(xq[0]) + let xsp = addr(xs[0]) + let xbsp = addr(xbs[0]) + var ytp = addr(ytile[0]) + var ygp = addr(ygemv[0]) + q8q8_amx_cfg_gen() + for (gu in range64(ng / gs)) { + let gg = gu * gs + var tk = 0l + while (tk + ts <= ntok) { + q8q8_tile_s16_gen(ytp, wp + gg * mr * n, sp + gg * mr * nb, xqp, xsp, xbsp, n, d, gg, tk) + tk += ts + } + while (tk < ntok) { + q8q8_gemv_s16_gen(ytp + tk * d, wp, sp, xqp + tk * n, xsp + tk * nb, n, gg * mr, (gg + gs) * mr) + tk++ + } + } + for (p in range64(ntok)) { + q8q8_gemv_s16_gen(ygp + p * d, wp, sp, xqp + p * n, xsp + p * nb, n, 0l, ng * mr) + } + } + identical(t, ytile, ygemv, "q8q8 s16 tile (mr={mr} tokstep={ts}) bit-matches the per-token s16 GEMV, n={n}") + held(t, ygemv, ref, bar, "q8q8 stamped s16 GEMV, n={n}") + held(t, ytile, ref, bar, "q8q8 stamped s16 tile, n={n}") + poisoned(t, ytile, ref, bar, "q8q8 stamped s16 tile, n={n}") +} + +def private s16_wrapper_gate(t : T?; n : int64) { + let d = 64l + let dreg = d / 2l + let nb = n / 32l + let ntok = 2l * int64(q8q8_tokstep_gen()) + 3l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x5F18Au) + fill_unit(xf, ntok * n, 0x5F18Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var ws16m : array + var ws32m : array + s16_planes(wq0.scales, d * nb, ws16m, ws32m) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, ws32m, xq, xs, ntok, d, n, ref, bar) + var gref : array + var gbar : array + groupn_ref(wq0.q, ws32m, xq, xs, 2l, dreg, n, gref, gbar) + var offs <- [0l, 0l, dreg * n, n] + let saved = active_kernel_backend() + for (name in kernel_backend_names()) { + pin_kernel_backend(name) + if (!kernel_backend_has_wscale16()) { + continue + } + var q : array + var s : array + q := wq0.q + s := ws32m + if (kernel_backend_needs_repack(name)) { + repack_q8q8_weight(q, s, 0l, n, d) + } + var s16 : array + narrow_scales(s, d * nb, s16) + var ymm : array + var ymm16 : array + gemv_image_q8(ymm, q, s, xq, xs, ntok, d, n) + gemv_image_s16(ymm16, q, s16, xq, xs, ntok, d, n) + var ybat16 : array + ybat16 |> resize(ntok * d) + matmul_q8q8_batch(ybat16, q, s16, 0l, xq, xs, n, d, ntok) + var ygn : array + var ygn16 : array + ygn |> resize(d) + ygn16 |> resize(d) + matmul_q8q8_groupn(ygn, q, s, offs, 2l, xq, xs, n, dreg) + matmul_q8q8_groupn(ygn16, q, s16, offs, 2l, xq, xs, n, dreg) + identical(t, ymm16, ymm, "{name} matmul_q8q8 s16 bit-matches the f32 plane, n={n}") + identical(t, ybat16, ymm16, "{name} matmul_q8q8_batch s16 bit-matches ntok x matmul_q8q8 s16, n={n}") + identical(t, ygn16, ygn, "{name} matmul_q8q8_groupn s16 bit-matches the f32 plane, n={n}") + held(t, ymm16, ref, bar, "{name} matmul_q8q8 s16, n={n}") + held(t, ybat16, ref, bar, "{name} matmul_q8q8_batch s16, n={n}") + held(t, ygn16, gref, gbar, "{name} matmul_q8q8_groupn s16, n={n}") + poisoned(t, ybat16, ref, bar, "{name} matmul_q8q8_batch s16, n={n}") + poisoned(t, ygn16, gref, gbar, "{name} matmul_q8q8_groupn s16, n={n}") + } + restore_backend(saved) +} + +[test] +def test_q8q8_wscale_f16(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + for (n in WIDTHS) { + t |> run("wscale_f16 dot/rows/groupN kernels vs their f32 twins and fp64, n={n}") @(t : T?) { + s16_kernel_gate(t, n) + } + t |> run("wscale_f16 stamped tile vs GEMV vs fp64, n={n}") @(t : T?) { + s16_stamped_gate(t, n) + } + t |> run("wscale_f16 matmul_q8q8 overloads on every backend, n={n}") @(t : T?) { + s16_wrapper_gate(t, n) + } + } + } +} + +def private group3_gate(t : T?; n : int64) { + let d0 = 32l + let d1 = 40l + let d2 = 44l + let dt = d0 + d1 + d2 + let nb = n / 32l + let woff0 = 0l + let woff1 = d0 * n + let woff2 = woff1 + d1 * n + var wf : array + var xf : array + fill_unit(wf, dt * n, 0x63A0Au) + fill_unit(xf, n, 0x63A0Bu) + let wq0 <- quantize_q8_0(wf, dt * n) + var ws16m : array + var ws32m : array + s16_planes(wq0.scales, dt * nb, ws16m, ws32m) + var xq : array + var xs : array + quantize_rows(xf, 1l, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, ws32m, xq, xs, 1l, dt, n, ref, bar) + let saved = active_kernel_backend() + for (name in kernel_backend_names()) { + pin_kernel_backend(name) + var q : array + var s : array + q := wq0.q + s := ws32m + if (kernel_backend_needs_repack(name)) { + repack_q8q8_weight(q, s, woff0, n, d0) + repack_q8q8_weight(q, s, woff1, n, d1) + repack_q8q8_weight(q, s, woff2, n, d2) + } + var s16 : array + narrow_scales(s, dt * nb, s16) + var y0 : array + var y1 : array + var y2 : array + y0 |> resize(d0) + y1 |> resize(d1) + y2 |> resize(d2) + matmul_q8q8_group3(y0, y1, y2, q, s, woff0, woff1, woff2, xq, xs, n, d0, d1, d2) + var g0 : array + var g1 : array + var g2 : array + g0 |> resize(d0) + g1 |> resize(d1) + g2 |> resize(d2) + matmul_q8q8(g0, q, s, woff0, xq, xs, n, d0) + matmul_q8q8(g1, q, s, woff1, xq, xs, n, d1) + matmul_q8q8(g2, q, s, woff2, xq, xs, n, d2) + identical(t, y0, g0, "{name} group3 region 0 bit-matches its own GEMV, n={n}") + identical(t, y1, g1, "{name} group3 region 1 bit-matches its own GEMV, n={n}") + identical(t, y2, g2, "{name} group3 region 2 (row tail) bit-matches its own GEMV, n={n}") + var fused : array + concat3(y0, y1, y2, fused) + held(t, fused, ref, bar, "{name} matmul_q8q8_group3, n={n}") + poisoned(t, fused, ref, bar, "{name} matmul_q8q8_group3, n={n}") + if (!kernel_backend_has_wscale16()) { + continue + } + var h0 : array + var h1 : array + var h2 : array + h0 |> resize(d0) + h1 |> resize(d1) + h2 |> resize(d2) + matmul_q8q8_group3(h0, h1, h2, q, s16, woff0, woff1, woff2, xq, xs, n, d0, d1, d2) + identical(t, h0, y0, "{name} group3 s16 region 0 bit-matches the f32 plane, n={n}") + identical(t, h1, y1, "{name} group3 s16 region 1 bit-matches the f32 plane, n={n}") + identical(t, h2, y2, "{name} group3 s16 region 2 bit-matches the f32 plane, n={n}") + } + restore_backend(saved) +} + +[test] +def test_q8q8_group3(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + for (n in WIDTHS) { + t |> run("matmul_q8q8_group3 (f32 and s16) vs three independent GEMVs and fp64, n={n}") @(t : T?) { + group3_gate(t, n) + } + } + } +} + +def private q8_f32x_gate(t : T?; n : int64) { + let d = 64l + let nb = n / 32l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x8F00Au) + fill_unit(xf, n, 0x8F00Bu) + let wq0 <- quantize_q8_0(wf, d * n) + var ref : array + var bar : array + ref |> resize(d) + bar |> resize(d) + for (r in range64(d)) { + let rf = ref_dot_q8f_f64(wq0.q, wq0.scales, r * n, xf, n) + ref[r] = float(rf.dot) + bar[r] = ref_bar(rf.env, nb) + } + var ymm : array + var yoff : array + var ydot : array + ymm |> resize(d) + yoff |> resize(d) + ydot |> resize(d) + matmul_q8(ymm, wq0.q, wq0.scales, xf, n, d) + matmul_q8(yoff, wq0.q, wq0.scales, 0l, xf, n, d) + unsafe { + let wp = addr(wq0.q[0]) + let sp = addr(wq0.scales[0]) + let xp = addr(xf[0]) + for (r in range64(d)) { + ydot[r] = dot_q8(wp + r * n, sp + r * nb, xp, n) + } + } + identical(t, ymm, ydot, "matmul_q8 bit-matches the per-row dot_q8, n={n}") + identical(t, yoff, ydot, "matmul_q8 blob+offset bit-matches the per-row dot_q8, n={n}") + held(t, ydot, ref, bar, "dot_q8, n={n}") + held(t, ymm, ref, bar, "matmul_q8, n={n}") + poisoned(t, ydot, ref, bar, "dot_q8, n={n}") +} + +[test] +def test_q8_f32_activation(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + for (n in WIDTHS) { + t |> run("matmul_q8 / dot_q8 vs fp64 dequant, n={n}") @(t : T?) { + q8_f32x_gate(t, n) + } + } + } +} + +def private mx4_planes(var wn : array; var we : array; cnt, nblk : int64) { + wn |> resize(cnt / 2l) + we |> resize(nblk) + for (i in range64(cnt / 2l)) { + wn[i] = uint8(mix32(0x3A4Bu, uint(int(i))) >> 24u) + } + for (i in range64(nblk)) { + we[i] = uint8(119u + mix32(0x3A4Cu, uint(int(i))) % 9u) + } +} + +def private mx4_ref_region(wn : array; we : array; woff : int64; + xq : array; xs : array; t0, te, d, n : int64; + var ref : array; var bar : array) { + let nb = n / 32l + for (p in range64(t0, te)) { + for (row in range64(d)) { + let rf = ref_dot_mx4_f64(wn, we, (woff + row * n) / 2l, (woff + row * n) / 32l, xq, xs, p * n, n) + ref[p * d + row] = float(rf.dot) + bar[p * d + row] = ref_bar(rf.env, nb) + } + } +} + +def private mx4_leaf_gate(t : T?; wn0 : array; we0 : array; + xq : array; xs : array; d, n : int64) { + let nb = n / 32l + var yleaf : array + var ymm : array + yleaf |> resize(d) + ymm |> resize(d) + let rxq <- slice_i8(xq, 0l, n) + let rxs <- slice_f(xs, 0l, nb) + pin_kernel_backend("portable") + matmul_mx4q8(ymm, wn0, we0, 0l, rxq, rxs, n, d) + var lut : int8[16] + mxfp4_fill_lut(lut) + unsafe { + let lutp = addr(lut[0]) + let wnp = addr(wn0[0]) + let wep = addr(we0[0]) + let xqp = addr(xq[0]) + let xsp = addr(xs[0]) + for (r in range64(d)) { + yleaf[r] = dot_mx4q8_scalar(lutp, wnp + r * (n / 2l), wep + r * nb, xqp, xsp, n) + } + } + identical(t, ymm, yleaf, "portable matmul_mx4q8 bit-matches the dot_mx4q8_scalar leaf, n={n}") +} + +def private mx4_batch_gate(t : T?; n : int64) { + let d = 32l + let nb = n / 32l + let ntok = 9l + let nreg = 2l + let cnt0 = 4l + let woff1 = d * n + var wn0 : array + var we0 : array + mx4_planes(wn0, we0, nreg * d * n, nreg * d * nb) + var xf : array + fill_unit(xf, ntok * n, 0x3A4Du) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var bref : array + var bbar : array + bref |> resize(ntok * d) + bbar |> resize(ntok * d) + mx4_ref_region(wn0, we0, 0l, xq, xs, 0l, ntok, d, n, bref, bbar) + var gref : array + var gbar : array + gref |> resize(ntok * d) + gbar |> resize(ntok * d) + mx4_ref_region(wn0, we0, 0l, xq, xs, 0l, cnt0, d, n, gref, gbar) + mx4_ref_region(wn0, we0, woff1, xq, xs, cnt0, ntok, d, n, gref, gbar) + var goffs <- [0l, 0l, cnt0, woff1, cnt0, ntok - cnt0] + let saved = active_kernel_backend() + mx4_leaf_gate(t, wn0, we0, xq, xs, d, n) + var names : array + names |> push("portable") + for (nm in kernel_backend_names()) { + if (kernel_backend_needs_repack(nm)) { + names |> push(nm) + } + } + for (nm in names) { + pin_kernel_backend(nm) + var wn : array + var we : array + wn := wn0 + we := we0 + if (kernel_backend_needs_repack(nm)) { + for (r in range64(nreg)) { + repack_mx4_weight(wn, we, r * d * n, n, d) + } + } + var ymm : array + var ygmm : array + var yrow : array + ymm |> resize(ntok * d) + ygmm |> resize(ntok * d) + yrow |> resize(d) + for (p in range64(ntok)) { + let rxq <- slice_i8(xq, p * n, n) + let rxs <- slice_f(xs, p * nb, nb) + matmul_mx4q8(yrow, wn, we, 0l, rxq, rxs, n, d) + for (row in range64(d)) { + ymm[p * d + row] = yrow[row] + } + matmul_mx4q8(yrow, wn, we, p < cnt0 ? 0l : woff1, rxq, rxs, n, d) + for (row in range64(d)) { + ygmm[p * d + row] = yrow[row] + } + } + var ybat : array + ybat |> resize(ntok * d) + matmul_mx4q8_batch(ybat, wn, we, 0l, xq, xs, n, d, ntok) + var ygb : array + ygb |> resize(ntok * d) + matmul_mx4q8_batch_groupn(ygb, wn, we, goffs, nreg, xq, xs, n, d) + identical(t, ybat, ymm, "{nm} matmul_mx4q8_batch bit-matches ntok x matmul_mx4q8, n={n}") + identical(t, ygb, ygmm, "{nm} matmul_mx4q8_batch_groupn bit-matches the per-region GEMVs, n={n}") + held(t, ybat, bref, bbar, "{nm} matmul_mx4q8_batch, n={n}") + held(t, ygb, gref, gbar, "{nm} matmul_mx4q8_batch_groupn, n={n}") + poisoned(t, ybat, bref, bbar, "{nm} matmul_mx4q8_batch, n={n}") + poisoned(t, ygb, gref, gbar, "{nm} matmul_mx4q8_batch_groupn, n={n}") + } + restore_backend(saved) +} + +[test] +def test_mx4q8_batch_family(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + for (n in WIDTHS) { + t |> run("mx4 batch / batch groupN vs per-token GEMVs, the scalar leaf and fp64, n={n}") @(t : T?) { + mx4_batch_gate(t, n) + } + } + } +} diff --git a/modules/dasLLAMA/tests/test_rope_apply.das b/modules/dasLLAMA/tests/test_rope_apply.das new file mode 100644 index 0000000000..b608213b61 --- /dev/null +++ b/modules/dasLLAMA/tests/test_rope_apply.das @@ -0,0 +1,481 @@ +//! The CPU rope APPLICATION leaves (dasllama_math's rope_scaled_* and rope_apply) against an in-test fp64 angle reference - a subject on every platform, not just the Apple Metal rope oracle. +options gen2 +options stack = 524288 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_math +require dasllama/dasllama_rope +require math + +var g_taint = 0.0 + +[sideeffects] +def private opaque(x : float) : float => x + g_taint + +let ROPE_BAR = 2.0e-3 +let ROPE_POISON = 0.05 +let TWIN_BAR = 1.0e-5 + +def private ff_none() : array { + var e : array + return <- e +} + +def private ff_ptr(ff : array) : float const? { + unsafe { + return !empty(ff) ? addr(ff[0]) : null + } +} + +def private fill_vec(var v : array; n : int64) { + v |> resize(n) + for (i in range64(n)) { + v[i] = opaque(1.3 * sin(float(i) * 0.31 + 0.2) - 0.7 * cos(float(i) * 0.077)) + } +} + +def private copy_vec(src : array; var dst : array; n : int64) { + dst |> resize(n) + for (i in range64(n)) { + dst[i] = src[i] + } +} + +def private ref_freq(hd, hs : int64; theta : float; ff : array; has_ff : bool; fj : int64) : double { + var f = 1.0lf / pow(double(theta), double(hd) / double(hs)) + if (has_ff) { + f /= double(ff[fj]) + } + return f +} + +def private ref_norm(v : array; var outv : array; pos, hs, n : int64; + theta, fscale, mscale : float; ff : array; has_ff : bool) { + copy_vec(v, outv, n) + let posf = double(pos) * double(fscale) + for (p in range64(n / 2l)) { + let i = p * 2l + let hd = i % hs + let a = posf * ref_freq(hd, hs, theta, ff, has_ff, hd / 2l) + let fcr = cos(a) * double(mscale) + let fci = sin(a) * double(mscale) + let v0 = double(v[i]) + let v1 = double(v[i + 1l]) + outv[i] = float(v0 * fcr - v1 * fci) + outv[i + 1l] = float(v0 * fci + v1 * fcr) + } +} + +def private ref_neox(v : array; var outv : array; pos, hs, rot, n : int64; + theta, fscale, mscale : float; ff : array; has_ff : bool) { + copy_vec(v, outv, n) + let half = rot / 2l + let posf = double(pos) * double(fscale) + for (h in range64(n / hs)) { + let base = h * hs + for (j in range64(half)) { + let a = posf * ref_freq(2l * j, rot, theta, ff, has_ff, j) + let fcr = cos(a) * double(mscale) + let fci = sin(a) * double(mscale) + let v0 = double(v[base + j]) + let v1 = double(v[base + j + half]) + outv[base + j] = float(v0 * fcr - v1 * fci) + outv[base + j + half] = float(v0 * fci + v1 * fcr) + } + } +} + +def private maxdiff(a : array; b : array; n : int64) : float { + var worst = 0.0 + for (i in range64(n)) { + worst = max(worst, abs(a[i] - b[i])) + } + return worst +} + +def private poisoned_maxdiff(a : array; b : array; n : int64) : float { + var worst = 0.0 + for (i in range64(n)) { + let w = i == n / 2l ? b[i] + ROPE_POISON : b[i] + worst = max(worst, abs(a[i] - w)) + } + return worst +} + +def private cmp_bar(t : T?; got : array; want : array; n : int64; bar : float; tag : string) { + let worst = maxdiff(got, want, n) + t |> success(worst <= bar, "{tag}: maxdiff {worst} exceeds the bar {bar}") + let pworst = poisoned_maxdiff(got, want, n) + t |> success(pworst > bar, "{tag}: a +{ROPE_POISON} poison on one expected element scored {pworst}, which must exceed {bar}") +} + +def private cmp_exact(t : T?; got : array; want : array; n : int64; tag : string) { + var bad = 0 + var poisoned = 0 + for (i in range64(n)) { + if (got[i] != want[i]) { + bad++ + } + if (got[i] != (i == n / 2l ? want[i] + ROPE_POISON : want[i])) { + poisoned++ + } + } + t |> success(bad == 0, "{tag}: {bad} of {int(n)} elements differ") + t |> success(poisoned > 0, "{tag}: a +{ROPE_POISON} poison on one expected element must be seen") +} + +def private norm_gate(t : T?; pos, hs, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var want : array + var ff <- ff_none() + ref_norm(v, want, pos, hs, n, theta, fscale, mscale, ff, false) + var got : array + copy_vec(v, got, n) + rope_scaled(got, pos, hs, n, theta, fscale, ff, mscale) + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled hs={int(hs)} nh={int(nh)} pos={int(pos)}") + delete v + delete got + delete want + delete ff +} + +def private neox_gate(t : T?; pos, hs, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var want : array + var ff <- ff_none() + ref_neox(v, want, pos, hs, hs, n, theta, fscale, mscale, ff, false) + var got : array + copy_vec(v, got, n) + rope_scaled_neox(got, pos, hs, n, theta, fscale, ff, mscale) + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled_neox hs={int(hs)} nh={int(nh)} pos={int(pos)}") + delete v + delete got + delete want + delete ff +} + +[test] +def test_rope_apply_norm(t : T?) { + for (pos in [0l, 1l, 7l, 37l, 512l]) { + t |> run("rope_scaled at pos {pos}: head 64x2, 96x1 and the 40-wide tail head vs the fp64 angles") @(t : T?) { + norm_gate(t, pos, 64l, 2l, 10000.0, 1.0, 1.0) + norm_gate(t, pos, 96l, 1l, 1000000.0, 1.0, 1.0) + norm_gate(t, pos, 40l, 3l, 10000.0, 0.5, 1.19) + } + } +} + +[test] +def test_rope_apply_neox(t : T?) { + for (pos in [0l, 1l, 7l, 37l, 512l]) { + t |> run("rope_scaled_neox at pos {pos}: head 64x2, 96x1 and the 40-wide tail head vs the fp64 angles") @(t : T?) { + neox_gate(t, pos, 64l, 2l, 10000.0, 1.0, 1.0) + neox_gate(t, pos, 96l, 1l, 1000000.0, 1.0, 1.0) + neox_gate(t, pos, 40l, 3l, 10000.0, 0.5, 1.19) + } + } +} + +def private tab_gate(t : T?; pos, hs, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var ff <- ff_none() + var want : array + ref_norm(v, want, pos, hs, n, theta, fscale, mscale, ff, false) + var direct : array + copy_vec(v, direct, n) + rope_scaled(direct, pos, hs, n, theta, fscale, ff, mscale) + var ct : array + var st : array + build_rope_tabs(ct, st, theta, fscale, mscale, ff, true, pos, 1l, hs) + var got : array + copy_vec(v, got, n) + unsafe { + rope_scaled_tab(addr(got[0]), hs, n, addr(ct[0]), addr(st[0])) + } + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled_tab hs={int(hs)} nh={int(nh)} pos={int(pos)}") + cmp_bar(t, got, direct, n, TWIN_BAR, "rope_scaled_tab twin of rope_scaled hs={int(hs)} pos={int(pos)}") + delete v + delete got + delete direct + delete want + delete ct + delete st + delete ff +} + +def private neox_tab_gate(t : T?; pos, hs, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var ff <- ff_none() + var want : array + ref_neox(v, want, pos, hs, hs, n, theta, fscale, mscale, ff, false) + var direct : array + copy_vec(v, direct, n) + rope_scaled_neox(direct, pos, hs, n, theta, fscale, ff, mscale) + var ct : array + var st : array + build_rope_tabs(ct, st, theta, fscale, mscale, ff, true, pos, 1l, hs) + var got : array + copy_vec(v, got, n) + unsafe { + rope_scaled_neox_tab(addr(got[0]), hs, n, addr(ct[0]), addr(st[0])) + } + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled_neox_tab hs={int(hs)} nh={int(nh)} pos={int(pos)}") + cmp_bar(t, got, direct, n, TWIN_BAR, "rope_scaled_neox_tab twin of rope_scaled_neox hs={int(hs)} pos={int(pos)}") + delete v + delete got + delete direct + delete want + delete ct + delete st + delete ff +} + +[test] +def test_rope_apply_tab(t : T?) { + for (pos in [0l, 1l, 37l, 512l]) { + t |> run("rope_scaled_tab at pos {pos}: the fp64 angles, and the un-tabled twin") @(t : T?) { + tab_gate(t, pos, 64l, 2l, 10000.0, 1.0, 1.0) + tab_gate(t, pos, 40l, 3l, 10000.0, 0.5, 1.19) + } + } +} + +[test] +def test_rope_apply_neox_tab(t : T?) { + for (pos in [0l, 1l, 37l, 512l]) { + t |> run("rope_scaled_neox_tab at pos {pos}: the fp64 angles, and the un-tabled twin") @(t : T?) { + neox_tab_gate(t, pos, 64l, 2l, 10000.0, 1.0, 1.0) + neox_tab_gate(t, pos, 40l, 3l, 10000.0, 0.5, 1.19) + } + } +} + +def private part_gate(t : T?; pos, hs, rot, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var ff <- ff_none() + var want : array + ref_neox(v, want, pos, hs, rot, n, theta, fscale, mscale, ff, false) + var got : array + copy_vec(v, got, n) + unsafe { + rope_scaled_neox_part(addr(got[0]), pos, hs, rot, n, theta, fscale, false, null, mscale) + } + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled_neox_part hs={int(hs)} rot={int(rot)} pos={int(pos)}") + var compact : array + compact |> resize(nh * rot) + for (h in range64(nh)) { + for (j in range64(rot)) { + compact[h * rot + j] = v[h * hs + j] + } + } + rope_scaled_neox(compact, pos, rot, nh * rot, theta, fscale, ff, mscale) + var full : array + copy_vec(v, full, n) + for (h in range64(nh)) { + for (j in range64(rot)) { + full[h * hs + j] = compact[h * rot + j] + } + } + cmp_exact(t, got, full, n, "rope_scaled_neox_part == the full leaf over the rotated prefix hs={int(hs)} rot={int(rot)}") + var tail_moved = 0 + for (h in range64(nh)) { + for (j in range64(rot, hs)) { + if (got[h * hs + j] != v[h * hs + j]) { + tail_moved++ + } + } + } + t |> success(tail_moved == 0, "the {int(hs - rot)} un-rotated dims of each head must pass through ({tail_moved} moved)") + delete v + delete got + delete full + delete want + delete compact + delete ff +} + +def private tab_part_gate(t : T?; pos, hs, rot, nh : int64; theta, fscale, mscale : float) { + let n = hs * nh + var v : array + fill_vec(v, n) + var ff <- ff_none() + var want : array + ref_neox(v, want, pos, hs, rot, n, theta, fscale, mscale, ff, false) + var direct : array + copy_vec(v, direct, n) + unsafe { + rope_scaled_neox_part(addr(direct[0]), pos, hs, rot, n, theta, fscale, false, null, mscale) + } + var ct : array + var st : array + build_rope_tabs(ct, st, theta, fscale, mscale, ff, true, pos, 1l, rot) + var got : array + copy_vec(v, got, n) + unsafe { + rope_scaled_neox_tab_part(addr(got[0]), hs, rot, n, addr(ct[0]), addr(st[0])) + } + cmp_bar(t, got, want, n, ROPE_BAR, "rope_scaled_neox_tab_part hs={int(hs)} rot={int(rot)} pos={int(pos)}") + cmp_bar(t, got, direct, n, TWIN_BAR, "rope_scaled_neox_tab_part twin of the direct part form hs={int(hs)} rot={int(rot)}") + delete v + delete got + delete direct + delete want + delete ct + delete st + delete ff +} + +[test] +def test_rope_apply_part(t : T?) { + for (pos in [0l, 1l, 37l, 512l]) { + t |> run("rope_scaled_neox_part at pos {pos}: 64 of 256 (qwen35), 32 of 96, 20 of 40") @(t : T?) { + part_gate(t, pos, 256l, 64l, 2l, 1000000.0, 1.0, 1.0) + part_gate(t, pos, 96l, 32l, 2l, 10000.0, 0.5, 1.19) + part_gate(t, pos, 40l, 20l, 3l, 10000.0, 1.0, 1.0) + } + t |> run("rope_scaled_neox_tab_part at pos {pos}: the fp64 angles, and the direct twin") @(t : T?) { + tab_part_gate(t, pos, 256l, 64l, 2l, 1000000.0, 1.0, 1.0) + tab_part_gate(t, pos, 40l, 20l, 3l, 10000.0, 1.0, 1.0) + } + } +} + +[test] +def test_rope_apply_dispatch(t : T?) { + t |> run("rope_apply lands on the leaf its `neox` flag names, pointer and array forms alike") @(t : T?) { + let hs = 64l + let n = 128l + let pos = 37l + var v : array + fill_vec(v, n) + var ff <- ff_none() + for (neox in [false, true]) { + var leaf : array + copy_vec(v, leaf, n) + if (neox) { + rope_scaled_neox(leaf, pos, hs, n, 10000.0, 1.0, ff, 1.19) + } else { + rope_scaled(leaf, pos, hs, n, 10000.0, 1.0, ff, 1.19) + } + var pform : array + copy_vec(v, pform, n) + unsafe { + rope_apply(addr(pform[0]), pos, hs, n, 10000.0, 1.0, false, null, neox, 1.19) + } + cmp_exact(t, pform, leaf, n, "rope_apply(ptr, neox={neox}) == the leaf") + var aform : array + copy_vec(v, aform, n) + rope_apply(aform, pos, hs, n, 10000.0, 1.0, ff, neox, 1.19, true) + cmp_exact(t, aform, leaf, n, "rope_apply(array, neox={neox}) == the leaf") + delete leaf + delete pform + delete aform + } + delete v + delete ff + } + t |> run("use_ff = false drops loaded freq-factors for this call (the gemma4 p-RoPE arm)") @(t : T?) { + let hs = 64l + let n = 128l + let pos = 37l + var v : array + fill_vec(v, n) + var ff : array + ff |> resize(hs / 2l) + for (j in range64(hs / 2l)) { + ff[j] = 1.0 + 0.25 * float(j) + } + var none <- ff_none() + for (neox in [false, true]) { + var off : array + copy_vec(v, off, n) + rope_apply(off, pos, hs, n, 10000.0, 1.0, ff, neox, 1.0, false) + var bare : array + copy_vec(v, bare, n) + rope_apply(bare, pos, hs, n, 10000.0, 1.0, none, neox, 1.0, true) + cmp_exact(t, off, bare, n, "use_ff=false (neox={neox}) == the no-factors apply") + var on : array + copy_vec(v, on, n) + rope_apply(on, pos, hs, n, 10000.0, 1.0, ff, neox, 1.0, true) + var moved = 0 + for (i in range64(n)) { + if (on[i] != off[i]) { + moved++ + } + } + t |> success(moved > 0, "the factors must actually move the rotation when use_ff is true (neox={neox})") + delete off + delete bare + delete on + } + delete v + delete ff + delete none + } +} + +def private ff_gate(t : T?; pos, hs, nh : int64; neox : bool) { + let n = hs * nh + var v : array + fill_vec(v, n) + var ff : array + ff |> resize(hs / 2l) + for (j in range64(hs / 2l)) { + ff[j] = 1.0 + 0.5 * float(j % 7l) + } + var want : array + if (neox) { + ref_neox(v, want, pos, hs, hs, n, 500000.0, 1.0, 1.0, ff, true) + } else { + ref_norm(v, want, pos, hs, n, 500000.0, 1.0, 1.0, ff, true) + } + var got : array + copy_vec(v, got, n) + unsafe { + rope_apply(addr(got[0]), pos, hs, n, 500000.0, 1.0, true, ff_ptr(ff), neox, 1.0) + } + cmp_bar(t, got, want, n, ROPE_BAR, "freq-factors (neox={neox}) hs={int(hs)} pos={int(pos)}") + var ones : array + ones |> resize(hs / 2l) + for (c in ones) { + c = 1.0 + } + var none <- ff_none() + var with_ones : array + copy_vec(v, with_ones, n) + rope_apply(with_ones, pos, hs, n, 500000.0, 1.0, ones, neox, 1.0, true) + var without : array + copy_vec(v, without, n) + rope_apply(without, pos, hs, n, 500000.0, 1.0, none, neox, 1.0, true) + cmp_exact(t, with_ones, without, n, "all-ones factors (neox={neox}) must divide away exactly") + delete v + delete got + delete want + delete ff + delete ones + delete none + delete with_ones + delete without +} + +[test] +def test_rope_apply_freq_factors(t : T?) { + for (pos in [1l, 37l, 512l]) { + t |> run("rope_freqs divide the per-pair frequency at pos {pos}, both conventions") @(t : T?) { + ff_gate(t, pos, 64l, 2l, false) + ff_gate(t, pos, 64l, 2l, true) + ff_gate(t, pos, 40l, 3l, true) + } + } +} diff --git a/modules/dasLLAMA/tests/test_tower_asr_kernels.das b/modules/dasLLAMA/tests/test_tower_asr_kernels.das new file mode 100644 index 0000000000..4d0f9c427e --- /dev/null +++ b/modules/dasLLAMA/tests/test_tower_asr_kernels.das @@ -0,0 +1,1276 @@ +options gen2 +options stack = 524288 +options _dasllama_internal = true + +require dastest/testing_boost public +require dasllama/dasllama_tower +require dasllama/dasllama_audio +require dasllama/dasllama_tts_blocks +require dasllama/dasllama_math +require dasllama/dasllama_plane +require daslib/jobque_boost +require math + +//! Model-free CPU units for the public tower, audio and TTS kernels the ASR and TTS paths run: every bar is an in-test fp64 reference over a seeded-LCG fixture, and every bar carries a must-EXCEED poison. +struct KCase { + name : string + max_abs : double + max_ratio : double + bad : int +} + +var private g_lcg = 0x2545F4914F6CDD1Dul + +[sideeffects] +def private lcg_seed(s : uint64) { + g_lcg = s * 2862933555777941757ul + 3037000493ul +} + +[sideeffects] +def private lcg_unit() : float { + g_lcg = g_lcg * 6364136223846793005ul + 1442695040888963407ul + return float(uint((g_lcg >> 40ul) & 0xFFFFFFul)) / 8388608.0 - 1.0 +} + +def private fill(var a : array; n : int64; scale : float) { + a |> resize(n) + for (v in a) { + v = lcg_unit() * scale + } +} + +def private score(name : string; got : array; want : array; env : array; + rel : double; floor : double) : KCase { + var res = KCase(name = name) + if (long_length(got) != long_length(want) || long_length(got) != long_length(env) || empty(got)) { + res.bad = 1000000 + return res + } + for (g, w, e in got, want, env) { + let d = abs(double(g) - w) + let bar = max(rel * e, floor) + res.max_abs = max(res.max_abs, d) + res.max_ratio = max(res.max_ratio, d / bar) + if (d > bar) { + res.bad++ + } + } + return res +} + +def private score_poison(name : string; got : array; want : array; env : array; + rel : double; floor : double; delta : double) : KCase { + var p := want + if (empty(p)) { + return KCase(name = name) + } + p[long_length(p) / 2l] += delta + return score(name, got, p, env, rel, floor) +} + +def private score_exact(name : string; got : array; want : array) : KCase { + var res = KCase(name = name) + if (long_length(got) != long_length(want) || empty(got)) { + res.bad = 1000000 + return res + } + for (g, w in got, want) { + res.max_abs = max(res.max_abs, abs(double(g) - double(w))) + if (g != w) { + res.bad++ + } + } + return res +} + +def private score_exact_poison(name : string; got : array; want : array; delta : float) : KCase { + var p := want + if (empty(p)) { + return KCase(name = name) + } + p[long_length(p) / 2l] += delta + return score_exact(name, got, p) +} + +def private report(t : T?; r : KCase) { + to_log(LOG_INFO, "tower kernel {r.name}: max |kernel - reference| {r.max_abs}, worst ratio to the bar {r.max_ratio}, past the bar {r.bad}\n") + t |> equal(r.bad, 0, "{r.name}: elements past the bar") +} + +def private report_poison(t : T?; r : KCase) { + to_log(LOG_INFO, "tower kernel POISON {r.name}: max diff {r.max_abs}, worst ratio to the bar {r.max_ratio}, past the bar {r.bad}\n") + t |> success(r.bad > 0, "{r.name}: the poisoned expectation must exceed the bar, {r.bad} elements did") +} + +def private plane_f(a : array) : PlaneF { + var p = PlaneF() + unsafe { + p.p = addr(a[0]) + } + p.n = long_length(a) + return p +} + +def private plane_u16(a : array) : PlaneU16 { + var p = PlaneU16() + unsafe { + p.p = addr(a[0]) + } + p.n = long_length(a) + return p +} + +def private to_bf16(x : float) : uint16 { + return uint16(unsafe(reinterpret(x)) >> 16u) +} + +def private from_bf16(h : uint16) : float { + return unsafe(reinterpret(uint(h) << 16u)) +} + +let MM_N = 128l +let MM_D = 96l +let MM_T = 40l +let MM_WOFF = 64l +let MM_NPAD = 192l +let MM_REL = 2.0e-5lf +let MM_FLOOR = 1.0e-6lf + +def private gemm_ref(w : array; woff : int64; x : array; n : int64; d : int64; npos : int64; + var want : array; var env : array) { + want |> resize(npos * d) + env |> resize(npos * d) + for (p in range64(npos)) { + for (j in range64(d)) { + var acc = 0.0lf + var mag = 0.0lf + for (i in range64(n)) { + let prod = double(w[woff + j * n + i]) * double(x[p * n + i]) + acc += prod + mag += abs(prod) + } + want[p * d + j] = acc + env[p * d + j] = mag + } + } +} + +def private mm_fixture(var blob : array; var x : array) { + lcg_seed(0xA5A5ul) + fill(blob, MM_WOFF + MM_N * MM_D, 1.0) + fill(x, MM_T * MM_N, 1.0) +} + +def private mm_bf16_fixture(var wq : array; var wf : array; var x : array) { + lcg_seed(0xB16Bul) + fill(wf, MM_WOFF + MM_N * MM_D, 1.0) + wq |> resize(long_length(wf)) + for (f, q in wf, wq) { + q = to_bf16(f) + f = from_bf16(q) + } + fill(x, MM_T * MM_N, 1.0) + for (v in x) { + v = from_bf16(to_bf16(v)) + } +} + +def private mm_blob_run(var y : array; blob : array; x : array) { + y |> resize(MM_T * MM_D) + mm_blob_b(y, blob, MM_WOFF, x, MM_N, MM_D, MM_T) +} + +def private mm_blob_case(poison : bool) : KCase { + var blob : array + var x : array + mm_fixture(blob, x) + var want : array + var env : array + gemm_ref(blob, MM_WOFF, x, MM_N, MM_D, MM_T, want, env) + var y : array + mm_blob_run(y, blob, x) + let nm = "mm_blob_b blob+woff" + if (poison) { + return score_poison(nm, y, want, env, MM_REL, MM_FLOOR, 0.05lf) + } + return score(nm, y, want, env, MM_REL, MM_FLOOR) +} + +def private mm_plane_case(poison : bool) : KCase { + var blob : array + var x : array + mm_fixture(blob, x) + var y_arr : array + var y_pl : array + mm_blob_run(y_arr, blob, x) + y_pl |> resize(MM_T * MM_D) + mm_blob_b(y_pl, plane_f(blob), MM_WOFF, x, MM_N, MM_D, MM_T) + let nm = "mm_blob_b borrowed plane == owned blob" + if (poison) { + return score_exact_poison(nm, y_pl, y_arr, 1.0e-5) + } + return score_exact(nm, y_pl, y_arr) +} + +def private mm_bf16_case(poison : bool) : KCase { + var wq : array + var wf : array + var x : array + mm_bf16_fixture(wq, wf, x) + var want : array + var env : array + gemm_ref(wf, MM_WOFF, x, MM_N, MM_D, MM_T, want, env) + var y : array + y |> resize(MM_T * MM_D) + mm_bf16_b(y, plane_u16(wq), MM_WOFF, x, MM_N, MM_D, MM_T) + let nm = "mm_bf16_b on bf16-exact operands" + if (poison) { + return score_poison(nm, y, want, env, MM_REL, MM_FLOOR, 0.05lf) + } + return score(nm, y, want, env, MM_REL, MM_FLOOR) +} + +def private mm_route_case(bf16 : bool; poison : bool) : KCase { + var wq : array + var wf : array + var x : array + mm_bf16_fixture(wq, wf, x) + var want : array + var got : array + want |> resize(MM_T * MM_D) + got |> resize(MM_T * MM_D) + if (bf16) { + mm_bf16_b(want, plane_u16(wq), MM_WOFF, x, MM_N, MM_D, MM_T) + } else { + mm_blob_b(want, plane_f(wf), MM_WOFF, x, MM_N, MM_D, MM_T) + } + mm_plane_b(got, bf16, plane_u16(wq), plane_f(wf), MM_WOFF, x, MM_N, MM_D, MM_T) + let nm = "mm_plane_b bf16={bf16} routes to its wrapper" + if (poison) { + return score_exact_poison(nm, got, want, 1.0e-5) + } + return score_exact(nm, got, want) +} + +def private mm_pad_case(poison : bool) : KCase { + lcg_seed(0x9AD0ul) + var w : array + var x : array + fill(w, MM_N * MM_D, 1.0) + fill(x, MM_T * MM_N, 1.0) + var wp : array + var xp : array + wp |> resize(MM_NPAD * MM_D) + xp |> resize(MM_T * MM_NPAD) + for (j in range64(MM_D)) { + for (i in range64(MM_N)) { + wp[j * MM_NPAD + i] = w[j * MM_N + i] + } + } + for (p in range64(MM_T)) { + for (i in range64(MM_N)) { + xp[p * MM_NPAD + i] = x[p * MM_N + i] + } + } + var y_tight : array + var y_pad : array + y_tight |> resize(MM_T * MM_D) + y_pad |> resize(MM_T * MM_D) + mm_blob_b(y_tight, w, 0l, x, MM_N, MM_D, MM_T) + mm_blob_b(y_pad, wp, 0l, xp, MM_NPAD, MM_D, MM_T) + let nm = "mm_blob_b zero-padded width {MM_NPAD} == unpadded {MM_N}" + if (poison) { + return score_exact_poison(nm, y_pad, y_tight, 1.0e-5) + } + return score_exact(nm, y_pad, y_tight) +} + +let LN_DIM = 197l +let LN_POS = 33l +let LN_OFF = 8l +let LN_EPS = 1.0e-5 +let LN_REL = 1.0e-5lf +let LN_FLOOR = 1.0e-6lf + +def private ln_fixture(var x : array; var wb : array) { + lcg_seed(0x1EAFul) + fill(x, LN_POS * LN_DIM, 3.0) + fill(wb, LN_OFF + 2l * LN_DIM, 1.0) +} + +def private ln_ref(x : array; wb : array; var want : array; var env : array) { + want |> resize(LN_POS * LN_DIM) + env |> resize(LN_POS * LN_DIM) + for (p in range64(LN_POS)) { + let off = p * LN_DIM + var mean = 0.0lf + for (i in range64(LN_DIM)) { + mean += double(x[off + i]) + } + mean /= double(LN_DIM) + var vr = 0.0lf + for (i in range64(LN_DIM)) { + let dv = double(x[off + i]) - mean + vr += dv * dv + } + vr /= double(LN_DIM) + let scale = 1.0lf / sqrt(vr + double(LN_EPS)) + for (i in range64(LN_DIM)) { + let nrm = (double(x[off + i]) - mean) * scale * double(wb[LN_OFF + i]) + let bias = double(wb[LN_OFF + LN_DIM + i]) + want[off + i] = nrm + bias + env[off + i] = abs(nrm) + abs(bias) + } + } +} + +def private ln_scalar_run(var out : array; x : array; wb : array) { + out |> resize(LN_POS * LN_DIM) + unsafe { + var op = addr(out[0]) + let ip = addr(x[0]) + let wp = addr(wb[LN_OFF]) + let bp = addr(wb[LN_OFF + LN_DIM]) + for (p in range64(LN_POS)) { + layernorm(op + p * LN_DIM, ip + p * LN_DIM, wp, bp, LN_DIM, LN_EPS) + } + } +} + +def private ln_scalar_case(poison : bool) : KCase { + var x : array + var wb : array + ln_fixture(x, wb) + var want : array + var env : array + ln_ref(x, wb, want, env) + var out : array + ln_scalar_run(out, x, wb) + let nm = "layernorm row form" + if (poison) { + return score_poison(nm, out, want, env, LN_REL, LN_FLOOR, 0.01lf) + } + return score(nm, out, want, env, LN_REL, LN_FLOOR) +} + +def private ln_batch_case(poison : bool) : KCase { + var x : array + var wb : array + ln_fixture(x, wb) + var want : array + var env : array + ln_ref(x, wb, want, env) + var out : array + out |> resize(LN_POS * LN_DIM) + layernorm_batch(out, x, wb, LN_OFF, LN_OFF + LN_DIM, LN_DIM, LN_POS, LN_EPS) + let nm = "layernorm_batch blob+offsets" + if (poison) { + return score_poison(nm, out, want, env, LN_REL, LN_FLOOR, 0.01lf) + } + return score(nm, out, want, env, LN_REL, LN_FLOOR) +} + +def private ln_plane_case(poison : bool) : KCase { + var x : array + var wb : array + ln_fixture(x, wb) + var y_arr : array + var y_pl : array + y_arr |> resize(LN_POS * LN_DIM) + y_pl |> resize(LN_POS * LN_DIM) + layernorm_batch(y_arr, x, wb, LN_OFF, LN_OFF + LN_DIM, LN_DIM, LN_POS, LN_EPS) + layernorm_batch(y_pl, x, plane_f(wb), LN_OFF, LN_OFF + LN_DIM, LN_DIM, LN_POS, LN_EPS) + let nm = "layernorm_batch borrowed plane == owned blob" + if (poison) { + return score_exact_poison(nm, y_pl, y_arr, 1.0e-5) + } + return score_exact(nm, y_pl, y_arr) +} + +let AB_D = 37l +let AB_POS = 41l +let AB_OFF = 5l +let AI_N = 4093l + +def private ab_fixture(var y : array; var blob : array) { + lcg_seed(0xB1A5ul) + fill(y, AB_POS * AB_D, 2.0) + fill(blob, AB_OFF + AB_D, 1.0) +} + +def private ab_ref(y : array; blob : array) : array { + var want : array + want |> resize(AB_POS * AB_D) + for (p in range64(AB_POS)) { + for (i in range64(AB_D)) { + want[p * AB_D + i] = y[p * AB_D + i] + blob[AB_OFF + i] + } + } + return <- want +} + +def private add_bias_case(as_plane : bool; poison : bool) : KCase { + var y : array + var blob : array + ab_fixture(y, blob) + var want <- ab_ref(y, blob) + if (as_plane) { + add_bias_rows(y, plane_f(blob), AB_OFF, AB_D, AB_POS) + } else { + add_bias_rows(y, blob, AB_OFF, AB_D, AB_POS) + } + let nm = "add_bias_rows plane={as_plane} d={AB_D} off the float4 block" + if (poison) { + return score_exact_poison(nm, y, want, 1.0e-5) + } + return score_exact(nm, y, want) +} + +def private add_inplace_case(poison : bool) : KCase { + lcg_seed(0xADD1ul) + var x : array + var r : array + fill(x, AI_N, 2.0) + fill(r, AI_N, 1.0) + var want : array + want |> resize(AI_N) + for (a, b, w in x, r, want) { + w = a + b + } + add_inplace_rows(x, r, AI_N) + let nm = "add_inplace_rows n={AI_N} off the float4 block" + if (poison) { + return score_exact_poison(nm, x, want, 1.0e-5) + } + return score_exact(nm, x, want) +} + +let GE_D = 64l +let GE_POS = 25l +let GE_REL = 1.0e-6lf +let GE_FLOOR = 1.0e-6lf + +def private erf_ref(z : double) : double { + var term = z + var sum = z + for (k in range(1, 60)) { + term *= -z * z / double(k) + sum += term / double(2 * k + 1) + } + return sum * 2.0lf / sqrt(DBL_PI) +} + +def private gelu_fixture(var x : array) { + lcg_seed(0x9E10ul) + fill(x, GE_D * GE_POS, 4.0) +} + +def private gelu_ref(x : array; var want : array; var env : array) { + want |> resize(long_length(x)) + env |> resize(long_length(x)) + for (v, w, e in x, want, env) { + let d = double(v) + w = 0.5lf * d * (1.0lf + erf_ref(d * 0.7071067811865476lf)) + e = abs(d) + } +} + +def private gelu_batch_case(poison : bool) : KCase { + var x : array + gelu_fixture(x) + var want : array + var env : array + gelu_ref(x, want, env) + gelu_erf_batch(x, GE_D, GE_POS) + let nm = "gelu_erf_batch vs the fp64 erf series" + if (poison) { + return score_poison(nm, x, want, env, GE_REL, GE_FLOOR, 0.001lf) + } + return score(nm, x, want, env, GE_REL, GE_FLOOR) +} + +def private gelu_shape_case(poison : bool) : KCase { + var x : array + gelu_fixture(x) + var flat : array + flat := x + gelu_erf_batch(x, GE_D, GE_POS) + gelu_erf(flat, GE_D * GE_POS) + let nm = "gelu_erf_batch == the whole-image gelu_erf" + if (poison) { + return score_exact_poison(nm, x, flat, 1.0e-5) + } + return score_exact(nm, x, flat) +} + +let RP_POS = 17l +let RP_HEADS = 5l +let RP_HS = 16l +let RP_REL = 2.0e-6lf +let RP_FLOOR = 1.0e-7lf + +def private rope_tabs(var ct : array; var st : array; live_row : int64) { + let half = RP_HS / 2l + ct |> resize(RP_POS * half) + st |> resize(RP_POS * half) + for (r in range64(RP_POS)) { + for (j in range64(half)) { + let live = live_row < 0l || r == live_row + let ang = live ? 0.37 * float(r + 1l) / pow(2.0, float(j)) : 0.0 + ct[r * half + j] = cos(ang) + st[r * half + j] = sin(ang) + } + } +} + +def private rope_ref(v : array; ct : array; st : array; + var want : array; var env : array) { + let d = RP_HEADS * RP_HS + let half = RP_HS / 2l + want |> resize(RP_POS * d) + env |> resize(RP_POS * d) + for (r in range64(RP_POS)) { + for (h in range64(RP_HEADS)) { + let base = r * d + h * RP_HS + for (j in range64(half)) { + let c = double(ct[r * half + j]) + let s = double(st[r * half + j]) + let v0 = double(v[base + j]) + let v1 = double(v[base + j + half]) + want[base + j] = v0 * c - v1 * s + want[base + j + half] = v0 * s + v1 * c + env[base + j] = abs(v0 * c) + abs(v1 * s) + env[base + j + half] = abs(v0 * s) + abs(v1 * c) + } + } + } +} + +def private rope_case(poison : bool) : KCase { + lcg_seed(0x40BEul) + var v : array + var ct : array + var st : array + fill(v, RP_POS * RP_HEADS * RP_HS, 2.0) + rope_tabs(ct, st, -1l) + var want : array + var env : array + rope_ref(v, ct, st, want, env) + rope_neox_tab_rows(v, RP_POS, RP_HEADS, RP_HS, ct, st) + let nm = "rope_neox_tab_rows one table row per token, shared by every head" + if (poison) { + return score_poison(nm, v, want, env, RP_REL, RP_FLOOR, 0.001lf) + } + return score(nm, v, want, env, RP_REL, RP_FLOOR) +} + +def private rope_row_index_case(live_row : int64; poison : bool) : KCase { + lcg_seed(0x41BEul) + var v : array + var ct : array + var st : array + fill(v, RP_POS * RP_HEADS * RP_HS, 2.0) + let before := v + rope_tabs(ct, st, live_row) + rope_neox_tab_rows(v, RP_POS, RP_HEADS, RP_HS, ct, st) + let d = RP_HEADS * RP_HS + var got : array + var want : array + got |> resize((RP_POS - 1l) * d) + want |> resize((RP_POS - 1l) * d) + var o = 0l + for (r in range64(RP_POS)) { + if (r == live_row) { + continue + } + for (i in range64(d)) { + got[o * d + i] = v[r * d + i] + want[o * d + i] = before[r * d + i] + } + o++ + } + let nm = "rope_neox_tab_rows: only token {live_row}'s table row rotates" + if (poison) { + return score_exact_poison(nm, got, want, 1.0e-5) + } + var moved = false + for (i in range64(d)) { + if (v[live_row * d + i] != before[live_row * d + i]) { + moved = true + } + } + var res = score_exact(nm, got, want) + if (!moved) { + res.bad += 1000000 + } + return res +} + +let AW_D = 32l +let AW_HEADS = 4l +let AW_SCALE = 0.35 +let AW_REL = 1.0e-5lf +let AW_FLOOR = 1.0e-6lf + +def private aw_win() : array { + return <- [0l, 7l, 23l, 26l, 47l, 48l] +} + +def private aw_fixture(win : array; var qq : array; var kk : array; var vv : array) { + lcg_seed(0x77A0ul) + let rows = win[long_length(win) - 1l] + fill(qq, rows * AW_D, 1.0) + fill(kk, rows * AW_D, 1.0) + fill(vv, rows * AW_D, 1.0) + for (wi in range64(long_length(win) - 1l)) { + let amp = 1.0 + 3.0 * float(wi) + for (idx in range64(win[wi] * AW_D, win[wi + 1l] * AW_D)) { + kk[idx] = kk[idx] * amp + float(wi) + vv[idx] = vv[idx] * amp - float(wi) + } + } +} + +def private aw_ref_head(qq : array; kk : array; vv : array; var sc : array; + var want : array; var env : array; + r0 : int64; wr : int64; hoff : int64; hs : int64) { + for (p in range64(wr)) { + var mx = -1.0e300lf + for (q in range64(wr)) { + var s = 0.0lf + for (j in range64(hs)) { + s += double(qq[(r0 + p) * AW_D + hoff + j]) * double(kk[(r0 + q) * AW_D + hoff + j]) + } + sc[q] = s * double(AW_SCALE) + mx = max(mx, sc[q]) + } + var sum = 0.0lf + for (q in range64(wr)) { + sc[q] = exp(sc[q] - mx) + sum += sc[q] + } + for (j in range64(hs)) { + var acc = 0.0lf + var mag = 0.0lf + for (q in range64(wr)) { + let wgt = sc[q] / sum + let vq = double(vv[(r0 + q) * AW_D + hoff + j]) + acc += wgt * vq + mag += wgt * abs(vq) + } + want[(r0 + p) * AW_D + hoff + j] = acc + env[(r0 + p) * AW_D + hoff + j] = mag + } + } +} + +def private aw_ref(qq : array; kk : array; vv : array; win : array; + var want : array; var env : array) { + let rows = win[long_length(win) - 1l] + let hs = AW_D / AW_HEADS + want |> resize(rows * AW_D) + env |> resize(rows * AW_D) + var sc : array + for (wi in range64(long_length(win) - 1l)) { + let r0 = win[wi] + let wr = win[wi + 1l] - r0 + sc |> resize(wr) + for (h in range64(AW_HEADS)) { + aw_ref_head(qq, kk, vv, sc, want, env, r0, wr, h * hs, hs) + } + } +} + +def private aw_case(poison : bool) : KCase { + var win <- aw_win() + var qq : array + var kk : array + var vv : array + aw_fixture(win, qq, kk, vv) + var want : array + var env : array + aw_ref(qq, kk, vv, win, want, env) + var out : array + var hp : array + var att : array + out |> resize(long_length(qq)) + attention_bidir_windows(out, qq, kk, vv, hp, att, win, AW_D, AW_HEADS, AW_SCALE) + let nm = "attention_bidir_windows vs an oracle attending strictly inside each ragged window" + if (poison) { + return score_poison(nm, out, want, env, AW_REL, AW_FLOOR, 0.05lf) + } + return score(nm, out, want, env, AW_REL, AW_FLOOR) +} + +let IG_REL = 5.0e-6lf +let IG_FLOOR = 1.0e-7lf + +def private interp_ref(src : array; src_off : int64; sw : int64; sh : int64; dw : int64; dh : int64; + d : int64; var want : array; var env : array) { + want |> resize(dw * dh * d) + env |> resize(dw * dh * d) + let sf0 = float(dw) / float(sw) + let sf1 = float(dh) / float(sh) + let sup0 = max(1.0, 1.0 / sf0) + let sup1 = max(1.0, 1.0 / sf1) + let iv0 = 1.0 / sup0 + let iv1 = 1.0 / sup1 + for (i1 in range64(dh)) { + let yc = (float(i1) + 0.5) / sf1 + let ylo = max(int64(yc - sup1 + 0.5), 0l) + let yhi = min(int64(yc + sup1 + 0.5), sh) + for (i0 in range64(dw)) { + let xc = (float(i0) + 0.5) / sf0 + let xlo = max(int64(xc - sup0 + 0.5), 0l) + let xhi = min(int64(xc + sup0 + 0.5), sw) + let obase = (i1 * dw + i0) * d + for (c in range64(d)) { + want[obase + c] = 0.0lf + env[obase + c] = 0.0lf + } + var tw = 0.0lf + for (sy in range64(ylo, yhi)) { + let wy = max(1.0 - abs((float(sy) - yc + 0.5) * iv1), 0.0) + for (sx in range64(xlo, xhi)) { + let wgt = max(1.0 - abs((float(sx) - xc + 0.5) * iv0), 0.0) * wy + if (wgt <= 0.0) { + continue + } + let sbase = src_off + (sy * sw + sx) * d + for (c in range64(d)) { + let term = double(src[sbase + c]) * double(wgt) + want[obase + c] += term + env[obase + c] += abs(term) + } + tw += double(wgt) + } + } + if (tw > 0.0lf) { + for (c in range64(d)) { + want[obase + c] /= tw + env[obase + c] /= tw + } + } + } + } +} + +def private interp_case(sw : int64; sh : int64; dw : int64; dh : int64; d : int64; src_off : int64; + poison : bool) : KCase { + lcg_seed(0x1E40ul + uint64(dw * 131l + sh)) + var src : array + fill(src, src_off + sw * sh * d, 2.0) + var want : array + var env : array + interp_ref(src, src_off, sw, sh, dw, dh, d, want, env) + var out : array + interpolate_grid_bilinear_aa(out, plane_f(src), src_off, sw, sh, dw, dh, d) + let nm = "interpolate_grid_bilinear_aa {sw}x{sh} -> {dw}x{dh} d={d} at plane offset {src_off}" + if (poison) { + return score_poison(nm, out, want, env, IG_REL, IG_FLOOR, 0.01lf) + } + return score(nm, out, want, env, IG_REL, IG_FLOOR) +} + +let FFT_REL = 2.0e-6lf +let FFT_FLOOR = 1.0e-6lf +let DFT_N = 400l + +def private dft_ref(reim : array; n : int64; var want : array; var env : array) { + want |> resize(2l * n) + env |> resize(2l * n) + var envsum = 0.0lf + for (j in range64(n)) { + envsum += abs(double(reim[2l * j])) + abs(double(reim[2l * j + 1l])) + } + for (k in range64(n)) { + var acc_re = 0.0lf + var acc_im = 0.0lf + for (j in range64(n)) { + let th = 2.0lf * DBL_PI * double((j * k) % n) / double(n) + let cs = cos(th) + let sn = sin(th) + acc_re += double(reim[2l * j]) * cs + double(reim[2l * j + 1l]) * sn + acc_im += double(reim[2l * j + 1l]) * cs - double(reim[2l * j]) * sn + } + want[2l * k] = acc_re + want[2l * k + 1l] = acc_im + env[2l * k] = envsum + env[2l * k + 1l] = envsum + } +} + +def private fft_case(n : int64; poison : bool) : KCase { + lcg_seed(0xFF70ul + uint64(n)) + var reim : array + fill(reim, 2l * n, 1.0) + var want : array + var env : array + dft_ref(reim, n, want, env) + var p <- build_fft_plan(n) + unsafe { + fft_pow2_run(n, addr(p.rev[0]), addr(p.tw[0]), addr(reim[0])) + } + let nm = "fft_pow2_run n={n} vs the fp64 O(n^2) DFT of the same forward convention" + if (poison) { + return score_poison(nm, reim, want, env, FFT_REL, FFT_FLOOR, 0.05lf) + } + return score(nm, reim, want, env, FFT_REL, FFT_FLOOR) +} + +def private fft_plan_tw_case(n : int64; poison : bool) : KCase { + let p <- build_fft_plan(n) + var want : array + var env : array + want |> resize(n) + env |> resize(n) + for (k in range64(n / 2l)) { + let th = 2.0lf * DBL_PI * double(k) / double(n) + want[2l * k] = cos(th) + want[2l * k + 1l] = -sin(th) + env[2l * k] = 1.0lf + env[2l * k + 1l] = 1.0lf + } + let nm = "build_fft_plan n={n} twiddles vs fp64 (cos, -sin)" + if (poison) { + return score_poison(nm, p.tw, want, env, FFT_REL, FFT_FLOOR, 0.001lf) + } + return score(nm, p.tw, want, env, FFT_REL, FFT_FLOOR) +} + +def private fft_plan_rev_bad(n : int64) : int { + let p <- build_fft_plan(n) + var seen : array + seen |> resize(n) + var bad = 0 + for (i in range64(n)) { + let r = int64(p.rev[i]) + if (r < 0l || r >= n || seen[r] || int64(p.rev[r]) != i) { + bad++ + } else { + seen[r] = true + } + } + if (p.rev[0] != 0 || int64(p.rev[1]) != n / 2l) { + bad++ + } + return bad +} + +def private fft_plan_refuses(n : int64) : bool { + var tripped = false + try { + let p <- build_fft_plan(n) + tripped = long_length(p.rev) < 0l + } recover { + tripped = true + } + return tripped +} + +def private dft_twiddle_case(poison : bool) : KCase { + let n_bins = DFT_N / 2l + 1l + let b <- build_dft_twiddles(DFT_N) + var want : array + var env : array + want |> resize(DFT_N * n_bins * 2l) + env |> resize(DFT_N * n_bins * 2l) + for (j in range64(DFT_N)) { + for (k in range64(n_bins)) { + let th = 2.0lf * DBL_PI * double((j * k) % DFT_N) / double(DFT_N) + let at = j * n_bins * 2l + k * 2l + want[at] = cos(th) + want[at + 1l] = -sin(th) + env[at] = 1.0lf + env[at + 1l] = 1.0lf + } + } + let nm = "build_dft_twiddles n={DFT_N} vs fp64 (cos, -sin) at (j*k) mod n" + if (poison) { + return score_poison(nm, b, want, env, FFT_REL, FFT_FLOOR, 0.001lf) + } + return score(nm, b, want, env, FFT_REL, FFT_FLOOR) +} + +let RS_C = 5l +let RS_T = 41l +let RS_REL = 2.0e-6lf +let RS_FLOOR = 1.0e-7lf + +def private rs_fixture(var x : array; scale : float) { + lcg_seed(0x2E51ul + uint64(int64(scale * 1024.0))) + fill(x, RS_C * RS_T, 2.0) +} + +def private rl_onnx_ref(x : array; scale : float; var want : array; var env : array) : int64 { + let t_out = int64(floor(float(RS_T) * scale)) + want |> resize(RS_C * t_out) + env |> resize(RS_C * t_out) + for (to in range64(t_out)) { + let in_x = clamp((float(to) + 0.5) / scale - 0.5, 0.0, float(RS_T - 1l)) + let i1 = min(int64(in_x), RS_T - 1l) + let i2 = min(i1 + 1l, RS_T - 1l) + var dx1 = double(abs(in_x - float(i1))) + var dx2 = double(abs(in_x - float(i2))) + if (i1 == i2) { + dx1 = 0.5lf + dx2 = 0.5lf + } + for (ci in range64(RS_C)) { + let a = double(x[ci * RS_T + i1]) + let b = double(x[ci * RS_T + i2]) + want[ci * t_out + to] = dx2 * a + dx1 * b + env[ci * t_out + to] = dx2 * abs(a) + dx1 * abs(b) + } + } + return t_out +} + +def private rl_torch_ref(x : array; scale : float; var want : array; var env : array) : int64 { + let t_out = int64(floor(float(RS_T) * scale)) + want |> resize(RS_C * t_out) + env |> resize(RS_C * t_out) + let inv = double(1.0 / scale) + for (to in range64(t_out)) { + let src = max(0.0, float(inv * (double(to) + 0.5lf) - 0.5lf)) + let i0 = min(int64(src), RS_T - 1l) + let i1 = min(i0 + 1l, RS_T - 1l) + let l1 = double(src - float(i0)) + let l0 = 1.0lf - l1 + for (ci in range64(RS_C)) { + let a = double(x[ci * RS_T + i0]) + let b = double(x[ci * RS_T + i1]) + want[ci * t_out + to] = l0 * a + l1 * b + env[ci * t_out + to] = l0 * abs(a) + l1 * abs(b) + } + } + return t_out +} + +def private resize_onnx_case(scale : float; poison : bool) : KCase { + var x : array + rs_fixture(x, scale) + var want : array + var env : array + let want_t = rl_onnx_ref(x, scale, want, env) + var y : array + let got_t = resize_linear(x, RS_C, RS_T, scale, y) + let nm = "resize_linear (ONNX half-pixel) scale={scale}" + if (got_t != want_t) { + return KCase(name = nm, bad = 1000000) + } + if (poison) { + return score_poison(nm, y, want, env, RS_REL, RS_FLOOR, 0.001lf) + } + return score(nm, y, want, env, RS_REL, RS_FLOOR) +} + +def private resize_torch_case(scale : float; poison : bool) : KCase { + var x : array + rs_fixture(x, scale) + var want : array + var env : array + let want_t = rl_torch_ref(x, scale, want, env) + var y : array + let got_t = resize_linear_torch(x, RS_C, RS_T, scale, y) + let nm = "resize_linear_torch (align_corners false) scale={scale}" + if (got_t != want_t) { + return KCase(name = nm, bad = 1000000) + } + if (poison) { + return score_poison(nm, y, want, env, RS_REL, RS_FLOOR, 0.001lf) + } + return score(nm, y, want, env, RS_REL, RS_FLOOR) +} + +let MP_BINS = 13l +let MP_FRAMES = 37l +let MP_EPS = 1.0e-9 +let MP_PHASE_ENV = 4.0lf +let MP_REL = 2.0e-6lf +let MP_FLOOR = 1.0e-6lf + +[sideeffects] +def private neg_zero() : float { + return unsafe(reinterpret(0x80000000u)) +} + +def private mp_fixture(var re : array; var im : array) { + lcg_seed(0x3A9Ful) + fill(re, MP_BINS * MP_FRAMES, 2.0) + fill(im, MP_BINS * MP_FRAMES, 2.0) + re[0] = -1.5 + im[0] = 0.0 + re[1] = -1.5 + im[1] = neg_zero() + re[2] = 2.5 + im[2] = 0.0 + re[3] = 2.5 + im[3] = neg_zero() + re[4] = neg_zero() + im[4] = neg_zero() +} + +def private mp_ref(re : array; im : array; var want : array; var env : array) { + let nb = MP_BINS * MP_FRAMES + want |> resize(2l * nb) + env |> resize(2l * nb) + for (i in range64(nb)) { + let r = double(re[i]) + let m = double(im[i]) + let mag = sqrt(r * r + m * m + double(MP_EPS)) + want[i] = mag + env[i] = mag + want[nb + i] = im[i] == 0.0 ? (re[i] < 0.0 ? DBL_PI : 0.0lf) : atan2(m, r) + env[nb + i] = MP_PHASE_ENV + } +} + +def private mp_case(poison : bool) : KCase { + var re : array + var im : array + mp_fixture(re, im) + var want : array + var env : array + mp_ref(re, im, want, env) + var out : array + magnitude_phase(re, im, MP_BINS, MP_FRAMES, MP_EPS, out) + let nm = "magnitude_phase {MP_BINS}x{MP_FRAMES} with the zero-imaginary branch planted" + if (poison) { + return score_poison(nm, out, want, env, MP_REL, MP_FLOOR, 0.01lf) + } + return score(nm, out, want, env, MP_REL, MP_FLOOR) +} + +let IS_REL = 4.0e-6lf +let IS_FLOOR = 1.0e-7lf + +def private is_window(var w : array; k : int64; zeroed : bool) { + w |> resize(k) + for (v in w) { + let u = 0.4 + 0.6 * abs(lcg_unit()) + v = zeroed ? 0.0 : u + } +} + +def private istft_ref(wave0 : array; n : int64; window : array; k : int64; hop : int64; + pad : int64; var want : array; var env : array) { + want |> resize(n) + env |> resize(n) + for (i in range64(n)) { + let m = i + pad + var acc = 0.0lf + var f = (m - k + hop) / hop + if (f < 0l) { + f = 0l + } + while (f * hop <= m) { + let j = m - f * hop + if (j < k) { + acc += double(window[j]) * double(window[j]) + } + f++ + } + let w0 = double(wave0[i]) + want[i] = acc > 1.0e-11lf ? w0 / acc : w0 + env[i] = abs(want[i]) + } +} + +def private istft_case(n : int64; k : int64; hop : int64; pad : int64; zeroed : bool; poison : bool) : KCase { + lcg_seed(0x157Ful + uint64(k * 97l + hop)) + var window : array + is_window(window, k, zeroed) + var wave : array + fill(wave, n, 2.0) + let wave0 := wave + var want : array + var env : array + istft_ref(wave0, n, window, k, hop, pad, want, env) + istft_envelope_divide(wave, n, window, k, hop, pad) + let nm = "istft_envelope_divide n={n} k={k} hop={hop} pad={pad} zero-window={zeroed}" + if (poison) { + return score_poison(nm, wave, want, env, IS_REL, IS_FLOOR, 0.01lf) + } + return score(nm, wave, want, env, IS_REL, IS_FLOOR) +} + +[test] +def test_tower_gemm_wrappers(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("the wrappers against an fp64 GEMM, and the plane, per-tensor-flag and zero-padded forms against them") @(t : T?) { + report(t, mm_blob_case(false)) + report(t, mm_bf16_case(false)) + report(t, mm_plane_case(false)) + report(t, mm_route_case(true, false)) + report(t, mm_route_case(false, false)) + report(t, mm_pad_case(false)) + } + t |> run("the GEMM bars' must-red controls") @(t : T?) { + report_poison(t, mm_blob_case(true)) + report_poison(t, mm_bf16_case(true)) + report_poison(t, mm_plane_case(true)) + report_poison(t, mm_route_case(true, true)) + report_poison(t, mm_route_case(false, true)) + report_poison(t, mm_pad_case(true)) + } + } +} + +[test] +def test_tower_layernorm(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("layernorm and layernorm_batch against the fp64 population-variance form") @(t : T?) { + report(t, ln_scalar_case(false)) + report(t, ln_batch_case(false)) + report(t, ln_plane_case(false)) + } + t |> run("the layernorm bars' must-red controls") @(t : T?) { + report_poison(t, ln_scalar_case(true)) + report_poison(t, ln_batch_case(true)) + report_poison(t, ln_plane_case(true)) + } + } +} + +[test] +def test_tower_row_adds(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("the bias and residual row adds against the scalar f32 loop") @(t : T?) { + report(t, add_bias_case(false, false)) + report(t, add_bias_case(true, false)) + report(t, add_inplace_case(false)) + } + t |> run("the row adds' must-red controls") @(t : T?) { + report_poison(t, add_bias_case(false, true)) + report_poison(t, add_bias_case(true, true)) + report_poison(t, add_inplace_case(true)) + } + } +} + +[test] +def test_tower_gelu_erf_batch(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("gelu_erf_batch against an fp64 erf series, and against its whole-image form") @(t : T?) { + report(t, gelu_batch_case(false)) + report(t, gelu_shape_case(false)) + } + t |> run("the gelu bars' must-red controls") @(t : T?) { + report_poison(t, gelu_batch_case(true)) + report_poison(t, gelu_shape_case(true)) + } + } +} + +[test] +def test_tower_rope_neox_tab_rows(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("the full-head table rope against the fp64 rotation, and its per-row table index") @(t : T?) { + report(t, rope_case(false)) + report(t, rope_row_index_case(0l, false)) + report(t, rope_row_index_case(11l, false)) + } + t |> run("the rope bars' must-red controls") @(t : T?) { + report_poison(t, rope_case(true)) + report_poison(t, rope_row_index_case(11l, true)) + } + } +} + +[test] +def test_tower_attention_windows(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("attention_bidir_windows against an oracle that attends strictly inside each window") @(t : T?) { + report(t, aw_case(false)) + } + t |> run("the window attention bar's must-red control") @(t : T?) { + report_poison(t, aw_case(true)) + } + } +} + +[test] +def test_tower_interpolate_grid_aa(t : T?) { + with_job_que() { + setup_dasllama_jobque_() + t |> run("the antialiased grid resample down and up, at a non-zero plane offset") @(t : T?) { + report(t, interp_case(48l, 48l, 14l, 14l, 8l, 0l, false)) + report(t, interp_case(48l, 48l, 27l, 13l, 4l, 37l, false)) + report(t, interp_case(4l, 4l, 9l, 9l, 3l, 11l, false)) + } + t |> run("the resample bars' must-red controls") @(t : T?) { + report_poison(t, interp_case(48l, 48l, 14l, 14l, 8l, 0l, true)) + report_poison(t, interp_case(4l, 4l, 9l, 9l, 3l, 11l, true)) + } + } +} + +[test] +def test_audio_fft_and_dft_tables(t : T?) { + t |> run("the radix-2 plan's bit-reversal permutation and its twiddles") @(t : T?) { + t |> equal(fft_plan_rev_bad(256l), 0, "build_fft_plan(256) rev is an involutive permutation with rev[1] = n/2") + t |> equal(fft_plan_rev_bad(64l), 0, "build_fft_plan(64) rev is an involutive permutation with rev[1] = n/2") + report(t, fft_plan_tw_case(256l, false)) + t |> success(fft_plan_refuses(6l), "build_fft_plan refuses a non-power-of-2 size") + t |> success(fft_plan_refuses(1l), "build_fft_plan refuses a size below 2") + } + t |> run("fft_pow2_run against an fp64 O(n^2) DFT of the same forward convention") @(t : T?) { + report(t, fft_case(64l, false)) + report(t, fft_case(256l, false)) + } + t |> run("the 400-point DFT twiddle matrix against fp64") @(t : T?) { + report(t, dft_twiddle_case(false)) + } + t |> run("the FFT and DFT bars' must-red controls") @(t : T?) { + report_poison(t, fft_plan_tw_case(256l, true)) + report_poison(t, fft_case(256l, true)) + report_poison(t, dft_twiddle_case(true)) + } +} + +[test] +def test_tts_linear_resamplers(t : T?) { + t |> run("both interpolation conventions against an fp64 two-tap reference") @(t : T?) { + report(t, resize_onnx_case(2.0, false)) + report(t, resize_onnx_case(1.5, false)) + report(t, resize_onnx_case(0.5, false)) + report(t, resize_torch_case(2.0, false)) + report(t, resize_torch_case(1.5, false)) + report(t, resize_torch_case(0.5, false)) + } + t |> run("the resampler bars' must-red controls") @(t : T?) { + report_poison(t, resize_onnx_case(1.5, true)) + report_poison(t, resize_torch_case(1.5, true)) + } +} + +[test] +def test_tts_magnitude_phase(t : T?) { + t |> run("magnitude_phase against an fp64 reference, the zero-imaginary branch planted at both zero signs") @(t : T?) { + report(t, mp_case(false)) + } + t |> run("the magnitude_phase bar's must-red control") @(t : T?) { + report_poison(t, mp_case(true)) + } +} + +[test] +def test_tts_istft_envelope(t : T?) { + t |> run("the istft window-envelope divide against an fp64 envelope, and its zero-envelope guard") @(t : T?) { + report(t, istft_case(200l, 32l, 8l, 16l, false, false)) + report(t, istft_case(137l, 24l, 7l, 9l, false, false)) + report(t, istft_case(96l, 32l, 8l, 0l, false, false)) + report(t, istft_case(200l, 32l, 8l, 16l, true, false)) + } + t |> run("the envelope bars' must-red controls") @(t : T?) { + report_poison(t, istft_case(200l, 32l, 8l, 16l, false, true)) + report_poison(t, istft_case(200l, 32l, 8l, 16l, true, true)) + } +} From efa727c8d7011477186fca9fa12528fe68ecf7a1 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 15:31:21 -0700 Subject: [PATCH 04/43] test_q8q8_family: the s16 plane's group3 form is bit-exact against the f32 plane on portable and envelope-bounded on a generated backend - the rebased tree stamps a different profile and the two stamps fold the row tail differently Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/tests/CLAUDE.md | 7 ++++--- modules/dasLLAMA/tests/test_q8q8_family.das | 18 +++++++++++------- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/modules/dasLLAMA/tests/CLAUDE.md b/modules/dasLLAMA/tests/CLAUDE.md index 3571497bee..b19b95c237 100644 --- a/modules/dasLLAMA/tests/CLAUDE.md +++ b/modules/dasLLAMA/tests/CLAUDE.md @@ -643,9 +643,10 @@ GEMVs. The dispatch wrappers run on every backend `kernel_backend_names()` lists backends through their own `repack_q8q8_weight`, never on row-major data - with `matmul_q8q8_batch` bit-exact against ntok x `matmul_q8q8`. The wscale_f16 arm covers `dot_q8q8_f16s`, the `_s16` rows and groupN kernels, the stamped s16 tile/GEMV twins and the -`matmul_q8q8` / `_batch` / `_groupn` s16 overloads, bit-exact against their f32 twins over -f16-exact scales. `matmul_q8q8_group3` (f32 and s16) runs against three independent GEMVs on -unequal regions 32/40/44 (the row tail); `matmul_q8` / `dot_q8` cover the fp32-activation rail; +`matmul_q8q8` / `_batch` / `_groupn` s16 overloads, against their f32 twins over f16-exact +scales - bit-exact on the portable backend, within the fp64 bar on a generated one whose s16 +and f32 stamps fold differently. `matmul_q8q8_group3` (f32 and s16) runs against three +independent GEMVs on unequal regions 32/40/44 (the row tail); `matmul_q8` / `dot_q8` cover the fp32-activation rail; the mx4 cell drives `matmul_mx4q8_batch` and `matmul_mx4q8_batch_groupn` against ntok independent `matmul_mx4q8` GEMVs and the `dot_mx4q8_scalar` leaf, pinned portable and swept over the repack backends that carry those slots. Nothing here self-skips off-JIT - every cell diff --git a/modules/dasLLAMA/tests/test_q8q8_family.das b/modules/dasLLAMA/tests/test_q8q8_family.das index da3410130e..35321793a7 100644 --- a/modules/dasLLAMA/tests/test_q8q8_family.das +++ b/modules/dasLLAMA/tests/test_q8q8_family.das @@ -626,10 +626,8 @@ def private group3_gate(t : T?; n : int64) { let saved = active_kernel_backend() for (name in kernel_backend_names()) { pin_kernel_backend(name) - var q : array - var s : array - q := wq0.q - s := ws32m + var q := wq0.q + var s := ws32m if (kernel_backend_needs_repack(name)) { repack_q8q8_weight(q, s, woff0, n, d0) repack_q8q8_weight(q, s, woff1, n, d1) @@ -670,9 +668,15 @@ def private group3_gate(t : T?; n : int64) { h1 |> resize(d1) h2 |> resize(d2) matmul_q8q8_group3(h0, h1, h2, q, s16, woff0, woff1, woff2, xq, xs, n, d0, d1, d2) - identical(t, h0, y0, "{name} group3 s16 region 0 bit-matches the f32 plane, n={n}") - identical(t, h1, y1, "{name} group3 s16 region 1 bit-matches the f32 plane, n={n}") - identical(t, h2, y2, "{name} group3 s16 region 2 bit-matches the f32 plane, n={n}") + if (name == "portable") { + identical(t, h0, y0, "{name} group3 s16 region 0 bit-matches the f32 plane, n={n}") + identical(t, h1, y1, "{name} group3 s16 region 1 bit-matches the f32 plane, n={n}") + identical(t, h2, y2, "{name} group3 s16 region 2 bit-matches the f32 plane, n={n}") + } else { + var fused16 : array + concat3(h0, h1, h2, fused16) + held(t, fused16, ref, bar, "{name} matmul_q8q8_group3 s16 within the fp64 bar, n={n}") + } } restore_backend(saved) } From 305ed7de09ba875be082ac6edbcc671f4915b89e Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 15:48:27 -0700 Subject: [PATCH 05/43] stage 1, wave 1: f16_cvt moves to daslib (37 requires follow; the JIT keys its half-convert intrinsics on the module name, which stays), the NEON tier's require in the engine takes the ?llvm/daslib/aarch64_neon path guard, and run.das gains --no-tune (DAS_TUNE_POLICY=reference in every child) - the portable tier's arm of the gate Co-Authored-By: Claude Fable 5.1 --- {modules/dasLLVM/daslib => daslib}/f16_cvt.das | 0 .../dasLLAMA/benchmarks/matmul/bench_metal_gemv_kernels.das | 2 +- modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das | 2 +- modules/dasLLAMA/dasllama/dasllama_common.das | 2 +- modules/dasLLAMA/dasllama/dasllama_convert.das | 2 +- modules/dasLLAMA/dasllama/dasllama_g2p.das | 2 +- modules/dasLLAMA/dasllama/dasllama_gguf.das | 2 +- modules/dasLLAMA/dasllama/dasllama_gpu_resident.das | 2 +- modules/dasLLAMA/dasllama/dasllama_kv_codec.das | 2 +- modules/dasLLAMA/dasllama/dasllama_load.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math_accelerate.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math_default.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math_gen.das | 2 +- modules/dasLLAMA/harness/accel_contention_probe.das | 2 +- modules/dasLLAMA/harness/gen_tune_probe.das | 2 +- modules/dasLLAMA/harness/moe_kq_probe.das | 2 +- modules/dasLLAMA/harness/p2_skw2_probe.das | 2 +- modules/dasLLAMA/harness/tq4_basis_probe.das | 2 +- modules/dasLLAMA/harness/tq4_order_probe.das | 2 +- modules/dasLLAMA/harness/tune_kernels.das | 2 +- modules/dasLLAMA/portable_tier_plan.md | 6 ++++-- modules/dasLLAMA/tests/CLAUDE.md | 4 +++- modules/dasLLAMA/tests/_metal_kernel_common.das | 2 +- modules/dasLLAMA/tests/_vk_kq_fixtures.das | 2 +- modules/dasLLAMA/tests/run.das | 6 ++++++ modules/dasLLAMA/tests/test_batch_decode.das | 2 +- modules/dasLLAMA/tests/test_kv_codec.das | 2 +- modules/dasLLAMA/tests/test_metal_decode_parity.das | 2 +- modules/dasLLAMA/tests/test_metal_misc_kernels.das | 2 +- modules/dasLLAMA/tests/test_metal_prefill_kernels.das | 2 +- modules/dasLLAMA/tests/test_parity.das | 2 +- modules/dasLLAMA/tests/test_prefill_cpu_kernels.das | 2 +- modules/dasLLAMA/tests/test_q8q8_family.das | 2 +- modules/dasLLAMA/tests/test_repack.das | 2 +- modules/dasLLAMA/tests/test_tower_helpers.das | 2 +- modules/dasLLAMA/tests/test_vulkan_dec_tail.das | 2 +- modules/dasLLAMA/tests/test_vulkan_moe_cm2.das | 2 +- modules/dasLLAMA/tests/test_vulkan_tier.das | 2 +- modules/dasLLVM/.das_module | 1 - modules/dasLLVM/daslib/llvm_jit_intrin.das | 2 +- modules/dasSpirv/spirv/spirv_builder.das | 3 ++- tests/aot/CMakeLists.txt | 4 ++-- tests/jit_tests/f16_cvt.das | 2 +- tests/metal/test_metal_tensor_ops.das | 2 +- 46 files changed, 56 insertions(+), 46 deletions(-) rename {modules/dasLLVM/daslib => daslib}/f16_cvt.das (100%) diff --git a/modules/dasLLVM/daslib/f16_cvt.das b/daslib/f16_cvt.das similarity index 100% rename from modules/dasLLVM/daslib/f16_cvt.das rename to daslib/f16_cvt.das diff --git a/modules/dasLLAMA/benchmarks/matmul/bench_metal_gemv_kernels.das b/modules/dasLLAMA/benchmarks/matmul/bench_metal_gemv_kernels.das index 0877b19b22..fdbbb4ba26 100644 --- a/modules/dasLLAMA/benchmarks/matmul/bench_metal_gemv_kernels.das +++ b/modules/dasLLAMA/benchmarks/matmul/bench_metal_gemv_kernels.das @@ -11,7 +11,7 @@ require ?das_metal dasllama/dasllama_metal_gemm // the production 32x32 GEMM MS require dasllama/dasllama_metal_kernels // the production GEMV MSL (v0 baseline) require daslib/typemacro_boost // [template_struct_instance] - the simdgroup-matrix lab stamps require ?das_metal dasllama/dasllama_metal_prefill // MetalKqMulMmK45TensorT - the Metal-4 tensor GEMM stamped at verify widths -require llvm/daslib/f16_cvt // f16 bit converts for the K-quant superblock fills +require daslib/f16_cvt // f16 bit converts for the K-quant superblock fills require math // used only inside the Apple static_if half require strings diff --git a/modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das b/modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das index 4883b8237b..93918eb9fe 100644 --- a/modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das +++ b/modules/dasLLAMA/benchmarks/matmul/bench_metal_moe_lab.das @@ -10,7 +10,7 @@ require dasllama/dasllama_env // the g_env_* knob globals ([EnvConfig] declarat require metal/msl_shader require dasllama/dasllama_metal_prefill // the production MoE mul_mm MSL (zero copy drift) require dasllama/dasllama_metal_kernels // the production MoE GEMV + geglu MSL (zero copy drift) -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/fio // used only inside the Apple static_if half require math // used only inside the Apple static_if half require daslib/math_bits // uint_bits_to_float in the lcppe kernel diff --git a/modules/dasLLAMA/dasllama/dasllama_common.das b/modules/dasLLAMA/dasllama/dasllama_common.das index 74148544ee..ed6dee16f6 100644 --- a/modules/dasLLAMA/dasllama/dasllama_common.das +++ b/modules/dasLLAMA/dasllama/dasllama_common.das @@ -18,7 +18,7 @@ require daslib/jobque_profile // JOBQUE_PROFILING_ENABLED — the compile-time require dasllama/dasllama_kv_codec public // the KV-cache codec families — kv_store_row/kv_load_row/kv_dot/kv_axpy dispatch into them require dasllama/dasllama_convert // cvt_f32_to_f16/cvt_f16_to_f32 — the f16 rows (wscale plane + the f16 KV codec) require dasllama/dasllama_math_default // registers the portable Q8·Q8 backend at [init] (the fallback everywhere) -require dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init]; no-op off-ARM +require ?llvm/daslib/aarch64_neon dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init] under the JIT alone; PATH guard: without dasLLVM its intrinsics are scalar fallbacks slower than portable, so the tier stays out require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen // [init]-only: registers "arm64-gen" (the generated GEMM family, arm64 load-select tier). PATH guard: the module hard-requires llvm_tune, so it loads exactly when dasLLVM is mounted require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): the auto-policy pass and tune_status reach every app that requires this module require ?das_metal dasllama/dasllama_metal_gemm // [init]-only: registers "metal" (the GPU prefill-GEMM batch donor). STRICT guard: the C++ das_metal module exists on Apple builds only, so non-Apple lanes never compile the Metal driver diff --git a/modules/dasLLAMA/dasllama/dasllama_convert.das b/modules/dasLLAMA/dasllama/dasllama_convert.das index e84a957117..5e243178fa 100644 --- a/modules/dasLLAMA/dasllama/dasllama_convert.das +++ b/modules/dasLLAMA/dasllama/dasllama_convert.das @@ -6,7 +6,7 @@ module dasllama_convert shared public require dasllama/dasllama_lint public require math -require llvm/daslib/f16_cvt // f16<->f32 halfword codecs — the Q5_K encoder + every f16 scale read +require daslib/f16_cvt // f16<->f32 halfword codecs — the Q5_K encoder + every f16 scale read require daslib/jobque_boost public // quantize_k5_plane threads its superblock loop require dasllama/dasllama_par // maybe_parallel_for (+ its emitted dispatch counters) require dasllama/dasllama_math // get_dispatch_lanes — the requant wrappers shape their fan-out diff --git a/modules/dasLLAMA/dasllama/dasllama_g2p.das b/modules/dasLLAMA/dasllama/dasllama_g2p.das index 63f34e7367..28ae44a3b9 100644 --- a/modules/dasLLAMA/dasllama/dasllama_g2p.das +++ b/modules/dasLLAMA/dasllama/dasllama_g2p.das @@ -10,7 +10,7 @@ require strings require daslib/strings_boost require daslib/fio require daslib/utf8_utils -require llvm/daslib/f16_cvt // the pack stores the GRU as f16; the reader widens it once at load +require daslib/f16_cvt // the pack stores the GRU as f16; the reader widens it once at load require dasllama/dasllama_unicode require dasllama/dasllama_postag require dasllama/dasllama_textnorm diff --git a/modules/dasLLAMA/dasllama/dasllama_gguf.das b/modules/dasLLAMA/dasllama/dasllama_gguf.das index 743b88a045..d033a9d8ab 100644 --- a/modules/dasLLAMA/dasllama/dasllama_gguf.das +++ b/modules/dasLLAMA/dasllama/dasllama_gguf.das @@ -12,7 +12,7 @@ require daslib/fio // shard-path expansion + mapping (split GGU require daslib/strings_boost // trim_suffix / pad_left (split shard names) require daslib/strings_convert // try_to_int (validating shard-index parse) require dasllama/dasllama_env // g_env_engine.mtp_head — the split NextN head sidecar override -require llvm/daslib/f16_cvt public // f16_to_f32 / f32_to_f16 (re-export: whisper/vad/parakeet/audio callers) +require daslib/f16_cvt public // f16_to_f32 / f32_to_f16 (re-export: whisper/vad/parakeet/audio callers) require dasllama/dasllama_convert public // the pure codec layer (readers/dequants/transcodes/encoder) — split out 2026-07-29 require dasllama/dasllama_par // maybe_parallel_for (+ its emitted dispatch counters) — the transcode hot loops thread when a jobque is up diff --git a/modules/dasLLAMA/dasllama/dasllama_gpu_resident.das b/modules/dasLLAMA/dasllama/dasllama_gpu_resident.das index 8341f857bd..44e83908c3 100644 --- a/modules/dasLLAMA/dasllama/dasllama_gpu_resident.das +++ b/modules/dasLLAMA/dasllama/dasllama_gpu_resident.das @@ -17,7 +17,7 @@ require dasllama/dasllama_kqformat require dasllama/dasllama_math require dasllama/dasllama_rope // build_rope_row_packed — the one source for every resident rope row require dasllama/dasllama_gemm_schema // Q8_BLOCK_ELEMS / KQ_SUPERBLOCK_ELEMS — device block sizing -require llvm/daslib/f16_cvt // f32_to_f16 — the trim pass re-packs the emb scale plane +require daslib/f16_cvt // f32_to_f16 — the trim pass re-packs the emb scale plane require dasllama/dasllama_par require dasllama/dasllama_env require math diff --git a/modules/dasLLAMA/dasllama/dasllama_kv_codec.das b/modules/dasLLAMA/dasllama/dasllama_kv_codec.das index fc90442883..42dfc43e52 100644 --- a/modules/dasLLAMA/dasllama/dasllama_kv_codec.das +++ b/modules/dasLLAMA/dasllama/dasllama_kv_codec.das @@ -6,7 +6,7 @@ module dasllama_kv_codec shared public require dasllama/dasllama_lint public require math -require llvm/daslib/f16_cvt // per-element f16 scale reads/writes inside the hot loops +require daslib/f16_cvt // per-element f16 scale reads/writes inside the hot loops require dasllama/dasllama_math // dot/axpy/copy_floats — the f32 arms of the dispatch below require dasllama/dasllama_convert // cvt_f16_to_f32 — the bulk f16 row dequant require dasllama/dasllama_tune // [tuned]: reconstitute kernels from templates with per-box loop hints diff --git a/modules/dasLLAMA/dasllama/dasllama_load.das b/modules/dasLLAMA/dasllama/dasllama_load.das index 60bd97f697..04748cfeb9 100644 --- a/modules/dasLLAMA/dasllama/dasllama_load.das +++ b/modules/dasLLAMA/dasllama/dasllama_load.das @@ -22,7 +22,7 @@ require dasllama/dasllama_tune require dasllama/dasllama_env require dasllama/dasllama_par require dasllama/dasllama_gpu_tier -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/json_boost require daslib/array_boost require dasllama/dasllama_tokenizer // load_tokenizer_auto — load_model_ populates the Model's tokenizer diff --git a/modules/dasLLAMA/dasllama/dasllama_math.das b/modules/dasLLAMA/dasllama/dasllama_math.das index 2d51af418b..2f7270e938 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math.das +++ b/modules/dasLLAMA/dasllama/dasllama_math.das @@ -16,7 +16,7 @@ require dasllama/dasllama_tune // [tuned]: reconstitute dot from dot_template w require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tuned] kernels pick their perms require dasllama/dasllama_gemm_schema // kq_qsb/kq_ssb — the fmt-id kq plane strides require dasllama/dasllama_gpu_tier public // the device-cooperation seam (split out 2026-07-29) — re-exported for the 112 requirers -require llvm/daslib/f16_cvt // f16<->f32 converts: JIT half intrinsics on aarch64 / x64+F16C, exact bit-twiddle elsewhere +require daslib/f16_cvt // f16<->f32 converts: JIT half intrinsics on aarch64 / x64+F16C, exact bit-twiddle elsewhere // dasLLAMA numeric primitives — naive fp32 baseline (curriculum step 1). // Correctness first; optimization (blocking, float4, threading) comes later. diff --git a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das index d1b2f268d7..cc7ae8d694 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das @@ -10,7 +10,7 @@ require dasllama/dasllama_math_default // dot_q8q8_idot4x4/_f16s — the plain- require dasllama/dasllama_tune // [tuned]: reconstitute the laneq4x4 tile from its template with per-box loop hints require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tuned] kernels pick their perms require llvm/daslib/aarch64_neon // tbl16/sdot4_w/sdot4_laneq: the mx4 + laneq tiles' JIT intrinsics -require llvm/daslib/f16_cvt // f16_to_f32 — the wscale_f16 twins widen the binary16 scale in-loop +require daslib/f16_cvt // f16_to_f32 — the wscale_f16 twins widen the binary16 scale in-loop require daslib/jobque_boost public // kernels expand parallel_for; consumers need its symbols require dasllama/dasllama_par // maybe_parallel_for diff --git a/modules/dasLLAMA/dasllama/dasllama_math_accelerate.das b/modules/dasLLAMA/dasllama/dasllama_math_accelerate.das index 992d2aab34..9d1a460acd 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_accelerate.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_accelerate.das @@ -10,7 +10,7 @@ require dasllama/dasllama_convert // cvt_f32_to_f16 — the BNNS f16 batch prep require dasllama/dasllama_env // the g_env_* knob globals ([EnvConfig] declarations; ENVIRONMENT.md is generated from them) require dasllama/dasllama_par require das_accelerate -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math require daslib/math_bits diff --git a/modules/dasLLAMA/dasllama/dasllama_math_default.das b/modules/dasLLAMA/dasllama/dasllama_math_default.das index df55a1fce2..71118c346b 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_default.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_default.das @@ -9,7 +9,7 @@ require dasllama/dasllama_math // KernelBackend + register_kernel_backend require dasllama/dasllama_tune // [tuned]: reconstitute dot_q8q8 from its template with per-box loop hints require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tuned] kernels pick their perms require dasllama/dasllama_gemm_schema // kq_qsb/kq_ssb — the fmt-id kq plane strides -require llvm/daslib/f16_cvt // dot_q8q8_f16s widens the binary16 weight scale in-loop +require daslib/f16_cvt // dot_q8q8_f16s widens the binary16 weight scale in-loop require daslib/jobque_boost public // parallel_for / notify_and_release (the kernels expand parallel_for) require dasllama/dasllama_par // maybe_parallel_for require math diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 3df6053d3e..9de55526da 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -10,7 +10,7 @@ require dasllama/dasllama_math require dasllama/dasllama_math_aarch64_neon require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails of the s16 twins require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) -require llvm/daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs +require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require llvm/daslib/llvm_tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code]; PUBLIC so the auto-policy infer pass and tune_status reach app roots that require this module require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema diff --git a/modules/dasLLAMA/harness/accel_contention_probe.das b/modules/dasLLAMA/harness/accel_contention_probe.das index 5608973df0..fe54094831 100644 --- a/modules/dasLLAMA/harness/accel_contention_probe.das +++ b/modules/dasLLAMA/harness/accel_contention_probe.das @@ -6,7 +6,7 @@ require das_accelerate require ?das_metal metal/das_metal_boost // the live-Metal-device ingredient (env METAL=1) require dasllama/dasllama_math require dasllama/dasllama_par -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/jobque_boost require math require dasllama/dasllama_env // nolint:STYLE030,LINT019 — the read sites sit in the apple static_if half; the linux lint flavor sees the require as unused diff --git a/modules/dasLLAMA/harness/gen_tune_probe.das b/modules/dasLLAMA/harness/gen_tune_probe.das index 916455c0bb..5ac110b09d 100644 --- a/modules/dasLLAMA/harness/gen_tune_probe.das +++ b/modules/dasLLAMA/harness/gen_tune_probe.das @@ -41,7 +41,7 @@ require dasllama/dasllama_repack require dasllama/dasllama_gemm_schema require dasllama/dasllama_convert require dasllama/dasllama_gguf // transcode_q*k_superblock — the kq fixture pack rail -require llvm/daslib/f16_cvt // f32_to_f16 — the kq fixtures pack disk superblock headers +require daslib/f16_cvt // f32_to_f16 — the kq fixtures pack disk superblock headers require llvm/daslib/llvm_tune require daslib/random require daslib/json // JsonValue — the null runtime arg of the deferred sidecar merge diff --git a/modules/dasLLAMA/harness/moe_kq_probe.das b/modules/dasLLAMA/harness/moe_kq_probe.das index e840709bb5..8ff3e7899d 100644 --- a/modules/dasLLAMA/harness/moe_kq_probe.das +++ b/modules/dasLLAMA/harness/moe_kq_probe.das @@ -12,7 +12,7 @@ require dasllama/dasllama_convert require dasllama/dasllama_repack require dasllama/dasllama_gemm_schema require daslib/jobque_boost -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math // ATTRIBUTION SWEEP: attributes the MoE CPU pp gap across kernel shape (dense vs narrow D), diff --git a/modules/dasLLAMA/harness/p2_skw2_probe.das b/modules/dasLLAMA/harness/p2_skw2_probe.das index 6003fb2034..d3ac38ef7c 100644 --- a/modules/dasLLAMA/harness/p2_skw2_probe.das +++ b/modules/dasLLAMA/harness/p2_skw2_probe.das @@ -5,7 +5,7 @@ options _dasllama_internal = true require metal/msl_shader require ?das_metal metal/das_metal_boost require ?das_metal dasllama/dasllama_metal_prefill // production MetalQ8MulMmTH128 MSL -require llvm/daslib/f16_cvt // f32_to_f16 - the blob's f16 scale halfwords +require daslib/f16_cvt // f32_to_f16 - the blob's f16 scale halfwords require strings // Phase-2 #16 sizing probe: the w2 (down-projection) shape at Qwen3-8B/512 geometry on the diff --git a/modules/dasLLAMA/harness/tq4_basis_probe.das b/modules/dasLLAMA/harness/tq4_basis_probe.das index 412759d89e..bbd67dd612 100644 --- a/modules/dasLLAMA/harness/tq4_basis_probe.das +++ b/modules/dasLLAMA/harness/tq4_basis_probe.das @@ -9,7 +9,7 @@ require dasllama/dasllama_transformer // umbrella fires each arch [init] regis require dasllama/dasllama_math require dasllama/dasllama_kv_codec // the tq4 codec + FWHT rotation under probe require dasllama/dasllama_tokenizer -require llvm/daslib/f16_cvt +require daslib/f16_cvt require dasllama/dasllama_env // Basis A/B on REAL cache rows: run SmolLM2 over real text with an f32 KV cache, then re-quantize diff --git a/modules/dasLLAMA/harness/tq4_order_probe.das b/modules/dasLLAMA/harness/tq4_order_probe.das index 652bc77c04..c58a9c07c3 100644 --- a/modules/dasLLAMA/harness/tq4_order_probe.das +++ b/modules/dasLLAMA/harness/tq4_order_probe.das @@ -9,7 +9,7 @@ require dasllama/dasllama_transformer // umbrella fires each arch [init] regis require dasllama/dasllama_math require dasllama/dasllama_kv_codec // quantize_tq4kv_row/cvt_tq4kv_to_f32 — the codec under probe require dasllama/dasllama_tokenizer -require llvm/daslib/f16_cvt +require daslib/f16_cvt require dasllama/dasllama_env // Is one channel order enough, or is it MoE-shaped? Run SmolLM2 over 5 text domains, learn a diff --git a/modules/dasLLAMA/harness/tune_kernels.das b/modules/dasLLAMA/harness/tune_kernels.das index 0d80684342..21657f7383 100644 --- a/modules/dasLLAMA/harness/tune_kernels.das +++ b/modules/dasLLAMA/harness/tune_kernels.das @@ -28,7 +28,7 @@ require ?das_metal dasllama/dasllama_metal_kernels // metal_tensor_race_decode require tuner_cli // the shared tuner CLI require tuner_noise // the noise gate + the measured floor, shared with the other half require llvm/daslib/aarch64_neon // sdot4_laneq (referenced by the cloned laneq template bodies) -require llvm/daslib/f16_cvt // f16 references + the cloned f16-kernel template bodies +require daslib/f16_cvt // f16 references + the cloned f16-kernel template bodies require math // mad (referenced by the cloned template bodies) require daslib/json require daslib/json_boost diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index e7f6fd6140..1cfe01ba45 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -1,6 +1,8 @@ # The portable tier - dasLLAMA without the JIT crowns: no-LLVM build, AOT, WASM64 - plan -**Status: decisions settled (2026-09-04 discussion), pre-implementation.** The arc has one +**Status: stage 0 landed (2026-09-04) - the coverage additions (a)-(h) are in `tests/` and green +on this box, `test_parity` reproduces the frozen ids on the portable backend, and the four owed +rule fixes are in; stage 1 in progress.** Decisions settled in the 2026-09-04 discussion. The arc has one product goal, reached in three stages that each prove the next one's premise: first a regular `-jit` build with the `[tune]` framework short-circuited (the portable tier running on JIT-compiled reference bodies), then an AOT compilation target under `examples/dasLLAMA/` on a very small @@ -199,7 +201,7 @@ file. ## Stage 1 - the refactor, and the portable tier on a regular `-jit` build -1. `daslib/f16_cvt.das`; 33 require spellings. +1. `daslib/f16_cvt.das`; 37 require spellings (landed). 2. The inert annotation module in daslib carrying `tune`, `tune_perm`, `tune_companion`, `tune_scope`, `tune_policy` and `llvm_code` as shells: `require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune`, then `static_if (typeinfo builtin_module_exists(llvm_tune))` in each diff --git a/modules/dasLLAMA/tests/CLAUDE.md b/modules/dasLLAMA/tests/CLAUDE.md index b19b95c237..0347ddcd2c 100644 --- a/modules/dasLLAMA/tests/CLAUDE.md +++ b/modules/dasLLAMA/tests/CLAUDE.md @@ -39,7 +39,9 @@ files and `test_model_image` - it omits `test_mtp_gemma_drafter.das`, `image-vul Every suite but `model-free` and `stocked` needs `--arm`. `--full` parses and is then refused, so `--arm` is the only way in. `--suite model-free` and `--suite stocked` take neither - each is a whole gate; `--exclude ` drops the files whose name contains a token and names each -on an `EXCLUDED` line, so a trimmed run cannot read as full. The runner redirects +on an `EXCLUDED` line, so a trimmed run cannot read as full. `--no-tune` composes with any suite: +every child runs with `DAS_TUNE_POLICY=reference`, so the `[tune]` families and the `[tuned]` +hints fall to their reference bodies - the portable tier's arm of the gate. The runner redirects the COMPLETE output to a log file, and prints that path on the DONE line. It owns the dastest timeout, and repeats a file only when `--nreps` is passed explicitly (default 1, never best-of-N). Every child runs `-jit -module-cache .jitted_scripts/module_cache/dastest.dascache`; diff --git a/modules/dasLLAMA/tests/_metal_kernel_common.das b/modules/dasLLAMA/tests/_metal_kernel_common.das index b058f6de6a..c66a069e7a 100644 --- a/modules/dasLLAMA/tests/_metal_kernel_common.das +++ b/modules/dasLLAMA/tests/_metal_kernel_common.das @@ -9,7 +9,7 @@ module _metal_kernel_common shared public require dasllama/dasllama_lint public require math require dasllama/dasllama_convert // dequant_k{4,5,6}_plane_superblock — the kq reference dequant -require llvm/daslib/f16_cvt // f32_to_f16 — packing synthetic scale planes +require daslib/f16_cvt // f32_to_f16 — packing synthetic scale planes require ?das_metal metal/das_metal_boost // ===== device buffers ===== diff --git a/modules/dasLLAMA/tests/_vk_kq_fixtures.das b/modules/dasLLAMA/tests/_vk_kq_fixtures.das index e4313e6bc7..35078df9e0 100644 --- a/modules/dasLLAMA/tests/_vk_kq_fixtures.das +++ b/modules/dasLLAMA/tests/_vk_kq_fixtures.das @@ -1,7 +1,7 @@ options gen2 options _dasllama_internal = true -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math // Synthetic K-quant expert stacks in the vulkan tier's DEVICE layout, shared by the tier suite diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index 5f69f6aa74..bd35b529ba 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -52,6 +52,9 @@ struct RunArgs { @clarg_short = "n" @clarg_doc = "Repeat count — explicit only (default 1, never best-of-N)" nreps : int = 1 + @clarg_name = "no-tune" + @clarg_doc = "Short-circuit the [tune] framework in every child (DAS_TUNE_POLICY=reference): the reference bodies run — the portable tier's arm" + no_tune : bool @clarg_doc = "daslang binary (default: the binary running this script — a hardcoded ./bin/daslang picked up a STALE top-level exe on the MSVC layout)" das : string = "" @@ -421,6 +424,9 @@ def private child_envs(cfg : RunArgs; win : bool; arm : string) : string { envs = "DASLLAMA_TEST_FAMILY='{cfg.family}' {envs}" } } + if (cfg.no_tune) { + envs = (win ? "{envs}set \"DAS_TUNE_POLICY=reference\"&& " : "DAS_TUNE_POLICY=reference {envs}") + } if (cfg.suite == "image-vulkan") { // the offline-bake suite: arm the tier so the probed config carries a vulkan section — // the bake itself is DRY (no device calls), so this is safe on GPU-less boxes too diff --git a/modules/dasLLAMA/tests/test_batch_decode.das b/modules/dasLLAMA/tests/test_batch_decode.das index 1f61e2535f..32c479c012 100644 --- a/modules/dasLLAMA/tests/test_batch_decode.das +++ b/modules/dasLLAMA/tests/test_batch_decode.das @@ -6,7 +6,7 @@ options _dasllama_internal = true require dastest/testing_boost public require dasllama/dasllama_transformer // umbrella fires each arch [init] registration require dasllama/dasllama_math // setup_dasllama_jobque_ + the runtime knob setters (engine-level, not re-exported) -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/jobque_boost require daslib/fio require math diff --git a/modules/dasLLAMA/tests/test_kv_codec.das b/modules/dasLLAMA/tests/test_kv_codec.das index a1a4ffb6ee..1c6299bf1c 100644 --- a/modules/dasLLAMA/tests/test_kv_codec.das +++ b/modules/dasLLAMA/tests/test_kv_codec.das @@ -8,7 +8,7 @@ require dasllama/dasllama_transformer // umbrella fires each arch [init] regis require dasllama/dasllama_math // setup_dasllama_jobque_ + target_chunk_work + fused knob require dasllama/dasllama_kv_codec // the codec kernel families under unit test require dasllama/dasllama_convert // cvt_f32_to_f16/cvt_f16_to_f32 — the generic f16 rows -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/jobque_boost require daslib/fio require math diff --git a/modules/dasLLAMA/tests/test_metal_decode_parity.das b/modules/dasLLAMA/tests/test_metal_decode_parity.das index 2f14d1fcb1..b04ec33458 100644 --- a/modules/dasLLAMA/tests/test_metal_decode_parity.das +++ b/modules/dasLLAMA/tests/test_metal_decode_parity.das @@ -12,7 +12,7 @@ require ?das_metal dasllama/dasllama_metal_decode // stats/mirror/shutdown rail require ?das_metal dasllama/dasllama_metal_common // the greedy chain's stats and mode (arm 15) require ?das_metal dasllama/dasllama_metal_prefill // the Metal-prefill→GPU-decode hybrid arm require ?das_metal metal/das_metal_boost // metal_live_object_count for the leak gate -require llvm/daslib/f16_cvt // f16 KV row decode in the tolerance gate (Apple half only) +require daslib/f16_cvt // f16 KV row decode in the tolerance gate (Apple half only) require daslib/jobque_boost require daslib/fio // nolint:STYLE030,LINT019 — path_join/stat, used only inside the das_metal static_if halves require math diff --git a/modules/dasLLAMA/tests/test_metal_misc_kernels.das b/modules/dasLLAMA/tests/test_metal_misc_kernels.das index 9256daadd5..c418c91e59 100644 --- a/modules/dasLLAMA/tests/test_metal_misc_kernels.das +++ b/modules/dasLLAMA/tests/test_metal_misc_kernels.das @@ -13,7 +13,7 @@ require ?das_metal metal/das_metal_boost // device/pipeline/buffer plumbing + t require ?das_metal _metal_kernel_common // shared buf helpers + eyeball-dump compares require ?das_metal _mtl_toy // the [metal_dispatch] multi-kernel fixture require ?das_metal dasllama/dasllama_metal_lens // set_metal_requires_hook - the requires= tripwire observer -require llvm/daslib/f16_cvt // f32_to_f16 — packing the synthetic scale planes +require daslib/f16_cvt // f32_to_f16 — packing the synthetic scale planes require math require daslib/fio // the lens tgmem gate's spawned compiles require strings diff --git a/modules/dasLLAMA/tests/test_metal_prefill_kernels.das b/modules/dasLLAMA/tests/test_metal_prefill_kernels.das index dae9c85c5e..7325121fb5 100644 --- a/modules/dasLLAMA/tests/test_metal_prefill_kernels.das +++ b/modules/dasLLAMA/tests/test_metal_prefill_kernels.das @@ -12,7 +12,7 @@ require ?das_metal dasllama/dasllama_metal_prefill // the Phase-6 prefill kerne require ?das_metal dasllama/dasllama_metal_kernels // the Wave C MoE routing kernel set (Apple builds) require ?das_metal metal/das_metal_boost // device/pipeline/buffer plumbing + the leak gate require ?das_metal _metal_kernel_common // shared buf helpers + fixtures; only the tag-less mismatch compares stay local -require llvm/daslib/f16_cvt // f32_to_f16: the q51 gates hand-pack f16 d/m scale halfwords +require daslib/f16_cvt // f32_to_f16: the q51 gates hand-pack f16 d/m scale halfwords require math // The Phase-6 GPU prefill kernels vs their dasLLAMA CPU twins on synthetic data. rmsnorm / diff --git a/modules/dasLLAMA/tests/test_parity.das b/modules/dasLLAMA/tests/test_parity.das index 567c822d30..a379f47795 100644 --- a/modules/dasLLAMA/tests/test_parity.das +++ b/modules/dasLLAMA/tests/test_parity.das @@ -4,7 +4,7 @@ options _dasllama_internal = true require dastest/testing_boost public require dasllama/dasllama // the public facade — public-path tests exercise the API through it (engine + arch registrations + chat) -require llvm/daslib/f16_cvt // f16_to_f32 — the tied-cls arm reads the live scale plane +require daslib/f16_cvt // f16_to_f32 — the tied-cls arm reads the live scale plane require dasllama/dasllama_math // pin_kernel_backend / kernel_backend_names / active_kernel_backend — the facade wraps none of them require dasllama/dasllama_env // g_env_engine.pin_backend require daslib/defer // the backend pin is restored on every path out of a pinned cell diff --git a/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das b/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das index cef0bb4e03..057d934bce 100644 --- a/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das +++ b/modules/dasLLAMA/tests/test_prefill_cpu_kernels.das @@ -11,7 +11,7 @@ require dasllama/dasllama_convert require dasllama/dasllama_kv_codec require dasllama/dasllama_repack require dasllama/dasllama_gemm_schema -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math var g_taint = 0.0 diff --git a/modules/dasLLAMA/tests/test_q8q8_family.das b/modules/dasLLAMA/tests/test_q8q8_family.das index 35321793a7..dba660e1b2 100644 --- a/modules/dasLLAMA/tests/test_q8q8_family.das +++ b/modules/dasLLAMA/tests/test_q8q8_family.das @@ -9,7 +9,7 @@ require dasllama/dasllama_math_aarch64_neon require dasllama/dasllama_math_gen require dasllama/dasllama_repack require dasllama/dasllama_convert -require llvm/daslib/f16_cvt +require daslib/f16_cvt require daslib/jobque_boost require math diff --git a/modules/dasLLAMA/tests/test_repack.das b/modules/dasLLAMA/tests/test_repack.das index ee58c4d08f..f6c56a18fd 100644 --- a/modules/dasLLAMA/tests/test_repack.das +++ b/modules/dasLLAMA/tests/test_repack.das @@ -13,7 +13,7 @@ require dasllama/dasllama_repack require dasllama/dasllama_math_default // dequant_kq_row_grp — the independent grp reader require dasllama/dasllama_gguf // dequant_*_plane_superblock — the independent disk reader require dasllama/dasllama_gemm_schema // kq_qsb/kq_ssb — the fmt-id plane strides -require llvm/daslib/f16_cvt // plausible f16 scale halfwords +require daslib/f16_cvt // plausible f16 scale halfwords // deterministic byte source (LCG) — no Math.random in tests struct private Rng { diff --git a/modules/dasLLAMA/tests/test_tower_helpers.das b/modules/dasLLAMA/tests/test_tower_helpers.das index 49acf1900b..cc33cc6f86 100644 --- a/modules/dasLLAMA/tests/test_tower_helpers.das +++ b/modules/dasLLAMA/tests/test_tower_helpers.das @@ -8,7 +8,7 @@ require dasllama/dasllama_math // setup_dasllama_jobque_ require dasllama/dasllama_rope // build_rope_tabs_rows — the 2-axis rope's table source require dasllama/dasllama_plane require daslib/jobque_boost -require llvm/daslib/f16_cvt // the binary16 round-trip the geglu table property checks +require daslib/f16_cvt // the binary16 round-trip the geglu table property checks require math // Model-free unit tests for the encoder-tower helpers both the audio (gemma4a) and vision diff --git a/modules/dasLLAMA/tests/test_vulkan_dec_tail.das b/modules/dasLLAMA/tests/test_vulkan_dec_tail.das index 84961aaee0..c1f43375d1 100644 --- a/modules/dasLLAMA/tests/test_vulkan_dec_tail.das +++ b/modules/dasLLAMA/tests/test_vulkan_dec_tail.das @@ -12,7 +12,7 @@ require dasllama/dasllama_common // MoeGate, select_decode_override require dasllama/dasllama_blocks // registers the vulkan_moe_span decode override at init require dasllama/dasllama_moe // moe_select_core: the host select the span's device top-k mirrors require ?vulkan dasllama/dasllama_math_vulkan // the decode attention block + FFN tail under test; the test installs the hooks explicitly (no env gate) -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math require _vk_kq_fixtures diff --git a/modules/dasLLAMA/tests/test_vulkan_moe_cm2.das b/modules/dasLLAMA/tests/test_vulkan_moe_cm2.das index cd909992a1..e0791c1668 100644 --- a/modules/dasLLAMA/tests/test_vulkan_moe_cm2.das +++ b/modules/dasLLAMA/tests/test_vulkan_moe_cm2.das @@ -7,7 +7,7 @@ options _dasllama_internal = true require dastest/testing_boost public require dasllama/dasllama_math require ?vulkan dasllama/dasllama_math_vulkan // the MoE GPU tier under test; the test installs its hooks explicitly (no env gate) -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math require _vk_kq_fixtures require dasllama/dasllama_vulkan_common // find_stack_idx: the fresh-base guard diff --git a/modules/dasLLAMA/tests/test_vulkan_tier.das b/modules/dasLLAMA/tests/test_vulkan_tier.das index 4546f9c714..dc4d3a9e50 100644 --- a/modules/dasLLAMA/tests/test_vulkan_tier.das +++ b/modules/dasLLAMA/tests/test_vulkan_tier.das @@ -7,7 +7,7 @@ require dastest/testing_boost public require dasllama/dasllama_math require ?vulkan dasllama/dasllama_math_vulkan // the MoE GPU tier under test; the test installs its hooks explicitly (no env gate) require ?vulkan dasllama/dasllama_vulkan_seams // the single-op seams (vk_add_rms / rope_kv / decode_attn) on the class rail -require llvm/daslib/f16_cvt +require daslib/f16_cvt require math require _vk_kq_fixtures // the synthetic kq stacks + pow2 scale sets (shared with test_vulkan_moe_cm2) diff --git a/modules/dasLLVM/.das_module b/modules/dasLLVM/.das_module index e2a22f0cad..e0b63b9295 100644 --- a/modules/dasLLVM/.das_module +++ b/modules/dasLLVM/.das_module @@ -12,7 +12,6 @@ def initialize(project_path : string) { "llvm_env", // [EnvConfig] environment-knob registry (ENVIRONMENT.md generates from it) "aarch64_neon", // public NEON-intrinsic header (portable fallbacks + name-based JIT recognition) "x64_avx", // public x86-64-intrinsic header (same contract, x64 mirror) - "f16_cvt", // public f16<->f32 convert header (same contract, aarch64 + x64/F16C) "llvm_code", // [llvm_code] user-side annotation "llvm_jit_code", // [llvm_code] JIT-side generator registry "llvm_user_modules", // user-extensible require list for generator modules diff --git a/modules/dasLLVM/daslib/llvm_jit_intrin.das b/modules/dasLLVM/daslib/llvm_jit_intrin.das index cc8b2df1bd..4c16103320 100644 --- a/modules/dasLLVM/daslib/llvm_jit_intrin.das +++ b/modules/dasLLVM/daslib/llvm_jit_intrin.das @@ -223,7 +223,7 @@ let g_x64_avx512vnni_intrin_lookup <- { // zmm VPDPBUSD (the bw gate is a prer "x64_avx::dot64_acc16_vnni" => @@intrinsic_dot64_acc16_vnni_x64 } -// Half<->float converts (the llvm/daslib/f16_cvt module) — one TARGET-INDEPENDENT table, consulted +// Half<->float converts (the daslib/f16_cvt module) — one TARGET-INDEPENDENT table, consulted // when the target lowers fpext/fptrunc through half in hardware: any aarch64 (scalar fcvt is // ARMv8.0 baseline, no finer gate) or x86-64 with F16C (g_target_x64_f16c). Elsewhere the calls // compile their exact bit-twiddle daslang bodies — a half convert without hardware support would diff --git a/modules/dasSpirv/spirv/spirv_builder.das b/modules/dasSpirv/spirv/spirv_builder.das index 9153a862fe..c3d83d7b22 100644 --- a/modules/dasSpirv/spirv/spirv_builder.das +++ b/modules/dasSpirv/spirv/spirv_builder.das @@ -714,7 +714,8 @@ def public const_double(var m : SpirvModule; v : double) : uint { // value), so this is type + result-id only. Keyed separately so it never collides with const_int 0/1. // f32 -> f16 bits, round-to-nearest-even — the das CPU reference semantics exactly (its // float16 converts are RNE too, so a folded CPU constant and this routine agree bit-for-bit). -// Pure das because the f16_cvt daslib lives in dasLLVM (not requireable from here). +// Its own body rather than daslib/f16_cvt's f32_to_f16: that one flattens every nan to sign|0x7E00, +// this one carries a payload bit through. def public float_to_half_bits(v : float) : uint { let bits = unsafe(reinterpret(v)) let s = (bits >> 16u) & 0x8000 diff --git a/tests/aot/CMakeLists.txt b/tests/aot/CMakeLists.txt index 9a7b21dd46..9c69c6c8f3 100644 --- a/tests/aot/CMakeLists.txt +++ b/tests/aot/CMakeLists.txt @@ -273,9 +273,9 @@ FILE(GLOB AOT_METAL_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "test list(FILTER AOT_METAL_FILES EXCLUDE REGEX "/_") SET(AOT_METAL_MODULE_FILES tests/metal/_metal_common.das - # f16_cvt is a generic dasLLVM daslib helper the tensor-ops test requires; it used to ride + # f16_cvt is a generic daslib helper the tensor-ops test requires; it used to ride # the (deleted) dasLLAMA module lib — dasLLAMA itself is NEVER AOT'd (see above) - modules/dasLLVM/daslib/f16_cvt.das + daslib/f16_cvt.das ) IF(APPLE) # das_metal_boost hard-requires the APPLE-gated das_metal C++ module; the GPU-behavioral diff --git a/tests/jit_tests/f16_cvt.das b/tests/jit_tests/f16_cvt.das index 4f6cb589b4..cd022e6ec7 100644 --- a/tests/jit_tests/f16_cvt.das +++ b/tests/jit_tests/f16_cvt.das @@ -1,7 +1,7 @@ options gen2 require dastest/testing_boost -require llvm/daslib/f16_cvt +require daslib/f16_cvt // f16_cvt intrinsic emission: under -jit on aarch64 / x64+F16C the module's f16_to_f32 / // f32_to_f16 lower to hardware half converts (fpext/fptrunc through half), while the ref_* diff --git a/tests/metal/test_metal_tensor_ops.das b/tests/metal/test_metal_tensor_ops.das index 22849ca33b..b280e793b8 100644 --- a/tests/metal/test_metal_tensor_ops.das +++ b/tests/metal/test_metal_tensor_ops.das @@ -11,7 +11,7 @@ require dastest/testing_boost public require strings require metal/msl_shader require daslib/math_bits // float_bits_to_uint — bf16 halfword construction -require llvm/daslib/f16_cvt // f32_to_f16 — the f16s scale plane +require daslib/f16_cvt // f32_to_f16 — the f16s scale plane require _metal_common // nolint:STYLE030,LINT019 — used only inside the Apple static_if half; off-Apple lint sees the branch compiled out let M = 8 From 72f5201e01c064b64096f95c6e6976c124d7054f Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 17:02:56 -0700 Subject: [PATCH 06/43] stage 1, wave 2: the six tune annotations become daslib shells (daslib/tune.das) that forward to the framework when dasLLVM is mounted and do nothing otherwise - the framework's classes turn into plain functions, the auto-policy infer pass moves with them because macro state is per module; [tuned] and dasllama_tune_scope take the path guard and fall back to the declared or default perm with no sidecar; the exchange is a guarded require in the server and the bench (404 without the framework); builtin_module_exists also sees a shared das module the compiling program required, so lint and the other tool-driven compiles audit the same arm the run takes Co-Authored-By: Claude Fable 5.1 --- daslib/tune.das | 106 ++++ modules/dasLLAMA/ARCHITECTURE_ENGINE.md | 14 +- modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md | 4 + modules/dasLLAMA/benchmarks/lcpp_bench.das | 29 +- modules/dasLLAMA/dasllama/dasllama_common.das | 13 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 5 +- modules/dasLLAMA/dasllama/dasllama_tune.das | 84 ++- .../dasLLAMA/dasllama/dasllama_tune_scope.das | 3 +- modules/dasLLAMA/harness/gen_tune_probe.das | 1 + .../dasLLAMA/harness/tune_confirm_prefill.das | 3 +- modules/dasLLAMA/harness/tune_kernels.das | 1 + .../performance/establish_baselines.das | 3 +- modules/dasLLAMA/performance/fetch_models.das | 3 +- .../performance/gen_bench_records.das | 3 +- modules/dasLLAMA/performance/gen_profile.das | 3 +- .../dasLLAMA/performance/gen_site_records.das | 3 +- .../dasLLAMA/performance/setup_asr_rig.das | 1 + modules/dasLLAMA/portable_tier_plan.md | 21 + modules/dasLLAMA/tests/test_box_profile.das | 6 +- modules/dasLLVM/daslib/llvm_code.das | 29 +- modules/dasLLVM/daslib/llvm_tune.das | 591 +++++++++--------- modules/dasLLVM/tests/cant_tune_bad_grids.das | 1 + modules/dasLLVM/tests/cant_tune_bad_scope.das | 1 + .../tests/cant_tune_scope_after_covered.das | 1 + modules/dasLLVM/tests/llvm_tune.das | 1 + .../tests/llvm_tune_manifest_client.das | 1 + .../dasLLVM/tests/llvm_tune_modes_client.das | 1 + .../dasLLVM/tests/llvm_tune_profiles_lib.das | 1 + .../dasLLVM/tests/llvm_tune_scope_client.das | 1 + .../tests/llvm_tune_scope_client_lib.das | 1 + .../tests/llvm_tune_scope_client_lib2.das | 1 + .../tests/llvm_tune_scope_client_resolver.das | 1 + .../tests/llvm_tune_scope_client_ver.das | 1 + .../tests/llvm_tune_scope_client_verlib.das | 1 + .../tests/llvm_tune_scope_client_verlib2.das | 1 + .../dasLLVM/tests/llvm_tune_scope_early.das | 1 + skills/internal/llvm_tune_internals.md | 22 +- skills/tune.md | 31 +- src/ast/ast_infer_type.cpp | 1 + tests/daslib/test_tune_shells.das | 43 ++ tests/jit_tests/exe_host_cpu.das | 2 +- tests/jit_tests/llvm_code.das | 2 +- utils/dasllama-server/README.md | 2 + utils/dasllama-server/main.das | 10 +- utils/dasllama-server/openai_server.das | 165 +++-- utils/daspkg/test_daspkg.das | 2 +- 46 files changed, 758 insertions(+), 463 deletions(-) create mode 100644 daslib/tune.das create mode 100644 tests/daslib/test_tune_shells.das diff --git a/daslib/tune.das b/daslib/tune.das new file mode 100644 index 0000000000..4274f6889d --- /dev/null +++ b/daslib/tune.das @@ -0,0 +1,106 @@ +options gen2 +options indenting = 4 +options no_aot = true + +module tune shared public + +require daslib/ast +require daslib/ast_boost + +require ?llvm/daslib/llvm_code llvm/daslib/llvm_code public +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public + +//! The six kernel-tuning annotations - `[tune_perm]`, `[tune_companion]`, `[tune]`, +//! `[tune_scope]`, `[tune_policy]` and `[llvm_code]` - plus the default-policy infer pass, as +//! shells every program can spell. The framework that implements them lives in +//! `llvm/daslib/llvm_tune` and `llvm/daslib/llvm_code`, reached through the path guards above +//! and re-exported, so `require daslib/tune` gives exactly what `require llvm/daslib/llvm_tune` +//! gave; the guarded require is also the witness that makes `builtin_module_exists` below +//! answer for this compile. Every tune annotation registers HERE and nowhere else, because a +//! module's macro state is per-module: the `[tune_scope]` bank and the pass that reads it have +//! to run in one macro context. Without dasLLVM every apply is a no-op - no sidecar read, no +//! stamp, no `_variants()` registry, no tuner spawn - and the reference body IS the function. +//! Framework semantics, arguments and the sidecar contract: `skills/tune.md`. + +[function_macro(name = "tune_perm")] +class private TunePermShell : AstFunctionAnnotation { + def override apply(var func : FunctionPtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_perm_apply(func, group, args, errors) + } else { + return true + } + } +} + +[function_macro(name = "tune_companion")] +class private TuneCompanionShell : AstFunctionAnnotation { + def override apply(var func : FunctionPtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_companion_apply(func, group, args, errors) + } else { + return true + } + } +} + +[function_macro(name = "tune")] +class private TuneShell : AstFunctionAnnotation { + def override apply(var func : FunctionPtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_apply(func, group, args, errors) + } else { + return true + } + } +} + +[structure_macro(name = "tune_scope")] +class private TuneScopeShell : AstStructureAnnotation { + def override apply(var st : StructurePtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_scope_apply(st, group, args, errors) + } else { + return true + } + } +} + +[function_macro(name = "tune_policy")] +class private TunePolicyShell : AstFunctionAnnotation { + def override apply(var func : FunctionPtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_policy_apply(func, group, args, errors) + } else { + return true + } + } +} + +[function_macro(name = "llvm_code")] +class private LlvmCodeShell : AstFunctionAnnotation { + def override apply(var func : FunctionPtr; var group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + static_if (typeinfo builtin_module_exists(llvm_code)) { + return llvm_code_apply(func, group, args, errors) + } else { + return true + } + } +} + +[infer_macro] +class private TuneDefaultPolicyShell : AstPassMacro { + def override apply(prog : ProgramPtr; mod : Module?) : bool { + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_default_policy_apply(prog, mod) + } else { + return false + } + } +} diff --git a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md index 32f123475e..1236923956 100644 --- a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md +++ b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md @@ -187,7 +187,12 @@ trunk-only and trunk+head images never collide and one image file serves both tr generator, the layout/perm schema shared by generator and runtime, and the `[tune]` family registration. A hand-written tile that the generator could emit belongs in the generator. - **`dasllama_tune.das`** - the per-box loop-hint tuner (`[tuned]` / `[dasllama_grid]`). Tuning - POLICY lives here; tuned VALUES live in the box's sidecar, never in source. + POLICY lives here; tuned VALUES live in the box's sidecar, never in source. The framework is + OPTIONAL: the `llvm_tune` require is path-guarded and every use of it sits behind + `static_if (typeinfo builtin_module_exists(llvm_tune))`, so a build without dasLLVM opens no + sidecar, reads no policy env and reports nothing - `[tuned]` stamps the declared `fallback=` + (or `DEFAULT_PERM`) and the AST rewrite is unchanged. `box_profile_verdict_at` answers the + reason NAME rather than llvm_tune's enum for the same reason: a signature cannot be gated. - **`dasllama_tune_scope.das`** - the one `[tune_scope]` declaration (tuner, covered modules, version pin, shipped defaults), in a module every kernel module requires: a kernel reads its class entry in the defaults profile at its own compile, so the scope must be on the AST first. @@ -247,7 +252,12 @@ file builds an `ArchDesc` (name * `configure` * the `ArchBlocks` fn-ptr quad * ` explicit `exchange_*` config counts as the expressed choice, otherwise the `.consent` sidecar-sibling file governs - unset asks on a real terminal, or emits `@sidecar consent state=needed` for the watchdog dialog / control page, and no request - leaves until a surface records "accepted". + leaves until a surface records "accepted". The client is meaningless without a sidecar, so + every requirer takes it through the path guard `require ?llvm/daslib/llvm_tune + dasllama/dasllama_exchange` and wraps each use in + `static_if (typeinfo builtin_module_exists(dasllama_exchange))`: with no tune framework in the + build the server's `/exchange` routes answer `not_found`, the boot resolver and submit check + are never registered, and `lcpp_bench --submit` refuses before it measures. - **`benchmarks/asr/mem_census.sh`** - the peak-memory census (`/usr/bin/time -l` around one asr_bench process per cell; macOS only) - the interim footprint instrument until a footprint leg lands in `gen_bench_records`; its numbers live in `PERF_LEDGER.md`, never the stores. diff --git a/modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md b/modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md index d13ea20828..2af245da35 100644 --- a/modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md +++ b/modules/dasLLAMA/ARCHITECTURE_MEASUREMENT.md @@ -242,6 +242,10 @@ profile (`performance/defaults`), the annotation's `fallback` `;`-chain, then `D FILE its answer came from, so a verbose compile names which of the two stamped each kernel. A box the shipped profile covers therefore compiles tuned kernels without racing anything, and a box it does not cover falls to the fallback chain - never to another box's winners. +With no tune framework in the build the first three steps do not exist - no policy env, no sidecar, +no profile - so the ladder is the `perm=` pin, then the `fallback` chain's first UNCONDITIONAL entry +(a `suffix:requires` seat cannot be judged with no feature probe), then `DEFAULT_PERM`, and the +compile reports nothing. ### 2.41 The mint's own wall rides in the sidecar's provenance {#mint-wall-provenance} diff --git a/modules/dasLLAMA/benchmarks/lcpp_bench.das b/modules/dasLLAMA/benchmarks/lcpp_bench.das index e7c248b933..25baf85d57 100644 --- a/modules/dasLLAMA/benchmarks/lcpp_bench.das +++ b/modules/dasLLAMA/benchmarks/lcpp_bench.das @@ -31,7 +31,7 @@ require daslib/module_path // get_this_module_dir — the default pro require daslib/jobque_profile // JOBQUE_PROFILING_ENABLED — --prof refuses on a build without the marker rail require ?das_accelerate dasllama/dasllama_math_accelerate // the --accel leg; Apple-only C++ module require ../performance/profile_common.das // profile_threads + affinity_on + bench records — the standing per-box methodology -require dasllama/dasllama_exchange // --submit: POST the stamped records to the exchange +require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: --submit POSTs the stamped records to the exchange, which rides the tune framework require llvm/daslib/llvm_tune // tune_app_script — the exe-vs-script discriminator behind --for-debug-purposes require daslib/jobque_boost require daslib/clargs @@ -1270,6 +1270,15 @@ def private tests_of(rows : array) : table { // flavor is stamped debug-jit so no board or oracle can mistake it for measurement var private g_debug_run = false +// the exchange client compiles only where the tune framework does, so --submit has a rail to refuse on +def private exchange_compiled_in() : bool { + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + return true + } else { + return false + } +} + def private flavor_stamp(base : string) : string { let b = tune_gate_bypassed() ? "untuned:{base}" : base return g_debug_run ? "debug-jit:{b}" : b @@ -1351,6 +1360,10 @@ def main() : int { print("error: --submit refused on a debug -jit run - the numbers are debug-grade; submit from the released exe\n") return 1 } + if (cfg.submit && !exchange_compiled_in()) { + print("error: --submit refused - this build carries no sidecar exchange client (no tune framework)\n") + return 1 + } // the standing per-box methodology, enforced here rather than left to the environment // (gen_profile's pattern): cap the future que's workers (computing main is the last lane) and // hard-pin lanes to distinct physical cores on the boxes that pin. -t is the explicit ask, but @@ -1844,13 +1857,15 @@ def main() : int { } print("{text}\n") if (cfg.submit) { - let url = empty(g_env_exchange.exchange_url) ? EXCHANGE_URL_DEFAULT : g_env_exchange.exchange_url - let serr = exchange_submit_records_doc(url, text) - if (!empty(serr)) { - say(fmt, "records NOT submitted to {url}: {serr}\n") - return 1 + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + let url = empty(g_env_exchange.exchange_url) ? EXCHANGE_URL_DEFAULT : g_env_exchange.exchange_url + let serr = exchange_submit_records_doc(url, text) + if (!empty(serr)) { + say(fmt, "records NOT submitted to {url}: {serr}\n") + return 1 + } + say(fmt, "records submitted to {url}\n") } - say(fmt, "records submitted to {url}\n") } return 0 } diff --git a/modules/dasLLAMA/dasllama/dasllama_common.das b/modules/dasLLAMA/dasllama/dasllama_common.das index ed6dee16f6..ff2c908bdb 100644 --- a/modules/dasLLAMA/dasllama/dasllama_common.das +++ b/modules/dasLLAMA/dasllama/dasllama_common.das @@ -20,7 +20,8 @@ require dasllama/dasllama_convert // cvt_f32_to_f16/cvt_f16_to_f32 — the f16 require dasllama/dasllama_math_default // registers the portable Q8·Q8 backend at [init] (the fallback everywhere) require ?llvm/daslib/aarch64_neon dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init] under the JIT alone; PATH guard: without dasLLVM its intrinsics are scalar fallbacks slower than portable, so the tier stays out require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen // [init]-only: registers "arm64-gen" (the generated GEMM family, arm64 load-select tier). PATH guard: the module hard-requires llvm_tune, so it loads exactly when dasLLVM is mounted -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): the auto-policy pass and tune_status reach every app that requires this module +require daslib/tune public // the tune annotations + the auto-policy pass, inert without dasLLVM; PUBLIC for the same reach as the line below +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): tune_status reaches every app that requires this module require ?das_metal dasllama/dasllama_metal_gemm // [init]-only: registers "metal" (the GPU prefill-GEMM batch donor). STRICT guard: the C++ das_metal module exists on Apple builds only, so non-Apple lanes never compile the Metal driver require ?das_accelerate dasllama/dasllama_math_accelerate // [init]-only: arms the "+AMX" float-batch override when DASLLAMA_ACCEL=1. Same Apple-only guard shape as the metal driver require dasllama/dasllama_gguf @@ -1476,16 +1477,16 @@ def apply_box_profile_runtime_checked(path : string) { g_box_profile_runtime_applied = false // both flags describe THIS apply, never an earlier one g_box_profile_runtime_declined = "" let reason = box_profile_verdict_at(path) - if (reason == TuneSidecarReason.absent) { - g_box_profile_runtime_declined = "{reason}" + if (reason == "absent") { + g_box_profile_runtime_declined = reason return } - if (reason == TuneSidecarReason.foreign_box || reason == TuneSidecarReason.unreadable) { - g_box_profile_runtime_declined = "{reason}" + if (reason == "foreign_box" || reason == "unreadable") { + g_box_profile_runtime_declined = reason to_log(LOG_INFO, "dasLLAMA: tune sidecar {path} not applied ({reason}) - its runtime knobs are the minting box's state\n") return } - if (reason == TuneSidecarReason.stale_binary) { + if (reason == "stale_binary") { to_log(LOG_INFO, "dasLLAMA: tune sidecar predates the binary - kernel winners fall back; the runtime knobs are box state and apply\n") } apply_box_profile_runtime_at(path) diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 9de55526da..e2d7eed233 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -11,8 +11,9 @@ require dasllama/dasllama_math_aarch64_neon require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails of the s16 twins require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs -require llvm/daslib/llvm_tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code]; PUBLIC so the auto-policy infer pass and tune_status reach app roots that require this module -require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms +require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module +require llvm/daslib/llvm_tune public // PUBLIC so the auto-policy infer pass and tune_status reach app roots that require this module +require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public require dasllama/dasllama_par diff --git a/modules/dasLLAMA/dasllama/dasllama_tune.das b/modules/dasLLAMA/dasllama/dasllama_tune.das index 80825c2d8e..9c50e6d419 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tune.das +++ b/modules/dasLLAMA/dasllama/dasllama_tune.das @@ -11,7 +11,8 @@ require daslib/rtti require daslib/ast_boost require daslib/templates_boost require daslib/json_boost // read_json / read_json_field (re-exports daslib/json) -require llvm/daslib/llvm_tune // the per-app tune sidecar (path, staleness, kernels read) +require daslib/strings_boost // nolint:STYLE030,LINT019 - split/find, reached only by the no-framework fallback-chain scan +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune // OPTIONAL: the per-app tune sidecar (path, staleness, kernels read) // ===== Tune spec ===== @@ -277,24 +278,36 @@ def private tuned_template_name(func : FunctionPtr; args : AnnotationArgumentLis return "{func.name}_template" // convention: dot -> dot_template } -//! The ONE per-app tune sidecar (`.tune.json` beside the root script/binary; -//! `DAS_TUNE_MANIFEST` env overrides) — shared by `[tuned]` compile-time reads, runtime apply, -//! `[tune]` generator winners, and the tuner harness write. Loop hints live under `"kernels"`, knobs under `"runtime"`. +//! The ONE per-app tune sidecar (`.tune.json` beside the root script/binary; `DAS_TUNE_MANIFEST` +//! env overrides) — shared by `[tuned]` compile-time reads, runtime apply, `[tune]` generator winners, +//! and the tuner harness write. Hints under `"kernels"`, knobs under `"runtime"`; "" with no framework. def box_profile_path() : string { - return tune_manifest_path() + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_manifest_path() + } else { + return "" + } } //! The sidecar is older than the running binary — its measurements are dead (the runtime //! apply skips it; the compile-time reads already treat it as absent). def box_profile_stale() : bool { - return tune_sidecar_stale(tune_manifest_path()) + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return tune_sidecar_stale(tune_manifest_path()) + } else { + return false + } } -//! Why the sidecar at `path` would not serve this box: `stale_binary` (a rebuild killed the -//! kernel winners; the runtime knobs are still this box's state), `foreign_box` / `unreadable` -//! (another box's state, or not a sidecar), `absent`; anything else reads as fresh and ours. -def box_profile_verdict_at(path : string) : TuneSidecarReason { - return tune_sidecar_verdict(path, "runtime").reason +//! Why the sidecar at `path` would not serve this box, as the `TuneSidecarReason` NAME - `stale_binary` +//! (a rebuild killed the kernel winners; the runtime knobs are still this box's state), `foreign_box` / +//! `unreadable`, `absent`; anything else reads as fresh and ours, and with no framework it is `absent`. +def box_profile_verdict_at(path : string) : string { // nolint:LINT012,LINT019 - `path` is read only in the framework arm + static_if (typeinfo builtin_module_exists(llvm_tune)) { + return "{tune_sidecar_verdict(path, "runtime").reason}" + } else { + return "absent" + } } @@ -306,15 +319,24 @@ def private fallback_perm(args : AnnotationArgumentList) : string { let dv = find_arg(args, "fallback") if (dv is tString) { // ';'-chain of `suffix` or `suffix:requires` — one annotation, a per-ISA default - let picked = tune_pick_fallback(dv as tString) - if (!empty(picked)) return picked + static_if (typeinfo builtin_module_exists(llvm_tune)) { + let picked = tune_pick_fallback(dv as tString) + if (!empty(picked)) return picked + } else { + // no CPU-feature probe here, so take the first unconditional entry - the chain's ISA-neutral tail + for (entry in split(dv as tString, ";")) { + if (!empty(entry) && find(entry, ":") < 0) return entry + } + } } return DEFAULT_PERM } -def private report_perm(fname, perm, source : string) : string { - if (tune_verbosity() == "verbose") { - print("dasllama_tune: {fname} <- {perm} ({source})\n") +def private report_perm(fname, perm, source : string) : string { // nolint:LINT012,LINT019 - `fname`/`source` are read only in the framework arm + static_if (typeinfo builtin_module_exists(llvm_tune)) { + if (tune_verbosity() == "verbose") { + print("dasllama_tune: {fname} <- {perm} ({source})\n") + } } return perm } @@ -325,19 +347,25 @@ def private resolve_perm(func : FunctionPtr; args : AnnotationArgumentList) : st let fname = "{func.name}" let fb = fallback_perm(args) g_fallbacks[fname] = fb - if (tune_policy_env() == "reference") { - // the true kill switch, [tune]'s twin: the template's own loops, no hints stamped - return report_perm(fname, "plain", "reference policy") - } - let av = find_arg(args, "perm") - if (av is tString) return av as tString - var pickSource = "" - let cfg = tune_kernel_pick(fname, pickSource) - if (!empty(cfg)) { - return report_perm(fname, cfg, pickSource) + static_if (typeinfo builtin_module_exists(llvm_tune)) { + if (tune_policy_env() == "reference") { + // the true kill switch, [tune]'s twin: the template's own loops, no hints stamped + return report_perm(fname, "plain", "reference policy") + } + let av = find_arg(args, "perm") + if (av is tString) return av as tString + var pickSource = "" + let cfg = tune_kernel_pick(fname, pickSource) + if (!empty(cfg)) { + return report_perm(fname, cfg, pickSource) + } + let declared = find_arg(args, "fallback") is tString + return report_perm(fname, fb, declared ? "fallback" : "default") + } else { + // no policy env and no sidecar here: the `perm=` pin still names a row, else the chain's answer + let av = find_arg(args, "perm") + return av is tString ? av as tString : fb } - let declared = find_arg(args, "fallback") is tString - return report_perm(fname, fb, declared ? "fallback" : "default") } // Template lives in the compiling module for production `[tuned]`; the profiler harness diff --git a/modules/dasLLAMA/dasllama/dasllama_tune_scope.das b/modules/dasLLAMA/dasllama/dasllama_tune_scope.das index e1ba93711d..e311d3f1f8 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tune_scope.das +++ b/modules/dasLLAMA/dasllama/dasllama_tune_scope.das @@ -5,7 +5,8 @@ options _dasllama_internal = true module dasllama_tune_scope shared public require dasllama/dasllama_lint public -require llvm/daslib/llvm_tune public // the scope annotation; PUBLIC so the auto-policy infer pass and tune_status reach app roots +require daslib/tune public // the [tune_scope] annotation, inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // OPTIONAL; PUBLIC so the auto-policy infer pass and tune_status reach app roots require dasllama/dasllama_version // nolint:STYLE030 — version_of= below names DASLLAMA_RELEASE by string; the module must be in this compile //! The scope reaches the AST before any covered module compiles - the scope apply refuses a covered diff --git a/modules/dasLLAMA/harness/gen_tune_probe.das b/modules/dasLLAMA/harness/gen_tune_probe.das index 5ac110b09d..727180fc4c 100644 --- a/modules/dasLLAMA/harness/gen_tune_probe.das +++ b/modules/dasLLAMA/harness/gen_tune_probe.das @@ -43,6 +43,7 @@ require dasllama/dasllama_convert require dasllama/dasllama_gguf // transcode_q*k_superblock — the kq fixture pack rail require daslib/f16_cvt // f32_to_f16 — the kq fixtures pack disk superblock headers require llvm/daslib/llvm_tune +require daslib/tune // [tune_policy] - inert without dasLLVM require daslib/random require daslib/json // JsonValue — the null runtime arg of the deferred sidecar merge require daslib/json_boost // JV over tuples — the per-variant race rows diff --git a/modules/dasLLAMA/harness/tune_confirm_prefill.das b/modules/dasLLAMA/harness/tune_confirm_prefill.das index 0e15b0ae36..9fc5124887 100644 --- a/modules/dasLLAMA/harness/tune_confirm_prefill.das +++ b/modules/dasLLAMA/harness/tune_confirm_prefill.das @@ -6,7 +6,8 @@ options _dasllama_internal = true require dasllama/dasllama_transformer // umbrella fires each arch [init] registration; requiring dasllama_common directly would drop them require dasllama/dasllama_math // setup_dasllama_jobque_ + apply_box_profile_runtime -require llvm/daslib/llvm_tune // [tune_policy]: the confirm arm measures WHAT THE MANIFEST SAYS — never self-tune +require daslib/tune // [tune_policy]: the confirm arm measures WHAT THE MANIFEST SAYS — never self-tune +require llvm/daslib/llvm_tune // the framework the annotation forwards to, and the visibility the auto-policy pass needs require daslib/jobque_boost require strings require math diff --git a/modules/dasLLAMA/harness/tune_kernels.das b/modules/dasLLAMA/harness/tune_kernels.das index 21657f7383..78e153a2cb 100644 --- a/modules/dasLLAMA/harness/tune_kernels.das +++ b/modules/dasLLAMA/harness/tune_kernels.das @@ -23,6 +23,7 @@ require dasllama/dasllama_common // the runtime-knob getters (the profile's "run require dasllama/dasllama_mtp_gemma // gemma_drafter_sidecar - the assistant-depth confirm's vehicle test require dasllama/dasllama_tune // [dasllama_grid] emitter require llvm/daslib/llvm_tune // tune_sidecar_merge — the app-sidecar upsert writer +require daslib/tune // [tune_policy] - inert without dasLLVM require ?das_metal dasllama/dasllama_metal_prefill // metal_tensor_race (Apple static_if half) require ?das_metal dasllama/dasllama_metal_kernels // metal_tensor_race_decode (Apple static_if half) require tuner_cli // the shared tuner CLI diff --git a/modules/dasLLAMA/performance/establish_baselines.das b/modules/dasLLAMA/performance/establish_baselines.das index b479ca08ac..cd89cad411 100644 --- a/modules/dasLLAMA/performance/establish_baselines.das +++ b/modules/dasLLAMA/performance/establish_baselines.das @@ -14,7 +14,8 @@ options _dasllama_internal = true require profile_common require daslib/clargs require strings -require llvm/daslib/llvm_tune // [tune_policy]: this tool benches the reference only — never auto-tune +require daslib/tune // [tune_policy]: this tool benches the reference only — never auto-tune +require llvm/daslib/llvm_tune // the framework the annotation forwards to, and the visibility the auto-policy pass needs require dasllama/dasllama_env [CommandLineArgs] diff --git a/modules/dasLLAMA/performance/fetch_models.das b/modules/dasLLAMA/performance/fetch_models.das index 81c36cd2fd..257090c3a5 100644 --- a/modules/dasLLAMA/performance/fetch_models.das +++ b/modules/dasLLAMA/performance/fetch_models.das @@ -17,7 +17,8 @@ require daslib/clargs require daslib/fio require strings require profile_common -require llvm/daslib/llvm_tune // [tune_policy]: provisioning only — never auto-tune +require daslib/tune // [tune_policy]: provisioning only — never auto-tune +require llvm/daslib/llvm_tune // the framework the annotation forwards to, and the visibility the auto-policy pass needs require dasllama/dasllama_env [CommandLineArgs] diff --git a/modules/dasLLAMA/performance/gen_bench_records.das b/modules/dasLLAMA/performance/gen_bench_records.das index c366381688..65cf66119a 100644 --- a/modules/dasLLAMA/performance/gen_bench_records.das +++ b/modules/dasLLAMA/performance/gen_bench_records.das @@ -7,7 +7,8 @@ options _dasllama_internal = true require profile_common require dasllama/dasllama_env // the g_env_* knob globals ([EnvConfig] declarations; ENVIRONMENT.md is generated from them) require dasllama/dasllama_image // dlim_wipe + dlim_inventory — the batch lifecycle and the re-mint tripwire -require llvm/daslib/llvm_tune // [tune_policy]: the orchestrator never measures — it must never self-tune +require daslib/tune // [tune_policy]: the orchestrator never measures — it must never self-tune +require llvm/daslib/llvm_tune // the framework the annotation forwards to, and the visibility the auto-policy pass needs require daslib/clargs require daslib/fio require math diff --git a/modules/dasLLAMA/performance/gen_profile.das b/modules/dasLLAMA/performance/gen_profile.das index 5ac8e7ee6d..71c1b49a16 100644 --- a/modules/dasLLAMA/performance/gen_profile.das +++ b/modules/dasLLAMA/performance/gen_profile.das @@ -14,7 +14,8 @@ options _dasllama_internal = true require dasllama/dasllama_transformer // the umbrella fires each arch [init] registration; requiring dasllama_common directly would drop them require dasllama/dasllama_math // setup_dasllama_jobque_ + apply_box_profile_runtime knobs (engine-level, not re-exported) require dasllama/dasllama_tune // box_profile_path (the applied per-box manifest, recorded in the platform block) -require llvm/daslib/llvm_tune // [tune_policy] + log_tune_status — the box tunes itself before measuring +require daslib/tune // [tune_policy] - inert without dasLLVM +require llvm/daslib/llvm_tune // log_tune_status — the box tunes itself before measuring require daslib/jobque_boost require profile_common require daslib/clargs diff --git a/modules/dasLLAMA/performance/gen_site_records.das b/modules/dasLLAMA/performance/gen_site_records.das index 64f70bdf58..5cd922f697 100644 --- a/modules/dasLLAMA/performance/gen_site_records.das +++ b/modules/dasLLAMA/performance/gen_site_records.das @@ -5,7 +5,8 @@ options _dasllama_internal = true require daslib/clargs require daslib/fio require profile_common -require llvm/daslib/llvm_tune // [tune_policy]: provisioning only — never auto-tune +require daslib/tune // [tune_policy]: provisioning only — never auto-tune +require llvm/daslib/llvm_tune // the framework the annotation forwards to, and the visibility the auto-policy pass needs // Merge every per-box record store (performance/records/.json) into the one file the site // renders: site/files/dasllama/bench_records.json. Boxes never collide on the run key (box is part diff --git a/modules/dasLLAMA/performance/setup_asr_rig.das b/modules/dasLLAMA/performance/setup_asr_rig.das index 6c1d837c90..c945198059 100644 --- a/modules/dasLLAMA/performance/setup_asr_rig.das +++ b/modules/dasLLAMA/performance/setup_asr_rig.das @@ -4,6 +4,7 @@ options _dasllama_internal = true require profile_common require dasllama/dasllama_env +require daslib/tune // [tune_policy] - inert without dasLLVM require daslib/clargs require daslib/fio require strings diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 1cfe01ba45..b60074655b 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -295,6 +295,27 @@ the honest wasm candidates, and smaller is on the table. ## Open questions +- **What "dasLLVM absent" means in a SOURCE tree.** The `?llvm/daslib/...` path guard asks whether + the guard's file resolves, and the `.das_module` folder scan (`src/ast/dyn_modules.cpp`) + resolves `modules/dasLLVM/` from the filesystem whatever `DAS_LLVM_DISABLED` says. So in a + source tree configured without LLVM the guard is TRUE, the generated tier loads, and its + `[extern(library="LLVM.dll")]` bindings fail at compile time; only an installed SDK without + dasLLVM (its install rules sit inside the CMake guard) has the guard false. The stage-1 gate + therefore runs in a worktree with `modules/dasLLVM` removed, the installed-SDK shape. If the + in-tree case should also work, the cheapest honest witness is a C++ module registered only + when LLVM is configured (the `?sqlite` pattern), and the guards become `? ...`. +- **`builtin_module_exists` on a shared das module flips under tool-driven compiles.** A file + with `require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune` and + `static_if (typeinfo builtin_module_exists(llvm_tune))` takes the framework arm under + `bin/daslang file.das` and the no-framework arm under `bin/daslang utils/lint/main.das -- file.das`: + `Module::requireEx` (`src/ast/ast_infer_type.cpp:2847`, `src/ast/ast_module.cpp:284`) scans the + bound process's promoted-module list, which a nested compile does not populate. C++-module + guards are immune. Consequence today: lint, and presumably ast-verify and the MCP checks, + audit the no-framework half of every guarded file and never the framework half (suppressed + per site with the house `nolint:...,LINT019` spelling). Two fixes, both C++: the witness module + above (then `builtin_module_exists()` is a linked-module question), or making the + trait answer "is this module in the compiling program" for shared das modules. + - Which small-LLM carrier gets the parity fixture for stages 1 and 2 (SmolLM2-135M already has a cls_q8 parity cell, `test_parity.das:76`). - Whether `test_groupn.das`'s independent reference is fp64 dequant (the kq ladder's choice) or diff --git a/modules/dasLLAMA/tests/test_box_profile.das b/modules/dasLLAMA/tests/test_box_profile.das index 9b79af0d3f..4f8a1ce95d 100644 --- a/modules/dasLLAMA/tests/test_box_profile.das +++ b/modules/dasLLAMA/tests/test_box_profile.das @@ -7,7 +7,7 @@ require dasllama/dasllama_math // the knob setters/getters (incl. set/get_q8_to require daslib/fio require dasllama/dasllama_env require dasllama/dasllama_tune // box_profile_verdict_at -require llvm/daslib/llvm_tune // tune_box_identity, TuneSidecarReason +require llvm/daslib/llvm_tune // tune_box_identity // The tune sidecar's "runtime" section: apply_box_profile_runtime(path) sets the runtime knobs // and logs each applied entry. The explicit-path form keeps the test independent of the app's @@ -260,7 +260,7 @@ def test_stale_binary_sidecar_keeps_runtime(t : T?) { t |> success(fwrite(path, js), "profile written") var err : string t |> success(set_mtime(path, mktime(2020, 1, 1, 0, 0, 0), err), "mtime set into the past ({err})") - t |> equal(box_profile_verdict_at(path), TuneSidecarReason.stale_binary, "reads as binary-stale") + t |> equal(box_profile_verdict_at(path), "stale_binary", "reads as binary-stale") apply_box_profile_runtime_checked(path) t |> equal(get_q8_token_block(), 320l, "runtime knob applied off a binary-stale sidecar") t |> success(box_profile_runtime_applied(), "the applied flag is up") @@ -276,7 +276,7 @@ def test_foreign_sidecar_applies_nothing(t : T?) { let path = path_join(dir, "dasllama_test_box_profile_foreign.json") let js = "\{\"kernels\":\{\},\"runtime\":\{\"q8_token_block\":320\},\"provenance\":\{\"box\":\"dasllama-test-foreign-box\"\}\}" t |> success(fwrite(path, js), "profile written") - t |> equal(box_profile_verdict_at(path), TuneSidecarReason.foreign_box, "reads as another box's") + t |> equal(box_profile_verdict_at(path), "foreign_box", "reads as another box's") apply_box_profile_runtime_checked(path) t |> equal(get_q8_token_block(), 128l, "runtime knob NOT applied off another box's sidecar") t |> equal(box_profile_runtime_declined(), "foreign_box", "the decline names its reason") diff --git a/modules/dasLLVM/daslib/llvm_code.das b/modules/dasLLVM/daslib/llvm_code.das index 355b95df37..14387ff4cc 100644 --- a/modules/dasLLVM/daslib/llvm_code.das +++ b/modules/dasLLVM/daslib/llvm_code.das @@ -14,22 +14,21 @@ require daslib/ast_boost //! IR instead of the body. The body stays the reference implementation — the interpreter, AOT, //! and any target the generator declines compile it verbatim. Extra annotation arguments are //! the generator's parameters (and fold into the JIT DLL cache key). -[function_macro(name = "llvm_code")] -class private LlvmCodeAnnotation : AstFunctionAnnotation { - def override apply(var func : FunctionPtr; var group : ModuleGroup; - args : AnnotationArgumentList; var errors : das_string) : bool { - for (arg in args) { - if (arg.name == "name") { - if (arg.basicType != Type.tString || empty(arg.sValue)) { - errors := "[llvm_code] on {func.name}: `name` must be a non-empty string (the generator key)" - return false - } - // force side effects, or constant-argument calls const-fold in the front-end and never reach the generator - func.sideEffectFlags.userScenario = true - return true +//! The annotation itself is registered by `daslib/tune` (inert without dasLLVM); this is the +//! body its shell forwards to. +def llvm_code_apply(var func : FunctionPtr; var _group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + for (arg in args) { + if (arg.name == "name") { + if (arg.basicType != Type.tString || empty(arg.sValue)) { + errors := "[llvm_code] on {func.name}: `name` must be a non-empty string (the generator key)" + return false } + // force side effects, or constant-argument calls const-fold in the front-end and never reach the generator + func.sideEffectFlags.userScenario = true + return true } - errors := "[llvm_code] on {func.name}: missing name=\"generator_key\" argument" - return false } + errors := "[llvm_code] on {func.name}: missing name=\"generator_key\" argument" + return false } diff --git a/modules/dasLLVM/daslib/llvm_tune.das b/modules/dasLLVM/daslib/llvm_tune.das index d772885818..ec7b53e1c7 100644 --- a/modules/dasLLVM/daslib/llvm_tune.das +++ b/modules/dasLLVM/daslib/llvm_tune.das @@ -1555,7 +1555,7 @@ def private stamp_llvm_code(var fn : FunctionPtr; genKey : string; perm : TunePe aargs |> push((argname = "tune_suffix", argvalue = RttiValue(tString = perm.suffix))) aargs |> push((argname = "tune_from", argvalue = RttiValue(tString = from))) aargs |> push((argname = "tune_source", argvalue = RttiValue(tString = source))) - fn |> append_annotation("llvm_code", "llvm_code", aargs) + fn |> append_annotation("tune", "llvm_code", aargs) // the annotation is registered by daslib/tune, not by the module that implements it fn.sideEffectFlags.userScenario = true delete aargs } @@ -1631,249 +1631,243 @@ def private emit_registry(var mod : Module?; base : FunctionPtr; perms : array)]` — one grid row for the `tune(...)` that must follow it in //! the same bracket. Args pass to the generator verbatim and fold into the JIT DLL cache //! key; `suffix="..."` overrides the auto-derived variant name. -[function_macro(name = "tune_perm")] -class private TunePermAnnotation : AstFunctionAnnotation { - def override apply(var func : FunctionPtr; var group : ModuleGroup; - args : AnnotationArgumentList; var errors : das_string) : bool { - g_perm_rows[perm_key(func)] |> emplace(build_perm_row(args)) - return true - } +def tune_perm_apply(var func : FunctionPtr; var _group : ModuleGroup; + args : AnnotationArgumentList; var _errors : das_string) : bool { + g_perm_rows[perm_key(func)] |> emplace(build_perm_row(args)) + return true } //! `[tune_companion(fn = "sibling", gen = "generator_key")]` — stamps `sibling` with the SAME //! perm from the same sidecar entry, so a kernel and (say) its repack-layout query can never //! desync. List these after the [tune_perm] grid and before tune(...). See `skills/tune.md`. -[function_macro(name = "tune_companion")] -class private TuneCompanionAnnotation : AstFunctionAnnotation { - def override apply(var func : FunctionPtr; var group : ModuleGroup; - args : AnnotationArgumentList; var errors : das_string) : bool { +def tune_companion_apply(var func : FunctionPtr; var _group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { // the bank doubles as the order guard: rows after tune(...) or before any perm are ignored - if (!(g_perm_rows |> key_exists(perm_key(func)))) { - errors := "[tune_companion] on {func.name}: list it AFTER the [tune_perm] rows and BEFORE tune(...) in the same bracket" - return false - } - var row : TuneCompanion - let fv = find_arg(args, "fn") - if (fv is tString) { row.fnname = fv as tString } - let gv = find_arg(args, "gen") - if (gv is tString) { row.genkey = gv as tString } - if (empty(row.fnname) || empty(row.genkey)) { - errors := "[tune_companion] on {func.name}: fn=\"\" and gen=\"\" are both required" - return false - } - g_companion_rows[perm_key(func)] |> emplace(row) - return true + if (!(g_perm_rows |> key_exists(perm_key(func)))) { + errors := "[tune_companion] on {func.name}: list it AFTER the [tune_perm] rows and BEFORE tune(...) in the same bracket" + return false + } + var row : TuneCompanion + let fv = find_arg(args, "fn") + if (fv is tString) { row.fnname = fv as tString } + let gv = find_arg(args, "gen") + if (gv is tString) { row.genkey = gv as tString } + if (empty(row.fnname) || empty(row.genkey)) { + errors := "[tune_companion] on {func.name}: fn=\"\" and gen=\"\" are both required" + return false } + g_companion_rows[perm_key(func)] |> emplace(row) + return true } //! `[tune(gen="key", fallback="suffix")]` — LAST in a bracket of `[tune_perm]` rows, which it //! consumes. Normal runs stamp one winner (sidecar > fallback > reference); tune/test runs //! stamp the whole grid plus a `_variants()` registry. Mode contract: module docstring. -[function_macro(name = "tune")] -class private TuneAnnotation : AstFunctionAnnotation { - def override apply(var func : FunctionPtr; var group : ModuleGroup; // nolint:STYLE037,STYLE038 — the [tune] resolution ladder: manifest/fallback/reference arms in precedence order - args : AnnotationArgumentList; var errors : das_string) : bool { - let gv = find_arg(args, "gen") - if (!(gv is tString) || empty(gv as tString)) { - errors := "[tune] on {func.name}: gen=\"<[llvm_code] generator key>\" is required" - return false - } - let genKey = gv as tString - var perms <- take_perms(func, errors) - if (empty(errors) && empty(perms)) { - errors := "[tune] on {func.name}: no [tune_perm(...)] rows - list them BEFORE tune(...) in the same annotation bracket" - } - var comps <- take_companions(func, errors) - if (!empty(errors)) { - delete perms - delete comps - return false - } - if (func.result.baseType == Type.autoinfer) { +def tune_apply(var func : FunctionPtr; var _group : ModuleGroup; // nolint:STYLE037,STYLE038 — the [tune] resolution ladder: manifest/fallback/reference arms in precedence order + args : AnnotationArgumentList; var errors : das_string) : bool { + let gv = find_arg(args, "gen") + if (!(gv is tString) || empty(gv as tString)) { + errors := "[tune] on {func.name}: gen=\"<[llvm_code] generator key>\" is required" + return false + } + let genKey = gv as tString + var perms <- take_perms(func, errors) + if (empty(errors) && empty(perms)) { + errors := "[tune] on {func.name}: no [tune_perm(...)] rows - list them BEFORE tune(...) in the same annotation bracket" + } + var comps <- take_companions(func, errors) + if (!empty(errors)) { + delete perms + delete comps + return false + } + if (func.result.baseType == Type.autoinfer) { // function_to_type runs pre-infer: an implicit result reaches the registry as `auto` - errors := "[tune] on {func.name}: an explicit return type is required (add ': void' or the real type)" - delete perms - delete comps - return false - } + errors := "[tune] on {func.name}: an explicit return type is required (add ': void' or the real type)" + delete perms + delete comps + return false + } // overrides grid modes too: a stale DAS_TUNE_MODE must not leak stubs into generated C++ - if (tune_stamp_free_gate()) { - let noPerms : array - var gok = emit_registry(compiling_module(), func, noPerms, errors) - for (c in comps) { - if (gok) { - gok = emit_registry(compiling_module(), c.fn, noPerms, errors) - } + if (tune_stamp_free_gate()) { + let noPerms : array + var gok = emit_registry(compiling_module(), func, noPerms, errors) + for (c in comps) { + if (gok) { + gok = emit_registry(compiling_module(), c.fn, noPerms, errors) } - delete perms - delete comps - return gok - } - let mode = tune_mode() - if (mode == "tune" || mode == "test") { - // visible generation cost: a silent permutation stamp reads as a hung compile from outside - let ctail = empty(comps) ? "" : " (+{length(comps)} companions each)" - print("llvm_tune: [tune] {func.name}: stamping {length(perms)} permutations{ctail}\n") - let ok = stamp_grid(func, genKey, perms, comps, errors) - delete perms - delete comps - return ok - } - if (mode != "normal") { - errors := "[tune] on {func.name}: unknown DAS_TUNE_MODE '{mode}' (normal | tune | test)" - delete perms - delete comps - return false } - if (g_env_tune.tune_policy == "reference") { - let noPerms : array - var rok = emit_registry(compiling_module(), func, noPerms, errors) - for (c in comps) { - break if (!rok) - rok = emit_registry(compiling_module(), c.fn, noPerms, errors) - } - delete perms - delete comps - return rok + delete perms + delete comps + return gok + } + let mode = tune_mode() + if (mode == "tune" || mode == "test") { + // visible generation cost: a silent permutation stamp reads as a hung compile from outside + let ctail = empty(comps) ? "" : " (+{length(comps)} companions each)" + print("llvm_tune: [tune] {func.name}: stamping {length(perms)} permutations{ctail}\n") + let ok = stamp_grid(func, genKey, perms, comps, errors) + delete perms + delete comps + return ok + } + if (mode != "normal") { + errors := "[tune] on {func.name}: unknown DAS_TUNE_MODE '{mode}' (normal | tune | test)" + delete perms + delete comps + return false + } + if (g_env_tune.tune_policy == "reference") { + let noPerms : array + var rok = emit_registry(compiling_module(), func, noPerms, errors) + for (c in comps) { + break if (!rok) + rok = emit_registry(compiling_module(), c.fn, noPerms, errors) } + delete perms + delete comps + return rok + } // no entry falls back rather than dropping to reference, so a sidecar written before a kernel family landed cannot untune it - let fname = "{func.name}" - var manifestFound = false - let mpath = tune_manifest_path() - var manifest <- read_manifest(mpath, manifestFound) - var suffix = "" - var tuneSource = "" // manifest | profile | fallback - the tune_source the stamp records - var tuneSourcePath = "" - if (manifestFound) { - suffix = manifest?[fname] ?? "" - if (!empty(suffix)) { - tuneSource = "manifest" - tuneSourcePath = mpath + let fname = "{func.name}" + var manifestFound = false + let mpath = tune_manifest_path() + var manifest <- read_manifest(mpath, manifestFound) + var suffix = "" + var tuneSource = "" // manifest | profile | fallback - the tune_source the stamp records + var tuneSourcePath = "" + if (manifestFound) { + suffix = manifest?[fname] ?? "" + if (!empty(suffix)) { + tuneSource = "manifest" + tuneSourcePath = mpath + } + } + var profileKlass = "" + var profilePath = "" + var profile <- empty(suffix) ? profile_kernels(profileKlass, profilePath) : no_kernels() + if (empty(suffix)) { + suffix = profile?[fname] ?? "" + if (!empty(suffix)) { + tuneSource = "profile" + tuneSourcePath = profilePath + } + } + // clear an adopted winner this box can't run (ISA-gated → would decline to reference; or not in this build's grid → would crash-loop) so the fallback ;-chain picks a legal perm and it boots + if (!empty(suffix) && suffix != "reference") { + var known = false + var eligible = true + for (p in perms) { + if (p.suffix == suffix) { + known = true + eligible = perm_requires_ok(p) + break } } - var profileKlass = "" - var profilePath = "" - var profile <- empty(suffix) ? profile_kernels(profileKlass, profilePath) : no_kernels() - if (empty(suffix)) { - suffix = profile?[fname] ?? "" - if (!empty(suffix)) { - tuneSource = "profile" - tuneSourcePath = profilePath + if (!known || !eligible) { + if (tune_verbosity() != "silent") { + let why = known ? "fails its requires= on this box" : "is not in this build's grid" + print("llvm_tune: {fname} {tuneSource} winner '{suffix}' {why} - using the fallback chain\n") } + suffix = "" + tuneSource = "" + tuneSourcePath = "" } - // clear an adopted winner this box can't run (ISA-gated → would decline to reference; or not in this build's grid → would crash-loop) so the fallback ;-chain picks a legal perm and it boots - if (!empty(suffix) && suffix != "reference") { - var known = false - var eligible = true + } + var companionOwnSuffix : array // a companion's OWN entry (the harness's gemv seat) when it names an eligible perm of this grid; else it follows the main + companionOwnSuffix |> resize(length(comps)) + if (tuneSource == "manifest" || tuneSource == "profile") { + for (ci, c in count(), comps) { + let own = tuneSource == "manifest" ? (manifest?["{c.fnname}"] ?? "") : (profile?["{c.fnname}"] ?? "") for (p in perms) { - if (p.suffix == suffix) { - known = true - eligible = perm_requires_ok(p) - break + if (p.suffix == own && perm_requires_ok(p)) { + companionOwnSuffix[ci] = own } } - if (!known || !eligible) { - if (tune_verbosity() != "silent") { - let why = known ? "fails its requires= on this box" : "is not in this build's grid" - print("llvm_tune: {fname} {tuneSource} winner '{suffix}' {why} - using the fallback chain\n") - } - suffix = "" - tuneSource = "" - tuneSourcePath = "" - } } - var companionOwnSuffix : array // a companion's OWN entry (the harness's gemv seat) when it names an eligible perm of this grid; else it follows the main - companionOwnSuffix |> resize(length(comps)) - if (tuneSource == "manifest" || tuneSource == "profile") { - for (ci, c in count(), comps) { - let own = tuneSource == "manifest" ? (manifest?["{c.fnname}"] ?? "") : (profile?["{c.fnname}"] ?? "") - for (p in perms) { - if (p.suffix == own && perm_requires_ok(p)) { - companionOwnSuffix[ci] = own - } - } - } - } - let fromPick = !empty(suffix) - var ok = true - if (empty(suffix)) { - tuneSource = "fallback" + } + let fromPick = !empty(suffix) + var ok = true + if (empty(suffix)) { + tuneSource = "fallback" // ';'-chain in declaration order, first passing requires= wins; none eligible => reference - let fv = find_arg(args, "fallback") - if (fv is tString) { - for (cand in split(fv as tString, ";")) { - continue if (empty(cand)) - var found = false - for (p in perms) { - if (p.suffix == cand) { - found = true - if (perm_requires_ok(p)) { - suffix = cand - } - break + let fv = find_arg(args, "fallback") + if (fv is tString) { + for (cand in split(fv as tString, ";")) { + continue if (empty(cand)) + var found = false + for (p in perms) { + if (p.suffix == cand) { + found = true + if (perm_requires_ok(p)) { + suffix = cand } + break } - if (!found) { - errors := "[tune] on {func.name}: fallback perm '{cand}' is not in the [tune_perm] grid" - ok = false - } - break if (!ok || !empty(suffix)) } + if (!found) { + errors := "[tune] on {func.name}: fallback perm '{cand}' is not in the [tune_perm] grid" + ok = false + } + break if (!ok || !empty(suffix)) } } - delete manifest - delete profile - if (ok && !empty(suffix) && suffix != "reference") { - ok = false - for (p in perms) { - if (p.suffix == suffix) { - stamp_llvm_code(func, genKey, p, fromPick ? tuneSourcePath : "", tuneSource) - for (ci, c in count(), comps) { - var stampedOwn = false - if (!empty(companionOwnSuffix[ci]) && companionOwnSuffix[ci] != suffix) { - for (q in perms) { - if (q.suffix == companionOwnSuffix[ci]) { - stamp_llvm_code(c.fn, c.genkey, q, tuneSourcePath, tuneSource) - stampedOwn = true - } - } - if (stampedOwn && tune_verbosity() == "verbose") { - print("llvm_tune: {c.fnname} <- {companionOwnSuffix[ci]} (its own entry, {tuneSourcePath})\n") + } + delete manifest + delete profile + if (ok && !empty(suffix) && suffix != "reference") { + ok = false + for (p in perms) { + if (p.suffix == suffix) { + stamp_llvm_code(func, genKey, p, fromPick ? tuneSourcePath : "", tuneSource) + for (ci, c in count(), comps) { + var stampedOwn = false + if (!empty(companionOwnSuffix[ci]) && companionOwnSuffix[ci] != suffix) { + for (q in perms) { + if (q.suffix == companionOwnSuffix[ci]) { + stamp_llvm_code(c.fn, c.genkey, q, tuneSourcePath, tuneSource) + stampedOwn = true } } - if (!stampedOwn) { - stamp_llvm_code(c.fn, c.genkey, p, fromPick ? tuneSourcePath : "", tuneSource) + if (stampedOwn && tune_verbosity() == "verbose") { + print("llvm_tune: {c.fnname} <- {companionOwnSuffix[ci]} (its own entry, {tuneSourcePath})\n") } } - if (tune_verbosity() == "verbose") { - print("llvm_tune: {fname} <- {suffix} ({fromPick ? tuneSourcePath : "fallback"})\n") + if (!stampedOwn) { + stamp_llvm_code(c.fn, c.genkey, p, fromPick ? tuneSourcePath : "", tuneSource) } - ok = true - break } + if (tune_verbosity() == "verbose") { + print("llvm_tune: {fname} <- {suffix} ({fromPick ? tuneSourcePath : "fallback"})\n") + } + ok = true + break } - if (!ok) { - errors := "[tune] on {func.name}: perm '{suffix}' is not in the [tune_perm] grid" - } } + if (!ok) { + errors := "[tune] on {func.name}: perm '{suffix}' is not in the [tune_perm] grid" + } + } // reference-row-only registries, so harness code referencing _variants() still compiles - if (ok) { - let noPerms : array - ok = emit_registry(compiling_module(), func, noPerms, errors) - for (c in comps) { - if (ok) { - ok = emit_registry(compiling_module(), c.fn, noPerms, errors) - } + if (ok) { + let noPerms : array + ok = emit_registry(compiling_module(), func, noPerms, errors) + for (c in comps) { + if (ok) { + ok = emit_registry(compiling_module(), c.fn, noPerms, errors) } } - delete companionOwnSuffix - delete perms - delete comps - return ok } + delete companionOwnSuffix + delete perms + delete comps + return ok } // ===== [tune_scope] / [tune_policy] — the tuner-wiring UX rail ===== @@ -1896,64 +1890,61 @@ def private module_compiled(name : string) : bool { //! `[tune_scope(name = "mylib", tuner = "harness.das")]` on a dummy struct in the library that //! owns the `[tune]` functions, declared ABOVE them: names the tuner that regenerates this //! library's winners in the ONE per-app sidecar. `tuner=` resolves against the declaring file. -[structure_macro(name = "tune_scope")] -class private TuneScopeAnnotation : AstStructureAnnotation { - def override apply(var st : StructurePtr; var group : ModuleGroup; - args : AnnotationArgumentList; var errors : das_string) : bool { - if (tune_aot_gate()) return true - let nv = find_arg(args, "name") - if (!(nv is tString) || empty(nv as tString)) { - errors := "[tune_scope]: name=\"\" is required" +def tune_scope_apply(var st : StructurePtr; var _group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + if (tune_aot_gate()) return true + let nv = find_arg(args, "name") + if (!(nv is tString) || empty(nv as tString)) { + errors := "[tune_scope]: name=\"\" is required" + return false + } + let sname = nv as tString + var tuner = "" + let tv = find_arg(args, "tuner") + if (tv is tString) { + tuner = resolve_decl_path(string(st.at.fileInfo.name), tv as tString) + } + var covers = "" + let cv = find_arg(args, "covers") + if (cv is tString) { + covers = cv as tString + } + var versionKey = "" + var versionExpected = "" + let vv = find_arg(args, "version_of") + if (vv is tString) { + var keyOverride = "" + let kv = find_arg(args, "version_key") + if (kv is tString) { + keyOverride = kv as tString + } + var verr = "" + if (!resolve_version_of(get_ptr(compiling_program()), vv as tString, keyOverride, versionKey, versionExpected, verr)) { + errors := "[tune_scope]: {verr}" return false } - let sname = nv as tString - var tuner = "" - let tv = find_arg(args, "tuner") - if (tv is tString) { - tuner = resolve_decl_path(string(st.at.fileInfo.name), tv as tString) - } - var covers = "" - let cv = find_arg(args, "covers") - if (cv is tString) { - covers = cv as tString - } - var versionKey = "" - var versionExpected = "" - let vv = find_arg(args, "version_of") - if (vv is tString) { - var keyOverride = "" - let kv = find_arg(args, "version_key") - if (kv is tString) { - keyOverride = kv as tString - } - var verr = "" - if (!resolve_version_of(get_ptr(compiling_program()), vv as tString, keyOverride, versionKey, versionExpected, verr)) { - errors := "[tune_scope]: {verr}" + } + var defaults = "" + let dv = find_arg(args, "defaults") + if (dv is tString) { + defaults = resolve_decl_path(string(st.at.fileInfo.name), dv as tString) + } + if (!empty(defaults)) { + for (m in split(covers, ";")) { + continue if (empty(m)) + if (module_compiled(m)) { + errors := "[tune_scope] '{sname}': covers= module '{m}' compiled before this scope, so its kernels could not read the defaults profile - require this scope's module from '{m}'" return false } } - var defaults = "" - let dv = find_arg(args, "defaults") - if (dv is tString) { - defaults = resolve_decl_path(string(st.at.fileInfo.name), dv as tString) - } - if (!empty(defaults)) { - for (m in split(covers, ";")) { - continue if (empty(m)) - if (module_compiled(m)) { - errors := "[tune_scope] '{sname}': covers= module '{m}' compiled before this scope, so its kernels could not read the defaults profile - require this scope's module from '{m}'" - return false - } - } - } - // overwrite, not error: shared-module macro state persists across compiles in one process - g_scopes["{compiling_module().name}"] = TuneScope(scopeName = sname, - modName = "{compiling_module().name}", covers = covers, - manifestPath = tune_manifest_path(), tunerPath = tuner, - versionKey = versionKey, versionExpected = versionExpected, - defaultsPath = defaults) - return true } + // overwrite, not error: shared-module macro state persists across compiles in one process + g_scopes["{compiling_module().name}"] = TuneScope(scopeName = sname, + modName = "{compiling_module().name}", covers = covers, + manifestPath = tune_manifest_path(), tunerPath = tuner, + versionKey = versionKey, versionExpected = versionExpected, + defaultsPath = defaults) + return true } // The tune flags, parsed off the APP's command line — we share an argv we do not own, so this @@ -2841,32 +2832,29 @@ def private apply_tune_policy(var func : FunctionPtr; declared : string; var err //! `[tune_policy(missing = "...")]` on the app's `main`: what to do about scopes whose sidecar //! entries are missing or stale — `fallback` | `warn` | `error` | `auto` (THE DEFAULT, declared //! or not) | `restart`. Full semantics and the escapes: `skills/tune.md`. -[function_macro(name = "tune_policy")] -class private TunePolicyAnnotation : AstFunctionAnnotation { - def override apply(var func : FunctionPtr; var group : ModuleGroup; - args : AnnotationArgumentList; var errors : das_string) : bool { - // the manifest warning is the default pass's job — it fires for every root, this apply's gate included - if (tune_policy_gated() && !exe_status_report()) { - return true - } - var errs = "" - if (tune_exe_gate()) { - // standalone build: no policy, but the artifact self-reports its stamps - emit_status_init(errs) - } else { - var policy = "auto" - let mv = find_arg(args, "missing") - if (mv is tString) { - policy = mv as tString - } - apply_tune_policy(func, policy, errs) - } - if (!empty(errs)) { - errors := errs - return false - } +def tune_policy_apply(var func : FunctionPtr; var _group : ModuleGroup; + args : AnnotationArgumentList; var errors : das_string) : bool { + // the manifest warning is the default pass's job — it fires for every root, this apply's gate included + if (tune_policy_gated() && !exe_status_report()) { return true } + var errs = "" + if (tune_exe_gate()) { + // standalone build: no policy, but the artifact self-reports its stamps + emit_status_init(errs) + } else { + var policy = "auto" + let mv = find_arg(args, "missing") + if (mv is tString) { + policy = mv as tString + } + apply_tune_policy(func, policy, errs) + } + if (!empty(errs)) { + errors := errs + return false + } + return true } // inert once-only marker the TuneDefaultPolicy pass stamps onto main — appending an @@ -2883,36 +2871,33 @@ class private TunePolicyDefaultMarker : AstFunctionAnnotation { //! The "untuned does not start" default: a program root with a `main`, some `[tune_scope]` in //! play and no declared `[tune_policy]` behaves as `missing = "auto"`. Escapes and the //! once-only marker: `skills/tune.md`. A root without `main` never tunes (dastest files). -[infer_macro] -class private TuneDefaultPolicy : AstPassMacro { - def override apply(prog : ProgramPtr; _mod : Module?) : bool { - // fires only for roots that SEE llvm_tune, so a scope-owning library must re-export it public - return false if (is_in_completion() || !empty("{compiling_module().name}") || empty(g_scopes)) - warn_manifest_untuned() // self-gates on DAS_TUNE_MANIFEST; covers exe and declared-policy roots too - return false if (tune_policy_gated() && !exe_status_report()) - var main_fn = compiling_module() |> find_unique_function("main", false) - return false if (main_fn == null) - for (ann in main_fn.annotations) { - // declared policy wins; the marker means this pass already ran - return false if (ann.annotation.name == "tune_policy" - || ann.annotation.name == "tune_policy_default_applied") - } - main_fn |> append_annotation("llvm_tune", "tune_policy_default_applied") - var errs = "" - var ok = true - if (tune_exe_gate()) { - // standalone build: no policy, but the artifact self-reports its stamps - emit_status_init(errs) - } else { - // O0 cannot represent winners raced under O3 - default fallback (DAS_TUNE_POLICY wins) - let flavor = jit_cli_opt_level() == 0 ? "fallback" : "auto" - ok = apply_tune_policy(main_fn, flavor, errs) - } - if (!ok || !empty(errs)) { +def tune_default_policy_apply(_prog : ProgramPtr; _mod : Module?) : bool { + // fires only for roots that SEE daslib/tune, so a scope-owning library must re-export it public + return false if (is_in_completion() || !empty("{compiling_module().name}") || empty(g_scopes)) + warn_manifest_untuned() // self-gates on DAS_TUNE_MANIFEST; covers exe and declared-policy roots too + return false if (tune_policy_gated() && !exe_status_report()) + var main_fn = compiling_module() |> find_unique_function("main", false) + return false if (main_fn == null) + for (ann in main_fn.annotations) { + // declared policy wins; the marker means this pass already ran + return false if (ann.annotation.name == "tune_policy" + || ann.annotation.name == "tune_policy_default_applied") + } + main_fn |> append_annotation("llvm_tune", "tune_policy_default_applied") + var errs = "" + var ok = true + if (tune_exe_gate()) { + // standalone build: no policy, but the artifact self-reports its stamps + emit_status_init(errs) + } else { + // O0 cannot represent winners raced under O3 - default fallback (DAS_TUNE_POLICY wins) + let flavor = jit_cli_opt_level() == 0 ? "fallback" : "auto" + ok = apply_tune_policy(main_fn, flavor, errs) + } + if (!ok || !empty(errs)) { // macro_error mid-infer-pass is dropped by the retry loop — print + panic actually stops it - print("llvm_tune: {errs}\n") - panic("llvm_tune default policy: {errs}") - } - return true // modified — re-run infer over the injected status/guard + print("llvm_tune: {errs}\n") + panic("llvm_tune default policy: {errs}") } + return true // modified — re-run infer over the injected status/guard } diff --git a/modules/dasLLVM/tests/cant_tune_bad_grids.das b/modules/dasLLVM/tests/cant_tune_bad_grids.das index 69bb887b51..671f0b578a 100644 --- a/modules/dasLLVM/tests/cant_tune_bad_grids.das +++ b/modules/dasLLVM/tests/cant_tune_bad_grids.das @@ -4,6 +4,7 @@ options gen2 expect 20800:12 +require daslib/tune require llvm/daslib/llvm_tune // 1. missing gen= diff --git a/modules/dasLLVM/tests/cant_tune_bad_scope.das b/modules/dasLLVM/tests/cant_tune_bad_scope.das index 960952638a..d296f32345 100644 --- a/modules/dasLLVM/tests/cant_tune_bad_scope.das +++ b/modules/dasLLVM/tests/cant_tune_bad_scope.das @@ -3,6 +3,7 @@ options gen2 expect 20800:3 +require daslib/tune require llvm/daslib/llvm_tune // 1. spec is not module/CONST diff --git a/modules/dasLLVM/tests/cant_tune_scope_after_covered.das b/modules/dasLLVM/tests/cant_tune_scope_after_covered.das index 2138a2351b..699a9df786 100644 --- a/modules/dasLLVM/tests/cant_tune_scope_after_covered.das +++ b/modules/dasLLVM/tests/cant_tune_scope_after_covered.das @@ -4,6 +4,7 @@ options gen2 expect 20800:1 +require daslib/tune require llvm/daslib/llvm_tune require llvm_tune_scope_early diff --git a/modules/dasLLVM/tests/llvm_tune.das b/modules/dasLLVM/tests/llvm_tune.das index b54667961a..f657e41940 100644 --- a/modules/dasLLVM/tests/llvm_tune.das +++ b/modules/dasLLVM/tests/llvm_tune.das @@ -1,6 +1,7 @@ options gen2 require dastest/testing_boost +require daslib/tune require llvm/daslib/llvm_tune // [tune] framework (llvm/daslib/llvm_tune): normal mode stamps the fallback perm's [llvm_code] diff --git a/modules/dasLLVM/tests/llvm_tune_manifest_client.das b/modules/dasLLVM/tests/llvm_tune_manifest_client.das index 750320fab5..5c915925b8 100644 --- a/modules/dasLLVM/tests/llvm_tune_manifest_client.das +++ b/modules/dasLLVM/tests/llvm_tune_manifest_client.das @@ -1,5 +1,6 @@ options gen2 +require daslib/tune require llvm/daslib/llvm_tune // Template fixture for llvm_tune_manifest.das (no [test] on purpose — the suite just diff --git a/modules/dasLLVM/tests/llvm_tune_modes_client.das b/modules/dasLLVM/tests/llvm_tune_modes_client.das index 5c92c5d4eb..f31ac7023a 100644 --- a/modules/dasLLVM/tests/llvm_tune_modes_client.das +++ b/modules/dasLLVM/tests/llvm_tune_modes_client.das @@ -1,6 +1,7 @@ options gen2 require dastest/testing_boost +require daslib/tune require llvm/daslib/llvm_tune // Doubles as the child fixture for llvm_tune_modes.das (spawned with DAS_TUNE_MODE=test, -jit: diff --git a/modules/dasLLVM/tests/llvm_tune_profiles_lib.das b/modules/dasLLVM/tests/llvm_tune_profiles_lib.das index 102da66cd2..3ec6a7ff06 100644 --- a/modules/dasLLVM/tests/llvm_tune_profiles_lib.das +++ b/modules/dasLLVM/tests/llvm_tune_profiles_lib.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_profiles_lib public +require daslib/tune public require llvm/daslib/llvm_tune public // Library-side fixture for llvm_tune_profiles.das (no [test] on purpose — the suite just diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client.das b/modules/dasLLVM/tests/llvm_tune_scope_client.das index a2e566a779..2b15ac1490 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client.das @@ -1,5 +1,6 @@ options gen2 +require daslib/tune require llvm_tune_scope_client_lib require llvm_tune_scope_client_lib2 diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_lib.das b/modules/dasLLVM/tests/llvm_tune_scope_client_lib.das index b60544c059..3058c1104c 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_lib.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_lib.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_scope_client_lib public +require daslib/tune public require llvm/daslib/llvm_tune public // Library-side fixture for llvm_tune_scope.das (no [test] on purpose — the suite just diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_lib2.das b/modules/dasLLVM/tests/llvm_tune_scope_client_lib2.das index f791540648..bfec8e622c 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_lib2.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_lib2.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_scope_client_lib2 public +require daslib/tune public require llvm/daslib/llvm_tune public // Second library fixture for llvm_tune_scope.das (no [test] on purpose): a SEPARATE scope diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_resolver.das b/modules/dasLLVM/tests/llvm_tune_scope_client_resolver.das index 65c237868b..26902396e3 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_resolver.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_resolver.das @@ -1,5 +1,6 @@ options gen2 +require daslib/tune require llvm_tune_scope_client_lib // Resolver fixture for llvm_tune_scope.das (no [test] on purpose — the suite just compiles diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_ver.das b/modules/dasLLVM/tests/llvm_tune_scope_client_ver.das index f80222fd38..48f18f7472 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_ver.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_ver.das @@ -1,5 +1,6 @@ options gen2 +require daslib/tune require llvm_tune_scope_client_verlib require llvm_tune_scope_client_verlib2 diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_verlib.das b/modules/dasLLVM/tests/llvm_tune_scope_client_verlib.das index 07335d681e..9f560623b0 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_verlib.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_verlib.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_scope_client_verlib public +require daslib/tune public require llvm/daslib/llvm_tune public require llvm_tune_scope_verlib // nolint:STYLE030 — version_of= below names TESTVER by string diff --git a/modules/dasLLVM/tests/llvm_tune_scope_client_verlib2.das b/modules/dasLLVM/tests/llvm_tune_scope_client_verlib2.das index dbc704bab6..75abac9127 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_client_verlib2.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_client_verlib2.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_scope_client_verlib2 public +require daslib/tune public require llvm/daslib/llvm_tune public require llvm_tune_scope_verlib // nolint:STYLE030 — version_of= below names TESTVER by string diff --git a/modules/dasLLVM/tests/llvm_tune_scope_early.das b/modules/dasLLVM/tests/llvm_tune_scope_early.das index 71e22f657c..d13df26d49 100644 --- a/modules/dasLLVM/tests/llvm_tune_scope_early.das +++ b/modules/dasLLVM/tests/llvm_tune_scope_early.das @@ -2,6 +2,7 @@ options gen2 module llvm_tune_scope_early public +require daslib/tune public require llvm/daslib/llvm_tune public // Helper for cant_tune_scope_after_covered.das: a kernel module with no scope of its own, so the diff --git a/skills/internal/llvm_tune_internals.md b/skills/internal/llvm_tune_internals.md index 87e1824a1e..34937636da 100644 --- a/skills/internal/llvm_tune_internals.md +++ b/skills/internal/llvm_tune_internals.md @@ -4,6 +4,16 @@ Read before touching `modules/dasLLVM/daslib/llvm_tune.das` or its tests. The fr itself - every annotation, the policy rail, the sidecar contract - is `skills/tune.md`; this file is only what maintaining the implementation needs on top of it. +`llvm_tune.das` registers no annotation of its own. All six names plus the default-policy +infer pass are registered by `daslib/tune`, whose shells forward here - `tune_apply`, +`tune_perm_apply`, `tune_companion_apply`, `tune_scope_apply`, `tune_policy_apply`, +`tune_default_policy_apply`, and `llvm_code_apply` in `llvm_code.das` - under +`static_if (typeinfo builtin_module_exists(llvm_tune))`, and do nothing when the guard is +false. Adding an annotation means adding both halves. A shell may not stay behind in this +module: **a module's macro state is per-module**, so `g_scopes`, `g_perm_rows` and +`g_companion_rows` are only consistent while every writer and reader runs in one macro +context - which is why the infer pass moved to `daslib/tune` with the annotations. + ## Macro-context traps - **Cross-module restamp does not work.** Mutating a *required* library's already-compiled @@ -29,7 +39,9 @@ file is only what maintaining the implementation needs on top of it. `collect_status` reads the stamps back off the AST. - **`stamp_llvm_code` records `tune_suffix` / `tune_from` / `tune_source`** as extra `[llvm_code]` args (generators ignore unknown args by contract), and `tune_status` reads that - stamped truth off the AST - there is no macro-state bank to keep coherent. + stamped truth off the AST - there is no macro-state bank to keep coherent. It appends the + annotation from module `tune`, the module that registers `[llvm_code]`; readers match it by + name, so nothing downstream cares which module it came from. - **`--tune` is decided at runtime** by the guard (`tune_cli_force`, the same `apply_cli_tune_flags` parse over `get_user_args()`), because a warm module cache can serve a compile whose macros never saw this run's argv; the policy pass reads it at macro time only to @@ -37,9 +49,11 @@ file is only what maintaining the implementation needs on top of it. ## Where the pieces live -- The default policy is injected by the `TuneDefaultPolicy` infer pass. A root is detected by - the anonymous module name, and the `tune_policy` marker annotation the pass appends is its - once-only guard. +- The default policy is injected by the `TuneDefaultPolicyShell` infer pass in `daslib/tune`, + which forwards to `tune_default_policy_apply` here. A root is detected by the anonymous + module name, and the `tune_policy` marker annotation the pass appends is its once-only guard. + The pass reaches a root that sees `daslib/tune`, so a scope-owning library re-exports THAT + (`require daslib/tune public`). - `scope_fnames_joined` appends a `version_of` pin to the joined kernel names as one `@version:key=value` token, so every carrier of that string - the policy guards, the resolver contract, daspkg's status walk - enforces the pin with no signature change. diff --git a/skills/tune.md b/skills/tune.md index 90c550b70e..9ebca8b49c 100644 --- a/skills/tune.md +++ b/skills/tune.md @@ -1,11 +1,20 @@ # Kernel tuning - `[tune]` and the per-app tune sidecar -`llvm/daslib/llvm_tune` turns **one reference function into a tuned kernel -family**: a grid of code-generation permutations, a per-app record of which -one wins, and a small policy rail that keeps an application honest about -whether it is running tuned code. It sits on top of `[llvm_code]` (the -JIT-time external code generator), so the family only *generates* under the -LLVM JIT; on every other tier the reference body runs verbatim. +`daslib/tune` turns **one reference function into a tuned kernel family**: a +grid of code-generation permutations, a per-app record of which one wins, and +a small policy rail that keeps an application honest about whether it is +running tuned code. It sits on top of `[llvm_code]` (the JIT-time external +code generator), so the family only *generates* under the LLVM JIT; on every +other tier the reference body runs verbatim. + +`daslib/tune` registers all six annotations - `[tune]`, `[tune_perm]`, +`[tune_companion]`, `[tune_scope]`, `[tune_policy]`, `[llvm_code]` - and the +default-policy pass; the framework behind them is `llvm/daslib/llvm_tune`, +which `daslib/tune` reaches through a path guard and re-exports. **Any program +can spell the annotations and compile whether or not dasLLVM is configured.** +Without dasLLVM every one of them does nothing at all - no sidecar read, no +stamp, no `_variants()` registry, no tuner spawn - and the reference +body IS the function. The design goal is that a shipped application reaches *its own box's* floor with **defaults that are data** - a small JSON sidecar - rather than a fork @@ -38,7 +47,7 @@ under `"kernels"`, library runtime knobs under `"runtime"`. ```das options gen2 -require llvm/daslib/llvm_tune +require daslib/tune // one reference function, a grid of [llvm_code] generator permutations, // and a per-ISA fallback for an app with no sidecar yet. @@ -169,10 +178,10 @@ CPU class's profile at compile time instead of racing. ```{warning} The default-`auto` policy fires only for app roots that **see** -`llvm_tune` - a library owning scopes must re-export it -(`require llvm/daslib/llvm_tune public`), or its apps silently get no -default policy. Re-export `llvm_tune` alone, not your module's whole -public surface (a blanket `public` on a module that also re-exports +`daslib/tune` - a library owning scopes must re-export it +(`require daslib/tune public`), or its apps silently get no default +policy. Re-export `daslib/tune` alone, not your module's whole public +surface (a blanket `public` on a module that also re-exports `jobque_boost` floods requirers with name ambiguities). ``` diff --git a/src/ast/ast_infer_type.cpp b/src/ast/ast_infer_type.cpp index 8cd047af78..d0ffc20209 100644 --- a/src/ast/ast_infer_type.cpp +++ b/src/ast/ast_infer_type.cpp @@ -2845,6 +2845,7 @@ namespace das { // allowPromoted: pairs with the `require ?guard` check (ast_parse.cpp), which // also accepts shared das modules compiled earlier in the process auto mod = Module::requireEx(evar->name, true); + if ( !mod ) mod = program->library.findModule(evar->name); reportAstChanged(); return new ExprConstBool(expr->at, mod != nullptr); } else { diff --git a/tests/daslib/test_tune_shells.das b/tests/daslib/test_tune_shells.das new file mode 100644 index 0000000000..a99bb0c9cb --- /dev/null +++ b/tests/daslib/test_tune_shells.das @@ -0,0 +1,43 @@ +options gen2 + +require dastest/testing_boost public +require daslib/tune + +// The daslib/tune shells: any program spells the tune annotations and compiles on every tier - +// with dasLLVM (stamped, or unstamped under DAS_TUNE_POLICY=reference) and without it, where +// every apply is a no-op and the reference body IS the function. The gate is that the answer +// never moves between those tiers, so the reference body and the generator agree by +// construction: `llvm_code_selftest::add_plus_k` emits `a + b + k` for an (int, int) : int +// stub, and the k7 perm this grid falls back to emits exactly what the body below computes. + +[tune_scope(name = "test_tune_shells")] +struct private TuneShellsScope {} + +def private ref_add(a, b : int) : int { + return a + b + 7 +} + +[tune_perm(k = 7), tune_perm(k = 0), + tune(gen = "llvm_code_selftest::add_plus_k", fallback = "k7")] +def shell_add(a, b : int) : int { + return a + b + 7 +} + +[tune_policy(missing = "fallback")] +def private shell_policy_probe(a : int) : int { + return a * 2 +} + +[test] +def test_tune_shell_result_matches_the_reference(t : T?) { + for (a in range(-3, 4)) { + for (b in range(-3, 4)) { + t |> equal(shell_add(a, b), ref_add(a, b), "shell_add({a}, {b})") + } + } +} + +[test] +def test_tune_policy_leaves_the_body_alone(t : T?) { + t |> equal(shell_policy_probe(21), 42) +} diff --git a/tests/jit_tests/exe_host_cpu.das b/tests/jit_tests/exe_host_cpu.das index 1021f05e22..715d795961 100644 --- a/tests/jit_tests/exe_host_cpu.das +++ b/tests/jit_tests/exe_host_cpu.das @@ -28,7 +28,7 @@ def private write_probe_script(path : string; with_kernel : bool) : bool { if (f != null) { fprint(f, "options gen2\n") if (with_kernel) { - fprint(f, "require llvm/daslib/llvm_code\n") + fprint(f, "require daslib/tune\n") fprint(f, "[llvm_code(name = \"llvm_code_selftest::host_features\")]\n") fprint(f, "def host_rail() : int \{ return -1 \}\n") fprint(f, "[export] def main() \{ print(\"HOSTF=[\{host_rail()}]\\n\"); \}\n") diff --git a/tests/jit_tests/llvm_code.das b/tests/jit_tests/llvm_code.das index e690eb6df3..29408e4d75 100644 --- a/tests/jit_tests/llvm_code.das +++ b/tests/jit_tests/llvm_code.das @@ -1,7 +1,7 @@ options gen2 require dastest/testing_boost -require llvm/daslib/llvm_code +require daslib/tune // [llvm_code] external-codegen rail: the selftest generator (llvm/daslib/llvm_code_selftest, // wired via llvm/daslib/llvm_user_modules) emits `a + b` for (int, int) : int stubs and diff --git a/utils/dasllama-server/README.md b/utils/dasllama-server/README.md index fcb37d3844..b16cd353b6 100644 --- a/utils/dasllama-server/README.md +++ b/utils/dasllama-server/README.md @@ -245,6 +245,8 @@ unverified NEVER auto-applies), `exchange_submit = ask | always | never` (defaul `always` shares it automatically), and `exchange_url` (baked default `https://dasllama.io`). `DASLLAMA_EXCHANGE_URL` / `DASLLAMA_EXCHANGE_ACCEPT` env override for tests and one-shot watchdog relaunches. Lookup failure is never fatal - the boot falls through to the local tuner. +The exchange client rides the tune framework, so a build without dasLLVM carries none of it: the +boot resolver and submit check are never registered and every `/exchange*` route answers 404. **First-contact consent (GDPR):** nothing is sent to the exchange until a choice is expressed. Setting any `exchange_*` key (TOML or env) IS that choice; on the zero-config diff --git a/utils/dasllama-server/main.das b/utils/dasllama-server/main.das index f1fd564608..523c273ddb 100644 --- a/utils/dasllama-server/main.das +++ b/utils/dasllama-server/main.das @@ -26,7 +26,7 @@ require openai_server // the server + route handlers (this directory) require model_catalog // catalog_models_dir — the models-dir resolution require dasllama/dasllama // load_model / QuantMode / load_asr_model require llvm/daslib/llvm_tune // [tune_policy] + tune_status (the injected guard resolves here) -require dasllama/dasllama_exchange // the sidecar exchange: boot resolver + submit check +require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the sidecar exchange (boot resolver + submit check), which rides the tune framework require jobque // set_jobque_threads_cap / set_jobque_affinity require daslib/clargs require daslib/toml @@ -177,7 +177,9 @@ struct ModelEntry { // untuned boot consults the exchange only when the config opts in (the exchange is closed; off by default) [init] def register_exchange_resolver { - exchange_register_resolver("dasllama-server.toml") + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + exchange_register_resolver("dasllama-server.toml") + } } // A clearly model-free boot (no --model/-m, no --config/-c, no discoverable toml) heads for @@ -851,7 +853,9 @@ def init() { // nolint:STYLE037,STYLE038 — boot sequence: the knobs are orde log_tune_status("dasllama-server") // a fresh local tune that has not been shared: submit now (always) or surface the offer // (ask — the watchdog balloon + the control page's exchange card); quiet otherwise - exchange_boot_submit_check(exchange_policy_from_config(g_cfg.config)) + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + exchange_boot_submit_check(exchange_policy_from_config(g_cfg.config)) + } // GPU backend lands BEFORE the load: the metal mode pins the row-major backend the GPU // kernels read, and the vulkan tier want must be armed before the load walk uploads stacks let gpu = effective_gpu(g_cfg) diff --git a/utils/dasllama-server/openai_server.das b/utils/dasllama-server/openai_server.das index 427ad579fc..744aaeddcb 100644 --- a/utils/dasllama-server/openai_server.das +++ b/utils/dasllama-server/openai_server.das @@ -9,7 +9,7 @@ require dasllama/dasllama_scheduler // continuous-batching scheduler (layer 3 require dashv/dashv_boost public require daslib/json_boost require daslib/toml // write_toml: POST /config persists the authoritative config -require dasllama/dasllama_exchange // the /exchange surface + submit rails +require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the /exchange surface + submit rails, which ride the tune framework require llvm/daslib/llvm_tune // tune_manifest_path — the /exchange apply target require daslib/defer // defer() { delete js } — free the parsed request JSON on every exit path require model_catalog // the curated list + the /catalog download rail @@ -3560,10 +3560,13 @@ def private handle_config_save(var req : HttpRequest?; var resp : HttpResponse?) // ===== the exchange surface (GET /exchange + the apply/submit/retune levers) ===== -def private exchange_policy_now : ExchangePolicy { - // the save path covers every start shape: --config, auto-discovered, and config-less - // (where the file may not exist yet and the baked defaults stand) - return exchange_policy_from_config(g_config_save_path) +// Every route below reads its policy from `exchange_policy_from_config(g_config_save_path)` — the +// save path covers every start shape: --config, auto-discovered, and config-less (where the file +// may not exist yet and the baked defaults stand). It is spelled at each site rather than hoisted +// because `ExchangePolicy` is a type of the optional exchange module: without the tune framework +// there is no such type to name in a signature, and each route answers `exchange_absent` instead. +def private exchange_absent(var resp : HttpResponse?) : http_status { // nolint:LINT022 — reached only from the no-framework static_if arm, which a build with dasLLVM drops + return resp |> JSON(error_body("the sidecar exchange is not in this build (no tune framework)", "not_found"), http_status.NOT_FOUND) } // The exchange control routes are an operator surface reached through the loopback control @@ -3619,7 +3622,11 @@ def private handle_exchange_get(var req : HttpRequest?; var resp : HttpResponse? // the status carries the local sidecar path (username in it) — operator-only, like the rest return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - return resp |> JSON(exchange_status_json(exchange_policy_now())) + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + return resp |> JSON(exchange_status_json(exchange_policy_from_config(g_config_save_path))) + } else { + return exchange_absent(resp) + } } def private handle_exchange_matches(var req : HttpRequest?; var resp : HttpResponse?) : http_status { @@ -3627,99 +3634,119 @@ def private handle_exchange_matches(var req : HttpRequest?; var resp : HttpRespo return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } // a live exchange lookup — seconds on the tick thread; the page requests it explicitly - return resp |> JSON(exchange_matches_json(exchange_policy_now())) + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + return resp |> JSON(exchange_matches_json(exchange_policy_from_config(g_config_save_path))) + } else { + return exchange_absent(resp) + } } def private handle_exchange_apply(var req : HttpRequest?; var resp : HttpResponse?) : http_status { if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - var jerr = "" - var js = read_json(string(req.body), jerr) - defer() { - unsafe { - delete js + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + var jerr = "" + var js = read_json(string(req.body), jerr) + defer() { + unsafe { + delete js + } } + let sha : string = js?["sha"] ?? "" + if (empty(sha)) { + return resp |> JSON(error_body("body must carry the sidecar sha", "invalid_request_error"), http_status.BAD_REQUEST) + } + // the listing box the page picked: passed through so the downloaded sidecar is verified to + // match what the exchange listed — the same lying-exchange gate the boot resolver applies + let box : string = js?["box"] ?? "" + let err = exchange_fetch_and_apply(exchange_policy_from_config(g_config_save_path).url, sha, tune_manifest_path(), box) + if (!empty(err)) { + return resp |> JSON(error_body(err, "invalid_request_error"), http_status.BAD_REQUEST) + } + // explicitly fetching a sidecar IS engaging the exchange - record the consent it expresses + exchange_record_consent(true) + to_log(LOG_INFO, "dasllama-server: exchange sidecar {sha} applied via the control page; draining for restart\n") + g_shutdown_requested = true + g_restart_requested = true + return resp |> JSON("\{\"ok\":true,\"restarting\":true}", http_status.OK) + } else { + return exchange_absent(resp) } - let sha : string = js?["sha"] ?? "" - if (empty(sha)) { - return resp |> JSON(error_body("body must carry the sidecar sha", "invalid_request_error"), http_status.BAD_REQUEST) - } - // the listing box the page picked: passed through so the downloaded sidecar is verified to - // match what the exchange listed — the same lying-exchange gate the boot resolver applies - let box : string = js?["box"] ?? "" - let err = exchange_fetch_and_apply(exchange_policy_now().url, sha, tune_manifest_path(), box) - if (!empty(err)) { - return resp |> JSON(error_body(err, "invalid_request_error"), http_status.BAD_REQUEST) - } - // explicitly fetching a sidecar IS engaging the exchange - record the consent it expresses - exchange_record_consent(true) - to_log(LOG_INFO, "dasllama-server: exchange sidecar {sha} applied via the control page; draining for restart\n") - g_shutdown_requested = true - g_restart_requested = true - return resp |> JSON("\{\"ok\":true,\"restarting\":true}", http_status.OK) } def private handle_exchange_submit(var req : HttpRequest?; var resp : HttpResponse?) : http_status { if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - let err = exchange_submit_current(exchange_policy_now()) - if (!empty(err)) { - return resp |> JSON(error_body(err, "invalid_request_error"), http_status.BAD_REQUEST) + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + let err = exchange_submit_current(exchange_policy_from_config(g_config_save_path)) + if (!empty(err)) { + return resp |> JSON(error_body(err, "invalid_request_error"), http_status.BAD_REQUEST) + } + to_log(LOG_INFO, "dasllama-server: this box's tune submitted to the exchange\n") + return resp |> JSON("\{\"ok\":true}", http_status.OK) + } else { + return exchange_absent(resp) } - to_log(LOG_INFO, "dasllama-server: this box's tune submitted to the exchange\n") - return resp |> JSON("\{\"ok\":true}", http_status.OK) } def private handle_exchange_consent(var req : HttpRequest?; var resp : HttpResponse?) : http_status { if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - var jerr = "" - var js = read_json(string(req.body), jerr) - defer() { - unsafe { - delete js + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + var jerr = "" + var js = read_json(string(req.body), jerr) + defer() { + unsafe { + delete js + } } + if (js == null) { + return resp |> JSON(error_body("invalid JSON: {jerr}", "invalid_request_error"), http_status.BAD_REQUEST) + } + let av = js?["accept"] + if (av == null || !(av.value is _bool)) { + return resp |> JSON(error_body("body must carry accept:true|false", "invalid_request_error"), http_status.BAD_REQUEST) + } + let accepted = av.value as _bool + if (!exchange_record_consent(accepted)) { + return resp |> JSON(error_body("could not write the consent file", "server_error"), http_status.INTERNAL_SERVER_ERROR) + } + let manifest = tune_manifest_path() + if (accepted && (!stat(manifest).is_valid || tune_sidecar_stale(manifest))) { + // untuned (or stale) box just enabled the exchange — restart so the boot resolver + // runs the lookup; a tuned box records the choice and keeps serving + to_log(LOG_INFO, "dasllama-server: exchange enabled via the control page; draining for restart to run the lookup\n") + g_shutdown_requested = true + g_restart_requested = true + return resp |> JSON("\{\"ok\":true,\"accepted\":true,\"restarting\":true}", http_status.OK) + } + to_log(LOG_INFO, "dasllama-server: exchange consent recorded ({accepted ? "accepted" : "declined"}) via the control page\n") + return resp |> JSON("\{\"ok\":true,\"accepted\":{accepted}}", http_status.OK) + } else { + return exchange_absent(resp) } - if (js == null) { - return resp |> JSON(error_body("invalid JSON: {jerr}", "invalid_request_error"), http_status.BAD_REQUEST) - } - let av = js?["accept"] - if (av == null || !(av.value is _bool)) { - return resp |> JSON(error_body("body must carry accept:true|false", "invalid_request_error"), http_status.BAD_REQUEST) - } - let accepted = av.value as _bool - if (!exchange_record_consent(accepted)) { - return resp |> JSON(error_body("could not write the consent file", "server_error"), http_status.INTERNAL_SERVER_ERROR) - } - let manifest = tune_manifest_path() - if (accepted && (!stat(manifest).is_valid || tune_sidecar_stale(manifest))) { - // untuned (or stale) box just enabled the exchange — restart so the boot resolver - // runs the lookup; a tuned box records the choice and keeps serving - to_log(LOG_INFO, "dasllama-server: exchange enabled via the control page; draining for restart to run the lookup\n") - g_shutdown_requested = true - g_restart_requested = true - return resp |> JSON("\{\"ok\":true,\"accepted\":true,\"restarting\":true}", http_status.OK) - } - to_log(LOG_INFO, "dasllama-server: exchange consent recorded ({accepted ? "accepted" : "declined"}) via the control page\n") - return resp |> JSON("\{\"ok\":true,\"accepted\":{accepted}}", http_status.OK) } def private handle_exchange_retune(var req : HttpRequest?; var resp : HttpResponse?) : http_status { if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - let err = exchange_request_retune() - if (!empty(err)) { - // the marker write failed and the sidecar is untouched — do NOT drain/restart - return resp |> JSON(error_body(err, "server_error"), http_status.INTERNAL_SERVER_ERROR) + static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + let err = exchange_request_retune() + if (!empty(err)) { + // the marker write failed and the sidecar is untouched — do NOT drain/restart + return resp |> JSON(error_body(err, "server_error"), http_status.INTERNAL_SERVER_ERROR) + } + to_log(LOG_INFO, "dasllama-server: re-tune armed (sidecar removed, exchange skipped once); draining for restart\n") + g_shutdown_requested = true + g_restart_requested = true + return resp |> JSON("\{\"ok\":true,\"restarting\":true,\"tune\":true}", http_status.OK) + } else { + return exchange_absent(resp) } - to_log(LOG_INFO, "dasllama-server: re-tune armed (sidecar removed, exchange skipped once); draining for restart\n") - g_shutdown_requested = true - g_restart_requested = true - return resp |> JSON("\{\"ok\":true,\"restarting\":true,\"tune\":true}", http_status.OK) } def private emit_asr_event(events : Channel?; event : AsrEvent) { diff --git a/utils/daspkg/test_daspkg.das b/utils/daspkg/test_daspkg.das index 03a4afefa9..b17e10be42 100644 --- a/utils/daspkg/test_daspkg.das +++ b/utils/daspkg/test_daspkg.das @@ -1881,7 +1881,7 @@ def test_cmd_release_ships_minted_scope_manifest(t : T?) { fwrite(f, payload) } fopen("{tmp_root}/main.das", "wb") $(f) { - fwrite(f, "options gen2\nrequire llvm/daslib/llvm_tune\n[tune_scope(name=\"stubscope\", tuner=\"stub_tuner.das\")]\nstruct StubScope \{\n unused : int\n\}\n[export]\ndef main() \{\n print(\"hi\\n\")\n\}\n") + fwrite(f, "options gen2\nrequire daslib/tune\nrequire llvm/daslib/llvm_tune\n[tune_scope(name=\"stubscope\", tuner=\"stub_tuner.das\")]\nstruct StubScope \{\n unused : int\n\}\n[export]\ndef main() \{\n print(\"hi\\n\")\n\}\n") } fopen("{tmp_root}/stub_tuner.das", "wb") $(f) { fwrite(f, "options gen2\nrequire daslib/fio\n[export]\ndef main() \{\n fopen(get_env_variable(\"DAS_TUNE_MANIFEST\"), \"wb\") $(mf) \{\n fwrite(mf, fread(\"{tmp_root}/stub_payload.json\"))\n \}\n\}\n") From f5bdfe5a3197afa050bb86ce8a3b622a69cac7fd Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 17:26:45 -0700 Subject: [PATCH 07/43] stage 1, wave 3: aarch64_neon and x64_avx move to daslib beside f16_cvt - pure das, zero requires, recognized by module name - so the NEON tier and the generated families' reference bodies compile without dasLLVM and their [init] registrations self-gate; the guards on both engine requires go, math_gen's llvm_tune re-export takes the path guard, and the three pure lattice table builders test_kquant borrowed from the IR generator move to dasllama_gemm_schema Co-Authored-By: Claude Fable 5.1 --- .../daslib => daslib}/aarch64_neon.das | 0 .../dasLLVM/daslib => daslib}/x64_avx.das | 0 .../benchmarks/matmul/matmul_variants.das | 2 +- modules/dasLLAMA/dasllama/dasllama_common.das | 4 +-- .../dasLLAMA/dasllama/dasllama_gemm_gen.das | 30 ------------------- .../dasllama/dasllama_gemm_schema.das | 30 +++++++++++++++++++ .../dasllama/dasllama_math_aarch64_neon.das | 2 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 2 +- modules/dasLLAMA/harness/smmla_probe.das | 2 +- modules/dasLLAMA/harness/tune_kernels.das | 2 +- modules/dasLLAMA/portable_tier_plan.md | 14 ++++++--- modules/dasLLAMA/tests/test_kquant.das | 2 +- modules/dasLLVM/.das_module | 2 -- 13 files changed, 48 insertions(+), 44 deletions(-) rename {modules/dasLLVM/daslib => daslib}/aarch64_neon.das (100%) rename {modules/dasLLVM/daslib => daslib}/x64_avx.das (100%) diff --git a/modules/dasLLVM/daslib/aarch64_neon.das b/daslib/aarch64_neon.das similarity index 100% rename from modules/dasLLVM/daslib/aarch64_neon.das rename to daslib/aarch64_neon.das diff --git a/modules/dasLLVM/daslib/x64_avx.das b/daslib/x64_avx.das similarity index 100% rename from modules/dasLLVM/daslib/x64_avx.das rename to daslib/x64_avx.das diff --git a/modules/dasLLAMA/benchmarks/matmul/matmul_variants.das b/modules/dasLLAMA/benchmarks/matmul/matmul_variants.das index 85918af16f..91f846d17d 100644 --- a/modules/dasLLAMA/benchmarks/matmul/matmul_variants.das +++ b/modules/dasLLAMA/benchmarks/matmul/matmul_variants.das @@ -4,7 +4,7 @@ module matmul_variants shared public require dasllama/dasllama_lint public require daslib/jobque_boost -require llvm/daslib/aarch64_neon // sdot4: JIT-emitted ARM SDOT (scalar fallback elsewhere) +require daslib/aarch64_neon // sdot4: JIT-emitted ARM SDOT (scalar fallback elsewhere) require math // mad: fused multiply-add in the decode-GEMV flush variants // Experiment ledger for the matmul kernel. Every optimization attempt lives here as diff --git a/modules/dasLLAMA/dasllama/dasllama_common.das b/modules/dasLLAMA/dasllama/dasllama_common.das index ff2c908bdb..b48dcfda6b 100644 --- a/modules/dasLLAMA/dasllama/dasllama_common.das +++ b/modules/dasLLAMA/dasllama/dasllama_common.das @@ -18,8 +18,8 @@ require daslib/jobque_profile // JOBQUE_PROFILING_ENABLED — the compile-time require dasllama/dasllama_kv_codec public // the KV-cache codec families — kv_store_row/kv_load_row/kv_dot/kv_axpy dispatch into them require dasllama/dasllama_convert // cvt_f32_to_f16/cvt_f16_to_f32 — the f16 rows (wscale plane + the f16 KV codec) require dasllama/dasllama_math_default // registers the portable Q8·Q8 backend at [init] (the fallback everywhere) -require ?llvm/daslib/aarch64_neon dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init] under the JIT alone; PATH guard: without dasLLVM its intrinsics are scalar fallbacks slower than portable, so the tier stays out -require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen // [init]-only: registers "arm64-gen" (the generated GEMM family, arm64 load-select tier). PATH guard: the module hard-requires llvm_tune, so it loads exactly when dasLLVM is mounted +require dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init] under arm64 + JIT alone; compiles everywhere (daslib/aarch64_neon carries scalar fallbacks) +require dasllama/dasllama_math_gen // [init]-only: registers "arm64-gen" / "x64-gen" (the generated GEMM family) under the JIT alone; compiles everywhere - its reference bodies are the portable tier's kq kernels require daslib/tune public // the tune annotations + the auto-policy pass, inert without dasLLVM; PUBLIC for the same reach as the line below require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): tune_status reaches every app that requires this module require ?das_metal dasllama/dasllama_metal_gemm // [init]-only: registers "metal" (the GPU prefill-GEMM batch donor). STRICT guard: the C++ das_metal module exists on Apple builds only, so non-Apple lanes never compile the Metal driver diff --git a/modules/dasLLAMA/dasllama/dasllama_gemm_gen.das b/modules/dasLLAMA/dasllama/dasllama_gemm_gen.das index e2640f7f5f..de690f75ed 100644 --- a/modules/dasLLAMA/dasllama/dasllama_gemm_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_gemm_gen.das @@ -1502,15 +1502,6 @@ def private grid_rows_path(te : TileEmit) : bool { return false } -//! eight +-1 bytes for sign byte `sb`, lane k negative when bit k is set -def public pm1_of(sb : int) : uint64 { // test_kquant round-trips the sign expansion - var v = 0ul - for (k in range(8)) { - v |= uint64(((sb >> k) & 1) != 0 ? 0xFF : 0x01) << uint64(8 * k) - } - return v -} - //! the row-pair decode's sign table: the 128 ksigns codes with the code's parity as the 8th bit //! (ksigns[i] == i | parity(i) << 7), each as its eight +-1 bytes def private sign_pm1_emit_globals(var te : TileEmit; mod : LLVMOpaqueModule?) { @@ -1876,13 +1867,6 @@ def private vbmi_push_iq2(te : TileEmit; var p0, p1 : array; w p1 |> push(LLVMConstInt(te.types.t_int8, uint64(vbmi_pack_word(w1, alpha, 2, 4)), 0)) } -//! the grid's symbol alphabet: the halved iq3xxs magnitudes, iq3s's odd magnitudes, the iq2 family's three -def public vbmi_alphabet(kq : int) : array { // test_kquant round-trips the code planes - if (kq == 34) return <- [2, 6, 10, 14, 18, 22, 26, 31] - if (kq == 33) return <- [1, 3, 5, 7, 9, 11, 13, 15] - return <- [8, 25, 43] -} - //! the grid gemv's module-level tables: the format's grid, the ksigns +-1 table when the row form reads it, the lattice's def private grid_emit_globals(var te : TileEmit; fmt : int; mod : LLVMOpaqueModule?) { if (fmt == 33) { @@ -1907,20 +1891,6 @@ def private vbmi_tag(kq : int) : string => kq == 25 ? "iq2xxs" : (kq == 24 ? "iq //! grid entries per format: the compact plane's byte count def private vbmi_entries(kq : int) : int => (kq == 25 || kq == 34) ? 256 : (kq == 23 ? 1024 : 512) -//! a grid word's low `nbytes` weight bytes as packed symbols (weight t at bits t*bits); a byte outside the alphabet is a broken table -def public vbmi_pack_word(word : uint; alpha : array; bits, nbytes : int) : int { - var code = 0 - for (t in range(nbytes)) { - let wb = int((word >> uint(8 * t)) & 0xFFu) - let si = alpha |> find_index(wb) - if (si < 0) { - panic("vbmi lattice: grid byte {wb} is outside the alphabet") - } - code |= si << (bits * t) - } - return code -} - def private vbmi_const_bytes(te : TileEmit; mod : LLVMOpaqueModule?; name : string; var elems : array) : LLVMOpaqueValue? { var g = LLVMAddGlobal(mod, LLVMArrayType(te.types.t_int8, uint(length(elems))), name) LLVMSetInitializer(g, LLVMConstArray(te.types.t_int8, array_data_ptr(elems), uint(length(elems)))) diff --git a/modules/dasLLAMA/dasllama/dasllama_gemm_schema.das b/modules/dasLLAMA/dasllama/dasllama_gemm_schema.das index f476357100..16ee216ab5 100644 --- a/modules/dasLLAMA/dasllama/dasllama_gemm_schema.das +++ b/modules/dasLLAMA/dasllama/dasllama_gemm_schema.das @@ -104,3 +104,33 @@ def kq_ssb(fmt : int) : int64 { def kq_reads_packed_planes(fmt : int64) : bool { return fmt == 4l || fmt == 40l || fmt == 44l || fmt == 3l || fmt == 45l || fmt == 2l } + +//! eight +-1 bytes for sign byte `sb`, lane k negative when bit k is set +def pm1_of(sb : int) : uint64 { + var v = 0ul + for (k in range(8)) { + v |= uint64(((sb >> k) & 1) != 0 ? 0xFF : 0x01) << uint64(8 * k) + } + return v +} + +//! the grid's symbol alphabet: the halved iq3xxs magnitudes, iq3s's odd magnitudes, the iq2 family's three +def vbmi_alphabet(kq : int) : array { + if (kq == 34) return <- [2, 6, 10, 14, 18, 22, 26, 31] + if (kq == 33) return <- [1, 3, 5, 7, 9, 11, 13, 15] + return <- [8, 25, 43] +} + +//! a grid word's low `nbytes` weight bytes as packed symbols (weight t at bits t*bits); a byte outside the alphabet is a broken table +def vbmi_pack_word(word : uint; alpha : array; bits, nbytes : int) : int { + var code = 0 + for (t in range(nbytes)) { + let wb = int((word >> uint(8 * t)) & 0xFFu) + let si = alpha |> find_index(wb) + if (si < 0) { + panic("vbmi lattice: grid byte {wb} is outside the alphabet") + } + code |= si << (bits * t) + } + return code +} diff --git a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das index cc7ae8d694..1a7df92f3f 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das @@ -9,7 +9,7 @@ require dasllama/dasllama_math require dasllama/dasllama_math_default // dot_q8q8_idot4x4/_f16s — the plain-das GEMV dots this backend dispatches require dasllama/dasllama_tune // [tuned]: reconstitute the laneq4x4 tile from its template with per-box loop hints require dasllama/dasllama_tune_scope // nolint:STYLE030 — the dasllama [tune_scope] must be on the AST before this module's [tuned] kernels pick their perms -require llvm/daslib/aarch64_neon // tbl16/sdot4_w/sdot4_laneq: the mx4 + laneq tiles' JIT intrinsics +require daslib/aarch64_neon // tbl16/sdot4_w/sdot4_laneq: the mx4 + laneq tiles' JIT intrinsics require daslib/f16_cvt // f16_to_f32 — the wscale_f16 twins widen the binary16 scale in-loop require daslib/jobque_boost public // kernels expand parallel_for; consumers need its symbols require dasllama/dasllama_par // maybe_parallel_for diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index e2d7eed233..89026489df 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -12,7 +12,7 @@ require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module -require llvm/daslib/llvm_tune public // PUBLIC so the auto-policy infer pass and tune_status reach app roots that require this module +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: this file names no framework symbol itself require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public diff --git a/modules/dasLLAMA/harness/smmla_probe.das b/modules/dasLLAMA/harness/smmla_probe.das index 78e397de0d..2e00799e59 100644 --- a/modules/dasLLAMA/harness/smmla_probe.das +++ b/modules/dasLLAMA/harness/smmla_probe.das @@ -1,7 +1,7 @@ options gen2 options _dasllama_internal = true require daslib/jobque_boost -require llvm/daslib/aarch64_neon +require daslib/aarch64_neon require dasllama/dasllama_env // SMMLA (i8mm) single-core scout: correctness + ALU-throughput ratio vs SDOT on the same diff --git a/modules/dasLLAMA/harness/tune_kernels.das b/modules/dasLLAMA/harness/tune_kernels.das index 78e153a2cb..8f3c4ee60b 100644 --- a/modules/dasLLAMA/harness/tune_kernels.das +++ b/modules/dasLLAMA/harness/tune_kernels.das @@ -28,7 +28,7 @@ require ?das_metal dasllama/dasllama_metal_prefill // metal_tensor_race (Apple require ?das_metal dasllama/dasllama_metal_kernels // metal_tensor_race_decode (Apple static_if half) require tuner_cli // the shared tuner CLI require tuner_noise // the noise gate + the measured floor, shared with the other half -require llvm/daslib/aarch64_neon // sdot4_laneq (referenced by the cloned laneq template bodies) +require daslib/aarch64_neon // sdot4_laneq (referenced by the cloned laneq template bodies) require daslib/f16_cvt // f16 references + the cloned f16-kernel template bodies require math // mad (referenced by the cloned template bodies) require daslib/json diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index b60074655b..af03def201 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -63,10 +63,16 @@ with LLVM on the host only; wasm32 cross is unsupported by design (pointer-width - **`f16_cvt` moves to `daslib/f16_cvt.das`.** The JIT recognizes it by mangled name from a target-independent table (`llvm_jit_intrin.das:232-234`), so the move changes nothing for the JIT; 33 require spellings change. -- **`aarch64_neon` and `x64_avx` stay in dasLLVM.** Nothing in the engine requires `x64_avx`. - The NEON tier file registers only under `jit_enabled()`, so its require becomes the path - guard `?llvm/daslib/aarch64_neon` from the umbrella; the harness, probe and benchmark that - require it are JIT-only by nature. +- **`aarch64_neon` and `x64_avx` move to daslib too** (ruled 2026-09-04, landed). The first + cut kept them in dasLLVM behind a path guard on the NEON tier; the no-LLVM gate then showed + the generated families' reference bodies in `dasllama_math_gen.das` call the NEON module's + `dot_q8q8_laneq4` and `idot4`, so the whole generated tier and the six tests that require it + fell out of a no-LLVM build. Both intrinsic modules are pure das with zero requires; the JIT + recognizes them by module name, unchanged. With them in daslib the NEON tier and `math_gen` + compile everywhere, their `[init]` registrations self-gate on `jit_enabled()` and the arch, + and the kq ladder runs on the no-LLVM tier. The three pure table builders `test_kquant` + borrowed from the IR generator (`pm1_of`, `vbmi_alphabet`, `vbmi_pack_word`) moved to + `dasllama_gemm_schema.das` for the same reason. - **`require ?G target` names a guard module, never the target** (`ds2_parser.ypp:849-861`, `parser_impl.cpp:1225-1279`). There is no module named `llvm` (dasLLVM is a pure-das dasbind package), so `?llvm` and `builtin_module_exists(llvm)` are permanently false; the working diff --git a/modules/dasLLAMA/tests/test_kquant.das b/modules/dasLLAMA/tests/test_kquant.das index 367da547a3..aa2c92f4f4 100644 --- a/modules/dasLLAMA/tests/test_kquant.das +++ b/modules/dasLLAMA/tests/test_kquant.das @@ -6,7 +6,7 @@ options _function_length = 300 // and each gate carries its whole fixture options _dasllama_internal = true require dastest/testing_boost public -require dasllama/dasllama_gemm_gen // vbmi_pack_word / vbmi_alphabet / pm1_of - the lattice's pure table builders +require dasllama/dasllama_gemm_schema // vbmi_pack_word / vbmi_alphabet / pm1_of - the lattice's pure table builders require dasllama/dasllama_layout // moe_gpu_gather_stack_kq - the device-plane gather oracle require _model_tier require dasllama/dasllama_gguf diff --git a/modules/dasLLVM/.das_module b/modules/dasLLVM/.das_module index e0b63b9295..1330ed28b0 100644 --- a/modules/dasLLVM/.das_module +++ b/modules/dasLLVM/.das_module @@ -10,8 +10,6 @@ def initialize(project_path : string) { "llvm_jit_intrin", "llvm_jit_common", "llvm_jit_lower", "llvm_dll_utils", "llvm_exe", "llvm_macro", "llvm_jit_cli", "llvm_jit_run", "llvm_aot", "llvm_env", // [EnvConfig] environment-knob registry (ENVIRONMENT.md generates from it) - "aarch64_neon", // public NEON-intrinsic header (portable fallbacks + name-based JIT recognition) - "x64_avx", // public x86-64-intrinsic header (same contract, x64 mirror) "llvm_code", // [llvm_code] user-side annotation "llvm_jit_code", // [llvm_code] JIT-side generator registry "llvm_user_modules", // user-extensible require list for generator modules From c0dd2c2ed1dcb4a8f0d4e2224e9bd8e723cd311c Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 17:31:54 -0700 Subject: [PATCH 08/43] run.das --no-tune runs its children with the image rail off: the reference policy is a different box identity, and a .dlim minted under it GC-purges the box's tuned images; test_audio_embedder's direct-image cell picks the image the rail would mint for THIS identity instead of the first gemma4a-tagged file in directory order Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/tests/CLAUDE.md | 5 ++++- modules/dasLLAMA/tests/run.das | 6 +++++- modules/dasLLAMA/tests/test_audio_embedder.das | 6 +++++- 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/modules/dasLLAMA/tests/CLAUDE.md b/modules/dasLLAMA/tests/CLAUDE.md index 0347ddcd2c..b088972fec 100644 --- a/modules/dasLLAMA/tests/CLAUDE.md +++ b/modules/dasLLAMA/tests/CLAUDE.md @@ -41,7 +41,10 @@ Every suite but `model-free` and `stocked` needs `--arm`. `--full` parses and is whole gate; `--exclude ` drops the files whose name contains a token and names each on an `EXCLUDED` line, so a trimmed run cannot read as full. `--no-tune` composes with any suite: every child runs with `DAS_TUNE_POLICY=reference`, so the `[tune]` families and the `[tuned]` -hints fall to their reference bodies - the portable tier's arm of the gate. The runner redirects +hints fall to their reference bodies - the portable tier's arm of the gate. That arm also runs +with `DASLLAMA_IMAGE=0`: the reference policy is a different box identity, and a `.dlim` minted +under it would GC-purge the box's tuned images, so image-rail cells skip there and keep their +coverage on the tuned arm. The runner redirects the COMPLETE output to a log file, and prints that path on the DONE line. It owns the dastest timeout, and repeats a file only when `--nreps` is passed explicitly (default 1, never best-of-N). Every child runs `-jit -module-cache .jitted_scripts/module_cache/dastest.dascache`; diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index bd35b529ba..da10bf5b1e 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -425,7 +425,11 @@ def private child_envs(cfg : RunArgs; win : bool; arm : string) : string { } } if (cfg.no_tune) { - envs = (win ? "{envs}set \"DAS_TUNE_POLICY=reference\"&& " : "DAS_TUNE_POLICY=reference {envs}") + // the reference policy changes the box identity every .dlim is stamped with, and a mint + // under a foreign identity GC-purges the box's tuned images as dead siblings - so the + // portable arm runs with the image rail off; image-rail coverage stays with the tuned arm + envs = (win ? "{envs}set \"DAS_TUNE_POLICY=reference\"&& set \"DASLLAMA_IMAGE=0\"&& " + : "DAS_TUNE_POLICY=reference DASLLAMA_IMAGE=0 {envs}") } if (cfg.suite == "image-vulkan") { // the offline-bake suite: arm the tier so the probed config carries a vulkan section — diff --git a/modules/dasLLAMA/tests/test_audio_embedder.das b/modules/dasLLAMA/tests/test_audio_embedder.das index c55ac79e61..7318d2663d 100644 --- a/modules/dasLLAMA/tests/test_audio_embedder.das +++ b/modules/dasLLAMA/tests/test_audio_embedder.das @@ -73,7 +73,11 @@ def test_audio_embedder_gemma4a_arm(t : T?) { // scan only skips under DASLLAMA_IMAGE=0 — a model gate, not an artifact gate if (empty(image_path) && starts_with(name, "{base_name(mm)}.0x") && ends_with(name, ".dlim")) { let cand = path_join(models_dir(), name) - if (image_family_tag(cand) |> starts_with("gemma4a-")) { + let tag = image_family_tag(cand) + // an image baked under another box identity (a different tune policy, another + // backend pin) sits beside the current one with its own hash - only the path the + // rail would mint for THIS identity is loadable by the direct route + if (tag |> starts_with("gemma4a-") && cand == image_path_for(mm, tag)) { image_path = cand } } From 64d24b0a655102047acb1a149d5c5add1c6d062e Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 19:28:20 -0700 Subject: [PATCH 09/43] math_gen's [init] guards its one framework call (tune_suppress_mint) under static_if - the last unguarded llvm_tune symbol in the engine's kernel closure; without dasLLVM the q8q8 family, the prefill CPU kernels and the whole kq ladder now run interpreted Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/dasllama/dasllama_math_gen.das | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 89026489df..7e84912dfa 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -12,7 +12,7 @@ require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: this file names no framework symbol itself +require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: the one framework call below sits under static_if require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public @@ -2750,7 +2750,9 @@ def dasllama_math_gen_register() { // line is unconditional-when-set — active overrides announce themselves if (g_env_bench.allow_untuned) { to_log(LOG_WARNING, "dasLLAMA: DASLLAMA_ALLOW_UNTUNED=1 - tune mint suppressed; untuned kernels run fallback, numbers are not publishable\n") - tune_suppress_mint("DASLLAMA_ALLOW_UNTUNED=1") + static_if (typeinfo builtin_module_exists(llvm_tune)) { + tune_suppress_mint("DASLLAMA_ALLOW_UNTUNED=1") + } } // the cross-box override's q51 reach: x64-gen keeps disk-order q51 slots, so the generic // grp q51 repack registers here for bakes whose TARGET has grp q51 planes From 8a906b5a1f9872c648bb2159a1f07e2a42324a2a Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 19:49:05 -0700 Subject: [PATCH 10/43] the aarch64 SDOT and SMMLA tables gate on DotProd and i8mm, not the arch alone: a kernel-free -exe targets the generic ARMv8.0 machine, where a name-recognized sdot4 was a fatal Cannot select under the reference tune policy A standalone exe targets the host CPU only when it carries [llvm_code] kernels; with the tune framework short-circuited there are none, so the NEON tier's reference bodies met a baseline machine that has no SDOT. g_target_arm64_dotprod (host rail: always, via the +dotprod append; generic and cross rails: the force env) now gates the sdot4 family and the idot ss form, g_target_arm64_i8mm gates smmla, and a force-env feature reaches the generic machine's string so the gates and the machine stay one truth. CODEGEN_VERSION 0x6e. Gates: tests/jit_tests/aarch64_neon.das compares every NEON intrinsic against a reference copy inside jitted helpers (a no-arg pure helper was const-folded, so they take a salt); exe_host_cpu.das builds a kernel-free exe that calls sdot4, smmla and the AVX2 dot on the baseline (red on the pre-fix emitter) and checks its answer; test_exe_smoke is green under DAS_TUNE_POLICY=reference. Co-Authored-By: Claude Fable 5.1 --- daslib/aarch64_neon.das | 8 +- modules/dasLLAMA/portable_tier_plan.md | 17 +- modules/dasLLVM/ARCHITECTURE.md | 7 +- modules/dasLLVM/daslib/llvm_jit_common.das | 32 ++- modules/dasLLVM/daslib/llvm_jit_intrin.das | 47 +++- modules/dasLLVM/daslib/llvm_jit_run.das | 2 +- tests/jit_tests/_exe_baseline_intrin.das | 26 +++ tests/jit_tests/aarch64_neon.das | 256 +++++++++++++++++++++ tests/jit_tests/exe_host_cpu.das | 21 +- 9 files changed, 388 insertions(+), 28 deletions(-) create mode 100644 tests/jit_tests/_exe_baseline_intrin.das create mode 100644 tests/jit_tests/aarch64_neon.das diff --git a/daslib/aarch64_neon.das b/daslib/aarch64_neon.das index 73ac587eb3..78b90e152b 100644 --- a/daslib/aarch64_neon.das +++ b/daslib/aarch64_neon.das @@ -6,11 +6,13 @@ module aarch64_neon shared public // // Every function here computes the correct result on ANY target via its daslang body — so the // interpreter, AOT, and non-ARM JIT builds all just run that body. On an aarch64 JIT target the -// LLVM JIT recognizes the call BY NAME (see g_aarch64_intrin_lookup in llvm_jit_intrin.das, gated -// on g_target_is_aarch64) and emits the hardware NEON instruction in place of the call. There is +// LLVM JIT recognizes the call BY NAME (the g_aarch64_*_intrin_lookup tables in llvm_jit_intrin.das, +// gated on the target arch and on the instruction's feature: TBL is baseline, SDOT needs DotProd, +// SMMLA needs i8mm) and emits the hardware NEON instruction in place of the call. There is // nothing to annotate and nothing to require from the llvm module — the recognition is purely // name-based at JIT codegen time, so the fallback runs verbatim everywhere the intrinsic isn't -// emitted. Callers therefore get the fast path for free on arm64 and stay correct elsewhere. +// emitted: a foreign arch, or a generic-CPU aarch64 exe whose baseline machine lacks the feature. +// Callers therefore get the fast path for free on arm64 and stay correct elsewhere. //! Signed dot-product (`SDOT` / `@llvm.aarch64.neon.sdot`): load 16 int8 from each of `w` and `x` //! (16-wide unaligned loads), multiply elementwise, and accumulate consecutive groups of 4 products diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index af03def201..be3a647d9d 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -225,10 +225,25 @@ file. 5. The NEON tier's require takes the path guard. 6. `tests/run.das` gets a no-tune arm (the framework's reference policy; `run.das:309` hard-codes `-jit` with no knob today), and the suite plus the parity rails run under it on - the stage-0 gates. + the stage-0 gates. (Landed as `--no-tune`; the whole-suite run under it is 2.5 h and was + cancelled - the suite rework in flight elsewhere is the gate, this arc runs single files.) 7. Measure the portable floor on the reference small models (decode and prefill, ASR and TTS cells), under the measurement discipline: it is the number every later tier is judged against. +8. The `-jit -exe` rail under the reference policy (landed). A standalone exe targets the host + CPU only when the program carries `[llvm_code]` kernels (`llvm_jit_run.das`, + `has_generated_kernel`); with the framework short-circuited there are none, the exe is the + generic ARMv8.0 baseline, and the NEON tier's reference bodies still name-resolved + `aarch64_neon::sdot4` to SDOT - a fatal `Cannot select` at codegen. The SDOT and SMMLA + tables now gate on `g_target_arm64_dotprod` / `g_target_arm64_i8mm` (host rail: always / + detected; generic and cross rails: the force env only), and a force-env feature reaches the + generic machine's string, so the gates and the machine stay one truth + (`modules/dasLLVM/ARCHITECTURE.md#aarch64-feature-truth`). The generic exe runs the NEON + tier on the scalar `sdot4` fallback - slower than the portable backend's auto-vectorized + `dot_q8q8`; whether the `[init]` backend gates should also consult the exe's feature rail is a + stage-2 question, alongside the same shape under AOT. Gates: `tests/jit_tests/aarch64_neon.das` + (hardware vs reference per intrinsic), the baseline probe in `tests/jit_tests/exe_host_cpu.das` + (red on the pre-fix emitter), and `test_exe_smoke` under `DAS_TUNE_POLICY=reference`. **Exit:** a `-DDAS_LLVM_DISABLED=ON` tree compiles the engine and its suite; a `-jit` run with the framework short-circuited is green on the stage-0 gates and parity; the floor is in diff --git a/modules/dasLLVM/ARCHITECTURE.md b/modules/dasLLVM/ARCHITECTURE.md index 5b4c7594cd..7030b3f89c 100644 --- a/modules/dasLLVM/ARCHITECTURE.md +++ b/modules/dasLLVM/ARCHITECTURE.md @@ -138,7 +138,12 @@ instead (sysctl / `AT_HWCAP` / `IsProcessorFeaturePresent`), so it answers for s never heard of. Both the tier gates (`init_jit_target_flags`) and the target machine's feature string (`create_default_target_machine`) therefore take the union of the two: an LLVM host-string hit OR a `cpu_supports` hit (fullfp16 additionally reads darwin-arm64 as always-on - every -Apple Silicon part has it). A cross-compile triple takes neither - only the force env. +Apple Silicon part has it). A cross-compile triple takes neither - only the force env - and so +does a generic-CPU standalone exe (one carrying no `[llvm_code]` kernel): its machine is the +ARMv8.0 baseline, which cannot select SDOT or SMMLA, so the DotProd and i8mm gates +(`g_target_arm64_dotprod`, `g_target_arm64_i8mm`) stay off there and every `aarch64_neon` call +that needs either compiles its daslang fallback body. The gates and the machine string are one +truth on both rails: a force-env feature raises the gate AND is appended to the generic machine. The two ways a feature reaches the target machine's string license different things. A detection-derived append - `+dotprod` always, `+i8mm` when `cpu_supports` confirms it - is diff --git a/modules/dasLLVM/daslib/llvm_jit_common.das b/modules/dasLLVM/daslib/llvm_jit_common.das index 908f849619..c18878aee6 100644 --- a/modules/dasLLVM/daslib/llvm_jit_common.das +++ b/modules/dasLLVM/daslib/llvm_jit_common.das @@ -444,6 +444,11 @@ var public g_target_x64_amx = false // targets. Gates emission that calls Linux-only externs (the amx witness's arch_prctl). var public g_target_os_linux = false +// ARMv8.2 DotProd (SDOT — the aarch64_neon sdot4 family, the idot ss form). Host targets always +// (the +dotprod append, ARCHITECTURE.md#aarch64-feature-truth); a generic-CPU exe or a cross triple +// only via DAS_JIT_ARM64_FORCE_FEATURES — the baseline machine cannot select SDOT. +var public g_target_arm64_dotprod = false + // ARMv8.6 i8mm (SMMLA — 2×2 s8 matrix-multiply-accumulate; Apple M2/A15+, Graviton3+; M1 has // DotProd only). Host truth is LLVMGetHostCPUFeatures "+i8mm" OR cpu_supports (ARCHITECTURE.md#aarch64-feature-truth). var public g_target_arm64_i8mm = false @@ -462,6 +467,11 @@ def private arm64_forced_feature(name : string) : bool { return false } +def public arm64_forced_plus_features() : string { + let plus <- [for (part in split(g_env_jit.jit_arm64_force_features, ",")); "+{part}"; where !empty(part)] + return join(plus, ",") +} + // "+name" present in LLVM's host-features string (comma-separated "+feat"/"-feat" entries; // exact match on the enabled spelling, so a "-i8mm" entry never reads as support) def private host_llvm_feature(name : string) : bool { @@ -634,6 +644,10 @@ def public init_jit_target_flags(target_triple : string = ""; host_features : bo g_target_os_linux = (empty(target_triple) ? get_platform_name() == "linux" : find(target_triple, "linux") >= 0) + // aarch64 dotprod (see declaration comment): the host rail force-appends +dotprod, so host + // truth IS the rail; the generic exe / cross rails take the force env only. + g_target_arm64_dotprod = g_target_is_aarch64 && (arm64_forced_feature("dotprod") + || (empty(target_triple) && host_features)) g_target_arm64_i8mm = g_target_is_aarch64 && (arm64_forced_feature("i8mm") || (empty(target_triple) && host_features && (host_llvm_feature("i8mm") || cpu_supports("i8mm")))) // aarch64 fullfp16 (see declaration comment): darwin-arm64 == Apple Silicon == always. @@ -922,20 +936,18 @@ def public create_default_target_machine(opt_level : uint; use_host_cpu : bool) feats = "{feats},+i8mm" } // a forced feature's artifact cache-hits on the target box without a linker there - let forced_arm <- [for (part in split(g_env_jit.jit_arm64_force_features, ",")); "+{part}"; where !empty(part)] + let forced_arm = arm64_forced_plus_features() if (!empty(forced_arm)) { - feats = "{feats},{join(forced_arm, ",")}" + feats = "{feats},{forced_arm}" } targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) - } elif (use_host_cpu && g_target_is_x64 && !empty(x64_forced_plus_features())) { - // DAS_JIT_X64_FORCE_FEATURES emission-only override (see the g_target_x64_* declaration - // comment): append AFTER the host features — the last occurrence of a feature wins in - // LLVM's SubtargetFeatures, so "+avx512f" here overrides the host list's "-avx512f". - let forced = x64_forced_plus_features() - let feats = empty(features_msg) ? forced : "{features_msg},{forced}" - targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) } else { - targetMachine = create_target_machine(triple_msg, cpu_msg, features_msg, opt_level) + // the force env (emission-only, see the g_target_x64_* declarations) reaches BOTH rails: + // AFTER the host features, where the last occurrence wins in LLVM's SubtargetFeatures, and + // onto the generic baseline, so the gates it raised never emit what the machine cannot select + let forced = g_target_is_x64 ? x64_forced_plus_features() : (g_target_is_aarch64 ? arm64_forced_plus_features() : "") + let feats = empty(forced) ? features_msg : (empty(features_msg) ? forced : "{features_msg},{forced}") + targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) } if (use_host_cpu) { diff --git a/modules/dasLLVM/daslib/llvm_jit_intrin.das b/modules/dasLLVM/daslib/llvm_jit_intrin.das index 4c16103320..7dbc9b5450 100644 --- a/modules/dasLLVM/daslib/llvm_jit_intrin.das +++ b/modules/dasLLVM/daslib/llvm_jit_intrin.das @@ -179,17 +179,24 @@ let g_intrin_lookup <- { "math::normalize" => @@intrinsic_math_normalize } -// AArch64 NEON intrinsics (the llvm/aarch64_neon header), consulted only when the JIT target is +// AArch64 NEON intrinsics (the daslib/aarch64_neon module), consulted only when the JIT target is // aarch64. On any other target has_intrinsic returns false for these keys and the call's own // daslang body — the portable fallback in the declaring module — compiles as a normal function. -// That silent degrade-to-fallback contract holds for every table in this section. +// That silent degrade-to-fallback contract holds for every table in this section. The baseline +// table is ARMv8.0 (TBL); the SDOT and SMMLA tables carry their own feature gates +// (g_target_arm64_dotprod / g_target_arm64_i8mm), because a generic-CPU exe targets the +// baseline machine, where either instruction is a fatal "Cannot select" at codegen. let g_aarch64_intrin_lookup <- { + "aarch64_neon::tbl16_lo" => @@intrinsic_tbl16, + "aarch64_neon::tbl16_hi" => @@intrinsic_tbl16 +} +let g_aarch64_dotprod_intrin_lookup <- { // ARMv8.2 DotProd "aarch64_neon::sdot4" => @@intrinsic_sdot4, "aarch64_neon::sdot4_laneq" => @@intrinsic_sdot4_laneq, "aarch64_neon::sdot4_w" => @@intrinsic_sdot4_w, - "aarch64_neon::sdot4_laneq_w" => @@intrinsic_sdot4_laneq_w, - "aarch64_neon::tbl16_lo" => @@intrinsic_tbl16, - "aarch64_neon::tbl16_hi" => @@intrinsic_tbl16, + "aarch64_neon::sdot4_laneq_w" => @@intrinsic_sdot4_laneq_w +} +let g_aarch64_i8mm_intrin_lookup <- { // ARMv8.6 i8mm "aarch64_neon::smmla" => @@intrinsic_smmla } @@ -276,6 +283,16 @@ def public has_intrinsic(expr : ExprCallFunc?) { result = true } } + if (!result && g_target_arm64_dotprod) { + g_aarch64_dotprod_intrin_lookup |> get(call_name) $(_pfun) { + result = true + } + } + if (!result && g_target_arm64_i8mm) { + g_aarch64_i8mm_intrin_lookup |> get(call_name) $(_pfun) { + result = true + } + } if (!result && g_target_x64_avx2) { g_x64_intrin_lookup |> get(call_name) $(_pfun) { result = true @@ -320,6 +337,16 @@ def public lookup_intinsic(g_ctx : LLVMContextRef; g_builder : LLVMOpaqueBuilder result = pfun |> invoke(JitCtx(ctx = g_ctx, builder = g_builder, types = types), expr, arguments) } } + if (result == null && g_target_arm64_dotprod) { + g_aarch64_dotprod_intrin_lookup |> get(call_name) $(pfun) { + result = pfun |> invoke(JitCtx(ctx = g_ctx, builder = g_builder, types = types), expr, arguments) + } + } + if (result == null && g_target_arm64_i8mm) { + g_aarch64_i8mm_intrin_lookup |> get(call_name) $(pfun) { + result = pfun |> invoke(JitCtx(ctx = g_ctx, builder = g_builder, types = types), expr, arguments) + } + } if (result == null && g_target_x64_avx2) { g_x64_intrin_lookup |> get(call_name) $(pfun) { result = pfun |> invoke(JitCtx(ctx = g_ctx, builder = g_builder, types = types), expr, arguments) @@ -1513,9 +1540,9 @@ def intrinsic_math_sinh_cosh_tanh(var ctx : JitCtx; expr : ExprCallFunc?; argume } // ===== the idot family: exact integer dots on the 8-bit lattice ===== -// Two lowerings: the signed x signed form emits ONE @llvm.aarch64.neon.sdot on the aarch64 -// host-features rail (g_target_host_features = exactly when +dotprod is force-appended, see -// init_jit_target_flags); everything else gets generic widen-multiply IR. The AArch64 backend +// Two lowerings: the signed x signed form emits ONE @llvm.aarch64.neon.sdot where the target has +// DotProd (g_target_arm64_dotprod: the host rail's +dotprod append, or the force env on the +// generic rail); everything else gets generic widen-multiply IR. The AArch64 backend // does NOT fold the generic form to SDOT (probe: zip/uzp/smull expansion), hence the native arm. // The unsigned x signed form needs USDOT (i8mm — absent on M1); it rides the i8mm un-gate item. def private idot_products(var ctx : JitCtx; expr : ExprCallFunc?; a, b : LLVMOpaqueValue?; aIdx : int) : LLVMOpaqueValue? { // nolint:LINT014 — shared by the idot intrinsics @@ -1545,7 +1572,7 @@ def private idot_sdot_aarch64(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : def intrinsic_lattice_idot(var ctx : JitCtx; expr : ExprCallFunc?; arguments : array) : LLVMOpaqueValue? { let aUnsigned = expr.arguments[0]._type.vectorBaseType == Type.tUInt8 - if (g_target_is_aarch64 && g_target_host_features && !aUnsigned) { + if (g_target_arm64_dotprod && !aUnsigned) { let v4i32 = ctx.types.LLVMInt4Type() var s = idot_sdot_aarch64(ctx, LLVMConstNull(v4i32), arguments[0], arguments[1]) if (s != null) { @@ -1577,7 +1604,7 @@ def intrinsic_lattice_idot4(var ctx : JitCtx; expr : ExprCallFunc?; arguments : let hasAcc = length(arguments) == 3 let aIdx = hasAcc ? 1 : 0 let aUnsigned = expr.arguments[aIdx]._type.vectorBaseType == Type.tUInt8 - if (g_target_is_aarch64 && g_target_host_features && !aUnsigned) { + if (g_target_arm64_dotprod && !aUnsigned) { let v4i32 = ctx.types.LLVMInt4Type() var acc = hasAcc ? arguments[0] : LLVMConstNull(v4i32) var s = idot_sdot_aarch64(ctx, acc, arguments[aIdx], arguments[aIdx + 1]) diff --git a/modules/dasLLVM/daslib/llvm_jit_run.das b/modules/dasLLVM/daslib/llvm_jit_run.das index 0b066b6c25..9363c7ca85 100644 --- a/modules/dasLLVM/daslib/llvm_jit_run.das +++ b/modules/dasLLVM/daslib/llvm_jit_run.das @@ -37,7 +37,7 @@ var LINK_WHOLE_LIB = false // when true, standalone exe links against the whole // invalidates cached DLLs (e.g. edits to llvm_jit.das, llvm_macro.das, llvm_jit_common.das, // runtime helper ABI, default target triple). Cache filenames fold this in, so a bump // makes every previously written DLL miss the cache on the next run and get GC'd. -let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x6dul // the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) +let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x6eul // the aarch64 SDOT / SMMLA tables gate on DotProd / i8mm, not the arch alone, and the force env reaches the generic exe machine (0x6d: the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) // Read by tests-cpp/small/test_jit_emitter_pin.cpp: FNV-1a64 of the emitter sources // (normalized to LF; file list in the test) diff --git a/tests/jit_tests/_exe_baseline_intrin.das b/tests/jit_tests/_exe_baseline_intrin.das new file mode 100644 index 0000000000..2b4c79c5cf --- /dev/null +++ b/tests/jit_tests/_exe_baseline_intrin.das @@ -0,0 +1,26 @@ +options gen2 + +// child for exe_host_cpu.das: a kernel-free program reaching every feature-gated intrinsic +// family through a plain call, built with -exe onto the generic baseline machine — the build +// must not emit an instruction that machine cannot select, and the artifact prints the +// fallback bodies' exact answers +require daslib/aarch64_neon +require daslib/x64_avx + +def lane_sum(v : int4) : int => v.x + v.y + v.z + v.w + +[export] +def main { + var w : array + w |> resize(16) + for (i in range(16)) { + w[i] = int8(i + 1) + } + unsafe { + let p = addr(w[0]) + let sdot = lane_sum(aarch64_neon::sdot4(int4(0), p, p)) + let mmla = lane_sum(aarch64_neon::smmla(int4(0), p, p)) + let dot = lane_sum(x64_avx::dot4(int4(0), p, p)) + print("INTRIN=[{sdot},{mmla},{dot}]\n") + } +} diff --git a/tests/jit_tests/aarch64_neon.das b/tests/jit_tests/aarch64_neon.das new file mode 100644 index 0000000000..895a9aa230 --- /dev/null +++ b/tests/jit_tests/aarch64_neon.das @@ -0,0 +1,256 @@ +options gen2 + +require dastest/testing_boost +require daslib/aarch64_neon + +// aarch64_neon intrinsic emission: under -jit on an aarch64 host the module's sdot4 family, the +// tbl16 pair and smmla lower to SDOT / TBL / SMMLA (smmla only where the host has i8mm) at the +// call sites inside the jitted run_* helpers, while the ref_* copies — the same arithmetic in a +// different module, so the name-keyed lookup never matches — compile as-is; every op is exact +// integer, so hardware-vs-reference isolates the emitter. Interp, foreign-arch and generic-exe +// lanes run the identical body on both sides (trivially green) — that IS the fallback contract +// the aarch64_neon header states. The helpers take no T? (dastest keeps such functions off the +// JIT), and each [test] witnesses that its helper is jitted before trusting the comparison. + +def sx8(v : int) : int => ((v & 0xff) ^ 0x80) - 0x80 + +// byte e (0..3) of packed lane `lane`, sign-extended +def packed_byte(lane : int; e : int) : int => sx8(lane >> (e * 8)) + +def ref_sdot4(acc : int4; w : int8 const?; x : int8 const?) : int4 { + var r = acc + unsafe { + for (j in range(4)) { + var s = 0 + for (e in range(4)) { + s += int(w[4 * j + e]) * int(x[4 * j + e]) + } + r[j] += s + } + } + return r +} + +def ref_sdot4_w(acc : int4; w : int4; x : int8 const?) : int4 { + var r = acc + unsafe { + for (j in range(4)) { + var s = 0 + for (e in range(4)) { + s += packed_byte(w[j], e) * int(x[4 * j + e]) + } + r[j] += s + } + } + return r +} + +def ref_sdot4_laneq(acc : int4; w : int8 const?; x : int8 const?; lane : int) : int4 { + var r = acc + unsafe { + for (j in range(4)) { + var s = 0 + for (e in range(4)) { + s += int(w[4 * j + e]) * int(x[4 * lane + e]) + } + r[j] += s + } + } + return r +} + +def ref_sdot4_laneq_w(acc : int4; w : int4; x : int8 const?; lane : int) : int4 { + var r = acc + unsafe { + for (j in range(4)) { + var s = 0 + for (e in range(4)) { + s += packed_byte(w[j], e) * int(x[4 * lane + e]) + } + r[j] += s + } + } + return r +} + +def ref_tbl16(lut : int8 const?; nib : uint8 const?; high : bool) : int4 { + var r = int4(0) + unsafe { + for (j in range(4)) { + for (e in range(4)) { + let n = int(nib[4 * j + e]) + let idx = high ? (n >> 4) : (n & 0x0f) + r[j] |= (int(lut[idx]) & 0xff) << (e * 8) + } + } + } + return r +} + +def ref_smmla(acc : int4; a : int8 const?; b : int8 const?) : int4 { + var r = acc + unsafe { + for (i in range(2)) { + for (j in range(2)) { + var s = 0 + for (k in range(8)) { + s += int(a[8 * i + k]) * int(b[8 * j + k]) + } + r[2 * i + j] += s + } + } + } + return r +} + +struct Lcg { + state : uint +} + +def next_byte(var g : Lcg) : int8 { + g.state = g.state * 1664525u + 1013904223u + return int8(int(g.state >> 24u) - 128) +} + +def fill_bytes(var g : Lcg; var dst : array) { + for (v in dst) { + v = next_byte(g) + } +} + +def fill_nibbles(var g : Lcg; var dst : array) { + for (v in dst) { + v = uint8(next_byte(g)) + } +} + +let TRIALS = 256 + +// the helpers seed off a global each [test] writes, so a pure call with constant arguments +// cannot be folded at compile time and reaped before the JIT sees its call sites +var g_salt = 0u + +// mismatch count over TRIALS random int8 vectors; the lane is a compile-time constant of the +// intrinsic, so the four lane forms are spelled out +def run_sdot4_family(salt : uint) : int { + var g = Lcg(state = 0x9e3779b9u ^ salt) + var w : array + var x : array + w |> resize(16) + x |> resize(16) + var bad = 0 + for (trial in range(TRIALS)) { + fill_bytes(g, w) + fill_bytes(g, x) + let acc = int4(trial, -trial, trial * 7, 3 - trial) + unsafe { + let wp = addr(w[0]) + let xp = addr(x[0]) + if (sdot4(acc, wp, xp) != ref_sdot4(acc, wp, xp)) { + bad++ + } + let packed = ref_tbl16(wp, reinterpret(xp), false) + if (sdot4_w(acc, packed, xp) != ref_sdot4_w(acc, packed, xp)) { + bad++ + } + if (sdot4_laneq(acc, wp, xp, 0) != ref_sdot4_laneq(acc, wp, xp, 0)) { + bad++ + } + if (sdot4_laneq(acc, wp, xp, 1) != ref_sdot4_laneq(acc, wp, xp, 1)) { + bad++ + } + if (sdot4_laneq(acc, wp, xp, 2) != ref_sdot4_laneq(acc, wp, xp, 2)) { + bad++ + } + if (sdot4_laneq(acc, wp, xp, 3) != ref_sdot4_laneq(acc, wp, xp, 3)) { + bad++ + } + if (sdot4_laneq_w(acc, packed, xp, 0) != ref_sdot4_laneq_w(acc, packed, xp, 0)) { + bad++ + } + if (sdot4_laneq_w(acc, packed, xp, 1) != ref_sdot4_laneq_w(acc, packed, xp, 1)) { + bad++ + } + if (sdot4_laneq_w(acc, packed, xp, 2) != ref_sdot4_laneq_w(acc, packed, xp, 2)) { + bad++ + } + if (sdot4_laneq_w(acc, packed, xp, 3) != ref_sdot4_laneq_w(acc, packed, xp, 3)) { + bad++ + } + } + } + return bad +} + +def run_tbl16(salt : uint) : int { + var g = Lcg(state = 0x2545f491u ^ salt) + var lut : array + var nib : array + lut |> resize(16) + nib |> resize(16) + var bad = 0 + for (_trial in range(TRIALS)) { + fill_bytes(g, lut) + fill_nibbles(g, nib) + unsafe { + let lp = addr(lut[0]) + let np = addr(nib[0]) + if (tbl16_lo(lp, np) != ref_tbl16(lp, np, false)) { + bad++ + } + if (tbl16_hi(lp, np) != ref_tbl16(lp, np, true)) { + bad++ + } + } + } + return bad +} + +def run_smmla(salt : uint) : int { + var g = Lcg(state = 0x7f4a7c15u ^ salt) + var a : array + var b : array + a |> resize(16) + b |> resize(16) + var bad = 0 + for (trial in range(TRIALS)) { + fill_bytes(g, a) + fill_bytes(g, b) + let acc = int4(-trial, trial, 11 - trial, trial * 5) + unsafe { + let ap = addr(a[0]) + let bp = addr(b[0]) + if (smmla(acc, ap, bp) != ref_smmla(acc, ap, bp)) { + bad++ + } + } + } + return bad +} + +[test] +def test_sdot4_family(t : T?) { + g_salt++ + t |> run("sdot4 / sdot4_w / sdot4_laneq / sdot4_laneq_w: hardware == reference") @(t : T?) { + t |> success(!jit_enabled() || is_jit_function(@@run_sdot4_family), "the helper is jitted, so its call sites reach the emitter") + t |> equal(run_sdot4_family(g_salt), 0, "every sdot4-family lane matches the reference over {TRIALS} random int8 vectors") + } +} + +[test] +def test_tbl16(t : T?) { + g_salt++ + t |> run("tbl16_lo / tbl16_hi: hardware == reference") @(t : T?) { + t |> success(!jit_enabled() || is_jit_function(@@run_tbl16), "the helper is jitted, so its call sites reach the emitter") + t |> equal(run_tbl16(g_salt), 0, "every tbl16 lookup matches the reference over {TRIALS} random tables") + } +} + +[test] +def test_smmla(t : T?) { + g_salt++ + t |> run("smmla: hardware == reference") @(t : T?) { + t |> success(!jit_enabled() || is_jit_function(@@run_smmla), "the helper is jitted, so its call sites reach the emitter") + t |> equal(run_smmla(g_salt), 0, "every smmla 2x2 product matches the reference over {TRIALS} random int8 matrices") + } +} diff --git a/tests/jit_tests/exe_host_cpu.das b/tests/jit_tests/exe_host_cpu.das index 715d795961..2909618d24 100644 --- a/tests/jit_tests/exe_host_cpu.das +++ b/tests/jit_tests/exe_host_cpu.das @@ -19,7 +19,10 @@ let KERNEL_SCRIPT = "{OUTPUT_DIR}/exe_host_cpu_kernel.das" let KERNEL_EXE = "{OUTPUT_DIR}/exe_host_cpu_kernel" let PLAIN_SCRIPT = "{OUTPUT_DIR}/exe_host_cpu_plain.das" let PLAIN_EXE = "{OUTPUT_DIR}/exe_host_cpu_plain" +let BASELINE_SCRIPT = "{get_das_root()}/tests/jit_tests/_exe_baseline_intrin.das" +let BASELINE_EXE = "{OUTPUT_DIR}/exe_baseline_intrin" let HOST_CPU_LINE = "targeting the current box" +let BASELINE_ANSWER = "INTRIN=[1496,2480,1496]" //! the fallback bodies over w = x = 1..16: sum of squares per lane group, the 2x2 block products, the same dot again def private write_probe_script(path : string; with_kernel : bool) : bool { var ok = false @@ -93,11 +96,25 @@ def test_exe_host_cpu(t : T?) { spawn_child("{neutral_env()}\"{bin}\" -exe -output \"{PLAIN_EXE}\" \"{PLAIN_SCRIPT}\"", lp) t |> success(!(lp |> has(HOST_CPU_LINE)), "kernel-free exe build must stay generic") - // run the built exe where the platform can load the shared lib (same gates as jit_exe.das: + // a kernel-free exe that calls the feature-gated intrinsic families (SDOT, SMMLA, the AVX2 + // dot) stays on the generic baseline: the build must not emit an instruction that machine + // cannot select — the gates decline to the fallback bodies — and the artifact prints their answers + var lb : array + let rcb = spawn_child("{neutral_env()}\"{bin}\" -exe -output \"{BASELINE_EXE}\" \"{BASELINE_SCRIPT}\"", lb) + t |> equal(0, rcb, "the intrinsic-calling kernel-free exe must build on the generic baseline: {join(lb, "\n")}") + t |> success(!(lb |> has(HOST_CPU_LINE)), "an intrinsic call alone never promotes the exe to the host cpu") + + // run the built exes where the platform can load the shared lib (same gates as jit_exe.das: // Windows lacks RPATH; static builds ship no runtime lib for the exe to call into) - if (get_platform_name() != "windows" && das_is_dll_build() && fexist("{KERNEL_EXE}.exe")) { + let can_run = get_platform_name() != "windows" && das_is_dll_build() + if (can_run && fexist("{KERNEL_EXE}.exe")) { var lr : array spawn_child("\"{KERNEL_EXE}.exe\"", lr) t |> success(lr |> has("HOSTF=[1]"), "the exe must bake the HOST feature rail, not generic (0) or reference (-1)") } + if (can_run && fexist("{BASELINE_EXE}.exe")) { + var lr : array + spawn_child("\"{BASELINE_EXE}.exe\"", lr) + t |> success(lr |> has(BASELINE_ANSWER), "the baseline exe computes the fallback bodies' answers: {join(lr, "\n")}") + } } From 9b7be379fa169d9fa1f2d4434d37230e65d1f0e2 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 20:26:57 -0700 Subject: [PATCH 11/43] stage 2 of the portable tier: dasllama_aot, the daslang host with the CPU engine's C++ stubs linked in, reproduces the JIT reference run token-exact on SmolLM2; five general AOT fixes rode along examples/dasLLAMA/CMakeLists.txt builds dasllama_aot (opt-in): DAS_AOT_LIB over the engine minus the Vulkan and Metal tiers, the LLVM IR generators and the exchange server, plus the daslib modules the CPU path calls that libDaScriptAot does not carry. Run it as bin/dasllama_aot -use-aot examples/dasLLAMA/run.das -- . What the engine needed from the rest of the tree: - -use-aot now compiles the script with policies.aot (and tune_frozen, the generator's policy) so the linked stubs bind; a function with no stub interprets. Only dastest honored the flag before. The per-PR AOT subset gate stays green (1724/1724). - guard_interp_inference and guard_interp_gguf_load read the AOT tier through aot_kernels_linked(): is_aot_function on one kernel, a runtime probe. aot_enabled() folds differently under generation and the consuming run and would desync every caller's hash. ARCHITECTURE_INVARIANTS.md sec.3 carries the adjusted rule. - das_accelerate had no aotRequire; a C++ module without one AOT-disables every das module whose require closure reaches it, silently - the engine above dasllama_math_accelerate emitted empty TUs. The module now ships dasAccelerate.h. - topoSortStructures read a function type's parameter types as by-value dependencies, saw a false cycle and fell back to source order, where C++ embeds an incomplete type. collectStructDeps skips function, lambda and block types; a residual container cycle keeps the sorted prefix. Gate: tests/aot/test_struct_order.das. - das_is_aot_function / das_is_jit_function and the jobque affinity pair are declared in the AOT headers the generated C++ includes. Decode runs 54 t/s against the JIT reference policy's 360 on the same model - the vectorization leg the plan names, not a measurement. Co-Authored-By: Claude Fable 5.1 --- CMakeLists.txt | 1 + examples/dasLLAMA/CMakeLists.txt | 65 +++++++++++++++++++ include/daScript/simulate/aot_builtin.h | 2 + .../daScript/simulate/aot_builtin_jobque.h | 2 + modules/dasAccelerate/src/dasAccelerate.cpp | 33 +++++----- modules/dasAccelerate/src/dasAccelerate.h | 20 ++++++ modules/dasLLAMA/ARCHITECTURE_IMAGE.md | 4 +- modules/dasLLAMA/ARCHITECTURE_INVARIANTS.md | 27 ++++---- modules/dasLLAMA/CMakeLists.txt | 3 +- modules/dasLLAMA/dasllama/dasllama_image.das | 2 +- modules/dasLLAMA/dasllama/dasllama_math.das | 17 +++-- modules/dasLLAMA/portable_tier_plan.md | 53 +++++++++++---- skills/internal/build_and_debug.md | 1 + src/ast/ast_program.cpp | 13 +++- tests/README.md | 1 + tests/aot/CMakeLists.txt | 8 +-- tests/aot/test_struct_order.das | 33 ++++++++++ utils/daslang/main.cpp | 14 ++-- 18 files changed, 245 insertions(+), 54 deletions(-) create mode 100644 examples/dasLLAMA/CMakeLists.txt create mode 100644 modules/dasAccelerate/src/dasAccelerate.h create mode 100644 tests/aot/test_struct_order.das diff --git a/CMakeLists.txt b/CMakeLists.txt index 06a9a9b819..4b17962023 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1446,6 +1446,7 @@ if (NOT ${DAS_TESTS_DISABLED}) # AOT examples are enabled — gate the inclusion accordingly. if(NOT ${DAS_AOT_EXAMPLES_DISABLED}) include(examples/pathTracer/CMakeLists.txt) + include(examples/dasLLAMA/CMakeLists.txt) endif() if(NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 4)) include(tests/aot/CMakeLists.txt) diff --git a/examples/dasLLAMA/CMakeLists.txt b/examples/dasLLAMA/CMakeLists.txt new file mode 100644 index 0000000000..c73c29d5da --- /dev/null +++ b/examples/dasLLAMA/CMakeLists.txt @@ -0,0 +1,65 @@ +### dasLLAMA AOT host: the daslang command-line binary with the engine's CPU modules compiled +### to C++ ahead of time and linked in - the portable tier without the JIT. +### +### cmake --build build --target dasllama_aot +### bin/dasllama_aot -use-aot examples/dasLLAMA/run.das -- +### +### Opt-in (EXCLUDE_FROM_ALL): ~90 AOT TUs. The GPU tiers, the LLVM IR generators and the +### exchange server are not AOT'd - they interpret as scaffolding (the Vulkan classes alone +### emit 40 MB of C++), and a call from AOT'd code into them is inlined C++ regardless. + +IF(DAS_LLAMA_INCLUDED AND NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 4)) + FILE(GLOB DASLLAMA_AOT_ENGINE_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "modules/dasLLAMA/dasllama/*.das") + list(FILTER DASLLAMA_AOT_ENGINE_FILES EXCLUDE REGEX "dasllama_(vulkan|metal|math_vulkan|gemm_gen|gemm_register|exchange)") + + # the daslib modules the CPU path calls at run time that libDaScriptAot does not carry + SET(DASLLAMA_AOT_DASLIB_FILES + daslib/aarch64_neon.das + daslib/x64_avx.das + daslib/f16_cvt.das + daslib/jobque_boost.das + daslib/jobque_profile.das + daslib/fio.das + daslib/array_boost.das + daslib/math_bits.das + daslib/sha_256.das + daslib/archive.das + daslib/clargs.das + daslib/env_registry.das + daslib/tty.das + daslib/toml.das + daslib/safe_addr.das + ) + + # an edit to any engine module changes its dependents' semantic hashes: every engine TU + # regenerates on every engine edit (the daslib glob is a global dependency already). The + # generator reads each C++ module's aotRequire out of the loaded shared module, so those + # are inputs too - a module without the override AOT-disables its whole require closure + FILE(GLOB DASLLAMA_AOT_ALL_ENGINE_FILES CONFIGURE_DEPENDS "${PROJECT_SOURCE_DIR}/modules/dasLLAMA/dasllama/*.das") + SET(DAS_AOT_EXTRA_DEPENDS ${DASLLAMA_AOT_ALL_ENGINE_FILES} ${DAS_DYN_MODULES_LIBS}) + add_custom_target(dasllama_aot_engine) + SET(DASLLAMA_AOT_ENGINE_GENERATED_SRC) + DAS_AOT_LIB("${DASLLAMA_AOT_ENGINE_FILES}" DASLLAMA_AOT_ENGINE_GENERATED_SRC dasllama_aot_engine daslang) + add_custom_target(dasllama_aot_daslib) + SET(DASLLAMA_AOT_DASLIB_GENERATED_SRC) + DAS_AOT_LIB("${DASLLAMA_AOT_DASLIB_FILES}" DASLLAMA_AOT_DASLIB_GENERATED_SRC dasllama_aot_daslib daslang) + unset(DAS_AOT_EXTRA_DEPENDS) + SOURCE_GROUP_FILES("aot generated" DASLLAMA_AOT_ENGINE_GENERATED_SRC) + SOURCE_GROUP_FILES("aot generated" DASLLAMA_AOT_DASLIB_GENERATED_SRC) + + add_executable(dasllama_aot EXCLUDE_FROM_ALL ${DAS_DASCRIPT_MAIN_SRC} + ${DASLLAMA_AOT_ENGINE_GENERATED_SRC} + ${DASLLAMA_AOT_DASLIB_GENERATED_SRC} + ) + target_sources(dasllama_aot PRIVATE ${PROJECT_SOURCE_DIR}/src/misc/alloc_tracker_overrides.cpp) + TARGET_LINK_LIBRARIES(dasllama_aot libDaScriptAot ${SRC_LIBRARIES} ${DAS_MODULES_LIBS}) + ADD_DEPENDENCIES(dasllama_aot libDaScriptAot dasllama_aot_engine dasllama_aot_daslib) + # the engine's AOT TUs include the C++ headers of every module in their require closure + # (dasAudio's miniaudio, dasMinfft, dasVulkan's volk) - test_aot's dirs and defines plus volk + target_include_directories(dasllama_aot PRIVATE ${NEED_MODULES_PATH} ${AUDIO_INCLUDE_DIR} ${CIPIC_HRTF_INCLUDE_DIR} + ${PROJECT_SOURCE_DIR}/modules/dasMinfft/minfft + ${PROJECT_SOURCE_DIR}/modules/dasVulkan/vendor/volk + ${PROJECT_SOURCE_DIR}/modules/dasVulkan/vendor/Vulkan-Headers/include) + target_compile_definitions(dasllama_aot PRIVATE MINFFT_SINGLE HRTF_SAMPLE_RATE=${DAS_AUDIO_SAMPLE_RATE}) + SETUP_CPP11(dasllama_aot) +ENDIF() diff --git a/include/daScript/simulate/aot_builtin.h b/include/daScript/simulate/aot_builtin.h index 3a04da7b5f..abf86f51ef 100644 --- a/include/daScript/simulate/aot_builtin.h +++ b/include/daScript/simulate/aot_builtin.h @@ -22,6 +22,8 @@ namespace das { DAS_API bool is_compiling ( ); DAS_API bool is_compiling_macros ( ); DAS_API bool is_standalone_exe ( ); + DAS_API bool das_is_jit_function ( const Func func ); + DAS_API bool das_is_aot_function ( const Func func ); DAS_API uint64_t get_context_share_counter ( Context * context ); DAS_API char * builtin_das_root ( Context * context, LineInfoArg * at ); diff --git a/include/daScript/simulate/aot_builtin_jobque.h b/include/daScript/simulate/aot_builtin_jobque.h index 4397436649..447d1256c8 100644 --- a/include/daScript/simulate/aot_builtin_jobque.h +++ b/include/daScript/simulate/aot_builtin_jobque.h @@ -276,6 +276,8 @@ namespace das { DAS_API int32_t get_jobque_worker_limit ( Context * context, LineInfoArg * at ); DAS_API void set_jobque_team_rank_gate ( bool on, Context * context, LineInfoArg * at ); DAS_API bool get_jobque_team_rank_gate ( Context * context, LineInfoArg * at ); + DAS_API void setJobqueAffinity ( int32_t mode ); + DAS_API int32_t getJobqueAffinity (); DAS_API uint64_t count_jobque_leaks(); DAS_API void new_job_invoke ( Lambda lambda, Func fn, int32_t lambdaSize, Context * context, LineInfoArg * lineinfo ); DAS_API void set_jobque_fork_pool ( bool keep, bool skipInit, Context * context, LineInfoArg * at ); diff --git a/modules/dasAccelerate/src/dasAccelerate.cpp b/modules/dasAccelerate/src/dasAccelerate.cpp index f2724bf583..5fe8965f04 100644 --- a/modules/dasAccelerate/src/dasAccelerate.cpp +++ b/modules/dasAccelerate/src/dasAccelerate.cpp @@ -7,8 +7,9 @@ #include #include "daScript/daScript.h" +#include "dasAccelerate.h" -using namespace das; +namespace das { // BLASSetThreading/BLASGetThreading exist only in the macOS 15 / iOS 18 SDK headers — the // __builtin_available checks below gate RUNTIME, but an older SDK fails at COMPILE time @@ -37,10 +38,10 @@ static inline void accel_pin_single_thread() { // C[m x n] = A[m x k] * B[n x k]^T, row-major — the exact call shape ggml-blas.cpp uses for // mul_mat (activations A stay token-major, weights B stay row-major, no pre-transpose). -static void accel_sgemm_nt(int32_t m, int32_t n, int32_t k, - const float * a, int32_t lda, - const float * b, int32_t ldb, - float * c, int32_t ldc) { +void accel_sgemm_nt(int32_t m, int32_t n, int32_t k, + const float * a, int32_t lda, + const float * b, int32_t ldb, + float * c, int32_t ldc) { accel_pin_single_thread(); cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans, m, n, k, @@ -51,9 +52,9 @@ static void accel_sgemm_nt(int32_t m, int32_t n, int32_t k, // y[n] = W[n x k] * x[k], row-major — the decode-lane probe (uncrowned on M1; here for the // M4/M5-SME applicability sweep, never gated out). -static void accel_sgemv_n(int32_t n, int32_t k, - const float * w, int32_t ldw, - const float * x, float * y) { +void accel_sgemv_n(int32_t n, int32_t k, + const float * w, int32_t ldw, + const float * x, float * y) { accel_pin_single_thread(); cblas_sgemv(CblasRowMajor, CblasNoTrans, n, k, 1.0f, w, ldw, x, 1, 0.0f, y, 1); } @@ -64,10 +65,10 @@ static void accel_sgemv_n(int32_t n, int32_t k, // Filter create+destroy is ~0.2us (bnns_hgemm_probe), so per-call creation is free and no // pointer-lifetime coupling exists. rc: 0 = ok, nonzero = BNNS unavailable or refused — // the caller falls back to the portable path. -static int32_t accel_bnns_hgemm_nt(int32_t m, int32_t n, int32_t k, - const uint16_t * a, int32_t lda, - const uint16_t * b, int32_t ldb, - float * c, int32_t ldc) { +int32_t accel_bnns_hgemm_nt(int32_t m, int32_t n, int32_t k, + const uint16_t * a, int32_t lda, + const uint16_t * b, int32_t ldb, + float * c, int32_t ldc) { if (__builtin_available(macOS 11.0, iOS 14.0, tvOS 14.0, watchOS 7.0, *)) { #pragma clang diagnostic push #pragma clang diagnostic ignored "-Wdeprecated-declarations" // classic BNNS: deprecated 15.0 for BNNSGraph, still the only C-callable dynamic matmul @@ -99,7 +100,7 @@ static int32_t accel_bnns_hgemm_nt(int32_t m, int32_t n, int32_t k, // Calling thread's effective BLAS threading mode after the pin: 0 = multi, 1 = single, // -1 = BLASSetThreading unavailable (pre-macOS-15). Diagnostics for the contention rig. -static int32_t accel_threading_mode() { +int32_t accel_threading_mode() { accel_pin_single_thread(); #if DAS_ACCEL_HAS_BLAS_THREADING if (__builtin_available(macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, *)) { @@ -109,8 +110,6 @@ static int32_t accel_threading_mode() { return -1; } -namespace das { - class Module_DasAccelerate : public Module { public: Module_DasAccelerate() : Module("das_accelerate") { @@ -131,6 +130,10 @@ class Module_DasAccelerate : public Module { addExtern(*this, lib, "accel_threading_mode", SideEffects::modifyExternal, "accel_threading_mode"); } + virtual ModuleAotType aotRequire ( TextWriter & tw ) const override { + tw << "#include \"../modules/dasAccelerate/src/dasAccelerate.h\"\n"; + return ModuleAotType::cpp; + } }; REGISTER_DYN_MODULE(Module_DasAccelerate, Module_DasAccelerate); diff --git a/modules/dasAccelerate/src/dasAccelerate.h b/modules/dasAccelerate/src/dasAccelerate.h new file mode 100644 index 0000000000..2580b57efc --- /dev/null +++ b/modules/dasAccelerate/src/dasAccelerate.h @@ -0,0 +1,20 @@ +#pragma once + +#include "daScript/misc/platform.h" + +// the das_accelerate binding surface, declared for AOT-generated C++ (Module::aotRequire) + +namespace das { + DAS_MOD_API void accel_sgemm_nt(int32_t m, int32_t n, int32_t k, + const float * a, int32_t lda, + const float * b, int32_t ldb, + float * c, int32_t ldc); + DAS_MOD_API void accel_sgemv_n(int32_t n, int32_t k, + const float * w, int32_t ldw, + const float * x, float * y); + DAS_MOD_API int32_t accel_bnns_hgemm_nt(int32_t m, int32_t n, int32_t k, + const uint16_t * a, int32_t lda, + const uint16_t * b, int32_t ldb, + float * c, int32_t ldc); + DAS_MOD_API int32_t accel_threading_mode(); +} diff --git a/modules/dasLLAMA/ARCHITECTURE_IMAGE.md b/modules/dasLLAMA/ARCHITECTURE_IMAGE.md index d2d97a3af4..110951f8c0 100644 --- a/modules/dasLLAMA/ARCHITECTURE_IMAGE.md +++ b/modules/dasLLAMA/ARCHITECTURE_IMAGE.md @@ -77,7 +77,9 @@ A gguf load's O(model) transform loops run about ten times slower interpreted - takes 53 s against 5.5 s jitted, and a 69 GB hybrid extrapolates to an hour (the repack itself is native tune kernels and costs the same either way). A prepared image costs nothing interpreted, because mapping and borrowing planes runs no such loop. That asymmetry is why the guard fires on -the gguf path and never on the image path. +the gguf path and never on the image path. The guard passes on every compiled tier - the JIT, a +standalone exe, and a host that linked the engine's AOT stubs (`aot_kernels_linked`, the probe +`guard_interp_inference` shares) - because the transform loops are native on each. ### 2.1e Publishing an image {#image-publish} diff --git a/modules/dasLLAMA/ARCHITECTURE_INVARIANTS.md b/modules/dasLLAMA/ARCHITECTURE_INVARIANTS.md index 9fb572e231..947e982d67 100644 --- a/modules/dasLLAMA/ARCHITECTURE_INVARIANTS.md +++ b/modules/dasLLAMA/ARCHITECTURE_INVARIANTS.md @@ -9,17 +9,22 @@ Durable "why it is built this way" facts harvested from the design docs archived **From `x64_arch.md`:** -- **The LIBRARY never runs interpreted or AOT - the scaffolding may.** Inference - anything that - runs a model (eval / decode / prefill / generate) - is `-jit`-only, hard stop: an interpreted or - AOT model run is not "slow but correct", it is out of contract, and the library panics on it. - Loop hints and intrinsic lowering exist only in the JIT, and the JIT tier is the only one the - oracles gate. **Scaffolding is exempt:** conversion utilities (`.dlim` bake, image processing), - debug scripts, and batch/driver tools that spawn the real runner as a child may run interpreted. - Enforcement is `guard_interp_gguf_load` (`dasllama_image.das` - big-load panic + - `DASLLAMA_ALLOW_INTERP_LOAD` escape for the conversion path) plus `guard_interp_inference` - (`dasllama_math.das`), which panics at the library seams: `make_run_state`, ASR session - creation, and `vad_chunk_prob`. Intrinsic fallback bodies exist for off-ARCH correctness - *inside* the JIT, not as runnable tiers. +- **The LIBRARY never runs interpreted - the scaffolding may.** Inference - anything that runs + a model (eval / decode / prefill / generate) - runs on a compiled tier, hard stop: an + interpreted model run is not "slow but correct", it is out of contract, and the library panics + on it. Three tiers pass: the JIT (the crown - loop hints, intrinsic lowering and the tuned + kernels exist only there, and it is the tier the oracles gate), a standalone `-exe`, and a host + that linked the engine's AOT stubs (the portable tier: the reference bodies as the C++ the AOT + emitter produced, `examples/dasLLAMA`, `-use-aot`). **Scaffolding is exempt:** conversion + utilities (`.dlim` bake, image processing), debug scripts, and batch/driver tools that spawn + the real runner as a child may run interpreted. Enforcement is `guard_interp_gguf_load` + (`dasllama_image.das` - big-load panic + `DASLLAMA_ALLOW_INTERP_LOAD` escape for the + conversion path) plus `guard_interp_inference` (`dasllama_math.das`), which panics at the + library seams: `make_run_state`, ASR session creation, and `vad_chunk_prob`. Both read the + AOT tier through `aot_kernels_linked` - a runtime probe of one kernel's binding, never + `aot_enabled()`, which folds at compile time and desyncs the semantic hash between AOT + generation and the AOT run. Intrinsic fallback bodies are what the AOT tier runs where the + JIT would lower an instruction; inside the JIT they exist for off-ARCH correctness. - **Correctness before speed, token-for-token.** The engine is validated against external oracles (llama2.c + upstream `simple_ids`) plus per-arch parity fixtures. A new kernel passes the suite *and* the oracles with the new backend active before any perf claim. diff --git a/modules/dasLLAMA/CMakeLists.txt b/modules/dasLLAMA/CMakeLists.txt index b6c7606089..5043067217 100644 --- a/modules/dasLLAMA/CMakeLists.txt +++ b/modules/dasLLAMA/CMakeLists.txt @@ -4,7 +4,8 @@ IF(NOT DAS_LLAMA_INCLUDED) ADD_MODULE_DAS_FROM_DESCRIPTOR(dasllama dasllama) - # dasLLAMA IS NEVER AOT'D — -jit only; no AOT source/depends exports (see tests/aot/CMakeLists.txt). + # dasLLAMA's tests are never in test_aot (see tests/aot/CMakeLists.txt); the engine's AOT + # tier is the opt-in examples/dasLLAMA target, which AOTs the CPU modules itself. install(DIRECTORY ${PROJECT_SOURCE_DIR}/modules/dasLLAMA/dasllama DESTINATION ${DAS_INSTALL_MODULESDIR}/dasLLAMA diff --git a/modules/dasLLAMA/dasllama/dasllama_image.das b/modules/dasLLAMA/dasllama/dasllama_image.das index a92440a09b..ae6c997a64 100644 --- a/modules/dasLLAMA/dasllama/dasllama_image.das +++ b/modules/dasLLAMA/dasllama/dasllama_image.das @@ -1525,7 +1525,7 @@ def private cache_via_image_(var m; var out; img : string; tag : string; quant : [arch(at = "../ARCHITECTURE_IMAGE.md#image-interp-load")] def private guard_interp_gguf_load(path : string) { - if (jit_enabled() || is_standalone_exe()) { + if (jit_enabled() || is_standalone_exe() || aot_kernels_linked()) { return } var st : FStat diff --git a/modules/dasLLAMA/dasllama/dasllama_math.das b/modules/dasLLAMA/dasllama/dasllama_math.das index 2f7270e938..a40cda8247 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math.das +++ b/modules/dasLLAMA/dasllama/dasllama_math.das @@ -25,11 +25,20 @@ require daslib/f16_cvt // f16<->f32 converts: JIT half intrinsics on aarch64 / // can be passed zero-copy as views into one big model blob and outputs can be written // straight into a view (e.g. the KV cache). Dims/indices are int64 for large models. -//! The library-seam ``-jit`` stop: dasLLAMA inference NEVER runs interpreted (kernels 10-40x -//! slower). Inference seams call it; scaffolding that only parses or peeks never crosses it. +//! True when the engine's kernels are the C++ the AOT emitter produced - the host linked this +//! module's stubs (examples/dasLLAMA, `-use-aot`). A runtime probe of one kernel's binding, not +//! `aot_enabled()`: that one folds at compile time, differently under AOT generation and the +//! AOT-consuming run, and would desync every caller's semantic hash (error[50101]). +def aot_kernels_linked() : bool { + return is_aot_function(@@ < (var x : float?; size : int64) : void > silu) +} + +//! The library-seam compiled-tier stop: dasLLAMA inference NEVER runs interpreted (kernels 10-40x +//! slower). The JIT, a standalone exe, and linked AOT stubs each pass; inference seams call it, +//! scaffolding that only parses or peeks never crosses it. def guard_interp_inference(what : string) { - if (!jit_enabled() && !is_standalone_exe()) { - panic("dasLLAMA: {what} requires -jit (or a compiled exe) - interpreted inference runs the kernels 10-40x slower") + if (!jit_enabled() && !is_standalone_exe() && !aot_kernels_linked()) { + panic("dasLLAMA: {what} requires -jit, a compiled exe, or an AOT host with the engine's stubs (-use-aot) - interpreted inference runs the kernels 10-40x slower") } } diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index be3a647d9d..32e31fee96 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -253,17 +253,48 @@ the framework short-circuited is green on the stage-0 gates and parity; the floo ## Stage 2 - an AOT compilation target under `examples/dasLLAMA/` -- One example program (a small-LLM chat, then ASR and TTS twins) registered as an AOT target, - the first exception to `modules/dasLLAMA/CMakeLists.txt:7` ("never AOT'd"), scoped to that - example. -- `guard_interp_inference` accepts `aot_enabled()` beside `jit_enabled()` and - `is_standalone_exe()`; `guard_interp_gguf_load` likewise. This is the rule adjustment the - invariants doc gets in the same change. -- The stamped-function hash question is moot under `-aot` (`tune_frozen`), and with the - stage-1 shells the AOT emitter sees ordinary functions. -- Expect heavy TUs and a long C++ compile: the engine closure is large. Known AOT traps from - earlier arcs to watch first: int64 pointer indexing, managed-vector for-in. -- Parity against the stage-1 JIT run on the same model is the pass criterion, ids exact. +**Status: the target exists and the exit's parity half holds (2026-09-04).** `examples/dasLLAMA/CMakeLists.txt` +builds `dasllama_aot` (opt-in, `EXCLUDE_FROM_ALL`): the daslang host with every CPU engine +module's C++ stubs linked in (`DAS_AOT_LIB` over `dasllama/*.das` minus the Vulkan and Metal +tiers, the LLVM IR generators and the exchange server, which interpret as scaffolding) plus the +daslib modules the CPU path calls that `libDaScriptAot` does not carry. `bin/dasllama_aot -use-aot +examples/dasLLAMA/run.das -- SmolLM2-135M-Instruct-Q8_0.gguf` reproduces the `-jit` reference-policy +run's 64 greedy tokens exactly. The run links 4637 functions; the 1178 it does not are the excluded +Metal tier plus 38 generic instantiations attributed to `json` / `strings_boost` / `strings_convert` +(libDaScriptAot's stubs, minted from another root - harmless, they interpret). + +What it took, each a general fix rather than a dasLLAMA one: + +- `-use-aot` now does what its help text says: the host compiles the script with `policies.aot` + (and `tune_frozen`, the generator's policy) so the linked stubs bind; a function with no stub + interprets (`fail_on_no_aot` stays off in the host). Before, only dastest honored the flag. +- The guards read the AOT tier through `aot_kernels_linked()` - `is_aot_function` on one kernel + (`dasllama_math::silu`), a runtime probe - never `aot_enabled()`, which folds differently under + generation and the consuming run and would desync every caller's hash. `ARCHITECTURE_INVARIANTS.md` + sec.3 carries the adjusted rule. +- `das_accelerate` had no `aotRequire`, and a C++ module without one AOT-disables every das + module whose require closure reaches it - the whole engine above `dasllama_math_accelerate` + emitted empty TUs (`// AOT disabled due to module requirements`) with no error anywhere. The + module now ships `dasAccelerate.h`. Lint candidate: every in-tree C++ module class overrides + `aotRequire` (a tests-cpp or CMake-time gate). +- The emitter's topological structure sort read a function type's parameter types as by-value + dependencies, saw a false cycle (`BatchWorkspace` embeds `Session`; a `function<(...BatchWorkspace...)>` + typedef names it) and fell back to source order - C++ then embedded an incomplete type. + `collectStructDeps` skips function / lambda / block types, and a residual (container-mediated) + cycle keeps the sorted prefix. Gate: `tests/aot/test_struct_order.das`. +- `das_is_aot_function` / `das_is_jit_function` and the jobque affinity pair had no AOT header + declaration (`aot_builtin.h`, `aot_builtin_jobque.h`); the generated C++ named them undeclared. +- The AOT TUs include every closure module's C++ header, so the target compiles with dasAudio's, + dasMinfft's and dasVulkan's (volk) include dirs; the generator reads `aotRequire` out of the + loaded shared modules, so those are regeneration inputs too (`DAS_AOT_EXTRA_DEPENDS`). + +The exit's numbers half is open: on the M5 box, contended by a build, the AOT decode ran 54 t/s +against the JIT reference policy's 360 t/s on the same model (prefill 74 vs 617) - not a +measurement, a gap of the size the vectorization leg predicts (clang over the emitted loop +shapes versus the JIT's auto-vectorizer). Next: the same rail for the ASR and TTS examples, a +model-free gate for the AOT target (CI has no models; the build itself is the compile gate, and a +synthetic-weights run is the cheapest runtime one), Linux and Windows builds of the target, and the +stage-2 profile that decides which kernels the vectorization leg rewrites first. **Exit:** the example compiles and runs the small model with exact parity to the JIT run; its decode/prefill numbers sit beside the stage-1 floor. diff --git a/skills/internal/build_and_debug.md b/skills/internal/build_and_debug.md index 6393f22bda..7abd7d5951 100644 --- a/skills/internal/build_and_debug.md +++ b/skills/internal/build_and_debug.md @@ -19,6 +19,7 @@ The repo builds on **Windows, Linux, macOS, iOS, Android, and WASM** (CI runs th - **Compile-only check:** ` -compile-only path/to/script.das` - compiles without simulation or execution, useful for syntax/type checking without needing a window or GL context. Use `-dry-run` to also simulate (but not execute). - **Run tests:** ` dastest/dastest.das -- --test path/to/test.das` - **AOT tests (full):** `cmake --build build --config Release --target test_aot` then ` -use-aot dastest/dastest.das -- --use-aot --test tests`. The full binary is `EXCLUDE_FROM_ALL` (~1080 AOT TUs); the default build only makes `test_aot_subset` (tests/language - the per-PR CI gate; `--target run_tests_aot_subset` sweeps it) +- **dasLLAMA on the AOT tier:** `cmake --build build --target dasllama_aot` (opt-in, `EXCLUDE_FROM_ALL`, ~90 engine TUs) then `bin/dasllama_aot -use-aot examples/dasLLAMA/run.das -- `. The host is `daslang` with the CPU engine's C++ stubs linked in; `-use-aot` compiles the script with `policies.aot` so they bind, and a function without a stub interprets. `options log_aot` in the script root lists every function as linked or `NOT FOUND` with its hash - the coverage probe for a stub set - **IMPORTANT:** When adding a new test directory under `tests/`, register it in `tests/aot/CMakeLists.txt` for AOT compilation. See `skills/internal/aot_testing.md` for the step-by-step pattern. The nightly CI + `preflight --full` run ALL tests with AOT enabled - unregistered test directories cause `error[50101]: AOT link failed` there (per-PR CI won't catch it) This skill uses `bin/Release/daslang.exe` in examples below (the dominant local-dev case); substitute the right path on other platforms. diff --git a/src/ast/ast_program.cpp b/src/ast/ast_program.cpp index 6c59fde820..548833d608 100644 --- a/src/ast/ast_program.cpp +++ b/src/ast/ast_program.cpp @@ -511,6 +511,8 @@ namespace das { static void collectStructDeps ( const TypeDeclPtr & type, Structure * owner, das_hash_set & deps ) { if ( !type ) return; + // a function, lambda or block value is a handle: the types in its signature are not stored by value + if ( type->baseType == Type::tFunction || type->baseType == Type::tLambda || type->baseType == Type::tBlock ) return; if ( type->baseType == Type::tStructure && type->structType && type->structType != owner ) { if ( type->isPointer() ) return; // pointers don't need full definition if ( !deps.insert(type->structType).second ) return; // already visited @@ -569,7 +571,16 @@ namespace das { } } } - if ( sorted.size() != structs.size() ) return; // cycle - keep original order + // a by-value cycle cannot exist (the layout would be infinite); what remains unsorted is a + // container-mediated cycle, which needs no definition order - it keeps the original order + // after everything the sort did place + if ( sorted.size() != structs.size() ) { + das_hash_set placed; + for ( auto s : sorted ) placed.insert(s); + for ( auto & sp : structs ) { + if ( !placed.count(sp) ) sorted.push_back(sp); + } + } // reorder structs to match sorted order das_hash_map byPtr; for ( auto & sp : structs ) byPtr[sp] = sp; diff --git a/tests/README.md b/tests/README.md index 875c1d6596..050b532062 100644 --- a/tests/README.md +++ b/tests/README.md @@ -44,6 +44,7 @@ Every `.das` file in this directory tree is listed below, grouped by subdirector | test_range64_ctor_args.das | AOT most-vexing-parse - a for-source of constructor-style casts only (`range64(int64(a), int64(b))`) must emit a variable, not a function declaration | | | test_op2_operand_order_emit.das | AOT binary-op operand sequencing - impure-operand ops wrap in `das_ordered2` (braced init = left-to-right), pure and short-circuit ops don't | | | _ordered_op2_fixture.das | *(helper)* the op2 shapes the sequencing test emits - impure policy/plain/xor, pure, `&&`/`\|\|` | | +| test_struct_order.das | AOT structure emission order - an outer struct embeds a later-declared inner whose function-typed field names the outer in its signature; a signature is not a by-value dependency, so the topological sort must still place the inner first | | ## apply/ diff --git a/tests/aot/CMakeLists.txt b/tests/aot/CMakeLists.txt index 9c69c6c8f3..3e1a08d586 100644 --- a/tests/aot/CMakeLists.txt +++ b/tests/aot/CMakeLists.txt @@ -51,10 +51,10 @@ FILE(GLOB AOT_DASPEG_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "tes FILE(GLOB AOT_DASGLTF_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "tests/dasGLTF/*.das") -# dasLLAMA IS NEVER AOT'D. No dasLLAMA test — old or new — joins test_aot: the engine is -# -jit only (its AOT story ends at the generic compile of the tree), and its require graph -# reaches dasVulkan, which is in-tree but carries no AOT registration at all, so a dasLLAMA -# AOT TU would have no Vulkan half to link against. Do not re-register it here. +# No dasLLAMA test — old or new — joins test_aot: the suite is -jit (tests/run.das), and its +# require graph reaches dasVulkan, which is in-tree but carries no AOT registration at all, so +# a dasLLAMA test TU would have no Vulkan half to link against. Do not re-register it here. +# The engine's own AOT tier is the opt-in examples/dasLLAMA target (CPU modules only). FILE(GLOB AOT_DECS_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "tests/decs/*.das") # Exclude expect-error tests (they intentionally fail to compile) diff --git a/tests/aot/test_struct_order.das b/tests/aot/test_struct_order.das new file mode 100644 index 0000000000..b8cbc4ac2a --- /dev/null +++ b/tests/aot/test_struct_order.das @@ -0,0 +1,33 @@ +options gen2 +options no_unused_function_arguments = false +require dastest/testing_boost public + +// AOT emits structures in by-value dependency order, not source order: Outer embeds Inner, +// which is declared later, and Inner's function-typed field names Outer in its signature. A +// sort that read the signature as a by-value dependency saw a cycle and fell back to source +// order, where the generated C++ embeds an incomplete type. The AOT build of this file is the +// gate; the assertions only keep the shapes alive. (An `array` field inside Inner would +// be a real C++ cycle - TArray takes sizeof(T) - so that shape stays out of the test.) + +struct Outer { + inner : Inner + tag : int +} + +struct Inner { + on_outer : function<(o : Outer) : int> + weight : float +} + +def outer_tag(o : Outer) : int { + return o.tag * 2 +} + +[test] +def test_struct_order(t : T?) { + t |> run("an outer struct embeds a later-declared inner whose signature names the outer") @(t : T?) { + var o = Outer(tag = 21, inner = Inner(on_outer = @@outer_tag, weight = 0.5)) + t |> equal(invoke(o.inner.on_outer, o), 42) + t |> equal(o.inner.weight, 0.5) + } +} diff --git a/utils/daslang/main.cpp b/utils/daslang/main.cpp index a41e01d831..3aebbb9f77 100644 --- a/utils/daslang/main.cpp +++ b/utils/daslang/main.cpp @@ -469,12 +469,16 @@ int compile_and_run ( const string & fn, const string & mainFnName, bool outputP // (--ast-verify-batch keeps only the post-infer one, read from argv by the module). access->addExtraModule("ast_verify", getDasRoot() + "/daslib/ast_verify.das"); } + // -use-aot links the stubs compiled into THIS binary against the script it runs; a function + // with no stub interprets (never fail_on_no_aot here - the host may be dastest, whose own + // framework has no stubs, and which sets both policies for the test files it compiles) + policies.aot = useAot; + policies.fail_on_no_aot = false; if ( useAot ) { - // don't set policies.aot here - the host program (e.g. dastest) doesn't need AOT linking - // the --use-aot flag (after --) tells dastest to enable AOT for test files it compiles - policies.fail_on_no_aot = false; - } else { - policies.fail_on_no_aot = false; + // the stubs were generated tune-frozen (a cross-box artifact carries no per-box [tune] + // stamps); a stamped compile here would hash every tune-family kernel differently and + // link none of them + policies.tune_frozen = true; } policies.fail_on_lack_of_aot_export = false; policies.aot_macros = aotMacros; // -aot-macros: force quote lowering (daslib/quote) in a normal run From d4bfb516dde06f020dedbfe8b65bf7a336cd3ad0 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 20:36:43 -0700 Subject: [PATCH 12/43] typeinfo module_exists(X): the compiling program's own library, so a path-guarded das target answers the same inside a tool's nested compile; builtin_module_exists keeps its process-registry meaning builtin_module_exists reads the process's registered and promoted modules. A running script promotes the shared das modules it requires, a nested compile (lint, the language server, a test harness) promotes nothing, so every static_if over a path-guarded das target took the framework arm at run time and the no-framework arm under every tool. The earlier one-line widening of builtin_module_exists is reverted; the new trait asks program->library instead. Every path-guarded das site in the tree (llvm_tune, llvm_code, dasllama_exchange, dasllama_gemm_gen) now asks module_exists; C++-module guards are unchanged. Gate: tests/language/optional_require.das. Docs and the skill references carry both traits. Co-Authored-By: Claude Fable 5.1 --- daslib/tune.das | 19 +++++++------- .../language/generic_programming.rst | 3 ++- doc/source/reference/language/modules.rst | 7 ++++-- modules/dasLLAMA/benchmarks/lcpp_bench.das | 4 +-- .../dasLLAMA/dasllama/dasllama_math_gen.das | 2 +- modules/dasLLAMA/dasllama/dasllama_tune.das | 12 ++++----- modules/dasLLAMA/portable_tier_plan.md | 25 ++++++++----------- modules/dasLLVM/daslib/llvm_user_modules.das | 4 +-- skills/das_macros.md | 3 ++- skills/daslang/references/generics.md | 4 ++- .../daslang/references/modules-and-stdlib.md | 4 ++- skills/internal/llvm_tune_internals.md | 2 +- src/ast/ast_infer_type.cpp | 21 ++++++++++++++-- tests/language/optional_require.das | 13 ++++++++++ utils/dasllama-server/main.das | 4 +-- utils/dasllama-server/openai_server.das | 12 ++++----- 16 files changed, 88 insertions(+), 51 deletions(-) diff --git a/daslib/tune.das b/daslib/tune.das index 4274f6889d..0b0ed7a2f1 100644 --- a/daslib/tune.das +++ b/daslib/tune.das @@ -15,8 +15,9 @@ require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public //! shells every program can spell. The framework that implements them lives in //! `llvm/daslib/llvm_tune` and `llvm/daslib/llvm_code`, reached through the path guards above //! and re-exported, so `require daslib/tune` gives exactly what `require llvm/daslib/llvm_tune` -//! gave; the guarded require is also the witness that makes `builtin_module_exists` below -//! answer for this compile. Every tune annotation registers HERE and nowhere else, because a +//! gave; the guarded require is also the witness that makes `module_exists` below answer for +//! this compile (`module_exists`, not `builtin_module_exists`: the program's own library, the +//! same answer inside a tool's nested compile). Every tune annotation registers HERE and nowhere else, because a //! module's macro state is per-module: the `[tune_scope]` bank and the pass that reads it have //! to run in one macro context. Without dasLLVM every apply is a no-op - no sidecar read, no //! stamp, no `_variants()` registry, no tuner spawn - and the reference body IS the function. @@ -26,7 +27,7 @@ require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public class private TunePermShell : AstFunctionAnnotation { def override apply(var func : FunctionPtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_perm_apply(func, group, args, errors) } else { return true @@ -38,7 +39,7 @@ class private TunePermShell : AstFunctionAnnotation { class private TuneCompanionShell : AstFunctionAnnotation { def override apply(var func : FunctionPtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_companion_apply(func, group, args, errors) } else { return true @@ -50,7 +51,7 @@ class private TuneCompanionShell : AstFunctionAnnotation { class private TuneShell : AstFunctionAnnotation { def override apply(var func : FunctionPtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_apply(func, group, args, errors) } else { return true @@ -62,7 +63,7 @@ class private TuneShell : AstFunctionAnnotation { class private TuneScopeShell : AstStructureAnnotation { def override apply(var st : StructurePtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_scope_apply(st, group, args, errors) } else { return true @@ -74,7 +75,7 @@ class private TuneScopeShell : AstStructureAnnotation { class private TunePolicyShell : AstFunctionAnnotation { def override apply(var func : FunctionPtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_policy_apply(func, group, args, errors) } else { return true @@ -86,7 +87,7 @@ class private TunePolicyShell : AstFunctionAnnotation { class private LlvmCodeShell : AstFunctionAnnotation { def override apply(var func : FunctionPtr; var group : ModuleGroup; args : AnnotationArgumentList; var errors : das_string) : bool { - static_if (typeinfo builtin_module_exists(llvm_code)) { + static_if (typeinfo module_exists(llvm_code)) { return llvm_code_apply(func, group, args, errors) } else { return true @@ -97,7 +98,7 @@ class private LlvmCodeShell : AstFunctionAnnotation { [infer_macro] class private TuneDefaultPolicyShell : AstPassMacro { def override apply(prog : ProgramPtr; mod : Module?) : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_default_policy_apply(prog, mod) } else { return false diff --git a/doc/source/reference/language/generic_programming.rst b/doc/source/reference/language/generic_programming.rst index 85bda003eb..c6e56c0435 100644 --- a/doc/source/reference/language/generic_programming.rst +++ b/doc/source/reference/language/generic_programming.rst @@ -152,7 +152,8 @@ All ``typeinfo`` traits can operate on either an expression or a ``type`` arg * ``typeinfo builtin_function_exists(expr)`` — true if a ``@@function`` exists * ``typeinfo builtin_annotation_exists(expr)`` — true if an annotation type exists -* ``typeinfo builtin_module_exists(expr)`` — true if a module is loaded +* ``typeinfo builtin_module_exists(expr)`` — true if a module is registered in the process: a linked C++ module, or a shared das module promoted by a running script +* ``typeinfo module_exists(expr)`` — true if the compiling program's own library holds the module, i.e. this program required it (directly or through another module); the answer is the same inside a tool's nested compile, which is what a ``require ?path`` witness needs * ``typeinfo is_argument(expr)`` — true if the expression is a function argument * ``typeinfo mangled_name(expr)`` — returns the mangled name of a ``@@function`` diff --git a/doc/source/reference/language/modules.rst b/doc/source/reference/language/modules.rst index 36f43fcad2..07a2401b00 100644 --- a/doc/source/reference/language/modules.rst +++ b/doc/source/reference/language/modules.rst @@ -99,8 +99,11 @@ unavailable, the require is skipped silently. This enables the contributor patte in ``llvm/daslib/llvm_user_modules.das``: a ``require ? `` line pulls a das package's registration glue exactly when that package is mounted. ``typeinfo builtin_module_exists`` additionally sees **shared das modules** -(``module X shared``) compiled earlier in the program, so the usual ``static_if`` -guard works for das-module contributors too. +(``module X shared``) promoted by the running script - but a tool that compiles the +same program in a nested context (lint, the language server, a test harness) +promotes nothing, and there the trait answers false for every das target. Guard a +path-guarded das target with ``typeinfo module_exists(target)`` instead: it asks the +compiling program's own library, so the answer is the same on both rails. Pair it with :ref:`typeinfo builtin_module_exists ` to guard code that uses the optional target's symbols — ``static_if`` drops the untaken diff --git a/modules/dasLLAMA/benchmarks/lcpp_bench.das b/modules/dasLLAMA/benchmarks/lcpp_bench.das index 25baf85d57..0648bc0644 100644 --- a/modules/dasLLAMA/benchmarks/lcpp_bench.das +++ b/modules/dasLLAMA/benchmarks/lcpp_bench.das @@ -1272,7 +1272,7 @@ var private g_debug_run = false // the exchange client compiles only where the tune framework does, so --submit has a rail to refuse on def private exchange_compiled_in() : bool { - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { return true } else { return false @@ -1857,7 +1857,7 @@ def main() : int { } print("{text}\n") if (cfg.submit) { - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { let url = empty(g_env_exchange.exchange_url) ? EXCHANGE_URL_DEFAULT : g_env_exchange.exchange_url let serr = exchange_submit_records_doc(url, text) if (!empty(serr)) { diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 7e84912dfa..104a84ae88 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -2750,7 +2750,7 @@ def dasllama_math_gen_register() { // line is unconditional-when-set — active overrides announce themselves if (g_env_bench.allow_untuned) { to_log(LOG_WARNING, "dasLLAMA: DASLLAMA_ALLOW_UNTUNED=1 - tune mint suppressed; untuned kernels run fallback, numbers are not publishable\n") - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { tune_suppress_mint("DASLLAMA_ALLOW_UNTUNED=1") } } diff --git a/modules/dasLLAMA/dasllama/dasllama_tune.das b/modules/dasLLAMA/dasllama/dasllama_tune.das index 9c50e6d419..a10a14bb1f 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tune.das +++ b/modules/dasLLAMA/dasllama/dasllama_tune.das @@ -282,7 +282,7 @@ def private tuned_template_name(func : FunctionPtr; args : AnnotationArgumentLis //! env overrides) — shared by `[tuned]` compile-time reads, runtime apply, `[tune]` generator winners, //! and the tuner harness write. Hints under `"kernels"`, knobs under `"runtime"`; "" with no framework. def box_profile_path() : string { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_manifest_path() } else { return "" @@ -292,7 +292,7 @@ def box_profile_path() : string { //! The sidecar is older than the running binary — its measurements are dead (the runtime //! apply skips it; the compile-time reads already treat it as absent). def box_profile_stale() : bool { - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return tune_sidecar_stale(tune_manifest_path()) } else { return false @@ -303,7 +303,7 @@ def box_profile_stale() : bool { //! (a rebuild killed the kernel winners; the runtime knobs are still this box's state), `foreign_box` / //! `unreadable`, `absent`; anything else reads as fresh and ours, and with no framework it is `absent`. def box_profile_verdict_at(path : string) : string { // nolint:LINT012,LINT019 - `path` is read only in the framework arm - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { return "{tune_sidecar_verdict(path, "runtime").reason}" } else { return "absent" @@ -319,7 +319,7 @@ def private fallback_perm(args : AnnotationArgumentList) : string { let dv = find_arg(args, "fallback") if (dv is tString) { // ';'-chain of `suffix` or `suffix:requires` — one annotation, a per-ISA default - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { let picked = tune_pick_fallback(dv as tString) if (!empty(picked)) return picked } else { @@ -333,7 +333,7 @@ def private fallback_perm(args : AnnotationArgumentList) : string { } def private report_perm(fname, perm, source : string) : string { // nolint:LINT012,LINT019 - `fname`/`source` are read only in the framework arm - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { if (tune_verbosity() == "verbose") { print("dasllama_tune: {fname} <- {perm} ({source})\n") } @@ -347,7 +347,7 @@ def private resolve_perm(func : FunctionPtr; args : AnnotationArgumentList) : st let fname = "{func.name}" let fb = fallback_perm(args) g_fallbacks[fname] = fb - static_if (typeinfo builtin_module_exists(llvm_tune)) { + static_if (typeinfo module_exists(llvm_tune)) { if (tune_policy_env() == "reference") { // the true kill switch, [tune]'s twin: the template's own loops, no hints stamped return report_perm(fname, "plain", "reference policy") diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 32e31fee96..9cc5f4fa54 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -210,10 +210,10 @@ file. 1. `daslib/f16_cvt.das`; 37 require spellings (landed). 2. The inert annotation module in daslib carrying `tune`, `tune_perm`, `tune_companion`, `tune_scope`, `tune_policy` and `llvm_code` as shells: `require ?llvm/daslib/llvm_tune - llvm/daslib/llvm_tune`, then `static_if (typeinfo builtin_module_exists(llvm_tune))` in each + llvm/daslib/llvm_tune`, then `static_if (typeinfo module_exists(llvm_tune))` in each `apply` delegating to the framework, else return true and do nothing. The shell's own - path-guarded require is the witness, so the order caveat on `builtin_module_exists` - (`ast_infer_type.cpp:2833-2848`) does not bite. Framework change: its annotation classes + path-guarded require is the witness (`module_exists`, not `builtin_module_exists` - see the + ruling under "Open questions"). Framework change: its annotation classes stop self-registering under those names (two registrations of `[tune]` would resolve ambiguously) and expose their `apply` as plain functions the shells forward to. The `_variants()` registries are emitted only by the framework; the engine references none @@ -356,17 +356,14 @@ the honest wasm candidates, and smaller is on the table. therefore runs in a worktree with `modules/dasLLVM` removed, the installed-SDK shape. If the in-tree case should also work, the cheapest honest witness is a C++ module registered only when LLVM is configured (the `?sqlite` pattern), and the guards become `? ...`. -- **`builtin_module_exists` on a shared das module flips under tool-driven compiles.** A file - with `require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune` and - `static_if (typeinfo builtin_module_exists(llvm_tune))` takes the framework arm under - `bin/daslang file.das` and the no-framework arm under `bin/daslang utils/lint/main.das -- file.das`: - `Module::requireEx` (`src/ast/ast_infer_type.cpp:2847`, `src/ast/ast_module.cpp:284`) scans the - bound process's promoted-module list, which a nested compile does not populate. C++-module - guards are immune. Consequence today: lint, and presumably ast-verify and the MCP checks, - audit the no-framework half of every guarded file and never the framework half (suppressed - per site with the house `nolint:...,LINT019` spelling). Two fixes, both C++: the witness module - above (then `builtin_module_exists()` is a linked-module question), or making the - trait answer "is this module in the compiling program" for shared das modules. +- **RULED: a second trait.** `builtin_module_exists` on a shared das module flipped under + tool-driven compiles (`Module::requireEx` scans the process's promoted-module list, which a + nested compile never populates), so lint, ast-verify and the MCP checks audited the + no-framework arm of every guarded file. The ruling keeps `builtin_module_exists` as it was + (the process registry) and adds `typeinfo module_exists(X)`: the compiling program's own + library, the same answer on both rails. Every path-guarded das target in the tree + (`llvm_tune`, `llvm_code`, `dasllama_exchange`, `dasllama_gemm_gen`) now asks `module_exists`; + C++-module guards keep `builtin_module_exists`. Gate: `tests/language/optional_require.das`. - Which small-LLM carrier gets the parity fixture for stages 1 and 2 (SmolLM2-135M already has a cls_q8 parity cell, `test_parity.das:76`). diff --git a/modules/dasLLVM/daslib/llvm_user_modules.das b/modules/dasLLVM/daslib/llvm_user_modules.das index 5a6ff6ec41..dbc53d5676 100644 --- a/modules/dasLLVM/daslib/llvm_user_modules.das +++ b/modules/dasLLVM/daslib/llvm_user_modules.das @@ -14,7 +14,7 @@ require ?dasllama/dasllama_gemm_gen dasllama/dasllama_gemm_register // nolint:S //! `require ? ` line above — a PATH guard loads exactly when the //! contributor's package is mounted (the guard's own file resolves), so this //! module never hard-depends on the contributor. Pair each line with a `static_if -//! (typeinfo builtin_module_exists())` around the registration call below; registered +//! (typeinfo module_exists())` around the registration call below; registered //! generators are name-keyed and inert for programs whose `[llvm_code]` annotations never //! name them. @@ -27,7 +27,7 @@ def public register_user_llvm_code_generators { } g_registered = true llvm_code_selftest::register_selftest() // worked example + self-test generator (add_i32, add_plus_k) - static_if (typeinfo builtin_module_exists(dasllama_gemm_gen)) { + static_if (typeinfo module_exists(dasllama_gemm_gen)) { dasllama_gemm_register::register_dasllama_gemm_code_generators() } } diff --git a/skills/das_macros.md b/skills/das_macros.md index 791285870d..a82c9ae39e 100644 --- a/skills/das_macros.md +++ b/skills/das_macros.md @@ -60,7 +60,8 @@ To contribute macro-time state into A's registry the contributor must compile in two pieces: an optional require (`require ? `; a plain `require` resolves before `static_if`, so `static_if` alone cannot gate it), plus `static_if (typeinfo builtin_module_exists())` around the registration call and any -dispatcher branch naming the contributor's symbols. Adapters may still *emit* code referencing +dispatcher branch naming the contributor's symbols - `typeinfo module_exists()` when the +guard is a path and the target a das module, so a tool's nested compile takes the same arm. Adapters may still *emit* code referencing those symbols by name - that resolves at the user's splice site and is unaffected. ## Pass macros - which hook sees what diff --git a/skills/daslang/references/generics.md b/skills/daslang/references/generics.md index dc2f546ae2..35e0145a9d 100644 --- a/skills/daslang/references/generics.md +++ b/skills/daslang/references/generics.md @@ -126,7 +126,9 @@ Annotation traits all carry the `struct_` prefix - plain `has_annotation` does n argument to already be a struct or handled type, `safe_has_field` does not. `` takes the bare field name, not a string: `typeinfo safe_has_field(obj)`. -**Existence** - `builtin_module_exists(name)` (how a binding is made optional), +**Existence** - `builtin_module_exists(name)` (a module registered in the process - how a +C++ binding is made optional), `module_exists(name)` (a module the compiling program required, +C++ or das - the guard for a path-guarded das target, same answer inside a tool's nested compile), `builtin_annotation_exists(type)` (true for registered native types), `builtin_function_exists(@@ name)` (native functions only - a daslang-defined function is a compile error here). diff --git a/skills/daslang/references/modules-and-stdlib.md b/skills/daslang/references/modules-and-stdlib.md index 953394810f..6cb5dd77e2 100644 --- a/skills/daslang/references/modules-and-stdlib.md +++ b/skills/daslang/references/modules-and-stdlib.md @@ -91,7 +91,9 @@ require ?pugixml pugixml/PUGIXML_boost // load only if module `pugixml` is avai - **`?guard` skips silently** when the guard module is unavailable - no dependency, no error, even if the target does not exist; with the guard present a missing target errors normally. A guard containing `/` is satisfied when that path resolves, a plain name when that module is registered. - Pair with `static_if (typeinfo builtin_module_exists(guard)) { ... }`. + Pair with `static_if (typeinfo builtin_module_exists(guard)) { ... }` for a C++ guard, and + with `static_if (typeinfo module_exists(target)) { ... }` for a path-guarded das target (the + program's own library answers, the same inside a tool's nested compile). ## Qualified calls diff --git a/skills/internal/llvm_tune_internals.md b/skills/internal/llvm_tune_internals.md index 34937636da..a4404ca590 100644 --- a/skills/internal/llvm_tune_internals.md +++ b/skills/internal/llvm_tune_internals.md @@ -8,7 +8,7 @@ file is only what maintaining the implementation needs on top of it. infer pass are registered by `daslib/tune`, whose shells forward here - `tune_apply`, `tune_perm_apply`, `tune_companion_apply`, `tune_scope_apply`, `tune_policy_apply`, `tune_default_policy_apply`, and `llvm_code_apply` in `llvm_code.das` - under -`static_if (typeinfo builtin_module_exists(llvm_tune))`, and do nothing when the guard is +`static_if (typeinfo module_exists(llvm_tune))`, and do nothing when the guard is false. Adding an annotation means adding both halves. A shell may not stay behind in this module: **a module's macro state is per-module**, so `g_scopes`, `g_perm_rows` and `g_companion_rows` are only consistent while every writer and reader runs in one macro diff --git a/src/ast/ast_infer_type.cpp b/src/ast/ast_infer_type.cpp index d0ffc20209..6e5e23ee6c 100644 --- a/src/ast/ast_infer_type.cpp +++ b/src/ast/ast_infer_type.cpp @@ -2313,7 +2313,7 @@ namespace das { } // verify bool allowMissingTypeExpr = false; - if (expr->trait == "builtin_function_exists" || expr->trait == "builtin_module_exists") { + if (expr->trait == "builtin_function_exists" || expr->trait == "builtin_module_exists" || expr->trait == "module_exists") { allowMissingTypeExpr = true; } bool allowMissingType = false; @@ -2845,7 +2845,24 @@ namespace das { // allowPromoted: pairs with the `require ?guard` check (ast_parse.cpp), which // also accepts shared das modules compiled earlier in the process auto mod = Module::requireEx(evar->name, true); - if ( !mod ) mod = program->library.findModule(evar->name); + reportAstChanged(); + return new ExprConstBool(expr->at, mod != nullptr); + } else { + error("unsupported module name subexpression ", expr->subexpr->__rtti, "", + expr->at, CompilationError::invalid_typeinfo_module_subexpression); + } + } + } else if (expr->trait == "module_exists") { + if (!expr->subexpr) { + error("module_exists requires subexpression", "", "", + expr->at, CompilationError::missing_typeinfo_subexpression); + } else { + if (expr->subexpr->rtti_isVar()) { + auto evar = static_cast(expr->subexpr); + // the compiling program's own library: a module this program required (directly or + // through another module), C++ or das, promoted or not - the same answer whether the + // program compiles as the running script or inside a tool's nested compile + auto mod = program->library.findModule(evar->name); reportAstChanged(); return new ExprConstBool(expr->at, mod != nullptr); } else { diff --git a/tests/language/optional_require.das b/tests/language/optional_require.das index 8776a22342..d318b0d813 100644 --- a/tests/language/optional_require.das +++ b/tests/language/optional_require.das @@ -43,6 +43,19 @@ def test_optional_require_path_guard(t : T?) { // path guard resolved (daslib/fio is always mounted): the target loaded and its symbols work t |> equal(optional_require_fixture_pathguard::pathguard_marker(), 43) t |> success(typeinfo builtin_module_exists(optional_require_fixture_pathguard)) + // module_exists asks the compiling program's own library, not the process registry: the + // loaded das target and a required builtin answer true, the skipped target and a name + // nobody required answer false - and a tool's nested compile, which promotes nothing, + // answers the same + t |> success(typeinfo module_exists(optional_require_fixture_pathguard)) + t |> success(typeinfo module_exists(math)) + t |> success(!typeinfo module_exists(optional_require_fixture_skipme)) + t |> success(!typeinfo module_exists(no_such_guard_xyz)) + static_if (typeinfo module_exists(optional_require_fixture_pathguard)) { + t |> equal(optional_require_fixture_pathguard::pathguard_marker(), 43) + } else { + this_symbol_must_never_resolve_module_exists_xyz() + } // path guard unresolvable: the require was SKIPPED even though the target file exists on disk — // the pin for path-guard semantics (a plain-name guard would have loaded it via the fallback) t |> success(!typeinfo builtin_module_exists(optional_require_fixture_skipme)) diff --git a/utils/dasllama-server/main.das b/utils/dasllama-server/main.das index 523c273ddb..44299d90c4 100644 --- a/utils/dasllama-server/main.das +++ b/utils/dasllama-server/main.das @@ -177,7 +177,7 @@ struct ModelEntry { // untuned boot consults the exchange only when the config opts in (the exchange is closed; off by default) [init] def register_exchange_resolver { - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { exchange_register_resolver("dasllama-server.toml") } } @@ -853,7 +853,7 @@ def init() { // nolint:STYLE037,STYLE038 — boot sequence: the knobs are orde log_tune_status("dasllama-server") // a fresh local tune that has not been shared: submit now (always) or surface the offer // (ask — the watchdog balloon + the control page's exchange card); quiet otherwise - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { exchange_boot_submit_check(exchange_policy_from_config(g_cfg.config)) } // GPU backend lands BEFORE the load: the metal mode pins the row-major backend the GPU diff --git a/utils/dasllama-server/openai_server.das b/utils/dasllama-server/openai_server.das index 744aaeddcb..558caae262 100644 --- a/utils/dasllama-server/openai_server.das +++ b/utils/dasllama-server/openai_server.das @@ -3622,7 +3622,7 @@ def private handle_exchange_get(var req : HttpRequest?; var resp : HttpResponse? // the status carries the local sidecar path (username in it) — operator-only, like the rest return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { return resp |> JSON(exchange_status_json(exchange_policy_from_config(g_config_save_path))) } else { return exchange_absent(resp) @@ -3634,7 +3634,7 @@ def private handle_exchange_matches(var req : HttpRequest?; var resp : HttpRespo return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } // a live exchange lookup — seconds on the tick thread; the page requests it explicitly - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { return resp |> JSON(exchange_matches_json(exchange_policy_from_config(g_config_save_path))) } else { return exchange_absent(resp) @@ -3645,7 +3645,7 @@ def private handle_exchange_apply(var req : HttpRequest?; var resp : HttpRespons if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { var jerr = "" var js = read_json(string(req.body), jerr) defer() { @@ -3679,7 +3679,7 @@ def private handle_exchange_submit(var req : HttpRequest?; var resp : HttpRespon if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { let err = exchange_submit_current(exchange_policy_from_config(g_config_save_path)) if (!empty(err)) { return resp |> JSON(error_body(err, "invalid_request_error"), http_status.BAD_REQUEST) @@ -3695,7 +3695,7 @@ def private handle_exchange_consent(var req : HttpRequest?; var resp : HttpRespo if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { var jerr = "" var js = read_json(string(req.body), jerr) defer() { @@ -3734,7 +3734,7 @@ def private handle_exchange_retune(var req : HttpRequest?; var resp : HttpRespon if (!is_loopback_req(req)) { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } - static_if (typeinfo builtin_module_exists(dasllama_exchange)) { + static_if (typeinfo module_exists(dasllama_exchange)) { let err = exchange_request_retune() if (!empty(err)) { // the marker write failed and the sidecar is untouched — do NOT drain/restart From 76d92d9d9b869b52600d55284d597eff0203d315 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:00:34 -0700 Subject: [PATCH 13/43] the llvm witness module: "no LLVM" is the build configuration's word, never the filesystem's - every LLVM guard is `require ?llvm`, and module_exists answers what is visible from the compiling module dasLLVM is a pure-das dasbind package, so its files sit on disk in every checkout and the path guard `?llvm/daslib/llvm_tune` satisfied itself in a build configured without LLVM: the tune framework loaded, read sidecars, and the IR generator was one call from LLVM.dll. The two shapes that matter, a wasm build where no tuning may exist and a console build with the LLVM sources present for the PC build but configured out, both need the configure to decide. modules/dasLLVM/src/dasLLVM.cpp is a C++ module named `llvm` with nothing in it, compiled in exactly when the package is (its .das_module registers the shared module silently, so a build without it registers nothing); every guard is now `require ?llvm `. module_exists tightens from "in the program's library" to "visible from the compiling module" (isVisibleDirectly): the first shape answered true for a module another module required while this one's guarded require had been skipped, and the taken arm then called into a module it could not see. The one site that asked about a guard rather than its target (llvm_user_modules, dasllama_gemm_gen for dasllama_gemm_register) asks about the target, which is what the JIT's generator registration needs. Co-Authored-By: Claude Fable 5.1 --- daslib/tune.das | 16 ++++++----- .../language/generic_programming.rst | 2 +- doc/source/reference/language/modules.rst | 7 ++--- modules/dasLLAMA/ARCHITECTURE_ENGINE.md | 10 +++---- modules/dasLLAMA/benchmarks/lcpp_bench.das | 2 +- modules/dasLLAMA/dasllama/dasllama_common.das | 2 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 4 +-- modules/dasLLAMA/dasllama/dasllama_tune.das | 2 +- .../dasLLAMA/dasllama/dasllama_tune_scope.das | 2 +- modules/dasLLAMA/portable_tier_plan.md | 25 ++++++++--------- modules/dasLLAMA/tests/test_batch_grid.das | 2 +- .../dasLLAMA/tests/test_kernel_backend.das | 2 +- modules/dasLLAMA/tests/test_mxfp4.das | 2 +- modules/dasLLVM/.das_module | 6 +++++ modules/dasLLVM/CMakeLists.txt | 8 ++++++ modules/dasLLVM/daslib/llvm_user_modules.das | 7 ++--- modules/dasLLVM/src/dasLLVM.cpp | 27 +++++++++++++++++++ skills/das_macros.md | 2 +- skills/daslang/references/generics.md | 4 +-- .../daslang/references/modules-and-stdlib.md | 4 +-- src/ast/ast_infer_type.cpp | 10 ++++--- tests/language/optional_require.das | 2 +- utils/dasllama-server/main.das | 2 +- utils/dasllama-server/openai_server.das | 2 +- 24 files changed, 100 insertions(+), 52 deletions(-) create mode 100644 modules/dasLLVM/src/dasLLVM.cpp diff --git a/daslib/tune.das b/daslib/tune.das index 0b0ed7a2f1..4305e25eca 100644 --- a/daslib/tune.das +++ b/daslib/tune.das @@ -7,17 +7,19 @@ module tune shared public require daslib/ast require daslib/ast_boost -require ?llvm/daslib/llvm_code llvm/daslib/llvm_code public -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public +require ?llvm llvm/daslib/llvm_code public +require ?llvm llvm/daslib/llvm_tune public //! The six kernel-tuning annotations - `[tune_perm]`, `[tune_companion]`, `[tune]`, //! `[tune_scope]`, `[tune_policy]` and `[llvm_code]` - plus the default-policy infer pass, as //! shells every program can spell. The framework that implements them lives in -//! `llvm/daslib/llvm_tune` and `llvm/daslib/llvm_code`, reached through the path guards above -//! and re-exported, so `require daslib/tune` gives exactly what `require llvm/daslib/llvm_tune` -//! gave; the guarded require is also the witness that makes `module_exists` below answer for -//! this compile (`module_exists`, not `builtin_module_exists`: the program's own library, the -//! same answer inside a tool's nested compile). Every tune annotation registers HERE and nowhere else, because a +//! `llvm/daslib/llvm_tune` and `llvm/daslib/llvm_code`, reached through the `?llvm` guards +//! above and re-exported, so `require daslib/tune` gives exactly what `require llvm/daslib/llvm_tune` +//! gave. The guard is the `llvm` C++ witness module, compiled in exactly when the build is +//! configured with dasLLVM - the das files being on disk never counts; the guarded require is +//! then what makes `module_exists` below answer for this compile (`module_exists`, not +//! `builtin_module_exists`: visible from this module, the same answer inside a tool's nested +//! compile). Every tune annotation registers HERE and nowhere else, because a //! module's macro state is per-module: the `[tune_scope]` bank and the pass that reads it have //! to run in one macro context. Without dasLLVM every apply is a no-op - no sidecar read, no //! stamp, no `_variants()` registry, no tuner spawn - and the reference body IS the function. diff --git a/doc/source/reference/language/generic_programming.rst b/doc/source/reference/language/generic_programming.rst index c6e56c0435..c8f5655206 100644 --- a/doc/source/reference/language/generic_programming.rst +++ b/doc/source/reference/language/generic_programming.rst @@ -153,7 +153,7 @@ All ``typeinfo`` traits can operate on either an expression or a ``type`` arg * ``typeinfo builtin_function_exists(expr)`` — true if a ``@@function`` exists * ``typeinfo builtin_annotation_exists(expr)`` — true if an annotation type exists * ``typeinfo builtin_module_exists(expr)`` — true if a module is registered in the process: a linked C++ module, or a shared das module promoted by a running script -* ``typeinfo module_exists(expr)`` — true if the compiling program's own library holds the module, i.e. this program required it (directly or through another module); the answer is the same inside a tool's nested compile, which is what a ``require ?path`` witness needs +* ``typeinfo module_exists(expr)`` — true if the module is visible from the compiling module: the module itself, one it requires, or one a require re-exports ``public``; the answer is the same inside a tool's nested compile, which is what a guarded ``require`` needs * ``typeinfo is_argument(expr)`` — true if the expression is a function argument * ``typeinfo mangled_name(expr)`` — returns the mangled name of a ``@@function`` diff --git a/doc/source/reference/language/modules.rst b/doc/source/reference/language/modules.rst index 07a2401b00..3c4db6d264 100644 --- a/doc/source/reference/language/modules.rst +++ b/doc/source/reference/language/modules.rst @@ -101,9 +101,10 @@ line pulls a das package's registration glue exactly when that package is mounte ``typeinfo builtin_module_exists`` additionally sees **shared das modules** (``module X shared``) promoted by the running script - but a tool that compiles the same program in a nested context (lint, the language server, a test harness) -promotes nothing, and there the trait answers false for every das target. Guard a -path-guarded das target with ``typeinfo module_exists(target)`` instead: it asks the -compiling program's own library, so the answer is the same on both rails. +promotes nothing, and there the trait answers false for every das target. Guard the +use of a guarded das target with ``typeinfo module_exists(target)`` instead: it asks +whether the target is visible from the compiling module, which is exactly what the +guarded ``require`` decided, so the answer is the same on both rails. Pair it with :ref:`typeinfo builtin_module_exists ` to guard code that uses the optional target's symbols — ``static_if`` drops the untaken diff --git a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md index 1236923956..c4b564ef98 100644 --- a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md +++ b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md @@ -253,11 +253,11 @@ file builds an `ArchDesc` (name * `configure` * the `ArchBlocks` fn-ptr quad * ` sidecar-sibling file governs - unset asks on a real terminal, or emits `@sidecar consent state=needed` for the watchdog dialog / control page, and no request leaves until a surface records "accepted". The client is meaningless without a sidecar, so - every requirer takes it through the path guard `require ?llvm/daslib/llvm_tune - dasllama/dasllama_exchange` and wraps each use in - `static_if (typeinfo builtin_module_exists(dasllama_exchange))`: with no tune framework in the - build the server's `/exchange` routes answer `not_found`, the boot resolver and submit check - are never registered, and `lcpp_bench --submit` refuses before it measures. + every requirer takes it through the guard `require ?llvm dasllama/dasllama_exchange` (`llvm` + is the C++ witness module dasLLVM compiles in exactly when the build is configured with it) + and wraps each use in `static_if (typeinfo module_exists(dasllama_exchange))`: with no tune + framework in the build the server's `/exchange` routes answer `not_found`, the boot resolver + and submit check are never registered, and `lcpp_bench --submit` refuses before it measures. - **`benchmarks/asr/mem_census.sh`** - the peak-memory census (`/usr/bin/time -l` around one asr_bench process per cell; macOS only) - the interim footprint instrument until a footprint leg lands in `gen_bench_records`; its numbers live in `PERF_LEDGER.md`, never the stores. diff --git a/modules/dasLLAMA/benchmarks/lcpp_bench.das b/modules/dasLLAMA/benchmarks/lcpp_bench.das index 0648bc0644..0187bd5767 100644 --- a/modules/dasLLAMA/benchmarks/lcpp_bench.das +++ b/modules/dasLLAMA/benchmarks/lcpp_bench.das @@ -31,7 +31,7 @@ require daslib/module_path // get_this_module_dir — the default pro require daslib/jobque_profile // JOBQUE_PROFILING_ENABLED — --prof refuses on a build without the marker rail require ?das_accelerate dasllama/dasllama_math_accelerate // the --accel leg; Apple-only C++ module require ../performance/profile_common.das // profile_threads + affinity_on + bench records — the standing per-box methodology -require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: --submit POSTs the stamped records to the exchange, which rides the tune framework +require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: --submit POSTs the stamped records to the exchange, which rides the tune framework require llvm/daslib/llvm_tune // tune_app_script — the exe-vs-script discriminator behind --for-debug-purposes require daslib/jobque_boost require daslib/clargs diff --git a/modules/dasLLAMA/dasllama/dasllama_common.das b/modules/dasLLAMA/dasllama/dasllama_common.das index b48dcfda6b..117445e47a 100644 --- a/modules/dasLLAMA/dasllama/dasllama_common.das +++ b/modules/dasLLAMA/dasllama/dasllama_common.das @@ -21,7 +21,7 @@ require dasllama/dasllama_math_default // registers the portable Q8·Q8 backend require dasllama/dasllama_math_aarch64_neon // registers the arm64 SDOT Q8·Q8 backend at [init] under arm64 + JIT alone; compiles everywhere (daslib/aarch64_neon carries scalar fallbacks) require dasllama/dasllama_math_gen // [init]-only: registers "arm64-gen" / "x64-gen" (the generated GEMM family) under the JIT alone; compiles everywhere - its reference bodies are the portable tier's kq kernels require daslib/tune public // the tune annotations + the auto-policy pass, inert without dasLLVM; PUBLIC for the same reach as the line below -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): tune_status reaches every app that requires this module +require ?llvm llvm/daslib/llvm_tune public // re-export ONLY llvm_tune (not math_gen's whole public surface — its jobque_boost would flood requirers with ambiguities): tune_status reaches every app that requires this module require ?das_metal dasllama/dasllama_metal_gemm // [init]-only: registers "metal" (the GPU prefill-GEMM batch donor). STRICT guard: the C++ das_metal module exists on Apple builds only, so non-Apple lanes never compile the Metal driver require ?das_accelerate dasllama/dasllama_math_accelerate // [init]-only: arms the "+AMX" float-batch override when DASLLAMA_ACCEL=1. Same Apple-only guard shape as the metal driver require dasllama/dasllama_gguf diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 104a84ae88..48420ae66c 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -12,7 +12,7 @@ require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: the one framework call below sits under static_if +require ?llvm llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: the one framework call below sits under static_if require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public @@ -45,7 +45,7 @@ require math // ISA-agnostic slot traversals: "arm64-gen" (priority 25, the arm64 load-select tier since // slice C) and "x64-gen" (priority 25, witness-gated — only selectable when the stamped // family emitted). dasllama_common pulls it in behind a path guard -// (`require ?llvm/daslib/llvm_tune`), so any Q8/mx4 model load on a gen-capable box runs the +// (`require ?llvm`), so any Q8/mx4 model load on a gen-capable box runs the // generated family. The generators live in dasllama/dasllama_gemm_gen.das (llvm-jit context, // wired via llvm/daslib/llvm_user_modules.das). diff --git a/modules/dasLLAMA/dasllama/dasllama_tune.das b/modules/dasLLAMA/dasllama/dasllama_tune.das index a10a14bb1f..c3d006771a 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tune.das +++ b/modules/dasLLAMA/dasllama/dasllama_tune.das @@ -12,7 +12,7 @@ require daslib/ast_boost require daslib/templates_boost require daslib/json_boost // read_json / read_json_field (re-exports daslib/json) require daslib/strings_boost // nolint:STYLE030,LINT019 - split/find, reached only by the no-framework fallback-chain scan -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune // OPTIONAL: the per-app tune sidecar (path, staleness, kernels read) +require ?llvm llvm/daslib/llvm_tune // OPTIONAL: the per-app tune sidecar (path, staleness, kernels read) // ===== Tune spec ===== diff --git a/modules/dasLLAMA/dasllama/dasllama_tune_scope.das b/modules/dasLLAMA/dasllama/dasllama_tune_scope.das index e311d3f1f8..2ffc1d69db 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tune_scope.das +++ b/modules/dasLLAMA/dasllama/dasllama_tune_scope.das @@ -6,7 +6,7 @@ module dasllama_tune_scope shared public require dasllama/dasllama_lint public require daslib/tune public // the [tune_scope] annotation, inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots -require ?llvm/daslib/llvm_tune llvm/daslib/llvm_tune public // OPTIONAL; PUBLIC so the auto-policy infer pass and tune_status reach app roots +require ?llvm llvm/daslib/llvm_tune public // OPTIONAL; PUBLIC so the auto-policy infer pass and tune_status reach app roots require dasllama/dasllama_version // nolint:STYLE030 — version_of= below names DASLLAMA_RELEASE by string; the module must be in this compile //! The scope reaches the AST before any covered module compiles - the scope apply refuses a covered diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 9cc5f4fa54..db90aae8b3 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -74,9 +74,13 @@ with LLVM on the host only; wasm32 cross is unsupported by design (pointer-width borrowed from the IR generator (`pm1_of`, `vbmi_alphabet`, `vbmi_pack_word`) moved to `dasllama_gemm_schema.das` for the same reason. - **`require ?G target` names a guard module, never the target** (`ds2_parser.ypp:849-861`, - `parser_impl.cpp:1225-1279`). There is no module named `llvm` (dasLLVM is a pure-das dasbind - package), so `?llvm` and `builtin_module_exists(llvm)` are permanently false; the working - spelling is the path guard `?llvm/daslib/llvm_tune`. `get_platform_name()`, + `parser_impl.cpp:1225-1279`). The guard for everything LLVM is `?llvm`: dasLLVM is a pure-das + dasbind package, so it now carries one C++ witness module named `llvm` + (`modules/dasLLVM/src/dasLLVM.cpp`), compiled in exactly when the build is configured with + dasLLVM. The build configuration decides, never the files on disk - a wasm build never + compiles the witness, and a console build with the LLVM sources present but configured out + loads no framework, no sidecar, no IR generator (ruled 2026-09-04; the path-guard spelling + `?llvm/daslib/llvm_tune` that preceded it satisfied itself off the filesystem). `get_platform_name()`, `get_cross_platform_name()` and `get_architecture_name()` fold at compile time (`module_builtin_runtime.cpp:2887-2907`), so a synthetic `wasm` module buys nothing for code gating. `require !?X` is about five parser lines plus the hand-written pre-parse scanner @@ -347,15 +351,12 @@ the honest wasm candidates, and smaller is on the table. ## Open questions -- **What "dasLLVM absent" means in a SOURCE tree.** The `?llvm/daslib/...` path guard asks whether - the guard's file resolves, and the `.das_module` folder scan (`src/ast/dyn_modules.cpp`) - resolves `modules/dasLLVM/` from the filesystem whatever `DAS_LLVM_DISABLED` says. So in a - source tree configured without LLVM the guard is TRUE, the generated tier loads, and its - `[extern(library="LLVM.dll")]` bindings fail at compile time; only an installed SDK without - dasLLVM (its install rules sit inside the CMake guard) has the guard false. The stage-1 gate - therefore runs in a worktree with `modules/dasLLVM` removed, the installed-SDK shape. If the - in-tree case should also work, the cheapest honest witness is a C++ module registered only - when LLVM is configured (the `?sqlite` pattern), and the guards become `? ...`. +- **RULED: "dasLLVM absent" is the build configuration's word.** The path guard + `?llvm/daslib/llvm_tune` satisfied itself off the filesystem, so a source tree configured + with `-DDAS_LLVM_DISABLED=ON` still loaded the framework. The two scenarios that matter - a + wasm build, where no tuning may exist, and a console build (PS5 in dagor) with the LLVM + sources on disk for the PC build but configured out - both need the configure to decide. The + `llvm` C++ witness module (see "Settled decisions") is that word; every guard is `?llvm`. - **RULED: a second trait.** `builtin_module_exists` on a shared das module flipped under tool-driven compiles (`Module::requireEx` scans the process's promoted-module list, which a nested compile never populates), so lint, ast-verify and the MCP checks audited the diff --git a/modules/dasLLAMA/tests/test_batch_grid.das b/modules/dasLLAMA/tests/test_batch_grid.das index dcce3e2fd7..975e3e7864 100644 --- a/modules/dasLLAMA/tests/test_batch_grid.das +++ b/modules/dasLLAMA/tests/test_batch_grid.das @@ -5,7 +5,7 @@ require dastest/testing_boost public require dasllama/dasllama_math require dasllama/dasllama_math_default // portable backend (knob-inert control) require dasllama/dasllama_math_aarch64_neon // arm64 backends (registered under arm64 + JIT only) -require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen // the gen batch walk (the knob's target) +require ?llvm dasllama/dasllama_math_gen // the gen batch walk (the knob's target) require dasllama/dasllama_convert require daslib/jobque_boost require math diff --git a/modules/dasLLAMA/tests/test_kernel_backend.das b/modules/dasLLAMA/tests/test_kernel_backend.das index da6453363b..65434586dd 100644 --- a/modules/dasLLAMA/tests/test_kernel_backend.das +++ b/modules/dasLLAMA/tests/test_kernel_backend.das @@ -5,7 +5,7 @@ require dastest/testing_boost public require dasllama/dasllama_math require dasllama/dasllama_math_default // registers the "portable" backend at [init] require dasllama/dasllama_math_aarch64_neon // registers "arm64-sdot" on arm64+jit -require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen +require ?llvm dasllama/dasllama_math_gen // The kernel-backend registry must self-populate at [init]: the portable Q8·Q8 backend is always // registered (the fallback everywhere), and one is auto-activated as the row-major default. A repack diff --git a/modules/dasLLAMA/tests/test_mxfp4.das b/modules/dasLLAMA/tests/test_mxfp4.das index 8de23c344b..5228e611d8 100644 --- a/modules/dasLLAMA/tests/test_mxfp4.das +++ b/modules/dasLLAMA/tests/test_mxfp4.das @@ -5,7 +5,7 @@ require dastest/testing_boost public require dasllama/dasllama_math require dasllama/dasllama_math_default // dot_mx4q8_scalar + the "portable" backend require dasllama/dasllama_math_aarch64_neon // dot_mx4q8 (TBL+SDOT under arm64 JIT, scalar fallback elsewhere) -require ?llvm/daslib/llvm_tune dasllama/dasllama_math_gen // nolint:STYLE030,LINT019 — registers the "arm64-gen"/"x64-gen" repack tier when dasLLVM is mounted +require ?llvm dasllama/dasllama_math_gen // nolint:STYLE030,LINT019 — registers the "arm64-gen"/"x64-gen" repack tier when dasLLVM is mounted require math // MXFP4·Q8 kernels: the E8M0 scale decode, the dot leaves (scalar reference vs the NEON-intrinsic diff --git a/modules/dasLLVM/.das_module b/modules/dasLLVM/.das_module index 1330ed28b0..d2e55ab784 100644 --- a/modules/dasLLVM/.das_module +++ b/modules/dasLLVM/.das_module @@ -25,4 +25,10 @@ def initialize(project_path : string) { for (path in bindings_paths) { register_native_path("llvm", "bindings/{path}", "{project_path}/bindings/{path}.das") } + if (das_is_dll_build()) { + // the `llvm` witness module (src/dasLLVM.cpp) exists only in a build configured with + // dasLLVM; the 2-arg form skips silently when the .shared_module was never built, and + // that silence is what `require ?llvm` reads as "this build has no LLVM" + register_dynamic_module("{project_path}/dasModuleLLVM.shared_module", "Module_LLVM") + } } diff --git a/modules/dasLLVM/CMakeLists.txt b/modules/dasLLVM/CMakeLists.txt index 0f26296b24..aa2498fb45 100644 --- a/modules/dasLLVM/CMakeLists.txt +++ b/modules/dasLLVM/CMakeLists.txt @@ -7,6 +7,14 @@ IF ((NOT DAS_LLVM_INCLUDED) AND ((NOT ${DAS_LLVM_DISABLED}) OR (NOT DEFINED DAS_ SET(DAS_LLVM_DIR ${PROJECT_SOURCE_DIR}/modules/dasLLVM) + # the `llvm` witness module (src/dasLLVM.cpp): compiled in exactly when this package is, so + # `require ?llvm ` follows the configure - the das files below are on disk in every + # checkout and would otherwise satisfy a path guard in a build without LLVM + ADD_MODULE_CPP(LLVM) + ADD_MODULE_LIB(libDasModuleLLVM dasModuleLLVM ${DAS_LLVM_DIR}/src/dasLLVM.cpp) + SETUP_CPP11(libDasModuleLLVM) + SETUP_CPP11(dasModuleLLVM) + LIST(APPEND CMAKE_MODULE_PATH ${DAS_LLVM_DIR}) set(DAS_LLVM_LIB_OUTPUT "${PROJECT_SOURCE_DIR}/lib") diff --git a/modules/dasLLVM/daslib/llvm_user_modules.das b/modules/dasLLVM/daslib/llvm_user_modules.das index dbc53d5676..31a9e58986 100644 --- a/modules/dasLLVM/daslib/llvm_user_modules.das +++ b/modules/dasLLVM/daslib/llvm_user_modules.das @@ -6,7 +6,7 @@ options no_global_variables = false module llvm_user_modules shared private require llvm/daslib/llvm_code_selftest -require ?dasllama/dasllama_gemm_gen dasllama/dasllama_gemm_register // nolint:STYLE030 — used inside the static_if below; on builds without the package both the require and the branch drop (path guard: pure-das package, nothing C++ to guard on) +require ?dasllama/dasllama_gemm_gen dasllama/dasllama_gemm_register // nolint:STYLE030,LINT019 — used inside the static_if below; on builds without the package both the require and the branch drop (path guard: pure-das package, nothing C++ to guard on) //! User-extensible wiring for `[llvm_code]` generator modules. The JIT calls //! `register_user_llvm_code_generators` (lazily, in the context that reads the registry) @@ -14,7 +14,8 @@ require ?dasllama/dasllama_gemm_gen dasllama/dasllama_gemm_register // nolint:S //! `require ? ` line above — a PATH guard loads exactly when the //! contributor's package is mounted (the guard's own file resolves), so this //! module never hard-depends on the contributor. Pair each line with a `static_if -//! (typeinfo module_exists())` around the registration call below; registered +//! (typeinfo module_exists())` around the registration call below - the +//! TARGET the guarded require loaded, which is what is visible from here; registered //! generators are name-keyed and inert for programs whose `[llvm_code]` annotations never //! name them. @@ -27,7 +28,7 @@ def public register_user_llvm_code_generators { } g_registered = true llvm_code_selftest::register_selftest() // worked example + self-test generator (add_i32, add_plus_k) - static_if (typeinfo module_exists(dasllama_gemm_gen)) { + static_if (typeinfo module_exists(dasllama_gemm_register)) { dasllama_gemm_register::register_dasllama_gemm_code_generators() } } diff --git a/modules/dasLLVM/src/dasLLVM.cpp b/modules/dasLLVM/src/dasLLVM.cpp new file mode 100644 index 0000000000..10eb63b578 --- /dev/null +++ b/modules/dasLLVM/src/dasLLVM.cpp @@ -0,0 +1,27 @@ +// llvm - the witness that this build was CONFIGURED with dasLLVM. The package itself is pure +// das (dasbind over LLVM.dll), so nothing else in it can answer "is LLVM part of this build" - +// its files sit on disk in every checkout, whatever DAS_LLVM_DISABLED says. This module is +// compiled in only when the package is, so `require ?llvm ` and +// `builtin_module_exists(llvm)` follow the configure, never the filesystem: a wasm or console +// build without LLVM loads no tune framework, no sidecar, no IR generator. + +#include "daScript/daScript.h" + +namespace das { + +class Module_LLVM : public Module { +public: + Module_LLVM() : Module("llvm") { + ModuleLibrary lib(this); + lib.addBuiltInModule(); + } + virtual ModuleAotType aotRequire ( TextWriter & ) const override { + return ModuleAotType::cpp; + } +}; + +REGISTER_DYN_MODULE(Module_LLVM, Module_LLVM); + +} // namespace das + +REGISTER_MODULE_IN_NAMESPACE(Module_LLVM, das); diff --git a/skills/das_macros.md b/skills/das_macros.md index a82c9ae39e..809942d18d 100644 --- a/skills/das_macros.md +++ b/skills/das_macros.md @@ -61,7 +61,7 @@ two pieces: an optional require (`require ? `; a plain `require` `static_if`, so `static_if` alone cannot gate it), plus `static_if (typeinfo builtin_module_exists())` around the registration call and any dispatcher branch naming the contributor's symbols - `typeinfo module_exists()` when the -guard is a path and the target a das module, so a tool's nested compile takes the same arm. Adapters may still *emit* code referencing +target is a das module, so a tool's nested compile takes the same arm. Adapters may still *emit* code referencing those symbols by name - that resolves at the user's splice site and is unaffected. ## Pass macros - which hook sees what diff --git a/skills/daslang/references/generics.md b/skills/daslang/references/generics.md index 35e0145a9d..9c43745b4f 100644 --- a/skills/daslang/references/generics.md +++ b/skills/daslang/references/generics.md @@ -127,8 +127,8 @@ argument to already be a struct or handled type, `safe_has_field` does not. `(obj)`. **Existence** - `builtin_module_exists(name)` (a module registered in the process - how a -C++ binding is made optional), `module_exists(name)` (a module the compiling program required, -C++ or das - the guard for a path-guarded das target, same answer inside a tool's nested compile), +C++ binding is made optional), `module_exists(name)` (a module visible from the compiling module, +C++ or das - the check behind a guarded require's target, same answer inside a tool's nested compile), `builtin_annotation_exists(type)` (true for registered native types), `builtin_function_exists(@@ name)` (native functions only - a daslang-defined function is a compile error here). diff --git a/skills/daslang/references/modules-and-stdlib.md b/skills/daslang/references/modules-and-stdlib.md index 6cb5dd77e2..3d1562f449 100644 --- a/skills/daslang/references/modules-and-stdlib.md +++ b/skills/daslang/references/modules-and-stdlib.md @@ -92,8 +92,8 @@ require ?pugixml pugixml/PUGIXML_boost // load only if module `pugixml` is avai if the target does not exist; with the guard present a missing target errors normally. A guard containing `/` is satisfied when that path resolves, a plain name when that module is registered. Pair with `static_if (typeinfo builtin_module_exists(guard)) { ... }` for a C++ guard, and - with `static_if (typeinfo module_exists(target)) { ... }` for a path-guarded das target (the - program's own library answers, the same inside a tool's nested compile). + with `static_if (typeinfo module_exists(target)) { ... }` around uses of the guarded target + (true when the target is visible from this module, the same inside a tool's nested compile). ## Qualified calls diff --git a/src/ast/ast_infer_type.cpp b/src/ast/ast_infer_type.cpp index 6e5e23ee6c..13eabd93ae 100644 --- a/src/ast/ast_infer_type.cpp +++ b/src/ast/ast_infer_type.cpp @@ -2859,12 +2859,14 @@ namespace das { } else { if (expr->subexpr->rtti_isVar()) { auto evar = static_cast(expr->subexpr); - // the compiling program's own library: a module this program required (directly or - // through another module), C++ or das, promoted or not - the same answer whether the - // program compiles as the running script or inside a tool's nested compile + // visible from the compiling module - the module itself, one it requires, or one a + // require re-exports public - C++ or das, promoted or not: the same answer whether + // the program compiles as the running script or inside a tool's nested compile, + // and "in the program somewhere" is not enough for the call the taken arm makes auto mod = program->library.findModule(evar->name); + bool visible = mod && program->thisModule->isVisibleDirectly(mod); reportAstChanged(); - return new ExprConstBool(expr->at, mod != nullptr); + return new ExprConstBool(expr->at, visible); } else { error("unsupported module name subexpression ", expr->subexpr->__rtti, "", expr->at, CompilationError::invalid_typeinfo_module_subexpression); diff --git a/tests/language/optional_require.das b/tests/language/optional_require.das index d318b0d813..fe7062117d 100644 --- a/tests/language/optional_require.das +++ b/tests/language/optional_require.das @@ -43,7 +43,7 @@ def test_optional_require_path_guard(t : T?) { // path guard resolved (daslib/fio is always mounted): the target loaded and its symbols work t |> equal(optional_require_fixture_pathguard::pathguard_marker(), 43) t |> success(typeinfo builtin_module_exists(optional_require_fixture_pathguard)) - // module_exists asks the compiling program's own library, not the process registry: the + // module_exists asks what is visible from this module, not the process registry: the // loaded das target and a required builtin answer true, the skipped target and a name // nobody required answer false - and a tool's nested compile, which promotes nothing, // answers the same diff --git a/utils/dasllama-server/main.das b/utils/dasllama-server/main.das index 44299d90c4..d84086c1a4 100644 --- a/utils/dasllama-server/main.das +++ b/utils/dasllama-server/main.das @@ -26,7 +26,7 @@ require openai_server // the server + route handlers (this directory) require model_catalog // catalog_models_dir — the models-dir resolution require dasllama/dasllama // load_model / QuantMode / load_asr_model require llvm/daslib/llvm_tune // [tune_policy] + tune_status (the injected guard resolves here) -require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the sidecar exchange (boot resolver + submit check), which rides the tune framework +require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the sidecar exchange (boot resolver + submit check), which rides the tune framework require jobque // set_jobque_threads_cap / set_jobque_affinity require daslib/clargs require daslib/toml diff --git a/utils/dasllama-server/openai_server.das b/utils/dasllama-server/openai_server.das index 558caae262..224569a2d7 100644 --- a/utils/dasllama-server/openai_server.das +++ b/utils/dasllama-server/openai_server.das @@ -9,7 +9,7 @@ require dasllama/dasllama_scheduler // continuous-batching scheduler (layer 3 require dashv/dashv_boost public require daslib/json_boost require daslib/toml // write_toml: POST /config persists the authoritative config -require ?llvm/daslib/llvm_tune dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the /exchange surface + submit rails, which ride the tune framework +require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the /exchange surface + submit rails, which ride the tune framework require llvm/daslib/llvm_tune // tune_manifest_path — the /exchange apply target require daslib/defer // defer() { delete js } — free the parsed request JSON on every exit path require model_catalog // the curated list + the /catalog download rail From 7b414bab9ee3068bc620098b045dd91f429f8747 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:09:00 -0700 Subject: [PATCH 14/43] the plan's stage 3 drops the Vulkan-guard item: the tier already hangs off require ?vulkan in the transformer, verified with --disable-module dasvulkan; a model-free no-GPU-modules gate is owed once the suite rework lands Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/portable_tier_plan.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index db90aae8b3..f190c9b1f2 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -309,6 +309,14 @@ decode/prefill numbers sit beside the stage-1 floor. - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + pthreads), under `wasmtime -W memory64=y -W exceptions=y` first, the browser after. +- The GPU tiers already self-gate the way a wasm build needs (verified 2026-09-04 with + `--disable-module dasvulkan`: the umbrella loads, `builtin_module_exists(vulkan)` is false, + kernel units green interpreted and under the JIT). The Vulkan tier hangs off one guarded + require in `dasllama_transformer.das` (`require ?vulkan dasllama/dasllama_math_vulkan`), the + same shape as the five `?das_metal` lines beside it; every unguarded `require vulkan` sits + inside the tier's own files, which only load through that guard. Owed once the suite rework + lands: a model-free gate that spawns the umbrella under `--disable-module dasvulkan + --disable-module dasmetal` (the wasm and console shape) so the seam cannot regress silently. - Repairs on the way, all small: the `wasm_cross` CI lane configures without `-DDAS_WASM_MEMORY64=ON` (`wasm_build.yml:209`) so `web/CMakeLists.txt:374` registers no examples and the lane is a green no-op; `get_architecture_name()` answers `wasm32` on a From 3bc34cf9e4c7b46748aeb8017049fd228c2abbb4 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:28:00 -0700 Subject: [PATCH 15/43] no tuning under a cross target, by construction: get_target_triple / get_target_architecture_name fold for the artifact a compile emits, the tune framework's inert gate fires on any --jit-target, and the engine's kernel-tier gates ask the target arch get_architecture_name is a #if on the host compiler, so under --jit-target=wasm64 on an arm64 box the engine's [init] gates registered the NEON tiers into the wasm artifact, and the tune framework stamped the host's sidecar into it. Two SideEffects::none builtins answer for the target instead: the --jit-target triple ("" native) and the arch parsed off it (the host's with no triple). tune_aot_gate adds the cross-target condition, so [tune], [tuned] and [llvm_code] emit reference bodies only; the three tier gates in the engine ask get_target_architecture_name. get_architecture_name now answers wasm64 on a memory64 build. The --jit-target argument sits after the -- separator, which the module cache's key ignored: a cached native compile served the cross one with the host's folds (get_cross_platform_name included). It keys the cache now. Gate: tests/jit_tests/cross_target_folds.das, three children with the default cache on - native, wasm64 and x86_64-linux compile-only. Co-Authored-By: Claude Fable 5.1 --- include/daScript/simulate/aot_builtin.h | 2 + .../dasllama/dasllama_math_aarch64_neon.das | 8 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 4 +- modules/dasLLAMA/portable_tier_plan.md | 16 +++- modules/dasLLVM/daslib/llvm_tune.das | 9 ++- src/builtin/module_builtin_runtime.cpp | 48 +++++++++--- tests/jit_tests/_cross_target_fixture.das | 18 +++++ tests/jit_tests/cross_target_folds.das | 76 +++++++++++++++++++ utils/daslang/main.cpp | 13 ++++ 9 files changed, 170 insertions(+), 24 deletions(-) create mode 100644 tests/jit_tests/_cross_target_fixture.das create mode 100644 tests/jit_tests/cross_target_folds.das diff --git a/include/daScript/simulate/aot_builtin.h b/include/daScript/simulate/aot_builtin.h index abf86f51ef..d9e67cc4b5 100644 --- a/include/daScript/simulate/aot_builtin.h +++ b/include/daScript/simulate/aot_builtin.h @@ -303,6 +303,8 @@ namespace das { const char * das_get_platform_name(); const char * das_get_cross_platform_name(); const char * das_get_architecture_name(); + const char * das_get_target_triple(); + const char * das_get_target_architecture_name(); DAS_API bool das_cpu_supports ( const char * feature ); DAS_API char * fmt_i8 ( const char * fmt, int8_t value, Context * context, LineInfoArg * at ); diff --git a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das index 1a7df92f3f..75779b9cd7 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das @@ -453,10 +453,10 @@ def mx4q8_rows_kernel_neon(var yp : float?; wn : uint8 const?; we : uint8 const? [init] def dasllama_math_aarch64_neon_register() { - // Only register the SDOT backends under JIT on arm64. Off-JIT (interpreter or AOT) sdot4/sdot4_laneq - // run their SCALAR fallback body — slower than the portable dot_q8q8, which the vectorizer handles — - // so the portable backend stays selected everywhere the hardware SDOT isn't emitted. - if (get_architecture_name() == "arm64" && jit_enabled()) { + // SDOT backends only under JIT for an arm64 TARGET (the artifact's arch, never the host's). + // Off-JIT the sdot4 family runs its SCALAR fallback, slower than the portable dot_q8q8 the + // vectorizer handles, so portable stays selected wherever hardware SDOT isn't emitted. + if (get_target_architecture_name() == "arm64" && jit_enabled()) { // arm64-sdot: idot4x4 on row-major weights (idot4 → SDOT) — the auto-selected default for direct callers // (tests, benchmarks). priority 10 out-ranks portable (0). register_kernel_backend(KernelBackend(name = "arm64-sdot", mm = @@q8q8_kernel_neon_idot4x4, diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 48420ae66c..5a2b149109 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -2763,7 +2763,7 @@ def dasllama_math_gen_register() { // needs_repack keeps this out of auto-select (direct callers stay on arm64-sdot). q8_layout // must be evaluated at selection time, not [init] — [init]s run before the JIT installs // generated code (the slice C mr8 bug); arm64 needs no availability predicate. - if (get_architecture_name() == "arm64" && jit_enabled()) { + if (get_target_architecture_name() == "arm64" && jit_enabled()) { register_kernel_backend(KernelBackend(name = "arm64-gen", mm = @@q8q8_kernel_gen, batch = @@q8q8_batch_kernel_neon_laneq_gen, group3 = @@q8q8_group3_gen, repack = @@repack_q8q8_gen, @@ -2795,7 +2795,7 @@ def dasllama_math_gen_register() { // x64 twin: the ONLY x64-specific wiring is the availability predicate — the witness must // be true only if the family actually emitted (a declined stamp falls back to slow NEON // scalar bodies here). avx2 is the family's minimum tier. - if (get_architecture_name() == "x86_64" && jit_enabled() && cpu_supports("avx2")) { + if (get_target_architecture_name() == "x86_64" && jit_enabled() && cpu_supports("avx2")) { register_kernel_backend(KernelBackend(name = "x64-gen", mm = @@q8q8_kernel_gen, batch = @@q8q8_batch_kernel_neon_laneq_gen, group3 = @@q8q8_group3_gen, repack = @@repack_q8q8_gen, diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index f190c9b1f2..d9d7275981 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -319,10 +319,18 @@ decode/prefill numbers sit beside the stage-1 floor. --disable-module dasmetal` (the wasm and console shape) so the seam cannot regress silently. - Repairs on the way, all small: the `wasm_cross` CI lane configures without `-DDAS_WASM_MEMORY64=ON` (`wasm_build.yml:209`) so `web/CMakeLists.txt:374` registers no - examples and the lane is a green no-op; `get_architecture_name()` answers `wasm32` on a - memory64 build (`module_builtin_runtime.cpp:2064`) and the engine keys backends off that - string; `DAS_MAX_HW_JOBS` is 4 on wasm (`platform.h:629`); the two emsdk pins contradict - (`wasm_build.yml:188` vs `pages.yml:73`). + examples and the lane is a green no-op; `DAS_MAX_HW_JOBS` is 4 on wasm (`platform.h:629`); + the two emsdk pins contradict (`wasm_build.yml:188` vs `pages.yml:73`). Done 2026-09-04: + `get_architecture_name()` answers `wasm64` on a memory64 build. +- **No tuning under a cross target, by construction (landed 2026-09-04).** Two folds answer for + the artifact a compile EMITS: `get_target_triple()` (the `--jit-target` triple, "" native) and + `get_target_architecture_name()` (the triple's arch, the host's with no triple). The tune + framework's inert gate (`tune_aot_gate`) fires on any cross target, so `[tune]`, `[tuned]` and + `[llvm_code]` emit nothing but reference bodies into a wasm or console exe; the engine's + kernel-tier `[init]` gates ask the target arch, so a wasm cross-compile from an arm64 box + registers no NEON tier. The `--jit-target` argument keys the module cache (it sits after the + `--` separator, which the key ignored, so a cached native compile served a cross one with the + host's folds). Gate: `tests/jit_tests/cross_target_folds.das`. - Limits to design around: SIMD is 128-bit through the `-msse2` shim and that flag is load-bearing (`web/CMakeLists.txt:34`; without it vecmath drops to scalar); a process-wide malloc spinlock under shared memory (`src/hal/wasm_thread_malloc.cpp`); the main-thread join diff --git a/modules/dasLLVM/daslib/llvm_tune.das b/modules/dasLLVM/daslib/llvm_tune.das index ec7b53e1c7..0c5a7e059b 100644 --- a/modules/dasLLVM/daslib/llvm_tune.das +++ b/modules/dasLLVM/daslib/llvm_tune.das @@ -70,12 +70,13 @@ def jit_cli_opt_level() : int { return cli.opt_level |> unwrap_or(compiling_program().policies.jit_opt_level) } - //! True when this compile must stay stamp-free: `policies.tune_frozen` (the cross-box - //! artifact mark) or an AOT-consuming run without the JIT (a stamp changes the semantic - //! hash). Every tune annotation goes inert; only reference-row registries are emitted. + //! True when this compile must stay stamp-free - `policies.tune_frozen`, an AOT-consuming run + //! without the JIT (a stamp changes the semantic hash), or a `--jit-target` cross target (the + //! artifact runs on a box this sidecar never measured): every tune annotation goes inert. def tune_aot_gate() : bool { return (compiling_program().policies.tune_frozen - || (compiling_program().policies.aot && !compiling_program().policies.jit_enabled)) + || (compiling_program().policies.aot && !compiling_program().policies.jit_enabled) + || !empty(get_target_triple())) } //! True when this compile must retain reference bodies. Documentation roots diff --git a/src/builtin/module_builtin_runtime.cpp b/src/builtin/module_builtin_runtime.cpp index 43b439339b..78ff3bbd73 100644 --- a/src/builtin/module_builtin_runtime.cpp +++ b/src/builtin/module_builtin_runtime.cpp @@ -2051,26 +2051,31 @@ namespace das // for that target (e.g. dasOpenGL registers its wasm GLES3 module for emscripten, // while a normal desktop run keeps the pure-das opengl.das). Only the wasm triple // is mapped today (-> "emscripten"); other cross targets return "". - const char * das_get_cross_platform_name() { + // the --jit-target triple this compile emits for, "" on a native compile. A property of the + // whole compile (the triple is fixed before the first module parses), so the folds below are + // compile-time constants: an artifact for another machine decides its tiers from the target. + const char * das_get_target_triple() { char ** argv = (char **) g_CommandLineArguments.data; uint64_t n = g_CommandLineArguments.size; for ( uint64_t i=0; i(*this, lib, "get_architecture_name", SideEffects::none, "das_get_architecture_name"); + // the two target-side folds (see the definitions): SideEffects::none like the pair above, + // so a kernel-tier [init] gate and a static_if see the artifact's target, not the host + addExternInline(*this, lib, "get_target_triple", + SideEffects::none, "das_get_target_triple"); + addExternInline(*this, lib, "get_target_architecture_name", + SideEffects::none, "das_get_target_architecture_name"); // accessExternal (NOT ::none) on purpose: CPU features are a property of the RUNNING box, // so this must never const-fold into AOT artifacts built on a different machine. addExternInline(*this, lib, "cpu_supports", diff --git a/tests/jit_tests/_cross_target_fixture.das b/tests/jit_tests/_cross_target_fixture.das new file mode 100644 index 0000000000..bea777c9c1 --- /dev/null +++ b/tests/jit_tests/_cross_target_fixture.das @@ -0,0 +1,18 @@ +options gen2 + +// child for cross_target_folds.das: prints the compile-time target folds, and which static_if arm +// the cross-target test the tune framework's inert gate uses selects, so the parent can compare a +// native compile against --jit-target= + +[export] +def main { + print("TRIPLE=[{get_target_triple()}]\n") + print("TARGET_ARCH=[{get_target_architecture_name()}]\n") + print("HOST_ARCH=[{get_architecture_name()}]\n") + print("CROSS_PLATFORM=[{get_cross_platform_name()}]\n") + static_if (!empty(get_target_triple())) { + print("CROSS=[yes]\n") + } else { + print("CROSS=[no]\n") + } +} diff --git a/tests/jit_tests/cross_target_folds.das b/tests/jit_tests/cross_target_folds.das new file mode 100644 index 0000000000..2f88aed494 --- /dev/null +++ b/tests/jit_tests/cross_target_folds.das @@ -0,0 +1,76 @@ +// The target-side folds: get_target_triple / get_target_architecture_name answer for the artifact +// a compile EMITS (the --jit-target triple), the host's values with no triple, and a static_if +// over the triple - the test the tune framework's inert gate makes - selects the cross arm, so a +// wasm or console artifact built on a tuned box carries reference bodies and no sidecar stamps. +// Compile-only keeps the cross run on this host: the folds are decided from argv before the +// first module parses, and main runs interpreted. The children run with the default module +// cache on purpose: a --jit-target after the separator keys the cache, or a cached native +// compile serves the cross one with the host's folds. +options gen2 +options no_aot + +require dastest/testing_boost +require daslib/fio +require daslib/strings_boost +require strings + +let FIXTURE = "{get_das_root()}/tests/jit_tests/_cross_target_fixture.das" + +def private spawn_child(cmd : string; var lines : array) : int { + var rc : int + unsafe { + rc = popen_timeout("{cmd} 2>&1", 300.0) $(f) { + if (f == null) { + return + } + while (!feof(f)) { + let ln = strip(fgets(f)) + if (!empty(ln)) { + lines |> push("{ln}") + } + } + } + } + return rc +} + +def private has(lines : array; needle : string) : bool { + for (ln in lines) { + if (find(ln, needle) >= 0) return true + } + return false +} + +[test] +def test_cross_target_folds(t : T?) { + if (!jit_enabled()) { + t |> success(true) + return + } + let args <- get_command_line_arguments() // locked view — do not delete + let bin = args[0] + t |> run("a native compile answers the host and keeps the framework live") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\"", ln) + t |> success(ln |> has("TRIPLE=[]"), "no triple on a native compile: {join(ln, " | ")}") + t |> success(ln |> has("TARGET_ARCH=[{get_architecture_name()}]"), "the target arch is the host's") + t |> success(ln |> has("CROSS_PLATFORM=[]"), "no cross platform") + t |> success(ln |> has("CROSS=[no]"), "a native compile takes the native arm") + } + t |> run("a wasm64 cross target answers wasm64 and turns the framework inert") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-target=wasm64-unknown-emscripten --jit-compile-only", ln) + t |> success(ln |> has("TRIPLE=[wasm64-unknown-emscripten]"), "the triple folds: {join(ln, " | ")}") + t |> success(ln |> has("TARGET_ARCH=[wasm64]"), "the target arch is the triple's") + t |> success(ln |> has("HOST_ARCH=[{get_architecture_name()}]"), "the host arch stays the host's") + t |> success(ln |> has("CROSS_PLATFORM=[emscripten]"), "the cross platform is emscripten") + t |> success(ln |> has("CROSS=[yes]"), "a cross compile takes the cross arm") + } + t |> run("an x86_64 linux cross target answers x86_64") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-target=x86_64-unknown-linux-gnu --jit-compile-only", ln) + t |> success(ln |> has("TARGET_ARCH=[x86_64]"), "the target arch is the triple's: {join(ln, " | ")}") + t |> success(ln |> has("CROSS_PLATFORM=[]"), "linux is not a cross platform name") + t |> success(ln |> has("CROSS=[yes]"), "any cross target takes the cross arm") + } +} diff --git a/utils/daslang/main.cpp b/utils/daslang/main.cpp index 3aebbb9f77..ea80b59d86 100644 --- a/utils/daslang/main.cpp +++ b/utils/daslang/main.cpp @@ -803,6 +803,19 @@ int MAIN_FUNC_NAME ( int argc, char * argv[] ) { hostOptions += argv[i]; hostOptions += '\n'; } + // a --jit-target after the separator is a compile input too: get_target_triple and every fold + // over it bake the target into each module, so a cached native compile must not serve a cross one + for ( int i=1, sep=0; i < argc; ++i ) { + if ( !sep ) { sep = strcmp(argv[i],"--")==0; continue; } + if ( strncmp(argv[i],"--jit-target",12)==0 ) { + hostOptions += argv[i]; + hostOptions += '\n'; + if ( strcmp(argv[i],"--jit-target")==0 && i+1 files; string mainName = "main"; bool scriptArgs = false; From 49c948d3e691dc7146602af1f84d093f50be45ea Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:43:54 -0700 Subject: [PATCH 16/43] stage 3: dasLLAMA runs on WASM64 - the dasllama_aot target builds in the wasm64 web tree from the host-generated engine C++, and the node driver reproduces the JIT run token-exact on SmolLM2 web/CMakeLists.txt adds this tree as a subdirectory, so the target already existed there with the wrong generator: the `daslang` target is a .wasm in that build. Under emscripten the repo's host daslang generates (one generated set serves both builds - 8-byte pointers on each side, every layout static_assert holds under em++), the executable links libDaScript instead of libDaScriptAot (whose stubs the wasm daslang would have to generate), the engine's das files and its .das_module embed into MEMFS beside daslib, ENV joins the exported runtime methods so the driver can hold the image rail off, and the initial heap grows to 512 MB for the AOT type tables. The Accelerate driver's TU stays out wherever das_accelerate is not built - its C++ calls the binding by symbol - which covers every non-Apple native build too. examples/dasLLAMA/wasm/run_node.js mounts the repo and the model's directory through NODEFS and calls main with -use-aot. Output matches the native JIT run; decode 46 t/s under node 25. No dasLLVM is embedded: require ?llvm is false inside the artifact. Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/CMakeLists.txt | 58 +++++++++++++++++++++++--- examples/dasLLAMA/wasm/run_node.js | 45 ++++++++++++++++++++ modules/dasLLAMA/portable_tier_plan.md | 17 +++++++- 3 files changed, 113 insertions(+), 7 deletions(-) create mode 100644 examples/dasLLAMA/wasm/run_node.js diff --git a/examples/dasLLAMA/CMakeLists.txt b/examples/dasLLAMA/CMakeLists.txt index c73c29d5da..3503ec65aa 100644 --- a/examples/dasLLAMA/CMakeLists.txt +++ b/examples/dasLLAMA/CMakeLists.txt @@ -7,10 +7,32 @@ ### Opt-in (EXCLUDE_FROM_ALL): ~90 AOT TUs. The GPU tiers, the LLVM IR generators and the ### exchange server are not AOT'd - they interpret as scaffolding (the Vulkan classes alone ### emit 40 MB of C++), and a call from AOT'd code into them is inlined C++ regardless. +### +### The same target exists in the wasm64 web build (web/CMakeLists.txt adds this tree as a +### subdirectory): the repo's HOST daslang generates the C++ (the `daslang` target is a .wasm +### there), em++ compiles it, and the engine's das files embed into MEMFS beside daslib - +### web/output64/dasllama_aot.{js,wasm}, driven by examples/dasLLAMA/wasm/run_node.js. No +### LLVM module is embedded, so `require ?llvm` is false and no tune framework exists in it. +### wasm32 is out: the host bakes 8-byte-pointer layouts into the generated C++. + +IF(DAS_LLAMA_INCLUDED AND NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 4) AND NOT (EMSCRIPTEN AND NOT DAS_WASM_MEMORY64)) + if(EMSCRIPTEN) + if(DEFINED DAS_HOST_DASLANG_OVERRIDE) + set(DASLLAMA_AOT_TOOL ${DAS_HOST_DASLANG_OVERRIDE}) + else() + set(DASLLAMA_AOT_TOOL ${PROJECT_SOURCE_DIR}/bin/daslang) + endif() + else() + set(DASLLAMA_AOT_TOOL daslang) + endif() -IF(DAS_LLAMA_INCLUDED AND NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 4)) FILE(GLOB DASLLAMA_AOT_ENGINE_FILES RELATIVE ${PROJECT_SOURCE_DIR} CONFIGURE_DEPENDS "modules/dasLLAMA/dasllama/*.das") list(FILTER DASLLAMA_AOT_ENGINE_FILES EXCLUDE REGEX "dasllama_(vulkan|metal|math_vulkan|gemm_gen|gemm_register|exchange)") + if(NOT DAS_ACCELERATE_INCLUDED) + # the Accelerate driver's C++ calls the das_accelerate binding by symbol; without the + # module (every non-Apple build, wasm included) the guarded require never loads it + list(FILTER DASLLAMA_AOT_ENGINE_FILES EXCLUDE REGEX "dasllama_math_accelerate") + endif() # the daslib modules the CPU path calls at run time that libDaScriptAot does not carry SET(DASLLAMA_AOT_DASLIB_FILES @@ -36,13 +58,18 @@ IF(DAS_LLAMA_INCLUDED AND NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CM # generator reads each C++ module's aotRequire out of the loaded shared module, so those # are inputs too - a module without the override AOT-disables its whole require closure FILE(GLOB DASLLAMA_AOT_ALL_ENGINE_FILES CONFIGURE_DEPENDS "${PROJECT_SOURCE_DIR}/modules/dasLLAMA/dasllama/*.das") - SET(DAS_AOT_EXTRA_DEPENDS ${DASLLAMA_AOT_ALL_ENGINE_FILES} ${DAS_DYN_MODULES_LIBS}) + SET(DAS_AOT_EXTRA_DEPENDS ${DASLLAMA_AOT_ALL_ENGINE_FILES}) + if(NOT EMSCRIPTEN) + LIST(APPEND DAS_AOT_EXTRA_DEPENDS ${DAS_DYN_MODULES_LIBS}) + endif() + # one generated set serves the native and the wasm64 build (same host tool, same sources, + # 8-byte pointers on both): the two trees regenerate the same files add_custom_target(dasllama_aot_engine) SET(DASLLAMA_AOT_ENGINE_GENERATED_SRC) - DAS_AOT_LIB("${DASLLAMA_AOT_ENGINE_FILES}" DASLLAMA_AOT_ENGINE_GENERATED_SRC dasllama_aot_engine daslang) + DAS_AOT_LIB("${DASLLAMA_AOT_ENGINE_FILES}" DASLLAMA_AOT_ENGINE_GENERATED_SRC dasllama_aot_engine ${DASLLAMA_AOT_TOOL}) add_custom_target(dasllama_aot_daslib) SET(DASLLAMA_AOT_DASLIB_GENERATED_SRC) - DAS_AOT_LIB("${DASLLAMA_AOT_DASLIB_FILES}" DASLLAMA_AOT_DASLIB_GENERATED_SRC dasllama_aot_daslib daslang) + DAS_AOT_LIB("${DASLLAMA_AOT_DASLIB_FILES}" DASLLAMA_AOT_DASLIB_GENERATED_SRC dasllama_aot_daslib ${DASLLAMA_AOT_TOOL}) unset(DAS_AOT_EXTRA_DEPENDS) SOURCE_GROUP_FILES("aot generated" DASLLAMA_AOT_ENGINE_GENERATED_SRC) SOURCE_GROUP_FILES("aot generated" DASLLAMA_AOT_DASLIB_GENERATED_SRC) @@ -52,8 +79,27 @@ IF(DAS_LLAMA_INCLUDED AND NOT ${DAS_AOT_EXAMPLES_DISABLED} AND NOT (WIN32 AND CM ${DASLLAMA_AOT_DASLIB_GENERATED_SRC} ) target_sources(dasllama_aot PRIVATE ${PROJECT_SOURCE_DIR}/src/misc/alloc_tracker_overrides.cpp) - TARGET_LINK_LIBRARIES(dasllama_aot libDaScriptAot ${SRC_LIBRARIES} ${DAS_MODULES_LIBS}) - ADD_DEPENDENCIES(dasllama_aot libDaScriptAot dasllama_aot_engine dasllama_aot_daslib) + if(EMSCRIPTEN) + # libDaScriptAot's stubs are generated by the `daslang` target, a .wasm here - the thirteen + # daslib modules it carries interpret in this host instead (none is on the inference path) + TARGET_LINK_LIBRARIES(dasllama_aot libDaScript ${SRC_LIBRARIES} ${DAS_MODULES_LIBS}) + ADD_DEPENDENCIES(dasllama_aot dasllama_aot_engine dasllama_aot_daslib) + set_target_properties(dasllama_aot PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${DAS_WEB_OUTPUT_DIR}) + # the engine's das files and its descriptor ride MEMFS beside daslib (the web build embeds + # daslib and the other module trees for every executable in its scope) + target_link_options(dasllama_aot PRIVATE + "SHELL:--embed-file ${PROJECT_SOURCE_DIR}/modules/dasLLAMA/dasllama@modules/dasLLAMA/dasllama" + "SHELL:--embed-file ${PROJECT_SOURCE_DIR}/modules/dasLLAMA/.das_module@modules/dasLLAMA/.das_module" + # ENV joins the web build's callMain,FS: the node driver sets the engine's knobs + # (DASLLAMA_IMAGE=0 - a mint under the wasm identity must not land beside the model) + -sEXPORTED_RUNTIME_METHODS=callMain,FS,ENV + # the engine's ~100 AOT TUs carry their type tables as static data on top of the + # embedded sources: the web build's 128 MB initial heap is short of it (growth stays on) + -sINITIAL_MEMORY=536870912) + else() + TARGET_LINK_LIBRARIES(dasllama_aot libDaScriptAot ${SRC_LIBRARIES} ${DAS_MODULES_LIBS}) + ADD_DEPENDENCIES(dasllama_aot libDaScriptAot dasllama_aot_engine dasllama_aot_daslib) + endif() # the engine's AOT TUs include the C++ headers of every module in their require closure # (dasAudio's miniaudio, dasMinfft, dasVulkan's volk) - test_aot's dirs and defines plus volk target_include_directories(dasllama_aot PRIVATE ${NEED_MODULES_PATH} ${AUDIO_INCLUDE_DIR} ${CIPIC_HRTF_INCLUDE_DIR} diff --git a/examples/dasLLAMA/wasm/run_node.js b/examples/dasLLAMA/wasm/run_node.js new file mode 100644 index 0000000000..20e9a2018d --- /dev/null +++ b/examples/dasLLAMA/wasm/run_node.js @@ -0,0 +1,45 @@ +// Runs the wasm64 dasllama_aot host under node: the repo mounts at /repo and the model's +// directory at /models through NODEFS, daslib and the engine's das files are embedded in +// the module, and main runs with -use-aot so the engine's AOT stubs bind. +// +// node examples/dasLLAMA/wasm/run_node.js [args...] +// +// output-dir - the directory holding dasllama_aot.js + dasllama_aot.wasm (web/output64) +// script.das - a path inside the repo (examples/dasLLAMA/run.das) +// model.gguf - a host path; its directory is mounted read-write, so the image rail is +// held off (DASLLAMA_IMAGE=0): a mint under the wasm identity would land +// beside the model +const path = require('path'); + +const outputDir = process.argv[2] && path.resolve(process.argv[2]); +const script = process.argv[3]; +const model = process.argv[4] && path.resolve(process.argv[4]); +const extra = process.argv.slice(5); +if (!outputDir || !script || !model) { + console.error('usage: node run_node.js [args...]'); + process.exit(1); +} +const repoRoot = path.resolve(__dirname, '../../..'); + +// emscripten implements exit() by throwing ExitStatus, which surfaces as an unhandled rejection +process.on('unhandledRejection', (reason) => { + if (reason && reason.name === 'ExitStatus') { + process.exit(reason.status); + } + console.error('WASM error:', reason); + process.exit(1); +}); + +const Module = require(path.join(outputDir, 'dasllama_aot.js')); +Module.onRuntimeInitialized = function() { + Module.ENV.DASLLAMA_IMAGE = '0'; + Module.ENV.DASLLAMA_ALLOW_UNTUNED = '1'; + Module.FS.mkdir('/repo'); + Module.FS.mount(Module.FS.filesystems.NODEFS, { root: repoRoot }, '/repo'); + Module.FS.mkdir('/models'); + Module.FS.mount(Module.FS.filesystems.NODEFS, { root: path.dirname(model) }, '/models'); + Module.callMain([ + '-use-aot', path.posix.join('/repo', script), '--', + path.posix.join('/models', path.basename(model)), ...extra + ]); +}; diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index d9d7275981..462b31afa7 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -307,8 +307,23 @@ decode/prefill numbers sit beside the stage-1 floor. ## Stage 3 - WASM64 +**Status: the AOT-through-emcc rail runs (2026-09-04).** The same `dasllama_aot` target builds in +the wasm64 web tree (`web/build64`, `DAS_WASM_MEMORY64` + pthreads): the repo's host daslang +generates the engine C++ (the host-generated set compiles for wasm64 unchanged - 8-byte pointers +on both sides, every layout `static_assert` holds), em++ compiles the ~100 TUs, and the engine's +das files embed into MEMFS beside daslib. `examples/dasLLAMA/wasm/run_node.js` mounts the repo +and the model's directory through NODEFS and calls main with `-use-aot`; on SmolLM2-135M-Q8 the +64 greedy tokens match the native JIT run exactly, decode 46 t/s under node 25 against 54 on the +native AOT host (same box, not a measurement). No dasLLVM is embedded, so `require ?llvm` is +false inside the artifact and no tune framework exists in it. Two link-time findings: the +Accelerate driver's TU stays out wherever `das_accelerate` is not built (its C++ calls the +binding by symbol), and the AOT type tables need a 512 MB initial heap on top of the web build's +embeds. The artifact is 306 MB: the playground's module embeds (fonts, glTF, audio, imgui) ride +along because they are directory-scoped link options in `web/CMakeLists.txt` - a dasLLAMA-only +embed set is the size item. Next: wasmtime (no JS host), the browser page, ASR and TTS examples. + - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + - pthreads), under `wasmtime -W memory64=y -W exceptions=y` first, the browser after. + pthreads), under node first (NODEFS mounts), the browser after. - The GPU tiers already self-gate the way a wasm build needs (verified 2026-09-04 with `--disable-module dasvulkan`: the umbrella loads, `builtin_module_exists(vulkan)` is false, kernel units green interpreted and under the JIT). The Vulkan tier hangs off one guarded From e0b3e57f9ed85a8ad0ce4c0e6d95b4764db6eb0d Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:53:00 -0700 Subject: [PATCH 17/43] the web build's embedded source trees exclude build residue: _aot_generated C++ and .jitted_scripts caches outweighed the .das files twenty to one, and rode into every wasm artifact - dasllama_aot drops from 306 MB to 60 MB, the playground's daslang_static loses the same 95 MB Co-Authored-By: Claude Fable 5.1 --- web/CMakeLists.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/web/CMakeLists.txt b/web/CMakeLists.txt index b6b36b51de..a5002205e0 100644 --- a/web/CMakeLists.txt +++ b/web/CMakeLists.txt @@ -120,6 +120,13 @@ add_link_options("SHELL:--embed-file ${CMAKE_CURRENT_SOURCE_DIR}/../daslib@dasli # itself with "module not found". add_link_options("SHELL:--exclude-file */daslib/spoof.das") add_link_options("SHELL:--exclude-file */daslib/linked_list.das") +# Build residue inside the embedded source trees is not source: the AOT emitter's C++ under +# _aot_generated/ and the JIT's .jitted_scripts/ caches land beside the .das files in a +# working checkout and outweigh them twenty to one (daslib: 3.6 MB of .das against 50 MB of +# generated C++ and 45 MB of cache). Every embedded tree in this scope carries the exclusion. +add_link_options("SHELL:--exclude-file */_aot_generated/*") +add_link_options("SHELL:--exclude-file */.jitted_scripts/*") +add_link_options("SHELL:--exclude-file */_llvm_aot_generated/*") # embed dastest runner so the playground's Test button can invoke # /dastest/dastest.das against user-authored [test] functions. add_link_options("SHELL:--embed-file ${CMAKE_CURRENT_SOURCE_DIR}/../dastest@dastest") From 822577f0e498cd6f9d12bad2fee25ab9598d7c50 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 21:53:11 -0700 Subject: [PATCH 18/43] the plan's stage 3 carries the artifact breakdown after the embed exclusions Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/portable_tier_plan.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 462b31afa7..30a3f7ce74 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -318,9 +318,12 @@ native AOT host (same box, not a measurement). No dasLLVM is embedded, so `requi false inside the artifact and no tune framework exists in it. Two link-time findings: the Accelerate driver's TU stays out wherever `das_accelerate` is not built (its C++ calls the binding by symbol), and the AOT type tables need a 512 MB initial heap on top of the web build's -embeds. The artifact is 306 MB: the playground's module embeds (fonts, glTF, audio, imgui) ride -along because they are directory-scoped link options in `web/CMakeLists.txt` - a dasLLAMA-only -embed set is the size item. Next: wasmtime (no JS host), the browser page, ASR and TTS examples. +embeds. The artifact is 60 MB - 35 MB of code, 24 MB of data (about 10 MB of embedded `.das`, +the rest the AOT type tables). It was 306 MB before the web build's embed step learned to +exclude build residue: `_aot_generated/` C++ and `.jitted_scripts/` caches sit inside the +embedded source trees in a working checkout and outweighed the sources twenty to one; the +playground's `daslang_static` carried the same 95 MB. Next: wasmtime (no JS host), the browser +page, ASR and TTS examples. - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + pthreads), under node first (NODEFS mounts), the browser after. From 362d5f01195a4d3370d1719848081e56ccf6d51c Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 22:30:04 -0700 Subject: [PATCH 19/43] examples/dasLLAMA/speak.das: speak a line locally through the TTS facade and report audio seconds against synthesis seconds; the node driver passes script arguments verbatim with @model standing for the mounted model path Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/speak.das | 77 ++++++++++++++++++++++++++++++ examples/dasLLAMA/wasm/run_node.js | 10 ++-- 2 files changed, 83 insertions(+), 4 deletions(-) create mode 100644 examples/dasLLAMA/speak.das diff --git a/examples/dasLLAMA/speak.das b/examples/dasLLAMA/speak.das new file mode 100644 index 0000000000..c77343cfa6 --- /dev/null +++ b/examples/dasLLAMA/speak.das @@ -0,0 +1,77 @@ +options gen2 +options stack = 524288 // every dasLLAMA program root takes this budget (options stack does not unify up from libs) +options _jit_fast_math = true // ggml-parity FP laxity (non-bit-exact, ~+10%); tests stay bit-exact + +require dasllama/dasllama // the facade: the TTS entry points, the WAV writer, the jobque setup +require daslib/clargs +require daslib/jobque_boost +require daslib/strings_boost +require strings + +// Speak a line locally: load a TTS GGUF (KittenTTS or Kokoro, with tts_g2p.bin and +// tts_postag.bin beside it), synthesize, write a WAV, and report audio seconds against the +// seconds the synthesis took. The voice defaults to the last one the model offers. +// +// bin/daslang -jit examples/dasLLAMA/speak.das -- --model --text "Hello." --out hello.wav + +[CommandLineArgs] +struct SpeakArgs { + @clarg_short = "m" + @clarg_doc = "The TTS model (GGUF or a prepared .dlim)" + model : string + + @clarg_short = "t" + @clarg_doc = "The text to speak" + text : string = "Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine." + + @clarg_short = "o" + @clarg_doc = "The WAV file to write" + out : string = "speech.wav" + + @clarg_short = "v" + @clarg_doc = "Voice name or alias (default: the model's last voice)" + voice : string + + @clarg_short = "s" + @clarg_doc = "Speech speed, 0.25 to 4.0" + speed : float = 1.0 + + @clarg_short = "?" + @clarg_name = "show-help" + @clarg_doc = "Show this help and exit" + help : bool +} + +[export] +def main : int { + var inscope r <- parse_args(type) + if (r |> is_err) { + print("error: {r |> unwrap_err}\n\n") + print_help(get_command_info(type), "speak") + return 1 + } + var inscope cfg <- r |> move_unwrap + if (cfg.help || empty(cfg.model)) { + print_help(get_command_info(type), "speak") + return cfg.help ? 0 : 1 + } + var inscope m <- load_tts_model(cfg.model) + allow_cpu_prefill() + var inscope c <- caps(m) + let voice = empty(cfg.voice) ? c.voices[length(c.voices) - 1] : cfg.voice + print("{length(c.voices)} voices: {join(c.voices, ", ")}; speaking as {voice}\n") + var rc = 0 + with_job_que() { + setup_dasllama_jobque() + var inscope a <- synthesize(m, cfg.text, voice, cfg.speed) + let audio_s = float(length(a.pcm)) / float(a.sample_rate) + let wall_s = float(a.timings.total_us) / 1000000.0 + if (!write_wav_pcm16(cfg.out, a.pcm, a.sample_rate)) { + print("cannot write {cfg.out}\n") + rc = 1 + } else { + print("{cfg.out}: {audio_s} s of audio at {a.sample_rate} Hz, synthesized in {wall_s} s - {audio_s / wall_s}x real time\n") + } + } + return rc +} diff --git a/examples/dasLLAMA/wasm/run_node.js b/examples/dasLLAMA/wasm/run_node.js index 20e9a2018d..7e6c27ccbe 100644 --- a/examples/dasLLAMA/wasm/run_node.js +++ b/examples/dasLLAMA/wasm/run_node.js @@ -9,6 +9,9 @@ // model.gguf - a host path; its directory is mounted read-write, so the image rail is // held off (DASLLAMA_IMAGE=0): a mint under the wasm identity would land // beside the model +// args - the script's own arguments; with none, the mounted model path is passed as +// the one positional argument (run.das), otherwise they are passed verbatim +// with every `@model` replaced by that path (speak.das takes --model @model) const path = require('path'); const outputDir = process.argv[2] && path.resolve(process.argv[2]); @@ -38,8 +41,7 @@ Module.onRuntimeInitialized = function() { Module.FS.mount(Module.FS.filesystems.NODEFS, { root: repoRoot }, '/repo'); Module.FS.mkdir('/models'); Module.FS.mount(Module.FS.filesystems.NODEFS, { root: path.dirname(model) }, '/models'); - Module.callMain([ - '-use-aot', path.posix.join('/repo', script), '--', - path.posix.join('/models', path.basename(model)), ...extra - ]); + const mounted = path.posix.join('/models', path.basename(model)); + const scriptArgs = extra.length ? extra.map((a) => a.replace(/@model/g, mounted)) : [mounted]; + Module.callMain(['-use-aot', path.posix.join('/repo', script), '--', ...scriptArgs]); }; From f523c76c7964e6f192c6a3f1f93d80c44ea304cd Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 22:52:36 -0700 Subject: [PATCH 20/43] the tune sidecar's module-cache pin moves out of the shared manifest reader into the stamping paths, so a standalone exe carries no ast_core extern; release wasm gains release_wasm_disable_module, and the exe rail skips the compiler-lib ast/network modules like ast_core read_manifest registered add_module_cache_dependency on every read - a no-op outside compilation by design, but the runtime shares the reader (the box-profile pin at load, tune_status), and a standalone exe binds every extern its functions name at startup: the ast_core module rode into every dasLLAMA exe, and a wasm cross-link, which sees only the compiler-free runtime archive, failed on jit_register_Module_Ast. The pin now sits beside the read in tune_apply and tune_kernel_pick (pin_module_cache_dependency), and the profile ladder pins through pin_profile_chain at its compile-time site; the runtime readers register nothing, which is all they ever did. The tune framework's manifest, profile, tune and tuned tests stay green. daspkg release wasm: release_wasm_disable_module("dasvulkan") keeps a host module out of the cross-compile so its guarded require resolves as absent, the way the wasm build sees it (the rail disabled dashv itself; an app reaching vulkan, das_metal or das_accelerate names those). The exe emitter's prophylactic skip of ast_core/network_core/jit extends to their das twins ast and network. With both, examples/dasLLAMA/speak.das cross-compiles on the JIT rail to a 19.5 MB wasm64 app and runs under node. Co-Authored-By: Claude Fable 5.1 --- daslib/daspkg.das | 8 +++++++ modules/dasLLVM/ARCHITECTURE.md | 16 ++++++++------ modules/dasLLVM/daslib/llvm_exe.das | 8 +++---- modules/dasLLVM/daslib/llvm_tune.das | 31 +++++++++++++++++++--------- skills/daspkg.md | 1 + utils/daspkg/commands.das | 15 ++++++++++---- utils/daspkg/package_runner.das | 4 ++++ 7 files changed, 59 insertions(+), 24 deletions(-) diff --git a/daslib/daspkg.das b/daslib/daspkg.das index 28e510f2c1..a380551d25 100644 --- a/daslib/daspkg.das +++ b/daslib/daspkg.das @@ -131,6 +131,7 @@ struct ReleaseSpec { emcc_args : array //!< extra emcc flags this module contributes to a `daspkg release wasm` link (e.g. "-sUSE_GLFW=3"); gathered from every referenced module embed_paths : array //!< "src@dst" embed pairs this module needs in the wasm MEMFS (e.g. a HUD font); gathered from every referenced module web_shell : string //!< optional emcc `--shell-file` HTML for `release wasm`; empty = daslang's minimal canvas shell + wasm_disabled_modules : array //!< host module folders (`dasvulkan`) kept unloaded on the `release wasm` cross-compile, so a guarded `require ?name` resolves as absent there wasm_build_command : string //!< shell command that builds this EXTERNAL module's wasm archives (run with cwd = module dir); empty = in-tree module built by `daspkg build --wasm` wasm_archives : array //!< archives required by wasm link; build outputs are module-relative, SDK archives may be named from the staged wasm lib dir requires_jit : bool //!< app is JIT-only (per-box [tune]/[llvm_code] kernels); a baked -exe would run broken, so `daspkg release` refuses it @@ -212,6 +213,13 @@ def release_embed_file(src, dst : string) { _release_spec.embed_paths |> push("{src}@{dst}") } +//! Keep a host module (folder name, case-insensitive) out of the `release wasm` cross-compile: +//! it runs on the host, where a module with no wasm build still satisfies a guarded `require ?name`. +//! dashv is disabled by the rail; an app reaching vulkan, das_metal or das_accelerate names those. +def release_wasm_disable_module(name : string) { + _release_spec.wasm_disabled_modules |> push(name) +} + //! Choose the emcc `--shell-file` HTML wrapper for `release wasm`. Empty (the //! default) ships daslang's minimal canvas-only shell. A site can point this at //! its own shell (e.g. the forge canvas frame) for in-page embedding. diff --git a/modules/dasLLVM/ARCHITECTURE.md b/modules/dasLLVM/ARCHITECTURE.md index 7030b3f89c..d93ae2895c 100644 --- a/modules/dasLLVM/ARCHITECTURE.md +++ b/modules/dasLLVM/ARCHITECTURE.md @@ -156,16 +156,20 @@ have, so the artifact is for another machine and executing it here traps. `[tuned]` and `[tune_policy]` stamp a function's hints at macro time out of the tune sidecar, and the module cache stores the stamped AST. A re-mint therefore has to invalidate the cached record, or a later run serves stamps minted against the old sidecar until some source file -changes. `read_manifest` (`daslib/llvm_tune.das`) registers the sidecar path with -`add_module_cache_dependency` on every read; the record carries the path with the file's byte -size and content hash, and the reader re-validates both before it trusts the payload. Content, -not mtime: an app that rewrites its sidecar byte-identically on exit must not churn the cache. +changes. The stamping paths (`tune_apply`, `tune_kernel_pick`) register the sidecar path with +`add_module_cache_dependency` through `pin_module_cache_dependency` before they read it; the +record carries the path with the file's byte size and content hash, and the reader re-validates +both before it trusts the payload. Content, not mtime: an app that rewrites its sidecar +byte-identically on exit must not churn the cache. The registration runs before the staleness gate, and for a path that does not exist yet, because the mints that matter most produce no successful read - the first mint has no sidecar, and a re-mint replaces one the gate rejected. An absent file registers as size -1 and hash 0, -which the next run's re-validation sees change. Registering is a no-op outside compilation, so -the manifest's runtime readers reach the same call unconditionally. +which the next run's re-validation sees change. The pin sits beside the read, not inside +`read_manifest`: the runtime shares that reader (the box-profile pin at load, `tune_status`), +and a standalone exe binds every extern its functions name at startup, so a reader carrying the +`ast_core` extern would drag the compiler module into every exe - and a wasm cross-link, which +sees only the compiler-free runtime archive, has nothing to bind it to. The shipped defaults profiles are the same kind of input: with no sidecar entry a kernel stamps its class entry out of `/.tune-defaults.json`, so `locate_profile_doc` diff --git a/modules/dasLLVM/daslib/llvm_exe.das b/modules/dasLLVM/daslib/llvm_exe.das index 1dc887ec8e..7152a1c15b 100644 --- a/modules/dasLLVM/daslib/llvm_exe.das +++ b/modules/dasLLVM/daslib/llvm_exe.das @@ -301,8 +301,8 @@ class public CollectExternVisitor : AstVisitor { if (empty(cpp_name)) return let reg_fn_name = "jit_register_{cpp_name}" to_log(LOG_INFO, "LLVM EXE: NEED_MODULE({cpp_name}) for `{mod_name}`\n") - // ast_core / network_core live in the compiler lib, so their jit_register thunk forces a whole-compiler-lib link. - if (mod_name == "ast_core" || mod_name == "network_core") { + // ast_core / ast / network_core / network live in the compiler lib, so their jit_register thunk forces a whole-compiler-lib link. + if (mod_name == "ast_core" || mod_name == "ast" || mod_name == "network_core" || mod_name == "network") { needs_whole_lib = true } // One call per thunk process-wide (not idempotent); get-or-add avoids a silent rename to an undefined symbol. @@ -918,9 +918,9 @@ def private emit_module_registration(m : Module?; dynamic_modules : table key_exists(mod_name)) reg_visited[mod_name] = true - // dylibs register via the runtime load path; ast_core / network_core / jit are prophylactic-skipped here (a real caller still gets them via ensure_module + needs_whole_lib). + // dylibs register via the runtime load path; the compiler-lib modules (ast_core, ast, network_core, network, jit) are prophylactic-skipped here (a real caller still gets them via ensure_module + needs_whole_lib) - an engine's macro-time require of daslib/ast reaches them with no runtime caller, and the wasm runtime archive has no thunk for them. return if (dynamic_modules |> key_exists(mod_name) - || mod_name == "ast_core" || mod_name == "network_core" || mod_name == "jit") + || mod_name == "ast_core" || mod_name == "ast" || mod_name == "network_core" || mod_name == "network" || mod_name == "jit") module_for_each_dependency(m) $(var dep : Module?; var _pub : bool) { emit_module_registration(dep, dynamic_modules, emitted_reg, reg_visited, builder, g_mod, register_mod_type, has_cpp_modules) diff --git a/modules/dasLLVM/daslib/llvm_tune.das b/modules/dasLLVM/daslib/llvm_tune.das index 0c5a7e059b..4d0892786e 100644 --- a/modules/dasLLVM/daslib/llvm_tune.das +++ b/modules/dasLLVM/daslib/llvm_tune.das @@ -352,18 +352,30 @@ def tune_sidecar_stale(path : string) : bool { return box_match_key(sidecar_box(path, "{sfs.mtime}")) != box_match_key(tune_box_identity()) } + //! The compile-time half of a sidecar or profile read: the module cache records the file, existing + //! or not, so a re-mint invalidates the stamps minted against it. Kept out of every reader the + //! runtime shares - a standalone exe binds each extern at startup (the cited section says why). +[arch(at="../ARCHITECTURE.md#tune-sidecar-cache-pin")] +def private pin_module_cache_dependency(path : string) { + add_module_cache_dependency(path) +} + +// every candidate on the class ladder a profile lookup tries, existing or not +def private pin_profile_chain(defaultsDir : string) { + var chain <- tune_class_chain() + for (c in chain) { + pin_module_cache_dependency(path_join(defaultsDir, "{c}.tune-defaults.json")) + } + delete chain +} + // the sidecar's "kernels" section as a flat { function name : perm suffix } map; // found = the file existed, parsed to an object, and is NOT stale vs the running binary // (a "kernels"-less but fresh sidecar counts as found-empty: runtime-knob-only files) -[arch(at="../ARCHITECTURE.md#tune-sidecar-cache-pin")] def private read_manifest(path : string; var found : bool&) : table { found = false var tab : table - if (empty(path)) { - return <- tab - } - add_module_cache_dependency(path) - if (tune_sidecar_stale(path)) { + if (empty(path) || tune_sidecar_stale(path)) { return <- tab } let text = fread(path) @@ -399,15 +411,11 @@ def tune_manifest_get(fname : string) : string { return r } -[arch(at="../ARCHITECTURE.md#tune-sidecar-cache-pin")] def private locate_profile_doc(defaultsDir, verKey, verVal : string; compileTime : bool; var klass, ppath : string&) : JsonValue? { var chain <- tune_class_chain() var doc : JsonValue? for (c in chain) { let p = path_join(defaultsDir, "{c}.tune-defaults.json") - if (compileTime) { - add_module_cache_dependency(p) - } let text = fread(p) continue if (empty(text)) var err = "" @@ -470,6 +478,7 @@ def private profile_kernels(var klass, ppath : string&) : table } var k = "" var p = "" + pin_profile_chain(dir) var doc = locate_profile_doc(dir, verKey, verVal, true, k, p) continue if (doc == null) let ks = doc?["kernels"] @@ -495,6 +504,7 @@ def private profile_kernels(var klass, ppath : string&) : table def tune_kernel_pick(fname : string; var source : string&) : string { var found = false let mpath = tune_manifest_path() + pin_module_cache_dependency(mpath) var tab <- read_manifest(mpath, found) var r = tab?[fname] ?? "" delete tab @@ -1739,6 +1749,7 @@ def tune_apply(var func : FunctionPtr; var _group : ModuleGroup; // nolint:STYLE let fname = "{func.name}" var manifestFound = false let mpath = tune_manifest_path() + pin_module_cache_dependency(mpath) var manifest <- read_manifest(mpath, manifestFound) var suffix = "" var tuneSource = "" // manifest | profile | fallback - the tune_source the stamp records diff --git a/skills/daspkg.md b/skills/daspkg.md index ec03e6a77e..c3d661f257 100644 --- a/skills/daspkg.md +++ b/skills/daspkg.md @@ -345,6 +345,7 @@ def release() { | `release_emcc_arg("-sUSE_GLFW=3")` | Append an emcc link flag (dasGlfw declares this) | | `release_embed_file(src, dst)` | `--embed-file src@dst` into MEMFS (dasStbImage embeds its HUD font) | | `release_web_shell("path.html")` | Override the default canvas shell (`web/templates/wasm_canvas_shell.html`) | +| `release_wasm_disable_module("dasvulkan")` | Keep a host module out of the cross-compile so its guarded `require ?name` resolves as absent, as the wasm build sees it (the rail disables dashv itself; an app reaching vulkan, das_metal or das_accelerate names those) | ### Game source contract (cross-compiles UNCHANGED) diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index 8815952fa1..81faf19b45 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -2665,7 +2665,7 @@ def private ensure_external_wasm_archives(pkg_dir, wasm_lib_dir : string; var bu // Build one app: cross-compile → discover modules → emcc-link → stage assets. def private release_one_wasm_app(root, out_dir, app_name, main_script : string; // nolint:STYLE037,STYLE038 — flat per-app wasm release steps + their error arms wasm_lib_dir, runtime_archive, shell_path : string; - include_globs, exclude_globs : array) : int { + include_globs, exclude_globs, disabled_modules : array) : int { if (!is_safe_pkg_name(app_name)) { to_log(LOG_ERROR, "release wasm: unsafe app name `{app_name}` (no path separators / `.` / `..`)\n") return 1 @@ -2693,14 +2693,21 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; // with --list-shared-modules writing the referenced-module set. // --disable-module dashv: libhv is native-only (no wasm archive), so keep it // unloaded on the cross-compile host — a guarded `require ?dashv` then resolves - // as absent instead of pulling in a module whose wasm archive can't exist. + // as absent instead of pulling in a module whose wasm archive can't exist. The + // package's release_wasm_disable_module() names the others of that kind it reaches. // -project-root {root}: dyn-module discovery roots at the PACKAGE dir, not the // main script's dir. They differ when release_main() points outside the package // (e.g. an example whose .das_package sits beside the source) — without this the // cross-compile scans the script's folder and misses the package's modules/. // --jit-threads: emit +atomics,+bulk-memory so the object's ABI matches the -pthread runtime // archive (the wasm build is threaded — see cmd_build_wasm). The link below adds -pthread. - let xc_cmd = "\"{daslang}\" -exe -output \"{obj_path}\" --list-shared-modules \"{deps_file}\" -project-root \"{root}\" --disable-module dashv \"{main_path}\" -- --jit-target=wasm64-unknown-emscripten --jit-emit-object --jit-threads --jit-runtime-lib=\"{runtime_archive}\"" + let disable_args = build_string() $(w) { + write(w, "--disable-module dashv") + for (m in disabled_modules) { + write(w, " --disable-module {m}") + } + } + let xc_cmd = "\"{daslang}\" -exe -output \"{obj_path}\" --list-shared-modules \"{deps_file}\" -project-root \"{root}\" {disable_args} \"{main_path}\" -- --jit-target=wasm64-unknown-emscripten --jit-emit-object --jit-threads --jit-runtime-lib=\"{runtime_archive}\"" var xc_out : string let xc_rc = run_cmd(xc_cmd, xc_out) if (xc_rc != 0 || !fexist(obj_path)) { @@ -2892,5 +2899,5 @@ def cmd_release_wasm(root : string; out_dir : string; wasm_lib_dir_override : st } mkdir_rec(out_dir) - return release_one_wasm_app(root, out_dir, bundle_name, main_script, wasm_lib_dir, runtime_archive, shell_path, spec.include_globs, spec.exclude_globs) + return release_one_wasm_app(root, out_dir, bundle_name, main_script, wasm_lib_dir, runtime_archive, shell_path, spec.include_globs, spec.exclude_globs, spec.wasm_disabled_modules) } diff --git a/utils/daspkg/package_runner.das b/utils/daspkg/package_runner.das index 132a38bd11..3071052394 100644 --- a/utils/daspkg/package_runner.das +++ b/utils/daspkg/package_runner.das @@ -43,6 +43,7 @@ struct PackageReleaseInfo { emcc_args : array embed_paths : array web_shell : string + wasm_disabled_modules : array wasm_build_command : string wasm_archives : array requires_jit : bool @@ -146,6 +147,9 @@ def run_das_package_release(das_package_path : string; var info : PackageRelease info.embed_paths |> push(clone_string(e)) } info.web_shell = clone_string(src.web_shell) + for (m in src.wasm_disabled_modules) { + info.wasm_disabled_modules |> push(clone_string(m)) + } info.wasm_build_command = clone_string(src.wasm_build_command) for (a in src.wasm_archives) { info.wasm_archives |> push(clone_string(a)) From 5e69cc149e19b4543dd7010bca07ad8c1286effe Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 22:59:06 -0700 Subject: [PATCH 21/43] the node driver sets the engine's knobs in preRun - emscripten builds the C environment from Module.ENV at startup, so a knob set after init never reaches getenv; the plan records the JIT-cross rail, the four-way numbers, and the three findings Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/wasm/run_node.js | 31 +++++++++++++--------- modules/dasLLAMA/portable_tier_plan.md | 36 ++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 12 deletions(-) diff --git a/examples/dasLLAMA/wasm/run_node.js b/examples/dasLLAMA/wasm/run_node.js index 7e6c27ccbe..8ef2694e88 100644 --- a/examples/dasLLAMA/wasm/run_node.js +++ b/examples/dasLLAMA/wasm/run_node.js @@ -8,7 +8,7 @@ // script.das - a path inside the repo (examples/dasLLAMA/run.das) // model.gguf - a host path; its directory is mounted read-write, so the image rail is // held off (DASLLAMA_IMAGE=0): a mint under the wasm identity would land -// beside the model +// beside the model, and the image GC would reap the box's own image // args - the script's own arguments; with none, the mounted model path is passed as // the one positional argument (run.das), otherwise they are passed verbatim // with every `@model` replaced by that path (speak.das takes --model @model) @@ -33,15 +33,22 @@ process.on('unhandledRejection', (reason) => { process.exit(1); }); -const Module = require(path.join(outputDir, 'dasllama_aot.js')); -Module.onRuntimeInitialized = function() { - Module.ENV.DASLLAMA_IMAGE = '0'; - Module.ENV.DASLLAMA_ALLOW_UNTUNED = '1'; - Module.FS.mkdir('/repo'); - Module.FS.mount(Module.FS.filesystems.NODEFS, { root: repoRoot }, '/repo'); - Module.FS.mkdir('/models'); - Module.FS.mount(Module.FS.filesystems.NODEFS, { root: path.dirname(model) }, '/models'); - const mounted = path.posix.join('/models', path.basename(model)); - const scriptArgs = extra.length ? extra.map((a) => a.replace(/@model/g, mounted)) : [mounted]; - Module.callMain(['-use-aot', path.posix.join('/repo', script), '--', ...scriptArgs]); +// the module script adopts a pre-existing global Module: preRun is the one hook that runs before +// emscripten builds the C environment from ENV, so the knobs land there, not after init +const Module = { + preRun: [function() { + Module.ENV.DASLLAMA_IMAGE = '0'; + Module.ENV.DASLLAMA_ALLOW_UNTUNED = '1'; + Module.FS.mkdir('/repo'); + Module.FS.mount(Module.FS.filesystems.NODEFS, { root: repoRoot }, '/repo'); + Module.FS.mkdir('/models'); + Module.FS.mount(Module.FS.filesystems.NODEFS, { root: path.dirname(model) }, '/models'); + }], + onRuntimeInitialized: function() { + const mounted = path.posix.join('/models', path.basename(model)); + const scriptArgs = extra.length ? extra.map((a) => a.replace(/@model/g, mounted)) : [mounted]; + Module.callMain(['-use-aot', path.posix.join('/repo', script), '--', ...scriptArgs]); + }, }; +global.Module = Module; +require(path.join(outputDir, 'dasllama_aot.js')); diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 30a3f7ce74..30baafd0c2 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -325,6 +325,42 @@ embedded source trees in a working checkout and outweighed the sources twenty to playground's `daslang_static` carried the same 95 MB. Next: wasmtime (no JS host), the browser page, ASR and TTS examples. +**The JIT-cross rail runs too (2026-09-04 night).** `daspkg release wasm` on a package whose +main is `examples/dasLLAMA/speak.das` or `run.das` produces a 20 MB wasm64 app (compiled code +only, no compiler, no embeds beyond the modules' own) that runs under node. Two rail fixes +made it so, both general: `release_wasm_disable_module("dasvulkan")` (and dasmetal, +dasaccelerate) keeps host-only modules out of the cross-compile so their guarded requires +resolve as absent, and the tune sidecar's module-cache pin moved out of the manifest reader the +runtime shares (`ast_core`'s `add_module_cache_dependency` rode into every exe; the wasm runtime +archive has no thunk for it). The tune framework is inert on the cross target by the +cross-target rule, so the artifact carries reference bodies through LLVM's own vectorizer. + +Kitten nano, the same line (6.7 s of audio), four workers on wasm, seven native; one box, +contended, not measurements: + +| host | Kitten nano, x real time | stories15M Q8, decode t/s | +|---|---|---| +| native JIT, tuned | 29-37 | 4550 | +| native AOT host (`dasllama_aot`) | 21 | - | +| wasm64, AOT C++ through emcc (node) | 4.0 | 357 | +| wasm64, JIT cross (`daspkg release wasm`, node) | 4.4 | 376-389 | + +Findings on the way: (1) the JIT-cross Kitten artifact dies with SIGILL under node's default +memory64 trap handling and runs clean under `--no-wasm-memory64-trap-handling`; the AOT +artifact never trips it, and the story artifact runs either way - a V8 trap-handler edge or an +access pattern of the emitted code, open, and a browser risk for that rail. (2) emscripten +builds the C environment from `Module.ENV` during startup, so a node driver must set the +engine's knobs in `preRun`, not after the runtime initializes: the first drivers set +`DASLLAMA_IMAGE=0` too late, the wasm runs minted wasm-identity `.dlim` files beside the box's +models, and the image GC reaped the box's own q8 Kitten image (re-minted by the next native +run - the hazard the image rail documents; both drivers now hold the knob in `preRun`, and a +run with the fixed driver mints nothing). (3) Parity is +numeric, not token-exact, across tiers on stories15M: with `_jit_fast_math` off, the JIT's +reference and tuned policies agree with each other, and the native AOT host, the wasm AOT host +and the JIT-cross artifact each diverge from it around token 40 - every tier orders its +reductions differently, and a 15M model's flat logits flip on that. SmolLM-135M matched +token-exact across all three; the frozen parity gates stay per tier. + - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + pthreads), under node first (NODEFS mounts), the browser after. - The GPU tiers already self-gate the way a wasm build needs (verified 2026-09-04 with From 4c435a89d2a4f40073e9e1d622b691f7c42a67fb Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 23:37:56 -0700 Subject: [PATCH 22/43] the storyteller: stories15M writes a children's story on screen two tokens a frame while KittenTTS reads each finished sentence aloud from its own thread; the jobque fork pool is per context, so the speech thread arms it itself or every parallel kernel clones the program (90x slower) Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/storyteller/.das_package | 18 + examples/dasLLAMA/storyteller/main.das | 383 +++++++++++++++++++++ modules/dasLLAMA/dasllama/dasllama.das | 6 +- 3 files changed, 404 insertions(+), 3 deletions(-) create mode 100644 examples/dasLLAMA/storyteller/.das_package create mode 100644 examples/dasLLAMA/storyteller/main.das diff --git a/examples/dasLLAMA/storyteller/.das_package b/examples/dasLLAMA/storyteller/.das_package new file mode 100644 index 0000000000..f98f73fa08 --- /dev/null +++ b/examples/dasLLAMA/storyteller/.das_package @@ -0,0 +1,18 @@ +options gen2 + +require daslib/daspkg + +[export] +def package() { + package_name("storyteller") + package_description("Storyteller: a tiny story LLM writes on screen while KittenTTS reads it aloud - dasGlfw + dasOpenGL + dasAudio over dasLLAMA") +} + +[export] +def release() { + release_main("main.das") // one source for the desktop run AND wasm64 (daspkg release wasm) + // host-only GPU tiers: absent from the wasm build, so their guarded requires resolve as absent + release_wasm_disable_module("dasvulkan") + release_wasm_disable_module("dasmetal") + release_wasm_disable_module("dasaccelerate") +} diff --git a/examples/dasLLAMA/storyteller/main.das b/examples/dasLLAMA/storyteller/main.das new file mode 100644 index 0000000000..231240b155 --- /dev/null +++ b/examples/dasLLAMA/storyteller/main.das @@ -0,0 +1,383 @@ +options gen2 +options persistent_heap +options stack = 524288 // every dasLLAMA program root takes this budget (options stack does not unify up from libs) + +require dasllama/dasllama // the facade: model, session, eval / sample / piece, the TTS entry points +require daslib/jobque_boost +require daslib/strings_boost +require daslib/clargs +require daslib/fio +require daslib/archive +require glfw/glfw_boost +require live/glfw_live +require opengl/opengl_boost +require opengl/opengl_cache +require opengl/opengl_ttf +require audio/audio_boost +require live_host +require math +require daslib/math_boost // ortho_rh / compose for the text layer +require strings + +// Storyteller. A tiny story model (Karpathy's stories15M, Q8) writes a children's story on +// screen a few tokens per frame, and KittenTTS reads each finished sentence aloud in a kid's +// voice while the next one is still being written. Space starts a new story, Escape quits. +// +// bin/daslang -jit examples/dasLLAMA/storyteller/main.das -- --models +// +// holds stories15M-Q8_0.gguf, kitten-nano.gguf, tts_g2p.bin and tts_postag.bin. The +// language model runs on the frame thread in per-frame token budgets, so the loop never +// blocks; speech synthesis runs on its own thread, fed sentences through one stream and +// answering with PCM through another, and the frame thread plays the clips back to back. + +[CommandLineArgs] +struct StoryArgs { + @clarg_short = "m" + @clarg_doc = "Directory holding the story model, the TTS model and its phoneme packs" + models : string = "/Users/borisbatkin/Work/llama.cpp/models" + + @clarg_doc = "The story model file inside --models" + story_model : string = "stories15M-Q8_0.gguf" + + @clarg_doc = "The TTS model file inside --models" + tts_model : string = "kitten-nano.gguf" + + @clarg_doc = "Voice name or alias (default: the model's last voice)" + voice : string + + @clarg_doc = "Start a story at once and quit when it has been read out - the smoke rail" + autoplay : bool + + @clarg_doc = "Stop after this many frames (0 = never) - the smoke rail" + max_frames : int + + @clarg_short = "?" + @clarg_name = "show-help" + @clarg_doc = "Show this help and exit" + help : bool +} + +let PROMPT = "Once upon a time" +let MAX_TOKENS = 220 +let STORY_CONTEXT = 256l // prompt + MAX_TOKENS must fit; stories15M was trained at 256 +let TOKENS_PER_FRAME = 2 // ~120 tokens a second at 60 Hz: readable typing, never a stall +let BOS_TOKEN = 1l // llama2 vocabulary; the TinyStories corpus separates stories with BOS, so it is the stop word too +let EOS_TOKEN = 2l +let WRAP_CHARS = 74 // droidsansmono is monospace: characters are the wrap unit +let TEXT_SIZE = 0.72 +let SMALL_SIZE = 0.52 + +enum Phase { + idle + writing + told +} + +struct Line { + text : string //! a sentence for the speech thread; "" asks it to stop +} + +struct Speech { + pcm : array + rate : int +} + +var g_args = StoryArgs() +var g_model = Model() +var g_session = Session() +var g_params = SamplingParams(temp = 0.8, top_k = 40l, penalty = 1.1) +var g_phase = Phase.idle +var g_lines : array // the story as wrapped screen lines +var g_sentence : array // the pieces of the sentence being written, handed to speech at its end +var g_ntokens = 0 +var g_stories = 0 + +var g_say : Stream? // frame thread -> speech thread: sentences +var g_speech : Stream? // speech thread -> frame thread: clips +var g_speech_done : Channel? // the speech thread signals it before it ends +var g_clips : array // clips waiting for the speaker +var g_now = 0.0 // seconds since start, from frame dt +var g_speaking_until = 0.0 +var g_pending_speech = 0 // sentences sent and not yet played +var g_audio_initialized = false +var g_asch : AudioSystemChannels +var g_font : Font? +var g_frames = 0 +var g_space_was = false +var display_w = 0 +var display_h = 0 + +// --- speech thread --- + +def start_speech_thread(tts_path, voice_pick : string; var say, speech : Stream?; var done : Channel?) { + new_thread() <| @capture(= tts_path, = voice_pick, = say, = speech, = done) { + setup_dasllama_jobque() // the fork-context pool is per context: without it every parallel kernel clones the program + var inscope m <- load_tts_model(tts_path) + var inscope c <- caps(m) + let voice = empty(voice_pick) ? c.voices[length(c.voices) - 1] : voice_pick + var running = true + while (running) { + say |> pop_archive() $(var l : Line&) { + if (empty(l.text)) { + running = false + return + } + var inscope a <- synthesize(m, l.text, voice, 1.0) + var clip = Speech(pcm := a.pcm, rate = a.sample_rate) + speech |> push_archive(clip) + } + } + say |> release() + speech |> release() + done |> notify_and_release() + } +} + +def say(text : string) { + var l = Line(text = text) + g_say |> push_archive(l) + g_pending_speech++ +} + +def poll_speech() { + g_speech |> try_pop() $(bytes) { + var clip : Speech + mem_archive_load(bytes, clip) + g_clips |> emplace(clip) + } + if (g_now >= g_speaking_until && !empty(g_clips)) { + let seconds = float(length(g_clips[0].pcm)) / float(max(g_clips[0].rate, 1)) + var pcm <- g_clips[0].pcm + play_sound_from_pcm(g_clips[0].rate, 1, pcm) + g_clips |> erase(0) + g_speaking_until = g_now + seconds + g_pending_speech-- + } +} + +// --- the story --- + +def start_story() { + delete g_session + g_session <- create_session(g_model) + g_lines |> clear() + g_lines |> push("") + g_sentence |> clear() + g_sentence |> reserve(64) + g_ntokens = 0 + g_stories++ + var prompt <- encode(g_model, PROMPT, true) + eval(g_model, g_session, prompt) + append_text(PROMPT) + g_sentence |> push(PROMPT) + g_phase = Phase.writing +} + +def append_text(piece : string) { + var inscope parts <- split(piece, "\n") + for (i, part in iter_range(parts), parts) { + if (i > 0) { + g_lines |> push("") + } + let li = length(g_lines) - 1 + g_lines[li] = "{g_lines[li]}{part}" + wrap_last_line() + } +} + +// break the last line at its last space before the wrap column, as often as it takes +def wrap_last_line() { + while (length(g_lines[length(g_lines) - 1]) > WRAP_CHARS) { + let li = length(g_lines) - 1 + var head = "" + var tail = "" + peek_data(g_lines[li]) $(d) { + var cut = rfind(slice(d, 0, WRAP_CHARS), " ") + if (cut <= 0) { + cut = WRAP_CHARS + } + head = clone_string(slice(d, 0, cut)) + tail = clone_string(slice(d, cut)) + } + g_lines[li] = head + g_lines |> push(strip(tail)) + } +} + +def sentence_ends(piece : string) : bool { + let t = strip(piece) + return ends_with(t, ".") || ends_with(t, "!") || ends_with(t, "?") || ends_with(t, "\"") && (find(t, ".") >= 0 || find(t, "!") >= 0 || find(t, "?") >= 0) +} + +def write_step() { + var one : array + one |> resize(1) + for (_i in range(TOKENS_PER_FRAME)) { + let tok = sample(g_session, g_params) + if (tok == EOS_TOKEN || tok == BOS_TOKEN || g_ntokens >= MAX_TOKENS) { + finish_story() + return + } + g_ntokens++ + let text = piece(g_model, tok) + append_text(text) + g_sentence |> push(text) + if (sentence_ends(text)) { + say(strip(join(g_sentence, ""))) + g_sentence |> clear() + } + one[0] = tok + eval(g_model, g_session, one) + } +} + +def finish_story() { + let rest = strip(join(g_sentence, "")) + if (!empty(rest)) { + say(rest) + } + g_sentence |> clear() + g_phase = Phase.told +} + +// --- text layer, in design pixels against a 1280x720 reference --- + +def hud_scale() : float { + return max(float(display_h) / 720.0, 0.5) +} + +def text_mvp(x, y, scale : float) : float4x4 { + let projection = ortho_rh(0.0, float(display_w), float(display_h), 0.0, -1.0, 1.0) + let model = compose(float3(x, y, 0.0), float4(0.0, 0.0, 0.0, 1.0), float3(scale, scale, 1.0)) + return projection * model +} + +def draw_text(text : string; x, y : float; size : float; tint : float3) { + return if (g_font == null || empty(text)) + var quads <- (*g_font) |> create_quads(text) + let s = hud_scale() * size + (*g_font) |> draw_quads(quads, text_mvp(x * hud_scale(), y * hud_scale(), s), tint) + delete quads +} + +def draw_story() { + let ink = float3(0.93, 0.9, 0.82) + let dim = float3(0.55, 0.52, 0.48) + let accent = float3(0.98, 0.75, 0.35) + draw_text("storyteller", 60.0, 60.0, 0.6, dim) + var y = 130.0 + let step = 34.0 * TEXT_SIZE / 0.72 + for (ln in g_lines) { + draw_text(ln, 60.0, y, TEXT_SIZE, ink) + y += step + } + let status = (g_phase == Phase.idle ? "press SPACE for a story" + : g_phase == Phase.writing ? "writing..." + : g_pending_speech > 0 || g_now < g_speaking_until ? "reading aloud..." + : "the end - press SPACE for another") + draw_text(status, 60.0, 660.0, SMALL_SIZE, g_phase == Phase.idle ? accent : dim) +} + +// --- lifecycle --- + +[export] +def init() { + var inscope r <- parse_args(type) + if (r |> is_err) { + panic("storyteller: {r |> unwrap_err}") + } + g_args <- r |> move_unwrap + live_create_window("Storyteller", 1280, 720) + cache_ttf_objects() + g_font = cache_font("{get_das_root()}/modules/dasStbImage/fonts/droidsansmono.ttf") + if (!g_audio_initialized) { + g_asch = audio_system_create() + g_audio_initialized = true + } + create_job_que() + setup_dasllama_jobque() + delete g_model + g_model <- load_model(path_join(g_args.models, g_args.story_model), QuantMode.q8) + g_model.config.seq_len = max(g_model.config.seq_len, STORY_CONTEXT) // the session's KV cache is sized to it + allow_cpu_prefill() + g_say = unsafe(stream_create()) + g_speech = unsafe(stream_create()) + g_speech_done = unsafe(channel_create()) + g_speech_done |> append(1) + start_speech_thread(path_join(g_args.models, g_args.tts_model), g_args.voice, g_say, g_speech, g_speech_done) + if (g_args.autoplay) { + start_story() + } +} + +[export] +def update() { + if (!live_begin_frame()) { + return + } + g_frames++ + g_now += get_dt() + live_get_framebuffer_size(display_w, display_h) + glViewport(0, 0, display_w, display_h) + glClearColor(0.07, 0.06, 0.09, 1.0) + glClear(GL_COLOR_BUFFER_BIT) + glDisable(GL_DEPTH_TEST) + glEnable(GL_BLEND) + glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) + + let space = glfwGetKey(live_window, GLFW_KEY_SPACE) == GLFW_PRESS + let space_just = space && !g_space_was + g_space_was = space + if (glfwGetKey(live_window, GLFW_KEY_ESCAPE) == GLFW_PRESS) { + glfwSetWindowShouldClose(live_window, 1) + } + if (space_just && g_phase != Phase.writing) { + start_story() + } + if (g_phase == Phase.writing) { + write_step() + } + poll_speech() + draw_story() + live_end_frame() +} + +[export] +def shutdown() { + say("") // the speech thread's stop word + g_speech_done |> join() + unsafe { + channel_remove(g_speech_done) + stream_remove(g_say) + stream_remove(g_speech) + } + if (g_audio_initialized) { + audio_system_finalize(g_asch.command, g_asch.next_sid) + g_audio_initialized = false + } + delete g_session + delete g_model + destroy_job_que() + live_destroy_window() +} + +def done_for_smoke() : bool { + return true if (g_args.max_frames > 0 && g_frames >= g_args.max_frames) + return g_args.autoplay && g_phase == Phase.told && g_pending_speech == 0 && g_now >= g_speaking_until +} + +// eval_main_loop drives the block once per frame: a blocking while-loop natively, the +// browser's requestAnimationFrame on the web - one main for both +[export] +def main() { + init() + eval_main_loop() { + update() + return false if (done_for_smoke()) + return !exit_requested() + } + shutdown() + if (g_args.autoplay) { + print("{join(g_lines, "\n")}\n") + } +} diff --git a/modules/dasLLAMA/dasllama/dasllama.das b/modules/dasLLAMA/dasllama/dasllama.das index 256d2d3ff2..dcf7c152c2 100644 --- a/modules/dasLLAMA/dasllama/dasllama.das +++ b/modules/dasLLAMA/dasllama/dasllama.das @@ -47,9 +47,9 @@ require dasllama/dasllama_gpu_resident // resident slot-switch/pin surfa // ===== Model, session, tokenizer ===== def setup_dasllama_jobque() { - //! Configure the job queue for dasLLAMA's fork/join matmul dispatch: pooled fork contexts, - //! batched dispatch, and the worker spin-before-park window (``jobque_spin_us``; 0 disables). - //! Call INSIDE ``with_job_que()``, before the first ``generate``/``eval``. + //! Configure the job queue for dasLLAMA's fork/join matmul dispatch: pooled fork contexts, batched dispatch, the worker spin-before-park window (``jobque_spin_us``; 0 disables). + //! Call INSIDE ``with_job_que()``, before the first ``generate``/``eval``. The fork-context pool is PER CONTEXT: + //! a ``new_thread`` body that runs inference calls this again, or every parallel kernel clones the whole program (90x slower). setup_dasllama_jobque_() } From c71e8291f434b33d7b4604acf2a1211ba2628f70 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Fri, 4 Sep 2026 23:58:21 -0700 Subject: [PATCH 23/43] the storyteller in the browser: its own web shell fetches the four model files into MEMFS and waits for the click audio needs, the release rail exports FS and ENV to shells; a fresh session sampled the same story every time, so the seed comes from the clock; BOS stops a TinyStories story, curly quotes turn ASCII for the font, a long story scrolls, a new story drops the old one's clips Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/storyteller/.das_package | 1 + examples/dasLLAMA/storyteller/main.das | 27 ++++- examples/dasLLAMA/storyteller/web_shell.html | 118 +++++++++++++++++++ modules/dasLLAMA/portable_tier_plan.md | 23 ++++ skills/daspkg.md | 2 +- utils/daspkg/commands.das | 4 +- 6 files changed, 167 insertions(+), 8 deletions(-) create mode 100644 examples/dasLLAMA/storyteller/web_shell.html diff --git a/examples/dasLLAMA/storyteller/.das_package b/examples/dasLLAMA/storyteller/.das_package index f98f73fa08..f9213f2e25 100644 --- a/examples/dasLLAMA/storyteller/.das_package +++ b/examples/dasLLAMA/storyteller/.das_package @@ -11,6 +11,7 @@ def package() { [export] def release() { release_main("main.das") // one source for the desktop run AND wasm64 (daspkg release wasm) + release_web_shell("web_shell.html") // fetches the models into MEMFS, then a click starts the program (audio needs the gesture) // host-only GPU tiers: absent from the wasm build, so their guarded requires resolve as absent release_wasm_disable_module("dasvulkan") release_wasm_disable_module("dasmetal") diff --git a/examples/dasLLAMA/storyteller/main.das b/examples/dasLLAMA/storyteller/main.das index 231240b155..f3134f8326 100644 --- a/examples/dasLLAMA/storyteller/main.das +++ b/examples/dasLLAMA/storyteller/main.das @@ -64,6 +64,7 @@ let TOKENS_PER_FRAME = 2 // ~120 tokens a second at 60 Hz: readable typ let BOS_TOKEN = 1l // llama2 vocabulary; the TinyStories corpus separates stories with BOS, so it is the stop word too let EOS_TOKEN = 2l let WRAP_CHARS = 74 // droidsansmono is monospace: characters are the wrap unit +let VISIBLE_LINES = 15 // what fits between the title and the status line at the reference size let TEXT_SIZE = 0.72 let SMALL_SIZE = 0.52 @@ -75,11 +76,13 @@ enum Phase { struct Line { text : string //! a sentence for the speech thread; "" asks it to stop + story : int //! which story it belongs to: a clip from an older story is dropped unheard } struct Speech { pcm : array rate : int + story : int } var g_args = StoryArgs() @@ -123,7 +126,7 @@ def start_speech_thread(tts_path, voice_pick : string; var say, speech : Stream? return } var inscope a <- synthesize(m, l.text, voice, 1.0) - var clip = Speech(pcm := a.pcm, rate = a.sample_rate) + var clip = Speech(pcm := a.pcm, rate = a.sample_rate, story = l.story) speech |> push_archive(clip) } } @@ -134,7 +137,7 @@ def start_speech_thread(tts_path, voice_pick : string; var say, speech : Stream? } def say(text : string) { - var l = Line(text = text) + var l = Line(text = text, story = g_stories) g_say |> push_archive(l) g_pending_speech++ } @@ -143,7 +146,9 @@ def poll_speech() { g_speech |> try_pop() $(bytes) { var clip : Speech mem_archive_load(bytes, clip) - g_clips |> emplace(clip) + if (clip.story == g_stories) { + g_clips |> emplace(clip) + } } if (g_now >= g_speaking_until && !empty(g_clips)) { let seconds = float(length(g_clips[0].pcm)) / float(max(g_clips[0].rate, 1)) @@ -160,12 +165,16 @@ def poll_speech() { def start_story() { delete g_session g_session <- create_session(g_model) + set_seed(g_session, int(ref_time_ticks() % 2147483647l)) // a fresh session samples the same story every time otherwise g_lines |> clear() g_lines |> push("") g_sentence |> clear() g_sentence |> reserve(64) g_ntokens = 0 g_stories++ + g_clips |> clear() // the previous story's unplayed clips; its sentences still in flight come back tagged and drop + g_pending_speech = 0 + g_speaking_until = g_now var prompt <- encode(g_model, PROMPT, true) eval(g_model, g_session, prompt) append_text(PROMPT) @@ -173,6 +182,11 @@ def start_story() { g_phase = Phase.writing } +// the corpus writes curly quotes; the screen font carries ASCII, so they would vanish +def plain_quotes(piece : string) : string { + return piece |> replace("“", "\"") |> replace("”", "\"") |> replace("‘", "'") |> replace("’", "'") +} + def append_text(piece : string) { var inscope parts <- split(piece, "\n") for (i, part in iter_range(parts), parts) { @@ -219,7 +233,7 @@ def write_step() { return } g_ntokens++ - let text = piece(g_model, tok) + let text = plain_quotes(piece(g_model, tok)) append_text(text) g_sentence |> push(text) if (sentence_ends(text)) { @@ -267,8 +281,9 @@ def draw_story() { draw_text("storyteller", 60.0, 60.0, 0.6, dim) var y = 130.0 let step = 34.0 * TEXT_SIZE / 0.72 - for (ln in g_lines) { - draw_text(ln, 60.0, y, TEXT_SIZE, ink) + let first = max(0, length(g_lines) - VISIBLE_LINES) // a long story scrolls: the newest lines stay on screen + for (i in range(first, length(g_lines))) { + draw_text(g_lines[i], 60.0, y, TEXT_SIZE, ink) y += step } let status = (g_phase == Phase.idle ? "press SPACE for a story" diff --git a/examples/dasLLAMA/storyteller/web_shell.html b/examples/dasLLAMA/storyteller/web_shell.html new file mode 100644 index 0000000000..45804596c4 --- /dev/null +++ b/examples/dasLLAMA/storyteller/web_shell.html @@ -0,0 +1,118 @@ + + + + + +storyteller + + + + + +
+

storyteller

+

fetching the models...

+
+ +
+ +{{{ SCRIPT }}} + + diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 30baafd0c2..83a3a343bb 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -361,6 +361,29 @@ and the JIT-cross artifact each diverge from it around token 40 - every tier ord reductions differently, and a 15M model's flat logits flip on that. SmolLM-135M matched token-exact across all three; the frozen parity gates stay per tier. +**The browser demo runs (2026-09-05).** `examples/dasLLAMA/storyteller/` is the first card of +the examples page: stories15M-Q8 writes a children's story on screen two tokens a frame +(dasGlfw + dasOpenGL text), each finished sentence goes to a `new_thread` speech worker, and +Kitten nano reads it in the kid voice through dasAudio while the next one is written; Space +starts a new story. One `main.das` for both targets, the same shape as `examples/games` +(`glfw_live` + `live_host` + `eval_main_loop`); `daspkg release wasm` gives a 27 MB wasm64 +app, the JIT-cross rail. The package's own shell (`web_shell.html`) fetches the four model +files (113 MB: the two ggufs and the g2p / POS-tag packs) into MEMFS with one progress bar +and waits for a click - browsers start audio only from a gesture - so the release rail now +exports `FS` and `ENV` to the shell (`-sEXPORTED_RUNTIME_METHODS=FS,ENV`). In Chrome under a +cross-origin-isolated server: stories at typing speed, Kitten nano at 0.37-0.43x real time +(three sentences of audio per second of synthesis, four workers), the AudioContext running, +no SIGILL - finding (1) did not reproduce in Chrome's trap handling. Three lessons from the +native run, general: the jobque fork-context pool is armed PER CONTEXT, so a `new_thread` body +that runs inference calls `setup_dasllama_jobque()` itself or every parallel kernel clones the +whole program (Kitten at rtf 15 instead of 0.05 - the facade doc-comment says so now); a +`new_thread` lambda's captured `Stream?` / `Channel?` are released by the thread body or the +lambda finalizer panics and takes the process down; a fresh `Session` samples from a fixed seed +(`set_seed` from the clock, or every story is the same story). The TinyStories corpus separates +stories with BOS, so BOS is a stop token beside EOS, and it writes curly quotes the screen +font has no glyphs for. Serving: the models sit beside the page (`models/`, or `?models=`); +the `.dlim` question (half the download for Kitten) and an OPFS reader stay open. + - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + pthreads), under node first (NODEFS mounts), the browser after. - The GPU tiers already self-gate the way a wasm build needs (verified 2026-09-04 with diff --git a/skills/daspkg.md b/skills/daspkg.md index c3d661f257..0b158375c5 100644 --- a/skills/daspkg.md +++ b/skills/daspkg.md @@ -344,7 +344,7 @@ def release() { |---|---| | `release_emcc_arg("-sUSE_GLFW=3")` | Append an emcc link flag (dasGlfw declares this) | | `release_embed_file(src, dst)` | `--embed-file src@dst` into MEMFS (dasStbImage embeds its HUD font) | -| `release_web_shell("path.html")` | Override the default canvas shell (`web/templates/wasm_canvas_shell.html`) | +| `release_web_shell("path.html")` | Override the default canvas shell (`web/templates/wasm_canvas_shell.html`). The link exports `FS` and `ENV`, so a shell's `Module.preRun` can stage files into MEMFS (`Module.FS.writeFile`) and set environment knobs (`Module.ENV.X = ...` - the C environment is built from it at startup, later writes never reach `getenv`); hold the run with `addRunDependency` while fetching | | `release_wasm_disable_module("dasvulkan")` | Keep a host module out of the cross-compile so its guarded `require ?name` resolves as absent, as the wasm build sees it (the rail disables dashv itself; an app reaching vulkan, das_metal or das_accelerate names those) | ### Game source contract (cross-compiles UNCHANGED) diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index 81faf19b45..697256b6d3 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -2807,7 +2807,9 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; // context/jobque mutexes). __wrap_emscripten_futex_wait (src/hal/wasm_thread_malloc.cpp) // busy-polls on the worklet instead, delegating all other threads to the real impl. No // emscripten fork needed — the wrap + shim live in libDaScript_runtime.a (like the malloc lock). - w |> write(" -sMEMORY64=1 -sFULL_ES3 -sMAX_WEBGL_VERSION=2 -sGL_ENABLE_GET_PROC_ADDRESS=1 -sALLOW_MEMORY_GROWTH=1 -sSTACK_SIZE=4MB -fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=0 -sEXIT_RUNTIME=0 -sFORCE_FILESYSTEM=1 -pthread -sPTHREAD_POOL_SIZE=16 -sAUDIO_WORKLET=1 -sWASM_WORKERS=1 -Wl,--wrap=emscripten_futex_wait") + // EXPORTED_RUNTIME_METHODS=FS,ENV: a custom shell stages files into MEMFS and sets the + // program's environment knobs in Module.preRun (the C environment is built from Module.ENV at startup). + w |> write(" -sMEMORY64=1 -sFULL_ES3 -sMAX_WEBGL_VERSION=2 -sGL_ENABLE_GET_PROC_ADDRESS=1 -sALLOW_MEMORY_GROWTH=1 -sSTACK_SIZE=4MB -fwasm-exceptions -sWASM_LEGACY_EXCEPTIONS=0 -sEXIT_RUNTIME=0 -sFORCE_FILESYSTEM=1 -sEXPORTED_RUNTIME_METHODS=FS,ENV -pthread -sPTHREAD_POOL_SIZE=16 -sAUDIO_WORKLET=1 -sWASM_WORKERS=1 -Wl,--wrap=emscripten_futex_wait") for (a in emcc_args) { w |> write(" {a}") } From 8ad4747a07500809e0f3cf6cccb9f40f8ab963e5 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 00:53:08 -0700 Subject: [PATCH 24/43] a TTS image serves without its gguf: the family's driver data (Kitten's speed priors and aliases, Kokoro's symbols) rides the image meta, load_tts_model takes a .dlim path on whichever lane the file was baked for, and dasllama-convert bakes one offline under a --config; the storyteller page fetches two images and the two English packs, no gguf Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/storyteller/main.das | 6 +- examples/dasLLAMA/storyteller/web_shell.html | 8 +- modules/dasLLAMA/ARCHITECTURE_TTS.md | 10 ++- modules/dasLLAMA/dasllama/dasllama_kitten.das | 34 +------- modules/dasLLAMA/dasllama/dasllama_kokoro.das | 27 +------ .../dasLLAMA/dasllama/dasllama_styletts2.das | 65 +++++++++++++-- modules/dasLLAMA/dasllama/dasllama_tts.das | 20 ++--- .../dasLLAMA/dasllama/dasllama_tts_types.das | 12 +++ modules/dasLLAMA/portable_tier_plan.md | 19 ++++- modules/dasLLAMA/tests/test_tts_kitten.das | 38 +++++++++ modules/dasLLAMA/tests/test_tts_kokoro.das | 11 ++- utils/dasllama-convert/main.das | 79 +++++++++++++++++++ 12 files changed, 237 insertions(+), 92 deletions(-) diff --git a/examples/dasLLAMA/storyteller/main.das b/examples/dasLLAMA/storyteller/main.das index f3134f8326..63d1b41268 100644 --- a/examples/dasLLAMA/storyteller/main.das +++ b/examples/dasLLAMA/storyteller/main.das @@ -25,8 +25,10 @@ require strings // // bin/daslang -jit examples/dasLLAMA/storyteller/main.das -- --models // -// holds stories15M-Q8_0.gguf, kitten-nano.gguf, tts_g2p.bin and tts_postag.bin. The -// language model runs on the frame thread in per-frame token budgets, so the loop never +// holds the two models and the front-end packs tts_g2p.bin and tts_postag.bin. A model +// is its gguf (the default names) or a prepared .dlim image baked for the running build's +// identity (dasllama-convert --config) - the web build ships images only, the shell names them. +// The language model runs on the frame thread in per-frame token budgets, so the loop never // blocks; speech synthesis runs on its own thread, fed sentences through one stream and // answering with PCM through another, and the frame thread plays the clips back to back. diff --git a/examples/dasLLAMA/storyteller/web_shell.html b/examples/dasLLAMA/storyteller/web_shell.html index 45804596c4..ed0f3c7427 100644 --- a/examples/dasLLAMA/storyteller/web_shell.html +++ b/examples/dasLLAMA/storyteller/web_shell.html @@ -41,7 +41,9 @@

storyteller

var SCROLL_KEYS = { ArrowUp:1, ArrowDown:1, ArrowLeft:1, ArrowRight:1, ' ':1, Spacebar:1, PageUp:1, PageDown:1, Home:1, End:1 }; window.addEventListener('keydown', function (e) { if (SCROLL_KEYS[e.key]) e.preventDefault(); }, { passive: false }); - var MODEL_FILES = ['stories15M-Q8_0.gguf', 'kitten-nano.gguf', 'tts_g2p.bin', 'tts_postag.bin']; + // the two prepared images (baked for this build's identity by dasllama-convert --config) and the + // two English front-end packs - no gguf crosses the wire + var MODEL_FILES = ['stories15M.dlim', 'kitten-nano.dlim', 'tts_g2p.bin', 'tts_postag.bin']; var modelsBase = new URLSearchParams(location.search).get('models') || 'models/'; if (!modelsBase.endsWith('/')) modelsBase += '/'; @@ -90,10 +92,10 @@

storyteller

}; return c; })(), - arguments: ['--models', '/models', '--autoplay'], + arguments: ['--models', '/models', '--story-model', 'stories15M.dlim', '--tts-model', 'kitten-nano.dlim', '--autoplay'], preRun: [function () { // the C environment is built from Module.ENV at startup: knobs set later never reach getenv - Module.ENV.DASLLAMA_IMAGE = '0'; // no prepared-image sidecar: MEMFS is per visit + Module.ENV.DASLLAMA_IMAGE = '0'; // never mint a sidecar into MEMFS: the images arrive prepared Module.ENV.DASLLAMA_ALLOW_UNTUNED = '1'; // the portable kernels, no tune sidecar Module.FS.mkdir('/models'); Module.addRunDependency('storyteller-models'); diff --git a/modules/dasLLAMA/ARCHITECTURE_TTS.md b/modules/dasLLAMA/ARCHITECTURE_TTS.md index b91f1950a2..e638b8b9d4 100644 --- a/modules/dasLLAMA/ARCHITECTURE_TTS.md +++ b/modules/dasLLAMA/ARCHITECTURE_TTS.md @@ -223,8 +223,14 @@ control). Two lanes, each its own image: `tts-q8`, the served default - the rows weights as Q8_0 quants in an int8 plane repacked for the box's backend, the tag config-bound - and `tts-f32`, the file's planes under a config-free tag, the reference lane the parity rail and the block test hold against, which carries no optimization duty: the rig held q8 at f32 -quality, and f32 costs memory for nothing. The meta blob carries the scalars, the spans and the -voice roster through the leaf structs' own `serialize` overloads; the loader binds every weight +quality, and f32 costs memory for nothing. The meta blob carries the scalars, the spans, the +voice roster and the family's driver data (Kitten's per-voice speed priors and alias names, +Kokoro's symbol table - staged from the GGUF's `kitten.*` / `kokoro.*` keys with the weights) +through the leaf structs' own `serialize` overloads, so an image serves without its GGUF: a +`.dlim` path maps the lane the file was baked on, whatever the policy asks (`load_styletts2` +tries the policy's tag first, then the other), and `dasllama-convert` bakes one offline +(`bake_styletts2_image`, `-o` anywhere) under a supplied `--config` for another box's identity. +The loader binds every weight array as a borrowed view over the mapped plane after the parse (post-load runs before the planes bind), so the carrier and every struct holding it take an explicit `finalize`. The layout fingerprint refuses a struct-shape change by name; a served-layout change that keeps the diff --git a/modules/dasLLAMA/dasllama/dasllama_kitten.das b/modules/dasLLAMA/dasllama/dasllama_kitten.das index 01753361b3..c472c81f56 100644 --- a/modules/dasLLAMA/dasllama/dasllama_kitten.das +++ b/modules/dasLLAMA/dasllama/dasllama_kitten.das @@ -7,10 +7,9 @@ module dasllama_kitten shared public require dasllama/dasllama_lint public require math require strings -require daslib/fio -require dasllama/dasllama_gguf require dasllama/dasllama_common require dasllama/dasllama_unicode +require dasllama/dasllama_tts_types require dasllama/dasllama_styletts2 //! The KittenTTS family (nano, mini): the reference driver's token table, re-spacing and @@ -95,37 +94,6 @@ def kitten_style_row(norm_text : string; rows : int) : int { return clamp(length(cps), 0, rows - 1) } -//! Kitten's driver data beside the shared model: per-voice speed priors and the alias names. -struct KittenFamily { - speed_priors : table - aliases : table -} - -//! Read the `kitten.*` metadata of a converted KittenTTS GGUF. -def load_kitten_family(path : string) : KittenFamily { - var inscope fam = KittenFamily() - let f = fopen(path, "rb") - if (f == null) { - panic("dasLLAMA kitten: cannot open model '{path}'") - } - fmap(f) $(var bytes : array#) { - var inscope m <- parse_gguf_meta(bytes) - var inscope voices <- gguf_str_array(m, bytes, "styletts2.voices") - for (v in voices) { - if (gguf_has(m, "kitten.speed_prior.{v}")) { - fam.speed_priors[v] = gguf_f32(m, bytes, "kitten.speed_prior.{v}") - } - } - var inscope anames <- gguf_str_array(m, bytes, "kitten.alias_names") - var inscope avoices <- gguf_str_array(m, bytes, "kitten.alias_voices") - for (an, av in anames, avoices) { - fam.aliases[an] = av - } - } - fclose(f) - return <- fam -} - //! Resolve a voice name or alias to a canonical voice; "" when unknown. def kitten_voice(fam : KittenFamily; m : StyleTts2Model; name : string) : string { if (styletts2_has_voice(m, name)) { diff --git a/modules/dasLLAMA/dasllama/dasllama_kokoro.das b/modules/dasLLAMA/dasllama/dasllama_kokoro.das index 8c2b02b834..62e3f01b6a 100644 --- a/modules/dasLLAMA/dasllama/dasllama_kokoro.das +++ b/modules/dasLLAMA/dasllama/dasllama_kokoro.das @@ -7,38 +7,13 @@ module dasllama_kokoro shared public require dasllama/dasllama_lint public require math require strings -require daslib/fio -require dasllama/dasllama_gguf require dasllama/dasllama_unicode +require dasllama/dasllama_tts_types //! The Kokoro family (Kokoro-82M): the reference pipeline's symbol table (the front end's own //! inventory - no rewrite), its token wrapping and style-row rule. The model itself is the //! shared StyleTTS2 assembly; the voice packs carry one style row per phoneme count. -//! Kokoro's driver data beside the shared model: codepoint -> token id. -struct KokoroFamily { - sym : table -} - -//! Read the `kokoro.*` metadata of a converted Kokoro GGUF. -def load_kokoro_family(path : string) : KokoroFamily { - var inscope fam = KokoroFamily() - let f = fopen(path, "rb") - if (f == null) { - panic("dasLLAMA kokoro: cannot open model '{path}'") - } - fmap(f) $(var bytes : array#) { - var inscope m <- parse_gguf_meta(bytes) - var inscope cpts <- gguf_int_array(m, bytes, "kokoro.symbol_cpts") - var inscope ids <- gguf_int_array(m, bytes, "kokoro.symbol_ids") - for (c, i in cpts, ids) { - fam.sym[int(c)] = int(i) - } - } - fclose(f) - return <- fam -} - //! The token ids the model reads for a phoneme string in the front end's inventory: every //! symbol the vocabulary knows, in order, wrapped in the pad token on both sides. def kokoro_token_ids(fam : KokoroFamily; ps : string) : array { diff --git a/modules/dasLLAMA/dasllama/dasllama_styletts2.das b/modules/dasLLAMA/dasllama/dasllama_styletts2.das index da09b0050c..d9e9213337 100644 --- a/modules/dasLLAMA/dasllama/dasllama_styletts2.das +++ b/modules/dasLLAMA/dasllama/dasllama_styletts2.das @@ -248,6 +248,8 @@ struct StyleTts2Model { decoder : St2Decoder = St2Decoder() source : SineSourceCfg = SineSourceCfg() q8 : bool // the lane this image was minted on: the rows GEMMs serve Q8_0 quants + kitten : KittenFamily // the family's driver data rides the meta, so an image serves without its gguf + kokoro : KokoroFamily blob : PlaneF = PlaneF() qblob : PlaneI8 = PlaneI8() image_map : void? @@ -266,9 +268,9 @@ def weights_walk(var io : TtsBlobIo; var m : StyleTts2Model) { } } -// serialize_image_meta covers 14 fields; voices (views bound from voice_spans), image_map and +// serialize_image_meta covers 16 fields; voices (views bound from voice_spans), image_map and // image_bytes are the deliberate skips -let private ST2_META_FIELDS = 14 + 3 +let private ST2_META_FIELDS = 16 + 3 let ST2_IMAGE_TAG = "tts-f32" let ST2_IMAGE_TAG_Q8 = "tts-q8" @@ -288,6 +290,9 @@ def private serialize_image_meta(var arch : Archive; var m : StyleTts2Model) { arch |> serialize(m.decoder) arch |> serialize(m.source) arch |> serialize_raw(m.q8) + arch |> serialize(m.kitten.speed_priors) + arch |> serialize(m.kitten.aliases) + arch |> serialize(m.kokoro.sym) } [unused_argument(m)] @@ -357,6 +362,8 @@ def finalize(var m : StyleTts2Model) { delete m.text_encoder delete m.predictor delete m.decoder + delete m.kitten + delete m.kokoro delete m.blob delete m.qblob if (m.image_map != null) { @@ -679,8 +686,8 @@ def private read_weight(m : GGUFMeta; bytes : array | #; name : string) : } //! Read a converted StyleTTS2-lineage GGUF (harness/convert_kitten.py, harness/convert_kokoro.py) -//! into the mint-side carrier: served layouts minted, voices and STFT convention read, every -//! weight moved into the staging blob. Family keys are the family file's to read. +//! into the mint-side carrier: served layouts minted, voices, STFT convention and the family's +//! driver keys read, every weight moved into the staging blob. [arch(at = "../ARCHITECTURE_TTS.md#tts-image-rail")] def stage_styletts2(path : string; q8 : bool = false) : St2Staging { var inscope st = St2Staging() @@ -712,6 +719,7 @@ def stage_styletts2(path : string; q8 : bool = false) : St2Staging { for (v in st.e.voice_names) { st.e.voices[v] <- read_arr(m, bytes, "voice.{v}") } + stage_family_data(m, bytes, st.e) read_albert(m, bytes, st.e.albert) read_text_encoder(m, bytes, st.e.text_encoder) read_predictor(m, bytes, st.e.predictor) @@ -723,6 +731,29 @@ def stage_styletts2(path : string; q8 : bool = false) : St2Staging { return <- st } +// the family's driver keys, staged with the model so they ride its image: Kitten's per-voice +// speed priors and alias names, Kokoro's symbol table +def private stage_family_data(m : GGUFMeta; bytes : array#; var e : StyleTts2Model) { + if (e.family == "kitten-tts") { + for (v in e.voice_names) { + if (gguf_has(m, "kitten.speed_prior.{v}")) { + e.kitten.speed_priors[v] = gguf_f32(m, bytes, "kitten.speed_prior.{v}") + } + } + var inscope anames <- gguf_str_array(m, bytes, "kitten.alias_names") + var inscope avoices <- gguf_str_array(m, bytes, "kitten.alias_voices") + for (an, av in anames, avoices) { + e.kitten.aliases[an] = av + } + } elif (e.family == "kokoro") { + var inscope cpts <- gguf_int_array(m, bytes, "kokoro.symbol_cpts") + var inscope ids <- gguf_int_array(m, bytes, "kokoro.symbol_ids") + for (c, i in cpts, ids) { + e.kokoro.sym[int(c)] = int(i) + } + } +} + //! Mint a served model from a staged read without touching disk - the in-memory image rail, //! the off-rail control for suites. The staged read's lane picks the tag. def mint_styletts2(var st : St2Staging; var out : StyleTts2Model) { @@ -731,13 +762,33 @@ def mint_styletts2(var st : St2Staging; var out : StyleTts2Model) { st2_bind(out) } -//! Load a converted StyleTTS2-lineage GGUF through its prepared image on the current lane: the -//! sibling `.dlim` maps when it exists, else the GGUF stages, bakes and maps -//! (``DASLLAMA_IMAGE=0`` mints in memory). +//! Bake a GGUF's prepared image to `out` on the lane `q8` names - the converter's TTS arm. The +//! box's backend pin (the converter's ``--config``) decides the q8 plane's repack. false = not written. +def bake_styletts2_image(path, out : string; q8 : bool) : bool { + var inscope st <- stage_styletts2(path, q8) + var inscope m = StyleTts2Model() + return cache_via_image_staged(st, m, out, q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG) +} + +//! Load a converted StyleTTS2-lineage GGUF through its prepared image on the current lane (the sibling `.dlim` maps when it +//! exists, else the GGUF stages, bakes and maps; ``DASLLAMA_IMAGE=0`` mints in memory), or a `.dlim` path directly: the +//! file names its lane, and the gguf is not needed. [arch(at = "../ARCHITECTURE_TTS.md#tts-image-rail"), arch(at = "../ARCHITECTURE_IMAGE.md#image-identity-backend-order")] def load_styletts2(path : string) : StyleTts2Model { var inscope m = StyleTts2Model() let q8 = st2_serve_q8() + if (path |> ends_with(".dlim")) { + apply_box_profile_runtime() + let ts = ref_time_ticks() + let first = q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG // the policy's lane first: no wrong-tag decline in the normal case + let second = q8 ? ST2_IMAGE_TAG : ST2_IMAGE_TAG_Q8 + if (!load_image(path, m, first) && !load_image(path, m, second)) { + panic("dasLLAMA styletts2: '{path}' is not a prepared TTS image for this box (identity {image_identity(ST2_IMAGE_TAG_Q8)} / {image_identity(ST2_IMAGE_TAG)}) - bake it from the source gguf with dasllama-convert") + } + st2_bind(m) + to_log(LOG_INFO, "dasLLAMA styletts2: prepared image mapped in {get_time_usec(ts) / 1000} ms on its {m.q8 ? "q8" : "f32"} lane - {path}\n") + return <- m + } let tag = st2_tag() st2_announce_lane() if (g_env_engine.image) { diff --git a/modules/dasLLAMA/dasllama/dasllama_tts.das b/modules/dasLLAMA/dasllama/dasllama_tts.das index 2f8745c79c..ede7d8f38d 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tts.das +++ b/modules/dasLLAMA/dasllama/dasllama_tts.das @@ -31,10 +31,8 @@ struct TtsModel { //! A loaded TTS model: the shared assembly, the family's driver data, and the front-end packs //! read beside the GGUF. Load with ``load_tts_model``; ``delete`` frees every plane and pack. kind : TtsKind //! the family the GGUF's architecture named - model : StyleTts2Model = StyleTts2Model() //! the shared StyleTTS2-lineage assembly and its voices + model : StyleTts2Model = StyleTts2Model() //! the shared StyleTTS2-lineage assembly, its voices and the family's driver data scratch : St2Scratch = St2Scratch() //! the activation carrier every synthesis reuses - kitten : KittenFamily //! the KittenTTS driver data (symbol table, aliases, speed priors) - kokoro : KokoroFamily //! the Kokoro driver data (vocabulary) g2p : G2pModel //! ``tts_g2p.bin``: lexicons, rules and the fallback chain tagger : PosTagger //! ``tts_postag.bin``: the tokenizer and part-of-speech tagger } @@ -45,24 +43,20 @@ def finalize(var m : TtsModel) { //! Free the model's planes, the carrier and both packs; ``delete m`` runs it. delete m.model delete m.scratch - delete m.kitten - delete m.kokoro delete m.g2p delete m.tagger } def load_tts_model(path : string) : TtsModel { - //! Load a TTS GGUF (KittenTTS or Kokoro, picked from its architecture); ``tts_g2p.bin`` and - //! ``tts_postag.bin`` come from the same directory. The rows GEMMs serve as the lane - //! ``tts_serves_q8`` names at the time of the load. + //! Load a TTS model: a converted GGUF (KittenTTS or Kokoro, picked from its architecture) or its prepared + //! ``.dlim`` image, which serves without the GGUF; ``tts_g2p.bin`` and ``tts_postag.bin`` come from the same directory. + //! A GGUF serves the rows GEMMs on the lane ``tts_serves_q8`` names at load time; an image serves the lane it was baked on. var inscope m = TtsModel() m.model <- load_styletts2(path) if (m.model.family == "kitten-tts") { m.kind = TtsKind.kitten - m.kitten <- load_kitten_family(path) } elif (m.model.family == "kokoro") { m.kind = TtsKind.kokoro - m.kokoro <- load_kokoro_family(path) } else { panic("dasLLAMA tts: '{path}' has architecture '{m.model.family}', which is not a TTS family") } @@ -329,9 +323,9 @@ def private synthesize_chunk(var m : TtsModel; norm : string; vname : string; sp let clock = ref_time_ticks() let ps_misaki = g2p_phonemize(m.g2p, m.tagger, norm, voice_lang(m, vname) == "en-gb") let is_kitten = m.kind == TtsKind.kitten - var inscope ids <- is_kitten ? kitten_token_ids(kitten_phonemes_from_misaki(ps_misaki)) : kokoro_token_ids(m.kokoro, ps_misaki) + var inscope ids <- is_kitten ? kitten_token_ids(kitten_phonemes_from_misaki(ps_misaki)) : kokoro_token_ids(m.model.kokoro, ps_misaki) let row = is_kitten ? kitten_style_row(norm, m.model.style_rows) : kokoro_style_row(ps_misaki, m.model.style_rows) - let sp = is_kitten ? speed * (m.kitten.speed_priors?[vname] ?? 1.0) : speed + let sp = is_kitten ? speed * (m.model.kitten.speed_priors?[vname] ?? 1.0) : speed var inscope style : array styletts2_style(m.model, vname, row, style) out.timings.front_end_us = int64(get_time_usec(clock)) @@ -345,7 +339,7 @@ def private synthesize_chunk(var m : TtsModel; norm : string; vname : string; sp } def private resolve_voice(m : TtsModel; voice : string) : string { - let vname = m.kind == TtsKind.kitten ? kitten_voice(m.kitten, m.model, voice) : (styletts2_has_voice(m.model, voice) ? voice : "") + let vname = m.kind == TtsKind.kitten ? kitten_voice(m.model.kitten, m.model, voice) : (styletts2_has_voice(m.model, voice) ? voice : "") if (empty(vname)) { panic("dasLLAMA tts: unknown voice '{voice}'") } diff --git a/modules/dasLLAMA/dasllama/dasllama_tts_types.das b/modules/dasLLAMA/dasllama/dasllama_tts_types.das index c18e62eec1..007d0efed1 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tts_types.das +++ b/modules/dasLLAMA/dasllama/dasllama_tts_types.das @@ -62,3 +62,15 @@ struct TtsNoise { @scratch normal : array //! [samples x harmonics], sample-major captured : bool //! supplied by an oracle dump: a synthesis reads it as is and never redraws } + +//! Kitten's driver data beside the shared model: per-voice speed priors and the alias names. +//! Staged from the GGUF's ``kitten.*`` keys into the carrier, so it rides the prepared image. +struct KittenFamily { + speed_priors : table + aliases : table +} + +//! Kokoro's driver data beside the shared model: codepoint -> token id, from ``kokoro.*``. +struct KokoroFamily { + sym : table +} diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 83a3a343bb..545bb5f82a 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -381,8 +381,23 @@ whole program (Kitten at rtf 15 instead of 0.05 - the facade doc-comment says so lambda finalizer panics and takes the process down; a fresh `Session` samples from a fixed seed (`set_seed` from the clock, or every story is the same story). The TinyStories corpus separates stories with BOS, so BOS is a stop token beside EOS, and it writes curly quotes the screen -font has no glyphs for. Serving: the models sit beside the page (`models/`, or `?models=`); -the `.dlim` question (half the download for Kitten) and an OPFS reader stay open. +font has no glyphs for. Serving: the models sit beside the page (`models/`, or `?models=`). + +**Images only across the wire (2026-09-05).** The page fetches two `.dlim` files and the two +English front-end packs, no gguf. The TTS carrier now serves without its gguf: the family's +driver data (Kitten's speed priors and aliases, Kokoro's symbol table) rides the image meta, +`load_tts_model` takes a `.dlim` path (the file names its lane), and `dasllama-convert` grew a +TTS arm (`-o` anywhere). An image is keyed by the build's identity, so the browser's images +are baked natively against the wasm build's configuration: a scratch script cross-compiled to +wasm64 prints `dlim_config_json` under node (identity +`v32p1|q8|portable|s16|q8 mr4 b0 g4|kq 4/../4|q51 mr1|nat 1101`), and `dasllama-convert +--config wasm.json -o serve/.dlim` bakes against it - the portable backend exists on +every host, so the fail-closed verify passes. Kitten's image is half its gguf (30 MB against +59). The story model's is not: 51 MB against a 26.7 MB gguf, because the image widens the +token-embedding table to f32 (9.2M of the 15.2M parameters) where the gguf holds it as Q8_0. +A q8 token-table plane in the image (the PLE rail already gathers rows from one) is the engine +item that would bring it to ~17 MB; until then the two-image download is 108 MB against the +gguf pair's 113. An OPFS reader stays open. - Route: the stage-2 example's AOT C++ through emcc against `web/output64` (memory64 + pthreads), under node first (NODEFS mounts), the browser after. diff --git a/modules/dasLLAMA/tests/test_tts_kitten.das b/modules/dasLLAMA/tests/test_tts_kitten.das index 37dd6ce43e..7062ab33d4 100644 --- a/modules/dasLLAMA/tests/test_tts_kitten.das +++ b/modules/dasLLAMA/tests/test_tts_kitten.das @@ -10,6 +10,7 @@ require dasllama/dasllama_tts_types require dasllama/dasllama_tts_blocks require dasllama/dasllama_styletts2 require dasllama/dasllama_image // image_path_for - the two lanes' file identities +require dasllama/dasllama_env // g_env_engine.image: the by-path load needs an image on disk require dasllama/dasllama_math // setup_dasllama_jobque_ require daslib/jobque_boost require daslib/defer @@ -291,6 +292,43 @@ def test_kitten_image(t : T?) { let q8_img = image_path_for(gguf_path("nano"), ST2_IMAGE_TAG_Q8) t |> success(f32_img != q8_img, "each lane is its own prepared image:\n {f32_img}\n {q8_img}") } + // ARCHITECTURE_TTS.md sec.2.32: an image serves without its gguf - the family's driver data + // rides the meta, and a .dlim path maps the lane the file was baked on, whatever the policy says + t |> run("nano: the image loads by its own path, the family data riding its meta") @(t : T?) { + if (!stat(gguf_path("nano")).is_valid) { + t |> skip("kitten-nano.gguf not present") + return + } + if (!g_env_engine.image) { + t |> skip("DASLLAMA_IMAGE=0: no image on disk to load by path") + return + } + with_job_que() { + setup_dasllama_jobque_() + var inscope from_gguf <- load_styletts2(gguf_path("nano")) // bakes the q8 sibling when it is not there yet + let img = image_path_for(gguf_path("nano"), ST2_IMAGE_TAG_Q8) + t |> success(stat(img).is_valid, "the q8 lane's image is on disk: {img}") + set_styletts2_q8(false) // the policy asks for f32: the file still serves its own lane + defer() { + reset_styletts2_q8() + } + var inscope m <- load_styletts2(img) + t |> success(m.q8, "the file's lane is the served lane") + t |> equal(length(m.voice_names), 8) + t |> equal(length(m.kitten.aliases), length(from_gguf.kitten.aliases), "the aliases rode the image ({length(m.kitten.aliases)})") + t |> equal(length(m.kitten.speed_priors), length(from_gguf.kitten.speed_priors), "the speed priors rode the image ({length(m.kitten.speed_priors)})") + t |> success(!empty(m.kitten.aliases) && !empty(m.kitten.speed_priors), "the gguf carries family data to compare") + for (an, av in keys(m.kitten.aliases), values(m.kitten.aliases)) { + t |> equal(from_gguf.kitten.aliases?[an] ?? "", av, "alias {an}") + } + if (stat(path_join(models_dir(), "tts_g2p.bin")).is_valid && stat(path_join(models_dir(), "tts_postag.bin")).is_valid) { + var inscope tm <- load_tts_model(img) // the facade: the packs still come from the directory + var inscope c <- caps(tm) + var inscope a <- synthesize(tm, "The image spoke.", c.voices[length(c.voices) - 1], 1.0) + t |> success(long_length(a.pcm) > 8000l, "the facade synthesized {long_length(a.pcm)} samples from the image") + } + } + } } // ARCHITECTURE_TTS.md sec.2.32: q8 is the served default, f32 the reference lane the parity diff --git a/modules/dasLLAMA/tests/test_tts_kokoro.das b/modules/dasLLAMA/tests/test_tts_kokoro.das index 22db4a7f8b..503d58ae63 100644 --- a/modules/dasLLAMA/tests/test_tts_kokoro.das +++ b/modules/dasLLAMA/tests/test_tts_kokoro.das @@ -54,7 +54,7 @@ def private panic_text_of(blk : block<() : void>) : string { // its dialect pick taken out, so a synthesis can be held against the string it should have read. // Noise never moves the length, so the seed here need not be the facade's. def private samples_for(var m : TtsModel; ps : string; voice : string) : int64 { - var inscope ids <- kokoro_token_ids(m.kokoro, ps) + var inscope ids <- kokoro_token_ids(m.model.kokoro, ps) var inscope style : array styletts2_style(m.model, voice, kokoro_style_row(ps, m.model.style_rows), style) var tm = TtsTimings() @@ -131,7 +131,10 @@ def test_kokoro_oracle(t : T?) { return } delete g_family - g_family <- load_kokoro_family(gguf_path()) + { + var inscope staged <- load_styletts2(gguf_path()) // the symbol table rides the model's own carrier + g_family := staged.kokoro + } t |> success(length(g_family.sym) > 100, "{length(g_family.sym)} symbols in the vocabulary") var inscope ids <- kokoro_token_ids(g_family, "hˈɛlO wˈɜɹld") // nolint:STYLE039 t |> equal(length(ids), 14, "two pads around twelve known symbols") @@ -210,8 +213,8 @@ def test_kokoro_dialects(t : T?) { // every British symbol has to be in the model's own vocabulary, or the token // builder drops it silently and the voice speaks a shortened line var inscope cps <- utf8_to_cpts(ps_gb) - var inscope ids_gb <- kokoro_token_ids(m.kokoro, ps_gb) - var inscope ids_us <- kokoro_token_ids(m.kokoro, ps_us) + var inscope ids_gb <- kokoro_token_ids(m.model.kokoro, ps_gb) + var inscope ids_us <- kokoro_token_ids(m.model.kokoro, ps_us) t |> equal(length(ids_gb), length(cps) + 2, "no British symbol falls out of the vocabulary") var same_tokens = length(ids_us) == length(ids_gb) for (a, b in ids_us, ids_gb) { diff --git a/utils/dasllama-convert/main.das b/utils/dasllama-convert/main.das index 32c6400f95..d06e2a40e6 100644 --- a/utils/dasllama-convert/main.das +++ b/utils/dasllama-convert/main.das @@ -30,6 +30,7 @@ require dasllama/dasllama_layout // convert_model_to_metal_blob — bake- require dasllama/dasllama_gguf // GGUFMeta — bake-side, deliberately not facade require dasllama/dasllama_math // pin_backend_for_bake — bake-side, deliberately not facade require dasllama/dasllama_gpu_tier // moe_gpu_bake_tag — bake-side, deliberately not facade +require dasllama/dasllama_styletts2 // bake_styletts2_image — the TTS arm, bake-side, deliberately not facade require daslib/clargs require daslib/jobque_boost require daslib/fio @@ -309,6 +310,80 @@ def private convert_asr(cfg : ConvertArgs; kind : string; qm : QuantMode) : int return 0 } +// "" = not a TTS carrier; else the StyleTTS2-lineage family the GGUF's architecture names +// (kitten-tts | kokoro) - a `styletts2.sample_rate` key is what makes it one +def private sniff_tts_family(path : string) : string { + var st : FStat + if (!stat(path, st) || st.is_dir || int64(st.size) < 8l) { + return "" + } + var family = "" + let f = fopen(path, "rb") + if (f == null) { + return "" + } + fmap(f) $(var bytes : array#) { + if (rd_u32(bytes, 0l) == 0x46554747u) { + var inscope m <- parse_gguf_meta(bytes) + if (gguf_has(m, "styletts2.sample_rate")) { + family = gguf_str(m, bytes, "general.architecture") + } + } + } + fclose(f) + return family +} + +// TTS carriers bake through the family's own staging on the lane -q names (q8 = the served +// default, fp32 = the reference lane); -o lands the image anywhere - a serving directory holds +// two .dlim files and the two front-end packs, no gguf. +def private convert_tts(cfg : ConvertArgs; family : string; qm : QuantMode) : int { + if (cfg.flavor != Flavor.planar) { + to_log(LOG_ERROR, "dasllama-convert: TTS sources have a single (planar) image flavor\n") + return 2 + } + if (cfg.trim || cfg.stream) { + to_log(LOG_ERROR, "dasllama-convert: --trim/--stream apply to decoder GGUFs, not TTS sources\n") + return 2 + } + let q8 = qm == QuantMode.q8 + if (!q8 && qm != QuantMode.fp32) { + to_log(LOG_ERROR, "dasllama-convert: TTS sources bake q8 or fp32\n") + return 2 + } + let tag = q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG + let out = cfg.out != "" ? cfg.out : image_path_for(cfg.model, tag) + to_log(LOG_INFO, "dasllama-convert: {cfg.model} ({family})\n") + to_log(LOG_INFO, "dasllama-convert: -> {out}\n") + to_log(LOG_INFO, "dasllama-convert: identity {image_identity(tag)}\n") + if (!cfg.keep_stale_tmp) { + sweep_stale_tmp(cfg.model) + } + if (stat(out).is_valid) { + if (!cfg.force) { + to_log(LOG_INFO, "dasllama-convert: image already present — nothing to do (use --force to rebuild)\n") + return 0 + } + if (!remove(out)) { + to_log(LOG_ERROR, "dasllama-convert: cannot remove '{out}' for the --force rebuild\n") + return 1 + } + } + let ts = ref_time_ticks() + var baked = false + with_job_que() { + baked = bake_styletts2_image(cfg.model, out, q8) + } + var st : FStat + if (!baked || !stat(out, st)) { + to_log(LOG_ERROR, "dasllama-convert: '{out}' did not land — see the log above\n") + return 1 + } + let total_ms = int64(get_time_usec(ts)) / 1000l + to_log(LOG_INFO, "dasllama-convert: wrote {int64(st.size) >> 20l} MB in {total_ms} ms total\n") + return 0 +} + def private flavor_tag(f : Flavor) : string { if (f == Flavor.metal) { return METAL_IMAGE_TAG @@ -429,6 +504,10 @@ def main() : int { if (asr_kind != "") { return convert_asr(cfg, asr_kind, qm) } + let tts_family = sniff_tts_family(cfg.model) + if (tts_family != "") { + return convert_tts(cfg, tts_family, qm) + } if (cfg.trim && cfg.flavor != Flavor.vulkan) { to_log(LOG_ERROR, "dasllama-convert: --trim is a vulkan-flavor property (add -f vulkan)\n") return 2 From 90896efbfd9c926e41acb8b34155045bb97a9b8e Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 05:25:13 -0700 Subject: [PATCH 25/43] the release rail forwards an app's own release_emcc_arg lines (only module packages' were gathered, so an app-level flag vanished silently); dasllama-convert arms the fork-context pool inside each of its job queues Co-Authored-By: Claude Fable 5.1 --- skills/daspkg.md | 2 +- utils/dasllama-convert/main.das | 1 + utils/daspkg/commands.das | 7 +++++-- 3 files changed, 7 insertions(+), 3 deletions(-) diff --git a/skills/daspkg.md b/skills/daspkg.md index 0b158375c5..814770924b 100644 --- a/skills/daspkg.md +++ b/skills/daspkg.md @@ -338,7 +338,7 @@ def release() { `release_wasm_main` exists because the desktop entry often `require`s host-only stacks (live-reload, libhv HTTP, threaded jobque) that don't cross-compile. The web variant swaps those for a `live_stub` shim (plain GLFW window + frame clock + no-op `[live_command]`) and guards music off (threaded strudel needs the jobque worker; SFX is single-threaded and stays on). One `.das_package` drives both targets. -**Module-side hooks** (in a module's own `.das_package`, accumulated across all linked modules): +**Module-side hooks** (in a module's own `.das_package`, accumulated across all linked modules; an app's own `.das_package` may call `release_emcc_arg` too - its flags come last on the link line, so they win): | Hook | Effect | |---|---| diff --git a/utils/dasllama-convert/main.das b/utils/dasllama-convert/main.das index d06e2a40e6..880d777723 100644 --- a/utils/dasllama-convert/main.das +++ b/utils/dasllama-convert/main.das @@ -372,6 +372,7 @@ def private convert_tts(cfg : ConvertArgs; family : string; qm : QuantMode) : in let ts = ref_time_ticks() var baked = false with_job_que() { + setup_dasllama_jobque() baked = bake_styletts2_image(cfg.model, out, q8) } var st : FStat diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index 697256b6d3..4dedcbd21a 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -2665,7 +2665,7 @@ def private ensure_external_wasm_archives(pkg_dir, wasm_lib_dir : string; var bu // Build one app: cross-compile → discover modules → emcc-link → stage assets. def private release_one_wasm_app(root, out_dir, app_name, main_script : string; // nolint:STYLE037,STYLE038 — flat per-app wasm release steps + their error arms wasm_lib_dir, runtime_archive, shell_path : string; - include_globs, exclude_globs, disabled_modules : array) : int { + include_globs, exclude_globs, disabled_modules, app_emcc_args : array) : int { if (!is_safe_pkg_name(app_name)) { to_log(LOG_ERROR, "release wasm: unsafe app name `{app_name}` (no path separators / `.` / `..`)\n") return 1 @@ -2784,6 +2784,9 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; } } + // the app's own release_emcc_arg() lines come last, after every module's: an app-level flag wins + emcc_args |> push_from(app_emcc_args) + // 4. emcc link → .{html,js,wasm}. run_cmd handles the Windows // cmd.exe outer-quote wrapping, so quoted paths survive on every host. let html_out = path_join(app_out_dir, "{app_name}.html") @@ -2901,5 +2904,5 @@ def cmd_release_wasm(root : string; out_dir : string; wasm_lib_dir_override : st } mkdir_rec(out_dir) - return release_one_wasm_app(root, out_dir, bundle_name, main_script, wasm_lib_dir, runtime_archive, shell_path, spec.include_globs, spec.exclude_globs, spec.wasm_disabled_modules) + return release_one_wasm_app(root, out_dir, bundle_name, main_script, wasm_lib_dir, runtime_archive, shell_path, spec.include_globs, spec.exclude_globs, spec.wasm_disabled_modules, spec.emcc_args) } From c6dc5e6342790b1076c86f55e0408cdc9e8b3775 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 06:23:53 -0700 Subject: [PATCH 26/43] the int8 dot on wasm: the JIT's signed idot family lowers to i32x4.relaxed_dot_i8x16_i7x16_add_s through the sign trick (extmul + pairwise adds where relaxed SIMD is absent), +relaxed-simd joins the wasm feature string, and the portable q8q8 kernels pick a vector-epilogue idot4 dot on a wasm target - 4.8 to 13.8 GMAC/s on one lane, Kitten in Chrome 0.4x to 0.25x real time Co-Authored-By: Claude Fable 5.1 --- .../dasllama/dasllama_math_default.das | 116 +++++++++++++++--- modules/dasLLAMA/portable_tier_plan.md | 27 ++++ modules/dasLLAMA/tests/test_q8q8_family.das | 52 ++++++++ modules/dasLLVM/daslib/llvm_jit_common.das | 9 +- modules/dasLLVM/daslib/llvm_jit_intrin.das | 79 +++++++++++- modules/dasLLVM/daslib/llvm_jit_run.das | 2 +- tests/jit_tests/_wasm_idot_fixture.das | 24 ++++ tests/jit_tests/wasm_idot_lowering.das | 66 ++++++++++ 8 files changed, 355 insertions(+), 20 deletions(-) create mode 100644 tests/jit_tests/_wasm_idot_fixture.das create mode 100644 tests/jit_tests/wasm_idot_lowering.das diff --git a/modules/dasLLAMA/dasllama/dasllama_math_default.das b/modules/dasLLAMA/dasllama/dasllama_math_default.das index 71118c346b..0193ac97bf 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_default.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_default.das @@ -146,6 +146,92 @@ def dot_q8q8_idot4x4_f16s(wq : int8 const?; ws : uint16 const?; xq : int8 const? return f0 + f1 + f2 + f3 } +//! The vector-epilogue idot4 dot: per 32-block one idot4 pair, then float4 accumulate under the block's +//! scale - no horizontal sum until the row ends, two rotating float4 accumulators. The wasm SIMD128 form (7.7 vs the +//! template's 4.8 GMAC/s on one lane: dot_q8q8_idot4x4's per-block hsum is four lane extracts there). Called from lifted lambdas. +[hint(unsafe_range_check, noalias = wq, noalias = xq, noalias = ws, noalias = xs)] +def dot_q8q8_idot4_ps(wq : int8 const?; ws : float const?; xq : int8 const?; xs : float const?; n : int64) : float { + let nb = n / 32l + var f0 = float4(0.0, 0.0, 0.0, 0.0) + var f1 = float4(0.0, 0.0, 0.0, 0.0) + var bi = 0l + unsafe { + let wv = reinterpret(wq) + let xv = reinterpret(xq) + while (bi + 2l <= nb) { + let v = bi * 2l + var a0 = idot4(wv[v], xv[v]) + a0 = idot4(a0, wv[v + 1l], xv[v + 1l]) + var a1 = idot4(wv[v + 2l], xv[v + 2l]) + a1 = idot4(a1, wv[v + 3l], xv[v + 3l]) + let s0 = ws[bi] * xs[bi] + let s1 = ws[bi + 1l] * xs[bi + 1l] + f0 += float4(a0) * float4(s0, s0, s0, s0) + f1 += float4(a1) * float4(s1, s1, s1, s1) + bi += 2l + } + while (bi < nb) { + let v = bi * 2l + var a0 = idot4(wv[v], xv[v]) + a0 = idot4(a0, wv[v + 1l], xv[v + 1l]) + let s0 = ws[bi] * xs[bi] + f0 += float4(a0) * float4(s0, s0, s0, s0) + bi++ + } + } + let f = f0 + f1 + return f.x + f.y + f.z + f.w +} + +//! The wscale_f16 twin of dot_q8q8_idot4_ps: the per-block weight scale is a raw binary16 halfword +//! widened in-loop. Not `private`: called from lifted worker lambdas. +[hint(unsafe_range_check, noalias = wq, noalias = xq, noalias = ws, noalias = xs)] +def dot_q8q8_idot4_ps_f16s(wq : int8 const?; ws : uint16 const?; xq : int8 const?; xs : float const?; n : int64) : float { + let nb = n / 32l + var f0 = float4(0.0, 0.0, 0.0, 0.0) + var f1 = float4(0.0, 0.0, 0.0, 0.0) + var bi = 0l + unsafe { + let wv = reinterpret(wq) + let xv = reinterpret(xq) + while (bi + 2l <= nb) { + let v = bi * 2l + var a0 = idot4(wv[v], xv[v]) + a0 = idot4(a0, wv[v + 1l], xv[v + 1l]) + var a1 = idot4(wv[v + 2l], xv[v + 2l]) + a1 = idot4(a1, wv[v + 3l], xv[v + 3l]) + let s0 = f16_to_f32(uint(ws[bi])) * xs[bi] + let s1 = f16_to_f32(uint(ws[bi + 1l])) * xs[bi + 1l] + f0 += float4(a0) * float4(s0, s0, s0, s0) + f1 += float4(a1) * float4(s1, s1, s1, s1) + bi += 2l + } + while (bi < nb) { + let v = bi * 2l + var a0 = idot4(wv[v], xv[v]) + a0 = idot4(a0, wv[v + 1l], xv[v + 1l]) + let s0 = f16_to_f32(uint(ws[bi])) * xs[bi] + f0 += float4(a0) * float4(s0, s0, s0, s0) + bi++ + } + } + let f = f0 + f1 + return f.x + f.y + f.z + f.w +} + +// On wasm SIMD128 the auto-vectorized template dot is the slow form: the ISA has no int8 dot for +// LLVM to find, while the idot4 builtin lowers there to the ISA's own widening multiply-adds. The +// portable kernels pick per target at compile time; every other host keeps the template. +let private WASM_TARGET = get_target_architecture_name() == "wasm64" || get_target_architecture_name() == "wasm32" + +def private dot_q8q8_p(wq : int8 const?; ws : float const?; xq : int8 const?; xs : float const?; n : int64) : float { + return WASM_TARGET ? dot_q8q8_idot4_ps(wq, ws, xq, xs, n) : dot_q8q8(wq, ws, xq, xs, n) +} + +def private dot_q8q8_f16s_p(wq : int8 const?; ws : uint16 const?; xq : int8 const?; xs : float const?; n : int64) : float { + return WASM_TARGET ? dot_q8q8_idot4_ps_f16s(wq, ws, xq, xs, n) : dot_q8q8_f16s(wq, ws, xq, xs, n) +} + // ----- parallel_for dispatch profiler (off by default) ----- // Records each worker's start time (rel. to pf-start) into g_pf_ws[rb]; a large first/last-worker gap // = workers trickling in (job pool waking from sleep) — the suspected prefill stall. @@ -185,7 +271,7 @@ def private q8q8_kernel(var yp : float?; wp : int8 const?; sp : float const?; xq maybe_parallel_for(0, int(d), matmul_chunks_gemv(int(d), 1, n * d)) $(rb, re) { unsafe { for (i in range(rb, re)) { - myp[i] = dot_q8q8(wp + int64(i) * n, sp + int64(i) * nb, xqp, xsp, n) + myp[i] = dot_q8q8_p(wp + int64(i) * n, sp + int64(i) * nb, xqp, xsp, n) } } } @@ -213,7 +299,7 @@ def private q8q8_batch_kernel(var yp : float?; wp : int8 const?; sp : float cons let wrow = wp + int64(i) * n let srow = sp + int64(i) * nb for (tk in range64(ntok)) { - myp[tk * d + int64(i)] = dot_q8q8(wrow, srow, xqp + tk * n, xsp + tk * nb, n) + myp[tk * d + int64(i)] = dot_q8q8_p(wrow, srow, xqp + tk * n, xsp + tk * nb, n) } } } @@ -242,7 +328,7 @@ def private q8q8_batch_kernel(var yp : float?; wp : int8 const?; sp : float cons let wrow = wp + int64(i) * n let srow = sp + int64(i) * nb for (tk in range64(ntok)) { - myp[tk * d + int64(i)] = dot_q8q8(wrow, srow, xqp + tk * n, xsp + tk * nb, n) + myp[tk * d + int64(i)] = dot_q8q8_p(wrow, srow, xqp + tk * n, xsp + tk * nb, n) } } } @@ -256,7 +342,7 @@ def q8q8_rows_kernel(var yp : float?; wp : int8 const?; sp : float const?; xqp : let nb = n / 32l unsafe { for (i in range64(rb, re)) { - yp[i] = dot_q8q8(wp + i * n, sp + i * nb, xqp, xsp, n) + yp[i] = dot_q8q8_p(wp + i * n, sp + i * nb, xqp, xsp, n) } } } @@ -273,13 +359,13 @@ def private q8q8_group3_kernel(var y0p : float?; var y1p : float?; var y2p : flo for (gi in range(rb, re)) { let g = int64(gi) if (g < d0) { - w0[g] = dot_q8q8(wp + woff0 + g * n, sp + woff0 / 32l + g * nb, xqp, xsp, n) + w0[g] = dot_q8q8_p(wp + woff0 + g * n, sp + woff0 / 32l + g * nb, xqp, xsp, n) } elif (g < d0 + d1) { let r = g - d0 - w1[r] = dot_q8q8(wp + woff1 + r * n, sp + woff1 / 32l + r * nb, xqp, xsp, n) + w1[r] = dot_q8q8_p(wp + woff1 + r * n, sp + woff1 / 32l + r * nb, xqp, xsp, n) } else { let r = g - d0 - d1 - w2[r] = dot_q8q8(wp + woff2 + r * n, sp + woff2 / 32l + r * nb, xqp, xsp, n) + w2[r] = dot_q8q8_p(wp + woff2 + r * n, sp + woff2 / 32l + r * nb, xqp, xsp, n) } } } @@ -295,7 +381,7 @@ def private q8q8_kernel_s16(var yp : float?; wp : int8 const?; sp : uint16 const maybe_parallel_for(0, int(d), matmul_chunks_gemv(int(d), 1, n * d)) $(rb, re) { unsafe { for (i in range(rb, re)) { - myp[i] = dot_q8q8_f16s(wp + int64(i) * n, sp + int64(i) * nb, xqp, xsp, n) + myp[i] = dot_q8q8_f16s_p(wp + int64(i) * n, sp + int64(i) * nb, xqp, xsp, n) } } } @@ -310,7 +396,7 @@ def private q8q8_batch_kernel_s16(var yp : float?; wp : int8 const?; sp : uint16 let wrow = wp + int64(i) * n let srow = sp + int64(i) * nb for (tk in range64(ntok)) { - myp[tk * d + int64(i)] = dot_q8q8_f16s(wrow, srow, xqp + tk * n, xsp + tk * nb, n) + myp[tk * d + int64(i)] = dot_q8q8_f16s_p(wrow, srow, xqp + tk * n, xsp + tk * nb, n) } } } @@ -322,7 +408,7 @@ def q8q8_rows_kernel_s16(var yp : float?; wp : int8 const?; sp : uint16 const?; let nb = n / 32l unsafe { for (i in range64(rb, re)) { - yp[i] = dot_q8q8_f16s(wp + i * n, sp + i * nb, xqp, xsp, n) + yp[i] = dot_q8q8_f16s_p(wp + i * n, sp + i * nb, xqp, xsp, n) } } } @@ -339,13 +425,13 @@ def private q8q8_group3_kernel_s16(var y0p : float?; var y1p : float?; var y2p : for (gi in range(rb, re)) { let g = int64(gi) if (g < d0) { - w0[g] = dot_q8q8_f16s(wp + woff0 + g * n, sp + woff0 / 32l + g * nb, xqp, xsp, n) + w0[g] = dot_q8q8_f16s_p(wp + woff0 + g * n, sp + woff0 / 32l + g * nb, xqp, xsp, n) } elif (g < d0 + d1) { let r = g - d0 - w1[r] = dot_q8q8_f16s(wp + woff1 + r * n, sp + woff1 / 32l + r * nb, xqp, xsp, n) + w1[r] = dot_q8q8_f16s_p(wp + woff1 + r * n, sp + woff1 / 32l + r * nb, xqp, xsp, n) } else { let r = g - d0 - d1 - w2[r] = dot_q8q8_f16s(wp + woff2 + r * n, sp + woff2 / 32l + r * nb, xqp, xsp, n) + w2[r] = dot_q8q8_f16s_p(wp + woff2 + r * n, sp + woff2 / 32l + r * nb, xqp, xsp, n) } } } @@ -363,7 +449,7 @@ def q8q8_groupn_kernel_s16(var yp : float?; wp : int8 const?; sp : uint16 const? let row = ii % d let woff = offs[r * 2l] let xoff = offs[r * 2l + 1l] - var v = dot_q8q8_f16s(wp + woff + row * n, sp + woff / 32l + row * nb, xqp + xoff, xsp + xoff / 32l, n) + var v = dot_q8q8_f16s_p(wp + woff + row * n, sp + woff / 32l + row * nb, xqp + xoff, xsp + xoff / 32l, n) if (bp != null) { v += bp[boffs[r] + row] } @@ -1570,7 +1656,7 @@ def q8q8_groupn_kernel(var yp : float?; wp : int8 const?; sp : float const?; off let row = ii % d let woff = offs[r * 2l] let xoff = offs[r * 2l + 1l] - var v = dot_q8q8(wp + woff + row * n, sp + woff / 32l + row * nb, xqp + xoff, xsp + xoff / 32l, n) + var v = dot_q8q8_p(wp + woff + row * n, sp + woff / 32l + row * nb, xqp + xoff, xsp + xoff / 32l, n) if (bp != null) { v += bp[boffs[r] + row] } diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index 545bb5f82a..c88484cb7f 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -438,6 +438,33 @@ browser demo is a stretch on the same artifact. ## The vectorization leg (between stages 1 and 3, measured) +**First rung landed (2026-09-05): the int8 dot on wasm.** The profile came from node's V8 tick +logger over the JIT-cross Kitten artifact (`--cpu-prof` hangs emscripten's pthread pool at +worker load; `--prof` with `DAS_JOBQUE_THREADS=2` does not): 77% of every tick in one lambda, +the portable `q8q8_batch_kernel`'s row loop, which is `dot_q8q8` inlined - the TTS rows GEMMs. +A one-lane micro-bench (4096-wide row, node, GMAC/s) sized the alternatives: + +| form | native arm64 JIT | wasm64, before | wasm64, after | +|---|---|---|---| +| `dot_q8q8` tuned template (auto-vectorized) | 36 | 4.8 | 4.8 | +| `dot_q8q8_idot4x4` (idot4 builtin, per-block hsum) | 52 | 1.9 | 9.1 | +| `dot_q8q8_idot4_ps` (idot4, vector epilogue, one hsum per row) | 46-54 | - | 13.8 | + +Two changes, both general: the JIT's signed `idot` family gets a wasm lowering - +`i32x4.relaxed_dot_i8x16_i7x16_add_s` through the sign trick (dot(w, x) = dot(sign(x)*w, |x|), +exact because Q8_0 quants sit in [-127, 127]), with the extmul + `extadd_pairwise` chain as the +fallback where relaxed SIMD is absent - and `+relaxed-simd` joins the wasm feature string (every +engine that runs memory64 shipped it first; no ABI moves). The portable kernels pick +`dot_q8q8_idot4_ps` (and its s16 twin) on a wasm target at compile time and keep the template +everywhere else. Gates: `tests/jit_tests/wasm_idot_lowering.das` (the cross dump carries the +relaxed dot, the host dump no wasm intrinsic) and the new `test_q8q8_idot4_ps` cell in the +kernel-family test (fp64 bar, s16 twin bit-identical). End to end: Kitten nano under node on 4 +lanes 0.27x -> 0.14x real time; in Chrome 0.37-0.43x -> 0.23-0.26x (the generator stage 1.1 s +-> 0.57 s per sentence). Not measurements, one box. Next rungs by the same profile: the +remaining 23% is spread thin (Snake's `sin` rows at 2.6%, libm `sinf`, the q8 quantizer), so the +next win is structural - the two-token GEMM shape for the rows conv (`q8q8_batch_kernel` reads +each weight row once per token) - or the story model's decode profile, not yet taken. + Under the JIT the reference loops are auto-vectorized, so stage 1 is the SIMD-achievable floor for the default kernels. AOT and WASM have no vectorizer we control: the emitted C++ goes through clang, gcc or emcc, whose auto-vectorization of daslang loop shapes is hit and miss, diff --git a/modules/dasLLAMA/tests/test_q8q8_family.das b/modules/dasLLAMA/tests/test_q8q8_family.das index dba660e1b2..b296f01d8e 100644 --- a/modules/dasLLAMA/tests/test_q8q8_family.das +++ b/modules/dasLLAMA/tests/test_q8q8_family.das @@ -600,6 +600,58 @@ def test_q8q8_wscale_f16(t : T?) { } } +// The vector-epilogue idot4 dots - the portable kernels' wasm form - on every host: each row of +// each token within the fp64 bar, the s16 twin bit-identical to its f32 twin over the widened plane. +// n=64 is the paired-block main loop alone, n=96 adds the single-block tail. +def private idot4_ps_gate(t : T?; n : int64) { + let d = 64l + let nb = n / 32l + let ntok = 3l + var wf : array + var xf : array + fill_unit(wf, d * n, 0x1D074u) + fill_unit(xf, ntok * n, 0x1D075u) + let wq0 <- quantize_q8_0(wf, d * n) + var ws16m : array + var ws32m : array + s16_planes(wq0.scales, d * nb, ws16m, ws32m) + var xq : array + var xs : array + quantize_rows(xf, ntok, n, xq, xs) + var ref : array + var bar : array + ref_image(wq0.q, ws32m, xq, xs, ntok, d, n, ref, bar) + var y : array + var y16 : array + y |> resize(ntok * d) + y16 |> resize(ntok * d) + unsafe { + let wp = addr(wq0.q[0]) + let sp = addr(ws32m[0]) + let sp16 = addr(ws16m[0]) + let xqp = addr(xq[0]) + let xsp = addr(xs[0]) + for (p in range64(ntok)) { + for (i in range64(d)) { + y[p * d + i] = dot_q8q8_idot4_ps(wp + i * n, sp + i * nb, xqp + p * n, xsp + p * nb, n) + y16[p * d + i] = dot_q8q8_idot4_ps_f16s(wp + i * n, sp16 + i * nb, xqp + p * n, xsp + p * nb, n) + } + } + } + held(t, y, ref, bar, "dot_q8q8_idot4_ps, n={n}") + identical(t, y16, y, "dot_q8q8_idot4_ps_f16s bit-matches its f32 twin over the widened plane, n={n}") + poisoned(t, y, ref, bar, "dot_q8q8_idot4_ps, n={n}") +} + +[test] +def test_q8q8_idot4_ps(t : T?) { + for (n in WIDTHS) { + t |> run("vector-epilogue idot4 dots vs fp64 and the s16 twin, n={n}") @(t : T?) { + idot4_ps_gate(t, n) + } + } +} + def private group3_gate(t : T?; n : int64) { let d0 = 32l let d1 = 40l diff --git a/modules/dasLLVM/daslib/llvm_jit_common.das b/modules/dasLLVM/daslib/llvm_jit_common.das index c18878aee6..abc9958305 100644 --- a/modules/dasLLVM/daslib/llvm_jit_common.das +++ b/modules/dasLLVM/daslib/llvm_jit_common.das @@ -395,6 +395,9 @@ var public g_handled_field_offset_globals : table starts_with("wasm") + g_target_wasm_relaxed_simd = g_target_is_wasm // wasm_target_features always asks for +relaxed-simd // Host path (empty triple): the module targets the host, so the daslang binary's own arch is the // JIT target arch. Explicit cross-compile triple: read the arch off the triple prefix. g_target_is_aarch64 = (empty(target_triple) diff --git a/modules/dasLLVM/daslib/llvm_jit_intrin.das b/modules/dasLLVM/daslib/llvm_jit_intrin.das index 7dbc9b5450..bede372963 100644 --- a/modules/dasLLVM/daslib/llvm_jit_intrin.das +++ b/modules/dasLLVM/daslib/llvm_jit_intrin.das @@ -1540,10 +1540,12 @@ def intrinsic_math_sinh_cosh_tanh(var ctx : JitCtx; expr : ExprCallFunc?; argume } // ===== the idot family: exact integer dots on the 8-bit lattice ===== -// Two lowerings: the signed x signed form emits ONE @llvm.aarch64.neon.sdot where the target has +// Three lowerings: the signed x signed form emits ONE @llvm.aarch64.neon.sdot where the target has // DotProd (g_target_arm64_dotprod: the host rail's +dotprod append, or the force env on the -// generic rail); everything else gets generic widen-multiply IR. The AArch64 backend -// does NOT fold the generic form to SDOT (probe: zip/uzp/smull expansion), hence the native arm. +// generic rail), the wasm SIMD128 form of idot_wasm_simd128 on a wasm target (the relaxed int8 +// dot through the sign trick, else extmul + pairwise adds); everything else gets generic +// widen-multiply IR. The AArch64 backend does NOT fold the generic form to SDOT (probe: +// zip/uzp/smull expansion) and the wasm backend runs it a fifth as fast, hence the native arms. // The unsigned x signed form needs USDOT (i8mm — absent on M1); it rides the i8mm un-gate item. def private idot_products(var ctx : JitCtx; expr : ExprCallFunc?; a, b : LLVMOpaqueValue?; aIdx : int) : LLVMOpaqueValue? { // nolint:LINT014 — shared by the idot intrinsics let v16i32 = LLVMVectorType(ctx.types.t_int32, 16u) @@ -1570,8 +1572,73 @@ def private idot_sdot_aarch64(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : return LLVMBuildCall2(ctx.builder, fnTy, decl, callArgs, "idot4") } +// The wasm SIMD128 form of the signed idot4: the ISA has no int8 dot, but it has the two halves of +// one - i16x8.extmul_{low,high}_i8x16_s (a sign-extending 8-lane multiply, which is what LLVM +// makes of mul(sext lo8, sext lo8)) and i32x4.extadd_pairwise_i16x8_s. Pairwise sums land as +// byte pairs (0-1, 2-3, ...), so one even/odd shuffle-add folds them into the quad lanes the +// generic form defines. Returns null when the LLVM at hand lacks the pairwise intrinsic. +def private idot_wasm_simd128(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : LLVMOpaqueValue? { + let v8i16 = LLVMVectorType(ctx.types.t_int16, 8u) + let v4i32 = ctx.types.LLVMInt4Type() + if (g_target_wasm_relaxed_simd) { + // relaxed SIMD has the dot itself: i32x4.relaxed_dot_i8x16_i7x16_add_s, exact when its second + // operand is in [0, 127]. The sign trick puts it there - dot(w, x) == dot(sign(x)*w, |x|) - and + // Q8_0 quants are in [-127, 127] by construction (d = amax/127), so no lane wraps on the flip. + let rid = LLVMLookupIntrinsicID("llvm.wasm.relaxed.dot.i8x16.i7x16.add.signed") + var noTypes : array + var rdecl = rid != 0u ? LLVMGetIntrinsicDeclaration(g_mod, rid, noTypes) : null + if (rdecl != null) { + let v16i8 = LLVMVectorType(ctx.types.t_int8, 16u) + let mask = LLVMBuildAShr(ctx.builder, b, build_const_int_splat(ctx.types.t_int8, 16, 7), "xsign") + let wsg = LLVMBuildSub(ctx.builder, LLVMBuildXor(ctx.builder, a, mask, ""), mask, "wsigned") + let xabs = LLVMBuildSub(ctx.builder, LLVMBuildXor(ctx.builder, b, mask, ""), mask, "xabs") + var fnArgTypes <- [v16i8, v16i8, v4i32] + var fnTy = LLVMFunctionType(v4i32, fnArgTypes) + var callArgs <- [wsg, xabs, acc != null ? acc : LLVMConstNull(v4i32)] + return LLVMBuildCall2(ctx.builder, fnTy, rdecl, callArgs, "idot4_relaxed") + } + } + let id = LLVMLookupIntrinsicID("llvm.wasm.extadd.pairwise.signed") + var declTypes <- [v4i32] // overloaded on the widened result vector type + var decl = id != 0u ? LLVMGetIntrinsicDeclaration(g_mod, id, declTypes) : null + if (decl == null) { + return null + } + var fnArgTypes <- [v8i16] + var fnTy = LLVMFunctionType(v4i32, fnArgTypes) + let alo = LLVMBuildSExt(ctx.builder, LLVMBuildShuffleVector(ctx.builder, ctx.types, a, a, [0, 1, 2, 3, 4, 5, 6, 7], ""), v8i16, "") + let ahi = LLVMBuildSExt(ctx.builder, LLVMBuildShuffleVector(ctx.builder, ctx.types, a, a, [8, 9, 10, 11, 12, 13, 14, 15], ""), v8i16, "") + let blo = LLVMBuildSExt(ctx.builder, LLVMBuildShuffleVector(ctx.builder, ctx.types, b, b, [0, 1, 2, 3, 4, 5, 6, 7], ""), v8i16, "") + let bhi = LLVMBuildSExt(ctx.builder, LLVMBuildShuffleVector(ctx.builder, ctx.types, b, b, [8, 9, 10, 11, 12, 13, 14, 15], ""), v8i16, "") + var plo <- [LLVMBuildMul(ctx.builder, alo, blo, "idot_lo")] + var phi <- [LLVMBuildMul(ctx.builder, ahi, bhi, "idot_hi")] + let qlo = LLVMBuildCall2(ctx.builder, fnTy, decl, plo, "idot_pairs_lo") + let qhi = LLVMBuildCall2(ctx.builder, fnTy, decl, phi, "idot_pairs_hi") + let even = LLVMBuildShuffleVector(ctx.builder, ctx.types, qlo, qhi, [0, 2, 4, 6], "") + let odd = LLVMBuildShuffleVector(ctx.builder, ctx.types, qlo, qhi, [1, 3, 5, 7], "") + var s = LLVMBuildAdd(ctx.builder, even, odd, "idot4") + if (acc != null) { + s = LLVMBuildAdd(ctx.builder, acc, s, "idot4_acc") + } + return s +} + def intrinsic_lattice_idot(var ctx : JitCtx; expr : ExprCallFunc?; arguments : array) : LLVMOpaqueValue? { let aUnsigned = expr.arguments[0]._type.vectorBaseType == Type.tUInt8 + if (g_target_is_wasm && !aUnsigned) { + let v4i32 = ctx.types.LLVMInt4Type() + var s = idot_wasm_simd128(ctx, null, arguments[0], arguments[1]) + if (s != null) { + let rid = LLVMLookupIntrinsicID("llvm.vector.reduce.add") + var rtys <- [v4i32] + var rdecl = rid != 0u ? LLVMGetIntrinsicDeclaration(g_mod, rid, rtys) : null + if (rdecl != null) { + var rtyp = LLVMFunctionType(ctx.types.t_int32, [v4i32]) + var rargs <- [s] + return LLVMBuildCall2(ctx.builder, rtyp, rdecl, rargs, "idot") + } + } + } if (g_target_arm64_dotprod && !aUnsigned) { let v4i32 = ctx.types.LLVMInt4Type() var s = idot_sdot_aarch64(ctx, LLVMConstNull(v4i32), arguments[0], arguments[1]) @@ -1604,6 +1671,12 @@ def intrinsic_lattice_idot4(var ctx : JitCtx; expr : ExprCallFunc?; arguments : let hasAcc = length(arguments) == 3 let aIdx = hasAcc ? 1 : 0 let aUnsigned = expr.arguments[aIdx]._type.vectorBaseType == Type.tUInt8 + if (g_target_is_wasm && !aUnsigned) { + var s = idot_wasm_simd128(ctx, hasAcc ? arguments[0] : null, arguments[aIdx], arguments[aIdx + 1]) + if (s != null) { + return s + } + } if (g_target_arm64_dotprod && !aUnsigned) { let v4i32 = ctx.types.LLVMInt4Type() var acc = hasAcc ? arguments[0] : LLVMConstNull(v4i32) diff --git a/modules/dasLLVM/daslib/llvm_jit_run.das b/modules/dasLLVM/daslib/llvm_jit_run.das index 9363c7ca85..3906aabe3f 100644 --- a/modules/dasLLVM/daslib/llvm_jit_run.das +++ b/modules/dasLLVM/daslib/llvm_jit_run.das @@ -37,7 +37,7 @@ var LINK_WHOLE_LIB = false // when true, standalone exe links against the whole // invalidates cached DLLs (e.g. edits to llvm_jit.das, llvm_macro.das, llvm_jit_common.das, // runtime helper ABI, default target triple). Cache filenames fold this in, so a bump // makes every previously written DLL miss the cache on the next run and get GC'd. -let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x6eul // the aarch64 SDOT / SMMLA tables gate on DotProd / i8mm, not the arch alone, and the force env reaches the generic exe machine (0x6d: the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) +let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x6ful // the signed idot family lowers to extmul + extadd_pairwise on wasm SIMD128 (0x6e: the aarch64 SDOT / SMMLA tables gate on DotProd / i8mm, not the arch alone, and the force env reaches the generic exe machine (0x6d: the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) // Read by tests-cpp/small/test_jit_emitter_pin.cpp: FNV-1a64 of the emitter sources // (normalized to LF; file list in the test) diff --git a/tests/jit_tests/_wasm_idot_fixture.das b/tests/jit_tests/_wasm_idot_fixture.das new file mode 100644 index 0000000000..89db0182bb --- /dev/null +++ b/tests/jit_tests/_wasm_idot_fixture.das @@ -0,0 +1,24 @@ +// The fixture wasm_idot_lowering.das compiles for the wasm64 cross target and for the host, dumping +// the IR: one signed idot4 over byte16 operands the emitter cannot fold, since the bytes arrive +// through the command line's length. +options gen2 + +def taint(n : int) : byte16 { + var b : byte16 + unsafe { + var p = addr(b) + for (i in range(16)) { + p[i] = int8(i * 3 - n) + } + } + return b +} + +[export] +def main() { + let args <- get_command_line_arguments() + let w = taint(length(args)) + let x = taint(length(args) + 5) + let d = idot4(w, x) + print("IDOT4=[{d.x} {d.y} {d.z} {d.w}]\n") +} diff --git a/tests/jit_tests/wasm_idot_lowering.das b/tests/jit_tests/wasm_idot_lowering.das new file mode 100644 index 0000000000..a373684a56 --- /dev/null +++ b/tests/jit_tests/wasm_idot_lowering.das @@ -0,0 +1,66 @@ +// The signed idot family on the wasm64 cross target lowers to the ISA's own int8 dot, +// i32x4.relaxed_dot_i8x16_i7x16_add_s (the sign trick makes the second operand 7-bit), and never +// to the generic widen-multiply chain that runs a fifth as fast there; a host compile of the same +// fixture emits no wasm intrinsic. Compile-only keeps both compiles on this host and never touches +// the artifact cache, so --jit-dump prints every time. +options gen2 +options no_aot + +require dastest/testing_boost +require daslib/fio +require daslib/strings_boost +require strings + +let FIXTURE = "{get_das_root()}/tests/jit_tests/_wasm_idot_fixture.das" +let RELAXED_DOT = "llvm.wasm.relaxed.dot.i8x16.i7x16.add.signed" + +def private spawn_child(cmd : string; var lines : array) : int { + var rc : int + unsafe { + rc = popen_timeout("{cmd} 2>&1", 300.0) $(f) { + if (f == null) { + return + } + while (!feof(f)) { + let ln = strip(fgets(f)) + if (!empty(ln)) { + lines |> push("{ln}") + } + } + } + } + return rc +} + +def private count_with(lines : array; needle : string) : int { + var n = 0 + for (ln in lines) { + if (find(ln, needle) >= 0) { + n++ + } + } + return n +} + +[test] +def test_wasm_idot_lowering(t : T?) { + if (!jit_enabled()) { + t |> success(true) + return + } + let args <- get_command_line_arguments() // locked view — do not delete + let bin = args[0] + t |> run("the wasm64 cross compile lowers idot4 to the relaxed int8 dot") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-target=wasm64-unknown-emscripten --jit-compile-only --jit-dump", ln) + let dots = count_with(ln, RELAXED_DOT) + t |> success(dots >= 2, "the declaration and the call of {RELAXED_DOT} are in the dump ({dots} lines, {length(ln)} lines total)") + t |> success(count_with(ln, "idot_prod") == 0, "no generic widen-multiply chain remains") + } + t |> run("a host compile emits no wasm intrinsic") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-compile-only --jit-dump", ln) + t |> success(length(ln) > 20, "the host dump printed ({length(ln)} lines)") + t |> success(count_with(ln, "llvm.wasm.") == 0, "no wasm intrinsic on the host") + } +} From 7bd180f6fecd1a742a01dcbf66ae2c85744bdcb9 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 06:42:39 -0700 Subject: [PATCH 27/43] the American-only phoneme pack tts_g2p_en_us.bin (10.3 MB against the full pack's 14.0): build_g2p_data.py --dialect us drops the British values and prunes CMUdict of every word the American lexicon carries, the reader refuses British on it by name, the facade takes whichever pack sits beside the model, the web shell fetches the American one; the publish set, the card and the provenance table carry it Co-Authored-By: Claude Fable 5.1 --- examples/dasLLAMA/storyteller/web_shell.html | 2 +- modules/dasLLAMA/THIRD_PARTY_NOTICES.md | 2 +- modules/dasLLAMA/dasllama/dasllama_g2p.das | 5 +++ modules/dasLLAMA/dasllama/dasllama_tts.das | 22 +++++++--- modules/dasLLAMA/harness/build_g2p_data.py | 18 ++++++-- modules/dasLLAMA/harness/publish_tts_hf.py | 2 +- modules/dasLLAMA/harness/tts_model_card.md | 5 ++- .../dasLLAMA/performance/build_tts_data.das | 6 ++- modules/dasLLAMA/performance/model_specs.das | 3 ++ modules/dasLLAMA/tests/test_tts_g2p.das | 41 +++++++++++++++++++ 10 files changed, 91 insertions(+), 15 deletions(-) diff --git a/examples/dasLLAMA/storyteller/web_shell.html b/examples/dasLLAMA/storyteller/web_shell.html index ed0f3c7427..a853eba811 100644 --- a/examples/dasLLAMA/storyteller/web_shell.html +++ b/examples/dasLLAMA/storyteller/web_shell.html @@ -43,7 +43,7 @@

storyteller

// the two prepared images (baked for this build's identity by dasllama-convert --config) and the // two English front-end packs - no gguf crosses the wire - var MODEL_FILES = ['stories15M.dlim', 'kitten-nano.dlim', 'tts_g2p.bin', 'tts_postag.bin']; + var MODEL_FILES = ['stories15M.dlim', 'kitten-nano.dlim', 'tts_g2p_en_us.bin', 'tts_postag.bin']; var modelsBase = new URLSearchParams(location.search).get('models') || 'models/'; if (!modelsBase.endsWith('/')) modelsBase += '/'; diff --git a/modules/dasLLAMA/THIRD_PARTY_NOTICES.md b/modules/dasLLAMA/THIRD_PARTY_NOTICES.md index 31916a541b..b5a7b38993 100644 --- a/modules/dasLLAMA/THIRD_PARTY_NOTICES.md +++ b/modules/dasLLAMA/THIRD_PARTY_NOTICES.md @@ -35,7 +35,7 @@ derive from the Silero VAD project (https://github.com/snakers4/silero-vad), Cop ## Text-to-speech front end The TTS front end and the two TTS families reimplement, and pack data from, the following -works. The packed files (`tts_g2p.bin`, `tts_postag.bin`, `kitten-nano.gguf`, +works. The packed files (`tts_g2p.bin` and its American-only twin `tts_g2p_en_us.bin`, `tts_postag.bin`, `kitten-nano.gguf`, `kitten-mini.gguf`, `kokoro-82m.gguf`, the `tts_oracle/` dumps) are built by the `harness/` scripts into the model store (`performance/build_tts_data.das`), each beside a license file naming these sources; nothing under this repository redistributes the data itself. diff --git a/modules/dasLLAMA/dasllama/dasllama_g2p.das b/modules/dasLLAMA/dasllama/dasllama_g2p.das index 28ae44a3b9..b9df30d5ef 100644 --- a/modules/dasLLAMA/dasllama/dasllama_g2p.das +++ b/modules/dasLLAMA/dasllama/dasllama_g2p.das @@ -38,6 +38,7 @@ struct LexTable { struct G2pModel { data : array source : string + us_only : bool //! the pack carries the American tier alone (build_g2p_data.py --dialect us): British is refused by name gold : LexTable silver : LexTable cmu : LexTable @@ -237,6 +238,7 @@ def load_g2p(path : string) : G2pModel { let src_len = rd_u16(g.data, o) o += 2l g.source = bytes_str(g.data, o, src_len) + g.us_only = find(g.source, "dialect us") >= 0 o += int64(src_len) g.gold = read_lex_table(g.data, o) g.silver = read_lex_table(g.data, o) @@ -1756,6 +1758,9 @@ def g2p_tokens(g : G2pModel; tagged : array; british : bool = false) : //! Phonemes for normalized text, in the inventory `british` names: tokens joined with the //! source spacing. def g2p_phonemize(g : G2pModel; tg : PosTagger; text : string; british : bool = false) : string { + if (british && g.us_only) { + panic("dasLLAMA tts: this phoneme pack carries the American tier only ({g.source}) - British English needs the full tts_g2p.bin") + } var inscope tagged <- pos_tokenize_tag(tg, text) var inscope toks <- g2p_tokens(g, tagged, british) return build_string() $(var w) { diff --git a/modules/dasLLAMA/dasllama/dasllama_tts.das b/modules/dasLLAMA/dasllama/dasllama_tts.das index ede7d8f38d..af55a07d4c 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tts.das +++ b/modules/dasLLAMA/dasllama/dasllama_tts.das @@ -61,12 +61,24 @@ def load_tts_model(path : string) : TtsModel { panic("dasLLAMA tts: '{path}' has architecture '{m.model.family}', which is not a TTS family") } let dir = dir_name(path) - m.g2p <- load_g2p(path_join(dir, "tts_g2p.bin")) + m.g2p <- load_g2p(g2p_pack_path(dir)) m.tagger <- load_pos_tagger(path_join(dir, "tts_postag.bin")) log_dropped_voices(m) return <- m } +// the directory decides the phoneme pack: the full one when present, else the American-only twin +// the web serving set ships (British voices are then not offered) +def private g2p_pack_path(dir : string) : string { + let full = path_join(dir, "tts_g2p.bin") + return full if (stat(full).is_valid) + let us = path_join(dir, "tts_g2p_en_us.bin") + if (!stat(us).is_valid) { + panic("dasLLAMA tts: no phoneme pack beside the model - neither {full} nor {us}") + } + return us +} + [cold_path] def private log_dropped_voices(m : TtsModel) { //! Name, once per load, the voice packs ``caps`` leaves out: the front end phonemizes none @@ -99,12 +111,12 @@ def tts_serves_q8() : bool { } def private front_end_langs(m : TtsModel) : array { - //! The languages the front end phonemizes for this model, as the reference pipeline's - //! codes. Both families ship American English; Kokoro adds British, which its ``bf_*`` and - //! ``bm_*`` packs speak. A family speaks a language only once its lexicon rung lands here. + //! The languages the front end phonemizes for this model, as the reference pipeline's codes. Both + //! families ship American English; Kokoro adds British (its ``bf_*`` / ``bm_*`` packs) when the phoneme + //! pack carries the British tier. A family speaks a language only once its lexicon rung lands here. var inscope langs : array langs |> push("en-us") - if (m.kind == TtsKind.kokoro) { + if (m.kind == TtsKind.kokoro && !m.g2p.us_only) { langs |> push("en-gb") } return <- langs diff --git a/modules/dasLLAMA/harness/build_g2p_data.py b/modules/dasLLAMA/harness/build_g2p_data.py index 13d559982f..42c39bc359 100644 --- a/modules/dasLLAMA/harness/build_g2p_data.py +++ b/modules/dasLLAMA/harness/build_g2p_data.py @@ -158,6 +158,9 @@ def main(): ap.add_argument("--local-additions", help="json {word: misaki phonemes} merged over the gold tier") ap.add_argument("--focus-words", help="also write the tag-keyed gold words, one per line, lowercased - the " "tagger's silver prose is sampled around them (mint_postag_silver.py --focus-words)") + ap.add_argument("--dialect", choices=["both", "us"], default="both", + help="'us' packs the American tier alone (the web serving set): the GB lexicon values and " + "GB-only keys are dropped, CMUdict and the speller stay; the reader refuses British on it by name") a = ap.parse_args() sys.path.insert(0, os.path.join(a.root, "scripts")) os.environ.setdefault("NLTK_DATA", os.path.join(a.root, ".nltk")) @@ -169,6 +172,8 @@ def main(): tables = load_lexicons(data_dir, a.local_additions, a.focus_words) gold, silver = tables["us_gold"], tables["us_silver"] gb_gold, gb_silver = tables["gb_gold"], tables["gb_silver"] + if a.dialect == "us": + gb_gold, gb_silver = {}, {} cmu_path = os.path.join(os.environ["NLTK_DATA"], "corpora", "cmudict", "cmudict") cmu = read_cmudict(cmu_path) @@ -178,7 +183,9 @@ def main(): # and only then this table. That is what makes the drop safe - a glued group sends its WHOLE # surface to the fallback as soon as one piece is unresolvable, so such a word DOES reach it # ("water-tzarina"), and a CMUdict row would only have offered the coarser reading. - covered = (set(gold) | set(silver)) & (set(gb_gold) | set(gb_silver)) + # an American-only pack is asked for American alone, so every word its lexicon carries is covered + covered = ((set(gold) | set(silver)) if a.dialect == "us" + else (set(gold) | set(silver)) & (set(gb_gold) | set(gb_silver))) dropped = sum(1 for w in cmu if w in covered) cmu_misaki = {w: G.arpabet_to_ipa(G.flap(ph, calib["flap"]), "misaki", calib["opts"]) for w, ph in cmu.items() if w not in covered} @@ -202,16 +209,19 @@ def main(): ("gru", tensor_section(arrays, graphemes, phonemes))] buf = bytearray() buf += struct.pack("misaki table; g2p_en 2.1.0 checkpoint20").encode("utf8") buf += struct.pack(" 0) { var inscope prose : array dir(path_join(args.root, "work/gutenberg")) $(name) { diff --git a/modules/dasLLAMA/performance/model_specs.das b/modules/dasLLAMA/performance/model_specs.das index 36754c514e..5f714bc2cb 100644 --- a/modules/dasLLAMA/performance/model_specs.das +++ b/modules/dasLLAMA/performance/model_specs.das @@ -557,6 +557,9 @@ def model_specs() : array { // nolint:STYLE038 — flat model-set t ProvEntry(name = "tts_g2p.bin", root = "llm", recipe = TTS_PACKS_RECIPE, url = "{TTS_HF}/tts_g2p.bin", bytes = 14011554l, sha256 = "d7b6afea7a0901a877d10531054d3f967845c58f415f0ea937decd8a3081ccd5"), + ProvEntry(name = "tts_g2p_en_us.bin", root = "llm", recipe = TTS_PACKS_RECIPE, // the American-only twin: the web serving set's pack + url = "{TTS_HF}/tts_g2p_en_us.bin", bytes = 10257455l, + sha256 = "6f69d2e74565bd7d876b8d1f4042bf8c1c5b615387fa26ff45215cf447932154"), ProvEntry(name = "tts_postag.bin", root = "llm", recipe = TTS_PACKS_RECIPE, url = "{TTS_HF}/tts_postag.bin", bytes = 12566510l, sha256 = "38c2e85f7fef3e57d561d2aa0af25fccda4276376ba1993c3dbc2ae0ebfa57b4"), diff --git a/modules/dasLLAMA/tests/test_tts_g2p.das b/modules/dasLLAMA/tests/test_tts_g2p.das index 9150152c89..88a97e50c0 100644 --- a/modules/dasLLAMA/tests/test_tts_g2p.das +++ b/modules/dasLLAMA/tests/test_tts_g2p.das @@ -11,6 +11,7 @@ require dasllama/dasllama_unicode // utf8_to_cpts: the inventory sweep reads c require _tts_corpus require _tts_corpus_gb require _model_tier // models_dir(): tts_g2p.bin and tts_postag.bin live in the model store +require daslib/rtti // this_context().last_exception: the American-only pack's British refusal is read as text // TTS grapheme-to-phoneme against the 200-sentence corpus: phoneme-identical with the reference // front end (arm E of the fidelity experiment, fed the same normalized text), the heteronym @@ -246,6 +247,46 @@ def private write_pack_header(path : string; version : int) { fclose(f) } +// The American-only twin (build_g2p_data.py --dialect us, the web serving set's pack): the same +// American reading as the full pack on the whole corpus, and British refused by name. +[test] +def test_us_only_pack(t : T?) { + t |> run("the American-only pack reads the corpus as the full pack does and refuses British") @(t : T?) { + return if (!data_present(t)) + let us_path = path_join(models_dir(), "tts_g2p_en_us.bin") + if (!stat(us_path).is_valid) { + t |> skip("tts_g2p_en_us.bin absent in {models_dir()}") + return + } + let full <- load_g2p(g2p_path()) + let us <- load_g2p(us_path) + let tg <- load_pos_tagger(tagger_path()) + t |> success(!full.us_only, "the full pack carries both tiers") + t |> success(us.us_only, "the American pack says so in its source line: {us.source}") + let rows <- load_tts_corpus() + var differs = 0 + for (r in rows) { + let a = g2p_phonemize(full, tg, r.norm) + let b = g2p_phonemize(us, tg, r.norm) + if (a != b) { + differs++ + to_log(LOG_INFO, "tts g2p us-only differs {r.id}: full {a}\n us {b}\n") + } + } + t |> equal(differs, 0, "American readings identical on {length(rows)} sentences") + var refused = "" + try { + let gb = g2p_phonemize(us, tg, "the water is warm", true) + to_log(LOG_INFO, "tts g2p us-only british read: {gb}\n") + } recover { + refused = this_context().last_exception + } + t |> success(find(refused, "American tier only") >= 0, "British on the American pack is refused by name: {refused}") + let gb_full = g2p_phonemize(full, tg, "the water is warm", true) + t |> success(!empty(gb_full), "the control: the full pack reads British") + } +} + [test] def test_missing_pack(t : T?) { t |> run("a missing pack panics by name") @(t : T?) { From d3b787e54210840241a57d704b9fb8eccf7a9d4b Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 06:49:00 -0700 Subject: [PATCH 28/43] the TTS set's Hugging Face pin moves to the commit that carries tts_g2p_en_us.bin Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/performance/model_specs.das | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/dasLLAMA/performance/model_specs.das b/modules/dasLLAMA/performance/model_specs.das index 5f714bc2cb..16d828a648 100644 --- a/modules/dasLLAMA/performance/model_specs.das +++ b/modules/dasLLAMA/performance/model_specs.das @@ -280,7 +280,7 @@ let REF_Q8_RECIPE = "derive: setup_asr_rig.das --refs quantizes it from the fp b let PARAKEET_V2_NOTE = "historic nemo-venv conversion (rail retired); present only on boxes that converted it - v3 is the fetched carrier" let CANARY_ENC_RECIPE = "convert: modules/dasLLAMA/harness/convert_canary_to_ggml.py over nvidia/canary-qwen-2.5b (nemo venv; encoder is a repack - sha canonical everywhere)" let CANARY_DEC_RECIPE = "convert: modules/dasLLAMA/harness/convert_canary_to_ggml.py over nvidia/canary-qwen-2.5b (nemo venv), then llama-quantize Q8_0 (a Q8_0 disk embedding lets the tied classifier serve cls_q8 on the Metal rail); bytes are per-arch (fp16 LoRA merge) - gate via asr_bench --text over jfk/jfk3/gb1 vs benchmarks/asr/canary_transcripts.expected" -let TTS_HF = "{HF}/borisbat/dasllama-tts/resolve/c5247896344155d36ce1d01f74ebfd9fef9dd862" // the published set, commit-pinned +let TTS_HF = "{HF}/borisbat/dasllama-tts/resolve/a469827cfb344d6a4c2dbfe6e259245af7020734" // the published set, commit-pinned let TTS_KITTEN_RECIPE = "convert: modules/dasLLAMA/performance/build_tts_data.das -- --root --out runs harness/convert_kitten.py over KittenML/kitten-tts-nano-0.8 @ 7a1db645b1f3ab9420761d87428e042b9cec3f26 and kitten-tts-mini-0.8 @ c02725660cea441db4c383af69f1f26f5cd00947 (the ONNX weights into GGUF, f32); THIRD_PARTY_NOTICES.md (repo root) carries the terms" let TTS_KOKORO_RECIPE = "convert: modules/dasLLAMA/performance/build_tts_data.das -- --root --out runs harness/convert_kokoro.py over hexgrad/Kokoro-82M @ f3ff3571791e39611d31c381e3a41a3af07b4987 (kokoro-v1_0.pth + the voice packs into GGUF, f32); THIRD_PARTY_NOTICES.md (repo root) carries the terms" let TTS_PACKS_RECIPE = "mint: modules/dasLLAMA/performance/build_tts_data.das -- --root --out runs harness/build_g2p_data.py (misaki 0.9.4 gold/silver + CMUdict 0.7a + the g2p_en 2.1.0 GRU + harness/g2p_local_additions.json into tts_g2p.bin) and harness/train_postag.py (UD English-EWT + spaCy-labelled silver prose into tts_postag.bin)" From 6051be3b2d9cfe0dfa030756a733969502afa961 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 10:31:57 -0700 Subject: [PATCH 29/43] the layout stamp follows the rebase onto the Q8 token table: version 33 stays, the closure hash moves with the TTS image meta Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/REVIEW.das | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/dasLLAMA/REVIEW.das b/modules/dasLLAMA/REVIEW.das index 64ecada2b7..915b5fdd6a 100644 --- a/modules/dasLLAMA/REVIEW.das +++ b/modules/dasLLAMA/REVIEW.das @@ -545,7 +545,7 @@ let private IMAGE_FILE = "modules/dasLLAMA/dasllama/dasllama_image.das" // in file order. A closure change with IMAGE_VERSION unmoved is red; the finding prints the // value to re-stamp with. let private IMAGE_LAYOUT_STAMP_VERSION = 33 -let private IMAGE_LAYOUT_STAMP_HASH = 0x16cc08ecdcac17c4ul +let private IMAGE_LAYOUT_STAMP_HASH = 0xdd91b563594dfb9ful // The helpers that decide WHERE bytes land: the page pad, the plane and total sizing, the // writer's append / zero-fill / header patch, and the header's scalar stores. Changing one From 3b84f840355559d6ae76886e2b778f12f95c3403 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 11:23:43 -0700 Subject: [PATCH 30/43] the runner census after the rebase: test_chat sits in the stocked list alone, and the six kernel units get their area rows (five llm, the ASR tower one audio) Co-Authored-By: Claude Fable 5.1 --- modules/dasLLAMA/tests/run.das | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index da10bf5b1e..f87c7a7628 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -102,7 +102,6 @@ def suite_files(name : string) : array { // nolint:STYLE038 - a flat s "modules/dasLLAMA/tests/test_bench_records_schema.das", "modules/dasLLAMA/tests/test_box_ident.das", "modules/dasLLAMA/tests/test_box_profile.das", - "modules/dasLLAMA/tests/test_chat.das", "modules/dasLLAMA/tests/test_mtp_snapshot.das", "modules/dasLLAMA/tests/test_mtp_sampled_walk.das", "modules/dasLLAMA/tests/test_dasllama_lint_contracts.das", @@ -232,7 +231,7 @@ let TESTS_DIR = "modules/dasLLAMA/tests" def area_tests(area : string) : array { if (area == "audio") { return <- [ "test_asr_verbs.das", "test_audio.das", "test_audio_embedder.das", "test_dasllama_lint_contracts.das", - "test_tower_helpers.das", "test_vad.das", "test_whisper.das" ] + "test_tower_asr_kernels.das", "test_tower_helpers.das", "test_vad.das", "test_whisper.das" ] } elif (area == "vision") { return <- [ "test_attn_span.das", "test_gemma3v.das", "test_gemma4uv.das", "test_gemma4v.das", "test_qwen25v.das", "test_qwen3v.das", "test_tower_helpers.das", "test_vision.das", "test_vision_chat.das", @@ -253,12 +252,15 @@ def area_tests(area : string) : array { "test_flash.das", "test_forward.das", "test_fused_decode.das", "test_gguf_quant.das", "test_gguf_shards.das", "test_gpu_model_swap.das", "test_gpu_slot_swap.das", "test_gpu_tier.das", "test_groupn.das", "test_jobque_tripwire.das", "test_kernel_backend.das", "test_kgroup_repack.das", "test_kqformat.das", "test_kquant.das", - "test_kv_codec.das", "test_kv_paged.das", "test_kv_prefix.das", "test_matmul.das", "test_matmul_batch.das", + "test_kv_codec.das", "test_kv_paged.das", "test_kv_prefix.das", "test_math_activations.das", + "test_math_elementwise.das", "test_matmul.das", "test_matmul_batch.das", "test_metal_float_a_gate.das", "test_metal_gemm.das", "test_metal_role_infer.das", "test_mtp.das", "test_mtp_gemma_drafter.das", "test_mtp_sampled_walk.das", "test_mtp_snapshot.das", "test_mxfp4.das", "test_par_indexed.das", "test_parity.das", "test_parity_cache.das", "test_parity_pregate.das", - "test_plane.das", "test_ple_check.das", "test_ple_modes.das", "test_prefill.das", "test_quant.das", - "test_repack.das", "test_repack_lane_context.das", "test_rmsnorm.das", "test_rope.das", "test_sampling.das", + "test_plane.das", "test_ple_check.das", "test_ple_modes.das", "test_prefill.das", "test_prefill_cpu_kernels.das", + "test_q8q8_family.das", "test_quant.das", + "test_repack.das", "test_repack_lane_context.das", "test_rmsnorm.das", "test_rope.das", "test_rope_apply.das", + "test_sampling.das", "test_scheduler.das", "test_silu.das", "test_softmax.das", "test_think_split.das", "test_tokenizer.das", "test_tool_formats.das", "test_unicode.das", "test_vulkan_dec_tail.das", "test_vulkan_kernels.das", "test_vulkan_moe_cm2.das", "test_vulkan_tier.das" ] From ea564c528fb1977672c1134b3c73f915a095c98f Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 12:31:28 -0700 Subject: [PATCH 31/43] dasllama.io gets an examples page: the storyteller card opens the wasm64 build on its own page under the site's nav; the model set is a rolling GitHub release the deploy downloads beside the page, never a file in git pages.yml builds the storyteller with the games (step 7) and stages it into the dasllama.io tree with `gh release download dasllama-web` - a missing build or set leaves a placeholder page, never a 404; the Caddy snippet sends the two cross-origin-isolation headers under /examples/ (the -pthread wasm64 needs SharedArrayBuffer), and serve.py mirrors them plus maps /examples// onto web/output64/examples for a local look that matches the deploy. The examples tab and footer link join every page and the story template; the page census (sitemap list, test_metadata, the deploy copy line) carries the new page. Co-Authored-By: Claude Fable 5.1 --- .github/workflows/pages.yml | 38 ++++++++- examples/dasLLAMA/storyteller/web_shell.html | 37 +++++++-- site-dasllama/_stories/template.html | 4 +- site-dasllama/build_news.py | 2 +- site-dasllama/examples.html | 75 ++++++++++++++++++ .../files/examples/storyteller-poster.png | Bin 0 -> 218177 bytes site-dasllama/index.html | 17 +++- site-dasllama/ladder.html | 4 +- site-dasllama/serve.py | 17 ++++ site-dasllama/sidecars.html | 4 +- site-dasllama/stories.html | 4 +- .../stories/2026-08-30-apple-m5.html | 4 +- .../stories/2026-09-03-mtp-on-mac.html | 4 +- .../stories/2026-09-03-text-to-speech.html | 4 +- site-dasllama/test_metadata.py | 1 + utils/internal/dasllama-ladder/caddy.snippet | 9 +++ 16 files changed, 197 insertions(+), 27 deletions(-) create mode 100644 site-dasllama/examples.html create mode 100644 site-dasllama/files/examples/storyteller-poster.png diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 7843b44a5f..0b0188da04 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -115,7 +115,7 @@ jobs: # The workflow file itself is a build input (it carries the configure flags # for web/build64 below), so hash it too — a change to the build steps must # miss a stale cache rather than skip the rebuild and reuse old archives. - key: wasm-${{ runner.os }}-${{ hashFiles('.github/workflows/pages.yml', 'CMakeLists.txt', 'web/CMakeLists.txt', 'web/stage_playground_imgui_samples.cmake', 'src/**', 'include/**', 'modules/**', 'daslib/**', 'utils/daslang/**', 'utils/daspkg/**', 'examples/games/**', 'examples/graphics/**', 'examples/pathTracer/**', 'web/examples/ui/samples/**', 'tutorials/**', 'dastest/**') }} + key: wasm-${{ runner.os }}-${{ hashFiles('.github/workflows/pages.yml', 'CMakeLists.txt', 'web/CMakeLists.txt', 'web/stage_playground_imgui_samples.cmake', 'src/**', 'include/**', 'modules/**', 'daslib/**', 'utils/daslang/**', 'utils/daspkg/**', 'examples/games/**', 'examples/graphics/**', 'examples/pathTracer/**', 'examples/dasLLAMA/storyteller/**', 'web/examples/ui/samples/**', 'tutorials/**', 'dastest/**') }} # Host daslang — one games-capable build (dasLLVM for cross-compile + dasGlfw # + dasOpenGL shared modules for the games) that serves BOTH das2rst and the @@ -316,6 +316,18 @@ jobs: echo "WARNING: physarum_lab wasm build failed — its examples card will be skipped this deploy." fi + # 7. dasllama.io/examples storyteller — dasLLAMA itself compiled to wasm64 (the story + # model + KittenTTS, both served as prepared .dlim images the dasllama.io stage step + # downloads from the rolling `dasllama-web` release). Its .das_package turns the GPU + # modules off, so the host needs no Metal/Vulkan. Non-fatal like the games; the + # dasllama.io stage step stages the card's page only when all three outputs exist. + if ./bin/daslang utils/daspkg/main.das -- \ + release wasm --root examples/dasLLAMA/storyteller --out "$REPO/web/output64/examples"; then + echo "storyteller wasm build OK" + else + echo "WARNING: storyteller wasm build failed — the dasllama.io storyteller page will be a placeholder this deploy." + fi + - name: "Compile-gate the curated playground samples" # The per-sample wasm64 builds used to live here and doubled as the proof # that every curated sample still compiles. They now build on demand @@ -555,6 +567,8 @@ jobs: fi - name: "Stage dasllama.io for deployment" + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} # gh release download of the browser examples' model sets run: | set -eux mkdir -p _site_dasllama/files @@ -566,7 +580,29 @@ jobs: python3 site-dasllama/test_metadata.py cp site-dasllama/index.html _site_dasllama/ + cp site-dasllama/examples.html _site_dasllama/ cp site-dasllama/stories.html _site_dasllama/ + + # The browser examples: the wasm64 build from step 7 above plus its model set from the + # rolling `dasllama-web` release (prepared .dlim images + the phoneme packs, ~70 MB - + # build artifacts like the tutorial MP4s, never in git; re-minted and re-uploaded with + # --clobber on every IMAGE_VERSION bump). Served same-origin, so the page needs no CORS; + # the Caddy vhost adds the isolation headers under /examples/ (caddy.snippet). A missing + # build or set stages a placeholder page, so the card never 404s. + mkdir -p _site_dasllama/examples/storyteller + if [ -f web/output64/examples/storyteller/storyteller.html ] \ + && [ -f web/output64/examples/storyteller/storyteller.js ] \ + && [ -f web/output64/examples/storyteller/storyteller.wasm ] \ + && gh release download dasllama-web --repo GaijinEntertainment/daScript \ + --dir _site_dasllama/examples/storyteller/models --clobber; then + cp web/output64/examples/storyteller/storyteller.html _site_dasllama/examples/storyteller/ + cp web/output64/examples/storyteller/storyteller.js _site_dasllama/examples/storyteller/ + cp web/output64/examples/storyteller/storyteller.wasm _site_dasllama/examples/storyteller/ + else + echo "WARNING: storyteller build or model set incomplete — staging a placeholder so the card doesn't 404." + rm -rf _site_dasllama/examples/storyteller/models + printf '%s' 'Storyteller — building

This example is being rebuilt and will be available shortly.

' > _site_dasllama/examples/storyteller/storyteller.html + fi mkdir -p _site_dasllama/stories cp -R site-dasllama/stories/. _site_dasllama/stories/ cp site-dasllama/ladder.html _site_dasllama/ diff --git a/examples/dasLLAMA/storyteller/web_shell.html b/examples/dasLLAMA/storyteller/web_shell.html index a853eba811..9952430861 100644 --- a/examples/dasLLAMA/storyteller/web_shell.html +++ b/examples/dasLLAMA/storyteller/web_shell.html @@ -3,13 +3,24 @@ -storyteller +storyteller — dasllama.io + + + + + + + + - + + +

storyteller

@@ -37,6 +59,7 @@

storyteller

+
+ + + +
+ + + +
+
+

The engine, running.

+

Every example here is dasLLAMA itself - compiled to WebAssembly and running in your browser, or built for your machine.

+
+
+ +
+
+
+ + +
+ Storyteller screenshot + +
+
+
+
+

Storyteller

+ llm + tts · in the browser +
+

A small story model writes a children's tale one token at a time while KittenTTS reads it aloud. Both models arrive as prepared .dlim images; the whole engine - decode, phonemizer, vocoder - runs in your browser on Web Workers.

+
llmttswasm64threads
+
space · next storyChrome, Edge or Firefox 133+ (memory64)
+
+ + +
+

Source for every example lives under examples/dasLLAMA in the repository; the browser builds are made with daspkg release wasm from the same sources.

+
+
+ + + +
diff --git a/site-dasllama/files/examples/storyteller-poster.png b/site-dasllama/files/examples/storyteller-poster.png new file mode 100644 index 0000000000000000000000000000000000000000..c944c6ce609cbd8d63d8d7e9f55b4faa95130720 GIT binary patch literal 218177 zcmeFZ1yfvI(=Ln#2p-&ny9c)fcS3M?cXto2!3PHS;O-J!26uONcQ}*x)LZr4_jCTi z_f+kfEo;}*>eZ`zU43=03I8rHf&3BwBLoBlvXrFg4+scY@FmndJQVn(gYVf4K0!MD zkPwEb7$ZCc--w#1OPR{aLC}G(;UOSHEg)e2z5@Q>gFg@u(Akg>(BL!V-)GrS|Jw=+ zkq!O7*HH6+Uz~&{EQWyi3Lz!>P1zmtBm>S{XTYT>AasrX8zvNt7pzxhuIP_^$?&6^ z7ArzNc_6oI{Huvu?O6Ze=&`L#vhmQ7PA*;g0w2QNhd>B;wD$X!^%X>ZehAd)&op1( zI=N^1+Lgw<+1*UXfb32JJt9T|g8O=rp%T~o41qN`kmzXtJaDz4P*73F|BU7|&a@Mu z{Cx2CVWe zI)Ws=e!SSIRvG@+bMUQ5m_OzQsWRRtF&NRW|2*_B;3Xl6`5UA){`nmInm#;w92Eag zI+5Ie^%H!HL<~vW6*Qnq8~INW{x*?9{qPU{ZX${K?~yX)!0?|409>_>$~a9Vh6y62xTA)9epSdgGx^J?(%YMl-}Cf zx(-VxG{s0+*xK=`ct|P!`1rU|haT)QB zg5!m4V2;v%;Z2^2A6T*TTT%>>4?j@&x8r?igCB%_dK${RdVGEq6AuA@=f2O*%p@hL z&T!1|@e}2s{|V1C9Eg=GE-$&gbI9()#!6CEnb-eLscJ6yT~rhoPc2amDmlwMmJGWa}zO+fS?}XZNof? z*obnPO8Ac-$spJg)s&a;VBuhYN`!duN&fzjVmjk&G--4Bxn!?qFQos2cAd^hY>keN zPPDQdqa367t?hfx<`2+8G5pPG%hFSKZv0co2YQ^?stJ8N=JVm-X<=b#etw<({e@dw zHAnbL3wIAKpZ?{a1IE+A%fxXRgMJb|yD;F(iHdc_7q~`9)D;^UDdrM9!^_PMF%NwYgv0qeH)wAwbuc5 zw$cI!E!o@Hf39*}-$_5iKJZ#W!Qz%Pn=JJ@?N@5cj%;icXG6Rn;pg3QaB`UHjJmjc z7!V5jv653$qq@}8mBjO49WM?_lJ}Csw-d0LWNGOu=@^*yEF>lMWtDf5 zK)~&i{qG8;zb%iBjuArsV2eHVmP9UCw-c>l3`C>AU-V^0H2?uF$S#-(S0%L;>LyY! zBP?wC;0rkQ=jU!sSvA87;53y^CdWUyKg$C61wEW!!j)30 z&iebr6tZ}UwQj=`a3n{Cx|n%b=avWP8DHtJ?Lr&XJUo)>)-!pIOa3MMC^cC2Cx8)C zA8}+bwut#R1J+ifQ^wVkCb3y01O#()O9oonk)aX0=??E>n-Mh(ad8QW;Qd`Jmn)mI z`j>nh%o8C8;U5+jCZ?t_i=8VIUtFeeqsEKIbuG)-ZD&#J1_10Sant!BC^ihUIx2^{ zwLc*KZTGmMV7yX^!SqqrR^H%f=z2R|qP*Qi(e~TrqSIktAEot&2A#-sMHog2L)(Rx z?4C_LOw6vmg}@iXO#9BlhQjI`3sKp8Mwn-78EHBzt0Ou7WgI|nn$|=`DV^m&tW29D zC7*>dA)TA2X~W_~q!ar$t!q|@NSLk$v>}k&$VkuFkgH-?K?eJ$V&{Ib~%~qa?^H@i|#MG)@0h9~gNG zf>GW45H4&}O9j>KP9Htqyf}4VG)X!x2|@jR7lL=Z&2~}w4R5_3AHm<2ZQJc_-^- zE|wR*&-{;gEluvBMcm|5p>meu!~zqQeJ0&ZK^I}`6L?2jrV<+FN|vSi3~8$$6@CDT z+glg2u_C93^N54LD@G-*tlW-4;gS$f|MDH439tP@ z^&u2y(_LFf$!bOuBQjKuV)yj4N=RH(e5Hb>)Y8)W^6GAQjFBbl@wQ848K)2DG>wFe zg9FdZ!dE+ozzC1(o9x6Kxlw;Zr5rXfag(~yxrv&SucDNV4dc@nY!l|t#`4m1wAa~U zIl|X}_Oj zdyC7Gb0haOR8_+%IZcjWW-KUZ85n5kz;35(c!)7o&;)91JSl0`yc;;_^7i9{SG-72 zka!H?Cdt7_V`AVDL6I!6nyxM&OHW%zi=YQExk;KxNi)IB#AJDth`?=q9fm2N+403} zE^$|8?*iGN=I-{EG^JFdS#reHMA=7+#Erbo_TOP#*Tm-ytPK)27z(`P52BdzDB>{@ zANT!$Eui_Ru?!h5-9K}vI(0b}kT&dWZ2T-VpF6ASTJVcD#RXpoFb{;-@Ne1ke;3}( z9Lm6o_2Yw>yT((33a>tX3}3d~sE^-jsjmJrK5p0jkS}9_hcnH_s9VQBc5-%xc_(L%$rSh4GwM6p6SGE%sVJ!^|6Or=fBP+fk&(Hw zwA75tQCK@_aa~JyXux+->|`?l+SS8HL?BdVex6HI97aM+LR>t=k;z#R-O`ovcSlyG zosEug zqoUsOcNL}R<*HgiMT_dh4{#qGO|kTlepghmb_~yrFE6K0SAF-iviu%Sp!pgq!1H@J z{QyX`>RBWwI-=MNU`A!s;eW2x31!MRac05X{&jK){}Jit`nsahs*7)F3?9;63FQX$ z;`Z9Y)_SY=Fhraaw-$17b9>upr!Gn5$Gy2lv0d*g9%Ng4yKKYWFU8@>I`quXm`LQ6 zS(SSF{v--4IP-I}fVlqWrS$kYuFrQ>WfDps|2@Y)r29L#xzNc5WdNRU3Lskdrv_T* zsMM8$$0Fs?PoIn9ZSV_j+fH$Rqi}f`a(&2iZ+M?@d42JQ3JSz#mc2B;Tc|$%E>@6U zdD38F+yOKOYlgON)eXICXN@#gwYL0Myo9K=-_qNl^f& zQn9VmBWpC7`S$G6e4abIM51n)QaDXcAAfBj@oUNPkT#0_H`Bk(T3_;Sc%1(`ua*T@ ze=*YQ?FV+|P+y;kw^bvbT&t7pE8{<7Jupo%@ylO3o9_xY&<`-$xMyzOu#IoF9YeK9 zN>eHNZ!`wU1+xQn`XR#Bg-pMLX2rEBv2I=7{-)^PuKot#-#$5R6p@gD?4SeHE* z#JD%jB<^7T3+#azl)sTbba&CEFXEs{i#b0|B2!_HWk>UUYE& z(j78#Qu6;^V$lL1|BlsX?}+~gZ=wB{w_xI+q1 z|JB$(?Bc&u@gGLb@?RVK|N2I-SKJWLhDidF*%QHNYHEh<9v*rSIS5tK^eFC)N2>T; z*QW(pe7dV_4Plj3nHX4E&OcuFgOl2OLtsr5hR$z&YPV(WT!@1UT5!p86|Vy<`>kDk zblIC4h}@io2*GR-{G`hA*9SK7{B@RL?CeX)$)@oaHy1nc@4SRPQuFxVH$z2?nL}~F z5xID&Ye+!vmwbN+KbZ3JasnncROm7rvZu$VhsVdO7SAt@GhAP0XXon$4lndd7rE2$ z)6)e&%d6`otAWduIA&*zqm0PN(cn1!w8h=;`T#c*3XCjx_=)OWZbwyPXD%N}R`k|% z*E%}ZzS^2wpVVKBj?M(7|Ngpql<)B9{v4-iq;zg^mv%Uc;Qw*wH*B-%Qi<~ozo6in zoLpB^Cg*wD6>{61BlpYeE0HeB2hEntB~4vCE)ExHrxQ}zH8vq3q0k;hF2Xf7&mjM= zLLa^&>P%O~HN!o4Qv7vMT*-e)h5e!4z-kt5ZVEDfq=JIqzgz0+%4&WWwjQ3}?&qhj zqUlfdV>Bv}Sq`j^pN}Co#z;qb-PWVlI63l-nzTw??X8~C@w0&ulh1K_?8DRJ%gO$D zN|M@Cmu==(y&GU!dfFaiX6h4uCeQ`J3=JeHT7x9mSpeJjn=XD!za_gv=-wfFY(_|7%eUg5>X_I4!9wpT^7 zUX2Z5N5f0;Q;wQpuQU@v8}$+`O!rOqVky zh~t9qrWO`FHJ2zYr+07)!au)Zbv~bQ3=QS!KD;gi06adn5T+a_dol!AL~j=&ZgOd< zNG*{#(^+0`8CK06KFJ;zU;l3s0{0Vmvg-M9Jh*#|XA0a-hMq^n#{SXWKR$N5oJumw z^4fOvoRcFyt>{GR;iQ^IH_gSMqEb^)1rsbMygEiT`@5%M3SmSUnEU7Fd@~;-f-R>K ztFd;m)GCZy<)z!&_{~76gUZVzTIYMw6<r?lzEP{eDIkP$NZJysqO4|jL5h_>RrOvfa$PVj^T z4}XPnCa0)O36AzIw4)K_-aUo(OgqUsPOWGtt-EiN-vEWQZ|A9YrI5wPv$(D|a+Cz} zjN~YIcs2?2yl4cOLRxx6W3lQAD>7$4TAJQ>kTbVm`8u$+RA`uA-`(PK&@s`lbPdce z1cdh4y}fAY`E2PbEQJ=C>1PYk`C>`iMM~({yfXW~P6Y%FL0#Y}u~-*Q zsz24@yhbw!NF_@gTWu>UyK{F6$6TNT;C28p((gS#2;NL*3E45DvOg>?D_pwon5mWn zq+IZzb!-#ZhY!rm}d=hf%LF=9daHsLS;b>j(dyC(y z=F~GY_i|`635L9|>Np&h^=2s6=WBiuu`3S>T@DI7qZ0G^2_g%bDM6|AsL3GCJ*ere z5`Mx@%t$!zwH;apWff|pRmqkyh9tp`1pEIn!5PKOd_OOBZ^v#csdg}yy$QOnSohml z=tERXbt330@4LS-zqDqZ=KI+lbI`H4@Y3ofElOTdRFnaQk3C;;IxKo#E!V;rm}HR- zJM`2u+^b*x#)g7kKCil^oerTh0q82!0m%AC82@DRyE&D}(Ogeq+CAcdtZ7cxi_`j^ zs;6FOv26o$mR3r-H+i*5&4)@NkOXikPtdPxsVa_ZoAERO3j&@!>!FAsY0@LwZ-bUR@fYe}nhc$L<5c6-#KtlhZDih`6L$$@V=Xoxiur*H6j%W@8=FPCw%+nY)bE!pSpLJ<%}RDkHHEBKiYw+6g~LzjztV zNt+rH{3f1S@uOaVgPG&gHpg-f3)8M;r!%*-4E-IEBr+;;A*x23@z@T+P?qhY6D2u8 zV9GSx(J{eSL#*G;Wo<``X6qEI@$rskXGFp0GgtFI~po-ngNYn&P=9U{sEpB7vaE_ zm$o*j(q{{-{xAdI&j%E31}nOEwogCIAm&|n?+TM5)BWgv|3)R0mm44KIGv+h@Lp}J zcHQ4TOL9n8oKN4235LK;99B_=sp(0F*jw7MV=8-ZMsM6^JeZWNACn?Fh7rbSl zr7%mE#35i`U;x$7WKpV*5rMSWO=uo;E3XOop5Egl>nA?YzOX$Mxb^ zkctWis!-ErSI4$m<=)fN166NdVv?FZVT?jpBs?mZEf@ne4Tqf(Doch!vT>S<%C3ZE zL-iCrh~4Mx={yNiGo; zW!f}8LJt~YG_M&DLLDCrtJ;XmUa^AB$st8RAV;svt~#((W#Stw<55A)trxtya_$|7 zjl#q0tN49%bObzU>yH1yB~sU#ST+4qR15^*@EJQPfeoglrBzb&Sv(Exv9$3K2!HT= z?u{kJBP1Lw*40rlGG&^JeSdg<>hBd*QC1EO`2ZUVW2QX>^_Pwbd)2@V|Vf^#0a!5fNvvfIh?9P4^`2oX#X-gVzOCC0A3wa%f7q2|Fv6*pL zIH?ck0n|#`JRke{9hCM#zGvyzd)`6)IwTTz2=Qs2H^7WXa` z9}I$`PPXEIG+10+)%Mo*xWKRvgCJz*(EQzP9U|+)4MVIPRP1HGAu7h~1Xj z*;ddGuA7P+tq|4=Mb0NgjbHQ&QjilN-v+Opl(10R;sq zDh!FavN$RtO5e(-KdH3rh=Q8BS|i1(C2;V9<<2=VIdO8$dgvEL*SHWbZ=m{1<^5LO zMK@=yTGo4$O<>UPF8sFa?|npk{Q--E-QYzp>o^2o?=yB$Gjgo$xz1FdmB9Fi&WO|FU@)AIt4tceV=;2qE<4eAl>}F@ zyI0HQ{s(=%$pmOzpZV=QU%pQLZ5eG$R*svrU^oPk2o4-eP-(|&Iw`{m(z zMOl^WTVl1gdac$wgXn^5Yhq#re6BmpM2j+afU}qv6<4E?cYvjFwE@d3E8P8$Cvk4g6p`}UmjM1o%>C_^u-%g z=y77gIF+i~UIG;w&sN)?S;)!QS9zQYnKilTZ+9L6>{QXj8E6ORddliGk9ntBjYJzO zfH}y8a(mN;BlL~)M$6e;jz&h4y`N`GQwu;>%IjM$AwicX;c*AKct+yu{UQb}UOVyO z{M6L20vS_0)Fa{`{YEV2^BAMI$E~iYC1Ub`TAM9GL z=V%iKB*er%ca==<>v=!=sM65jD@^LomFcvYzcIAgl$3zVPVU#1Jzsix3i&Wu;-LiF z4z~)N^H=K}T&!Q<<7^je?hnoM8nuLtMIWAS>2a7-u|5%s>g1al`@9TQ^hn6xvdmH` zkqBiK$m~;5_P^dO&Zi2FW$+ojo{o%XDEK|j-di3YsWN26F5N91ll>3&g!kg@gTnxH z891d+JzrAAEGr@Lu5&uiB*zVIja)bsH2v0CiF)W{3YscFOlHi)z}*?Y8}xg;*E2EQ zardmRK%5vWX#(q>DweoU8|`&{d6@wDJxOg{$m5Bh(Y|l^kv^FzmCH_3+4$h~zQPC@ zs{BO#=x~1hr7GWMgyu)QAN8+?g!E#vgrcbBD&O5EgrT<%Uk;agi5TF^M$q^HQ%i`O zSEv|}qco*|Ai6gvW&UFD*Yvr~S|CySTKHJ#2ZeTKpb$9)1x&jD`SlSTiI3~ffyI%) z7lxBb2~N{}%(lzDP=*BnS3yB=^7StDc{?yU_(cwRwW1Y$YJUAbStarl#`g471t^`J zh$wxyJwEC6PAiL;-!YJL?{qhn!|JG?PcSLqrYP-p?ZZb4gG)b%JXiazE zR(Qfdwx#dWq3yfB|7;^uMOoR?sgqrPVWtUfAj0Dn7NV_2v%aV@X1==Z^TD=`*FAB) zTQ!v59EkRQt)0`t*;~$KbvLT)NmFb8uwCV|ALCQft?&ea`as84%5XD}wd#<lp!noaNO+`P$5L5$52}?q-$qB2Zv@ikCK?jNsk8BI zmOHZQ8)i#Wveks{UzVTmNeI|Ef2H`0%n5ki5!K@ux|5rZX4d$g4&QUx_0%m(n4qTM zHew;n4q*(exa^lV>E$fu_GR#UTn3@K259Jt*=$gIW^wd$W!}%>IkNVI$ApzsnyKOPxcFarbvfedX=7;Iv z$pbm8pV0g6$X-RO5A1fU#n33@A)ea0=W~KliJ5z+c;CR9(|B4CoD$@26tBl{su$O4 zhtqsc`SSf;^%l^>sKqTpx- z@#6}j9iWb7-_y$pB;~C9P?_Z$43HCE8ALu-iy}#BGFe{h2~~(8N@zN)SSBOyc|R4( z>b64JRJ7=L&Y+GV5_)~oB4Jx$(qRh~0<_t_>GK5>;bDl~0N9HNx$R$W=Y@-lBCPbk`f zoQ4KX)ve47rSX2{@r&yj>+;*Qkk3`t`>7gcuX20mdK6v)HZ#zJ=atUaZQ1oKG6IR2 zfuZk8s@8D(CNFj+*tXMUxubS}nx(_xYILDG-0vM#NTwC|ZL|f_cD4I=5G$cHiC+!D zsprs^Mq`C$6y;rXqhL3N`(^H;vE1}6>Rk_&>&AwIECwjZPNVstTH?H-qQdWR_WrvO zxLknKw(WK-y0PQ-t_Zh~WoDpn06fgI5#hdhy#@0+2Vh6XYU zoe@+k13Vh29`@gl!V}CQA|mwJtmcdpS@EN$UsDcm4{LF_Wy0`z{8)Xi_lQzq!UF~1 zF5^y@>H~v9V+N?BzQ$*?qU@!*Fu|b^HAq|F)KDVv*3ak}xo#ZqkgxqCd$SQlFjuKn z?KS>uX66(VIV1Gs?Rj84CpRS}g*rMyufce&Vik^^&5;IKtjVNL7(3*R{FeVIHY0f1 zuf&ZG9P*D|oivYl-4U-=UwAffmE zX?4KB?$GZ^_N4eI6e?SN+yorp=g% zc!APtU+1O^vjqKU1cVyMBBr~Ru6yFae3nXS#TaT|9 z@wpNY3DvhMxeswY~_lO%W%`Z(88F^ zfHRcK7hh#itlyRP-Rw%{v#_)BOSfvJF2C(cv5r?v4GTI6fz!KB<22>l>iTsa5uJtOkxCgt!0U_fgtTJ`8{5#DKl43CQa? zLqTZS#sBidD4Np6jRgjtv?BEdS;zj_thu1ydtO#`WYXr!=0&#zIu;@zH;-(7wWF<~ zViRmp4p%p@U)G%VnFa=Y_A8JZi4_FWlafTp1R~`FU$lFiTY?U6531QgAnA)<#0#?o z+=QS@BZ&lV&P`^4rhtS5@Y<82u2$%8Hm3Wu?)OXt1>|-hhUSQX%-wQ)IWE%e$L)sH zXL)>5JC|^w$hRugsgyz%`O})-xD^CPQV6vG$6e!i7geHOf$Ot5!I6BNF>S_Vh$)uS zt?haQ5{p8>Edcys)2KVnEo^w3*u(&pgX?{GEw;?(n?O3>w`-lQ)_mP$q@-Xyp^jfG zJKh2s8@DrC=h%csMo0WEI&Ke0->=RUSWk`a{4O>SVnQ{XmJQjqWPin!+@3RkOa~+d zL51h-=@7`NblksjQr(2fxlH_gQ2eMlLSJd~hU?_)L#_!az6}mx-jBEfaNm^*6&9-A zutD$&0=|wP7U2lO| z-l)vZtU&~P zMqww(`;7;K=?yA_%?3~S+r`k1!xz)}3){+}uJQHzS5&vQxBgIMWI5~i$e0-80;2&l zI7NqnScML=X`56Mzsr&J<+%}1Y4iJq-y6`R(g}x2O-040#pc|t^O~F2-u2GZao}@E zKyg$4%iyHa$>2e~>j9cM(%>V=snXy{yuORD<*9+P4!Z??Y3-L0wE~b|B62rPj0W2< z)7^QxsXEW1js~MGULrP2+=ucPvyj91<&0C7^H=%OArQ@uAv%TGY>MdeX`)*Z z_w7((U9ZialS&nT{L19o9J=%2cBIqqP~c6L-_dfamvs|8$M)OpJ(bJA9T zhl`@9sAwoANdB3Mcb{wREmf%on>j7Ke@sTeW!N2`S_EPk+LM96_yTnRDi&~GptdFh z5Cq`hH9BeQYOsC1)~M6(zno!D%?_`(-z)=KJr2ZVRb%tQ6+*bh)nAifMjv%ad;kPN zK_BArhF<7b*IXtrwk}H6xK9$!4BiWgzFPqJyy%`8K1NC(YN<=~IKsPr5m=fUsq|s- z)t&+_PCKyV)ZCr%h0OKF_fF;!O74Y254JT=e~+J@X7|)MI3bfRBDm2(+#d%DSkckL zGIu@+GBX!_sdd;aRaleglyfmQ-XoL|7=hHRL-r)d&0AB*5)2@XO2S#gQDw5fcR!7e zt)8uI>)XQjncR+D;V^TRR&~`?7Jc}wg$*tne?5C_)Nml@M;vzAm)-X{b16ZoDqA{P z@;R6)kS#kX%k`~sQTSD_7_2trna&R`8Lsy}Tq*K>e!6Dv7_|;)1IT*cj@*x?rw5`6 zy-JPqvOqJf%0DHY?EBt-SVx5a*j>MJzScU=puwLaX8GlmxP`E{M`UNZyjAGU8Rwb! zz|Y*pogin^NmtF*RxsjnOeb`{LLHk&~A@M;4aW|px zVZVM_pka`;jmlH@;{Whz4}->N4&F?@b+E+~`CgLG-a`z_>GRz(%NSdrp#UNx@>=^o zO%~0DLWmO;%$+bt9!{&m02MVG)7DD8?`Z_z+$x*Nu=~`$0+Oe8-M*)f+MIJ1Ee<{+ zRCUJHP6^^kpUK#$cLv`6&XCdK!5~ry4VoNz9)YZ-Ck(?Hd^oDYoU$Binqrl2wk2d6i3kb7aWV1N`T%5c~ zh%nPLnvIgNvSQCmctdFjlvLVeOKreLXmL1Ey*#Dhz@j*}_ zPKkdS4(5F05_%w4#%CCxA!E#8PLg*H++>ZcGsls^9@l6wW{SY3txC%F`tyAM(^MpQ z;oPn$wJo!7(m}p)(qdVQ=;Acawf4(WRf*yB%SnW-*BWj6oe(X~M;6~yiV;Y15_ETc@mepBz&$PuoCYxKB2`Q zGMIg4ukS+|e1Z)j|BX}xah1kt1I1{0f0pNc9=ft@;^Nh3WxnhC((By=8j2`T+c>S(9#lqLtk=oV!F?&wSbLvzEo{g z;9B=bOc|CLzUaF<4tkAGxwQR#(b+_SF}6Q_PTXrIq)B6y&VCsc4NdM=kv2Ns-xk&i zx8_2~iTFSxTAKUcQfcCZjX!wh0>@mxe*ZhXP$iT++o=DEVF&dzQcJs{TsR8|+NNQu%`W zB&8){bdFG9fR$8GXT|gQpk4bQ)_0j!tKay(;qZl?u!t5B+2{<2`67x!#M77iRV~#P z*4_)Fn}NMr3TBz_Mm>+z!Lxa0#wnlQ^n|9@$b9+gkT^tU<;@}pzRK|t%Jbngizu6M zlH91WA4>$n%t2A3q0?WhMBDDi z?T(~q#L)N+>>TP6{P?S4o+0{eZ+Pt&f}dus+}pG$-)WVotLo+r3^&tkmpLr91`<8D z_z|<6uaqY`UL`zC3gvRBixzJLMf9EZGg31B7UP={_eS$mOfkiOnR0e(fu=s$VfVn? zN^WYH>pQRcZMds&v<7+|qb>cM@%vlYr$6{94)($GcSv>gA85)5^gt;jk!XR>RbeBa z0}P^vqVaAn`pU~|S8xDOX#}7nT(|NpII9YzpK7(08gH)UJy@}5Vb4e)n69zA!4%G1 zcnv&#qs#Fz&b`D2q-Rq8Ls$G}DL7~bB|P>PK07IOKpZ>W>{G($B z1n}ll6lBEy!aqA$B+67yt{SPV2`E$Q0nz6`6_op}OH4ny8{HQIhauCs?nHBELm+~U zlQYSA$ZBwX`{Fp6jq3nue2$SXan$_*YIQYhtbljQH=8q$3)o*}rOl>F8`+WO2b;X+ z=kxU=*1bW|w<0q4_t!er-fK69390(is>RH8m{cOlWRx{@X|dkPju7!WZmI~fd0T7?Gssb@^__}E-*4E6q57GTsHr9#MJJT{E3x~(Or zVoKB+j1vz~IXF!m=fhRoPLS`F0nxE=2wuBhk7-gY%#%CaTx|stT<9^(xTbBA+p1`c zXc~WZ2?eT(nr+*R>S^R=XW-OlI7(cBk`#$q=5m5R5p$R>$3PRDtMk|#^$K{!R`mc+ zaa->R*`CFo3ngljf10cfuhtpBWcMFxNCKxuB>|v#xI!1Jq zRx{6jZ0yZKFol&TL5qgz(aG9Qli>8A#n7J22yjeAR>R^wF~<1N!X`SQEHY~R{q`oo zwd3^(N6P#NGeG|5b&}0)#F3L>UisAK*PsHfs`Oqt^Xnf>tSqPo|n?*06QfFXc6lOKxkr{SiZ@2j{zF>M@@P{Yv5oJu(Yli5(eKqGFZ0 zUGk`jJIpTwOcz=RNW#+ck(U?DWQNAX^!V;@IazQHrj`J?%?{mSlJ+dyIBAJ+f5uqsmllR;_cQ49neAmh*vY|_e zpqYO+M#fvhePo~9wF(`*16_M=S^O<5b~zOEi?p=d8`Rs;e$sjN`1dlwh-F`?BpDJh0t%j0OATo&bYbeIR>HgldX)3e_{9lW3D5? z<&<+rvK?7XJ;R}I>~&opd)ZwRLI@f)Z3s}AQ?*_qclO#PERM>fI_|9y!I@X*kN#w` zXGHwq#74(n$d+Z498%GpHFhEErasybbICI8uvHZu{Z+@Ykb($tni5778ZL=)aqRdei9JhTlIDG^Xt*3=GQ1=lA(i!Qk_`P=St9R-bRR<3Q zX?6i>T|6hSfzD0NKP+1XCf>W}lp9(}?k#Zb+G|bT-?nAv2Uijcn_)XukvJNzMEL1p zYbhoVCA4Sq8`oAnA1)9|9n>eaG*iPB4+SuPN~rsG1;VBZEYpC1iXhh}2P27Du9-aP z>cbS;Z4p#!eZQ+cR3Q=m6ZZx7_e+wFePG-#cnsB*90!!%JjbD}#aJOvoM$AztJ-yk z`1*1Nmro0i)Wri)^2X%6;u|TtzVC*bR1#)J3?|!#sD^xbsF#S+p^gC}+X{YnP}7E` z#B$rhC4(c(!VevZzi@!_E54A2B4v99WoHyWqfuZXs1>N;>R{Ux-AALl_rlMmUhl^5 zYKtJKb;k_);a%^vmv-sZ!cS&PCixQV+b@g}kO(TW^#1q10G=r=GmmxhpneIpZ)N#D zv#Q?cdYp>522Mb+ZU{X-tTuF5l6@u<;``lfZ}S4wfdht=FW+rzo>*VQ&@wU?5lu}` zQAiHcwMs#uACU4%85RjhzH8c*mV$_`sx7nAi^S<|dF@%Sn9WgSd42XiEg#m_UMu=% zoY!PvUw#{^zqUDw8_0g$EW>tW!I~~1EL>VDEuB{49Y3*pEWTF|^hLEInc+La&Vb%~ z3K$tV2r7*D6bUD1Wm%`f&kaicSeq(*cA~ymeT~+tMx{WkaNoVPN}YAqr*0E!mS{Ba zFx}~s(I+R{p$a(~HcC4nUTSA)irGIo@_5wTPz^b(z*MGEyFs?(?~^nPdV82&{$-hRPJ@G-3ZuwoX- zY#y2&jfIn9zc#D)x;tO}DvWjib9-QXXAe7We$>0Rg*L$Ue=4)SBGg3_Ld)%=(hK&; z8Ym8dnM1BMhmodgDQkK5p7I`6TWhk`2X6GAE{0LS)o%tb`kF(uL@*oh68-$i$r%As zz!FrH0PA`a=kdYe%>^W_?fQOVWkrl`ytPwUC&fKV^?W>XfG>JTpY>b_o>|5w-c^El z-=!EKCN9*r)(90>6djE);ZuByT5mjy8Ts-v<6}My{T~{hm2*-zzT3~=|5U$~7s;f(Qk z-l<~e^h|=#qD#nK_|;eF10Iw4B4ypuF5IRv5(Tg9CH#3hA~V&zWTiXyeWq&7N#0Sj zA5)>K0daLI&Iosr)ugpL+ovMVy+#QBa#ieR@`i%FRO%~&mtOn%Tu2XJ_Fe*^IAgAC zSg52reUB(X=wYM8Cl!$#vh+Lzw8yujK&q$L&WEM)Mh5Te(^$~ZezM#UR^m1On{gVD zFm^O_yKLS2*~mZV5e5!UeYrCO(yO`3V)+7L=|g8myA{NK{dKj90~I2E=4yYFs~`s| zzdAn~Bcwe3ejcTwR&%Tru|x8*H#_Q2?# zh2`kSlEVv0(Ra2aVs~JHV1e2y- z@4aTOg$rxuxZ6~Ranxvb_;oaJ+V_2WfK$nCP0@XAhXt0@U(F2iL8S?+Lqb?RE@b}X+7ZzRr5motZYc*!5zgyVPU`6-^tjOd2+c0hH>%!Y~NB_l0z>u>OLm0oej5j zV*(e7?+Zt-nT&Ofq=IGP_1{kjm*3dBrxEPua8~=6eILop@Ggcoe*J+10Ab*R_<{8f zkJ8vSY;#Y_Qw>DVRhx~ja;~{gYJa!3gfK#yh4Nd=TzJ5bJp^brz3bD7r-u?Z1+98I zIFP3>X867P98ackBYaR`ks&d_JZ5p>@Jr__Tf8#J5k|33<8pf<*K2XQIZ$-Jq>PD3 z=P~;D><1s@`lo{bm}21;qo2C=SGYo_z4^dR2?lPT7vq;}3K0C|oGV+Mx%gZ(foimT zL}M)#Zx1PnZubGN)1L9&;bzE5+mi>w;HkFiFzU!}3^eC^!-v3$>(n|$CI)J2wY?r4^P+8VO+%^; zcn5epj7Xa7)}nPk6Nez0<;n?T5QUP8mc>b0$p{g9z4{Xwr1!5GkUb2#qx!du(%0WTI+J<6Y@G*$bc`!sy$CIlXcJ zO3$l0*i?jCY#qCbQ`z^iRHx|v3K~H9v}IT*ODqwzKO6K#)*g@B_r+RQm(|QrA&WB{ zTdY#@8+S&4$OD63=G~H;YVt8G?L_dFD7)VplCCk#c70z;x&^{2t;Y4%MLBs@Y62rJ z%p$uDBc!?s7fEk-Nu)w+=m$8ZwPjVt{viPN2?&aG(XQU^-N9A*nuE42`w<13T+$2P z?7Pn2KN;#`p_pi;<3K`mVHY)Z*B^ZY?5}l)UZ211yK!rVgyT@vWW66zIwP@nSF<7s zEIfUvWJGg&SYFCDV~|W8h|EAUwo-Y$$;cS+9^y)-x<%w!eHq_jAhQ?gg}$Uaef*slBP z??!n1AKKn2y0R|%_Kj`ZPAYa%v8{^jRBYQ$Dz;UzZCe%Fwv(OY?(dv)A5Qz<*W229 z-fi!_)|zvUF?#=XlhL^SEcCXe&x_t~Uanf07WXQh@wF@)R!Rg-qUXKQyaJFN&rv!g2D6~hpdqz`+dWlk#oU8*YVY4t zE>)m?{faSH(LYzD%A68*hnwhq{DUHg52Z~aW?%<_0+pr7vka8{#9epEpc&2AGE#i^ za;+(s>~Fy6j?W>V|GlP_G_I=AU+@h7_vjNg*2YK?=EX~taopZ3FiRY5|F?^@o~LQa zuSZ29CYsHFt-wZ6Seu^i#H5vn!>~0}B(Q{`tN>@eZ$d4c>CB;ghQLfVDDqyhh z`-#{;U^;|{zLL3YR#(D9hq*~WK|s=ePgAc+>v*$?UZ6LIY}%z-UXJUV(vdFOIqOyF zdCMIaIPu^(cF6W0oRQw(_$SYvrXz)qiF&vDd0{b|J77x85d*I9eU@P?a?f+$=cu(d)g1cimED;i-Mqqaf6q64sY<<73^j6eVG169hoM)_X z+hTu_> z|F|v6Nsd!I=Kub1C;D2*5d`e`M?Jwp&+I1jO{Ud0tl%9}T&v)5rD^)|C?kggZoaQ) zEioCi3x;~$*TaDu2$%WW4&b@j-v42*_v>rrPNS?c5a6BVz8fTbP>(M8w9mG?#`D$| zBk(zeuXz?}nFjA=*Wj{kY+Kk9i0+$BPsqR5T)54XlGfHV_3z(9*I#LANAtVMMaR8? zl!RR&zAi#EFm;B;_%n1IzcOqj~bJYpq09 zWA{pXFGLbFi_KfcN(&M^_O9*v%+*m<9gRZr>*PJg^o+mvZVB2F!REwB`cxKqyI+Q; z^K-4C(|1(hutb{=)}C5?vE*{Ma5{-C=HAlfvBMuQ9q#ye^vF^E2etc7@N1E& z?sMVcA&~4hAaeBjnB6t3y5atB(dZ0oz9!`bMOBs{Efo{1FZ_hy4WYWPS5~~hTKC%^ zVg7+Aluv%HN#6Gja<9%zYyW@<%r^kuez6RfyZgsBN;(_HsozhSdeQ%sW zTH2BmYiEKBdBG7CAWlW_b+UF|&|vT1>B)}q8=8eT(xdTngVQirCh=q%LjURajEpc; zCL{FZR*x5MjV*BM<(nOU8C6t1V`vvut@FP|(F8Ca5@9Bo03Uysr_ZcXpPqeQXA1;F zs619%6`2VNezy5qZnX^IP%R82=8Ne}p6Z5%KX);LxU3oaayhJliIGNGut^*;p8Gv! z0!wUG4AE`b+x`93KpDpIBNBSaTDnVmX=B|q9Q#gsy9X_JBQvjD?I1F!CX;m6C-_yNQ?Y+rki5^ukq32qhJ&Wj z2QypH8Ow4wY1v-)48PM=2QL{r9t0liK#j|Wr(@ z8Hw^vhYFdeS1&uAapR+lvt2U`6UN8Ml`rtY?qV2N?c+%?Uv7H~m9@ z8d$c3j6-lESOaGW-TT~rV zpvI*WItELB&Wj<_)Ey_5qp<B+*I)u( zr=TcNe{y#$bxYw9$$4Iqs7Iz!GbL5s1Zp&e4E8uAQWjbw*t^T4tV)``G4ryWtzNlD zi`i_MYe-1(u125t&=RLanz`y2(8Myka0?xwcXvN&=({*O=L zuMq3V-+R1}xqZp@nb!Ejo=yave8)80_7m zXKsWL3w{i7Jvm}9+whclm&*u%Q)2}V6OYeH8Fb#}I1V%0>TvdJPL4$MQ-zFIvjy33 z!BmkUTI;dgO#)+Y!ym9pj%@pSlLv@nI;nrz_+@i%=6ky>Z~dCw$9H2}u+p4WqdKO~ zgo6|y8Xh1ub1YWs!tmO*aEoK-!F0)~$lT-@Q_{5sW6vLwrL+1L|E{1YzU)DFbcDTN zBA3gJWg8VHL%EMLP)I>-Joo9;^ctypautfdO*sY_?fv4=TXgUF5OsTeRQ7-ontE-y z5HAi$clQ3Ar4ekT9<#@=|afe=|Z3hr-NF$G%7#Eginz16k`)eb)TI}r` zu(8*z57%#){7G`a8}JPtxEJJeL?d-~Gkdt>+-Qydh561HCekv-+jt5p5ORmND+Pdr zx7ag{(Fd9b=h-Y6l?aFeb4=bGHTyK8S3+Ly{$;>d(JH||3mtHpk+Ht-azFR^Uc5%3 z?J%2&k@p{rDwtS>w-Z_~gw!|T>f8NI;R+G|1viD$Pcqp-BrDYxb(>=fCE(FuqoJ

u%NIWvna86AH=Crl_G79 z{uv2(!ePnq&#!1HyFoa)23s485zevgg9-coo6*8EMI9l1xUu8&IH|b(LA~=yN<1m7 zLCuf^bqrU-=aUc;nEFQifDA!y93B}>tLHtB0-86fzfx^0jmU-U41onYxe*m%f(e>VSQ5Q4N;Z(Wx&9S*)p9S14_-n ze(Dc+YSqNEKn+}SIVq|UadQthMbsBGV8&}(&V&inqi~A+ZEk(#zy53RUsR)CL_|T@ zKV7`AU+;0vi$rVM*d*44tS1C;dB4bEBA!Y8~?`#pk+W{Ir>NB#l<3B zmaOV=cc~_fghWdUCQr2&8)kSyiw9w?ye;=dSYH#R>dUr`iQSgRjW%y_b0^rwjI{Kb z!fP|!V(WJsZ75~##;QU_jv}8#>yWzuIpIoL8zNF92}R9cOFW;W>kt1YCYmd~aaEP% zp94;DIo7O<<1b!or2ZC`)S@SZ&U*}5olUWNB=N zW&&1f!?AksPO5>&IBOjp znB2sjGS^O5biIVB8SGH>-blx4m<$DS?8}!d$G7b>ellCyf3HK_6U(~+$s-)toxfj& zr~|Kp_~0#~S1K}^sZI5su)0-m^}irO8`n{SpKWw;)*(bc8!vNaNQhdj=_9o%e{&2z z{|SFuWh^4xV&rK3s3AkcQSN+~?aV(g8hu$jytv;4jC&jPoP-3|KUF=LbSk+`*4J!+ z&yBrlSv7qhWOi(gq+94TIt-3JS$Pwio;3YqCK7EB}T2#44AXV1B+s;*%US0EY7s;U^%epx6Z&HFafJ8`E$<=_la5{t%- zOfM*xS(*bGShkrRH< zH=1`j)1G>AFTjJ&2}gUw=WifcuJ;J+c?n7!hu=`=m8M`1zOp1RTuy85C##TM(=RE{QzmT5XBqnNNH%WQL}D)BkQek5MycBsag^_Guy@5C19(U zO7EY`+LN!b5!rY@&^aq_G10BR8ki&6m00+r<#ViA)Y^a9xpk95lp0!a_y*|w)pX6< zT4E=>&6azf-%4@EHcMf1Y^0lt8_qRrK~ru=$SGRyPp33VJ^QAk#iz&*KDX-`i_eKk zj?~70nJish$=`ZMtrRZu%};nP*(py`H`|7GA%Qo%_YLUhjGr}HZa<20tX4ks4m!Ho z`c7-C-1g1kh7VdgpA85HTEiT=SK_VFJAXJP#a|HnN|m16{uFejQmo5uykZn$ny&)F z4KAj(u0q+4a*N!3qgdH^X#-7`m<8hq+0J(h&}~FrIgmQ1YnZtBNY9g-RSo}~V>EOQ zUD0eoy2;PUES99Wq#37uYz?0!d!%_(b9F>tbMhzLb&T%u^UwxP?_EZlqeBzIZ$)q1 z`X=|=ZW7sP^M6o!cpLu|69pPF_(DyP9FqEzoD+dZA9@m4`N@BdvGAm5{KOlCKEffha-pELv6q1@m&fY}RN<^j2iiY( z-q`+hdURoeb(Pn4|BpdsXbEKG@i$#qI24;?n+V3QJm_b4AM&G`IyAfwlt!(EB=yni z*CxdJZN?T?HT~CE2LZhTB?3#i^^EAFlK4Vq5+*1YhzFzq@l|~& zWPQ(qYE0HIRcpgPLnuB9qmAw9hj|Drc9?KHwK2dEtwWP%p1Ix&HIAq0iKn4j+U&B} zYM}N;w+;W|DkMP zP@JE6=9l+l%y&u~b45}KjtVGKmV+Hz zvQ1ad9{QiS>r-MNVe9jtWbVHr^8daDlYyQWBW*`Co&Q1D%J&DF)L-J$ME*YrTdMyN zwycJwZT{OG{;wkZse~apL;A{V5jQk``>xPdha4jf>zjsdZ($a!enssZ-#i%RwWzVzAo}+4JgWO>(;dieNu5Mk} zIXMFh4N$5wIWFHZD}y0?KL#f!;UR(gSS$ckNW-|!5#Hs%6U7htg}t^lvl|7HbeEU0 z6x4o8oLu&buP&vn5`NPgRnzdaqV@_!+XV}nq17)*b6OWNVQXNNgvtIajjhRIa3LajU-BlFO zkjKjAtSh@ddJzEfokEWV^z5W(X($=&%yFn%@G2*wwr;g>(=gZ7)tNGv3IMqSE1NAg z!}o6wC#7;(tUXOtg~i2H(4N^rj{Q?M>;GWkk7pU^E6e}>y*6Q3!Fcx|#KLO;`q&~O z>MGaTePdV?3^`AB^Yc$nVGM|GI^Mn3`*u@mpq1x;Nr;Kp^w`px&CW8}*of(!k-qK< z6n2OP1zUFb0SrdJS?tZ9hGvMoj0*Z&t8{VD2{Y0R{#~HW?!x^{uK)^l&z5ztFfr5A z)zqFh`GM*c+Mr=>qk>BIaFkU2y+FjA!-qh69sq*;ZN9}?Y5&$%dDH7ym8z`x1Xspl* z`7aVL?|Gg99Tx3w)2#0&$;spFk(-+X_z1 zzct|j8TMOaq_4cDKy@5%{#G50oMiev>kSaxIW<1s9aoTsF%nPwnKF9geX;c;ZGkl_ zbMwNxO7P?DZQFhJ3P_L;-0-X`%`|GmcE0O_ACF-s>9e0cH$$hUC^Hymq+{%7-v7fu zzpuYo7I)%jr(fWlDaD}HKuy%icja_^y%mn1^G^w#wNhPtiob=>w3-|nMisB|P%XSt)- zNL|tdVUDk^nmy((H&z%r-Kt5rTJy;}7*T6ppe8Il#7@(Qdrv&h+rgpZcQj%2LJM^n zc&oI!Da_uBBW~w1*MVH9sd&LHSDv4h*1PzA;a|(Hp%H1uhhwLs#NB*PL+z?%O!<6U zA!!4)1qi9-5w-lD#J_D-ohxgs!w=_yW3!!|t|d-tydU(efckK*yM{b&*Mfrn&yQ=) zG$WTJXg5+~p|I>`WT;6O?v^h2R ze&*_EjbmGL^$^aK9ikBs_^gcRXu)$kKLbIA#u7tmT{zF6t2L8AED}}`rJu6KZ zhh^BM|B7ACGh07*7)7!QHv-t&{Emu?@_5|qbi3wFPDzQ46C=x^a!|joX2T&{cf3Nl zl(d-De?Ma~7#kZKIhuK@AcVqY&@fJFO!59OjQ~-HJMIParnq(9y1BG|Yo9fxPRbN) z$*(-2Is$3c+rIwRnwn$Hn?98;S7#g?%^~5?g0Fpqz{7?4)>iILXYv+Bj4qbw5F6!% zcE|G}ul|~2yM;KHesYeCg^In^_{7B98I%6l__%we)T+^C2%q>(y3R>&gj5f?WSl0q zKh^q8@7IZ{relWHFZF`htTjx~*EDv$2If$I_!1&EC(GMO@s{^cEqh3yi)$-Oi1dZ&vO>Ylhj5S6#d*TV66?)$F>?t%L5A z(x(bso#^@D9~Ankda7Dl;v+b0qjA!kExv+bGRuzV7w3sxKj0PZzRoMswALzWYwbJN z)Xz$psgk~Dw>|qTTy-P$36*XTpQ1X1Fi1Ou9*z5;Z0a22y|d3LIzOJTzBsc?8$URw z>kry?gXj(M$0j7c1)^EDDCGe_R(X4teRNtr|DNv@guv*om26Q)+wusx&L0}@- z2y4jA&mh<`Ki1mPy^^otbEr9*pRBhH;$!7lq;TD2taSIOB$M&Lo9GQx6OSgBmiz`@ zmAUQxU&7sI4aSCyK8~T*wq~|5a^|TSp7RzQ$zwegt(-=<3QfIlu0X`&h6m|tYMdO@O^Sm_eoi zzXzjiI@*bUyls;jI=XYJn-L2&ty|?b7Y*R*h|yi7^I83<*9nWkQIgBymfnVwF2!_gw`I1 zg(bbcbvmSBPMcLFkV*2<@BQ3JN6;FE*3q#L4lYl`6 z^p$IAYNw{?{a|FYw>q15Hg%l~2oNXLxHg5<#WLyr+=3cWuK;R z=6w~$R*(_>MKn>gymcc^iyA@yvE9Gge7|G(&basTg1sLr7co*6dvW&td7T3YLZgDg zN`OwAl?3^GO_H%hm_ADsB3fV5T%;MR>;`95JcSwCSk-ilK;6I9zS8!Wubi#e<}j*e zsMYJ7an4lTaq1_)D6+Gaw?T4_U0%YSmQoI&UcbkHB*BsCS?a+9aJ;Vk``7)j4F%8U zENeY^lBX3Z`Z}oCGr{k?J+Y3~AhjQiAIBGV_L$*ZUD!`#P`^t}Kl$9^1{KE69gH#6n5P=%avZY zMa8q4;#Pn*XJZ=w8*vk&kR`7`nPQKLK6!r75ZE5%pH# zo^})Wzh43oE?C*S!d!p-hPP{Nk@!Ik7hE!uk0}i8{6M;IF-TAfLS8uKs;tD%J}R>? zVa{wC!Ljm#Yp7U>E)q-)dvaK*KE5e}8M=Tj8hcC7&In|i9$$%CRMdAl;reCdJ0-{1 zHhBocIV_TdTdEG_Ct7-wL!WRTsB|5+$99+Jffqa-;R+ffCg?B&G&EUQGOI0@eM>ny z`lFbu8IQ3Ng<$1UEO{A!34&!`)Gmk#N)$rypevmj%`kngE|7H)Kt-p?D}Hh47*KIa zTdfK?2DrQmXNlwA(@qWVPX~38WQ|uDiOXrND?gOcCtX_llWZ};0!o^glEQH?%V;>q zr&QRL^TRraV{mo5G$sPa8V-*&GOF$wOaa^zY6Qg77WY%TEI|s-SZR=gytHN;LcDy~EovkP>0{i&V2O*pR6dj2{<;Y< z&tA~ul>+Pq(7wp@;f&!5sBMbMC z|6b6Ubdho5$waMswNCF7U|2Vj2s3FkHj~Ad-R_f3zR{sMk4XGMTdmn^^l`N5 zlj4E^suo64$JCnu-b+|J^JiFg?Cbnvg?=r;qK zb2x6y7g5ngAms=Ll}xHk zfe0H54}@b^$0JvC9nI5nKrVyB4{x<<#nFm0wnliZ*Ssv-FM#|6X$iZ z8zRjJ%SZBI@xuLtW6{CJ`i6jBD`nq~`*n83&}{EZf7ry|b?DS7ab!anV73Giw*wj5 z#Xun>8yNU!i|r1wJp>ezL5GK%+w$-gAF@mivwu_j-dkb7;$CCw~<;mH2hiDov*)&5XhQZl$67X&X| z>B9+)r|Dw#x^=1M**s-pLlJJ}NR8ql(6cgGLr$hvW{z)8q-H! zZn;uZ)oivLL4IwD?PR3a)Wk(&m)8GciKFqWP*tTWj@j(IdA;tIBsmkM$1*O``hHVS zwWETe7T(_{EBIxnzo(JqZqPQCh;DRenv}`zhybzHe!*zCtO`%n1;96&J3qw!*mx%3Pi92_iMLzTE)M2sMk^~#baZ|^RHLD$TMuN$BbjDB6g)*X9YXaXmQy%?NN(M z6^QtLQw07Ji4va4$6en!#nb!aGAIpJDHpXF;~b=4sZzoBwCMn3?5 zfUCxS8|_w(>2?VPQ%gr@#pkL+M036)zV&0O306_yt8d_Ku?}|kual#rVNCf2^y zSscLDg6r|kf>`@)myCcx>K8_v|Jh}lfi|KMk5{bC`|X4{5s}?3J|hesp1|fu$qBJ^ z4sXrhj#MEt8+shnun#YlKFc)dZxpvZ7+$2LpdghtqI#lt4K=kw<2R3XKAzPB@~CJ; zZ_}lLo1e|uJ%i7pVcO2I=RukXb_k3AXvYOV=aW$P0)dRl>6Kb0-GukSG-7CrACd2( zkyt0Qvc9;vFK!3e_D2<;DCoV=vH5f)fl-Z2>!L z8N9ZetM#ydbUcjh?OQE{fB%a>fzqIHY<)5L3L+Kcfqbp(4t?3B42!7{pMrs1$j`Mw znfm#2GnGLhOVHwPGlzLm2=VtW?96uX!sFTaX^BUIM0}jD!yA7L-r=0r{v9-qv4#dU zviphI%PbrsQeO`W5ni3~n(mO-yeb|W7cm=^(a0l40KJd(UwagkP6_Sv*FF$pR^`M_ zq@TB=f*wBG*;aj^B-X=^{|>Qpb)M#;AlpJozaqwV^S&R?<_GR6Q$ZEoTTk-zh8+x# zE2}FMvvU7%`hsN^h>fUiOz_Ep;AEu=GDBabQKE*1w100dBD0`VLX^V3mG5>dM=C>f z4p_-BiJK*i;)yH-0s&L`tGY8k3^C1@MC@@9WJGAoRg^TSowk-%I0~WY zR~?rJjTS=Yr#eT8^<=JFM@p%X`UV%u*)c#wQw8Rno!rtBSSMyzbhIqIR%vOe{!Gqa`+p5POBd}^C;;ISiJk8A8$V(U+t(r9u6aV@Y&2UgsWes{@4#_mxZ1(ku=fXKNUG$-~R$ z{ase`s#lkf5)T1;keQ6)l15tI+H1--ov z$8a#AgTKyOc_IHRzT3)Z=1gFF$~t^Ma>RM%6%>Y?pN>ZKZmK`K>I)*v#fRd6>=FI0 zWT1T-_RwqytXf5;C{<((bAU37{(2dB2%O1E!ox*vGTZo??OhEsoCdD+k-L)yDv}M3 zU&pQX4nYwer^q;wX1tW*^5^{C&pztQ!)J>s2@K5@xpU5AG|^p0ho&zVJ6|c_BQ$~! z$8y5XLQ3dx>XMQ+E<-_Yi>$0OpKnao^z_!3m$n8TZkPfn3WvseoWjKCXv_Gxs4+!o zz6%?R-ED1P(lDWLAFX9+A3#(TDtkY%#>jT}D1(6y1wZ>7h@qFnazzXSyMuwj><^t6 zyX_81iSLFB*?OY0k^b$kvH8Wknw=nbUQS+sMlZl|VeFKt7l8M291umb1P<^K_;Q)- zm~ACR9T|>!J;@UczKB$9ew{6NXn#m^E-NrW5!cmK2^0C43x-!0^x4@nDVtw2 z~2%#ID8QQMu#u4iTkT1)byT*pvx=8muRWuV+4@#9}m2XgC^ZNoe%D}P`}^!ZmP-I zL5>3#y_${e!cxMyqvHj>&hz}fHeU~izCS1>^B>gLz1}hvM5E0Aj~cN6zR6g>Kh85W-5crSiCwRs24W7?|?$0>z>?zCh>* z?%H|>R2w@L9l+o3gAGk}&l*#>%$&@#xx4zxA8^j0pa&-Cdj4wAb^lnUm86h8C3cm; zZ^cl`-gbS(ad8`@@@Y}ae|2$C5F=L`ht~~CvdrbM3;5y2#idcE;dhTvDk`CyVPMYh za}zv!7jmaOmVv+TNZ~PIj4ox@yrFDx0BcLWNPFSjINn+TwUT28K^_$9MY~ zJzI??)9FSFjSuf^LH!|{W|!b|F;Kz}*oYT&`~C}!4F!p0;Tq&J1ijb%HT&)Ld65IX zMp8$WH1g+);{>iXtc9HV9Qsi9eS0)o_s7)Kv~gIePi*H&(@^s6On6!}0wzXkgc4|Vn}vhS~l!fUii zK2bIycOQ+_v(XHic8FXLo%OFNM3Uy9jWxqIPwb95QkC40!68Y@F22+h!c<^09$&4w zV+;xkDP5tIC4_Uou0yAT+IY$iY4AkBZBrjZwF*?ITZ!KyWBLB*Hl{%hZ1!h$0Pr*OD{dB{Z6o zO$aJL#)=WX)B{d^Yjv9%Bjf#&Ba|$L7o1~R1v0=X-kYGO^$dP zIL6;L^x2d2sV&5D`Bf@i%<}%DG-lzm;hwHAPT72phes$kH!;>5{W=K4!A7&(-$)tP zjpAJa?dI=aUI4R4MgT&yykUqUB^eova+apG69!9<5|mZLUv054dikvRH2|P+J(D-T zy`$}xRC@q@*Wv7b7~19h9}fx%QA$fhtfV;m#VNnbDMf0~Mp-=U;SoLFuVW0{px8$Y zXo_p$AOP&YA=t%;J`DaV)069h0fwfx|sgM0n#uBIUk01C}#&?`m4ih4DJE! zC5XUvQAEV=ZP6r)fuSYhfdP?TB; z@M>nJx2q}e%rw*S5FLp$yaSK48h?7Qb{Wl%=#Qi#BWYEv`hlE1R4VGKYg(pY28t4& z&93d5G&5GOpI~7Xis9&p+GwtD|NI<6>PS9G4i1h42WgoI%ZQDR#aA76hg8nyRT%h!Sf!l|i72zdSo`Aw4W()J<{ih74!Hughn&t&Oy6yl|s!He_$XNkOL}Hksb9^B8-V)=0PDWpq)?hWFnQv{*TXX8$VHnDsHC?A`FBXL5zT&7mO- zMxT06Mz*w(g?;!DDw%3bNkw^iw(6kAWxX2S{_o44w{?!0KZbt?0m8dD^^3P*(T!|c zd~$KsESm3ZIbi=SMw_k}b3gl9S&J`;FZ0t^=_f)X#@%SRy3RAb4)I&uSmV;MNGyM; zCvFr!Ym*Hn0ilcjb#4dy9}9l>gVb*dvwP&Y(8y)zc$LGN>i-0kYo^!gtUwA-nt^Iv zm?X@w?u2-O2r#hMH7i{DJsz?RnFL zOJ4mxj!L+#Y?)}8DK3hx?h`N+rr?+l*T(w>%`|8ryrh3nnnM2oBA5<6@V`J&f3K@5 zRU3H(t`@gR2ABq7h`5k})h|2LkQr(&P9eMu@L6+iw$cD|TdI2;XuntQ*s`yoGI}Uu z1riC*zBa7$%{;r@5fP^erX*}c=FZ^7_Nhp5x{fM`aSVV|%5-+R%xf6+p+GWNM+xR7%kbgnin+=cNPAPe_LXJXBU>UxdNK6paWR< z07$G(n>a#<4vrY0`!B>u`a!m-94-k<-LW}L8$k~wSDm=ENE`KBfTLs?*%(+3kPbKB zwoD76vAf+*1tusYA+d&`o?{+IHnAQiq_m2qEzJKIb^f~mpo(fK^#FkokB=S?o?b?1oc5p!o5i=P%C>-n->L_^`k zn{^sU0JCZEdak#XP*<(x4?`m9e|m2b6=N*YIDkia2aWriE)7m6FRz$g7MAej1C0d~8!uab8jj`Vd61IHtox ztOXH`zz2XwgHjt20cIJ5z%7z-jTI8W7RT3!%o%&NBax&PGuSOZe4ex!`ACBM;dfro z!_!ct2Byfk_&QEQW%pxh=4bvQ_d`rFEkZ%p*2Z%Mhfbod{3NA5?JL*06e!ncoOzy^ z*%S17V@W80bLL^5!`35A%7z$r-dxeR=Q$t`(}yRX#0G^Q4EJ_Y3nx+z_zFkMtZ|6b zUKoOevgmmWZ_HFD!Q`L4N5eXf=SToSN5}LO*J&H&k&vYU)xpO&V28`1QibAmOV2Q& zT5UcVi=GvnvI36p_Vb1Od3&5+P+xUJ%9ZEd zMgPxAfQd1OxvN7Y(SlMR`}lPSlH_w^vE^N>=hK7TJCh0FsyBwZ8@_2=E`XSxCs5pSen z6oj-=6A#2RIVF7G zgr5LusYkrfVw1~$OIDw^;n<-iTS;0#K_biz#@eeHl%_*iMrI*;wr0So{-y8kzM&Gr z`Zku{7TxmT#}C_X?UdS1Vo=FR>6n<}0xYTTmE$^;%_X*irn>i9s+~D3o;+sIq0v~w zk|Q=ff;HcR>Ub@pn}09^NUyfN((x3(HEANStY*7ahN>uxAQnM`KoQBYuYrTC)+zVU z1B6bxWEtg>tci&Xi)85v5LLWg+_J{{G?xA1X>v-ViC1M{2V(ysCp_6|m5FdKBdT&K zWvMwt$OM6ou|3MUxuQJ-#dn4uJw(M|=kj1sj*EJ8w{qHht}~#7gb6{hiDpTUC@9$Y zl36Sl4=|J08hI5(_2E$7M6*DvGELvuAo*}Z zx3eZa^f9igMUX`-Xh$&u|G0{1;N!a|!npD|pr#+duZ~tYIzE=-%lNVyi9l8`20kiX zB1~$I*YTzPzbByXkpE3U)x#3E9cp$S+lm^p7B?8Su^IuyaO^`Y5mTo`u))S|{)4~8d zFI@Q4(a}w4uAdUbG!7K~W1{5;J(^DFRC0>8MKkgu|^kPov-{!cW_l9J*>Iv$?$ z-}v`%Q4@DeYRh6_;UIe|c0|`=m|VZ8l6+P3CgGn77T9OK-Wlx>WOgR5K$zK2pMsDC z2*jo)dW7E1?)SHQO`n4&kEeTB-=;(ZLYa4Ty9lZr}F zI}r*V;Dqyv8M=g=RV^7O@+TwhPh7YLoo%4GotZ-u#?mMh75U#m)_k37F-YA1c9E{Z za*|uvGp#@Qi$3}Rm7D^aL68lNOf1&v7-Oa$2@&`dzYtbk_Ut40RJvX*Ii&-XE*uP! z=$pf>_=ZjzpfKP91IS?(K|JRjs21hR0~OQz#64qM!`b|#Zfkp<``fc+Jq!m#z|;l1 zG|w7@BB;{4Hry{y$^|(qURPm65N6Ev+B_TOwaNa~sQz&P=Z%0X5a?3HrZWj>fkW?j z+ALS>9f$OSe9L1rppA(|oMe*P6&)%j5}H=hi1`W#grbT$QypEr;M)1RT#a0GfGlf9 z-CVl6tg%LroSA58DcVl>9YO4CY+62MTq4GfQ@Mb6o%L0TCfBV*YN$5QI)bY`gCv)o z@HluQC=uivlMlukohBLm8Zd_XIj z6=mrU!8Lc+ALq3<_dLg(B0MC7m`3LA?iTMGb94FCCHKKz{fEOjS*e_Xsv`bgwziJS zwD4f1pjZ6CUx#i~iIVG&krbB^aY)mXOzj?La6;^Q=*w_#s!UDt)}X)h7Wqbi9#&=Z z@c3Lt+6o#P2C9+33|7O@vx=-rAkqmof~NNSXb>1oRZ3BFsgLDgYx3UwpUi+4^7X{ulXrR3leNh%(S4OjM8& zEEm-&%`SzBFrs;<3@i0Zgn*<0!KtyM+eqsKUg%W?WcX_olyoLFaVhw(nnjXmmxN6T zzn~fj0S$y!7^8rN{&^ z=zRi`>IW3?{(yxR?H4yjj0ISc$YY2zh`8ObIdDxo9FDjwr&n}*&@xcP(me5Lav1sK zk7xg=eG2Kad~;fb6V_!oqiLsF46MVnsce?O#89r+0D2w`@Xzoh2q{r*6V0f9);Ykm zzQi8a^W^$+8lN~!p2&1X7r`Z1IgsA}hpu;wuC(dec4MvBHak|wwr$%sJ2pGEopfxQ z9VZ=jY}?6R_k-`-W54hIn^hTCRi##CUUQ!3F~1xgc2exfbdiY+f#vod&kC4YK0_yu z_g#w_1}$EIoF-2xABDGp5cjbRyg{x6 zmf)?$w0TkkjJTN4z#c+U@QzgNVMyqBNB?wPp$tiE8Gp=~u=|%DH1H*~yDqM+{=IFzUDT zJWQL{veG#!uEYCCiVzZpQ_?sb)~dp6gF{BJ&ct`^lw1}k1a}b&?T$RN$2=97(v#m- zQP3~e&|N2z_F<`*sFF4}^|)vhwe&>l7_wk2JnjH8Fp~Y4vd*rf@mp%oMFnFsi)c8=3FeHx=%bEur-R1bK|S~#wMg=_L6n*rLdLtnkvjD_)Mf4%6G@_LVb3n`alR}8n?Yu9pQiC>7#Pon5$e9LQ;;Ju3UT#t7pp$1 zk=+{lv|9N)%uzI1F=O7La|$ghYK31v3^jVl^LW1pzH5><67S&naFbC(|N8RQb68P+ zF)Ec%>?ey&VWluj?#*j8tpu$)`&Y37s1*F8BKQQdwTs+{gc2UvB`Gf5BL1j)dv+aJ zgkW$ksUQ%2Ph32%7oO|=4%*}5xGPFI|6Q4fo2rxQt-v;A6DG;a$mylJ3HChM6Kh?2 zox@>WMz3Wb(naNlY0p@gLFg@$&5CYST9Lg%POL@e(ai#@?0H3!UtyvwjC+c^Ji;M1 zL_u@wkKM}1A4NyA?Y)C)0wOixaS~|QrMc8doxjgZmnG2_J+}*2i3wr0y3f0cg}{~z zNsYeL%1M!n0fNNwEb7Y2VLlo38J@Oqy6wuGQ|>T%hM;`ZA~>jY;o*?pZJjst&lM;kHf4r+y+4ED%gnkueeQ{?wo1(347|_wN?PCnpT1f{L(mprKLq z`;Fucmgrs|)Y;bdvmIiTEK-M$PZt$c{MyV!sKX^Kp|uXHSUwBL<`&i$Vm3Gyf+v;v zmJuZ-66IJ6%GF0<7s-!}w&+=s?SCFx05N)uYy@B=xdsX`jB!XzzR6?E<0jf8%%dns zTAxX%)B7J#Ddcdu5ca|m+_-wYUGK&26G}v&C7;Q0H(z=FiP#OJztUmK2|j>l3yyn_ z!lqi{HN6>68ofc#FNDALb4o|cKpIczDrF;IO}&BKP98V8GGIxH@3DWi#zFyg7L-#7 zVJY6H*x|~n67mXD>d&4ow7gjV0iTo@$^{B=PWL9B!!#yxLJ(#Z7ZY@VN_JDm^?ikr=h#bPp(z0^4EyD2$mSA|Masy7%3!f71|f z-l~XkJU34GRh*veLkC&-)3SU<7X-n-3xYGeDoK3gx_}b(WXJqy(v%K2n9H9|sW3j^ z=ZZVu2_;O#-u2O&OW-Haf}#%!%Fm&royNxmef0e?`Cesp=&y@!u%S}WUn%hE*8nTn zkC zeX_QGtm}S&%r_>@(8RaBTElddxRbN8!hI-~R0=H}*ke3PoPH%H1j{!!Dpn!Pye|A! zi(ou;a8|^~;){<%2JNV9atzNu6*UQ5K9zQByJ>Y3~u4i9TpRY^Cs-vwki)p(2S*qj|jvS?fuqHBl^ zIgvB!!=eeJfqWhL`JP9YgVlwWbDPxIFIb!qlf+5IC8fNtW5$#$dG1JX*TaMbQ=DXK z_Y(gp0{o#ijGyK-cuR|

Dw&XJ<)62h?MkHaJa&RpqB8-?w&+`>mF&_PYubxWE}wrga^8q`LH1%k5Bv{kjrruiEIc-S(UvB+w8Al|XdV-1~h9w5G*_1siZGiT? z6^#Hx(*CNL;80Qap{BiYAzIC+7kBx#=8ZNQNP$Qu2IE4EF=hrcemh1sy$d%foPBvP z942sP{IQSO@Q8(GB0E%dq-#v=7|4xi+^}mZqusWC$edjHE8GGkLV8tx!rkBbZ}9gA zw2;2afb>$QZF+opeH?TRRRiBc@}RXYGe&r+ZQ5`jsZ8lqH(kCEt802{PK&+6%_hM5 zzHXcF!q4hd73<4 zFHCp}$37MsV1-Q%OlQuFg;zthNGAW?gpG_l#|VnK2TOX9dML0~@*_4dAlYQ>W~D43 z_yW8LT&~YV7U3AaTsI+i}j`mW)O~_xbIIE z*?>rSqt{u_2ed+Amyzl~IaQN`48YcFn^=f>b_xs=Us&mw;E^#q+>y2hE$_$WhM8@2 z$R8FzD0qYbdLyxp@wJj~Vjva-WGQS>zhYA`wlsHbI*x^zkmw+iM7VK~LD5)*uFshX zW*s%*Z+z~oXd6O5Je2zId%<(sTW&3IG z93BpdoT7$%ZxVA*V%X65)~DdHk?xL$xLW(3;E9;%)f}AkH;YKS%Ms9y<36~8ni*AD z4{$=nq=;=HW}j=Xmfv8pc4R!`T_oELrZ&#?@#+kRc&U$zOwUx&lH~>oVk8R<#rlh^ z1>*fB7!Vrm7o|1sO;O3S!c3%4oJV1}9`LifDoQR2Hs~?kKyji+HXn@^YP=Pwd0=f} zO$7@`s*8;`Pa4QP_f-tuF$bq`VK-3dyNBgl6)~bmorZ>folA5(Yc=Rn6>PcV9X_wx zc0fdc&FEL{_PJuVoP-BsscB3tc)BG@uC1)>N!ID>8r&Z}&Q@_}_MNZei>z2`24Ha% zh^MnlaUy|VgTi2GP(6y0Ltt8~llOoPLw7eeFy6hmqGiK^#Q@fEqA?9p_Bg?pfI--F zPk?L?f#QxrQsY`c=4;J=-tp)lhwL9X^h2lga{R7r%X+xj$bAbCKFK~;j#h}Bve#}y z^$+d}oGnuPcIdT{D{R77;Eh~LK&%Q^_J8ShmRgG@8+8u6&-6^Tgsk-UOo|X+XS+Co zP?fPYpv=d7`#atmpbSCy-G@YKj7ad=sfHJQHFr)8xe|SN{%5(;t&4P7sx5A0$cn&_ zO|jhuqUA`_I%L(?4{8w%Vq&IoWf)~$MZLe$6zcj5IIt)mQD6-ow-`SN2qe)?Ln1`Z z#snjzLDIskF*uj#VNo=|QV=Su9GkG3rfVdx$YE&wc|qwi1r*R1I@=D~Q(ZX)iF1`V zVxBGT@u?M7xaPrw3*jStxypN+gb$57$ihW*pjSf?Z?jw1j&3tGutCVjRqxvbL4NwB z{pd%YgUbR7^7qpN?+1@6B}I4GE-&804Y1^cV~5}>M`RzW7sRufUMaA4mPHiO9VZ#` z;g{w`Z`l!~R8BtUQ6q2nr2k=AH}>0+-FNB3R`Clg>e7W2WD+n>m~Nvb504)%%wBLy zj6|_xy=oImO#Xu~Dj=eCG^Wob`eX>C`%@7VCg${bKO(3SpRp^90fLCBH(*Lfm?H8B zJ-MQ0G(P|d-wG?oKwnl|WcDibV%?Y$s(r|z1uTC=a65QOA;g_0oVe^U-xLVk~23Bz3*snOiXHQ>3-P$uhsjz0N=vG20&j^1HCn@bi!z- zh%Z)xWy9BVD^cQGuvnNrt;6pZM5gI$gU?g;pnyJIyRl7R)1 zno3?~O;MkU|IUzZp+JfXFXt}Z@>n-{u!2_?bOX*j!q5#{O;`s(W{y-+a^eN}$R^ka zkf9tZBe9~=pl(6|7zthGDzz#-6U1bk%;tXn!Z{G$Bhc&w7wckay>d zOgB?86Kz#O85yrJh*2ZIO?1DigPmM4)}YmG>PaOJBo=mu(jD9Eid`8PJWG-jOVQ|$ zw}MElCMlTOf#C3-AS_+~axl6M#6{%~VsvGyUJwiTH9E#E34bfzUu7}&s?PQrq8?$L zMvsdwxGU7e{u=c&LbOsnzotAj4GSB9-keRkEA*rzGJliVf;`y?i1^tz+D-yHP;=ky zigmmwc!`^Qslj(l%y)Nyl>6kjq%yXGaggZRiKD-`EIf;tRE=QlXgT!6-wZTom#oyc${zO47vFcmXCn(i6W965beWxGR)=`pycxYk;%ZfWs%pTUg zL*p(E1=Jto`E@UHUC37UW^h| z%Th*o@83143#NMLOe5NM!h(PLIIt^vek*CJ)>ijt%;=On8qTRqG=jM2_be}BFGK1S zxk!t}1E!*{3)UD?S-cQPjfIZc07yu?LvO3q(Y}I+0oHm^DEf12t7a&N>L24Q)ID?$ zYqz+{hHotgNg?>q$Y24GXAxZAB?wVfMw0sPiN-{m9kHX_+YcoqBf8X?=l<(>oKenC zyLX>a-f`#tbxsGu)I&kz;G^$k(?lO^Y@P9$I|=q7q*CQl5Ng=e)e_LM8}J8e_H`p_ z1R8=Ajz%E(C)BJdb1(~~eXJT5)8aQjKzZ$cXppLcS8!`^7yDdPg@)i&xI)v#V|anm z7!*=UMn}m<{1{A~i{|p>*ucm3Tg~^AyV)@`@N9(2^tKE8c0eg8{(+r> z%c%~-b2;S^6&f1pxYMwQG0ZvJtiKhyR$i^?*Bj%PXdk)t3~|^TOsRAYA^sxjH_NETDTwOrK`{nlnV4ud8z;T=*&o6JIOdtcBJqg)GZr$s21wq3nw zMFSy#CMwqx1`L^1Z+9LtSqd}j#nqnubgum8wUm^^(z)ANf#NyC@Y6}D?K!ErHzfQc zCcrOajEG1;(O4FBZSOoyzKUR3U$+RT9%zrpi1t;*KjB8JFzCKsQ~&WX6Ds;n$&2lW zdIQ9c8`~+_aQJvVANbszy7tGfQ{>m*FVV0xKsk#qr)$UKo}4e_YnFBI7Dj_N4>u%~ z3}}P+jab;3+>Hjq4tk7wdI0*s+P6m7CX3DVBudA1^Vjt&}U~r@XDe;RsVg1YQt^cp;!#) z=tM43sCKSv6w%+td&kmd{oP^MBV(9wvLp+~`c!%=-Sewe1|dU)j_1nWa2q*_n+L^6 zR^Tpxa>)#X>>o-@UAIa_d#L7;RhSM$u8|x0?NCv$E~H#r1>Bwh`9KHRz}{fiz7DYW zSP8k>f9)BcBq%TMd`bw%q5Zu7f#Ss9%@n;(XfL&^hKVIzUztX=9F{G6H0kZOCP@^P z5Bdi&3+#WFmwn67pLrAm9)O+TU56e<{t+xcLzjRQmW_k^edTYS3cp=mZVEiE9ItGB zgX~$tc_Wa;nG2bJ%2gR_m4@ri{dYj<3TH_(OKTDQ80ug1uQ~p!6!35B@jvtXmjw}6 z8=!MC($N#yW{+ESq+ZP7>0@;!)s(2Fn?~eU{p2Q9L zS^^yNB!l}A`v1G~4Y_|51D#fE4podfIjw4XwblRo)B}56;50vS_R5N$>{LLSTN{fO z1_rXfxKr}l+h1BY%MrgGsrx`q4x`q|j0Bn?uRJnofjA5rI`j67jNz?SHjM-pd<;K& zmSV}TC-`q{rmDRZg>Fa*DA&6vHGqxFHLV4Kod=HpZpn4L1n8uZ2*D>}-Ll8;-c@2v zZB0+F_EFLBVNo$pA1_x|S2hlVt5mlhw`$9{?=caolsb@ ziFp==@AC@_H60z7JVhb~qE?xCpk|)C2Rnp#cwUVq&FwFl{p&q~_f)}UAWa0{U8#LB zFfa(Qx6e*?)HO7IG&Vozkh1(@s@-e!&A9*j&dKHZ zuItCE+eQAYuBD`)uB@rywxfeX{A0^w>7k8BmmBk`-w46<^`mP%Ap#Zw-Qfk8e(9oYOGs#bP`6c*@|I=GQY(;cT}pkZKDy2aEdStpVBykpEI z%0TY6*kTqPzY0L~yAfp;B64(cx3#kqIo1F)wR*^AavoyU^5&#zxZWu97poYYV6`fM zQc1i4`_@~O%B8dV+&=Foh9j1lK)j?z#7q&OA3yC8@fkVroU4v-+4q9Wn_RZFw}0-3 zE5zXAB;_W$eZ0nA<thXT5xDbS1i-urb+dN^6_l6G0`Lv-ls9uoXe|dl4gv4wHS1!mO&SyWgo<%43~!M$;fJWP^t1C?aSmglxcG zh#p&CUAxvLsVswG2sihRVx-A)VWo3;vS*YN>7HirXAY0Hg4^JjOml>8FMedd0><1P zV)N`)ZVxngdcOw9(5PZY&{hr5TDb~v*qI=DvZ?^mIqtr1HgntUR=eq*ljl9zIvf_Xd-NaIx;01o)6}NU{-0ZQ zJ5D*@0QM@b#rY4IiI2L!x$V4di87sdcFMA)d$auXANQy|_Xhl#k~*1jd5YK8EVkWx zIBARsFVel=`#R0u9yOq!YLy;dsSvn@Hw-(W8pN_gMw$^ntX7xwI^)qXQ%H^HA%>v> z)<3j^bI%PG(5$yoFvS?rD=l>#o^BrTxmdUbdSNu;3qdA7ADZF_CDGs@f=I+9sD<_# z40unDEj5ajBDNT4J&KmEdKY(cLW%;eK&wE|9S$HxpnaRrwL+I)Djb(OZX+LS=om}f zb7}pcRB?XJi%hLv$4S6hJLXK!dDhaA5%bRAC z_QG)Dg-xj~6KJdenK1dTpFd*1n_eqos=rd`K(!I03@OP(v2>kVY@@^EARxs4_?!$^ zRXu%-B2IuLKafL0pr4=Sz{SEC@4+%u)>FG%!?UT6;Olw)e7P&_ejFKndm;+%nt+kc zJ?YzKXJM{1aV7sJ49s>XzZECr-w zt=>P{kq`pb~k(y(jz!4G<%RxpO z1lo4~$(6m?;oofrtW+45kAZ@-4TJ$5G@RdoOcgwme!nq%! z?2r_Ri<|68Fgw_H)uXA{!B z8A_gmSq8pxSkxI84n{V0IqpcCFGG?SFc<9+5S+VA8=zQ=T384MIM_~^BDOeff9E56 zlRfmZGv&0ut2aUA<<;}d%JG90XyHwO52-iVy`flMUPQAz-Mf(A|C=!duwa=%z*a;k za10$D4Gmn`;8Nk^)6<~K)6w|TcPEo}7f&Ei1aT5bFkDR~hn<9sWo5(BRJ?z>Q-l+P z>lfZqha5FDHgtSkK3gCXrrWLUa{qiXFrbVREEt%u9dKQjUo^_X*}~!Oihp^38+8f} zt&o~x00~Ag+D+Gv>%h8X#nGF;!X~2_4ZaWLTEI+=IB^V(K#@9}|WsI%x-U{4N{tJ+%C9 zi|Yw5e*s%#R@B&l!1&=Lp=06TG`Kt&nK$j9rXNSt*JTv(eKvU)n0r)kXyd=?gGbS@ zdb4WnERKCL$29tja%+E*2}vN9VvIWOF0;o3ufy7^5RFJn5F?|7L~2=v|v z?tEGBU&i)#`blEjE z|Gv|+>1}B6+$SA@heAk5zM9p}>+%{he=A*bMPEP?s6-|lmp4{jQ3F7@QIL(ey>%vo z6f{v`b)BB*DbKkA7Krb2z#Jv+n(>Yp56#RE4XJVJ-nOsK_s5*EP*6gGn~J&{+lsSk zwDk2Unrc3Os16~2x0fwUX{LYx9SDu~GK5rt41c!B>4y1$OrfLunIs`D7S+hOSCm-m z!s@EiT9YXU7L|Izz`(B>xpkXgCE|8Xw1scealqpm{{l)-9m`l}#pjB|QcEjT zehL|0^%RwV+&CpxThZAADQJNMMdBlM&I1~%78LQ3a7AuYFm#Q)$yxb9?V2M(cq2)u zV6nMeF1aN=2RvLsrcECIM>YAI97o2akBI@$5zM9*!klT2slFF9~cA&vRu^FmKvN$iyR40 zVmSoN%bsp-wCipE7Q;%RiwI_4mn5MFFZxOMv#v3aZa~rZ3<}mQtE7I#ks(?5@5{^*6aZ<93?++ZT31Pus zaxQ{ohuz#Z^~P-8hWIR&%j+u_f`CV`x1Yw}_dm-~m{U{S+Rg)Nf@U(g>Kg~Eo0~Dn zAU|W-ul)SKiKs$*q3?yO(}A=C?~D9qFo-Nja1CurRZClj=&LL)k4N+&9yWJiIc)m5 z`{l6=r_7}T7Yhdag^P1dpw4v~Xiig#1(8RCg;hGhP!q+3fJ8L-xZXt)_zU4tQX1Fm zV1zqhW@6IVnRPb+sh(G}{<-=}ZyDT2MecbZ0$5O1VH)jwbVUd%Dw|`^-VxVc;tjIv z;@waeXuvLLZ~281cl&U&ZMl2(bFDSxz>*jZzQ-R+Pobep5~f{GsUUg#_^7Kc5jov&Tl8KX{@y z5JnU-n>R65+IpXhZlIYQ!5kL@!|hI&R3R!%#HAQ%^*hD0yC>|onvZfC;ob7%^WU?Z?rUa#qAQfW;a5P6F>!Mp2@10)wpo`B`K!t=nfEAZ2MR zJwJtmrbx5CPJqQIoyFoMtFp)5=k3!8?8x!5^hZDxX!3;-jfL@h^g(hm$6J7s7ntQt zOVZI)gu@G{6e@yhRaKCV@-sa!ZXzE3W<8o;^tQ5*+lq0LJ{(% zPm7f@hc_jZgzVclS6k(cr3veExoQ|fd;6RE#B}rbkFFnIWl!7g7~g7G9ToU{;Mw6n z;I^k|rijXefUw9We%E6~b*k^UzgCCwbI$8CjfR0{a9fR_{c$-=RCGZ%c;sw`9#1Z! zs&M1XS!3(>%;oSm?^<6+quUh-Tg)g0h3VO7ogA>?4vCQgRIw+=S zS9C+h0$PR*uFkTdaba!5yZ_y4MY=Dq`@AQVwGOM+%fZjChDkUys@CP^vJl|2{gANitcVwrtvh@5!IOgvxm6zpA=>?pC zIiw4Mb3$!?B89b1B44LlQ_2BQ)cp?`^8&Z`hZ=<&+w9*xW8!Crr|%0ye34EcO9H*| zi}Gb-+#*H`qC!wkb**A+xryBVXOABSSKJ|S($2U+v)^c|N_Mh)YPafYdI}PI7FH>q zes?*{#mq_7iw&3DAH0Wd+(u1guu=j&`;Uk8J|0&*QKrEh!IF~N68NKf*WFYa$bs*` z23pb6F-#%Png&||_uy>yP6b7yJs+3ebcock@iqEU=nY?e<3JTpkp7QC0N6*$d*Si1 zVzRR1od{cn4Ltn)aJZq3O|%UF>3hGwYya=1ii-M|;AR8Ty|6H-uG>8-EILU(XSyPw z_3-_amMA+u_XTJ(AE%`j3YU2G@;ngc%?pn&M#G54!qQlJo$>I!o=c11cbY?);p>E> zc)34uvD~B_llh|<1thMfdwn0h3|&G*Z294}SJV(y)n~em{sWyQFQO9wb3Ke6q=Kt- z9$3;SxhA?Gjoc}wY&`YzZT|6`?{z7)+mpuWyzBXW5Q#odqMS=)YK$~24C3v#bk_C5 zuR)9Hec~gMYz%bFNg)NCJkf_upE&#VdLjz+{@3x*D}#1pBPS^_F|jnf1!|vde<~g& zO-thQkeIkg=wS@JKxg?G3<@|m$R$eJGg3-yL3Van`ZFJeTFnN|SS68W!wn~a={(jH zB|#l1FDQZ?_TKzOh&ff$e0@EUz2%FyWxJI)IXLrNPLdN~7DrY=PRQ7Nf*83fwD$y( zE}ZBsQ7u|=?d$m)uU1GbxD04gM{zMe5#yMSj?O?7d@3A1&`d^mF_1()2aGi0I-hNUT)%(gz9)*Yp4RS<5}F<2x%wq>KEl8R#Md1>&15iDbVsW$TP2 zJ5hbZ$U!I?44%YFPSqEi6BeSTsd}C?5cmh_QzqaojGuKbq_Q1NWBR#K@e*nVG*o~d z@$OjfyEy~uk#8{Bt#;l|EV~?PwMKXEiQpP1W6F767`3Ri#Lv6 z{b&;L7TYN$Op$H+p6+IA_vpKF);4E#A}D9f-R^w$x17A?w9;QKk)Wxf>BaSz0P(n2X;3W zSMEpuEC-Rei__7tyCKAM;;Tfw|9!(F-h9f{o*r4uGf-VG?=lxRbRijqm{vOv-cL6) z8GwIX+?%vKe&CR9Mttm9%3!yA#^bo)_2u+?cAQyW&icCYaF}eeTUu%cf##z`w28>+ z-h8v^<^A!=GC4emvTFb(elVjTMYJ4^;nA zFmlk$+B9}?NIBYcn}d%a1>6UGLDN8ZC39-Vj7olK{y?Ey8I+cuL~%lXUG+}nnl)L? z5)pnkjULlqWLKz#fM5GyyY@?|YZ!{=u#^+3qyAM`? zEkk>BAg}Xh=HLKa-}Q6&d=8|@8C(d57LjMt-~1~6w%X*SAS48P;}DS&i$HpW!Py8( zMp1W0<__QXHt*4s9pdkQ>C3R%IO}GHp`~Z6rmj!J7bR@o(8TF~OBW~FyTcDr(bA$P zu@0IS@_d}du|BKzYagzx$~Nm1A`GSV@MP!q^o+>v-5P5G9LY*}7@-|=2R0pFPn_)F zVp=bJZ;uo?aZbr8OxtPv<%!3g$vqk?wJjp*@cv-3Z!&8>3;e$#_RLlgnT+)_BLinu zvGsz}6DWZ%U;gtQl{XSIR$OSd6SJh}4-+c$pe3=~??1p7DL6Qp=qQ=KY@&OV4`pvC z)&w6YFWh@y-;G2T*jF*B!S)dSkd`jHH=$&-kUf>MnVg9+sfc*;ZM1fkCTe*F?OmSN z>m<3|JCaOR2SxL#V@<=$^&dfw5^?lsRa{o{KwU5@$}PHqfkAR-u5Pb;QFS%kBp4~d zw3KPjT8DSG`7t3AYZ#PgR!coHiuSnT8hpKw#fZNUdPi_kP6#Qtl~v&U7{P@JPN3;rL@Jl*6-sd*4HH7QcHCsQ)pos?U{aXXsQd*&ju>>IQRqP~ zM(5~YG8E|YG#i=2>frRVW~r|gsjkVvKfke1|Fig`$H~I|J4)T*ODM%NIP-60uSsE1 zT6+2#St*!k`aTdT`vRr{{NokN0H6@9D!L%xb^=y;@gzApafmqIXTe61%k>yt=$Ywz z%g?G5(L2NjGf2-{B#JnVG{~!wjpF>?uoK5ihnp*_VrTvm#!bsoA;b%YF8s^ zDl=*Awjbm}ztdc>V4EE?emRkXw1h99L*D96zvw2LE{9tOSrx=Gff};a3C-*Xq#8uc zJ%g0f*7lQdw1V)^G)4|5=pc#$-99MW!A6o3jp)d~E&r~XFYy|+)Rtps6`?of-^^@EPFDt97$7is*3UZw2oUm!RxTfk6 zm634!IGkvkY>+r84TY3&qA?_dEq8|^8ygoWQIcezKy5OEVt7ctBjc;!nz&NbN@mtC zI+bzPoNu^Up0#vWXV0<{8$Kt~m?);7a28jUm#=l%F?Lc>;`x(5Qi%rbVN|m&iVB$t zA;Lb6Qcgk+ied!ReP$hFL9PJdH@RG7!;=ep9eW7q4yf&{Im~!-jOutX#@mC-Eha}K zlmI+v7-T;=O*Gjt_IfV1Vd{wT=V|$BrdUV{F0O8B2UO^flK)mNH!%OHo2HIa$z#JY z7UpqmU0YgPi{Fo)h5DAB+wIz7oQi5n8d_QsCu$sP=;zvtgd+$I{gi8PL3<`grFTon zD2-(Ob$(W3M=pZ-^H*6Jk3pwcq5tlU*-CLmkTH<;CYVo?Y66$um!-EcxgH$Gpg$$y zESOnPhfL}M;CejSI|u}m#StZS_hv}82^tGgH3h4Fg@eC1w?l!h-Nz(ltxtOc9f5u4`vgMydI%8V;pr zt7!j+co5U)()WkfCHY?NpRU56b3u=nzaVmwUD2t~0KDrEvSVpeE^ zL|s2$Uh+SJzSHdoKnb!VyK%u9lkf5IafIr{>8U7wgKS%r;|L8qIdAs9Hrt~1DP#~! zkGDLiXItlv`B1d7wtDCcvW}Evf{XhE3Yj|;C+(LixB9sMc+?s!A-PDjafWr9k% zz>C4qB^k%oZUxa$IE;?C59`&WUx{M0HkYBT`g!C#?;-IdLiM0)9%mq=|un!$1wJ zc&NY*H#IE8fH-G|*Qx4q)WIF+V1v(-5u(2dVGx3+4Z6{VNcrf7#AMJ9&7})lJ6MN6 zP)FE#GCQfgKZoYkw&X@5bz!KIgcVzJDn(C)K z-VAQH>iAYH23Og@(9m6Mlwet1Z_l_4~>TI@EU8ES#Iuw#!0dR17GHD43R#L`=5TgcNB+kdkiD z0__J#g0YhN#>&nt91{D&+kz)*Mx+RiP4PJ?idE}>4Su6+38o`!Ve}jd=Vx)#n>|#% zDWXP9)>IW;5eO5`p`TtUOVX#NLXa1A7<4Yjp9(XmC_gN7xCtdYLMFZjSMqa=E(Zw4 zFbAby*!;$ZgcD@67R4?Wk2Z%$%p)dHN({@(fe4!i$)vj^lGf6as;$PkyeBXq;>~i4 z1P_736dp9)@Yw@*2Hz_qUHb|J85k=RBKd0-6?qhe=74PwTVKc4ihA4Y88H}TD}K9z z*Xr3c8<($7oOLN;3OUjU=812-Hjm% zyoBg3vfB2aY)_X#XPeurBGhFwdG$;-GFDWAY#vX|V21|{&zQ-ZIq-Z00?jZPv+K>? z{DbJ~!iZ}goBkZZwG{NV&B4YCcnc^~q96=V4mfGb3vWdd5r{Soa~5W(7)uutpaAGG zo_j9(h<-ws#|1*7fs-I@>Jqao-!|9uG{*>#P&E9cj;km>pgj=~uQZ=vHPBQh&r*W2 zU{r^_z01JnT*ZVu{-a~VzZY|JZ#BL@K?;qx+Qt8DNoG!rN&KFtsd?Snf7dXaRYMOQ zUz}7JBV`KG`{$cQ7wgul%1-s5Bo-ejJUH^X-|eE7a)ek|ctUJU{W#Z;3^3U*i(LN% zv@FpL!=C|P!`@3ZVRefp^si4}@2r=vK(1w;m$#Uj+-5kD-K+Oe{~5Sg)j(=O+{&K= zq$#nY5nxAPumS+$w_81Q2plG1oDgeMXcb31I?x^2d99Db)*lqnpeN+H-o~roZm~pu z4LE6Z&rFVVjalEX4rBNN;qiZsu**eNAPKBSRAuV|pvZVu)H^5p@7nme661!&r$gf6 zT9T+{epbQ_KYS0I6rbJSx#Q0d(L-fVpTf$vwK&+phD=n{g}28#y%hZV`p({n!eQt( zkQ%szr(<{9<{6~b7pD01K46S#A=zPl;MWuZ0_=8F3SdD5<=1UOn128u{vZ{aZrL494V zf2swX*}yBmfqxtN7C_dP-Z^>t@#hM%1p-0Q06 zwyUc!i$f;?Wt<@^&p=q?S}3tTto*aL=YuAc4*~!w4MStldxg zWDj6E zc@Am(260Cnewx1eN1#8jD?c9u3rf6hxkg7D2DO9hVKBb8z#a8-BSJid{6^s z8$4^wxgYsZn&9ys_NB-Ur!()Tk^rCA0uBqS)mzCWrtKII5THVW$d_Yd}Zh z;68u=SP=^^)>@Stf^Kott4<>5mKN4t#Hvvqe-wUgzu{sZE5L~&6mu8$?z)Yu+EK+H zC?T*f-Tu1$&e)1v#BmE@T#{;Hs2r^_#t{NV5`Y$NS9gbygk}45I+r0={JjMr!qyuJ z7ih0WF+{yuh`k}{fIq({lmUa29;(z%JL$0BTNf3Cm6w2xf-*ta8!}`HV$@5-f7;gI z!U=u$>LzIxA*E<|#5##qeQ9%~j0@dKOG_+A-7p)Sz~8J@3zzZw4I#!dzRd>p4gH1i zA;lit=N`BFrH&sA@f^17sA5ceEG zsxknJadamX3p(XZ`wI<(y+m`^>R~~1tKbav9X9wh!sz3s`jCVU#XeJv!a9mr2JDso zmf@556;RmFwhSIhIn5v^Y8n)>0~;xZwA7?lm0DmVgs@)|XX?u9&brI@x5Ngz>-R}v zUHJW0kp$HMF4|RVgL@@|I`^r^0a`==Zyt6opj?EA?jRs;AlGBp3|Q_QEv+Fdieiv_l?n&bxXS)JLz<6 zc8rc~Cmq|iZFKB(I<~!I+qP}n*4^(pAK!Dvxa0o4zjnsR+H=me*IYHLo~n8ty&y7K zC{|t&K`Nx1j2}IwpY;8NP(G1TH3AnB_zulSEq!-56e=eQuU%MZOS}0Z zoMc=sxv#k&8=IyYstP^*VV_wPkW!2JrUtX%emV3wDlY_IrS*`ou5rV(dV{97HV_gt z&ggdW^->a#UFjH2a2;4);N^je!fXUUD+wy~_$k`(y_Vot{blHo9jHzf)yTqKgSE!D zU2U-~L(1@ZI`j`atHRBjk}870u)UKc6l}+P390a@Rj=t z4*qLZINQhigcYtS21p9HBlD9&K2j;&vZpRQa&$U=$rEOw$2T++7q6QLC9r9WeZ+n@ zw+^zvn=m#K8Kx^_IUXjtbM01qor|LeE&}2eBg=O$fq;8Lh6tSA1>4brZqVm!&3($Q zQu+8?HyTlRS-ZnwG%jY+`?Zzy}#Yd$W>e9O@lF1gpiEkyBn;_HffMtb37D6#N z$#c6k2}I^V;(l=S;G>hrCk6(NCZ8>Bj5Q0VM}{Y$O*_6VQK+)5Rx$JybyKIAl?B>% ze~Iu4+@$$KVN*>Lp2dk0ArPY&B**Dg$jGPFoNc>OuS@u+V+8{^kEO`j#;#Ck z7SrKA;rHBNqCrwJve#G7TD?xr1hW>ZKd^nxfMs*)ckmV5)RCWD`62Yu}Ts;wR8F-J~M_sqT0Km6z z3P0p?`*TJNUULuU!m3CvLb232P{`oF&v6g|s;a6x_$f@drH+52ic^>?DX!5>$C>x( z_d?pqu_jxZ$RRojOvW;m|G|0VYL0Y(l}ELHR!7ggngqn7*& z<#vrZ{Sp_R_pjY(0$(TAkiuLR1cC~gi=5LN*fp~VJT9c48H_K-;!@q@3XR5yHhu7 z>D=h_N2=fty#_x6N!Qb|-==zwQKJl=rpZU0nFBStDjqPMBX|Y^3rUnP_YE2+Cq|^D zNBNC|*fLN_s7R=ChRCEb!&(vY5VUeO=HLh+IazH{5Z`a<7pl-DCil`raF*L2F$2O* z>W-J$)23XuB0zifM0OcN1+bL+Mi(7#_T;n0pa}n1jb?UX%*RjUs8Jvo6p_J*YP8{bXrlSgS6ij!Mv4~x^ zK1sR<1#OD$Zuu>&&7&80UnfFPCZ@;RJMU1aIC?x@^gLV8RR7AV!1fjJ3$L3HZOd`K2aypDw7k@#uG ztxfkjb3FWWAA_FstKBcK-d2~>FG%8WslI7~`$dF=(9od%{3yhPuy!QfMzF$!{3<3j zhX9Q}l0M|flw!wZe;Bi0YR}tZqv$-$0g6U;KgY+$w5uIZ4nwKlXP53qQKQ?VcaT-S ziv2EaKBV;|L7-*~(Zn_IIH=2UcO<(Bl!zVvHXIY;%v4@@I5UHQ<4UN7g7Wlyh)x{S z(~)NkcRa5cu*TML-D%M9u@m#FJnicW^oa> zXHaAVf^t#6oh7)i(5>{}eVSV$=B7!2FeblzVeFspEEDNkE&-7sb>FAAIdkx7mFAdl?`{=xv?jr|$7| zx|ziNHU(B}T8yw2klbqh+CAb@J6hu*K7ECShv&IJLO7v3K`-|LHVBtDP{yyKI-l6- z4HfO8v1)R*L{#~v$KZ|Rt0eP@&Dg}I0liEKDushT7XW#u&=SX}%T=z`t}LQZeyNa+ zhT&UlfFi&1q*Ex1<=oyB7-#rY^TRJiTAdB+Oi7j|Gl1IJFAK@rc>)0A^lfL^>!BYT zQC$E<=Vq8uhFwT1273*sJdwAndxd7ok2b?)bfF!2J14zj>_o_3g0g|0*Ifn0FDQJ9 z|ExWb{Z$<5?lLn@(NFm+>=`C=S|Q@Y(fN@l4N^*&xmF^~k5hTBDg*7{JWCAP^y{Ns ztTxawTekD(o3i>+L-u0B-h;`EnO0TPv4%)e{0lm-GywDYhNsi;AT@tlh<0lo0f)ht zHAXGHZ+qgVi1q>ze(t=J^`{LC!Cwb`fK3S=?77k$F(IBSGP>y@eaXR04dJNTS66ke4W9s zD!x?~xt>)tg6e>SzJY`+Q1G1OczuLh^_>$vHADym|GC74e24_cr8N#1ut_9C5Otp+c{ovzF)TVlCY zgtc7mTIhn*C1h1i1GKU$C1&u9U)#~!@j>_l5>^dnolTsNM;kfLdxjJam!*a?21B78 zlC_L~ra}QO=t+q_RYuD|p6V{?`n)_6e1h~@IBAO7lRl7XU9aXumk?^$?<;(!817B2 zFVF6IGE6IA*w~aI^RtgF&ZnUPY@$-|nc#UBv#pM=F35n{X*7%-(F~+h5m? zhd+C+e?`L@Lkf+QmM<(E{@}OQrC;^qng^R5oDyZ8YQ6OCm$%p(ycj7SD$4OB^lcve zCY4nrckk}g_BI_t(1n~@!o*hO$0DgR%WhR!(_4yW8hk37RN<@I1aN>2=I(`}mM3pdfnD{-ax~@*{hvWq6B;-!*gJ{McW=jh5#Gv~ zj?_FqvZ64h?aNECHB_(2O%GE_Ny=N!9^tHwb)T0JMJ^WtkKOS}JAhy^e-#PzS4BK> zWa7%N^iawu-N^P>S=2^|`5@d=q0fC|QEfr4-?-nhEf9}5>S|x84(?{yx2gVXgVy^&#NIxAGo2Q3~#LG-Ex10L5 zjR%EARq+0~Z*YLUpDA|5+7m{&+#^Cj8h2{byQlpk^; zR)s+>85IC82R?%XUVZW#Z?U^g%R?KaPnsEUY<*#`*!P;DQ(itkms(ZEi~|4r8*CjV zYhCsi-xnqy1Tz*dXwgGe>ETBH=49*RM7s$Y5hY&Qh*eah$07?!U4kA)!w~&x5EwK& zJ4W!XU?*E*;Rz$>XLvQkS_ee7akCse(O?*)&oOfUQ2tQ*yuY8jhLgevRZJg5dQ+NKdW^M_<#Xc?W;X zK}s%lj$}^%BVC!R&cc{L!x8M@mC{LHChxS!S!#G2K0{QtCO|o$b{4?aL6~Unrbo6= zuP3|@Pb&%rnI2pmYgEM5+{D>ij<54ngUP=%T$xftV}XbLtuHA8qs|2zN@(U* zga?un5?{@LW?tK0mCUw&y^Bv5x1%X}PypW+^h=RySJ-lV{r)~jMo3~ za4(+)W{xmW?s=U@t`-g^0gz*;WPD;&%ECni25miVQBx>8J%Eba<$$LDmTZ9e@|7)F z4DcQv6JfX~E82z$)THxEo7E(ElcuHFy&A16#yCZ%bNV z0UHX~oFeeV3%piSs#=(+V<*(n0A^*-1m)>!-pa=gGS7G$A>jZ$#%vun0O~tO^06~h z8Xr;dY7jYUj=2I-?MuRbDO;(3>H55~ZGjUgQ|pKMS97yg%YnNkLE15gaZl;CTy-j} zTxcd^N$_UZ*KcpCgesTQKoLy>P)Ji$F_R$#nvnq4kVvpuos?nmW z9EIj06(8d72lS-Zkf7JVV5^$=*$PP*#B%O-3L_@><|Oj2ulCw{w)lM?f_8p|p-?8G z@VuXPiegD(SG@uLRJfV(Hg25W9Z<8_ZtfThjoa8b-ZZ>6T!hRYs~T1wP`92f{xqST z{>m?1a}ebpJDOT^fO5K;`p8;e@k+j2=9qh=OkK^%RJdl9s7YJ>T-Sjkm7|U`b z0vy?ux_wdoS5-+*ePS!r^vG~M0lh)+BvXK%U zd!}FCU3nUfs)vog>$sZ4G5k@t)ZbVnwGo<1?FvT7s0l~yw6rrZ?h3An)rzfrUVF*c zfdqtEwz8$vxo}e(XG|YR+-HT$wvK81nODTXd7#lMVPoZ^HBufv`J5kC?Oiy?$R$tu zkNC!sh!A~Clz4Pa6g8zkW}K{r5nubK^^;2Vd)p9PL=KRSN>FjTo(-KJ;JuRxUlp6frSR0U3ns<#a0 zrI`WZ+{maiZ#^A5_P4SBIn#I)zerc|0iEa(+}|hv=LN19Ao6~fbRP;W|7&IcvCb|$ zL4MRNF7P^3lK-*n|GEh~SLCk{PZvA%1^;&RH6n0fkPab!DTaT$w*P;pPQR4nc=Tl0 z-`{7mP2^&~%F2t2Bf(&4;C9*yi&G>?!pR0>k-o(TQBW>#TPrRuJ#KedZ*xaU#71yi zpPwGLXmD5$OS`Alj*fPbb&fPCul$PfzxLc$a2pihW3xr!Pr!>B8Xu=6C7oIyFK=pM zCujRysXsl^yRfOq(eI{B*Z95VAusRZVqqQ&F<5~Rb?47Nu zWh&bd1SveW6Y=uBP>47ymBmfWL<|M_`E|x_LxvFtkvLpVZ%sMn#65-+{W+MMe?|fm zclM93x86#GARl>L={ky^Y%H`|e(9BnM4`lK4T(opb1}l+00ULTi9i7ZFOmu%PU?fV<03dXlZM`+?}ixsbZ+A zCW+R%?hcG!taFZe=?Uuj@v$e1I81sB4k3eqfxZDI$H86C2KxJ(9g7;C@R)P^aM)~| zPG`1NJH7Ouras^W5?@&SlZ}x7wyiptphMxDcGFvG5kPu;USsdi#vODWZZ*oa%Kc;g<>RF(fQU2%~?9-czL&QGv zzKx8J*Jo!>S?mju?9&$TLqR~EW70+$SkGWK2q-g_%<~PP8#`b<)_=(7M}dmMpTk7@KKlJJ7@g7lN5w*E zT!Jl;3~o}Cafsbvk{XuHTm{suE}}Xd#NlAo!}h0{v#78@LZ^Io^JH=ZVvzj^YY>r z5(Ja~u~Hue4+lp@-5Q&isjTLvd`u7>y%bVCH?QQg#Dt6eO-?~0F(oBR5CY|m0Ik&d zG?9Xfvc%ksMJ=+o*Z&!Aylk;T0HM9#XmN6KaCJ1`eE`Uv{nVXHSiR3e%TwFb6d2Em zwdy4Auv8U3;&hIk=B(`VmMCF9Q{pZ}d9s?45BSbkouKsw>Yi11k^M#q}lcmpjMnep&b@^A-nIdO5h&5m^;LC~ri54V;a zzQcWt(w0+nzW|4j(9o)@s;_5bbJX{e!RoFX-9O#j#PzbX%E~N#bU?6{d$et*Q$cPZ zAt1n?UT3_xbJijwtvr+^Tbs*5*(sP8^Mn}dVzx*g6 z+}V=TP*72G%jmSzBp@{*YyOv4WEDCTbZzN@n*MUV_EZRI;_C77crFJ)$>+NE&?a&7 z=j714-7wo=bQXhk0t+1fK0odAbxh_!@N?^@2M-rHQb9xrQBHT~%YMZ3v#RfVmT$9$ zN?>N~QBI1tr>jVBB(LXn*602_s8A&HNs!Z~3^&b4-jAuZ-nfkp2h4J%ry4^5q zL%VngNmnCMC#vvq{3a^Pw;B@fE4~}AkzJO#;$WQ>R%EG)meS2KhNVj7vgA)PNipA# z;h4u zd(9?2P6Q}>z0La9E7eVal#H}kTqC;bE&h(uNzfts{lzktb4DuY7Eqy+@68Qsgs(gp zcD2$r=Gl=fBnVze*UDQMzCNcn)xr*0qZa8&?11hORxQLbLuSY^U)m2!|vsWCO{w)(iN0gdb zqD#2X=d(yM_4I5sE{}3!Pe40%r==5Xk4=ud0m6sXF?O=~_F4LGx0J@EB!urAgcjnL z<*P^lt#*zdU6w-&8d~kp z-lL}H`{mQ*Fy8R=l<#6qOpwg`P;kljuvyvR&A^M?%gFlEpD9JAH(DkdvxT8?cMwh` zu-U`lnww_KwxgZ-L*bizNv5YqcYj2ZDqv*{8>EE|ma3F%yxON%W^9P!q?Q0PCiWq! zv7(BhHF^OAt8V!nDw@0~-_Y~%%I>Y zXh^G`m6De?-g_MSL%9FAGR<&256 z0)>>By>J&bI;^UOUfsYIe0RDSF*^6+e>0mX^`9}tfB5-NoqoXZAj-U-nDaR&}U%{I0HxLLsv49}lKatvlvXHV?oqx%oWJ!8|e9*5@V> zk^52lrWUV$vbu!``>=LTXOS4-F+Mp-4-f(i)La^!q+jyg!epp0;r+2N+Z*S}OmS9M zYLJQ~21OG$7(LkbabU{+*uu`0mzHCQ!aOSHca1s_e{yiR`#dO{c)9F4XLoH9K$l_< zXUkylq86} zY7I12t58uXlb}_o>muZY;=69|6f8x+fW5fJL_&-88$8+;MO({c@WzyuJe*LmlnL6F zOmURWiX@GV({`CQWmkRmv{g4(*YJSU8PU7;PM0AaR3rD+L zG5-pJWp;B)wcAzdtOtMjc(+F3;RS)NHOjK#CIHXdTpc)Th{5RSzWNdRrr_MtEQo)axc=p*M#sxHVcRz7mbIdJ_ zfB47I!i*>Z4yeXaosky}?l`TcRh6|U`ni4KU@P_y_IHoZkGFN!@56 z;)`+EMr6{t(_&+poXk3-(p(?Uro+4&4h`vZ$mUJzYj46c0~AbgF^s&m?2%%3$qk@m zS$x*y1VL)u@6Vc_<}Dk{66R%sHNEns*v>-g51A2Rfh~7w!?;pXlJc26Jls{PIQZjK z`H8J*aB1d>e`f6v2g$TzTNI9sb*XvV{eo)-bhUhw%9oyi_Us)PE~?JVObd3tS@b5B zd(N>gN*d#Uhf0_njhakLy9A7=xquz&qxu+^H|kI%p7HQ{FbDUREe!zbY?OXN^h?9{ zJ2wJ!@Z4^$-P4)gw*Ty(Q4CbAF>*6A-yM78nX0>QWoG_Ts2FK*?M7Q|iy?^V_>Aa+Eel#7G>OJYd`Ty1#sjF&!e$QazZfV-eMY2#Ng^9^CuI z@~Di83XQv~d6_vuxb(CG($H$^J3#eRh1On<&-n##qsgiu04sLqG2I)`ZWyuv14B8g zYLn~vo25osD&7~fmGZaIc-aZbLQr()!s(z#1hBpc4Uo4_1&>|jK z=sLl3t5#bChd7GHVzA++jGA)-`-anx!&!wu=eR);QNFleq=&5m8TAvJgqzwdGTXR$ zroV)RL@MlUZ%;Q!QyY9bEYU3IDAI49eSDW;^mOvPKl2Y4)+-;2#BEq{pi%&1`EFo| z+}>=N^w4A)*WbI2@EyAx5S@Ff94((EB_#!POBvi}#Ql{H_t8m<@{ti!`E}KoIe?r? zTW#O{xvF{G`mR0{$)i?Wx0)J@2J0a?O%)`O%iC=9UW}G&ni03IdIC z^>gqRx^=*G<<;$aIafM$8$sSZv(0N^`MqrYt<>-2xcb!6hsH(@y7E_pXO;0)@Wj;kLB~dy?MfT zWLynUm^}kl>uewl?qFYnwNqILas_KiJT}gTg|&A1P~TQH7mC&Wx&I#GT!;X1I~#u$ z0fJhFvRu8VC%1`A4)d#Qo%GGwc1@@r7W-7XG^H4Pu5nRaZ!&~tX!m;8O@Iwcq>r8pdw?n^KE8VLZal%M!VVM z7DKnf(NTG&RXveRLEQrajJr-kFA;*g*x$Cj{qC6j@bh-EGbtfq6y10MV#DVU&Yv{T zLSY<8y{~u*tfezL*_&o(aqPI*&NGeobTmGnA|)a2(ebgbw!fnrvtySV6O&3r2}$XP z01LLrY_EdgBSUiDl;awk12UQIRvAxZr!wnnxJRG(UfQa)VNkf)a^8g!VfuN4F$p9>sCDQeGiBw@bE|Fo zVjsWV>`3N(y23y)w7EH%bo=}BK4}vQJk?|}lO(yD2$&|d#WS8}STcNDt3}#Dt#bXW z#$l5zQ!a{5zmfc#!Fi%10h(m%dVCGY(lfCv0Dtq&o0<}dbhNMi&-QeO{^vl>Mwi6{j2AD2f*^v{TV4D1)D$$r_0e~lHu2e=a0AYlYcZJ1-rCdC9~L|5 zbA+|I8j?F+yH$aS`O%Rx3RNYG(S~+h`BC%sr5_wLk9R4zMU#CuxC+IlgR=c^A2GH! z=&oix(gb|==uvc}%)f6_rY;Jl@y54XvV1zQO%466uBnKL?{c7u_?q-qmT&F*7BpV= zHI$6!1PfqrH0>5EHhuP%uzohl$lM3(t8~f1&GZUYW@c=Ic(Wg@LDWZY?Kxdd*<0-< z(b<#M%oOXh)6rdDR{H*2w?8}}nr`cO^LT$=dZNo%1>q812b{IV6-_vZ*9Dic?zzK( zj;eU2{c#8qN`NNFNKYRymlXzzUfD~)%oDX?)fYzpsC8LashKLbdz%;;`(^sa*m%F~ zmUjq)BEXJz{3ppy<~u>>`|9*A1NQBT+MmO}dob^Teumk;ed`rSGoC0-(a+dN1WBw;i zL?q$p5)70MLUiLqk5!ZH>YS9-(KnjpTjXQence!nwr=2O1#?SXl-Iu4Qw>ykZ*`Gi6(b51yLZF-OyN$(Om2Y>AeMw~B^SAnx*!6?) zmY?9$WC6y@?IFbfJ+xIemMfk`7J~btj#V8)1>WT-Y_1F>vd>0(K|Y zdUSruY;fr)Qri+Z)C2=vQGB&C~Hpj{{XKiZv@L z=Dj@bQBX<$=zl-ZeHgD%RXHx8O<$72Wyz<>GNXRFI+3)l%v z9$3WcUkuKckI?9@lK}XtRC+2aE&#ZO)Vhg0>)V(OoYyyO><@%UY(?x_q}2RW<4++Q zHo(#9Ymco>t{wiK(^>0;4uyE4tE1J>8co!X;bSe5y{rHM={TJ1-{tmV;GGr?vtTxz zA{U%_%<|2bN_wEAJjQ)Ix!6X>-HRb>bJ#y{-n?LTA!JK^)1C-5;4{7{9z-ACJCM{M zd#UQoLISD2;s=UP(R0W@)7_Hy|7*@j8~g$Uu8N||MYGVIAU7Kl@&%|TU8>zGjyh+( zibSd{V}3)-@QjGH)~GnhYEx0M7!ff$Qv<{-5`O>In%r)eo)BaD+6TVUtoTY4@cI=R zmQSzYt0CXuTUuP)B76ai#NRv&U~2>{@Tg;zdz-ifvC-1qa?)!E|AUdGTrq%LLV{tz znk4OSLN!_#fAHZ>pjzIXgYxqC7f)Mg3t!H!%!PG>1@)s69k7GK7O-L2%1D^XW#@DE1-d?|1R(5M`%a?ms8z71=|zw?=Il41sZs}S|J*UzoSb3 zSl}8Q5Tx{?uk*k9pXtW`%G*ty{e@wkeU$?F+wJ`Gibqf&FtrI+PRRXl2WRH`3%BCx zQ~0m({^ulJ?SFw5kAl?DNdI>5@o3<`#}TVh|L2$fYo!0T+kPSkRR1Rzz~6`azyAd6 zY@RR*egQ)6?i@eZIXO3y^yIj-1+)h>%L^ww%bIP_BmxE<$cwnDck)JaMWvNXxQ4yf zyI9@lPv|ZXJC*Al?d)DIe2eDtN?tbbZ2UbUm{N~A@T4ruKE~77?x_vsN}TZ-x|-J$ z4{<2yJJnqU>aQs$aYFcOA5K<6Z?!u8P0cR`Skm{t0eb~5=|E|@*Ss%4E)sm6)?1P} zY&Mj9=WrF?bBS3QL(9q{9N|o7HlI0ruzOG#5fs`HRHb|1?=uV6|gWuZh17tkK3hXST$#dAS zMc8a@t)M}pWO}KPc;padqsa8>^De+zdd*f!^VC{kk;Xfmq{fRV+3aWveqq{C4B^K- z9@e7MrUSDeSRhU58tn3}Uu#RKki-jDv`Z=*Y^}d7zRxbJM;+E##d+;Q@|7+gRpcgD zb`0dV!g8x>&!fzZm;02#{%8rT7h)ydZT}6KY0eI*{M8K6M=L!;TU*OUB`b+?WPLr# zr(f5V#hy*tkqzooP!Z@AT?m24U+Ga>Y;0SEgRWtAr4;|}?dcWt5BMuh_6GI_`sNM% z1FRY5YFAl80;UxGTuQQT6J|H~@pO^o+TrB>wBOYsxX`Jvv8w8;E$P?kY9D^EJFTtwh=0`S^ z;$UV(dwiDe6u?C6?{V+gaj`EQ`C55(0*jl=SK#Dh-dT-u*bJf3XsyU$^CKZ|Fj)kUl#)ynhHX6lUg{L$P*$LNZ)95uM9-1VHmP}ek5}#ocHt?Hz_Jm7 zS-F{KWns_f^KRoM@*Hnr9UO1dN>TfaF))&T|Mm+_%-P;1f_I`#PnVRGECHwoR#qat zhmaQ&)AkAh!F#ouZ9b-7>o_xcD?z~!Wz}<8)pJRmYtu|%pkBZSbR5-!`r+a3K9^Ot+1Ng5ZolsOpbRxgKU|4?^lQfcDJXi;}|~O*u=!+qhef}>Ig*q zIcmn-{fqVxN5|V#m*BEA&%oe?at)u$COS9bmf@0xL&fG>}^#hmUn)CIN9$s z0j0sU)(nwN{b98l{VuJ5ZlHUqqaBQN`uP!}Yq};(VAHk&(DQ zI1I)#AdAU$7dNS5y;4)8Y&4pr<7Rbu5kReCT~nf(G;xkGmeu05Peub7M~)B^);x7o z8aut*-z;r;zfGzmQ=*QQX&C1H(JV6bQu%(~+w{~|m@wQ`E1f28CoZ8sUeNPfJO==G zcRstmIqCV{$i-MwqseT+Aj-7aAZ;|WucSh4rDt`K6)$t%;&RUJqlMbkuPbTs=NnOk z45P{2)bXsSSRMTBl^rb|9gi!o&;H5yV6+$pTFL&mnMiEjtFV5@&F3v@{0^X#OEG5K z*?M=@9O2AHfHl2_ugl$Nod(^@S*+51r#3SA4kZJmM_YslQoYxJsrZ^FhOqNoINsXf zSSn+<+EbmMA34v=+ujZ4$>?Xh*4Em_J=mSIA|CB4d%CbaTNpVqJj~@o++s6!x&^-X zd@&G-<;C#~D+I!abC0JK&%X-^3CRugmLJZEKR;f`-`<9)W%~zpeeUaBBO($CM{-wl zgP6EnEy}#;*R+nMfSEm6kwKrPJ4$FT5zn#|sjX~E6tS6_S43P(4IeY!gN2WGPhca#_Ej|#R*JfbBdEOoc75F) zd{>cfKe^UM`=Q)yBHi3S3YeOb{z9b~%6s?ZwUoRV=yI+sh-z(PY%*G0%zclpjsx?J zz+VFMQH9YR5$W1ItnXLOe_pApyB`Qi&euTPvOT^u-*=dRzBe#hgDumUYBw9cE)*EE zsd+c175xEtm>W;p7@P{lL%AY70`b(=mS)1WAMU#r#uJtKfi?9yEXQ$#3=Gc!-$x_s zO*MxHva>&~eCcpDPQJ^a&m@E5*36YteNs8@8!NW>Xla6eQ%q!ukTLhpU7mHa%G3-y zjTHY3&-StR^p(daoG+41vJ_dYF0Y4&JDDviTa=VNb8*WP77Zpwsy0$oQqng71sxIL z5P;FGwqVNUEGsS9fV0KkNPV+xIA1`7^lEF*bo0{j@HjZN^(jTep@zr+x|j>Jn!$B; z6=eHq%9Q~fDFYV>1wig>8Vz<5F0RWw*+ew7l1rr|85V@}dHSO`g@})ra*6RH)CVp| z+2K&{o07pp`TE=HYqWTrLZPFJLa9oci9);kB|vM2E20G$s4lU_)bY_R5K4QuG&wBM zhBW~_)x=O;w9#IjMSCFN_Nl*0Eh>8L0^#v}$usr&9ArZz;E@i=){1O=e|3HQm>Ix$@MsO5wNq5*KBMBG*F-)%K z`!t|6-VV?+j!}J^OFuna@YL2a?k#y_**sy6`X~#-;iiyb!5a^gP{hz!r6#G&h`bvH zoTbb1YEFE!a8vsYWG1{9Q0b(|K*T|DypC$*<@4m7DAdYmh@O#jBm&3vd#zmq_` zb&UamqddsRk4D|jfj&vhkW{q3;I&%SLm;iWl7X#pPHyvkUaEW9_xi^?o1?S+VzzneM0PuX@^9jXh#q4zDi*w54(wztV86V$fT|LhL3j6{AaO0jg9n{=;RpV@(j(@OC~^72^|Ao)|!q_PQs;; zHn~d&z%uyH!(CiLs=Z&M6HCgQ8$`PCztq?#b<(hX0imSu@8lMsn_KuoFDq-rg36i} z;wHiq&AO@=iusamF8T7q})0mt5eOhb)>8#%UppJXb;GXbk!u z*k@G+2F^YQeR`HW^aD`MDzD_UF4stfGcyz|Ff-c=Rx{+4zo-rnVSd3+Rb{BuqW4cJ zTjZ}uh7~N&-m=i}NbJm_H}Tr&38M4&!No-7cNHnWdAP|g&MuK($M<;+au+_H`;!s()Bb_IMF9xix9EA?ba_WTxseL##WMY$SE{&02Il*;q- zT>{oAd4_gKk1gh<3UfYClrIRpiAN3$rn@=(^Ll?=x>W@a6+5Q)feL9*wv?nW)qR1M28bF} zRoR)6p472u)pmYoOA=B8{*m8=`@3R9(?>zVHMS=9h?D;vi0-P~%0nlQ3h?%Pr zbc{0O0CFuW+^4ru7XFO5xH8-}JX~@a-0RUSzR4bf2OUPi;AkZ^@iMeH_>6_=>DeO@ zR;{IvKsQ04#wmdp3`{ssGzAJs4Bhc$u^z1r<(E3gO^)!6FsFZbRLulWY2s;)Si%Ap zm6pb(R$Pz6V$-zy07}hC&kqdT0W>ealYAmvCfcV@QT-ze&6A#f$wW6nUX2`igfr`c=W)oIxh@s5`L|Wnt25dXAvU@!>sxG% zjMA@FpJq#k%%O&gb!mM2qCkQz9woS@tia~xcyoPSumUU+JDTzGYhx+ZO)0MaoB^L( zd`gdqP;)lqrD`f2L_cCe-=-7_(TB%n1$-WqdoEP03OLU_+58yNT^J1(xkrG%+YK;& z*g`liDNGVxvrjUy;0oE3J&=muHldy@VYzb%6!0R?A8;o0GG`(soK?!C*V|DI=0 zNmOcKV%(u0cLhwO1qXsN!07M2SskB5M$(v|{;4)KAE>kEXnt<@aDjopGTu)Mvp$KJ zXB)Ap(5i*--2#>x3JV7c*iLn2hJ{hCy5eIqS=fa{mrtgLX*U?4fR2zLO^%I^>Na?h zpit$4F2|l7PNw9-Pgph(H{qZpkQmtL=z!Fcuq~9piIR}?6v^N(7g&1Mu|xl{pX}J> za>3{4swu1kfj)?RczHNx@UcUN^S-(|26n*_!*`z_#RdTJ{9mg$C|46en)hyHf9C=@NU-%5mvnA> z;nM}$6z}EnCdbEE@7EZH!Cs^^tJlX^ddlM5~(N-7SGcqrq271yV~&BM0P~SVgjUU@JOi>WGNI>a6v~&psfXD^jqi{$dRSbApD!W5oZNIa?H^uSg0@I)8V4E!eSWYkOOHLd9ltu!lYc``ce1 zTZX!;&eO#O(b<&E3YdFowbRzpo|$3%+5yzZi1qnF<_Bqwfsa2zcF5b_N~|!;T(^F~ zBX)Fkhv2nRB#pINFU3_yW3&V;w2ko_E7T4Of=vM?d7|(+ih>+>W=%?)6RPDnO0QJ(R(iqs>aqf>e30le=f-vPQzD#cA`LFP^|^ zj0|pjqtyoFr`}y9EZj4(B%zr1lhb=(#o2U$9OvtPi<8YNBh8-ZvouYd<${gj;&ekk8zr!p~%mU;@nT)V~PopsL z#R)*fJ>(j{!^2qEk#Q28Ha>@e9FW7$JywfS+mkUan3vneYHq6A+uN(%EdEJq_>}dm z$%!&7J{2@NTPXNWBxfh?)8u&|xfWwgK|Fw%!Q>|@&gYTLm+>ZK$A_I#7R}lkhY}W4 zwd7=0XZSL(qerJO60tls+vT)0)$PVVsi`u%**@9ggyX=pTb0oORbpJ;Y}5@-v7Pvk z6V4TQ)Y+LAHw1R>7fVD+?Wj!Vst?1rJRfWf?Py-6*I!`(CIt^ItsNjebQn%U1v}ll z(%VeIUMvYDCl^<_c4q*QYhTp|yxC}yEvNIrS{tp3Q3E z{x*qehkD1a%&w}i+lc|`R(;G=)WsU(B>>cbVg|5V87QeZB|LKgNrv^sbaDsGmDlNM z4j4bLxSEMf#ze@zJk2Jkt8?rX9KSvsi;{+3UFm%RRtRpiSbt6%7X6AnnOeIWj=-Qt z!e483F0-4;ld9D2)IOZ=7YH0JG`pP#YJdPOVczW&_0hM;?cXe$xkHcVvd6tWn!!~Y zaj_}2&ab~n#AK@Jz@(oB4YPnWZN^s|Rn43%v;iT(FO!<+!DmU9NmywiQ>F<)!_c?9 zyL{Rtom*4^G0QdiP`FL=x*_(HS>ZuQ50_O!Xal*yci%_peLr}ph7c+!_@zUp)2{>3 z)E-0*_xEw)sWA`me+I)}aM}Ds-GvOQ%1(599sxearx7c)^5?kuV8nAxCW-A%>CYS{ zbW~JED)M-@ z#r0HAj>7c$`03iL@LP5;yHv`VW_Mo5uTurV5w=lQEg2b-y!t{^(?l~Mp8kAPD1$wo z)N>-?c`JSg0$z?OChdNn3?g$IZP{<89oh@pgra6Fy*JLG)&sOqM zXZ=rHqr?F*rt)Z`k$>po{*7too9F}ZnG<4N2o7zx{dmZ_;N6=guqa|)h6Z~vaTKwUQ|Jmsk{H3D zDEas}nW-Tls#1q>fzKcr`mWD^M<_&4Xl}fN%?j1KQK@eb2gy2o9%vH{AK+Gd^TMG=PM6O1L|Mw)mkgY!lGERxY#r|taJBx zpUY{eF&1b0bG_y-q%M?6$X&dPsPBgtyk4stPHnfDvcTu<(&?s3J@>3fbbfsunH!*p z?)yRl!C{i`=j(NJd~&?ADxE4a&uEFdQ^y?oK%;?>&j0{hIHQK`)^#}>peJSnU&e?cJ)3roe=Tq zRYY3c43EW&QEzfmQjg&hUeds2>hiMhZ)hIzYqeVSRy_;3W@>bGTC(zdnL7ta_WOrd z&)XoW(7?BSZwuO=fvF{Ywt(6QqacQ_Vd$ZAkA{Zy?QQ5g_G;zuR;tD7O(HZDQm?!G z9gC?9jX&C)u>@>brUa8V)g}AwQawrMm#3f=fO((uDe75ahTR#l8W%28x?QBSF%}Aq zm6+RR5Z~$9&grZ!uOBZ&MYDA}Ld2~`HI9<&@M_93oEN_y8trOMyj(uf_Ygq&A^{vg z#1D>1Pm#mvnl{tSDYF#)eg8o@`jMzyrAGHIn?O;gBV}$G<`+uwlfB!Qn^OohJ;NX= z-8mQ*aZs2f13iI_y+M?dlw^nFV`ZGA7x*1hOu$zBIub4AgJWLEpg2DWzNEqB^*nb= z>PrGI@GuYh)Hyt?q6Ibw3z1-_%kw#xw<9GhEo7cgLN#pwauN*jqO|Mk3hr^Y$G_`s zMm$|~ZH#lVU7>rqimalUSrL;OO4$44QMaQThsTAF>E2;JC$AvDm3N&Fc(}ze%ZfE? z38Tj#4p-^--MFCO83rg;&|*2EDyh}|9i6q}-01noc!>#7-=ezt;>kA9{UIZ5{^xl4 z+*$GnQGl~YE<5ciyWdC5KiHXlj9GY)C0iyO@lpm>uRTFU(ZhD|&5#wWb`KQtg@k{% zR#!r5+Hbi)K7FP=!TOYp1kp4S$37zFrAi1y05~jeY3$@&En#r*MnM z?UU5aI;^ZP9ov%-G3)c9qjQ_9o9pSskGAu(N0Y5Oby2SJS*TR*td~pBd>eCKRR3(d z+o0c9B#L7_zPH2B_lA{qQb0=SiBRSH@8xZ4S75W_l<~7c`rl_0VlrlGQmUHx?@?Rv z019kdDuh$~w0EF8dT1@Df%8OSR6gmlOc&CQ7r0m`;qbQ+Z(Xid5oHOdzTf4v z-IA%PxvqyNny%;|%2w_IF3Mz|#bLv#K2fkRmm?-v47`~uxA$<^w6s3k&L z{f}VS(zy(F2oGbUgU?Ujb)J4Bq)X@knZFqv=kl~B_4RQ!^`@8nUS{MrD_phr(ZcU> zNHS0|ZhL#sX_A_2-shnbygj^tN?~*hhwCWt|5WQt|H)NqAw(!d&EMP}?RY=xIa};^ z!#o#gW4tkZla*Q4BgXmLeqO86S8H|~o~GD5gp9%Z8DNlsH!J{Zgi{6`qg@rj{9F5< zOTxrNLxF$yECmCB|6pMNTM-t&t9e6_f5;CZ(2?gz73r2Jd|_C-3l znlM|W+{6P_aEIC;AHS@i3Dk-?P7RW^vvH$H37~(F_llLvS`+Dp>K#W=45uo14{eN2$ zP)He@Ue`CDgg@Vt#rH$6H$0O!a#(^vkUM>!*F(d_DkiJlK|$ds3YSY&G(3OG|E)Lt zt|1-z`Qq^Z+i}VGfW$7+EZtHyvk1nX%Iu#+Aauvk_XGLV0u~?h@ZrQEV4bkxt5CuFJr~`Yp!pEZEPN$Zs>LZmJs!@kS?P= z%*@BdPj9#c&<9o!1BFYobP9Zy-%B-g2EX5{zWvpvGgcSY8$cbhxiz+QbmwMwmS&-x z=hLcWZCdWB0z7Kr;*;v^Ydy@I zpJ^-;$cp!Uq~97R2zZ=!*rt4I~i6yj*J=GY&Xay#hUEl6q#hD6%!X zpM#Y%X_zwElfZ|AL4E=RPWkS)wTZ}vDeEQ9;hHRCO;mRPtHDr`$X_P*-nY-fiew4@ z&vTexAl~heMYu}>+G1gvumyU}S^%GxrjT~)=hk&tDAL)TEH#y)nN=-QQBfG&`Qvx^ zF488<*job84`2}xPUmvvh7BEP71&Zm;LYEkOlNUgk9u^n@?&-q0r6tuDyXm|1SY%g z+DXimZ>0}U8vcE)PtVP5@jI076^${=h^#qZ;n3BK3;ZCo_t8;O>V9k3SO%dA3I^E_ zpPdZ)zUb3z$W1g9{JHVj%cIv_DbS@!^MLwnbzT#SvA4j1LuSPWWyllU-O)W)!LmG` zcoMO8bh~|D9})(w!_@tmxfL*`MzUrsb!>%8z|zmiM2tBs&lLE4)h?6&6Z}3TA^-Df zJ~!)S08&XrBT;SJkkw)K_l09ub5dQz{o@`nw_i1o98N_oYuYX{E&Q03n}#9*hs7y@ zX;8J?Ek?KXPKdU-Lr3q+~WK?W)aBkY<33Kuvxu7JqO% z5S3r177oov(Zy0m3zA>qW&VQX$r)vyk|!TEEWG=1XDz1RMJ9&`OQgEgl?FN68-^Pd z^MkEEh^&cf_yVR##C)MIZ=0_QTKn*B?=Z ztwvi0A^<_ADP|mGwHSJ+A}JMW1eq}V`M7QO>tTp|Hvwx5kG3aFKav|J8arxHAcMm*E*WizxfpBZe9Hq>uuS4LZlzpXrqWSi0cmU`vIej2>$&J}7a}kx>esFR z)8|5O2Ye}w*1XWIUvARz-PCC@g>_{HxQ&BUQ7|w zb+<4}BiNp-l~%TEy~i*TI%DPR!Yj^6ki4sf6O6&%pHL;`b5Pi#@bfXPf{%leFy1<+ zx3S5&Ak5a`_WKZT6tuftERWw6Kvs6VKqI82^ub^7o6C(AaJ}FKfkdZtQgD&ANteH_ zN;RQdaZ>OSg{1X}pnX$svWCswM}BOF2>)8lSg79z%r;k-mf)!|Xb9^PnB_8StDhiN zL9gV;xn=nEdMzLA*oQ7A+wAbM@lZU@#w}U*0W3qH;o^P_HlGzLt?UyAWiqT7D^3ZC zgLIu3g+br7!9&}>N+f@x8ru~qR&u2d3*%e_7$F4_$Oo*ET>%>hwSf3*K2n-+by^ZC zvCB8?S(5|GMjp9<)EEj-a{6*$e1ve@Ezx?`0P!*BVu!ysXftaDSCOi-`-=0vsWGd+ z7RVD!->Y^n)Qg3nMhQKh#sizo{B#&bC{qT<%p;lfgJWAz0tBLYX{GuY0t`cF86?R6d5DZ4GRiElauT!B8Ot6 z`q)Q35Ih7!0=>3gK&iY<3K8?+P}5>aSLd}g0GX8VAA3WlK=PVVNWGJ^Bdf*HbqE| zE~2fY?UPbiDvCmP4~F4y!<5qHuKl#)C!Q&hPV}Q=uPqCqN{CI9k)Q+Cu}2! z{k(~3fcAzCy69UZ3p*(mA3F~v0}BfW7aKh0$Mp17!aPtIYhyoU?oLcjcTZAoBye6X zG=+tzi3CeGRywQM6Oyd$1hu+W+YO_KVjii)tU-nKo`>XC;OR-tKddM=B32+7AQu+d zIIIiJvY}WIN{tK(;ArcN`E&IE*(Gz6qkhn@*qzYcX6M@iWKsw>i`|RNPw%#ub>Zj3iWg@oK;D~E{xbWHu5Slo2JzvC~TC* z`@2l0%Y`N;D&AoBg{#vjb#i7**j=gxq=H)~lE6p>s5hM#btxr6s+T6Z< zcu%}mM_O+=$5}%{^2-XO53In07l7hpto{o3)qd4!fdYiU)Bz;zUN0)jjK3W9>H4hm6xNoyOzU)4aBE3e2f-|Gnab0*o`nyVR}pJ!)Cfb+nYa zk<`3tKh|Y@l>^_dFoZ~}#MH#8b`f@0J0uV}v)$OcWx2=ZouX%EUCT%g!Xs$$c|`qk z;kWrLRyfUb?cZ=f7EA!S5JQ7@)0!J1GX0m@Znl7D1Z`+;uwPJVta4hKpN-_X4m)CI z6Y?Z7@@7l)+lT;P9c=#OIxwdsc2l-IqK+gB41F-9A4x4TUuYXYf9@OOVL@ynhGsID zWTqr>>hva>#VIAq+}T1fG9K3-lpSPW2V_V;(hAZ^x-M};l-u12R3>S{t=>Z*2&pSd z^c7Z4&%1#N+Rxh^XEc$UmdlX{3J00Jy7~m7Di4{riqAL-jxw#QXRWAduock9yA+++0aq6(a^u!}p*LcBsz;sOvHyIRsP z_+h?`5-OA?KOt}Grani@pjB@g$0oFY96pS3aW>4u8P(e1c{WSO4D!IABmEJRGstxj zdt?C|2N?}Tjp}o0ymePOFT^ELk^}Ak8cznrk9WU6#%K-q(jCv{rGCK*gj^hcQU)P+ zzkQNc19+F~meb>hQVCpxNfo}(Dk|6VV; zH6KT@xG^O-7~IezG}H^w@OCK&@@uO4bwz*x4~a8A*ZY9F?3WS9)Ah)+4HnAzYazf# zVZWsRa;i~Ft2SM6n|lD~wzaCX;-P23o(XYHw+~F^5eM~qfY}KO`dl|G7^Cq^O7Hna zYdC)myz! z?*|B!2Iep@DsIXg&nj*84%0#S!r}5kl0laRb0Ns-xQqK;rP^}bz)uM@3@^NASfj>I zQ=m6FtO#M_K5h8PQ;5xap4l9ZU3H8B@@xwyDHOxL_{ zuJ7ZNMk5UX%0s|q2Py$s8*s&K=W;n(&h-sXL|YK#c8M@) z2%xQ`k3+}_caiQ@z-OgN3YJ#G>V)qi0c@0a5z$D%DlsbY6@}4D*eu5U=m8(4In0PHFHe;4tn?a z;#6m5M$oN!e>-bB6iofb7$;phFRU4P2>G&nJdriC=km^ey%7k4N1f>)WeTMI705PK zv!l$7QQ6mb4_!#$=qTuCvHUn32q;XPh6gAH0vWk<+&ZflO@jRH4vb_EJisY+q$!TB z?g~H|>DiD!E5n=G114=TFRoXv8V0U}8$HQxF@*Kok2)KhFp5K32lo5j%}c$&#bfaYFuD#Eei%Csr9zMo zXI0mFleA$wep{CUONZSrl^H>gj1lEau&kT-7|;ZcChW)ksQTT**rrZ@>L#Dn@CKsQ z9{S9pRc2Y#t$k9h_L5>&ql5)>n#-mPg*6WJ&P_4!Pww{h zeczXe@=zvdSBbeB(%CPF09d-bQGXYP$;?@;mwt8YW<#y8uNJrLvZ!PkIBCNlZ#tiI zFvkDtvU)b#jH^R$-H;QB==#r%zT@}T_Ks+%=}*ruJdzOuPNz3P1n7Scmo3$qJANz; zrNI*)(RJ*M-WT#RBOBv2!4k`3z1{!gxa{fuXZ19*Atno(lu1?cw5Q2oivu;mT&@R& zjBkQ%P>-Melo-}xP18wniX>%K0}YQI<;jt;|Sm3L&m$?zEcg@J?WN6-{- zFE*@N34tD(fBpVkLu{!5>LMg*gs<1YXA(Hzj&Od6*Y`YVkT7@l=Vy;&-UCKH$~MK_ z7#z#BWhXaMA4bPBj37r0s&;xA4uypVRPxdj7^yDz9VJ0I{J~U&C|B_WakE1*9O{@x ziw2dI6a1x?CQQcIB_~rKG}g&MjdwUB5{YkH8S`_weG##Ls3d&7Tt{!?@WUp51XbrO z;IgO1Utb;`$R5~V+(15o@C^)27?y_7B(Ie{xlkzaQWwGOlQv zP?gg2z5P%fI;(0%$F`xuSvSI_C`R2%f(Gzb85PTlAl}}KOH3CwSO-tsizTr!In2%( zqcWf7_fc(Z$uW=}E6ZeYB8zTY@&BD=c5wZYXV_wk1P|per6w|e9I>>-cFizw^@rWvOeQP2(Z7@3CPT+b~}M;3N21g&n5mY0j?W|hCSIUCx5`!rv~4t4#5ZmdKJ;v%5R7Od5f?e;e`{x#MS zuT3u|P*Mz$vES}@Gc!zA&IM6^(A_tAm`f>U&VTn=p*l@(tUI4F?D00<4V)t~U(siIZUYBg`zy=sJeyo=H;DG^ zfGY5oLuUU^~M&KVM)!KDyCOB+jl7MaC}&mLcnOsM>La zZHEf{ZaGILg@&55`TZqXV=yHOadPX=u*?P_7S?lUOBr5Vbl9 zrS>MhA;T&3x|{p86W_8=U_p|vS3Ag&Wvx1^r*{xR7O9_?*N0+Ig#cF2Ln z_P&cH6744QffKRipeSCfI5W&4mRHK zq6qp!eWh@8Do?!x4lXPq6v93{Q_F7=4Dik7xM37nfm%CyYW2nYrLg>aIX1MwoNZp( z^*adUmtik&uRGUOt&&O1C+N*Aa-f8Lno|9SyFhAmWdXO`NtH>saW~~wSyb_evzAWv zt>!x{cs}`dI;U;>lJj4m6lIZsNx4j5gPHAkiPK>pgvTIwnw8qIG!GZkl_A4 zMm1dVG_)Bp25}M7Uf8fmc6Pt=ft+6Abi*ZI&W4)SL1pubAyHy0WH4RHk|ro^^2_eA z7adex+4VV2W4Y3l*yzEl!*M%}l2fD(A6gF7`qg~!y3cz7{!P}^ZXDLM)dA3Iqf#E< z58nGIfQWPO#?eJrys0vvC(Ip2pSz&4p>~P&4|gl+rh~9B^M2wuf@5b}u7sjcM~u8G zTRfvtp;%S8-9?7}sH&Av*CqWxzr11_6DcGrb8P>z_&M+&o>p>xLRdS_g>oeKo12W@ zA(3E!DVrcjtQ z7AG9_&Uunse0Md*t~nn)oyv4xj!O%+joPD&ZVkyY=Rn%!F{XfJ1=9t>6VWs)N}%CM z=JE}ojvG-P{D%#UeAa;e2jheo(swVYw71Ortn)EFu(0q6L;l1?woY(jOekP3aQ5zJ z!yC4rJI?w?-zIRRRim-L(bUY$EQ)$>7(qnDR==aJ?l$jr>e3pz;p8C^iE@!iQ-6h}zMlcsG2tO4Q zPLpgQSw1x=DljyVHdGK>Qo(I`wqEpZm^`WiwsTArQ2nQY;17hewct68_!}mlV0GSl zBxm^FN{pRW2%XRRzvzq)-DN0^+L4FJeS`m{B6%Lp=@2Qwrr}{!D&%m7f6|#!i0Hw# zk-;Ls_`D5>Np_yu`y6r^5wlYqRKag?m5gH87DNA@&`g$i2tUS5iAyw`J|iBkB+o$7 zUXEzHZ~pt0qcdo>*>t)dt~2T=;jzEtev3!4WB?;RBdBg(^ssg7r>9vFsgGhWjdUVv z?8q6eZiGwL%lca{QXqy15N?n{RM5*t*3w`767ibicQwp482oi0KtQU2KpE8N2ulNl z3n?W$RhYv6J+8!BK3ohp9u^MnWCTxhJWLP5t;;PMvp1OE{4lekv8GO|KVB;!JB*#V z0cHsC38;{$vD}AiMtq~%Z9PpS9u8jLj1lhK#zu+ z5|t$8mE0(wC}xLXlbD-rk7<8$TUW!gd5U#1Pm%;)IebEcN(+tbJ^i!m$)rl zMntHL-{0>J;d7iJEDS~#p%yGQJbOL2^k~}{1LQ|CjHE(I&5@%m!M%_jU0Q8+quG!5 z4`(K{`MgBSO3fzP`T4v@>rV+GWzLj&WV;+47i_F4jZk5^o58cyBX3?7^>?cCV?}w( zE~(U}ZuJ9H^LD7_923#)m-?1l2o$6+cGi!0T@(e>g4VflF;JlJ1iTLXbQqxm5I4;> zBXV_#%m;CsYcbg{yt3|>s+y}Q1Av*g0gw-hATG`|rZ}x5%4^8S8EONQl^cqPpiO~x*LsU! zaj9ur?;<}{ct4xN9;Xo;0*T7~auw-f3^H{YUb}>?fl5Zd_5gYEbAfqvC@1>oPzO{V zrg=Ug3?h(ITb)7!*-VSH$JLs59X$s%`kGXnPdzl}0=G|Y70wo4$Pn)3wFb{~Tc z9}p=g2i8ucfPd8`Gnb;@ylQIETjdr}Q=a!Fi&iFmAtmzjoZJYJzQNQk$!Y_|ui ziGy?6xP)9WKFNLeCZ`ojHa3z~-N$D&do&Oj;Lv{CfPkWGYhS@Z!W2yw>=Vf2Qu5G8 z;{Ksk)X=oUx8}Q==MGK1<0tjF2V>1YHK{&w3lWl$(nAX=P`c|H^tw_c&7?P_Skat= z1Wj-cG~DQ?CiO?M_2dL7Jf!(e`dZ@bg}-+FR)gPRoDiZufsmuAn3>@cG4sSlHDc`9 z{W9Nt4B-W74?hK0-MLC5Kts4@%IhVEAzP7w$v~}yt~p)6>Wq!W_s7>Z;?Hz)IE)9; z!$hm$ul7ZO#YF`-M=jHCX{^B{-+hXy@!1uxrKe~b2tmLfIQ}KR;B%%W4Qe~KLiz*a zfs13nj1ONK1``UgQ{!<51qSr2e6{9bqPIMV1IPTMj-T1EV2kEs71g!M&?lC0EFiXY!!59ZI2u;CD6C^#u2ly+@+YxWnM zn>Woh$XKKDL5E=3^F8{XG{4EL@v=PMnf```4>C~y*&)$e^gp}WDlaOZVlGayG%8># zL9GUGN_d}&npc4?_7_2l5$6Ox2^35M1e^{JfX9%BOm$Xu^~Yn6B~B|kBgvUZj42KW zIa@SoorY%pEHM$PYtAuHXZiYeZR#u(xDu<+6A2XyHx@V}bBvMLZNFh(a3?;msF6{2!-y81xau#-0+OiUc_Whgm8S2(lpqD}9G03t z{Jyy*71NofIZ4-dA`ap zOz;Lc@EZ#{&~a$=+Q)@cIBk}qDu8*FoH<3g!c-^hAPFUz8)v~{fiD?ICdVS+ZCeRX z7nVG^k9OuRU8&sylVU$=(&~xT$SaV&W|Wzvt~!N$haC+6>G{#Da}QNSZuiFmKOC1S ztNQ2|eM2bh9OuwVF+hM2lOKk#27?eJDCFL;q0lcGb5^6lRMrs)cnN>M*ZHwFkZ#ZU zj@bq&Ao%t$Vc<(cf|PBSjY9AUsKA)Al&O3l2^j zla1P$#tsP_K{SOrsxp@HQIMg{?OBiPKUqatQ&AsTDPpxy*Slry&;VgUVh2*NVOC39 zgudV3>>l>Fr>Kh%CU#L^lg8SeDu2-!Y9?a{#zPHa8Ea7YNXg{PjT+L+KKBz}xK5X5 zL3fgJ>v)XF589{W3??a~z~+xgLu5&njoge|^cGV(Sd$LQhfZ_#2Bcp5zWz=@#^;FI z9(LHK-`xtwp^+C49MV-`* zlP4~W|8{gpu=D`8AFrT2I;^5DL}kOHGDYAlCd*~!@iYWFgf3B}2AHwRGasp^n15u% zXK3btg_5t1z_EG&n=f-rN^|yrsB0w~EsSG!K?(B!?8&4rU^5R8HqL!wQ>hn&xXp!a z&eq?KULbR>@??*S2`_w5N#YcH4EFxd(Ucp>FH^gpk)R&h%T@qXm3=wRPmw^rce0?h z#6G3)U+Q%E$SQ)=5_)Ey&n9}%JP_dPAk}jjg=OXXC8L070+QVe|BBu!n*C^SQ=~K+ z*pICfJFq0J^Z=%fMnyZZXYe=t5Do)5S9n+?CvYKq(;i|IgF0b1C5%u8HT)z9lnp#z7J9wenDH2!NK$C!GbV~NIPOmj-qDhxQ1%fBzNnxR&#%DbrIev~!2Y9c969S0pHp~&cKf#bZ=BXl88JCwa_1~xsVRtm1 zq3dv95^uGCFu9+oPLMqI{0o0WQ>xLh<6tE&bGo=(_rHrcEu&Hu+$&p`cyRIjcIW0p zA6tvupN~O|Hf46o3Sv4jq=2CcERTyh=G|1TKbfMZw53a5pP$0jLIoPgLm1`r7n6Hm z3#fR3NApc37%|bF^Qp05gHIE`!$vk@ZTu&m&0!EfyCf*yTtTG{R3!(4(~bs@|IIvE zj-y-R*h!QLRZIN=(Wb*F9!`wyGEAxAEvEOb_NDs2N~%Ue5bQOPuxfRweIjQP{DT4* zZR6eGPT}WHGRlV+N%4(>jaty>bys!Q3;g% zJ)=`V2=NnCU^>VgEd^%q&C|f)`6_6pDDUZ%oE586!A9Elu*=>HZaLdk@CxPU*Cg+u zz{6;V>$V1R-Ql+MU(g(Av6hl*(mz$E&!T1jmjzI<3PR!#kdm8Ask53X@#N@!?Qpe@ zo%U$@(fUEt5Sn!);l}kvDQqI=8Ty@!3r0%%z>6HDMJ`R<_8d!PKWsllUd7OrC-oP; zLe`$V^}$l+5y$d3x6P&OgY@OJQ8P{^)43%lgF*gbo?bIMdn}hIF>4#3^LW-)S|^6G zB(oP|Vb|hvmJma)t41M_2CgPIn$n<8?EhFNn?t87kS~jMNSTwK{#a)?%yta<0wgyy zv;Hb~>BQ2;O8ahiGTb%d%1fqfztNAzI{I^(hj@64S(R@X>qGU?eAQrhfcY28Dqx5PPWo za#^X=A_}A@7-||>Y1TIGQqn22N{J=>(9{#zYT&<{zef^P7J?i);^X^5RaE zQ$qE%{4uhXW7755z6wSS+QJACEwel%%S;b#u4#U!_&>9bBOdse z9rE13e?{r1l*r+1ETAFod0Wpg7S z78=hzNXMDFTF))^b&yec zl!Q@OoUS1Ig#xkjs-L%y?xD=gfKr4nV}C%Jo90?dsUTx{P))X zO0hl;kZS|PdZhsSHTqI9;057aqB`oI;;j~J^xgknM~nozj9h0g+1Wbz?rBwtSu_kR zTw%3?)%g_*xTbu-*RlNq)eH)*(IL27+XH3ayp`%Y()PcXEC`|U-MCR^zQViaeXg&T z#lr||fmJ5{cTWShMW@91DdjwMGk}ng|E}b}kNDr={%ddjcVhg=0HuK2aWnPQ|I_mN zf4#+>07M$N_igoy~AHSWdw6?akv{Wt4+@2x#u?>Ah1Wze0F0N+WJG!wSB|R8a z)mARn4q2UhV{=nWYw&!PNPNW7nv!yCwWV~{q{`mp8V}zfe}(ntDaJtZal-NAHXOXg zc-A5#PK#Y~M29rYjfRp2cOax=GYiK|W;8KYbCWESU1p03UAv58j8oa}U(Rkd%Wh$t z5ka5dpRAq2jhG#>6PnHJQ4GQX+-WW5p0^;Y-Fb|a5?fU393AsGTXp@N)|#Ti(*NBy zHyU7ge|7k_gpOP0jCuBoTxp5Oo+G0Fuo-W!H;8z>UZ|;?=jG`u>gwVn&0F3No|A?l zVzJh%D;fx@x!IYTs>&S0fr&Y3eFyXc%oY zY;YmlIy``whUzLo+H^HW!(K5(Mc?Er-t!F>Hvn1B(-VAzyoADDTeRCX^3R_f@mQSY z8fz6dc|BFtOKS-+R(vVR5{jI=9RCNViFQeY+-b7(&x*(;X5}w^Irt{FF8SnX_B4xW zVR}xEzVc}K*fs#Y1|ao%e14vt=qWY-*|7sl7TkP**&HWk8O;y$HIv2VFcl&T#ITtG?c@K{ORIlAm9nsd1z^===i2Ss?jlU zSZyaYBqYkV(_&`4RN#lU$HUnJTWt6P^XuYY0ah;%?td&xtIJSw8oK|_TAyc!ilBam z%vDVVe>2NQ0XDtM?^WP;L)8BmJWx8?jEbs+#jJY(zur5*L­k&$d$ACFE)ksrxp zprthg-+m!Mj^Ezqrq^Yj+MWCNkXqc`{i*#6F_YzlmY(v-O;@j2FAhQe>*HCc*+I+j zH}u0L;!b3CqweX+4Y&C!7^HW&@;#=s^rRM8dt!>3YgyE!bk9w9$;a2f7oV#{a`jpY z;^~&G_6DM@Tk5|fBc2y2Ou6`c2Xi{KmNgJ@a;55X>j1F!&9PvBufDz8Iu5TEnQ z+u2zJz`gm4mMV+K+euUN9pL^2#7xv|nT&5W6#Y90Wx;~VdTyxg>@3aCe*;(y%;)s! z8S3hLvep4kQe7bee%ooW^S@nU3b~w%V{0-O=^n-r14j1Y{LY`l8~D5U301_!SKAY} z_Yya~Ss70EN=5*l-YZurRgZc6I$+dfVs>h`-j1)aS?`jCoP7*n&!q1P^c96mSXz3@ z$-$EX;z&lkw$6!$rV|Ey)p+ag?CW~7Gdt#L$&*v%kHR4fF_oGS$dFUk3mS5#+rZ3T5G zXAB8^yBIqtnyXxjNXP@eqxF>*(7EU+nZEBA9xdt999n=M5kMJ{l9lK4I3p8yR=4#4 z)kNFd7x^cb#mUS=!@x&cHDm4+bv8c`Y8qF|r0ew@BQGm0;B$nQ)tYMAB$Gamj=25& zo+SscIIJ$u;`-A|Fe~kokVs3 z>%{#Q18|-8O-;^h*qwTKj&4qZJ!7Cas$)W&e$i7Qjfs3p2}@)Ez~9TXz9pIcoy1&i z*%C~nyQ5p{nhvKF(Nc2?ztS=w{N&^Ly5nR@?#z$5>+|yHOt8q>Vi~o}CHl#9VWwxr zv(=*rx+f?=@gQau(wk?P>$i}%w=TV%CH?|5>0od=L-t(KPx&*QxsV!Z|SpS#7r#U*EM7YDOUaY_(f&pOP4zMwd@>c1-?%l7Qo(o!?aU7|}Q zPx9u|(t1Jx5<$-&RZ<}Dvc1O)!@As@MVS_;A|i3PUv0!)8~O9ht9<}jGS${j9C%UB zzxWN^IvP3}4hCs8A%1rwKWEpDg0f~VdWsPpG3d{m9msjY zs(H?{j0MRY9M0A_>?RJdL&ZwCb+|42O@yPbaVFHO4Z$}`rRS}PhQKK^&P$fMM{;KF zOCR&?{-1qie$Rh9p=>MU-ql)s?vQZs<0Fjv)>r|?JReqIFxcJ{)fF~?90S0YfZDD; z!<9Lp1Brcc<>0q-LWIY}q)Z|vAkNRgiuM3Oj-mX-9045&RJAhktu{}J693`Wcfk9h zDsL6Hi9QxSlb}ej%=#zeWi@c>4{)Z_?)Z0Es`oar{Ne+oZvz$u8YK@;jO(4Erquq5 zl5QYvH}SBAScC2=bV|ab24+AtZ3;;{>J=A{`kKliCZ#jmSki76~qdhSknh0avv{K1;vhM zP*GlP(!@b@woH|k*KL~3I!Lc`K7M9dy&b?}qRZ1x5~RMiyx7*YLH# zzf)6>`;eH=7Yom+Yy1%c&VlOd!X=rB-kqYFo+Ih{wzZ~`7CeFEe4~v$MSKd}E@wdB ztk~O&182HUDZtEg;NX;$S}7^?^uiL0)gnqv0seL|pHh3VRt5NREzQ)tf^yRDUJz1* zh_;jxlo`RyvPy-y{gd!~L9-{v=L;LV9865tZ}-k(2y$gzuDF~`D=RBY8!MAKr`njYRZBaw+pZ{> ze6{__Wfm$=pu8RQbd^X(#g>T8OiD{$8(|l+KI2V^-jh)(#MIpMJUkR75Zi)+gkg>d zV}7^B1D-l{95sTl3%dr8<-lq2ATAj)u1B_pVFU7Q<|w7{OX7<8ZjD7&p)gMwx@R6K z^k-|V1&CEo@LoKoOqh2|ql@4zmA^2MXeMdSUi`G?m<%-zMd4WA3XJcRyF`CAvLG>z zPc?{NS_c;@^)fFSvfft~7l*{<3xp$FM>I|MfA@5XhWT|_y8r$Soiii=_H3eN$0{$X zo`8>Na-@=PjtqTuWs#3SPY+m(lxiIRcf;d{qzL0cvlWD>?_%fDDVuwTz4Lse4211*&m*gYVWpNP>sjy zgMth;Y4gYEk?{bbv>u-)6tPSmvafM`yj~_fxpPe1^_D=4T1v_iklrkdF3v*ipzN}i zaW(+@4-6bqu%^5m%@8Z~bp8H~#hf9tv3O~P%#4t*&hN`hR6GQbTQ&#!)AO$ExFa6I z?MC~(w~wk6EnNQaa-YL&siLC;x9RBQG+IDfrlic0FM7Vd8wP)!mz5O?Cg36|;O2&o zj+w{pyMNXIsIrM|p3nc{OX&Cu>{`Erb3Wz9^-+N-<2!KQv7qhU-g7YS=R`M;T zU*&;7yqMVx?CR3?@>*>`HLuf?5>I03w|Q540_)W zwB)$#$Yehbl9)HkK`eV3VOAQ@yfs5>5T;Eix#&wHG7JXTZFKlfXEH@0^P$)MuyJuc z&z(rjqz1TydgC#%FfcFxQBB~)Nna}g?3M=+dwa82$fiYXNgUEw0P1uC18zsNOPQ@U zoml7d{k_=)^a&tp3MFYVr^Ufk=`J?%hS^3^Uq=ZWS-qBwMP(n@=o231Mt0#nR7?c& zwP2OUoiOSOWwXydzWM3uuQviF8U{S!YTjS?j1!H!LaxTf2gcp^cdvyfO8MNehw1O1 z;-jZ3_2hk_MH8sk>!mL(zlHZl^8GYRm?u-{ZZg)IW#v|6dekASefK}lT;4jw7#x|VO z`+d$5dhf*1;{X(P8z5gwN^@@BCYS?$JV3h1;Id=qEPbO4mIx^?Z^+YA1F3{UQC}c_ zCu^0M&83{&)YVcIE`HLgxfKa;S9Qxo4FWv&5I_FP+8AopUMkoxF*O=jD<3gtmzWGY zWYB-FaPaVh_aSkP_lv40sBmS)B97O{n6Y)zk`6@EN4ltw&_jXzL=^yexPadkBST%B zY(RP)*uM|pzH`|0(nr0jp0lH^f#RxlHJR<(N)RGk7cFnhv8WSUa9(J5&jx= z%^L?Ie5DZ+xpZXVC@3IOnoE52yRJJl!oouvNFPOmW=K(=PZv6swuCY?8vK}W!@$Cu zB~3F>6-N^YIMQ>}`GNAXnMP$W`Rdr%pb^q-{<)c1Ux<_t3q{14r=g|I(0qPKHR)xp zhPIlq8oMEAmBHh3iV7=54I*&HTzZ&TI$c`YU={l1y=p?|1IBhyY)(<8zSlJ%+2@i8 z4FlfdC%LExs6Z8@CLG)xO%A0_XZZ0mOf!#)U&Xt`9OYQ()LSWLR=&LLGY>q_s3Jr~Lu5 zZp%6w^3n&QDJdBSSV=tFl!+FLrlm z?7#L|!5Q{-rS5;!B<3d z!1pn}7>$YA)6F+EIjM2lYdw%sv-Q&VaN!-nnG#q1v%NIs`_;{$(;k(^X`)9M5&$Ty zM3?FZaqEyEk>ex~2$4`w0(2s-f+9x672l-f!t^>ceB}(!}b3D|HIZh$4B;UTi>xdwr$(C(cQ6~bc~K|+jcr;$7aWNI<{@R`*-g- z&vVaxpFi_Ss%q6v)!K8-ImUO~f6VMY)=Qur6PZze@jNSNFL`}(_Szm?nyJyr$^x{b zEl<_QG?q9PrSGplrF_~PIZRKRI$;c9sF?jy`qU&B5t-X-5%3;9ULIYgK|~O2#;}z! zMXAAX@MHr6c=K;v&vE^sV6cKZM6<-MNDxqgT-x3DH}9F{nIQ8i>`=+;!_-aD`cPY8 zs&3EAbUxd0$v!dMQO=K z6HYn@4e_Lce8bX$aI|?_7n)kBa}O&lLxzVth8D{>Nc8kW6s|K#Yxq747V%{7V9}r1 zOawqfBB~|A;T-@X;3xQ5XTKQ`y8{ zYYkdKdnCHNQot7yjV&0rpSG9e+~$w*W|*_UNXucOOcdlJnHB3g+;S0rqzFGtHmo=| zu%5Uz6N&`{fe=!xjEx1BNYxLsQ(+jqa9;ZySv$Mo?mvEh8(?(pZ2y{?BhV)Ja5YUGc6tCMGXe#TaW zbbe&a#>2rGB}J($xY~lA3|nh7MJj|iV*Zp2f<}>zyAJ~oRm}Ufg~y~$^apWeFw%@s zhdEhA@)Zb7e0_0SDbLp4YdIDR0jDG8;aRefItcQHR-7In@4bbDm`vCA)F_ajmEybA+No6fWe#_7j{x;J(bQDFEPGTfZ9d*er;YL?IdMn>c?A% zz3CDo52MDLfXf*?Em!Lvni)wKYk$anYy_M!dcTIXHC715rbU->5hzaj*IAZNy3&^% zfr9j2_Ij5@N_ZjxN948L@M{rMLqe%!XmxtW^DX&%?~cxc$@Hfctj+2V=fEmPwu9nw zs()3WSDnNL=dX92%=Yb=ea*K(evbL1-cZWxFqUsfDW<1i9XqKCvDPuGtJ6Zi=IK@Z z93b~&wh*?Xk*SpzbX*1*Cxb@J7N?e)He5R|cNLVH_^8?B-@TaW&HAqMNKLOREy|ru zngL)}zOqwPs%ke~NKoZ+`#_Lh=IZ%)zQ(+B&%X~V5gso!zd{q$@OfUa4os9CD;W^x zV_wH|TyB2cz;balcX!_Cc}OC9>T9}6%~P`7@JeEg_tkvd_J1p#B=Zw&G(yR#_;t$X zN-#e_yFD1{@efo30&MyCrT9AWHyp^iq>yIR5b520Mi;itWZn#f2w9lmEW ztcN3%+euG-pOxCtu0gx|7 z8L!$1q{IanO&&j99+x)E%fJXOw=p71l<;ovoB3uCddznQ;sF4OZo6$onB#X978B*o z(wvr5x2@^;`c%Z<&j9(tkmA6kqVF+%Lf8RalTJASC{L^2 zPkSRyVFyJbp}iUA4g8_gVcE=kITQ`?OqSvg?Z6kdP(U_5UU5<^1<6^FI^L|fj^(tM z{MCnL>f?+GX_pA}h8}bp&!J=xVMVs$c)oH|_{90m9n){8P2&BzSYM@F#TP{T%-Rj{ z^E5iv_hC+|L>~WnhiFAI^Zhng02c!X?(nH>_VDxL-pMcASy2%#ZU={iu$CE~v;^RyLq{z63@OK_`SKSHPxH$I9o$O4OKWKIs?^waq zrk;DMDj`6A)~B&d|8wEN8i}#TuzL1ziFo$c7oXxm$@k6O@$1lM=N**z)jd63O^`df z!4QL-R^_GFZ`-vEZz;`0qhwk>DGL$lTW(jMrh`m`&9fHIk#Q=*_E35c&H2HxIj5OE zoNbhRy3Mz}qnU5;K>9G{-%dBPzJ;V%LCx5J=u|m?OMCoT^L{I~$O=%{b9~KsEUY)( zug23O(VZ^^oWQ`mK5uT;q+*7Q1G;=y_?47gZMaYOHM;VhJLW{{wU_>MzrCMVtvwo6 zBEp`G6YAzCSgOl7tgPN^YhD|M%~)!hKMzh3W^y=zeGq4m1rX5a`|RsN4f=k(NXgST z2I^t|&AhYdmPZ7r&>@O{vfc`*E=~JsB~C}Qyukw-x71IYd3ky2)`e4j_J2R7!Itqw zDr@q%S64G;^F7ZpS~SUTRl>45{$1y*eG`AOMM6?v{CK&)_;~QNdN|`7ee0hEC;W)4 zR`3KKy`S(4V*@)xc-W%gMPAui6w_HlX<(d=kDSe2+TNTW@@O0&LYYl|&Y(Ra5Y6}k zd~Wd*Bj#CqpsoGkwEqJPbehio+5MJ(p<0I-a8!czSs1Vw1(Kz^bv(KwD+0X=Q zn5D<@4i5Fhc=2+UCeHJS6J19K-y#`PN4+i-(#8Ey`Bb-7cW~_AU-5xC?l*!3F4?@^ zb+uu-bN8Kh0AUG&hPk>F#M4J)2KaPEzgXmb0xTP)G{^rTz|DfJU5_b`*%^4rr6p zz2aH#FrAtv@xJnzM>mK^zNKL_C#u$?@}}92h;YikG`P9He$Z>bAn3al#k{sU_sIRc zbe%%PnT5t;{W>bg6qETvm5~%2o)%#s+5Zbn?fM#vmDRB}v4?p?Iw~zKC%wV8bH4p- z+i54r@TuJ0^Jh^dRN07h*EQkFuV25g#c*+%w74FW5Pv63TURJ&NT2566 zBYE{4sm`P;GoTj2$9`P_I0|OJHUQXXreAQ>Zk6JBl?R7dPJxUJY8jO9!%yH-Yu$W* zMj?G>g1n?W4e9SLa6GrBX@ZP9(QSouP(&Pbh7KEvldc2I%3xG-N{U7-zO5h+B^^9! zb<$cW-<_}Ga;8R*#QGFWNZuDuPilm!+zLKDRd~2U)!DSfBS;FoqRR$lT6D6KeF>30+C8U_jlxCdwO$75P!shg_>Og9{5@rc?DZ|MuI z7?F$bZ$WoLZls9K3hopoKSA3p-g&1>>MR^enB;e^&8vfgq3h!D2$DG)7 zaMOizykC4CmhwPcqc3LL)z3b_slvdOQcLof7Hx64TwtI+nU-t~5ny~;8EBs)7@PVj|P*9{mEEG_*huH|GqE~3%_2h{j1unTtA<94&u&xMH_3X>;EiQ|&3nmz?=)`zwbO8J-CDfu zjC$@uFY{VDXOc-x$>R$}3B;Y1_gylQlA0V|^i%)jjfcv>q(+%6%BzKFh5HSviiU=~ zgJQ;hA;S=KR>ZD$(LN+@pN3#|5*B8#2O>BtO~~v0#Tk!qq+q`WiZjTXBp6TU>rC~^ zb=deoGf^)A3h*r1(F3dqH_m2igt+*aLVMOSaP1U&8Ch}imDZCRAYytX=?<|hK?)%- zzSGuQeP`VdebavSu@JE~_;!XDD?y7Rn}<95=s&8|87O$b4H1(JkfPxr;oBh=VSgA% z&fkfMc?at)+v`ttOGg}+G+}Du9Dl3uB*?Txsf^!qJ3c)f95NOXsv#0;$1jT1y}@Jrk_Jc@M@Htby4Kv03k7nvJ!tRHtSwq(9wQ>3`{nFv z6Z+^h9dH8(no=35c5@=eNBe+N!@CCAlkF18NR?vHVy{atUE`Pc{Em3llt+*L^rE_( z+r6?cpHGKU5E+J>lzg023fKVADbl^qlC*>(U4^BuaaP@MoB z4AVjzh6YVPn8a%6m;Tt(o2cD?Og=E~Qn9B|<1^ zkb-hLm7F>q*0BD*iKF_}OH-yP@4#&dK<@BoN#pnB8nsNYXZC<7KpB!_D)OQL88Nx- zILGXa`nYn1y$5(~wRih_q2!4^Sl{#ah_8VY=6ety&b9}VD(dkLf z38BL~F_&r5ykDxuODT?mC*$=RTry2gbOV*q%8ZN6Z!IH2D-Nxrw~bFAA9k`+e+a!p z-$$*cb1su8Buxa7U#%2jr|mP-WCM-Kl<}~pFC=kQ1@KI3Gpe<_F{;FB2o9LuJPpOJ zmP)YD%dA=QjeZLc|CvtcpodxraH;%N$%vV6TRte}$tV968Yz2pJWfhRa~J9(zFDpF zc4gbRv{VU)&0|1Vet216*2arT2xLgaHC0&ikDBwwyQ`oG42ml&u4nO~ zn03RbGP-9wiX+o85K?w53vSU#t~Sz+%1R?J_sK>4&w6=q#zPuzehP5%3bVf!9*GKv9p;P#@343wc?U!@ z4$`dOKwq~(%R4FqTEA-?_*abE-f|iy4zL}Cz=P?^%D(7-JaBI30cntIRzoe}yP;_M z*>itu!$!}%r1F@ye#~O-K>uLgO-xys@iQ;jy^av*SD0J6C0)cn1xJ_{vb+$A;%<7G zqi94D@un4B*=}a?g+WZSj2wGPYP_f4!2cFKMS=t5iBL{wdx2TH9Q4 zC=;!P&R!^Wh=w%hN5?VJ{*vRYLjD>s{DIv#V93xxq=5yf2C8C*elVy2Is^&RdXJ>V zP>MPyg-r5pco2nYpKN$?t6M4!N2Ch0>maVKj%_63y15lst4Qfw*;rXBer(g@@b>Hs zUj1OfSk^L}6ph8gr4TXAU*NucXPHRed^D_(1QYdh zKzdjpSpquCu};HwwLTLiy2biS^L&aV`8d62h(jsW0cLdA#dT?Sb$!dE=@*4L_NsEi z^RPMY1ExNyx?H06=4rQ6v20zb{5y0`kNa?p_P7P^%FyjAs13hdE{mnbY)9K9Ua)Q`QdM3#ge-+-AEJYXn&LBU zW-xYS@G&zfjH+so2=>~n;xO)xMo0XUv`HvUqhP7td^+!Fd|c-(Q8TT@p2c5a9D*b7 zP#J<(NDt2$1jGalf{&oQqMYK!V2BZZ9ogD25Eqa&?R3duOMp2rb=wg_qkMz=QEQ^k z9#9+-DB$%$11Sad!u*p;pb)PcPa`4cij+*AJ+BwR_5t2E5<+2m`-?O^qrF*Z>qi6w z`57H2Bg2ZQ?9|4(!fyKUL@~1`BOhgF!EdGrqdE&;pCWdHOufbLnAza0!Exd4zaGgFVQBw0- zWBW~iV;G2+<~jOyNjYhQn>lL)tnVT*;HG>#{@_%f4<~$d>Q$8Jn3z!BENfb3=LDn( zW+uz$%QFc1&_mDFYdM`fFY}1`phUVchZ2*g)BaUWs!yX2r*jD1^|x5n1-ztd+nw^< zxp{dY^lbBxxx)11qU0F$8QF;qY@T&_R|nd;oewXsK$LpQmL$e%Yu-fi+U%i0H0|li z3_&5g3!8DjCW*wkGX1rFbU2(j5@(L*z2@iwudupK-T%z$_X$@D7FGB7mR zk|w7|s^7Q#SkHQ^fE^+7?y#NeGKOfO#BWC598omhhE$OME-w>_sc2lv>xpa0-Kl^4 zmTFv5St}E=*s$yFb%u6^Ck5w({6WLNn5g;EgOfXVA|k(f}h8irCL1Fm$U` z$rigR+`~f&TLQU0sP5Z@qLOI|>Y0c@w!wRj*NtIe0B)0(msY}_HDE2)8Aj}9N9%L% z<0I%S*Tz08w2R(G&a94}nW8>it=qu{vl{9lHoV19nt_sY9>POGfDM9!ZuX|{;AMbz zHe~1Of`|;-rpZ@W5flpDke&i_8zo4I>;qWTf6Tm`;ILS^c9xKmqn2|l^5e^38y)qZ z4VtPmkFispkAbeY(+bc*V7OVN!!uS zw@w#=_=i{ceiBxug}IE-Z6R}*RUD*5*)V)9{`TLxZh!6Ap6I1Nb^=Jv-mow@IzvCM z-YM-7sqR98pil=>(a={?IE_~|5I2BG89@vtFrT>KSQf%q@pZt&4Z}+k>O>47?TNC- z9Zp3{8_+U6kUS{M>H1&-9|)w3koObBfCc09X;e4ms`>V%rllH7B8Ue{tOz6ae5Tt| zHhF$0<%&yNo{>u(C+%an__0h~%eaan(EP!wro|sb@g6T<@HuaXj0+5#fR#&Lj`U5R?V*-lvLDS(+9A)!79c`Yvrb@UC8XfWt zPE5s_D8J_gEgj4mLJ9G0fr8sydYzk?k+s9};czToo5{Abi6E)Lu9i%t3g!nS4O>MS zef;5B9xxHXQuCgCJto8pCnAGFf%kxwDckgF0DS2*449*0L4u;X0DHYiD%Qf9=}1T% zi-@<8sqdof={5sL*`$hTY=(u4?J(PQ>FL$oDG!|hypV;J!VV=HiBOQkXc=Tj#k<`S zNm4oyb){g$ow@t%d6%eaqxNO){*Q}V3+ss0Jr$_udHCY%QNP@BO&Bi<3d-nl{hvE) z+fo?feTXfZL`fr5@*}zkI9}CnpOPrkX=JG)If>TBNVz-g-xZlDC*31Wj9}woD~{HJ zh7rHOil_F}qM=~&+g*H*;t3Bg%d&-g(qfDsNz?6KvKvP`f>c#gvst9$1N4StvKGbKPp=Tkc+KYU>r3q zsF`X6JkG{jzo?H)YN}-|VW6SSG)Pr)uH&W$B2c59<~W(=x_>}2r0DIgmC&!fe3?$W z3Wrw;9Ur0T84FY;E72M9q?(G!TA$27ar!t>;zCW1P#*e+D(=%?2>tW!COHW~->!Z8 zPce*<+C3`O_tI@(48}ng88E#guC(R5YPy%EH4^$}+G^N8M1aJo|1|z>Y}sStZPEjC zls0q0FbYFva(+MRr9E|$bec=Q_~LINPh)H}MnS4&9X70Q5A*O7(hcFo1zTyXaeP4H zCrDOtXke&fPInx$=K=Sko(6ve3{rRmqWKAOmSrmB!#Fl4tDU2g7HKZa`Ff`t=J1WB z$Rz&CcX?}R?E|y}h@rlK=WpDGk-=J%!UdNoN+S3kae(#ApehZxQJ1hu(#l zeN$5#;gq(flx~#PMes-h*MI-}8Y7>^41toHXl?3+-ZgQgk%hr-{K7pT4uOif%Ci7| zDQI1Ee5@HJLf@k=Bfa3#`9rm~nGAidIC5mE8iSOd_{jbNX~XiYabFr00?UMN((-7j zW;pkwJ_0<&_Q{8pb9}lM45jdx7y0sPU$-E(nx=RBJjxi zG^lb=njTwQYM5Z;o(kBkuOfP1jVwgjhyXQ}HZRYr^Xqn=#)(- z-i;mthFSXsuw&4_Vk9p56l3mklM)_kxCGtb;){2O5r_S=)I~=eYK7c)@JX4dn9->4 z`7<%eWP72;^!7cb1QM63USv2x`B`WLI5{0JvPOCkE2KyCeSQb~^(;iBcQV=W~N>N{Hgi0$7kD!RP>)Xl)BGN`@G{bk*wr;7UPiF`Xo`L zx3@^UC9mJXrmT43!f@c8U%%pZ^!r)X+|6U@HtAY905jE>e(2sr9!fgS-9W z<72hw6`qz-C)vYa*pvi09Hs5JzRVCp2+tj&AVyv|#Bd^(x=D~byTM=%lN@;WhlOK^ zaT!iEnOIezF0d9ppCL=JIe|vZGQd}nah^$rlFuLMqrCI}9Z1XD3L=^wB^a9bqdy`*q#*j*v=4t%!T zv%pf|D-u$@BBd%VBW&2UuqT>7DOYY%*e3ue3EHC=$)|dBsu%>$_g}&5&nIqP2 z#NfM6M0B>|8HPcSXu3^}39wTjI7cNjonq0eHN7nM@H+wm4-ZA+mVMHK@*MA# z`P=LiUXI2q6L{!_eC^QxEGHLSF>JP^*sEoM1dG(+!; z{J0HFl36gmacD*So_yT+OlZySuXuSl?{x&xabPAi;8arURQXp0e3+KWrV86B=JTGrPl-1eoB^O@+*~98LRGR%j&Fq}xH+ z>*r;0@+qfn;(>m8+8nXQ@gZN8ITcIAnML@YY#>9hWyFwbk8Rv(wiYUOIhs2;(-mPb zp}`!9fOoK2|41kZ&tWix-=Xg*pQ+rbm!)8jHRUtQT1F{D#=( z?yl7!1fg54P!#VPJo0Z1$v`zuNip&SKJD_k+chxFN1qzXG#9JCNwx4XMAd>A2edu1 zD+$sq5!>Jpi!B6MDUn+{I{5XUYRtt%1AK07KL>v&+zlCRAz!iZkeV2aa&`T<7hpfW z1q$u5dUDahg8Mx6&!s!a4-(-1 zL=2XKC&{elm4dD#4mhO3Io3ku8X;6EyZ(eA68D9ItZ7k(wb1dv1&2%F_6gZ_3BwER zj8AH;G>Pq!rA*1i{z5|H&rMrM1qC&9CTs$cg~3u1WH%N0eiZV zy*i4Trhkma@V2OWuZ|>{EZn$0zSVqLA`^Z=R;_$oV&ZUPPvxNeAiS_^L88^bX-8*J zE(0B__0u6NJoBu?qrH>qXo1`}Qs=K_C0 z#VI$4*+gJNrSvv(d473D{)9!=*f%uw&!)WUkwDqoAvKL~v1La3#k39u=}ZbrN6<)F z7{!5zSV&5tc?5oYsJNZ9I@P`Ww%DFKS7V(enpqL=rRac-kxn~PsDT9nCrUqqppgY8 z&yyu1#sp?K$$z>Pfv7*m-#<#DD6kkl*4+4|YG}KS9Ss*Z8}n$d98R@gB)k=jqM;Fe z6G!D6e_Xl}H5fr!!Smae$t3%qoiL5#KO^8oaj68E5K$g+=3S(E=pr+MjQY>>Lwmy0aV4YN?TwVFIzI|hR$~g(l*pQH8VPnp| ziDRHRJ30_ElS^%y^z*3rFHhq~Hp+BN)srFJFZ3x7Gt(Yo07cE6I#osIOdD9Ai@d~O)Q z<6nWxoObi~?c!uWg|p2=PqJ9q@>$K(K-F>8T|>m{JnM3dK)3~gf_n`vAnQoY-L5x0 zRZ>ib0ZWm)Lye|@fzdC1Kij)4Nd^Wd>Vx5o0HB_KJN|A?>xI-Z>h!D1ujzQpom89@ z`go3%e%{+5QL`m1c>df`XCmPrp~cgN_{z`DYQ0y~>d9?ga)IAw72sw3m*RKGD(rfT zH50Dj?+%m3EcN>NMi(u68ir-mi-`_WjSDuRHX`s3Xd>_6TSEA>`S^pv=ALBOGs+RI zmv7idjUSSdqv#wB*VIe#`pO-+|(=j_Si4qJ&AmQSN#D>u28)+gX)5LHpk zZ{y@R6MJ$u3Vt%wDpd9=I#eCRShLil(MQiFC$scRpYHPa$XouO0X~l zzuIFL(XkZ(p|v?*Z?%acqUjX67#|P8VV`kjKyG_Y_+n3iJUvv;Fd=okH`~Ugzc%DU z$Je13kq^u_tIgH_+qtl%D4s_^-7^Z=hQOP1t!+y_>=fu_>SY}29ggNvN3fkP*;u5m z=SehIKBAsp({UySOwRDHU{+_4(_gl@O#N3lP_CZ(s0;@>TPIP zSO%%P(&9Ki%U~&@f3(Wy30Nnn(RHe16v0fmx69xr5@FJuTFP(!kSDW&IXo$HUf;V? zC)Z=-MJ*|U-IX+OTW>u;mV>ny)yt0Qs0%2Pk=<4LM%d_jfTVE3NRvjz_O;0oiGxlc zH0t1I{5!li8q@W1bvy(#DJ!Pp&Dj(LvJ*RT0c?xHODsl;wQsO~@%;dMq0BiwVuD&B zl^j>-ukF^s*#S9YvWQtzD23gH+fh7YS%;b`a}?+>gsHSh^QkPq)ZQkWSM8#ASt$89 zOM`sKPl-c|YFPHBOdAL77jX29U1fUEqd3dA3^Z{x7iN6NMf>V>h2IU6{N(by@W;{4 znZ|3fxM z`2=l&+t^Y&{g2i)H7)_->2K@nG{DR<)@I6`<=lpPaf4?}6G~Qa;0~yNLNXk1aMAF7 z7%Y)XMXiNg@51^`Ry$hnc{l_B5kzu7^PlRlGQ3T*zC`R=X!p9UiHHw|B3?Y6u|4iu zMNn-3gueCPugsJyoF{6{1caMJ{b{Ys&VIphHV@)Jri9@uSEkFc-svV0QF)PeIEXBI zL$rO4dg}k+1XM}p5oq`&gRP$|T6s%CP;=rB4LMSWA!V*q>nQx_9s0f&At7fXN1~m! zd@prpM#@ z5%VyLGw!GTv4#ajr^f;%4DNai_+QWhXwqp!0G;jHwC+D~Eufw*;7Hxw zTs;l-eu#1tZ4>PyZp;% z_#bcwfN8Qw4A>5}11k#3|1b05pEvn$1jIQ!Xfa501C+|y|F{DG{w45oJyjyW5}Tpa zB~<^qG^ssHy9Z4tl}n?ouKXpw(aRI~%ms%)r`&d$!l zeg;ijBor*SE8Dm2yV=n$~uj1oFT8)V&sPb_g6448a ztNhn?}?*YW95xMyZ!p@CptoHqR2FTTkbbg*ZlO6>c0Hv@BDUm<-iuBC_w*}Lr~ zy8N7k3aik`?6{^VDIY0z(Ch6 z19fr&GL#fzE)_c+NBrbZH|Ki3)m-xRrgTJtL94vF;jX7!&7_e4e5?31I23ZuWPcx{ z$-IN@+iXFlY8^9Wg>QxS;Ut9Sg4X}NHvTTre#jFOMYor!#=w=O?U<>Vt|#<-y0DxZ z0(PBNEBU?fMDxf9**d-^!}zfI_m>B+lWWh`){Hp$dE;+hmB)!x~smtys@$4!lNquB2sWVKXB}+t{*jVmPuEU znY!&K2nH&?pTrEg+?ah`L*yCiA_M1-r5v$n-X73SLZ7!K35mg&n36A`m~ghwWFxQ!FE1XE5>c7fwJB`&c>8(jq-6cR zH*bNe$qS$FLN$4?LZ4?h9Qw9C&&jTSGs{ADg|aki{{f0wcY#?)O+IT9%l`>a*2HNx zP=`XL`rQvoA`;IlM=7^8E)CPYm6)arctfi zTnuhjZ3QW*O_x3m4aV!L)#i5a63tgTfXfD_7nQ2k!E;NggX1fjwKEip!|Ax^7Z|FP zl$6B78-cLwv0yI|Yy;OzTq%Indf&V*=QA)DBIi3sH$G3m*S6W+Gh+y^NdNA7dn@d*B|i)&CQVQEy_Tol*|PZE0a!waa8C>itzIOTcx-==iDM z_}%4vtAOWkhT~VIh+$5yw)miNM$kPyJr6LSlgArLDM`<{C$&>@^@v|Xmph9=-GyAj7JTW zcJUwyiICRz_5)oyZr2Z1PJ#3iCvzboq1?vC@qa`(pdvWGshDe>Y$rL^!&P9!!~$z+ zt~=Vwd1)v*3+lgOmDC~cIA`bT3%gY2x#?nJVhj%J4;YnM&Bq}i!iAj@?(Q$3kdeHR zjbmY@=EbC_v$Le*t_(%<(37GMvIq!Nw3URM4~zI`&-C1WiJ0aA2YbuL&juJAwL31c z(GPbgFmjNP!KS7d)<+AGE)EWeNBBY6Z1nWpoSZ*^ptd7R)BSzGF{#3(iLb=#a+enX zGz#FpL{4RO!iY;MX0(^;ZEWb}@p_Es`GIr;jVWLi+wM7OX>hrs4k)S9gMFd_9uE5Kj?Bu+3QbE`bKa5lri{mbe6{uId0#bTnE1>4j{L>k)Q%Y$GS8&0>1Rc`3dkJW0O_7Dhukg zv~UC7FnSt>vGMUCPkc(vpf(Q<9vYfts04mDx0Yd-=%Qm1#hI0r*6F?E!bD?x0}x*t z7_zE)>-TpcdcC8b)hR;mSAeow8HX4pJAIf>QgX8A_UuAH0BH^$NUfI~3Owj_O=Hvp zEmMt&^0Z;&Jf(t3dVt6Osygg%|G8CjV3VE~fW~02`ko}s0DVQ*OTutjzptqYDbEMx zQ!#`B-eUU7Br={J{4dk9DrC@bOBS9akLIQaU0pZ3g~K~mKYqMCgnOQCQ(Sg@?l)(; z^cA4jmMRi@10p?N@8@&C;>69zlX%eOd0-GBuBvfAiG|m_;;#wIgXi6|*1o$N%fRr5 zsk*G}Vy+#J7ms%MITE(!V>fV29hD@kp4V_a%=02g;-P#LS5(#Aqpfg&P!K=%8AbQ| zk1v!*Jxzg$bHH?}lV9NTZAsO_qQP9jFZ7=*#9NL{mpgm?E^AQt&)<$5CWjN*^X=+f z+&sP{+m|(6k@ZD^%ae zr%;4r@WZ!uFN)`hWrqPxJMn`70x$UjbewVe2KBG$uyc=griQ@Pg{48B$M9L2=zue{nR6ndH+; z$h9TleBrs-c}vp>QizD#M^O4$Ze2hV(^(3*Q!^-0rqKB56UUurwjA$XpOikO* z$QQt0`7<28Se=KPw|?6G&f;xT&$BtNcG)Yj3ATT0u|l1`)m_f+fC0ZG1S z)%E0fyR_*QS}Jh&_wQdddVOVO<%EnB9;+d44~{n?0iTCrq3+%jSnHu7alw|iYCQqt z;wdn_{eK3#R#V$2<+S=lHJ+C_{nT#fxkLrce9J3r%*VgJQAKODp3I4bDt%2&OJj@` zs&CX@l9t6{*fYTLeYhF(^`#;s8xHAwe{dt+JJ?5q0r`#3k+x9nWdmM&&uB9XaZ>u9 zC3DZjKh^wVVQ3}4;kvTP#Z>otR$AID1LlgyHgFs?Gdm!@_;S2_5{zdf`{N+vxArtV zO4#eevCn7yGw^;tZ<5KAK*NZ?D&cdTU2P|xQG_HA)x|?SO5lt!p%3x0jL ztw!e=cp^Kc!W6ON8I^p)!T?UYmB&#lA_Hhb9;2H^-~R&^9FG*!8f{b^gw+W_2g&JQ%mtBiPw)w_1-UcLNk- zNCZ5=oQ|Ef7k?j@f`UWxsv)3%_L15^>^i?e7JXhDhXUSScVMUr3}Z!p_tQfW-_%mC zw?f~=ijn9)AC$JXE&;rQ=C=>X2J@KVENRw8Br=X&fg0tvJol}wYe1k3c6za?=PeZ} za1aM17txRl@890vf?Y8!T5H`u90I}F7J{x5xe+AJ!>+F~ep)|2y!wFs#8`KLyv4q^ zQ3fp}SHRU2m${R#Zy>g8(xzO<(D#gLngZ3)g+l2Y3{cJb%7Q6HXN zH6eHHW4-5r^_tIC=0y7ss~;?OB_(D@#or)3mVnuETYD?)q0_{Brs(!ufB!k)_S)Sk z_-u9LkRzdIsGVzRYilYm-*~YJ=2D~Jb$tM6@au#QEJQvXAA4m^DBHadJYpD2>;8kh z2q6=4sR=F@yh&LJF+x?X#=u=UzZ0Ni<^ zVvb<9_pncZ<6Qq`r}-ohLMZh9hKKtMI3i1x@LsQ}@Qz4`V{M)<1{)+}v7Z)RY@}kc zhdzyFbmxd#?DPQq5?|+b76}?EDzoDNF@Jyev&&L)kjd<{;nVP%qEDBRzVl~Jh0tpT zs%E2>^HO7Ju2u+Ro9`zJ;EsM`i_ytCHJ30Tw^mJM%)cWI$@GM7)?}lcUJ%mi%<*$k zoSfXw7a`zv?ulFT^0f{^+c--9XQqK+Xa$1bb+seJ-2F|~nDl4qf1VQUBY_rx{VC|; z8_Q{v?Pl}!>G_|3tZUbgx9;|XqV&}=BV}!zMs*-XdPy5Ki*^#YluSZssBeRkd zQWWQMJE*yx7|mKOtAtOcWM_BVzBd6Zu&+igiiC>YEoTKYGNwb)6MKXiPi&<{Ei=oLm4}Fh#2irvj^}HMcr%&MxUu#k%(wOI(gbanaF0~YGbDEpEY)_VX zAe}Run4n}jUC`mSTD-n_gIbcI{ru$)By$*W)9~z$uZ#^S^7*{MBT>W;1*KdZ1wA6E(vk&XPT>T;y<}fN{m&}10Q! z7*UO$QfLvf!N3B3W_iiZqNHTdmT`ycXWh5bsas4oK1QgklGcm0&=4rabbiOj4YEt` zmXy%6u8JQ&KJG&GYrPuGH&JjSUuNrn|KSNgE;t>N2gBS>EccG#O-E)XNDr1%bv!cs8QNbR>er|Sj*z0t!6;@qKeZAx5hA{w76s8lz@CDsG*yuENcfFYp z23Vxq991jr?8P&ATiCL}J|2=63j^pm`4LG;$qjR}n^W2c!Gb@^OGDcW1JOnkGAk-V z+v#2z5FqH+`>;)FblyK#_>g+aW9Tj`9wU@Rh+)b7k);bBX+3I6u(7PEVzje%tm{X3 zSFF3{|28Vr#CY0a%U^cFibI1YD2x1h)SG_HIUnI0`*ahHL2q-Rl%j>0;d*n-6V@6H(^atyx;!SK38i$PP=rG-@0l@kT_Ag}BDl zge{zSZkjzUR*=xPKV-njti83pJ}5=m7fU@A6Y>09Q!g{i3v{!~OLTw_QDbq1G$jlZ z;`#L8q+E40W`ubCo}Yvzn}VD=F3dv_OS{vtt1ysEjUh4uwrM!`a_?+@{yUXQk1&N% zdVel)z%!hkNhp_#5!L$pPVsjz(lBqov%$zR`8j(%^c+2pP`z)rc+@>>Kxm>7w4~r; za;UK}X`+(8xZv-+#~8kL#pz{5%BsRrmH&NSbVUR;1=+-G8i;{g(yWM$K95|E9bLh( zDiF(V!EwEg*mt5*#D+!j^3a$8GGwdP^X%5pjAT3LO8+9epKfh&`<&VLQC`EZ3G2l5 zM)a?uoP|$#8r1C%XhnSsGcGA9(joI|t1LkErKfr^_=V5sQ_@k8&|GD$@q|_1vJ3NS zl5Gyq4rltLVsi?&`cYXkbl&z?Yh@cUSXdCkg_D1*J!e=g_ot zR;Z;hhCCNQbpjN&{~ulN994O=#|>w@CfjyRHYeBQCfl}c+s0&0w(U-~&B;CI>Ur0_ z&wBsvtaJ8R{m$O|`{B#!D}lQ9_q`TnWQdueRY+JY;U3nUE&_;iVdb|Nis?_~!K4v{ zaC#a#vk+!o6jA9nyp`kij2l7e!Q|^1Cp~&WfI7O3 zP8X)3`dvj+Q(JKl$VzlrM)8!e?jaqXoc7H2!ovCP-7X-`0AnoImaV0x)i*2x_^2AD zr)8Y0M&g$#vtw6hBdN%lHrIx-aLXtwIqdA*SQh9`=5l@7AZ^cLsPg&nh7A}dWK*#F zhzPF7n>DY+hPPSQoliYx4)STW8Tl+mH>+5Dz8yH8%$S^>6nr_&Za zmD-Smi=Um4SBY(*4b_=01p-4h-_IZ86tejg!hL5z1edk&#z^CW2}g0D(RHPV1U#l`W!%Z0o!pgn2p$I+! zTwu_=oDNROwbG%Y>EfNncI9>vKIRtm43NL@cy;UqY+^x&sTOX(d^$Q}rgaPS|}7FI8+{jgkK=0;Q7EV$RyAeuE>Vbt)soKl#d zZ>R%}{P6=8YWV`r%0^fMNtP7gwHyr#v_VV8QY3w+dmElSMQ0Jw(4dn;bw{V8HQ8*w zjB0L=8a2-0vE^YlMiZvQ)a_LZ%gQaC_zsqpHB|)@LM^Uh65~4}QKgZ>lW*tg$w}7) z>k!u88=xd6DXM4|>V+SH;_Ql<(DW01D@K)!Vrme{01)|n3Cp!|@9GZjfNlE?k3k&uXk9_|+=Dc6Z9`>2)<0$xur3ptc*neok9j85Lc7gv~HWOD?K}{qq2nwi$ZK zom?OMIO0m-BU<@~5f zI5XbwseSv$Wnj+G8xuD|UnQCJR%RpCCss#w#APo0yM0?-$5P7knx!44lh|0xRP4OG z>P#{0^tNU;e6G9@Hr5#Y9qs=^khB)TP=h~n?ku@(Wbj*|b&!^nm`O;48dXtJMWqB} z%3t-4OSJqzfY}?BK{mi8(iMA)S?`7o1}>Y2vDYZ37NvU#s}^**wvm8+MqKnfZ7_tJmqY z!a8K}9UB?{cDz9|R_XEY--0UxAJ5%cTWgdKoH3J(c#~S1{+`v{47q~(wER%;w%_gT zDNCG~F&-|H*jD%c%3NGjJgm`|wVJ1qd1xw;Ibl)|UK$+lOKjm(YqrvS29k&4k-vL| zzJ%*PZYP0PJ11u3Cxk~}?&wUw%p?{9A%@&6(;LYgBOfTJbP5E1#{I`N|A^YI;M>qP zG7+~2x?g_o_8DRr_zT2~zHviPeOelC8OvN%6^F?RT?VpObVFUQWSl&}YX>{*ToG?l z{Jc_Q3|_s;R6d&(Qzr2Rqe?jS#BqatkXvPiR%MSuE6bbD5Ka6ZU1ibYf>#?RYKE<;M(rbsW6)C;|-%)r7z zG*iFv@xe?3&X2YFb7o0&V7uSNx?l2%RjgcL03iA8g1g=VQ{Br^1e`#I$L+-aPE^JW z2x{hZ*oQd9snO@{pY3(JZ7ZGZBbxFvgQDUqOpzLta&qNnCN)uqaMR)D>2g~1Q^)R@ zCb`D?i~P=b2SUny3Tz)Ofg~4imP(tB)RCC;Zw9OIvTH^=ZDz%H*}B*~vO#$zo};)qH9$F8@*c>6Siuqc6*83DPE2E#Wk#vRxKa`0cOPzLolShxYE< z<+3!!vb~KURuql+ zro_XX9p4Ayaw7n1RaH9m8!LS^zZ+f~5j(ifpAWAE)Kz~4RmRI@G3WjG5puiXxVr9? ze?C0r1q>Gd+9&c!awWO0^?m!{>6v;ZGLqk)7LGWft4K=tRBEr@a`flP?s`_YH)L}H zQ%BM3J%J)*sN}k<=k_53-jnl42Z(*~xlJE@{ms)Gc|U@nK;e z{@YRhmI5oW7j!Os5V%8{7Fk>L<1A!tz>^97@%I_Z`4sN^;%GEwN3-N^5+aPgN9Zp2udsqJ22rZ~WWWJmsN>z%W>PGs7^qiM@6m znWm)K9pCjrAjPq10r4jby9Ncoxe*cOX272tIOX!56xvYb@hB+PH%Ta-c!fZGoc$chVb&4VB4C%%2Twfdjdi;70Wy}b9yw8yRd{u^XTEYky0Se|)-t=2*GBg&} zf1=1dr9|jU-(K$1>Z|lwd1wpW>~?Q4C92pzE-r7Xa(UDGT~`}0XZyyuO&8;RKl+)6 zl@}W1Df0$23^_rs>3A9H+tb-~T04qJ2kO1Kw!`<FM5OC(5c%oO zcR-Bs<2)Hp&{dfo#rRlj-z!lI259>I3BK*Bv)n|1)o;F=wEJb=%6&B4nXmCaOrhr3diNcqp14c zEpkT>$L|QmE=`TL0@>9%xgPyke_E85_|#P4s=+8GzEE|Kg(Dx*?m2mCJU zf#Z&N11<>H8-qNCe!3qrbklh~jxUzDD&p~?@)}XGW^eN)m0HnFnjsgL;9jpPKJA2W zU32+`1DqR~ss#@T1N*nMPOt2vvl}0;Au5veemiFiWmq1ffu0R>1ZYE`M#MsMJZCq( z()uctF{P#Wa#xiK zr5hbsAKgZ08+8w91w%%LO4G}x-}73ir1r`FGuIz)Z0FxEhQC+7G~22h68C$E3qHqT z)rI&zZTn44&>e)vU-|)E{G@mu6$E0tomhvLE>aK5k1G} zimvPZLyqUX!Re(ES&@Df$37N6RF-E_$&AtO0FM5la5Dhern1yb)Yki%C zSovTnF|;#TA+hNgF#g(l7(&-l7~z5VoOR9|`EHwL>vaz+Jm>kJn1>ZI|DSl{pG`%+ zB0@O5zwBa$OS7t=tRqx77K&|(t!e_3P%kq=GP9`%H2W@A9%|JE4J)dLk~vciWkbJB z*-Edw5nkb;K3#A7W|~YUOKQ+mV*VxKs<#;twxOa@8R(TNSikP5w8X?YXLZ11(~_pN z5s%VwSrMfx(WcpZ!fDj4dHjQkGjN#YNb#}nq7AeiylxAAaP}Q!Qx8M^s*?9R&zYId zm7Bv-pSbu%4Ri>mTsUqGrUTsV$eh(q@G2_2{`bBS?GG1l#j^um_%=K!2E6nP9F1-6 zXW1M=LK9F`AP{k;x6Up$O;%RmTzp&XqNJ&$@Ct)^j5!&&Q@^lrZ7^XXm1m<}d6Kp@ z5&Nh_Y4w634zpHf;Oi~uv(6j7K*JN>f17Gc7zw<&x-@jFRr^i;-=7ak5b#C}p@q}y zY32MH%29a1EoRIb=~c$I9uIn_xctOFoVVEZRBO*Vz=em)&7(A4%@%psua4KMB8>sI z83$6oAVoI4g}F;LUYIfZ#Ysiw;#Uz__}}PX@22pM0JFa;jPptsu1SthR>vm$5>Efa z;U|)2aoyz!b+l!hX3dj|wO$otNH4#b&<|ezD0t2bf(wGnL0gS8<^MGG{O1sJhW`f) zz<_<7{Rpq^mfWg)*`X4tn+=>UoWuPS$B;;rVgE-9)*kBb`09fAJ>`EMqyMPD2$><0 zLErs3(hb%B`#Jm13xIFH>faHXHD2=H*!zDJ3CP&MCHeN^%GUo&PW?Zhj{){e^dHab zfZXB#^KJgAItzeC076uP`iP}~{~w)O4M{%2PR(wqBh7!G=>Hb2^@9Ing+z5wDi{BA zR)$<@*3 zuO{1|X>fh~%$4c!KT}&|W=>96(**cyKSxLh$ekP55 z8DX8Qr$bhRc4t-6@fTY4^6>Dmue)rVS|5xUvX*zWwsX0vbZFD>33!Zxgz6I?znU*x zMq4ao8QRavv}J+g8Xo=VLFZSHaxts)u*>MlY01yx;_Xn}o>qHtg=^+#n(9+8dMSmf zyIfsUpb-#}mj_b(>&IIEcJP?Oz|fse({h`NX9_4n)zOf0eoCy|JU%`>J>5M#+}zmz zwRih_^RScq^(?``$uVAw;WrnU#+b`(u~+`^Q1$yPv!eogRRb&0fNgMOi0^YWF*9>* zV*{)k07Er-KlH1?UScnqg+_he%JOW@DEGqW^}+M$bjh}c(c)CIB@NzLKRjV}cGldh zrXxccyzjK2#Hmak#|Nm4DQ#)tN4cU@-&DnHK3{DdSY^t>8Pe-%iRgfEuM6IrHT(u_f?_X=p7rZ2?2r0I1+Ytcff8+imK-7cb=t3>?R;!HwMNfV1K;} zdMqdBmi@905td58!D?`6Bm)mGbAJakDIc!Ww+~$0gg)|Gih%_Q+1eUNOKSqF`=k}Z z9&hZmx%^(6D)w0$ZIjc~)5%ZbDN_Emt87TZY(PWR0k{81zI^HjIK0CX;9wNy(b}=AWgZ9dcz6Gp%VMK5=`S_klrMAMI565{Ipn?ZspsMOcEHL;g_?c zb0&wAh?tn&$AS^O2^tx=-`B*Kc9VT(W##!wPs@{^@8@N!FiXop*Re(afT-D2`YauJ z{PhaI*>EiyXtWsJ1;jkvNSfA$GnuDUakC?-|>Vp_dj>X-Y*}B?9XfU zTckx4o6^$KqxXP)}^ww=W-h8CHF$1xt098r7Z(w>(&Lu9xm`MSy4KJzs zBU7b*cUxOa%TtQFppZHvEn|{wCPtaraod3+W&A}%SYVlHYx1LI*5>rc2pcEm*<=Pr zk_Jq}4tg%11BMf{-^)60Ny(v5*~+HW;9gf%C@OSJl&L192U7t$pH+CqVr-aDQ3*z_ zy*+E7Xu+B&>ZWPEWy32*AEL22MfIjFM{0g~LKwza6Q`XAogSiG_^(d;S0=xYAL8N^ zuubfLsyKvy>WGNQ>hGpVDBA-Fz4lU$as$D>L{+ewc-o~zX=!}dx|q*e$#Lxm5&bxp z*$rlKmpV7sS(*IjWKxYYY>CbH)O%he%ho?k8>73IEvcSa!g5G$H%8y);=VL~X-7p% zf4V?5(PG!z4CJz8yznOrCsq`)6rtaUd_5fd)s$C}hl=3#7Ox0fF$^`G8sI%}+O4~t zJq%gltPbyrFv%Cs;=)hI%-*J>xtDTusJT_&c}k4=%Tk8VYBY6FG7Ph0UW+@fX;%6t ze;*5#-oMLDmP)3)W@sotU)+jZ$(+?P?l$2gkJ7xfMSE|v|78FC{7Cux+scw=4Pf=4 zOuxZ?ZYT{N(F;N5P%Zz5Tyv&|WRdjp71#!>*vpIcz4hw_ULBiRR1~}otG;&XL5yU5 zlw4OW9T^t(d5GytNn0EGSmJ~nc})*A_BMq+K^;tk<@20gNcd;)cA30h{>gCY@a_Qg zd99r78af`x?SuV3dW+L>LHp}#H)@yB>t7?v3G%ngcMQjhZ!;QXnYxv^3HmaZYp6g= zNs<@qcs;y*ISk}jT(C4NBG2wDXG@24=96#BP3a>SZ0n$oLM;V~vK%z(!;7C4+2-Pz zTCVygx6Z25bNT2mAX&BgsNdKoMkQXu64cL9BPI<9RF?E1lm>gWV^GlzRp%>hWudN-}F3<+f&6jJsQ$pmjXR8;TXV!u%1AR5u;PO|oTV6$v1J6!i6xfvTkF z^1yyWQe>Ad=)msoPj*`u7da{W{JcCjQBf5o2b$6<`oJD8?Z7~=PQJ~8=%l*xa)?V6 zG_zCc7*>b=QGqT|n1vD&bAwW)ALHX2@iv< z{XN6FO5)<*IOa3loScR_kch)sEVB-N9P3fCy~wx%bA#voNow`cu+PtNfsgl2Bn?_U zU$qJBo*X$X7xnD+Hncn(+?qc)B+vx#Md(Y9Xl`zfGMT1+VsbmXG7}~>{rN%GKj5XJ zZW$b|4=Y3Yn0&bzj{pjTbS3>&_(q%|R6zA~cbD%hqN*9)>qp66G7j6KPmQ=T-{2U@ zY2lNTIfh7FhXIBT2B*2|;cW0~#P34SNJ5blv>iNxnRc^4*hicX0$TtZUP%=IB%Y^9 znHc0D63B{ReUvz$!-z>zbpdL?D#N-FQQ~jquU-quhu^+^`tqN&J=hK^V2{Ah@daoK7FX(?j`)lxrIZ)cEwh*P@}#_$6`TPFmQq6`}_DP_H@E^B$}5iyCsxS z0F^pEPJ3#b@@;9>uG~y6th_^iYk;guVf5KyfOv_i(5+cR#xU&XPt_$UIE+C6Qp-lK zgs?CwrU1^Nc4kBk?x4`w{_3K8qb?q`KH*rgPDW_B)vFd=_MPU0-YW1%^ns0n?HkZl z+tI%1ClepXr*^kJ}OH!;bIH9iLM;d=L4@4*` za$i300T#vzn~yb07LtcQh7o-fmr!kMtE}tyj41V9DQ9IN1;1A;v>`60^)Ar{au5hZ zd$HOFj5%$86-5m(vNO4&u}*CipYeg5xlm6MfUJ84y5ua5S%ROM?A#kj2D`DFgTool zY!I@VkEoACv2)pxM#G68gVK-V+SqjCO^g^9SwTAu%?m zl#(hL8JJzW|C~beEOeK0IPd}>lL1cG^*k(O_pgFqkD#xlYakIX) z$=qXoTLlP602BfPhk8D@)qy};^qKW`xWF^J1Ond1;zo92VrD|dz+(5$7ey0&WS4@L z?%2p+2_a}#akbiiV)m-2O^w~|?e*CGa3{T640!p>`PLh8*=QTCwXdrt+7bla6~9g+_7(EF zunE4Qes>2MX8xwrV{n{MRA#JVs3DL4-Fv!f45%wL&yc2?Da`kWy%;NGDJ+%835tmn z2^#mh>OYjraxE)0OAMWuSuuNQ;7-$2Uf<}n*CoYN@)6x{e_UU&0rX`6N4?!a z$Ha`(j!^6H=J44x<7F%=D4K;-h zL)m9B`$C+*Qp#h;G7G*uJ1z>iuXi1f0zDn&6$g69wFxG*#IWCJTrr?rP{oa_KTRw=bfBi z13eB8!`kOaggl_nwpTR^=v2`-DgHRp3a^*fy8Ldta7l290`Eb9;s`QL2q`+^@QR8I z?UqYHpCd4|@WM%_vmq`v(+3o>SSl>WQr`yK^Fz8*MHE9|Xaq1di}a;_zOi<@H90X9 zWFs?ap>pmiG@2cBuwG_UXHz0JTF~?Tr}BJT$bE>u-L{MQrrA19`0QFnc_y!!qlkA^ zT$-7tIK|Bsz=ir)f^e4Q1oZ=06j=-im_tBng1D=5c?&2SNz8~3RDH!=;9qoG9DVNI z_0iGEVbbk<9c#A%rq`S#78LYVr4Z8!2>23n&cn~}|6qJw44rYiSarkLv(aq1iPGx* z8vk~Wwg+V>Ul-}PklH0JZI8GY@lJnlhvdtSTXuc)6NovfefbXO|vVvO`ml&k1qb&>&-^ufMX2L*(Q%cx!SgeKz4Md??3WlYCG*iVTFA}re660J8 z)5%?Z{Tu`wVr-1*&E(`bFf|eyWPX)n&ILb_IeckuZq6qoe`1Ddqvd`lAV@k||H!_X zg1$;6U}lNzKUn~!dI^)QeJsA+x=lX~4Ca9R9#D>?G^R_aWqHJb$(Sxc?n}H>ER&j= zY_siQzmUrBEKSDYKZJu%uxWRA*QWnAN2Sf|Z-CrLw2Ce0iv7gB{pxhg#HNTYIZVg+ zd%#^}U~YRA$$v|jW{<4+AC2e#n5SE9fDqn>3k;A^?xFmiNV;&$^$l2l4s3{v&WVsY zyL4N*S52~;GZ~4F97!JFgF!rf*@KTG;Pp)foDPl^!G4Odop17*KSs87xDs`_UcPQ{ zC%sBa7MGeG9%IpG3#QOhB{88mf47?^uSzQ#kuO-j35bbA@N=FjR9LwdnrhIc1Kj-m z>-CU8kwh5{X#8vb^vLJD8r+KQEcO6(*UyK98oqC%(WI|so+reKyOfT#?K59jG?|vG z6VTB8rH{4>G#_CyWe4oiR*O9gA4bMTOLe5WyiS(fV=0GHB6waT3@N(bU(PrW!7Hb! z)OOu(h29@C21NxYux~JD*?&k7YWuVvH|+Shy4LtBkRK9FU@_GLMq&vfuUcK+ z1fpH}aAYfrNDOog(6&2$*{JJA3LztaG+vwi?O8(67`5&K`UtReX8_P{Hmze^Kn!n~ z=l7&^)@}FYfI+|9+~BXcFa`nD$}^Xq#*h<|>GGr#*^~EuoNnjqFVk0UN)EVo`7=Ne zsMwTP08%L%9beep1V0&39U^_Mj@s?}qA8+Efw}f*jmt|*8A~hXA`s${?dt`bocf&k zu0=->LuE00wFaSZnfqYKk4PWA#Qj;iBzw=9=tE3s1LTjj|zO0S^P>*P>*`Hd>wk(&DMw~;lA0vSl@uXpw3 zos;3jFl~AYQy#$SbZBD8!~8Z|Lxar34~vifLNqWHgDhHROkMnp#Z%`Q>2E1dFKz_%jJD zfo?P_)c9Kr@2tVwuKd$jV=uWL*xNTf+1c$m)Cv0W_G@OkB&bS*lZrSpy07$OHiB$HkX)(VF8>|5taXF-M41!CXz|P3WU0vl zl}gsi4f*>Kyg#GES^DUxr0?s!BUf3`E@m#P1I9gMIHCkD)a7h&K&ASW-{YeK555cy z*qm|D!1t~9WCNV4rz>9HLrbVF3*3-Di#ZldmXT2@8lfs?jI6D<_*m?dWF)c{kLP&9 zqQyu95!@rqaZD!3&*$6QkOpRwBV*{2f-R<~qs)mr$7Z$(GthruAO{g$>;vY@6Y96$ zU(s)zo~2mHjz>`)fah?;_UEUIgl{lhP$qY#U)R&^Y9q}}htmp)DO&`=iGNK@ZPsOa zi~HxQr0v1c<(gBttaV!$wKDJ3De9-MfZod;E$J{aKA&)!0i6Eh_>1@3=R?|`>KWeA zy9oZ>BGI1Sasl^ehy_13vuGe*@V?6QV|6=rH8_A}j;Z^Gni!woGm-|DQWcnpZ#3y- zv)SD#XEjb|}<)`9c)d_HR==t2+ci;R=#5Zjp!`*K9vXQ|?$qetbF9G^^ zwnP^X1CO2|#;0MyY%g@crXVPb0j}2zY;_N3n;E(?0p{w2ctDlp2Jn~45)xo8pa1Fc zVqiPxA>z(|dyTS+I0g6-M?@eSC#I-DECRy2ew6;3Q&9sl6m{S1ZO^}DqAX+C@?TV@ zr%RdUug5R!z1oZA8=07FKA*<*yqub?zpyu?Pe(WGQ9;L`3%b}BQFp~S)DOx1IPX;R8PpwXWQRu z0PJx3T+rJhEk*ryBiQ|R(F^dMpWQ)6x(70bLaGrFr{C@Mi)|h64ofZF*?>Zf7B@dM zS1f_!o%r5s@AHZBp5(ySQN0Vy3uwOeR#tl{q!Xf-MxvfzBNR6xTg;rU=;-qm_L<@u zcA?IyPOuryOtG(W7%)S@Ph2x{_VY9X9$TfV|HrKOP=3xKB|;5Z)X(z5fvd{Hhn%F zEeS7FRyqFN&bV>o}R??=zqTAhQNiz){kQK!rw5QZULO8=+Nuq-Csc9EDt z&b_M|v1k=H9px;}>%J=ymx0Thxe;w$8ULoXTv;v4=RzrJI&Wo~;m>AuV1+U=;2Q0?o&m+ID6rh~W>8a^7=`##l{S`@$L}h6C!Y|zS3giMHeIM4^s)Mn;J_&PQPMYvB1sAq5M+nCzUyEU!X+kr(M4n25E>9!ho6(k0&DLx>H_3o zINC4-Sn<5{nIZ{E;LvgVRl`pMdTyu6XTSzRk)O2YH-e03^0-WPd;GsNm#5h=6%;c) zqRmA?wWChcf%fY?IKuCqQtQ7t_Ju^(8L6pXHi`7gR(j`4CQh0B{$@Zrp|i0$h5Lzf zD%A?UyTgM+@>6rwQLzb=g^uhN8&0u-78;N64cB{fJmP+ME=A7o0^I|e$mhciYaM1= zXZLa)09`yL_DTts8DM|cR|?Q-iUY}L7@UVwvaTt9$$|GSzbMAWmasG z?4j!;dEp10A6HQCyWyD27i6_ln270nY+LlSze&MBI?Vd@O}aOaYOFAK19o0+T1 zx9VT_*HKvVMKL>lJPvgxR~!rsQLYL1#Ok%Cp9hlL`VFCROy}bT3dFLyx_!LYzSE1} zZdg_%z)!FA1n#`8($|2>>FYMLWoA$@vO75{&;f>ps>uOh*KFbp3R@?>$&lN@*ht#Y z$fIO-`bR;`3FO!FCA4=av4F|EIVDqTeRaxMNaLgdPWT5x&X{^XymQTqS(DRmq?Rl06gwI85A!u{LbZ2nys7_b&BQ4Asql}wC{jdVqDMi2%aj#ARg0*q#wEUBhiiIE8Tlqyz( zPZR)ueqvfSbGTqJ>5?!L{xW+9pjJ%{w2)&HcB!z|s{Y!McktYdVGY*UQR(-X7=)`v zHX4*De22AdmNKPRb$)GM(-JC@V=B?VX=VNc#<6IX30@69S9m3bk;?bCiC4}Pl7Fkg zGrsXx|68#BRf8HE%?rMP*W?8RBs2>1jI`iSbTafPNL~~q@zcX^;h>ZdHS*yq;>*>R zbORr&%-%v@-FG+2i-t7cG7a>#+?)mIqBlKD`l8ZzwMKAcE>Wg8@F2s#UxGapL#VbE zV%#q!@)Z~T>Pn1|2@_3Z{=-bna!TS-2~+8IQ-O+QBQ9+Ib$84lqwS2$ySSQEmfnO` zurdhgA1bfP{%2?U=X3(1wYaOg;dG%}?amw-x=Gf9zHp<~aIj2DWqodc|wef$>P3_eOmEGa9sv~l4Gm`veKIdr2{R2oDp6@PkE`ureiAIK_@^)XOTxJlSODymE%tL>SWfDudr`+By)6}p>heDYI~t9l&3Ilgf!JTSDrll;*y|ZRCkwpT z6V-0lD5kOLs6|&olC)Y$eW@vJh4q8O!7f4v7kh;&=3z7jAV%)!?}589He<3V(D%$H z61=4HeSOxs^!q*#>WR319H32dlxuVvUP!QFf!xAt&bNi9GFUTbo4nYslN&4tiNQ}f z>=xufP?yse*y%Su>utx(&6JjzJX#b?y2@hWs^9WRB#g@EQkcawJA2Dw)MdR8fhDq$ zRy?fV%m~rZlXug?@eb-l_sBtY!Mn(>tjhR`j^VWUnJiNh8o%a_~vYQaT2je``$pbS{SjW8eImT zBSzAj;H9PLB%k_5l-_;eX-NoKLSgS44;)@e{s}s1h&lwasI#-!Os+3|JDHfGc_hIv zb9Hm7^5SHs?9l$hiBwt!nXIaVQ0<@S0%ui6hB4Wr71f!dRG4qwO}^O`SieDHX|_aw z8Gm15hImUhbokDvDWMiJ^64P0nZ#V-ZGFDUML_30V9s499yt z`{+b8!C)vo2iJai3zs)aWs<=^>HDG__i=Htb8I6Z!VU`A93HlO*)}E$W_}j>hB7=2&{5w_$gU zhCw!Ym9-!C*0dpDbC^h*%uVHrorTglhXEnM=CW%p7{=uSGt*-#B^Zg84v)U5CZXAL zOFsq9t|i;ec>1h70=|X9{{dY+}FS zwV!w2hW>z4r&E}>pyvr%rD5Axa%5nM(yflkV4g{t{L=Zgx$z4MLfrdpySo?_RUCvk zft3|%9oOWNz=08%9FfE45d5yQw_|b>#`+kDp~7iRD@-Ss!Dl%?xO>c0?0a|Gl2kSw zVO#QR6c0V5ss)?VdiHh3&0w7LAUvD$fK+~DeIclXoy|-hWpbUvsQf#7#Lz&tdC1Ql z2mzm#z&IEaRN?;7;cPBGslBt>e+%q3kia&}Jhzar^cFiaZ!1-E75-$flFAM*hk3JV zKd3QqZ_9K)l*#`xzQj?ArH3{Npu@ZXR*c##Vv!)C)~H>yC1A>&2f2Py5Ol(Pr7W`P z%aj?6_8TS3lg)&tx!ts+V2!0dRc7RHwqM1dIC|BY|?Ey2FySJq<8E05BGy3he zwH~gm5h^+F{yi%bJ}OevII4k^P0^?8Wi7t$kL1_ex!~4j%RceCa10*X0W_1~-_x@7 zua`ARN90`2i1p9jVuOmOKRu5oKaE<9cHb4EmDLg$G+9*?q>|=&E@(`=3%Akze|)e* z)*ogpOI$9g&QK)_$UZDCR^=1>@y)e&oX1Cao81)NF4)(?Lx^t%C@`zbyvsvuH zq&9q=&e$~`%*?9r6dFEuEmvr-HfGKrQ0zrQc9^UOkVz(@(pyRl+@f6S4TC2UE-W_u zjG7aFa1-r-r@6LY9vB7BJ~x`J=1mYu??{f~>m*tJj;Zf-pUjG?&Y-m(tMOpx`yoC3 zeXG@Crh+Za9=1W(XzJ`m;wLJe!Gk44^SH&I&4;i+!0iO{ngwMDwqIYsSRiG~iBTS3 z3(LghsKFN}m>ze;xOtu%IvM6@9W)w$=1arht!-xpwy+Y+s)ykUd1b)xg+culfJHR# zZTt);VqcR~J_=atc5IEPFE#`A>oTGD!Sw#~gNm6Kk{?ks6TXplU(2;@(rfjk1>-lH zsA1@jWsccu%4j?RV5)+o8WX;N^X&EKGgQa6aQtq#L7~fNa9=9hJ0LBpw7%pmGGW8M zGf=mwRan<YMZ3w>9=iyY% zMz^_yz&s>QaMSf@!$i#!-nW@?;LMXD=0v4hw!gKLUUWXB)Pkb?^ItMbJw+fY>}`gZ z%aruar4-${B53kQlX>CTn#BcZu25@04Z&l}-#(mJ3AbAmoKZU|!mblRBa38^&L_h; z$LY1CQTR2ogp=y5mA=wC(zHJ?BhtOq1;(+Yo~?>bNH`uyQQ+jfFmz1xP3ZSUgEPz@ zT8QOweIm-w`$u3{Q(ahvygy@KOlE@>Ly`9gsVQ4UXU(^Nk+#7WOQ+SmU!73uS3AEf}f(%~8qaR?kvKhOZLh zJaw*_Q!_Qix<()NzkWen>7}}%Ra%5VJnDy;JrRBZ2dQ`K5BWd~7G|K)H(X(z94k7a z#YigvulhCFHnDU<5Hi0ye%y=9K`acgx#@*W#N(KlvLxvK5;8c*5^SDW^rm5+<~A00 zQ1htUgo?x3=%~s21HPdh@494*=|J1YMEK445)faDnT8umZNBeBMk5(ef1#FDa*}_# zUQ*fR&l`?@GW{EllbP54B=|u}Cf-6)!zAFf7xcsRg6A$yeT(WS3rXC)tO${%X1MSa zHN2|aosOLo5xTzyMVQVE z(E6ZY;_rOR2%gsL+J{QWJNkra*{0Jf!KOOb7wY zMbPO8t##}a;fBN_j6sF~R(#l8&|&X=`|G7!qho7P3I^&e2~Vc5E1cZ0=HUHh0vt;0 za|Ky9o8O64*R~Chw4_dVN3Yf|JH=z)imAp#n3YB4qC?moiU>)b+T7fdmY$l&`vk&> zz^0&{(sl=!QZ6EX6K@oTen}x7ayjeLH@ljTLy(hWAdewmN;sv+rOu|#9Iin z>7Q$TG)CfkW9UuAbRt+%ZyE7i8y&7lGN9LNz!z;5HnP&QCG>}Nof;Q z$XC*+gv?8TY|4iWtiLe_J@l`-xrE%!(MAu3NljJ{5n$sTDJQImk-CB!0kFCQ`%mag zq(Zc3y(Hs0y}?IA7E;vtEYE|ob39PryB0Bek3NRQ^(PURg`#(jn@qF|8ws#~JXE;DQGjUt_N5^PX9l-P{WY{~C(*>a;F_7BY zerAo+dP-YIt38PRZrD`}TZ#U{_?k43nI$M7d}_wg={~|JS^R!Qs&^C_zFG;c0{#}T z)BBrG)(~tu=Ja-{?j}{W?}HSDfmU9h_{m=`t)4!k?;W;v4&6e|lVUF>ej>R^Tv1UW zw$h*h+(VR?iLO>Gbx>QW0O=+{>_G2c^mhnO1E&$nX zyjTgSUmWt?N6_UA$3-KO!LOB2IRTc+G5kKT_7E|)P_h_LvBIyKQj!(=Ay+1cGlFEm zxhz|>+26L>1nk1^X*opSMs3Ev_7=^mNAtRKHd4VXaZBcN4L-<~J#Kccx{Xy7zR(fmLW0 z@It7er|&Z+#rc~H>|h!aRG8lujH?!NSbPF|n8JQW)q}KX8u59b=tnDu(&RGN-1S+C zYGzedR8}j@jQT!4#%c6^0re>`vhPO-lSzTfd<> zNqRf!l4`wC9~U=_SSaOj2S^p1a{vJ9egQpqSHwKycZ4bo(tz9{_qa6%0#ENTDdJ{Z zLPz0jvN2beQQZwwJ{WjMy8JwCv+$?t1qip7Az#YsyaVmy2t^zucvNxX&R? z=oxb43KH&A4kOSAo3NF<|nc?PLHAJ`Fti!PKp70v}Z$OIzaj&QH5FiXOM`v#q~+T zWE_}jtWYBmrbgQXNGLXGad-AkObPH`Al?)wwt87o#GcT;hPf3ce;a}8bqtg67wYWD zDa?5PAG+QtO0uqPvrgN#S!vt0s?xS?+qPYaO52sTZQHgwo)eQQ8|pNN_aM%D!CJXG?~iNdt#e%+-}bull&|i{;xVP83x_+GZFUO^BdN(x zKM`$aUeYEp)FIyzKzpb>Z+l_&Pn!$bz@gXRMijh0sbirvIq=7JamwseQni z{d=c-&elgW?y@(q>bA96>%DsM?fnW)@=>?0w2FM%{k6}>SWJY#qL9s<8RIlM^)&<~ z@Z+zpPt2=$k!4(GroPw&IPPPWyZMz{_dTnJg<^5wzSF=&^5ghc?1>M;9}n}@?2P$g zgGqh3JRU!j_v)a5OcKX~5)00l(9dKawc=m7!AE-?S)4WR#ud69R_P zHe2;^^KPwX50TItm&e??n+OjgM(ZT;Dk!+G4ABs6B7~mA*{Ws&wn$B@@O> z)yg~r zQS3+>2Tm?ilsS4%krykjVG+WmR|Y3H*q zkn@*&zjUp1jBf>eE@H>Ot~ufXNkH?qxl;EoE<|0cNlSB=v?7~^YE7B%zzN9u;9aZ@ z4<99rNlI?g{8NJpV88;2y|QJ`ar6i?m~DFI(Ux~hO55fVuHx)VOHz9sI3uqmOk8nW zo&|OZ8*Cr=QKXZS90C9Hwcd~(Wk`Gj|F}1Qz7_~hr5SEo%>~{}^3G-8d-QEeO6H2dT5R9` z;>9BM*g+cRcSA2sQ_op{WnU}G1R-BYBAu(>Bu((IHXckF^3f?;&KS450}l*Oew(KEL8a^=q%1uacdb=QZeL>6CcV=TZH8}>z!3slvhYEut zxLbF2c7@|^w0b-zkQ%P9M5QS+VU{z%@MVqF(;?~7CjY_3d+NBxLPcV063!E+Hn1W= zfo&?eB<4B@r%mL`-Blw<9Pu?YCk1C51nmq>W$)#D%qM$kX@AqGyJ zpEgX^_(<;|a0+-67Nr~X7m4iD*#Id*hkP1IlK5q(yIy#A1C$f}I91caU`Mf2R}#^9 zE}R?D1E8doFj>+an2PFZqlObYt>;H z0@S25rH#db)DX5<>iH8ttls!%YdNQ1~Mf-v!>W`O+ z?bmCmi$f?6qs4Lfsm!xHUkOy(rj?ut_Cv7@{NXK65{zB@9$wf7RM?N{jj&wl~5dWQ|24YfRvyn z+`MmQDNqXvt#+!TV-d`bS(?LJ28SB7b5~V3gE@B&jxkZ0fbSdenMSdMI}61hoMrX& z2=g{RNGLEIKADi1Qb9oa-`9pyl?FGx;^wphkaw}Um}9{g@pvW(zBt+WHaD?#PWvDc zbHP|4Ecxk);*iDYpcuLQzF@$xr=`ZVfpGl^d6UVc(^vvc;j(!1*;Z|Q2kTe8Lzk+y zp;r-@!U8MVNFny&di!OONgyO?NBaNB3dZd#(iexzQ<@qxJMn)R(`a$wOhzFaPXXrR~Q$Il*F=WBtPNSXs&V+wV58x*AfD0-DO13zKio*%&fTz1#}PM;6A$eBm}3 zy0xYz>kMOlmqv|?;J@?2w6(UBd$jmV;onuFO)j^%S z9aq?6(t(i75tHIUj_Bhd%8exPv-!?NM}5j0?D=bOYK!&}-1_*$Zi+TC{J$N5AWAtP z!_@tLS5qfq?eKl}AsBFCDnDZ|kFz9w%Vid;ac^qHgJn_R;c4hm#4@dLbvsqyOWaRY zzJgc@94fV7xP{}9>H&_^nrPij-bi9w@4@tjFOSQPIux2PKrTe%w+sJJ>+2w-Uv2bK z=K&`Tw}v%_=MEqgz@*88382&*puS?BqvgQG!D^zhks~0?3o8g3qU1N-MjjIpY006W zRhcU!Z5GyGf`RAv_$LrpG!mZd3&w{xNtNK!Ml{4ygWAjq0m-;ZTAdZ*t`Xq2-vnC* z3SdG{Bw%)$zl+K2B?9*ZIIVeH%{DBx=dX6iu!*ILI>oF!_iO@qG|=RdKC-s>RpOyPBqgKp1M`-m23m zCIh8?478g%hUGn|KKQ;W{uMjg#dBp;GxzZXLZNwnckKl%k-y?oxD;!}E%>fSB z!8A=cK~p|@)W)zgNBA%e=a|qBWjY~#a?BJgIpzXqnturTb`M}F`@ZDq?qMzfgwoqT z;+3%i<1N#}u%D3QC+@Q~pr_V$B^fXLdS=CODStpgRS;x$doBX*lpNZybAl0~%)8~f zdvoQE=6ybI$5Q5$xMoIYIaUZQlfaG$oabS)9ta9OVndIb8R-zEaJpP5e4XRcn)v{( zVQePEm|MjL4NSwDO&38EIML)PuydY@3o?V%QLQK7lzz#v?ZWZaRiPo+Th14ZNOT1+ zG>*Yf1eLyEn4jZN6V4?0bmdvdKUdY zc9tu{0$Zu1bpG9j=9dt(_BG$<^eeTTqsTszN5AvSp(p(DS!7a4gZMaId)xNWSuiea`LHL6z%w0D@0EXomicmDe8}tj=!B<^90gfoh?*B0qwPR>yPGDHQKT z{}a?>S`SN5>Tu|QB*VK8ej+(F!U$2!^l|_J@I?7|a|t?EBc<>%vVe8ROaQg8qa;EV zQda%A@{NlkVg@X1ZBIyAuPNLwb#cr3O6}s=X_kG+BGH zxDXNxO7Y=(1=Xl~)FmaJ8(oHlf{MvcvIeUmiOtMV$il0;np_>;#rWeIS&L+hSJ4sW z{i?S_6bO4ENlWGqc@5LLyR=S}Um4=>9#O5y3OU$w;eQKGTbcQ4(rpr4j&osSlDmgc z4i&t=@YMUA63T2qj12d?+M*-sn_|X{nKOFQL%ec0Im()US3wX9beMtz2g=W@rvhfR zYgM;xzII*1324%B53I`&uNn61vHQjwEiO(Nw78JRHcwSP|Q>gTrL1$^nl2JNETjKV{Kjyo8g!NL3 zqYWA64WgOGhYmJJkfF0@o9w=scwL$!vMhv_{Q7||WsrhXFH+5Kvu_FYs=u^xZ3cS! zwvY*O!Nhn1+tMBNr)GNt*v_mC!=~{lDJ{ShQ%%uLeAqG#>)_CgH$D1j?tvvr8eh^7 z;g<_YOVxux(wmP{cpDLl1KAa#mukHG^b8xc22f6H%vFSBv`a|22vznaHS=|(+EZrAV+>(mc(8w=D=opm?D|8@`%3Zg~Lo`LAq7mm*<_m(&8== zRp4DbfJELCr`TagWtjtD4*yoDvB>%_&i`jvmyg6KA?V>h$<(%;cQb3!5e0rMy4z3 zAlHkV@44l?0ZBkaA5F&i#1c z0f^>|sUT*&#yIa|O=bwOfSp$A57SY+zTQpz1b(CRG7f$8OTl?Ri=f zxBny~d8wc_zKzP8FK78;F}xHtRc`;iwSO*u5Re%+y_);segypIj^O{HFd`8BhipOA zoVlm*eA()2l~I1kpKNw>ukiTzdWlBiwM4SBMuHUpVTL=Ggvi zX(>+`P+T5RUz-derD#?xKDok)ZLfpblpMoA`ib-3dW8a#y|df>Uqtn~Q&>;J8;h~P zq@<#ThK8Un(8$7uq@<(v~1wyiJS!qQ#G zCt1Bpj8U%?AJx!MR8%BpN^9Jj2auMg1AhYj0Z_*O{w?U@c1<`m<*{R^A%+G8V5`Py zez`K0%&ORMgZpRW&VL&KJV0%5H8P3=9ftYHGgh?5v48rT%cO z?X4?3oQkOw!t&|~+@MPpBAy4bNtGCH62`vU4Kbe9V zLOx~#b75MWgWX+SRn^)2Un&aoTQQ>UCo@6-PW~Xd$tC%D>hTS`yS<;ZkZ*g4ep1izXw|5hQYRbN7c3?(N-ZXn)itBo7)p=89cQV9Yl@BUPS62N@Ox1$w5Y z{bCFuDs7EE<-w8}ETbP*mQgVk7j`?(m4=Uwz~MGlR5Fl~v(fPyAu^VaMF=u{e?Hxe z`b$)?O2;cMAr_5Cpr)*;tD`fodx|!2BFYgBb*-YQyfw2zMlsp$$wd1c7bzHzgJb@ z?N3LRmWJ5i!YJTE5?5D4>6g0tT&Y4$#ysUuwZ+ub6kJ%M@t*XNRTj!$s!J0&d?uIs zG61036a&@S-7@B2p_@6>sui72-zy_z7#Mwe78WY1M!ecjC5_F)!04t{o@f1#y5~Ju zD?*)ylb(X&zCObr>oH;?0u&|NfYfJTMV8fFc@!rZIVbP)?UN>}-&(p7H0&{Dp*8`Z zYkPkFNb_7)ox;1sb51I!{0E22#fg&0%jwBZcq|ngJI-%~ym&wtFlK%~cCLJ+X>)}E-$Kn1wKYW?)FY(GD}&14A)R~jG}jOpA*l805S2;b(?RPlJ- z3-j~e3yf6vco=*@Oq|*rp{YgYR#Qv6sa52CAL7CcW~Ze+ zI2QQAU>EpR0>iF6|2k_2_~rEW=>mXXb`<^0_u&sAAi;f)wG@o*@npDFEZDK;O#1qS zZxMt3aLMg)m92GhsT9768JldMSode3UZhgmzb~L7cHLZL0cN`H3wm*z~o%T80^?^*n^0FZ| zsdJ4*Kme5by#H9^ebS;RA^z*w-h&An@ZJOD`esPHRn;U`7Zrtj1L&J~eeWIt*bZ34 zMdCxp+S9`*C)hjRBmGeqmw44Crspt(ggSrgNkid%-48z&8fmo$m5p||04j7*wuL+` zZ`ebBg0BkIiS?H6mEEGzR$wX9VnGm2V&J4JKuv0Fblk1hs;Ef4bemXHXR6vn6cVK= z6;4K3{KzBZccasC0W4) zZa;6(6UFBA+MR;Iamnr`j3-$OgNJHeE&b|&i^(D!h=;040R>+2E; z()0G69Flual+Vp$ws;8iP@(w&z{$zXiVYe%@OG|xJB`GzFbFh}JX(qa3a{!RkP96E z@XBw!@sbD^a=1zN?_Sj9nqy3~RP}|+=!q6~VXwhvkJy!Uo)t=Y?D=lLZNI;7yVaW# z;XXY(2@qUTdp1Hw|q*pjIyuUs|XaHI7&x_%0GU0(v^V}1UHV)I}S896t z&zpU^-{imHZH!1JWLhM9xqV(!yS0<)^I7Z&Mk81CM630-`4MY0Z?+)w0A>>Gq@EF= z<_uP^eA3-mgMiSLVV>GA z)huV2c3nyPogzdA%@$uC8?sZJawG1LU@-npO9Ks!*IrV}+CABcuI33X^A7fn^v@GV zSDn6s8R>zMR;md0;xOA6@3e}nC^0n~t08vl3}uV^`a?)$aVY4_XjS(PI+;^kj?3G- zCo&?j$N+g+g#BcbgWh8ESLv_cyVlpUi4}lA<^Wu{HvkwXd&Fd*m6$k6CW8^R)W+r! zH313sQsZr|I~TzM_#!cVgPmhnZH#!HLPv|Kdx=3*CE?6t-JgLi{#U4)PO7f~&7uH+ z(TW0@6ti@hySJ(l&mIrz+OrG%xD=;}F_ucK~V=*<4=GnAsTqo2ILj7M51$ zV?Tam;bgTkDVb1f&bZk|sj2Ddy3#GCBej-Z;Q+CVxd=zb`*m)- zJeVi5m?^`+-d1wkGhoO6mdO<6X7$cj02u|jJlxxpt$BTDxJ=#%WuXIi)|aga1a*{2 z+lijeP_^VtRF0+u;ojF$KP2x2#lpKw zl0mt?VQ?P7A|zz7G~NJoeYCZ8sGyD>mjcjoqK(ySlZ8*QZsIvcO7e-$=qVxwP#n(6 zy2T1L6d9e(t$lQg0Qhkca&twVvvp#?q?$%(L%X2OXqlKAYe?HZL>5IWD;_C`23>$% z0SRwFK>|n~YrPHYGJz(^12~bGAaf>i7NUAbcG`-zqRXr5iq41{-pu;>sea^GziiZ< z1ua~xa1(eqU1z0iYU(B~L|i&aJS^Ne_o8@#JKCJWXkEhq=+&R%dv?S9rA_EaBOd`g zBMP@Yh|1?0nYhtLES9xuZVSx+rU#YOQ&~7)@Pe!MK62bu|Fkw-PFP!3Jx`amRli5V zT0c%ntQJyzO>CAnn3#!qRM8eM**(0~@l;0~Q}YkRJ*^p_lB}-jNs-6c8cYDSK1EDT zERT#f&{cQE=w9J`>tM~Y6ak0u#7oAr6U11AFBawbzuA@C|1c}@i5I@S__=>2Gt zJWHHQOG$12y%&6X)E+xPSJEm{H{v2&0;s!d>`Yy!VspZ(OHA`qcLPZY>^9O3k|<-M zV_~!a?#OWDTkn#F>SNp0dJcx}+F#$b!@)sPXknR}^EW>U39svl!lDV-7M6!m<%l?1 zpcL*hoT$^ys#e#lcnPiXG%C-Hh^g{?E;5RX8z2!Wc6r#8;N>Lq|2|ydGdp~gb$$fi z@D~^ubTwb|0>-ptT!eQn=Wg1a?jh|B@^OL=XW^j%R<~V%mgX^LU~zFMOrR!<=>ifK zfM~a8<|}jy@KFMIykf6G9@XLD{PU$TaTm*w6INE|y{^O6L9N~au@8W>R*3?@vI(+T zQUWMjp@he7iPPt3#f07le2H)5pVX$?j~&c*2?v_L0U!_80`A%NZXey+FQzK<%(L>X ziYwtzBNLs#YgYtUD+0v^g`jLWIBVavTy8z>YqYyOXsTGx$a(G0G$MI z9OMkKqg_|S6xNOeNFTCBGl#{Vr1mfrj?7Lh9-qqr)ov@1eN^%*{@3wL{0<=C2o4Dz zjfJt?zkPfwkv7Bq^Bi|1`xBuAg$4ecXmFT(4l_ohfq%*K z(^IQMk^cG9XPWX{87A~gkTWsYXAvoqS09PJ2eBi!&p8JYai42nD{_C>u*aw0(-bKG zpMA|26uB-Aq_ZyKV5F|+EeFtj+hMlYq_nFvP7giZyKKYZSkz%xZJ>3mEK}Lxhpi~V zE|P`#V4SX#_i6mgMN79PO@j?HzuI=M>xm42_hurS{kWa8_a~r3$?3oh3rkYd2>o9c z0Ju(6R0I~6qrR!?))W5wDVh!VBV0O@4=rbw=% zLBZp^wkz`9*2;Z+gaMBiFTB7v!0F1ne_!S0&^%9~Q>6OSc%`=BlXGKyy7B^qD4B^A z?gZ@R9iSs5s25dEAt*VZ81jlKhn($Qw! zI=cX;9yP`3f6F5(D*iINqr$1+I?QYMMU#aN`F#9=MtsbQW5YZ$; zyx;y`)J5VCh=}J>op7`8iqY|ra#?N;{9<7Yw5n<~i?HA??E$8fbb!w(0)F;Viyg^v z_H0e0%-HTaK;QF#jSm@x2}eWY=WPu(Y$hD&#TZ)~lO6{dTQ90^jPAra6Q7^Bz%K=T zo3LE7xdK49XEX4OKtViRFai8^xdC6!!ILJV5_vkE3um3xEiKCIw43aO`gyr@?qo-{glj@=Dq4pa5_(npBE zmKL7o%=)>=m@GH5W3(9)`+Y$+s0MHeaR{d=Dt`7v)gM19)cl@2+H6W8fIv<|5zmKh z_s-6u1GAF+OeZtx%w%ZX4|*Z^@)h!u!5Hj_foeT$-n^$zqwyHSY<)Z_k^CVlYUv;4 zBf@TBV5ef@t=NFDE$Up41?y0W6 zf=d2;qgcwe6P%L{kHcavQ_x53cEhAgTMInQY=sTxJm)gGlYE#N8_R5c2|v(f0txxB%vQ?}FRh@G0|&c~AhI+0)rky0ok(*x}^j6dMlJ zcj|cN%&v*&A`y`E#m!ibMhdzJpACpw?x#R<4_ zpjcY0a$%ZD*pmY7p4835WfOPigXiC($8T!uC>CaCBtNXJoPvwPcqa5v@7m1E=>=n5em7ID4VPO;|(Y@R$45Ta0eR?8EKv?(PPFfv7pl z##i|Kt`P)=>>k8lwhZ{w>+^F_Y~Af*{(*|Ah!`KW+b$2xGgt%>#1Zuzoir8{)vX}u`$;r*hiDIRatXBHLS&9{{#1y@JtvL&rmzU^kS(##| zJtK-VIHdU3v7a8PCwQ~L#_2wCZw=-u;8K(J3tSRcs^( zwfYaSvWX_#vRHlF{V*|5lOlyYFa-s*3!F7$G`y%OY@9U~3W|;8b=W)R%DFCWjZ@*u zDr9@F)ZvXz!-xW3La1tb`#$Ftf~I3{o1mH+N?2GC{}Sgf+8pj5`-RYLA}e+ zXPX(JeheKIXB0dWJ2x$DV<{sk%)|OC5KL|&>T!~O;R`L zA*N$v;^IQs;G_*2=3r)ih~+|^e|s{VOOQsT5cLg@ihAiM)SIu5Y*a_D0+jx#VW^yJxpgS|KF|UYF#m#`1%C&c157u+a!~70gb=96uK~HtDkkmP`pgM7^g!_*g@L#H@K0|vaOE2DT%@eYAx7O$&-|5pYQ%;Tx!i) zIQ{P4MX+s9lW-)3V1IeI5k}skon+u*RYl5FCU2n&b9Mm%(a`6s-8VHgb^qJViHMtg z9(yKgE}+sd<3cCr-!R|FiP@Yf)f$Es)}_5SVRM=zX{UO(C1n`*hd}IsaBg=>k*0$9ihw zwp8JGPt9!X;ez;RK~K-qZ2B)LR7itqwBJENTlkbeIADMrw@Hh`sS_ zCg3qJT9*Z%*jbB{Ujx{(;cF!P2|*fzp7_AW;WS|$mklk-aR$rpkpV(ljnAk&{>WJv zI?QIW-!mn2UT%P?2xp%r5fUnX|Lt}OZ}Y^b5+#fiX4ESx&1LMLG={g zVE*p&{_yq*J8jROJ>n5lcPTxU`gy+{rL_2U)8|C>UG)oSAzEJa#<$*jzkB-wson_m zt>%~jhsTrAlbHs3YqR4zJ2LpQRG{18!K9ljTobpI6kNYU5g+G`-2%;DPf6Z;a zgwL+C39JG;@DUKs1yxg{jERX^nZ7`27V6dQGQAI#ThDX1&aIhV0jOyr`9cn%e39ry zd3OCmZ)>0Fv>Ofgk2?&y9ri;%U&0V~IQq@OAJuQSzU|p6qq;K(;mwW|{O+JE4%GMT zaSIVU-=?1xCnSr54ZyQN4hCli>a99UG76R)gX)U1M3X;`mZxi%PD?1_%}%$PeF&L| z;siW7m`Lrq-$(2AOWWc|F=Ld&7$C~tKBD}5lGreo=C0@2Op$ya`}P{9zmLep|73}% z5i(4}Vlv3&+6nc|q3c)tu@d+&e=J0_MM49mPDC;EeeP>$rRT8T5c|V3>1|YAGRtR* z=YaS9ax8q|KupZ%_j*qlZ)67;iDDUNq#PcmT+`{sF78bGyKL#D)x$ZfrNpP!vy zH>JyU`E_!<_YXcr5(weZPzm%K(UlvR-*2kFT;O+g@E(X>m;63o3?{^fBeYu`_DAYe z0AflC9g?TmI020g*H70H^!Pdvs2x>-TKJ^%skWrKxkL}Ca zy5Rajq_=@pm%`%F%-4oIUS87$K}|^Z>>wR~&NR&LyQA_6JTr+>3Btct8|gySU8oVH z4Qs8QWmNh9TKYF#RMHzb;P#@mY|<(m%E-&%64t&WOyQk)G{T!EZlW>+EO^KSfic)_ zws~eLrTL-Pt>%rwvVF;GcEnQIr<>w5ew_Nnl&|5p}&R(X?3me zJ7Wjpb8_M)#uO&D#KeAYw!1huTe4ScRhDfR!&vk~F_Rghb|u?Mica8yaC2q!Hnyq6 zU=~zC9{~y37B&0@M9tsGNPPCx1ywn{n(+!mQF$*Mq)C;?$;Bi>A&u_Bnz=kepe7q> z9Q=!sqWr?!s!j@vR))YwkQyX*o>Qyd3j}n*A!tZh2>k~pNe<7!is_11^1coXZa$WA z2@8i#s#G98e*1+wfH8Hc;!ZYiKxlzAjiQDLJdNlW_j?}f^4x+V0hOynz^Ws+u*W}U zdHSq?ZllSA%2ODj5ixPDfe)wj>Mwk3A#;cCr-3sKf^JD_dIxgjms2mJ_h;y>Y{lT3rLG-uofr0B5S; zR$#VFEeUO26Fh8{R^cgC2Y5Aeye;APELUs|4=WC9$_TW8A4sLQdxP;a1Z0wsARi*z z5nH0qLzmtq#dxGjaaX>3idIRSXfT4@gH-u7eRep@&=l5$TvfpXNOKV{x|>AC?gO%D zR@4$^S+;?lKvSBYtIm^P;jMPfsePl-ut)NE&Un5<570*OK2zyk>wQTMX7H5JP_PL2 z#x^xocC=Jf204@_X!L8%KhBU{1VmCB70j5gUo}qt4#6Y zcC;}3C=ghZsmLnz?ZIDa`O_~*J)I~pD@)@UUW*iePgc zTI5uPp+HcD(X&pkX#*1T5jRN*PIQn*koNZ^i|e1_TYr6qIraOzDbq=KN|9V`c4<(9 zqkZQbTxB^UUqJ-v4@T;FrCG?i>9Te*U3k>V-NBCUx&5OsYZ$G}K_pj91~9Z$rCh`# zS&*aod16^jN9oe(aDTp#Ox}s z33m?;Mbd@J2EgGdVv)XVtMRCj-yVC<|NR_}!;>+aG#(^<$f-u25_HjWpJp~at1k2h zte)o}kTKAYkfDs6&^Ryy?-;b$QV>E(S!Z`iNqtjOf;;EDWKnOP5t=)b^kfOck%=jz zJkt(vI3B__m0Bs#c=%kP9saI;hveOJCujQuB+LX&O{rOuMKSU!7mrDWsGJ0oXhH`_ ziTS%P6I8l$mj+zf}l! z`2qhGosi1F%iRAVt)w9RiYh6GS%LsYyUi1vau@V@+V|=7-F_`U6_3*%&<27E&|MPl zJVL_`Q)1%Ayx+d@g=&4fds!PD4av6UyJ4G@)>0Wu=_x&6JxGX`?p5!)1;LiyuZ+wiwjKN+AY>eM(EqNVf4*PL&$!La_ zmTWPhAJDh3XB3UKJOT1kQ5gEey9;9?FJML%o0SkiaN*%X8jsd;Pd;&b!(HU13x-r&bbapf# zY0+Jk8oDV5AKx7w-MCoiL@J%BaWDi*@dYdg`&UM7LUX3oqN~?9vf#~9YDzBYXnZ!a zT8*|1%4*+0&#apv5~ncWu2R?_5+xRuLrMOdH$LT)iO3P{a^$&eZQVmsXk%Q6*#<|z zG(PMhYfILskMy1GI=Rg9REHT(wVL)e3H$Mk6jqRL1{(Xf6m~1-G5c^9Ac&wZycHKR z`Ma=C6N`U>CVd-uHO^MwG;p15-5j-TTZx*B0SL_2vMUvzwMzEfT1xc0kPSIF8eyqJ zjrW$#Y+6-uwy|+=`^i-{b!|-(D>SzDwxGEntVqqOA}PJWoApk!60yZq*rQ*42&SCj z3oXPw_}u7kZ_>mR$T|?D8jG2(u&|eAsEMnZm3B1wH#ttC&B2U)lGK=z4MkLEoLn-! zzH0%~PkZOhLaqdjur6m@Fm26;Sz1%aNhF4ZV|j1FdmRM?ob0aXUyChx|{X^W?c zTb9zeL9&U0?L8KgGO8}m!AANO<=;hWJK~Y}#f@JUU7-k4UN60E06wZltNURtpG38e zodjlKP{J;;few>MF2Q4~Fas6Pd7SA!81?&u_Y>2WCk`NCf<0 zg6RahxwDOK{iBuHAPk?I(^!1S_y(akUK-eKJr`!au>N2?tAhn9I7gw*NB$9dG~!-*jWLYO1(ftZ@G-?k+{xqcsvl0(6xN??_?1$3~#X2`2`NiBiW zm}8~Y!=oi!T#9T+I|b46ARK%?=M%Y1Jg0M1hGTICtXyW!ovWsWi|{)`L}T$*>!jsK z;+XoLU)wzq#86R*sm150IRz+76;ZFmN>q}mqT;C$BMfMm3K|iG8PfAFG-&M{lM0+K zmYV)n2V5x5L5l!?Sh!b9LXl~e0>3=n{idNZWh`h@k+J&on@b!ruw~pa8JIu%?FQ6I z0|pSQ(NR|ywp5z}f^FLbQyLc#qei)lfb%e|>|wD7a^gLsrGQYiHq;4>Icy~jgjL?f zx3R6^%BVeD;`U9W77?K~;jc7k!tk39&>})XyM;X^+-0>N$@VN)tJqT%*wtQw%r#eFt4xs1B!Kc98TH`MPfFBC?;|pD*+pHRF(g zjx)j@i$9nyfu#pGqxmHavogmx4#q27{bELXfFWq37r!zt&`6;i-18 zHzaM6-^rxe0xr}=?Nacs6XzTJVt-}-?qneo(Ed#MOpHY3{EvK`Iva%?=EQ8gaM9x_ z=^#{X)X_DR0HsX0?@F$mg^0_1M|0bcplCq@j`1+Kj%k=SS$xm zFdEx+5Bv96P~nzQBa|BY%s`e24h{3;cxloTI1Pl@Lb?| zJrUl}!!$>=<=8JM3G>{pKTh8f@s9nq!*;N)mc7bkhnYO zFmn;nf}iFQu@MTLF0sLPGtv>$?2C}X{i30??9YJA?iisYOd`-9`13ubqWo{iiT ztr@n)8r>tkR3Q|kyi~I-MFfN$6d|*aSZ&n`ZB|371cV}kL=l<7gc3>k$HWO4mJ;xButjgCzD6cVa^EN+tM=L#xer`=Pamzn*Aa2&VAV(>1c zjy@gvwTQa;j=+j3y*71yLOYRby?2jS3<}3*&DC zZ|(Vf2}vmX6KtQ}x(NZzI^>Qc!}?ZZT$afE^Gp$=*`ea`)00Riwwf;|WZ<)h@a&27 zVmTOGeGXD^MlB4Zu^lf;?T}}Dq-(*IHtlVH5OO91#?R|?o`|07?#>6oDH!C*gf$+A zEj|^wG*G7lt)nLBlC#o(oz(dk3c`Y~VT@^xl~o|9_}jo?VnBv*25GA3X#I3Q>T2u9LZsJdDt&I&v{$azQm z;8HOi;7ADc%DHj3pA7Ff>@$+f8<=lY9Ysm8sJ7XRVDKRL9^h!$BfJ9zGpZwgQj&qu z`JGt9M;NSQK|AnJQa1;qDjJ=F=K9~pc!rAbO-W?}tW3N9P10iK(f4`tL5%j-x(*}B zl99y30azybGF@%WZ-bm9R*p~tWW|s$(OFSS=VZ7^?JYU@!1&MGJg7CN0HX2K*rJxQ z3Tmkwe$WGh1+0DS1qJyK(!eZM&(+SRvI=dav!tO$bNZj;>Je|p?sgn_3-=Y7{o@?kz>24<)5icE6I4zi$N`%w4x zNr1~ZVERsku{gtVQkd59Gc{`QzN42>JtV@W!82Z=G0f=1@L4KZbqfCpUF;+lXw~!kE3+tm_{^ zd4KyTC#nOLgWyB$Q;P0slUrPZgY6H7bjdVcECjeU4I#f zNBvz1#or90yuH0k1!L!d5*E|KFxDjMASMHclrO$FWZa8i4P12-Wr(Kg2}R&9ZVSDK zw>Elcjghe@vcd_4m@0@51;v>Y!ipGxq#aa@u#n`J-0P~7h+5*?7+n*jw7&Gfz5KHt z3G(li2-0G>qt@6E0J2w0V!FArF4>FrK9g2f^|=X)9Vil7JFZuAT~6+ z*KvM)3>;RoB~eiuE+&Myt1nPWNvXAJj2Ns)8jHCB0|08x-u`57_iBoV5x+g6 z1z5{Y%}@}#J;JtM!9y;r?Fb>4>gom>299VJExx}$s1`gLkBNw?j7RD4ga9cuP2lsU z{M~(DQq`5952Hq3Dm;qUl^0bAlJf

    32p(7hj~8RBO-^nMs|max>^V!nA>JAe&I z`ibg2!qm{ZVqZ^=$_CFQBR#TY#h}|X+rJIS-C**Pti)jP`|I1?;U-+bIE4ND?VfTx zk$ZEl$J=K1trQz`%f1snvg5X;Bkn67FX3+J?XbP+bsc>;&hhs)kDN2Sm7(Jsyjy7~ zKk*g|7<*O53@)+E!51Y?Q5J2R9`ELeh~|o@Gc&wRS2uV1^G4{7-Ng97QIm~?B-nX% zg?O`ywCfcnV7#}A|oyd*Xt zVFT>r6>*Cr%Kvh0{?vq4>2xDF1 zxU>DWGdbWhEn`saColpUElmiW1-cx6{CJKhF7AK=P&a>d57Ss$&Ln+iQ9*YI5YSqS zWwXjjgm*dJh%B+|21Uz6<3*RHwMNaeHEirH2h(|kp|aDCzY@4h(`(?R;O}&vsIjBH0ufgwE5A)6j$|$Bff3glHUY-L?XAzT z<=7DPAe7jiim|UCv=57({84bBg?p%)Zg;Cam&b|USEIPw&5_TxOIHA%7AT}D^g)~N zABbBp0;Z~6VZ^DEYU=}gK}}}y21S*lttK&^YwWG*3ZX>01HqdfBN&ejBB)G$c5oCm zY>1lWjgosw%qyIsCZi~9uBvbRMoMt}GXfO6F)a_tva;!-REGly8RL~>hayNP-++$=teN!re8`6~mOtv`C)>kOW*tay zv3xQ$?sT7aum~9YMo^t@J2I@~(U??j#kV8XqqylBi14YXQBOlZfLH6U?qH}^P-6|0 z9nSW89wn}TTdJ{=MVrG6>yR{t_WX3qreKYvXOb6x*yo#>n-f)7AT~&}aRR2r$wLF^ zDQtI)G(Zp?8f9- zVb-xB;U(IQp21j`i;LA8hKxe?Gvt6G`#WUr0Dulb;GGBmf{-fIHrFR+3MO6 z{H}zXQ^08FPwLOBhvGyisJqX{#%ZKH5eMS}QtG|#Kc%G5`iKYNV8T_n05*4F#ULbw z+hdGCZ!P`5$4E5rDeAfP@%ibG8#Uk2wZgX)j;~|Zv6XPI5M^PlmK))re*4Lr;R1z* z4HDmz@=oPi4qu#Pk@15RMe!sdt$k= z?YD~HkcbaFEbI|}mN|1dLAIuG(n-RTZ9?l0CgQ{&U92gX(l4)GggmLQ`{H@?P4^IX z2w-DSQAhU|E6rNQB1nAKW8r&;l#MG@+Xximab|nN4GSH0RXjZV!DROoue>2en~UhO z=z1JE;8uTl zTiYWNnr{&}7U&6ahjDyVj3vkUCs)$n`{TX?p9j$~AaWo}<+j5J5NBs0VJyU~%Mh_M zEP!+o`aDAOlM5}(FxpuHSab9_&MF+Q!2R%?4`7Eu*cm7erivFK7ED@q+rwh+SaB<8 z5)lbiU?L@r`(p&xgr1p>x;}xg!h^@>^^8A^MJr4NYL#x)| zf%R9;ZaOH;!@^3r=RpdQolm%Dk&`ggc0eP*l*&vV4&_kC$ljwd z*wOAW2=a06&D6-ldDZJ3j?1QwT>-cS)kFJncw9g}{u~7R%ljp5)YSLtql^MXKS?Ev zoKPhDy~%(aw?}fiGE-3JjXO@3+XPOYq*;OO@@lI?T9_0IqS)IV@#Hp2oHqxgaFUq$K!I<9O-P>+dD7h=|(ZK*JdGH zt=g2!To(>oWDCk+TL9Mfoy#_Gc?;Q_!PTC(AA&Wq-3c}tPaUl?z%u})8o||oW+F!; zEW-UIVGUtMK`!PLow}rFe%K@p1TG$UY++xa5bdl?3nF79frf(@z7LeM_~t36P~m+f(OK(Au& z-Q?9oC}Aw}I5|7yMUnr-o-K-nm=fax%hn>NV;t^)(N9|ji$QF}ZfXgm9j->##JsAu z#NXf$4i1j$rd{fZ1t_ zx^^6SWO88IpUa_+tM^|SKbIH+rZB`>-W-Kha#96HV6})9r2g>(SQ5>B#cu@ejn<)Z{BDG*zbe!vUa^Ai`HvKqX`)`(h982>PQPzQeO-s&^z^%z>_N^f;o{td|}lU z9*?w4^qRlWAV|g-9yp~7;Yt!95N<`O;lN<2FZu4=OlzFhy`?G;9I3-#;%cFpWs){Y z(=|N?-Rb&#N_(9r)0{ZCLkUd*@Olwb6rfem`-5Y0dyW0#vIr*FO}=z}DR$D*JiF&A zSaZIBttcjzvivc}J6JW8W1quWsWYT- z+iYbJvBUyJkczq9$RXu+j8_;*se(=*SRoD`j-?$DCO}kss1ZGMbaVm)x`0sXcDQAt zdZkYVpo3TXfk*aEzd!eTQBaL?eV^UpmsV%9K3u!zG%wV6Q^tf@g?HO2%VQmBFu^29 zjLsvDPJjI}Khz&67MO#ikz(o1tV~jt-Q+5t(FFeOA!o3HC~%HCm;K9M$ecV^%zQwF zoJ7D7!Je;apempwW0J~F0o@Can~`2fBY)d&jwn@2I`Rgh0E#t!ilIGlar+IA=`JFD zzQKZOTvMFz^5T-1$XaGMB8r`W^X@dcp*0ON3XusG{waoA;IfU0^eDm6^LGkbG9|q_ zX-*fZEsGe=LrvEz&cRwk=D2Hs{1o^SX$6)|q)R%wETGvB+0eMRN9y9}I&>ZF0LDPb zt8a7(M}BRO5qc45VYlUf?NU?ZlPblBLlPLXhiAifXk)BK^?j-LGh$^(+CM5dMb#VS zpwRbw0MQ|}=k8r?J*(<(U~JJ*QVI4otodVRNjlTARlNdZ4%=8@R6B##>PbS|G{snr zIGX=VD-OoA-jacW`nU$X`k)3pn2|H_MtCcHbL0G(Ju3|C4M}0J#`@eGQO;~tFl)8J z0-V$yNsIoK;&dg=@AL>15Hyy+lCk-DXbk3DHr_({E76zMVNdW#%(Xz90r;WvNvH)u zSJb|N52}wqrHwF9e7+ssDn=~zfG8V;MGSEF@@co7TKX0_4J;u+Yu|>{37!kB{|Hp~ zaHW6^@LMdRJ|6DWVD8eywPa-zg=z|r+v_cuf#lmat9-C?aV@|%=q4tsk$wkgZXjxC z)e_S;8TzBf(em2%Avw>TD?r2barL($+r@|%1N^5f54y@dl3ac&HjvRO+)kp=3(rm3}Nnt0{t8xy&2I@W}KX`>8ZLIHus<% zq8OY#PSe-?$;1GLq_`01@Y(wy1M|+b=(ki?%p@*MR@0VJFo)==7|NC9Z^- zl6vYvsjg0M(Y^H_pD20qV=@p5@F^UQZsLF_T}FF|Eh^*K+;MzgOCQYednS(x4kIO* zUb{~rAr=NDF zE^aG5Z2HmxCGvZ4NbmFCT@3UTK7de4!SFXY7|P-2IU+x?-2kB;z>pdvm;sv5Cn|a^ zb&A(rOq>6`#1P%DH1blWE7ptD*Njbvd(iXr;3*2Yh`BTn4V_btZY<1XRtVd2s?ZUs zOHC$0UyUg&{m#{;leJ!=6KT$-F%GEv%Gfd61v>QDv7K^?v!JJSW(|t891gD7mH+-qI znDotoNN1_Fx|gvX)AiM_{a&foqB5+fo3KEsx+6RwIiVbGW39E*-7ba$k7lxaD*CyQ zftDISZRaQ+KiGy%N|u?2bCa&d!E5oS7R(l52QPUXf4)goBaq#rqSfp_OH)W^e$veP zf^UT7FY(gSYPe8S&@pf01cURFwPy+Ucz|TbW~)Wmj65(X>A6@;i_y>2)D?IXhVyL3 zLBr59BAa8iHhMEd`Id88>yz=_LN~t}W7+3Fw;OQFOMqwy{@FMc(Awb9h{6;k zCAkD6Mwl;c{3`_j@>rX1<4)(;L1;-8kxt}o2m`@3g?T}0Cqo($XlviG}JndeA|N zFWE14^y?2BLXnW(|7p&KQRQc&~7e?NCsVW_}gHn|;P?EdHG|39Ar2#NyKegFSL{;dW7sRsy( z`O;xhl>;`4`M-aUza(^bcYj!T>-Ab_KXiK1@z_eI*)Ukh&{(d6Y_&xWUus!xlTdBi z?a&|D*v+>ctuC101Fv^{DfcpBPuF5f;?ua?i&EV+|7nL-b}47TNNCNWqf@oR4A81v zrdR)!Zv!(%yqxQY>}t^&!0I&7L9f?iNLs$B9!)PU7R<={^b`ZqTl)-Z zX{_oq}{#umG~w!}?;vnb5Z-Q6A_HuCl5bCu;-FS2Wg)gw6G z-qr?SxodU0O&wU8CAi(GIorZo(?>&ou$0h2DA78DS(iI>InjTv+o}!Kl8=2tY@RY#W$z^(MN_4qCgmVd?P8|z; zWtmA%PD#yS39xvV-xCoH{W{Bl7Ke##whmsSfYtFnvK`qMj zuoU#!4%zBDs9nPW2>}p6e4mrVladYrp^dJ9G{Viz<5XMC4xvA?(y};ARenN3&O+WQ z+SbN~Le5HtaNmq($!aP(?If(^W6RA}s~RjLyDp;uf<&UzFE%~$#8f1;zuYpw5cPJJ z3yt!-YO1a^`E|w0Wwo|>MHbPtpYLTKd-I)@Ru>NsQQQ;!t9DyhY=(VKvwZ5_rL4Vs za2;TukX4mYu#92wk~|*F?*k!GD73o|LS@MOkcbyv-#(4z_)u>?mzYaZvXU2RsnPRg zH8nf#)A7|joX&@YQq3-$daKG;o~%ArCqLE($LE2s^OhfI2K5e;ubObbr%L37?ot;DIX;1HT z_cA^eg_-5C?5TC}`(F)|ns|VnKS-5Oe(#*U)w}ZQ`J&j8Lz)2OT9y8=TIlxv5gD0{!X2-s5+8=d8HYARSo7r-7$aXsO>rx24qOLNE zC9<`(j3;^jl#bLcr_SbQ2;KdpPMf=;`0wh(*1vo#AsEPG6w!pC)xe57v#XW$_as7; zy0r(14@RT%q<^LFV+Y)K=yf{0kE1+Z?CdbnSU0KrpbRY|A`nPf#f+RbD|d1e34)JA zLJT)M!0m}{&(8@6arbX-A`dEs+Z_Ht;EflvDY8F;Xe(e6j3`b8A|h&Mm)xr~E`1cD&19BG~>^tJ(lm6pl%0=m&cG8ARIjMyuHbR7#c? zi-Itl-2oRFteN0@QW%&>w`;D)lxk@>RN322KQbOlzqvj>es9Zwygav21-nk=)_X~1 zR7JQ|dfhJXYwngfRUU|9Q(xIEZU98o6424qiFVa$+1tzA#?Qg4A9S@BuqC~LN&a^o zInqF+fP0?m1*wRgF=9w~S<4H%EYB&etSl`RU)LPhhS&8N^m-z#>QB7!|8{rt#%LK{ zcKCim1R^u_jr=vCrLSnXm;GZ^QDo%SaBJLk>VP^Pjo%=BI@3WCz!CI*mDRM|&Jx}{ zW=PaWN~G_WhC?*l7uM0YPZ1YAjQl2z{x*4H8JjqcUjbXk!HVK?0jA0-E8CrJP12kH z(%S16@-sJXNP~eWGwB8)peJ7D>KY7e&%dTmevkA`An%N1EmL4lFhkG$Y`c%=ao|?3 z;P!*HW|HbIDHMp|QV7yWDGPMz*1U+QtLO9WNyt>^Z5+qhdd^hgML~v!s1CDjlTA?S zZG`l&Ai?CU6wHbl$y3WlM?r@)I)1qQWD~2WxJy@!^br1C?)k2|i`_y_Y18=S@q%5~ zAJj*;+x2qibF@aUlK>O8)~_z}gh=t%lZV6%XW-7V)v9XEPLt~=yQiS& zdT?>X-XX=ld7ysu9DGbzdub33CQEFNyA(PVg4TFK3G`kc-HA8J@BZjT+=eyMnA|BS zMHwYYGiat%j$z|{I@vBLefw^oU|tQ>OsuVB#M({3m3+M zXgueh<8sNDME~egbDcw&h{m1xk>@~kax;*;M9s9UC?GYI?A?tbRThMF8TEB0&4lUu z>cLju0=MaGXOM>{Tz9Af>LCn!Fq_vVT9MEbcaja(kEUi9VGm#}T&sCZV2SrR{B%I^ z*>Cov3q9em)jrkYOWgHz(KMZERy41fAr0uFVo2ozo`>@oTX$}ZsCRa8TA{8IF6DMu zx(am0`A>?ox5FSK(k8Jdy*Li zvS%1VA5ysGpreJzNl%*rr{U5otI#gRqKM#UKM09R1D)#=ovxN#5PkKO#pCy0 zJjqdq>vX-GP%1nGJwWjwLP94#TOH56xm>G&{r4&PmIYjM<>r71y=~+lL!!7=XBr?ed^`KnU_Z zk)i$uvvX@Wl)$~O7mv+JL}_2a>M5!b)dpk zBdU(<8psT=WDe;%!pnz3K)$_s=JD%cqN0Wl2BSi5gL-2hJEgFN#gzsLOWnGM*={e$ zpPo@73sAFDkdU>FCK#2?=GBUNw1o3kQ~r${rWs>oW7XC__h6xD zotcs0w|_ux*zUk9!~_951yG}@ETkm0bjn&|@_-W*>GrUCs@a8H#q@Sa;D0ePUX z_k4yU^55E=#OJd%?^0KBXK&4axkx!taP0t1gm#z*-uBTqS}9~9Ki1FyXNE`&hu%u8 zf{>e$gA=r{7zVfvgC9Wf#-c&KJSqU16caL9RwS_>bU)o5Uo1?iih?}-#nFzu9`8&D zkhY|60ectODr7E7BsL+gBr4#EcXDj%G&a87#t75j9*2ed#{xE1&;PIhrdCIjz~c*M zCr~cotoww73`5$|!vX#snb+VKC^wGPj<=Dd^L04jD4Lf=G*W489}$tcm`jf4Hj{z} z4dT*!QYoRD;I@~$aQ&KQ(dH6vm8U;Xa*`2dT%ueFmGt}2o~xOaqm&}SkPbOXT2oi4 z8)*FC5B2+Xz{iJ&BtLnzadGL3%TAM%Q!hCs5X`0n?Dk1CmjVW#vr!AI@M;X|N@cNS z^}(bZZGG#kwlB3t~`P7X^GAnEs+a(USQ zYrKX&HX1W~_G(adKjW?R4l@ERwk6;S%5PtOGYtcvAv6ntX$;z z{9qhL)NHbu@zDlKpO8Tn1YnS7?d@+k(9zDNrnXG*Gq*rI(?2e>V68yb{*V%hh|er- zV8GL}t+T;AuaXaOxT3mc{jrwz)3tT3cj~Y4_A$C+Q8$#1ji53?r3ht5sv}_P98SxGl=7P`I zezz^X?%^SltC6A*s1fv^$l52_x}LSbEN}1b!^0wVXboe)^D7M%XHd)xiQ$Muh1GxS zEHB`6?|msOlbXzPn=V6Jxt4W(b(775U46CH{^!@Si-_Q1U;_wFzu{LGHdIdU+ifp0 z2@QKO1rL~&+_WSe8gz5@#nlj1A3(V%qVl%={Vx9N7d$R!CUpG@|K~j^c!-%jPZaY_ z4ItV+o!NqHlP=ZCd9Ql&nag|!kb2o@jzV;G2`}e2TABuyqnymcz8JfcVy6cYs+@q~l5W=Bw-(ZE` zJ}C9nMEtUOe9wSfpLna|ikM6gID=pfugvi$5a_2U zfY#>}SQ&+S;0Gw|2zb0-d2VS1+?p1e2#`?j?(Tp;Cpg@0G_xo=5!lw)qaAIx80AHX z^-(d3-R;ly`=rt7#rcD4ccK3pCWt~v5nCo8{#ce&^cufK*3NXK=I3sQE zN$*cL?TOLIJm`pBuQ%wLSLQ zoE)Pd9&honk!41GOJOxCMrT z$BT`o!ly0X$n=|awubNZ{$ikF7wgGZ#W$W4kHs1@_|p6Ot7m29gx$IO@peAjdi8lJ zZC6wwo6CvPeCPN5$UXQ8z&8koJ_0;bPZ`CO3yt6RwW86TchVee27f&FdbrTUW<}D3 zT>zAxz02>fIvI22YE9Pf{xySCPk^F)N<&C`k8163ZeQ8Y=hdbPeaM^f5wEhxXlAHD zctKs){+*MD3nk@k!s?cdoqYi>*ym%ulZ($EL3rwiTyIHU)w}sQj@-F&+K>1&)#oc7 zZkq`ztib}U)x-1k-k0MjeONwX!Jj8nxtxw7el=`^6X+kWm3-ULj>AwL9o({+jE*}x z%77*T4i0^4>MzyRB^cdRYnzj8rkV`WpC%_;q7#mo^j>B(#!z4Kv5=*5NRBlnaR3h-geRBdyQ z8!o@=8R*|>2y35D@Z?|F%|s-ygN^_l)8Q!K>E#j|Zwdd;+#iOnp>}99V)Dxss&Ki{2}%mx!JImm4+HC@0v32-@ zDrYVp#_cF8YJf^zA!jc=Jvj3F@`4`CS7QGSZY%$-EC$%audlBo2%-i?l4oZ5l~uGS zu+-PVfX4`yIGR@h44Om~5`z=qPtUNhFn=)*jS^SKCloZIG_1}wfa$n6Hj>Nk@I+@) z)q2jg8)M;|b&0`$)}Hr$>NDlklE~UyJ2Qw(_HjVZe?1FbARyg!SkK4 zTf>0db?u3oyI=P7b_aN&Q@y|hac;0w$a+=!xJ*NaJ7ErYZp|k*Sft!*^z@m35 zI<`btZ*y;5T>LRs%HT3$&F#4633eAmr=7qP)@(JP47I0OR&+zA2OPM0{dSu8sP6&% z3z04{BC(rM@cY-)ZS^)Lf;jx2?f}mRk1N4e{U0F5H2_|R#Voo%lr_ESyxA@Mtf=J=RZ(g;Sij0gDHC4rsS%byXP5@gG$;j*Fz0f+|`NwSNlI&GC z5*UEJy5oLHnyg;+UJOV$>yLu+4EF)Rgajmnu-z~2J=_JK00sd(WPKjbrTi2cj%wXb z@M*~Qyu`{?>z|ISIX6W+MUT%?ks3XYq`I^fyF7j)R~d&0gp%TFOM=Zl266r&W<>33 zct3z2ni2rA6O|~507(MKZCpFLfTb6H0Gq&eA-*_oVjyFVUAqFs@3!AmSY zTATHU7#hCq;1^h?dm%t1H4t^unX@|zG$$P!GRi(Tk<^^JE{A`IjbDFUP;Xv!2yNNP zHX9p2_*5QjdWLhbvvaiP;`tdvjwL9OaC~yoaa1(>Rzl7MJI0MLS&+Ba>*=-+L*;sk zpM*iL`vh=3^F5E{kcJ|z`Q3rd!viAhhS&rALa9STgN)!?Px}Wc<29;LRtUb~Y`qB? z5S>lERYe?(Y93ZeC>2@mYL>EVO%`$_X1QIN8y~H~SMcQgcYQD`hT{(km6BzyaE!zd zDeMooLne>gjQsjb7Ttl9WO-PZRv_Q87f1Ju-)DIy?W5hU7tjxpnK>whhNI_5C=^b_ zW6pXn8tKGivjY{8J?@h&V>t~cvnndBR;o5VTbe*%F)iURSlbZ3m+w)rSrK74n^6u( z@9BM8OkopQHOVdF8FYo-n{QW|X~f}2qp_O|mq&`knHeW!vN^rKwihictm^aoxrU^8 zANR6ZhD&O{tQ-KiEWDFHC$g0Bg6YW{-)nl^u0L5=7$`gI+KQvMZa&}0cU?}CB~iue zwEfxZZ}@fy042?EqU-U2C#^;so6T`=+f61=jcG&>%yf|yahk4{^KaZL2JxqndE4pD z@3n1f3vHvIGsxR4onOkZR`tw{v|BEx#&Cg1@LT&HsgJ9>AMy8|C*!X}#DnFTAI?vv2|e`5+LUj;SSDS}`Ai^J;3f zmzwVzS7V|CO;vd!6%i2gRDSEu7go*9B9Cu>V1|=vOjv(7r6tWq30uQO+J7r_>VH~| z1=ZyhTw1OOtB^H6Tt^Qn|;-mYi`tZUx z{Ps8(oU}G6&p`0bF~(i`KhD#`TJ8A#g)WbeLlp^US>%XiR)ejHIqK%=by{kglI>MH z0dy(ipm+V^@4dn0G^;xW;?02KpF4G;x3y$%u^55`dygSHsF17Oe`QEZkq?=*7RVos4X@kmXaC-ob{jMI)khy;PYoL)E!Srqhm$y_}0D0RM(p>Myi z2oDGP#L(+JUDEzhGe6K`F@Ck}-j8|TtVp4&i`C;A9 z064UCMq8>Mi>F$DNvIJB(j#kwTDbhfPA&iL6h%NQ0$Ox<u-9taUZXK#v;)y_`)|ppvme-fxNX`ZH1|) zmp4k-O~Xur29DK(G#SwSQwp360GwfwhUxZKY!LT=={FUb$3k=IX_$zCH zCYQ}@^+p)XsH~J?pFq3b@=xh>rv>c8$cPvm?t+>?Y11;IEMx7xKXSJW5;x~zGpTM4 zHRAc*lcIm@;-$Q5)$GhfnB*Cs1I>2lyP!ZW%78z^&7lb{%Z2zr5t>;c>2S4oi>2%^CCb%x~v6 z@DV~=M~?F{S76~-A>=!ODjJR9cA>)l%hC#0!S{~-U|X~iqz6o0MO#Thy=uM-+#wyt z3sNR_HpA&*RNhp7X>P@$--w$6gJI9|0RnY%svYa2H6)~DXs59Wa6->Ub2ghvu6IX~ z>+&6Rm6*6|l*y*6#`|f>7=68WHdY6cYC3m;Nw1rckpuZ4BTqG306j$x$j}=rXfnDb z`T?ATT}soh?+AkRW`fLN3lc+RyFo6OrH)Rr!vrUpdYDzlNJ$Geg_!Cc35@LC{wg|j zR}^xy`>YIhk6!7~fhnneM>!>eFRSr@Y{)b}k89xWQ}#=w+WEV|WwOLb}z z2-1m~84v+4#f)mvnr_ls*w9dznMR9)$UfAsuc)FDh@6BEqg1z2-y%Q~<6lVoGRQ(m z_ZjW(IMG;UpgBDl;v(_FbDQz? z)zx=i-J-rK3U5shyVua^b{t+`pRdtu>NA!6SWX~44|luD@hk!|`+MAW2-s!Dx*YfB zpqpfMvxBmttk`{x`MT?dVO#W)bwMf=)sO}f6p>Txe{n&)StSOq<&qB%*>2;2@y~b0 zQ{HiVdU%oSkysGN3o;e!T8?*Q=4M2#08gLee<6J&73-ATR_|-PkO#~)nd^X>D?w$T zXE4;}s|)Lr{W?&=rN=k=k)bm0*YX%eBa4?HW2g-zPG5+=Ef^4>K!o^H8w-;skPH8g zrm%k|lP}htpJcArk$1(-vC zW@v~rTO~bAVK$%O4JdkopGu~f8;S@vx3>C}fIv-MVhwig(>4+{SKWB#*D8?#YU*rO z`2*}MlHUyS!PHgYVn(|_OCUd}M&YYBBdDOR91d!)|J-cwKvrbxc)JD>qx#FHA|n8C zfeam|7<%VBg0spLCjc$m>8JUyD8IAWt9N_N^K53kKgd=4%i2s`5U6tH?TLqUFeu_e zpUAXOp_WSIoZvg|&!8*axj z99ArgA)aG}#J=<5VAmyCXD9~2Ip`@%b6;eThTnjtm68At=vfF4J%?|$8O!IRFmL)? zBye?r}z!Gp{kU8^wk}_(YMg_B*6_#_53m8d7C^o1Zr@PZ5!|V%b12 zfun2E+bw=BjXHx0Gn$8(@0*u2hb4zRii(>l!FZ<{Vpq(^-FsyGlw0Hk}&E%GMQ{!zQn$wn0W}9BuwaQE#mLjhO9#|<_ zwY&@*C^w6SGHnC$>+Q?^%FuXChqH%*bkbklz1{0aeatkxX_FN8bi_-OiP1*YATm5T zCdTw)Dg)TA3qD|R2^cSN&xo?1zzQsoATU31w^&Cl-_InQepB(Ipe?_Si7SNqBd81k zzA&Yvv?=ijIEjpgp0(<{n_|gHY_xtX!qD*gvmp99!bSkG6*kx%Tx%tea_&4F)FV9n z_4@X19EOP!!~srUMwy8T`eq9u7LO_8NlQ*A!v2g2?93!jP&Js4#)w!a&2w6&W><4p zSsGOVy$+9&CW%V!Bd{|eDK>(KDp!q##Cr=+j5yF@FEReNCQdUZ_>_ ztAoRm#JrM%&VMm={M2)?6n{R+Ce-Gvp6QJ_c@={opRidh8Hq>B8zg>cwGz$^XlJ>k zv@jZM3On<;-*M_96k@pFKV@dlHFW5 z0+#inowmJ-UOaN(PD}AUma(m-N8| zzqLgk=(5X2zDalvus7>0%1(aT+9FxO{x&fWVRV;OWtp4ai8Ys>q7tzP4TVT}MklmT zRHTk5!cxcz?fLxj3qX01Nf>Q*Q)N2iY|Zk(t(kYme~E#UN!;B>BI&b;PJE!w8RB9w zSqMiSd@u4VY$YQTaYSfoYx`t@ol-@BMg@yXhs{EkjX~)z&?X6x`&M$=eOg!-PNCE^98EQV=ehHrC=56uN0Hqw&r z)6h9Re*W1t+=2%R>mX2=h5ZRJuKsH0iYqk~7jg+V~?G6H% z0tvE%DRv`S;#m``F(b9&VQj zs%KkQF=pn5Lfh?@UXw%fBKQI3It&L^2xT-#@48hqjGTaz{eqtwi-6PH$=oNyGqNBd zR$Kj{gm-huT~vTd^S`pm9$~aP2h3N5rx)A>^um0*9p0RbAh7jXJ*A@&Mq`I2Wy3tA z1Cy-+bQxp--Cw@=G@DB~HZo8zN?kP(`gYR|jVS;?17pIw;dbWy?A3FJrWTJh+&U#a zfD759Vs~)Wh{B}em4 z=EujsvR_5SjOSka=!UQm@e?QJ7U}CVhq|2>7DV1dUhIi*lE-~$)P3&7^LITJWW>g< zJT57SDGFXh7!EVYptOwd%kl~O3f zjHe)pNC-;t7SeowT?oY``8-Sul7;_f>9kjWLuU!p(zd`9;q0UN$C@T?C0-piiSj!|iJmsn6r8e1nWgDYn|jv!q3+Y{O4ljLzDz+ue)>w*034aRI+v1vJlT01_c$!lj&2%t`F^CLNkzZ0x6I4vaCh4PsplkTJn z*33B*yAMgktd!0l5^;}M3aekj{2q3$+ur#nCpOVLmIDfhC?L|CSu4@Y+ ziN7ld>zWs%3<-jgJ-|OjZxQr<-KOP!PfOQ!wIDHY3>V)SIFZ_5fT>733E|qjuAAn> z@*@4Vw%Z}P!sADTKey2c6TA&a6DPQs-^T2REr>aDw%LQ?9j&)WVkM1;08-QSuw9RQ z0{k~9J7_VC>T^>lG8U~?gZIK=8c7S+-ENloBy3xJzWUPj_F+6;is&Rwvd2OX)T#M? zQ#nF$D5i}--TC4vx{u;06uO}DZ@(0OomgGk+82gYqvl2oqNg{^w{LFUP-8G}Var3@*9^)dG z&lxNZsTsyttk{mD+)pHCSRb9O<)6dwkaH1u-v*3Uitf+0&vCW#ItLaB_Pk(_#@Q< zr-H%VmP^#6a5mK*Dz1I3o#gL85*Tuha z1pp#S>KK6P4zL3(+S7tI>>4er2L}s%eL&3OC&%ap=A~}jwvLnRV)EhyRq7oXf{az_Oo@iDRU<0i^4R9{CpXzBo1c60U@c~xuoi-=bM zl3V}q)Jp<${5SuQLqiLR35o%ffK;HXmJnE+#GK=u{TraSCf)#;kL4a5puo3r(PP{I zt%0?;iYrOtiIV}t9Y8@MbG=74j8gZEghojqKERJaU2cg49I**e(R~b#yI!qX9JOFf z#el_}^k8k*`zP&Vz{Hh>Ve*lHFjzo=f@v5-my1pGr-)E9WmiKkL3x0pRr<|`OHEe3 z{gozQalE_f7Dr+|$N3Q~<`Q#Ntysb(sWqhi;F0Kb_L~x8&+}Ne-JO@yHzLXMurw#y zy;Ha^%EqGr%1*t1a8Pn4;8WROLtVXMpGGXJr7l+>_@j`;uBSU0?*x&bF|AzZ`l+lC ztp`B~`}1Y}9gyWFjew;hvK=$7td0q_SQ_hj_r>M)jQ2VG$8SDZQE~nmv3s@CT03gL zFDJZf^2|v)2AE{?2QBcfQuL2$850#9LQfB3e6{IS9PElx^NtBIHce%V4V1ayQPvp$bF-5sxorYq-;waTmagI~aXe&K zndbb{j|valNY{rwiV!ble1wD2%FN6q5O+|b#bgn*|dvif(iMXevw*yw`2*_1f)yNd#3#wpE%2Q?#5( zSn%Z?X0ymM=36{=+}3>w(zJq7FoXtcO7m6n47~fXe26Vt}=#{~8$shx&sY3K4HE>x(&Ry^JCnf8gAT zRc70*#vuebEuA4wmSmLpf|@#0(%K;*&6I83VJp;v2HG54n1IRY3DYGCad&Lnwr#Ux+qTiMZQD-A`gY&<-uT}8-TT)W z$=;(*o!YglR?RinoEQn+7WDG~1AC?O_R|@71_${Iz-7QOREHk&$yNLX?Z0OeP%e>9 zpZ>s2quqd#nPd7#Lo@VVTbzKK*dmV0}Z}(Gz3pSJ`0f}{8}b-PBF#B^r=QH23dZyK4V9>zn)TW@0zoJUi5!=KAz*K+JawUk!Ha+=T}^t zcJiQ|`yby5pg2Z59%?DT2FjLsNClJsvyG$kZbg1yTI?JHVRup=N&9GU22A8bg>zDD zE7Z0RP5OEi558pR1IGRVmUzmrNcz$?du6Cs&!{Xw%a;<7>>~oyvr_C%7d+3h+mH94 zqSr#71NuaeaooxwH9{o8lVce|ntrR8S{+XK%acU{86-I2{czn1^aTP_=9X^rZ;_^c zQ_%-VhL3;26I9_>;XNk-kwX?-zNcxyFMR5?4hW3kwY&l_a?l4h28==y`5@2zAjc2& z%ItnH*)cJHFk_Efdif>IZ?cs>4-+$g(|PBab>aGYIV23ZQ&9U2v(@ft#Q3zaLbC}W zKbAhG_npt(8YFAccHcXkI()hIn>u6M*cg0h9D9_0a7vEkEb0FGhw;;Tk5k)u;W{e< z+ZOiDQmDR>QKDh^u%3}um%V6fs~7%p?3ja<{1dOe@kIL5_*%>17>9jf;FAd1nEcsB zfPI|^6x-*IhE2;;9E=)+Yt7lB=)9s%tq#|DvxY_VMqNd5omA+I0%mXGKCFZtt3g-F z$64NoF@FR7=>Xpz6pHxyx6^F1FtlOF^LQh@LvZy*QLBv+BAhe|h#-US$x7cD6w=dC z-Znk0ae`fK9O!fDArZ#8ty%}RQ#_SCJ5VTMgwn)dkiaMwNht{CESZ+}nx2M59F)H3 z^rj+Cl1mjc{BTc}#(|9Hc3^u?NiOAm(L;FVe#<4EU07VOuVMg~4q`C>`5mH~+mBR^ z*Zlhke5ux?U9OXR3AOw>Ly zSk#R7=_#AV3f_F)Dw8y?4>2SxB1#dP^*S8fYp_^1Ha>GP(#9gSZw+rx8;szna^bMGr;_D^CdU2;G_#RD2@{S@ed1U3(es8J1tl$4c6IppdTw+Gtfrs2VLLEaAO?I^> zMe%qZOJPUs*L{;=5`@%3E;wP+&etc=4>+|llm&Y*ouGYwH=ol2JU9lD3D~TP9m7m$ z&0gp(p~d;q5k|!}PpfzbOooT4*1A5}4VPnvKk3o#jM)$xsyLmaO1vj!8k1K2rhj~z zv6gv+vFTr~PUpW{p+Y}gD`6z9LFt8*NgqkpndWwFV=2qG>2(!g-&+y198JGY!^%3a z_o4ARo8;*TasNZIV8t%_3HH;H83ig1R)k6vTxIbWS6*J2+14^Zg?JWaD(!96Y<;9Z z)Z}9t?4>$3!5UfnTlq3o*GhL-TIP@WVtOskIM-&fKypYk1vYI423z&UbK{@;=L8+y zM`7X8hH#M2$&TCQ998FC^scnD$7PDR>?O_oC7G znqOuUxrnC`P{;hJZxRgL@U<#13_YtsN2d^+j=KA2Rs8+$)(h{+0Yn02IX}VOkshQTiL*QSUjBSlN=^1itazj)VeyC8 z5F#K916hCAZOVkIfy3eW0zW}&Ku$kkW){D`Cgh(IT=L6wK_V8H|2r}=_=O61`ZZGP zbfhR}d!VrauR!d2@6*9`2Qwe5hZpi@XxYzK_=}&SPC|A&0wWS)6NE5Ju0s6qTzntH zqg^j=P~u5dB$FhLQ2naaWXT4)x-hU2!#<=@IX)~#eha7_{#C?EyzrnNpe0H0)GqEX zF56E=3Dgx0aR^YOAJ5~H%PNn(O&|Uk=8sKACa0N7gaQ;?-4KXEiVRI#Xv|ws{OS9a zkD+G^3HnI7$(Ib$6auV?GuO++^=H!FuZ$b{<_o2}uI%lYrZ1Uoif6f-?-x81VSV=0 zh0I)p_pKJJJb)74YRbyWD8q0AKft1ysq)TR_1c5 zMf-?fkzyVH%($!fRUGR;JeK2aA*yLhXErK3HsF#rgZ6>D{26Km?4A0_-zLraZ;_WU{SV6)mewzj*-3x z1)UyJe?(A0xOfH@EW=NCZ}WXWnEZ?S$}?j^F+;Xm^7BO*FeDl zJeS)aR>A>thGV=~{ej+VmhP2VByc4$;tsI< z09*D;r{2mS0g7{!SF=6fa8fy8deH)DB6wd>PpSe8SEicSmWQ=*=4%&`*ii_ zU|P|ZBU)39>`HY6rW)$runz#EVw_eTfw*H(343Z*l=1K=!G~^>pGKvSW+K$uU^03- z^>B#75s*$nf@dl9n#z)=KblO#-=qs;r)Ymmw4c0wQ^vy1OKESZj~XMCPMWrL*^BXg z+l#=|c#Gs$AFQok=f(|37&oIIvMue#CfV4rcqnlUgIC4Ds$SfSGsQr1!jH@HX9fbIF#mS zGke}QHGyB|Una8n)SvDkd6Ca=jN%R8>&CD~c1Vrvzq;21``PyRQP0ASk`Rssr}+*V z$0M+8byxfHaC5N(0%wT=!4ABI;MYd=b#nuYasE@sSgjb+uv%X=aL252In^9WG-v0e zo4hhb8GqP|r#Y7e~OVdropvKfwn zZSB{+gsDoRkLSzF>eNxTiJ%{Ce2z5K8>>n44fI2gHg?h(a}ynfC1?5wlAJ-3G*<(G zHZ0}`eNA4EYpr)&e6KFTP=*bpJ0fdRRN~nI)rA#`)w%U_T*n9-IyNhFH9`+hr|#Nz zV|15LDLGGDV9sw>P?N8d1biY?q-bNNa;==gQz63!jT42!3t5O}i~Z+Z=sWa!oBc|) z_Yd&`Gk%4xg81%A>XIPdjJp+M@@Y|ZRYR#W5|X+lVxL*RQc<&3%(`ZOKb;Gq52)Dg z&HcFD-0b0frTcjATBzPO5{?NZHRK{aPuWkkN=>oAiF^hzXtA*8I=(~MU+mT~l2KhP z_=yurITwSEflQ>yuj3+!UFB1Q)7FXKa$A5!(Bghk|MC3x@|WrP?Y%-Dn{-}UUsoK> z^gv9p%srjkfPP8FvewS9LwS5F^VJI8TuNN%%=~Ibn8u%Qyhg%Y&5CGRzuwZNj6aQcHvjf7 z?R<*e>8CEsVQm|baJHG15JfNrGlp(Nq6Sx##)13d7JrgOuYaRIA;ryzEyJBA$)uu4 zmocnfwi{3zpDS!;Q-W)DitL_RuByROSL5r*S;r~7h+W)juG2;wPZZMmFb&luB_-wk z7U^xY9bsSoV-BsB6mffjSFp5*xlv@?ELR^UDV?3WH~lBbm*y+!19J?X;!D2Q*-Mx) zD~J&bfikvK_GolZ{pQbv#rqiR-@WXX+VnuS2fJ8Or0e(au5cx80l)eQR$ijeyWj(u=ivFs31BpB7q>wkA01!Qu2^>T6I1w zft2N6qf&x#d7o4YR9cP5@QG5Z(~)%wm8-0fsMC8vx!>J?K-uW^Hh(qYftVJXzS!VD z=d4PYKR{1|Jr8M2)Nv|y6{P(;TNzxpC_LvsKP}9b&o#3hC%0J2jd4R8e9;egBxfJW zTJLfIw_066)K6rwc0j^zPccoh(Knn?lejCl+-&4#<6v_=NckxSQ#k%zHB=_7a69(5 zk#z(mcw>wE##|hkesmW5tU#;2GbtEn+qg89@&Z1asd`K8MjFnS&0?&^r`ip+q7rW+ z^Ml&$-yS~9(Has)xbK!;YQrb)5RP?5dL=~^c?C~}2eZ?*<`iYAI`R#r4?9h(shfKh zDc*HXxQtO=YS640O^mvPrjpephT3amx~>L=%U?Hdvd_7w@X|6Z+hW{u-9|u5#-$mP z=ct#}=q*GV3Q%9rHO1u-57Nc87um`RQx4S2VHmehw@2DGdV~fqvJ0Velc-4Q3JO}8 zqsOk-D0vu@BP|7n*_$$-0%(N$?2LDx17L_@X8SYEm@O+%ietD?>tZdZX>GRTSaHJXL9aGdj|C zOw6Z7c3Sm!yqauD+2hu0`(TBDP=C!rX`ZL-FwM24YKP^f0QOePm5E=o>&}u{-VgO# z*`K6P)t>j&UK6Vhi#si2;)9?;o4`RIiEO?QB2p`7sCWK4!w0C5!v0VX#OSZ@{$zm9 ztEokeQ@xno{4%nipAR7+G&E>Zq)+pAZB~aQZRADX`*UPFFn!!K?9e@nbr{zthbpTZo}F=YR$#i#g6N#tf8 z^V<{g^r$#?S}4SbjHY*H&D6=9IopyJNA@iJ8#i4MX3j%qRH(jqBSYc}EwS|90ZkP4 zQO>041oZI9isu3F?a>t@4V1+R(O!;|D{eWuA~(c%pL`Iu1-SQb(97x9Ex83s18XD7 zd2({N4Ohyx_xz)_LaYFH^#&| z{KtN+!h*>9SeISI5B!H8MF?I0w{1%z;^Y6=uOlBgydc*#H+yCx@2^`S& zs({GMzn;SXy@m{KMFqZQRkpBZ5Lr@XZR6x*XWR0-3?A?j`?L#BwRrHCw^+~PnR>Z5 zr*MY=wY|cuSq%Hnaqq4O0Q$F>-!#PmD0s}#d~Gf)WM!+`N}Zc#$R+Q}#K%9%N>Ef( zL%UFEi8`>eb-KU5DXpleD5+>jJf0h-`O3Z2VNa&86*e;B=eM)6(rRAluWp{`{vU=) zO|*M{wzuhyY>`EA8OD0Otg;gEpn_CJVVstT-@5bK;=xKcI66}Ru7jwRz{25T&GjOk zBCaSxN_vzM$d@p>?)&KPo^Mty<{@puEl0p{ARwr;CuF3!`GbN1Wc>G+%QPM!Vye^> z*g=&8r1q<<&Xp&+Wd1p7;J&B9K603q3EE~*>bCwJ_FO~zl4%H%Nb-m2%W-*MiOVH^ zT>!O9fSGQN;4_Y=M=!Gn7S>^>XD*OKI@{leQC1?Ydd2qxG-Y=O5Tiqm%nE{X)fF`z zZ}yhui}VEBzyU?#snNKV3e`QWe>_j1FTSV$T6%bQioKV(dwT^5T#quP`P4JgfBEhV z<_0}?*t~y|VNl8IE)4-EYq|Ph(^YrH=uouYKespPnkdTp8b#`ue2)UV> zJ^}UsfQvRzcA#Jr<6Sn+aI9^%iqSM+NMdf80V!&1J~8V@BmJ}f;IJsjPa>7sl1TWW z5XP@zNEm*~c08l9%skLH{5H9trCMoZ9DcOx-{!VW0tzz3;|<8&)zR7cK8MxVl@&^k zukZV|uBMEinfa+W;K)!~#-cfgi-hE?O+Z@}7FPOerZy~0tU!p9DiVuksCTw^a&a}! z-nel(F+TCOo*2HRwH1(7-f}|#tE{Aj2qZxcfHJ|v#9W-4%PwtgX$6OXxN2(8t*tIz z65*e3=*DQ!FRv^sD>{K?g;iEG_l)kc(7(UCyT87-v$vm*9+K}>((2t@9Ub)-H3>B} z#iFDtvsfw&w6e02F%fT&x4c}#dCt_rw;+T}zPW+L{fg)%m*~e#0LN{Tl%xohnVL+L ztZZ&3(F~g8cy;nZzHx%NI`Ol0D}MJOI4|JL$WkNUIXzt-9tM*rvxf)mYa3}AE#YbgXRWO=NZV@@08?~Ga zUO!h`oBZ0UsAU!{O+`1~8o&O2?7yvK8W`Y0Ae|D}vME7Gxb;_^s)E6YM<^KYWAfpc zYiy(a%Ot!+j>omuyQsf%R;QM>fZC&}9d|P6&w;FgTJ_GG({zay{aRwSj1H%mjawTj z&B$$SZ7i(JncR-=rz~kZXmV4(A`UBVgDArX??&l(f2AhQ7dacKs9dpaT8ZkW@mW)j z+;8%!*V*3EI`DmD`zUL9)H_#Qn6!lc#JTM3VrF1Kuw4GO(qtXoAAS*qzl57GIX~V| zT57Z&q-nlrIXi+)I}6xLr+Pmyd-D-6b0tfQjXZiBr^XfOwTOZda2Y-R625(X^(gt{ zAFX7yQd^{mw^}~o=D)I5AU*{KcMyTg1~2@%%=N6Lk4_nQXOr~Q@++Py9!#Rii*FC4 zGPvC3WUBcR&F9Vn0_LQO6P;J!00)GCP?g#wV)|W`UTGM654-|0&2F8bynas7#%?{_!c*Mb? zfQ@W`*Hc%QS7v6c@bb{b_IEaqrBAsYH7L)Ip-YNoY=!v--@}Sz+@FU74<|_G7v|Ty zbd*ezblh%NFB|FJ>}amLhmf(c=|3t>fl8@|KSyJe@$q8?qg^~`XhjX?a(k%%XOsB5 zsF?fufCDPc3e!4VZO)~YmHmB-4dz1!G=O(KI|^2o9W5;b3r$^VEhRbq*2WW4-q+q~ zBP0S2`@Ez;K;d+C4FoJw z5u>5kTTbc2*jlNWW$UNHv^}3{V0MEpmmFOtH}d|bPuEJ0u~1Rv=#}*d@%XIz(l)(0 zyRuDs3)hV&`jT|_$}2f=(_*w`pi$0}ms0HQ;X2ye0eu>;XNk3v*+U|8L-qNca~8uJ z?74P8Hd!}t0dYg*e1nenVM_D1FrC}g3Lw>!m!XkGePPbF9$RML5pFt#io$T73;7+x z<3@dObd6oUFk2XnfCR_ma`eqf^XwdnHa$G(_;}y-^-* zOJ{LNPj)^HFj1RrZFM+*gx1!EmBfhxd9h&Hfibvte*Smh>@zxXSX>;Ep8lG=1a8rO zxEnW_A)QZaZCzD_ul?BF!@{uywk*g{%0~xbpIg{2wvZS}8O+ zm3G|XUZj~{xIlx_|7ZcI_|T!b);IU%*YCwUaAyT0UNn7d)vUWmnVk2}fz_1fqEx(^ z^-ioqO?x2XaR83USq!+pqF$INzc}?Ll(Bl<98fInV1+R?gG;c zp{(B5n|$4?Q#2`?_RKqCo$}t>V$Fcfr#A_hENV91#%+8d-$K?RjEf-<*H26txDu zao-g`FxWk=tG&JYM=|f{93FopOjU6@1Vt^@+7CT6~mEV{S*Te_~2^72Ar6=;{I{$QZg^JjScN6T;iE-#*s{mu7!#}{5~AJQ1R>wxa9 zrFqr-$MegLyz`|E2E315I_)eLeLW42S`HUdqS|9@DxRr(bOl=pi>E9JDS9B6)A zHE|fp!SOo?Ta=PXWeQvGQQWJX`KFJAk6LMZ;;mSJ_n|Y|YdesfZ>0^R;;SJ6R!(&) zahiJC&R%2e3>^P&TAjkeCYFFIMN7J!f`W$M*KzNI5!lT6)ZTEIVnW5k2{M{yXUx>q zdpszHxLm&;dMpmvB{?}djiHhL;S`#l7o?03t}GQr`EmL2(!#XP4*eRIv6`T)s6UPImGL z)><-lQevGp_?cy;)AUSrwR8pt5qLc1_QuZSWF*FCd^Yf>fOk3u> z58V~rfzn!Uf5Qd4ypA)bqNKFJD)JSfBE0l_sn5znYM)4Di4}qraalwru%0X~wsDbn z86o+{iv|t42nj^#RnW?(QkoYPd^w}i9y!xevKD`cHNr!&&UQ|A*0#QUK}SYE>U#X$ zCS%Bgc5!hL{~LQXewE0aTuO!*juNdEM=Z}43B;2H%W45S&AZHd*mpmUrsT-wk!aTs z7dNfh^|bSx3n@N6{suh*!FmtC%Ak;VTpnhnd3^d&`OX_o8wWF`^LeRC#5;seiw z3;_n+KUvY8FeXz-pdq-_y)@HhO-vDkl_Rn7BM(_gRjE77JJ(a=(yFHHob+1!rj#Nr zcB~w}_9*$o+En0YccLOi6f4OX9sh=Yl+2aqmxXcJ37?n=3JUIWk&H9ZWc0CL!AC_6 z8Q`@XNuDOobHr7(%BF%zg>biQQ2X~Y&eq7^AaaDkK?r0X-LiE4O91o)9l6)T8{IXD z2n8upqr#v1wKdQJvZgXP99H8v#*z7gRK2)gvwW`x`;|PwgHbq~?hE;Bogd$POA=|c z#hdlb2Ax{X7jqt) zT^v_UKM@s9aVH?2TmedYkE3)v8Y&85;S+;v-hR(+43Lsbksu?e2 z)*av5%K)CLDMWKhrI1s;}pmH`H-hSbBTPY`j`oLjH2N4*bv-~*pB(>*}2j-uiNSI5pY z9=|m*@$Nmc_PQE0Re$p4qZcYO{#Gv>4Ec*760dz^l+x*fGvJp+qo!pKtIlHoZ(qte z9A$|9f!y!v3MBuI5^#nA2VFej5#B60&GdA}y4xw^AVeG&URsR+<_|f?vjaR2XFH`Qr4bPp3S5pymC?!?8g@0?!D~v_h=ukN47~Fe@pt$4eHE!p zzP7C*PkZIs_3RTpkzo5g@Ew;UkY{Rtg?>2X4Y zh7}pPg1Jt)5>Swjx%o9|Z298!?;CsF30R1V=NX(3P*AnjiE!G6_hhqX?S8)_1w*G= z37WkTZKHSD?@MxGvo(ZL!X zi2JSkkl71_)hf+8tgU};5>7et7kzgu6N8^TU!t~o-==;&355+zrl_pJU1p7!_BUog z5$^c7hWOaq*@@`MJ-4?Nek(9Q#%j%Mbaw^RmU6In8!M
  1. =mM-*1Ylo}eDzuf6#I zvx%bwm37I3eUX1m(dOQNy~U8<^>>HkYQE^)UJ2e_Ba@QbikTgZD4ted)mPa5)t3 z;ZRz!gtZt(VSz_qba4I-hx~uG|ou$-Xq(_6quei69d6>9$mLJ+sqxf{$x8?zHvv>@+v$ zBk^fHjmaau@10{!yG(a4BSxw|B3*i;gaU%UQ)yzwtk%9jfngahg^j1$ih znojR5@wI=uxm^GdGs}v}%_ZO{lgt4miuxHP@Wg`coCfyq)>^(rkqXG`S<_oJoeAwz zr-y}6QrD!n)#)$h*dL;OW9-sg7c4W2Uy)55@Hjqy+0IVzN=OS9h!VrW!8y1* z9CwoEc9c$MPfJUZ-YV?rLH)y)X}5;*-uj4iOW-Xd$W>FjrQZI^VpoeQ|C~T83r3( z^9R+0lCSHNT(RpCcL~;a=zfV+n0yJrY=W$0fz`?MP=AU78_xErl-ZfOrRULQfup#- zs+C$#K4a1Tz5(1IjXPmvV{V9MTiZvu7L4jmZ#03enrjYxY-| zq4pDzXN~oCj#E!cDG^6@uqh{|I9+;L)~!XKL7!nQNrxK zS0|x2^L5O_Y@ZNwO;PthFr0I4$;J1u8`P7s17KoCg9bL(|yRPiIU_>}heD$!`Z9GFv}7 zRrx-GH}QDfQXRoH^A5z43ea&{4pAPi3eQ`=$!Q&X;7+fOMlJ4qNCz7<$pk} z)xiyrs%gI7yxfa?(o}{TH^QRb?ph+DSpkCl-5JjRgA4nnL4YiSdDQ(0xUwB2^yx^V zpQ8j%tm#my-{dMhEXet1ui~1&j}kWTB7Z;bm>chP{vtA?^MBd{GAd3&*%XN9=S$3{ zasM_j|H9{|{D3fkMrA1DKV9y&i~z1S*&d3`{$u_7uY`lz-y8%4TnpMi=Fq=xZjSuB z(Xn1=n5O=xw!g#u9Z?-3=YRgHf8E@~@HY`KkU=H+AKNzJ2T~B2)O%9@*j)bwZf~)Q z{7rr|+ZUMr$F^zyrd|93*I@n${{PLrtPKEHJ#d{XnI->I+u0BxO^NC)ZS%+`78VXp zPMtW-L1^OxhF5~ar&Eyrzi~e)L#Y>iYKK3w3hKnd2jFc#AC6&}w70fgZJY6sx7pL4 zYz4al=wwZgvOk3Io0R2Rrkg)p|Ml3c=>7thukU-igN77Uyn&17_CJIHY!?B6m0M-0 zX6@ILo)XPu<=Fz^>Dj@_$(aC#6au7RlF-b4>a~#}Sou-L^E0IRj$`ws>5k28)|= zAPEa8{BVRvFc{TQQUt*-6BDL*I#M)zG z2vg#ZwglSKZl)(E+ZsoHGkn&2Nv8w69KtFXSBq*;#iaggF8ZGO0COk_sWw>ja3W(J;d&zl$;s2x(@|2= zQnI2$;V>I9UnQS|sZR=nsHn6<(h?r;= zp6c&F4v@%mq^t0?Jv(^G*gk%74uPl+Mo+y4A>v`9Ogk=C`WF~tSUS?RLqI_Bz7P4; z)xfAXojXd@5lbKIiIBd@X(}piPHmm;j&GG07tdYY46S#>=FD%T@m5L*DHa?#I zNN{q+j=|61(OlO6;PaD{lk?27KvdK!Y0;%cFSNo>fzi=TWpdDwNx@hl^UMW@oD3xj zii+l$$3nG{xLoD!?K?x$Ok*5uy6R`DxkMQ}kJ`qmLTGd8!?ntS#7seo35LjSZf@BS ziptYRYH^6JfZ`mF@x6gJpEh~lV39|MQ5=5Lo^N(h(c#>%z9>9Fe(>6eB5qvVtgMs- zEL$%^6;ZoZ9)tB%XZq#PxDPzfP?1C5)ln##NB}qldQv91c*S$s$ zpj$<2aG+Aa35D?;N|=P5J-aqHYzJ?SpS=ID(RuPB{36SqmlkGzx?gx%J+=H*h zNH8U{dwOzG8H?@f!0z-1S!j=D_1?hna3!1(&-9FBAH)?)c@YqTM8cc7w2;s+Rc(ey%In{8j|0zPL*##mB#GXGBv{ zmU24UwYhIFFMwh@Ml1nnBoNHzk?G0FF;?c5|3vcx^-rYE+^n*mdm*S~N3-8Bn)CIx zu_4siJH%>)9L7ruG6SptURT7U9nU|{$ZYD6hXh@in>YiaFKXBG&p|4$_^cxYr7&SN_w4~gQx z_7|H~LL2*v*iC_Iz1@@fvKN34FyyqvaB^}=+yfLrp6z74?d|h+xNQExq&AT(5#;Sg^`c|SoCSWohLR@XGt1zX;cm4L)08c5U^3# zi%)59yzLC~eauUHU#HERv!>yU;hLPPjiKKL^@9eizd=d+sFX1ZsY6J@A~GI{zLGG( zVRp}(VUSY1;8F#Od#Wu?RqHSQu3EI1pfb@MXW|H^a`6{@p`>LAWI=xqkATnOwX^_| zgZlozs70f`1nhuQP@XI{kx>913wL*pJnjJi_L2MXFuEukIxWj0V-!j1ZWJ~i=?_m^ zNelM!kj`xypOsb1rw%tWja`uCr8f z=KT|1Th1sMeAllh_py&TmC5Xn&e~VtT_hzfSX`J>`N@!WcQS1`(> zO0)iPc;0FJK4zTd7Q?2A^l}hP^WpcaaO#(V^P)n3zkXuh)m3Fx=bxnz-RW^0+6i6J zLq^n<{c}bWIdEo=_Ktxl{4nbgf{#BBP2TrqS?@PZ+TL|78&*7X{Ig%Ub73=6+f^^e z`x>o=#%xj5zUn}wcrn%D)`X#sEhs6^|@ZTFe8>nv?q5URwqjebP zwX($Ikk>;8T|;De)gxN;`o~-x+yenSK5#VK^V)42#hNov_J(i_TZvvdLSJYYO`i6b z&Iof9jB|3_23tvVF+gRoj;Jbq-rE5Q84GTs5T8>cA&phjTwi;1KU0Bc{BGG%OPFHV zGTqMrsi#(AU0hXJnPXPh%u@8$VfG_0F1pECCYvuYeJ?*^Cl}Qsv(=ZA4-i|!8yug9 zKx$fPb@Q;2#Eg+!W{WU&ICoxE;jPA;>t{gYV4{`lvGtN{d9u55NIJa&t<zEa~fAtriiy0XWs zyadtctm~VbbY|T>CTSYC^Mt|L+8e2isQHAGMke|p3ESkl2-yAPGR}tSVUx}qr3&+6 zHVBPF#5KNE=H!dV;(5c}I%#TBCKV>7xNR`NJ1wL3dj_I$ZJ^U8{pmJgz-Um zkEy>g=rA9&ntmV89;DOXdEBo5DT!kxffZmaa}s|bjv@Nf$)bF zw5yQT+5t#c>*qc7Y^GL;hCshRKkPr8DrK|$!(CWQRIP^uQIL?LYxIMvmn6G(+do+c zVXoER`EU>Ho%d;o#lv7$V{E=H}+>o=nQ}+Q<~gPtQtAdl+A-7!XC?RHy=SH6)*>Q>~kJK-ewA zR_ZbF)|=d$rra-6rn|hQ=X_uNcIY}^N%^{(iACTIoe<$XPeP$`q3-xWe~6E+)LLqv zwYUhj_8`m?u+3FhBlEUWlwh%1t;A(y82{SujllPFcKFlH`{+zWfR35)h2yjR)f7y? z4_wzFNaD_YGpL*V>Ex;QNP%TtGcx{vP1N2h?op z2HOZ5&R_4=3|*-G$Y9i+i!j|aVf=*8?qGM>Nvt%CaqlMNq#vv}nYq5s*>bRtt~1DM z%ZSOVYcx0X8w~Ofuj|d1)##V|P@%%Hk>S-=OFpkTW+i(d3!1_-0I0MA%#UroNcen% z*eFeAn=Kz+A2bw{uql6)xY*bZ&q0t^DP7b|psFQo7<9IYaaK`@tiZrxqy138h!vuQ zgq~h!<7^oC8Uc*o<1WXwm!Ad$(0zwDH=UpV*yaU^I@?{wr>EM|vckxY${cYHWc6x5 zyO9?F3N?xeViWo=3_i2OMnrv9YJJ&;ce>+E>M^9=uM2QUG%dJLUP9K7XX{*M1NGOk zvD<`MfY5~bQzs3kNcU1OP|GR_{AT`8-%+x z7!l_aE;~Vwx}oqsfLprV9}R2`7ID_P_m!uhx!*^0=VvBqH@hSC9#C5bP(2c`*bNP# zOmjKoovpSFCNl_fbMu6Si!W5J;-jHSN4DoUa5j#trS~k?k@|fSj>6+=vmLeiGIThZ z_8ArRu|eL3|9*=gr9Zt%>QfK{(Ix9aUc5?li9OVhZ)tM?m|#S7v#cs@%tbX3#;+wbflQuOv~u6`Pt9;O*mn!jnH5#?r=A`VQt8}Ko~`(s);EAO z8nUta#$sI2#QBQnQabaHMV}Zpz{$??_CXSt5$-PlBub62!FKvW^9G{vSPg7GGI`o) z+;k?6rl$>tCMG`2?jYAHDxg=lKB?X-(6xIz$;uA5A~@8zJ>aPMf^QaN1K`O;==3T?Z4c_T)y^JA8ZnTNU0)vYMi+4o7AzHk%VV+S zvoU70S-HxO59J>iHupRp6k}Bx;IUyjp-qgH6 z2|w(M$J!FPuik?JSf_*j(^fV8?{G{Ej0q@CwkDg04PpmcZEq;WRWK5Y{|~CV;mn-k*&>rgeAJQ6D;n#hC(BQ*eqy)xvS@lZs`l9ck ziShHsg=icF%kUcs3>`sB7c&5Ai{Y3P<3RG$^$3s)+}nM4IGcZS?(*iNqQYACqr9<$ z15NOt$87l+ie&;=2yRXEeh@)X!33>`z7Y2Gql90;A)jl0R+X2Z48_V{?QY#lsNU7f zs$$DnpQzYdn*jMB3eoM-@&!~5R}T+vtTnc>=H|88I3Ecsa+=s!lqXm6dokev&>k@> zFaD2Cp3634QWzj>Xl|jP25Tw$cr;6O5_!qz{{vXJT1rb(OXLTm2-raNU!pXjhUmYw z@@*VYn5K}c*}_m@cN1uNJWLV20VIPBesJHG?cO}^KLbqUrb9omeF;2W__KJ|Ps}VV zw<>nv-(_QUtvjB+tRUJ3iG_{I1h_q%*#d+YL4TzK8A9PBi60VxMaju#*Wwz7yiCx? z{YLH1FVsK}OBykBrE-osGNK2MmvyIl>UHLo33`=!n+>$+&FJZbaebhakw`<2P%ox_ zWU_$NPfI}VWWf>&8+&N5?EzmHr9g`cFL_%)laR?y*T{&rhb`B@$?0P2=)J1_ksNlu zX4iOGCPaYU@??px{dTzXP4w6_+XV>E|g}EtJM!L0zT-N=>{Er z4tv(l_T1Lu;^eTgQq7v@vaAuU(3}VcL6knRpqP}w_W^AB3Bc6I;3otLc!AZQVw3U( zD5NZ2*XJ*VIW6|2#6eaYt$5hC-Wm_LfggZ z+ULEar?>TLK(^633m;G&%No!vZP%RQE0I10Pl+?FVQCmRtFLOE<$7!=aQAa0~w>&aH7O}?5+6nV&~e8u4|Cb*==7VzB;E`ndsHz~dR5oi}-EL*aE2_0_=cVcexLuPCl zygd=V9kg_ zhQnh|LC!8U=Mx+1=aZ+y#BaqJH863lrRurPb~JZ=LN zL=J~N<_T4MBdJ1v6)c~LVp)Htx?AT`>Cileffew_V%3x=eNWhdQU&YUsJsp6edt*L zt?>NtQ*CEt<|7~a0J4#|Q&kV%ug`QTDIP)Tt&VUn+en$h`;Y{wtd=Lu*Sn%X%@;4H z-5GQDXOmHa|3)Nn{Q*Q$+0R|yWibXIsHHNR!uz2p#cT1jd3PAWK{C*I5Nh3KHGmpW zyshsV0Rlz<4ZHL30;f0V&t~m;_o6xL*Y8)F2GbeRRSgPd4|{zB?$3Xox?a!2?GNLS zWwczJU^d;UvoiRQk9h*gNq7oK!5>N7bbTm&?p)YB4Q8${Bs^aSAzWWK& zosJQlMh^$S1a278JX2U$4wdM_02tKsS+1K7q(tJjJn1NWo;?Du>y_8GqV47~ZZh1O z6qjx5$6!7$baz^mu4b$(@1u>USt8*GD)w|H`>&-F)*mk&K=lgoLY_Vn!9rJaqk<{T zxn@0R0kr=skSphVex&-s2nP1*%bo{&&M*NO5b7I*ybY8}ot)Vl z{(Uj1E1ZQaFA5dk#||ze?)vY%g^?`|?MRIfQwTg%bz}q_#{A8!kDKMhb13@RNNm=t zaRTn!$Git3?ZSMc?@%d7a@rnGG!506H{EY_p?*PMBA^;BR5N$ds$EaDG;D0Cra~1Y zIU7-!(;4!DB5=9TT~pOu#FDl`%#F+we%3>9!x@dIPtkmZC&>{vu`yFiOj14sxE6a} z@$-+52JHagp+^L{;k>CH^b?aL-RpH&oh~~slPgQ-NWsh!0#|dhlV{^Yha*VF3!nxV zAMO#A_~RiOtpKaypLpsoXqYM7(Yusu<{VjTuF^)q1|7GSNI!G8}fQws|hYdhpD-;lv&$a=R=J z%hxa%@45{BB#TYK@s4rc)E@Y@%B?uF=jMj?5e{%J-u64_;NZekj9uRn21COJ;eRsI zDRB-&<9bcL9;+nqz3-I)>=$M&NE!?jL!TzdEIg;Ft+U?GYe5KYtkwXcIpH4WJGaf% z|LEvDv{3QV`O!f}7DX^@hwmRoL$*7JSmgVHlqn?3*7IxJ`2J=5^4)1ii2AglIM%xB z>8w}$;;EO5Kt4UEhwA^a^_Ee2H0`!75Zv7zg1ft0aCdiicXx;24#5c;EI@D%?(XjH zoPIw!YoD{%uV*}es>kT=s;;{4Ij?C#5VuxkHuX@-N&r^pt)aCxg-ISeTr&of-rI@b#-I`yL)$aGfI#(q%71_3a&OljT;}e6;*I+kO4sa#Y9zRyexX2F*vJ|iIeXFKT!H?q< zK(DG8eV6}!w_ld;yQQm(DkC57b9fT(SV-}2d7fd2P|Cm{g_^%RIB3_V|HlTIq{Qw{ z3`oB`iLGQs1&(cp?d|UsF%3U=6c(ELW9-+YsfJcb?ztj&Noz@cvBZ^Mhr>WRgQN3+ zu{8$=Y3c?F8RRw4v$U8PB=IFaS+;0aFp?o$y8*&3 zk-I7ewg?+VWt_zNa&o|`sQ{rj>2DWLn5^(jP(+k!tNP3$n{CSI_c`ifC9`QAus6TD zeoUSM?Wq_Zbov^=>VFBZV z=sI|og!o-;z299Cj}cNMi1Al~i1E+_8(m)(`5VhZ4JATSY`O)1Y$3#nk-uYOjDZ%f z@^jRl9L4ot5L>rDqpq*+GPy9FgfJd7OP2wD21otyFjH>+J`$=DSpBxB*7J|W`3P{R zCFqnh=z`x#g>gwss@x)N+QvpU_9kQrcgU5!sHan>!XrHhaq#jIxO%Y~B-U3zh%Ixe z(XhB1(#_-K=qRNotq<(XOk<1fs*J5s#!!;m;RV?dq2>>G&v~C7T!b$sr}6L9Rya>^ zQ*d0VCyx*eZ6gzZH=DeW#9R0DUY|(^YX;sjEW++#bse{Uha-}zflr?cPILK zou$7OXdR@cNR7>r&ib%TLy=#7hL6oaM-$JDm!Iq&#BlX^ZdJ+6C4tIoE>-R3vs&jX3a8_dhdd|u)zYhdwrnGT_IDj zb*8GS8J#U;tW0ye1jwH}i61ZK2_(H&etKl@jyeGWg~6 zBE;+Auo^O=r@OlJT5>Bq*D|(ko75Y1XZt7C21Bb$({vAA}#{COw{(WoB2 zb!cbkw|gLwWsnlYmBt1C9i0Od)Ny~lv>Vv#`}-Qh*5;|RWNv7wtEO515>D@Yw5Aw@ zs94tz@!AMOD>HU!xP6YUCkxleK5uPZWtvOKC3X%)tD=8@c>qCJiT9BKcw7hczcaO? z{IJo7X#;*>F~)-;PSegg*IyHWl4QdaCY|@lCLiviW-f3&frzKVfQB-!v#N5~t&f$P z8-J6M>aqSC%t%R7Z<4M+EYmKlSS@VGr0{J-N@hQ9SncoKqf&JXKkbd88z#RQkiqU| zKGf4_&k@a&vb^Cpq|{sysey!Kp^nTXESeF&+scwQsI1ITC%^cWJdprrJld7B5R@I_ znfBjP-?11O8SZMIZ1RzjnR@rn=V9N{l4ooSSLJ%Z=GkGMSEA1Yo4e45(2_b9{t=$v zrf~SMF2I^;CLJYPGx=8jvtrOJMpK*4`Lv3@p~Qqir{4LC#0szzgKV13Dd_%{QFS z7X^m~UUV1IpM~WJ+J?czr!UZ)@2~e1Y*c|rxKnuB80t0P=Q=})fkd_kObY0_U(n3z z#1TYKex-Hx`c#&#XGIo1aTiHKmgr}Nhc5n*VH!iw{E2*~cvTPr9giwLyOdk5-QV@t z&XXpeSZ4*t!fB2iiwDoQYa}CHvru)L5zsdMNI$UBDhgAGrgd+K0xlTC&;zv7@Ow@R zHEN}}y6azh!)*7@w)T@!$6?W{V60t0`==nailSm;19fWDr;AcecaRoOAXikSgx$fn zawk;6Yo1eG$?xrW5oPn(lJ(N7JCM8Lb0ac}ul-HWey!*?KFzl~1!CJMk8+r`S~j43 zLDpn6bo)Y^0om59Pw-j|@9uX0W9b6=V0W&3jZf{2rz~~u&V+IT5qBwD?-9R_&Hl`B zq0q4h#D${0Y9QdZtKmF;`?-0SHV*bYn+mB7~6q@6f4^dQ&Sz>cI;@1rLfd)>?HG&~5{cg7kZV1$-!k zlL%(t`xAO__7X!?Rd6^`+h$3t604Fc)I!;Z3yi4Q#G3u1HavJ6-0suk{T>$3Y_O$xP?kC{|_*5!kz#y1Rb0kmY|Pxv=Ul^*i=}?P!YFyj zf_5{Emy|@Rp`K}PjF&2-ly5))UhBPl4ihRCZ;tb_R8%?^VZsqdowt{+mJEq6E%h2Q zdiC-7h6=Bz^9FFpA42%0%-n}S$-H9{9vjIQ{S-eBfwGyGg&LNlw`6Z2MSa3u6lLUa ztXN#$+0yoblD-|<`fCLW`^#2>{}o;K&jg}`9OieXZom75Rec|^*Zln|x@m=Qq+N`2|u%-9Y^u7T~ zP$xgW*o0cMZ-aytUg@Q(RhW*73Wrs5W+QcgoS%dbE+tzE{MumBK;es$a?|vPBMy@u zG3^&Yhq5)%gW>sEX~THILYp+!#re{Cau5oNdg(l)dahRHK?j;>v~vFwes=!e*gM;5 zblZb$^Y>t(#Jr@oohrJABj<;MDn`K<^50)~lewH{CHCxMdVfhtvIyyRQ>RPGTnRfxZy`n~M@aH?7nh-!hx1QT%%z>#t#cS(nDCM|PhG6{0!o9& zUWi5}u7JA{Sh~Gd3$&?w&d>LamCNQvO(6plPCQQY@=h*GNxgID{wNd~H%qC4>D`R2 zoMq`6(&Z&R>p3>kFdL(Zi)#$>qaY|ZsYbA1t?yq{HFdi^<7=X*xUdBeDHT(Cz?nLV7yi;c}rFra#M>*QBT zChKjld&C~qg%6LsQsjYNCq!GofX$5+?ErxedfmB2N+4u+U8)1c_H{()hMGU{E41c$ z(RS5^+HRk-CH~S^Uk{V>m)F)ZAe=|(2zpNjpIsvIJ1&2Cy@3sC{D!Darp`8@55bt` zOQ)d)ul;t0zSr;h438!G*pshc_qUA-C7jq#u$Rt6K{@Gu6j)nzlA4+-j-kJSxX)2; zaHZ)ku_j=(8{yfG_GcWX1WhMvhI?{u6;->>aXJ;}Jh#bOpXeC-YtX4R`;3(z@bh@z z!86dydVG0#j0MnU+%Th47Mq}~gzkoEiSX|R1hW`~5?n{6_mUD|%T2YR_PkOo*Vcn8 zHSe%2T)RI+w%()oC2rIPJaV0)yJ9XC6zhoWC37|xqJ#b0?x(eUtEg*uC&-JFwg2GO z^EE~WzK%l?)lgBZZTF9q8!7R7grz8*ewVKNw|GxT5~WqD_1KbR#6ma_Df1(PBtQY1 zGlxLMcFK4R)245D-?Ie;5!We5Pv2J2G~vffDsH}&3~7sjdPKadn+15x|Z-O3h7V`W0JxVq-Nx_1Cc;`svJ%v zb0Q&~7MkWj_FQ99!t3KDf}b)xXhj9&YCHU%QC?DFQs!R+(j*oR#nb``Su1q;Y|=S) z@43X5fIF~YE6JsRLWmFy9i&J5Dg>; z?VN8QO1vQQlXrd|6=iOP50N5{L(g*5*vGKt2IplaW`J4xz1(&G%0o+-$X6)J_I`}* zRsW_+^YVVSET+33fk*&tmSHi*Ri?HLAo0>e!FVC@%4}AUFIDW04;|^B;Q5(8v={Q6ht6 z9b{qHTDN9CTfrbNY%Yx-AfpJ_*qOLEDOn2yaT~@tWvMO@{)_!3t~@DL2s2{}@mQSS zxI2Y)^x6!>4w2C0#mqbqOcJ8eCY3s?fvy30I9KC)abac}y5Php<4Q(BmcV9)5Vud% z+qe>XJ!mESPn{XhznPy%1l$;y3t>{)UH)C=JJo`{co_9~nU*eSLkWhLok=@D@3YdPcOh`rCq>_ZNqq5*YXicwrzn@=lqcKa2%ODyJ)xI~9)1 zf!ue}^Aa;t5-VV9Ei16S1j^)(TJil#mf{HR+OSIT0x0pKtFq}{aq++zj`ZP$VAn$Ph6#~nUI9C ze$jO=vPX;wOlXI4TkhpdB?dIEySG2XNXhVg#V&GX)92i&CGZjiV zDA^!l$#BYjlfEhc`IA(i%70x<;~?jo5CsJ$K?#TDFAtIu!@r`~`|w7pT{&yM2@`}S zgYcyWkw9m0ITmE>D=_Fc<3!{0Arve))*X4E2;uj7x3@-JGT#&O5)KX}a9z%G4Q-e} z$Q$;azg~&I)K^IH`4qD_IaF4+YcKRi-_k>UVQYD4^$WrYP`Di+y6^7U<=~L8Ign`| zlIj(1030LO@Qr7^-;}V&0dnZ^^ln+9=uW#EFZY+{zn6l2N(`2jIkBU#6Pp5>5=++X zz(oFjJnD?&=cE+9UwBThOuVp?u_RV<(z4+j)Xj8pVQ=$iHC%kLUKg5zsxww|%Dj<> zxoN2(QtNAx&mv$EWG;8qqmWSA^ylw-JVCqTJttkDePt5-c%_N7Nz2AF9Q*#f>ZGHo z&_mJu*Y-s!|Fj4{)lY< z8n?o|>f3)6q#$_T-5unIy$T*yV0e713W+S((}6f_!4&o0P0cPmH3S z=iF_oqTT7jWrxki9>BZM+#8(|Q?Uu1QYu_h@g@7B#;wUO zfp)4w&E^JU&6=)9&`zfCI_8$dP%;US=lmmH!n zLyiN{oPZRPhn^u??yg`5)*_fP#<_Wp0%DX1Ou+>}1z^(Ho7Nie2}2J5jw4|yG)>9@ zY2hs4rXd{5-HtiX*R!Z3NSdbXD1{u+iYgN2z?_U}dl!4+esx8ib`Ar|;x@_^Uk&gu zCNwhTlB?s3?YzA_@wwe02~fil-%~6t`F&g^Irdw{<>U35ed%$tLMrW$#oe}TqoPNx z<9D_ck*i7)(jD?Q_@RZx*z`=rXyv-2&Ip+SNCQ4ff113f$u(hHvj*eWe&pB)ZRCzf z0r60^sgQF3rU(RsKq2OeOo)cSA|U|)5{Vb#Dst8tNRCy&LpL}`(eQwTUM^8lp>-85 zoX0eglQHV0#8uKm$rJKLT;?_k`I}TmmmiQ>hK_*|RC#-hWfSQW8!oh*lhe^~BaMB! zN(ozx5a@ykrCShTcuV;u_`~mU-0(c#D9((6cq#!ja7AKIQ%t#CfP~~bMKL0Sbi{vI z02anfBz+nN4#fTP&dM(pta7NV$kTsv*n{mX=BXA0#KxISP|Y}r9UKkz4F zDYO(78@$H!@7N55TNdl;%HDl_Q3H#sx;YjK6$A|$-BtxSU_^87(FbWnQ=1cs+*T$R zv1pka1fX?%+UVuCf;0k!RKOd0rE3$fuLXSX;H#vvf|_BIRJ?88`Mt7sSv z#l5}?XCFIRPcqW+1PI_zARLgDA2t54;jmkY8=g>sgd8|Vbl*oI`x~w^h!M^{2u~Jj z5~}lEkvf=?!@Xmfm6Az7qh@)0sm>ii?dj#&kqvixx=Xv0nwk0U3n03@BO6F=WEDl! zA^m-Q@3@0vx@`wYy784lz@icYQGQUPJpYO49c6ojbx%hBMr8rD#e#)#e`w!?Lc(e4tN`bcXxCd2I37i zwxnyQCO{uB&%wKDpsw1Xn;mndZam+XhPxw#6X}Dt4%BE6EuLjS4|s)f$9k`%GP7kG zs?p6d2~4+YQk7uHGIsiiI|w<(>!rFrq5>|Aoa2FTT*1sfqrP7$IkzEjE{Mw{v+}vZRot zPXF||KZ=L+Iho_eRGDIh5s)0VKLA}&(5f`)j2d~qJLc**#R=^|2d5LWDH_Uj^cItF zg+nr{MqSG&{xe>Ip$Z2p<2j0AP@T*~6FD1>Ynz+0dP2R_R}nN@$F=JFtbvD3{&Fkk z@16ct8}pl*>J`vJ@BK|qY4LXc&WOW0u5_iIXfS@(pLjaUS!rX*WC3=aX;S*N0N1AH zyldLqwsE+;G^ zdam+s*CN+y{O)v+g1E|_0neDM^QaubNX(g9xLe&|>db}}oudSe6bk=3x3+#qQDimb zWxe`kpXZ_|)c@;eG5-tKgBGXsuk`m)uZduWWGxjW4xa1z!*Os9ik^ZGeA3bas{%Z- zPb1f_nuy_AsS>IJp)~7=UkTcsz~YE_%dWDjDy!9hnqi&e+jRC=W-tX(nByWXzeHZe z$#w_$9!%46p``X!GGMewEEi3=T%m^v{3%BnqxsP)DJnEx^dN(qI6M@pSi~SERDwoZ zOl}-4j3^4tif3aklx5MFmR)rdsHPD2!*vM+C&OW98M-8jbo=6B!rjbN7~DWrF^^Qz z*h0lRFdDR!K}^CoYMz={Y*3kAJzV!rlQb+@@>>zZMCvxNar-wcr>k_r>r$hO!*ThJLL#v<-}Yw2q%5jIc;yPv ztk7lfuKM_KV&hF>>w9WOauCJXz+IrYc_$#}m0;zPN$G}=aWKqR=n;`T`B>tW&YRfa zA;uNAL?w@Qt`ta%W5#1~T*>Xe&MM%1xkKbqsqk|{XEt8l>C&8D;FWS@M>KmMGU$qb zeoRr&Sccw*f4W%z-E}MY*?-ah`8XLsM@PpJ^sDw~4LSe_D%3vRE@>2oibHv`B0fcw ztBpl-uxIDS`GnVO4|6vv+4;`A@craliHQsJ2fIRP76m6J{*=$C3nrbBF};LyN?eA#6kJQPnwJj$bR|cr~bn z27uLuknkjzg`=p-V{|IO(}Os2V{8nO0yamY4>rQCxsdQzVrGq67EjU|a=`9AB_K3< zJ^Z{fz+b||KAMz?ZUCq8z6J}%znwS*G4}RuA(4X2691BU?MYCcq%@>!swNQEQb{eu zUyf=SO%v;eBM!fJ%o2TTf0HsmVNR1*If#w*f|}s#*QtrqGn5&X#jHS?g)=NZPj%(% zBot|dOqikU>NFI<4vmlFcgfGX@o`*>L?8f4VUF^SKFWsdUY^bf5#?>@Vlv(&kjH0d zY524JXQdHKsA#9D`HYjJsGGGdw(xhm4PHV}xYLkW<&y@=U`Y&nF6>X7OhZHm*OY_& zo=Ip6j1c7$@hw$y6Vm+{_>Z+!6&o^f4+h@wHEcZJAq0B8T=j0((Z!w|*(HtMromG& zNGYrEkv?boGbiOQ9pBT{+)aPlgrI%H{70d?)hr}5Al<=Zd~Vi9!Z4MNkh>BRnrf9M zIi)J1)krGL(;{uIY+&y;NhXkwfg!!uklM?on6Mm&o(p_N#?1pl+~BOn%j>M*NgP!x z3)pGm6d0pji=9%#e$m-lzL<8?CT!tn8B{Jn)MT!9O47~lgz--fGiy5a2*D+Blq6y& zww>?eq(Ef}p_syf>)OkKxbY-{by=4kY;fg}{n4hz;|co%y?_t4mQkzEOdpjMoP@yo zAUIvBs0RzHTDz<&XC7I6X=Xu-BKf(;=uSDiJa2A@-IfIzIFgqf3Llq5BPAAdz^LJ3 z{>wi|3sUFn`W#I-&*6^`@w4LDY#SxD)w|pSP&pXxNlnf+DkK68t!1`BHk3N!JDP>; zOn{XSFNnPR$K11KR3sZ5Ta|9}N>daeX~+1z@rU~baxhB2+(Aw3@uSkdeVoow3wvHg zYFSQsC<=UwZ?%2|Qoab1*lAq@oSJP({3ihz);EX6pINKPEcq?&3Dv3wBJ{6pqK3CArerchf`f9QXEI5~`$JFNyrir4 z;pS-s?>5qS51IIU4Hg?+T+~vi@_!GPwON<2E)Aow|kllc#frF#OzfEOZRe6anMm+IpKZXsf%!3!6bk>wo}_KfKITLUVfW1 z(~+A?*vcOh#<2TtMkSyBj~7v@nJXTj+w(Ymv*8|PH64Mv-M5rgs=_DX(#o*&REu?xjbn^9H4eQTm@_j1ueCkb!d8Hpmye zk3SrR`mash!usX#EuDd$6R$CWJ_QO?oA%eAw=Fsy>m1E8J>krQl<}yI=Ca0P*Gu)H z^-Q(T=Cu01&zPG>j-LoxYSYp2V9!5X17kxZvut&SHE~!Cz@q_^Ro<$4ME-YwI7)Xr zhhNox7|Epn=`r&E|IVRO*6B4Lwrl%W8!)dc1#)rL3 z@N(XMo)K{V)vBF_l2VY?r`8IhodaJ_uCggUqi?n>_I$e7N7)%yI6*#Fz{4vBS0kn7 zR0AGhS`8yYk4x9b7ht%NQ^Xf+s)Ce)>t%9pMcpsM7zJMx@IL7_YC~ zLbPcBI$V$r251@y7iG9;h9eUY0NBI5ssGxkwrXx$IrKJ`_ptUR(i^oc<^x6ln~OIr z%$-C!!1@Kkq;DX%VI#Dd9h902F_8Zd{IGa4N~Ivj{(pdTKdHeudk75y>8%wKoD*=e zYn4Mbe55lvxq2EDe&?7)VuqaI5s|GAnGv{}x0?yDRpR?gy(FqI)hf zy}52x?wu=kzMZ1PLg#;6cs~%2O{_})U9Sx!0Bb4Zvsje<_AgKL|DUAVI{VK9lP3b5 z@&ETV|33uT3*>*C$f7*jS=9f%pZ^1M?aqw{_;g*pB1Lq2WmAwDY)&61G{_Ee` zzl9d}<3ssB@5vV8KW1{?lHwnk|J%UF|2S645p<|_igk!B$Y=-ZK|kbYr-Q+%4(((2 zjgJSmjh$Tzf@JoLU97fxnsDDn$MPVp_QOT$3IawA)_PNODUAKG;W~NCUB-eb)$oQT z^?Ik5sTeXzFEUBja$8 zfvvf^p4#l%n)4w~tbQo^&NGZ$o4-rn56Yv9@YpOG5el!_dn_0buy8Tb-VjmI;L0i+ z9UUEnd=JStqksviy2f9*O#V-GBu@3rNdf};^XzVMadM?@9*%V}MCdk`h6Z|iE($jM zR17rCBtMH9M(Oa@G302oZ}8EN09>V^*h$t(`vuEJ@nLyWP*9hN1dP8at14G`fvL*uSk&9JBj<(&ebT zJNu5^RqD&{TwFb0M$A-F2J?75+A7-*E`Bmh5plRqxlH~PsgU}UR?`+PmM)^{YkCvy*w9 zy88NVEg76fC&J!S8}a794L|OvQzZXp2oS0r8auDtpK`{w$8`&mU!{NE1^><-%WO=) zbN}<3y$~H*pv!#Pqza2b(5b7Wq&od4Yx45S%IF!7%-slpz7zPYQ3G5lnObdcY?Y~b zGdBo)%R8j?&1pMX`~5Gg3(d_=#S!mkLzCu*=(^v=nzl@Y5p}r&aMik8pJ_U;A2Q~~ z*xNzT7!@8Z7Jho#-uL!D1;Ok4_`Hr$A5*P%Qb*E2fj{|O@+yFXKk2qQ{@5oxI%I-R zY1#_*QpxxA9a(ls_fdU|J$W+x0Ovt(a}aFdu{WF>h|xH~cpuh*rkH#?Jm&8nbba`(8_lje~Nte#zHtx*aPV0tx>Oe`#*Xci&pVWVch? z$8jxiBt^mmCYdD)n_xpf64w6b#K2dx@b#?#{*TIFSkSO;KcOv9#XvRayWHmAiYPp} zdd|yC$*78QfT5VS${H8fsU%}s@P1&Bs(2zZ3_Mj|ODi`mO-t8zeM%&uAbiYY{na2e z6ujEc8b=Vz{nz~5b?J#Xf!w5EohjT}(8dY-%^aprORWQL(h3Xk01jGytf z{vvk!W%Kjc$s`;Bj|VwCH|M48H=KdGoE$%&f&55oqVJD02?#Uc2wL7UWO~^Yb%e3Y zR>~mLY&=37HQ@l-3Y<=M(Tirnk)pV<1*yz^X~q(=VwlxP0X%_K@4chU`KkjbsMpn{ zr3{{`VI7Ul4BumGbM-Ex->3aPF0*7lSG6AK|Wz#8RwR(uVDWcYKGz$3L_3$o`G{N<-ZjZRPhhfS=;1QsU7gInoNH z+_5}t_~q~4zveEdKgL>uMSSwR7W~%NbaP&~n?3GuSyd2jjijWu?kaXC4Wzg|K3ERE z%+CBxvwTuy?EKxa0>Y-!PF`fu;!@}0<|2W|Np8PQDD>v*hdGh&eX#aJr7Yz-Q|chf?}i_cK1ReLTD@Hcp`AFCkqHv>=d2 z@9*zz-R74I+@(yU&_NJaS(x{&)sBQRI=X2doUOI-U+`o{eJVbL#3+a39-;#ryeu=i zuk;V*p&ycNcI29M@RqO&B)nn2{#Z8uL0q#Qwih%A&6rFP;FX-BqNuO0rOFnaVbtfm z&u?|aWM;M>@cJtGSMF~~y4QVbgX?>lyNe9!vIF~k`+A*grg(tN5Axds=nd(vTdZ+M zJiOHa?-{PQ2j%viZ%`0&5nFj_Aoy9D#kB~eabUHro`?>71;YD3_0E3~_oj3U{n28j zEY)vozU+q4^2i$l$NYpH)onrzpbj1@cWr>Pt*LSH$mh*3x^LH!ie6jUFvPRLi$))Z z`G&&2on~qtU}+Y^S$6J7ls=AO1)^ehmYFrMcaKaH6y0AX?q6GkQY}YO-$n3)K!@} z+oX80O$vEuytkWXt*Lj7EOVA&RIg5jXPKAG;nWQR`~1n!aB}V;<`+;1G|$gti&ewY z=AZfvWoUCt&Bhd_#KEy2x*L`q^+v^=wQ<r-!4yc_h4io4d`^um|Zy^kYy*M8X zYg)NzQUyXg=d*R1dv404-x6U_iDyhqBR>T~Ntg=lm{>x?f)@L`*^3Gb5e7!P=eDZ| z>(N$&%n>>`{WMO0FL=b75;&dHH|yj8!=zJaYOjrZ^Ertsl#zlpV!fAG3!w<>;<--e^ z9S?Cn7LLpN<@ZP@?Ym8VB`PgAIv8w_Pf(z8paYcJ0=uLT*>&#K`FW#RZGAXK+diSvwU<$H<748^y2gLdjOr5!fkceGxxM{aQPJuAl@$;#Zbp(s@KDcYr=@W`FmSN9bl0r)N3P>S z(pi)}JQ+1K-l9NiI#md?!NN*AnM*t2`%{k%418r4z)VDGE@zZC{~*P?ie6gj2`Qqb zsf_G-xO3$7J3tL8Ad@67Xy56gqM?a`M{;s@vbM3q!=C+9UIKh)$yiv(d1!*`R^hNz zj1tasO-DxPEv>A+-;312!5})iHc<*RPl$1G)cCl5Ix;#W?8#^Zhk&6%4sFpE&&%hw zMRF3pxFXH@H0w)BgipUfgo#sA<46rQxRw7(NT783zHQWFOV=+VoUPxPHfvaCl)zTB z4WEjZ3l%wa!TjZAbxC{H;^rns3F6-UF#LVP>C5Zvk#a-Q%vIctw66hAQa6af3O|5n zYL=3jl>U=e9F`mNsCaebh}Ox#hCCiMF#u8+W#oNf?*M4jMMRHPQz+tr-L5gEi>D?r zuBed_wYIQDEvEFuR*=&4G>WY$V2~_=wk1uKp^lG7JbFU`QVu``REbnolxIg6Zg(P! zoh0L=Wh04%95MwxXaF9GTU@&GjFrb2fdWg7o6qS|#s|zbv*H*dRhuhyP2-)rB-HED zQhD8!kT-#;mExfIqUStYV&YaoP<_%Fbs#zpv!I#z{&a3Tgt(9J`cg6zG0j@vy4_TF zx&J%91Y*MD^+Q3GqA-=vUm%}iTUL}QaTjWcsgnuYiv0kgnN&3VZp4tT-N{L*PD$69>{*#+>16r( zu182r4B7;Wl9gX^>T;&hv{&atnUSJC;|RK0O~yeGzV&xosfB5yS4Q1-U1Gt?i}#Mo znr?1yUoi1RfxV@r#6lVLpbS*ogMj?|i@0PM0?=IHTwLK1q9UyeGeqv=*-&;rMDpxj z9Vsa7Gvvl)qw*GV1%H3iF2%m=UswD>p;`17=etmPzJcttoZQ*P1zkrj8d?t5x59u= zCMX}D)*@lE$KM$_G%sIgvofkhEU~nMDl+OPr*<(q5jPoMQA*`;(D3(UsGr7_AgX9= z2%nTEzCR?yoP1aC*^0A$wPZ2Qo>lT#vrUh=|ZBRa|vTZ||OK=_|Qi&wO9g z6*1{YK4jDjhA|Ry<(eJbPpx8OE-pqRl=)Pa9REfl49Z#>XA!~YqgW`#Nb_6Y>BW|1 z+~zKf0h16{j425KOK*`INSoZGYV*)BDTGOY5A}-ydL=b=wZ>30(}QU@k_i+@|1SMI zi&C2rsI?i)?d_g0h_9F;r{50F&r$G?@^f{h^l64O?5yp9;*QB^5o@1|zA(x-h`Xkx z(0HKK!?_AiqZZ(-wF)F4WwSBv6{5gt2b~@u>G?SOQgpuz-lGH9DR=qp4V!9fv6Q5& zG*~sk&D>?Wtu70?jr)JDXBQL*pEJb`FW@jAXcZ};0Y-*DZ}{rWgdl)ih;cUky6kbH|O(il-JTP8KIH?3#2En z-%6{M%8>uVKs+^Ys5vO$D8~^W_?p384yx2;dz8iL0o#?ckj9~*Q8qsQw)pv>q^%99 z$<4Duug~T3Fb-q+VY#IpTG-c)#9{a3YvTaUh%I!R&+fvP{I!z8Kg8gZqWIS=Oze2* z$?%7=4!tAp$ol*Mb!EU3VD@|o6B#i`)K&1CzaQ1W;kobTl;RjZLFMB zWDiw9$cQqFlM{v?R1TXN3vJLQz>i1|{ecO#Ag>*HY)-Vh@lBeMw)QNK@Y!)9_{TlhN~&Pje4}`k9-1-(>`A9Gq2rO<87&epJoJL z{Bj|j;qh!vy^iD#-n%h^Z%E{}8E3=x^A=YHpqqZ%%npnm@)dfxyc0$9EYY$pZjT6z0GQQAb5F$NJ61+TgUw!t~?K*5l#fu{Odkfg<{$OOd;| zo(;m1O!;_uk;~!4$$_SF*l*m<=JQtoWE}AHPKiXw{r-%_I}}GS zp*u7VGb!t8Dg4`==zTT&R2bFSGY^w7tKM~Fyj0>Y()=V?|F2lP)9giQ=}*h?pTP(h z*eKxhF)4^(e#wk_KpGRNn`={xi3i9O3ho!za#8rX{t%?2AG#xkaDl3U*L4m4eut|zy1FmZMA$55X5at3 z_hnL(c`@IikV;iKxTm$F; zz6Mv05X6W^T?7Q&#@#gk;O5VD2f>GfRU(H?CurW?+Rxje*gJ#Ii@4NOo3jF~+as`x z%z6L_bGODyt2vOtjQ`4w+SL0qC-*m!Nfe|`+V16_C$3ZU;*?urr#Mz-Ak5{(+MAJ|{77F+~&J1fj{dESfzKqas# zBjz6D+s;cq`C1Sh?!}&wvD&V_5>OO)23OAw<3RKKd~O#$1WBT9M15W5(Jd3`T3rFG zR&3NJfrH*A+DwOzdzj9Z8{1ieNr}k{FfKUSy2!(wlczh75l`{soNH!>2~+j2Y4Yw$ z_5FYDCnz;$8ZM$C?fYY2DuCQMD)26V&kN?89oXm|9{%ETFe|w6Zu_xT0vxr@*8J`s z89T$QnT?mW2zU`24lpnb0*?1v{CqnCK2j`zsGyM|tU<88(Flscl+Wk&O=)R|Rvumx zn8BGz?~v-iqYylV`(3hyU4jT`jegaHyI8l(frJ>0>#J>*BMhKMSL5dk(;gw;F@zT> z844xP=#JAByr}sQXpj(ZH!Dq#|OP!|F2{6hg_glrk_j)>wYTlMFgeW?g(jY;L#KkZ#Xp zK8GBr*X8<96`&On8rSXg;R}1H{o{w3+Wg!W?8i0I0)u9v>(Z0zW~-aC9qxW8ai_>zmy4L2UDKm+H{mvaE72A2R0iS`EL3AR|R zUFZCa&nF)dpCcih%jYz=#+?EP3Ga^&SCu5pCo+f?9~Vl z0zvm9s8h~!K}S^oB?!&{{h6qg!`qt}kO-mh$*v{&IsURyBQU1a35X0t#H$e4yB9p7 zZ6eX@?q=~};;t^juOi+Md0(64+F`xtQlrQ=UxqA>up{$aVPe5N&pg9oP!V9=E_M}~ z3;s-h3db^l`+bnW+R8I(kX3O0ddaP(*TpPLn);LorWCb~#E`cB?f!XxZt{DDKm%d7c8{?whF zGueph%4+k5P@}gWRGpf^{?{poIl2Au>YbR-kDgZDcA5Z=i& zUl=%@>wT8S6m{Ufx1<%{@Y|Uk?meITeBe@Gkl`_ZwY8Pmqf9$+`#MfM2hN9Bj52i z`Jv13gfz+s7R5RCmX+%Ft;SZupu|d}OO9XV>-!~K51@{`$jlaq&eRsbd>nNHmMj2H za%QlpnP9QnH(OJrtBK|rgZF&=Ys~iC*5G;URV%X%pK-`m)O`?cs03%CO5*HVWo}@| zZ)7L)t|Z;SFX4(*vzKK62GJS9q9FtUJ~iWkX_C1d$DY-~vFmfIs7M)-IF0&ko+^z_ zx%!N;^9>PKAa?s2^ZIIGL4`6wzugiN2$QUNRT{=g2E@?N9Pxr3Z_ON(uJ=YN58l4L z<0C7~zE6h1h#9%q5GY1H_VyFn5=OZSS$y8hGg?3v*qik%A+FO|Mxg?Kz3VfX%g5DF zi;q{JZUSF9I8eC^6xk9WEdcn{P((e*Bp_yx+@9k{{b-a7$cLHiX6T!&M-kdZb||I} zNE@sVdCxC6BT01E>gCqeE}FSoC~)>^x@}oeh7|hH&u0{HfB3W0s%vOzs}e&Kr}t^~ z%1Nu?J|(P15XiqQG;S<{>sj_+frBB|OBDCkAECa)oE23>7N)DrYUQyLwtpSb>1|`w zlTV>D8UvD4;jLTH9sEFLs30}>tE#L~*x?Pp5JV2=JMbrdxjQzAlFG`rMYo24U+%Kv zbBj`8%>Rls;A*|OiFVS>;qyab?%aDp=x4+}!06e7CjI?D_*HrC@4Q=oBsdVQnEOq_ zRved!ZsM)eoOkHuYFjd!hkp>2i8j;+og))(RY;)tl#i0pH@MY0hfYH<7};_c&efPvhrI=?@~h?9$h9&NbW zU;=sX71`dG_(i-@)TA0n#xnY8wTlJ$|FHIs@s)M$w`Rq*ZQHi3s@O@zwr$%^MU_-+ z+qP}nIJ=(byxslw`S-W}xPNuB1etY2(h(;>8NK;c+9>5>2jUsw2*W^a+3Mn z&LHzZ%}CC#d)nxW$KC?NxKKV5`9s-a5=_FR;+B<=doVFFGk8ulAn*-wbvX}vDZ%T@ z0KfKJaH0Zh5%DAbsF@|cjonD9LEtD+=JGrdv>Cl;=k>Cm}K69eRe z0e9dAF_>a7h}YTyPz!Zy$*H&!+rmvtBWck+O%9$KEfn%2{bG|W~9#=Q#>AfbeD$b7qe>)m9!C( zQ(#&vfxNy#WQPht8(kn0C7L=Ap6^Zp9qGr@H)av7be>03*~lD}&>Z}M@n-PE(w`q) zx_l1ztSt_=&^2}M=M+2e)x4rGs(V87SIpN^e^}dt{2}#J;?c_ovR2_6JT_FsucL}bkbcJ@Exd_VY)*liq zfb*3_h1*d_NN8=LTqBBXG!8G8iQa=mL@Sf8x*{5riL=!gQ`f9jQBDW%P0wQ8A{`5I z!_Yy0TsSduQm}dG{saXR@=Rb;wnH$dlE>pau>X4LitMENM5{rCTC7-we<}_UMHD9_ z)8Mguf8#WWKkykZ3IpET2SQhYgiySO{rj@P`6rsow^zw6q!iO_I7 zsiYzp>mVGW)TW&n3zAA$-^irZeGd~&@F}G%NZA7slCPs&-R^ha`{fsy5vt)WW~PFz zn!Tgt+a|LeR^Smp*|a~N3-dXiVPq{`eEnEJPNwFXl&~8o9l;cs!VWP^RtK_TV5iT9 z%-Ac%6vq8QEkq1w^9<^8A3R|7r?iM$SNxiOLJ@XZ*c1a1L3LarsuZ>;q@)Otk;ag+ zjWr3e@J!kAN?3Ei08kn5&)8$;PP@oup>d*7mRcEUYNVMC&E@zprSFqDds(ESK=x9A ziu(X5vs7Uej?MH0S|Hg%ei0sTErx4``THGtAT+Zhzi-+ zS=Ue~(~-@EWn!xL7X8eqZga}x{4;Ie2hcB7)ilg*tUkDL!lQgHL?>tjRuloC z#i=3B<7puSBLA3=b0%oS$8a9tO=J56@+WS4CG{$Xh-Rk~>VRy*n*>O8=VH=(Z?$BrSw-@*ZtoiZ4qVC`7knp?$Tmp`=!5y*ruB)yo^ ze(bRPObOPm<6qD5$wir3sQjzZ9C!ggw|My}&F|i1S6~joRS9fm^Em(B=H2JnVAA?0= zc@Du`#;h7x4-f^KpV%J3A8ax`7ObgX-9JFy*Ca?1~eIbMS&CFQH)!k)q2KoM3 z)%xyR5Cce*HRgA^!g;>0g(V8P9pfHytcWGxeucEbK4JFO%W*xQK%4jOwgdE~|w z#(7wX!s2Wk;#cz6(M{G5|85_4ImE@z#}Pmf)NWD?RdgBn1h2#zM82%{kx>{H&(WbkF$_HdKK1u+7-;tq$|TiFC13t+3eQ1hBTQ(TN+Pb{xQbl z^V~+{PF&&OQa zg__Z=4w0`=E2U>6Aa+ae9tlhVeH!^Z%YD<8OrcI>__K@|--2{k7s!dP6^<)m4i$dZ z5g(b#W2_QdBt1bH=E|_#-xSy zNi=igaEWXX{#46Z>*_lZF6y7u{Zln9Vq@N80|iqDGr92f{$S~4noPGYN{Z|(HT-n5 zcjV!K?kqEZ43#GF_lK|6my(;bm#K}Ym^^wg8Xa{%mM1mP zk#o%oux7?!NQ;=A@Oi!+rB2e?Ia(SmaQpo}GnBb9&+YoS9}H7m6Z=Gp%ILFdCxED?jx17Y~uF2u+_UijdQ`kSfh zrfMQKq{1uPGlG!ut;JMzuwNW~KXuo-PgP8#*L>NXiIHmc*G{+mJX%rer9^7V2w7T( z0Bk{BoXuB}lSe=&<*To2>E#f!`h3%evTHP>#8i-v%gys(WaE0d zG&MCO(a3`n>w3W*F-W6E%LvR71o1e2SpO#&|=}7!&w% zILgvMQ5sPMxNm)RDY(B|&<^{8BI5IUar<2R0jAAcz|P!W)Q&R(aqH?_c+$jP=WC>x z#grb2W-I{wNzw?iILq&MRdGdya>Y*a=k=s?r%a0{c-tE4{ILZ$kG~2NJ)xRC>Fh&o zgh~m$kItlQFf~!|5(lveO55|%(qjINba;AF*c&ScG1w5pR5r^CAO!cS{f+BW0hEn> z=H<yp22~~ryiw=*KfSy?Kd}7_5&wh7V$f@D0soiTwm8Y z(HACLx-ZylE9Uujj{Y3;L@86O*vC%qz^;tE{WO}pWu(2G)R)O>@vNWyrRCr(z1zfl zK7c@KG8b|t#-DG!-Fc{Zlaljo$qUW1{H!OX)}*S%{%(21oLsGMXN6REC-j^pt)RZKfP z41#8>+hNv9B4hviUbce$l*MbC!K%$=!>h|(<=PM;g-5!(-&_k@R9Q%*~^$q?G+#e*3x|%Sr&KwpJf0?jP&tx?sLi)YK zs=dTh-+&0;KhKP}&jbpaZ$HV|>pVz(j)*H4>x!O;dbj(FU-c!}(3MH)d)DIT2?h@! z%sIHTI92`~n8ijyfSuUl8pRPxX!xrA35yBixSW{hE1c2^7BXco7YxmMoy`!{i0$RQ zJUa_Vf&${@>&>>RI-sqYr%Ib?Xk+0(!1n{ga7l=Yp50=3+F!i29Eb>f(7*_>51;I;Y<*kS?6CXL z=;0lZIu4S^cO@E4;rdmKyXCOE54MhkG6+Y(|8jhD2dS0f_;5&uGm zq?O*}*QeEP9T~)OT;sx*dAUhqTWxZkW0~4 zm5Wicazy_N+ZdIg+2o1&rtW(D4=Mc%holQn)AhGwLU6_Xt8-_=QsG>I$4B_m40S|I zW~+L5IJkX8Dq&W8uq}D_Z1+3{nR74wPd%>mZZxl12=2A)b?+GtPE8k&S@jBhC#yrC z)ns~qc^T=ueeT?ZgxQ%>ls%(ehyi^=qgA5%WXIn00LpTf{1;y#a5dzjSWJFQOT3TJ z+?EzG+(@pQw2&&a&E?I}@xYv76jKMIoZioXLfBZQqllX1BwcAz zy0;h4dE;?Mlj|gC=w{Yq%J0Gy{XTTez-WFOlejyRTv97sFXw67%|6f<9og~Wo&JgM zTy;H@DW6nHkoNex3MSHY`aw|n=i&!9Di|yJvf=qe{#Hfk#WkBc!NE)BhB3X;Xj?88 zCh$k)T%h^K%a4-clSCCbR{hg{$gO%)>g$RM|KL>{;WqC-lSs)d`PQV+r^qp47>7+gEqJs7 zg2`!M55uR9pL_jA@C03|{#uz+$=cf>aWvbTHXh=C+fR?E0Z)K=Lj$K4XC*vF z)lUOKeZI$#pf;>Ec%0SS=!p&+mQ;nt+OmEh<5JyvYSP_4|%jJ*scvR?m2gwz?2-YVOJpNdK+Mol5DI#GC#D6p5%E^co z?B#`6-X%#G>P{ovT7X8~kW#S&qC%GOxz}c%xJkrVZXL8|9`>`_JJpJ$CTY*RGm_{z<|6BpvdOAflK@4ZEuXtSuO$sD0Fa{>OdEP$0}3W*)|E=e-f&EM#OWs^bW}sjzZ$pF^8<>Q&6vU|ZL#_qcrB8J_$0>?rSx!YP1AghAq~YhdiWFnT!>I0k{Ag|2=Mx}Dw>wTsaa@yT^dme`^cxm(KdNFxTD zlVlTBdtNC`X8FF?49Mb{Rhxl1(cQP=pNRn%;NRXttI}o$qlH)h0Eo7%Xi}#-laz;? zL=7;Xk7Yzc`sK5k(Sb5lQeP74iVdBeJbn-M%=F7K4iH&8jx(sBH*(!$_bqZnMtM=< zELj0ZF=Sls$VV?LU(O#(9z(!M*q64My}|OaVns1{$%GfaF@5g^KLvGxqotjm_n$q% z;m`(0Z+nL+_uM?+(5Xsz0r zS${{4CSaEPh@37~>BMWssWoQUuW!X4ZT8P;)~rGY?J{xT`3uWL(FVRX6Rl0^+PrGE|qmFv6H z^1~n>NS3*RM2~HAG zfHNaHh7vPtGASp0%penzxI$2}(rSs65*rTw1YHQYCSj_HRIhcEdf;@-Ky24fn?NDz zBg?ukUv(JBxdAR~PDSy%h0&hhb50y$B<>Vp5<|Pa&7xE=?ybJ3~Ia zc_@0r-OG?SgaV|NV<`J#3{XzC3>_!KT3o3OM7hnK*}nRciZZhz{DU#7JQxXy@QnNb zhtKQ1zv;Z#y|278Jsnk_XEy~arzk4Wx8(4+?|}Z_E|MX~jpi}MIa7JLZEs0Zw$8AI zv~j+8f7vXwpoF1j#C_OLFObBK?kv9n$I*qAYOlbhU={W2)C}@KW{HfB$5)70+ME#y zv*++U|9gd`HoLWkCP(ATN0fvads$QAdm(!r@lpA_Bq>~6T!=8;Wy6Ky zWbiSuI~BI~xW5GXr#KDXC3!*G$r!AAOI+PMt=E9&5lH2oOoXdy%&y&)p+m_LxT-Zt z8!brioi`x#cIkE`6+S^A`TNId<|*PR*d$uEXZQ2e_eB@0PO$~@ zMdS8)tviWX>~|%j_>XsnVsgO@{w)2N{enySF^O!uuSK|ehTWVM691Fo`YKnBFyJ#_xd5yswG@#t{N;5z1e2-EAxfQ6cBrE3&2zx`P&-#3C z!vFK9Y-)mGKddufWbe2jTVwnSrn*!YS7p3CHOmW)26iDonc+@PntM6no9)Yy-i3+p z6#^h(vJO!q*mljw?!DQ7wMMn|xBxeC@})ZpI3oZm?rPdj6j)F6%KiK7AFfiWsx+$9 z%2_T~XS;GC)4u#C-1oun+B!7y$+=8Mz!XPJ&idGb8AFr9#$&V-29lWJoN#M%8_{#a zy6zWz0Z4X`j>k6gx;5-l!i&&6Nc3{#h)gOLDrB&Q_W2t(Q1!W=#~R;E>O*3N4FRGD_skgdtc0fnSsB$*^aSFudRexf=@lAJA1_lU~Tf3~@uK&ud(=NGULOIM}-dApX8{a*3s+pv|m zEyWn1FMcwQDrDN`#gz@FR#LXb_|C-F#~TVvDu>Nt`6}8|{e!3SyUSz5$n>le@xAG3 zcj_qyLsq`?Y9{5*67wDBR!(2dF+y|F3o%CMY{&I_;kb#}K>uFY1TfkUV#lL_O8rOL zFmE7mkZ4GRX8~9KhbiBWZzKLAEyLLYlq1;#zh}62=I{8cfmmue<%W zAtl$ImUx&J)6EVC*h7|c0q*aKt~o`> zbp_B0-VKl|)0YA()yFC(`z-+XB4*9Ln>z#)W{gr30^AlCX1PTp)J`kEU^~W$uMa59 zR&HJqT3Q$l`HWbYV$?6p1qL1#X0Kigf>98oBPBhup6&p+)-DI`sPLK(R@C|zToT8= z(fO#ufe2SW+fXN7OPzw&pMeNTP!eWJU0T^mN#~}dO7Awvk&sz3BsB~SdE8)kP{nz! zUc$(T=Vo|Yp5G%>Bv4QwN?k9%gXXUXBHLVR7T}y=7|jFa-iQJ`5__wz? z476kDso$k83GCP5Vv}>j@U2gevT_@CByC44&k?(!_YnBw-+?B@+dhXu7Q>|{==L0i zY3pQHdtY0xWt?gf3)eZq#6`zZL7_07vVYgJ;FSE8_C7Or>~l}0fwU=%-h+YJ(i2U9 zcwA^6h=+^|)eTWqCOf6))LvR@M+|(d329nt_j0d3u`;05iI2-d2<$0v^U@pNYpVI- z&0O1WD)3W7{DJKarHLJ0`4u4nOE4dnFqdd-S3iGeTjFq+lU^PlS?`XTZVo&cE*=K= zHG&Uv0Rt)e6!9=FJw0-js;)b~qk}JO2P~MN(=Xz3sHymv%!F&kac1QhAx-`l0yKA8 zC@{!}xo8ON2M_{Ue3LNzVqP#OsS+utw+@TU!@yA>Lwp*122@L;8>pffLXBeO2n~Ph z-q55VKc)y~3AtAUX+1emX+5oWK9MCsb88VdX#5ib!j5HWSc$grOfyO@1(5Ct$dtEt z$D2e5qq7~tu$ja?bX-ZX!A{TjtQR2~c-V_DTT^}mC&-9ZpYMtX_iESyj(&;)X$GQQ zXv(!kk?s9R(`#4*ZWm@$h9xKZPF8Mvbm{3;%OE2{&4?1aC1{i|TFLdsD{CvEAz(;g zB2i=`JBjclZ*@fh2RyPSi{4CN%&gEb(uzs-=;xzDS|*yjiPuEwNEa3h5FBbQE;((X zYcNxPiWf2UI0^8bg3X!v4{H`+y4IVIE?n951ERuU%PN2Ov`Fw%F0ZYG1cByT$sBVx zo7oQ8^g?(Zi_IfNqOpUGLljh8GN>Dt7F;r`e}jv}Phr#IX}1RS4qSu)Mn8RUDtwa> zUPn)LGs33|l6qQ>@lAEuzTV?jZ)TR;>apqC#4$t(pT{_GkFiAckb3I-P5sF||u6g9z7 zFW}#K291cbh-nk#sD_^!R2CLz201z(%NXOuO9ZoaX4`4<&tCK>s2flRDe{qGf@fF6 zL~FWX;{|FedTu5M2a1-vFlnsEM72iKF6T@@M_E~3^6ztR&>i6VBg0vU`1s?Z&@#)~ zoFeeiI;Md*z`=$_Z;meH0BOa%KNWT}XK{Rp`1Ljgr#^lFik%6hND4*LWRR4jBPW+X z`|-F}7OqjN!O5xazt1js;9k}W1-B-V?a4bzKu$Z8vHSEQ#Uea_$sa;EWC!K{3*ku^ zD1J6v3Fy~MsxNXFD_)1-vD+(+gSi;wi@0(e)(efVG`QqXQ37b@=)CoGH3cJeZiEM7 zR{SWeIH+H^fp&Qqe^lD;a5P||546Mq3DGCxDr^SvDZ`eZ#Jcgpk*9Y{d0at=r7ZAU9-8Fos2p<^b9N+V(JX|yI z9xqTZU|_M1Hf@u;O`j5@#D#p4V=Jn_x|!Xpri_Y2w|?t^)JpF@*P~0dKS3qe@~2Ix zkkW-C_>=#$NK|-qVS1+3FD6S$)e(V|-h>CCYB*oSAW5sLcE_aaV{V z52O^^emBG23-P*Bcy2^YZqm!H&kqTSm>l;L1=~W?UQC1)#~a(P+uctQiY>1*lnYU^ zRt*eAJ7A$tY&3~KBQQAa0r0KSKXV?MG|NX?BMsms>jPrim004(>YZlI;3(7+0EU$Z zW!f#sLmnK+=ZBZ+dfJ@s{G4yKwJk5@(J)N9q|Cc5n0l%*=o8O9kqYgPO9)-d^@*4e zob}^TpQ<$FmbdX>SU=?W_p6jnu#JFe3LEFb$8ca)qLni~gPPCXip3JuOUC(rgjgeI zGLuN@yD$|^p=bnY(~=^Q+)E1f#b=Mvg4H3gZOW31^yHZF^U1^(olz}B|FcDT#)jnm z23-y9|A~F^GcCy(*s{rs_>xD>$@=ElZF4W+*B^}ydQDBFjR?gMSWM>PttWMEDmn}z zl+JlBils?Lf1fHf6<~53Hks-B{FPJ7 zq_$r)4mUw#jBVuVt_>M8TImU)e2X)2#X^-@qsUIM(pT{nE7Ixc#xLQ)j}6z*87W(% z!fkts<*Rhg6dn2SeN&UHI@?e9#w5p4KGM5xYYgL zVQVhnJbodKZ>I&v8QCFv@ z(c4WcCxZpM*^>kIXD+Fho&|H7aV|sLmJU@&Iy$s+(kN}Iw_|XP4kh3uQGOSFxK!<5 zd76Qkpt~QzVv35_7C3PK0FuTBKn#sUKp;22vQkG?)m6jal{cJ#>WDK?O8eJ}D-!Tc zx;u%=i1T4$WFj(yk!>=A>n5BnGK|Rt z0g{F8NMNsst(tQ~3m?__pMqN+qMIO++c(~BX&@iLw-CcKT|z(gLGd0UD0?f5gFu>H zLGr+kp{D{-F%`gL&Mgi}U%vZpWYCJAqR$^D%ddM;10KkPg(6dbCX~SVkidA;kDis4 zk%_;2bO*%kn3WM4r? zMGT9C_$3VC*Mh5@Q-fae0b6J`pJCESUUC15$oprofCVWRu-@BsHxGld(qE}K;Lrc+ z3I8G<{;!wa9I=4!>vA9e5&nO^`S-JS1b|b%_Wq`W_OIUj|M-$EUSMgU&qF3PGV1^N z2;PE%RNYTNPCtJA_o4qY60bW*z=V@Iq@7&)e~;@wZ~h(i6VPAh6U6k7`2Tr`*Apaw zs;e!&`3UO}f18X1f@)&_A4B~A<40iM@{0e;l(t%;*C3&}@Nl!FICE&Su)jN; z;PSWK+T!J*dH+`PlFyUS(ZJGdj{G%-E> z$N9OHxvRL7wROjejYzFej0?xw8Wzn{lV@gmkVo-<|7Y}}fGi~-Fl4T}@?D=)Bw8=1 z3l1kTSq^x%bnD!D?$hTe)|-{GvlA8M@Woy6cRx$xMSPw=N-QjJ92{W~&urJ|q;$uD z1ztZ_k@`^RwPmfXtXi#=n>JXDgW&F8BvQt6*esRBO>CTMV@*Yj zNGwj5MmI^0%v9r%^0(Z5q7&;_&i%hOind5Vo;ZN6H(@e+v%g26-`wB3IUT&03ZGyp`j5T24+_PCvy#fiExyal+^AxPH1xSgR(r*qfmy`A zv?&xP--Y>DZGP$px-hh~QtS72TJo!otcdFMpoo5Jo<%xv`vx zrcxP8+FzJER*;;Q+8VDEqf1Av7n^B9t@@edPR3TieK19j-qsShx45W^`|Fvp>0&+C z;~=AhPugUuST8!^Igp&n{5WeYS8@820-Ci#?P$#=c{--7FD6*io`CQyL(;JUhDsIPA`s-6>1v_N*ldQY81bx3_n9Nj1mw7M$S4 z9_k59)+!?r5_!_Eq%{v^7`k+HbO2bPin=8!<;JpY$;O13Ka}-z-qa{KwSO>*$XK>6 zS7}-!l4VKWaw>yhGE`Loc5hk@-Vcnf@lsne($BS0$iArJDpCY&)&QVF(7e;b)0_L% zpBgLfsq}iuYjtm$4{T!URgpVz#pnfRYJbvFGxefCQZ^TQvcCP^OE@hj+YO^=U{KAoEfXvuPNa!5NwK0cdlY!3iETXOOOWEm&IhBko8D8sw1 zbJMW8x_pR%esqB5vH&4gT$LOfs|MJ{fuW%#qobnZ1mAkNdjuseEt2z*NQ5oyg68S! z09Z)6IhTbJWP^jE+R`l%85Xv-T6$Y?nQCF`3{rydOP1q)pPxRP&JWQ`%(cJEl`y$c z(O4mTeRH=7y8zzra0PLKX8S_nDjozVRykQrOw@eCe~cPrF(*j6h(<-Ntfq!wD>p$`suXen zoLpFF^!~{Hv zhX;3eF*33w`rk74{y;wWH8=;!LjFEL{+$5K2dLYe(ST(%({GVF@_qzDw=|qp;LYO& ztL6o+q;R>fS$A!n^}#4CWm7XEP>Ra7zeqpN?}2 zQ-@^z{ElnNf8T_0~j$r*GU-|5R4C2S64gU6s*?Uaf$+haPt}rSF&yZ zHT;5k(bLS~wyTb>pVFsn`)*E3^;>M^7+C_j5e1w-ZzU02UEw!g^gjkCw4U;LVWluR z9S4IVN6bY+U_~j1fV6)6xLX+90OWcv?K+Ej-R?^j9!|*J-aMXLTK~{eY5LkOt75+( z$ExL26ZWFbsUHrD%e3ZjyoE0RZ7Eme9tiEc9^tXFLN15T<9=J(?Ee9{bcz-5Ffeee zElm^Svlnov z;3)g#a}Nk4OJ_>_8=$x+m{Bs?F6>OePwO+&YiL34;4W@&0Eqs$IP!0UFJsL*8Cm6vZTWpN9l z%mhN5<7QP;9*Gk18Zy;ryV@R_I7I4+LJkW6jWKdV$q%n)yz;%pN{EH%n^98IdcQ2V z&Q<*+>mQ=k={Bgmlx%%=B%U4f29fd8JAIQOLTUJCU)z$U)s zWAnpO2FH$o({491y6u?jJW^g7yUwhKRXKo^Jm+j_OyI-o&0mCKS`sQ)y}^+G5fwK| zg*;YViDIH5>imJgx+N}4`YMCdht$uA6Y~0r+zePQxiSoDd!ehM(6c)eN>c}+n}VI) z;$?QF)b5-!uP-zd;;#DKV?Y9jJr|IX@{2c%(_lZKMXJ+hb1|y9^|eo8(rUfFZM?g2 zR;(tI$IRuf7-8nqGS>@K|D+qX9uSZqWLpCLivq(;A)DC*AYCQUV*oI9xN*Wh_By5j z$s8gZ0I_+MI=`#ncQ<3v>dZEnRayk|-G^K&et=s+Fh2WGz0nE10Yn7#?G8g3V9JfI z88M4ECHS1*i!}dfWq@e|z(nbraXRh878Vy2%sNyBqRg)jEz#m*qDbl5+9Fioj5DgM zcP=+Oe)S|e;aC|1g0^+q?KB1R2zFK;;NU3ab>GkLE-)ns;e#(3m8SlCp`rbEV~|si zaZ9Ycx6CBc^>(p9RhFLFcrNCX#cV#G7)vMaDyEWfOA3^H#Gu{cLWiSc32>mxpigy& z0+egsIv(SS?BGLU6oW%48>Q>M+o<>yytbeU$evLZ7l(%cGLNxSHAtxZ#j-tQe%77m zB5!LP+z7y)OsG((&s*o>E^o5~@YACDkjr7V0BoAlU-x&`VUupB(>>q=V&3frTB@q} ztAwZZU0sNybb!{QHHWQ<$x;5pj!R@1B$vzkdM60QGD8G!G`}CU;Dw?I_`rh%$V%s( zzPme`;&?hA6yDnd2Y*G`9}+Lr2BHg##e@0#*40;kkV4hUw8%CpiV{Qxk|Xn6PIvrV zMrvZ8Nd}HfWPb6ibTBeLi!Mfl&<1B`t){CXwhoJz_R`OMN|RdU_5hCc;Wby>-{sbj zu-9V2oX)Sr>56?3k@i*2rB%Nws;VS-2EuieHjDKANPqX?n z7PDM?3@lfj%w)82GZBZ^Jm`lanT~f0I}L!J64IOE?ZCy0z24~b{u2s^bDwQnRg44h z50x*bs3sjz;OhmLgjomYfPma*0oLiD&u>Y|YI%A}zw7E$eu|-u8yV!^C8?>lcs6g7 zVze2Y1wk}dSEDMB$SuyUQje3hh-ozuh_?03ygp}$U56wS@leEy$3{ToaCuFa&85QQ z^G81=r+7OZXrCshAPBsE{7@MJL^YiUiA|gDMBlWzUfl!) zD4%PjCREvM*dx}L@t81he?udBgPE!tY`q^)Ys0*?ulZnNo&M7EIUvAPU8c!N#)DrgITo34-VSIXcMv^}L`gzFce+lz2c#pFIEs zG5yD}=Hepm-u>EEmp#!7G(P)#G-WRD5gjpuLCQ$#?54+C;_p-Jdj%Y=%q!og8vw)S zXj1R-+HL?tqWPh`>JN&kN>h_wOTx?A);sRg{Ne?KB9S6m2weD^Ti;?V|7Mi9$9uoK zp5D}PsiN{@7-UxU2eJZN@xIY?;8tB;Q z0$1}ByT0;KELbuyeOekbFPtA9+zRA#TlV)Z{@OL`nGKm|uwrm|#%{(EDWb(A1$)G$ z{0M*Xls@$n!!N*K%t0dJu~;1YnNcq!1FeUxPK=L}1bIG=ho>@>Ty40@3R{R09Dut0 zl<#v;@)HpOey*RLF**XEsS0Kb^b3XW`R5z&J=_iK&1GcP07~ZxV4ve*s`{knW;3V1 ziW4++^r=i9tkkN8Xf~Z-Nu&Ai@0D&x`XTr?ff%`%oYlUg$(H=Om*oUHVV6&@IA>y% zLY_Px9&X-3mc=2r|q4@ zT})#o_X`z8&D{QNhvQ-MmvF)Y%&H)hS>NA{&lEwiNqYLe zJ~Ef4vK4;zXLBcx&)D1@$!<1%aq^`|JBc(m#bTuavv5UO{Sv;KH@SE+S`LBQn|Vlm$ntNv+f3ecIm3FuV` zbiC5zdxngku#A+B!eT{1!UC-*KuWopZXBJQtlS@u{>F~tG5g(qp(H_9RmyF*?Iz+) zg8x~O1uRP;E4WRbU)N9vK$n7N()!#?b@8~r5`(<&mqh|7mvtslTWwF6(aTzX-?{&* zhKBIB{M9KTH$O>KGc#jQ%H0p;cuOr5b-pVyD>ip``&c?Qz~F=e=PxO#N1$Ljv2b#E zXlNQ5Gj{@AcN&-r0d}ZqQhCUH5vf~J7pbYNtCa0U)d2v`4lbfwC-N0o^^=ui?qK>V znWgy}iLfBmfk24ws*|gxd`%~2XX7gQt!PD#dR%%z$JbCQJ=@%T7pazz$TI%clAbhM zB>rCA%yGS~Bv((HHMPX-hig$;JTugFL7`p^%L~Mor^DLg2UQssCwpdQ zCabT$Eo9h5-lF$*>}Yz+GKMq=(nECQ!)vs|cdC13;yxfrRft5O;M%Da?#B8?ef1sw z?kSu%NQ0vr`R^eF;Te`KZv(g?fVe>Fh-g}MX@2JZ8N@$~qM7*NR7P-r z3Vt0<5R|f&xDl$frIj0Le~RABYrC{(c655FmBCo4vBq9eD2m4|_T;XfF1xy}JS<`? zfE48Ye4^+lk~li@*!gWpS7}>w#SA8s^>)e;#iX4qUDg~v0fW5GeP8NYn|A zpHTbk2EL&pp*y_yb5XF)-@k|*8z{m&*w58_x!xPKc+^dN#3X!rso@5|T)COo$<|7&uH7C(`-S2~BYWK?U|m+iA$QEQ#|XJ??Hw zvO()HS!X-+2>JF@~thN0*d9c)XjVA)>buZw5RRh z;N<#kVR!=m&;iwdWZ{}g$WmmukTjDO;O_wo0VS{;BeF-ChxnAvox{!D<62Sf4Z>?N z+EU|;g&PVw4w_e`Q*lF1KM_ujSSv2RwxfAt$}4Jls`Oiz%y9n@_^r^+_oLWJ$&3Iv zmIez0S2oxr2H0!u-g8{i9hB?q;{s08O{}-xeD8Y4cEe@+dF1Yu-vCB5?BV*>sSkIM zA9BXU8`MM88PPiCMTENPdl25ze2hsTDo2{Iv2hUCXoq`nvGtWcMRh`SQOj$UURMw0 zU~%4rS_e_#?0;3=-5~$gRyr(5fBirWnWsswYHyTJ9OQu}zDJdVs&O9OX96U&MLF7p zi5x;wzDEeW>}awdU~@T469o@aJ8OfOxCcU-CfnN$lSUS3j%uJ<%7GEC z@%~6rp0^@&8mkz~2{s1k?L{H?W3K$_VjU0=E1}*SC=^pe0*C}^SQ04x{X3bOexAu^r*8AJ0KtYSZe;hW-Qth6`lhB*AlxAqO)rxm@)|V1;hx9ut^ca>;%olnGL{Q{^{<$+ z(Jl)e&V3)K0-t$(9jwLqA^Lucge|JR6F1vAL;TUeD*;8i{?hT()oRC-+ypATwzB4+ zU03|rX+@+IxViV_sJX-*_1#2!7I=Y5Eg2WgfZzVkb^U(BWkqK%KE$uGJ>A}0H- zB-cCgqCeNWMZ^bt2v($tpP1oYvXn;y4XvFm`9)6{qvmuBBW*=*^w7Gt?qPmyEnCu@ zm9qUA6LmTK^n{2%SyU5)!j$-WW}J)vkLz}LntC3FALM>bXV|ly`^^J6yxpNJRij;C zer6<0E*tU6(rla^3vx*Gl*%F=tF!l%SGKJ?H8%(>G7+ujBl%71Hn^+zGK}D~8vqum z1;sfSJ{hRvr)Uj5IzD+&B58aRKrn*rD@EBBttu=b{*rRkzHs@o3wilEsOwsovk$<- zrq^y4=DcU{y?O7yAz3b@(b@eagfjU0O(vl8TGI92;#@CFi`_{Q_ipe7Cba>Xka9Og z%Dstx<;Ll_-prwg>nIDbzTf}G(mVVjU5)A5*m$1fH|2zc8qHM*aeRp$8WHnfjTg`8 z?v13%OSUQmZ0~+cQR_)%!!K%RI^P))NLI3YtdWL_3a`CGJ|rYBZ{rz1ASBTU2d9f+ zBq)*)3PmJPZBiXA92{0?7fEo4*NM!;paR2RA1=q`hZdDmYfZ;iuP8YVw2ObqAwhtW z2b9AfO1_}kJ+3$DRtT=;5L-wFZN_MZDn!qbJEg*qZE9Avq!&}jIj^OrntUic5>{47 zv9Q>%(_8K=7_iapgmWk=vlB5l zg>_KT@d^-3?g6|J!V(R2ZZc zQj(gQ;)Env4ycHhV~R5(j*tU`g7R{2_xruR|H1pNb@_<}YaPyKpS_=b_QN^PzAS$O zptsWQoNzH2+n;evZ+F&YqQW7vZT5GKBNiC$mj~@?y79GNe&G6Hi~Dw4{h^2P3wffa zP?k&k27?0nl;U${N*r*p$rhi5)o1H}%zJy#L$<|LLIzaL**{Mm{+(g!>JwrY#YVg` z$8w&^9U!>%JH54Y3Jx%y&@%ZF*Y%Yrk}KA`oIYOxAKAZlZ=k!B*7S>#u~e8WlEoY{ z>&UWHE_;z&-LU7$q+mFAToXn6R1<7WA(5ORc{jFfH}mC#*xt^Apg*J?it|Fl&R@~L zSVs?aJMLiYfc2b>-ZGePjwU^ozvSj9*;iI+96Wkl-Sw@HamG?lo)Nrm(p7ZV3~qcv zbcC0VRL5|yT0N(a89v(KS~2EG`eS{n96prrhkur{kK20>_*s%QrtxIb6Y(e%=>0jB zeI}J&_ilFD32u0d)$3QI*`22QUu6PN*CtOJn?x*_r``TK+U-g>(zEp<7xECPNLVVm zRjz7GUz}$syz_Zh5GkxL(MvTy!6TNCPT}tqDA|tG>P=sNFi)N34J+ z(hsW&80!1}eTV(y&6XXL!Drrv6%@|yL?J_`EYkO%^S`6^vBG)0RV6w5u!tK}2~0r6 z^F#WWiU$m3RA06xyxNpxlO`$&iAzk`_t`gL$9^%|4k>Lyo*9Ii@+O~?%hSAZrT3V{=CYQHf%>DtR} zhYk1qYFW2Vy@8ezU6S}d!Eb>JcRuO|VOz_7K#T2klHdChIvb$3ME|>-wEMOT^Js5yW%tFsI1mY?oz7g{H0Qrg&8gtWtvqReAkLJ$L zIl)iSq;``sk4d6%hQH;x?d`sWU%cDzl0F5myS0i0HPu~p__jjER>{`9)Lc1NbMloq zhI{+dPzOuup21B|FW~d!w_OoVHJXw&w@4}&ZB7-g3M(Lt+K{ZB(Xmk1eM%_?Ct3^w z$W=y_q|}2R_bHO%swu<=@WibgspE<8NATD!o5YXDsE!9b+j`%54)A=aWv;Ek-P`N! z3#I-%$r%}?8`7-P#F=A`+8Wf9AtgLY=0bSif_w*6dbRI1yO+3{dR^f;WPl&DZW|*c z?*Ut{#t4T>o++552pfi@XXz^|?>+@C=ClykAThn?VRVb`yHes?cI-WJA-1mS!fPxM z_~W3@Byv@ZySp6hE_DYVS#t+r3tnMnU}dA%6O&Ues5=wF*{!19hM87~mF7$<4lciK zWyxJa0w<2GZV9}^aurfKKy$hZkEAxYqt=ew@|5K!w3vJ$T|>*Yy+W=E>^OQn3+_O9 z1l8Y*l-og1A-`?XtbM!HK0PNxDI#EQ0GDiH8Sba*YbDk`e;eeZ%e@%bZJbGqX0%5k z22vB2^o`FdE0C5-@h9N<)oNXjB#T17N; z1z^&izH;29bU|4-u^C#d-A*1&!=Sv(nFA@laVajHC2jOnMT_Z6k>K_5?;^_~aqgD6 z(~VXd=@g6YU5^IC&uzLJQu++E}!6lkYg?)uhsXeM9F;U|2*@Nio*6E)h%s~uJ zH(=cVHyoefSvqY?*l8YM2wBGtcAcZ8hr8fFMu&G>49hVzT-x8g^>w*#llM*X((fwD z$}x~h+Lo7nu10IACW3E7;U1Q7ekAI-`Nzfh6|7KT0>;vtGEcFqRBnni>lj*6pl>C3 zu5c0)BUBJ}#SlSkhf;NCM@rb}k~@;LL^mb8{hp73lC$-}XMwEHnZ+KtgY?V{ctqGa z2&i4u0w#5{U;!>9uJ?4TcWj3ul@%ua)6x(t@3pS$>wh}f3JMU=B-~ff&GrOOIhjdW z&Uav)g#-bJ>bLE#)GkwC2nt zC}dljB1>F|+Bzt{*1Pu}ji8i*M=Om_ZG&XeB^sCVuDQ3O-Yu+L6ZKxOY9ax>?j4X@ z;iNXj$TqG$2Yb(bQpn5m0z*pSN7>k%nB|f6@sg!G=3@u{I8D=S7p%J}_xow%jf5eC z1ei$-O+Q^xm;is{g67Z#Ef_;6sw!yRf0yn0nI%ov#D`~fEy((;&uxKOUA#}`ztO6x zt_{kUHFo|PT4I~T@jIn+o}60u`DUyc8c?NN>E`@wU!X~<&FNDyPYf3G&nC0QQ^xvG zDk>8<)H_lGx?Z{#RF?pa*L6$^WC}r5s6#L3r<%XO1(x?3mwZ0N9JNM_s0P*+FG5N| zT?Lj@+1qp0uqP+_WRQwFmrPyqvR5)ljMD@2w{-y9I}^)s-*6Rg2D?+FOGC^#t4-M5@+t4Sjg$g)E9 zaQ`e?wx-k~IWKx4C`s)%Lj=XYvgp7wucOeEU&!hKa1={#z0pMI9kZ$-s;}z5HRHsM6fhQznU3 z9yFtjXYU?PGU=&v!`~};rD5ja9)MMNiP2V_ZSShTMpBztw?5E+0;E`+hLW|DVV@vv z{T+ulpbOfyoSHg`_Apis8X~U5ohxi*$f=9N1-=no1&j2i4t#$UEn!G)|Nf95cjx{i z^ZnHxHv1`@G}Cl{Svg&o)}`}0J%o2nfiluCI(G330|cz34lJs+m~k}`UL^rt9w%Lc zibQ>cD$h`n8dbgx2-AFD{F{EK3oe+>^R41KCBrx8nzmA!yzhkzuJLuohbJKb`%9O7 z1MMVA*kB)S9>UWPnw#94=g3&?JNCsFGX-nA!vW z3gi1q6-MFMq{g5@Pufxvb#7fiX5DTo z@bb+JA6y1uD6hznm@*sROwMXbX5heSjhqNl6QMD=$P-~vsfII0K1al+q!DdtBM$ZY5&E)&2Z$x>#gF&n}Rl# zo!S9MZb2ZlTphfAE#YNiS^M${e`Cj%9o=Si=jzx8{f*YtcSwJ>U0qXu`qucr-`+Yh z`rDUXUvHcLchEoI6v^(5bp#e){X4z?H2eDt7sQIioYVjI=)acY$k)hCo2uE9e#iep zs{c6qt}Rj45Bd-IygKkz*2c=CvtiB^5@aBu%GZlJQJv$9O)B9%Ho zrP|tJ=j4DMFqos6nRkJOr3Kl-%GLdbiy42hb+Jp9odj{&x&i&WFaIq1rlxgsexTw2 z%;AR*J5n*vIGkmGRR_XRa`&==KKyP-h_A12X;Bdr9?o?`1qVVPeJ|=0tWbJ7=Rd{x z&D=PpR{ujvrmt7$d>M-eMknWaiJBzz^M&{4cL}eJ z9QzP-;*DOD)2&ia=yDx*B>EkZs8L~3yjW;#(J~*1-oA&x=lHL!85Jhw68#QVZm?Wz zU7z}>^c3dEe6#KWeN(8?iwbm6l&>eQIVnjB0bT{#dV2%J+KFCU`)jb+(-(Ca2TF?1 z1704^)?caC$_!&KPy8Mb@=K6A8PAo!9?kRd3Rh?BlYR8ItnwMBC!G^8@(uUt)2Ext zw1+c2J;lYvZue+uwDdGW*W<2}z7<8^iD)f3HAVoV33S*wc5o06c&H3rS+Rw zF~LKgc`{m2blZ;)&Srjoz&FGj07q(I>tEhE5e*}!Mq8xJq=fsdfy1Z+cx}|zYG5#H z8VOkDa>BwZJ1A=Ka9;s)h~JW+Pv=!2Q5{SPumpeGcII%@^CUdJ#_F(%fkEUE-&HQd z7;mz2<}X-nOzbvirzVnYIYIg9MxzjQW*OtAR2= zxqo?#lak_^$l(!MWjThLk#{|WBw2NeMjHeYPUL+7bD37dgeV|u6oqs(r~n3A0bruD z)fOiJ@CA+ks&a0U--mU14<(=~g{U@*MB82}UE;4~xInAo=pe4lOs5Twj0kJ1T;vZN zc!@%pr@BB_Zhe`N#GH{eOEDZU1a(Jxg;!##_(Kqve#4his4BDO8{<_jG1I)qXxwo< zSS`Ehv`D`4cV&}kf#~Qbp_l4-XxcK=DpgWJ-n=_%x-k_!nZNf zHFi{L4Bgzg4=le^_v@$%7a*9Ooh5+SO(J$DwJjMuA942t z$-z|*I@g*$yGL1Jjnmdx{OsAW<0?w5!@oZp()tSF{rEbvK8NmTDrWX;1@r2xLn^^A z{&?p&-GXN1?2JZl*R9kb`I6L~*H%iRQ_&6j`(Vr8KdT;`VzHRhD?C94*|r!0fuIaR z=7wuq4z*JXlK7KQ8C7+SBxjsX^jf%NhvfVSWOPq_Bn(y+U^jj&pds!rtyKEpJ!jiD=xNn6$rsRhZPkkGklg@}blsXpKc-}IFTne?h5BZSZ6CqjKH zi(cc1DjPs>ug_yq0BN_)r`l+x?om)0-EX^zajHK(eT~n3x_dr~QDo~HwQ|eM+gq}! zT|?F5!RGDp!|?g#QNlXCG5s4_kw&WmM+O&Do$c&uV2tQ-x*On%i-Vt6FURRIwOWP? zHuJW%Rj};HLu7FTK9PCfmN_DTJ9NhwD)@cZA^C#<7ziHWzpBR1kO*~+SZ|n?O=hfL z>yPdjCTw8_?gU+1ZvL@;Wwyz*`7mxg0_104hUGF3{=TRIX(`FgmHs%;+dq_{INVn} zbSg87^|_uVvDEm4ZI&BcxliDGw!eT2cEkGl! zHznhv)jQSHRZU*L(g$Yj30mGW4Aa6~RypK# zf8yZ;ihwf1YMG`D9xzSZYj-QnGskgou;9gSSaRE(fSoC&pKQ=a2Mod@)>BKblPUsB zH2r(0n&T8|xlK3;WRrUG9_G*z%?PMVwm9eM?7vc38yFsB+~uK^g((*_pLcvbyJR8J zI5y&j9-m^?p$Kt=xN&<#aq)6EB>hnB&=6fDQi&+9;&M6as;b5WatIozPc~?rbQn+i ztVfxeZW-(=OyZjaQuj-21kE4mTSYl2y`}YV>xb&Iue_jNCW`P<77`MdFF()+Wa3^& z*1&4;Pr6=#B+$ROe9UG>fCpa3*IzqGJQg;b_hf9H~~@>C6sJ6u0Kok}uGNZ=kehEGkscmPpm zT3G_D0DzM4as`7kDLdB2^BI!z-;rKz6g4l0l6v#+^`cw&+eAlcZ#i{3alFIMNsPd_ z>_ZoV-p*BQqdUeTUC%GHJC^KTHYP~wn;Y1kBl0v(9wn<-B?=s)_wiiPR4pI>6tuWSk5SPbPPS+l3=rW za}+5#iAg1DSa5bPtK1*1hC=Q|pM~lj1Vwnnmj#IW9x`D6bdrl9(52a zY_W=diOhNCyawO%>DWI3IO=%X`gj`U=)1=3Ut@C)!9zmMc3B#|%xYzqQa)zfg)^A7 zPJd+bm%SV&7iO!1hO?TN3?T|_?d=S^A4!>z_H2dBww`({7#GNDXn2rsrq%M0M$;96 zZO7U}O~p_^hWr$ZtYcd;oCVcRT`2}N0`uHZuU_GVqxTlQK>_N^QNb~OkzP!Su4{Y9 z5SC$VE^Sq89Jc5q&fryKW>i6KM)sZ5GF{3xV1`uJiwpC$V*q%ddHL#Ptkfu6#xE?!$%eXS^y}o+e|YgVduqITLoC^7JgL zOoz;+oJ4HuQuB42B;1F^P*43KRRB;u)kTX{6gU{6J7I=EDgjlos>ex)#H#CxCuFG7 zTx~xgCT5xTt}$K6pVa~{No3qvRamQs1P#~lKbLgK*aQc1PlN}wjb$O|zSqcTJ3F0- zC?tvEa&r3#qmz}YeX@?glgA5tk+O0ODE4{NVo#D%8eMyppcBW5q%=ai3QJp)(7n^I z3eO+i;1bkR)`xPXVpAc6>`lCbP^#x@p{`3ze)Cf@*~rMqW{-FD5`IaFD=!VXwfK`; z{+Y?@w2mDS)7^GSEV_W}?hPXiM{QoKjXJ5c{T6s;ad~dgCf&z-;~oG$FD@SZ;FehQ z0OXGPQ5Oxj`LszkGFocGn4|rBBescu}E z6{vk1=JUE&H07cMSH?yhw^k6bSH|u|n2ibjeMPL8)9hFubmJ3un~BJ;4!hZt8~26( zPV--D{&cZ_ee?f(2*nkrr>7Ud@B_VfpmGH|)+QSxnLrgcj}Dyg311Ly-d8S{6jE@( RLrnBCHw9cNzI6Bg{{ZGaI>!J2 literal 0 HcmV?d00001 diff --git a/site-dasllama/index.html b/site-dasllama/index.html index 2780b653ad..d50bd44ff5 100644 --- a/site-dasllama/index.html +++ b/site-dasllama/index.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -62,8 +62,17 @@

    Local inference, written in daslang.

    -
    -
    2026-09-03engine

    dasLLAMA speaks - KittenTTS nano and mini, Kokoro-82M, and a text front end that is nothing but data.latest

    ▾
    +
    +
    2026-09-05examples

    dasLLAMA runs in the browser - the storyteller, compiled to wasm64, opens the new examples page.latest

    ▾
    +

    The engine behind the ladder now also ships as a WebAssembly build. daspkg release wasm +compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano +reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared .dlim +images, 76 MB with the two English phoneme packs, the same format the native engine maps. +Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. +Chrome, Edge or Firefox 133+ (memory64). Try it.

    +
    +
    +
    2026-09-03engine

    dasLLAMA speaks - KittenTTS nano and mini, Kokoro-82M, and a text front end that is nothing but data.

    ▾

    Three text-to-speech models serve from the same engine and the same tuned kernels as the language models, on the CPU, through /v1/audio/speech on dasllama-server and a speech studio on its control page. On an M1 Max the served 8-bit lane reads a real-time factor of @@ -145,7 +154,7 @@

    Latest measurements.

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - +
    diff --git a/site-dasllama/ladder.html b/site-dasllama/ladder.html index 81827de611..1b258bb599 100644 --- a/site-dasllama/ladder.html +++ b/site-dasllama/ladder.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -66,7 +66,7 @@ dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/serve.py b/site-dasllama/serve.py index 6c2192e8e5..f73495e8a9 100644 --- a/site-dasllama/serve.py +++ b/site-dasllama/serve.py @@ -4,6 +4,11 @@ github-star.js come from ../site/files (single source of truth in the repo), everything else from here. +/examples// is served from ../web/output64/examples// - where `daspkg release wasm` +writes a browser example and where the deploy step downloads its model set - with the two +cross-origin-isolation headers the Caddy vhost sends there (the -pthread wasm64 builds need +SharedArrayBuffer). Build one, drop its models beside it, and the card works here as deployed. + /api/* is proxied to a locally running ladder service (utils/internal/dasllama-ladder on :8201), mirroring the Caddy vhost — start one with real data to preview the live pages: @@ -19,6 +24,7 @@ HERE = os.path.dirname(os.path.abspath(__file__)) SITE_FILES = os.path.normpath(os.path.join(HERE, "..", "site", "files")) +EXAMPLES_OUT = os.path.normpath(os.path.join(HERE, "..", "web", "output64", "examples")) SHARED = {"forge.css", "nav-dropdown.css", "dasllama-table.css", "github-star.js"} PORT = 8932 LADDER = "http://127.0.0.1:8201" @@ -33,6 +39,13 @@ def do_GET(self): return self.proxy_api() return super().do_GET() + def end_headers(self): + # mirrors the `header /examples/*` block of utils/internal/dasllama-ladder/caddy.snippet + if self.path.startswith("/examples/"): + self.send_header("Cross-Origin-Opener-Policy", "same-origin") + self.send_header("Cross-Origin-Embedder-Policy", "credentialless") + super().end_headers() + def proxy_api(self): try: with urllib.request.urlopen(LADDER + self.path, timeout=30) as up: @@ -63,6 +76,10 @@ def translate_path(self, path): name = os.path.basename(clean) if name in SHARED: return os.path.join(SITE_FILES, name) + if clean.startswith("/examples/"): + rel = os.path.normpath(clean[len("/examples/"):]) + if rel and not rel.startswith(".."): + return os.path.join(EXAMPLES_OUT, rel) return super().translate_path(path) diff --git a/site-dasllama/sidecars.html b/site-dasllama/sidecars.html index 615a098945..4221acaa1c 100644 --- a/site-dasllama/sidecars.html +++ b/site-dasllama/sidecars.html @@ -25,7 +25,7 @@
    dasllama.io -
    homestoriesladdersidecars
    +
    homeexamplesstoriesladdersidecars
    GitHub built on daslang.io ↗v0.6.4
    @@ -61,7 +61,7 @@

    The sidecar exchange is closed.

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/stories.html b/site-dasllama/stories.html index 9b337dafed..beeb50a2ff 100644 --- a/site-dasllama/stories.html +++ b/site-dasllama/stories.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -56,7 +56,7 @@

    Stories from the build.

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/stories/2026-08-30-apple-m5.html b/site-dasllama/stories/2026-08-30-apple-m5.html index 61d47e42fe..68ca0ea610 100644 --- a/site-dasllama/stories/2026-08-30-apple-m5.html +++ b/site-dasllama/stories/2026-08-30-apple-m5.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -157,7 +157,7 @@

    Sixty-two kernels walked into a new chip.

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/stories/2026-09-03-mtp-on-mac.html b/site-dasllama/stories/2026-09-03-mtp-on-mac.html index f6a8943c59..e6622083c1 100644 --- a/site-dasllama/stories/2026-09-03-mtp-on-mac.html +++ b/site-dasllama/stories/2026-09-03-mtp-on-mac.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -341,7 +341,7 @@

    What a draft is worth

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/stories/2026-09-03-text-to-speech.html b/site-dasllama/stories/2026-09-03-text-to-speech.html index 04510e4776..4128281d8f 100644 --- a/site-dasllama/stories/2026-09-03-text-to-speech.html +++ b/site-dasllama/stories/2026-09-03-text-to-speech.html @@ -24,7 +24,7 @@
    dasllama.io -
    homestoriesladder
    +
    homeexamplesstoriesladder
    GitHub built on daslang.io ↗v0.6.4
    @@ -304,7 +304,7 @@

    Where it ends

    dasllama.io

    LLM + speech-to-text inference written in daslang.
    © 2018–2026 Gaijin Entertainment · BSD 3-Clause

    - + diff --git a/site-dasllama/test_metadata.py b/site-dasllama/test_metadata.py index 14842ea1bb..d93f082d95 100644 --- a/site-dasllama/test_metadata.py +++ b/site-dasllama/test_metadata.py @@ -11,6 +11,7 @@ # and REVIEW.das reads this list to hold each served .html to it PAGES = { "index.html": "https://dasllama.io/", + "examples.html": "https://dasllama.io/examples.html", "stories.html": "https://dasllama.io/stories.html", "ladder.html": "https://dasllama.io/ladder.html", "sidecars.html": "https://dasllama.io/sidecars.html", diff --git a/utils/internal/dasllama-ladder/caddy.snippet b/utils/internal/dasllama-ladder/caddy.snippet index d3780086b7..cb7bbe9e95 100644 --- a/utils/internal/dasllama-ladder/caddy.snippet +++ b/utils/internal/dasllama-ladder/caddy.snippet @@ -59,6 +59,15 @@ handle /api/sidecar/* { reverse_proxy 127.0.0.1:8201 } +# The browser examples (/examples//) are -pthread wasm64 builds: SharedArrayBuffer needs +# a cross-origin-isolated page, so the two isolation headers ride every response under +# /examples/. `credentialless` (not require-corp) keeps the analytics script loading; every +# memory64-capable browser supports it, and the model files are same-origin. +header /examples/* { + Cross-Origin-Opener-Policy same-origin + Cross-Origin-Embedder-Policy credentialless +} + # /admin/* and /shutdown are deliberately absent: they are operator surfaces. # The service also refuses them from a non-loopback peer, so adding a route # here does not merely expose them — it breaks them. From 07a4164d8935f54e1e76428e1007bb305a679382 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 12:31:28 -0700 Subject: [PATCH 32/43] news on both sites: dasLLAMA runs in the browser - the storyteller compiled to wasm64 opens the dasllama.io examples page Co-Authored-By: Claude Fable 5.1 --- .../_news/2026-09-05-storyteller-in-the-browser.md | 12 ++++++++++++ site-dasllama/feed.xml | 14 +++++++++++++- site-dasllama/sitemap.xml | 3 ++- site/_news/2026-09-05-dasllama-in-the-browser.md | 6 ++++++ 4 files changed, 33 insertions(+), 2 deletions(-) create mode 100644 site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md create mode 100644 site/_news/2026-09-05-dasllama-in-the-browser.md diff --git a/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md b/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md new file mode 100644 index 0000000000..7c34a883e4 --- /dev/null +++ b/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md @@ -0,0 +1,12 @@ +--- +date: 2026-09-05 +tag: examples +title: dasLLAMA runs in the browser - the storyteller, compiled to wasm64, opens the new examples page. +--- + +The engine behind the ladder now also ships as a WebAssembly build. `daspkg release wasm` +compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano +reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared `.dlim` +images, 76 MB with the two English phoneme packs, the same format the native engine maps. +Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. +Chrome, Edge or Firefox 133+ (memory64). [Try it](examples.html). diff --git a/site-dasllama/feed.xml b/site-dasllama/feed.xml index ecb7f0071a..82cd1f68d0 100644 --- a/site-dasllama/feed.xml +++ b/site-dasllama/feed.xml @@ -4,9 +4,21 @@ dasllama.io -2026-09-03T00:00:00Z +2026-09-05T00:00:00Z https://dasllama.io/ +dasLLAMA runs in the browser - the storyteller, compiled to wasm64, opens the new examples page. + +https://dasllama.io/#n-2026-09-05-storyteller-in-the-browser +2026-09-05T00:00:00Z +<p>The engine behind the ladder now also ships as a WebAssembly build. <code>daspkg release wasm</code> +compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano +reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared <code>.dlim</code> +images, 76 MB with the two English phoneme packs, the same format the native engine maps. +Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. +Chrome, Edge or Firefox 133+ (memory64). <a href="examples.html">Try it</a>.</p> + + dasLLAMA speaks - KittenTTS nano and mini, Kokoro-82M, and a text front end that is nothing but data. https://dasllama.io/#n-2026-09-03-text-to-speech diff --git a/site-dasllama/sitemap.xml b/site-dasllama/sitemap.xml index b855b18620..0dfdd433f4 100644 --- a/site-dasllama/sitemap.xml +++ b/site-dasllama/sitemap.xml @@ -1,6 +1,7 @@ -https://dasllama.io/2026-09-03 +https://dasllama.io/2026-09-05 +https://dasllama.io/examples.html https://dasllama.io/stories.html2026-09-03 https://dasllama.io/ladder.html https://dasllama.io/sidecars.html diff --git a/site/_news/2026-09-05-dasllama-in-the-browser.md b/site/_news/2026-09-05-dasllama-in-the-browser.md new file mode 100644 index 0000000000..7a4095d6f9 --- /dev/null +++ b/site/_news/2026-09-05-dasllama-in-the-browser.md @@ -0,0 +1,6 @@ +--- +date: 2026-09-05 +tag: dasllama +title: You can now run dasLLAMA in the web browser - the whole engine compiled to wasm64, a story model writing while KittenTTS reads it aloud. Try it out on dasllama.io. +link: https://dasllama.io/examples.html +--- From fb294dfac7e723980728ffa3e1e3ca756cc31e16 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 12:38:24 -0700 Subject: [PATCH 33/43] the dasLLVM descriptor census admits the witness module registration: Module_LLVM is a .shared_module name, not a module file Co-Authored-By: Claude Fable 5.1 --- modules/dasLLVM/REVIEW.das | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/modules/dasLLVM/REVIEW.das b/modules/dasLLVM/REVIEW.das index ee9ac057a1..47fe021b91 100644 --- a/modules/dasLLVM/REVIEW.das +++ b/modules/dasLLVM/REVIEW.das @@ -26,9 +26,10 @@ var private SINK_CALLS <- { "fwrite", "copy_file", "copy_file_result", "set_mtime", "set_mtime_result" } -// The require category every register_native_path row in the descriptor carries; each of its -// other identifier-shaped quoted tokens names a module file. -var private DESCRIPTOR_CATEGORY_TOKENS <- { "llvm" } +// The require category every register_native_path row in the descriptor carries, and the C++ +// witness module's registration name (register_dynamic_module - a .shared_module, no .das file); +// each of the descriptor's other identifier-shaped quoted tokens names a module file. +var private DESCRIPTOR_CATEGORY_TOKENS <- { "llvm", "Module_LLVM" } def private is_ident_byte(b : int) : bool { return is_alnum(b) || b == '_' From df4c5799fa3bc6f48b16cf17867c51146a1d23f1 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 13:04:17 -0700 Subject: [PATCH 34/43] the comment harvest and the formatter pass over the arc: four rules and four architecture sections land where the comments pointed, with their anchors and citations; the storyteller and the tests name what the comments used to say daslib/REVIEW.md: the JIT recognizes the three intrinsic modules by function name, so a new or renamed intrinsic changes its lookup row; an x64_avx tier reaches its narrower sibling, never scalar code. dasLLVM: ARCHITECTURE.md sec.9 (the idot family's three target lowerings, the relaxed-simd sign trick) cited from intrinsic_lattice_idot, idot_wasm_simd128 and wasm_target_features; REVIEW.md: forced target features append after the detected ones. dasLLAMA: ARCHITECTURE_CPU_KERNELS.md 2.42 (a CPU tier selects on the target, not the host) and ARCHITECTURE_TTS.md 2.43 (the two-tier phoneme pack); REVIEW_MEASUREMENT.md: a kernel-backend pin lands before the load it governs. dasllama-server: README.md gets its first anchor (the exchange policy's config path) and REVIEW.md the rule that no signature names an optional module's type. test_silu4_batch arms the job queue in its with_job_que blocks - the bare-queue tripwire caught it on the no-LLVM tier. Co-Authored-By: Claude Fable 5.1 --- daslib/REVIEW.md | 12 +++ daslib/aarch64_neon.das | 19 ++-- daslib/f16_cvt.das | 4 +- daslib/x64_avx.das | 4 - examples/dasLLAMA/storyteller/main.das | 88 +++++++++---------- modules/dasLLAMA/ARCHITECTURE.md | 10 ++- modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md | 12 +++ modules/dasLLAMA/ARCHITECTURE_TTS.md | 12 ++- modules/dasLLAMA/REVIEW_MEASUREMENT.md | 6 ++ modules/dasLLAMA/benchmarks/lcpp_bench.das | 1 - .../dasllama/dasllama_math_aarch64_neon.das | 5 +- .../dasllama/dasllama_math_default.das | 8 +- .../dasLLAMA/dasllama/dasllama_styletts2.das | 10 +-- modules/dasLLAMA/dasllama/dasllama_tts.das | 3 +- modules/dasLLAMA/harness/parity.das | 7 +- .../dasLLAMA/performance/build_tts_data.das | 1 - modules/dasLLAMA/performance/model_specs.das | 2 +- modules/dasLLAMA/tests/run.das | 3 - .../dasLLAMA/tests/test_audio_embedder.das | 4 +- modules/dasLLAMA/tests/test_groupn.das | 14 ++- .../dasLLAMA/tests/test_math_activations.das | 2 + modules/dasLLAMA/tests/test_parity.das | 5 -- modules/dasLLAMA/tests/test_q8q8_family.das | 4 +- modules/dasLLAMA/tests/test_tts_g2p.das | 2 - modules/dasLLAMA/tests/test_tts_kitten.das | 8 +- modules/dasLLAMA/tests/test_tts_kokoro.das | 2 +- modules/dasLLVM/ARCHITECTURE.md | 20 ++++- modules/dasLLVM/REVIEW.md | 5 ++ modules/dasLLVM/daslib/llvm_jit_common.das | 18 +--- modules/dasLLVM/daslib/llvm_jit_intrin.das | 22 +---- modules/dasLLVM/daslib/llvm_tune.das | 14 +-- tests-cpp/REVIEW.das | 4 +- tests/aot/test_struct_order.das | 2 +- tests/jit_tests/_cross_target_fixture.das | 6 +- tests/jit_tests/aarch64_neon.das | 10 +-- tests/jit_tests/cross_target_folds.das | 4 +- tests/jit_tests/exe_host_cpu.das | 4 +- tests/language/optional_require.das | 5 +- utils/dasllama-server/README.md | 5 ++ utils/dasllama-server/REVIEW.md | 5 ++ utils/dasllama-server/openai_server.das | 6 +- utils/daspkg/commands.das | 3 +- 42 files changed, 184 insertions(+), 197 deletions(-) diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index 173f4ad68a..218c3b9f19 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -262,3 +262,15 @@ a value. marker annotation to the instance it finishes, and returns unchanged when it sees that marker.** A chain of template ancestors delivers one copy of the annotation per level, so a macro without the marker runs a second time over a structure it already rewrote. + +**A diff that adds or renames a public intrinsic in `aarch64_neon.das`, `x64_avx.das` or +`f16_cvt.das` changes the matching row of the lookup tables in +`modules/dasLLVM/daslib/llvm_jit_intrin.das` (repo root) in the same change.** The JIT recognizes +these calls by function name alone, so a missing row runs the portable fallback on every target +with every test green - the `tests/jit_tests` twins compare hardware against a reference body and +catch a wrong emitter, never a missing one. + +**A function in `x64_avx.das` that exists to reach a wider or newer CPU tier than another +function in that file computes its result by calling that function - never by a hand-written +scalar loop.** A box whose CPU lacks the tier then runs the recognized narrower path instead of +scalar code. diff --git a/daslib/aarch64_neon.das b/daslib/aarch64_neon.das index 78b90e152b..62e3dc225a 100644 --- a/daslib/aarch64_neon.das +++ b/daslib/aarch64_neon.das @@ -45,17 +45,16 @@ def sdot4_w(acc : int4; w : int4; x : int8 const?) : int4 { var s3 = 0 unsafe { for (e in range(4)) { - s0 += sxb(w.x, e) * int(x[e]) - s1 += sxb(w.y, e) * int(x[4 + e]) - s2 += sxb(w.z, e) * int(x[8 + e]) - s3 += sxb(w.w, e) * int(x[12 + e]) + s0 += sign_extend_byte(w.x, e) * int(x[e]) + s1 += sign_extend_byte(w.y, e) * int(x[4 + e]) + s2 += sign_extend_byte(w.z, e) * int(x[8 + e]) + s3 += sign_extend_byte(w.w, e) * int(x[12 + e]) } } return acc + int4(s0, s1, s2, s3) } -// sign-extend byte e (0..3, little-endian) of a packed int lane -def private sxb(lane : int; e : int) : int => (((lane >> (e * 8)) & 0xff) ^ 0x80) - 0x80 +def private sign_extend_byte(lane : int; e : int) : int => (((lane >> (e * 8)) & 0xff) ^ 0x80) - 0x80 //! Byte table lookup of LOW nibbles (`TBL` / `vqtbl1q_s8(table, nib & 0x0f)`): each of 16 `nib` //! bytes has its low nibble (masked to 0..15, never out of table) looked up in `table`; the 16 @@ -125,10 +124,10 @@ def sdot4_laneq_w(acc : int4; w : int4; x : int8 const?; lane : int) : int4 { let x1 = int(x[b + 1]) let x2 = int(x[b + 2]) let x3 = int(x[b + 3]) - r.x += sxb(w.x, 0) * x0 + sxb(w.x, 1) * x1 + sxb(w.x, 2) * x2 + sxb(w.x, 3) * x3 - r.y += sxb(w.y, 0) * x0 + sxb(w.y, 1) * x1 + sxb(w.y, 2) * x2 + sxb(w.y, 3) * x3 - r.z += sxb(w.z, 0) * x0 + sxb(w.z, 1) * x1 + sxb(w.z, 2) * x2 + sxb(w.z, 3) * x3 - r.w += sxb(w.w, 0) * x0 + sxb(w.w, 1) * x1 + sxb(w.w, 2) * x2 + sxb(w.w, 3) * x3 + r.x += sign_extend_byte(w.x, 0) * x0 + sign_extend_byte(w.x, 1) * x1 + sign_extend_byte(w.x, 2) * x2 + sign_extend_byte(w.x, 3) * x3 + r.y += sign_extend_byte(w.y, 0) * x0 + sign_extend_byte(w.y, 1) * x1 + sign_extend_byte(w.y, 2) * x2 + sign_extend_byte(w.y, 3) * x3 + r.z += sign_extend_byte(w.z, 0) * x0 + sign_extend_byte(w.z, 1) * x1 + sign_extend_byte(w.z, 2) * x2 + sign_extend_byte(w.z, 3) * x3 + r.w += sign_extend_byte(w.w, 0) * x0 + sign_extend_byte(w.w, 1) * x1 + sign_extend_byte(w.w, 2) * x2 + sign_extend_byte(w.w, 3) * x3 } return r } diff --git a/daslib/f16_cvt.das b/daslib/f16_cvt.das index 009fa78303..629d5f7d1f 100644 --- a/daslib/f16_cvt.das +++ b/daslib/f16_cvt.das @@ -25,7 +25,7 @@ def f16_to_f32(h : uint) : float { } let sign = (h & 0x8000u) << 16u var bits : uint - if (exp == 0x1Fu) { // inf / nan + if (exp == 0x1Fu) { bits = sign | 0x7F800000u | (mant << 13u) } else { // normal: rebias exponent 15 -> 127 (+112), widen mantissa 10 -> 23 bits bits = sign | ((exp + 112u) << 23u) | (mant << 13u) @@ -39,7 +39,7 @@ def f32_to_f16(f : float) : uint { let bits = unsafe(reinterpret(f)) let sign = (bits >> 16u) & 0x8000u let absb = bits & 0x7FFFFFFFu - if (absb > 0x7F800000u) { // nan -> quiet nan + if (absb > 0x7F800000u) { return sign | 0x7E00u } if (absb >= 0x47800000u) { // inf, or >= 65536 (overflows past f16 max) diff --git a/daslib/x64_avx.das b/daslib/x64_avx.das index f7b47b0db8..ac819ea90c 100644 --- a/daslib/x64_avx.das +++ b/daslib/x64_avx.das @@ -109,10 +109,6 @@ def mx4_dequant32(lut : int8 const?; wn : uint8 const?; var dst : int8?) { } } -// ===== AVX kernel matrix tiers (VNNI / AVX-512), each gated on its own g_target_x64_* flag ===== -// Every body DELEGATES to the established intrinsic of the same math, so a box without the tier -// degrades to the recognized ymm sign-trick path rather than to scalar code. - //! dot32 with the VNNI dot step: same signature, same exact int4 result, lane for lane. With //! 256-bit VNNI (AVX-VNNI, or AVX512-VNNI+VL) the VPMADDUBSW+VPMADDWD pair becomes ONE VPDPBUSD //! (u8xs8 quad dot into i32, no 16-bit intermediate); sign trick and high/low fold are unchanged. diff --git a/examples/dasLLAMA/storyteller/main.das b/examples/dasLLAMA/storyteller/main.das index 63d1b41268..2c9fd96c95 100644 --- a/examples/dasLLAMA/storyteller/main.das +++ b/examples/dasLLAMA/storyteller/main.das @@ -92,18 +92,18 @@ var g_model = Model() var g_session = Session() var g_params = SamplingParams(temp = 0.8, top_k = 40l, penalty = 1.1) var g_phase = Phase.idle -var g_lines : array // the story as wrapped screen lines -var g_sentence : array // the pieces of the sentence being written, handed to speech at its end +var g_wrapped_lines : array +var g_sentence_pieces : array var g_ntokens = 0 var g_stories = 0 var g_say : Stream? // frame thread -> speech thread: sentences var g_speech : Stream? // speech thread -> frame thread: clips -var g_speech_done : Channel? // the speech thread signals it before it ends -var g_clips : array // clips waiting for the speaker -var g_now = 0.0 // seconds since start, from frame dt +var g_speech_done : Channel? +var g_clips : array +var g_elapsed_s = 0.0 var g_speaking_until = 0.0 -var g_pending_speech = 0 // sentences sent and not yet played +var g_unplayed_sentences = 0 var g_audio_initialized = false var g_asch : AudioSystemChannels var g_font : Font? @@ -112,8 +112,6 @@ var g_space_was = false var display_w = 0 var display_h = 0 -// --- speech thread --- - def start_speech_thread(tts_path, voice_pick : string; var say, speech : Stream?; var done : Channel?) { new_thread() <| @capture(= tts_path, = voice_pick, = say, = speech, = done) { setup_dasllama_jobque() // the fork-context pool is per context: without it every parallel kernel clones the program @@ -141,7 +139,7 @@ def start_speech_thread(tts_path, voice_pick : string; var say, speech : Stream? def say(text : string) { var l = Line(text = text, story = g_stories) g_say |> push_archive(l) - g_pending_speech++ + g_unplayed_sentences++ } def poll_speech() { @@ -152,35 +150,33 @@ def poll_speech() { g_clips |> emplace(clip) } } - if (g_now >= g_speaking_until && !empty(g_clips)) { + if (g_elapsed_s >= g_speaking_until && !empty(g_clips)) { let seconds = float(length(g_clips[0].pcm)) / float(max(g_clips[0].rate, 1)) var pcm <- g_clips[0].pcm play_sound_from_pcm(g_clips[0].rate, 1, pcm) g_clips |> erase(0) - g_speaking_until = g_now + seconds - g_pending_speech-- + g_speaking_until = g_elapsed_s + seconds + g_unplayed_sentences-- } } -// --- the story --- - def start_story() { delete g_session g_session <- create_session(g_model) set_seed(g_session, int(ref_time_ticks() % 2147483647l)) // a fresh session samples the same story every time otherwise - g_lines |> clear() - g_lines |> push("") - g_sentence |> clear() - g_sentence |> reserve(64) + g_wrapped_lines |> clear() + g_wrapped_lines |> push("") + g_sentence_pieces |> clear() + g_sentence_pieces |> reserve(64) g_ntokens = 0 g_stories++ - g_clips |> clear() // the previous story's unplayed clips; its sentences still in flight come back tagged and drop - g_pending_speech = 0 - g_speaking_until = g_now + g_clips |> clear() + g_unplayed_sentences = 0 + g_speaking_until = g_elapsed_s var prompt <- encode(g_model, PROMPT, true) eval(g_model, g_session, prompt) append_text(PROMPT) - g_sentence |> push(PROMPT) + g_sentence_pieces |> push(PROMPT) g_phase = Phase.writing } @@ -193,21 +189,20 @@ def append_text(piece : string) { var inscope parts <- split(piece, "\n") for (i, part in iter_range(parts), parts) { if (i > 0) { - g_lines |> push("") + g_wrapped_lines |> push("") } - let li = length(g_lines) - 1 - g_lines[li] = "{g_lines[li]}{part}" + let li = length(g_wrapped_lines) - 1 + g_wrapped_lines[li] = "{g_wrapped_lines[li]}{part}" wrap_last_line() } } -// break the last line at its last space before the wrap column, as often as it takes def wrap_last_line() { - while (length(g_lines[length(g_lines) - 1]) > WRAP_CHARS) { - let li = length(g_lines) - 1 + while (length(g_wrapped_lines[length(g_wrapped_lines) - 1]) > WRAP_CHARS) { + let li = length(g_wrapped_lines) - 1 var head = "" var tail = "" - peek_data(g_lines[li]) $(d) { + peek_data(g_wrapped_lines[li]) $(d) { var cut = rfind(slice(d, 0, WRAP_CHARS), " ") if (cut <= 0) { cut = WRAP_CHARS @@ -215,8 +210,8 @@ def wrap_last_line() { head = clone_string(slice(d, 0, cut)) tail = clone_string(slice(d, cut)) } - g_lines[li] = head - g_lines |> push(strip(tail)) + g_wrapped_lines[li] = head + g_wrapped_lines |> push(strip(tail)) } } @@ -237,10 +232,10 @@ def write_step() { g_ntokens++ let text = plain_quotes(piece(g_model, tok)) append_text(text) - g_sentence |> push(text) + g_sentence_pieces |> push(text) if (sentence_ends(text)) { - say(strip(join(g_sentence, ""))) - g_sentence |> clear() + say(strip(join(g_sentence_pieces, ""))) + g_sentence_pieces |> clear() } one[0] = tok eval(g_model, g_session, one) @@ -248,16 +243,15 @@ def write_step() { } def finish_story() { - let rest = strip(join(g_sentence, "")) + let rest = strip(join(g_sentence_pieces, "")) if (!empty(rest)) { say(rest) } - g_sentence |> clear() + g_sentence_pieces |> clear() g_phase = Phase.told } -// --- text layer, in design pixels against a 1280x720 reference --- - +//! the text layer's coordinates are design pixels against a 1280x720 reference def hud_scale() : float { return max(float(display_h) / 720.0, 0.5) } @@ -283,20 +277,18 @@ def draw_story() { draw_text("storyteller", 60.0, 60.0, 0.6, dim) var y = 130.0 let step = 34.0 * TEXT_SIZE / 0.72 - let first = max(0, length(g_lines) - VISIBLE_LINES) // a long story scrolls: the newest lines stay on screen - for (i in range(first, length(g_lines))) { - draw_text(g_lines[i], 60.0, y, TEXT_SIZE, ink) + let first = max(0, length(g_wrapped_lines) - VISIBLE_LINES) // a long story scrolls: the newest lines stay on screen + for (i in range(first, length(g_wrapped_lines))) { + draw_text(g_wrapped_lines[i], 60.0, y, TEXT_SIZE, ink) y += step } let status = (g_phase == Phase.idle ? "press SPACE for a story" : g_phase == Phase.writing ? "writing..." - : g_pending_speech > 0 || g_now < g_speaking_until ? "reading aloud..." + : g_unplayed_sentences > 0 || g_elapsed_s < g_speaking_until ? "reading aloud..." : "the end - press SPACE for another") draw_text(status, 60.0, 660.0, SMALL_SIZE, g_phase == Phase.idle ? accent : dim) } -// --- lifecycle --- - [export] def init() { var inscope r <- parse_args(type) @@ -333,7 +325,7 @@ def update() { return } g_frames++ - g_now += get_dt() + g_elapsed_s += get_dt() live_get_framebuffer_size(display_w, display_h) glViewport(0, 0, display_w, display_h) glClearColor(0.07, 0.06, 0.09, 1.0) @@ -361,7 +353,7 @@ def update() { [export] def shutdown() { - say("") // the speech thread's stop word + say("") g_speech_done |> join() unsafe { channel_remove(g_speech_done) @@ -380,7 +372,7 @@ def shutdown() { def done_for_smoke() : bool { return true if (g_args.max_frames > 0 && g_frames >= g_args.max_frames) - return g_args.autoplay && g_phase == Phase.told && g_pending_speech == 0 && g_now >= g_speaking_until + return g_args.autoplay && g_phase == Phase.told && g_unplayed_sentences == 0 && g_elapsed_s >= g_speaking_until } // eval_main_loop drives the block once per frame: a blocking while-loop natively, the @@ -395,6 +387,6 @@ def main() { } shutdown() if (g_args.autoplay) { - print("{join(g_lines, "\n")}\n") + print("{join(g_wrapped_lines, "\n")}\n") } } diff --git a/modules/dasLLAMA/ARCHITECTURE.md b/modules/dasLLAMA/ARCHITECTURE.md index f3b5df8d1e..be6a80ebed 100644 --- a/modules/dasLLAMA/ARCHITECTURE.md +++ b/modules/dasLLAMA/ARCHITECTURE.md @@ -66,11 +66,13 @@ re-transcoding `$LCPP/src/unicode-data.cpp`). rig, the tune gate, the sanctioned instrumentation rails, kernel-race fidelity, the gemv's own tune seat, the CPU kernel bench's fixture conditions, the speculative round's ruler record, the `[tuned]` perm precedence, and the mint wall in the sidecar's provenance. -- `ARCHITECTURE_CPU_KERNELS.md` - sec.2.22-2.24: the sub-block-packed k3/k6 planes, the grid - formats' panel and row-group decodes, and the VBMI symbol lattice. -- `ARCHITECTURE_TTS.md` - sec.2.28-2.35: the TTS block home's two layouts, tap stacking, the +- `ARCHITECTURE_CPU_KERNELS.md` - sec.2.22-2.24, 2.42: the sub-block-packed k3/k6 planes, the grid + formats' panel and row-group decodes, the VBMI symbol lattice, and the tier that selects on the + target rather than the host. +- `ARCHITECTURE_TTS.md` - sec.2.28-2.35, 2.43: the TTS block home's two layouts, tap stacking, the padded concat width, the per-synthesis carrier, the served carrier on the image rail, the - sine source's operation order, the heteronym context, and the rig's scoring. + sine source's operation order, the heteronym context, the rig's scoring, and the two-tier + phoneme pack. ## 3. Inherited invariants diff --git a/modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md b/modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md index 9a32bf3b44..6c0b32dacd 100644 --- a/modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md +++ b/modules/dasLLAMA/ARCHITECTURE_CPU_KERNELS.md @@ -43,3 +43,15 @@ code bytes in its qword, `VPMULTISHIFTQB` spreads the symbols into bytes, one `v magnitudes, and the signs ride the activation copy as a mask `(x ^ m) - m`. The lattice row shares its tile body and planes with the 512/mr16 row, so only the gemv differs - what the gemv's own seat (`ARCHITECTURE_MEASUREMENT.md` sec.2.26) races. + +### 2.42 A CPU tier selects on the TARGET, not the host {#cpu-tier-target-select} + +The arm64 SDOT tier registers its backends only under the JIT and only for an arm64 TARGET - the +artifact's architecture, never the running host's. Off the JIT the `sdot4` family runs its scalar +fallback bodies, which are slower than the portable `dot_q8q8` the vectorizer handles, so the +portable tier stays selected wherever hardware SDOT is not emitted. + +The portable tier picks its dot form on the target at compile time. On wasm SIMD128 the +auto-vectorized template dot is the slow form - the ISA carries no int8 dot for LLVM to find, +while the `idot4` builtin lowers there to the ISA's own widening multiply-adds - so a wasm target +takes `dot_q8q8_idot4_ps`. Every other target keeps the template. diff --git a/modules/dasLLAMA/ARCHITECTURE_TTS.md b/modules/dasLLAMA/ARCHITECTURE_TTS.md index e638b8b9d4..50f0e5f4aa 100644 --- a/modules/dasLLAMA/ARCHITECTURE_TTS.md +++ b/modules/dasLLAMA/ARCHITECTURE_TTS.md @@ -1,7 +1,7 @@ # dasLLAMA architecture - text to speech The companion `ARCHITECTURE.md` indexes: the TTS file charters (sec.1.7c) and the mechanisms the -TTS files implement (sec.2.28-2.35). `ARCHITECTURE_COMMON.md` (repo root) is the contract. +TTS files implement (sec.2.28-2.35, 2.43). `ARCHITECTURE_COMMON.md` (repo root) is the contract. ## 1. File charters @@ -272,3 +272,13 @@ exposes `expected` so the normalizer cell and the rig read the one correction; t (`harness/mint_tts_g2p_fixture.py`) carries the table so a re-mint reproduces it. The reference line is the reference arms' own WAVs re-scored from the experiment's transcripts on the same forms with the same scorer, the clock suffix ("a m", "am") counted as one word on both sides. + +### 2.43 The phoneme pack ships in two tiers {#tts-g2p-pack-tiers} + +`build_g2p_data.py` mints two packs from one source: `tts_g2p.bin`, carrying both dialect tiers, +and `tts_g2p_en_us.bin` under `--dialect us`, the American tier alone - four megabytes smaller, +and what the web serving set ships. A pack names its own tier in its source line, which `load_g2p` +reads into `G2pModel.us_only`. `load_tts_model` takes the full pack from the model's directory when +it is there and the American-only twin otherwise, and panics when neither is. On the American-only +pack `g2p_phonemize` refuses a British request by name and `caps` offers no British voice, so a +Kokoro `bf_*` / `bm_*` pack is simply not on the list. diff --git a/modules/dasLLAMA/REVIEW_MEASUREMENT.md b/modules/dasLLAMA/REVIEW_MEASUREMENT.md index 7a253eef13..6633687b55 100644 --- a/modules/dasLLAMA/REVIEW_MEASUREMENT.md +++ b/modules/dasLLAMA/REVIEW_MEASUREMENT.md @@ -83,3 +83,9 @@ board cell or ruler record produced names the record and row instead of the harn sits in the figure's own sentence, in a table heading that covers the table's rows, in a section-level provenance line that covers the paragraphs under it, or in a citation of the `PERF_LEDGER.md` entry whose provenance line covers it. + +**A diff that pins the kernel backend - `pin_kernel_backend`, `select_kernel_backend`, or +`DASLLAMA_PIN_BACKEND` - pins it before the load it governs, and on a name the box has not +registered refuses or skips rather than running on the default.** The loader repacks weights +into the pinned backend's layout, so a pin set after the load never reaches them, and a +misspelled pin that falls through measures the default backend under the pinned one's name. diff --git a/modules/dasLLAMA/benchmarks/lcpp_bench.das b/modules/dasLLAMA/benchmarks/lcpp_bench.das index 0187bd5767..4a0d88528f 100644 --- a/modules/dasLLAMA/benchmarks/lcpp_bench.das +++ b/modules/dasLLAMA/benchmarks/lcpp_bench.das @@ -1270,7 +1270,6 @@ def private tests_of(rows : array) : table { // flavor is stamped debug-jit so no board or oracle can mistake it for measurement var private g_debug_run = false -// the exchange client compiles only where the tune framework does, so --submit has a rail to refuse on def private exchange_compiled_in() : bool { static_if (typeinfo module_exists(dasllama_exchange)) { return true diff --git a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das index 75779b9cd7..371471177e 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_aarch64_neon.das @@ -451,11 +451,8 @@ def mx4q8_rows_kernel_neon(var yp : float?; wn : uint8 const?; we : uint8 const? } } -[init] +[init, arch(at = "../ARCHITECTURE_CPU_KERNELS.md#cpu-tier-target-select")] def dasllama_math_aarch64_neon_register() { - // SDOT backends only under JIT for an arm64 TARGET (the artifact's arch, never the host's). - // Off-JIT the sdot4 family runs its SCALAR fallback, slower than the portable dot_q8q8 the - // vectorizer handles, so portable stays selected wherever hardware SDOT isn't emitted. if (get_target_architecture_name() == "arm64" && jit_enabled()) { // arm64-sdot: idot4x4 on row-major weights (idot4 → SDOT) — the auto-selected default for direct callers // (tests, benchmarks). priority 10 out-ranks portable (0). diff --git a/modules/dasLLAMA/dasllama/dasllama_math_default.das b/modules/dasLLAMA/dasllama/dasllama_math_default.das index 0193ac97bf..44162a2cf8 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_default.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_default.das @@ -147,8 +147,8 @@ def dot_q8q8_idot4x4_f16s(wq : int8 const?; ws : uint16 const?; xq : int8 const? } //! The vector-epilogue idot4 dot: per 32-block one idot4 pair, then float4 accumulate under the block's -//! scale - no horizontal sum until the row ends, two rotating float4 accumulators. The wasm SIMD128 form (7.7 vs the -//! template's 4.8 GMAC/s on one lane: dot_q8q8_idot4x4's per-block hsum is four lane extracts there). Called from lifted lambdas. +//! scale - no horizontal sum until the row ends, two rotating float4 accumulators. The wasm SIMD128 form +//! (dot_q8q8_idot4x4's per-block hsum is four lane extracts there). Called from lifted lambdas. [hint(unsafe_range_check, noalias = wq, noalias = xq, noalias = ws, noalias = xs)] def dot_q8q8_idot4_ps(wq : int8 const?; ws : float const?; xq : int8 const?; xs : float const?; n : int64) : float { let nb = n / 32l @@ -219,11 +219,9 @@ def dot_q8q8_idot4_ps_f16s(wq : int8 const?; ws : uint16 const?; xq : int8 const return f.x + f.y + f.z + f.w } -// On wasm SIMD128 the auto-vectorized template dot is the slow form: the ISA has no int8 dot for -// LLVM to find, while the idot4 builtin lowers there to the ISA's own widening multiply-adds. The -// portable kernels pick per target at compile time; every other host keeps the template. let private WASM_TARGET = get_target_architecture_name() == "wasm64" || get_target_architecture_name() == "wasm32" +[arch(at = "../ARCHITECTURE_CPU_KERNELS.md#cpu-tier-target-select")] def private dot_q8q8_p(wq : int8 const?; ws : float const?; xq : int8 const?; xs : float const?; n : int64) : float { return WASM_TARGET ? dot_q8q8_idot4_ps(wq, ws, xq, xs, n) : dot_q8q8(wq, ws, xq, xs, n) } diff --git a/modules/dasLLAMA/dasllama/dasllama_styletts2.das b/modules/dasLLAMA/dasllama/dasllama_styletts2.das index d9e9213337..4daa0e99ec 100644 --- a/modules/dasLLAMA/dasllama/dasllama_styletts2.das +++ b/modules/dasLLAMA/dasllama/dasllama_styletts2.das @@ -248,7 +248,7 @@ struct StyleTts2Model { decoder : St2Decoder = St2Decoder() source : SineSourceCfg = SineSourceCfg() q8 : bool // the lane this image was minted on: the rows GEMMs serve Q8_0 quants - kitten : KittenFamily // the family's driver data rides the meta, so an image serves without its gguf + kitten : KittenFamily kokoro : KokoroFamily blob : PlaneF = PlaneF() qblob : PlaneI8 = PlaneI8() @@ -731,8 +731,6 @@ def stage_styletts2(path : string; q8 : bool = false) : St2Staging { return <- st } -// the family's driver keys, staged with the model so they ride its image: Kitten's per-voice -// speed priors and alias names, Kokoro's symbol table def private stage_family_data(m : GGUFMeta; bytes : array#; var e : StyleTts2Model) { if (e.family == "kitten-tts") { for (v in e.voice_names) { @@ -780,9 +778,9 @@ def load_styletts2(path : string) : StyleTts2Model { if (path |> ends_with(".dlim")) { apply_box_profile_runtime() let ts = ref_time_ticks() - let first = q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG // the policy's lane first: no wrong-tag decline in the normal case - let second = q8 ? ST2_IMAGE_TAG : ST2_IMAGE_TAG_Q8 - if (!load_image(path, m, first) && !load_image(path, m, second)) { + let preferred_tag = q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG + let other_tag = q8 ? ST2_IMAGE_TAG : ST2_IMAGE_TAG_Q8 + if (!load_image(path, m, preferred_tag) && !load_image(path, m, other_tag)) { panic("dasLLAMA styletts2: '{path}' is not a prepared TTS image for this box (identity {image_identity(ST2_IMAGE_TAG_Q8)} / {image_identity(ST2_IMAGE_TAG)}) - bake it from the source gguf with dasllama-convert") } st2_bind(m) diff --git a/modules/dasLLAMA/dasllama/dasllama_tts.das b/modules/dasLLAMA/dasllama/dasllama_tts.das index af55a07d4c..8ac6ab35d5 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tts.das +++ b/modules/dasLLAMA/dasllama/dasllama_tts.das @@ -67,8 +67,7 @@ def load_tts_model(path : string) : TtsModel { return <- m } -// the directory decides the phoneme pack: the full one when present, else the American-only twin -// the web serving set ships (British voices are then not offered) +[arch(at = "../ARCHITECTURE_TTS.md#tts-g2p-pack-tiers")] def private g2p_pack_path(dir : string) : string { let full = path_join(dir, "tts_g2p.bin") return full if (stat(full).is_valid) diff --git a/modules/dasLLAMA/harness/parity.das b/modules/dasLLAMA/harness/parity.das index 1f2a8d8b3e..69c8937256 100644 --- a/modules/dasLLAMA/harness/parity.das +++ b/modules/dasLLAMA/harness/parity.das @@ -58,10 +58,7 @@ struct Args { help : bool } -// `--backend`, defaulting to DASLLAMA_PIN_BACKEND: resolve the name against the registry and pin -// it, or refuse. Called before load_gguf — select_matmul_backend_for_load_ reads the pin to pick -// the weight layout the loader repacks into — and after the --ngl block, so the flag outranks the -// portable pin metal mode applies. +//! applied after the --ngl block, so --backend outranks the portable pin metal mode applies def private apply_backend_pin(flag : string) : bool { let name = empty(flag) ? g_env_engine.pin_backend : flag if (empty(name)) { @@ -123,7 +120,7 @@ def main : int { if (t.config.seq_len > 2048l) { t.config.seq_len = 2048l } - print("GEN_BACKEND: {active_kernel_backend()} (batch: {active_batch_backend()})\n") // the GEN_ block's header — a log names the backend its ids came off + print("GEN_BACKEND: {active_kernel_backend()} (batch: {active_batch_backend()})\n") var s = make_run_state(t.config, cfg.kv, cfg.kv) var rc = 0 with_job_que() { diff --git a/modules/dasLLAMA/performance/build_tts_data.das b/modules/dasLLAMA/performance/build_tts_data.das index 009f276403..0830abfdae 100644 --- a/modules/dasLLAMA/performance/build_tts_data.das +++ b/modules/dasLLAMA/performance/build_tts_data.das @@ -53,7 +53,6 @@ def main() : int { let fixture = path_join(get_das_root(), "modules/dasLLAMA/tests/_tts_fixtures/g2p_corpus.json") let silver = path_join(args.root, "work/silver_focus") let focus = path_join(args.root, "work/focus_words.txt") - // the full pack, then its American-only twin - the web serving set's, no British tier let additions = path_join(harness, "g2p_local_additions.json") return 1 if (!step("pack the G2P data", [py, path_join(harness, "build_g2p_data.py"), args.root, "--out", path_join(store, "tts_g2p.bin"), "--local-additions", additions, "--focus-words", focus]) diff --git a/modules/dasLLAMA/performance/model_specs.das b/modules/dasLLAMA/performance/model_specs.das index 16d828a648..0bad12b6e9 100644 --- a/modules/dasLLAMA/performance/model_specs.das +++ b/modules/dasLLAMA/performance/model_specs.das @@ -557,7 +557,7 @@ def model_specs() : array { // nolint:STYLE038 — flat model-set t ProvEntry(name = "tts_g2p.bin", root = "llm", recipe = TTS_PACKS_RECIPE, url = "{TTS_HF}/tts_g2p.bin", bytes = 14011554l, sha256 = "d7b6afea7a0901a877d10531054d3f967845c58f415f0ea937decd8a3081ccd5"), - ProvEntry(name = "tts_g2p_en_us.bin", root = "llm", recipe = TTS_PACKS_RECIPE, // the American-only twin: the web serving set's pack + ProvEntry(name = "tts_g2p_en_us.bin", root = "llm", recipe = TTS_PACKS_RECIPE, url = "{TTS_HF}/tts_g2p_en_us.bin", bytes = 10257455l, sha256 = "6f69d2e74565bd7d876b8d1f4042bf8c1c5b615387fa26ff45215cf447932154"), ProvEntry(name = "tts_postag.bin", root = "llm", recipe = TTS_PACKS_RECIPE, diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index f87c7a7628..99e7e3318e 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -427,9 +427,6 @@ def private child_envs(cfg : RunArgs; win : bool; arm : string) : string { } } if (cfg.no_tune) { - // the reference policy changes the box identity every .dlim is stamped with, and a mint - // under a foreign identity GC-purges the box's tuned images as dead siblings - so the - // portable arm runs with the image rail off; image-rail coverage stays with the tuned arm envs = (win ? "{envs}set \"DAS_TUNE_POLICY=reference\"&& set \"DASLLAMA_IMAGE=0\"&& " : "DAS_TUNE_POLICY=reference DASLLAMA_IMAGE=0 {envs}") } diff --git a/modules/dasLLAMA/tests/test_audio_embedder.das b/modules/dasLLAMA/tests/test_audio_embedder.das index 7318d2663d..59684015d1 100644 --- a/modules/dasLLAMA/tests/test_audio_embedder.das +++ b/modules/dasLLAMA/tests/test_audio_embedder.das @@ -74,9 +74,7 @@ def test_audio_embedder_gemma4a_arm(t : T?) { if (empty(image_path) && starts_with(name, "{base_name(mm)}.0x") && ends_with(name, ".dlim")) { let cand = path_join(models_dir(), name) let tag = image_family_tag(cand) - // an image baked under another box identity (a different tune policy, another - // backend pin) sits beside the current one with its own hash - only the path the - // rail would mint for THIS identity is loadable by the direct route + //! only the path the rail mints for THIS box identity is loadable by the direct route - a foreign-identity bake sits beside it under its own hash if (tag |> starts_with("gemma4a-") && cand == image_path_for(mm, tag)) { image_path = cand } diff --git a/modules/dasLLAMA/tests/test_groupn.das b/modules/dasLLAMA/tests/test_groupn.das index 67c2e72e72..32c441ace0 100644 --- a/modules/dasLLAMA/tests/test_groupn.das +++ b/modules/dasLLAMA/tests/test_groupn.das @@ -22,8 +22,8 @@ def private close(t : T?; got, want : float; what : string) { let N = 64l let D = 4l // one interleaved group per region (laneq-compatible), no tail let NREG = 3l -let REF_ULP = 2.0e-6 // per-32-block fp32 fold margin — the bar scales with the accumulation length -let REF_POISON = 0.05 // added to one expected element; the bar must not swallow it +let REF_ULP_PER_BLOCK = 2.0e-6 +let REF_POISON = 0.05 def private build_offs() : array { return <- [N * D * 0l, 0l, N * D * 1l, 0l, N * D * 2l, N] @@ -41,10 +41,7 @@ def private slice_f(src : array; off, n : int64) : array { return <- [for (i in range64(off, off + n)); src[i]] } -// The independent truth for one q8 row: an fp64 dot over the region's dequantized Q8_0 blocks — -// int8 products summed exactly in int, both block scales applied in double. `env` is that sum -// taken on block magnitudes: the envelope the kernels' fp32 per-block folds round inside, which is -// what the bar is derived from. Layout-free, so it judges repacked and row-major backends alike. +//! The independent fp64 truth for one q8 row; `env` is the same sum on block magnitudes, the envelope the bar derives from. Layout-free, so repacked and row-major backends judge alike. def private ref_dot_q8_f64(wq : array; ws : array; woff : int64; xq : array; xs : array; xoff : int64; n : int64) : tuple { var res : tuple @@ -77,8 +74,7 @@ def test_q8q8_groupn(t : T?) { let wq0 <- quantize_q8_0(wf, NREG * D * N) let xq <- quantize_q8_0(xf, 2l * N) var offs <- build_offs() - // the fp64 reference and its bar, built once off the row-major quants: both are - // backend-free, since a repack only permutes the same weights + //! reference and bar built once off the row-major quants - a repack only permutes the same weights, so both are backend-free var refy : array var refbar : array refy |> resize(int(NREG * D)) @@ -87,7 +83,7 @@ def test_q8q8_groupn(t : T?) { for (row in range64(D)) { let rf = ref_dot_q8_f64(wq0.q, wq0.scales, r * D * N + row * N, xq.q, xq.scales, offs[r * 2l + 1l], N) refy[r * D + row] = float(rf.dot) - refbar[r * D + row] = REF_ULP * float(N / 32l) * max(float(rf.env), 1.0) + refbar[r * D + row] = REF_ULP_PER_BLOCK * float(N / 32l) * max(float(rf.env), 1.0) } } let saved = active_kernel_backend() diff --git a/modules/dasLLAMA/tests/test_math_activations.das b/modules/dasLLAMA/tests/test_math_activations.das index 9691b9580a..c915ea2495 100644 --- a/modules/dasLLAMA/tests/test_math_activations.das +++ b/modules/dasLLAMA/tests/test_math_activations.das @@ -577,6 +577,7 @@ def test_silu4_batch(t : T?) { let inline_wide = silu4_batch_case(r, 256, 512) var threaded = BatchCase() with_job_que() { + setup_dasllama_jobque_() threaded = silu4_batch_case(r, 256, 512) } report_moved(t, "silu4_batch", r) @@ -589,6 +590,7 @@ def test_silu4_batch(t : T?) { t |> run("silu4_batch falls back whole where the row length or the row count forbids a split") @(t : T?) { var r = ActSweep() with_job_que() { + setup_dasllama_jobque_() let odd = silu4_batch_case(r, 250, 64) let single = silu4_batch_case(r, 64, 1) let tail = silu4_batch_case(r, 4, 1025) diff --git a/modules/dasLLAMA/tests/test_parity.das b/modules/dasLLAMA/tests/test_parity.das index a379f47795..bbc64eda69 100644 --- a/modules/dasLLAMA/tests/test_parity.das +++ b/modules/dasLLAMA/tests/test_parity.das @@ -18,11 +18,6 @@ require ../performance/model_specs.das // the model-set table — parity evide // frozen under; regenerate with harness/parity.sh. The board's --parity pregate reads the same rows. // Models are gitignored multi-GB files: absent = clean skip (a real local regression, a no-op in CI). -// DASLLAMA_PIN_BACKEND points these gates at one kernel backend, the way the benches read it — -// the pin lands BEFORE the load, so select_matmul_backend_for_load_ picks the layout it repacks -// into. pin_kernel_backend is process-wide, so the entry backend is restored on every path out. -// Unknown name => the body never runs and the cell skips naming it, never a silent run on the -// default backend. def private with_parity_backend(t : T?; blk : block<() : void>) { let name = g_env_engine.pin_backend if (empty(name)) { diff --git a/modules/dasLLAMA/tests/test_q8q8_family.das b/modules/dasLLAMA/tests/test_q8q8_family.das index b296f01d8e..a18de7488f 100644 --- a/modules/dasLLAMA/tests/test_q8q8_family.das +++ b/modules/dasLLAMA/tests/test_q8q8_family.das @@ -600,9 +600,7 @@ def test_q8q8_wscale_f16(t : T?) { } } -// The vector-epilogue idot4 dots - the portable kernels' wasm form - on every host: each row of -// each token within the fp64 bar, the s16 twin bit-identical to its f32 twin over the widened plane. -// n=64 is the paired-block main loop alone, n=96 adds the single-block tail. +//! The portable kernels' wasm form, gated on every host: n=64 is the paired-block main loop alone, n=96 adds the single-block tail. def private idot4_ps_gate(t : T?; n : int64) { let d = 64l let nb = n / 32l diff --git a/modules/dasLLAMA/tests/test_tts_g2p.das b/modules/dasLLAMA/tests/test_tts_g2p.das index 88a97e50c0..532b2c47f8 100644 --- a/modules/dasLLAMA/tests/test_tts_g2p.das +++ b/modules/dasLLAMA/tests/test_tts_g2p.das @@ -247,8 +247,6 @@ def private write_pack_header(path : string; version : int) { fclose(f) } -// The American-only twin (build_g2p_data.py --dialect us, the web serving set's pack): the same -// American reading as the full pack on the whole corpus, and British refused by name. [test] def test_us_only_pack(t : T?) { t |> run("the American-only pack reads the corpus as the full pack does and refuses British") @(t : T?) { diff --git a/modules/dasLLAMA/tests/test_tts_kitten.das b/modules/dasLLAMA/tests/test_tts_kitten.das index 7062ab33d4..187f589d1b 100644 --- a/modules/dasLLAMA/tests/test_tts_kitten.das +++ b/modules/dasLLAMA/tests/test_tts_kitten.das @@ -264,7 +264,7 @@ def test_kitten_generator_hook(t : T?) { } } -[test] +[test, arch(at = "../ARCHITECTURE_TTS.md#tts-image-rail")] def test_kitten_image(t : T?) { t |> run("nano: the staged read, the mapped image and the in-memory mint agree on both lanes") @(t : T?) { if (!stat(gguf_path("nano")).is_valid) { @@ -292,8 +292,6 @@ def test_kitten_image(t : T?) { let q8_img = image_path_for(gguf_path("nano"), ST2_IMAGE_TAG_Q8) t |> success(f32_img != q8_img, "each lane is its own prepared image:\n {f32_img}\n {q8_img}") } - // ARCHITECTURE_TTS.md sec.2.32: an image serves without its gguf - the family's driver data - // rides the meta, and a .dlim path maps the lane the file was baked on, whatever the policy says t |> run("nano: the image loads by its own path, the family data riding its meta") @(t : T?) { if (!stat(gguf_path("nano")).is_valid) { t |> skip("kitten-nano.gguf not present") @@ -308,7 +306,7 @@ def test_kitten_image(t : T?) { var inscope from_gguf <- load_styletts2(gguf_path("nano")) // bakes the q8 sibling when it is not there yet let img = image_path_for(gguf_path("nano"), ST2_IMAGE_TAG_Q8) t |> success(stat(img).is_valid, "the q8 lane's image is on disk: {img}") - set_styletts2_q8(false) // the policy asks for f32: the file still serves its own lane + set_styletts2_q8(false) defer() { reset_styletts2_q8() } @@ -322,7 +320,7 @@ def test_kitten_image(t : T?) { t |> equal(from_gguf.kitten.aliases?[an] ?? "", av, "alias {an}") } if (stat(path_join(models_dir(), "tts_g2p.bin")).is_valid && stat(path_join(models_dir(), "tts_postag.bin")).is_valid) { - var inscope tm <- load_tts_model(img) // the facade: the packs still come from the directory + var inscope tm <- load_tts_model(img) var inscope c <- caps(tm) var inscope a <- synthesize(tm, "The image spoke.", c.voices[length(c.voices) - 1], 1.0) t |> success(long_length(a.pcm) > 8000l, "the facade synthesized {long_length(a.pcm)} samples from the image") diff --git a/modules/dasLLAMA/tests/test_tts_kokoro.das b/modules/dasLLAMA/tests/test_tts_kokoro.das index 503d58ae63..e09bce79f9 100644 --- a/modules/dasLLAMA/tests/test_tts_kokoro.das +++ b/modules/dasLLAMA/tests/test_tts_kokoro.das @@ -132,7 +132,7 @@ def test_kokoro_oracle(t : T?) { } delete g_family { - var inscope staged <- load_styletts2(gguf_path()) // the symbol table rides the model's own carrier + var inscope staged <- load_styletts2(gguf_path()) g_family := staged.kokoro } t |> success(length(g_family.sym) > 100, "{length(g_family.sym)} symbols in the vocabulary") diff --git a/modules/dasLLVM/ARCHITECTURE.md b/modules/dasLLVM/ARCHITECTURE.md index d93ae2895c..99358818b2 100644 --- a/modules/dasLLVM/ARCHITECTURE.md +++ b/modules/dasLLVM/ARCHITECTURE.md @@ -172,7 +172,7 @@ and a standalone exe binds every extern its functions name at startup, so a read sees only the compiler-free runtime archive, has nothing to bind it to. The shipped defaults profiles are the same kind of input: with no sidecar entry a kernel -stamps its class entry out of `/.tune-defaults.json`, so `locate_profile_doc` +stamps its class entry out of `/.tune-defaults.json`, so `pin_profile_chain` registers every candidate on the class ladder it tries, existing or not - a profile that appears, or is re-exported after a re-mint, must invalidate the stamps minted without it. The staleness gate itself compares the sidecar's mtime with the running binary's, which no content @@ -278,3 +278,21 @@ vecmath carries no vector sinh/cosh/tanh, so `SimPolicy` binds `vsinh`/`vcosh`/` exp polynomial's error rather than agreeing with them - the opposite trade from every other emitter on the rail, taken because the consumer (GELU over float4 rows) otherwise pays four libm calls per vector. `tests/llvm_vector_math.das` asserts the size of that divergence. + +## 9. The idot family's target lowerings {#idot-lowerings} + +The exact integer dots on the 8-bit lattice have three lowerings, picked by target: one +`@llvm.aarch64.neon.sdot` where the target has DotProd (`g_target_arm64_dotprod` - the host rail's +`+dotprod` append, or the force env on the generic rail), the SIMD128 form on a wasm target +(`idot_wasm_simd128`), generic widen-multiply IR everywhere else. The native arms exist because +neither backend produces them from the generic form: AArch64 expands it to zip/uzp/smull instead +of folding to SDOT, and the wasm backend runs it a fifth as fast. The wasm form has two arms. With +`+relaxed-simd` the ISA carries the dot itself, `i32x4.relaxed_dot_i8x16_i7x16_add_s`, exact only +while its second operand is in [0, 127]; the sign trick puts it there, since +`dot(w, x) == dot(sign(x)*w, |x|)` and Q8_0 quants are in [-127, 127] by construction +(`d = amax/127`), so no lane wraps on the flip. Without the feature the ISA still has the two halves +of an int8 dot, `i16x8.extmul_{low,high}_i8x16_s` and `i32x4.extadd_pairwise_i16x8_s`; the pairwise +sums land as byte pairs, and one even/odd shuffle-add folds them into the quad lanes the generic +form defines. `wasm_target_features` always asks for `+relaxed-simd`: every engine that runs +memory64 shipped relaxed SIMD first, and the feature changes no ABI, so a runtime archive built +without it links unchanged. diff --git a/modules/dasLLVM/REVIEW.md b/modules/dasLLVM/REVIEW.md index 46cc279952..7b82e999ba 100644 --- a/modules/dasLLVM/REVIEW.md +++ b/modules/dasLLVM/REVIEW.md @@ -111,3 +111,8 @@ directory from its tracked-fixture list or removing the last tracked file under one (a guard over nothing), or a finding text that no longer names what failed. What the gate enforces is read from the gate itself. + +- **A diff that appends target features to a machine's feature string appends the forced ones + (`x64_forced_plus_features` / `arm64_forced_plus_features`, `daslib/llvm_jit_common.das`) AFTER + the detected host features.** LLVM's `SubtargetFeatures` takes the last occurrence of a name, so + a forced feature placed first is silently overridden by detection. diff --git a/modules/dasLLVM/daslib/llvm_jit_common.das b/modules/dasLLVM/daslib/llvm_jit_common.das index abc9958305..92aad0eda1 100644 --- a/modules/dasLLVM/daslib/llvm_jit_common.das +++ b/modules/dasLLVM/daslib/llvm_jit_common.das @@ -395,8 +395,7 @@ var public g_handled_field_offset_globals : table starts_with("wasm") - g_target_wasm_relaxed_simd = g_target_is_wasm // wasm_target_features always asks for +relaxed-simd + g_target_wasm_relaxed_simd = g_target_is_wasm // Host path (empty triple): the module targets the host, so the daslang binary's own arch is the // JIT target arch. Explicit cross-compile triple: read the arch off the triple prefix. g_target_is_aarch64 = (empty(target_triple) @@ -651,8 +646,6 @@ def public init_jit_target_flags(target_triple : string = ""; host_features : bo g_target_os_linux = (empty(target_triple) ? get_platform_name() == "linux" : find(target_triple, "linux") >= 0) - // aarch64 dotprod (see declaration comment): the host rail force-appends +dotprod, so host - // truth IS the rail; the generic exe / cross rails take the force env only. g_target_arm64_dotprod = g_target_is_aarch64 && (arm64_forced_feature("dotprod") || (empty(target_triple) && host_features)) g_target_arm64_i8mm = g_target_is_aarch64 && (arm64_forced_feature("i8mm") @@ -949,9 +942,6 @@ def public create_default_target_machine(opt_level : uint; use_host_cpu : bool) } targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) } else { - // the force env (emission-only, see the g_target_x64_* declarations) reaches BOTH rails: - // AFTER the host features, where the last occurrence wins in LLVM's SubtargetFeatures, and - // onto the generic baseline, so the gates it raised never emit what the machine cannot select let forced = g_target_is_x64 ? x64_forced_plus_features() : (g_target_is_aarch64 ? arm64_forced_plus_features() : "") let feats = empty(forced) ? features_msg : (empty(features_msg) ? forced : "{features_msg},{forced}") targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) diff --git a/modules/dasLLVM/daslib/llvm_jit_intrin.das b/modules/dasLLVM/daslib/llvm_jit_intrin.das index bede372963..74a5f5841d 100644 --- a/modules/dasLLVM/daslib/llvm_jit_intrin.das +++ b/modules/dasLLVM/daslib/llvm_jit_intrin.das @@ -182,10 +182,7 @@ let g_intrin_lookup <- { // AArch64 NEON intrinsics (the daslib/aarch64_neon module), consulted only when the JIT target is // aarch64. On any other target has_intrinsic returns false for these keys and the call's own // daslang body — the portable fallback in the declaring module — compiles as a normal function. -// That silent degrade-to-fallback contract holds for every table in this section. The baseline -// table is ARMv8.0 (TBL); the SDOT and SMMLA tables carry their own feature gates -// (g_target_arm64_dotprod / g_target_arm64_i8mm), because a generic-CPU exe targets the -// baseline machine, where either instruction is a fatal "Cannot select" at codegen. +// That silent degrade-to-fallback contract holds for every table in this section. let g_aarch64_intrin_lookup <- { "aarch64_neon::tbl16_lo" => @@intrinsic_tbl16, "aarch64_neon::tbl16_hi" => @@intrinsic_tbl16 @@ -1540,12 +1537,6 @@ def intrinsic_math_sinh_cosh_tanh(var ctx : JitCtx; expr : ExprCallFunc?; argume } // ===== the idot family: exact integer dots on the 8-bit lattice ===== -// Three lowerings: the signed x signed form emits ONE @llvm.aarch64.neon.sdot where the target has -// DotProd (g_target_arm64_dotprod: the host rail's +dotprod append, or the force env on the -// generic rail), the wasm SIMD128 form of idot_wasm_simd128 on a wasm target (the relaxed int8 -// dot through the sign trick, else extmul + pairwise adds); everything else gets generic -// widen-multiply IR. The AArch64 backend does NOT fold the generic form to SDOT (probe: -// zip/uzp/smull expansion) and the wasm backend runs it a fifth as fast, hence the native arms. // The unsigned x signed form needs USDOT (i8mm — absent on M1); it rides the i8mm un-gate item. def private idot_products(var ctx : JitCtx; expr : ExprCallFunc?; a, b : LLVMOpaqueValue?; aIdx : int) : LLVMOpaqueValue? { // nolint:LINT014 — shared by the idot intrinsics let v16i32 = LLVMVectorType(ctx.types.t_int32, 16u) @@ -1572,18 +1563,12 @@ def private idot_sdot_aarch64(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : return LLVMBuildCall2(ctx.builder, fnTy, decl, callArgs, "idot4") } -// The wasm SIMD128 form of the signed idot4: the ISA has no int8 dot, but it has the two halves of -// one - i16x8.extmul_{low,high}_i8x16_s (a sign-extending 8-lane multiply, which is what LLVM -// makes of mul(sext lo8, sext lo8)) and i32x4.extadd_pairwise_i16x8_s. Pairwise sums land as -// byte pairs (0-1, 2-3, ...), so one even/odd shuffle-add folds them into the quad lanes the -// generic form defines. Returns null when the LLVM at hand lacks the pairwise intrinsic. +// Returns null when this LLVM lacks the pairwise intrinsic, so callers fall through to the generic lowering. +[arch(at = "../ARCHITECTURE.md#idot-lowerings")] def private idot_wasm_simd128(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : LLVMOpaqueValue? { let v8i16 = LLVMVectorType(ctx.types.t_int16, 8u) let v4i32 = ctx.types.LLVMInt4Type() if (g_target_wasm_relaxed_simd) { - // relaxed SIMD has the dot itself: i32x4.relaxed_dot_i8x16_i7x16_add_s, exact when its second - // operand is in [0, 127]. The sign trick puts it there - dot(w, x) == dot(sign(x)*w, |x|) - and - // Q8_0 quants are in [-127, 127] by construction (d = amax/127), so no lane wraps on the flip. let rid = LLVMLookupIntrinsicID("llvm.wasm.relaxed.dot.i8x16.i7x16.add.signed") var noTypes : array var rdecl = rid != 0u ? LLVMGetIntrinsicDeclaration(g_mod, rid, noTypes) : null @@ -1623,6 +1608,7 @@ def private idot_wasm_simd128(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : return s } +[arch(at = "../ARCHITECTURE.md#idot-lowerings")] def intrinsic_lattice_idot(var ctx : JitCtx; expr : ExprCallFunc?; arguments : array) : LLVMOpaqueValue? { let aUnsigned = expr.arguments[0]._type.vectorBaseType == Type.tUInt8 if (g_target_is_wasm && !aUnsigned) { diff --git a/modules/dasLLVM/daslib/llvm_tune.das b/modules/dasLLVM/daslib/llvm_tune.das index 4d0892786e..5953a49c91 100644 --- a/modules/dasLLVM/daslib/llvm_tune.das +++ b/modules/dasLLVM/daslib/llvm_tune.das @@ -360,7 +360,6 @@ def private pin_module_cache_dependency(path : string) { add_module_cache_dependency(path) } -// every candidate on the class ladder a profile lookup tries, existing or not def private pin_profile_chain(defaultsDir : string) { var chain <- tune_class_chain() for (c in chain) { @@ -1566,7 +1565,7 @@ def private stamp_llvm_code(var fn : FunctionPtr; genKey : string; perm : TunePe aargs |> push((argname = "tune_suffix", argvalue = RttiValue(tString = perm.suffix))) aargs |> push((argname = "tune_from", argvalue = RttiValue(tString = from))) aargs |> push((argname = "tune_source", argvalue = RttiValue(tString = source))) - fn |> append_annotation("tune", "llvm_code", aargs) // the annotation is registered by daslib/tune, not by the module that implements it + fn |> append_annotation("tune", "llvm_code", aargs) fn.sideEffectFlags.userScenario = true delete aargs } @@ -1642,9 +1641,6 @@ def private emit_registry(var mod : Module?; base : FunctionPtr; perms : array)]` — one grid row for the `tune(...)` that must follow it in //! the same bracket. Args pass to the generator verbatim and fold into the JIT DLL cache //! key; `suffix="..."` overrides the auto-derived variant name. @@ -1752,7 +1748,7 @@ def tune_apply(var func : FunctionPtr; var _group : ModuleGroup; // nolint:STYLE pin_module_cache_dependency(mpath) var manifest <- read_manifest(mpath, manifestFound) var suffix = "" - var tuneSource = "" // manifest | profile | fallback - the tune_source the stamp records + var tuneSource = "" // manifest | profile | fallback var tuneSourcePath = "" if (manifestFound) { suffix = manifest?[fname] ?? "" @@ -1771,7 +1767,7 @@ def tune_apply(var func : FunctionPtr; var _group : ModuleGroup; // nolint:STYLE tuneSourcePath = profilePath } } - // clear an adopted winner this box can't run (ISA-gated → would decline to reference; or not in this build's grid → would crash-loop) so the fallback ;-chain picks a legal perm and it boots + // clear a winner this box can't run - ISA-gated (declines to reference) or absent from this build's grid (crash-loop) - so the fallback chain picks a legal perm if (!empty(suffix) && suffix != "reference") { var known = false var eligible = true @@ -1792,7 +1788,7 @@ def tune_apply(var func : FunctionPtr; var _group : ModuleGroup; // nolint:STYLE tuneSourcePath = "" } } - var companionOwnSuffix : array // a companion's OWN entry (the harness's gemv seat) when it names an eligible perm of this grid; else it follows the main + var companionOwnSuffix : array // a companion's own entry when it names an eligible perm of this grid; empty follows the main companionOwnSuffix |> resize(length(comps)) if (tuneSource == "manifest" || tuneSource == "profile") { for (ci, c in count(), comps) { @@ -2884,7 +2880,6 @@ class private TunePolicyDefaultMarker : AstFunctionAnnotation { //! play and no declared `[tune_policy]` behaves as `missing = "auto"`. Escapes and the //! once-only marker: `skills/tune.md`. A root without `main` never tunes (dastest files). def tune_default_policy_apply(_prog : ProgramPtr; _mod : Module?) : bool { - // fires only for roots that SEE daslib/tune, so a scope-owning library must re-export it public return false if (is_in_completion() || !empty("{compiling_module().name}") || empty(g_scopes)) warn_manifest_untuned() // self-gates on DAS_TUNE_MANIFEST; covers exe and declared-policy roots too return false if (tune_policy_gated() && !exe_status_report()) @@ -2902,7 +2897,6 @@ def tune_default_policy_apply(_prog : ProgramPtr; _mod : Module?) : bool { // standalone build: no policy, but the artifact self-reports its stamps emit_status_init(errs) } else { - // O0 cannot represent winners raced under O3 - default fallback (DAS_TUNE_POLICY wins) let flavor = jit_cli_opt_level() == 0 ? "fallback" : "auto" ok = apply_tune_policy(main_fn, flavor, errs) } diff --git a/tests-cpp/REVIEW.das b/tests-cpp/REVIEW.das index 66f7b4296c..8a6a0febe2 100644 --- a/tests-cpp/REVIEW.das +++ b/tests-cpp/REVIEW.das @@ -61,9 +61,7 @@ def private check_test_targets(f : string; text : string) { } } -// Every source a `set_source_files_properties` names exists beside its CMakeLists. The -// property applies to nothing when the file moves, so a test whose compile flag went -// missing builds the ordinary way and stays green. +//! every source a `set_source_files_properties` names exists beside its CMakeLists - a property naming a moved file applies to nothing and its test builds green without the flag def private check_source_property_paths(f : string; text : string) { let folder = dir_name(f) for (blk in cmake_command_blocks(text, "set_source_files_properties")) { diff --git a/tests/aot/test_struct_order.das b/tests/aot/test_struct_order.das index b8cbc4ac2a..2283192463 100644 --- a/tests/aot/test_struct_order.das +++ b/tests/aot/test_struct_order.das @@ -4,7 +4,7 @@ require dastest/testing_boost public // AOT emits structures in by-value dependency order, not source order: Outer embeds Inner, // which is declared later, and Inner's function-typed field names Outer in its signature. A -// sort that read the signature as a by-value dependency saw a cycle and fell back to source +// sort that reads the signature as a by-value dependency sees a cycle and falls back to source // order, where the generated C++ embeds an incomplete type. The AOT build of this file is the // gate; the assertions only keep the shapes alive. (An `array` field inside Inner would // be a real C++ cycle - TArray takes sizeof(T) - so that shape stays out of the test.) diff --git a/tests/jit_tests/_cross_target_fixture.das b/tests/jit_tests/_cross_target_fixture.das index bea777c9c1..553ac303d6 100644 --- a/tests/jit_tests/_cross_target_fixture.das +++ b/tests/jit_tests/_cross_target_fixture.das @@ -1,8 +1,8 @@ options gen2 -// child for cross_target_folds.das: prints the compile-time target folds, and which static_if arm -// the cross-target test the tune framework's inert gate uses selects, so the parent can compare a -// native compile against --jit-target= +// child for cross_target_folds.das: prints the compile-time target folds and which arm a static_if +// over the target triple selects (the same test the tune framework's inert gate makes), so the +// parent can compare a native compile against --jit-target= [export] def main { diff --git a/tests/jit_tests/aarch64_neon.das b/tests/jit_tests/aarch64_neon.das index 895a9aa230..1757a9acca 100644 --- a/tests/jit_tests/aarch64_neon.das +++ b/tests/jit_tests/aarch64_neon.das @@ -14,8 +14,7 @@ require daslib/aarch64_neon def sx8(v : int) : int => ((v & 0xff) ^ 0x80) - 0x80 -// byte e (0..3) of packed lane `lane`, sign-extended -def packed_byte(lane : int; e : int) : int => sx8(lane >> (e * 8)) +def packed_byte_sx(lane : int; e : int) : int => sx8(lane >> (e * 8)) def ref_sdot4(acc : int4; w : int8 const?; x : int8 const?) : int4 { var r = acc @@ -37,7 +36,7 @@ def ref_sdot4_w(acc : int4; w : int4; x : int8 const?) : int4 { for (j in range(4)) { var s = 0 for (e in range(4)) { - s += packed_byte(w[j], e) * int(x[4 * j + e]) + s += packed_byte_sx(w[j], e) * int(x[4 * j + e]) } r[j] += s } @@ -65,7 +64,7 @@ def ref_sdot4_laneq_w(acc : int4; w : int4; x : int8 const?; lane : int) : int4 for (j in range(4)) { var s = 0 for (e in range(4)) { - s += packed_byte(w[j], e) * int(x[4 * lane + e]) + s += packed_byte_sx(w[j], e) * int(x[4 * lane + e]) } r[j] += s } @@ -126,8 +125,7 @@ def fill_nibbles(var g : Lcg; var dst : array) { let TRIALS = 256 -// the helpers seed off a global each [test] writes, so a pure call with constant arguments -// cannot be folded at compile time and reaped before the JIT sees its call sites +//! each [test] bumps this before calling a helper: a pure call with constant arguments folds away at compile time, before the JIT ever sees its call sites var g_salt = 0u // mismatch count over TRIALS random int8 vectors; the lane is a compile-time constant of the diff --git a/tests/jit_tests/cross_target_folds.das b/tests/jit_tests/cross_target_folds.das index 2f88aed494..c3a6e145ea 100644 --- a/tests/jit_tests/cross_target_folds.das +++ b/tests/jit_tests/cross_target_folds.das @@ -1,7 +1,7 @@ // The target-side folds: get_target_triple / get_target_architecture_name answer for the artifact // a compile EMITS (the --jit-target triple), the host's values with no triple, and a static_if -// over the triple - the test the tune framework's inert gate makes - selects the cross arm, so a -// wasm or console artifact built on a tuned box carries reference bodies and no sidecar stamps. +// over the triple selects the cross arm (the tune framework's inert gate makes the same test), so +// a wasm or console artifact built on a tuned box carries reference bodies and no sidecar stamps. // Compile-only keeps the cross run on this host: the folds are decided from argv before the // first module parses, and main runs interpreted. The children run with the default module // cache on purpose: a --jit-target after the separator keys the cache, or a cached native diff --git a/tests/jit_tests/exe_host_cpu.das b/tests/jit_tests/exe_host_cpu.das index 2909618d24..796598263a 100644 --- a/tests/jit_tests/exe_host_cpu.das +++ b/tests/jit_tests/exe_host_cpu.das @@ -96,9 +96,7 @@ def test_exe_host_cpu(t : T?) { spawn_child("{neutral_env()}\"{bin}\" -exe -output \"{PLAIN_EXE}\" \"{PLAIN_SCRIPT}\"", lp) t |> success(!(lp |> has(HOST_CPU_LINE)), "kernel-free exe build must stay generic") - // a kernel-free exe that calls the feature-gated intrinsic families (SDOT, SMMLA, the AVX2 - // dot) stays on the generic baseline: the build must not emit an instruction that machine - // cannot select — the gates decline to the fallback bodies — and the artifact prints their answers + //! an intrinsic call alone must not promote the exe off the generic baseline - the gates decline to the fallback bodies, so the build emits nothing that machine cannot select var lb : array let rcb = spawn_child("{neutral_env()}\"{bin}\" -exe -output \"{BASELINE_EXE}\" \"{BASELINE_SCRIPT}\"", lb) t |> equal(0, rcb, "the intrinsic-calling kernel-free exe must build on the generic baseline: {join(lb, "\n")}") diff --git a/tests/language/optional_require.das b/tests/language/optional_require.das index fe7062117d..84a73a82ab 100644 --- a/tests/language/optional_require.das +++ b/tests/language/optional_require.das @@ -43,10 +43,7 @@ def test_optional_require_path_guard(t : T?) { // path guard resolved (daslib/fio is always mounted): the target loaded and its symbols work t |> equal(optional_require_fixture_pathguard::pathguard_marker(), 43) t |> success(typeinfo builtin_module_exists(optional_require_fixture_pathguard)) - // module_exists asks what is visible from this module, not the process registry: the - // loaded das target and a required builtin answer true, the skipped target and a name - // nobody required answer false - and a tool's nested compile, which promotes nothing, - // answers the same + //! module_exists is scoped to what THIS module can see, not the process registry - a loaded das target and a required builtin answer true, a skipped target and an unrequired name answer false t |> success(typeinfo module_exists(optional_require_fixture_pathguard)) t |> success(typeinfo module_exists(math)) t |> success(!typeinfo module_exists(optional_require_fixture_skipme)) diff --git a/utils/dasllama-server/README.md b/utils/dasllama-server/README.md index b16cd353b6..0c149396b1 100644 --- a/utils/dasllama-server/README.md +++ b/utils/dasllama-server/README.md @@ -235,6 +235,8 @@ Config precedence: `defaults < config TOML < explicit CLI flags` - unless the TO `authoritative = true` (what the control page saves), which flips the top: `defaults < CLI < authoritative TOML`. +### The sidecar exchange {#exchange-policy} + The sidecar exchange rides three config-only keys (no CLI flags - one code path). **The exchange is closed: both policies default to off, and only an explicit key turns them on** - a box its shipped class profile covers stamps its kernels at compile time and never needed the @@ -245,6 +247,9 @@ unverified NEVER auto-applies), `exchange_submit = ask | always | never` (defaul `always` shares it automatically), and `exchange_url` (baked default `https://dasllama.io`). `DASLLAMA_EXCHANGE_URL` / `DASLLAMA_EXCHANGE_ACCEPT` env override for tests and one-shot watchdog relaunches. Lookup failure is never fatal - the boot falls through to the local tuner. +Every `/exchange*` route resolves its policy from `g_config_save_path`, the path a `POST /config` +writes. That path covers every start shape - `--config`, auto-discovered, and config-less, where +the file may not exist yet and the baked defaults stand. The exchange client rides the tune framework, so a build without dasLLVM carries none of it: the boot resolver and submit check are never registered and every `/exchange*` route answers 404. diff --git a/utils/dasllama-server/REVIEW.md b/utils/dasllama-server/REVIEW.md index 53af926b18..7b26a739c0 100644 --- a/utils/dasllama-server/REVIEW.md +++ b/utils/dasllama-server/REVIEW.md @@ -35,3 +35,8 @@ fixture for the route that answers with that key first.** **A diff that adds a key to what a route answers lists it in that route's `README.md` row, in the same change.** The row is where a consumer learns the key exists. + +**A function in this folder that names a type from `dasllama_exchange` in its signature is a +defect.** That module is optional (`require ?llvm`) and its types do not exist in a build without +dasLLVM; call `exchange_policy_from_config(g_config_save_path)` inside a +`static_if (typeinfo module_exists(dasllama_exchange))` arm at each site instead. diff --git a/utils/dasllama-server/openai_server.das b/utils/dasllama-server/openai_server.das index 224569a2d7..d6010594a5 100644 --- a/utils/dasllama-server/openai_server.das +++ b/utils/dasllama-server/openai_server.das @@ -3560,11 +3560,6 @@ def private handle_config_save(var req : HttpRequest?; var resp : HttpResponse?) // ===== the exchange surface (GET /exchange + the apply/submit/retune levers) ===== -// Every route below reads its policy from `exchange_policy_from_config(g_config_save_path)` — the -// save path covers every start shape: --config, auto-discovered, and config-less (where the file -// may not exist yet and the baked defaults stand). It is spelled at each site rather than hoisted -// because `ExchangePolicy` is a type of the optional exchange module: without the tune framework -// there is no such type to name in a signature, and each route answers `exchange_absent` instead. def private exchange_absent(var resp : HttpResponse?) : http_status { // nolint:LINT022 — reached only from the no-framework static_if arm, which a build with dasLLVM drops return resp |> JSON(error_body("the sidecar exchange is not in this build (no tune framework)", "not_found"), http_status.NOT_FOUND) } @@ -3617,6 +3612,7 @@ def private deny_operator(var resp : HttpResponse?) : http_status { return resp |> JSON(error_body("loopback-only (operator surface)", "forbidden"), http_status.FORBIDDEN) } +[arch(at = "README.md#exchange-policy")] def private handle_exchange_get(var req : HttpRequest?; var resp : HttpResponse?) : http_status { if (!is_loopback_req(req)) { // the status carries the local sidecar path (username in it) — operator-only, like the rest diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index 4dedcbd21a..9d1ddf4712 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -2693,8 +2693,7 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; // with --list-shared-modules writing the referenced-module set. // --disable-module dashv: libhv is native-only (no wasm archive), so keep it // unloaded on the cross-compile host — a guarded `require ?dashv` then resolves - // as absent instead of pulling in a module whose wasm archive can't exist. The - // package's release_wasm_disable_module() names the others of that kind it reaches. + // as absent instead of pulling in a module whose wasm archive can't exist. // -project-root {root}: dyn-module discovery roots at the PACKAGE dir, not the // main script's dir. They differ when release_main() points outside the package // (e.g. an example whose .das_package sits beside the source) — without this the From 8ca7b8fcf45c2be00db8313ee613597194fb0149 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 14:02:57 -0700 Subject: [PATCH 35/43] the review-round fix batch: the framework is optional everywhere a portable program reaches it - the server, the bench, the profiling helpers and the modes fixture read the registry and the stamps only inside a module_exists arm, so every program root compiles with dasLLVM absent and the exe-rail lint sees no phantom registry; the team-lane wasm select is a def, the tune policy reads pin the module cache, forced JIT features announce themselves, the pages deploy checks every asset it stages, the deploy script splices the examples header idempotently, the program-roots census recurses, the no-tune arm refuses the image suites, and twenty-two rule and architecture documents say what their gates and code now do Co-Authored-By: Claude Fable 5.1 --- .github/workflows/REVIEW.md | 8 +- .github/workflows/pages.yml | 6 +- daslib/ARCHITECTURE.md | 15 ++++ daslib/REVIEW.md | 22 +++--- daslib/tune.das | 26 ++++++ doc/REVIEW.md | 7 +- doc/reflections/das2rst.das | 2 +- doc/source/reference/language/modules.rst | 6 +- examples/dasLLAMA/storyteller/main.das | 14 ++-- modules/dasLLAMA/ARCHITECTURE_ENGINE.md | 4 +- modules/dasLLAMA/ARCHITECTURE_TTS.md | 29 ++++--- modules/dasLLAMA/CLAUDE.md | 6 +- modules/dasLLAMA/REVIEW_IMAGE.md | 11 ++- modules/dasLLAMA/REVIEW_MEASUREMENT.md | 11 +-- modules/dasLLAMA/REVIEW_TTS.md | 3 +- modules/dasLLAMA/benchmarks/REVIEW.md | 9 ++- modules/dasLLAMA/benchmarks/lcpp_bench.das | 8 +- modules/dasLLAMA/dasllama/dasllama_math.das | 2 +- .../dasllama/dasllama_math_default.das | 8 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 6 +- .../dasLLAMA/dasllama/dasllama_styletts2.das | 7 ++ modules/dasLLAMA/dasllama/dasllama_tts.das | 4 +- modules/dasLLAMA/harness/REVIEW.md | 11 ++- modules/dasLLAMA/performance/REVIEW.md | 9 ++- modules/dasLLAMA/performance/model_specs.das | 9 ++- .../dasLLAMA/performance/profile_common.das | 79 +++++++++++++------ modules/dasLLAMA/portable_tier_plan.md | 15 ++-- modules/dasLLAMA/tests/CLAUDE.md | 15 +++- modules/dasLLAMA/tests/REVIEW.md | 14 ++-- modules/dasLLAMA/tests/run.das | 9 ++- .../dasLLAMA/tests/test_math_activations.das | 2 +- modules/dasLLAMA/tests/test_program_roots.das | 11 ++- modules/dasLLAMA/tests/test_tts_g2p.das | 24 ++++++ modules/dasLLVM/ARCHITECTURE.md | 31 ++++---- modules/dasLLVM/REVIEW.md | 35 ++++---- modules/dasLLVM/daslib/llvm_jit_code.das | 4 +- modules/dasLLVM/daslib/llvm_jit_common.das | 10 +-- modules/dasLLVM/daslib/llvm_jit_intrin.das | 18 +---- modules/dasLLVM/daslib/llvm_jit_run.das | 8 +- modules/dasLLVM/daslib/llvm_tune.das | 24 +----- .../dasLLVM/tests/llvm_tune_modes_client.das | 27 ++++--- site-dasllama/README.md | 8 ++ site-dasllama/REVIEW.md | 5 +- .../2026-09-05-storyteller-in-the-browser.md | 6 +- site-dasllama/examples.html | 4 +- site-dasllama/feed.xml | 6 +- site-dasllama/index.html | 6 +- site-dasllama/serve.py | 10 ++- site/files/examples.js | 2 +- skills/tune.md | 2 +- src/ast/ast_program.cpp | 6 +- src/builtin/ARCHITECTURE.md | 6 +- src/builtin/REVIEW.md | 25 +++--- src/builtin/module_builtin_runtime.cpp | 29 ++++--- tests-cpp/REVIEW.md | 7 +- tests/aot/test_struct_order.das | 27 ++++++- tests/jit_tests/_cross_target_fixture.das | 5 ++ tests/jit_tests/cross_target_folds.das | 7 ++ tests/jit_tests/wasm_idot_lowering.das | 22 +++--- tests/language/optional_require.das | 11 +++ .../optional_require_fixture_hidden.das | 8 ++ .../optional_require_fixture_private.das | 8 ++ tests/metal/REVIEW.md | 4 +- utils/daslang/main.cpp | 7 +- utils/dasllama-convert/main.das | 7 ++ utils/dasllama-server/REVIEW.md | 10 +-- utils/dasllama-server/ask.das | 2 +- utils/dasllama-server/main.das | 11 ++- utils/dasllama-server/openai_server.das | 2 +- utils/dasllama-server/test_model_catalog.das | 6 +- utils/dasllama-server/txt2wav.das | 2 +- utils/dasllama-server/wav2txt.das | 2 +- utils/daspkg/REVIEW.md | 16 ++-- utils/daspkg/commands.das | 6 ++ .../daspkg/fixtures/test_release.das_package | 2 + utils/daspkg/test_daspkg.das | 6 ++ .../dasllama-ladder/dasllama-deploy.sh | 28 +++++++ web/REVIEW.md | 3 +- 78 files changed, 585 insertions(+), 308 deletions(-) create mode 100644 tests/language/optional_require_fixture_hidden.das create mode 100644 tests/language/optional_require_fixture_private.das diff --git a/.github/workflows/REVIEW.md b/.github/workflows/REVIEW.md index a385ce8870..012d13838b 100644 --- a/.github/workflows/REVIEW.md +++ b/.github/workflows/REVIEW.md @@ -23,6 +23,8 @@ lane's platform, in its PR body or commit message; a green run of that lane on t commit is that evidence.** A check that fails for a non-defect turns a green branch red for everyone. -**A step in `pages.yml` that names the deployed games spells the list as a `for g in ; do` -loop, never inline.** `examples/games/REVIEW.das` (repo root) reads the deployed list from -those loops; an inline list beside a surviving loop is one nothing cross-checks. +**A step in `pages.yml` that names more than one id under `examples/games/` spells them as a +`for g in ; do` loop, never inline.** `examples/games/REVIEW.das` (repo root) reads the +deployed list from those loops; an inline list beside a surviving loop is one nothing +cross-checks. An example outside `examples/games/` (a graphics showcase, a dasLLAMA app) is not +on that list and is named on its own. diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 0b0188da04..e81ace6350 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -594,7 +594,11 @@ jobs: && [ -f web/output64/examples/storyteller/storyteller.js ] \ && [ -f web/output64/examples/storyteller/storyteller.wasm ] \ && gh release download dasllama-web --repo GaijinEntertainment/daScript \ - --dir _site_dasllama/examples/storyteller/models --clobber; then + --dir _site_dasllama/examples/storyteller/models --clobber \ + && test -s _site_dasllama/examples/storyteller/models/stories15M.dlim \ + && test -s _site_dasllama/examples/storyteller/models/kitten-nano.dlim \ + && test -s _site_dasllama/examples/storyteller/models/tts_g2p_en_us.bin \ + && test -s _site_dasllama/examples/storyteller/models/tts_postag.bin; then cp web/output64/examples/storyteller/storyteller.html _site_dasllama/examples/storyteller/ cp web/output64/examples/storyteller/storyteller.js _site_dasllama/examples/storyteller/ cp web/output64/examples/storyteller/storyteller.wasm _site_dasllama/examples/storyteller/ diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index 97fe1f9703..fd1c76d356 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -222,3 +222,18 @@ Three companions carry a concern each; a section number is unique across all fou - **`_table_index_and_init` exists for infer's `default_init_containers` rewrite of non-store `tab[key]`** - it has no daslib call site and is not dead. + +## 37. tune {#tune-shells-module-exists} + +- **Every tune annotation - `[tune]`, `[tuned]`, `[tune_perm]`, `[tune_scope]`, `[tune_policy]`, + `[llvm_code]` - registers HERE and nowhere else, as a shell that forwards to the framework in + `llvm/daslib/llvm_tune` when that module is visible and answers "unchanged" when it is not.** A + module's macro state is per module, so a shell and the `[tune_scope]` bank it reads have to + share one macro context; `tune_status` and `log_tune_status` are shells of the same kind. +- **The shells decide with `typeinfo module_exists(llvm_tune)`, a pair with the trait's C++ half + in `src/ast/ast_infer_type.cpp` (`module_exists`): visible from the compiling module, which is + what the `require ?llvm llvm/daslib/llvm_tune public` line above them decided.** Nothing fails + when the two sides drift - `tests/language/optional_require.das` pins the trait's visibility + half on fixtures, and `tests/daslib/test_tune_shells.das` is tier-agnostic by design - so the pair + is recorded here: a trait that answered off the process registry instead would take the + framework arm inside a tool's nested compile, where the framework is not mounted. diff --git a/daslib/REVIEW.md b/daslib/REVIEW.md index 218c3b9f19..0dc0e20b8a 100644 --- a/daslib/REVIEW.md +++ b/daslib/REVIEW.md @@ -264,13 +264,15 @@ A chain of template ancestors delivers one copy of the annotation per level, so the marker runs a second time over a structure it already rewrote. **A diff that adds or renames a public intrinsic in `aarch64_neon.das`, `x64_avx.das` or -`f16_cvt.das` changes the matching row of the lookup tables in -`modules/dasLLVM/daslib/llvm_jit_intrin.das` (repo root) in the same change.** The JIT recognizes -these calls by function name alone, so a missing row runs the portable fallback on every target -with every test green - the `tests/jit_tests` twins compare hardware against a reference body and -catch a wrong emitter, never a missing one. - -**A function in `x64_avx.das` that exists to reach a wider or newer CPU tier than another -function in that file computes its result by calling that function - never by a hand-written -scalar loop.** A box whose CPU lacks the tier then runs the recognized narrower path instead of -scalar code. +`f16_cvt.das` changes its row in the lookup table of `modules/dasLLVM/daslib/llvm_jit_intrin.das` +(repo root) that is gated on the CPU feature the instruction needs, in the same change.** The JIT +recognizes these calls by function name alone, so a missing row runs the portable fallback on +every target with every test green, and a row in a table gated on a weaker feature emits an +instruction the generic machine cannot select - the `tests/jit_tests` twins compare hardware +against a reference body and catch a wrong emitter, never a missing or misplaced one. + +**A function in `aarch64_neon.das`, `x64_avx.das` or `f16_cvt.das` that is the wider- or +newer-tier variant of the same math another function in that file computes calls that function - +never a hand-written scalar loop.** A box whose CPU lacks the tier then runs the recognized +narrower path instead of scalar code. A tier function with no same-math sibling (`smmla`'s 2x8 +by 8x2 shape) keeps its own fallback body. diff --git a/daslib/tune.das b/daslib/tune.das index 4305e25eca..04200df6bf 100644 --- a/daslib/tune.das +++ b/daslib/tune.das @@ -107,3 +107,29 @@ class private TuneDefaultPolicyShell : AstPassMacro { } } } + +//! Log the tune status table at `LOG_INFO`, prefixed by `appName`: `/ kernels tuned for +//! this box`, one line per function, a `--tune` hint when any kernel sits on a fallback tier. +//! Silent when the table is empty, and in a build without the framework - nothing was stamped. +[arch(at = "ARCHITECTURE.md#tune-shells-module-exists")] +def log_tune_status(appName : string) { + static_if (typeinfo module_exists(llvm_tune)) { + var st <- tune_status() + if (empty(st)) { + delete st + return + } + var tuned = 0 + for (s in st) { + tuned += (s.source == "manifest" || s.source == "profile") ? 1 : 0 + } + to_log(LOG_INFO, "{appName}: tune status ({tuned}/{length(st)} kernels tuned for this box):\n") + for (s in st) { + to_log(LOG_INFO, " {s.fname}: {empty(s.suffix) ? "reference" : s.suffix} ({s.source})\n") + } + if (tuned < length(st)) { + to_log(LOG_INFO, "{appName}: run with --tune to tune the remaining kernels for this box\n") + } + delete st + } +} diff --git a/doc/REVIEW.md b/doc/REVIEW.md index 0b31b8883d..c3733304cb 100644 --- a/doc/REVIEW.md +++ b/doc/REVIEW.md @@ -1,9 +1,8 @@ # doc (the Sphinx manual) Code Review Checklist -**Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** RST -mechanics are `skills/internal/documentation_rst.md` (repo root) and the prose itself is -`skills/internal/tutorial_prose.md` (repo root); how a page reads is reviewed there, not -here. +**Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture +doc: `skills/internal/documentation_rst.md` (repo root). The prose itself is +`skills/internal/tutorial_prose.md` (repo root); how a page reads is reviewed there, not here. **Weakening `REVIEW.das` (beside this file) is a defect - a diff that makes it read fewer files, or report less than it did before the change with the same `ALLOWED_HOSTS`, is weaker.** diff --git a/doc/reflections/das2rst.das b/doc/reflections/das2rst.das index aeed88d1b8..10808c50f6 100644 --- a/doc/reflections/das2rst.das +++ b/doc/reflections/das2rst.das @@ -199,7 +199,7 @@ get_value|insert_clone|emplace_new|insert_default|emplace_default|get_with_defau group_by_regex("Binary serializer", mod, %regex~(binary_load|binary_save)$%%), group_by_regex("Path and command line", mod, %regex~(get_command_line_arguments|with_argv)$%%), group_by_regex("Time and date", mod, %regex~(get_time_usec|ref_time_ticks|get_clock|get_time_nsec|mktime|iso8601_now|format_time|clock)$%%), - group_by_regex("Platform queries", mod, %regex~(get_context_share_counter|das_is_dll_build|das_is_exceptions_enabled|is_standalone_exe|get_platform_name|get_running_platform_name|get_cross_platform_name|get_architecture_name|cpu_supports)$%%), + group_by_regex("Platform queries", mod, %regex~(get_context_share_counter|das_is_dll_build|das_is_exceptions_enabled|is_standalone_exe|get_platform_name|get_running_platform_name|get_cross_platform_name|get_architecture_name|get_target_triple|get_target_architecture_name|cpu_supports)$%%), group_by_regex("String formatting", mod, %regex~(fmt)$%%), hide_group(group_by_regex("Temp string reclaim", mod, %regex~(_temp_string_result)$%%)), group_by_regex("Argument consumption", mod, %regex~(consume_argument)$%%), diff --git a/doc/source/reference/language/modules.rst b/doc/source/reference/language/modules.rst index 3c4db6d264..d3d4335a4c 100644 --- a/doc/source/reference/language/modules.rst +++ b/doc/source/reference/language/modules.rst @@ -104,7 +104,11 @@ same program in a nested context (lint, the language server, a test harness) promotes nothing, and there the trait answers false for every das target. Guard the use of a guarded das target with ``typeinfo module_exists(target)`` instead: it asks whether the target is visible from the compiling module, which is exactly what the -guarded ``require`` decided, so the answer is the same on both rails. +guarded ``require`` decided, so the answer is the same on both rails. Two edges of that +rule: the name must be the module's own name (an alias from ``require X as Y`` answers +false, like any name the library does not know), and inside a generic function the +compiling module is the one INSTANTIATING the generic, not the one that declared it - a +guard in a generic body reads the caller's view. Pair it with :ref:`typeinfo builtin_module_exists ` to guard code that uses the optional target's symbols — ``static_if`` drops the untaken diff --git a/examples/dasLLAMA/storyteller/main.das b/examples/dasLLAMA/storyteller/main.das index 2c9fd96c95..2d38b2d32a 100644 --- a/examples/dasLLAMA/storyteller/main.das +++ b/examples/dasLLAMA/storyteller/main.das @@ -25,7 +25,8 @@ require strings // // bin/daslang -jit examples/dasLLAMA/storyteller/main.das -- --models // -// holds the two models and the front-end packs tts_g2p.bin and tts_postag.bin. A model +// holds the two models and the front-end packs - tts_g2p.bin (or its American-only twin +// tts_g2p_en_us.bin, which the web set ships) and tts_postag.bin. A model // is its gguf (the default names) or a prepared .dlim image baked for the running build's // identity (dasllama-convert --config) - the web build ships images only, the shell names them. // The language model runs on the frame thread in per-frame token budgets, so the loop never @@ -47,9 +48,12 @@ struct StoryArgs { @clarg_doc = "Voice name or alias (default: the model's last voice)" voice : string - @clarg_doc = "Start a story at once and quit when it has been read out - the smoke rail" + @clarg_doc = "Start a story at once (the browser shell's start button) - the loop then runs on, space for the next" autoplay : bool + @clarg_doc = "Start a story at once and quit when it has been read out - the smoke rail" + smoke : bool + @clarg_doc = "Stop after this many frames (0 = never) - the smoke rail" max_frames : int @@ -314,7 +318,7 @@ def init() { g_speech_done = unsafe(channel_create()) g_speech_done |> append(1) start_speech_thread(path_join(g_args.models, g_args.tts_model), g_args.voice, g_say, g_speech, g_speech_done) - if (g_args.autoplay) { + if (g_args.autoplay || g_args.smoke) { start_story() } } @@ -372,7 +376,7 @@ def shutdown() { def done_for_smoke() : bool { return true if (g_args.max_frames > 0 && g_frames >= g_args.max_frames) - return g_args.autoplay && g_phase == Phase.told && g_unplayed_sentences == 0 && g_elapsed_s >= g_speaking_until + return g_args.smoke && g_phase == Phase.told && g_unplayed_sentences == 0 && g_elapsed_s >= g_speaking_until } // eval_main_loop drives the block once per frame: a blocking while-loop natively, the @@ -386,7 +390,7 @@ def main() { return !exit_requested() } shutdown() - if (g_args.autoplay) { + if (g_args.smoke) { print("{join(g_wrapped_lines, "\n")}\n") } } diff --git a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md index c4b564ef98..7f6ecdd8b5 100644 --- a/modules/dasLLAMA/ARCHITECTURE_ENGINE.md +++ b/modules/dasLLAMA/ARCHITECTURE_ENGINE.md @@ -188,8 +188,8 @@ trunk-only and trunk+head images never collide and one image file serves both tr registration. A hand-written tile that the generator could emit belongs in the generator. - **`dasllama_tune.das`** - the per-box loop-hint tuner (`[tuned]` / `[dasllama_grid]`). Tuning POLICY lives here; tuned VALUES live in the box's sidecar, never in source. The framework is - OPTIONAL: the `llvm_tune` require is path-guarded and every use of it sits behind - `static_if (typeinfo builtin_module_exists(llvm_tune))`, so a build without dasLLVM opens no + OPTIONAL: the `llvm_tune` require is `?llvm`-guarded and every use of it sits behind + `static_if (typeinfo module_exists(llvm_tune))`, so a build without dasLLVM opens no sidecar, reads no policy env and reports nothing - `[tuned]` stamps the declared `fallback=` (or `DEFAULT_PERM`) and the AST rewrite is unchanged. `box_profile_verdict_at` answers the reason NAME rather than llvm_tune's enum for the same reason: a signature cannot be gated. diff --git a/modules/dasLLAMA/ARCHITECTURE_TTS.md b/modules/dasLLAMA/ARCHITECTURE_TTS.md index 50f0e5f4aa..1582b47487 100644 --- a/modules/dasLLAMA/ARCHITECTURE_TTS.md +++ b/modules/dasLLAMA/ARCHITECTURE_TTS.md @@ -44,7 +44,8 @@ TTS files implement (sec.2.28-2.35, 2.43). `ARCHITECTURE_COMMON.md` (repo root) the DRESS vowel before a linking rhotic as SQUARE, having nothing in the string to tell merry from Mary. A vowel the two lexicons give no evidence for before a dropped rhotic keeps that rhotic rather than losing it. The bath-trap split reaches only lexicon - words. Loads `tts_g2p.bin`, pack + words. Loads a phoneme pack - `tts_g2p.bin` (both dialect tiers) or `tts_g2p_en_us.bin` (the + American tier alone, sec.2.43) - pack version 2 (`harness/build_g2p_data.py`: the gold tier extended by `harness/g2p_local_additions.json`, the US and GB keys merged into one string table per tier, the GRU stored as f16, CMUdict pruned of the words both dialects' lexicons carry - @@ -56,7 +57,9 @@ TTS files implement (sec.2.28-2.35, 2.43). `ARCHITECTURE_COMMON.md` (repo root) parity rails for all three stages. - **`dasllama_tts_types.das`** - the TTS floor: `TtsCaps`, `TtsAudio` (f32 PCM + rate), `TtsNoise` (the source noise a synthesis consumed - captured from the oracle, or drawn into a reused - carrier). Family files require this, never each other. + carrier), and the two family data records the shared carrier serializes - `KittenFamily` (speed + priors, voice aliases) and `KokoroFamily` (the symbol vocabulary) - plain data, no family logic. + Family files require this, never each other. - **`dasllama_tts_blocks.das`** - the StyleTTS2-lineage block home, the TTS twin of `dasllama_tower.das`, in the two layouts of sec.2.28: Conv1d (dense, depthwise, forward and transposed), the dense layer, LayerNorm over rows and over channels, InstanceNorm and AdaIN, @@ -85,23 +88,27 @@ TTS files implement (sec.2.28-2.35, 2.43). `ARCHITECTURE_COMMON.md` (repo root) rows-form input, the style and the source spectrum as rows and answers with the waveform or declines; the SineGen phase chain and the harmonic STFT stay on the CPU in both routes (sec.2.33), the trace rail keeps the CPU chain, and engage is read from the counters. No - driver fills the slot yet. The assembly names no family; a family's quirk lives in its family - file. + driver fills the slot yet. The carrier also holds each family's DATA - the `KittenFamily` / + `KokoroFamily` records of `dasllama_tts_types.das`, read from the GGUF's `kitten.*` / + `kokoro.symbol_*` metadata by `stage_family_data` - because the image meta serializes them and a + `.dlim` load has no GGUF to read them from (sec.2.32); the family LOGIC that interprets those + records lives in the family files. - **`dasllama_kitten.das`** - the KittenTTS family (nano and mini): the reference driver's symbol - table, re-spacing rule and style-row rule (the chunk's character count), its speed priors and - voice aliases (`kitten.*` metadata), its 5000-sample tail trim, and the rewrite of the front - end's inventory into the espeak-style IPA these models consume. A phoneme the symbol table + table, re-spacing rule and style-row rule (the chunk's character count), how the speed priors + and voice aliases of its `KittenFamily` record are applied, its 5000-sample tail trim, and the + rewrite of the front end's inventory into the espeak-style IPA these models consume. A phoneme the symbol table does not carry takes its separator space with it, so a dropped symbol never leaves a doubled space token where the model was trained on none. - **`dasllama_kokoro.das`** - the Kokoro family (Kokoro-82M): the reference pipeline's - vocabulary (`kokoro.symbol_*` metadata - the front end's own inventory, no rewrite), its token - wrapping and style-row rule (the phoneme string's character count less one). Fifty-four voice + vocabulary (the `KokoroFamily` symbol record - the front end's own inventory, no rewrite), its + token wrapping and style-row rule (the phoneme string's character count less one). Fifty-four voice packs of 510 rows, each named `_`; a pack's language is read from that shape and only from it, because the language letters are ordinary first letters of ordinary names, so a name of any other shape has no language rather than the one its first letter spells. - **`dasllama_tts.das`** - the TTS facade: `load_tts_model` (the shared model plus the family - picked by `general.architecture`; `tts_g2p.bin` and `tts_postag.bin` read from the GGUF's - directory; the packs it leaves out are named once in the log), `caps` (the voices the front + picked by `general.architecture` - from a GGUF or from a prepared `.dlim`; the phoneme pack + and `tts_postag.bin` read from the model's directory, the full pack preferred over the + American-only twin, sec.2.43; the packs it leaves out are named once in the log), `caps` (the voices the front end can drive - a Kokoro pack's name carries its language, and only the languages the family declares are listed or accepted; the rest refuse with the language in the message, or, where the name carried none, with the fact that the front end cannot phonemize it), `tts_voice_lang` diff --git a/modules/dasLLAMA/CLAUDE.md b/modules/dasLLAMA/CLAUDE.md index e7b5d330bd..6e38212224 100644 --- a/modules/dasLLAMA/CLAUDE.md +++ b/modules/dasLLAMA/CLAUDE.md @@ -16,8 +16,10 @@ directly). A changed file in a model suite (decode | mtp | prefill | matrix | im area - `--changed` names it and runs nothing for it; run that suite with `--arm` scoped to the change. Before a PR: `--suite model-free`, then `--suite stocked` on a box with models - a PR's `stocked` run carries no `--exclude`; `--exclude test_ple_modes` is the between-PR iteration -form. The runner's `REPORT` and `NEXT` lines say what ran, what the run skipped (a filtered arm, -an absent device, or a missing model), and what is still owed. The rest: `tests/CLAUDE.md`. +form. `--no-tune` runs any suite but the image suites on the reference kernel bodies +(`DAS_TUNE_POLICY=reference`, the image rail off) - the portable tier's arm. The runner's `REPORT` +and `NEXT` lines say what ran, what the run skipped (a filtered arm, an absent device, or a +missing model), and what is still owed. The rest: `tests/CLAUDE.md`. Follow the daslang **gen2** conventions - the root `CLAUDE.md` rules apply to every `.das` file here. diff --git a/modules/dasLLAMA/REVIEW_IMAGE.md b/modules/dasLLAMA/REVIEW_IMAGE.md index 4f2883f99b..06158d13b5 100644 --- a/modules/dasLLAMA/REVIEW_IMAGE.md +++ b/modules/dasLLAMA/REVIEW_IMAGE.md @@ -53,10 +53,13 @@ property shapes never does.** A config-bound identity on a property-free family image per tune state and reaps the others on every switch; a config-free identity on a property-shaped family serves the wrong bytes under a changed property. -**A diff that moves a byte in a `.dlim`, adds or drops a serialized field, or changes what an -image at an UNCHANGED path contains, bumps `IMAGE_VERSION` (`dasllama/dasllama_image.das`) in -the same change; weakening the `REVIEW.das` layout-stamp check - narrowing its closure or -dropping the cell - is a defect.** Without the bump a stale image stays structurally valid and +**A diff that changes what an image at an UNCHANGED path contains without changing the meta +closure's struct shape - a moved byte, a re-meaning of a serialized field, a serializer body +change that the layout fingerprint cannot see - bumps `IMAGE_VERSION` (`dasllama/dasllama_image.das`) +in the same change; weakening the `REVIEW.das` layout-stamp check - narrowing its closure or +dropping the cell - is a defect.** A struct field added or dropped moves `layout_fingerprint()`, +which the load compares by name, so that image refuses loudly and a re-stamp of the closure hash +discharges it; without the fingerprint's help a stale image stays structurally valid and silently serves a different model. What the stamp covers is the layout stamp section of `ARCHITECTURE_IMAGE.md`. diff --git a/modules/dasLLAMA/REVIEW_MEASUREMENT.md b/modules/dasLLAMA/REVIEW_MEASUREMENT.md index 6633687b55..5629a1a1dc 100644 --- a/modules/dasLLAMA/REVIEW_MEASUREMENT.md +++ b/modules/dasLLAMA/REVIEW_MEASUREMENT.md @@ -47,11 +47,12 @@ decision in the report of the run that took it and in the PR that lands the kern winner enters the ledger only through a re-measured board cell. **A diff that makes the engine serve tokens by a route no board cell exercises mints that cell -in the same change.** A route is what a request takes end to end: a weight format, a modality, -a family, a backend, a serving lane (q8 or f32), a GPU tower (a GPU-run vision or audio -encoder), a sampler class (greedy, sampled) through the speculative round (a decode step that -drafts several tokens and verifies them in one pass), or the path a run with no flags and no -environment overrides takes. +in the same change.** A route is anything that changes which code serves the tokens end to end: +a weight format, a modality, a family, a backend, a serving lane (q8 or f32), a GPU tower (a +GPU-run vision or audio encoder), a sampler class (greedy, sampled) through the speculative +round (a decode step that drafts several tokens and verifies them in one pass), a compile tier +(JIT, AOT, interpreted reference bodies), a cross target (wasm64), or the path a run with no +flags and no environment overrides takes. **A diff that claims to make an already-served path faster, from an author whose box mints that path, re-mints a board row (`performance/records/.json`) that exercises that path, in diff --git a/modules/dasLLAMA/REVIEW_TTS.md b/modules/dasLLAMA/REVIEW_TTS.md index e65974f7f4..74c8d3d293 100644 --- a/modules/dasLLAMA/REVIEW_TTS.md +++ b/modules/dasLLAMA/REVIEW_TTS.md @@ -65,7 +65,8 @@ family's quirk goes in its family file.** **A change to a kernel, a lexicon, a normalizer rule, or a phoneme rule ships the WER and UTMOS of `harness/tts_rig.py`, before and after, on every model the change reaches at the -rig's voice, in the PR body.** +rig's voice, in the PR body - unless a corpus-identity cell in `tests/test_tts_g2p.das` proves +the change moves no phoneme of the 200-sentence corpus, which pins the audio bit for bit.** **A text normalization or grapheme-to-phoneme error `harness/tts_rig.py`'s transcripts expose lands as a failing-first case in `tests/test_tts_textnorm.das` or diff --git a/modules/dasLLAMA/benchmarks/REVIEW.md b/modules/dasLLAMA/benchmarks/REVIEW.md index e5b635f1f4..825b800031 100644 --- a/modules/dasLLAMA/benchmarks/REVIEW.md +++ b/modules/dasLLAMA/benchmarks/REVIEW.md @@ -71,10 +71,11 @@ no attribution across stages - is not a difference. and corpus - or withdraws the affected rows and names the withdrawal and its reason in the PR body.** A board cell is a timed cell of the published results board: one `../performance/gen_bench_records.das` spawns, or a manual `lcpp_bench.das` cell with its own -`../PROFILE.md` section. What a cell times changes when a change to its code, to its input -corpus, or to the pinned reference build - anything that decides which reference binary or -reference environment the run measures, `DEFAULT_REF_SHA` in `setup_lcpp_ref.das` included - -moves the measured quantity. The re-mint or withdrawal lands in +`../PROFILE.md` section. What a cell times changes when a change inside its timed body, to its +input corpus, or to the pinned reference build (`DEFAULT_REF_SHA` in `setup_lcpp_ref.das`, or +anything else deciding which reference binary or environment the run measures) moves the +measured quantity; a change outside the timed body - a flag, a require, the submit path - does +not. The re-mint or withdrawal lands in `../performance/records/.json`, the file the affected rows live in. **A diff that adds or changes an instrument makes it exit non-zero on a run that ends diff --git a/modules/dasLLAMA/benchmarks/lcpp_bench.das b/modules/dasLLAMA/benchmarks/lcpp_bench.das index 4a0d88528f..dc8c80efc2 100644 --- a/modules/dasLLAMA/benchmarks/lcpp_bench.das +++ b/modules/dasLLAMA/benchmarks/lcpp_bench.das @@ -32,7 +32,7 @@ require daslib/jobque_profile // JOBQUE_PROFILING_ENABLED — --prof refuses o require ?das_accelerate dasllama/dasllama_math_accelerate // the --accel leg; Apple-only C++ module require ../performance/profile_common.das // profile_threads + affinity_on + bench records — the standing per-box methodology require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: --submit POSTs the stamped records to the exchange, which rides the tune framework -require llvm/daslib/llvm_tune // tune_app_script — the exe-vs-script discriminator behind --for-debug-purposes +require ?llvm llvm/daslib/llvm_tune // nolint:STYLE030,LINT019 — OPTIONAL: tune_app_script, the exe-vs-script discriminator behind --for-debug-purposes require daslib/jobque_boost require daslib/clargs require daslib/fio @@ -1306,7 +1306,11 @@ def main() : int { } // measurement exists only from the released exe: a -jit SCRIPT run reads whatever sidecar // sits beside the script and re-keys nothing — its numbers must never enter records - g_debug_run = !empty(tune_app_script()) + static_if (typeinfo module_exists(llvm_tune)) { + g_debug_run = !empty(tune_app_script()) + } else { + g_debug_run = true // no framework, no released exe: a reference-kernel run is never a record + } if (g_debug_run && !cfg.for_debug_purposes) { print("lcpp_bench: this is a -jit SCRIPT run - measurement runs the released exe (daspkg release).\n") print(" pass --for-debug-purposes to proceed as a debug instrument; output is stamped debug-jit and is not record-grade\n") diff --git a/modules/dasLLAMA/dasllama/dasllama_math.das b/modules/dasLLAMA/dasllama/dasllama_math.das index a40cda8247..8bab029d7b 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math.das +++ b/modules/dasLLAMA/dasllama/dasllama_math.das @@ -296,7 +296,7 @@ def dasllama_jobque_threads_cap() { // Elsewhere mode 2 is a REAL mask — stays opt-in. DAS_JOBQUE_AFFINITY and app calls override. [init] def dasllama_jobque_affinity_default() { - if (get_platform_name() == "darwin" && get_jobque_affinity() < 0) { + if (get_platform_name() == "darwin" && empty(get_target_triple()) && get_jobque_affinity() < 0) { // the HOST's darwin, never a cross artifact's set_jobque_affinity(2) } } diff --git a/modules/dasLLAMA/dasllama/dasllama_math_default.das b/modules/dasLLAMA/dasllama/dasllama_math_default.das index 44162a2cf8..93f1120af2 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_default.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_default.das @@ -219,15 +219,17 @@ def dot_q8q8_idot4_ps_f16s(wq : int8 const?; ws : uint16 const?; xq : int8 const return f.x + f.y + f.z + f.w } -let private WASM_TARGET = get_target_architecture_name() == "wasm64" || get_target_architecture_name() == "wasm32" +//! a def, not a module global: team lanes run no global initializers, and the fold below is a +//! compile-time constant either way +def private wasm_target() : bool => get_target_architecture_name() == "wasm64" || get_target_architecture_name() == "wasm32" [arch(at = "../ARCHITECTURE_CPU_KERNELS.md#cpu-tier-target-select")] def private dot_q8q8_p(wq : int8 const?; ws : float const?; xq : int8 const?; xs : float const?; n : int64) : float { - return WASM_TARGET ? dot_q8q8_idot4_ps(wq, ws, xq, xs, n) : dot_q8q8(wq, ws, xq, xs, n) + return wasm_target() ? dot_q8q8_idot4_ps(wq, ws, xq, xs, n) : dot_q8q8(wq, ws, xq, xs, n) } def private dot_q8q8_f16s_p(wq : int8 const?; ws : uint16 const?; xq : int8 const?; xs : float const?; n : int64) : float { - return WASM_TARGET ? dot_q8q8_idot4_ps_f16s(wq, ws, xq, xs, n) : dot_q8q8_f16s(wq, ws, xq, xs, n) + return wasm_target() ? dot_q8q8_idot4_ps_f16s(wq, ws, xq, xs, n) : dot_q8q8_f16s(wq, ws, xq, xs, n) } // ----- parallel_for dispatch profiler (off by default) ----- diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 5a2b149109..61420ce10e 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -12,7 +12,7 @@ require dasllama/dasllama_math_default // dot_q8q8_f16s — the row-major tails require dasllama/dasllama_repack // the grp repack family + kq panel unpack (pure transforms; the _gen/_bake twins below feed them numbers) require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module -require ?llvm llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; PATH guard: the one framework call below sits under static_if +require ?llvm llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; witness guard: the one framework call below sits under static_if require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public @@ -2749,9 +2749,11 @@ def dasllama_math_gen_register() { // untuned box still pays the minutes-long auto-tune before the gate ever runs. The log // line is unconditional-when-set — active overrides announce themselves if (g_env_bench.allow_untuned) { - to_log(LOG_WARNING, "dasLLAMA: DASLLAMA_ALLOW_UNTUNED=1 - tune mint suppressed; untuned kernels run fallback, numbers are not publishable\n") static_if (typeinfo module_exists(llvm_tune)) { + to_log(LOG_WARNING, "dasLLAMA: DASLLAMA_ALLOW_UNTUNED=1 - tune mint suppressed; untuned kernels run fallback, numbers are not publishable\n") tune_suppress_mint("DASLLAMA_ALLOW_UNTUNED=1") + } else { + to_log(LOG_WARNING, "dasLLAMA: DASLLAMA_ALLOW_UNTUNED=1 - no tune framework in this build; every kernel runs its reference body, numbers are not publishable\n") } } // the cross-box override's q51 reach: x64-gen keeps disk-order q51 slots, so the generic diff --git a/modules/dasLLAMA/dasllama/dasllama_styletts2.das b/modules/dasLLAMA/dasllama/dasllama_styletts2.das index 4daa0e99ec..5c5be57347 100644 --- a/modules/dasLLAMA/dasllama/dasllama_styletts2.das +++ b/modules/dasLLAMA/dasllama/dasllama_styletts2.das @@ -290,6 +290,10 @@ def private serialize_image_meta(var arch : Archive; var m : StyleTts2Model) { arch |> serialize(m.decoder) arch |> serialize(m.source) arch |> serialize_raw(m.q8) + // the family records are hand-listed too, so each pins its own count: a field added to one of + // them and not written here would read back zero on every image load + verify(count_meta_fields(m.kitten) == 2) // grew KittenFamily? extend this list + verify(count_meta_fields(m.kokoro) == 1) // grew KokoroFamily? extend this list arch |> serialize(m.kitten.speed_priors) arch |> serialize(m.kitten.aliases) arch |> serialize(m.kokoro.sym) @@ -780,6 +784,9 @@ def load_styletts2(path : string) : StyleTts2Model { let ts = ref_time_ticks() let preferred_tag = q8 ? ST2_IMAGE_TAG_Q8 : ST2_IMAGE_TAG let other_tag = q8 ? ST2_IMAGE_TAG : ST2_IMAGE_TAG_Q8 + if (!stat(path).is_valid) { + panic("dasLLAMA styletts2: '{path}' does not exist") + } if (!load_image(path, m, preferred_tag) && !load_image(path, m, other_tag)) { panic("dasLLAMA styletts2: '{path}' is not a prepared TTS image for this box (identity {image_identity(ST2_IMAGE_TAG_Q8)} / {image_identity(ST2_IMAGE_TAG)}) - bake it from the source gguf with dasllama-convert") } diff --git a/modules/dasLLAMA/dasllama/dasllama_tts.das b/modules/dasLLAMA/dasllama/dasllama_tts.das index 8ac6ab35d5..fca143ae95 100644 --- a/modules/dasLLAMA/dasllama/dasllama_tts.das +++ b/modules/dasLLAMA/dasllama/dasllama_tts.das @@ -67,8 +67,10 @@ def load_tts_model(path : string) : TtsModel { return <- m } +//! The phoneme pack a model directory serves: the full pack when present, else the American-only +//! twin; neither is a panic naming the directory. [arch(at = "../ARCHITECTURE_TTS.md#tts-g2p-pack-tiers")] -def private g2p_pack_path(dir : string) : string { +def g2p_pack_path(dir : string) : string { let full = path_join(dir, "tts_g2p.bin") return full if (stat(full).is_valid) let us = path_join(dir, "tts_g2p_en_us.bin") diff --git a/modules/dasLLAMA/harness/REVIEW.md b/modules/dasLLAMA/harness/REVIEW.md index f567b1389c..fe00d1405a 100644 --- a/modules/dasLLAMA/harness/REVIEW.md +++ b/modules/dasLLAMA/harness/REVIEW.md @@ -6,12 +6,11 @@ the Metal backend, `../followup_vulkan.md` for anything about the Vulkan backend `../PERF_LEDGER.md` for any other performance followup, `../followup_general.md` for everything else. -**A diff that adds or moves a timed unit in `tune_kernels.das` - a row in the `benches` list, -or a race that runs after the list - keeps `dot_q8q8_laneq4x4` last among the timings that use -the CPU matmul backend.** Running that bench pins one matmul backend for the rest of the -process, so a CPU timing after it runs against the pinned backend instead of the one it would -have picked. The Metal crown race after the list uses no CPU matmul backend and is out of the -rule's scope. +**A diff that adds or moves a timed unit in `tune_kernels.das` keeps `dot_q8q8_laneq4x4` the +LAST row of the `benches` list and adds no CPU timing after it; the Metal crown race after the +list is the one thing allowed there.** Running that bench pins one matmul backend for the rest of +the process, so a CPU timing after it runs against the pinned backend instead of the one it would +have picked. **A diff that points a `dashv` call in this folder at the sidecar exchange - dasllama.io's tune-sidecar service: sidecar lookup, download, or submit - is a defect; it calls the exchange diff --git a/modules/dasLLAMA/performance/REVIEW.md b/modules/dasLLAMA/performance/REVIEW.md index e78c1330de..64263fd50c 100644 --- a/modules/dasLLAMA/performance/REVIEW.md +++ b/modules/dasLLAMA/performance/REVIEW.md @@ -95,10 +95,11 @@ beside it: a projector, a draft head, an assistant sidecar, an image fixture - p `companions` of the row that pins its carrier, and names it from every other row that consumes it.** Uniqueness itself is `../tests/test_model_specs.das`'s to enforce. -**A diff that adds or changes a `serve_*` field on a row of `model_specs()`, or any `serve_*` -function in `model_specs.das`, lands `utils/dasllama-server/test_model_catalog.das` (repo -root) green in the same change** - the serving catalog is a view over these rows, so its gates -red on a row this folder ships. +**A diff that changes what any `serve_*` function in `model_specs.das` returns - a `serve_*` +field on a row, the function's body, or a `companions` entry with a `url` on a row a `serve_*` +view reads - lands `utils/dasllama-server/test_model_catalog.das` (repo root) green in the same +change** - the serving catalog is a view over these rows, so its gates red on a row this folder +ships. **A convert, a bench, or a tune-state write reached from `fetch_models.das --fetch` is a defect - `--fetch` downloads only.** Each has its own home: a conversion recipe runs under diff --git a/modules/dasLLAMA/performance/model_specs.das b/modules/dasLLAMA/performance/model_specs.das index 0bad12b6e9..2e371cb2ef 100644 --- a/modules/dasLLAMA/performance/model_specs.das +++ b/modules/dasLLAMA/performance/model_specs.das @@ -577,10 +577,11 @@ def model_specs() : array { // nolint:STYLE038 — flat model-set t ] } -//! The text-to-speech set the serving catalog offers: the three converted GGUFs and the two -//! front-end packs every one of them loads, as pinned on the borisbat/dasllama-tts repository - -//! a view over the rows above (the packs ride the kitten-nano row's companions). Every entry -//! carries url + bytes + sha256; `pack` tells the two apart from the models. +//! The text-to-speech set the serving catalog offers: the three converted GGUFs and the three +//! front-end packs (the full phoneme pack, its American-only twin, the tagger), as pinned on the +//! borisbat/dasllama-tts repository - a view over the rows above (the packs ride the kitten-nano +//! row's companions). Every entry carries url + bytes + sha256; `pack` tells the packs apart from +//! the models. def serve_tts_set() : array> { var out : array> var all <- model_specs() diff --git a/modules/dasLLAMA/performance/profile_common.das b/modules/dasLLAMA/performance/profile_common.das index e0ef0f8070..083cca45ad 100644 --- a/modules/dasLLAMA/performance/profile_common.das +++ b/modules/dasLLAMA/performance/profile_common.das @@ -21,7 +21,7 @@ require daslib/json public // JV(...) for logger field payloads require daslib/json_boost // ?[] / ?? navigation — the sidecar provenance reader require dasllama/dasllama_common public // QuantMode require dasllama/dasllama_math // active_kernel_backend — the exec_fmt layout receipt -require llvm/daslib/llvm_tune // tune_status — stamp provenance for the platform block +require ?llvm llvm/daslib/llvm_tune // nolint:STYLE030,LINT019 — tune_status, the stamp provenance; empty without the framework require model_specs public // the model-set table + its llm/official views require box_ident // derive_box_name — the hardware-keyed box tag @@ -474,7 +474,8 @@ def private platform_os() : string { return strip("{run_capture(". /etc/os-release && echo $PRETTY_NAME", true)} {run_capture("uname -r", true)}") } -def private tune_row_str(r : TuneStatus) : string { +//! generic on purpose: the row type lives in the framework, and this file compiles without it +def private tune_row_str(r) : string { let sfx = empty(r.suffix) ? "reference" : r.suffix return "{r.fname}={sfx} ({r.source})" } @@ -482,12 +483,16 @@ def private tune_row_str(r : TuneStatus) : string { //! One-line gen-kernel stamp provenance — "" (omitted from JSON) when the driver declares no //! `[tune_policy]` (tune_status is empty then). def tune_summary() : string { - var st <- tune_status() - var parts <- [for (r in st); tune_row_str(r)] - let s = join(parts, "; ") - delete parts - delete st - return s + static_if (typeinfo module_exists(llvm_tune)) { + var st <- tune_status() + var parts <- [for (r in st); tune_row_str(r)] + let s = join(parts, "; ") + delete parts + delete st + return s + } else { + return "" + } } //! The bench cells' provenance gate: REFUSE to measure on fallback kernel winners. A @@ -499,7 +504,9 @@ def tune_summary() : string { //! * **``DAS_TUNE_MANIFEST`` pinned**: stamps apply, status stays empty (framework gap, //! ledgered) — gate on the FILE: present, newer than the binary, carrying kernels. //! * **plain script**: the policy rail ran, so every row must stamp a manifest winner. -//! ``DASLLAMA_ALLOW_UNTUNED=1`` is the dev-run escape in all three (it also suppresses the +//! A build without the framework (no dasLLVM, the portable tier) has no winners at all - every +//! kernel is its reference body - and the gate refuses by name. +//! ``DASLLAMA_ALLOW_UNTUNED=1`` is the dev-run escape in every world (it also suppresses the //! auto-mint at ``[init]`` — see ``dasllama_math_gen_register`` — so the gate is reached at all). var private g_tune_gate_bypassed = false @@ -510,6 +517,30 @@ def tune_gate_bypassed() : bool { } def tune_gate() : bool { + static_if (typeinfo module_exists(llvm_tune)) { + return tune_gate_verdict(tune_gate_why(0)) + } else { + return tune_gate_verdict("this build carries no tune framework (dasLLVM absent) - every kernel runs its reference body") + } +} + +def private tune_gate_verdict(why : string) : bool { + if (empty(why)) { + return true + } + if (g_env_bench.allow_untuned) { + g_tune_gate_bypassed = true + to_log(LOG_WARNING, "tune gate: {why} - measuring anyway (DASLLAMA_ALLOW_UNTUNED=1, numbers are NOT publishable)\n") + return true + } + to_log(LOG_ERROR, "tune gate: {why}. Mint with: DAS_TUNE_MODE=tune bin/daslang -jit modules/dasLLAMA/harness/dasllama_tuner.das -dasroot ; DASLLAMA_ALLOW_UNTUNED=1 overrides for dev runs\n") + return false +} + +//! The three-world refusal text, "" when the cell may measure. Generic on purpose (the unused +//! parameter is what keeps it one): it names framework functions, so its body must instantiate +//! only from `tune_gate`'s framework arm - a plain function would compile in every build. +def private tune_gate_why(_) : string { var why = "" let mpath = g_env_core.tune_manifest if (is_standalone_exe()) { @@ -553,16 +584,7 @@ def tune_gate() : bool { why = "{bad}/{total} kernels stamp non-manifest winners (first: {sample}) - the sidecar is stale or absent; a class-profile stamp covers the kernels but is not this box's mint" } } - if (empty(why)) { - return true - } - if (g_env_bench.allow_untuned) { - g_tune_gate_bypassed = true - to_log(LOG_WARNING, "tune gate: {why} - measuring anyway (DASLLAMA_ALLOW_UNTUNED=1, numbers are NOT publishable)\n") - return true - } - to_log(LOG_ERROR, "tune gate: {why}. Mint with: DAS_TUNE_MODE=tune bin/daslang -jit modules/dasLLAMA/harness/dasllama_tuner.das -dasroot ; DASLLAMA_ALLOW_UNTUNED=1 overrides for dev runs\n") - return false + return why } //! True when a das run cannot name the winners it measured under — an empty ``tune`` or @@ -903,14 +925,25 @@ def private compute_sha256(path : string) : string { //! sha256 of this process's tune sidecar — the sidecar GENERATION a measurement runs under. //! Rows minted under different generations are not comparable; "" when the sidecar is absent. def tune_manifest_sha() : string { - return sidecar_generation_sha(tune_manifest_path()) + static_if (typeinfo module_exists(llvm_tune)) { + return sidecar_generation_sha(tune_manifest_path()) + } else { + return "" + } } //! The mint's noise verdict from the sidecar provenance ("ok" | "overridden"; "" when the -//! sidecar or its provenance is absent) — records copy it so a mint taken through a failing -//! noise gate is visible on the board, not only inside the sidecar. +//! sidecar or its provenance is absent, and in a build without the framework) — records copy it +//! so a mint taken through a failing noise gate is visible on the board, not only inside the sidecar. def tune_manifest_noise() : string { - let path = tune_manifest_path() + static_if (typeinfo module_exists(llvm_tune)) { + return sidecar_noise_verdict(tune_manifest_path()) + } else { + return "" + } +} + +def private sidecar_noise_verdict(path : string) : string { if (!stat(path).is_valid) { return "" } diff --git a/modules/dasLLAMA/portable_tier_plan.md b/modules/dasLLAMA/portable_tier_plan.md index c88484cb7f..1aae804540 100644 --- a/modules/dasLLAMA/portable_tier_plan.md +++ b/modules/dasLLAMA/portable_tier_plan.md @@ -442,7 +442,9 @@ browser demo is a stretch on the same artifact. logger over the JIT-cross Kitten artifact (`--cpu-prof` hangs emscripten's pthread pool at worker load; `--prof` with `DAS_JOBQUE_THREADS=2` does not): 77% of every tick in one lambda, the portable `q8q8_batch_kernel`'s row loop, which is `dot_q8q8` inlined - the TTS rows GEMMs. -A one-lane micro-bench (4096-wide row, node, GMAC/s) sized the alternatives: +A one-lane micro-bench sized the alternatives - a scratch `q8bench` daspkg package (not checked in) +timing all three forms over one 4096-wide row in ONE node process, `release wasm` with +`--profiling-funcs`, GMAC/s; the native column is the same package under `-jit` on this M1 Max: | form | native arm64 JIT | wasm64, before | wasm64, after | |---|---|---|---| @@ -450,14 +452,13 @@ A one-lane micro-bench (4096-wide row, node, GMAC/s) sized the alternatives: | `dot_q8q8_idot4x4` (idot4 builtin, per-block hsum) | 52 | 1.9 | 9.1 | | `dot_q8q8_idot4_ps` (idot4, vector epilogue, one hsum per row) | 46-54 | - | 13.8 | -Two changes, both general: the JIT's signed `idot` family gets a wasm lowering - -`i32x4.relaxed_dot_i8x16_i7x16_add_s` through the sign trick (dot(w, x) = dot(sign(x)*w, |x|), -exact because Q8_0 quants sit in [-127, 127]), with the extmul + `extadd_pairwise` chain as the -fallback where relaxed SIMD is absent - and `+relaxed-simd` joins the wasm feature string (every -engine that runs memory64 shipped it first; no ABI moves). The portable kernels pick +Two changes, both general: the JIT's signed `idot` family gets a wasm lowering - the exact +extmul + `extadd_pairwise` chain (the 9.1 row above); the relaxed 7-bit dot behind the 13.8 row was +withdrawn in review, because the sign trick that feeds it wraps at -128 in either operand and +`+relaxed-simd` re-means float-vector min/max/mad engine-wide - and the portable kernels pick `dot_q8q8_idot4_ps` (and its s16 twin) on a wasm target at compile time and keep the template everywhere else. Gates: `tests/jit_tests/wasm_idot_lowering.das` (the cross dump carries the -relaxed dot, the host dump no wasm intrinsic) and the new `test_q8q8_idot4_ps` cell in the +pairwise adds and no relaxed dot, the host dump no wasm intrinsic) and the new `test_q8q8_idot4_ps` cell in the kernel-family test (fp64 bar, s16 twin bit-identical). End to end: Kitten nano under node on 4 lanes 0.27x -> 0.14x real time; in Chrome 0.37-0.43x -> 0.23-0.26x (the generator stage 1.1 s -> 0.57 s per sentence). Not measurements, one box. Next rungs by the same profile: the diff --git a/modules/dasLLAMA/tests/CLAUDE.md b/modules/dasLLAMA/tests/CLAUDE.md index b088972fec..6151a82893 100644 --- a/modules/dasLLAMA/tests/CLAUDE.md +++ b/modules/dasLLAMA/tests/CLAUDE.md @@ -43,8 +43,10 @@ on an `EXCLUDED` line, so a trimmed run cannot read as full. `--no-tune` compose every child runs with `DAS_TUNE_POLICY=reference`, so the `[tune]` families and the `[tuned]` hints fall to their reference bodies - the portable tier's arm of the gate. That arm also runs with `DASLLAMA_IMAGE=0`: the reference policy is a different box identity, and a `.dlim` minted -under it would GC-purge the box's tuned images, so image-rail cells skip there and keep their -coverage on the tuned arm. The runner redirects +under it would GC-purge the box's tuned images, so the runner refuses `--no-tune` with the +`image` and `image-vulkan` suites, and the two image-reading cells outside them (`test_tts_kitten`'s +image cell, `test_audio_embedder`'s direct-route cell) skip on the knob and keep their coverage on +the tuned arm. The runner redirects the COMPLETE output to a log file, and prints that path on the DONE line. It owns the dastest timeout, and repeats a file only when `--nreps` is passed explicitly (default 1, never best-of-N). Every child runs `-jit -module-cache .jitted_scripts/module_cache/dastest.dascache`; @@ -528,13 +530,18 @@ pack-derived rhotic rows and its keep-the-r fallback for a vowel with no row, an rewrite on the fixture's probe words with the reference's espeak readings logged beside ours, and the British lexicon tier itself - the bath-trap split, the LOT vowel and the non-rhotic vowels answered from the British tier rather than the rewrite, with the American reading of the same -line beside it. +line beside it. Then the AMERICAN-ONLY pack (`tts_g2p_en_us.bin`, skips when absent): phoneme-identical +with the full pack on the 200 sentences, British refused by name on it (the full pack reading +British is the control), and the directory rule of `g2p_pack_path` - the full pack wins where +both sit, the American twin serves alone, an empty directory panics naming it. `test_tts_kitten.das` - stocked suite; the symbol-map and token-rule cells run everywhere (the front end's inventory into espeak-style IPA against the reference rewrite over the corpus, the reference driver's re-spacing and wrapping), the model-gated cells (`kitten-.gguf` + `tts_oracle/kitten_/` under the models dir, both from `performance/build_tts_data.das`) run the parity rail of `_tts_parity.das` per size and a facade smoke cell that speaks one -sentence and checks the PCM is finite, non-silent, of speech length, and carries its timings. +sentence and checks the PCM is finite, non-silent, of speech length, and carries its timings; +the image cell (skips under `DASLLAMA_IMAGE=0`) loads kitten-nano through `load_styletts2` on a +`.dlim` - the speed priors and aliases ride the image meta, the q8 sibling bakes when absent. `test_tts_kokoro.das` - stocked suite; model-free cells: the symbol map over a synthetic phoneme string, the out-of-vocabulary drop, the style-row clamps, and the pack-name language rule (`_` for all nine codes, and every other shape reading "" whatever letter it opens diff --git a/modules/dasLLAMA/tests/REVIEW.md b/modules/dasLLAMA/tests/REVIEW.md index 855be65458..2b0f1ac245 100644 --- a/modules/dasLLAMA/tests/REVIEW.md +++ b/modules/dasLLAMA/tests/REVIEW.md @@ -45,8 +45,8 @@ or skip condition.** A clause that only names the file (a brace list, a suite ro nothing to correct. **A diff that changes `run.das`'s flag surface - a flag, a suite name, an area name, or what a -flag runs - corrects `CLAUDE.md`'s "Run suites ONLY through the runner" block and -`../CLAUDE.md`'s "Test workflow" section in the same change.** Both restate the surface for an +flag runs - adds it to or corrects it in `CLAUDE.md`'s "Run suites ONLY through the runner" block +and `../CLAUDE.md`'s "Test workflow" section in the same change.** Both restate the surface for an agent that reads them cold; a copy the code has left behind sends that agent to a flag that no longer does what the text says. @@ -172,8 +172,9 @@ which can wrongly satisfy a tolerant compare. transcript against a reference leg, external dump or CPU control alike. **A test in this folder that loads a stocked artifact whose producer the test cannot name is a -defect.** Stocked artifacts are model files, mmprojs, image fixtures, and oracle dumps. Any -one of these names the producer: a row in `../performance/model_specs.das`; a row's +defect.** A stocked artifact is anything the test reads out of `models_dir()` that this repo's +build does not produce - model files, mmprojs, front-end packs, image fixtures, oracle dumps. +Any one of these names the producer: a row in `../performance/model_specs.das`; a row's `companions` list; a row in `asr_catalog` (`../performance/profile_common.das`); a convert script beside `model_specs.das` in `../performance/`. For an oracle dump, the mint script stocked beside the dumps under `models_dir()` also counts, and the test that loads the dump @@ -213,8 +214,9 @@ one CPU inference reads) and the stages a decode override selects on that model' (`blob_twin(t, path, seq_cap)`, `test_metal_decode_parity.das`), in one session.** The planar model and its blob twin share one shape, so one session serves both. -**A diff that adds a model-loading block to a `run.das` MODEL suite (every suite the -`--family` filter reaches - not the model-free suite) tags it with its family.** The family +**A diff that adds a model-loading block to a `run.das` MODEL suite (`decode`, `mtp`, `prefill`, +`matrix`, `image`, `image-vulkan`, `coverage` - not `model-free` or `stocked`) tags it with its +family.** The family tag is the token passed to `family_on(t, name)` (`_model_tier.das`). An untagged block silently joins every family's gate. diff --git a/modules/dasLLAMA/tests/run.das b/modules/dasLLAMA/tests/run.das index 99e7e3318e..a47cc38a2c 100644 --- a/modules/dasLLAMA/tests/run.das +++ b/modules/dasLLAMA/tests/run.das @@ -441,8 +441,9 @@ def private child_envs(cfg : RunArgs; win : bool; arm : string) : string { // the perf rig's. Without this every child auto-mints its own sidecar tune, computes a // FOREIGN image identity, and its save GC-purges the rig's images as dead siblings // (observed 2026-07-30: fam-gemma4 freed 34 GB of bench flavors). A caller-set - // DAS_TUNE_MANIFEST inherits; a box without a minted manifest keeps sidecar behavior. - if (empty(g_env_core.tune_manifest)) { + // DAS_TUNE_MANIFEST inherits; a box without a minted manifest keeps sidecar behavior. The + // --no-tune arm pins nothing: an explicit manifest disables the policy rail it runs on. + if (empty(g_env_core.tune_manifest) && !cfg.no_tune) { let manifest = "modules/dasLLAMA/performance/{derive_box_name(g_env_prof.box)}.tune.json" var mst : FStat if (stat(manifest, mst)) { @@ -558,6 +559,10 @@ def select_areas(cfg : RunArgs) : tuple> { //! the suite forms' argument contract: 0 = run, else main's exit code def check_suite_args(cfg : RunArgs) : int { + if (cfg.no_tune && (cfg.suite == "image" || cfg.suite == "image-vulkan")) { //! the arm runs with the image rail off, and the image suites ARE the rail + to_log(LOG_ERROR, "--no-tune sets DASLLAMA_IMAGE=0 in every child, so it cannot drive the image suites - drop --no-tune or pick another suite\n") + return 2 + } if (cfg.suite == "model-free" || cfg.suite == "stocked") { if (cfg.full || !empty(cfg.arm)) { //! the per-PR gates have no arms to scope: a filter would silently shrink them to_log(LOG_ERROR, "--suite {cfg.suite} takes neither --arm nor --full: it is a whole per-PR gate (--exclude names what it drops)\n") diff --git a/modules/dasLLAMA/tests/test_math_activations.das b/modules/dasLLAMA/tests/test_math_activations.das index c915ea2495..ffe175056e 100644 --- a/modules/dasLLAMA/tests/test_math_activations.das +++ b/modules/dasLLAMA/tests/test_math_activations.das @@ -550,7 +550,7 @@ def test_sigmoid_gate(t : T?) { [test] def test_swiglu_oai(t : T?) { t |> run("swiglu_oai against an fp64 reference carrying the trained clamps") @(t : T?) { - report(t, "swiglu_oai", sweep_swiglu_oai(false)) + report_moved(t, "swiglu_oai", sweep_swiglu_oai(false)) report(t, "swiglu_oai4", sweep_swiglu_oai(true)) } t |> run("the gate limit and the up-branch band are the activation, not an optimization") @(t : T?) { diff --git a/modules/dasLLAMA/tests/test_program_roots.das b/modules/dasLLAMA/tests/test_program_roots.das index ba5f96f9cf..ec0a0e78ab 100644 --- a/modules/dasLLAMA/tests/test_program_roots.das +++ b/modules/dasLLAMA/tests/test_program_roots.das @@ -25,11 +25,18 @@ def private read_lines(path : string) : array { return <- split(replace(text, "\r\n", "\n"), "\n") } +//! every .das under a root dir, subdirectories included: a program root may sit in its own folder +//! (examples/dasLLAMA/storyteller/main.das) def private das_files_of(dir_path : string) : array { var files : array fio::dir("{get_das_root()}/{dir_path}") $(name) { + return if (name == "." || name == "..") + let rel = "{dir_path}/{name}" if (name |> ends_with(".das")) { - files |> push("{dir_path}/{name}") + files |> push(rel) + } elif (stat("{get_das_root()}/{rel}").is_dir) { + var sub <- das_files_of(rel) + files |> push_from(sub) } } return <- files @@ -57,6 +64,8 @@ def test_program_roots(t : T?) { requires_dasllama ||= line |> starts_with("require dasllama/") has_main ||= (line |> starts_with("def main")) || (line |> starts_with("def main(")) has_stack ||= line |> starts_with(STACK_LINE) + // the loaders whose model prefills (LLM, ASR decoder, audio tower); a TTS + // carrier runs no prefill, so a TTS-only root owes no intent loads_model ||= find(line, "load_model(") >= 0 || find(line, "load_asr_model(") >= 0 || find(line, "load_audio_tower(") >= 0 // CODE only: roots' allow_cpu_prefill() tail comments NAME set_metal_mode, so a // raw-line scan would let a comment satisfy the check diff --git a/modules/dasLLAMA/tests/test_tts_g2p.das b/modules/dasLLAMA/tests/test_tts_g2p.das index 532b2c47f8..3314c8d7da 100644 --- a/modules/dasLLAMA/tests/test_tts_g2p.das +++ b/modules/dasLLAMA/tests/test_tts_g2p.das @@ -4,6 +4,7 @@ options _dasllama_internal = true require dastest/testing_boost public require dasllama/dasllama_g2p require dasllama/dasllama_postag +require dasllama/dasllama_tts // g2p_pack_path - the directory rule the facade loads by require strings require daslib/strings_boost require daslib/fio @@ -283,6 +284,29 @@ def test_us_only_pack(t : T?) { let gb_full = g2p_phonemize(full, tg, "the water is warm", true) t |> success(!empty(gb_full), "the control: the full pack reads British") } + t |> run("the directory rule: the full pack wins where both sit, the American twin serves alone, an empty directory panics") @(tt : T?) { + return if (!data_present(tt)) + let us_path = path_join(models_dir(), "tts_g2p_en_us.bin") + if (!stat(us_path).is_valid) { + tt |> skip("tts_g2p_en_us.bin absent in {models_dir()}") + return + } + tt |> equal(g2p_pack_path(models_dir()), g2p_path(), "both packs beside the models: the full pack wins") + let only_us = tmp_path("packs_us") + mkdir(only_us) + write_pack_header(path_join(only_us, "tts_g2p_en_us.bin"), 2) // the path rule stats, it never reads + tt |> equal(g2p_pack_path(only_us), path_join(only_us, "tts_g2p_en_us.bin"), "the American twin serves alone") + let none = tmp_path("packs_none") + mkdir(none) + var refused = "" + try { + let p : string = g2p_pack_path(none) + to_log(LOG_INFO, "tts g2p pack path on an empty dir answered {p}\n") + } recover { + refused = this_context().last_exception + } + tt |> success(find(refused, "no phoneme pack") >= 0, "an empty directory panics naming the packs it lacks: {refused}") + } } [test] diff --git a/modules/dasLLVM/ARCHITECTURE.md b/modules/dasLLVM/ARCHITECTURE.md index 99358818b2..2cd7fa826a 100644 --- a/modules/dasLLVM/ARCHITECTURE.md +++ b/modules/dasLLVM/ARCHITECTURE.md @@ -135,10 +135,13 @@ An aarch64 host target reads its CPU features from two sources, because neither the features - and a part this LLVM cannot name maps to the generic CPU, where SDOT and SMMLA have no instruction to select and codegen aborts. `cpu_supports` reads the operating system instead (sysctl / `AT_HWCAP` / `IsProcessorFeaturePresent`), so it answers for silicon LLVM has -never heard of. Both the tier gates (`init_jit_target_flags`) and the target machine's feature -string (`create_default_target_machine`) therefore take the union of the two: an LLVM host-string -hit OR a `cpu_supports` hit (fullfp16 additionally reads darwin-arm64 as always-on - every -Apple Silicon part has it). A cross-compile triple takes neither - only the force env - and so +never heard of. The tier gates (`init_jit_target_flags` - `g_target_arm64_dotprod`, `_i8mm`, +`_fullfp16`) and the target machine's feature string (`create_default_target_machine`) therefore +take the union of the two: an LLVM host-string hit OR a `cpu_supports` hit (fullfp16 additionally +reads darwin-arm64 as always-on - every Apple Silicon part has it). One asymmetry: the host rail's +machine string carries `+dotprod` unconditionally (every part the JIT has run on has it), while the +DotProd GATE probes like its siblings - on an ARMv8.0 host the gate declines and the `sdot4` family +compiles its fallback, whatever the string says. A cross-compile triple takes neither - only the force env - and so does a generic-CPU standalone exe (one carrying no `[llvm_code]` kernel): its machine is the ARMv8.0 baseline, which cannot select SDOT or SMMLA, so the DotProd and i8mm gates (`g_target_arm64_dotprod`, `g_target_arm64_i8mm`) stay off there and every `aarch64_neon` call @@ -286,13 +289,13 @@ The exact integer dots on the 8-bit lattice have three lowerings, picked by targ `+dotprod` append, or the force env on the generic rail), the SIMD128 form on a wasm target (`idot_wasm_simd128`), generic widen-multiply IR everywhere else. The native arms exist because neither backend produces them from the generic form: AArch64 expands it to zip/uzp/smull instead -of folding to SDOT, and the wasm backend runs it a fifth as fast. The wasm form has two arms. With -`+relaxed-simd` the ISA carries the dot itself, `i32x4.relaxed_dot_i8x16_i7x16_add_s`, exact only -while its second operand is in [0, 127]; the sign trick puts it there, since -`dot(w, x) == dot(sign(x)*w, |x|)` and Q8_0 quants are in [-127, 127] by construction -(`d = amax/127`), so no lane wraps on the flip. Without the feature the ISA still has the two halves -of an int8 dot, `i16x8.extmul_{low,high}_i8x16_s` and `i32x4.extadd_pairwise_i16x8_s`; the pairwise -sums land as byte pairs, and one even/odd shuffle-add folds them into the quad lanes the generic -form defines. `wasm_target_features` always asks for `+relaxed-simd`: every engine that runs -memory64 shipped relaxed SIMD first, and the feature changes no ABI, so a runtime archive built -without it links unchanged. +of folding to SDOT, and the wasm backend runs it a fifth as fast. The wasm form is the ISA's two +halves of an int8 dot, `i16x8.extmul_{low,high}_i8x16_s` (what LLVM makes of `mul(sext, sext)`) and +`i32x4.extadd_pairwise_i16x8_s`; the pairwise sums land as byte pairs, and one even/odd shuffle-add +folds them into the quad lanes the generic form defines - exact for every int8 lane, like the +builtin's contract. The relaxed-SIMD dot (`i32x4.relaxed_dot_i8x16_i7x16_add_s`) is deliberately +NOT used: its second operand is 7-bit, so the sign trick that would feed it (`dot(w, x) == +dot(sign(x)*w, |x|)`) wraps at -128 in either operand and answers the wrong sign there, and +`+relaxed-simd` is a whole-module switch that also turns float-vector `min`/`max` and `mad` into +engine-defined instructions (NaN and signed-zero answers, fusion) - the feature string stays +`+simd128,+nontrapping-fptoint`, matching the runtime archive. diff --git a/modules/dasLLVM/REVIEW.md b/modules/dasLLVM/REVIEW.md index 7b82e999ba..1344bdc807 100644 --- a/modules/dasLLVM/REVIEW.md +++ b/modules/dasLLVM/REVIEW.md @@ -89,8 +89,9 @@ `ConstantExpr`, and the setter's cast writes through the wrong type into it (`ARCHITECTURE.md#gep-constant-fold`). -- **A feature name used in a `requires=` list or a `g_target_x64_*` gate has its cpuid line in - `das_cpu_supports` (`src/builtin/module_builtin_runtime.cpp`, repo root) in the same diff** +- **A feature name used in a `requires=` list or a `g_target_*` tier gate + (`daslib/llvm_jit_common.das`) has its cpuid line in `das_cpu_supports` + (`src/builtin/module_builtin_runtime.cpp`, repo root) in the same diff** (`ARCHITECTURE.md#x64-tier-gates`). A name the cpuid table does not know answers false on every box, so every perm that requires it silently declines to its fallback and no error names the cause. @@ -101,18 +102,22 @@ sign-alternating chain moves the last few bits of the result, and the interpreter and AOT answers do not move with it. -- **A diff that adds or changes a `build_vector_*` emitter also adds two cells to - `tests/llvm_vector_math.das` (beside this file): one comparing the emitted result lane for - lane with the interpreted result, at every vector width the emitter serves, and one asserting - both answer NaN in the same lanes.** A clamp or a conversion written with ordered compares - turns a NaN lane into a number, and an accuracy bound reads that as success. - -- **Weakening `REVIEW.das` (beside this file) is a defect:** dropping a check, dropping a - directory from its tracked-fixture list or removing the last tracked file under one (a guard - over nothing), or a finding text that no longer names what failed. What the gate enforces is - read from the gate itself. - -- **A diff that appends target features to a machine's feature string appends the forced ones +- **A diff that adds or changes an intrinsic emitter whose daslang body is the reference + implementation - a `build_vector_*` emitter, an `idot` lowering - also adds a cell comparing + the emitted result with the interpreted result over the operand range the emitter serves + (every vector width for `build_vector_*`, the full int8 lattice for a dot), and for a float + emitter one asserting both answer NaN in the same lanes; a lowering only a cross target runs + states in the PR body the artifact that compared them.** A clamp or a conversion written with + ordered compares turns a NaN lane into a number, and an accuracy bound reads that as success; + an IR-shape test names the instruction and never a number. + +- **A change that makes `REVIEW.das` (beside this file) report fewer inputs is a defect:** + dropping a check, shrinking a scanned set or a tracked-fixture directory (a guard over nothing), + widening an exemption list without naming the exempted input's reason beside it, or a finding + text that no longer names what failed. What the gate enforces is read from the gate itself. + +- **A diff that builds an x64 or aarch64 machine's feature string appends the forced ones (`x64_forced_plus_features` / `arm64_forced_plus_features`, `daslib/llvm_jit_common.das`) AFTER the detected host features.** LLVM's `SubtargetFeatures` takes the last occurrence of a name, so - a forced feature placed first is silently overridden by detection. + a forced feature placed first is silently overridden by detection. A wasm machine has no force + knob and no detected features, so the rule does not reach it. diff --git a/modules/dasLLVM/daslib/llvm_jit_code.das b/modules/dasLLVM/daslib/llvm_jit_code.das index 4cf992501d..e6772691e9 100644 --- a/modules/dasLLVM/daslib/llvm_jit_code.das +++ b/modules/dasLLVM/daslib/llvm_jit_code.das @@ -9,8 +9,8 @@ require llvm/bindings/llvm_struct require llvm/daslib/llvm_jit_intrin require daslib/ast_boost -//! JIT-side half of the `[llvm_code]` rail (the user-side annotation lives in -//! `llvm/daslib/llvm_code.das`): a name-keyed registry of das generator functions that emit a +//! JIT-side half of the `[llvm_code]` rail (the user-side annotation registers in +//! `daslib/tune.das`, forwarding to `llvm_code_apply` in `llvm/daslib/llvm_code.das`): a name-keyed registry of das generator functions that emit a //! function's LLVM IR wholesale. Generator modules compile into the JIT's context via the //! require list in `llvm/daslib/llvm_user_modules.das` and register themselves from `[init]`. diff --git a/modules/dasLLVM/daslib/llvm_jit_common.das b/modules/dasLLVM/daslib/llvm_jit_common.das index 92aad0eda1..e21bb93a2e 100644 --- a/modules/dasLLVM/daslib/llvm_jit_common.das +++ b/modules/dasLLVM/daslib/llvm_jit_common.das @@ -395,8 +395,6 @@ var public g_handled_field_offset_globals : table starts_with("wasm") - g_target_wasm_relaxed_simd = g_target_is_wasm // Host path (empty triple): the module targets the host, so the daslang binary's own arch is the // JIT target arch. Explicit cross-compile triple: read the arch off the triple prefix. g_target_is_aarch64 = (empty(target_triple) @@ -647,7 +644,7 @@ def public init_jit_target_flags(target_triple : string = ""; host_features : bo ? get_platform_name() == "linux" : find(target_triple, "linux") >= 0) g_target_arm64_dotprod = g_target_is_aarch64 && (arm64_forced_feature("dotprod") - || (empty(target_triple) && host_features)) + || (empty(target_triple) && host_features && (host_llvm_feature("dotprod") || cpu_supports("dotprod")))) g_target_arm64_i8mm = g_target_is_aarch64 && (arm64_forced_feature("i8mm") || (empty(target_triple) && host_features && (host_llvm_feature("i8mm") || cpu_supports("i8mm")))) // aarch64 fullfp16 (see declaration comment): darwin-arm64 == Apple Silicon == always. @@ -944,6 +941,9 @@ def public create_default_target_machine(opt_level : uint; use_host_cpu : bool) } else { let forced = g_target_is_x64 ? x64_forced_plus_features() : (g_target_is_aarch64 ? arm64_forced_plus_features() : "") let feats = empty(forced) ? features_msg : (empty(features_msg) ? forced : "{features_msg},{forced}") + if (!empty(forced)) { + to_log(LOG_INFO, "LLVM JIT: {g_target_is_x64 ? "DAS_JIT_X64_FORCE_FEATURES" : "DAS_JIT_ARM64_FORCE_FEATURES"} appends {forced} to the generic machine {cpu_msg}\n") + } targetMachine = create_target_machine(triple_msg, cpu_msg, feats, opt_level) } diff --git a/modules/dasLLVM/daslib/llvm_jit_intrin.das b/modules/dasLLVM/daslib/llvm_jit_intrin.das index 74a5f5841d..41c3d7b754 100644 --- a/modules/dasLLVM/daslib/llvm_jit_intrin.das +++ b/modules/dasLLVM/daslib/llvm_jit_intrin.das @@ -1563,26 +1563,12 @@ def private idot_sdot_aarch64(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : return LLVMBuildCall2(ctx.builder, fnTy, decl, callArgs, "idot4") } -// Returns null when this LLVM lacks the pairwise intrinsic, so callers fall through to the generic lowering. +// The exact wasm SIMD128 int8 dot: extmul halves + pairwise widening adds + one even/odd fold. Returns +// null when this LLVM lacks the pairwise intrinsic, so callers fall through to the generic lowering. [arch(at = "../ARCHITECTURE.md#idot-lowerings")] def private idot_wasm_simd128(var ctx : JitCtx; acc, a, b : LLVMOpaqueValue?) : LLVMOpaqueValue? { let v8i16 = LLVMVectorType(ctx.types.t_int16, 8u) let v4i32 = ctx.types.LLVMInt4Type() - if (g_target_wasm_relaxed_simd) { - let rid = LLVMLookupIntrinsicID("llvm.wasm.relaxed.dot.i8x16.i7x16.add.signed") - var noTypes : array - var rdecl = rid != 0u ? LLVMGetIntrinsicDeclaration(g_mod, rid, noTypes) : null - if (rdecl != null) { - let v16i8 = LLVMVectorType(ctx.types.t_int8, 16u) - let mask = LLVMBuildAShr(ctx.builder, b, build_const_int_splat(ctx.types.t_int8, 16, 7), "xsign") - let wsg = LLVMBuildSub(ctx.builder, LLVMBuildXor(ctx.builder, a, mask, ""), mask, "wsigned") - let xabs = LLVMBuildSub(ctx.builder, LLVMBuildXor(ctx.builder, b, mask, ""), mask, "xabs") - var fnArgTypes <- [v16i8, v16i8, v4i32] - var fnTy = LLVMFunctionType(v4i32, fnArgTypes) - var callArgs <- [wsg, xabs, acc != null ? acc : LLVMConstNull(v4i32)] - return LLVMBuildCall2(ctx.builder, fnTy, rdecl, callArgs, "idot4_relaxed") - } - } let id = LLVMLookupIntrinsicID("llvm.wasm.extadd.pairwise.signed") var declTypes <- [v4i32] // overloaded on the widened result vector type var decl = id != 0u ? LLVMGetIntrinsicDeclaration(g_mod, id, declTypes) : null diff --git a/modules/dasLLVM/daslib/llvm_jit_run.das b/modules/dasLLVM/daslib/llvm_jit_run.das index 3906aabe3f..29d37215ad 100644 --- a/modules/dasLLVM/daslib/llvm_jit_run.das +++ b/modules/dasLLVM/daslib/llvm_jit_run.das @@ -37,7 +37,7 @@ var LINK_WHOLE_LIB = false // when true, standalone exe links against the whole // invalidates cached DLLs (e.g. edits to llvm_jit.das, llvm_macro.das, llvm_jit_common.das, // runtime helper ABI, default target triple). Cache filenames fold this in, so a bump // makes every previously written DLL miss the cache on the next run and get GC'd. -let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x6ful // the signed idot family lowers to extmul + extadd_pairwise on wasm SIMD128 (0x6e: the aarch64 SDOT / SMMLA tables gate on DotProd / i8mm, not the arch alone, and the force env reaches the generic exe machine (0x6d: the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) +let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x70ul // the wasm feature string drops +relaxed-simd and the idot family keeps only the exact extmul + extadd_pairwise lowering on wasm SIMD128 (0x6f: the first wasm idot lowering; 0x6e: the aarch64 SDOT / SMMLA tables gate on DotProd / i8mm, not the arch alone, and the force env reaches the generic exe machine (0x6d: the inline polynomial rail carries NaN: tanh selects the operand back over its ordered clamp, and the sincos quadrant / tan octant convert through llvm.fptosi.sat instead of poisoning on NaN and out-of-range (0x6c: aarch64 vector tan/exp2/log2/log/pow join the inline polynomial rail bit-exactly with the interpreter, sinh/cosh/tanh ride the exp one; 0x6b: aarch64 vector sin/cos ride the inline polynomial; 0x6a: srem/urem for 32-bit %; 0x69: every string argument of an extern is substituted, not just the ones which asked) // Read by tests-cpp/small/test_jit_emitter_pin.cpp: FNV-1a64 of the emitter sources // (normalized to LF; file list in the test) @@ -785,6 +785,12 @@ def public run_jit(prog : Program?; var ctx : Context?) : bool { // nolint:STYL // script-level `options jit_target = "..."`. let opt_target = (prog._options |> find_arg("jit_target")) ?as tString ?? "" let target_triple = cli_opts.target |> unwrap_or(opt_target) + // the compile-time target folds (get_target_triple, get_target_architecture_name) read the + // command line, never this option: a script-level triple the command line does not carry + // would emit for one target with every tier gate folded for the host + if (!empty(target_triple) && get_target_triple() != target_triple) { + panic("LLVM JIT: the target triple `{target_triple}` is not on the command line - pass --jit-target={target_triple} after the `--` separator (the compile-time target folds read argv, so `options jit_target` alone folds the host's tiers into a cross artifact)") + } let gen_wasm = gen_exe && (target_triple |> starts_with("wasm")) // Optional CLI override for the wasm runtime archive location. Falls // back to `options jit_runtime_lib = "..."` then write_wasm's diff --git a/modules/dasLLVM/daslib/llvm_tune.das b/modules/dasLLVM/daslib/llvm_tune.das index 5953a49c91..b3fa21409e 100644 --- a/modules/dasLLVM/daslib/llvm_tune.das +++ b/modules/dasLLVM/daslib/llvm_tune.das @@ -917,28 +917,6 @@ def tune_status() : array { return clone_to_move(g_tune_status) } - //! Log the tune status table at `LOG_INFO` — the shared "am I tuned?" surface, prefixed by - //! `appName`. Prints `/ kernels tuned for this box`, one line per function, and a - //! `--tune` hint when any kernel sits on a fallback/reference tier. No-op when empty. -def log_tune_status(appName : string) { - var st <- tune_status() - if (empty(st)) { - delete st - return - } - var tuned = 0 - for (s in st) { - tuned += (s.source == "manifest" || s.source == "profile") ? 1 : 0 - } - to_log(LOG_INFO, "{appName}: tune status ({tuned}/{length(st)} kernels tuned for this box):\n") - for (s in st) { - to_log(LOG_INFO, " {s.fname}: {empty(s.suffix) ? "reference" : s.suffix} ({s.source})\n") - } - if (tuned < length(st)) { - to_log(LOG_INFO, "{appName}: run with --tune to tune the remaining kernels for this box\n") - } - delete st -} // ===== [tune_scope] data — library-owned tuner wiring ===== @@ -2656,6 +2634,7 @@ def private warn_manifest_untuned() { for (scope in values(g_scopes)) { let key = "{scope.scopeName}|{scope.manifestPath}" continue if (g_manifest_warned |> key_exists(key)) + pin_module_cache_dependency(scope.manifestPath) let v = tune_sidecar_verdict(scope.manifestPath, scope_fnames_joined(scope)) continue if (v.complete) g_manifest_warned |> insert(key) @@ -2739,6 +2718,7 @@ def private apply_tune_policy(var func : FunctionPtr; declared : string; var err seen |> insert(scope.scopeName) // per-scope KEY completeness, not file existence — the sidecar is shared let scopeFnames = scope_fnames_joined(scope) + pin_module_cache_dependency(scope.manifestPath) let v = tune_sidecar_verdict(scope.manifestPath, scopeFnames) continue if (!force && v.complete && policy != "auto") if (force || policy == "auto") { diff --git a/modules/dasLLVM/tests/llvm_tune_modes_client.das b/modules/dasLLVM/tests/llvm_tune_modes_client.das index f31ac7023a..791d1d1b92 100644 --- a/modules/dasLLVM/tests/llvm_tune_modes_client.das +++ b/modules/dasLLVM/tests/llvm_tune_modes_client.das @@ -2,12 +2,13 @@ options gen2 require dastest/testing_boost require daslib/tune -require llvm/daslib/llvm_tune // Doubles as the child fixture for llvm_tune_modes.das (spawned with DAS_TUNE_MODE=test, -jit: // main prints every grid variant's result) and as an ordinary suite test (normal mode: the // [test] below checks the fallback contract). The variants registry exists in every mode — -// reference-row-only under normal — so both faces compile from the same source. +// reference-row-only under normal — so both faces compile from the same source. The framework +// is reached through `daslib/tune` alone, so `module_exists(llvm_tune)` below is the witness +// answer: in a build without it the shells are inert, no registry exists, and main prints nothing. // companion sibling — grid modes clone it per perm too (modes_kv__k0 ... ), so the child run // proves per-perm args reach the companion generator and the declined row falls back with it @@ -24,17 +25,19 @@ def modes_add(a, b : int) : int { [export] def main { - var n = 2 // nolint:LINT003 — var defeats const-fold so the calls survive to codegen - var vs <- modes_add_variants() - for (v in vs) { - print("VARIANT {v._0} -> {invoke(v._1, n, 3)}\n") - } - delete vs - var kvs <- modes_kv_variants() - for (v in kvs) { - print("KVARIANT {v._0} -> {invoke(v._1)}\n") + static_if (typeinfo module_exists(llvm_tune)) { + var n = 2 // nolint:LINT003 — var defeats const-fold so the calls survive to codegen + var vs <- modes_add_variants() + for (v in vs) { + print("VARIANT {v._0} -> {invoke(v._1, n, 3)}\n") + } + delete vs + var kvs <- modes_kv_variants() + for (v in kvs) { + print("KVARIANT {v._0} -> {invoke(v._1)}\n") + } + delete kvs } - delete kvs } [test] diff --git a/site-dasllama/README.md b/site-dasllama/README.md index 3c8a1991f8..37c357d7ef 100644 --- a/site-dasllama/README.md +++ b/site-dasllama/README.md @@ -8,6 +8,10 @@ project "daslang.io Forge") on the daslang.io Forge system. Arc plan + follow-up ## Layout - `index.html` - news-first home (identity masthead, feed, how-to-get, ladder teaser) +- `examples.html` - the browser examples: one card per example, each opening the example on its own + page under `examples//` (the storyteller's page is `examples/dasLLAMA/storyteller/web_shell.html` + in the repo, built by `daspkg release wasm`; its model set is the rolling `dasllama-web` GitHub + release the deploy downloads beside it); posters under `files/examples/` - `stories.html` - the stories index (title, date, tag, lede per story), generated from `_stories/` between the `stories:begin` markers by `build_news.py`; the index rows use `forge.css`'s shared blog-list language (`forge-blog-*`), so their styling is shared with @@ -50,6 +54,10 @@ ledgered follow-up. py -3 serve.py # http://127.0.0.1:8932/ - stages shared css like the deploy does +`serve.py` maps `/examples//` onto `../web/output64/examples//` and sends the two +cross-origin-isolation headers the Caddy vhost sends under `/examples/` (`utils/internal/dasllama-ladder/caddy.snippet`), +so a locally built example runs here as deployed once its model set sits in `/models/`. + `serve.py` also proxies `/api/*` to a locally running ladder service on 127.0.0.1:8201 (mirroring the Caddy vhost), so a preview against real data is: start `utils/internal/dasllama-ladder/main.das` with `--official-dir modules/dasLLAMA/performance/records`, diff --git a/site-dasllama/REVIEW.md b/site-dasllama/REVIEW.md index 79146fae7a..ede3cb522c 100644 --- a/site-dasllama/REVIEW.md +++ b/site-dasllama/REVIEW.md @@ -37,8 +37,9 @@ top-level `.html` page here, or in a script or stylesheet under `files/`, that n `site/files/dasllama-table.css` (repo root) defines and no `id="..."` in those files declares - is a defect.** -**A claim in any text this folder publishes or documents - a page, a `_news/*.md` or -`_stories/*.md` entry, or `README.md` - that is not true at publish time is a defect.** A dated +**A claim in any text under this folder a reader or an operator reads as fact - a page, a +`_news/*.md` or `_stories/*.md` entry, `README.md`, a tool's docstring - that is not true at +publish time is a defect.** A dated `_news` or `_stories` entry is read as a claim about its own date; standing page copy - a masthead, section prose, a meta tag - is read as a claim about now, and a diff that falsifies standing copy fixes it in the same change. Check a claim about what a shipped tool does against diff --git a/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md b/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md index 7c34a883e4..c7d96538fa 100644 --- a/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md +++ b/site-dasllama/_news/2026-09-05-storyteller-in-the-browser.md @@ -7,6 +7,6 @@ title: dasLLAMA runs in the browser - the storyteller, compiled to wasm64, opens The engine behind the ladder now also ships as a WebAssembly build. `daspkg release wasm` compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared `.dlim` -images, 76 MB with the two English phoneme packs, the same format the native engine maps. -Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. -Chrome, Edge or Firefox 133+ (memory64). [Try it](examples.html). +images, 76 MB with the two English front-end packs, the same format the native engine maps. The +speech thread and the parallel kernels run on Web Workers in the page; nothing is interpreted. +Chrome or Edge 133+, Firefox 134+ (memory64). [Try it](examples.html). diff --git a/site-dasllama/examples.html b/site-dasllama/examples.html index 4ece91a601..e1656fd7e5 100644 --- a/site-dasllama/examples.html +++ b/site-dasllama/examples.html @@ -51,9 +51,9 @@

    The engine, running.

    Storyteller

    llm + tts · in the browser -

    A small story model writes a children's tale one token at a time while KittenTTS reads it aloud. Both models arrive as prepared .dlim images; the whole engine - decode, phonemizer, vocoder - runs in your browser on Web Workers.

    +

    A small story model writes a children's tale one token at a time while KittenTTS reads it aloud. Both models arrive as prepared .dlim images; the whole engine runs in your browser, the speech thread and the parallel kernels on Web Workers.

    llmttswasm64threads
    -
    space · next storyChrome, Edge or Firefox 133+ (memory64)
    +
    space · next storyChrome or Edge 133+, Firefox 134+ (memory64)
    diff --git a/site-dasllama/feed.xml b/site-dasllama/feed.xml index 82cd1f68d0..7a7a55acac 100644 --- a/site-dasllama/feed.xml +++ b/site-dasllama/feed.xml @@ -14,9 +14,9 @@ <p>The engine behind the ladder now also ships as a WebAssembly build. <code>daspkg release wasm</code> compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared <code>.dlim</code> -images, 76 MB with the two English phoneme packs, the same format the native engine maps. -Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. -Chrome, Edge or Firefox 133+ (memory64). <a href="examples.html">Try it</a>.</p> +images, 76 MB with the two English front-end packs, the same format the native engine maps. The +speech thread and the parallel kernels run on Web Workers in the page; nothing is interpreted. +Chrome or Edge 133+, Firefox 134+ (memory64). <a href="examples.html">Try it</a>.</p> dasLLAMA speaks - KittenTTS nano and mini, Kokoro-82M, and a text front end that is nothing but data. diff --git a/site-dasllama/index.html b/site-dasllama/index.html index d50bd44ff5..eb12553974 100644 --- a/site-dasllama/index.html +++ b/site-dasllama/index.html @@ -67,9 +67,9 @@

    Local inference, written in daslang.

    The engine behind the ladder now also ships as a WebAssembly build. daspkg release wasm compiles the storyteller - llama2.c's stories15M writing a children's tale while KittenTTS nano reads it aloud - into one 26 MB wasm64 module, and the two models arrive as prepared .dlim -images, 76 MB with the two English phoneme packs, the same format the native engine maps. -Decode, phonemizer and vocoder all run on Web Workers in the page; nothing is interpreted. -Chrome, Edge or Firefox 133+ (memory64). Try it.

    +images, 76 MB with the two English front-end packs, the same format the native engine maps. The +speech thread and the parallel kernels run on Web Workers in the page; nothing is interpreted. +Chrome or Edge 133+, Firefox 134+ (memory64). Try it.

    2026-09-03engine

    dasLLAMA speaks - KittenTTS nano and mini, Kokoro-82M, and a text front end that is nothing but data.

    ▾
    diff --git a/site-dasllama/serve.py b/site-dasllama/serve.py index f73495e8a9..e9ca151887 100644 --- a/site-dasllama/serve.py +++ b/site-dasllama/serve.py @@ -5,9 +5,10 @@ else from here. /examples// is served from ../web/output64/examples// - where `daspkg release wasm` -writes a browser example and where the deploy step downloads its model set - with the two -cross-origin-isolation headers the Caddy vhost sends there (the -pthread wasm64 builds need -SharedArrayBuffer). Build one, drop its models beside it, and the card works here as deployed. +writes a browser example - with the two cross-origin-isolation headers the Caddy vhost sends +there (the -pthread wasm64 builds need SharedArrayBuffer). Build one, copy its model set into +/models/ (the deploy downloads the same set from the `dasllama-web` release into the staged +tree), and the card works here as deployed. /api/* is proxied to a locally running ladder service (utils/internal/dasllama-ladder on :8201), mirroring the Caddy vhost — start one with real data to preview the live pages: @@ -78,7 +79,8 @@ def translate_path(self, path): return os.path.join(SITE_FILES, name) if clean.startswith("/examples/"): rel = os.path.normpath(clean[len("/examples/"):]) - if rel and not rel.startswith(".."): + # an absolute remainder (a doubled slash) would make os.path.join drop the base + if rel and not rel.startswith("..") and not os.path.isabs(rel): return os.path.join(EXAMPLES_OUT, rel) return super().translate_path(path) diff --git a/site/files/examples.js b/site/files/examples.js index 00ba7b85f1..d1c4f8bab9 100644 --- a/site/files/examples.js +++ b/site/files/examples.js @@ -258,7 +258,7 @@ '' + '
    Needs a memory64 browser
    ' + '

    ' + esc(ex.name) + ' bundles the Dear ImGui ' + - 'module compiled to wasm64 — it runs on Chrome, Edge or Firefox 133+ (engines with ' + + 'module compiled to wasm64 — it runs on Chrome or Edge 133+ and Firefox 134+ (engines with ' + 'WebAssembly memory64). The universal interpreter can’t bind a native module, so ' + 'this one has no interpreted fallback.

    ' + 'view source ↗' + diff --git a/skills/tune.md b/skills/tune.md index 9ebca8b49c..bb991b0e0f 100644 --- a/skills/tune.md +++ b/skills/tune.md @@ -10,7 +10,7 @@ other tier the reference body runs verbatim. `daslib/tune` registers all six annotations - `[tune]`, `[tune_perm]`, `[tune_companion]`, `[tune_scope]`, `[tune_policy]`, `[llvm_code]` - and the default-policy pass; the framework behind them is `llvm/daslib/llvm_tune`, -which `daslib/tune` reaches through a path guard and re-exports. **Any program +which `daslib/tune` reaches through the `?llvm` witness guard (present exactly when the build is configured with dasLLVM) and re-exports. **Any program can spell the annotations and compile whether or not dasLLVM is configured.** Without dasLLVM every one of them does nothing at all - no sidecar read, no stamp, no `_variants()` registry, no tuner spawn - and the reference diff --git a/src/ast/ast_program.cpp b/src/ast/ast_program.cpp index 548833d608..b49aaad2c6 100644 --- a/src/ast/ast_program.cpp +++ b/src/ast/ast_program.cpp @@ -572,8 +572,10 @@ namespace das { } } // a by-value cycle cannot exist (the layout would be infinite); what remains unsorted is a - // container-mediated cycle, which needs no definition order - it keeps the original order - // after everything the sort did place + // container-mediated cycle's members plus every struct that depends on one of them. The + // members need no definition order (TArray holds no T), so they keep the original order + // after everything the sort did place; a by-value dependent of a cycle member still emits + // an incomplete type there, as it did before this fallback existed if ( sorted.size() != structs.size() ) { das_hash_set placed; for ( auto s : sorted ) placed.insert(s); diff --git a/src/builtin/ARCHITECTURE.md b/src/builtin/ARCHITECTURE.md index 29c383e29c..0e0ceb1250 100644 --- a/src/builtin/ARCHITECTURE.md +++ b/src/builtin/ARCHITECTURE.md @@ -44,8 +44,10 @@ current directory, and it sits beside the JIT DLL cache. `` is the script' without its extension. `` is the first 8 hex digits of a 64-bit hash over the normalized script path, the running executable's mtime and size (the host resolves its own path through the OS, so a launch by bare name through `PATH` keys the same file as a launch by path), the host's -own command line - every argument up to the first `--`, the arguments after it belong to the -script - and every `NAME=VALUE` environment pair whose name starts with `DAS`, sorted. +own command line - every argument up to the first `--`, plus a `--jit-target` after it (the one +script argument that changes what the host compiles: the target's folds); the other arguments after +`--` belong to the script - and every `NAME=VALUE` environment pair whose name starts with `DAS`, +sorted. The binary, the command line and the environment are in the key because each changes the compile without changing a source file: macros read the tune and JIT environment at compile diff --git a/src/builtin/REVIEW.md b/src/builtin/REVIEW.md index 7276861558..d2c6d55a0c 100644 --- a/src/builtin/REVIEW.md +++ b/src/builtin/REVIEW.md @@ -3,19 +3,11 @@ **Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture doc: `ARCHITECTURE.md`. -- **A diff that adds or changes an `addExtern...`/`addInterop` registration under this folder - uses, for a plain-value bind, `addExternInline` or `addExternInlineEx` when the name its - file passes to `Module(...)` is an Inline module, and an `addExtern...` entry point whose - name does not contain `Inline` in every other module.** A bind is a C++ function registered - into a module with an `addExtern...` or `addInterop` entry point. The Inline modules are `$` - (builtin), `math`, `strings` and `jit`. A plain-value bind returns nothing at all, or a value - that is neither a reference nor a result the callee writes into the caller's result slot; it - is not an interop bind (`addInterop`), and not a generic container or equality helper - (`das_vector_*`, `das_equ*`, `das_nequ*`, `das_handle_equ*`, `das_handle_nequ*`). - - **Weakening `review_nttp.das`'s bind-flavor scan, which `REVIEW.das` runs, is a defect** - fix a bind the scan reports by switching the bind, and never drop a module from - `review_nttp.das`'s `require` list, which sets the modules the scan covers. + `review_nttp.das`'s `require` list, which sets the modules the scan covers. The scan reads the + binds compiled into the RUNNING binary, so a diff that changes a bind runs the gate against a + binary built from the diff - a stale binary yields a false green. - **A diff that adds a module under this folder adds it to `review_nttp.das`'s `require` list, in the same change** - a module off the list is a module the scan never sees. @@ -30,12 +22,15 @@ outside `DAS_MODULE_CACHE_POLICY_FIELDS` is a defect - put the field on the list instead** - the list drives both the record's policy stream and the compare that refuses a record written under other policies, so a field handled outside it is written without being compared, or compared - without being written. A diff that adds a field to `CodeOfPolicies` itself applies - `include/daScript/simulate/REVIEW.md`. + without being written. - **A diff that adds a diagnostic to `AstSerializer::serializeProgram` or `AstSerializer::serializeProgramImpl` in `module_builtin_ast_serialize.cpp`, or drops the `quietCache` gate from one already there, is a defect - gate every line those two functions print on the serializer's `quietCache`** - the default cache is on unasked for an ordinary run, - so an ungated line becomes output every user sees. A diff to `trySerializeProgramModule` - (`src/ast/ast_parse.cpp`) applies `src/ast/REVIEW.md` too. + so an ungated line becomes output every user sees. + +- **A diff that changes what `ModuleFileCache::defaultPath` folds into the module-cache key - + the binary, the command line, the environment names, or which script arguments count - updates + the cache-key paragraph of `ARCHITECTURE.md` in the same change.** The key is what stops a + native-compiled module serving a cross compile, so a wrong description of it gets trusted. diff --git a/src/builtin/module_builtin_runtime.cpp b/src/builtin/module_builtin_runtime.cpp index 78ff3bbd73..91ede69f87 100644 --- a/src/builtin/module_builtin_runtime.cpp +++ b/src/builtin/module_builtin_runtime.cpp @@ -2042,31 +2042,30 @@ namespace das #endif } - // The cross-compilation TARGET platform, or "" on a normal (non-cross) run. - // Unlike get_platform_name() (the host, a compile-time #if), this reads the - // active jit cross-compile target from the command line (`--jit-target=` - // after `--`) -- already in g_CommandLineArguments at process start, so it is - // valid at .das_module-initialize time, before the jit codegen macro runs. - // Lets a .das_module register a target-native module only when cross-compiling - // for that target (e.g. dasOpenGL registers its wasm GLES3 module for emscripten, - // while a normal desktop run keeps the pure-das opengl.das). Only the wasm triple - // is mapped today (-> "emscripten"); other cross targets return "". - // the --jit-target triple this compile emits for, "" on a native compile. A property of the - // whole compile (the triple is fixed before the first module parses), so the folds below are - // compile-time constants: an artifact for another machine decides its tiers from the target. + // The --jit-target triple this compile emits for, "" on a native compile. Read from the + // command line (`--jit-target=` or `--jit-target ` after `--`), already in + // g_CommandLineArguments at process start, so it is valid at .das_module-initialize time, + // before the jit codegen macro runs. A property of the whole compile (the triple is fixed + // before the first module parses), so the folds below are compile-time constants: an artifact + // for another machine decides its tiers from the target. das_get_cross_platform_name maps the + // triple to a platform name - only the wasm triples are mapped today (-> "emscripten"), which + // lets a .das_module register a target-native module only when cross-compiling for that + // target (dasOpenGL registers its wasm GLES3 module for emscripten; a desktop run keeps the + // pure-das opengl.das). const char * das_get_target_triple() { char ** argv = (char **) g_CommandLineArguments.data; uint64_t n = g_CommandLineArguments.size; + const char * found = ""; // the LAST occurrence wins, as every other argv reader here takes it for ( uint64_t i=0; i` field inside Inner would -// be a real C++ cycle - TArray takes sizeof(T) - so that shape stays out of the test.) +// order, where the generated C++ embeds an incomplete type. Holder and Inc are the other shape: +// a genuine container cycle (each holds an array of the other), which the sort cannot place and +// appends unsorted - legal C++, because TArray holds no T by value. The AOT build of this +// file is the gate; the assertions only keep the shapes alive. struct Outer { inner : Inner @@ -19,6 +20,16 @@ struct Inner { weight : float } +struct Holder { + arr : array + n : int +} + +struct Inc { + back : array + w : float +} + def outer_tag(o : Outer) : int { return o.tag * 2 } @@ -30,4 +41,14 @@ def test_struct_order(t : T?) { t |> equal(invoke(o.inner.on_outer, o), 42) t |> equal(o.inner.weight, 0.5) } + t |> run("two structs holding arrays of each other sort as a container cycle") @(t : T?) { + var h = Holder(n = 3) + var inc = Inc(w = 1.5) + h.arr |> emplace(inc) + var inner = Holder(n = 7) + h.arr[0].back |> emplace(inner) + t |> equal(h.arr[0].back[0].n, 7) + t |> equal(h.arr[0].w, 1.5) + delete h + } } diff --git a/tests/jit_tests/_cross_target_fixture.das b/tests/jit_tests/_cross_target_fixture.das index 553ac303d6..469328062b 100644 --- a/tests/jit_tests/_cross_target_fixture.das +++ b/tests/jit_tests/_cross_target_fixture.das @@ -4,10 +4,15 @@ options gen2 // over the target triple selects (the same test the tune framework's inert gate makes), so the // parent can compare a native compile against --jit-target= +// the module-level `let` gate shape the engine's tier files use (dasllama_math_default's wasm dot +// pick), folded through a different path than the static_if below +let TARGET_IS_WASM = get_target_architecture_name() == "wasm64" || get_target_architecture_name() == "wasm32" + [export] def main { print("TRIPLE=[{get_target_triple()}]\n") print("TARGET_ARCH=[{get_target_architecture_name()}]\n") + print("LET_WASM=[{TARGET_IS_WASM}]\n") print("HOST_ARCH=[{get_architecture_name()}]\n") print("CROSS_PLATFORM=[{get_cross_platform_name()}]\n") static_if (!empty(get_target_triple())) { diff --git a/tests/jit_tests/cross_target_folds.das b/tests/jit_tests/cross_target_folds.das index c3a6e145ea..4889680486 100644 --- a/tests/jit_tests/cross_target_folds.das +++ b/tests/jit_tests/cross_target_folds.das @@ -56,6 +56,7 @@ def test_cross_target_folds(t : T?) { t |> success(ln |> has("TARGET_ARCH=[{get_architecture_name()}]"), "the target arch is the host's") t |> success(ln |> has("CROSS_PLATFORM=[]"), "no cross platform") t |> success(ln |> has("CROSS=[no]"), "a native compile takes the native arm") + t |> success(ln |> has("LET_WASM=[false]"), "a module-level let over the target folds false natively") } t |> run("a wasm64 cross target answers wasm64 and turns the framework inert") @(t : T?) { var ln : array @@ -65,6 +66,12 @@ def test_cross_target_folds(t : T?) { t |> success(ln |> has("HOST_ARCH=[{get_architecture_name()}]"), "the host arch stays the host's") t |> success(ln |> has("CROSS_PLATFORM=[emscripten]"), "the cross platform is emscripten") t |> success(ln |> has("CROSS=[yes]"), "a cross compile takes the cross arm") + t |> success(ln |> has("LET_WASM=[true]"), "a module-level let over the target folds true for wasm64") + } + t |> run("the last --jit-target on the command line wins, as every other argv reader takes it") @(t : T?) { + var ln : array + spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-target=x86_64-unknown-linux-gnu --jit-target=wasm64-unknown-emscripten --jit-compile-only", ln) + t |> success(ln |> has("TRIPLE=[wasm64-unknown-emscripten]"), "the last triple folds: {join(ln, " | ")}") } t |> run("an x86_64 linux cross target answers x86_64") @(t : T?) { var ln : array diff --git a/tests/jit_tests/wasm_idot_lowering.das b/tests/jit_tests/wasm_idot_lowering.das index a373684a56..bab3f1a312 100644 --- a/tests/jit_tests/wasm_idot_lowering.das +++ b/tests/jit_tests/wasm_idot_lowering.das @@ -1,8 +1,10 @@ -// The signed idot family on the wasm64 cross target lowers to the ISA's own int8 dot, -// i32x4.relaxed_dot_i8x16_i7x16_add_s (the sign trick makes the second operand 7-bit), and never -// to the generic widen-multiply chain that runs a fifth as fast there; a host compile of the same -// fixture emits no wasm intrinsic. Compile-only keeps both compiles on this host and never touches -// the artifact cache, so --jit-dump prints every time. +// The signed idot family on the wasm64 cross target lowers to the ISA's exact int8 dot halves - +// the i16x8 extmul products summed by i32x4.extadd_pairwise_i16x8_s and folded even/odd - and never +// to the generic widen-multiply chain that runs a fifth as fast there; the relaxed 7-bit dot is +// refused by design (it wraps at -128, and +relaxed-simd re-means min/max/mad), so the dump carries +// no relaxed intrinsic either. A host compile of the same fixture emits no wasm intrinsic. +// Compile-only keeps both compiles on this host and never touches the artifact cache, so --jit-dump +// prints every time. options gen2 options no_aot @@ -12,7 +14,8 @@ require daslib/strings_boost require strings let FIXTURE = "{get_das_root()}/tests/jit_tests/_wasm_idot_fixture.das" -let RELAXED_DOT = "llvm.wasm.relaxed.dot.i8x16.i7x16.add.signed" +let PAIRWISE_ADD = "llvm.wasm.extadd.pairwise.signed" +let RELAXED_DOT = "llvm.wasm.relaxed.dot" def private spawn_child(cmd : string; var lines : array) : int { var rc : int @@ -50,11 +53,12 @@ def test_wasm_idot_lowering(t : T?) { } let args <- get_command_line_arguments() // locked view — do not delete let bin = args[0] - t |> run("the wasm64 cross compile lowers idot4 to the relaxed int8 dot") @(t : T?) { + t |> run("the wasm64 cross compile lowers idot4 to the exact pairwise int8 dot") @(t : T?) { var ln : array spawn_child("\"{bin}\" -jit \"{FIXTURE}\" -- --jit-target=wasm64-unknown-emscripten --jit-compile-only --jit-dump", ln) - let dots = count_with(ln, RELAXED_DOT) - t |> success(dots >= 2, "the declaration and the call of {RELAXED_DOT} are in the dump ({dots} lines, {length(ln)} lines total)") + let pairs = count_with(ln, PAIRWISE_ADD) + t |> success(pairs >= 3, "the declaration and the two half calls of {PAIRWISE_ADD} are in the dump ({pairs} lines, {length(ln)} lines total)") + t |> success(count_with(ln, RELAXED_DOT) == 0, "the relaxed 7-bit dot is never emitted") t |> success(count_with(ln, "idot_prod") == 0, "no generic widen-multiply chain remains") } t |> run("a host compile emits no wasm intrinsic") @(t : T?) { diff --git a/tests/language/optional_require.das b/tests/language/optional_require.das index 84a73a82ab..b566a841a3 100644 --- a/tests/language/optional_require.das +++ b/tests/language/optional_require.das @@ -22,6 +22,7 @@ require ?daslib/no_such_file_xyz optional_require_fixture_skipme // path guard // scan identically to ast_requireModule (it once stopped at '%', mangling guard AND target) require ?%/tests/language/optional_require_fixture.das optional_require_fixture_pctguard // nolint:STYLE030 — %/ guard resolves -> target loaded; referenced only under static_if require ?%/tests/language/no_such_file_xyz.das optional_require_fixture_pctskip // %/ guard unresolvable -> SKIPPED, target file resolves +require optional_require_fixture_private // requires optional_require_fixture_hidden PRIVATELY: the module_exists visibility pin below [test] def test_optional_require_guard(t : T?) { @@ -53,6 +54,16 @@ def test_optional_require_path_guard(t : T?) { } else { this_symbol_must_never_resolve_module_exists_xyz() } + //! the half that separates module_exists from builtin_module_exists: a module another module + //! requires privately is IN the program (the registry says so) and NOT visible from here + t |> equal(optional_require_fixture_private::private_marker(), 46) + t |> success(typeinfo builtin_module_exists(optional_require_fixture_hidden), "a privately required module is in the process registry") + t |> success(!typeinfo module_exists(optional_require_fixture_hidden), "and module_exists answers for THIS module's view, where it is not visible") + static_if (typeinfo module_exists(optional_require_fixture_hidden)) { + this_symbol_must_never_resolve_hidden_xyz() + } else { + t |> success(true) + } // path guard unresolvable: the require was SKIPPED even though the target file exists on disk — // the pin for path-guard semantics (a plain-name guard would have loaded it via the fallback) t |> success(!typeinfo builtin_module_exists(optional_require_fixture_skipme)) diff --git a/tests/language/optional_require_fixture_hidden.das b/tests/language/optional_require_fixture_hidden.das new file mode 100644 index 0000000000..dc3aa0b889 --- /dev/null +++ b/tests/language/optional_require_fixture_hidden.das @@ -0,0 +1,8 @@ +options gen2 +// the module optional_require_fixture_private requires PRIVATELY: loaded into the program, so the +// process registry (builtin_module_exists) knows it, yet not visible from the program root +module optional_require_fixture_hidden shared public + +def hidden_marker() : int { + return 45 +} diff --git a/tests/language/optional_require_fixture_private.das b/tests/language/optional_require_fixture_private.das new file mode 100644 index 0000000000..6e163770b1 --- /dev/null +++ b/tests/language/optional_require_fixture_private.das @@ -0,0 +1,8 @@ +options gen2 +module optional_require_fixture_private shared public + +require optional_require_fixture_hidden // private on purpose: the root sees this module, not that one + +def private_marker() : int { + return hidden_marker() + 1 +} diff --git a/tests/metal/REVIEW.md b/tests/metal/REVIEW.md index a23d3e3326..4d4648338e 100644 --- a/tests/metal/REVIEW.md +++ b/tests/metal/REVIEW.md @@ -9,5 +9,5 @@ produces its expected values by running the kernel method on the CPU, in a drive sets `gl_GlobalInvocationID` once per thread id. Kernel members are per-thread state, so a reused instance carries the previous thread's values into the next. -**A file in this folder that creates any Metal object asserts `metal_live_object_count() == 0` -before it returns.** +**A diff that adds or changes a `[test]` in this folder that creates any Metal object asserts +`metal_live_object_count() == 0` before that test returns.** diff --git a/utils/daslang/main.cpp b/utils/daslang/main.cpp index ea80b59d86..7ab98fd4ea 100644 --- a/utils/daslang/main.cpp +++ b/utils/daslang/main.cpp @@ -470,8 +470,9 @@ int compile_and_run ( const string & fn, const string & mainFnName, bool outputP access->addExtraModule("ast_verify", getDasRoot() + "/daslib/ast_verify.das"); } // -use-aot links the stubs compiled into THIS binary against the script it runs; a function - // with no stub interprets (never fail_on_no_aot here - the host may be dastest, whose own - // framework has no stubs, and which sets both policies for the test files it compiles) + // with no stub interprets (never fail_on_no_aot here - the host may be dastest, whose driver + // has no stubs even where its testing.das does, and which sets aot / fail_on_no_aot itself + // for the test files it compiles) policies.aot = useAot; policies.fail_on_no_aot = false; if ( useAot ) { @@ -807,7 +808,7 @@ int MAIN_FUNC_NAME ( int argc, char * argv[] ) { // over it bake the target into each module, so a cached native compile must not serve a cross one for ( int i=1, sep=0; i < argc; ++i ) { if ( !sep ) { sep = strcmp(argv[i],"--")==0; continue; } - if ( strncmp(argv[i],"--jit-target",12)==0 ) { + if ( strcmp(argv[i],"--jit-target")==0 || strncmp(argv[i],"--jit-target=",13)==0 ) { hostOptions += argv[i]; hostOptions += '\n'; if ( strcmp(argv[i],"--jit-target")==0 && i+1> 20l} MB in {total_ms} ms total\n") + if (cfg.out == "") { + dlim_gc_stale(cfg.model, tag, q8 ? "q8" : "fp32") // the fresh bake beside the source proves ITS LANE's siblings dead + } return 0 } diff --git a/utils/dasllama-server/REVIEW.md b/utils/dasllama-server/REVIEW.md index 7b26a739c0..c567907240 100644 --- a/utils/dasllama-server/REVIEW.md +++ b/utils/dasllama-server/REVIEW.md @@ -26,7 +26,7 @@ before - a new route requires all of its fields - adds that field to every `cont request to that route, in the same change.** **A diff that changes what a route answers - an item, a field, or a value, including one a -required `dasllama/*` module supplies - re-captures every fixture under `tests/fixtures/` that +`dasllama/*` module supplies - re-captures every fixture under `tests/fixtures/` that records that route, in the same change.** The fixtures are the recorded response shape. **A diff that makes `control.html` or a Playwright `.spec.js` under `tests/` (beside this file) @@ -36,7 +36,7 @@ fixture for the route that answers with that key first.** **A diff that adds a key to what a route answers lists it in that route's `README.md` row, in the same change.** The row is where a consumer learns the key exists. -**A function in this folder that names a type from `dasllama_exchange` in its signature is a -defect.** That module is optional (`require ?llvm`) and its types do not exist in a build without -dasLLVM; call `exchange_policy_from_config(g_config_save_path)` inside a -`static_if (typeinfo module_exists(dasllama_exchange))` arm at each site instead. +**A reference to a `dasllama_exchange` or `llvm_tune` symbol in this folder outside a +`static_if (typeinfo module_exists())` arm is a defect - a signature cannot carry that +guard, so it never names one of their types.** Both modules are optional (`require ?llvm`) and do +not exist in a build without dasLLVM; the unguarded reference fails that build's compile. diff --git a/utils/dasllama-server/ask.das b/utils/dasllama-server/ask.das index cbbfc9da44..980285a718 100644 --- a/utils/dasllama-server/ask.das +++ b/utils/dasllama-server/ask.das @@ -12,7 +12,7 @@ options stack = 524288 // every dasLLAMA program root takes this budget (optio // the three tools), and every sibling is tuned. require dasllama/dasllama // load_model / generate / the shared [tune_scope] -require llvm/daslib/llvm_tune // [tune_policy] + tune_status +require daslib/tune // [tune_policy] + log_tune_status - the shells; the framework rides in when the build has dasLLVM require daslib/jobque_boost require daslib/clargs require daslib/fio diff --git a/utils/dasllama-server/main.das b/utils/dasllama-server/main.das index d84086c1a4..dbff780cf7 100644 --- a/utils/dasllama-server/main.das +++ b/utils/dasllama-server/main.das @@ -25,7 +25,8 @@ options stack = 524288 // every dasLLAMA program root takes this budget (optio require openai_server // the server + route handlers (this directory) require model_catalog // catalog_models_dir — the models-dir resolution require dasllama/dasllama // load_model / QuantMode / load_asr_model -require llvm/daslib/llvm_tune // [tune_policy] + tune_status (the injected guard resolves here) +require daslib/tune // [tune_policy] + log_tune_status (the injected guard resolves here) +require ?llvm llvm/daslib/llvm_tune // nolint:STYLE030,LINT019 — OPTIONAL: tune_suppress_mint, reached under module_exists(llvm_tune) require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the sidecar exchange (boot resolver + submit check), which rides the tune framework require jobque // set_jobque_threads_cap / set_jobque_affinity require daslib/clargs @@ -196,9 +197,11 @@ def suppress_tune_for_setup { } } delete args - if (!configured && !stat("dasllama-server.toml").is_valid - && !stat(path_join(get_this_module_dir(), "dasllama-server.toml")).is_valid) { - tune_suppress_mint("no model configured - setup mode boots on fallback kernels") + static_if (typeinfo module_exists(llvm_tune)) { + if (!configured && !stat("dasllama-server.toml").is_valid + && !stat(path_join(get_this_module_dir(), "dasllama-server.toml")).is_valid) { + tune_suppress_mint("no model configured - setup mode boots on fallback kernels") + } } } diff --git a/utils/dasllama-server/openai_server.das b/utils/dasllama-server/openai_server.das index d6010594a5..b4593b4a10 100644 --- a/utils/dasllama-server/openai_server.das +++ b/utils/dasllama-server/openai_server.das @@ -10,7 +10,7 @@ require dashv/dashv_boost public require daslib/json_boost require daslib/toml // write_toml: POST /config persists the authoritative config require ?llvm dasllama/dasllama_exchange // nolint:STYLE030,LINT019 — OPTIONAL: the /exchange surface + submit rails, which ride the tune framework -require llvm/daslib/llvm_tune // tune_manifest_path — the /exchange apply target +require ?llvm llvm/daslib/llvm_tune // nolint:STYLE030,LINT019 — OPTIONAL: tune_manifest_path, the /exchange apply target, reached only inside the exchange arms require daslib/defer // defer() { delete js } — free the parsed request JSON on every exit path require model_catalog // the curated list + the /catalog download rail require daslib/jobque_boost diff --git a/utils/dasllama-server/test_model_catalog.das b/utils/dasllama-server/test_model_catalog.das index 0260210c7f..71a51c0fdb 100644 --- a/utils/dasllama-server/test_model_catalog.das +++ b/utils/dasllama-server/test_model_catalog.das @@ -92,9 +92,9 @@ def test_catalog_table(t : T?) { packs++ } } - t |> equal(length(set), 5, "the set is three models and two packs") - t |> equal(packs, 2, "two front-end packs") - for (want in ["kitten-nano.gguf", "kitten-mini.gguf", "kokoro-82m.gguf", "tts_g2p.bin", "tts_postag.bin"]) { + t |> equal(length(set), 6, "the set is three models and three packs") + t |> equal(packs, 3, "three front-end packs: the full phoneme pack, its American-only twin, the tagger") + for (want in ["kitten-nano.gguf", "kitten-mini.gguf", "kokoro-82m.gguf", "tts_g2p.bin", "tts_g2p_en_us.bin", "tts_postag.bin"]) { t |> success(key_exists(names, want), "the set carries {want}") } } diff --git a/utils/dasllama-server/txt2wav.das b/utils/dasllama-server/txt2wav.das index 9eb5a6b474..2d675194a5 100644 --- a/utils/dasllama-server/txt2wav.das +++ b/utils/dasllama-server/txt2wav.das @@ -8,7 +8,7 @@ options stack = 524288 // every dasLLAMA program root takes this budget (optio // bin/daslang -jit utils/dasllama-server/txt2wav.das -- --tts --text "Hello." --out hello.wav [--voice expr-voice-2-f] [--speed 1.0] require dasllama/dasllama // load_tts_model / synthesize / write_wav_pcm16 / the shared [tune_scope] -require llvm/daslib/llvm_tune // [tune_policy] + log_tune_status +require daslib/tune // [tune_policy] + log_tune_status - the shells; the framework rides in when the build has dasLLVM require daslib/jobque_boost require daslib/clargs require daslib/defer // defer() { reset_tts_q8() } — the weight-lane pin never outlives the run diff --git a/utils/dasllama-server/wav2txt.das b/utils/dasllama-server/wav2txt.das index cb1bc2a20b..c47422ea8f 100644 --- a/utils/dasllama-server/wav2txt.das +++ b/utils/dasllama-server/wav2txt.das @@ -10,7 +10,7 @@ options stack = 524288 // every dasLLAMA program root takes this budget (optio // bin/daslang -jit utils/dasllama-server/wav2txt.das -- --asr --file [--out t.txt] require dasllama/dasllama // load_asr_model / transcribe / the shared [tune_scope] -require llvm/daslib/llvm_tune // [tune_policy] + log_tune_status +require daslib/tune // [tune_policy] + log_tune_status - the shells; the framework rides in when the build has dasLLVM require daslib/jobque_boost require daslib/clargs require daslib/fio diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index b9f04facd1..f24c60612c 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -20,10 +20,6 @@ vs flat directory). **A diff that adds a command or a flag also adds its `print_usage` line and its `README.md` table row, in the same change.** -**A diff that removes `daspkg` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a -defect.** The removal drops the prebuilt daspkg exe from the release bundle, and the gate -cannot see a removed entry. - **A `cmd_release` bundle that ships an exe without a tune sidecar beside it is a defect** - the tune sidecar is the `.tune.json` file of measured kernel choices the exe reads at run time. @@ -39,12 +35,14 @@ missing a scope key, that is, an entry of the `tune_scopes` list in the deps JSO one the package's `release()` declares that way: a starter file deployed once, then owned by the user. -**A release path that finishes a bundle without writing `.daspkg_release.manifest` is a -defect.** +**A `cmd_release` bundle that finishes without writing `.daspkg_release.manifest` is a +defect.** `release wasm` is out: it wipes its output directory first, so it has no stale file to +account for. **A test in `test_daspkg.das` that reaches the network is a defect** - network coverage belongs in `test_daspkg_git.das`. -**A shell command built from a package, bundle, or app name outside `commands.das`, or without -an `is_safe_pkg_name` check on that name first, is a defect** - `is_safe_pkg_name` is private to -`commands.das`. +**A shell command built from any string a `.das_package` supplies - a package, bundle, app or +module name - outside `commands.das`, or without an `is_safe_pkg_name` check on that string +first, is a defect** - `is_safe_pkg_name` is private to `commands.das`. The emcc flags of +`release_emcc_arg` are the carve-out: free-form shell text by design, appended verbatim. diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index 9d1ddf4712..f28c9e74c7 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -2670,6 +2670,12 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; to_log(LOG_ERROR, "release wasm: unsafe app name `{app_name}` (no path separators / `.` / `..`)\n") return 1 } + for (m in disabled_modules) { // a .das_package string that reaches the cross-compile command line + if (!is_safe_pkg_name(m)) { + to_log(LOG_ERROR, "release wasm: unsafe module name `{m}` in release_wasm_disable_module (no path separators / `.` / `..` / spaces)\n") + return 1 + } + } let main_path = path_join(root, main_script) if (!fexist(main_path)) { to_log(LOG_ERROR, "release wasm: app `{app_name}` main script not found: {main_path}\n") diff --git a/utils/daspkg/fixtures/test_release.das_package b/utils/daspkg/fixtures/test_release.das_package index 2b390a33f0..0c43c50fb1 100644 --- a/utils/daspkg/fixtures/test_release.das_package +++ b/utils/daspkg/fixtures/test_release.das_package @@ -20,4 +20,6 @@ def release() { release_wasm_build("emcmake cmake -S . -B _wasm && cmake --build _wasm") release_wasm_archive("_wasm/liblibDasModuleTest.a") release_wasm_archive("_wasm/liblibTestApp.a") + release_wasm_disable_module("dasvulkan") + release_emcc_arg("--profiling-funcs") } diff --git a/utils/daspkg/test_daspkg.das b/utils/daspkg/test_daspkg.das index b17e10be42..92cb69306d 100644 --- a/utils/daspkg/test_daspkg.das +++ b/utils/daspkg/test_daspkg.das @@ -1781,6 +1781,12 @@ def test_run_das_package_release(t : T?) { tt |> equal(2, length(info.wasm_archives)) tt |> equal("_wasm/liblibDasModuleTest.a", info.wasm_archives[0]) tt |> equal("_wasm/liblibTestApp.a", info.wasm_archives[1]) + // the two wasm-only knobs cross the same hand-written marshaller: a dropped copy would + // cross-compile with the module mounted and the flag missing, silently + tt |> equal(1, length(info.wasm_disabled_modules)) + tt |> equal("dasvulkan", info.wasm_disabled_modules[0]) + tt |> equal(1, length(info.emcc_args)) + tt |> equal("--profiling-funcs", info.emcc_args[0]) } t |> run("returns true with empty spec when release() is missing") @(tt : T?) { var info : PackageReleaseInfo diff --git a/utils/internal/dasllama-ladder/dasllama-deploy.sh b/utils/internal/dasllama-ladder/dasllama-deploy.sh index a04dd739fe..38f653dc44 100644 --- a/utils/internal/dasllama-ladder/dasllama-deploy.sh +++ b/utils/internal/dasllama-ladder/dasllama-deploy.sh @@ -103,6 +103,34 @@ caddy_apply() { snippet="$APP/current/caddy.snippet" [ -f "$snippet" ] || { echo "no $snippet - install a release first"; exit 1; } if grep -q "reverse_proxy 127.0.0.1:$PORT" "$CADDYFILE"; then + # the routes are in; a later snippet may still carry a block the vhost lacks - today the + # `header /examples/*` isolation block - so splice each such block on its own + if grep -q "header /examples/\*" "$snippet" && ! grep -q "header /examples/\*" "$CADDYFILE"; then + ts=$(date +%Y%m%d-%H%M%S) + cp "$CADDYFILE" "$CADDYFILE.bak-$ts" + awk -v snip="$snippet" ' + /^dasllama\.io \{/ && !done { + print + inblock = 0 + while ((getline line < snip) > 0) { + if (line ~ /^header \/examples\/\*/) inblock = 1 + if (inblock) print "\t" line + if (inblock && line ~ /^\}/) inblock = 0 + } + close(snip); done=1; next + } + { print } + ' "$CADDYFILE.bak-$ts" > "$CADDYFILE" + if ! grep -q "header /examples/\*" "$CADDYFILE"; then + echo "caddy: header splice inserted nothing (dasllama.io vhost not matched) - restoring $CADDYFILE.bak-$ts"; cp "$CADDYFILE.bak-$ts" "$CADDYFILE"; exit 1 + fi + if ! caddy validate --config "$CADDYFILE" --adapter caddyfile >/dev/null 2>&1; then + echo "caddy validate FAILED - restoring $CADDYFILE.bak-$ts"; cp "$CADDYFILE.bak-$ts" "$CADDYFILE"; exit 1 + fi + systemctl reload caddy + echo "caddy: /examples/* isolation headers spliced + reloaded (backup $CADDYFILE.bak-$ts)" + return 0 + fi echo "caddy: /api already spliced - nothing to do"; return 0 fi ts=$(date +%Y%m%d-%H%M%S) diff --git a/web/REVIEW.md b/web/REVIEW.md index e6a0b3876a..8a5268e218 100644 --- a/web/REVIEW.md +++ b/web/REVIEW.md @@ -1,6 +1,7 @@ # web (the WASM build and its shells) Code Review Checklist -**Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** A file +**Read `REVIEW_COMMON.md` (repo root) first - its contract binds this checklist.** Architecture +doc: `README.md`. A file under this folder is served when the deploy (`.github/workflows/pages.yml`, repo root), `daspkg release wasm`, or a build step that feeds either copies it into a page a visitor loads. From 5dae7d7d3b9b88f8ba1b2f1da540eae73863254a Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 14:20:37 -0700 Subject: [PATCH 36/43] the gate residue of the fix batch: the two rules the folder gates parse come back (the Inline module list in src/builtin, the shipped-exe decision in daspkg), the two documents that crossed 300 lines lose the lines they grew, the image layout stamp follows the TTS serializer's verify lines, the target folds get their reference text, g2p_pack_path is demonstrated in the TTS tutorial and grouped on its page, and the deploy script asks the dasllama.io vhost - not the whole Caddyfile - whether a block is already spliced Co-Authored-By: Claude Fable 5.1 --- doc/reflections/das2rst.das | 2 +- .../tutorials/dasLLAMA_16_text_to_speech.rst | 6 +++++- ...target_architecture_name-0x5a46cb8acf2fed30.rst | 1 + ...builtin-get_target_triple-0x6b5a8940c4c62c1.rst | 1 + modules/dasLLAMA/REVIEW.das | 2 +- modules/dasLLAMA/dasllama/dasllama_styletts2.das | 3 +-- modules/dasLLAMA/tests/REVIEW.md | 10 ++++------ modules/dasLLVM/ARCHITECTURE.md | 13 ++++++------- src/builtin/REVIEW.md | 6 +++++- tutorials/dasLLAMA/16_text_to_speech.das | 6 ++++++ utils/daspkg/REVIEW.md | 4 ++++ utils/internal/dasllama-ladder/dasllama-deploy.sh | 14 ++++++++++---- 12 files changed, 45 insertions(+), 23 deletions(-) create mode 100644 doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst create mode 100644 doc/source/stdlib/handmade/function-builtin-get_target_triple-0x6b5a8940c4c62c1.rst diff --git a/doc/reflections/das2rst.das b/doc/reflections/das2rst.das index 10808c50f6..d8625f9485 100644 --- a/doc/reflections/das2rst.das +++ b/doc/reflections/das2rst.das @@ -401,7 +401,7 @@ def document_module_dasllama_tts(_root : string) { } var mod = [find_module("dasllama_tts"), find_module("dasllama_tts_types")] var groups <- array( - group_by_regex("Loading and capabilities", mod, %regex~(load_tts_model|caps|tts_voice_lang|finalize)$%%), + group_by_regex("Loading and capabilities", mod, %regex~(load_tts_model|g2p_pack_path|caps|tts_voice_lang|finalize)$%%), group_by_regex("Text front end", mod, %regex~(tts_normalize|tts_phonemize|tts_chunks)$%%), group_by_regex("Synthesis", mod, %regex~(synthesize|synthesize_stream)$%%), group_by_regex("The served lane", mod, %regex~(set_tts_q8|reset_tts_q8|tts_serves_q8)$%%), diff --git a/doc/source/reference/tutorials/dasLLAMA_16_text_to_speech.rst b/doc/source/reference/tutorials/dasLLAMA_16_text_to_speech.rst index 1c5c11970e..7af66adfc2 100644 --- a/doc/source/reference/tutorials/dasLLAMA_16_text_to_speech.rst +++ b/doc/source/reference/tutorials/dasLLAMA_16_text_to_speech.rst @@ -38,11 +38,15 @@ One loader, the packs beside the model so no family name appears in the call. Two more files travel with the model: ``tts_g2p.bin`` and ``tts_postag.bin``, and the loader reads them from the model's directory. A file that is not a TTS family panics and names the -architecture it found. +architecture it found. ``g2p_pack_path`` names the phoneme pack the loader +takes from a directory: ``tts_g2p.bin`` when it is there, else the +American-only ``tts_g2p_en_us.bin`` - the smaller pack a browser build ships, +which reads no British voice. .. code-block:: das var m <- load_tts_model("kitten-nano.gguf") // tts_g2p.bin + tts_postag.bin sit beside it + print("phoneme pack: {base_name(g2p_pack_path(dir_name("kitten-nano.gguf")))}\n") caps(): ask, don't assume ========================= diff --git a/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst b/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst new file mode 100644 index 0000000000..2129d13caa --- /dev/null +++ b/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst @@ -0,0 +1 @@ +Returns the architecture component of the compile target, such as `"wasm64"`, `"wasm32"`, `"aarch64"` or `"x86_64"`; the host's own architecture when no `--jit-target` names another. Folded to a constant like `get_target_triple`, it is the switch a kernel uses to pick a per-target body. diff --git a/doc/source/stdlib/handmade/function-builtin-get_target_triple-0x6b5a8940c4c62c1.rst b/doc/source/stdlib/handmade/function-builtin-get_target_triple-0x6b5a8940c4c62c1.rst new file mode 100644 index 0000000000..9be8661cc0 --- /dev/null +++ b/doc/source/stdlib/handmade/function-builtin-get_target_triple-0x6b5a8940c4c62c1.rst @@ -0,0 +1 @@ +Returns the LLVM target triple the program is being compiled for, such as `"wasm64-unknown-emscripten"`, read from the command line's `--jit-target` (the last one wins); `""` for the host. The compiler folds the call to a constant, so a `static_if` or an `if` on it costs nothing at run time and the code for the other target is not compiled. diff --git a/modules/dasLLAMA/REVIEW.das b/modules/dasLLAMA/REVIEW.das index 915b5fdd6a..0074d68336 100644 --- a/modules/dasLLAMA/REVIEW.das +++ b/modules/dasLLAMA/REVIEW.das @@ -545,7 +545,7 @@ let private IMAGE_FILE = "modules/dasLLAMA/dasllama/dasllama_image.das" // in file order. A closure change with IMAGE_VERSION unmoved is red; the finding prints the // value to re-stamp with. let private IMAGE_LAYOUT_STAMP_VERSION = 33 -let private IMAGE_LAYOUT_STAMP_HASH = 0xdd91b563594dfb9ful +let private IMAGE_LAYOUT_STAMP_HASH = 0xec0727cff52428eeul // The helpers that decide WHERE bytes land: the page pad, the plane and total sizing, the // writer's append / zero-fill / header patch, and the header's scalar stores. Changing one diff --git a/modules/dasLLAMA/dasllama/dasllama_styletts2.das b/modules/dasLLAMA/dasllama/dasllama_styletts2.das index 5c5be57347..97cfbae80e 100644 --- a/modules/dasLLAMA/dasllama/dasllama_styletts2.das +++ b/modules/dasLLAMA/dasllama/dasllama_styletts2.das @@ -290,8 +290,7 @@ def private serialize_image_meta(var arch : Archive; var m : StyleTts2Model) { arch |> serialize(m.decoder) arch |> serialize(m.source) arch |> serialize_raw(m.q8) - // the family records are hand-listed too, so each pins its own count: a field added to one of - // them and not written here would read back zero on every image load + // the family records are hand-listed too, so each pins its count - an unwritten field reads back zero verify(count_meta_fields(m.kitten) == 2) // grew KittenFamily? extend this list verify(count_meta_fields(m.kokoro) == 1) // grew KokoroFamily? extend this list arch |> serialize(m.kitten.speed_priors) diff --git a/modules/dasLLAMA/tests/REVIEW.md b/modules/dasLLAMA/tests/REVIEW.md index 2b0f1ac245..ca053e605a 100644 --- a/modules/dasLLAMA/tests/REVIEW.md +++ b/modules/dasLLAMA/tests/REVIEW.md @@ -176,9 +176,8 @@ defect.** A stocked artifact is anything the test reads out of `models_dir()` th build does not produce - model files, mmprojs, front-end packs, image fixtures, oracle dumps. Any one of these names the producer: a row in `../performance/model_specs.das`; a row's `companions` list; a row in `asr_catalog` (`../performance/profile_common.das`); a convert -script beside `model_specs.das` in `../performance/`. For an oracle dump, the mint script -stocked beside the dumps under `models_dir()` also counts, and the test that loads the dump -names it. +script beside `model_specs.das` in `../performance/`; for an oracle dump, the mint script stocked +beside the dumps under `models_dir()`, named by the test that loads the dump. **A test that reads a vision encode oracle dump without naming the minting arm in its header - the backend, the flash-attention setting, and the mmproj precision the dump came from - is a @@ -216,9 +215,8 @@ model and its blob twin share one shape, so one session serves both. **A diff that adds a model-loading block to a `run.das` MODEL suite (`decode`, `mtp`, `prefill`, `matrix`, `image`, `image-vulkan`, `coverage` - not `model-free` or `stocked`) tags it with its -family.** The family -tag is the token passed to `family_on(t, name)` (`_model_tier.das`). An untagged block -silently joins every family's gate. +family.** The family tag is the token passed to `family_on(t, name)` (`_model_tier.das`). An +untagged block silently joins every family's gate. **No CPU-control batch parity runs against `Llama-3.3-70B-Instruct-Q4_K_M.gguf`.** The batched code paths get their parity on small models, through pins. diff --git a/modules/dasLLVM/ARCHITECTURE.md b/modules/dasLLVM/ARCHITECTURE.md index 2cd7fa826a..492f3e42c7 100644 --- a/modules/dasLLVM/ARCHITECTURE.md +++ b/modules/dasLLVM/ARCHITECTURE.md @@ -292,10 +292,9 @@ neither backend produces them from the generic form: AArch64 expands it to zip/u of folding to SDOT, and the wasm backend runs it a fifth as fast. The wasm form is the ISA's two halves of an int8 dot, `i16x8.extmul_{low,high}_i8x16_s` (what LLVM makes of `mul(sext, sext)`) and `i32x4.extadd_pairwise_i16x8_s`; the pairwise sums land as byte pairs, and one even/odd shuffle-add -folds them into the quad lanes the generic form defines - exact for every int8 lane, like the -builtin's contract. The relaxed-SIMD dot (`i32x4.relaxed_dot_i8x16_i7x16_add_s`) is deliberately -NOT used: its second operand is 7-bit, so the sign trick that would feed it (`dot(w, x) == -dot(sign(x)*w, |x|)`) wraps at -128 in either operand and answers the wrong sign there, and -`+relaxed-simd` is a whole-module switch that also turns float-vector `min`/`max` and `mad` into -engine-defined instructions (NaN and signed-zero answers, fusion) - the feature string stays -`+simd128,+nontrapping-fptoint`, matching the runtime archive. +folds them into the quad lanes the generic form defines - exact for every int8 lane. The +relaxed-SIMD dot (`i32x4.relaxed_dot_i8x16_i7x16_add_s`) is NOT used: its second operand is 7-bit, +so the sign trick that would feed it (`dot(w, x) == dot(sign(x)*w, |x|)`) wraps at -128 in either +operand and answers the wrong sign there, and `+relaxed-simd` is a whole-module switch that also +turns float-vector `min`/`max` and `mad` into engine-defined instructions (NaN and signed-zero +answers, fusion) - the feature string stays `+simd128,+nontrapping-fptoint`, the runtime archive's. diff --git a/src/builtin/REVIEW.md b/src/builtin/REVIEW.md index d2c6d55a0c..57258ef318 100644 --- a/src/builtin/REVIEW.md +++ b/src/builtin/REVIEW.md @@ -7,7 +7,11 @@ fix a bind the scan reports by switching the bind, and never drop a module from `review_nttp.das`'s `require` list, which sets the modules the scan covers. The scan reads the binds compiled into the RUNNING binary, so a diff that changes a bind runs the gate against a - binary built from the diff - a stale binary yields a false green. + binary built from the diff - a stale binary yields a false green. The Inline modules are `$` + (builtin), `math`, `strings` and `jit`. In those, a plain-value bind - one returning nothing, or + a value that is neither a reference nor written into the caller's result slot - registers + through `addExternInline` or `addExternInlineEx`; `REVIEW.das` holds this list and the scan's + `inlineOnlyModules` to the same names. - **A diff that adds a module under this folder adds it to `review_nttp.das`'s `require` list, in the same change** - a module off the list is a module the scan never sees. diff --git a/tutorials/dasLLAMA/16_text_to_speech.das b/tutorials/dasLLAMA/16_text_to_speech.das index 6d8ead3cbc..55cfa1ea11 100644 --- a/tutorials/dasLLAMA/16_text_to_speech.das +++ b/tutorials/dasLLAMA/16_text_to_speech.das @@ -5,6 +5,7 @@ require dasllama/dasllama require daslib/jobque_boost require daslib/defer require daslib/strings_boost +require daslib/fio require strings // Tutorial dasLLAMA-16: Text to Speech @@ -127,6 +128,11 @@ def main { // nolint:STYLE038 - one linear tutorial walkthrough // letters and two different sounds. synthesize runs both passes for you; run them alone and // you see what the model is asked to say. // + // g2p_pack_path names the phoneme pack the loader took from the model's directory: the full + // tts_g2p.bin when it is there, else the American-only tts_g2p_en_us.bin - the smaller pack a + // browser build ships, which reads no British voice. + print("phoneme pack: {base_name(g2p_pack_path(dir_name(model_path)))}\n") + // // A synthesis phonemizes in the DIALECT ITS VOICE WAS TRAINED ON, so this pass has to be // asked in the same one, or you are reading a document about a different sound. Every voice // carries its language: c.voice_langs[i] beside c.voices[i], and tts_voice_lang answers for diff --git a/utils/daspkg/REVIEW.md b/utils/daspkg/REVIEW.md index f24c60612c..b0a6525d07 100644 --- a/utils/daspkg/REVIEW.md +++ b/utils/daspkg/REVIEW.md @@ -35,6 +35,10 @@ missing a scope key, that is, an entry of the `tune_scopes` list in the deps JSO one the package's `release()` declares that way: a starter file deployed once, then owned by the user. +**A diff that removes `daspkg` from `DAS_UTILS_SHIPPED_EXES` in `utils/CMakeLists.txt` is a +defect.** The removal drops the prebuilt daspkg exe from the release bundle, and the parent's +gate cannot see a removed entry - it reads this sentence to know the decision stands. + **A `cmd_release` bundle that finishes without writing `.daspkg_release.manifest` is a defect.** `release wasm` is out: it wipes its output directory first, so it has no stale file to account for. diff --git a/utils/internal/dasllama-ladder/dasllama-deploy.sh b/utils/internal/dasllama-ladder/dasllama-deploy.sh index 38f653dc44..06a3012215 100644 --- a/utils/internal/dasllama-ladder/dasllama-deploy.sh +++ b/utils/internal/dasllama-ladder/dasllama-deploy.sh @@ -97,15 +97,21 @@ EOF echo "provisioned: user=$SVCUSER data=$DATA unit=$UNIT; restic snapshot line added" } +vhost_has() { + # true when the dasllama.io block of the shared Caddyfile carries $1 - the other vhosts + # (daslang.io) may spell the same directive, so a whole-file grep is not the question + awk '/^dasllama\.io \{/ { b = 1 } b { print } b && /^\}/ { b = 0 }' "$CADDYFILE" | grep -q "$1" +} + caddy_apply() { # Splice caddy.snippet (from the installed release) into the dasllama.io vhost, ahead of # root/file_server. Idempotent, validated before reload, with a timestamped Caddyfile backup. snippet="$APP/current/caddy.snippet" [ -f "$snippet" ] || { echo "no $snippet - install a release first"; exit 1; } - if grep -q "reverse_proxy 127.0.0.1:$PORT" "$CADDYFILE"; then + if vhost_has "reverse_proxy 127.0.0.1:$PORT"; then # the routes are in; a later snippet may still carry a block the vhost lacks - today the # `header /examples/*` isolation block - so splice each such block on its own - if grep -q "header /examples/\*" "$snippet" && ! grep -q "header /examples/\*" "$CADDYFILE"; then + if grep -q "header /examples/\*" "$snippet" && ! vhost_has "header /examples/\*"; then ts=$(date +%Y%m%d-%H%M%S) cp "$CADDYFILE" "$CADDYFILE.bak-$ts" awk -v snip="$snippet" ' @@ -121,7 +127,7 @@ caddy_apply() { } { print } ' "$CADDYFILE.bak-$ts" > "$CADDYFILE" - if ! grep -q "header /examples/\*" "$CADDYFILE"; then + if ! vhost_has "header /examples/\*"; then echo "caddy: header splice inserted nothing (dasllama.io vhost not matched) - restoring $CADDYFILE.bak-$ts"; cp "$CADDYFILE.bak-$ts" "$CADDYFILE"; exit 1 fi if ! caddy validate --config "$CADDYFILE" --adapter caddyfile >/dev/null 2>&1; then @@ -146,7 +152,7 @@ caddy_apply() { # The awk match is exact (`dasllama.io {`); if the vhost is ever reformatted (shared address, # renamed) it matches nothing, validate still passes on the unchanged file, and we would # wrongly report success. Confirm the proxy line actually landed before reloading. - if ! grep -q "reverse_proxy 127.0.0.1:$PORT" "$CADDYFILE"; then + if ! vhost_has "reverse_proxy 127.0.0.1:$PORT"; then echo "caddy: splice inserted nothing (dasllama.io vhost not matched) - restoring $CADDYFILE.bak-$ts"; cp "$CADDYFILE.bak-$ts" "$CADDYFILE"; exit 1 fi if ! caddy validate --config "$CADDYFILE" --adapter caddyfile >/dev/null 2>&1; then From e1bc2b187ad2f745578e3a904478510fb0ef14fe Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 14:44:25 -0700 Subject: [PATCH 37/43] the exe-rail residue: the llvm descriptor registers the llvm/daslib paths only with the witness, so a build without dasLLVM refuses a direct framework require instead of compiling it hollow; log_tune_status is one function in every build with the framework's row type kept behind the guard; the tune-scope ordering require is named as one; the image stamp follows the serializer's one-line comment Co-Authored-By: Claude Fable 5.1 --- daslib/ARCHITECTURE.md | 6 +++ daslib/tune.das | 40 ++++++++++++------- modules/dasLLAMA/REVIEW.das | 2 +- .../dasLLAMA/dasllama/dasllama_math_gen.das | 2 +- modules/dasLLVM/.das_module | 21 ++++++---- skills/tune.md | 4 +- 6 files changed, 49 insertions(+), 26 deletions(-) diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index fd1c76d356..43e0ba77be 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -237,3 +237,9 @@ Three companions carry a concern each; a section number is unique across all fou half on fixtures, and `tests/daslib/test_tune_shells.das` is tier-agnostic by design - so the pair is recorded here: a trait that answered off the process registry instead would take the framework arm inside a tool's nested compile, where the framework is not mounted. +- **`modules/dasLLVM/.das_module` registers the `llvm/daslib/*` paths only after the witness + registered (or was compiled in), so without dasLLVM a direct `require llvm/daslib/x` is a missing + prerequisite and never a hollow compile.** The framework's das files compile without their C++ + module, which is how a tool that reads a `_variants()` registry through a direct require + once compiled in a witness-less world with inert shells and no registry; a world that cannot + mount the framework now refuses the require, and a lint skips the file as one it cannot compile. diff --git a/daslib/tune.das b/daslib/tune.das index 04200df6bf..c6bf82d485 100644 --- a/daslib/tune.das +++ b/daslib/tune.das @@ -113,23 +113,33 @@ class private TuneDefaultPolicyShell : AstPassMacro { //! Silent when the table is empty, and in a build without the framework - nothing was stamped. [arch(at = "ARCHITECTURE.md#tune-shells-module-exists")] def log_tune_status(appName : string) { + var inscope rows <- tune_status_rows() + if (empty(rows)) { + return + } + var tuned = 0 + for (r in rows) { + tuned += (r.source == "manifest" || r.source == "profile") ? 1 : 0 + } + to_log(LOG_INFO, "{appName}: tune status ({tuned}/{length(rows)} kernels tuned for this box):\n") + for (r in rows) { + to_log(LOG_INFO, " {r.fname}: {empty(r.suffix) ? "reference" : r.suffix} ({r.source})\n") + } + if (tuned < length(rows)) { + to_log(LOG_INFO, "{appName}: run with --tune to tune the remaining kernels for this box\n") + } +} + +//! The status table as plain strings - the framework's row type never crosses the guard, so the +//! caller above is one function in every build; empty without the framework. +def private tune_status_rows() : array> { + var rows : array> static_if (typeinfo module_exists(llvm_tune)) { - var st <- tune_status() - if (empty(st)) { - delete st - return - } - var tuned = 0 + var inscope st <- tune_status() + rows |> reserve(length(st)) for (s in st) { - tuned += (s.source == "manifest" || s.source == "profile") ? 1 : 0 - } - to_log(LOG_INFO, "{appName}: tune status ({tuned}/{length(st)} kernels tuned for this box):\n") - for (s in st) { - to_log(LOG_INFO, " {s.fname}: {empty(s.suffix) ? "reference" : s.suffix} ({s.source})\n") - } - if (tuned < length(st)) { - to_log(LOG_INFO, "{appName}: run with --tune to tune the remaining kernels for this box\n") + rows |> push((fname = s.fname, suffix = s.suffix, source = s.source)) } - delete st } + return <- rows } diff --git a/modules/dasLLAMA/REVIEW.das b/modules/dasLLAMA/REVIEW.das index 0074d68336..47b633896e 100644 --- a/modules/dasLLAMA/REVIEW.das +++ b/modules/dasLLAMA/REVIEW.das @@ -545,7 +545,7 @@ let private IMAGE_FILE = "modules/dasLLAMA/dasllama/dasllama_image.das" // in file order. A closure change with IMAGE_VERSION unmoved is red; the finding prints the // value to re-stamp with. let private IMAGE_LAYOUT_STAMP_VERSION = 33 -let private IMAGE_LAYOUT_STAMP_HASH = 0xec0727cff52428eeul +let private IMAGE_LAYOUT_STAMP_HASH = 0x18e58f5384bf57e0ul // The helpers that decide WHERE bytes land: the page pad, the plane and total sizing, the // writer's append / zero-fill / header patch, and the header's scalar stores. Changing one diff --git a/modules/dasLLAMA/dasllama/dasllama_math_gen.das b/modules/dasLLAMA/dasllama/dasllama_math_gen.das index 61420ce10e..475a901694 100644 --- a/modules/dasLLAMA/dasllama/dasllama_math_gen.das +++ b/modules/dasLLAMA/dasllama/dasllama_math_gen.das @@ -13,7 +13,7 @@ require dasllama/dasllama_repack // the grp repack family + kq panel unpack require daslib/f16_cvt // the kq reference bodies widen the f16 (s, o) scale pairs require daslib/tune public // [tune]/[tune_perm]/[tune_companion] + [llvm_code], inert without dasLLVM; PUBLIC so the auto-policy infer pass reaches app roots that require this module require ?llvm llvm/daslib/llvm_tune public // PUBLIC so tune_status reaches app roots that require this module; witness guard: the one framework call below sits under static_if -require dasllama/dasllama_tune_scope // the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms +require dasllama/dasllama_tune_scope // nolint:STYLE029,LINT019 — an ORDER dependency, not a symbol use: the dasllama [tune_scope] must be on the AST before this module's [tune] kernels pick their perms require dasllama/dasllama_gemm_schema require daslib/jobque_boost public require dasllama/dasllama_par diff --git a/modules/dasLLVM/.das_module b/modules/dasLLVM/.das_module index d2e55ab784..a8df4f3d66 100644 --- a/modules/dasLLVM/.das_module +++ b/modules/dasLLVM/.das_module @@ -2,8 +2,21 @@ options gen2 require daslib/fio -[export] +[export, arch(at = "../../daslib/ARCHITECTURE.md#tune-shells-module-exists")] def initialize(project_path : string) { + // the `llvm` witness module (src/dasLLVM.cpp) exists only in a build configured with dasLLVM: + // registered here in a DLL build (the 2-arg form skips silently when the .shared_module was + // never built), compiled in ahead of this script in a static one. Without it nothing under + // `llvm/` resolves - a direct `require llvm/daslib/x` is then a missing prerequisite and + // `require ?llvm ...` reads "this build has no LLVM" - so the das files being on disk never + // stands in for the framework + var witness = typeinfo builtin_module_exists(llvm) + if (das_is_dll_build()) { + witness = register_dynamic_module("{project_path}/dasModuleLLVM.shared_module", "Module_LLVM") != null || witness + } + if (!witness) { + return + } let daslib_paths = [ "llvm_boost", "llvm_debug", "llvm_jit", "llvm_targets", "llvm_dsl", @@ -25,10 +38,4 @@ def initialize(project_path : string) { for (path in bindings_paths) { register_native_path("llvm", "bindings/{path}", "{project_path}/bindings/{path}.das") } - if (das_is_dll_build()) { - // the `llvm` witness module (src/dasLLVM.cpp) exists only in a build configured with - // dasLLVM; the 2-arg form skips silently when the .shared_module was never built, and - // that silence is what `require ?llvm` reads as "this build has no LLVM" - register_dynamic_module("{project_path}/dasModuleLLVM.shared_module", "Module_LLVM") - } } diff --git a/skills/tune.md b/skills/tune.md index bb991b0e0f..298c98d218 100644 --- a/skills/tune.md +++ b/skills/tune.md @@ -10,8 +10,8 @@ other tier the reference body runs verbatim. `daslib/tune` registers all six annotations - `[tune]`, `[tune_perm]`, `[tune_companion]`, `[tune_scope]`, `[tune_policy]`, `[llvm_code]` - and the default-policy pass; the framework behind them is `llvm/daslib/llvm_tune`, -which `daslib/tune` reaches through the `?llvm` witness guard (present exactly when the build is configured with dasLLVM) and re-exports. **Any program -can spell the annotations and compile whether or not dasLLVM is configured.** +which `daslib/tune` reaches through the `?llvm` witness guard (present exactly when the build is configured with dasLLVM) and re-exports; the `llvm/daslib/*` paths themselves resolve only with the witness, so a direct `require llvm/daslib/llvm_tune` marks a program as framework-only - it fails to compile without dasLLVM instead of compiling hollow. **Any program +that reaches the framework through `daslib/tune` alone can spell the annotations and compile whether or not dasLLVM is configured.** Without dasLLVM every one of them does nothing at all - no sidecar read, no stamp, no `_variants()` registry, no tuner spawn - and the reference body IS the function. From e78657d5d7139cfca355060b402ee30d1ac765d0 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 15:05:52 -0700 Subject: [PATCH 38/43] the exe lint rail retires: an exe carries the host's module paths without the modules, so its lint world was never one in-tree files answered to - CI lints the changed set once through daslang, preflight's lint gate runs the host and linux-mirror rails, the rail-skip flag and its helpers go, das-lint is still built and the bundle smoke still runs the shipped lint.exe on daslib Co-Authored-By: Claude Fable 5.1 --- .github/workflows/extended_checks.yml | 4 +- modules/dasUnitTest/unit_test.das.inc | 64 +++++------ skills/internal/babysit.md | 2 +- skills/internal/preflight.md | 7 +- tests/dastest/test_preflight_config.das | 17 --- .../cpp/class_adapters_module.das.inc | 106 +++++++++--------- utils/internal/preflight/README.md | 5 +- utils/internal/preflight/config.das | 14 --- utils/internal/preflight/main.das | 60 +--------- 9 files changed, 99 insertions(+), 180 deletions(-) diff --git a/.github/workflows/extended_checks.yml b/.github/workflows/extended_checks.yml index 006b8589ca..690fe1bff7 100644 --- a/.github/workflows/extended_checks.yml +++ b/.github/workflows/extended_checks.yml @@ -327,7 +327,8 @@ jobs: ci/nano_arm_build.sh "$BIN/daslang" - name: "Build standalone executables" - # core: the formatter and lint steps below run the das-fmt / das-lint exes this builds + # core: the formatter step below runs the das-fmt exe this builds; das-lint is built so the + # shipped lint.exe is proven to compile (the bundle smoke test runs it on daslib) if: matrix.role != 'modules' run: | set -eux @@ -403,7 +404,6 @@ jobs: # outside the PR, and >=16 files parallelize into per-file ledgers - both produce # false stales. The nightly whole-tree -j 1 pass owns staleness. $BIN/daslang ./utils/lint/main.das -- "${CHANGED[@]}" --quiet --disable LINT019 - $BIN/das-lint.exe "${CHANGED[@]}" --quiet --disable LINT019 - name: "Run ast-verify on changed .das files" if: matrix.role != 'modules' diff --git a/modules/dasUnitTest/unit_test.das.inc b/modules/dasUnitTest/unit_test.das.inc index 9a909d6317..9db5163853 100644 --- a/modules/dasUnitTest/unit_test.das.inc +++ b/modules/dasUnitTest/unit_test.das.inc @@ -1,32 +1,32 @@ -static unsigned char unit_test_das[] = { -0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, -0x67,0x65,0x6e,0x32,0x0a, -0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, -0x72,0x65,0x6d,0x6f,0x76,0x65,0x5f,0x75, -0x6e,0x75,0x73,0x65,0x64,0x5f,0x73,0x79, -0x6d,0x62,0x6f,0x6c,0x73,0x20,0x3d,0x20, -0x66,0x61,0x6c,0x73,0x65,0x0a, -0x0a, -0x72,0x65,0x71,0x75,0x69,0x72,0x65,0x20, -0x55,0x6e,0x69,0x74,0x54,0x65,0x73,0x74, -0x0a, -0x0a, -0x6c,0x65,0x74,0x20,0x4f,0x54,0x48,0x45, -0x52,0x5f,0x55,0x4e,0x49,0x54,0x5f,0x54, -0x45,0x53,0x54,0x5f,0x43,0x4f,0x4e,0x53, -0x54,0x41,0x4e,0x54,0x20,0x3d,0x20,0x55, -0x4e,0x49,0x54,0x5f,0x54,0x45,0x53,0x54, -0x5f,0x43,0x4f,0x4e,0x53,0x54,0x41,0x4e, -0x54,0x20,0x2b,0x20,0x31,0x3b,0x0a, -0x0a, -0x64,0x65,0x66,0x20,0x73,0x74,0x61,0x72, -0x74,0x5f,0x65,0x66,0x66,0x65,0x63,0x74, -0x28,0x6e,0x61,0x6d,0x65,0x3b,0x20,0x74, -0x6d,0x29,0x20,0x7b,0x0a, -0x20,0x20,0x20,0x20,0x72,0x65,0x74,0x75, -0x72,0x6e,0x20,0x73,0x74,0x61,0x72,0x74, -0x5f,0x65,0x66,0x66,0x65,0x63,0x74,0x28, -0x6e,0x61,0x6d,0x65,0x2c,0x20,0x74,0x6d, -0x2c,0x20,0x31,0x2e,0x30,0x29,0x0a, -0x7d,0x0a, -}; +static unsigned char unit_test_das[] = { +0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, +0x67,0x65,0x6e,0x32,0x0a, +0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, +0x72,0x65,0x6d,0x6f,0x76,0x65,0x5f,0x75, +0x6e,0x75,0x73,0x65,0x64,0x5f,0x73,0x79, +0x6d,0x62,0x6f,0x6c,0x73,0x20,0x3d,0x20, +0x66,0x61,0x6c,0x73,0x65,0x0a, +0x0a, +0x72,0x65,0x71,0x75,0x69,0x72,0x65,0x20, +0x55,0x6e,0x69,0x74,0x54,0x65,0x73,0x74, +0x0a, +0x0a, +0x6c,0x65,0x74,0x20,0x4f,0x54,0x48,0x45, +0x52,0x5f,0x55,0x4e,0x49,0x54,0x5f,0x54, +0x45,0x53,0x54,0x5f,0x43,0x4f,0x4e,0x53, +0x54,0x41,0x4e,0x54,0x20,0x3d,0x20,0x55, +0x4e,0x49,0x54,0x5f,0x54,0x45,0x53,0x54, +0x5f,0x43,0x4f,0x4e,0x53,0x54,0x41,0x4e, +0x54,0x20,0x2b,0x20,0x31,0x3b,0x0a, +0x0a, +0x64,0x65,0x66,0x20,0x73,0x74,0x61,0x72, +0x74,0x5f,0x65,0x66,0x66,0x65,0x63,0x74, +0x28,0x6e,0x61,0x6d,0x65,0x3b,0x20,0x74, +0x6d,0x29,0x20,0x7b,0x0a, +0x20,0x20,0x20,0x20,0x72,0x65,0x74,0x75, +0x72,0x6e,0x20,0x73,0x74,0x61,0x72,0x74, +0x5f,0x65,0x66,0x66,0x65,0x63,0x74,0x28, +0x6e,0x61,0x6d,0x65,0x2c,0x20,0x74,0x6d, +0x2c,0x20,0x31,0x2e,0x30,0x29,0x0a, +0x7d,0x0a, +}; diff --git a/skills/internal/babysit.md b/skills/internal/babysit.md index 745a97c733..f729d24175 100644 --- a/skills/internal/babysit.md +++ b/skills/internal/babysit.md @@ -56,7 +56,7 @@ matrix between rounds. 6. Repeat until Section 0 holds; **only then** wait for CI green and merge. **Every push that changes a `.das` file since the last push runs the pre-push lint gate -first**: `daslang utils/internal/preflight/main.das -- --only lint,review-md --lint-skip-exe-rail` +first**: `daslang utils/internal/preflight/main.das -- --only lint,review-md` (rails and cost: `skills/internal/preflight.md`, the lint row). MCP `lint` on the file just edited is NOT this gate: this run lints every `.das` the branch changed against `origin/master`, exactly as CI does, so a clean single-file lint still pushes red. **A push diff --git a/skills/internal/preflight.md b/skills/internal/preflight.md index d4e996d40a..6d9cffeaac 100644 --- a/skills/internal/preflight.md +++ b/skills/internal/preflight.md @@ -13,8 +13,7 @@ sequence`, `--only dasllama-model-free` when the module is the work. A gate with a **reach set** skips, with the reason, when nothing under its paths or the core (`src/`, `include/`, `daslib/`, `dastest/`, `CMakeLists.txt`, `cmake/`) changed; `--only` runs a gate whatever changed. `--list-gates` prints tier, reach -and description; `--skip ` drops gates; `--lint-skip-exe-rail` trims the -lint gate to its interp rails. A gate whose host tool or module is missing +and description; `--skip ` drops gates. A gate whose host tool or module is missing reports `SKIP` with an install/rebuild hint. The budget the tiers serve: a full run fits 20 minutes on the M5 box, or the gate is not in preflight (`plans/ci_preflight_budget.md`). @@ -172,10 +171,10 @@ cmake -B build -DDAS_HV_DISABLED=OFF -DDAS_LLVM_DISABLED=OFF -DDAS_AUDIO_DISABLE | Run examples - **nightly** | `cmake --build build --config Release --target run_examples`; per PR, preflight's `compile-sweep` gate compiles every example root | the run form is 5-8 minutes a lane | | Utils tests | `cmake --build build --config Release --target run_utils_tests` - preflight's `utils-tests` lane | | | Tutorial dry-runs - **nightly** | `cmake --build build --config Release --target dry_run_tutorials`; per PR, preflight's `compile-sweep` gate compiles every tutorial root | compile rot in `tutorials/` after daslib API changes; the run form is 8-11 minutes a lane | -| Standalone exes | `cmake --build build --config Release --target all_utils_exe`, plus ` -exe -output bin/das-fmt utils/das-fmt/dasfmt.das` and `... bin/das-lint utils/lint/main.das` | `-exe` needs dasLLVM + lld-link on PATH | +| Standalone exes | `cmake --build build --config Release --target all_utils_exe`, plus ` -exe -output bin/das-fmt utils/das-fmt/dasfmt.das` and `... bin/das-lint utils/lint/main.das` | `-exe` needs dasLLVM + lld-link on PATH; das-lint is built, not run - an exe carries the host's module paths without the modules, so its lint world is not one in-tree files answer to; the shipped `lint.exe` is exercised by the bundle smoke test on daslib | | Sequence smoke | Windows: `pwsh examples/games/sequence/ci_smoke_test.ps1 "$(pwd)"`; linux/mac: `bash examples/games/sequence/ci_smoke_test.sh "$(pwd)"` | build the runtime modules first: `cmake --build build --config Release --target dasModuleGlfw dasModuleLiveHost dasModuleHV dasModuleAudio dasModulePUGIXML dasModuleStbImage`. **The only pre-merge lane compiling GLFW-gated `.das` like dasOpenGL** - run it for type-system / daslib-generics changes | | Formatter `--verify` | preflight's `format` gate runs it exactly (tracked files via `--files-from`); manual: ` utils/das-fmt/dasfmt.das -- --path ./ --verify --exclude-mask build/` | CI's second verify pass uses an `-exe`-compiled `bin/das-fmt.exe`; the mask skips generated `.das` under the build dir (nightly doc-verify extracts RST snippets there) | -| Lint changed `.das` | preflight's `lint` gate - THREE rails: host-flavor interp, the LINUX-lane mirror (`--disable-module dasMetal` - CI's verdict comes only from linux, where dasMetal's platform `static_if` halves compile out so its requires/args read unused there and nowhere else; dasVulkan is in-tree on linux, so NOT disabled), and the `-exe`-compiled `das-lint` (different module resolution). `--lint-skip-exe-rail` drops the exe rail (per-push cadence - `skills/internal/babysit.md`). Single rail: `git diff --name-only origin/master..HEAD -- '*.das' \| xargs utils/lint/main.das -- --quiet` | zero warnings on ALL rails; a mirror-only STYLE030/LINT012 takes the both-worlds `nolint:...,LINT019` spelling | +| Lint changed `.das` | preflight's `lint` gate - TWO rails: host-flavor interp, then the LINUX-lane mirror (`--disable-module dasMetal` - CI's verdict comes only from linux, where dasMetal's platform `static_if` halves compile out so its requires/args read unused there and nowhere else; dasVulkan is in-tree on linux, so NOT disabled). Single rail: `git diff --name-only origin/master..HEAD -- '*.das' \| xargs utils/lint/main.das -- --quiet` | zero warnings on BOTH rails; a mirror-only STYLE030/LINT012 takes the both-worlds `nolint:...,LINT019` spelling | | ast-verify changed `.das` | preflight's `ast-verify` gate - ` -dry-run --ast-verify-batch ` per changed `.das` plus the `tests/linq/test_linq_fold.das` qmacro canary, parallel, 300 s per-file timeout, skipping `cant_`/`failed_`/`invalid_` and `utils/internal/ast-fuzz/selftest/`. An `AST verify` line, crash or timeout fails; a compile error belongs to whoever owns the file; a file inside the verifier's own require closure (`daslib/ast*.das`, `daslib/rtti.das`, `daslib/strings_boost.das` - `error[20510]` under the force-include) is reported *not verifiable*, never clean | mirrors the workflow's "Run ast-verify on changed .das files". Batch mode is the ruled gate form (`skills/das_macros.md`); with no pre-infer walk, a tree a macro breaks mid-inference surfaces as a compiler crash instead of a located report - hence crash = red, and plain `--ast-verify` on that file locates it. Each item is a whole-engine compile (2-3x a plain one). Width is physical cores halved; `-j` only lowers it | | REVIEW.das gates | ` utils/internal/review-md/all.das` | every `REVIEW.das` in the tree, fail-fix; also run per-diff in the make_pr step-0a walk | | dastest own suite | ` dastest/dastest.das -- --failures-only --test dastest/tests` | framework suite + `review_gate` library tests; whole-directory, so a new file needs no CI row | diff --git a/tests/dastest/test_preflight_config.das b/tests/dastest/test_preflight_config.das index e677bf3133..2ebe3815d9 100644 --- a/tests/dastest/test_preflight_config.das +++ b/tests/dastest/test_preflight_config.das @@ -1,7 +1,6 @@ options gen2 options no_aot -require strings require dastest/testing_boost require daslib/fio require ../../dastest/worker_policy.das @@ -52,22 +51,6 @@ def test_preflight_config(tt : T?) { t |> equal(0, length(blanks)) } - tt |> run("lint rail selection and its Pass detail are pinned") <| @(t : T?) { - t |> success(lint_runs_exe_rail(false)) - t |> success(!lint_runs_exe_rail(true)) - t |> equal("3 file(s) clean on all three rails (host, linux-mirror, exe)", lint_pass_detail(3, true)) - t |> equal("1 file(s) clean on interp rails (host, linux-mirror); exe rail skipped", lint_pass_detail(1, false)) - } - - tt |> run("--show-help lists the rail-skip flag the skills hardcode") <| @(t : T?) { - let exe = get_command_line_arguments()[0] - let tool = path_join(get_das_root(), "utils/internal/preflight/main.das") - var out : string - let rc = run_and_capture([exe, tool, "--", "--show-help"], out, 120.0) - t |> equal(0, rc) - t |> success(find(out, "--lint-skip-exe-rail") >= 0, "help lists --lint-skip-exe-rail") - } - tt |> run("invalid settings fail closed to defaults") <| @(t : T?) { let temp = create_temp_file_result("preflight_config_bad", ".toml") t |> success(temp is value, "temporary config created") diff --git a/tutorials/integration/cpp/class_adapters_module.das.inc b/tutorials/integration/cpp/class_adapters_module.das.inc index b79fa630fd..908c829486 100644 --- a/tutorials/integration/cpp/class_adapters_module.das.inc +++ b/tutorials/integration/cpp/class_adapters_module.das.inc @@ -1,53 +1,53 @@ -static unsigned char class_adapters_module_das[] = { -0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, -0x67,0x65,0x6e,0x32,0x0a, -0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, -0x72,0x65,0x6d,0x6f,0x76,0x65,0x5f,0x75, -0x6e,0x75,0x73,0x65,0x64,0x5f,0x73,0x79, -0x6d,0x62,0x6f,0x6c,0x73,0x20,0x3d,0x20, -0x66,0x61,0x6c,0x73,0x65,0x0a, -0x0a, -0x2f,0x2f,0x20,0x41,0x62,0x73,0x74,0x72, -0x61,0x63,0x74,0x20,0x62,0x61,0x73,0x65, -0x20,0x63,0x6c,0x61,0x73,0x73,0x20,0xe2, -0x80,0x94,0x20,0x64,0x61,0x73,0x6c,0x61, -0x6e,0x67,0x20,0x73,0x69,0x64,0x65,0x2e, -0x0a, -0x2f,0x2f,0x20,0x43,0x2b,0x2b,0x20,0x77, -0x69,0x6c,0x6c,0x20,0x69,0x6e,0x63,0x6c, -0x75,0x64,0x65,0x20,0x74,0x68,0x69,0x73, -0x20,0x76,0x69,0x61,0x20,0x63,0x6f,0x6d, -0x70,0x69,0x6c,0x65,0x42,0x75,0x69,0x6c, -0x74,0x69,0x6e,0x4d,0x6f,0x64,0x75,0x6c, -0x65,0x20,0x61,0x6e,0x64,0x20,0x74,0x68, -0x65,0x20,0x58,0x44,0x44,0x20,0x2e,0x64, -0x61,0x73,0x2e,0x69,0x6e,0x63,0x20,0x66, -0x69,0x6c,0x65,0x2e,0x0a, -0x2f,0x2f,0x20,0x64,0x61,0x73,0x6c,0x61, -0x6e,0x67,0x20,0x63,0x6c,0x61,0x73,0x73, -0x65,0x73,0x20,0x63,0x61,0x6e,0x20,0x64, -0x65,0x72,0x69,0x76,0x65,0x20,0x66,0x72, -0x6f,0x6d,0x20,0x74,0x68,0x69,0x73,0x20, -0x61,0x6e,0x64,0x20,0x6f,0x76,0x65,0x72, -0x72,0x69,0x64,0x65,0x20,0x74,0x68,0x65, -0x20,0x76,0x69,0x72,0x74,0x75,0x61,0x6c, -0x20,0x6d,0x65,0x74,0x68,0x6f,0x64,0x73, -0x2e,0x0a, -0x0a, -0x63,0x6c,0x61,0x73,0x73,0x20,0x54,0x75, -0x74,0x6f,0x72,0x69,0x61,0x6c,0x42,0x61, -0x73,0x65,0x43,0x6c,0x61,0x73,0x73,0x20, -0x7b,0x0a, -0x20,0x20,0x20,0x20,0x64,0x65,0x66,0x20, -0x61,0x62,0x73,0x74,0x72,0x61,0x63,0x74, -0x20,0x75,0x70,0x64,0x61,0x74,0x65,0x28, -0x64,0x74,0x20,0x3a,0x20,0x66,0x6c,0x6f, -0x61,0x74,0x29,0x20,0x3a,0x20,0x76,0x6f, -0x69,0x64,0x0a, -0x20,0x20,0x20,0x20,0x64,0x65,0x66,0x20, -0x61,0x62,0x73,0x74,0x72,0x61,0x63,0x74, -0x20,0x67,0x65,0x74,0x5f,0x70,0x6f,0x73, -0x69,0x74,0x69,0x6f,0x6e,0x20,0x3a,0x20, -0x66,0x6c,0x6f,0x61,0x74,0x33,0x0a, -0x7d,0x0a, -}; +static unsigned char class_adapters_module_das[] = { +0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, +0x67,0x65,0x6e,0x32,0x0a, +0x6f,0x70,0x74,0x69,0x6f,0x6e,0x73,0x20, +0x72,0x65,0x6d,0x6f,0x76,0x65,0x5f,0x75, +0x6e,0x75,0x73,0x65,0x64,0x5f,0x73,0x79, +0x6d,0x62,0x6f,0x6c,0x73,0x20,0x3d,0x20, +0x66,0x61,0x6c,0x73,0x65,0x0a, +0x0a, +0x2f,0x2f,0x20,0x41,0x62,0x73,0x74,0x72, +0x61,0x63,0x74,0x20,0x62,0x61,0x73,0x65, +0x20,0x63,0x6c,0x61,0x73,0x73,0x20,0xe2, +0x80,0x94,0x20,0x64,0x61,0x73,0x6c,0x61, +0x6e,0x67,0x20,0x73,0x69,0x64,0x65,0x2e, +0x0a, +0x2f,0x2f,0x20,0x43,0x2b,0x2b,0x20,0x77, +0x69,0x6c,0x6c,0x20,0x69,0x6e,0x63,0x6c, +0x75,0x64,0x65,0x20,0x74,0x68,0x69,0x73, +0x20,0x76,0x69,0x61,0x20,0x63,0x6f,0x6d, +0x70,0x69,0x6c,0x65,0x42,0x75,0x69,0x6c, +0x74,0x69,0x6e,0x4d,0x6f,0x64,0x75,0x6c, +0x65,0x20,0x61,0x6e,0x64,0x20,0x74,0x68, +0x65,0x20,0x58,0x44,0x44,0x20,0x2e,0x64, +0x61,0x73,0x2e,0x69,0x6e,0x63,0x20,0x66, +0x69,0x6c,0x65,0x2e,0x0a, +0x2f,0x2f,0x20,0x64,0x61,0x73,0x6c,0x61, +0x6e,0x67,0x20,0x63,0x6c,0x61,0x73,0x73, +0x65,0x73,0x20,0x63,0x61,0x6e,0x20,0x64, +0x65,0x72,0x69,0x76,0x65,0x20,0x66,0x72, +0x6f,0x6d,0x20,0x74,0x68,0x69,0x73,0x20, +0x61,0x6e,0x64,0x20,0x6f,0x76,0x65,0x72, +0x72,0x69,0x64,0x65,0x20,0x74,0x68,0x65, +0x20,0x76,0x69,0x72,0x74,0x75,0x61,0x6c, +0x20,0x6d,0x65,0x74,0x68,0x6f,0x64,0x73, +0x2e,0x0a, +0x0a, +0x63,0x6c,0x61,0x73,0x73,0x20,0x54,0x75, +0x74,0x6f,0x72,0x69,0x61,0x6c,0x42,0x61, +0x73,0x65,0x43,0x6c,0x61,0x73,0x73,0x20, +0x7b,0x0a, +0x20,0x20,0x20,0x20,0x64,0x65,0x66,0x20, +0x61,0x62,0x73,0x74,0x72,0x61,0x63,0x74, +0x20,0x75,0x70,0x64,0x61,0x74,0x65,0x28, +0x64,0x74,0x20,0x3a,0x20,0x66,0x6c,0x6f, +0x61,0x74,0x29,0x20,0x3a,0x20,0x76,0x6f, +0x69,0x64,0x0a, +0x20,0x20,0x20,0x20,0x64,0x65,0x66,0x20, +0x61,0x62,0x73,0x74,0x72,0x61,0x63,0x74, +0x20,0x67,0x65,0x74,0x5f,0x70,0x6f,0x73, +0x69,0x74,0x69,0x6f,0x6e,0x20,0x3a,0x20, +0x66,0x6c,0x6f,0x61,0x74,0x33,0x0a, +0x7d,0x0a, +}; diff --git a/utils/internal/preflight/README.md b/utils/internal/preflight/README.md index 59604852dc..a6e1eec6db 100644 --- a/utils/internal/preflight/README.md +++ b/utils/internal/preflight/README.md @@ -27,9 +27,8 @@ daslang utils/internal/preflight/main.das -- --list-gates daslang utils/internal/preflight/main.das -- --only docs,ci-das daslang utils/internal/preflight/main.das -- --skip tests-aot --full -# pre-push lint gate: the changed-set lint without the -exe rail (rails and -# cost: skills/internal/preflight.md, the lint row) -daslang utils/internal/preflight/main.das -- --only lint --lint-skip-exe-rail +# pre-push lint gate: the changed-set lint (rails and cost: skills/internal/preflight.md, the lint row) +daslang utils/internal/preflight/main.das -- --only lint ``` A gate with a reach set (`dasgen`, `docs`, `utils-tests`, `ci-matrix`, the module gates) skips diff --git a/utils/internal/preflight/config.das b/utils/internal/preflight/config.das index 170991d6ed..e7bc42dbd0 100644 --- a/utils/internal/preflight/config.das +++ b/utils/internal/preflight/config.das @@ -184,20 +184,6 @@ def public valid_jit_max_file_time_override(value : float) : bool { return value == -1.0 || value >= 0.0 } -//! The pure half of the lint gate's rail selection: `--lint-skip-exe-rail` -//! keeps the host + linux-mirror interp rails and drops the -exe compiled one. -def public lint_runs_exe_rail(skip_exe_rail : bool) : bool { - return !skip_exe_rail -} - -//! The lint gate's Pass detail — the only observable that distinguishes a -//! full three-rail pass from a rail-skipping one (both return GateStatus.Pass). -def public lint_pass_detail(n_files : int; ran_exe_rail : bool) : string { - return (ran_exe_rail - ? "{n_files} file(s) clean on all three rails (host, linux-mirror, exe)" - : "{n_files} file(s) clean on interp rails (host, linux-mirror); exe rail skipped") -} - //! Resolve the read-only per-user config. ``DAS_PREFLIGHT_CONFIG`` overrides //! the platform default for hermetic tests and unusual installations. def public default_preflight_config_path() : string { diff --git a/utils/internal/preflight/main.das b/utils/internal/preflight/main.das index e553bf0b9b..564a3ab8d9 100644 --- a/utils/internal/preflight/main.das +++ b/utils/internal/preflight/main.das @@ -74,9 +74,6 @@ struct Config { @clarg_doc = "Stop at the first failing gate" fail_fast : bool - @clarg_doc = "Lint gate: skip the -exe compiled rail, keeping the host + linux-mirror interp rails — run before every push; PR-open preflight runs all three" - lint_skip_exe_rail : bool - @clarg_doc = "List gates with tier and description, then exit" list_gates : bool @@ -121,7 +118,6 @@ struct PreflightCtx { jit_jobs : int jit_max_file_time : float verbose : bool - lint_skip_exe_rail : bool } // ===== subprocess + small helpers ===== @@ -140,17 +136,6 @@ def run_argv(args : array) : tuple { return run_argv(args, 0.0) } -// A dynamically-linked artifact spawned outside the daslang bin dir (the exe rail's -// temp-path das-lint) resolves libDaScriptDyn only through the loader's environment — -// prepend the bin dir so spawned children inherit it. -def prepend_loader_search_path(bin_dir : string) { - let platform = get_platform_name() - let vname = platform == "windows" ? "PATH" : (platform == "darwin" ? "DYLD_LIBRARY_PATH" : "LD_LIBRARY_PATH") - let sep = platform == "windows" ? ";" : ":" - let cur = get_env_variable(vname) - set_env_variable(vname, empty(cur) ? bin_dir : "{bin_dir}{sep}{cur}") -} - def tool_available(exe, probe_flag : string) : bool { let r = run_argv([exe, probe_flag], 30.0) return r.rc == 0 @@ -583,11 +568,9 @@ def gate_format(ctx : PreflightCtx) : GateResult { // CI's lint verdict is rendered ONLY by the linux lane (extended_checks gates the step on // matrix.target == 'linux'), in the linux MODULE WORLD (no dasMetal — platform static_if // halves compile out, so their requires/args read unused there and nowhere else; dasVulkan -// is in-tree and present on linux, so it is NOT disabled), -// and TWICE (interp, then the -exe compiled das-lint, which resolves modules differently). -// A host-flavor pass alone is structurally blind to both — so this gate runs three rails -// (--lint-skip-exe-rail keeps just the interp two). -// LINT019 is OFF on all three (mirrors CI): a changed-set ledger cannot see consumers +// is in-tree and present on linux, so it is NOT disabled). A host-flavor pass alone is +// structurally blind to that — so this gate runs two rails, host then linux-mirror. +// LINT019 is OFF on both (mirrors CI): a changed-set ledger cannot see consumers // outside the PR — staleness belongs to the nightly whole-tree -j 1 pass. def gate_lint(ctx : PreflightCtx) : GateResult { let t0 = ref_time_ticks() @@ -617,38 +600,8 @@ def gate_lint(ctx : PreflightCtx) : GateResult { return GateResult(name = "lint", status = GateStatus.Fail, seconds = seconds_since(t0), detail = "LINUX-lane lint warnings (host pass was clean — a platform static_if half hides the symbol here; nolint:...,LINT019 both-worlds spelling, see skills/internal/preflight.md)", output = mirror.out) } - if (!lint_runs_exe_rail(ctx.lint_skip_exe_rail)) { - return GateResult(name = "lint", status = GateStatus.Pass, seconds = seconds_since(t0), - detail = lint_pass_detail(length(files), false)) - } - // the exe rail: CI compiles utils/lint into a standalone das-lint and lints AGAIN — the - // -exe world resolves modules differently (dasAudio et al are not baked in). `-exe` - // appends .exe on every platform (CI's LINUX lane invokes $BIN/das-lint.exe verbatim) - let lint_exe_base = unique_temp_path("preflight_das_lint", "") - let build = run_argv([ctx.daslang, "-exe", "-output", lint_exe_base, "utils/lint/main.das"]) - if (build.rc != 0) { - remove("{lint_exe_base}.exe") - remove("{lint_exe_base}.o") - return GateResult(name = "lint", status = GateStatus.Fail, seconds = seconds_since(t0), - detail = "das-lint -exe build failed — the CI exe rail would fail the same way", output = build.out) - } - // the -exe artifact is dynamically linked (libDaScriptDyn); from the temp path the - // loader resolves it only via the environment — point it at the daslang bin dir - prepend_loader_search_path(dir_name(ctx.daslang)) - var exe_args <- ["{lint_exe_base}.exe"] - exe_args |> push_from(files) - exe_args |> push("--quiet") - exe_args |> push("--disable") - exe_args |> push("LINT019") - let exe = run_argv(exe_args) - remove("{lint_exe_base}.exe") - remove("{lint_exe_base}.o") - if (exe.rc != 0) { - return GateResult(name = "lint", status = GateStatus.Fail, seconds = seconds_since(t0), - detail = "das-lint.exe rail warnings (interp rails were clean — the -exe module world differs)", output = exe.out) - } return GateResult(name = "lint", status = GateStatus.Pass, seconds = seconds_since(t0), - detail = lint_pass_detail(length(files), true)) + detail = "{length(files)} file(s) clean on both rails (host, linux-mirror)") } // ===== ast-verify on changed .das (mirrors extended_checks "Run ast-verify on changed .das files") ===== @@ -1606,7 +1559,7 @@ def gate_table() : array { return <- [ GateInfo(name = "untracked", tier = "fast", doc = "no untracked files at PR time — commit, delete, or ignore each"), GateInfo(name = "format", tier = "fast", doc = "formatter --verify on tracked .das (mirrors CI)"), - GateInfo(name = "lint", tier = "fast", doc = "lint changed .das on three rails (host, linux-mirror, -exe), zero warnings; --lint-skip-exe-rail drops the exe rail"), + GateInfo(name = "lint", tier = "fast", doc = "lint changed .das on two rails (host, linux-mirror), zero warnings"), GateInfo(name = "hash-refs", tier = "fast", doc = "no bare #N in branch commit messages that GitHub would mislink - ledger cites spell out or backtick"), GateInfo(name = "review-md", tier = "fast", doc = "REVIEW.das gates of every folder the diff touches (utils/internal/review-md; CI's extended_checks step runs every gate)"), GateInfo(name = "md-ascii", tier = "fast", doc = "ci/fix_md_ascii.py --check when the diff touches .md (mirrors CI's Markdown ASCII gate in extended_checks)"), @@ -1779,8 +1732,7 @@ def main() : int { // nolint:STYLE037,STYLE038 — the gate loop + CLI surface; var ctx = PreflightCtx(base = cfg.base, jobs = cfg.jobs, jit_jobs = jit_jobs, jit_max_file_time = jit_max_file_time, - verbose = cfg.verbose, - lint_skip_exe_rail = cfg.lint_skip_exe_rail) + verbose = cfg.verbose) ctx.daslang = find_daslang(cfg.daslang_bin) if (empty(ctx.daslang)) { to_log(LOG_ERROR, "daslang binary not found — build it (cmake --build build --target daslang) or pass --daslang / set DASLANG\n") From 1ccf6cc8bab56d352053658872845a2befa00be0 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 15:43:14 -0700 Subject: [PATCH 39/43] get_target_architecture_name's reference text lists what the function returns: arm64 for both arm triples, never aarch64 Co-Authored-By: Claude Fable 5.1 --- ...-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst b/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst index 2129d13caa..e051e987f2 100644 --- a/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst +++ b/doc/source/stdlib/handmade/function-builtin-get_target_architecture_name-0x5a46cb8acf2fed30.rst @@ -1 +1 @@ -Returns the architecture component of the compile target, such as `"wasm64"`, `"wasm32"`, `"aarch64"` or `"x86_64"`; the host's own architecture when no `--jit-target` names another. Folded to a constant like `get_target_triple`, it is the switch a kernel uses to pick a per-target body. +Returns the architecture component of the compile target: `"wasm64"`, `"wasm32"`, `"x86_64"`, `"arm64"` (for both the `aarch64-` and `arm64-` triples), `"x86"`, `"arm"`, or `"unknown"`; the host's own architecture when no `--jit-target` names another. Folded to a constant like `get_target_triple`, it is the switch a kernel uses to pick a per-target body. From 588aa9c72ce045b9e5c9b2244f949859b137acca Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 16:02:48 -0700 Subject: [PATCH 40/43] the two remaining daslib-side doors to llvm/ spell the witness guard: das2rst and just_in_time compile in a build without dasLLVM - the docs lane's daslang carries none, and the interpreter sweeps reach the JIT tests through just_in_time Co-Authored-By: Claude Fable 5.1 --- daslib/ARCHITECTURE.md | 2 ++ daslib/just_in_time.das | 9 +++------ doc/reflections/das2rst.das | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index 43e0ba77be..d253383d96 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -243,3 +243,5 @@ Three companions carry a concern each; a section number is unique across all fou module, which is how a tool that reads a `_variants()` registry through a direct require once compiled in a witness-less world with inert shells and no registry; a world that cannot mount the framework now refuses the require, and a lint skips the file as one it cannot compile. + The daslib doors to `llvm/` - `tune` and `just_in_time` - and the documentation generator + spell the guard, so a program reaching LLVM only through them compiles in every build. diff --git a/daslib/just_in_time.das b/daslib/just_in_time.das index 2750391988..cdbda5d9cd 100644 --- a/daslib/just_in_time.das +++ b/daslib/just_in_time.das @@ -6,10 +6,7 @@ module just_in_time shared private //! JIT compilation configuration. //! //! Bootstraps the Just-In-Time compiler by requiring the platform-appropriate -//! LLVM-based JIT backend module. +//! LLVM-based JIT backend module - through the `llvm` witness guard, so a program +//! requiring this module still compiles, and runs interpreted, in a build without dasLLVM. -/* - this is a JIT configuration file. We require platform appropriate jit module, as well as set necessary options. -*/ - -require llvm/daslib/llvm_macro +require ?llvm llvm/daslib/llvm_macro // nolint:STYLE030,LINT019 — a side-effect require: the module's presence is what arms the JIT diff --git a/doc/reflections/das2rst.das b/doc/reflections/das2rst.das index d8625f9485..551f26c5ef 100644 --- a/doc/reflections/das2rst.das +++ b/doc/reflections/das2rst.das @@ -145,7 +145,7 @@ require strudel/strudel_sf2_voice require strudel/strudel_pattern require spirv/spirv_reflect require dasllama/dasllama -require llvm/daslib/llvm_tune // documentation context keeps [tune] fully inert +require ?llvm llvm/daslib/llvm_tune // nolint:STYLE030,LINT019 — the framework's is_building_documentation check keeps [tune] inert; a build without dasLLVM has no framework to keep inert def document_module_math(_root : string) { var mod = get_module("math") From ba6f3f2f51b64bf50579dd850216136bc4fe438d Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 16:46:25 -0700 Subject: [PATCH 41/43] the descriptor path gate is withdrawn and the JIT emitter pin re-stamped: the llvm/daslib files resolve wherever modules/dasLLVM sits and only the witness decides the guards, because a static host that never registers the witness still runs the JIT through the LLVM library (the C API JIT test), so just_in_time keeps its direct require; LLVM_JIT_EMITTER_HASH follows the idot and feature-string edits Co-Authored-By: Claude Fable 5.1 --- daslib/ARCHITECTURE.md | 15 +++++++-------- daslib/just_in_time.das | 9 ++++++--- modules/dasLLVM/.das_module | 21 +++++++-------------- modules/dasLLVM/daslib/llvm_jit_run.das | 2 +- skills/tune.md | 2 +- 5 files changed, 22 insertions(+), 27 deletions(-) diff --git a/daslib/ARCHITECTURE.md b/daslib/ARCHITECTURE.md index d253383d96..9dba5c40eb 100644 --- a/daslib/ARCHITECTURE.md +++ b/daslib/ARCHITECTURE.md @@ -237,11 +237,10 @@ Three companions carry a concern each; a section number is unique across all fou half on fixtures, and `tests/daslib/test_tune_shells.das` is tier-agnostic by design - so the pair is recorded here: a trait that answered off the process registry instead would take the framework arm inside a tool's nested compile, where the framework is not mounted. -- **`modules/dasLLVM/.das_module` registers the `llvm/daslib/*` paths only after the witness - registered (or was compiled in), so without dasLLVM a direct `require llvm/daslib/x` is a missing - prerequisite and never a hollow compile.** The framework's das files compile without their C++ - module, which is how a tool that reads a `_variants()` registry through a direct require - once compiled in a witness-less world with inert shells and no registry; a world that cannot - mount the framework now refuses the require, and a lint skips the file as one it cannot compile. - The daslib doors to `llvm/` - `tune` and `just_in_time` - and the documentation generator - spell the guard, so a program reaching LLVM only through them compiles in every build. +- **The `llvm/daslib/*` files resolve in every build that carries `modules/dasLLVM/`; only the + witness decides what `?llvm` and `module_exists(llvm_tune)` answer.** The framework's das files + compile without their C++ module, so a direct `require llvm/daslib/llvm_tune` in a build without + dasLLVM compiles with the shells inert and no `_variants()` registry - a program that + reads one is framework-only and says so with that direct require. `daslib/just_in_time` keeps + its direct require for the opposite reason: a static host that never registered the witness + still runs the JIT through the LLVM library, and the guard would switch it off. diff --git a/daslib/just_in_time.das b/daslib/just_in_time.das index cdbda5d9cd..2750391988 100644 --- a/daslib/just_in_time.das +++ b/daslib/just_in_time.das @@ -6,7 +6,10 @@ module just_in_time shared private //! JIT compilation configuration. //! //! Bootstraps the Just-In-Time compiler by requiring the platform-appropriate -//! LLVM-based JIT backend module - through the `llvm` witness guard, so a program -//! requiring this module still compiles, and runs interpreted, in a build without dasLLVM. +//! LLVM-based JIT backend module. -require ?llvm llvm/daslib/llvm_macro // nolint:STYLE030,LINT019 — a side-effect require: the module's presence is what arms the JIT +/* + this is a JIT configuration file. We require platform appropriate jit module, as well as set necessary options. +*/ + +require llvm/daslib/llvm_macro diff --git a/modules/dasLLVM/.das_module b/modules/dasLLVM/.das_module index a8df4f3d66..d2e55ab784 100644 --- a/modules/dasLLVM/.das_module +++ b/modules/dasLLVM/.das_module @@ -2,21 +2,8 @@ options gen2 require daslib/fio -[export, arch(at = "../../daslib/ARCHITECTURE.md#tune-shells-module-exists")] +[export] def initialize(project_path : string) { - // the `llvm` witness module (src/dasLLVM.cpp) exists only in a build configured with dasLLVM: - // registered here in a DLL build (the 2-arg form skips silently when the .shared_module was - // never built), compiled in ahead of this script in a static one. Without it nothing under - // `llvm/` resolves - a direct `require llvm/daslib/x` is then a missing prerequisite and - // `require ?llvm ...` reads "this build has no LLVM" - so the das files being on disk never - // stands in for the framework - var witness = typeinfo builtin_module_exists(llvm) - if (das_is_dll_build()) { - witness = register_dynamic_module("{project_path}/dasModuleLLVM.shared_module", "Module_LLVM") != null || witness - } - if (!witness) { - return - } let daslib_paths = [ "llvm_boost", "llvm_debug", "llvm_jit", "llvm_targets", "llvm_dsl", @@ -38,4 +25,10 @@ def initialize(project_path : string) { for (path in bindings_paths) { register_native_path("llvm", "bindings/{path}", "{project_path}/bindings/{path}.das") } + if (das_is_dll_build()) { + // the `llvm` witness module (src/dasLLVM.cpp) exists only in a build configured with + // dasLLVM; the 2-arg form skips silently when the .shared_module was never built, and + // that silence is what `require ?llvm` reads as "this build has no LLVM" + register_dynamic_module("{project_path}/dasModuleLLVM.shared_module", "Module_LLVM") + } } diff --git a/modules/dasLLVM/daslib/llvm_jit_run.das b/modules/dasLLVM/daslib/llvm_jit_run.das index 29d37215ad..8c0f1b5f2c 100644 --- a/modules/dasLLVM/daslib/llvm_jit_run.das +++ b/modules/dasLLVM/daslib/llvm_jit_run.das @@ -41,7 +41,7 @@ let LLVM_JIT_CODEGEN_VERSION : uint64 = 0x70ul // the wasm feature string drop // Read by tests-cpp/small/test_jit_emitter_pin.cpp: FNV-1a64 of the emitter sources // (normalized to LF; file list in the test) -let LLVM_JIT_EMITTER_HASH : uint64 = 0x6330d3756a092eebul +let LLVM_JIT_EMITTER_HASH : uint64 = 0x6edb28d5b95231c2ul let JIT_FNV_PRIME : uint64 = 1099511628211ul diff --git a/skills/tune.md b/skills/tune.md index 298c98d218..30d05886ba 100644 --- a/skills/tune.md +++ b/skills/tune.md @@ -10,7 +10,7 @@ other tier the reference body runs verbatim. `daslib/tune` registers all six annotations - `[tune]`, `[tune_perm]`, `[tune_companion]`, `[tune_scope]`, `[tune_policy]`, `[llvm_code]` - and the default-policy pass; the framework behind them is `llvm/daslib/llvm_tune`, -which `daslib/tune` reaches through the `?llvm` witness guard (present exactly when the build is configured with dasLLVM) and re-exports; the `llvm/daslib/*` paths themselves resolve only with the witness, so a direct `require llvm/daslib/llvm_tune` marks a program as framework-only - it fails to compile without dasLLVM instead of compiling hollow. **Any program +which `daslib/tune` reaches through the `?llvm` witness guard (present exactly when the build is configured with dasLLVM) and re-exports; a direct `require llvm/daslib/llvm_tune` marks a program as framework-only - without dasLLVM its files still compile, but the shells are inert and no `_variants()` registry exists. **Any program that reaches the framework through `daslib/tune` alone can spell the annotations and compile whether or not dasLLVM is configured.** Without dasLLVM every one of them does nothing at all - no sidecar read, no stamp, no `_variants()` registry, no tuner spawn - and the reference From 01ca6a291017828164c512091f1cb498239720a9 Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 20:07:14 -0700 Subject: [PATCH 42/43] the daspkg name guard is an allowlist: letters, digits, underscore, dash and dot, never dot or dot-dot - the strings it admits are spliced unquoted into run_cmd lines, so a space split argv and a semicolon ran; the three refusals say what the gate admits, and four unsafe bundle names are refused red-first before anything builds Co-Authored-By: Claude Fable 5.1 --- utils/daspkg/commands.das | 14 ++++++++++---- utils/daspkg/test_daspkg.das | 28 ++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/utils/daspkg/commands.das b/utils/daspkg/commands.das index f28c9e74c7..544dee14f7 100644 --- a/utils/daspkg/commands.das +++ b/utils/daspkg/commands.das @@ -1299,9 +1299,15 @@ def cmd_check(root : string; json : bool = false; is_global : bool = false) { return 0 } +//! The one gate a `.das_package` string passes before it reaches a path or a shell command line: +//! letters, digits, `_`, `-` and `.`, and not `.` or `..` - an allowlist, because the strings +//! are spliced unquoted into `run_cmd` lines, where a space splits argv and `;` or `$(` run. def private is_safe_pkg_name(name : string) : bool { if (empty(name) || name == "." || name == "..") return false - return !(find(name, "/") >= 0 || find(name, "\\") >= 0) + for (c in name) { + if (!(is_alpha(c) || is_number(c) || c == '_' || c == '-' || c == '.')) return false + } + return true } def cmd_cleanup(root : string; force : bool; is_global : bool = false) : int { // nolint:STYLE038 — flat removal checklist (modules/, lock, caches), one arm per artifact @@ -2150,7 +2156,7 @@ def cmd_release(root : string; out_dir : string; paranoid = false; quick = false // separators / `..` so a hostile or typo'd name can't escape out_dir // when path_join + force_rmdir run below. if (!is_safe_pkg_name(bundle_name)) { - to_log(LOG_ERROR, "Error: unsafe bundle name `{bundle_name}` (must not contain path separators or be `.` / `..`)\n") + to_log(LOG_ERROR, "Error: unsafe bundle name `{bundle_name}` (letters, digits, `_`, `-` and `.` only; not `.` or `..`)\n") return 1 } // macOS: emit a clickable .app bundle. The exe + dylibs + modules go into @@ -2667,12 +2673,12 @@ def private release_one_wasm_app(root, out_dir, app_name, main_script : string; wasm_lib_dir, runtime_archive, shell_path : string; include_globs, exclude_globs, disabled_modules, app_emcc_args : array) : int { if (!is_safe_pkg_name(app_name)) { - to_log(LOG_ERROR, "release wasm: unsafe app name `{app_name}` (no path separators / `.` / `..`)\n") + to_log(LOG_ERROR, "release wasm: unsafe app name `{app_name}` (letters, digits, `_`, `-` and `.` only; not `.` or `..`)\n") return 1 } for (m in disabled_modules) { // a .das_package string that reaches the cross-compile command line if (!is_safe_pkg_name(m)) { - to_log(LOG_ERROR, "release wasm: unsafe module name `{m}` in release_wasm_disable_module (no path separators / `.` / `..` / spaces)\n") + to_log(LOG_ERROR, "release wasm: unsafe module name `{m}` in release_wasm_disable_module (letters, digits, `_`, `-` and `.` only; not `.` or `..`)\n") return 1 } } diff --git a/utils/daspkg/test_daspkg.das b/utils/daspkg/test_daspkg.das index 92cb69306d..8f717e1144 100644 --- a/utils/daspkg/test_daspkg.das +++ b/utils/daspkg/test_daspkg.das @@ -1871,6 +1871,34 @@ def test_cmd_release_pure_daslang(t : T?) { } } +[test] +def test_cmd_release_refuses_unsafe_bundle_name(t : T?) { + // the bundle name is a .das_package string that reaches path_join, force_rmdir and shell + // command lines; the guard admits letters, digits, `_`, `-` and `.` only - a space or a + // shell metacharacter is refused before anything is built or removed + for (bad in ["bad name", "bad;name", "bad$(name)", "bad\"name"]) { + t |> run("refuses bundle name `{bad}`") @(tt : T?) { + let tmp_root = "{get_das_root()}/_test_release_badname" + let out_dir = "{get_das_root()}/_test_release_badname_out" + force_rmdir(tmp_root) + force_rmdir(out_dir) + mkdir(tmp_root) + fopen("{tmp_root}/main.das", "wb") $(f) { + fwrite(f, "options gen2\n[export]\ndef main() \{\n print(\"hello\\n\")\n\}\n") + } + let escaped = replace(replace(bad, "\\", "\\\\"), "\"", "\\\"") + fopen("{tmp_root}/.das_package", "wb") $(f) { + fwrite(f, "options gen2\nrequire daslib/daspkg\n[export]\ndef package() \{\n package_name(\"{escaped}\")\n\}\n[export]\ndef release() \{\n release_main(\"main.das\")\n\}\n") + } + let rc = cmd_release(tmp_root, out_dir) + tt |> equal(1, rc, "cmd_release refuses the name") + tt |> success(!fexist(out_dir), "nothing was built or removed under {out_dir}") + force_rmdir(tmp_root) + force_rmdir(out_dir) + } + } +} + [test] def test_cmd_release_ships_minted_scope_manifest(t : T?) { t |> run("a tune scope's RESOLVED manifest ships beside the exe") @(tt : T?) { From b1baf4b34e6fa65ab6ade69dfbdac9560736e3ab Mon Sep 17 00:00:00 2001 From: Boris Batkin Date: Sat, 5 Sep 2026 20:26:29 -0700 Subject: [PATCH 43/43] the preview server decides the /examples/ sandbox by containment, not by pattern: the resolved real path must sit under the examples root, so a doubled slash, a dot-dot or a Windows drive-qualified remainder falls through to the site tree instead of leaving it Co-Authored-By: Claude Fable 5.1 --- site-dasllama/serve.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/site-dasllama/serve.py b/site-dasllama/serve.py index e9ca151887..b1ddfad89c 100644 --- a/site-dasllama/serve.py +++ b/site-dasllama/serve.py @@ -25,7 +25,7 @@ HERE = os.path.dirname(os.path.abspath(__file__)) SITE_FILES = os.path.normpath(os.path.join(HERE, "..", "site", "files")) -EXAMPLES_OUT = os.path.normpath(os.path.join(HERE, "..", "web", "output64", "examples")) +EXAMPLES_OUT = os.path.realpath(os.path.join(HERE, "..", "web", "output64", "examples")) SHARED = {"forge.css", "nav-dropdown.css", "dasllama-table.css", "github-star.js"} PORT = 8932 LADDER = "http://127.0.0.1:8201" @@ -78,10 +78,13 @@ def translate_path(self, path): if name in SHARED: return os.path.join(SITE_FILES, name) if clean.startswith("/examples/"): - rel = os.path.normpath(clean[len("/examples/"):]) - # an absolute remainder (a doubled slash) would make os.path.join drop the base - if rel and not rel.startswith("..") and not os.path.isabs(rel): - return os.path.join(EXAMPLES_OUT, rel) + rel = clean[len("/examples/"):] + # containment, not pattern checks: `..`, an absolute remainder (a doubled slash) and a + # drive-qualified one on Windows (`C:foo`) all resolve outside EXAMPLES_OUT and fall + # through to the site tree, where no such path exists + full = os.path.realpath(os.path.join(EXAMPLES_OUT, rel)) + if rel and full.startswith(EXAMPLES_OUT + os.sep): + return full return super().translate_path(path)