Feat/edge0 strata

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -23,6 +23,9 @@ body:
         - Android (CPU)
         - Windows (CPU)
         - Windows (Vulkan)
+        - Linux (CPU)
+        - Linux (NVIDIA GPU, edge0-strata)
+        - Windows (NVIDIA GPU, edge0-strata)
         - Other (describe below)
     validations:
       required: true
@@ -33,6 +36,8 @@ body:
       label: Model tier
       options:
         - edge0-35b
+        - edge0-145b
+        - edge0-120b
         - other / custom checkpoint
     validations:
       required: true
@@ -59,7 +64,7 @@ body:
     id: os
     attributes:
       label: OS version
-      placeholder: macOS 15.1 / Windows 11 23H2 / Android 15 / iOS 18.1
+      placeholder: macOS 15.1 / Windows 11 23H2 / Ubuntu 24.04 / Android 15 / iOS 18.1
     validations:
       required: true
 
@@ -68,7 +73,7 @@ body:
     attributes:
       label: Engine build
       description: "The engine.version and engine.source_commit lines printed by: edge0-infer info"
-      placeholder: "engine.version: 0.1.0, engine.source_commit: ..."
+      placeholder: "engine.version: 0.2.0, engine.source_commit: ..."
     validations:
       required: false
 
@@ -78,7 +83,7 @@ body:
       label: How are you running it?
       options:
         - edge0-infer CLI (run / chat / bench)
-        - edge0-infer serve (OpenAI-compatible server)
+        - edge0-infer serve (local OpenAI- and Anthropic-compatible server)
         - Native C++ API in my own application
         - Chat app example (desktop / Android / iOS)
         - Other (describe below)

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -15,7 +15,7 @@ body:
       options:
         - New backend (GPU / NPU / other accelerator)
         - New platform support
-        - Server / OpenAI-compatible API
+        - Server / OpenAI- and Anthropic-compatible API
         - CLI / native C++ API
         - Quantization / adapters (LoRA / prerouter)
         - Documentation

.github/ISSUE_TEMPLATE/performance_report.yml

@@ -7,7 +7,9 @@ body:
       value: |
         Use this for "slower than the README numbers" or "high memory / OOM" reports
         (e.g. #17, #110). The reference figures and the devices they were measured on are in
-        [Benchmarking](https://github.com/Edge0-AI/Edge0/blob/main/edge0-infer/docs/benchmarking.md#reference-figures).
+        [Benchmarking](https://github.com/Edge0-AI/Edge0/blob/main/edge0-infer/docs/benchmarking.md#reference-figures)
+        (`edge0-35b`) and on the [model page](https://github.com/Edge0-AI/Edge0/blob/main/docs/models/edge0-145b.md)
+        (`edge0-145b`, `edge0-120b`; include the GPU and its memory).
         Throughput depends on memory and storage speed — please include your total RAM and storage type.
 
   - type: dropdown
@@ -16,6 +18,8 @@ body:
       label: Model tier
       options:
         - edge0-35b
+        - edge0-145b
+        - edge0-120b
     validations:
       required: true
 

.github/workflows/ci.yml

@@ -29,3 +29,5 @@ jobs:
         run: cmake --build --preset cpu
       - name: Test (L0, L1)
         run: ctest --preset cpu --no-tests=error --output-on-failure -L "L0|L1"
+      - name: Test (L2, delegated engine)
+        run: ctest --preset cpu --no-tests=error --output-on-failure -L L2 -R strata

CONTRIBUTING.md

@@ -20,7 +20,8 @@ platform/backend work.
 
 | Path | What |
 |---|---|
-| [`edge0-infer/`](edge0-infer/) | The inference engine (C++, GGML), its CLI and OpenAI-compatible server, the model asset tools, and the chat app examples for desktop, Android and iOS |
+| [`edge0-infer/`](edge0-infer/) | The program you run: the native inference engine (C++, GGML), its CLI and local OpenAI- and Anthropic-compatible server, the model asset tools, and the chat app examples for desktop, Android and iOS |
+| [`edge0-strata/`](edge0-strata/EDGE0.md) | The engine for `edge0-145b` and `edge0-120b` (NVIDIA GPU, Linux or Windows), a fork of Strata that `edge0-infer` starts for these models |
 | [`docs/`](docs/) | Model design docs: MoE routing, the prerouter, and the model tiers |
 | [`paper/`](paper/) | The technical report |
 
@@ -32,7 +33,8 @@ is applied there first and arrives here with the next sync.
 
 Build the engine and get the model assets as described in
 [Getting started](edge0-infer/docs/getting-started.md). It covers macOS, iOS,
-Android and Windows.
+Android, Linux and Windows. For `edge0-145b` and `edge0-120b`, also build
+`edge0-strata/` as [`edge0-strata/EDGE0.md`](edge0-strata/EDGE0.md) describes.
 
 ## Running the tests
 
@@ -84,4 +86,5 @@ instead.
 ## License
 
 By contributing, you agree that your contributions will be licensed under the
-project's [Apache-2.0](LICENSE) license.
+project's [Apache-2.0](LICENSE) license. Contributions to `edge0-strata/` are
+licensed under that directory's [MIT](edge0-strata/LICENSE) license.

NOTICE

@@ -19,15 +19,27 @@ llama.cpp tokenizer code (MIT License)
 
 cpp-httplib (MIT License)
   - `edge0-infer/third_party/cpp-httplib/` (version 0.60.1), used only by the
-    CLI and the OpenAI-compatible server.
+    CLI and the local server.
     License: `edge0-infer/third_party/cpp-httplib/LICENSE`.
   Copyright (c) 2017 yhirose.
 
 nlohmann/json (MIT License)
   - `edge0-infer/third_party/nlohmann-json/` (version 3.12.0), used only by the
-    CLI and the OpenAI-compatible server.
+    CLI and the local server.
     License: `edge0-infer/third_party/nlohmann-json/LICENSE.MIT`.
   Copyright (c) 2013-2025 Niels Lohmann.
 
+Strata (MIT License)
+  - `edge0-strata/` is a fork of Strata (https://github.com/Niko1221/Strata),
+    imported from the fork at commit f1933b1 plus the source files recorded in
+    `edge0-strata/edge0/IMPORTED_UNCOMMITTED.txt`, whose merge base with upstream
+    Strata is commit c62d5e3. The license is kept verbatim in
+    `edge0-strata/LICENSE`; `edge0-strata/EDGE0.md` records the origin and the
+    changes. Third-party code in it keeps its own license and notice files
+    (`edge0-strata/third_party/`): ggml, project-maya, the libstrix notice, and
+    `ggml-tq`, which is derived from the ggml of the agentionai/llama.cpp fork
+    (MIT License, the ggml authors).
+  Copyright (c) 2026 Niko1221 and the Strata contributors.
+
 All other files are Copyright (c) 2025 edge0 contributors, licensed under
 the Apache License, Version 2.0 (see LICENSE).

README.md

@@ -6,7 +6,7 @@
 
 **An open-source streaming MoE inference framework — SSD expert offload + Recover-LoRA + prerouter routing prediction.**
 
-**macOS** · **iOS** · **Android** · **Windows** — one engine, one native API.
+**macOS** · **iOS** · **Android** · **Linux** · **Windows** — one program for every Edge0 model.
 
 [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--35B--A3B--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-35B-A3B-preview)
 [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Edge0--8B--A1B--preview-yellow?style=for-the-badge)](https://huggingface.co/Edge0/Edge0-8B-A1B-preview)
@@ -23,6 +23,7 @@ English | [中文](README_zh.md) | [日本語](README_ja.md) | [Español](README
 
 ## News
 
+- **[2026-11-13]** We released **`edge0-145b`** and **`edge0-120b`**, latent mixture-of-experts models derived from Qwen3.8-Flash-Next for a single NVIDIA GPU on Linux or Windows. They run on [edge0-strata](edge0-strata/EDGE0.md), which `edge0-infer serve` and `edge0-infer chat` start for them; see the [model page](docs/models/edge0-145b.md).
 - **[2026-11-04]** We released the **unified inference engine [`edge0-infer`](edge0-infer/README.md)**: built on [GGML](https://github.com/ggml-org/ggml), one engine and one native C++ API run the same model on macOS, iOS, Android and Windows, with a CLI, an OpenAI-compatible local server, and chat app examples for desktop, Android and iOS. The first release supports `edge0-35b`; `edge0-8b` follows in a later release. It replaces the Python framework and the per-platform engines; the previous code is kept at tag [`pre-edge0-infer`](https://github.com/Edge0-AI/Edge0/tree/pre-edge0-infer).
 - **[2026-09-30]** We released the **edge0 inference engines for four platforms — iOS, macOS, Android and Windows** (source at tag [`pre-edge0-infer`](https://github.com/Edge0-AI/Edge0/tree/pre-edge0-infer)).
 - **[2026-09-16]** Our technical report is on arXiv: [The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction](https://arxiv.org/abs/2609.18063).
@@ -52,33 +53,37 @@ memory is bounded by the *active* expert set, not the parameter count.
 
 ### Platforms
 
-One engine and one native API, with the same code on every platform — see [`edge0-infer/`](edge0-infer/README.md):
+[`edge0-infer`](edge0-infer/README.md) is the program you run (CLI, local OpenAI- and Anthropic-compatible server, chat apps), with the same commands for every model. Each model runs on the engine built for it, and edge0-infer picks the engine from the model folder; there is nothing to choose: `edge0-35b` runs on the **native engine** in `edge0-infer/` (built on GGML, with a native C++ API), and `edge0-145b` and `edge0-120b` run on **edge0-strata** in [`edge0-strata/`](edge0-strata/EDGE0.md), a fork of Strata.
 
-| Platform | Backends | Verified on |
+| Platform | `edge0-35b` (native engine) | `edge0-145b`, `edge0-120b` (edge0-strata) |
 |---|---|---|
-| **macOS** (Apple Silicon) | Metal, CPU | Apple M5, 16 GiB |
-| **iOS** | Metal | iPhone 17 Pro Max |
-| **Android** | CPU | OnePlus PKR110 (Snapdragon 8 Elite) |
-| **Windows** (x86-64) | CPU, Vulkan | Intel Core i5-12400, NVIDIA GeForce RTX 4070 SUPER |
+| **macOS** (Apple Silicon) | Metal, CPU | — |
+| **iOS** | Metal | — |
+| **Android** | CPU | — |
+| **Linux** | CPU | NVIDIA GPU (CUDA 13) |
+| **Windows** | CPU, Vulkan | NVIDIA GPU (CUDA 13) |
 
-Prefill and decode can run on different backends of the same device, for example Metal prefill with CPU decode. Chat app examples for desktop (macOS, Windows), Android and iOS are in [`edge0-infer/examples/`](edge0-infer/examples/README.md#chat-applications).
+Measurements: [Benchmarking](edge0-infer/docs/benchmarking.md#reference-figures) for `edge0-35b`, and the [model page](docs/models/edge0-145b.md) for `edge0-145b` and `edge0-120b`. On the native engine, prefill and decode can run on different backends of the same device, for example Metal prefill with CPU decode. Chat app examples for desktop (macOS, Windows), Android and iOS are in [`edge0-infer/examples/`](edge0-infer/examples/README.md#chat-applications).
 
 ### Models
 
-Two model tiers ship with the framework. Each tier is an end-to-end
+Two model tiers, `edge0-35b` and `edge0-8b`, ship with the framework. Each tier is an end-to-end
 release: the released checkpoint, the trained LoRA adapters, and the
 trained prerouter heads work together as one unit.
 
-| Tier | Released checkpoint | Inference profile | edge0-infer |
+| Tier | Released checkpoint | Inference profile | Engine |
 |---|---|---|---|
-| `edge0-35b` | [`Edge0/Edge0-35B-A3B-preview`](https://huggingface.co/Edge0/Edge0-35B-A3B-preview) · [ModelScope](https://www.modelscope.cn/models/Edge0/Edge0-35B-A3B-preview) | 4-bit, 40 layers, 256 experts, prerouter K=4 | ✅ Supported |
-| `edge0-8b` | [`Edge0/Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview) · [ModelScope](https://www.modelscope.cn/models/Edge0/Edge0-8B-A1B-preview) | 4-bit, 24 layers, 128 experts, prerouter K=8 | Planned |
+| `edge0-35b` | [`Edge0/Edge0-35B-A3B-preview`](https://huggingface.co/Edge0/Edge0-35B-A3B-preview) · [ModelScope](https://www.modelscope.cn/models/Edge0/Edge0-35B-A3B-preview) | 4-bit, 40 layers, 256 experts, prerouter K=4 | Native engine ✅ |
+| `edge0-8b` | [`Edge0/Edge0-8B-A1B-preview`](https://huggingface.co/Edge0/Edge0-8B-A1B-preview) · [ModelScope](https://www.modelscope.cn/models/Edge0/Edge0-8B-A1B-preview) | 4-bit, 24 layers, 128 experts, prerouter K=8 | Native engine (planned) |
+| [`edge0-145b`](docs/models/edge0-145b.md) | coming soon | Q3_K or NVFP4 latent experts (1792), 48 layers, 512 experts, K=10 | edge0-strata ✅ (Linux or Windows, NVIDIA GPU) |
+| [`edge0-120b`](docs/models/edge0-145b.md) | coming soon | Q3_K or NVFP4 latent experts (1280), 48 layers, 512 experts, K=10 | edge0-strata ✅ (Linux or Windows, NVIDIA GPU) |
 
-Both checkpoints are built on open sparse-MoE base models (Qwen3.6-35B-A3B
+The two released checkpoints are built on open sparse-MoE base models (Qwen3.6-35B-A3B
 and the Ling 3.0 bailing hybrid respectively) and ship with the
 LoRA and prerouter training done for this framework. edge0-infer downloads
 the released checkpoint with its asset tools and converts it into the
 engine's model assets (see [Getting Started](#getting-started)).
+`edge0-145b` and `edge0-120b` are a separate latent-MoE tier for a single NVIDIA GPU; they run on edge0-strata, which `edge0-infer serve` and `edge0-infer chat` start for them (see [edge0-145b and edge0-120b](docs/models/edge0-145b.md)).
 
 ### Design
 
@@ -87,7 +92,7 @@ engine's model assets (see [Getting Started](#getting-started)).
   downloads nothing;
 - **One native API**: applications link the CMake target `edge0::edge0`
   and include the headers in `include/edge0/` to use the engine in
-  process; the reference CLI and the OpenAI-compatible server use the same
+  process; the reference CLI and the local server use the same
   settings and defaults as the native API;
 - **Experts on demand**: routed experts are read from storage on demand
   into a fixed number of expert slots per layer, so a model larger than the
@@ -118,6 +123,17 @@ edge0-35b, 2.8 for edge0-8b** (MMLU-Pro is even above the base). Max 100:
 | IFBench | 57.9 | 61.7 | 53.9 | 60.6 |
 | **Average** | **79.2** | **83.2** | **69.9** | **72.7** |
 
+`edge0-145b` and `edge0-120b` (**bold** = best in column; see the [model page](docs/models/edge0-145b.md#quality)):
+
+| Model | deepswe 1.1 | terminal-bench 2.1 | AA-LCR | IFBench | GPQA-Diamond | HLE |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| **Edge0-145B** | **64.6** | **86.5** | 79.3 | 80.6 | 90.4 | 35.4 |
+| **Edge0-120B** | 50.0 | 75.0 | 81.0 | 79.3 | 87.4 | 31.0 |
+| DeepSeek-V4-Flash-0731 | 54.4 | 82.7 | 79.7 | 79.2 | 90.8 | 33.8 |
+| Qwen3.8-Flash-Next | 58.7 | 83.3 | 79.1 | **81.3** | **91.7** | 35.9 |
+| Qwen3.8-27B | 42.2 | 73.0 | **82.0** | 79.5 | 89.2 | 30.8 |
+| Claude-Opus-4.6 (Max) | / | 78.2 | 78.0 | 62.5 | 91.3 | **40.0** |
+
 ### Benchmark
 
 `edge0-35b` with `predicted` routing, measured with edge0-infer (each range is the minimum to maximum of three processes):
@@ -133,6 +149,33 @@ edge0-35b, 2.8 for edge0-8b** (MMLU-Pro is even above the base). Max 100:
 
 Memory metrics differ by platform and cannot be compared directly. The measurement method, the `actual` routing figures and the commands to reproduce them are in [Benchmarking](edge0-infer/docs/benchmarking.md#reference-figures).
 
+`edge0-145b` and `edge0-120b` on edge0-strata, one NVIDIA GPU (tok/s):
+
+#### edge0-145b
+
+| Machine | Build | 4K prefill | 16K prefill | Chat decode | Agent decode |
+| --- | --- | ---: | ---: | ---: | ---: |
+| RTX 5090 D 24 GB | Q3_K | 2169.9 | 3554.9 | 99.7 | **121.2** |
+| RTX 5090 D 24 GB | NVFP4 | 1839.6 | 3165.7 | 79.6 | 98.3 |
+| RTX 4090 24 GB | Q3_K | 2042.0 | 3179.4 | 69.9 | 91.7 |
+| RTX Spark N1X 60 GiB | Q3_K | 674.0 | 770.3 | 37.7 | 36.7 |
+| RTX Spark N1X 60 GiB | NVFP4 | 821.2 | 922.4 | 40.1 | 46.0 |
+| 16 GiB VRAM + 20 GiB RAM | Q3_K | 249 | 335 | 36.8 | 44.9 |
+
+#### edge0-120b
+
+| Machine | Build | 4K prefill | 16K prefill | Chat decode | Agent decode |
+| --- | --- | ---: | ---: | ---: | ---: |
+| RTX 5090 D 24 GB | Q3_K | 3236.7 | 4576.5 | 130.2 | **144.6** |
+| RTX 5090 D 24 GB | NVFP4 | 2668.5 | 4357.1 | 124.9 | 135.5 |
+| RTX 4090 24 GB | Q3_K | 2637.5 | 3551.1 | 95.0 | 104.9 |
+| RTX Spark N1X 60 GiB | Q3_K | 987.0 | 1081.7 | 47.2 | 68.4 |
+| RTX Spark N1X 60 GiB | NVFP4 | 996.4 | 1066.9 | 45.1 | 68.9 |
+| 16 GiB VRAM + 20 GiB RAM | Q3_K | 430 | 778 | 62.4 | 73.3 |
+| 16 GiB VRAM + 20 GiB RAM | NVFP4 | 361.1 | 619.7 | 51.3 | 59.9 |
+
+`16 GiB VRAM + 20 GiB RAM` rows are the same RTX 5090 D 24 GB, limited by cgroup to 16 GiB of GPU memory and 20 GiB of system memory. Details are on the [model page](docs/models/edge0-145b.md#speed-measured-one-card).
+
 ## Getting Started
 
 Run the following commands in `edge0-infer/`. You need CMake 3.21+,
@@ -151,7 +194,9 @@ build/mac-metal/bin/edge0-infer info        # expect a line: model.0: edge0_35b
 On machines without Metal use `--preset cpu`, `build/cpu/bin/edge0-infer`,
 and `--device cpu` in the commands below.
 
-### 2) Download and convert the model assets
+### 2) Get a model
+
+**edge0-35b (native engine).** Download and convert the model assets:
 
 ```bash
 export MOE_SOURCE_35B=<directory for the original weights>
@@ -168,17 +213,21 @@ assets_py tools/assets/check_assets.py check "$MOE_ASSETS_35B"
 This downloads `Edge0/Edge0-35B-A3B-preview` from Hugging Face and checks
 it against the locked revision and digests.
 
+**edge0-145b and edge0-120b (edge0-strata).** Build edge0-strata as [`edge0-strata/EDGE0.md`](edge0-strata/EDGE0.md) describes and use a released model folder (see [edge0-145b and edge0-120b](docs/models/edge0-145b.md)).
+
 ### 3) Run
 
 ```bash
 # one prompt
 build/mac-metal/bin/edge0-infer run   --model "$MOE_ASSETS_35B" --device metal --user "Introduce yourself in one sentence."
 # multi-turn chat
 build/mac-metal/bin/edge0-infer chat  --model "$MOE_ASSETS_35B" --device metal
-# serve (OpenAI-compatible, default http://127.0.0.1:8000/v1)
+# serve (OpenAI- and Anthropic-compatible, default http://127.0.0.1:8000)
 build/mac-metal/bin/edge0-infer serve --model "$MOE_ASSETS_35B" --device metal
 ```
 
+For edge0-145b and edge0-120b, `chat` and `serve` take the model folder the same way (`--model <model folder>`); `run` is for the native engine.
+
 For the Android, Windows and iOS builds, and for using the engine from
 your own CMake project, see [Getting started](edge0-infer/docs/getting-started.md).
 
@@ -187,17 +236,22 @@ your own CMake project, see [Getting started](edge0-infer/docs/getting-started.m
 **Models**
 
 - [MoE routing and experts](docs/moe.md) / [prerouter](docs/prerouter.md)
-- [edge0-35b](docs/models/edge0-35b.md) / [edge0-8b](docs/models/edge0-8b.md)
+- [edge0-35b](docs/models/edge0-35b.md) / [edge0-8b](docs/models/edge0-8b.md) / [edge0-145b and edge0-120b](docs/models/edge0-145b.md)
 - Technical report: [The Other Half of the Memory Wall](https://arxiv.org/abs/2609.18063) ([PDF](paper/main.pdf))
 
-**Engine (edge0-infer)**
+**edge0-infer**
 
 - [Getting started](edge0-infer/docs/getting-started.md): build, model assets, run, chat and serve on every platform
-- [Configuration](edge0-infer/docs/configuration.md) / [Native API](edge0-infer/docs/native-api.md) / [OpenAI-compatible server](edge0-infer/docs/openai-compatible-server.md)
+- [Configuration](edge0-infer/docs/configuration.md) / [Native API](edge0-infer/docs/native-api.md) / [Local server](edge0-infer/docs/local-server.md)
 - [Architecture](edge0-infer/docs/architecture.md) / [Known limitations](edge0-infer/docs/limitations.md)
 - [Benchmarking](edge0-infer/docs/benchmarking.md) / [Testing](edge0-infer/docs/testing.md)
 - [Model asset tools and adding a model](edge0-infer/tools/assets/README.md)
 
+**Engines**
+
+- Native engine: [Architecture](edge0-infer/docs/architecture.md#native-engine) / [Native API](edge0-infer/docs/native-api.md)
+- edge0-strata: [build and launch contract](edge0-strata/EDGE0.md) / [edge0-145b](edge0-infer/docs/edge0-145b.md) and [edge0-120b](edge0-infer/docs/edge0-120b.md) in edge0-infer
+
 ## Roadmap
 
 ### Q4 2026