Repository navigation
Feat/edge0 strata - #180
Merged
Merged
Conversation
edge0-strata is a fork of Strata (https://github.com/Niko1221/Strata) with latent MoE support. It is the engine that edge0-infer starts for Edge0 145B and Edge0 120B. - Import the fork snapshot, without the directories that the CUDA build and the server do not use (apps, docker, integrations, metal, packaging, spark, sycl) and without the root experiment files. - Add the launch contract with edge0-infer (version 4) and the model templates edge0/models/edge0-145b.json and edge0-120b.json, which read the released model folders as provided. - Build on Linux and Windows with an NVIDIA GPU and CUDA 13; on Windows with MSVC (x64) or clang-cl (Arm64). EDGE0.md gives the build steps and the changes from upstream. Co-authored-by: wjiahao1256 <wjiahao1256@gmail.com>
Export edge0-infer from the edge0-infer development repository. - serve, chat and inspect start edge0-strata for a released Edge0 145B or Edge0 120B model folder, chosen by model_name in its model.json; the native API refuses such a folder. - Model options du and ple-io, --strata-arg passthrough of engine arguments, live /status progress for delegated models, and delegated models in the desktop app. - Windows x64 and Arm64 support for delegated models. - Documentation: the two layers (edge0-infer and the engine of each model), docs/edge0-145b.md and docs/edge0-120b.md. Co-authored-by: wjiahao1256 <wjiahao1256@gmail.com>
- docs/models/edge0-145b.md: model profile, quality and measured speed of both models, model folders, requirements and how to run them. - READMEs: platforms and models by engine, release status, getting started and documentation links for both models. - docs/benchmarks/README.md points to the model page for these models. - NOTICE: Strata.
Add POST /v1/messages (text, function tools, thinking; complete and streaming) and POST /v1/messages/count_tokens to edge0-infer serve for native models. LAN authentication also accepts x-api-key, in native and delegated mode. The server guide is renamed to docs/local-server.md; links are updated.
…iew image bench/results/ holds upstream's published results for other models, AGENTS.md describes the upstream product and its installers, and docs/media/pagoda-preview.webp is a 9.6 MB image that nothing in Edge0 uses. None of them takes part in the build or in serve/server.py. EDGE0.md records the removal; SECURITY.md now sends reports to Edge0.
…r docs CONTRIBUTING.md lists edge0-strata/ and states that contributions to it are MIT-licensed like the directory; the issue forms offer Linux, NVIDIA GPU and the two models. The README no longer calls the models "coming soon" next to the release note.
edge0-infer (engine and CLI) and the desktop app move from 0.1.0, the Android and iOS examples and the CLI's iOS bundle from 1.0. The bug report template's example follows.
… and GitHub configuration Remove data/experimental-speed-projection/ (a control vector for Qwen3.8-Flash-Next that removes the model's refusals; not made for the Edge0 models), docs/work/ (upstream's working notes and test logs), .github/ (upstream's funding file, issue forms and workflows), the Dockerfile, which copies from the removed docker/, and the product README in seven other languages. README.md becomes a short pointer to EDGE0.md. Also: the MIT license text in third_party/libstrix/NOTICE.md, two source comments translated to English, links to the removed AGENTS.md and packaging/ fixed, and EDGE0.md lists the removals.
Edge0-120B's AA-LCR score (81.0) was bold although Qwen3.8-27B has 82.0 in the same column. NOTICE names the third-party code under edge0-strata/ and calls the server the local server.
…rchitecture field is delegated) Delegate mode no longer passes STRATA_API_KEY to edge0-strata, reports an engine that stops mid-response as 502/503 or a trailing error event, forwards /v1/messages/count_tokens, and sets TCP_NODELAY. Anthropic Messages accepts tool_result without content and keeps rejected tool calls out of the stream. A model.json is treated as a native Edge0 manifest only when it has format_version.
The fake engine's child no longer holds the test's output pipe, the engine-exit check uses the fake's own record and exit code instead of what the client happens to read, and the test has a 120 s timeout.
…its bound address) http.server looks up the host name of the bound address (socket.getfqdn), which can take tens of seconds on GitHub's macOS runners; the fake engine does not need it. A fake that is alive but not ready now prints its stacks before the self-test fails.
…nchors, model card placeholders, strata product tests - edge0-infer: local-server.md and serve_delegate.h describe delegation by format_version; nine links follow the renamed getting-started sections; a Messages tool block starts only after an open text block is closed; a non-streaming answer is parsed once. - README, model pages: "coming soon" instead of the <MODEL_CARD_URL> placeholder; the 35B page names the Anthropic-compatible server. - edge0-strata: remove the tests of upstream's terminal app, desktop app, Intel engine and Docker/SYCL/AMD setup; setup.py no longer offers the removed control vector; the project-maya NOTICE says the port history is upstream's. - CI: run the delegated-engine L2 tests (fake engine, no GPU).
…links On GitHub's macOS runners the fake engine reports its interpreter under /private/var/folders while the test built the path under /var/folders, the same folder through a link.
shuk777
pushed a commit
that referenced
this pull request
Nov 11, 2026
* edge0-strata: add the engine for Edge0 145B and Edge0 120B edge0-strata is a fork of Strata (https://github.com/Niko1221/Strata) with latent MoE support. It is the engine that edge0-infer starts for Edge0 145B and Edge0 120B. - Import the fork snapshot, without the directories that the CUDA build and the server do not use (apps, docker, integrations, metal, packaging, spark, sycl) and without the root experiment files. - Add the launch contract with edge0-infer (version 4) and the model templates edge0/models/edge0-145b.json and edge0-120b.json, which read the released model folders as provided. - Build on Linux and Windows with an NVIDIA GPU and CUDA 13; on Windows with MSVC (x64) or clang-cl (Arm64). EDGE0.md gives the build steps and the changes from upstream. Co-authored-by: wjiahao1256 <wjiahao1256@gmail.com> * edge0-infer: run Edge0 145B and Edge0 120B on edge0-strata Export edge0-infer from the edge0-infer development repository. - serve, chat and inspect start edge0-strata for a released Edge0 145B or Edge0 120B model folder, chosen by model_name in its model.json; the native API refuses such a folder. - Model options du and ple-io, --strata-arg passthrough of engine arguments, live /status progress for delegated models, and delegated models in the desktop app. - Windows x64 and Arm64 support for delegated models. - Documentation: the two layers (edge0-infer and the engine of each model), docs/edge0-145b.md and docs/edge0-120b.md. Co-authored-by: wjiahao1256 <wjiahao1256@gmail.com> * docs: add Edge0 145B and Edge0 120B - docs/models/edge0-145b.md: model profile, quality and measured speed of both models, model folders, requirements and how to run them. - READMEs: platforms and models by engine, release status, getting started and documentation links for both models. - docs/benchmarks/README.md points to the model page for these models. - NOTICE: Strata. * docs: Edge0 145B and Edge0 120B in the README news, quality and benchmark sections * edge0-strata: add the Arm64 and CUDA host compiler options to the clang-cl build * edge0-infer: serve Anthropic Messages for native models Add POST /v1/messages (text, function tools, thinking; complete and streaming) and POST /v1/messages/count_tokens to edge0-infer serve for native models. LAN authentication also accepts x-api-key, in native and delegated mode. The server guide is renamed to docs/local-server.md; links are updated. * docs: describe the local server as OpenAI- and Anthropic-compatible in the README * edge0-strata: remove upstream benchmark results, AGENTS.md and a preview image bench/results/ holds upstream's published results for other models, AGENTS.md describes the upstream product and its installers, and docs/media/pagoda-preview.webp is a 9.6 MB image that nothing in Edge0 uses. None of them takes part in the build or in serve/server.py. EDGE0.md records the removal; SECURITY.md now sends reports to Edge0. * docs: cover edge0-strata, Linux and Edge0 145B/120B in the contributor docs CONTRIBUTING.md lists edge0-strata/ and states that contributions to it are MIT-licensed like the directory; the issue forms offer Linux, NVIDIA GPU and the two models. The README no longer calls the models "coming soon" next to the release note. * chore: set the version to 0.2.0 edge0-infer (engine and CLI) and the desktop app move from 0.1.0, the Android and iOS examples and the CLI's iOS bundle from 1.0. The bug report template's example follows. * edge0-strata: remove upstream's refusal-removal vector, working notes and GitHub configuration Remove data/experimental-speed-projection/ (a control vector for Qwen3.8-Flash-Next that removes the model's refusals; not made for the Edge0 models), docs/work/ (upstream's working notes and test logs), .github/ (upstream's funding file, issue forms and workflows), the Dockerfile, which copies from the removed docker/, and the product README in seven other languages. README.md becomes a short pointer to EDGE0.md. Also: the MIT license text in third_party/libstrix/NOTICE.md, two source comments translated to English, links to the removed AGENTS.md and packaging/ fixed, and EDGE0.md lists the removals. * docs: fix the bold marks in the quality table and name ggml-tq in NOTICE Edge0-120B's AA-LCR score (81.0) was bold although Qwen3.8-27B has 82.0 in the same column. NOTICE names the third-party code under edge0-strata/ and calls the server the local server. * edge0-infer: export eb50e81 (serve review fixes; model.json with an architecture field is delegated) Delegate mode no longer passes STRATA_API_KEY to edge0-strata, reports an engine that stops mid-response as 502/503 or a trailing error event, forwards /v1/messages/count_tokens, and sets TCP_NODELAY. Anthropic Messages accepts tool_result without content and keeps rejected tool calls out of the stream. A model.json is treated as a native Edge0 manifest only when it has format_version. * edge0-infer: export 71fda2b (fake engine self-test no longer hangs CI) The fake engine's child no longer holds the test's output pipe, the engine-exit check uses the fake's own record and exit code instead of what the client happens to read, and the test has a 120 s timeout. * edge0-infer: export 8e89512 (fake engine skips the reverse lookup of its bound address) http.server looks up the host name of the bound address (socket.getfqdn), which can take tens of seconds on GitHub's macOS runners; the fake engine does not need it. A fake that is alive but not ready now prints its stacks before the self-test fails. * Address the second review: delegated-model wording, getting-started anchors, model card placeholders, strata product tests - edge0-infer: local-server.md and serve_delegate.h describe delegation by format_version; nine links follow the renamed getting-started sections; a Messages tool block starts only after an open text block is closed; a non-streaming answer is parsed once. - README, model pages: "coming soon" instead of the <MODEL_CARD_URL> placeholder; the 35B page names the Anthropic-compatible server. - edge0-strata: remove the tests of upstream's terminal app, desktop app, Intel engine and Docker/SYCL/AMD setup; setup.py no longer offers the removed control vector; the project-maya NOTICE says the port history is upstream's. - CI: run the delegated-engine L2 tests (fake engine, no GPU). * edge0-infer: compare the engine's interpreter folder after resolving links On GitHub's macOS runners the fake engine reports its interpreter under /private/var/folders while the test built the path under /var/folders, the same folder through a link.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds Edge0 145B and Edge0 120B, two latent-MoE models for a single NVIDIA GPU on Linux or Windows, and the engine that runs them.
edge0-strata/: a fork of Strata (MIT) with latent-MoE support for these models.edge0-strata/EDGE0.mdrecords the origin, the changes from upstream, the build steps and the launch contract withedge0-infer;NOTICElists it. Parts of upstream that the Linux/Windows CUDA build and the server do not use are removed (other engines, the desktop app, packaging, upstream's benchmark results,AGENTS.md).edge0-inferdelegated models:serve,chatandinspectrecognise a model folder for these models bymodel_namein itsmodel.json.servestarts edge0-strata on a loopback port and proxies it;chattalks to it as a client./health,/v1/modelsand/statusare answered in Edge0's format, generation endpoints are forwarded. Other subcommands refuse a delegated folder with a clear message.POST /v1/messages(text, function tools, thinking; complete and streaming) andPOST /v1/messages/count_tokens. LAN authentication also acceptsx-api-key, in native and delegated mode.docs/models/edge0-145b.md,edge0-infer/docs/edge0-145b.mdandedge0-120b.md, the server guide renamed todocs/local-server.md, README news, quality and benchmark sections in all five languages, and the contributor docs and issue forms updated for edge0-strata, Linux and the new models.Known gaps, to follow up:
<MODEL_CARD_URL>in the READMEs and the model page; they are replaced when the model cards are online.edge0-inferon macOS only. The Linux and Windows code paths, including the Windows process code for delegated models, are not compiled in CI.Please merge with Squash and merge: the first commit on this branch contains about 22 MB of upstream benchmark results that a later commit removes, and squashing keeps them out of the history of
main.Area
edge0-infer/)edge0-infer/tools/assets/)edge0-infer/examples/)docs/)edge0-strata/(new engine directory), root READMEs,CONTRIBUTING.md, issue formsHow was it tested?
On macOS (Apple Silicon),
cpupreset, at the head of this branch:ctest --preset cpu -L "L0|L1": 104 of 104 passed (3 Vulkan tests skipped).ctest --preset cpu -L L2 -R strata:moe_test_serve_strataandmoe_test_chat_stratapassed. These runserveandchatin delegated mode against the fake engine intests/integration/fake_strata.py.edge0-strata/edge0/IMPORTED_UNCOMMITTED.txtverify against the imported files.On real hardware, on Linux and Windows with an NVIDIA GPU (RTX 5090 D 24 GB, RTX 4090 24 GB, RTX Spark N1X 60 GiB), both models in both expert formats (Q3_K and NVFP4):
edge0-infer/tests/platform/strata/(smoke, real, model formats and bench), which build edge0-strata, start it throughedge0-infer serveandedge0-infer chat, and compare outputs and speed;Checklist
ctest --preset cpu -L "L0|L1"passes locally inedge0-infer/(engine changes)edge0-infer/docs/testing.md)