[hattori7243]

What & why

Edge0 145B and Edge0 120B ran only on Linux and Windows with an NVIDIA GPU. This PR runs them on macOS (Apple Silicon) with Metal, through edge0-infer serve / chat and the Edge0 Chat desktop app, and adds a script that builds the macOS package of Edge0 Chat.

• edge0-strata: restore metal/ from Strata f1933b1 and apply the Metal latent Q4 handoff patch; add the Metal launch templates edge0-145b-metal.json / edge0-120b-metal.json, tools/mtp_gguf.py (rebuilds mtp-q2_0.gguf, the MTP file the Metal engine reads, from the released mtp/ folder), contract 5, and "Build (macOS, Apple Silicon)" in EDGE0.md. The released Edge0-{145B,120B}-q3k folders are used as they are; no weights are re-exported.
• edge0-infer (export of the development repository):
  • On macOS the delegated device is metal (also the default). cuda is accepted only on Linux and Windows, metal only on macOS.
  • On metal, du accepts only q4, ple-io only direct, and -fp4 folders are rejected, all before anything starts.
  • The template is chosen per device (<name>-metal.json on Metal).
  • A template can declare a prepare step. The Metal templates use it to write mtp-q2_0.gguf into the model folder on the first start; later starts leave it in place.
  • /status and the logs name the real device.
  • Desktop app: with no executable configured, it also uses the edge0-infer in Contents/Resources of its macOS app bundle.
• docs: README (and translations), model page and CONTRIBUTING name macOS; Apple M2 Ultra rows in the speed tables; the download badge reads "Windows · macOS".
• scripts/package/macos-arm64: package.sh fetch | build | verify.
  • fetch downloads the pinned inputs.
  • build packs offline: Edge0 Chat.app with edge0-infer, edge0-strata (Metal) and its Python runtime inside Contents/Resources, signed ad hoc, as edge0-chat-<version>-macos-arm64.zip.
  • verify checks the package and can serve one prompt with a model.

Linux and Windows behavior is unchanged. The patch's edits to shared engine sources are inside STRATA_GPU_METAL, and the CUDA templates are unchanged.

Not in this PR: Q8 D/U and NVFP4 on Metal, 128K context on Metal, notarization.

Area

• Engine, CLI or server (edge0-infer/)
• Model asset tools (edge0-infer/tools/assets/)
• Chat app examples (edge0-infer/examples/)
• Model docs (docs/)
• Other (edge0-strata/, scripts/package/, README)

How was it tested?

edge0-infer (development repository): L0–L2 161/161 on macOS. New L1 cases cover the macOS / Linux / Windows rules. New L2 cases with a fake engine cover the prepare step, the Metal limits and /status. Desktop app Rust tests 29/29.

Mac Studio, Apple M2 Ultra 192 GB, macOS 27.0.1:

• edge0-strata built from a clean checkout as EDGE0.md describes. moe_mm_parity 1280 and 1792 PASS. The patch's verify_metal_latent.py 14/14 for each model.

• mtp_gguf.py output vs the original mtp-q2_0.gguf: 21 tensors byte-identical; 10 Q8_0 tensors with 0 blocks different from the engine's own quantization.

• Original vs converted MTP, 4 fixed prompts at temperature 0, both models: identical answers and identical accepted drafts per prompt.

• Through edge0-infer, from a folder without mtp-q2_0.gguf, both models:

  • the first start generates the file and becomes ready; the second start does not regenerate it;
  • 4 fixed prompts at temperature 0 give identical answers through the proxy and directly;
  • multi-turn chat, /v1/responses, /v1/messages and thinking all answer.

• Speed with strata_bench.py through the proxy (4K prefill / 16K prefill / chat decode / agent decode, tok/s):

  Model	Measured	Model card
  145B	754.5 / 799.0 / 57.5 / 58.4	755.6 / 800.3 / 60.9 / 57.0
  120B	868.8 / 918.3 / 60.5 / 60.4	867.6 / 908.7 / 60.4 / 63.0

  The docs use the model-card figures.

• Desktop app: both models load and chat.

Package (package.sh):

• Built on the Mac Studio.
• verify passes:
  • 145B;
  • 120B from a folder without mtp-q2_0.gguf; the bundled Python generated it, byte-identical to the one above;
  • 35B on Metal and on the CPU (Apple M5, macOS 26.5).
• First launch walked through from the zip with the quarantine flag set, through Privacy & Security > Open Anyway.

Regression:

• Linux (RTX 5090 D): 145B through serve answers.
• Linux and Windows: build plus the delegation L1/L2 tests pass.
• Not run on a Windows RTX GPU: the CUDA code path and templates are unchanged.

Checklist

• ctest --preset cpu -L "L0|L1" passes locally in edge0-infer/ (engine changes)
• Ran the higher test levels this change affects (see edge0-infer/docs/testing.md)
• Added/updated tests for new behavior
• Updated relevant docs / READMEs (incl. translations if user-facing)

[reneleonhardt]

Thank you for restoring Metal support! 🚀

Not in this PR: 128K context on Metal, notarization

What context size is supported at the moment? Are those improvements planned?
Will there be official MLX weights?

[hattori7243]

Thank you for restoring Metal support! 🚀

Not in this PR: 128K context on Metal, notarization

What context size is supported at the moment? Are those improvements planned? Will there be official MLX weights?

Thanks! On Metal the default context is 32K, but it is not a hard limit. The desktop app currently starts the model with that fixed default, while the CLI lets you set it yourself, e.g. edge0-infer serve --model <folder> --context 131072.

We tested 128K on a Mac Studio (Apple M2 Ultra, 192 GB) with Edge0 145B: a 127K-token prompt with a fact placed near the beginning was answered correctly, prefill averaged about 760 tok/s (about 168 s in total), and the engine process peaked at about 32 GB of memory.

Notarization and official MLX weights are not available yet; we'll consider both for future releases. Until then, the first launch of the app needs Privacy & Security > Open Anyway, and on macOS the models run on the Metal engine directly from the released Q3_K folders.