supported-models.md

Supported models and hardware

This page distinguishes validated serving paths from experimental integrations. Start with the CPU decision walkthrough. The rolling model tracker (#83) records available paths separately from proposed integrations and hardware evidence. Models that are being added are also tracked in issues labeled new model.

Model	Worker	CPU	NVIDIA CUDA	Apple GPU	Requirements
LAYA, English checkpoint	External worker running the upstream Laya runtime, for text requests	Validated (#2); CPU demo	Unverified (#39)	Use the dedicated MPS worker below	Python 3.12, pinned CPU dependencies
LAYA, English checkpoint	Python MPS/CPU worker in this repository	Contract checks documented (#67)	No validation recorded for this path	PyTorch MPS validated on M1 Pro; M4/M5 checks documented (#30, #67)	Python 3.12, MPS dependency versions; native Metal execution remains planned
LAYA, English checkpoint	Native Rust worker	Processing/checkpoint checks; no CPU inference	Hopper sm_90a; validation scope	Not supported	CUDA toolkit, TileLang for AOT generation, pinned checkpoint and rotary tables
Cua-S1 4B 0.2, text adapter	Reference worker on Transformers and PEFT	Unverified	Validated (#13)	Unverified	Python 3.12, the versions in requirements-text.txt
Cua-S1 4B 0.2, text adapter	Native Rust worker on the Qwen3.5 CUDA kernels	Not supported	Validated on compute capability 8.9 (#19, #52)	Not supported	Compute capability 8.0 or newer, the CUDA toolkit to build, weights merged with export_text_merged.py
Cua-S1 4B 0.2, multimodal adapter	Reference worker on Transformers and PEFT, src/frontend/cua_s1.py; no recipe yet	Not supported	Validated (#17, #18)	Not supported	The state is one PNG or JPEG image; upstream's weights.lock.json next to the base weights
Cua-S1 4B 0.2, multimodal adapter	Native Rust/CUDA screenshot worker	Preparation checks only; no CPU inference	RTX 4090 / sm_89, native vision and language Graph regressions; scoped evidence and controls	Not supported	Verified BF16 language export, original vision checkpoint and FP32 vision adapter, rebuilt ABI 9 CUDA library; one PNG/JPEG and choice
Valen-Preview-0923	Python reference worker; text state or one inline PNG/JPEG image; choice, noul, and score questions	Not supported	Author-run on RTX 5060 Laptop (sm_120, WSL2): one text and one 2×2 PNG request; input and readout parity ≤1e-6 against reference (model outputs not compared); noul/score parity ≤1e-6 with exact legend and token agreement on the recipe's text fixtures. Owner: community.	Not supported	Python 3.12, pinned Valen source/checkpoint, Qwen3.5-2B base, BF16 CUDA
Open-Jev-27B-v1.1	Native Rust/CUDA worker on the shared Qwen3.5/3.8 executor	Not supported	Validated on H200 (sm_90) for the 74 single-candidate workload	Not supported	Compute capability 8.0 or newer, CUDA toolkit to build, exported merged weights and trained head; rebuild worker and library together for the 27B candidate optimizations
Open-Jev-9B	The same native Rust/CUDA worker	Not supported	Validated on compute capability 8.9 against the reference for 253 requests	Not supported	Compute capability 8.0 or newer, CUDA toolkit to build, exported merged weights and trained head
StartLux-Decision v1.1 4B Instant	Native Rust/CUDA worker	Contract checks only; no CPU inference	Unverified. Build target is L20X / Ada sm_89. The latency recipe is not a result	Not supported	Pinned CC BY-NC 4.0 Instant 4B checkpoint; rebuilt ABI 9 library with cs1_attention_gated_bidirectional. Instant only: AutoThink and 27B are refused
Decider-2B v11	Native Rust/CUDA worker	Contract checks only; no CPU inference	RTX 4090 (sm_89) synthetic parity corpus; limits and provenance	Not supported	Pinned original BF16 checkpoint; ABI 9 library and consumers rebuilt together
JEMM / Qwen3.8-27B	Native Rust/CUDA worker	Host checks only; no CPU inference	A800 (sm_80) text and 1–4-image frozen corpus	Not supported	Pinned jemm-native/2 export with BF16 base and unmerged FP32 LoRA, selected untied LM head, cuDNN 9 and additive reference CUDA symbols
autotrust/JEV-27B-VL	Experimental native worker, single-question API	Contract tests only; no CPU inference	H800 frozen-corpus check; experimental scope and limits	Not supported	Pinned merged checkpoint, rebuilt ABI 9 CUDA library; image inputs require offline Transformers preencoding
CLM-v0.1-8B	External clm-serve recipe with a CPU stub embeddings server	Stub-encoder contract checks only (#23); not real Qwen3-8B decisions	Real encoder unverified by the merged recipe	Unverified	Python, upstream CLM and head checkpoint; a real encoder requires a separate embeddings server
LFM2.5-350M	Transformers choice worker	Tiny-model regression tests; full checkpoint unverified	L40S FP16 recorded execution checks	Unverified	Python 3.12, pinned dependencies, explicit candidate batch size
CLM-v0.1-8B	Native Rust engine, clm-run, in front of a Qwen3-8B /v1/embeddings server	Engine-only agreement 4.5e-06 on identical vectors (#29, validation); contract checks documented (#23)	Validated on compute capability 8.9 (#29, validation)	Unverified	Rust to build and a separate Qwen3-8B /v1/embeddings server for the encoder; the engine itself has no device-specific code
OmniJev-4B v1.1	Native Rust/CUDA worker on the shared Qwen3.5 vision and language path, eager	Contract, preparation, pixels, heads and finishing checked against the reference; no CPU inference	Compared with the reference on compute capability 8.9 for 140 answers	Not supported	Compute capability 8.0 or newer, CUDA toolkit to build, the export from export.py
D3 (Decision 3.0)	Python reference worker on the checkpoint's pinned runtime — text, image and video state over choice/noul/score, with the deliberate transport differences documented in the recipe; the native worker is in progress (#136)	Unverified	Author-run on NVIDIA A100 (sm_80): serving smoke and reference parity for text and media (exact decisions, max delta 0.0); evidence in the integration and media PRs. Owner: @MrDongsls.	Unverified	Python 3.12, the pinned runtime requirements, the pinned package (~52 GB); media budgets and the 64 MiB total request cap
Vela 2.0 0.3B	Python reference worker running the checkpoint's pinned runtime	CPU FP32 validated (author-run, x86 WSL2, 24 cores): startup with all-kind warmup, the four example requests plus five synthetic rejections, and exact-match reference parity against the pinned upstream server on all nine (four 200s byte-equal incl. key order, agreed 422/413); byte-identical frontend forwarding; evidence in the integration PR. Owner: @MrDongsls.	Unverified (bf16-autocast is an untested deployment option)	Not supported	Python 3.12, the pinned requirements, the pinned package (1.24 GB FP32 weights; ~4.4 GB with ONNX/assets)
Vela 2.0 0.8B hybrid	Python reference worker running the checkpoint's pinned runtime (same wire layer as 0.3B)	Same surface as 0.3B plus the span_heads field (router/broad dispatch); CPU FP32 parity (author-run; evidence in the integration PR)	Unverified (bf16-autocast is the evaluated upstream setting but untested here)	Not supported	Python 3.12, the pinned requirements, the pinned package (2.02 GB shard + 4.5 MB broad head; ~3 GB FP32 parameters)
Clef-flash (9B)	Rust omni-clef server with the pinned Python executor on one persistent thread; text choice/noul/score; non-thinking prompt; thinking.enabled: false admitted and omitted from the prompt; 16384-token refusal. The joint head is still the checkpoint module	Contract tests only, including the persistent-thread serve path; no inference	Kept Public231 primary on the L20X-labeled lab host is the previous ThreadingHTTPServer worker. Baseline 9030065 run1 p50 247.518 ms, p95 413.025 ms, 231/231. Tip 594528a9 remeasure, warmup 5, concurrency 1, stock Public231 with thinking.enabled false, metric successful_latency: 231/231, p50 164.969 ms, p95 216.755 ms, /health fla+causal_conv1d. Versus run1: Δp50 −82.549 ms (−33.35%), Δp95 −196.270 ms (−47.52%). Operator-reported omni-clef tip f9127a0: smoke 4/4, /health fla+causal_conv1d, R1 p50 169.605 ms, p95 220.003 ms (231/231), R2 53.580/97.031, R3 52.950/97.608, quality choice_acc=0.8417 noul_acc=0.7973 score_mae=0.2488 (matches 594528a9). Not a latency KEEP. R1 does not pass the dual gate versus 164.969/216.755. R2 and R3 are not a new primary. Not re-run here	Not supported	Python 3.12, card-tested torch 2.11 and transformers 5.10.2, a Rust build of omni-clef, pinned Cloudflare/clef-flash snapshot (~19.1 GB)
Clef (27B)	Same omni-clef server and text contract as Clef-flash, selected with --variant clef	Contract tests only; no inference	Not run	Not supported	Same stack; pinned Cloudflare/clef snapshot (~55.0 GB). Weights and joint-head config differ from flash
pplx-decider-v1.1-27b	Python reference worker on the checkpoint's own runtime. Text and up to four images; choice/noul/score. Native CUDA executor is not this worker. Lab L20X-labeled GPUs are the H200 target. The causal-conv1d build keeps sm_90 and sm_89. Capability 8.9 is ready on FLA chunk. Capability 9.0 uses all-sequence FlashInfer. The kept primary rotates 48 cu_seqlens buffers per device and length, passes use_cp false, skips the q/k repeat, and packs q and k before the L2-norm view. 801ef7b fused the 48 linear gated RMSNorms and missed the median (p50 89.172 ms, p95 274.790 ms), so that kernel stays off. 2bbbe4e replaced silu(gate) * up with FLA swiglu_fwd and the owner closeout rejected it (p50 97.613 ms, p95 292.795 ms), so that kernel stays off. Gate and up GEMMs stay separate. Residual RMSNorm and the gated output norm stay eager. An unknown PPLX_GDN_PREFILL fails closed on every device, including capability 8.9. 3ecdf5a crashed in warmup (0/231). Always-on fusion 3afb472 and the 512-token gate 705ae089 missed the median. The 48-slot FlashInfer state ring 17a7b120 missed the gate. PPLX_GDN_PREFILL=fla reproduces the prior f968e576 chunk. PPLX_GDN_PREFILL=short is the rejected ≤256 dispatch. SDPA stays on the checkpoint order. The fused gate-up GEMM is refused. The forward stays eager. PPLX_COMPILE=dynamic and PPLX_COMPILE=dynamic-tail are rejected opt-ins	Contract checks only; no inference	Reported load on 1× lab L20X (sm_89, ~140 GiB) at head 1ba07d1: BF16 resident ~50333 MiB, no OOM. Historical primary: 231/231, p50 98.777 ms, p95 341.202 ms. Prior primary is the f968e576 remeasure on the lab L20X-labeled GPU, compute_capability=9.0, fla-chunk, flashinfer null: 231/231, p50 94.077 ms, p95 300.200 ms (p50 −4.76%, p95 −12.02%). Prior primary ffae182: 231/231, p50 92.844 ms, p95 296.458 ms (p50 −1.31%, p95 −1.25%). Prior primary 4b99b73: 231/231, p50 88.942 ms, p95 295.797 ms (p50 −4.20%, p95 −0.22%), 48-slot cu_seqlens, quality choice_acc=0.9065 (139), noul_acc=0.8784 (74), score_mae=0.2035 (18). Kept primary is fcf5e2b: 231/231, p50 87.782 ms, p95 291.183 ms (p50 −1.30%, p95 −1.56%), grouped-value FlashInfer with q and k packed, same quality digits. Uncached all-sequence FlashInfer 3c72f02: p50 88.808 ms, p95 300.751 ms, gate failed. Short-sequence FlashInfer a50b9aa: p50 95.446 ms, p95 300.871 ms, gate failed. cuDNN-first SDPA 0dcaa0ac: p50 138.501 ms, p95 332.859 ms, gate failed. Fused gate-up 8b4ee326: p50 102.437 ms, p95 304.970 ms, gate failed. Always-on dynamic compile e36b32ed: p50 97.751 ms, p95 263.124 ms, gate failed (p95 improved). 512-token compile 7e3b1be: p50 100.037 ms, p95 257.084 ms, gate failed. Isolated-median compile 34ba5de: p50 97.084 ms, p95 256.679 ms, gate failed (median probe was eager). FlashInfer state-workspace ring 17a7b120: p50 89.165 ms, p95 295.723 ms, gate failed. Always-on FLA residual RMSNorm 3afb472: p50 93.935 ms, p95 274.902 ms, gate failed. Length-gated FLA residual RMSNorm 705ae089: p50 94.604 ms, p95 274.835 ms, gate failed. use_cp=auto a6f4d9a: p50 112.294 ms, p95 299.419 ms, gate failed. Grouped-value 3ecdf5a crashed in warmup (0/231, no p50 or p95). The fused linear gated-norm candidate has no sample. Does not enter the H800 board	Not supported	Python 3.12, pinned checkpoint revision 14c9d90137962d300f3de95633f7b6535e80c5ce, checkpoint requirements.txt, causal-conv1d 1.7.0, flash-linear-attention 0.5.2, optional flashinfer-python 0.6.16.post3. Shard sizes in the release manifest sum with the readout to about 48.6 GiB before activations

• Validated: covered by the recipe on main or by the checks in the linked merged pull request.
• Author-run: recorded by the recipe author on the stated hardware and not independently reproduced; the row names its owner.
• Unverified: the worker accepts this device, but no recipe or merged pull request covers it.
• Planned: not implemented yet; the linked issue tracks it.

The Cua-S1 workers answer choice questions only. LAYA's English worker, Open-Jev, CLM, and StartLux Instant 4B support choice, score, and noul text questions. StartLux serves Instant only; AutoThink is not implemented, and the weights are CC BY-NC 4.0. D3's reference worker serves the same three kinds for text, image and video state (inline base64 media under the pinned runtime's per-item budgets); its media serving evidence is author-run on the A100 above. Vela 2.0's reference workers (0.3B and the 0.8B hybrid) serve all five kinds (choice, noul, score, plus the span/set extensions with their Noul views, and span_heads on the hybrid) for text state on CPU FP32; the larger hybrid members (4B/9B) are separate slices. Clef-flash and Clef serve the same three kinds for text only. The worker refuses image and video inputs. The flash CUDA cell records two operator-reported Public231 concurrency-1 results on the L20X-labeled lab host. Baseline 9030065 run1 is p50 247.518 ms, p95 413.025 ms, metric successful_latency, clock post_to_body_json_and_schema_validation. Tip 594528a9 is the primary: warmup 5, concurrency 1, stock Public231 with thinking.enabled false, metric successful_latency, p50 164.969 ms, p95 216.755 ms, 231/231, /health linear_attention fla+causal_conv1d, Δp50 −82.549 ms (−33.35%), Δp95 −196.270 ms (−47.52%). The tip paste does not restate the clock name. Neither campaign was re-run for this document. Both used the previous Python HTTP server. Operator-reported omni-clef tip f9127a0 on that host is a serve-path paste, not a latency KEEP: smoke 4/4, /health fla+causal_conv1d, R1 p50 169.605 ms, p95 220.003 ms (231/231), R2 53.580/97.031, R3 52.950/97.608, quality choice_acc=0.8417, noul_acc=0.7973, score_mae=0.2488 (matches the 594528a9 baseline). R1 does not pass the dual gate versus 164.969/216.755. R2 and R3 are not a new primary. The primary remains tip 594528a9. The worker starts on that host when the capability prints 8.9. The default prefill is the FLA chunk. --gdn-prefill flashinfer is opt-in and is the named follow-up blocker: it has no measured forward and no answer parity against fla+causal_conv1d. The 27B variant has no recorded timing.

pplx-decider's reference worker admits those three kinds for text and up to four images. The checked path is the CPU contract. The kept Public231 primary is fcf5e2b in the table: p50 87.782 ms, p95 291.183 ms, grouped-value FlashInfer with q and k packed and a 48-slot cu_seqlens ring. The prior primary 4b99b73 is p50 88.942 ms, p95 295.797 ms. ffae182 is p50 92.844 ms, p95 296.458 ms. f968e576 is p50 94.077 ms, p95 300.200 ms, fla-chunk at compute capability 9.0. Uncached all-sequence FlashInfer, the ≤256 FlashInfer dispatch, cuDNN-first SDPA, the fused gate-up GEMM, and the three compile tips missed the gate and are not the primary. Compile improved p95 and raised p50, including after the median route proved eager. The FlashInfer state-workspace ring 17a7b120 missed the gate (p50 89.165 ms, p95 295.723 ms) and is not the primary. Always-on FLA residual RMSNorm 3afb472 missed the gate (p50 93.935 ms, p95 274.902 ms) and is not the primary. Length-gated FLA residual RMSNorm 705ae089 missed the gate (p50 94.604 ms, p95 274.835 ms) and is not the primary. use_cp=auto a6f4d9a missed the gate (p50 112.294 ms, p95 299.419 ms) and is not the primary. Grouped-value 3ecdf5a crashed in warmup (0/231) and is not a latency result. FLA fused gated RMSNorm 801ef7b missed the gate (p50 89.172 ms, p95 274.790 ms) and is not the primary. FLA swiglu_fwd 2bbbe4e missed the gate (owner closeout p50 97.613 ms, p95 292.795 ms) and is not installed. CLM has a row per path: the merged clm-serve recipe exercises those answer shapes with stub embeddings and says nothing about decision quality, while the native engine's CUDA row is the path through a real Qwen3-8B encoder. Its engine-only figure comes from the same comparison run against the deterministic stub, where both sides get identical vectors; neither number validates decision quality. MPS validation above is for a Python/PyTorch worker, not a native Metal backend.

The architecture contracts describe the native target. Shared processing orchestration and dynamic batching remain planned. Native workers reuse serial admission. Qwen workers run independent single-prompt prefills with CPU heads; Laya packs questions within one request and runs its scorer/action head on CUDA. These target layers do not expand the validated model or hardware coverage above.