Repository navigation

All pull requests

fix(engine): isolate per-request cancellation from concurrent decoding in multi-slot mode (--parallel >= 2)
#791nicholas631 opened Nov 2, 2026
ds4: add opt-in strict NUMA placement for locked Engram pages
#784oscarlius opened Nov 2, 2026
1
ds4: optionally prepare CPU row checks during model load
#781oscarlius opened Nov 2, 2026
1
MiniMax-M3.1-Flash on Apple Silicon: a Metal engine (maya-metal-mm3)
#778Franzferdinan51 opened Nov 1, 2026
1
strata-dense: Qwen3.8-27B support (dense engine, MTP / DFlash2, K8V8 KV)
#775merbanan opened Nov 1, 2026Contributor
DeepSeek --tp 2 on CUDA: the exact peer-VRAM prompt all-reduce (MAYA_DS4_HIP_PEER_AR) on NVIDIA, NVLink pairs ~1.3x faster long prompts
#773olobarius opened Nov 1, 2026
1
docs: add 4x RTX 4090 DeepSeek benchmark with CPU startup patches
#767oscarlius opened Nov 1, 2026
2
DeepSeek expert placement: recent slots that survive per-byte placement (DS4_RECENT_FIT, off by default; rounding-level)
#761bitaccesscomau opened Nov 1, 20267 of 8 tasks
EXL3 CPU kernels, AVX2: the byte-aligned trellis windows by byte shuffles (STRATA_EXL3_AVX2_SHUF, off by default; bit-identical)
#760bitaccesscomau opened Nov 1, 20267 of 8 tasks
EXL3 CPU kernels, AVX2: the codebook products and byte sums in 16-bit lanes for calls of 1-3 activations (DS4_CPU_W16, off by default; bit-identical)
#759bitaccesscomau opened Nov 1, 20267 of 8 tasks
DeepSeek CPU lane: a small EXL3 call's gate / up and down pieces cut by columns too (DS4_CPU_SPLIT_COLS, off by default; bit-identical)
#758bitaccesscomau opened Nov 1, 20267 of 8 tasks
ppc64le: build and run on POWER9 (IBM AC922)
#753jefflaichunyin opened Nov 1, 2026
HIP: drain startup peer probes after sampling expires
#748LostVector opened Nov 1, 2026Contributor
ds4: add opt-in direct NH copies for RAM-tier prompt flow
#747oscarlius opened Nov 1, 2026
ds4: an expert gives the same bits on the CPU lane and in VRAM (MAYA_DS4_LANE_PARITY, off, #644)
#741cresslank opened Oct 31, 2026Contributor
ds4: the engine's end sets aside every parallel conversation, not only the bound one
#732hajekt2 opened Oct 31, 2026Contributor
GLM decode: CPU-expert deferral on every part of a split, opt-in (MAYA_GLM_DEFER_CPU=2): +7.7% decode on 2x RTX 4060 Ti (+13% with a re-biased router)
#722oisee opened Oct 31, 2026
2
A GLM setup run again keeps the config's host, API key, CORS origins and MCP servers
#720tobitege opened Oct 31, 2026Contributor
GLM --parallel: keep the conversation checkpoints for each conversation (MAYA_GLM_MULTI_CKPTS, #711)
#714coco-famihiro opened Oct 31, 2026
ds4 GGUF: rebase Q5_K int8 tensor-core GEMVs onto v1.1.42
#712handmade0octopus opened Oct 31, 2026Contributor
1
ds4: tensor-parallel layers with every expert in VRAM can leave the host out too (MAYA_DS4_HOST_FREE_TP=1)
#701loktar00 opened Oct 30, 2026Contributor
ds4: say when another program holds a GPU's memory, and never plan around it silently
#700loktar00 opened Oct 30, 2026Contributor
CPU lane: IQ1_S rows decode a segment's indices, scales and delta sums in vectors - one token too, the same bits
#695sociolog opened Oct 30, 2026Contributor
1
HIP: EXL3 prompt path on gfx950 matrix cores (MFMA)
#692yagop opened Oct 30, 2026Contributor
DeepSeek: a request that leaves a GPU in a fault ends the engine, and the next request starts a fresh one
#688sociolog opened Oct 30, 2026Contributor

Navigation Menu

Repository navigation

All pull requests

fix(engine): isolate per-request cancellation from concurrent decoding in multi-slot mode (--parallel >= 2)

ds4: add opt-in strict NUMA placement for locked Engram pages

ds4: optionally prepare CPU row checks during model load

MiniMax-M3.1-Flash on Apple Silicon: a Metal engine (maya-metal-mm3)

strata-dense: Qwen3.8-27B support (dense engine, MTP / DFlash2, K8V8 KV)

DeepSeek --tp 2 on CUDA: the exact peer-VRAM prompt all-reduce (MAYA_DS4_HIP_PEER_AR) on NVIDIA, NVLink pairs ~1.3x faster long prompts

docs: add 4x RTX 4090 DeepSeek benchmark with CPU startup patches

DeepSeek expert placement: recent slots that survive per-byte placement (DS4_RECENT_FIT, off by default; rounding-level)

EXL3 CPU kernels, AVX2: the byte-aligned trellis windows by byte shuffles (STRATA_EXL3_AVX2_SHUF, off by default; bit-identical)

EXL3 CPU kernels, AVX2: the codebook products and byte sums in 16-bit lanes for calls of 1-3 activations (DS4_CPU_W16, off by default; bit-identical)

DeepSeek CPU lane: a small EXL3 call's gate / up and down pieces cut by columns too (DS4_CPU_SPLIT_COLS, off by default; bit-identical)

ppc64le: build and run on POWER9 (IBM AC922)

HIP: drain startup peer probes after sampling expires

ds4: add opt-in direct NH copies for RAM-tier prompt flow

ds4: an expert gives the same bits on the CPU lane and in VRAM (MAYA_DS4_LANE_PARITY, off, #644)

ds4: the engine's end sets aside every parallel conversation, not only the bound one

GLM decode: CPU-expert deferral on every part of a split, opt-in (`MAYA_GLM_DEFER_CPU=2`): +7.7% decode on 2x RTX 4060 Ti (+13% with a re-biased router)

A GLM setup run again keeps the config's host, API key, CORS origins and MCP servers

GLM --parallel: keep the conversation checkpoints for each conversation (MAYA_GLM_MULTI_CKPTS, #711)

ds4 GGUF: rebase Q5_K int8 tensor-core GEMVs onto v1.1.42

ds4: tensor-parallel layers with every expert in VRAM can leave the host out too (MAYA_DS4_HOST_FREE_TP=1)

ds4: say when another program holds a GPU's memory, and never plan around it silently

CPU lane: IQ1_S rows decode a segment's indices, scales and delta sums in vectors - one token too, the same bits

HIP: EXL3 prompt path on gfx950 matrix cores (MFMA)

DeepSeek: a request that leaves a GPU in a fault ends the engine, and the next request starts a fresh one

Pull requests