kv-cache-offloading

Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding.

QSA HiSparse as an SGLang plugin: 256K CPU KV offload and text/image host prefix reuse for Qwen Sparse Attention on unmodified SGLang

Persistent KV cache sessions for LLM serving on edge and on-premises hardware: restore the KV cache instead of recomputing it, lossless long-context overflow archive, cross-engine sessions (llama.cpp, vLLM, SGLang). IEEE Access 2026.