Popular repositories Loading

1. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

2. vllm-omni vllm-omni Public

   Forked from vllm-project/vllm-omni

   A framework for efficient model inference with omni-modality models

   Python

3. LMCache LMCache Public

   Forked from LMCache/LMCache

   LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

   Python

4. nano-lmcache nano-lmcache Public

   Forked from Dami-wq-k03/nano-lmcache

   📚 An educational, step-by-step implementation of LLM KV cache reuse — learn prefix caching, chunk keys, storage & eviction, from pure Python to GPU tensors and vLLM integration. Inspired by LMCache.

   Python

5. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python

6. vllm-rlt vllm-rlt Public

   Forked from InferMatrixLab/vllm-rlt

   A vLLM-inspired inference engine for looped transformers, with loop-level scheduling and execution across composable stages

   Python