Pinned Loading

1. ome-projects/ome ome-projects/ome Public

   Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

   Go 523 106

2. sgl-project/genai-bench sgl-project/genai-bench Public

   Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.

   Python 335 69

3. cachebox-project/inference-cache cachebox-project/inference-cache Public

   A Kubernetes-native cache plane for LLM inference

   Go 16