aiha-lab

Popular repositories Loading

1. MX-QLLM MX-QLLM Public

   LLM Inference with Microscaling Format

   Python 35 5

2. InfiniPot-V InfiniPot-V Public

   [NeurIPS 25] InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding

   Python 23 1

3. Attention-Head-Pruning Attention-Head-Pruning Public

   Layer-wise Pruning of Transformer Heads for Efficient Language Modeling

   Python 22 1

4. TSLD TSLD Public

   [NeurIPS 2023] Token-Scaled Logit Distillation for Ternary Weight Generative Language Models

   Python 18 1

5. TernGEMM TernGEMM Public

   TernGEMM: General Matrix Multiply Library with Ternary Weights for Fast DNN Inference

   C++ 14 1

6. tangram tangram Public

   A high-throughput LLM serving engine with non-uniform KV cache compression, built on vLLM

   Python 13 1