SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
Paper • 2610.02953 • Published
SlimKV
SlimKV (EMNLP 2026 Oral): joint token-feature KV compression with reconstruction-free decoding. r2/r4/r8 = 2×/4×/8× feature compression.
Note SlimKV learns layer-adaptive, low-rank Beacon memories and enables reconstruction-free latent-space attention by removing Beacon K-RoPE. r2/r4/r8 denote 2×/4×/8× feature compression; token compression is configurable. Nominal total compression = token × feature compression. At 128K, t4f4 achieves 7.34× attention and 3.38× end-to-end speedups on Llama-3.1-8B (A100 80GB, BF16 eager). Code and usage: https://github.com/Strandding/SlimKV
Zihant0513/SlimKV-Llama3.1-8B-r2
Text Generation • 9B • Updated • 412Note 2× feature compression · SlimKV-f2
Zihant0513/SlimKV-Llama3.1-8B-r4
Text Generation • 9B • Updated • 470Note 4× feature compression · SlimKV-f4
Zihant0513/SlimKV-Llama3.1-8B-r8
Text Generation • 9B • Updated • 481Note 8× feature compression · SlimKV-f8
Zihant0513/SlimKV-Qwen2.5-14B-r2
Text Generation • 16B • Updated • 560Note 2× feature compression · SlimKV-f2
Zihant0513/SlimKV-Qwen2.5-14B-r4
Text Generation • 16B • Updated • 675Note 4× feature compression · SlimKV-f4
Zihant0513/SlimKV-Qwen2.5-14B-r8
Text Generation • 16B • Updated • 715 • 1Note 8× feature compression · SlimKV-f8