Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
Paper • 2512.03324 • Published • 2
TrimKV
A set of models that can run with bounded memory
-
-
ngocbh/TrimKV-Qwen3-4B-Math
Text Generation • Updated • 49 -
ngocbh/TrimKV-Qwen3-1.7B-Math
Text Generation • Updated • 36 -
ngocbh/TrimKV-Qwen3-4B-Instruct-2507
Text Generation • Updated • 32 -
ngocbh/TrimKV-Phi-3-mini-128k-instruct
Text Generation • Updated • 35 -
ngocbh/TrimKV-Qwen3-8B-Math
Text Generation • Updated • 44 -
ngocbh/TrimKV-Qwen3-14B-Math
Text Generation • Updated • 38 -
ngocbh/TrimKV-DeepSeek-R1-Distill-Llama-8B
Updated • 21 -
Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
Paper • 2605.09649 • Published • 11 -
ngocbh/DBTrimKV-Qwen3-4B-Math
Text Generation • Updated • 53 -
ngocbh/DBTrimKV-Qwen3-4B-Instruct-2507
Text Generation • Updated • 39 -
ngocbh/DBTrimKV-Qwen3-VL-8B-Thinking
Image-Text-to-Text • Updated • 129 -
ngocbh/DBTrimKV-Qwen3-VL-4B-Instruct
Image-Text-to-Text • Updated • 68