Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,7 @@ license: apache-2.0
|
|
| 4 |
|
| 5 |
# AttnVQ — Attention-Aware KV Cache Quantization
|
| 6 |
|
| 7 |
-
Training-free **product vector quantization** of the KV cache for long-context LLMs. AttnVQ fits small per-subspace codebooks with **attention-weighted batched LBG** (centroids weighted by key attention mass from GQA causal attention), but scores distortion by
|
| 8 |
|
| 9 |
Primary target: Laguna-XS.2 (model-agnostic). Only the 10 full-attention layers are compressed; 30 sliding-window layers stay fp16.
|
| 10 |
|
|
|
|
| 4 |
|
| 5 |
# AttnVQ — Attention-Aware KV Cache Quantization
|
| 6 |
|
| 7 |
+
Training-free **product vector quantization** of the KV cache for long-context LLMs. AttnVQ fits small per-subspace codebooks with **attention-weighted batched LBG** (centroids weighted by key attention mass from GQA causal attention), but scores distortion by attention-output error (and key cosine / inner-product bias), not cache MSE. Calibration is light: 10–15 agent traces, ~15 s on GPU — enough to capture the model's K/V geometry (data-aware, not corpus-dependent).
|
| 8 |
|
| 9 |
Primary target: Laguna-XS.2 (model-agnostic). Only the 10 full-attention layers are compressed; 30 sliding-window layers stay fp16.
|
| 10 |
|