adirik commited on
Commit
a169aae
·
verified ·
1 Parent(s): 7d67145

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -4,7 +4,7 @@ license: apache-2.0
4
 
5
  # AttnVQ — Attention-Aware KV Cache Quantization
6
 
7
- Training-free **product vector quantization** of the KV cache for long-context LLMs. AttnVQ fits small per-subspace codebooks with **attention-weighted batched LBG** (centroids weighted by key attention mass from GQA causal attention), but scores distortion by **attention-output error** (and key cosine / inner-product bias), not cache MSE. Calibration is light: **10–15 agent traces, ~15 s on GPU** — enough to capture the **model's** K/V geometry (data-aware, not corpus-dependent).
8
 
9
  Primary target: Laguna-XS.2 (model-agnostic). Only the 10 full-attention layers are compressed; 30 sliding-window layers stay fp16.
10
 
 
4
 
5
  # AttnVQ — Attention-Aware KV Cache Quantization
6
 
7
+ Training-free **product vector quantization** of the KV cache for long-context LLMs. AttnVQ fits small per-subspace codebooks with **attention-weighted batched LBG** (centroids weighted by key attention mass from GQA causal attention), but scores distortion by attention-output error (and key cosine / inner-product bias), not cache MSE. Calibration is light: 10–15 agent traces, ~15 s on GPU — enough to capture the model's K/V geometry (data-aware, not corpus-dependent).
8
 
9
  Primary target: Laguna-XS.2 (model-agnostic). Only the 10 full-attention layers are compressed; 30 sliding-window layers stay fp16.
10