Add calibrated KV-cache scales (51 F32 scalars, weights unchanged)

model-kvscales.safetensors: {q,k,v}_scale for the 16 full-attention layers + the MTP draft.
Captured at --kv-cache-dtype auto on Attention.forward (post-norm, post-RoPE) over 1.4M tokens
/ 170,905 calls; scale = amax*1.1/{200,200,100}. Weight shards are byte-identical -- only the
index gains 51 entries. Inert under bf16 KV; active only with --kv-cache-dtype fp8.
Largest amax is 147 vs e4m3's 448, so scale-1.0 was never clipping: this is hygiene, not a
quality fix. Vision tower not calibrated (text-only corpus); no accuracy re-run yet.

README.md

@@ -123,6 +123,52 @@ branch `rdna4-port-0.26.1`. `VLLM_RDNA_MXFP4_FP8=0` falls back to the weight-onl
 Sampling follows the base model card: thinking `temp 1.0, top_p 0.95, top_k 20, min_p 0`;
 non-thinking `temp 0.7, top_p 0.8, top_k 20, presence_penalty 1.5`.
 
+## KV-cache scales (added 2026-08-18)
+
+The weight shards are **unchanged**. This repo additionally carries **51 F32 KV-cache scalars** in
+`model-kvscales.safetensors` (5,708 bytes) plus matching `model.safetensors.index.json` entries —
+`{q,k,v}_scale` for each of the 16 full-attention layers and for the MTP draft layer.
+
+**They do nothing unless you serve with `--kv-cache-dtype fp8`.** Under the default bf16 KV cache —
+which is every number elsewhere on this card — they load and sit unused. Without them vLLM falls back
+to `k_scale = v_scale = q_scale = 1.0` and logs `Using KV cache scaling factor 1.0 for fp8_e4m3`:
+correct bytes, wrong numbers. With them, that line is **absent**, and its absence is the only reliable
+proof they loaded — `.q_scale/.k_scale/.v_scale` are in vLLM's `_ignore_unexpected_suffixes`, so a
+mis-named scale is dropped silently instead of erroring.
+
+**How they were captured.** Served at `--kv-cache-dtype auto` with a recorder on `Attention.forward`
+taking a running amax of q/k/v *as the cache sees them* — post `q_norm`/`k_norm`, post-RoPE — over
+**1.4M tokens / 170,905 attention calls**: 354 requests spanning long documents (8k–96k), multi-subject
+QA, maths, code, instruction-following and a Spanish business slice, plus production traffic. Both
+tensor-parallel ranks recorded, per-layer maximum taken. `scale = amax × 1.1 / {200, 200, 100}` for
+q/k/v, matching vLLM's `Q/K/V_SCALE_CONSTANT`.
+
+| | q | k | v |
+|---|---|---|---|
+| range over the 16 attention layers | 11.50 – 19.50 | 12.19 – 23.12 | 8.44 – **147.00** |
+| MTP draft layer | 19.50 | 20.88 | 51.50 |
+
+`v` grows monotonically with depth (12.06 at layer 3 → 147.00 at layer 63).
+
+**What this is worth — read this before assuming it is an upgrade.** The largest amax anywhere is
+**147 against e4m3's maximum of 448**, so the uncalibrated cache was never *clipping*. The scales shift
+where the distribution sits rather than rescue it, and on a float format that is a small effect —
+consistent with the published finding of ≤0.7 points from an uncalibrated fp8 KV cache on a model of
+this class. Treat this as hygiene that closes a question, not as a quality fix.
+
+**Not measured, not covered:**
+
+* **No accuracy re-run of the calibrated build** (GSM8K / long-context / task suites). The numbers under
+  *Quality* below are the bf16-KV build and are unaffected by this addition.
+* **The vision tower is not calibrated** — the capture corpus is text-only.
+* **No `prob_scale` is shipped.** No v1 ROCm attention backend consumes one, so vLLM's
+  `…and/or prob_scale 1.0 with fp8 attention` warning still fires and can be ignored.
+* **fp8 KV doubles the attention block on this architecture.** vLLM aligns the attention page to the
+  mamba page, so halving the bytes per token pushes the block from 800 to 1600 tokens, which coarsens
+  prefix-cache reuse: only whole blocks are reused. If your requests share a long prefix, pad it to a
+  multiple of the block size — otherwise the remainder is recomputed on every request. Measured here on
+  a 7,299-token prompt: 99 tokens recomputed at block 800 vs 899 at block 1600.
+
 ## What it took to run this on RDNA4
 
 These weights are the small half of the work. RDNA4 (gfx1200/gfx1201) is outside the official ROCm
@@ -335,6 +381,8 @@ will be revised as they land; the commit history is the record of what was known
 ## Reproducing the quantisation
 
 Data-free, CPU-only, file-to-file — no calibration set, no GPU, ~3 minutes for this model.
+(This describes the **weights**. The KV-cache scalars above are the one calibrated artefact in the
+repo and came from a separate capture pass on the served model; they do not touch the shards.)
 
 ```python
 from quark.torch.export.api import direct_quantize_checkpoint

model-kvscales.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8e738b76c92943bcccbccf2f204e0d327caff970fabc889a6daee6e01107aaa
+size 5708