Zhongzhu commited on
Commit
35c854c
·
verified ·
1 Parent(s): b1f6435

Add Qwen3-30B-A3B OSCAR KV rotations

Browse files
Files changed (1) hide show
  1. Qwen3-30B-A3B/README.md +6 -2
Qwen3-30B-A3B/README.md CHANGED
@@ -9,14 +9,18 @@ near-orthogonal rotations (`mean |diag(R0^T R1)| ~= 0.07`).
9
  | `k_rotation_perhead.pt`, `v_rotation_perhead.pt` | `format_version: 2` | `rotation: [4, 128, 128]` per layer, 48 layers -- **use these** |
10
  | `k_rotation_qqt_r_h_pbr.pt`, `v_rotation_sst_r_h_pbr.pt` | `format_version: 1` | one shared `[128, 128]` per layer -- for comparison only |
11
 
12
- 3 seeds, TP=2, 32K generation:
13
 
14
  | Benchmark | BF16 | INT2 shared | INT2 per-head |
15
  |---|:---:|:---:|:---:|
16
  | GPQA | 60.8 +- 3.2 | 43.9 +- 2.8 | **58.6 +- 0.9** |
17
  | HumanEval | 90.9 +- 2.1 | 25.8 +- 2.5 | **89.0 +- 1.2** |
18
- | AIME 2025 | 70.0 +- 6.7 | 18.9 +- 10.7 | **65.6 +- 1.9** |
19
  | MATH500 | 93.6 +- 0.9 | 60.8 +- 2.2 | **92.1 +- 0.4** |
20
 
 
 
 
 
21
  Serving per-head needs a runtime that reads `format_version: 2`; the pool logs
22
  `[per-head: 4 kv heads]` at startup when it is active.
 
9
  | `k_rotation_perhead.pt`, `v_rotation_perhead.pt` | `format_version: 2` | `rotation: [4, 128, 128]` per layer, 48 layers -- **use these** |
10
  | `k_rotation_qqt_r_h_pbr.pt`, `v_rotation_sst_r_h_pbr.pt` | `format_version: 1` | one shared `[128, 128]` per layer -- for comparison only |
11
 
12
+ TP=2, 32K generation; 3 seeds except AIME (8 seeds):
13
 
14
  | Benchmark | BF16 | INT2 shared | INT2 per-head |
15
  |---|:---:|:---:|:---:|
16
  | GPQA | 60.8 +- 3.2 | 43.9 +- 2.8 | **58.6 +- 0.9** |
17
  | HumanEval | 90.9 +- 2.1 | 25.8 +- 2.5 | **89.0 +- 1.2** |
18
+ | AIME 2025 (8 seeds) | 72.5 +- 4.6 | 14.2 +- 7.1 | **66.2 +- 3.8** |
19
  | MATH500 | 93.6 +- 0.9 | 60.8 +- 2.2 | **92.1 +- 0.4** |
20
 
21
+ AIME is the one benchmark where per-head still loses measurably (-6.3, Welch
22
+ t=2.96, p~0.01 over 8 seeds); the other three sit within 1.5-2.2 of BF16. Long
23
+ chain-of-thought math is the hardest case for INT2 KV.
24
+
25
  Serving per-head needs a runtime that reads `format_version: 2`; the pool logs
26
  `[per-head: 4 kv heads]` at startup when it is active.