h4x3rotab commited on
Commit
e42e1ae
·
verified ·
1 Parent(s): 438e39c

Calibrate MTP layer with runtime-aware AWQ

Browse files

Replace the MTP shard with runtime-aware AWQ calibration from SWE-chat code-generation activations. The 24-session SGLang evaluation improved aggregate throughput by 3.97%.

Files changed (2) hide show
  1. CHANGELOG.md +13 -0
  2. model-00041-of-00041.safetensors +1 -1
CHANGELOG.md ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Changelog
2
+
3
+ ## 2026-07-21
4
+
5
+ Replaced `model-00041-of-00041.safetensors` with a runtime-aware AWQ-calibrated MTP layer. Calibration used code-generation activations from session-disjoint SWE-chat data and selected scales with the deployed SGLang CUTLASS and DeepGEMM kernels.
6
+
7
+ On 24 untouched SWE-chat sessions at concurrency 8 with three speculative steps and four draft tokens:
8
+
9
+ - Draft acceptance increased from 55.42% to 56.57%.
10
+ - Aggregate output throughput increased from 4.890 to 5.084 tokens per second.
11
+ - Average request latency decreased from 88.64 to 84.45 seconds.
12
+
13
+ All other model weights are unchanged.
model-00041-of-00041.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:da5447af3c6c7d1cdbd8fa3a05ffa0e6c4c6f5e983fd04a2d794610b2cafd7aa
3
  size 5349720144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8f0616dbb1d82c16823f9d7f21e5e74bb2e178db271f4bedf2ad19b4e0bebd9
3
  size 5349720144