h4x3rotab commited on
Commit
3cd8b80
·
verified ·
1 Parent(s): d975c19

Model card: trim the 2026-06-19 fix note to the essentials

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -35,7 +35,7 @@ Two things make it useful:
35
  **Tested with SGLang only** ([v0.5.13.post1](https://github.com/sgl-project/sglang)). The 4-bit layout and speculative-decoding path are SGLang-specific and untried on vLLM or other engines.
36
 
37
  > [!IMPORTANT]
38
- > **Update (2026-06-19): speculative-decoding (MTP) fix.** Revisions of this repo before this date shipped a `model.safetensors.index.json` that did **not** reference the MTP / next-token-prediction layer shard (`model-00041-of-00041.safetensors`). The shard was present in the repo but **unindexed**, so SGLang's EAGLE speculative decoding loaded an uninitialized draft head and accepted almost nothing (~0.1% acceptance — **no speedup**). Outputs were always correct regardless, because the target model verifies every drafted token. This is now fixed: the index references the full MTP layer (layer 78 — 256 experts + `eh_proj`/`enorm`/`hnorm`/`shared_head`), and speculative decoding reaches **~80%+ draft acceptance**. **If you downloaded the repo before 2026-06-19, re-pull `model.safetensors.index.json`** — it is the only file that changed; the weights were already present.
39
 
40
  ## Benchmarks
41
 
 
35
  **Tested with SGLang only** ([v0.5.13.post1](https://github.com/sgl-project/sglang)). The 4-bit layout and speculative-decoding path are SGLang-specific and untried on vLLM or other engines.
36
 
37
  > [!IMPORTANT]
38
+ > **2026-06-19 fix:** if you downloaded this repo before this date, re-download `model.safetensors.index.json` it's the only changed file, and it enables EAGLE/MTP speculative decoding (previously a no-op). The weights are unchanged.
39
 
40
  ## Benchmarks
41