Add speculators training note
Browse files
README.md
CHANGED
|
@@ -10,6 +10,8 @@ private: true
|
|
| 10 |
DSpark speculative-decoding draft model for **meta-models/Muse-Spark-Open-816B-A42B**
|
| 11 |
(816B-A42B MoE target, NVFP4). **Built with Muse Spark.**
|
| 12 |
|
|
|
|
|
|
|
| 13 |
## DSpark architecture
|
| 14 |
|
| 15 |
- 5-layer decoder-only draft (hidden 8192, 128/16 attention/KV heads, FFN 16384), BF16,
|
|
|
|
| 10 |
DSpark speculative-decoding draft model for **meta-models/Muse-Spark-Open-816B-A42B**
|
| 11 |
(816B-A42B MoE target, NVFP4). **Built with Muse Spark.**
|
| 12 |
|
| 13 |
+
Trained with [speculators](https://github.com/vllm-project/speculators) from vllm-project.
|
| 14 |
+
|
| 15 |
## DSpark architecture
|
| 16 |
|
| 17 |
- 5-layer decoder-only draft (hidden 8192, 128/16 attention/KV heads, FFN 16384), BF16,
|