| --- |
| license: other |
| license_name: muse-spark-open |
| license_link: LICENSE.md |
| private: true |
| --- |
| |
| # Muse-Spark-Open-DSpark |
|
|
| DSpark speculative-decoding draft model for **meta-models/Muse-Spark-Open-816B-A42B** |
| (816B-A42B MoE target, NVFP4). **Built with Muse Spark.** |
|
|
| Trained with [speculators](https://github.com/vllm-project/speculators) from vllm-project. |
|
|
| ## DSpark architecture |
|
|
| - 5-layer decoder-only draft (hidden 8192, 128/16 attention/KV heads, FFN 16384), BF16, |
| causal sliding-window attention (2048) |
| - Full 202,048-token target vocabulary; frozen target LM head |
| - Conditioned on target hidden states tapped at layers [13, 29, 45, 57, 61] plus the |
| final hidden state |
| - Proposes 7 tokens per step (sample-from-anchor); learned Markov confidence head (rank 512) |
| - ~10.6 GB, 74 tensors, safetensors |
|
|
| ## Acceptance rates |
|
|
| Measured against [`meta-models/Muse-Spark-Open-816B-A42B-NVFP4`](https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B-NVFP4)(default chat template, reasoning effort `xhigh`), with the serving config |
| below: 7 probabilistic draft tokens, block verification, temperature 1.0 / top-p 0.95. |
|
|
| | Dataset | Requests | Acceptance | Tokens/cycle | |
| |---|---:|---:|---:| |
| | MT-Bench (chat) | 80 | 28.46% | 2.99 | |
| | GSM8K | 1319 | 42.04% | 3.94 | |
| | MATH-500 | 500 | 39.69% | 3.78 | |
| | HumanEval | 164 | 37.23% | 3.61 | |
| | MBPP | 974 | 35.24% | 3.47 | |
| | AIME26 | 30 | 33.15% | 3.32 | |
| | SWE-bench Pro | 731 | 23.69% | 2.66 | |
| | SPEED-Bench (low entropy) | 512 | 24.76% | 2.73 | |
| | BEAM 100K | 20 | 29.07% | 3.03 | |
| | BEAM 500K | 35 | 29.85% | 3.09 | |
| | BEAM 1M | 35 | 28.55% | 3.00 | |
|
|
| Tokens/cycle is the mean number of tokens emitted per verification step, including the target's |
| bonus token (acceptance length). SPEED-Bench uses the `throughput_16k` low-entropy split with |
| inputs truncated to 10,240 tokens; BEAM requests generate 96 tokens each. |
|
|
| ## Serving |
|
|
| ```json |
| {"method":"dspark","model":"<this repo>","num_speculative_tokens":7, |
| "draft_sample_method":"probabilistic","enable_adaptive_verification":false, |
| "rejection_sample_method":"block"} |
| ``` |
|
|
| Requires a Muse Spark–compatible serving runtime. |
|
|
