| --- |
| license: apache-2.0 |
| base_model: netease-youdao/Confucius3-Math |
| tags: |
| - eagle3 |
| - speculative-decoding |
| - draft-model |
| - vllm |
| --- |
| |
| # Confucius3-Math EAGLE3 Draft Head |
|
|
| An [EAGLE3](https://arxiv.org/abs/2503.01840) speculative-decoding draft head for |
| [**netease-youdao/Confucius3-Math**](https://huggingface.co/netease-youdao/Confucius3-Math) |
| (Qwen2.5-14B, R1-style inline-think). Trained with [SpecForge](https://github.com/sgl-project/SpecForge) |
| on the same rollout data as a matched DFlash / D-PACE drafter, for a method comparison. |
|
|
| > ⚠️ **Intermediate checkpoint.** This is `epoch_2 / step_12000` of a planned 6-epoch |
| > (31,734-step) run — roughly **38%** through training. Acceptance length will still |
| > improve with further training. A final checkpoint will replace this when the run completes. |
|
|
| ## Architecture |
|
|
| | | | |
| |---|---| |
| | architecture | `LlamaForCausalLMEagle3` (1 layer) | |
| | hidden size | 5120 | |
| | heads (q/kv) | 40 / 8 | |
| | intermediate | 13824 | |
| | draft vocab | 32000 (with `d2t`/`t2d` mapping to the 152064 target vocab) | |
| | aux hidden layers | `[1, 23, 44]` (SpecForge default) | |
| | dtype | bfloat16 | |
|
|
| ## Training |
|
|
| - SpecForge EAGLE3 defaults: auto-generated draft config, lr 1e-4, warmup 0.015, grad-norm 0.5, TTT length 7. |
| - Copied from the matched DFlash/D-PACE run: same preformatted Confucius3-Math rollout data (148,112 rows), `confucius-r1` chat template, max-length 4096, batch-size 4, 6 epochs. |
|
|
| ## Usage (vLLM) |
|
|
| ```bash |
| vllm serve netease-youdao/Confucius3-Math \ |
| --speculative-config '{"method":"eagle3","model":"noctuashap/Confucius3-Math-Eagle3","num_speculative_tokens":7}' \ |
| --attention-backend flash_attn --trust-remote-code |
| ``` |
|
|
| `num_speculative_tokens` is the draft length — lower it (e.g. 2–3) for high-concurrency / |
| compute-bound serving, raise it (up to ~7) for low-batch / latency-oriented serving. |
|
|
| ## Measured acceptance (gsm8k / math500, thinking mode, vLLM) |
|
|
| At an earlier checkpoint (step_8000) with `num_speculative_tokens=7`: |
| |
| | dataset | accept length | tok/s (single stream) | |
| |---|---|---| |
| | gsm8k | ~4.3 | ~365 | |
| | math500 | ~4.5 | ~386 | |
| |
| (Confucius3-Math is R1-style always-thinking, so thinking and non-thinking modes are equivalent.) |
| |
