| --- |
| license: mit |
| base_model: |
| - Inferact/MiniMax-M3-EAGLE3 |
| tags: |
| - eagle3 |
| - speculative-decoding |
| - compressed-tensors |
| - int4 |
| --- |
| |
| # MiniMax-M3 EAGLE3 drafter — RTN INT4 |
|
|
| EAGLE3 speculative-decoding draft head for |
| [Sebesky/MiniMax-M3-W4A16-GPTQ](https://huggingface.co/Sebesky/MiniMax-M3-W4A16-GPTQ): |
| a single Llama-style decoder layer (hidden 6144, 64-head full MHA, head_dim 128) |
| with its **own** `embed_tokens` and `lm_head` (vocab 200,064) — not shared with |
| the target. Everything is RTN-quantized to int4 (compressed-tensors |
| `pack-quantized`, group 128): Linear, embedding, and lm_head. ~1.6 GB. |
|
|
| ## Usage (vLLM) |
|
|
| ``` |
| --speculative-config '{"method":"eagle3","model":"Sebesky/MiniMax-M3-EAGLE3-RTN-INT4","num_speculative_tokens":3}' |
| ``` |
|
|
| Validated on 2x DGX Spark (GB10) at TP2 with `draft_tensor_parallel_size: 2`: |
| acceptance length 3.36 (78.6% draft acceptance) at 120k-token context, |
| decode 39.7 / 30.4 / 26.4 tok/s @512/65k/120k with 4-bit KV quantization. |
