File size: 3,562 Bytes
69fc97a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | ---
license: other
library_name: transformers
pipeline_tag: text-generation
tags:
- dflash
- speculative-decoding
- block-diffusion
- draft-model
- efficiency
- minimax
- minimax_m2
- diffusion-language-model
---
# MiniMax-M2.5-DFlash
[**Paper**](https://arxiv.org/abs/2602.06036) | [**GitHub**](https://github.com/z-lab/dflash) | [**Blog**](https://z-lab.ai/projects/dflash/)
**DFlash** is a speculative decoding method that uses a lightweight **block diffusion** model to draft multiple tokens in parallel. This is the drafter model, which must be paired with [MiniMaxAI/MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5).
<div align="center">
<img src="https://huggingface.co/z-lab/gemma-4-31B-it-DFlash/resolve/main/assets/dflash_system.png" alt="DFlash Architecture" width="85%">
</div>
## Quick Start
### Installation
vLLM:
Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
SGLang:
```bash
uv pip install "git+https://github.com/sgl-project/sglang.git#subdirectory=python"
```
### Launch Server
vLLM:
Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
SGLang:
```bash
python -m sglang.launch_server \
--model-path MiniMaxAI/MiniMax-M2.5 \
--tp-size 4 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path z-lab/MiniMax-M2.5-DFlash \
--attention-backend trtllm_mha \
--speculative-draft-attention-backend fa4 \
--mem-fraction-static 0.8 \
--trust-remote-code \
--host 0.0.0.0 \
--port 30000
```
### Usage
For SGLang, use port `30000`.
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="MiniMaxAI/MiniMax-M2.5",
messages=[{"role": "user", "content": "Write a quicksort in Python."}],
max_tokens=4096,
temperature=0.0,
extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(response.choices[0].message.content)
```
## Benchmark Results
**Setup:** 4 NVIDIA B200 GPUs per server/run, SGLang, tensor parallel size 4, target attention backend `trtllm_mha`, draft attention backend `fa4`, thinking enabled, max output length 4096, greedy decoding. Concurrency 1 uses 128 prompts; concurrency 32 uses 1024 prompts.
### Throughput
_Generated tokens/sec_
**Block Size = 8**
| Task | Concurrency | **DFlash** |
|---|---:|---:|
| Math500 | 1 | **355.17** |
| | 32 | **4619.18** |
| GSM8K | 1 | **347.84** |
| | 32 | **4161.22** |
| HumanEval | 1 | **331.03** |
| | 32 | **4329.96** |
| MT-Bench | 1 | **385.45** |
| | 32 | **4658.84** |
### Acceptance Length
| Task | c1 | c32 |
|---|---:|---:|
| Math500 | 4.503 | 4.516 |
| GSM8K | 4.342 | 4.338 |
| HumanEval | 3.923 | 3.979 |
| MT-Bench | 4.382 | 4.184 |
## Acknowledgements
Special thanks to [David Wang](https://davidwa.ng/) for his outstanding engineering support on this project. We are also grateful to [Modal](https://modal.com/), [InnoMatrix](https://innomatrix.ai), and [Yotta Labs](https://www.yottalabs.ai/) for providing the compute resources used to train this draft model.
## Citation
If you find DFlash useful, please cite our work. To share feedback on DFlash or request new model support, please fill out this form: [DFlash Feedback](https://forms.gle/4YNwfqb4nJdqn6hq9).
```bibtex
@article{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
journal = {arXiv preprint arXiv:2602.06036},
year = {2026}
}
```
|