jianchen0311 commited on
Commit
69fc97a
·
verified ·
1 Parent(s): bcf6644

Update DFlash model card

Browse files
Files changed (1) hide show
  1. README.md +126 -0
README.md ADDED
@@ -0,0 +1,126 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: transformers
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - dflash
7
+ - speculative-decoding
8
+ - block-diffusion
9
+ - draft-model
10
+ - efficiency
11
+ - minimax
12
+ - minimax_m2
13
+ - diffusion-language-model
14
+ ---
15
+
16
+ # MiniMax-M2.5-DFlash
17
+
18
+ [**Paper**](https://arxiv.org/abs/2602.06036) | [**GitHub**](https://github.com/z-lab/dflash) | [**Blog**](https://z-lab.ai/projects/dflash/)
19
+
20
+ **DFlash** is a speculative decoding method that uses a lightweight **block diffusion** model to draft multiple tokens in parallel. This is the drafter model, which must be paired with [MiniMaxAI/MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5).
21
+
22
+ <div align="center">
23
+ <img src="https://huggingface.co/z-lab/gemma-4-31B-it-DFlash/resolve/main/assets/dflash_system.png" alt="DFlash Architecture" width="85%">
24
+ </div>
25
+
26
+ ## Quick Start
27
+
28
+ ### Installation
29
+
30
+ vLLM:
31
+
32
+ Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
33
+
34
+ SGLang:
35
+
36
+ ```bash
37
+ uv pip install "git+https://github.com/sgl-project/sglang.git#subdirectory=python"
38
+ ```
39
+
40
+ ### Launch Server
41
+
42
+ vLLM:
43
+
44
+ Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
45
+
46
+ SGLang:
47
+
48
+ ```bash
49
+ python -m sglang.launch_server \
50
+ --model-path MiniMaxAI/MiniMax-M2.5 \
51
+ --tp-size 4 \
52
+ --speculative-algorithm DFLASH \
53
+ --speculative-draft-model-path z-lab/MiniMax-M2.5-DFlash \
54
+ --attention-backend trtllm_mha \
55
+ --speculative-draft-attention-backend fa4 \
56
+ --mem-fraction-static 0.8 \
57
+ --trust-remote-code \
58
+ --host 0.0.0.0 \
59
+ --port 30000
60
+ ```
61
+
62
+ ### Usage
63
+
64
+ For SGLang, use port `30000`.
65
+
66
+ ```python
67
+ from openai import OpenAI
68
+
69
+ client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
70
+
71
+ response = client.chat.completions.create(
72
+ model="MiniMaxAI/MiniMax-M2.5",
73
+ messages=[{"role": "user", "content": "Write a quicksort in Python."}],
74
+ max_tokens=4096,
75
+ temperature=0.0,
76
+ extra_body={"chat_template_kwargs": {"enable_thinking": True}},
77
+ )
78
+ print(response.choices[0].message.content)
79
+ ```
80
+
81
+ ## Benchmark Results
82
+
83
+ **Setup:** 4 NVIDIA B200 GPUs per server/run, SGLang, tensor parallel size 4, target attention backend `trtllm_mha`, draft attention backend `fa4`, thinking enabled, max output length 4096, greedy decoding. Concurrency 1 uses 128 prompts; concurrency 32 uses 1024 prompts.
84
+
85
+ ### Throughput
86
+
87
+ _Generated tokens/sec_
88
+
89
+ **Block Size = 8**
90
+
91
+ | Task | Concurrency | **DFlash** |
92
+ |---|---:|---:|
93
+ | Math500 | 1 | **355.17** |
94
+ | | 32 | **4619.18** |
95
+ | GSM8K | 1 | **347.84** |
96
+ | | 32 | **4161.22** |
97
+ | HumanEval | 1 | **331.03** |
98
+ | | 32 | **4329.96** |
99
+ | MT-Bench | 1 | **385.45** |
100
+ | | 32 | **4658.84** |
101
+
102
+ ### Acceptance Length
103
+
104
+ | Task | c1 | c32 |
105
+ |---|---:|---:|
106
+ | Math500 | 4.503 | 4.516 |
107
+ | GSM8K | 4.342 | 4.338 |
108
+ | HumanEval | 3.923 | 3.979 |
109
+ | MT-Bench | 4.382 | 4.184 |
110
+
111
+ ## Acknowledgements
112
+
113
+ Special thanks to [David Wang](https://davidwa.ng/) for his outstanding engineering support on this project. We are also grateful to [Modal](https://modal.com/), [InnoMatrix](https://innomatrix.ai), and [Yotta Labs](https://www.yottalabs.ai/) for providing the compute resources used to train this draft model.
114
+
115
+ ## Citation
116
+
117
+ If you find DFlash useful, please cite our work. To share feedback on DFlash or request new model support, please fill out this form: [DFlash Feedback](https://forms.gle/4YNwfqb4nJdqn6hq9).
118
+
119
+ ```bibtex
120
+ @article{chen2026dflash,
121
+ title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
122
+ author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
123
+ journal = {arXiv preprint arXiv:2602.06036},
124
+ year = {2026}
125
+ }
126
+ ```