File size: 4,326 Bytes
1ef5392 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | ---
base_model: z-lab/MiniMax-M2.7-DFlash
tags:
- transformers
- safetensors
- qwen3
- dflash
- speculative-decoding
- block-diffusion
- draft-model
- efficiency
- minimax
- minimax_m2
- diffusion-language-model
- text-generation
- arxiv:2602.06036
- license:other
- text-generation-inference
- endpoints_compatible
- region:us
---
# MiniMax M2.7 DFlash GGUF
GGUF quantizations of [**z-lab DFlash draft model**](https://huggingface.co/z-lab/MiniMax-M2.7-DFlash) for [**MiniMax M2.7**](https://huggingface.co/MiniMaxAI/MiniMax-M2.7).
Use with [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp), a llama.cpp fork with advanced quantization features.
> **Conversion note:** The gated z-lab checkpoint was not downloadable during conversion. These GGUFs were converted from the compatible public [NVIDIA/MiniMax-M2.7-DFlash](https://huggingface.co/nvidia/MiniMax-M2.7-DFlash) checkpoint.
---
# MiniMax-M2.7-DFlash
[**Paper**](https://arxiv.org/abs/2602.06036) | [**GitHub**](https://github.com/z-lab/dflash) | [**Blog**](https://z-lab.ai/projects/dflash/)
**DFlash** is a speculative decoding method that uses a lightweight **block diffusion** model to draft multiple tokens in parallel. This is the drafter model, which must be paired with [MiniMaxAI/MiniMax-M2.7](https://huggingface.co/MiniMaxAI/MiniMax-M2.7).
<div align="center">
<img src="assets/dflash_system.png" alt="DFlash Architecture" width="85%">
</div>
## Quick Start
### Installation
vLLM:
Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
SGLang:
```bash
uv pip install "git+https://github.com/sgl-project/sglang.git#subdirectory=python"
```
### Launch Server
vLLM:
Check out [vLLM issue #46105](https://github.com/vllm-project/vllm/issues/46105).
SGLang:
```bash
python -m sglang.launch_server \
--model-path MiniMaxAI/MiniMax-M2.7 \
--tp-size 4 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path z-lab/MiniMax-M2.7-DFlash \
--attention-backend trtllm_mha \
--speculative-draft-attention-backend fa4 \
--mem-fraction-static 0.8 \
--trust-remote-code \
--host 0.0.0.0 \
--port 30000
```
### Usage
For SGLang, use port `30000`.
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="MiniMaxAI/MiniMax-M2.7",
messages=[{"role": "user", "content": "Write a quicksort in Python."}],
max_tokens=4096,
temperature=0.0,
extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(response.choices[0].message.content)
```
## Benchmark Results
**Setup:** 4 NVIDIA B200 GPUs per server/run, SGLang, tensor parallel size 4, target attention backend `trtllm_mha`, draft attention backend `fa4`, thinking enabled, max output length 4096, greedy decoding. Concurrency 1 uses 128 prompts; concurrency 32 uses 1024 prompts.
### Throughput
_Generated tokens/sec_
**Block Size = 8**
| Task | Concurrency | **DFlash** |
|---|---:|---:|
| Math500 | 1 | **331.12** |
| | 32 | **4422.52** |
| GSM8K | 1 | **304.07** |
| | 32 | **4202.09** |
| HumanEval | 1 | **333.44** |
| | 32 | **4394.23** |
| MT-Bench | 1 | **350.84** |
| | 32 | **4549.75** |
### Acceptance Length
| Task | c1 | c32 |
|---|---:|---:|
| Math500 | 3.561 | 3.658 |
| GSM8K | 3.481 | 3.586 |
| HumanEval | 3.610 | 3.657 |
| MT-Bench | 3.550 | 3.624 |
## Acknowledgements
Special thanks to [David Wang](https://davidwa.ng/) for his outstanding engineering support on this project. We are also grateful to [Modal](https://modal.com/), [InnoMatrix](https://innomatrix.ai), and [Yotta Labs](https://www.yottalabs.ai/) for providing the compute resources used to train this draft model.
## Citation
If you find DFlash useful, please cite our work. To share feedback on DFlash or request new model support, please fill out this form: [DFlash Feedback](https://forms.gle/4YNwfqb4nJdqn6hq9).
```bibtex
@article{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
journal = {arXiv preprint arXiv:2602.06036},
year = {2026}
}
```
|