Minimax-M3.1-Flash-Dflash2

Overview

A DFlash2 draft for MiniMax M3.1 Flash that proposes tokens in parallel for verification by the target model in SGLang.

This repository provides the BF16 checkpoint and its configuration. Serving also requires the target model, which provides the embedding, output head, tokenizer, and native chat template.

Model Specifications

• Base model: MiniMaxAI/MiniMax-M3.1-Flash
• Architecture: MiniMaxDFlash2DraftModel, 5 GQA layers
• Parameters: 2,054,677,760 across 81 BF16 tensors
• Format: One BF16 Safetensors file
• Hidden size: 6,144; 64 query heads and 4 KV heads
• Attention: Sliding window of 4,096 tokens
• Verification width: 8 tokens, comprising an anchor and 7 proposals
• Selector: Rank 256, top-k 16
• Auxiliary target layers: [3, 17, 31, 45, 59]
• Configured maximum positions: 1,048,576

The evaluations below used a 16,384-token serving context. They do not establish acceptance quality at the full configured context length.

Evaluation Results

acc_len is the unweighted mean of each request's completion-token count divided by its number of verification steps. It includes responses that reached the output cap.

Dataset	Samples	acc_len
GSM8K	1,319	5.163813
AIME26	30	4.309412
MBPP	257	4.383587
MT-Bench	160 turns (80 conversations)	3.580651

Measured with temperature=1.0, top_p=0.95, top_k=-1, and native adaptive thinking. The evaluation used TP4 on GB300, a native NVFP4 target with TRTLLM MHA and FP8 KV cache, and a BF16 draft with FA4 and BF16 KV cache. DFLASH verified 8 tokens per step.

GSM8K, AIME26, and MBPP used concurrency 16 and an 8,192-token output cap; MT-Bench used concurrency 2 and a 2,048-token cap. The cap was reached by 3, 8, 3, and 54 responses, respectively. MT-Bench's second turn used the model's generated first-turn history. Acceptance length measures decoding behavior, rather than answer quality or throughput.

Serving with SGLang

The following cookbook and example show how to serve MiniMax M3.1 Flash with this DFlash2 draft using SGLang:

SGLANG_RAGGED_VERIFY_MODE=static \
FMHA_SM100_ALLOW_JIT=1 \
SGLANG_OPT_USE_BF16_ROUTER_GEMM=0 \
SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=32768 \
python -m sglang.launch_server \
  --model-path MiniMaxAI/MiniMax-M3.1-Flash \
  --trust-remote-code \
  --reasoning-parser auto \
  --tool-call-parser auto \
  --tp-size 4 \
  --dp-size 1 \
  --ep-size 1 \
  --context-length 16384 \
  --chunked-prefill-size 2048 \
  --max-prefill-tokens 65536 \
  --max-running-requests 16 \
  --cuda-graph-max-bs-decode 16 \
  --cuda-graph-backend-prefill disabled \
  --mem-fraction-static 0.8 \
  --max-total-tokens 131072 \
  --moe-runner-backend flashinfer_trtllm_routed \
  --moe-a2a-backend none \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path RadixArk/Minimax-M3.1-Flash-Dflash2 \
  --speculative-draft-attention-backend fa4 \
  --speculative-draft-kv-cache-dtype bf16 \
  --speculative-draft-model-quantization unquant \
  --speculative-num-draft-tokens 8 \
  --random-seed 42 \
  --host 127.0.0.1 \
  --port 30000

SGLang selects the target's native NVFP4 loading, TRTLLM MHA attention, and FP8 KV cache automatically. Use the target's native chat template and adaptive thinking to match the evaluation settings.