---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model:
  - openbmb/MiniCPM5-2B
inference: false
tags:
  - dflash2
  - block-diffusion
  - speculative-decoding
  - draft-model
  - sglang
  - vllm
  - minicpm
---

# MiniCPM5-2B-DFlash2

A community **DFlash 2** block-diffusion drafter for [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B),
trained by self-distillation with the recipe of [DFlash 2](https://inco.ai/blog/dflash2/)
([`z-lab/dflash`](https://github.com/z-lab/dflash)). It is **not** a standalone language model: it drafts blocks of
tokens for the frozen target to verify inside a speculative-decoding server. Decoding is lossless — greedy output
matches the target exactly, and sampling preserves its distribution.

## Architecture

| | |
|---|---|
| draft backbone | 5 qwen3-style decoder layers, hidden 2048, GQA |
| block size | 8 (7 drafted tokens per verify step) |
| target taps | layers [1, 10, 20, 30, 39] of 42, fused by a linear + RMSNorm |
| block diffusion | bidirectional attention inside the block (`is_causal: false`), two-tap dynamic grouped convs (k=2, groups=16) |
| path selector | low-rank Markov selector, rank 256, top-k 16 |
| mask token | `75982` (the same unused id OpenBMB's official `MiniCPM5-2B-DSpark` uses) |
| embeddings / LM head | **not stored** — reused from the target at runtime |
| parameters | ~335M (0.67 GB bf16) |

## Training (this repo's `dflash2_training_meta.json` has the raw settings)

- **Method:** target frozen; forward-KL distillation of the draft's block logits against the target's teacher-forced
  distribution (+ CE anchor weight 0.1), and the candidate-path selector trained toward the greedy-acceptance
  answer with teacher-forced predecessor chains. **4 blocks trained per row** with cache-streamed
  full-prefix contexts — the exact cache discipline of the DFlash2 inference loop, so training and serving match.
- **Data:** two-phase mixture — phase 1 (general): HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k, ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, HuggingFaceFW/fineweb-edu; phase 2 (code/agentic
  anneal): ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, bigcode/self-oss-instruct-sc2-exec-filter-50k, HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k. Chat formatted with the target's template, packed into
  1536-token rows, 4600 steps at an effective batch of 8 rows.
- **Hardware:** Tesla T4, fp16/bf16 autocast + GradScaler.
- **Validation mode:** no — full release recipe.

## Measured speed (greedy, batch 1, 6 prompts/domain, this notebook's run)

| domain | plain tok/s | spec tok/s | speedup | proj | acceptance length | greedy-identical |
|---|---:|---:|---:|---:|---:|---:|
| coding | 17.81 | 19.36 | **1.087x** | 1.005x | 1.234 | 6/6 |
| agentic | 17.98 | 19.94 | **1.109x** | 1.032x | 1.267 | 5/6 |
| math | 17.91 | 19.28 | **1.076x** | 0.998x | 1.226 | 5/6 |
| general | 17.9 | 19.26 | **1.076x** | 1.0x | 1.228 | 5/6 |
| **overall** | 17.89 | 19.42 | **1.086x** | 1.006x | 1.236 | 21/24 |

Measured per-cycle costs: target decode 57.714 ms/tok, target verify(8) 55.433 ms, draft block 15.442 ms. The **proj** column is AL * t_decode / (t_verify8 + t_draft) — what an engine-grade serving stack (SGLang/vLLM) should approach; the measured speedup additionally carries the python-side loop overhead of this eval harness.

Speed targets: coding >= 2.0x, agentic >= 2.0x, overall >= 1.5x — not met within caps (measured overall 1.086x).
Decoding is lossless by construction: the target verifies every drafted token, so greedy output is token-identical to
plain decoding (the identity rate above is limited only by bf16 near-tie argmax flips) and sampling preserves the
target distribution. The published DFlash 2 drafters report ~4-5.5 acceptance length on Qwen3.8-27B with full-scale
training.

## Serving

**SGLang** (needs the DFLASH support build, see [`z-lab/dflash`](https://github.com/z-lab/dflash)):

```bash
python -m sglang.launch_server   --model-path openbmb/MiniCPM5-2B   --speculative-algorithm DFLASH   --speculative-draft-model-path applepie69/MiniCPM5-2B-DFlash2   --speculative-num-draft-tokens 8
```

**vLLM:**

```bash
vllm serve openbmb/MiniCPM5-2B   --speculative-config '{"method": "dflash", "model": "applepie69/MiniCPM5-2B-DFlash2", "num_speculative_tokens": 7}'
```

**Transformers / offline eval** with the `dflash` package:

```python
from dflash.model import DFlash2DraftModel, dflash_generate
draft = DFlash2DraftModel.from_pretrained("applepie69/MiniCPM5-2B-DFlash2")
out = dflash_generate(draft, target_model, input_ids, max_new_tokens=256,
                      stop_token_ids=[1, 130073], temperature=0.0)
```

## Files

| file | what |
|---|---|
| `config.json` | DFlash2 draft config (`dflash_config` block) |
| `model.safetensors` | the trained drafter (bf16) |
| `dflash2_training_meta.json` | training settings + measured results |
| `measured_results.json` | per-domain speed/acceptance measurements + gate status |

## License

Apache-2.0, matching the base model. Check the base model's terms before use.