Instructions to use applepie69/MiniCPM5-2B-DFlash2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use applepie69/MiniCPM5-2B-DFlash2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="applepie69/MiniCPM5-2B-DFlash2")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("applepie69/MiniCPM5-2B-DFlash2")
model = AutoModel.from_pretrained("applepie69/MiniCPM5-2B-DFlash2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use applepie69/MiniCPM5-2B-DFlash2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "applepie69/MiniCPM5-2B-DFlash2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applepie69/MiniCPM5-2B-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/applepie69/MiniCPM5-2B-DFlash2

SGLang

How to use applepie69/MiniCPM5-2B-DFlash2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "applepie69/MiniCPM5-2B-DFlash2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applepie69/MiniCPM5-2B-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "applepie69/MiniCPM5-2B-DFlash2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "applepie69/MiniCPM5-2B-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use applepie69/MiniCPM5-2B-DFlash2 with Docker Model Runner:
```
docker model run hf.co/applepie69/MiniCPM5-2B-DFlash2
```

MiniCPM5-2B-DFlash2 / README.md

applepie69

DFlash2 drafter for openbmb/MiniCPM5-2B (overall 1.086x, coding 1.087x, agentic 1.109x, 4600 steps)

c93e263 verified about 21 hours ago

preview code

Raw History Blame Contribute Delete

5.16 kB

metadata

license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model:
  - openbmb/MiniCPM5-2B
inference: false
tags:
  - dflash2
  - block-diffusion
  - speculative-decoding
  - draft-model
  - sglang
  - vllm
  - minicpm

MiniCPM5-2B-DFlash2

A community DFlash 2 block-diffusion drafter for openbmb/MiniCPM5-2B, trained by self-distillation with the recipe of DFlash 2 (z-lab/dflash). It is not a standalone language model: it drafts blocks of tokens for the frozen target to verify inside a speculative-decoding server. Decoding is lossless — greedy output matches the target exactly, and sampling preserves its distribution.

Architecture


draft backbone	5 qwen3-style decoder layers, hidden 2048, GQA
block size	8 (7 drafted tokens per verify step)
target taps	layers [1, 10, 20, 30, 39] of 42, fused by a linear + RMSNorm
block diffusion	bidirectional attention inside the block (`is_causal: false`), two-tap dynamic grouped convs (k=2, groups=16)
path selector	low-rank Markov selector, rank 256, top-k 16
mask token	`75982` (the same unused id OpenBMB's official `MiniCPM5-2B-DSpark` uses)
embeddings / LM head	not stored — reused from the target at runtime
parameters	~335M (0.67 GB bf16)

Training (this repo's `dflash2_training_meta.json` has the raw settings)

Method: target frozen; forward-KL distillation of the draft's block logits against the target's teacher-forced distribution (+ CE anchor weight 0.1), and the candidate-path selector trained toward the greedy-acceptance answer with teacher-forced predecessor chains. 4 blocks trained per row with cache-streamed full-prefix contexts — the exact cache discipline of the DFlash2 inference loop, so training and serving match.
Data: two-phase mixture — phase 1 (general): HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k, ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, HuggingFaceFW/fineweb-edu; phase 2 (code/agentic anneal): ise-uiuc/Magicoder-Evol-Instruct-110K, HuggingFaceTB/smoltalk, bigcode/self-oss-instruct-sc2-exec-filter-50k, HuggingFaceH4/ultrachat_200k, open-r1/OpenR1-Math-220k. Chat formatted with the target's template, packed into 1536-token rows, 4600 steps at an effective batch of 8 rows.
Hardware: Tesla T4, fp16/bf16 autocast + GradScaler.
Validation mode: no — full release recipe.

Measured speed (greedy, batch 1, 6 prompts/domain, this notebook's run)

domain	plain tok/s	spec tok/s	speedup	proj	acceptance length	greedy-identical
coding	17.81	19.36	1.087x	1.005x	1.234	6/6
agentic	17.98	19.94	1.109x	1.032x	1.267	5/6
math	17.91	19.28	1.076x	0.998x	1.226	5/6
general	17.9	19.26	1.076x	1.0x	1.228	5/6
overall	17.89	19.42	1.086x	1.006x	1.236	21/24

Measured per-cycle costs: target decode 57.714 ms/tok, target verify(8) 55.433 ms, draft block 15.442 ms. The proj column is AL * t_decode / (t_verify8 + t_draft) — what an engine-grade serving stack (SGLang/vLLM) should approach; the measured speedup additionally carries the python-side loop overhead of this eval harness.

Speed targets: coding >= 2.0x, agentic >= 2.0x, overall >= 1.5x — not met within caps (measured overall 1.086x). Decoding is lossless by construction: the target verifies every drafted token, so greedy output is token-identical to plain decoding (the identity rate above is limited only by bf16 near-tie argmax flips) and sampling preserves the target distribution. The published DFlash 2 drafters report ~4-5.5 acceptance length on Qwen3.8-27B with full-scale training.

Serving

SGLang (needs the DFLASH support build, see z-lab/dflash):

python -m sglang.launch_server   --model-path openbmb/MiniCPM5-2B   --speculative-algorithm DFLASH   --speculative-draft-model-path applepie69/MiniCPM5-2B-DFlash2   --speculative-num-draft-tokens 8

vLLM:

vllm serve openbmb/MiniCPM5-2B   --speculative-config '{"method": "dflash", "model": "applepie69/MiniCPM5-2B-DFlash2", "num_speculative_tokens": 7}'

Transformers / offline eval with the dflash package:

from dflash.model import DFlash2DraftModel, dflash_generate
draft = DFlash2DraftModel.from_pretrained("applepie69/MiniCPM5-2B-DFlash2")
out = dflash_generate(draft, target_model, input_ids, max_new_tokens=256,
                      stop_token_ids=[1, 130073], temperature=0.0)

Files

file	what
`config.json`	DFlash2 draft config (`dflash_config` block)
`model.safetensors`	the trained drafter (bf16)
`dflash2_training_meta.json`	training settings + measured results
`measured_results.json`	per-domain speed/acceptance measurements + gate status

License

Apache-2.0, matching the base model. Check the base model's terms before use.