metadata
library_name: transformers
tags:
- speculative-decoding
- diffusion-drafter
- adaflash
- qwen3
license: apache-2.0
base_model: z-lab/Qwen3-8B-DFlash-b16
AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
Overview
This is the AdaFlash drafter for Qwen3-8B — an on-policy distilled diffusion drafter with an adaptive length head, trained using PerfectBlend data and reverse-KL loss. It is initialized from Qwen3-8B-DFlash-b16.
AdaFlash accelerates LLM inference through speculative decoding. The adaptive length head dynamically adjusts the number of draft candidates per step, balancing speed and acceptance rate.
Quick Start
AdaFlash requires the patched SGLang PR #34171 for serving.
# Install
conda create -n adaflash python=3.12 -y && conda activate adaflash
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/34171/head#subdirectory=python"
# Serve
export MODEL_PATH=Qwen/Qwen3-8B
export DRAFT_MODEL_PATH=AdaFlash/Qwen3-8B-AdaFlash
# Launch the server & benchmark
bash scripts/tools/benchmark.sh
For full setup, training, and benchmark details, see the GitHub repo.
Citation
@article{arxiv'26:adaflash,
title = {AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters},
author = {Qian, Yu-Yang and Wu, Hao-Cong and Chen, Chen and Sun, Jiacheng and Dong, Zhenhua and Zhao, Peng and Zhou, Zhi-Hua},
journal = {ArXiv preprint},
volume = {arXiv:2607.19223},
year = {2026}
}