File size: 2,157 Bytes
3eb8d1e 6d9b5c2 3eb8d1e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
library_name: transformers
tags:
- speculative-decoding
- diffusion-drafter
- adaflash
- qwen3
license: apache-2.0
base_model: z-lab/Qwen3-8B-DFlash-b16
---
<p align="center">
<img src="https://raw.githubusercontent.com/ZinYY/AdaFlash/main/asset/imgs/AdaFlash_Logo.jpg" width="50%" alt="AdaFlash" />
</p>
<div align="center">
**AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters**
[](https://arxiv.org/abs/2607.19223)
[](https://github.com/AdaFlash-project/AdaFlash)
</div>
---
## Overview
This is the **AdaFlash drafter** for [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) — an on-policy distilled diffusion drafter with an adaptive length head, trained using PerfectBlend data and reverse-KL loss. It is initialized from [Qwen3-8B-DFlash-b16](https://huggingface.co/z-lab/Qwen3-8B-DFlash-b16).
AdaFlash accelerates LLM inference through speculative decoding. The adaptive length head dynamically adjusts the number of draft candidates per step, balancing speed and acceptance rate.
## Quick Start
AdaFlash requires the patched [SGLang PR #34171](https://github.com/sgl-project/sglang/pull/34171) for serving.
```bash
# Install
conda create -n adaflash python=3.12 -y && conda activate adaflash
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/34171/head#subdirectory=python"
# Serve
export MODEL_PATH=Qwen/Qwen3-8B
export DRAFT_MODEL_PATH=AdaFlash/Qwen3-8B-AdaFlash
# Launch the server & benchmark
bash scripts/tools/benchmark.sh
```
For full setup, training, and benchmark details, see the [GitHub repo](https://github.com/AdaFlash-project/AdaFlash).
## Citation
```bibtex
@article{arxiv'26:adaflash,
title = {AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters},
author = {Qian, Yu-Yang and Wu, Hao-Cong and Chen, Chen and Sun, Jiacheng and Dong, Zhenhua and Zhao, Peng and Zhou, Zhi-Hua},
journal = {ArXiv preprint},
volume = {arXiv:2607.19223},
year = {2026}
}
``` |