d3LLM-model commited on
Commit
3eb8d1e
·
verified ·
1 Parent(s): 9119eba

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +62 -0
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - speculative-decoding
5
+ - diffusion-drafter
6
+ - adaflash
7
+ - qwen3
8
+ license: apache-2.0
9
+ base_model: Qwen/Qwen3-8B
10
+ ---
11
+
12
+ <p align="center">
13
+ <img src="https://raw.githubusercontent.com/ZinYY/AdaFlash/main/asset/imgs/AdaFlash_Logo.jpg" width="50%" alt="AdaFlash" />
14
+ </p>
15
+
16
+ <div align="center">
17
+
18
+ **AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters**
19
+
20
+ [![Paper](https://img.shields.io/badge/Paper-arXiv%3A2607.19223-b31b1b.svg)](https://arxiv.org/abs/2607.19223)
21
+ [![GitHub](https://img.shields.io/badge/GitHub-AdaFlash-blue.svg)](https://github.com/AdaFlash-project/AdaFlash)
22
+
23
+ </div>
24
+
25
+ ---
26
+
27
+ ## Overview
28
+
29
+ This is the **AdaFlash drafter** for [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) — an on-policy distilled diffusion drafter with an adaptive length head, trained using PerfectBlend data and reverse-KL loss. It is initialized from [Qwen3-8B-DFlash-b16](https://huggingface.co/z-lab/Qwen3-8B-DFlash-b16).
30
+
31
+ AdaFlash accelerates LLM inference through speculative decoding. The adaptive length head dynamically adjusts the number of draft candidates per step, balancing speed and acceptance rate.
32
+
33
+ ## Quick Start
34
+
35
+ AdaFlash requires the patched [SGLang PR #34171](https://github.com/sgl-project/sglang/pull/34171) for serving.
36
+
37
+ ```bash
38
+ # Install
39
+ conda create -n adaflash python=3.12 -y && conda activate adaflash
40
+ pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git@refs/pull/34171/head#subdirectory=python"
41
+
42
+ # Serve
43
+ export MODEL_PATH=Qwen/Qwen3-8B
44
+ export DRAFT_MODEL_PATH=AdaFlash/Qwen3-8B-AdaFlash
45
+
46
+ # Launch the server & benchmark
47
+ bash scripts/tools/benchmark.sh
48
+ ```
49
+
50
+ For full setup, training, and benchmark details, see the [GitHub repo](https://github.com/AdaFlash-project/AdaFlash).
51
+
52
+ ## Citation
53
+
54
+ ```bibtex
55
+ @article{arxiv'26:adaflash,
56
+ title = {AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters},
57
+ author = {Qian, Yu-Yang and Wu, Hao-Cong and Chen, Chen and Sun, Jiacheng and Dong, Zhenhua and Zhao, Peng and Zhou, Zhi-Hua},
58
+ journal = {ArXiv preprint},
59
+ volume = {arXiv:2607.19223},
60
+ year = {2026}
61
+ }
62
+ ```