# Hy3-A21B DFly High-Thinking Drafter
## Model Summary
This repository contains a **DFly drafter model** trained for speculative decoding with **Hy3-A21B** under the **high-thinking** inference setting. The model is trained with **AngelSpec** and is designed to propose candidate tokens that are verified by the Hy3-A21B target model.
This drafter is not a standalone chat model. It should be deployed with the matching Hy3-A21B target model and the corresponding **high-thinking** target-side inference mode.
- **Target model:** Hy3-A21B
- **Drafter type:** DFly
- **Inference mode:** High-thinking
- **Training framework:** AngelSpec
- **Recommended serving backend:** vLLM speculative decoding
- **Recommended speculative tokens:** 8
## Training and Thinking-Mode Specialization
High-thinking and no-thinking decoding induce different hidden-state and acceptance distributions. As a result, a drafter trained for one thinking mode should not be assumed to transfer optimally to the other.
This checkpoint is the **high-thinking DFly drafter**: it is trained with high-thinking target-side data and should be evaluated and deployed with high-thinking prompts / target-side generation behavior. The matched high-thinking configuration is important for preserving acceptance length and speculative-decoding efficiency.
## Evaluation: Accepted Length
The following table reports this model's mean accepted length on Hy3-A21B under the **high-thinking** setting at **temperature 1**. The average is computed over the six displayed benchmarks.
| Target Model | Drafter | Train Data | Test Data | Math500 | GSM8K | HumanEval | MBPP | LiveCodeBench | MT-Bench | Avg. |
|---|---|---|---|---:|---:|---:|---:|---:|---:|---:|
| Hy3-A21B | DFly | High-think | High-think | 4.29 | 4.47 | 4.76 | 4.74 | 3.92 | **3.42** | 4.27 |
The high-thinking drafter is specialized for high-thinking inference. It is especially important for reasoning- and chat-oriented workloads where the target-side thinking mode changes the acceptance distribution.
## Deployment
Set `TARGET` to the Hy3-A21B target model path or repository and `DRAFT` to the corresponding drafter checkpoint. The relevant inference code can be found in PR https://github.com/vllm-project/vllm/pull/50246.
```bash
export TARGET=""
export DRAFT=""
vllm serve "$TARGET" \
--tensor-parallel-size 8 \
--speculative-config "{"method":"dspark", "model":"$DRAFT", "num_speculative_tokens":7}"
```
## Technical Report
For more details, see the technical report:
> **AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding**
> arXiv: https://arxiv.org/abs/2607.25852
