Add pipeline tag and link to paper (#1)
Browse files- Add pipeline tag and link to paper (dac7bbec412591f6a683f5bd9e0c71c7fa5bf7db)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,24 +1,27 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
datasets:
|
| 4 |
-
- HuggingFaceH4/ultrachat_200k
|
| 5 |
base_model:
|
| 6 |
- Qwen/Qwen2.5-7B-Instruct
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
|
|
|
| 8 |
# Qwen2.5-7B-Instruct_EAGLE3_UltraChat
|
| 9 |
|
| 10 |
### Introduction
|
| 11 |
**Qwen2.5-7B-Instruct_EAGLE3_UltraChat** is trained based on the open-source Qwen2.5-32B-Instruct model using the [SpecForge](https://github.com/sgl-project/SpecForge) framework,
|
| 12 |
and can be used for the Eagle-3 speculative decoding algorithm to speed up the inference of large language models during the decoding stage.
|
| 13 |
|
|
|
|
| 14 |
|
| 15 |
### Training Configuration
|
| 16 |
We adopted the default training hyperparameters in SpecForge and trained EAGLE-3 to match the target model's output until convergence.
|
| 17 |
|
| 18 |
This model checkpoint is obtained after five epochs of training ($\sim$260k training steps with bs=4). We find that even though further training improves training-time accuracy, they have a negligible impact on the end-to-end speedup of EAGLE-3.
|
| 19 |
|
| 20 |
-
- Dataset
|
| 21 |
-
- Training environment
|
| 22 |
|
| 23 |
### Model Inference Launch Command
|
| 24 |
|
|
@@ -48,10 +51,19 @@ We run our evaluations on two NVIDIA A6000-48GB GPUs connected via PCIe 4.0 x16.
|
|
| 48 |
| **Qwen2.5-7B-Instruct** | 2.19x | 2.05x | 2.02x | 1.78x | 2.25x | **2.06x** |
|
| 49 |
|
| 50 |
|
| 51 |
-
### Relevant
|
| 52 |
|
| 53 |
-
|
|
|
|
|
|
|
| 54 |
|
| 55 |
-
|
| 56 |
|
| 57 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
base_model:
|
| 3 |
- Qwen/Qwen2.5-7B-Instruct
|
| 4 |
+
datasets:
|
| 5 |
+
- HuggingFaceH4/ultrachat_200k
|
| 6 |
+
license: apache-2.0
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
---
|
| 9 |
+
|
| 10 |
# Qwen2.5-7B-Instruct_EAGLE3_UltraChat
|
| 11 |
|
| 12 |
### Introduction
|
| 13 |
**Qwen2.5-7B-Instruct_EAGLE3_UltraChat** is trained based on the open-source Qwen2.5-32B-Instruct model using the [SpecForge](https://github.com/sgl-project/SpecForge) framework,
|
| 14 |
and can be used for the Eagle-3 speculative decoding algorithm to speed up the inference of large language models during the decoding stage.
|
| 15 |
|
| 16 |
+
This model is an artifact for the paper: [Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs](https://huggingface.co/papers/2512.20573).
|
| 17 |
|
| 18 |
### Training Configuration
|
| 19 |
We adopted the default training hyperparameters in SpecForge and trained EAGLE-3 to match the target model's output until convergence.
|
| 20 |
|
| 21 |
This model checkpoint is obtained after five epochs of training ($\sim$260k training steps with bs=4). We find that even though further training improves training-time accuracy, they have a negligible impact on the end-to-end speedup of EAGLE-3.
|
| 22 |
|
| 23 |
+
- **Dataset**: Utilized the [UltraChat-200K](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) dataset.
|
| 24 |
+
- **Training environment**: The training was conducted on 4 NVIDIA H100 GPUs with 80 GB VRAM each, leveraging the DeepSpeed framework. Each training epoch took approximately 3.5 hours.
|
| 25 |
|
| 26 |
### Model Inference Launch Command
|
| 27 |
|
|
|
|
| 51 |
| **Qwen2.5-7B-Instruct** | 2.19x | 2.05x | 2.02x | 1.78x | 2.25x | **2.06x** |
|
| 52 |
|
| 53 |
|
| 54 |
+
### Relevant Links
|
| 55 |
|
| 56 |
+
- **Paper**: [Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs](https://huggingface.co/papers/2512.20573)
|
| 57 |
+
- **GitHub Repository**: [ruipeterpan/failfast](https://github.com/ruipeterpan/failfast)
|
| 58 |
+
- **Base Model**: [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 59 |
|
| 60 |
+
### Citation
|
| 61 |
|
| 62 |
+
```bibtex
|
| 63 |
+
@article{pan2025failfast,
|
| 64 |
+
title={Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs},
|
| 65 |
+
author={Pan, Rui and Chen, Zhuofu and Liu, Hongyi and Krishnamurthy, Arvind and Netravali, Ravi},
|
| 66 |
+
journal={arXiv preprint arXiv:2512.20573},
|
| 67 |
+
year={2025}
|
| 68 |
+
}
|
| 69 |
+
```
|