ruipeterpan nielsr HF Staff commited on
Commit
bf2887d
·
verified ·
1 Parent(s): 5908703

Add pipeline tag and link to paper (#1)

Browse files

- Add pipeline tag and link to paper (dac7bbec412591f6a683f5bd9e0c71c7fa5bf7db)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +21 -9
README.md CHANGED
@@ -1,24 +1,27 @@
1
  ---
2
- license: apache-2.0
3
- datasets:
4
- - HuggingFaceH4/ultrachat_200k
5
  base_model:
6
  - Qwen/Qwen2.5-7B-Instruct
 
 
 
 
7
  ---
 
8
  # Qwen2.5-7B-Instruct_EAGLE3_UltraChat
9
 
10
  ### Introduction
11
  **Qwen2.5-7B-Instruct_EAGLE3_UltraChat** is trained based on the open-source Qwen2.5-32B-Instruct model using the [SpecForge](https://github.com/sgl-project/SpecForge) framework,
12
  and can be used for the Eagle-3 speculative decoding algorithm to speed up the inference of large language models during the decoding stage.
13
 
 
14
 
15
  ### Training Configuration
16
  We adopted the default training hyperparameters in SpecForge and trained EAGLE-3 to match the target model's output until convergence.
17
 
18
  This model checkpoint is obtained after five epochs of training ($\sim$260k training steps with bs=4). We find that even though further training improves training-time accuracy, they have a negligible impact on the end-to-end speedup of EAGLE-3.
19
 
20
- - Dataset: Utilized the UltraChat-200K dataset.
21
- - Training environment: The training was conducted on 4 NVIDIA H100 GPUs with 80 GB VRAM each, leveraging the DeepSpeed framework. Each training epoch took approximately 3.5 hours.
22
 
23
  ### Model Inference Launch Command
24
 
@@ -48,10 +51,19 @@ We run our evaluations on two NVIDIA A6000-48GB GPUs connected via PCIe 4.0 x16.
48
  | **Qwen2.5-7B-Instruct** | 2.19x | 2.05x | 2.02x | 1.78x | 2.25x | **2.06x** |
49
 
50
 
51
- ### Relevant Link
52
 
53
- Qwen2.5-7B-Instruct Open-source Weights: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct
 
 
54
 
55
- "Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs" [arXiv '25]: https://arxiv.org/pdf/2512.20573
56
 
57
- Artifact of FailFast: https://github.com/ruipeterpan/failfast
 
 
 
 
 
 
 
 
1
  ---
 
 
 
2
  base_model:
3
  - Qwen/Qwen2.5-7B-Instruct
4
+ datasets:
5
+ - HuggingFaceH4/ultrachat_200k
6
+ license: apache-2.0
7
+ pipeline_tag: text-generation
8
  ---
9
+
10
  # Qwen2.5-7B-Instruct_EAGLE3_UltraChat
11
 
12
  ### Introduction
13
  **Qwen2.5-7B-Instruct_EAGLE3_UltraChat** is trained based on the open-source Qwen2.5-32B-Instruct model using the [SpecForge](https://github.com/sgl-project/SpecForge) framework,
14
  and can be used for the Eagle-3 speculative decoding algorithm to speed up the inference of large language models during the decoding stage.
15
 
16
+ This model is an artifact for the paper: [Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs](https://huggingface.co/papers/2512.20573).
17
 
18
  ### Training Configuration
19
  We adopted the default training hyperparameters in SpecForge and trained EAGLE-3 to match the target model's output until convergence.
20
 
21
  This model checkpoint is obtained after five epochs of training ($\sim$260k training steps with bs=4). We find that even though further training improves training-time accuracy, they have a negligible impact on the end-to-end speedup of EAGLE-3.
22
 
23
+ - **Dataset**: Utilized the [UltraChat-200K](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) dataset.
24
+ - **Training environment**: The training was conducted on 4 NVIDIA H100 GPUs with 80 GB VRAM each, leveraging the DeepSpeed framework. Each training epoch took approximately 3.5 hours.
25
 
26
  ### Model Inference Launch Command
27
 
 
51
  | **Qwen2.5-7B-Instruct** | 2.19x | 2.05x | 2.02x | 1.78x | 2.25x | **2.06x** |
52
 
53
 
54
+ ### Relevant Links
55
 
56
+ - **Paper**: [Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs](https://huggingface.co/papers/2512.20573)
57
+ - **GitHub Repository**: [ruipeterpan/failfast](https://github.com/ruipeterpan/failfast)
58
+ - **Base Model**: [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
59
 
60
+ ### Citation
61
 
62
+ ```bibtex
63
+ @article{pan2025failfast,
64
+ title={Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs},
65
+ author={Pan, Rui and Chen, Zhuofu and Liu, Hongyi and Krishnamurthy, Arvind and Netravali, Ravi},
66
+ journal={arXiv preprint arXiv:2512.20573},
67
+ year={2025}
68
+ }
69
+ ```