yukiwayx commited on
Commit
1794b59
·
verified ·
1 Parent(s): cfa2dd1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -18,11 +18,11 @@ language:
18
  FastMTP is a simple yet effective method that enhances Multi-Token Prediction (MTP) for speculative decoding during inference. Our approach fine-tunes a single MTP head with shared weights across multiple causal draft steps, enabling it to capture longer-range dependencies and achieve higher acceptance rates in speculative decoding. By incorporating language-aware vocabulary compression, we further reduce computational overhead during draft generation. Experimental results across diverse benchmarks demonstrate that FastMTP achieves an average of 2.03× speedup over vanilla next token prediction while maintaining lossless output quality. With low training cost and seamless integration into existing inference frameworks, FastMTP offers a practical and rapidly deployable solution for accelerating LLM inference.
19
 
20
  <!-- ![Model](./assets/mtp-overview.png){width=50%} -->
21
- <img src="./assets/mtp-overview.png" width="50%">
22
 
23
  Speedup comparison of different methods across subtasks, evaluated on a single A10 GPU:
24
 
25
- <img src="./assets/radar_chart.png" width="30%">
26
 
27
  ## What's Included
28
 
 
18
  FastMTP is a simple yet effective method that enhances Multi-Token Prediction (MTP) for speculative decoding during inference. Our approach fine-tunes a single MTP head with shared weights across multiple causal draft steps, enabling it to capture longer-range dependencies and achieve higher acceptance rates in speculative decoding. By incorporating language-aware vocabulary compression, we further reduce computational overhead during draft generation. Experimental results across diverse benchmarks demonstrate that FastMTP achieves an average of 2.03× speedup over vanilla next token prediction while maintaining lossless output quality. With low training cost and seamless integration into existing inference frameworks, FastMTP offers a practical and rapidly deployable solution for accelerating LLM inference.
19
 
20
  <!-- ![Model](./assets/mtp-overview.png){width=50%} -->
21
+ <img src="./assets/mtp-overview.png" width="75%">
22
 
23
  Speedup comparison of different methods across subtasks, evaluated on a single A10 GPU:
24
 
25
+ <img src="./assets/radar_chart.png" width="55%">
26
 
27
  ## What's Included
28