AQ-MedAI/DeepSeek-V4-Flash-0731-eagle3

Thanks to the SpecForge framework for their foundational contributions. Stay tuned for further updates.

The currently trained EAGLE3 model is a preliminary version. In future iterations, we plan to further train it on long-context Chinese datasets, with the goal of achieving a longer average acceptance length.

Model Overview

DeepSeek-V4-Flash-0731-eagle3 is an advanced and highly specialized draft model meticulously engineered to significantly accelerate the inference process of the Kimi-K2.7-Code ecosystem, leveraging the powerful EAGLE3 framework.

Architected upon the robust Llama architecture, this model functions as an exceptionally efficient drafter. It has undergone rigorous training on 0.5 million high-quality samples sourced from the comprehensive open-perfectblend dataset and some Chinese-language data. This extensive training ensures precise and strict alignment with the teacher model's distribution, thereby guaranteeing high fidelity and performance.

Performance & Acceleration

The core value of this EAGLE3 model is its ability to predict multiple future tokens that are subsequently verified by the base model. High acceptance lengths indicate significant latency reduction. Continuous future iterations.

Speculative Decoding Configuration:

• --speculative-num-steps 3: Configures the number of speculative decoding steps.
• --speculative-eagle-topk 1: Sets the top-k value for the Eagle draft model during speculative decoding.
• --speculative-num-draft-tokens 4: Specifies the number of draft tokens generated in each speculative step.

Average Token Acceptance Lengths:

Benchmark	Eagle3
HumanEval (Code)	3.16
Math500 (Complex Math)	3.20
Ceval (Chinese Exam)	3.12
Mtbench (Dialogue)	2.86
GSM8K (Math)	2.73
AIME2024 (Competition Math)	2.62

These metrics demonstrate robust acceleration performance across diverse and complex domains.

Quick Start

Requirements

• NVIDIA GPU
• CUDA 12.9
• PyTorch 2.11

Installation

pip install sglang==0.5.16

and PR #33344

Inference with SGLang

EAGLE3

sglang serve \
   --trust-remote-code \
   --mem-fraction-static 0.8 \
   --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
   --moe-runner-backend marlin \
   --speculative-draft-model-path AQ-MedAI/DeepSeek-V4-Flash-0731-eagle3 \
   --tp-size 8 \
   --speculative-algo EAGLE3 \
   --speculative-num-steps 3 \
   --speculative-eagle-topk 1 \
   --speculative-num-draft-tokens 4 \
   --speculative-draft-attention-backend fa3 \
   --host 0.0.0.0 \
   --port 30019

DSpark

sglang serve \
  --trust-remote-code \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 8 \
  --moe-runner-backend marlin \
  --speculative-algorithm DSPARK \
  --speculative-draft-attention-backend fa3 \
  --mem-fraction-static 0.8 \
  --host 0.0.0.0 \
  --port 30019

Citation

If you use this model in your research or application, please cite the following:

@misc{deepseekv4flash0731eagle3,
  title={DeepSeek-V4-Flash-0731-eagle3: Accelerating Instruction Following with EAGLE3},
  author={Ant AQ Team},
  year={2026},
}