Request: variant trained against DeepSeek-V4-Flash-0731 (different weights - acceptance collapses to ~3%)
Thanks for publishing this speculator - the numbers on the base model look
great. We ran it end-to-end against DeepSeek-V4-Flash-0731 and found it
does not transfer, which is probably worth a note on the model card.
The cause: deepseek-ai/DeepSeek-V4-Flash (your training target) andDeepSeek-V4-Flash-0731 are different checkpoints - 46 vs 48 shards, and the
shared shards differ in content (e.g. model-00002 has the same byte size but
a different SHA-256). Measured on 0731 at k=7: 2.9% draft acceptance
(3.8% greedy) vs your 41.6%, mean accepted length 1.2 vs ~3.9.
We ruled out integration errors: the aux hidden-state layout was verified
against your fc weights (reshaping the 81,920 input dim as stream-major[hc_mult, hidden] shows four clearly distinct per-stream norm blocks per
layer slot; interleaved shows none), greedy vs sampled acceptance is
identical, and layer ids follow the standard eagle3 convention. The target
itself generates coherently.
The request: could you retrain - or fine-tune from the existing
checkpoint - a variant against deepseek-ai/DeepSeek-V4-Flash-0731? It is
the current release most deployments will pick up.
The runtime side is ready and public: our vLLM fork runs your speculator
against an MLA target out of the box (draft KV-cache dtype override,
KV-cache grouping for a plain-attention draft on an MLA target, and related
fixes) - https://github.com/lrozewicz/vLLM-Moet-GB10 - serving
DS4-Flash-0731 on a single NVIDIA GB10 (121.6 GiB unified memory), where a
working DFlash would roughly double decode speed. Happy to test candidate
checkpoints.
Small config note: config.json here has "target_hidden_size": null; a
concrete value (16384 for hc_mult=4) would let integrations size the fc
input without guessing.
Environment: vLLM v0.24.0 fork (link above), fp8_ds_mla KV, method:"dflash",
k=7, greedy drafts, NVIDIA GB10 (sm_121, aarch64), CUDA 13.0.