README.md

metadata

license: mit
library_name: pytorch
pipeline_tag: other
tags:
  - speculative-decoding
  - diffusion-language-model
  - masked-diffusion
  - block-diffusion
  - drafter
  - lossless-acceleration

SPOT drafters

On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction

Eliezer Shlomi^1,2*, Eliron Rahimi^1,2*, Avi Mendelson¹, Chaim Baskin²

¹Technion – Israel Institute of Technology
²INSIGHT Lab, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev

_{^*Equal contribution. Correspondence: {eliezer, elironrahimi}@campus.technion.ac.il}

Commit-order drafters for SPOT (Speculating Positions Order Trajectory). Each checkpoint is the same 0.62M-parameter order model (3 layers, d_model 128, 8 heads, stale top-32 inputs). It reads only the target's own output distributions from the previous pass and predicts which masked position a block-diffusion target commits next. SPOT grows those predictions into a tree of candidate block states, and the target verifies the whole tree in one forward, exactly: an accepted state equals the target's own next state, and coupled-noise match keeps this exact at any temperature. On SDAR-8B this cuts target passes by up to 6.8× and reaches up to 3.4× the throughput of standard decoding, with the target's output unchanged. Paper: arXiv:2610.22543.

Files

file	target	block	held-out next-commit acc.
sdar8b_b32.pt	SDAR-8B-Chat-b32	32	0.750
sdar8b_b16.pt	SDAR-8B-Chat-b16	16	0.746
sdar4b_b32.pt	SDAR-4B-Chat-b32	32	0.669
sdar1.7b_b32.pt	SDAR-1.7B-Chat-b32	32	0.665
llada2mini_b32.pt	LLaDA2.0-mini	32	0.636
llada2mini_b16.pt	LLaDA2.0-mini	16	0.665

sets/ holds the SPOT-Sets drafters for dynamic threshold decoding (τ = 0.9, block 32):

file	target	drafter
sets/sdar8b_b32.pt	SDAR-8B-Chat-b32	trained on general chat
sets/sdar4b_b32.pt	SDAR-4B-Chat-b32	trained on general chat
sets/llada2mini_b32.pt	LLaDA2.0-mini	trained on general chat
sets/llada8b_b32.pt	LLaDA-8B-Instruct	the SDAR-8B drafter, calibrated on 98 target prompts
sets/dream_b32.pt	Dream-v0-Instruct-7B	the SDAR-8B drafter, calibrated on 98 target prompts

Each file is a 5.1M-parameter set drafter (6 layers, d_model 256) that predicts which positions the next step commits and with which tokens, stored with the target's token-embedding PCA and hidden-state basis it reads. The first three were trained on the target's own threshold-decoded trajectories on general chat prompts only (ShareGPT and UltraChat-200K). The LLaDA-8B and Dream-7B drafters start from the SDAR-8B one and are calibrated on the target's own trajectories for 98 train prompts of GSM8K, MATH, MBPP and CodeAlpaca. On SDAR-8B at T = 0 SPOT-Sets raises throughput over standard decoding to 5.2–9.9× (7.9× on average), 59% above the threshold decoder alone (Table 2 of the paper).

tuned/ holds the per-setting comparison drafters of the paper's appendix; the six checkpoints above are all you need to use SPOT. A drafter can also drive other targets directly; LLaDA-8B and Dream-7B run with transferred drafters.

Results

SDAR-8B, block width 32, generation length 256, draft budget 8, T = 0: throughput ratio over draft-free decoding (1.00), mean of three seeds (Table 2 of the paper).

Task	Spiffy graph	SPOT	SPOT-γ
GSM8K	2.48	2.95	3.03
MBPP	2.40	3.38	3.48
HumanEval	2.64	3.35	3.42
MATH500	2.29	2.50	2.53

At budget 20, SPOT needs 4.1–6.8× fewer target forwards than draft-free decoding on SDAR-8B. At T = 1 it uses 16–35% fewer forwards than Spiffy at 1.08–1.37× its throughput. SPOT beats or matches every baseline in every setting at both temperatures (a margin under 5% counts as a match).

Limitations

• Gains shrink when verification is costly or acceptance is low: at wide blocks, on low-acceptance tasks, and at T ≥ 2, where noise-aware ranking beats the order model trained at T = 0.
• SPOT-Sets needs a short calibration on each new target, its larger drafter (5.1M parameters) matters more when the target forward is cheap, and its main rows use max-commit, which changes the output.
• LLaDA-8B and Dream-7B run with an approximate prefix cache, so those results measure reach; losslessness is shown on the exact-cache targets.
• Exact tree packing under bidirectional attention remains open.

Use

The checkpoints are loaded by the decoder in the GitHub repository:

git clone https://github.com/Eliezer318/SPOT-dLLM.git && cd SPOT-dLLM
pip install torch --index-url https://download.pytorch.org/whl/cu124   # match your driver
pip install -r requirements.txt
huggingface-cli download EliezerS/SPOT-dLLM --local-dir drafters
python run_sweep.py --model sdar --block-length 32 --gen-length 256 --task gsm8k --limit 100 \
  --temperature 0.0 --verify match --layout concat --cache on --unmasking single --suffix skip \
  --pad-pieces -1 --draft slot --slot-mode ddtree_b --slot-ckpt drafters/sdar8b_b32.pt \
  --slot-tokgate 0.6 --budgets 0,8 --seeds 0 --out-root runs/sdar8b/gsm8k/spot
# SPOT-Sets on threshold decoding (budget 0 = the threshold decoder without drafts)
python run_sweep.py --model sdar --block-length 32 --gen-length 256 --task gsm8k --limit 100 \
  --temperature 0.0 --layout concat --cache on --unmasking threshold --threshold 0.9 \
  --draft sets --sets-ckpt drafters/sets/sdar8b_b32.pt --sets-accept product --cross-block on \
  --budgets 0,8 --seeds 0 --out-root runs/sdar8b/gsm8k/sets
# other SPOT-Sets targets: --model sdar4b | llada2 | llada | dream with the matching --sets-ckpt

Citation

@article{shlomi2026spot,
  title         = {On the {SPOT}: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction},
  author        = {Shlomi, Eliezer and Rahimi, Eliron and Mendelson, Avi and Baskin, Chaim},
  journal       = {arXiv preprint arXiv:2610.22543},
  year          = {2026},
  doi           = {10.48550/arXiv.2610.22543},
  eprint        = {2610.22543},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.22543}
}