--- license: mit library_name: pytorch pipeline_tag: other tags: - speculative-decoding - diffusion-language-model - masked-diffusion - block-diffusion - drafter - lossless-acceleration ---
# SPOT drafters **On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction** Eliezer Shlomi1,2\*, Eliron Rahimi1,2\*, Avi Mendelson1, Chaim Baskin2 1Technion – Israel Institute of Technology
2INSIGHT Lab, School of Electrical and Computer Engineering, Ben-Gurion University of the Negev \*Equal contribution. Correspondence: {eliezer, elironrahimi}@campus.technion.ac.il [![Project page](https://img.shields.io/badge/Project-page-1D4ED8)](https://eliezer318.github.io/SPOT-dLLM/) [![arXiv](https://img.shields.io/badge/arXiv-2610.22543-B31B1B)](https://arxiv.org/abs/2610.22543) [![Code](https://img.shields.io/badge/GitHub-SPOT--dLLM-181717?logo=github)](https://github.com/Eliezer318/SPOT-dLLM) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Eliezer318/SPOT-dLLM/blob/main/colab/spot_quickstart.ipynb) [![License](https://img.shields.io/badge/license-MIT-blue)](https://github.com/Eliezer318/SPOT-dLLM/blob/main/LICENSE)
![Recorded SDAR-8B runs on one GSM8K prompt: draft-free decoding, the Spiffy baseline, SPOT and SPOT-γ write the same 256 tokens; SPOT finishes in a quarter of the forwards.](assets/spot_replay.gif) Commit-order drafters for **SPOT** (Speculating Positions Order Trajectory). Each checkpoint is the same 0.62M-parameter order model (3 layers, `d_model` 128, 8 heads, stale top-32 inputs). It reads only the target's own output distributions from the previous pass and predicts which masked position a block-diffusion target commits next. SPOT grows those predictions into a tree of candidate block states, and the target verifies the whole tree in one forward, exactly: an accepted state equals the target's own next state, and coupled-noise match keeps this exact at any temperature. On SDAR-8B this cuts target passes by up to 6.8× and reaches up to 3.4× the throughput of standard decoding, with the target's output unchanged. Paper: [arXiv:2610.22543](https://arxiv.org/abs/2610.22543). ## Files | file | target | block | held-out next-commit acc. | |---|---|---|---| | `sdar8b_b32.pt` | SDAR-8B-Chat-b32 | 32 | 0.750 | | `sdar8b_b16.pt` | SDAR-8B-Chat-b16 | 16 | 0.746 | | `sdar4b_b32.pt` | SDAR-4B-Chat-b32 | 32 | 0.669 | | `sdar1.7b_b32.pt` | SDAR-1.7B-Chat-b32 | 32 | 0.665 | | `llada2mini_b32.pt` | LLaDA2.0-mini | 32 | 0.636 | | `llada2mini_b16.pt` | LLaDA2.0-mini | 16 | 0.665 | `sets/` holds the **SPOT-Sets** drafters for dynamic threshold decoding (τ = 0.9, block 32): | file | target | drafter | |---|---|---| | `sets/sdar8b_b32.pt` | SDAR-8B-Chat-b32 | trained on general chat | | `sets/sdar4b_b32.pt` | SDAR-4B-Chat-b32 | trained on general chat | | `sets/llada2mini_b32.pt` | LLaDA2.0-mini | trained on general chat | | `sets/llada8b_b32.pt` | LLaDA-8B-Instruct | the SDAR-8B drafter, calibrated on 98 target prompts | | `sets/dream_b32.pt` | Dream-v0-Instruct-7B | the SDAR-8B drafter, calibrated on 98 target prompts | Each file is a 5.1M-parameter set drafter (6 layers, `d_model` 256) that predicts which positions the next step commits and with which tokens, stored with the target's token-embedding PCA and hidden-state basis it reads. The first three were trained on the target's own threshold-decoded trajectories on general chat prompts only (ShareGPT and UltraChat-200K). The LLaDA-8B and Dream-7B drafters start from the SDAR-8B one and are calibrated on the target's own trajectories for 98 train prompts of GSM8K, MATH, MBPP and CodeAlpaca. On SDAR-8B at T = 0 SPOT-Sets raises throughput over standard decoding to 5.2–9.9× (7.9× on average), 59% above the threshold decoder alone (Table 2 of the paper). `tuned/` holds the per-setting comparison drafters of the paper's appendix; the six checkpoints above are all you need to use SPOT. A drafter can also drive other targets directly; LLaDA-8B and Dream-7B run with transferred drafters. ## Results SDAR-8B, block width 32, generation length 256, draft budget 8, T = 0: throughput ratio over draft-free decoding (1.00), mean of three seeds (Table 2 of the paper). | Task | Spiffy graph | SPOT | SPOT-γ | |---|---|---|---| | GSM8K | 2.48 | 2.95 | **3.03** | | MBPP | 2.40 | 3.38 | **3.48** | | HumanEval | 2.64 | 3.35 | **3.42** | | MATH500 | 2.29 | 2.50 | **2.53** | At budget 20, SPOT needs 4.1–6.8× fewer target forwards than draft-free decoding on SDAR-8B. At T = 1 it uses 16–35% fewer forwards than Spiffy at 1.08–1.37× its throughput. SPOT beats or matches every baseline in every setting at both temperatures (a margin under 5% counts as a match). ## Limitations - Gains shrink when verification is costly or acceptance is low: at wide blocks, on low-acceptance tasks, and at T ≥ 2, where noise-aware ranking beats the order model trained at T = 0. - SPOT-Sets needs a short calibration on each new target, its larger drafter (5.1M parameters) matters more when the target forward is cheap, and its main rows use max-commit, which changes the output. - LLaDA-8B and Dream-7B run with an approximate prefix cache, so those results measure reach; losslessness is shown on the exact-cache targets. - Exact tree packing under bidirectional attention remains open. ## Use The checkpoints are loaded by the decoder in the [GitHub repository](https://github.com/Eliezer318/SPOT-dLLM): ```bash git clone https://github.com/Eliezer318/SPOT-dLLM.git && cd SPOT-dLLM pip install torch --index-url https://download.pytorch.org/whl/cu124 # match your driver pip install -r requirements.txt huggingface-cli download EliezerS/SPOT-dLLM --local-dir drafters python run_sweep.py --model sdar --block-length 32 --gen-length 256 --task gsm8k --limit 100 \ --temperature 0.0 --verify match --layout concat --cache on --unmasking single --suffix skip \ --pad-pieces -1 --draft slot --slot-mode ddtree_b --slot-ckpt drafters/sdar8b_b32.pt \ --slot-tokgate 0.6 --budgets 0,8 --seeds 0 --out-root runs/sdar8b/gsm8k/spot # SPOT-Sets on threshold decoding (budget 0 = the threshold decoder without drafts) python run_sweep.py --model sdar --block-length 32 --gen-length 256 --task gsm8k --limit 100 \ --temperature 0.0 --layout concat --cache on --unmasking threshold --threshold 0.9 \ --draft sets --sets-ckpt drafters/sets/sdar8b_b32.pt --sets-accept product --cross-block on \ --budgets 0,8 --seeds 0 --out-root runs/sdar8b/gsm8k/sets # other SPOT-Sets targets: --model sdar4b | llada2 | llada | dream with the matching --sets-ckpt ``` ## Citation ```bibtex @article{shlomi2026spot, title = {On the {SPOT}: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction}, author = {Shlomi, Eliezer and Rahimi, Eliron and Mendelson, Avi and Baskin, Chaim}, journal = {arXiv preprint arXiv:2610.22543}, year = {2026}, doi = {10.48550/arXiv.2610.22543}, eprint = {2610.22543}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2610.22543} } ```