---
title: DSSD exact self-speculative decoding
emoji: ⚡
colorFrom: indigo
colorTo: pink
sdk: gradio
sdk_version: 6.29.1
python_version: "3.12"
app_file: app.py
license: apache-2.0
short_description: Exact self-speculative decoding with exit heads + copying
models:
- FlorianVal/DSSD-Qwen3-8B-Base
- FlorianVal/Llama-3.1-8B-DSSD
- Qwen/Qwen3-8B-Base
- meta-llama/Llama-3.1-8B
---

# DSSD demo

Exact self-speculative decoding: a small exit head on layer 4 of the frozen model, plus n-gram copies from the
context, propose a tree of draft tokens. The full model verifies it in one forward pass. Outputs are identical to
greedy decoding.

The demo shows:
- each token coloured by its source: copy, exit head, head top-4 candidate, or full model;
- the same prompt decoded autoregressively for comparison;
- the measured speedup, tokens per verification, and a check that both outputs are identical;
- a side-by-side replay of both generations at the measured speed.

Models: Qwen3-8B-Base and Llama-3.1-8B (base models, plain-text completion). The heads are in
[FlorianVal/DSSD-Qwen3-8B-Base](https://huggingface.co/FlorianVal/DSSD-Qwen3-8B-Base) and
[FlorianVal/Llama-3.1-8B-DSSD](https://huggingface.co/FlorianVal/Llama-3.1-8B-DSSD). The Llama heads are built with Llama.

## Configuration

| Variable | Default | Effect |
|---|---|---|
| `HF_TOKEN` (secret) | — | needed for the gated Llama base model and for private head repositories |
| `DSSD_DEMO_MODELS` | both | comma-separated subset of `Qwen3-8B-Base,Llama-3.1-8B` |
| `DSSD_GRAPHS` | off on ZeroGPU, on otherwise | `1` / `0` forces CUDA graphs |
| `DSSD_DEMO_BACKEND` | `hub` | `tiny`: random tiny model on CPU, for local UI work |

## Local development

```bash
uv venv .venv && uv pip install --python .venv/bin/python -r requirements.txt pytest
DSSD_DEMO_BACKEND=tiny .venv/bin/python app.py      # http://localhost:7860, no GPU needed
.venv/bin/python -m pytest -q tests
scripts/vendor_dssd.sh ../BranchyLLM-pub           # refresh the vendored dssd package
```

The `dssd/` directory is a copy of the inference package from BranchyLLM (see `dssd/VENDORED.txt`); do not edit it here.