File size: 4,013 Bytes
a3d2204
 
 
 
 
 
98cddf2
a198824
 
 
98cddf2
 
 
a3d2204
 
a198824
a3d2204
a198824
 
 
 
 
 
 
a3d2204
a198824
 
 
 
 
 
 
 
 
 
 
 
a3d2204
 
 
 
 
 
a198824
 
 
 
 
 
 
a3d2204
 
 
 
 
a198824
 
 
a3d2204
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
license: apache-2.0
tags:
- discrete-diffusion
- jax
- flax
datasets:
- sapientinc/sudoku-extreme
- rodcasnog/sudoku-extreme-split
- AlignmentResearch/boxoban-astar-solutions
- afmck/text8
- roneneldan/TinyStories
- Skylion007/openwebtext
---

# Binary Discrete Diffusion: models

The models of [Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning
Tasks](https://arxiv.org/abs/2610.28543): diffusion transformers for each task and noise kernel (`masked`,
`uniform`, `binary_symmetric`, `binary_absorbing`), five training seeds on the logic tasks and one on text. Each
run directory of [binary-discrete-diffusion](https://github.com/rodcasnog/binary-discrete-diffusion) was trained
with `configs/paper/<task>/<kernel>.yaml` and holds the composed config in `.hydra/config.yaml` and the EMA
parameters the paper evaluates in `checkpoints_ema/<step>/`. The logic runs also hold their final training state
(parameters, optimizer and time-sampler state) in `checkpoints/<step>/`, from which `train.py` resumes.

| Task | Directory | EMA checkpoint | Final training state | Data |
|---|---|---|---|---|
| Sudoku | `sudoku/<kernel>/seed<0-4>` | `checkpoints_ema/1000000` | `checkpoints/1000000` | `scripts/prep_sudoku.py` |
| Boxoban | `boxoban/<kernel>/seed<0-4>` | `checkpoints_ema/2500000`; binary absorbing seeds 1 and 3 at `2000000` and `1000000` | `checkpoints/2500000` | `scripts/prep_boxoban.py` |
| Graph coloring | `graphcolor/<kernel>/seed<0-4>` | `checkpoints_ema/300000` | `checkpoints/300000` | generated on the fly |
| Text8 | `text8/<kernel>` | `checkpoints_ema/1000000` | | characters, written by `scripts/prep_text8.py` |
| TinyStories | `tinystories/<kernel>` | `checkpoints_ema/500000` | | tokenizer [rodcasnog/tinystories-bpe-1k](https://huggingface.co/rodcasnog/tinystories-bpe-1k) |
| OpenWebText | `owt/<kernel>` | `checkpoints_ema/250000` | | tokenizer [dvruette/openwebtext-bpe-1k](https://huggingface.co/dvruette/openwebtext-bpe-1k) |

The Boxoban EMA checkpoints were selected on validation NELBO, as described in the paper's appendix. Since
`train.py` resumes only from a step that holds both checkpoints, Boxoban binary absorbing seeds 1 and 3 do not resume
as downloaded.

## Usage

From a clone of the code repository, after `uv sync`:

```bash
uv run hf download rodcasnog/binary-discrete-diffusion --include "sudoku/binary_symmetric/seed0/*" --local-dir outputs/hf
uv run python -m binary_diffusion.eval.solve checkpoint_dir=outputs/hf/sudoku/binary_symmetric/seed0/checkpoints_ema/1000000 \
    sampler.nfe=256 out=outputs/sudoku-bsym/solve
# Train the same run 200k steps longer.
uv run python -m binary_diffusion.train --config-name paper/sudoku/binary_symmetric training.seed=0 \
    training.num_steps=1200000 hydra.run.dir=outputs/hf/sudoku/binary_symmetric/seed0

uv run hf download rodcasnog/binary-discrete-diffusion --include "text8/binary_symmetric/*" --local-dir outputs/hf
uv run python -m binary_diffusion.sample checkpoint_dir=outputs/hf/text8/binary_symmetric/checkpoints_ema/1000000 \
    sampler.nfe=256 sampler.spacing=log sampler.eta=1 sampler.lam=1 out=outputs/text8-bsym/samples
```

Sudoku and Boxoban models read their task data in `data/sudoku` and `data/boxoban`. Text8 models read the tokenizer
in `data/text8`, TinyStories samples take their lengths from `data/tinystories`, and NELBO and MAUVE read the corpus;
the code's README lists the data and evaluation commands.

## License

Apache-2.0.

## Citation

```bibtex
@misc{casadonoguerales2026binarydiffusion,
  title         = {Binary Discrete Diffusion: Candidate-Set Diffusion for
                   Logic and Planning Tasks},
  author        = {Casado Noguerales, Rodrigo and von R{\"u}tte, Dimitri
                   and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas
                   and Raoufi, Aran},
  year          = {2026},
  eprint        = {2610.28543},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.28543},
}
```