File size: 4,013 Bytes
a3d2204 98cddf2 a198824 98cddf2 a3d2204 a198824 a3d2204 a198824 a3d2204 a198824 a3d2204 a198824 a3d2204 a198824 a3d2204 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
license: apache-2.0
tags:
- discrete-diffusion
- jax
- flax
datasets:
- sapientinc/sudoku-extreme
- rodcasnog/sudoku-extreme-split
- AlignmentResearch/boxoban-astar-solutions
- afmck/text8
- roneneldan/TinyStories
- Skylion007/openwebtext
---
# Binary Discrete Diffusion: models
The models of [Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning
Tasks](https://arxiv.org/abs/2610.28543): diffusion transformers for each task and noise kernel (`masked`,
`uniform`, `binary_symmetric`, `binary_absorbing`), five training seeds on the logic tasks and one on text. Each
run directory of [binary-discrete-diffusion](https://github.com/rodcasnog/binary-discrete-diffusion) was trained
with `configs/paper/<task>/<kernel>.yaml` and holds the composed config in `.hydra/config.yaml` and the EMA
parameters the paper evaluates in `checkpoints_ema/<step>/`. The logic runs also hold their final training state
(parameters, optimizer and time-sampler state) in `checkpoints/<step>/`, from which `train.py` resumes.
| Task | Directory | EMA checkpoint | Final training state | Data |
|---|---|---|---|---|
| Sudoku | `sudoku/<kernel>/seed<0-4>` | `checkpoints_ema/1000000` | `checkpoints/1000000` | `scripts/prep_sudoku.py` |
| Boxoban | `boxoban/<kernel>/seed<0-4>` | `checkpoints_ema/2500000`; binary absorbing seeds 1 and 3 at `2000000` and `1000000` | `checkpoints/2500000` | `scripts/prep_boxoban.py` |
| Graph coloring | `graphcolor/<kernel>/seed<0-4>` | `checkpoints_ema/300000` | `checkpoints/300000` | generated on the fly |
| Text8 | `text8/<kernel>` | `checkpoints_ema/1000000` | | characters, written by `scripts/prep_text8.py` |
| TinyStories | `tinystories/<kernel>` | `checkpoints_ema/500000` | | tokenizer [rodcasnog/tinystories-bpe-1k](https://huggingface.co/rodcasnog/tinystories-bpe-1k) |
| OpenWebText | `owt/<kernel>` | `checkpoints_ema/250000` | | tokenizer [dvruette/openwebtext-bpe-1k](https://huggingface.co/dvruette/openwebtext-bpe-1k) |
The Boxoban EMA checkpoints were selected on validation NELBO, as described in the paper's appendix. Since
`train.py` resumes only from a step that holds both checkpoints, Boxoban binary absorbing seeds 1 and 3 do not resume
as downloaded.
## Usage
From a clone of the code repository, after `uv sync`:
```bash
uv run hf download rodcasnog/binary-discrete-diffusion --include "sudoku/binary_symmetric/seed0/*" --local-dir outputs/hf
uv run python -m binary_diffusion.eval.solve checkpoint_dir=outputs/hf/sudoku/binary_symmetric/seed0/checkpoints_ema/1000000 \
sampler.nfe=256 out=outputs/sudoku-bsym/solve
# Train the same run 200k steps longer.
uv run python -m binary_diffusion.train --config-name paper/sudoku/binary_symmetric training.seed=0 \
training.num_steps=1200000 hydra.run.dir=outputs/hf/sudoku/binary_symmetric/seed0
uv run hf download rodcasnog/binary-discrete-diffusion --include "text8/binary_symmetric/*" --local-dir outputs/hf
uv run python -m binary_diffusion.sample checkpoint_dir=outputs/hf/text8/binary_symmetric/checkpoints_ema/1000000 \
sampler.nfe=256 sampler.spacing=log sampler.eta=1 sampler.lam=1 out=outputs/text8-bsym/samples
```
Sudoku and Boxoban models read their task data in `data/sudoku` and `data/boxoban`. Text8 models read the tokenizer
in `data/text8`, TinyStories samples take their lengths from `data/tinystories`, and NELBO and MAUVE read the corpus;
the code's README lists the data and evaluation commands.
## License
Apache-2.0.
## Citation
```bibtex
@misc{casadonoguerales2026binarydiffusion,
title = {Binary Discrete Diffusion: Candidate-Set Diffusion for
Logic and Planning Tasks},
author = {Casado Noguerales, Rodrigo and von R{\"u}tte, Dimitri
and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas
and Raoufi, Aran},
year = {2026},
eprint = {2610.28543},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.28543},
}
```
|