--- license: apache-2.0 tags: - discrete-diffusion - jax - flax datasets: - sapientinc/sudoku-extreme - rodcasnog/sudoku-extreme-split - AlignmentResearch/boxoban-astar-solutions - afmck/text8 - roneneldan/TinyStories - Skylion007/openwebtext --- # Binary Discrete Diffusion: models The models of [Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks](https://arxiv.org/abs/2610.28543): diffusion transformers for each task and noise kernel (`masked`, `uniform`, `binary_symmetric`, `binary_absorbing`), five training seeds on the logic tasks and one on text. Each run directory of [binary-discrete-diffusion](https://github.com/rodcasnog/binary-discrete-diffusion) was trained with `configs/paper//.yaml` and holds the composed config in `.hydra/config.yaml` and the EMA parameters the paper evaluates in `checkpoints_ema//`. The logic runs also hold their final training state (parameters, optimizer and time-sampler state) in `checkpoints//`, from which `train.py` resumes. | Task | Directory | EMA checkpoint | Final training state | Data | |---|---|---|---|---| | Sudoku | `sudoku//seed<0-4>` | `checkpoints_ema/1000000` | `checkpoints/1000000` | `scripts/prep_sudoku.py` | | Boxoban | `boxoban//seed<0-4>` | `checkpoints_ema/2500000`; binary absorbing seeds 1 and 3 at `2000000` and `1000000` | `checkpoints/2500000` | `scripts/prep_boxoban.py` | | Graph coloring | `graphcolor//seed<0-4>` | `checkpoints_ema/300000` | `checkpoints/300000` | generated on the fly | | Text8 | `text8/` | `checkpoints_ema/1000000` | | characters, written by `scripts/prep_text8.py` | | TinyStories | `tinystories/` | `checkpoints_ema/500000` | | tokenizer [rodcasnog/tinystories-bpe-1k](https://huggingface.co/rodcasnog/tinystories-bpe-1k) | | OpenWebText | `owt/` | `checkpoints_ema/250000` | | tokenizer [dvruette/openwebtext-bpe-1k](https://huggingface.co/dvruette/openwebtext-bpe-1k) | The Boxoban EMA checkpoints were selected on validation NELBO, as described in the paper's appendix. Since `train.py` resumes only from a step that holds both checkpoints, Boxoban binary absorbing seeds 1 and 3 do not resume as downloaded. ## Usage From a clone of the code repository, after `uv sync`: ```bash uv run hf download rodcasnog/binary-discrete-diffusion --include "sudoku/binary_symmetric/seed0/*" --local-dir outputs/hf uv run python -m binary_diffusion.eval.solve checkpoint_dir=outputs/hf/sudoku/binary_symmetric/seed0/checkpoints_ema/1000000 \ sampler.nfe=256 out=outputs/sudoku-bsym/solve # Train the same run 200k steps longer. uv run python -m binary_diffusion.train --config-name paper/sudoku/binary_symmetric training.seed=0 \ training.num_steps=1200000 hydra.run.dir=outputs/hf/sudoku/binary_symmetric/seed0 uv run hf download rodcasnog/binary-discrete-diffusion --include "text8/binary_symmetric/*" --local-dir outputs/hf uv run python -m binary_diffusion.sample checkpoint_dir=outputs/hf/text8/binary_symmetric/checkpoints_ema/1000000 \ sampler.nfe=256 sampler.spacing=log sampler.eta=1 sampler.lam=1 out=outputs/text8-bsym/samples ``` Sudoku and Boxoban models read their task data in `data/sudoku` and `data/boxoban`. Text8 models read the tokenizer in `data/text8`, TinyStories samples take their lengths from `data/tinystories`, and NELBO and MAUVE read the corpus; the code's README lists the data and evaluation commands. ## License Apache-2.0. ## Citation ```bibtex @misc{casadonoguerales2026binarydiffusion, title = {Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks}, author = {Casado Noguerales, Rodrigo and von R{\"u}tte, Dimitri and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas and Raoufi, Aran}, year = {2026}, eprint = {2610.28543}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2610.28543}, } ```