Binary Discrete Diffusion: models

The models of Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks: diffusion transformers for each task and noise kernel (masked, uniform, binary_symmetric, binary_absorbing), five training seeds on the logic tasks and one on text. Each run directory of binary-discrete-diffusion was trained with configs/paper/<task>/<kernel>.yaml and holds the composed config in .hydra/config.yaml and the EMA parameters the paper evaluates in checkpoints_ema/<step>/. The logic runs also hold their final training state (parameters, optimizer and time-sampler state) in checkpoints/<step>/, from which train.py resumes.

Task Directory EMA checkpoint Final training state Data
Sudoku sudoku/<kernel>/seed<0-4> checkpoints_ema/1000000 checkpoints/1000000 scripts/prep_sudoku.py
Boxoban boxoban/<kernel>/seed<0-4> checkpoints_ema/2500000; binary absorbing seeds 1 and 3 at 2000000 and 1000000 checkpoints/2500000 scripts/prep_boxoban.py
Graph coloring graphcolor/<kernel>/seed<0-4> checkpoints_ema/300000 checkpoints/300000 generated on the fly
Text8 text8/<kernel> checkpoints_ema/1000000 characters, written by scripts/prep_text8.py
TinyStories tinystories/<kernel> checkpoints_ema/500000 tokenizer rodcasnog/tinystories-bpe-1k
OpenWebText owt/<kernel> checkpoints_ema/250000 tokenizer dvruette/openwebtext-bpe-1k

The Boxoban EMA checkpoints were selected on validation NELBO, as described in the paper's appendix. Since train.py resumes only from a step that holds both checkpoints, Boxoban binary absorbing seeds 1 and 3 do not resume as downloaded.

Usage

From a clone of the code repository, after uv sync:

uv run hf download rodcasnog/binary-discrete-diffusion --include "sudoku/binary_symmetric/seed0/*" --local-dir outputs/hf
uv run python -m binary_diffusion.eval.solve checkpoint_dir=outputs/hf/sudoku/binary_symmetric/seed0/checkpoints_ema/1000000 \
    sampler.nfe=256 out=outputs/sudoku-bsym/solve
# Train the same run 200k steps longer.
uv run python -m binary_diffusion.train --config-name paper/sudoku/binary_symmetric training.seed=0 \
    training.num_steps=1200000 hydra.run.dir=outputs/hf/sudoku/binary_symmetric/seed0

uv run hf download rodcasnog/binary-discrete-diffusion --include "text8/binary_symmetric/*" --local-dir outputs/hf
uv run python -m binary_diffusion.sample checkpoint_dir=outputs/hf/text8/binary_symmetric/checkpoints_ema/1000000 \
    sampler.nfe=256 sampler.spacing=log sampler.eta=1 sampler.lam=1 out=outputs/text8-bsym/samples

Sudoku and Boxoban models read their task data in data/sudoku and data/boxoban. Text8 models read the tokenizer in data/text8, TinyStories samples take their lengths from data/tinystories, and NELBO and MAUVE read the corpus; the code's README lists the data and evaluation commands.

License

Apache-2.0.

Citation

@misc{casadonoguerales2026binarydiffusion,
  title         = {Binary Discrete Diffusion: Candidate-Set Diffusion for
                   Logic and Planning Tasks},
  author        = {Casado Noguerales, Rodrigo and von R{\"u}tte, Dimitri
                   and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas
                   and Raoufi, Aran},
  year          = {2026},
  eprint        = {2610.28543},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2610.28543},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train rodcasnog/binary-discrete-diffusion

Paper for rodcasnog/binary-discrete-diffusion