| --- |
| library_name: pytorch |
| license: mit |
| pipeline_tag: text-generation |
| tags: |
| - gdds |
| - discrete-diffusion |
| - language-modeling |
| - research |
| - pytorch |
| --- |
| |
| # GDDS Checkpoints |
|
|
| Official checkpoint bundle for the paper **Generalized Discrete Diffusion from Snapshots**. |
|
|
| Generalized Discrete Diffusion from Snapshots (GDDS) is a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. It introduces a training objective based on snapshot latents rather than the entire noising path, allowing for efficient training and high-quality generation. |
|
|
| ## Model Sources |
|
|
| - **Paper:** [Generalized Discrete Diffusion from Snapshots](https://huggingface.co/papers/2603.21342) |
| - **arXiv:** [2603.21342](https://arxiv.org/abs/2603.21342) |
| - **Code:** [GitHub - ozekri/gdds](https://github.com/ozekri/gdds) |
| - **Project Page:** [https://oussamazekri.fr/gdds](https://oussamazekri.fr/gdds) |
|
|
| ## Included Checkpoints |
|
|
| | File | Method | Notes | |
| | --- | --- | --- | |
| | `checkpoints/gdds_gauss_500k.ckpt` | GDDS | 500k-step checkpoint with the Gaussian SIK forward process | |
| | `checkpoints/gdds_uniform_500k.ckpt` | GDDS | 500k-step checkpoint with the uniform forward process | |
| | `checkpoints/gdds_absorb_500k.ckpt` | GDDS | 500k-step checkpoint with the absorbing forward process | |
| | `checkpoints/mdlm_500k.ckpt` | MDLM | 500k-step baseline checkpoint | |
| | `checkpoints/udlm_500k.ckpt` | UDLM | 500k-step baseline checkpoint | |
| | `checkpoints/ar_500k.ckpt` | AR | 500k-step autoregressive baseline checkpoint | |
|
|
| ## Usage |
|
|
| These files are PyTorch Lightning checkpoints intended to be used with the [`gdds`](https://github.com/ozekri/gdds) codebase. |
|
|
| ```bash |
| git clone https://github.com/ozekri/gdds.git |
| cd gdds |
| pip install -r requirements.txt |
| pip install -e . |
| |
| # Example evaluation using a checkpoint |
| PYTHONPATH=src python -m discrete_diffusion.evaluations.ppl_eval \ |
| data=openwebtext \ |
| model=small \ |
| algo=mdlm \ |
| eval.checkpoint_path=/path/to/checkpoints/mdlm_500k.ckpt |
| ``` |
|
|
| For sampling and other evaluations, use the same repository and pass the relevant checkpoint path through the Hydra evaluation config. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{zekri2026generalizeddiscretediffusionsnapshots, |
| title={Generalized Discrete Diffusion from Snapshots}, |
| author={Oussama Zekri and Th{\\'e}o Uscidda and Nicolas Boull{\\'e} and Anna Korba}, |
| year={2026}, |
| eprint={2603.21342}, |
| archivePrefix={arXiv}, |
| primaryClass={stat.ML}, |
| url={https://arxiv.org/abs/2603.21342}, |
| } |
| ``` |
