pipeline_tag: text-generation
RADD-Medium (lambda-dce)
Reparameterized Absorbing Discrete Diffusion (RADD) medium model with lambda-dce loss trained for 400k iterations.
This model was introduced in the paper Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data.
Model Description
RADD is a dedicated diffusion model without time-condition that characterizes time-independent conditional probabilities. This architecture unifies training objectives for absorbing discrete diffusion and any-order autoregressive models (AO-ARMs). The removal of the time condition allows for caching strategies that significantly improve sampling speed. This specific checkpoint is the medium version (approx. 405M parameters) trained using the $\lambda$-DCE loss function.
Links
- Paper: Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
- Code: Official GitHub Repository
Sample Usage
To load the model and noise schedule, you can use the following code (requires the load_model.py script from the official GitHub repository):
from load_model import load_model
# Load model and noise schedule
model, noise = load_model('JingyangOu/radd-lambda-dce-medium', device='cuda')
Citation
@misc{ou2024absorbingdiscretediffusionsecretly,
title={Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data},
author={Jingyang Ou and Shen Nie and Kaiwen Xue and Fengqi Zhu and Jiacheng Sun and Zhenguo Li and Chongxuan Li},
year={2024},
eprint={2406.03736},
archivePrefix={arXiv},
primaryClass={cs.LG},
}