---
license: bsd-2-clause
language:
- en
base_model:
- GSAI-ML/LLaDA-8B-Instruct
library_name: transformers
tags:
- DLM
- EvoToken
- lora
- text-generation
---
# EvoTokenDLM LoRA adapter training from pretrained weights LLaDA-8B-Instruct

Starting from the original MDLM (Masked Discrete Diffusion Language Model) LLaDA-8B-Instruct, we trained the EvoTokenDLM LoRA adapter using the **Continuous Trajectory Supervision** method.

Our implementation replaces traditional hard binary masks with evolving soft token distributions. This allows EvoTokenDLM to facilitate a progressive transition from masked states to discrete outputs, effectively supporting revisable decoding.

The method and its results are detailed in the paper: [Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models](https://arxiv.org/abs/2601.07351).

## How to Use

⚠️ **Important:** This is a LoRA adapter and requires the official EvoTokenDLM codebase for inference. 

For detailed instructions and code, please refer to the official GitHub repository: [EvoTokenDLM GitHub Repository](https://github.com/aim-uofa/EvoTokenDLM)


## Citation

If you find this work helpful for your research, please cite:

```BibTeX
@article{zhong2026beyond,
    title={Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models},
    author={Zhong, Linhao and Wu, Linyu and Fang, Bozhen and Feng, Tianjian and Jing, Chenchen and Wang, Wen and Zhang, Jiaheng and Chen, Hao and Shen, Chunhua},
    journal={arXiv preprint arXiv:2601.07351},
    year={2026}
}
```