Add pipeline tag, project links and sample usage
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,35 +1,46 @@
|
|
| 1 |
---
|
| 2 |
-
license: bsd-2-clause
|
| 3 |
-
language:
|
| 4 |
-
- en
|
| 5 |
base_model:
|
| 6 |
- GSAI-ML/LLaDA-8B-Instruct
|
|
|
|
|
|
|
| 7 |
library_name: transformers
|
|
|
|
|
|
|
| 8 |
tags:
|
| 9 |
- DLM
|
| 10 |
- EvoToken
|
| 11 |
- lora
|
| 12 |
---
|
| 13 |
-
# EvoTokenDLM LoRA adapter training from pretrained weights LLaDA-8B-Instruct
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
|
|
|
|
|
|
| 20 |
|
| 21 |
## How to Use
|
| 22 |
|
| 23 |
-
⚠️ **Important:** This is a LoRA adapter and requires the official EvoTokenDLM codebase for inference.
|
|
|
|
|
|
|
| 24 |
|
| 25 |
-
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Citation
|
| 29 |
|
| 30 |
If you find this work helpful for your research, please cite:
|
| 31 |
|
| 32 |
-
```
|
| 33 |
@article{zhong2026beyond,
|
| 34 |
title={Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models},
|
| 35 |
author={Zhong, Linhao and Wu, Linyu and Fang, Bozhen and Feng, Tianjian and Jing, Chenchen and Wang, Wen and Zhang, Jiaheng and Chen, Hao and Shen, Chunhua},
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
base_model:
|
| 3 |
- GSAI-ML/LLaDA-8B-Instruct
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
library_name: transformers
|
| 7 |
+
license: bsd-2-clause
|
| 8 |
+
pipeline_tag: text-generation
|
| 9 |
tags:
|
| 10 |
- DLM
|
| 11 |
- EvoToken
|
| 12 |
- lora
|
| 13 |
---
|
|
|
|
| 14 |
|
| 15 |
+
# EvoToken-DLM (LoRA Adapter)
|
| 16 |
|
| 17 |
+
[**Project Page**](https://aim-uofa.github.io/EvoTokenDLM/) | [**GitHub**](https://github.com/aim-uofa/EvoTokenDLM) | [**Paper**](https://arxiv.org/abs/2601.07351)
|
| 18 |
|
| 19 |
+
EvoToken-DLM is a novel diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. While most Diffusion Language Models (DLMs) rely on hard binary masking and discrete token assignments, which can hinder the revision of early decisions, EvoToken-DLM enables a progressive transition from masked states to discrete outputs, supporting revisable decoding.
|
| 20 |
+
|
| 21 |
+
This repository provides the LoRA adapter weights trained from [LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct) using **Continuous Trajectory Supervision**.
|
| 22 |
|
| 23 |
## How to Use
|
| 24 |
|
| 25 |
+
⚠️ **Important:** This is a LoRA adapter and requires the [official EvoTokenDLM codebase](https://github.com/aim-uofa/EvoTokenDLM) for inference.
|
| 26 |
+
|
| 27 |
+
### Sample Inference Command
|
| 28 |
|
| 29 |
+
Once you have set up the environment following the instructions in the official repository, you can run progressive inference using the following command:
|
| 30 |
|
| 31 |
+
```bash
|
| 32 |
+
python generate.py --model_path GSAI-ML/LLaDA-8B-Instruct \
|
| 33 |
+
--checkpoint_path zhongzero/EvoToken_LLaDA_Instruct_8B_Lora \
|
| 34 |
+
--prompt "Lily can run 12 kilometers per hour for 4 hours. After that, she runs 6 kilometers per hour. How many kilometers can she run in 8 hours?" \
|
| 35 |
+
--k_soft 3 \
|
| 36 |
+
--alpha_soft_mask 0.7
|
| 37 |
+
```
|
| 38 |
|
| 39 |
## Citation
|
| 40 |
|
| 41 |
If you find this work helpful for your research, please cite:
|
| 42 |
|
| 43 |
+
```bibtex
|
| 44 |
@article{zhong2026beyond,
|
| 45 |
title={Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models},
|
| 46 |
author={Zhong, Linhao and Wu, Linyu and Fang, Bozhen and Feng, Tianjian and Jing, Chenchen and Wang, Wen and Zhang, Jiaheng and Chen, Hao and Shen, Chunhua},
|