Add pipeline tag, project links and sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +21 -10
README.md CHANGED
@@ -1,35 +1,46 @@
1
  ---
2
- license: bsd-2-clause
3
- language:
4
- - en
5
  base_model:
6
  - GSAI-ML/LLaDA-8B-Instruct
 
 
7
  library_name: transformers
 
 
8
  tags:
9
  - DLM
10
  - EvoToken
11
  - lora
12
  ---
13
- # EvoTokenDLM LoRA adapter training from pretrained weights LLaDA-8B-Instruct
14
 
15
- Starting from the original MDLM (Masked Discrete Diffusion Language Model) LLaDA-8B-Instruct, we trained the EvoTokenDLM LoRA adapter using the **Continuous Trajectory Supervision** method.
16
 
17
- Our implementation replaces traditional hard binary masks with evolving soft token distributions. This allows EvoTokenDLM to facilitate a progressive transition from masked states to discrete outputs, effectively supporting revisable decoding.
18
 
19
- The method and its results are detailed in the paper: [Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models](https://arxiv.org/abs/2601.07351).
 
 
20
 
21
  ## How to Use
22
 
23
- ⚠️ **Important:** This is a LoRA adapter and requires the official EvoTokenDLM codebase for inference.
 
 
24
 
25
- For detailed instructions and code, please refer to the official GitHub repository: [EvoTokenDLM GitHub Repository](https://github.com/aim-uofa/EvoTokenDLM)
26
 
 
 
 
 
 
 
 
27
 
28
  ## Citation
29
 
30
  If you find this work helpful for your research, please cite:
31
 
32
- ```BibTeX
33
  @article{zhong2026beyond,
34
  title={Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models},
35
  author={Zhong, Linhao and Wu, Linyu and Fang, Bozhen and Feng, Tianjian and Jing, Chenchen and Wang, Wen and Zhang, Jiaheng and Chen, Hao and Shen, Chunhua},
 
1
  ---
 
 
 
2
  base_model:
3
  - GSAI-ML/LLaDA-8B-Instruct
4
+ language:
5
+ - en
6
  library_name: transformers
7
+ license: bsd-2-clause
8
+ pipeline_tag: text-generation
9
  tags:
10
  - DLM
11
  - EvoToken
12
  - lora
13
  ---
 
14
 
15
+ # EvoToken-DLM (LoRA Adapter)
16
 
17
+ [**Project Page**](https://aim-uofa.github.io/EvoTokenDLM/) | [**GitHub**](https://github.com/aim-uofa/EvoTokenDLM) | [**Paper**](https://arxiv.org/abs/2601.07351)
18
 
19
+ EvoToken-DLM is a novel diffusion-based language modeling approach that replaces hard binary masks with evolving soft token distributions. While most Diffusion Language Models (DLMs) rely on hard binary masking and discrete token assignments, which can hinder the revision of early decisions, EvoToken-DLM enables a progressive transition from masked states to discrete outputs, supporting revisable decoding.
20
+
21
+ This repository provides the LoRA adapter weights trained from [LLaDA-8B-Instruct](https://huggingface.co/GSAI-ML/LLaDA-8B-Instruct) using **Continuous Trajectory Supervision**.
22
 
23
  ## How to Use
24
 
25
+ ⚠️ **Important:** This is a LoRA adapter and requires the [official EvoTokenDLM codebase](https://github.com/aim-uofa/EvoTokenDLM) for inference.
26
+
27
+ ### Sample Inference Command
28
 
29
+ Once you have set up the environment following the instructions in the official repository, you can run progressive inference using the following command:
30
 
31
+ ```bash
32
+ python generate.py --model_path GSAI-ML/LLaDA-8B-Instruct \
33
+ --checkpoint_path zhongzero/EvoToken_LLaDA_Instruct_8B_Lora \
34
+ --prompt "Lily can run 12 kilometers per hour for 4 hours. After that, she runs 6 kilometers per hour. How many kilometers can she run in 8 hours?" \
35
+ --k_soft 3 \
36
+ --alpha_soft_mask 0.7
37
+ ```
38
 
39
  ## Citation
40
 
41
  If you find this work helpful for your research, please cite:
42
 
43
+ ```bibtex
44
  @article{zhong2026beyond,
45
  title={Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models},
46
  author={Zhong, Linhao and Wu, Linyu and Fang, Bozhen and Feng, Tianjian and Jing, Chenchen and Wang, Wen and Zhang, Jiaheng and Chen, Hao and Shen, Chunhua},