Fetchniche commited on
Commit
c3a0730
·
verified ·
1 Parent(s): c7720d8

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +80 -0
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - GSAI-ML/LLaDA-8B-Instruct
4
+ pipeline_tag: text-generation
5
+ ---
6
+ # RemeDi: <u><b>Rem</b></u>asking-<u><b>e</b></u>nabled <u><b>Di</b></u>ffusion Language Model
7
+
8
+
9
+ <div align="center">
10
+
11
+
12
+ [![weixin](https://img.shields.io/badge/-WeChat@MAPLE实验室-000000?logo=wechat&logoColor=07C160)](https://mp.weixin.qq.com/s/UefnjlCSi6YvzVe-Xu9jjQ)
13
+ [![RemeDi](https://img.shields.io/badge/Paper-RemeDi-2b9348.svg?logo=arXiv)](https://arxiv.org/abs/2509.23653)&#160;
14
+
15
+ [![Static Badge](https://img.shields.io/badge/Model(9B)-yellow?logoColor=violet&label=%F0%9F%A4%97%20RemeDi-Instruct%20checkpoint)](https://huggingface.co/maple-research-lab/RemeDi-Instruct)&#160;
16
+ [![Static Badge](https://img.shields.io/badge/Model(9B)-yellow?logoColor=violet&label=%F0%9F%A4%97%20RemeDi-RL%20checkpoints)](https://huggingface.co/maple-research-lab/RemeDi-RL)&#160;
17
+
18
+ </div>
19
+
20
+ # 🔬 Method Overview
21
+
22
+ RemeDi lets every token be revised at every diffusion step. Instead of fixing in an early guess, the model evaluates the quality of each token and can remask low-confidence positions, allowing later steps to resample them with richer context—built-in self-correction.
23
+
24
+ RemeDi extends the original model with a dual-stream transformer:
25
+
26
+ - Token Prediction Stream (TPS) predicts masked tokens as usual.
27
+
28
+ - Unmasking Policy Stream (UPS) outputs per-token confidence scores, deciding which tokens to unmask or remask.
29
+
30
+
31
+ At each denoising step, tokens with low confidence can be remasked and resampled, enabling iterative refinement.
32
+ For the training and RL algorithms, see the Methods section of the paper.
33
+
34
+ <p align="center">
35
+ <!-- Replace with the actual image path -->
36
+ <img src="figures/figure1.png" alt="RemeDi architecture and performance radar" width="600">
37
+ </p>
38
+
39
+
40
+
41
+ # 📈 Key Results
42
+
43
+
44
+ <p align="center">
45
+ <!-- Replace with the actual image path -->
46
+ <img src="figures/figure2.png" alt="RemeDi performance table" width="600">
47
+ </p>
48
+
49
+
50
+
51
+ # 📂 Repository Structure
52
+
53
+ ```
54
+ ├── inference.py # inference scripts
55
+ ├── remedi/ # networks configs
56
+ └── README.md
57
+ ```
58
+
59
+ # 🚀 Inference
60
+
61
+ To run inference, execute:
62
+
63
+ ```sh
64
+ git clone https://github.com/maple-research-lab/RemeDi.git
65
+ cd RemeDi
66
+
67
+ # chat with remedi
68
+ python inference.py
69
+ ```
70
+
71
+ # 📥 Citation
72
+
73
+ ```
74
+ @article{huang2025don,
75
+ title={Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models},
76
+ author={Huang, Zemin and Wang, Yuhang and Chen, Zhiyang and Qi, Guo-Jun},
77
+ journal={arXiv preprint arXiv:2509.23653},
78
+ year={2025}
79
+ }
80
+ ```