Improve model card: Add metadata, project page link, and sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +45 -10
README.md CHANGED
@@ -1,17 +1,56 @@
1
  ---
2
  license: mit
 
 
3
  ---
4
 
5
  # Introduction to TraDo
6
 
7
- [Paper](https://arxiv.org/abs/2509.06949) | [Code](https://github.com/Gen-Verse/dLLM-RL)
8
 
9
  We introduce **TraDo**, SOTA diffusion language model, trained with **TraceRL**.
10
 
11
- * **TraDo-4B-Instruct** and **TraDo-8B-Instruct** outperform similarly sized strong AR models across math reasoning tasks.
12
- * **TraDo-8B-Thinking** is the first Long-CoT diffusion language model.
13
-
14
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  <p align="center">
17
  <img src="https://github.com/yinjjiew/Data/raw/main/dllm-rl/figure1.png" width="100%"/>
@@ -23,8 +62,6 @@ We introduce **TraDo**, SOTA diffusion language model, trained with **TraceRL**.
23
  </p>
24
 
25
 
26
-
27
-
28
  # Citation
29
 
30
  ```
@@ -34,6 +71,4 @@ We introduce **TraDo**, SOTA diffusion language model, trained with **TraceRL**.
34
  journal={arXiv preprint arXiv:2509.06949},
35
  year={2025}
36
  }
37
- ```
38
-
39
-
 
1
  ---
2
  license: mit
3
+ pipeline_tag: text-generation
4
+ library_name: transformers
5
  ---
6
 
7
  # Introduction to TraDo
8
 
9
+ [Paper](https://arxiv.org/abs/2509.06949) | [Code](https://github.com/Gen-Verse/dLLM-RL) | [Project Page](https://huggingface.co/collections/Gen-Verse/trado-series-68beb6cd6a26c27cde9fe3af)
10
 
11
  We introduce **TraDo**, SOTA diffusion language model, trained with **TraceRL**.
12
 
13
+ * **TraDo-4B-Instruct** and **TraDo-8B-Instruct** outperform similarly sized strong AR models across math reasoning tasks.
14
+ * **TraDo-8B-Thinking** is the first Long-CoT diffusion language model.
15
+
16
+ ## Usage
17
+
18
+ You can download and try our model:
19
+ ```python
20
+ from transformers import AutoModelForCausalLM, AutoTokenizer
21
+ from generate import block_diffusion_generate
22
+
23
+ model_name = "Gen-Verse/TraDo-8B-Instruct"
24
+
25
+ model = AutoModelForCausalLM.from_pretrained(
26
+ model_name, trust_remote_code=True, torch_dtype="float16", device_map="cuda"
27
+ )
28
+ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
29
+
30
+ prompt = "What's the solution of x^2 - 2x + 1 = 0\
31
+ Please reason step by step, and put your final answer within \\\\boxed{}.\
32
+ "
33
+ messages = [{"role": "user", "content": prompt}]
34
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
35
+
36
+ tokens = tokenizer.batch_encode_plus([text], return_tensors='pt', padding=True, truncation=True, max_length=200)
37
+ tokens = {k: v.to(model.device) for k, v in tokens.items()}
38
+
39
+ output_ids = block_diffusion_generate(
40
+ model,
41
+ prompt=tokens,
42
+ mask_id=151669,
43
+ gen_length=200,
44
+ block_length=4, denoising_steps=4,
45
+ temperature=1.0, top_k=0, top_p=1.0,
46
+ remasking_strategy="low_confidence_dynamic",
47
+ confidence_threshold=0.9
48
+ )
49
+
50
+ output_text = tokenizer.decode(output_ids[0], skip_special_tokens=False)
51
+ cleaned_text = output_text.replace('<|MASK|>', '').replace('<|endoftext|>', '')
52
+ print(cleaned_text)
53
+ ```
54
 
55
  <p align="center">
56
  <img src="https://github.com/yinjjiew/Data/raw/main/dllm-rl/figure1.png" width="100%"/>
 
62
  </p>
63
 
64
 
 
 
65
  # Citation
66
 
67
  ```
 
71
  journal={arXiv preprint arXiv:2509.06949},
72
  year={2025}
73
  }
74
+ ```