Improve model card: Add pipeline tag, library, links, and sample usage

README.md

@@ -1,9 +1,70 @@
 ---
-license: mit
-datasets:
-- openai/gsm8k
 base_model:
 - GSAI-ML/LLaDA-8B-Instruct
+datasets:
+- openai/gsm8k
+license: mit
+pipeline_tag: text-generation
+library_name: transformers
 ---
 
-Post-Training Lora models on gsm8k task based on LLaDA-8B-Instruct for the paper Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
+# ESPO: Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
+
+This repository provides the official implementation and checkpoints for the paper:
+[**Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective**](https://huggingface.co/papers/2512.03759)
+
+**Project Page:** [https://jingyangou.github.io/ESPO-Demo/](https://jingyangou.github.io/ESPO-Demo/)
+**Code Repository:** [https://github.com/ML-GSAI/ESPO](https://github.com/ML-GSAI/ESPO)
+
+We introduce **ESPO (ELBO-based Sequence-level Policy Optimization)**, a principled reinforcement learning framework that adapts to diffusion large language models (dLLMs). This framework addresses the fundamental challenges of applying traditional RL methods to dLLMs by viewing the **entire sequence generation as a single action** and leveraging the **ELBO** as a tractable proxy for sequence-level likelihood.
+
+ESPO introduces:
+-   **Sequence-level optimization** for diffusion LLMs via the ELBO objective.
+-   **Per-token normalized ratio estimation** and **robust KL regularization** for stable large-scale training.
+-   **Consistent gains** across math, coding, and planning benchmarks, significantly outperforming token-level baselines.
+
+## Model Details
+This specific model is an ESPO-fine-tuned LoRA adapter built on `GSAI-ML/LLaDA-8B-Instruct` for the GSM8K task. ESPO-GSM8K, ESPO-Math, ESPO-Countdown, and ESPO-Sudoku are provided as LoRA adapters, which can be loaded on top of the base LLaDA-8B-Instruct model for lightweight and efficient fine-tuning.
+
+## Sample Usage (Quick Start)
+You can load and use the ESPO-fine-tuned LoRA adapters with the `transformers` and `peft` libraries. Here's an example for inference using an ESPO adapter (e.g., ESPO-Math):
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+from eval.generate_utils import generate # Note: This `generate` function is a custom utility from the ESPO codebase.
+import torch
+
+base_model_path = 'GSAI-ML/LLaDA-8B-Instruct'
+peft_model_path = 'GSAI-ML/ESPO-Math' # Use 'GSAI-ML/ESPO-GSM8K' for this model, or other ESPO adapters.
+
+tokenizer = AutoTokenizer.from_pretrained(base_model_path)
+model = AutoModelForCausalLM.from_pretrained(
+    base_model_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="cuda")
+
+peft_model = PeftModel.from_pretrained(model, peft_model_path, device_map="cuda")
+
+prompt = "The point $(0,0)$ is reflected over the vertical line $x=1$.  When its image is then reflected over the line $y=2$, what is the resulting point?\
+\
+Write your answer in the form $(x, y)$ where $x$ and $y$ are real numbers."
+messages = [{"role": "user", "content": prompt}]
+
+input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
+
+output_ids = generate(peft_model, input_ids, tokenizer, steps=128, gen_length=256, temperature=0.9, remasking="low_confidence")
+
+output_text = tokenizer.batch_decode(output_ids[:, input_ids.shape[1]:], skip_special_tokens=True)[0]
+print(output_text)
+```
+
+## Citation
+If you find ESPO useful in your research, please consider citing our paper:
+
+```bibtex
+@article{ou2025principledrldiffusionllms,
+      title={Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective}, 
+      author={Jingyang Ou and Jiaqi Han and Minkai Xu and Shaoxuan Xu and Jianwen Xie and Stefano Ermon and Yi Wu and Chongxuan Li},
+      journal={arXiv preprint arXiv:2512.03759},
+      year={2025},
+}
+```