Improve model card: Add tags, paper link, code, and usage example
Browse filesThis PR significantly improves the model card for the RLVER model. It includes:
- The `pipeline_tag: text-generation`, enhancing discoverability on the Hub.
- The `library_name: transformers`, ensuring the correct inference widget and usage instructions are displayed.
- A direct link to the official Hugging Face paper page: [RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents](https://huggingface.co/papers/2507.03112).
- A link to the GitHub repository for the project.
- A comprehensive description of the RLVER framework, drawing from the paper abstract and GitHub README.
- Relevant images from the GitHub repository to visualize the framework and results.
- A clear Python usage example using the `transformers` library, tailored for chat-based interaction.
- A BibTeX entry for easy citation.
These changes make the model much more discoverable, understandable, and usable for the community.
|
@@ -1,8 +1,90 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
license: other
|
| 3 |
license_name: license
|
| 4 |
license_link: LICENSE
|
| 5 |
-
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- Qwen/Qwen2.5-7B-Instruct
|
| 4 |
license: other
|
| 5 |
license_name: license
|
| 6 |
license_link: LICENSE
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
+
library_name: transformers
|
| 9 |
+
tags:
|
| 10 |
+
- reinforcement-learning
|
| 11 |
+
- empathetic-agent
|
| 12 |
+
- dialogue
|
| 13 |
+
- llm
|
| 14 |
+
- qwen
|
| 15 |
+
- instruct
|
| 16 |
---
|
| 17 |
+
|
| 18 |
+
# RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
|
| 19 |
+
|
| 20 |
+
This repository hosts the `Qwen2.5-7B-Instruct` model fine-tuned using **RLVER**, a novel reinforcement learning framework for cultivating higher-order empathetic abilities in Large Language Models (LLMs).
|
| 21 |
+
|
| 22 |
+
**RLVER** is the first end-to-end reinforcement learning framework that leverages verifiable emotion rewards from simulated users to guide the LLM's learning. By engaging self-consistent affective simulated users in dialogue rollouts that produce deterministic emotion scores, RLVER significantly boosts emotional intelligence while largely preserving other cognitive competencies, as demonstrated by an increase in Sentient-Benchmark scores from 13.3 to 79.2.
|
| 23 |
+
|
| 24 |
+
**[\ud83d\udcda Paper](https://huggingface.co/papers/2507.03112)** | **[\ud83d\udcbb Code](https://github.com/Tencent/digitalhuman/tree/main/RLVER)**
|
| 25 |
+
|
| 26 |
+
<p align="center"><img width="90%" src="https://github.com/Tencent/digitalhuman/raw/main/RLVER/code/figs/framework.png" alt="RLVER Framework Overview" /></p>
|
| 27 |
+
<p align="center"><em>The overview of RLVER. This work presents the first end-to-end reinforcement-learning framework that equips an LLM with human-level empathetic skills by optimizing against verifiable emotion rewards.</em></p>
|
| 28 |
+
|
| 29 |
+
<p align="center"><img width="90%" src="https://github.com/Tencent/digitalhuman/raw/main/RLVER/code/figs/main_result.png" alt="RLVER Main Result" /></p>
|
| 30 |
+
<p align="center"><em>The main result of RLVER, showcasing the improvement in Sentient-Benchmark score.</em></p>
|
| 31 |
+
|
| 32 |
+
## Usage
|
| 33 |
+
|
| 34 |
+
This model can be easily loaded and used for text generation with the `transformers` library. The model uses a chat template, so it's recommended to use `apply_chat_template` for optimal performance in conversational settings.
|
| 35 |
+
|
| 36 |
+
```python
|
| 37 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 38 |
+
import torch
|
| 39 |
+
|
| 40 |
+
model_id = "RLVER/RLVER-Qwen2.5-7B-Instruct" # Replace with actual repo name if different
|
| 41 |
+
|
| 42 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 43 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 44 |
+
model_id,
|
| 45 |
+
torch_dtype=torch.bfloat16, # Or torch.float16 depending on your hardware/preference
|
| 46 |
+
device_map="auto"
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
# Example conversation following the Qwen chat template
|
| 50 |
+
messages = [
|
| 51 |
+
{"role": "system", "content": "You are a helpful and empathetic assistant."},
|
| 52 |
+
{"role": "user", "content": "I'm feeling a bit overwhelmed with work lately. Any advice?"},
|
| 53 |
+
]
|
| 54 |
+
|
| 55 |
+
# Apply the chat template and tokenize
|
| 56 |
+
text = tokenizer.apply_chat_template(
|
| 57 |
+
messages,
|
| 58 |
+
tokenize=False,
|
| 59 |
+
add_generation_prompt=True
|
| 60 |
+
)
|
| 61 |
+
|
| 62 |
+
model_inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
| 63 |
+
|
| 64 |
+
# Generate a response
|
| 65 |
+
generated_ids = model.generate(
|
| 66 |
+
model_inputs.input_ids,
|
| 67 |
+
max_new_tokens=512,
|
| 68 |
+
do_sample=True,
|
| 69 |
+
temperature=0.7,
|
| 70 |
+
top_p=0.9,
|
| 71 |
+
eos_token_id=tokenizer.eos_token_id
|
| 72 |
+
)
|
| 73 |
+
|
| 74 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 75 |
+
print(response)
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
## Citation
|
| 79 |
+
|
| 80 |
+
If you find this work useful for your research, please consider citing the paper:
|
| 81 |
+
|
| 82 |
+
```bibtex
|
| 83 |
+
@article{zhou2024learning,
|
| 84 |
+
title={RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents},
|
| 85 |
+
author={Zhou, Yulin and Li, Yiran and Wang, Ruipeng and Feng, Yuanhui and Chen, Jingsheng and Zhu, Haoran and Lu, Ming and Lv, Qingwei and Liu, Hong and Li, Weichao and Wu, Xing and Chen, Wenliang},
|
| 86 |
+
journal={arXiv preprint arXiv:2507.03112},
|
| 87 |
+
year={2025},
|
| 88 |
+
url={https://arxiv.org/abs/2507.03112}
|
| 89 |
+
}
|
| 90 |
+
```
|