nielsr HF Staff commited on
Commit
deb6754
·
verified ·
1 Parent(s): afbf61c

Improve model card: Add tags, paper link, code, and usage example

Browse files

This PR significantly improves the model card for the RLVER model. It includes:
- The `pipeline_tag: text-generation`, enhancing discoverability on the Hub.
- The `library_name: transformers`, ensuring the correct inference widget and usage instructions are displayed.
- A direct link to the official Hugging Face paper page: [RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents](https://huggingface.co/papers/2507.03112).
- A link to the GitHub repository for the project.
- A comprehensive description of the RLVER framework, drawing from the paper abstract and GitHub README.
- Relevant images from the GitHub repository to visualize the framework and results.
- A clear Python usage example using the `transformers` library, tailored for chat-based interaction.
- A BibTeX entry for easy citation.

These changes make the model much more discoverable, understandable, and usable for the community.

Files changed (1) hide show
  1. README.md +85 -3
README.md CHANGED
@@ -1,8 +1,90 @@
1
  ---
 
 
2
  license: other
3
  license_name: license
4
  license_link: LICENSE
5
- base_model:
6
- - Qwen/Qwen2.5-7B-Instruct
 
 
 
 
 
 
 
7
  ---
8
- https://www.arxiv.org/abs/2507.03112
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model:
3
+ - Qwen/Qwen2.5-7B-Instruct
4
  license: other
5
  license_name: license
6
  license_link: LICENSE
7
+ pipeline_tag: text-generation
8
+ library_name: transformers
9
+ tags:
10
+ - reinforcement-learning
11
+ - empathetic-agent
12
+ - dialogue
13
+ - llm
14
+ - qwen
15
+ - instruct
16
  ---
17
+
18
+ # RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents
19
+
20
+ This repository hosts the `Qwen2.5-7B-Instruct` model fine-tuned using **RLVER**, a novel reinforcement learning framework for cultivating higher-order empathetic abilities in Large Language Models (LLMs).
21
+
22
+ **RLVER** is the first end-to-end reinforcement learning framework that leverages verifiable emotion rewards from simulated users to guide the LLM's learning. By engaging self-consistent affective simulated users in dialogue rollouts that produce deterministic emotion scores, RLVER significantly boosts emotional intelligence while largely preserving other cognitive competencies, as demonstrated by an increase in Sentient-Benchmark scores from 13.3 to 79.2.
23
+
24
+ **[\ud83d\udcda Paper](https://huggingface.co/papers/2507.03112)** | **[\ud83d\udcbb Code](https://github.com/Tencent/digitalhuman/tree/main/RLVER)**
25
+
26
+ <p align="center"><img width="90%" src="https://github.com/Tencent/digitalhuman/raw/main/RLVER/code/figs/framework.png" alt="RLVER Framework Overview" /></p>
27
+ <p align="center"><em>The overview of RLVER. This work presents the first end-to-end reinforcement-learning framework that equips an LLM with human-level empathetic skills by optimizing against verifiable emotion rewards.</em></p>
28
+
29
+ <p align="center"><img width="90%" src="https://github.com/Tencent/digitalhuman/raw/main/RLVER/code/figs/main_result.png" alt="RLVER Main Result" /></p>
30
+ <p align="center"><em>The main result of RLVER, showcasing the improvement in Sentient-Benchmark score.</em></p>
31
+
32
+ ## Usage
33
+
34
+ This model can be easily loaded and used for text generation with the `transformers` library. The model uses a chat template, so it's recommended to use `apply_chat_template` for optimal performance in conversational settings.
35
+
36
+ ```python
37
+ from transformers import AutoModelForCausalLM, AutoTokenizer
38
+ import torch
39
+
40
+ model_id = "RLVER/RLVER-Qwen2.5-7B-Instruct" # Replace with actual repo name if different
41
+
42
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
43
+ model = AutoModelForCausalLM.from_pretrained(
44
+ model_id,
45
+ torch_dtype=torch.bfloat16, # Or torch.float16 depending on your hardware/preference
46
+ device_map="auto"
47
+ )
48
+
49
+ # Example conversation following the Qwen chat template
50
+ messages = [
51
+ {"role": "system", "content": "You are a helpful and empathetic assistant."},
52
+ {"role": "user", "content": "I'm feeling a bit overwhelmed with work lately. Any advice?"},
53
+ ]
54
+
55
+ # Apply the chat template and tokenize
56
+ text = tokenizer.apply_chat_template(
57
+ messages,
58
+ tokenize=False,
59
+ add_generation_prompt=True
60
+ )
61
+
62
+ model_inputs = tokenizer(text, return_tensors="pt").to(model.device)
63
+
64
+ # Generate a response
65
+ generated_ids = model.generate(
66
+ model_inputs.input_ids,
67
+ max_new_tokens=512,
68
+ do_sample=True,
69
+ temperature=0.7,
70
+ top_p=0.9,
71
+ eos_token_id=tokenizer.eos_token_id
72
+ )
73
+
74
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
75
+ print(response)
76
+ ```
77
+
78
+ ## Citation
79
+
80
+ If you find this work useful for your research, please consider citing the paper:
81
+
82
+ ```bibtex
83
+ @article{zhou2024learning,
84
+ title={RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents},
85
+ author={Zhou, Yulin and Li, Yiran and Wang, Ruipeng and Feng, Yuanhui and Chen, Jingsheng and Zhu, Haoran and Lu, Ming and Lv, Qingwei and Liu, Hong and Li, Weichao and Wu, Xing and Chen, Wenliang},
86
+ journal={arXiv preprint arXiv:2507.03112},
87
+ year={2025},
88
+ url={https://arxiv.org/abs/2507.03112}
89
+ }
90
+ ```