Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ base_model:
|
|
| 24 |
</a>
|
| 25 |
<br>
|
| 26 |
<a href="https://huggingface.co/HeAAAAA/Crab" style="margin: 0 10px;">
|
| 27 |
-
💬 <strong>Role-
|
| 28 |
</a> |
|
| 29 |
<a href="https://huggingface.co/HeAAAAA/RoleRM" style="margin: 0 10px;">
|
| 30 |
💬 <strong>Role-palying Evaluation Model</strong>
|
|
@@ -38,8 +38,10 @@ base_model:
|
|
| 38 |
</a> |
|
| 39 |
<a href="https://huggingface.co/datasets/HeAAAAA/Crab-manually-annotated-role-playing-evaluation-dataset" style="margin: 0 10px;">
|
| 40 |
💬 <strong>Annotated Role-playing Evaluation Dataset</strong>
|
|
|
|
|
|
|
|
|
|
| 41 |
</a>
|
| 42 |
-
|
| 43 |
</p>
|
| 44 |
|
| 45 |
</div>
|
|
@@ -58,7 +60,8 @@ Thus, to validate RP-LLMs' effectiveness, we introduced a new benchmark containi
|
|
| 58 |
Sufficient experiments reveal that RoleRM significantly outperforms ChatGPT and other evaluation methods in conducting fine-grained evaluations of RP.
|
| 59 |
Also, RP-LLMs powered by Crab demonstrate superior performance across various fine-grained aspects.
|
| 60 |
|
| 61 |
-
More details can be seen at
|
|
|
|
| 62 |
|
| 63 |
|
| 64 |
|
|
@@ -119,33 +122,32 @@ Table 2: The ablation study for Crab. Due to missing attributes in our dataset,
|
|
| 119 |
|
| 120 |
<br>
|
| 121 |
|
| 122 |
-
# 4.
|
| 123 |
-
We publish three datasets, including
|
| 124 |
-
|
| 125 |
-
## 4.1 Crab role-playing train set:
|
| 126 |
-
{https://huggingface.co/datasets/HeAAAAA/Crab-role-playing-train-set}
|
| 127 |
-
|
| 128 |
-
## 4.2 Crab role-playing evaluation benchmark:
|
| 129 |
-
{https://huggingface.co/datasets/HeAAAAA/Crab-role-playing-evaluation-benchmark}
|
| 130 |
-
|
| 131 |
-
## 4.3 Crab manually annotated role-playing evaluation dataset:
|
| 132 |
-
{https://huggingface.co/datasets/HeAAAAA/Crab-manually-annotated-role-playing-evaluation-dataset}
|
| 133 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
|
| 135 |
<br>
|
| 136 |
|
| 137 |
# 5. Fine-tuned Role-playing Model
|
| 138 |
-
We release a fine-tuned
|
| 139 |
-
|
|
|
|
| 140 |
|
| 141 |
<br>
|
| 142 |
|
| 143 |
# 6. Role-palying Evaluation Model
|
| 144 |
-
We release a trained
|
| 145 |
-
|
|
|
|
| 146 |
|
| 147 |
<br>
|
| 148 |
|
|
|
|
|
|
|
| 149 |
# 7. Citation
|
| 150 |
|
| 151 |
```bibtex
|
|
@@ -155,7 +157,3 @@ We release a trained model to automate the evaluation of role-playing tasks.
|
|
| 155 |
year={2025},
|
| 156 |
}
|
| 157 |
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
|
|
|
| 24 |
</a>
|
| 25 |
<br>
|
| 26 |
<a href="https://huggingface.co/HeAAAAA/Crab" style="margin: 0 10px;">
|
| 27 |
+
💬 <strong>Role-playing Model</strong>
|
| 28 |
</a> |
|
| 29 |
<a href="https://huggingface.co/HeAAAAA/RoleRM" style="margin: 0 10px;">
|
| 30 |
💬 <strong>Role-palying Evaluation Model</strong>
|
|
|
|
| 38 |
</a> |
|
| 39 |
<a href="https://huggingface.co/datasets/HeAAAAA/Crab-manually-annotated-role-playing-evaluation-dataset" style="margin: 0 10px;">
|
| 40 |
💬 <strong>Annotated Role-playing Evaluation Dataset</strong>
|
| 41 |
+
</a> |
|
| 42 |
+
<a href="https://huggingface.co/datasets/HeAAAAA/Crab-human-preference" style="margin: 0 10px;">
|
| 43 |
+
💬 <strong>Human-preference Dataset</strong>
|
| 44 |
</a>
|
|
|
|
| 45 |
</p>
|
| 46 |
|
| 47 |
</div>
|
|
|
|
| 60 |
Sufficient experiments reveal that RoleRM significantly outperforms ChatGPT and other evaluation methods in conducting fine-grained evaluations of RP.
|
| 61 |
Also, RP-LLMs powered by Crab demonstrate superior performance across various fine-grained aspects.
|
| 62 |
|
| 63 |
+
More details can be seen at [GitHub](https://github.com/KaiHe-better/Crab?tab=readme-ov-file).
|
| 64 |
+
|
| 65 |
|
| 66 |
|
| 67 |
|
|
|
|
| 122 |
|
| 123 |
<br>
|
| 124 |
|
| 125 |
+
# 4. Four Datasets
|
| 126 |
+
We totally publish three datasets, including :
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
+
1. [Crab role-playing train set](https://huggingface.co/datasets/HeAAAAA/Crab-role-playing-train-set) : the dataset used for fine‑tuning a role‑playing LLM.
|
| 129 |
+
2. [Crab role-playing evaluation benchmark](https://huggingface.co/datasets/HeAAAAA/Crab-role-playing-evaluation-benchmark) :the dataset used for evalauating a role‑playing LLM.
|
| 130 |
+
3. [Manually annotated role-playing evaluation dataset](https://huggingface.co/datasets/HeAAAAA/Crab-manually-annotated-role-playing-evaluation-dataset): the dataset used for training a evaluator for role‑playing tasks.
|
| 131 |
+
4. [Crab Human preference dataset](https://huggingface.co/datasets/HeAAAAA/Crab-Human-preference): the dataset used to train a role‑playing LLM via reinforcement learning
|
| 132 |
|
| 133 |
<br>
|
| 134 |
|
| 135 |
# 5. Fine-tuned Role-playing Model
|
| 136 |
+
We release a fine-tuned Role-playin LLM to achieve configurable Role-Playing tasks:
|
| 137 |
+
|
| 138 |
+
[Download Link](https://huggingface.co/HeAAAAA/Crab)
|
| 139 |
|
| 140 |
<br>
|
| 141 |
|
| 142 |
# 6. Role-palying Evaluation Model
|
| 143 |
+
We release a trained LLM to automate the evaluation of role-playing tasks:
|
| 144 |
+
|
| 145 |
+
[Download Link](https://huggingface.co/HeAAAAA/RoleRM)
|
| 146 |
|
| 147 |
<br>
|
| 148 |
|
| 149 |
+
|
| 150 |
+
|
| 151 |
# 7. Citation
|
| 152 |
|
| 153 |
```bibtex
|
|
|
|
| 157 |
year={2025},
|
| 158 |
}
|
| 159 |
|
|
|
|
|
|
|
|
|
|
|
|