Update README.md
Browse files
README.md
CHANGED
|
@@ -13,16 +13,13 @@ pipeline_tag: text-generation
|
|
| 13 |
|
| 14 |
# 🐷SUS-Chat: Instruction tuning done right
|
| 15 |
|
| 16 |
-
<p align="left">
|
| 17 |
-
<a href="README_CN.md">中文</a>  |  English 
|
| 18 |
-
</p>
|
| 19 |
|
| 20 |
-
<br><br>
|
| 21 |
|
| 22 |
<div align="center">
|
| 23 |
|
| 24 |
<p align="center">
|
| 25 |
-
<img
|
|
|
|
| 26 |
</p>
|
| 27 |
|
| 28 |
<div style="display: inline-block;">
|
|
@@ -44,7 +41,7 @@ pipeline_tag: text-generation
|
|
| 44 |
<div style="display: inline-block;">
|
| 45 |
|
| 46 |
<a rel="noopener nofollow" href="https://www.modelscope.cn/organization/sustc/">
|
| 47 |
-
<img src="https://img.shields.io/badge/ModelScope-
|
| 48 |
</a>
|
| 49 |
|
| 50 |
</div>
|
|
@@ -78,14 +75,15 @@ pipeline_tag: text-generation
|
|
| 78 |
# News
|
| 79 |
|
| 80 |
- 2023-12-05: SUS-Chat is ranked 2nd in [Open LLM
|
| 81 |
-
leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
|
|
|
| 82 |
|
| 83 |
- 2023-12-01: SUS-Chat-34B is now avaliable on HuggingFace🤗.
|
| 84 |
|
| 85 |
# Inrtoduction
|
| 86 |
|
| 87 |
<img src="https://hackmd.io/_uploads/HJlDtzhBa.png" id="fig-sus"
|
| 88 |
-
alt="Figure
|
| 89 |
|
| 90 |
**SUS-Chat** is a 34B bilingual Chinese-English dialogue model, jointly
|
| 91 |
released by the **Southern University of Science and Technology** and
|
|
@@ -96,8 +94,8 @@ model, the SUS-Chat-34B model has improved the model’s response to human
|
|
| 96 |
instructions through high-quality instruction fine-tuning and excels at
|
| 97 |
imitating human thought processes through chains of thought. It
|
| 98 |
introduces inter-instruction attention sharing in long texts, expanding
|
| 99 |
-
the window
|
| 100 |
-
|
| 101 |
|
| 102 |
It has surpassed all models of the same size in almost all benchmark
|
| 103 |
tests and is better suited to meet the practical needs of complex
|
|
@@ -142,7 +140,9 @@ similar scale and achieved the most advanced comprehensive performance.
|
|
| 142 |
| OrionStar-34B | 68.51 | 66.88 | 65.13 | 54.36 | 62.88 | 12.8 | 77.27 | 80.19 | 84.54 | 53.24 | 62.58 |
|
| 143 |
| Yi-34B-Chat | 66.96 | 55.16 | 77.16 | 63.76 | 61.54 | 10.02 | 76.64 | 70.66 | 82.29 | 54.57 | 61.876 |
|
| 144 |
|
| 145 |
-
<img
|
|
|
|
|
|
|
| 146 |
|
| 147 |
# Usage
|
| 148 |
|
|
@@ -158,7 +158,7 @@ def chat_template(messages):
|
|
| 158 |
history = ""
|
| 159 |
for message in messages:
|
| 160 |
match message:
|
| 161 |
-
case {"role": "
|
| 162 |
history += f"### Human: {message}\n\n### Assistant: "
|
| 163 |
case {"role": "assistant", "content": message}:
|
| 164 |
history += message
|
|
@@ -222,4 +222,4 @@ model.
|
|
| 222 |
This model is developed entirely for academic research and free
|
| 223 |
commercial use, but it must adhere to the
|
| 224 |
[license](https://github.com/SUSTech-IDEA/SUS-Chat/blob/main/MODEL_LICENSE_AGREEMENT.txt)
|
| 225 |
-
from 01-ai.
|
|
|
|
| 13 |
|
| 14 |
# 🐷SUS-Chat: Instruction tuning done right
|
| 15 |
|
|
|
|
|
|
|
|
|
|
| 16 |
|
|
|
|
| 17 |
|
| 18 |
<div align="center">
|
| 19 |
|
| 20 |
<p align="center">
|
| 21 |
+
<img src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/sustech.svg?sanitize=true" width="200px">
|
| 22 |
+
<img src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/ccnl.png?sanitize=true" width="200px">
|
| 23 |
</p>
|
| 24 |
|
| 25 |
<div style="display: inline-block;">
|
|
|
|
| 41 |
<div style="display: inline-block;">
|
| 42 |
|
| 43 |
<a rel="noopener nofollow" href="https://www.modelscope.cn/organization/sustc/">
|
| 44 |
+
<img src="https://img.shields.io/badge/ModelScope-sustc-blue" style="margin: 0 0;">
|
| 45 |
</a>
|
| 46 |
|
| 47 |
</div>
|
|
|
|
| 75 |
# News
|
| 76 |
|
| 77 |
- 2023-12-05: SUS-Chat is ranked 2nd in [Open LLM
|
| 78 |
+
leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
| 79 |
+
and surpassed all models under 70B.
|
| 80 |
|
| 81 |
- 2023-12-01: SUS-Chat-34B is now avaliable on HuggingFace🤗.
|
| 82 |
|
| 83 |
# Inrtoduction
|
| 84 |
|
| 85 |
<img src="https://hackmd.io/_uploads/HJlDtzhBa.png" id="fig-sus"
|
| 86 |
+
alt="Figure 1: DALL·E 2023-12-01 11.03.28 - An imposing, majestic wild boar combined with elements of a futuristic transformer robot. The boar itself should be intricately blended with these tra" />
|
| 87 |
|
| 88 |
**SUS-Chat** is a 34B bilingual Chinese-English dialogue model, jointly
|
| 89 |
released by the **Southern University of Science and Technology** and
|
|
|
|
| 94 |
instructions through high-quality instruction fine-tuning and excels at
|
| 95 |
imitating human thought processes through chains of thought. It
|
| 96 |
introduces inter-instruction attention sharing in long texts, expanding
|
| 97 |
+
the window size from 4K to 8K, significantly enhancing the usability of
|
| 98 |
+
multi-round dialogues.
|
| 99 |
|
| 100 |
It has surpassed all models of the same size in almost all benchmark
|
| 101 |
tests and is better suited to meet the practical needs of complex
|
|
|
|
| 140 |
| OrionStar-34B | 68.51 | 66.88 | 65.13 | 54.36 | 62.88 | 12.8 | 77.27 | 80.19 | 84.54 | 53.24 | 62.58 |
|
| 141 |
| Yi-34B-Chat | 66.96 | 55.16 | 77.16 | 63.76 | 61.54 | 10.02 | 76.64 | 70.66 | 82.29 | 54.57 | 61.876 |
|
| 142 |
|
| 143 |
+
<img
|
| 144 |
+
src="https://github.com/SUSTech-IDEA/SUS-Chat/raw/main/assets/radar.png"
|
| 145 |
+
id="fig-bench" alt="Figure 2: Benchmark" />
|
| 146 |
|
| 147 |
# Usage
|
| 148 |
|
|
|
|
| 158 |
history = ""
|
| 159 |
for message in messages:
|
| 160 |
match message:
|
| 161 |
+
case {"role": "user", "content": message}:
|
| 162 |
history += f"### Human: {message}\n\n### Assistant: "
|
| 163 |
case {"role": "assistant", "content": message}:
|
| 164 |
history += message
|
|
|
|
| 222 |
This model is developed entirely for academic research and free
|
| 223 |
commercial use, but it must adhere to the
|
| 224 |
[license](https://github.com/SUSTech-IDEA/SUS-Chat/blob/main/MODEL_LICENSE_AGREEMENT.txt)
|
| 225 |
+
from 01-ai.
|