Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ datasets:
|
|
| 6 |
language:
|
| 7 |
- zh
|
| 8 |
library_name: transformers
|
| 9 |
-
pipeline_tag:
|
| 10 |
metrics:
|
| 11 |
- perplexity
|
| 12 |
- bleu
|
|
@@ -102,7 +102,7 @@ T5模型(Text-to-Text Transfer Transformer),详情见论文: [Exploring th
|
|
| 102 |
|
| 103 |
模型源码来自huggingface,见:[T5ForConditionalGeneration](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L1557)。
|
| 104 |
|
| 105 |
-
模型配置见[model_config.json](https://huggingface.co/charent/ChatLM-Chinese
|
| 106 |
|
| 107 |
模型参数:0.2B。词表大小:29298,仅包含中文和少量英文。
|
| 108 |
|
|
@@ -145,7 +145,7 @@ CPU: Intel(R) i5-13600k @ 5.1GHz
|
|
| 145 |
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
| 146 |
import torch
|
| 147 |
|
| 148 |
-
model_id = 'charent/ChatLM-Chinese
|
| 149 |
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
| 150 |
|
| 151 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
@@ -207,12 +207,12 @@ conda install --yes --file ./requirements.txt
|
|
| 207 |
从`Hugging Face Hub`下载模型权重及配置文件,需要先安装[Git LFS](https://docs.github.com/zh/repositories/working-with-files/managing-large-files/installing-git-large-file-storage),然后运行:
|
| 208 |
|
| 209 |
```bash
|
| 210 |
-
git clone --depth 1 https://huggingface.co/charent/ChatLM-Chinese
|
| 211 |
|
| 212 |
mv ChatLM-Chinese-0.2B model_save
|
| 213 |
```
|
| 214 |
|
| 215 |
-
也可以直接从`Hugging Face Hub`仓库[ChatLM-Chinese-0.2B](https://huggingface.co/charent/ChatLM-Chinese
|
| 216 |
|
| 217 |
## 3.3 Tokenizer训练
|
| 218 |
|
|
|
|
| 6 |
language:
|
| 7 |
- zh
|
| 8 |
library_name: transformers
|
| 9 |
+
pipeline_tag: text-generation
|
| 10 |
metrics:
|
| 11 |
- perplexity
|
| 12 |
- bleu
|
|
|
|
| 102 |
|
| 103 |
模型源码来自huggingface,见:[T5ForConditionalGeneration](https://github.com/huggingface/transformers/blob/main/src/transformers/models/t5/modeling_t5.py#L1557)。
|
| 104 |
|
| 105 |
+
模型配置见[model_config.json](https://huggingface.co/charent/ChatLM-mini-Chinese/blob/main/config.json),官方的`T5-base`:`encoder layer`和`decoder layer `均为为12层,本项目这两个参数修改为10层。
|
| 106 |
|
| 107 |
模型参数:0.2B。词表大小:29298,仅包含中文和少量英文。
|
| 108 |
|
|
|
|
| 145 |
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
| 146 |
import torch
|
| 147 |
|
| 148 |
+
model_id = 'charent/ChatLM-mini-Chinese'
|
| 149 |
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
|
| 150 |
|
| 151 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
|
|
| 207 |
从`Hugging Face Hub`下载模型权重及配置文件,需要先安装[Git LFS](https://docs.github.com/zh/repositories/working-with-files/managing-large-files/installing-git-large-file-storage),然后运行:
|
| 208 |
|
| 209 |
```bash
|
| 210 |
+
git clone --depth 1 https://huggingface.co/charent/ChatLM-mini-Chinese
|
| 211 |
|
| 212 |
mv ChatLM-Chinese-0.2B model_save
|
| 213 |
```
|
| 214 |
|
| 215 |
+
也可以直接从`Hugging Face Hub`仓库[ChatLM-Chinese-0.2B](https://huggingface.co/charent/ChatLM-mini-Chinese)手工下载,将下载的文件移动到`model_save`目录下即可。
|
| 216 |
|
| 217 |
## 3.3 Tokenizer训练
|
| 218 |
|