Upload README.md
Browse files
README.md
CHANGED
|
@@ -153,12 +153,21 @@ This format must be adhered to strictly, as deviations may result in less optima
|
|
| 153 |
The template used to construct a prompt for the Instruct model is specified as follows:
|
| 154 |
|
| 155 |
```
|
| 156 |
-
<s>[INST] <<SYS>>\n{
|
| 157 |
```
|
| 158 |
|
|
|
|
| 159 |
Please be aware that ``<s>`` and ``</s>`` are special tokens used for the beginning of string (BOS) and end of string (EOS), respectively, while [INST] and [/INST] are considered regular strings.
|
| 160 |
|
| 161 |
-
For the "{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
|
| 163 |
|
| 164 |
### Use the instruct model Ver0.1
|
|
@@ -228,3 +237,15 @@ Here are the team members:
|
|
| 228 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
| 229 |
- [Takumi Okamoto](https://www.linkedin.com/in/takumi-okamoto)
|
| 230 |
- [Ishida Shigeki](https://www.wantedly.com/id/reborn27)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
The template used to construct a prompt for the Instruct model is specified as follows:
|
| 154 |
|
| 155 |
```
|
| 156 |
+
<s>[INST] <<SYS>>\n{SYSTEM_PROMPT}\n<</SYS>>\n\n{USER_MESSAGE_1} [/INST] {BOT_MESSAGE_1} </s>[INST] {USER_MESSAGE_2}[/INST]
|
| 157 |
```
|
| 158 |
|
| 159 |
+
|
| 160 |
Please be aware that ``<s>`` and ``</s>`` are special tokens used for the beginning of string (BOS) and end of string (EOS), respectively, while [INST] and [/INST] are considered regular strings.
|
| 161 |
|
| 162 |
+
For the "{SYSTEM_PROMPT}" part, We recommend using "あなたは誠実で優秀な日本人のアシスタントです。"
|
| 163 |
+
|
| 164 |
+
For the "{USER_MESSAGE_1}" part, We recommend using {instruction}\n{input}
|
| 165 |
+
|
| 166 |
+
In other words, We recommend the following:
|
| 167 |
+
|
| 168 |
+
```
|
| 169 |
+
<s>[INST] <<SYS>>\nあなたは誠実で優秀な日本人のアシスタントです。\n<</SYS>>\n\n{instruction1}\n{input1} [/INST] {BOT_MESSAGE_1}</s>[INST] \n\n{instruction2}\n{input2} [/INST]
|
| 170 |
+
```
|
| 171 |
|
| 172 |
|
| 173 |
### Use the instruct model Ver0.1
|
|
|
|
| 237 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
| 238 |
- [Takumi Okamoto](https://www.linkedin.com/in/takumi-okamoto)
|
| 239 |
- [Ishida Shigeki](https://www.wantedly.com/id/reborn27)
|
| 240 |
+
|
| 241 |
+
## How to cite
|
| 242 |
+
```
|
| 243 |
+
@misc{fujii2024continual,
|
| 244 |
+
title={Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities},
|
| 245 |
+
author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae Mizuki and Rio Yokota and Naoaki Okazaki},
|
| 246 |
+
year={2024},
|
| 247 |
+
eprint={2404.17790},
|
| 248 |
+
archivePrefix={arXiv},
|
| 249 |
+
primaryClass={cs.CL}
|
| 250 |
+
}
|
| 251 |
+
```
|