Update README.md
Browse files
README.md
CHANGED
|
@@ -86,14 +86,19 @@ print(bot_message)
|
|
| 86 |
```
|
| 87 |
|
| 88 |
|
| 89 |
-
# Training
|
| 90 |
-
This model was trained using the ChatML format, so it should be used for inference using the ChatML chatbot format.
|
| 91 |
-
We chose this format as the base model ([Open-Orca/Mistral-7B-SlimOrca](https://huggingface.co/Open-Orca/Mistral-7B-SlimOrca)) was trained with this format, and we find the chatbot format more compelling for practical use compared to the Alpaca style instruction format.
|
| 92 |
|
|
|
|
| 93 |
* [JASTER](https://github.com/llm-jp/llm-jp-eval)
|
| 94 |
* [kunishou/oasst1-89k-ja](https://huggingface.co/datasets/kunishou/oasst1-89k-ja/)
|
| 95 |
* [kunishou/databricks-dolly-15k-ja](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja/)
|
| 96 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
We trained for 1 epoch using the following Axolotl config. (Early stopping was not performed during our training.)
|
| 98 |
<details><summary>Axolotl config .yaml</summary>
|
| 99 |
|
|
|
|
| 86 |
```
|
| 87 |
|
| 88 |
|
| 89 |
+
# Training details
|
|
|
|
|
|
|
| 90 |
|
| 91 |
+
We trained on the following 3 datasets:
|
| 92 |
* [JASTER](https://github.com/llm-jp/llm-jp-eval)
|
| 93 |
* [kunishou/oasst1-89k-ja](https://huggingface.co/datasets/kunishou/oasst1-89k-ja/)
|
| 94 |
* [kunishou/databricks-dolly-15k-ja](https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja/)
|
| 95 |
|
| 96 |
+
using the ([Open-Orca/Mistral-7B-SlimOrca](https://huggingface.co/Open-Orca/Mistral-7B-SlimOrca)) model as our base checkpoint.
|
| 97 |
+
|
| 98 |
+
This model was trained using the ChatML format, so it should be used for inference using the ChatML chatbot format.
|
| 99 |
+
We chose this format as the base model ([Open-Orca/Mistral-7B-SlimOrca](https://huggingface.co/Open-Orca/Mistral-7B-SlimOrca)) was trained with this format, and we find the chatbot format more compelling for practical use compared to the Alpaca style instruction format.
|
| 100 |
+
|
| 101 |
+
|
| 102 |
We trained for 1 epoch using the following Axolotl config. (Early stopping was not performed during our training.)
|
| 103 |
<details><summary>Axolotl config .yaml</summary>
|
| 104 |
|