End of training
Browse files- README.md +30 -1
- adapter_model.bin +2 -2
README.md
CHANGED
|
@@ -2,6 +2,7 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: peft
|
| 4 |
tags:
|
|
|
|
| 5 |
- generated_from_trainer
|
| 6 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
| 7 |
model-index:
|
|
@@ -117,7 +118,9 @@ weight_decay: 0
|
|
| 117 |
|
| 118 |
# mixtral-lora
|
| 119 |
|
| 120 |
-
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on
|
|
|
|
|
|
|
| 121 |
|
| 122 |
## Model description
|
| 123 |
|
|
@@ -150,6 +153,32 @@ The following hyperparameters were used during training:
|
|
| 150 |
- lr_scheduler_warmup_steps: 10
|
| 151 |
- num_epochs: 2
|
| 152 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
### Framework versions
|
| 154 |
|
| 155 |
- PEFT 0.8.2
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: peft
|
| 4 |
tags:
|
| 5 |
+
- axolotl
|
| 6 |
- generated_from_trainer
|
| 7 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
| 8 |
model-index:
|
|
|
|
| 118 |
|
| 119 |
# mixtral-lora
|
| 120 |
|
| 121 |
+
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on the None dataset.
|
| 122 |
+
It achieves the following results on the evaluation set:
|
| 123 |
+
- Loss: 0.4469
|
| 124 |
|
| 125 |
## Model description
|
| 126 |
|
|
|
|
| 153 |
- lr_scheduler_warmup_steps: 10
|
| 154 |
- num_epochs: 2
|
| 155 |
|
| 156 |
+
### Training results
|
| 157 |
+
|
| 158 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
| 159 |
+
|:-------------:|:-----:|:----:|:---------------:|
|
| 160 |
+
| 3.3559 | 0.0 | 1 | 3.6627 |
|
| 161 |
+
| 0.548 | 0.1 | 43 | 0.5124 |
|
| 162 |
+
| 0.2747 | 0.2 | 86 | 0.4845 |
|
| 163 |
+
| 0.4202 | 0.31 | 129 | 0.4760 |
|
| 164 |
+
| 0.4662 | 0.41 | 172 | 0.4690 |
|
| 165 |
+
| 0.4605 | 0.51 | 215 | 0.4640 |
|
| 166 |
+
| 0.2909 | 0.61 | 258 | 0.4620 |
|
| 167 |
+
| 0.3941 | 0.71 | 301 | 0.4600 |
|
| 168 |
+
| 0.4185 | 0.82 | 344 | 0.4573 |
|
| 169 |
+
| 0.395 | 0.92 | 387 | 0.4558 |
|
| 170 |
+
| 0.2725 | 1.0 | 430 | 0.4534 |
|
| 171 |
+
| 0.2789 | 1.1 | 473 | 0.4525 |
|
| 172 |
+
| 0.4126 | 1.21 | 516 | 0.4511 |
|
| 173 |
+
| 0.3277 | 1.31 | 559 | 0.4506 |
|
| 174 |
+
| 0.3591 | 1.41 | 602 | 0.4493 |
|
| 175 |
+
| 0.3665 | 1.51 | 645 | 0.4487 |
|
| 176 |
+
| 0.5551 | 1.62 | 688 | 0.4471 |
|
| 177 |
+
| 0.3363 | 1.72 | 731 | 0.4473 |
|
| 178 |
+
| 0.3117 | 1.82 | 774 | 0.4471 |
|
| 179 |
+
| 0.496 | 1.92 | 817 | 0.4469 |
|
| 180 |
+
|
| 181 |
+
|
| 182 |
### Framework versions
|
| 183 |
|
| 184 |
- PEFT 0.8.2
|
adapter_model.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3acbe23bf41e60f8d5a30cb06a7c41a6d155e59af7c1563c4d945cf45e90c9b6
|
| 3 |
+
size 109144714
|