Hugging Face's logo

Weni
/
ZeroShot-3.3.6-Mistral-7b-Multilanguage-3.2.0

PEFT
Safetensors
mistral
trl
sft
Generated from Trainer
Model card Files
xet
 Community
ironrock commited on
Commit
42144d8
·
verified ·
1 Parent(s): 5860ec6

Model save

Browse files
Files changed (1) hide show
  1. README.md +96 -0
README.md ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: peft
4
+ tags:
5
+ - trl
6
+ - sft
7
+ - generated_from_trainer
8
+ base_model: mistralai/Mistral-7B-v0.1
9
+ model-index:
10
+ - name: ZeroShot-3.3.6-Mistral-7b-Multilanguage-3.2.0
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # ZeroShot-3.3.6-Mistral-7b-Multilanguage-3.2.0
18
+
19
+ This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.2603
22
+
23
+ ## Model description
24
+
25
+ More information needed
26
+
27
+ ## Intended uses & limitations
28
+
29
+ More information needed
30
+
31
+ ## Training and evaluation data
32
+
33
+ More information needed
34
+
35
+ ## Training procedure
36
+
37
+ ### Training hyperparameters
38
+
39
+ The following hyperparameters were used during training:
40
+ - learning_rate: 0.0002
41
+ - train_batch_size: 8
42
+ - eval_batch_size: 2
43
+ - seed: 42
44
+ - gradient_accumulation_steps: 2
45
+ - total_train_batch_size: 16
46
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
47
+ - lr_scheduler_type: cosine
48
+ - lr_scheduler_warmup_ratio: 0.1
49
+ - num_epochs: 1
50
+ - mixed_precision_training: Native AMP
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss |
55
+ |:-------------:|:-----:|:----:|:---------------:|
56
+ | 0.8669 | 0.03 | 50 | 0.4141 |
57
+ | 0.4025 | 0.06 | 100 | 0.4020 |
58
+ | 0.4016 | 0.09 | 150 | 0.4082 |
59
+ | 0.3981 | 0.12 | 200 | 0.4152 |
60
+ | 0.4065 | 0.16 | 250 | 0.3998 |
61
+ | 0.4003 | 0.19 | 300 | 0.3989 |
62
+ | 0.3964 | 0.22 | 350 | 0.3991 |
63
+ | 0.3825 | 0.25 | 400 | 0.3861 |
64
+ | 0.379 | 0.28 | 450 | 0.3804 |
65
+ | 0.3696 | 0.31 | 500 | 0.3756 |
66
+ | 0.3658 | 0.34 | 550 | 0.3662 |
67
+ | 0.3474 | 0.37 | 600 | 0.3615 |
68
+ | 0.3575 | 0.4 | 650 | 0.3527 |
69
+ | 0.346 | 0.43 | 700 | 0.3470 |
70
+ | 0.3486 | 0.47 | 750 | 0.3394 |
71
+ | 0.3326 | 0.5 | 800 | 0.3317 |
72
+ | 0.3253 | 0.53 | 850 | 0.3228 |
73
+ | 0.3151 | 0.56 | 900 | 0.3156 |
74
+ | 0.3031 | 0.59 | 950 | 0.3100 |
75
+ | 0.3106 | 0.62 | 1000 | 0.3028 |
76
+ | 0.2994 | 0.65 | 1050 | 0.2963 |
77
+ | 0.2974 | 0.68 | 1100 | 0.2901 |
78
+ | 0.2742 | 0.71 | 1150 | 0.2847 |
79
+ | 0.2873 | 0.74 | 1200 | 0.2789 |
80
+ | 0.2694 | 0.78 | 1250 | 0.2747 |
81
+ | 0.2738 | 0.81 | 1300 | 0.2699 |
82
+ | 0.2719 | 0.84 | 1350 | 0.2662 |
83
+ | 0.2525 | 0.87 | 1400 | 0.2637 |
84
+ | 0.2538 | 0.9 | 1450 | 0.2620 |
85
+ | 0.2576 | 0.93 | 1500 | 0.2610 |
86
+ | 0.258 | 0.96 | 1550 | 0.2605 |
87
+ | 0.2524 | 0.99 | 1600 | 0.2603 |
88
+
89
+
90
+ ### Framework versions
91
+
92
+ - PEFT 0.8.2
93
+ - Transformers 4.38.1
94
+ - Pytorch 2.1.0+cu121
95
+ - Datasets 2.17.1
96
+ - Tokenizers 0.15.2