Hugging Face's logo

ninagroot
/
Llama-360M-RUN1

Text Generation
Transformers
TensorBoard
Safetensors
llama
Generated from Trainer
text-generation-inference
Model card Files
xet
Metrics Community
ninagroot commited on
Commit
3bd22bc
·
verified ·
1 Parent(s): de34bae

ninagroot/Llama-360Mtest

Browse files
Files changed (4) hide show
  1. README.md +62 -27
  2. model.safetensors +1 -1
  3. runs/Apr17_12-23-14_gcn20.local.snellius.surf.nl/events.out.tfevents.1713349405.gcn20.local.snellius.surf.nl.1176517.0 +3 -0
  4. training_args.bin +1 -1
README.md CHANGED
@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
- - Loss: 5.3649
17
 
18
  ## Model description
19
 
@@ -33,44 +33,79 @@ More information needed
33
 
34
  The following hyperparameters were used during training:
35
  - learning_rate: 0.0003
36
- - train_batch_size: 32
37
  - eval_batch_size: 8
38
  - seed: 42
39
- - gradient_accumulation_steps: 4
40
  - total_train_batch_size: 128
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: cosine
43
  - lr_scheduler_warmup_steps: 50
44
- - num_epochs: 40
45
  - mixed_precision_training: Native AMP
46
 
47
  ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
- | 9.5781 | 0.57 | 1 | 9.5646 |
52
- | 9.4151 | 1.71 | 3 | 9.3660 |
53
- | 8.6797 | 2.86 | 5 | 8.9242 |
54
- | 7.7829 | 4.0 | 7 | 8.3276 |
55
- | 7.4363 | 4.57 | 8 | 8.0833 |
56
- | 6.9645 | 5.71 | 10 | 7.6931 |
57
- | 6.6265 | 6.86 | 12 | 7.4268 |
58
- | 6.2878 | 8.0 | 14 | 7.1965 |
59
- | 6.1084 | 8.57 | 15 | 7.0904 |
60
- | 5.7872 | 9.71 | 17 | 6.8732 |
61
- | 5.43 | 10.86 | 19 | 6.6048 |
62
- | 5.0767 | 12.0 | 21 | 6.4381 |
63
- | 4.9363 | 12.57 | 22 | 6.3377 |
64
- | 4.5648 | 13.71 | 24 | 6.1173 |
65
- | 4.2417 | 14.86 | 26 | 5.9199 |
66
- | 3.926 | 16.0 | 28 | 5.7397 |
67
- | 3.7467 | 16.57 | 29 | 5.6383 |
68
- | 3.4837 | 17.71 | 31 | 5.5448 |
69
- | 3.2985 | 18.86 | 33 | 5.4371 |
70
- | 2.9882 | 20.0 | 35 | 5.4290 |
71
- | 2.8515 | 20.57 | 36 | 5.4213 |
72
- | 2.5933 | 21.71 | 38 | 5.3916 |
73
- | 2.3065 | 22.86 | 40 | 5.3649 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
 
75
 
76
  ### Framework versions
 
13
 
14
  This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
+ - Loss: 5.6445
17
 
18
  ## Model description
19
 
 
33
 
34
  The following hyperparameters were used during training:
35
  - learning_rate: 0.0003
36
+ - train_batch_size: 16
37
  - eval_batch_size: 8
38
  - seed: 42
39
+ - gradient_accumulation_steps: 8
40
  - total_train_batch_size: 128
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: cosine
43
  - lr_scheduler_warmup_steps: 50
44
+ - num_epochs: 100
45
  - mixed_precision_training: Native AMP
46
 
47
  ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:----:|:---------------:|
51
+ | 9.595 | 0.57 | 1 | 9.6036 |
52
+ | 9.4191 | 1.71 | 3 | 9.4004 |
53
+ | 8.6679 | 2.86 | 5 | 8.9609 |
54
+ | 7.7889 | 4.0 | 7 | 8.3870 |
55
+ | 7.4852 | 4.57 | 8 | 8.1495 |
56
+ | 6.9951 | 5.71 | 10 | 7.7565 |
57
+ | 6.6337 | 6.86 | 12 | 7.4558 |
58
+ | 6.2744 | 8.0 | 14 | 7.1806 |
59
+ | 6.052 | 8.57 | 15 | 7.0535 |
60
+ | 5.69 | 9.71 | 17 | 6.8455 |
61
+ | 5.4046 | 10.86 | 19 | 6.6445 |
62
+ | 5.1682 | 12.0 | 21 | 6.5058 |
63
+ | 5.0522 | 12.57 | 22 | 6.3872 |
64
+ | 4.6834 | 13.71 | 24 | 6.2011 |
65
+ | 4.2821 | 14.86 | 26 | 5.9424 |
66
+ | 3.9781 | 16.0 | 28 | 5.7461 |
67
+ | 3.7742 | 16.57 | 29 | 5.6778 |
68
+ | 3.525 | 17.71 | 31 | 5.5370 |
69
+ | 3.3434 | 18.86 | 33 | 5.4445 |
70
+ | 3.0161 | 20.0 | 35 | 5.3650 |
71
+ | 2.848 | 20.57 | 36 | 5.4065 |
72
+ | 2.5819 | 21.71 | 38 | 5.3697 |
73
+ | 2.2761 | 22.86 | 40 | 5.3867 |
74
+ | 2.0201 | 24.0 | 42 | 5.3975 |
75
+ | 1.8269 | 24.57 | 43 | 5.4014 |
76
+ | 1.5501 | 25.71 | 45 | 5.3687 |
77
+ | 1.3036 | 26.86 | 47 | 5.4212 |
78
+ | 1.0129 | 28.0 | 49 | 5.4374 |
79
+ | 0.8856 | 28.57 | 50 | 5.4521 |
80
+ | 0.6592 | 29.71 | 52 | 5.4968 |
81
+ | 0.5508 | 30.86 | 54 | 5.4760 |
82
+ | 0.4567 | 32.0 | 56 | 5.4806 |
83
+ | 0.4057 | 32.57 | 57 | 5.5026 |
84
+ | 0.3211 | 33.71 | 59 | 5.5319 |
85
+ | 0.289 | 34.86 | 61 | 5.5295 |
86
+ | 0.2501 | 36.0 | 63 | 5.5913 |
87
+ | 0.2088 | 36.57 | 64 | 5.5563 |
88
+ | 0.1661 | 37.71 | 66 | 5.5826 |
89
+ | 0.1405 | 38.86 | 68 | 5.5947 |
90
+ | 0.1031 | 40.0 | 70 | 5.6525 |
91
+ | 0.0882 | 40.57 | 71 | 5.6373 |
92
+ | 0.0609 | 41.71 | 73 | 5.6135 |
93
+ | 0.0544 | 42.86 | 75 | 5.6294 |
94
+ | 0.0415 | 44.0 | 77 | 5.6271 |
95
+ | 0.0358 | 44.57 | 78 | 5.6269 |
96
+ | 0.0284 | 45.71 | 80 | 5.6244 |
97
+ | 0.0241 | 46.86 | 82 | 5.6265 |
98
+ | 0.0207 | 48.0 | 84 | 5.6290 |
99
+ | 0.0201 | 48.57 | 85 | 5.6310 |
100
+ | 0.0194 | 49.71 | 87 | 5.6346 |
101
+ | 0.0182 | 50.86 | 89 | 5.6376 |
102
+ | 0.0166 | 52.0 | 91 | 5.6402 |
103
+ | 0.0159 | 52.57 | 92 | 5.6413 |
104
+ | 0.0156 | 53.71 | 94 | 5.6430 |
105
+ | 0.0151 | 54.86 | 96 | 5.6440 |
106
+ | 0.0151 | 56.0 | 98 | 5.6444 |
107
+ | 0.0151 | 56.57 | 99 | 5.6445 |
108
+ | 0.0144 | 57.14 | 100 | 5.6445 |
109
 
110
 
111
  ### Framework versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b5565075cfacf013834e10d49bc325bd1a80185b85099c949d8dad85d1e04cb1
3
  size 1408774432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8cf188330e20a958fc19fa166f852613d26d71a3348dc3f34de0a2a3960e8ce6
3
  size 1408774432
runs/Apr17_12-23-14_gcn20.local.snellius.surf.nl/events.out.tfevents.1713349405.gcn20.local.snellius.surf.nl.1176517.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97dbf587554a639e0af1a979d99e8a71ed252ffdb5f253b8f43c8c36677fa3c3
3
+ size 41039
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:29cdcf928fd468e90e2f2a6126f8cfcf3d4abe128edd25f38df98c1fe01aa3b6
3
  size 4984
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dc0fe5f1565284ba751ed376d47d7be6f894736965a81620a6e9253651d4135
3
  size 4984