ninagroot/Llama-360Mtest

README.md

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 6.4242
+- Loss: 5.4878
 
 ## Model description
 
@@ -33,13 +33,14 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0003
-- train_batch_size: 32
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 128
+- total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 50
 - num_epochs: 20
 - mixed_precision_training: Native AMP
 
@@ -47,18 +48,26 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 0.57  | 1    | 8.5126          |
-| No log        | 1.71  | 3    | 8.2629          |
-| No log        | 2.86  | 5    | 7.6468          |
-| No log        | 4.0   | 7    | 7.0674          |
-| No log        | 4.57  | 8    | 6.9461          |
-| No log        | 5.71  | 10   | 6.7528          |
-| No log        | 6.86  | 12   | 6.5911          |
-| No log        | 8.0   | 14   | 6.5001          |
-| No log        | 8.57  | 15   | 6.4729          |
-| No log        | 9.71  | 17   | 6.4385          |
-| No log        | 10.86 | 19   | 6.4254          |
-| 6.4414        | 11.43 | 20   | 6.4242          |
+| No log        | 1.0   | 7    | 8.6294          |
+| No log        | 2.0   | 14   | 7.4266          |
+| 7.6351        | 3.0   | 21   | 6.6887          |
+| 7.6351        | 4.0   | 28   | 6.2073          |
+| 7.6351        | 5.0   | 35   | 5.6607          |
+| 4.8066        | 6.0   | 42   | 5.3291          |
+| 4.8066        | 7.0   | 49   | 5.1600          |
+| 4.8066        | 8.0   | 56   | 5.1858          |
+| 3.1317        | 9.0   | 63   | 5.1762          |
+| 3.1317        | 10.0  | 70   | 5.2201          |
+| 3.1317        | 11.0  | 77   | 5.2886          |
+| 1.8969        | 12.0  | 84   | 5.2965          |
+| 1.8969        | 13.0  | 91   | 5.3368          |
+| 1.8969        | 14.0  | 98   | 5.3904          |
+| 0.8088        | 15.0  | 105  | 5.3941          |
+| 0.8088        | 16.0  | 112  | 5.4356          |
+| 0.8088        | 17.0  | 119  | 5.4636          |
+| 0.2919        | 18.0  | 126  | 5.4765          |
+| 0.2919        | 19.0  | 133  | 5.4868          |
+| 0.1383        | 20.0  | 140  | 5.4878          |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4e14fb55de9560eda325034afcdd7fae057ae106ac32853d5897adabb745e7dd
+oid sha256:e7fb7700f319bcff174e69f505b43d1841caf03b1ec4149cd0542baf1bfad830
 size 1408774432

runs/Apr16_13-36-00_gcn62.local.snellius.surf.nl/events.out.tfevents.1713267369.gcn62.local.snellius.surf.nl.2830994.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf319605f9297de7355870fb5303658b1147c4f7ff072113f76b15348f70b554
+size 11699

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:491428fabd67cabc14c94ed8858607844951fa3c6a8e25d22ae8ed1426778506
+oid sha256:ab223116eee66b276404500050526bc6c4f2abc8dc04f0188816a7c40db9dfad
 size 4984