ninagroot/GPT2-705Mtest

README.md

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 7.4496
+- Loss: 5.7957
 
 ## Model description
 
@@ -33,14 +33,14 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.00025
-- train_batch_size: 16
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - gradient_accumulation_steps: 8
-- total_train_batch_size: 128
+- total_train_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 100
+- lr_scheduler_warmup_steps: 50
 - num_epochs: 10
 - mixed_precision_training: Native AMP
 
@@ -48,12 +48,15 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 0.57  | 1    | 9.7861          |
-| No log        | 1.71  | 3    | 8.5878          |
-| No log        | 2.86  | 5    | 8.2583          |
-| No log        | 4.0   | 7    | 7.9939          |
-| No log        | 4.57  | 8    | 7.8503          |
-| No log        | 5.71  | 10   | 7.4496          |
+| No log        | 0.86  | 3    | 9.1712          |
+| No log        | 2.0   | 7    | 8.7604          |
+| No log        | 2.86  | 10   | 7.5925          |
+| No log        | 4.0   | 14   | 7.3624          |
+| No log        | 4.86  | 17   | 6.9348          |
+| 7.3776        | 6.0   | 21   | 6.4774          |
+| 7.3776        | 6.86  | 24   | 6.4065          |
+| 7.3776        | 8.0   | 28   | 5.9677          |
+| 7.3776        | 8.57  | 30   | 5.7957          |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ac4dca2e1fb9d9cdba14ce6b42938c577249cf20a62d6333dd44fbf7cadca77e
+oid sha256:1a97e4f33ec53e9ee3587a4c8d037c3739a5750718be3288f0206a3d76e6baa7
 size 2796386080

runs/Apr16_10-58-57_gcn18.local.snellius.surf.nl/events.out.tfevents.1713257946.gcn18.local.snellius.surf.nl.2054051.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:962590a9b90bce76e8e77e8f33182c398d760bada3409d8174ee9f1f6b95ad0b
+size 7627

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0f1af8c960cef3e70b4e192c4c07ce00ff29c5f09cee895c2e3708179dad2ee8
+oid sha256:9737ed2ac8b321c46a74a7040cd59b1107ef509adb81edf26aaff17959f3223c
 size 4984