ninagroot/GPT2-705Mtest

README.md

@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 6.0408
+- Loss: 6.3448
 
 ## Model description
 
@@ -32,7 +32,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0025
+- learning_rate: 2.5e-05
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
@@ -41,23 +41,25 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 300
-- num_epochs: 10
+- num_epochs: 15
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| No log        | 0.91  | 2    | 8.3080          |
-| No log        | 1.83  | 4    | 8.4099          |
-| No log        | 2.74  | 6    | 7.3996          |
-| No log        | 3.66  | 8    | 7.2565          |
-| No log        | 4.57  | 10   | 6.9514          |
-| No log        | 5.94  | 13   | 6.5453          |
-| No log        | 6.86  | 15   | 6.3409          |
-| No log        | 7.77  | 17   | 6.1577          |
-| No log        | 8.69  | 19   | 6.5267          |
-| 7.123         | 9.14  | 20   | 6.0408          |
+| No log        | 0.74  | 2    | 8.7412          |
+| No log        | 1.86  | 5    | 8.4430          |
+| No log        | 2.98  | 8    | 7.9292          |
+| No log        | 3.72  | 10   | 7.6564          |
+| No log        | 4.84  | 13   | 7.4596          |
+| No log        | 5.95  | 16   | 7.1794          |
+| No log        | 6.7   | 18   | 7.0752          |
+| 7.7805        | 7.81  | 21   | 6.8572          |
+| 7.7805        | 8.93  | 24   | 6.6855          |
+| 7.7805        | 9.67  | 26   | 6.5781          |
+| 7.7805        | 10.79 | 29   | 6.4065          |
+| 7.7805        | 11.16 | 30   | 6.3448          |
 
 
 ### Framework versions

config.json

@@ -14,7 +14,7 @@
   "n_head": 16,
   "n_inner": null,
   "n_layer": 24,
-  "n_positions": 256,
+  "n_positions": 210,
   "pad_token_id": 0,
   "reorder_and_upcast_attn": false,
   "resid_pdrop": 0.0,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bd662d2dccbe4570d5feeec8fb6f2f2af6de8a84d02c8c69e363fae122e7a31e
-size 2747934496
+oid sha256:f97b3f20c97d0b50fdeab6a7b95d0c7b113c5a7c0f25298052f05581ecc54a96
+size 2747651872

runs/Mar20_16-04-43_gcn27.local.snellius.surf.nl/events.out.tfevents.1710947092.gcn27.local.snellius.surf.nl.697666.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd5470fb8be17ccd528863d3dcd3c8340b116031e84d30b99dfbb95454581885
+size 8131

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f1f1bf6fb4d7253fbad755965768a42ded4a1786e5b4ec84633c943289b78953
+oid sha256:7a605625002f9a8b862cd5bf5287017ed7500d7286426e4a0e094ae6cdd05bad
 size 4728