training solidity generator11/20/2023, 11:40:49

README.md

@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [ckandemir/solidity-generator](https://huggingface.co/ckandemir/solidity-generator) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.9415
+- Loss: 0.2695
 
 ## Model description
 
@@ -34,33 +34,31 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 7e-06
-- train_batch_size: 40
-- eval_batch_size: 40
+- learning_rate: 7e-05
+- train_batch_size: 26
+- eval_batch_size: 26
 - seed: 100
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 104
+- total_eval_batch_size: 104
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 10
+- num_epochs: 4
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| No log        | 1.0   | 3    | 3.6609          |
-| No log        | 2.0   | 6    | 3.4262          |
-| No log        | 3.0   | 9    | 3.2868          |
-| No log        | 4.0   | 12   | 3.1867          |
-| No log        | 5.0   | 15   | 3.1092          |
-| No log        | 6.0   | 18   | 3.0489          |
-| No log        | 7.0   | 21   | 3.0032          |
-| No log        | 8.0   | 24   | 2.9704          |
-| No log        | 9.0   | 27   | 2.9497          |
-| No log        | 10.0  | 30   | 2.9415          |
+| Training Loss | Epoch | Step   | Validation Loss |
+|:-------------:|:-----:|:------:|:---------------:|
+| 0.3698        | 1.0   | 33010  | 0.3371          |
+| 0.3292        | 2.0   | 66020  | 0.2977          |
+| 0.3067        | 3.0   | 99030  | 0.2782          |
+| 0.2956        | 4.0   | 132040 | 0.2695          |
 
 
 ### Framework versions
 
 - Transformers 4.33.0
 - Pytorch 2.1.0+cu121
-- Datasets 2.15.0
+- Datasets 2.14.7
 - Tokenizers 0.13.3

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "./checkpoint-110496",
+  "_name_or_path": "ckandemir/solidity-generator",
   "activation_function": "gelu_new",
   "architectures": [
     "GPT2LMHeadModel"

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a7fcf3fe360c3cbcfb21f7f14c59c2a324c84b07cfe08ecc00049e2bc03f774a
-size 444079386
+oid sha256:f9b16985325a8687877293b11c4d71beb3dbf20b12394b43cb33410c53a7a7fd
+size 444081498

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:50e1d5b19679205905927268d187ab3df5a4465e6a5fed21f92fbf1e0e45cd3b
+oid sha256:f2f32f5715cb41600bd425db6d7b478e9e99870e399b3ced0a33b5e4f98e8f74
 size 4472