alex-apoha/esm_finetuned

README.md

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model was trained from scratch on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.0238
+- Loss: 0.0143
 
 ## Model description
 
@@ -33,11 +33,11 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0004
-- train_batch_size: 4
+- learning_rate: 5e-05
+- train_batch_size: 16
 - eval_batch_size: 8
 - seed: 42
-- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1
 - num_epochs: 50
@@ -46,20 +46,24 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.0478        | 1.0   | 32   | 0.0808          |
-| 0.0265        | 2.0   | 64   | 0.0346          |
-| 0.0412        | 3.0   | 96   | 0.0195          |
-| 0.0201        | 4.0   | 128  | 0.0522          |
-| 0.0286        | 5.0   | 160  | 0.0484          |
-| 0.0175        | 6.0   | 192  | 0.0190          |
-| 0.0163        | 7.0   | 224  | 0.0356          |
-| 0.0094        | 8.0   | 256  | 0.0248          |
-| 0.0047        | 9.0   | 288  | 0.0238          |
+| No log        | 1.0   | 8    | 0.0703          |
+| 0.0992        | 2.0   | 16   | 0.0554          |
+| 0.0516        | 3.0   | 24   | 0.0550          |
+| 0.0533        | 4.0   | 32   | 0.0460          |
+| 0.0359        | 5.0   | 40   | 0.0361          |
+| 0.0359        | 6.0   | 48   | 0.0268          |
+| 0.027         | 7.0   | 56   | 0.0201          |
+| 0.0151        | 8.0   | 64   | 0.0151          |
+| 0.0138        | 9.0   | 72   | 0.0116          |
+| 0.0124        | 10.0  | 80   | 0.0094          |
+| 0.0124        | 11.0  | 88   | 0.0102          |
+| 0.0085        | 12.0  | 96   | 0.0106          |
+| 0.0062        | 13.0  | 104  | 0.0143          |
 
 
 ### Framework versions
 
-- Transformers 4.48.2
-- Pytorch 2.6.0+cu124
-- Datasets 3.2.0
+- Transformers 4.48.1
+- Pytorch 2.4.1.post300
+- Datasets 2.2.1
 - Tokenizers 0.21.0

best_model/config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "./esm_mlm_cdr_pretrained_150M",
+  "_name_or_path": "../data/esm_mlm_cdr_pretrained_150M",
   "architectures": [
     "EsmForSequenceClassification"
   ],
@@ -30,7 +30,7 @@
   "problem_type": "regression",
   "token_dropout": true,
   "torch_dtype": "float32",
-  "transformers_version": "4.48.2",
+  "transformers_version": "4.48.1",
   "use_cache": true,
   "vocab_list": null,
   "vocab_size": 33

best_model/model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:99425f643e72ecf325312e502f93376c9b4b0b0a110c693d39049f9899885705
+oid sha256:fa8a5dacfcac7c48b3acfc2aa6b6914459ea12d533992a016288cef590141320
 size 595246640

runs/Feb05_16-42-06_default/events.out.tfevents.1738773729.default.5357.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1d58581306bb737e5aba049e73b8f64c5fe3f4ed5f2ead32bcac0652bc2b3d5
+size 11139

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:16d67b0277df85854a22985e15f61d4e5f706e9026cd95869d643bb8b8030d82
-size 5304
+oid sha256:5a332706d4c590ec3fb8c53bcbb1d55e3525d17285c406d010f904a02508c99c
+size 5368