training latge t5 comment 2 code done 12/10/2023, 10:32:14

README.md

@@ -0,0 +1,80 @@
+---
+license: apache-2.0
+base_model: Salesforce/codet5-base
+tags:
+- generated_from_trainer
+model-index:
+- name: SolCoder
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# SolCoder
+
+This model is a fine-tuned version of [Salesforce/codet5-base](https://huggingface.co/Salesforce/codet5-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6043
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 37
+- eval_batch_size: 37
+- seed: 100
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 148
+- total_eval_batch_size: 148
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 20
+
+### Training results
+
+| Training Loss | Epoch | Step   | Validation Loss |
+|:-------------:|:-----:|:------:|:---------------:|
+| 1.0314        | 1.0   | 7440   | 0.9257          |
+| 0.919         | 2.0   | 14880  | 0.8329          |
+| 0.8463        | 3.0   | 22320  | 0.7831          |
+| 0.8025        | 4.0   | 29760  | 0.7471          |
+| 0.764         | 5.0   | 37200  | 0.7218          |
+| 0.738         | 6.0   | 44640  | 0.6986          |
+| 0.7101        | 7.0   | 52080  | 0.6823          |
+| 0.6915        | 8.0   | 59520  | 0.6681          |
+| 0.6738        | 9.0   | 66960  | 0.6560          |
+| 0.6543        | 10.0  | 74400  | 0.6480          |
+| 0.6438        | 11.0  | 81840  | 0.6380          |
+| 0.6285        | 12.0  | 89280  | 0.6312          |
+| 0.6163        | 13.0  | 96720  | 0.6250          |
+| 0.6093        | 14.0  | 104160 | 0.6187          |
+| 0.5967        | 15.0  | 111600 | 0.6149          |
+| 0.5914        | 16.0  | 119040 | 0.6114          |
+| 0.5827        | 17.0  | 126480 | 0.6079          |
+| 0.5758        | 18.0  | 133920 | 0.6059          |
+| 0.5724        | 19.0  | 141360 | 0.6045          |
+| 0.5653        | 20.0  | 148800 | 0.6043          |
+
+
+### Framework versions
+
+- Transformers 4.33.0
+- Pytorch 2.1.0+cu121
+- Datasets 2.11.0
+- Tokenizers 0.13.3

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "./training_models/checkpoint-305868",
+  "_name_or_path": "Salesforce/codet5-base",
   "architectures": [
     "T5ForConditionalGeneration"
   ],

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dae6b99711b7c8249b100a7af2a9ba1f2a984a9b4c7e4726425d351431d85fc6
-size 891613774
+oid sha256:cc1c1ffe877771eb7458541517e1caae752e550fece1146fbfe61a5ec9a64cf4
+size 891617358

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2d73b509a28e59cbf3d5a6f990bd34c3850e4771a5fd6a62fc23cd76f6bc6b0
+size 4600