Finished finetuning grade 2

README.md

@@ -12,13 +12,12 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/uds/Graded%20text%20simplification%20training/runs/1mkbeo5j)
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/uds/Graded%20text%20simplification%20training/runs/vd7wa4iw)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/uds/Graded%20text%20simplification%20training/runs/42n7wyjl)
 # text-simplification
 
 This model is a fine-tuned version of [openai-community/gpt2](https://huggingface.co/openai-community/gpt2) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5178
+- Loss: 0.3608
 
 ## Model description
 
@@ -49,16 +48,16 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.546         | 1.0   | 775  | 0.5205          |
-| 0.5405        | 2.0   | 1550 | 0.5186          |
-| 0.5368        | 3.0   | 2325 | 0.5162          |
-| 0.5338        | 4.0   | 3100 | 0.5176          |
-| 0.5322        | 5.0   | 3875 | 0.5189          |
-| 0.5304        | 6.0   | 4650 | 0.5179          |
-| 0.5284        | 7.0   | 5425 | 0.5184          |
-| 0.5278        | 8.0   | 6200 | 0.5176          |
-| 0.5276        | 9.0   | 6975 | 0.5177          |
-| 0.527         | 10.0  | 7750 | 0.5178          |
+| 0.4017        | 1.0   | 469  | 0.3665          |
+| 0.3951        | 2.0   | 938  | 0.3640          |
+| 0.3907        | 3.0   | 1407 | 0.3641          |
+| 0.3892        | 4.0   | 1876 | 0.3623          |
+| 0.3864        | 5.0   | 2345 | 0.3606          |
+| 0.3847        | 6.0   | 2814 | 0.3614          |
+| 0.383         | 7.0   | 3283 | 0.3606          |
+| 0.3821        | 8.0   | 3752 | 0.3609          |
+| 0.3815        | 9.0   | 4221 | 0.3610          |
+| 0.3806        | 10.0  | 4690 | 0.3608          |
 
 
 ### Framework versions

gpt2-grade-2-finetuned/adapter_config.json

@@ -3,6 +3,8 @@
   "auto_mapping": null,
   "base_model_name_or_path": "openai-community/gpt2",
   "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
   "fan_in_fan_out": false,
   "inference_mode": true,
   "init_lora_weights": true,
@@ -11,6 +13,7 @@
   "layers_to_transform": null,
   "loftq_config": {},
   "lora_alpha": 32,
+  "lora_bias": false,
   "lora_dropout": 0.01,
   "megatron_config": null,
   "megatron_core": "megatron.core",
@@ -20,10 +23,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "c_proj",
-    "c_attn",
     "c_fc",
-    "lm_head"
+    "c_attn",
+    "lm_head",
+    "c_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

gpt2-grade-2-finetuned/adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0192c65a91bf678670e23d3d4f7dded3020605d04f62c90431a9971fb531a4de
-size 83581271
+oid sha256:4befae71d3ecf37f12917a3a0825b40c7f14e9517932cb7691543736d133d327
+size 160776023

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:73d8229af7e9a6c1c4932cebc80c86e66cb19d5f222f394f37cbd4a55f5c82de
-size 5368
+oid sha256:b5faedaca63599a9ee5c38b36992fc6e39d2ddac460d824c93451e91a85592a3
+size 5496