mistral-lp2-org_aug_a

README.md

@@ -16,10 +16,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6260
-- F1 Micro: 0.5230
-- F1 Macro: 0.5155
-- F1 Weighted: 0.5274
+- Loss: 0.8627
+- F1 Micro: 0.6194
+- F1 Macro: 0.6193
+- F1 Weighted: 0.6193
 
 ## Model description
 
@@ -39,18 +39,18 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0001
-- train_batch_size: 32
-- eval_batch_size: 32
+- train_batch_size: 16
+- eval_batch_size: 16
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- training_steps: 25
+- training_steps: 2001
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss | F1 Micro | F1 Macro | F1 Weighted |
 |:-------------:|:------:|:----:|:---------------:|:--------:|:--------:|:-----------:|
-| 1.8454        | 0.0154 | 25   | 1.6260          | 0.5230   | 0.5155   | 0.5274      |
+| 0.9832        | 0.2548 | 2000 | 0.8627          | 0.6194   | 0.6193   | 0.6193      |
 
 
 ### Framework versions

adapter_config.json

@@ -21,9 +21,9 @@
   "revision": null,
   "target_modules": [
     "q_proj",
+    "v_proj",
     "k_proj",
-    "o_proj",
-    "v_proj"
+    "o_proj"
   ],
   "task_type": "SEQ_CLS",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c436c187cbabace0ab368aea37ad0c4711742d5d7324ad76d9c8544cf59472b4
+oid sha256:2a7ce785314115beac1aae1ab678454b2674d1c5038a9794150c26376427ccf4
 size 578881968

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3745867cf9811ba4f8713c7ef3ed9214e2a6decf9a602ecbc1d1be5e8770d17c
+oid sha256:69ac6d1e5d379cf776d79ba2cf05b41561bc972eda7cb863d7c1b269b7d18c06
 size 4920