mistral-lp2-org_org_a

README.md

@@ -16,10 +16,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.2466
-- F1 Micro: 0.4688
-- F1 Macro: 0.3637
-- F1 Weighted: 0.5576
+- Loss: 0.8942
+- F1 Micro: 0.6
+- F1 Macro: 0.5040
+- F1 Weighted: 0.6262
 
 ## Model description
 
@@ -44,19 +44,19 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- training_steps: 10
+- training_steps: 801
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss | F1 Micro | F1 Macro | F1 Weighted |
 |:-------------:|:------:|:----:|:---------------:|:--------:|:--------:|:-----------:|
-| 1.949         | 0.0016 | 10   | 1.2466          | 0.4688   | 0.3637   | 0.5576      |
+| 0.9474        | 0.2462 | 800  | 0.8942          | 0.6      | 0.5040   | 0.6262      |
 
 
 ### Framework versions
 
-- PEFT 0.11.1
-- Transformers 4.41.2
-- Pytorch 2.1.2
-- Datasets 2.19.2
+- PEFT 0.10.0
+- Transformers 4.40.2
+- Pytorch 2.3.0+cu118
+- Datasets 2.19.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -14,19 +14,16 @@
   "lora_dropout": 0.05,
   "megatron_config": null,
   "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "classifier",
-    "score"
-  ],
+  "modules_to_save": null,
   "peft_type": "LORA",
   "r": 16,
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
+    "o_proj",
     "q_proj",
-    "v_proj",
-    "o_proj"
+    "k_proj",
+    "v_proj"
   ],
   "task_type": "SEQ_CLS",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6c13ecad7e01d239461f44c05df1643720178d19b0fec5e2991cda349584a054
+oid sha256:07ce652557a155d475012cf7017aef328b6513da1a747354e3c649d0e5eaa356
 size 578881968

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c051214d89b5bc1865551b6773d12b7d08ec9239978e1b2b3ad0323998b83019
-size 5048
+oid sha256:197939ee0a341d8f338a04d40532ae09af8b5516cdc77fb21359b12112aad973
+size 4920