End of training

DPO/adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
+    "up_proj",
+    "o_proj",
     "v_proj",
     "q_proj",
-    "o_proj",
-    "up_proj",
-    "down_proj",
-    "gate_proj"
+    "k_proj",
+    "gate_proj",
+    "down_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

DPO/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1151a09027155c3f53f87e166ca10e73d207bdb9f5268c7b1cab861429e181bb
+oid sha256:ff6e040a505a298435ee58552b938292bf111ae714f41cc7d826e18909606676
 size 167832240

README.md

@@ -17,6 +17,16 @@ should probably proofread and complete it, then remove this comment. -->
 # mistral_dpo
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6867
+- Rewards/chosen: 0.0388
+- Rewards/rejected: 0.0248
+- Rewards/accuracies: 0.5990
+- Rewards/margins: 0.0141
+- Logps/rejected: -54.3813
+- Logps/chosen: -67.8578
+- Logits/rejected: -0.5593
+- Logits/chosen: -0.6842
 
 ## Model description
 
@@ -44,10 +54,17 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 20
-- training_steps: 10
+- training_steps: 100
 
 ### Training results
 
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.6925        | 0.0214 | 20   | 0.6915          | 0.0109         | 0.0076           | 0.5925             | 0.0034          | -54.5535       | -68.1368     | -0.5594         | -0.6846       |
+| 0.6903        | 0.0427 | 40   | 0.6893          | 0.0276         | 0.0195           | 0.5909             | 0.0081          | -54.4341       | -67.9702     | -0.5593         | -0.6844       |
+| 0.6886        | 0.0641 | 60   | 0.6877          | 0.0353         | 0.0236           | 0.5974             | 0.0117          | -54.3930       | -67.8934     | -0.5592         | -0.6842       |
+| 0.6862        | 0.0855 | 80   | 0.6869          | 0.0385         | 0.0250           | 0.6055             | 0.0136          | -54.3795       | -67.8606     | -0.5593         | -0.6842       |
+| 0.6865        | 0.1068 | 100  | 0.6867          | 0.0388         | 0.0248           | 0.5990             | 0.0141          | -54.3813       | -67.8578     | -0.5593         | -0.6842       |
 
 
 ### Framework versions

reference/adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
+    "up_proj",
+    "o_proj",
     "v_proj",
     "q_proj",
-    "o_proj",
-    "up_proj",
-    "down_proj",
-    "gate_proj"
+    "k_proj",
+    "gate_proj",
+    "down_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

runs/Jun12_04-15-39_129-146-74-240/events.out.tfevents.1718165740.129-146-74-240.216284.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5527dec82746117e10e341d7d638da3014f5f58ea726a94a0a73f327b984e44
+size 13347

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6325425ba2bebd1a6250d9cbc13a9ddb5572c82908ddf88e4128910bb08aac46
+oid sha256:a501fa5a4ae26c6409a60d5b0c60df4bcc4cbaea877a0cec2e000b74cd6d6627
 size 5688