End of training

README.md

@@ -17,6 +17,14 @@ should probably proofread and complete it, then remove this comment. -->
 # llama3_false_positives_1101_KTO_optimised_model
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5288
+- Rewards/chosen: 0.5981
+- Logps/chosen: -45.9608
+- Rewards/rejected: -0.3258
+- Logps/rejected: -56.8765
+- Rewards/margins: 0.9238
+- Kl: 0.0439
 
 ## Model description
 
@@ -48,6 +56,14 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Logps/chosen | Rewards/rejected | Logps/rejected | Rewards/margins | Kl     |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:------------:|:----------------:|:--------------:|:---------------:|:------:|
+| 0.4886        | 0.96  | 12   | 0.6709          | 0.1420         | -50.5211     | 0.0978           | -52.6405       | 0.0442          | 0.1381 |
+| 0.5124        | 2.0   | 25   | 0.5966          | 0.3060         | -48.8816     | -0.1760          | -55.3786       | 0.4819          | 0.0375 |
+| 0.3647        | 2.96  | 37   | 0.5552          | 0.4908         | -47.0331     | -0.2555          | -56.1745       | 0.7464          | 0.0447 |
+| 0.3725        | 4.0   | 50   | 0.5239          | 0.5506         | -46.4352     | -0.3964          | -57.5829       | 0.9470          | 0.0317 |
+| 0.3249        | 4.96  | 62   | 0.5300          | 0.5839         | -46.1019     | -0.3309          | -56.9280       | 0.9148          | 0.0409 |
+| 0.3262        | 5.76  | 72   | 0.5288          | 0.5981         | -45.9608     | -0.3258          | -56.8765       | 0.9238          | 0.0439 |
 
 
 ### Framework versions

adapter_config.json

@@ -20,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
     "q_proj",
+    "k_proj",
     "v_proj",
     "o_proj"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a3cf09ef35cbadf601e50b2b2ef137156dbdc7b0b8b448d805facb6275ede0bc
+oid sha256:8e4738ee6e1f67ef060f8c6f99326f9e014ad020aa305af94c963aa81ade0029
 size 27297544

metrics.jsonl

@@ -1,6 +1,6 @@
-{"epoch": 0.96, "precision": 0.7142857040816327, "recall": 0.9999999800000003, "fold": 0}
+{"epoch": 0.96, "precision": 0.7142857040816327, "recall": 0.8333333194444447, "fold": 0}
 {"epoch": 2.0, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
-{"epoch": 2.96, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
-{"epoch": 4.0, "precision": 0.9999999000000099, "recall": 0.24999999375000015, "fold": 0}
-{"epoch": 4.96, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
-{"epoch": 5.76, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
+{"epoch": 2.96, "precision": 0.9999999666666678, "recall": 0.7499999812500004, "fold": 0}
+{"epoch": 4.0, "precision": 0.9999999500000026, "recall": 0.4999999875000003, "fold": 0}
+{"epoch": 4.96, "precision": 0.9999999000000099, "recall": 0.24999999375000015, "fold": 0}
+{"epoch": 5.76, "precision": 0.9999999500000026, "recall": 0.39999999200000014, "fold": 0}

metrics_epoch_0.96_fold_0_lr_1e-05_seed_42_weight_2.0.json

@@ -1 +1 @@
-{"epoch": 0.96, "precision": 0.7142857040816327, "recall": 0.9999999800000003, "fold": 0}
+{"epoch": 0.96, "precision": 0.7142857040816327, "recall": 0.8333333194444447, "fold": 0}

metrics_epoch_2.96_fold_0_lr_1e-05_seed_42_weight_2.0.json

@@ -1 +1 @@
-{"epoch": 2.96, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
+{"epoch": 2.96, "precision": 0.9999999666666678, "recall": 0.7499999812500004, "fold": 0}

metrics_epoch_4.0_fold_0_lr_1e-05_seed_42_weight_2.0.json

@@ -1 +1 @@
-{"epoch": 4.0, "precision": 0.9999999000000099, "recall": 0.24999999375000015, "fold": 0}
+{"epoch": 4.0, "precision": 0.9999999500000026, "recall": 0.4999999875000003, "fold": 0}

metrics_epoch_4.96_fold_0_lr_1e-05_seed_42_weight_2.0.json

@@ -1 +1 @@
-{"epoch": 4.96, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
+{"epoch": 4.96, "precision": 0.9999999000000099, "recall": 0.24999999375000015, "fold": 0}

metrics_epoch_5.76_fold_0_lr_1e-05_seed_42_weight_2.0.json

@@ -1 +1 @@
-{"epoch": 5.76, "precision": 0.9999999666666678, "recall": 0.5999999880000002, "fold": 0}
+{"epoch": 5.76, "precision": 0.9999999500000026, "recall": 0.39999999200000014, "fold": 0}

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:15e787b3ed7794fc3362c63f030ed16d4d402ef371b0735fb2442a2a62ada08c
-size 5368
+oid sha256:4141f8c2c200a30b85d1dce107a6769a026fa17850ece625ace19de9dfa46f0e
+size 5304