End of training

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [stabilityai/stablelm-zephyr-3b](https://huggingface.co/stabilityai/stablelm-zephyr-3b) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 6.4023
+- Loss: 3.4473
 
 ## Model description
 
@@ -50,12 +50,31 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| No log        | 0.56  | 1    | 2.9512          |
+| No log        | 1.67  | 3    | 2.9727          |
+| No log        | 2.78  | 5    | 3.2578          |
+| No log        | 3.89  | 7    | 2.9238          |
+| No log        | 5.0   | 9    | 3.3867          |
+| 0.916         | 5.56  | 10   | 3.3066          |
+| 0.916         | 6.67  | 12   | 3.2090          |
+| 0.916         | 7.78  | 14   | 3.4980          |
+| 0.916         | 8.89  | 16   | 3.5098          |
+| 0.916         | 10.0  | 18   | 3.4434          |
+| 0.916         | 10.56 | 19   | 3.4375          |
+| 0.1354        | 11.67 | 21   | 3.4238          |
+| 0.1354        | 12.78 | 23   | 3.4336          |
+| 0.1354        | 13.89 | 25   | 3.4473          |
+| 0.1354        | 15.0  | 27   | 3.4492          |
+| 0.1354        | 15.56 | 28   | 3.4492          |
+| 0.0754        | 16.67 | 30   | 3.4473          |
 
 
 ### Framework versions
 
 - PEFT 0.9.0
 - Transformers 4.38.2
-- Pytorch 2.1.2+cu121
+- Pytorch 2.1.0+cu121
 - Datasets 2.18.0
 - Tokenizers 0.15.2

adapter_config.json

@@ -19,13 +19,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "down_proj",
     "k_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
     "gate_proj",
-    "v_proj"
+    "v_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
-size 48
+oid sha256:ff067eb0d24d6ac677cadfb8c1d21e9e59f8757a225366830ffdefb6711bca47
+size 801172960

runs/Mar13_10-17-13_6e2c9b113eb3/events.out.tfevents.1710325034.6e2c9b113eb3.153.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2a2107835eead15477ebad78e5ebe4ce72f8d9112d4f3d1d935df4d76024d80
+size 5104

runs/Mar13_10-17-41_6e2c9b113eb3/events.out.tfevents.1710325062.6e2c9b113eb3.153.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83bfcbc5f4ed8aec3c46ed95e400ea108c0df213f7233f7833a54fe7316cc9c8
+size 13935

runs/Mar13_10-17-41_6e2c9b113eb3/events.out.tfevents.1710325549.6e2c9b113eb3.153.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20ef89c3a4db7f2a03ca1717df043d5c8aa5f16c24f4727e7b1c8877728826e0
+size 354

runs/Mar13_10-26-32_6e2c9b113eb3/events.out.tfevents.1710325593.6e2c9b113eb3.153.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5957b224a966b23f239b9269b97cbeaea98cfe2aaf4ee6677bf7008910dfec0
+size 10595

runs/Mar13_10-26-32_6e2c9b113eb3/events.out.tfevents.1710325885.6e2c9b113eb3.153.4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b952392a880e41e94dde187919c6ef0690a75f9e9fa48c489595215596a376a9
+size 354

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3aa663e6169809153a242be034068df0397a9a316586710ff0a0bab8c5f39dda
+oid sha256:0e1b006c398a102784e93bbad6860bfc91328a0b6e4dfe64dead070a31c3d334
 size 4920