ca-finetuned-phi-2

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.9920
+- Loss: 1.7293
 - Perplexity: 0.0000
 
 ## Model description
@@ -53,14 +53,16 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Perplexity |
 |:-------------:|:-----:|:----:|:---------------:|:----------:|
-| No log        | 1.0   | 1    | 2.3882          | 0.0000     |
-| No log        | 2.0   | 2    | 2.2933          | 0.0000     |
-| No log        | 3.0   | 4    | 2.1606          | 0.0000     |
-| No log        | 4.0   | 5    | 2.0964          | 0.0000     |
-| No log        | 5.0   | 6    | 2.0452          | 0.0000     |
-| No log        | 6.0   | 8    | 2.0022          | 0.0000     |
-| No log        | 7.0   | 9    | 1.9943          | 0.0000     |
-| No log        | 8.0   | 10   | 1.9920          | 0.0000     |
+| No log        | 0.97  | 7    | 2.4415          | 0.0000     |
+| No log        | 1.95  | 14   | 2.1788          | 0.0000     |
+| 2.5859        | 2.92  | 21   | 2.0108          | 0.0000     |
+| 2.5859        | 3.9   | 28   | 1.9063          | 0.0000     |
+| 2.0806        | 4.87  | 35   | 1.8347          | 0.0000     |
+| 2.0806        | 5.98  | 43   | 1.7810          | 0.0000     |
+| 1.84          | 6.96  | 50   | 1.7497          | 0.0000     |
+| 1.84          | 7.93  | 57   | 1.7358          | 0.0000     |
+| 1.7413        | 8.9   | 64   | 1.7306          | 0.0000     |
+| 1.7413        | 9.74  | 70   | 1.7293          | 0.0000     |
 
 
 ### Framework versions

adapter_config.json

@@ -19,8 +19,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "fc1",
     "fc2",
+    "fc1",
     "Wqkv"
   ],
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1432ae232ff114e9236553c8e1b7fb40e331a47b287af43df046e4c7a42bbcbb
+oid sha256:7da1c87977f29bb80d79cab7c2ff73c7efa3d4fa997857e93588de8f87d16732
 size 146825352

tokenizer.json

@@ -2,7 +2,7 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 512,
+    "max_length": 364,
     "strategy": "LongestFirst",
     "stride": 0
   },

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:56de8dff970b4a55858bf24335ebf386cad56ec256f65a50c85e55523233977c
+oid sha256:3ff98dac56699d34eb15c09aa1525894f3514760317ece52cb2ce3e2477dc9a0
 size 4347