Sreenath/sqlm

README.md

@@ -17,6 +17,8 @@ should probably proofread and complete it, then remove this comment. -->
 # sqlm
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3178
 
 ## Model description
 
@@ -42,10 +44,22 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- training_steps: 10
+- training_steps: 200
 
 ### Training results
 
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 0.539         | 0.0333 | 20   | 0.4674          |
+| 0.3767        | 0.0667 | 40   | 0.3872          |
+| 0.3646        | 0.1    | 60   | 0.3610          |
+| 0.3425        | 0.1333 | 80   | 0.3428          |
+| 0.3554        | 0.1667 | 100  | 0.3416          |
+| 0.3363        | 0.2    | 120  | 0.3343          |
+| 0.2963        | 0.2333 | 140  | 0.3316          |
+| 0.3263        | 0.2667 | 160  | 0.3278          |
+| 0.3226        | 0.3    | 180  | 0.3222          |
+| 0.3217        | 0.3333 | 200  | 0.3178          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj"
+    "v_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7ec986660d4833849f913000d4eda432fdc3160e8d419ef4ab43ff2c13d91891
+oid sha256:0eed634a9e5a25ccd00d839d86445ee731efafcc7c5563686dc6f6cd078aaa7e
 size 109069176

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca5cdfe75083b98aafa476a651e0c39529955a8449f3b559389fa02c3c8d679d
+oid sha256:f1d8a5922e689d59ddf7cb9289b1ab27f8384b696ffbe936bc0771d7df5655bf
 size 4984