End of training

README.md

@@ -2,24 +2,19 @@
 base_model: bofenghuang/vigogne-2-13b-instruct
 tags:
 - generated_from_trainer
-- lora
 model-index:
-- name: PointCon-vigogne-2-13b-instruct-QLoRA-Instruct
+- name: PointCon-Vigogne-13B-LoRA
   results: []
-datasets:
-- IUseAMouse/POINTCON-QA-Light
-language:
-- fr
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# PointCon-vigogne-2-13b-instruct-QLoRA-Instruct
+# PointCon-Vigogne-13B-LoRA
 
-This model is a fine-tuned version of [bofenghuang/vigogne-2-13b-instruct](https://huggingface.co/bofenghuang/vigogne-2-13b-instruct) on the .CON corpus of books.
+This model is a fine-tuned version of [bofenghuang/vigogne-2-13b-instruct](https://huggingface.co/bofenghuang/vigogne-2-13b-instruct) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.8921
+- Loss: 1.8656
 
 ## Model description
 
@@ -38,24 +33,34 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0002
+- learning_rate: 5e-05
 - train_batch_size: 1
 - eval_batch_size: 1
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - num_epochs: 1
+- mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.9628        | 1.0   | 91   | 1.8921          |
+| 2.0885        | 0.1   | 30   | 2.0357          |
+| 2.0024        | 0.19  | 60   | 1.9733          |
+| 1.9995        | 0.29  | 90   | 1.9406          |
+| 1.9752        | 0.38  | 120  | 1.9285          |
+| 1.9235        | 0.48  | 150  | 1.9060          |
+| 1.9345        | 0.57  | 180  | 1.8924          |
+| 1.8576        | 0.67  | 210  | 1.8818          |
+| 1.8693        | 0.76  | 240  | 1.8734          |
+| 1.8686        | 0.86  | 270  | 1.8695          |
+| 1.8814        | 0.95  | 300  | 1.8656          |
 
 
 ### Framework versions
 
-- Transformers 4.32.1
-- Pytorch 2.0.1+cu118
-- Datasets 2.14.4
-- Tokenizers 0.13.3
+- Transformers 4.35.0
+- Pytorch 2.1.0+cu118
+- Datasets 2.14.6
+- Tokenizers 0.14.1

adapter_config.json

@@ -1,4 +1,5 @@
 {
+  "alpha_pattern": {},
   "auto_mapping": null,
   "base_model_name_or_path": "bofenghuang/vigogne-2-13b-instruct",
   "bias": "none",
@@ -12,15 +13,18 @@
   "modules_to_save": null,
   "peft_type": "LORA",
   "r": 16,
+  "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
+    "lm_head",
+    "up_proj",
     "v_proj",
-    "k_proj",
+    "q_proj",
     "o_proj",
     "gate_proj",
-    "up_proj",
-    "down_proj"
+    "embed_tokens",
+    "down_proj",
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b44a7489fad91183427e318cafbcb9e768cbd9b684bec66426178a8cb65681c3
+size 255174752

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5c90dbcd2b199ecf3a4eb172d4c54532485f65157b7fcf5b84a47567c76fea92
-size 4091
+oid sha256:4de3abff4bda1c5672b4aa7e681b04a90b6b6cd86a05336b0d7ebd6312befbd3
+size 4600