Dev-SriramB/qa_bot

README.md

@@ -1,7 +1,7 @@
 ---
-base_model: TheBloke/Mistral-7B-Instruct-v0.2-GPTQ
 library_name: peft
 license: apache-2.0
+base_model: TheBloke/Mistral-7B-Instruct-v0.2-GPTQ
 tags:
 - generated_from_trainer
 model-index:
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7438
+- Loss: 1.4698
 
 ## Model description
 
@@ -41,7 +41,7 @@ The following hyperparameters were used during training:
 - seed: 42
 - gradient_accumulation_steps: 4
 - total_train_batch_size: 4
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 2
 - num_epochs: 2
@@ -51,14 +51,14 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 2.3683        | 1.0   | 5    | 1.9824          |
-| 1.8801        | 2.0   | 10   | 1.7438          |
+| 1.7883        | 1.0   | 75   | 1.4686          |
+| 1.3538        | 2.0   | 150  | 1.4698          |
 
 
 ### Framework versions
 
-- PEFT 0.12.0
-- Transformers 4.42.4
-- Pytorch 2.3.1+cu121
-- Datasets 2.20.0
-- Tokenizers 0.19.1
+- PEFT 0.13.2
+- Transformers 4.46.2
+- Pytorch 2.5.1+cu121
+- Datasets 3.1.0
+- Tokenizers 0.20.3

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:70dbe7f37b3e622482e56daada842f7060763bfeb5897c3bf11c0a4509d75c33
+oid sha256:7dc4beae180f98870b8f030af2f2b6dd85e388dc8e02ece83f50d6239b99f905
 size 8397056

runs/Nov17_15-42-50_7d0f570af10f/events.out.tfevents.1731858198.7d0f570af10f.385.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d647daf3898c3ecc0a0354d205ab53e2e667639dd2a893e5e08127ff159c412
+size 7036

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7f16530a26e9eeaf46cd2342eb2d3f723fda67fae1d47541856bb6063444f68b
-size 5112
+oid sha256:906ba9651c579e6c98415ad58fd192e61fb134fed30421d648ce04edc4ce8794
+size 5304