End of training
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ base_model: NousResearch/Meta-Llama-3-8B-Instruct
|
|
| 5 |
tags:
|
| 6 |
- axolotl
|
| 7 |
- generated_from_trainer
|
|
|
|
|
|
|
| 8 |
model-index:
|
| 9 |
- name: llama31-8b-sft-sitcom-lora
|
| 10 |
results: []
|
|
@@ -73,7 +75,9 @@ weight_decay: 0.0
|
|
| 73 |
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/deepakkarkala-personal/finetuning_llama31_8b_sitcom/runs/sft_trial_3)
|
| 74 |
# llama31-8b-sft-sitcom-lora
|
| 75 |
|
| 76 |
-
This model is a fine-tuned version of [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) on
|
|
|
|
|
|
|
| 77 |
|
| 78 |
## Model description
|
| 79 |
|
|
@@ -103,6 +107,17 @@ The following hyperparameters were used during training:
|
|
| 103 |
- lr_scheduler_warmup_steps: 10
|
| 104 |
- training_steps: 200
|
| 105 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
### Framework versions
|
| 107 |
|
| 108 |
- PEFT 0.15.2
|
|
|
|
| 5 |
tags:
|
| 6 |
- axolotl
|
| 7 |
- generated_from_trainer
|
| 8 |
+
datasets:
|
| 9 |
+
- deepakkarkala/sft_sitcom_chandlerbing_jsonl
|
| 10 |
model-index:
|
| 11 |
- name: llama31-8b-sft-sitcom-lora
|
| 12 |
results: []
|
|
|
|
| 75 |
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/deepakkarkala-personal/finetuning_llama31_8b_sitcom/runs/sft_trial_3)
|
| 76 |
# llama31-8b-sft-sitcom-lora
|
| 77 |
|
| 78 |
+
This model is a fine-tuned version of [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) on the deepakkarkala/sft_sitcom_chandlerbing_jsonl dataset.
|
| 79 |
+
It achieves the following results on the evaluation set:
|
| 80 |
+
- Loss: 1.8431
|
| 81 |
|
| 82 |
## Model description
|
| 83 |
|
|
|
|
| 107 |
- lr_scheduler_warmup_steps: 10
|
| 108 |
- training_steps: 200
|
| 109 |
|
| 110 |
+
### Training results
|
| 111 |
+
|
| 112 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
| 113 |
+
|:-------------:|:------:|:----:|:---------------:|
|
| 114 |
+
| 2.9323 | 0.0050 | 1 | 2.8320 |
|
| 115 |
+
| 2.0701 | 0.2506 | 50 | 1.9194 |
|
| 116 |
+
| 1.9102 | 0.5013 | 100 | 1.8692 |
|
| 117 |
+
| 1.9795 | 0.7519 | 150 | 1.8487 |
|
| 118 |
+
| 1.8136 | 1.0 | 200 | 1.8431 |
|
| 119 |
+
|
| 120 |
+
|
| 121 |
### Framework versions
|
| 122 |
|
| 123 |
- PEFT 0.15.2
|