End of training

README.md

@@ -56,10 +56,10 @@ lora_r: 16
 lora_target_linear: true
 lr_scheduler: cosine
 max_steps: 150
-micro_batch_size: 2
+micro_batch_size: 10
 mlflow_experiment_name: mhenrichsen/alpaca_2k_test
 model_type: LlamaForCausalLM
-num_epochs: 3
+num_epochs: 5
 optimizer: adamw_bnb_8bit
 output_dir: miner_id_besimray
 pad_to_sequence_len: false
@@ -91,7 +91,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [unsloth/Llama-3.2-1B-Instruct](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.1380
+- Loss: 1.1848
 
 ## Model description
 
@@ -111,31 +111,30 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 10
+- eval_batch_size: 10
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- total_train_batch_size: 40
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
-- training_steps: 30
+- training_steps: 10
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.3356        | 0.0952 | 1    | 1.2672          |
-| 1.2159        | 0.2857 | 3    | 1.2534          |
-| 1.2716        | 0.5714 | 6    | 1.2098          |
-| 1.2697        | 0.8571 | 9    | 1.1910          |
-| 1.2243        | 1.1190 | 12   | 1.1755          |
-| 1.1981        | 1.4048 | 15   | 1.1604          |
-| 1.1662        | 1.6905 | 18   | 1.1496          |
-| 1.1879        | 1.9762 | 21   | 1.1424          |
-| 1.198         | 2.2381 | 24   | 1.1413          |
-| 1.1981        | 2.5238 | 27   | 1.1411          |
-| 1.1377        | 2.8095 | 30   | 1.1380          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.2878        | 0.5   | 1    | 1.2576          |
+| 1.294         | 1.0   | 2    | 1.2571          |
+| 1.2719        | 1.375 | 3    | 1.2468          |
+| 1.2869        | 1.875 | 4    | 1.2302          |
+| 1.2828        | 2.25  | 5    | 1.2147          |
+| 1.2449        | 2.75  | 6    | 1.2145          |
+| 1.2385        | 3.125 | 7    | 1.2129          |
+| 1.2142        | 3.625 | 8    | 1.2061          |
+| 1.2725        | 4.125 | 9    | 1.1927          |
+| 1.2282        | 4.5   | 10   | 1.1848          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:267858af8019eb7e782699cae15101562b55ccdf93cf3dc7d1d602d0c507ea90
+oid sha256:663e8219297d65bc184eb8f4abcf0507f9efed03f9d35d5b9fc1a1f5ffdd8295
 size 45169354