End of training

README.md

@@ -31,7 +31,7 @@ deepspeed: null
 early_stopping_patience: null
 eval_max_new_tokens: 128
 eval_table_size: null
-evals_per_epoch: 4
+evals_per_epoch: 2
 flash_attention: true
 fp16: null
 fsdp: null
@@ -55,10 +55,10 @@ lora_r: 16
 lora_target_linear: true
 lr_scheduler: cosine
 max_steps: 10
-micro_batch_size: 2
+micro_batch_size: 10
 mlflow_experiment_name: mhenrichsen/alpaca_2k_test
 model_type: LlamaForCausalLM
-num_epochs: 1
+num_epochs: 3
 optimizer: adamw_bnb_8bit
 output_dir: miner_id_besimray
 pad_to_sequence_len: true
@@ -90,7 +90,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [unsloth/Llama-3.2-1B-Instruct](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.2174
+- Loss: 1.1916
 
 ## Model description
 
@@ -110,11 +110,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
-- train_batch_size: 2
-- eval_batch_size: 2
+- train_batch_size: 10
+- eval_batch_size: 10
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 8
+- total_train_batch_size: 40
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_steps: 10
@@ -124,10 +124,12 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 1.3218        | 0.0042 | 1    | 1.2625          |
-| 1.3035        | 0.0126 | 3    | 1.2594          |
-| 1.492         | 0.0253 | 6    | 1.2116          |
-| 1.2852        | 0.0379 | 9    | 1.2174          |
+| 1.2983        | 0.0211 | 1    | 1.2586          |
+| 1.4576        | 0.0421 | 2    | 1.2597          |
+| 1.4211        | 0.0842 | 4    | 1.2397          |
+| 1.3893        | 0.1263 | 6    | 1.2157          |
+| 1.2216        | 0.1684 | 8    | 1.2087          |
+| 1.3764        | 0.2105 | 10   | 1.1916          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d7dac8b42797c0eba41eeece25e187c08bb4a7765066ff147ecb71b9a2c50e22
+oid sha256:51f99ff1b462cd062f5ddf5de2aff51080840a6e9b6f60e6151cf655eeec5836
 size 45169354