End of training

README.md

@@ -1,7 +1,5 @@
 ---
 tags:
-- trl
-- sft
 - generated_from_trainer
 model-index:
 - name: minillama2
@@ -15,12 +13,12 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- eval_loss: 8.6515
-- eval_runtime: 0.0987
-- eval_samples_per_second: 101.294
-- eval_steps_per_second: 10.129
-- epoch: 0.0567
-- step: 34000
+- eval_loss: 3.9366
+- eval_runtime: 0.8539
+- eval_samples_per_second: 149.905
+- eval_steps_per_second: 2.342
+- epoch: 0.1147
+- step: 4300
 
 ## Model description
 
@@ -39,13 +37,16 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0001
+- learning_rate: 0.0005
 - train_batch_size: 32
-- eval_batch_size: 16
+- eval_batch_size: 64
 - seed: 42
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 256
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- training_steps: 600000.0
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 100
+- training_steps: 37500.0
 
 ### Framework versions
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:db4634f8628007f7d029bf4bae32af527075b2d64f1e5348329382be52f1ead7
+oid sha256:f144a986bd7ce2fc8ac8f5a6a309585a3a28bcad354d83a90a67c634fb45387f
 size 100963088