anilguleroglu/trained_llama_model_with_gqa

README.md

@@ -12,6 +12,8 @@ should probably proofread and complete it, then remove this comment. -->
 # results
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 2.5183
 
 ## Model description
 
@@ -31,11 +33,11 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
-- train_batch_size: 256
-- eval_batch_size: 256
+- train_batch_size: 128
+- eval_batch_size: 128
 - seed: 42
 - gradient_accumulation_steps: 8
-- total_train_batch_size: 2048
+- total_train_batch_size: 1024
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 500
@@ -43,6 +45,9 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 2.9463        | 0.7733 | 400  | 2.5183          |
 
 
 ### Framework versions

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d01d56e74c091b12c467d4ccaff4dd2c91aa447f4893e7f366fbbbbca68c2c98
+oid sha256:7f03d5eb1de64bfb1ed49557d268ecb64b15c42c3d18564e372482a6b44f5f93
 size 649681952

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:862313b7a7ad712b17810ce1dcb2dcc921a1152c8bd52424e7820142a53746f8
+oid sha256:f6403dfadea2e25d15043079b9aaabfe969833c458b9ae3ab0ef19780f25d084
 size 5112