Model save

README.md

@@ -0,0 +1,65 @@
+---
+library_name: transformers
+license: llama3.1
+base_model: meta-llama/Meta-Llama-3.1-8B
+tags:
+- llama-factory
+- generated_from_trainer
+model-index:
+- name: llama3-1_8b_mlfoundations-dev-stackoverflow_10000tasks__5p
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# llama3-1_8b_mlfoundations-dev-stackoverflow_10000tasks__5p
+
+This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6614
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 512
+- total_eval_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: constant
+- num_epochs: 3.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 0.7103        | 0.9724 | 33   | 0.6921          |
+| 0.6364        | 1.9742 | 67   | 0.6614          |
+
+
+### Framework versions
+
+- Transformers 4.45.2
+- Pytorch 2.5.1+cu124
+- Datasets 3.1.0
+- Tokenizers 0.20.1

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.45.2"
+}

model-00001-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a61e606eafe10fc6743398a3b9bb85d56f6b6882fc510dcf4c2ba7ccf399336f
+oid sha256:52eb957f46d636d293d306f68a7500e06cf9f45cc1816bac7b09a9755ee9eaa9
 size 4976698672

model-00002-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3734c29639c7f6c6f0692af54514d0ab98848d60aac3f80feb54b76d23d65ee8
+oid sha256:07552d37fa4e0ecb3bc83e0675c67ec0a22fc16c55bdf1401af6b0323548580a
 size 4999802720

model-00003-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3fc59f99d68f605ba19730e736dc1fa5626c4d8e5b85cd133f6f350b12c810b1
+oid sha256:153ac2d15a6297c3654309f77298e82f1f85b8deea42a9ac93df123cb39d4976
 size 4915916176

model-00004-of-00004.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1292baab890f9ffcbcac85b59a160194b06c63b63973bdf67357afc347524e3e
+oid sha256:353072c08ab52f693c61376b41c1c1d832f621ef767b3ca15a39f059573dceee
 size 1168138808

trainer_log.jsonl

@@ -1,8 +1 @@
-{"current_steps": 10, "total_steps": 99, "loss": 0.8598, "lr": 5e-06, "epoch": 0.2946593001841621, "percentage": 10.1, "elapsed_time": "0:26:05", "remaining_time": "3:52:14"}
-{"current_steps": 20, "total_steps": 99, "loss": 0.7522, "lr": 5e-06, "epoch": 0.5893186003683242, "percentage": 20.2, "elapsed_time": "0:52:01", "remaining_time": "3:25:29"}
-{"current_steps": 30, "total_steps": 99, "loss": 0.7103, "lr": 5e-06, "epoch": 0.8839779005524862, "percentage": 30.3, "elapsed_time": "1:17:56", "remaining_time": "2:59:15"}
-{"current_steps": 33, "total_steps": 99, "eval_loss": 0.6921380162239075, "epoch": 0.9723756906077348, "percentage": 33.33, "elapsed_time": "1:29:53", "remaining_time": "2:59:46"}
-{"current_steps": 40, "total_steps": 99, "loss": 0.6749, "lr": 5e-06, "epoch": 1.1786372007366483, "percentage": 40.4, "elapsed_time": "1:46:59", "remaining_time": "2:37:49"}
-{"current_steps": 50, "total_steps": 99, "loss": 0.6447, "lr": 5e-06, "epoch": 1.4732965009208103, "percentage": 50.51, "elapsed_time": "2:12:54", "remaining_time": "2:10:14"}
-{"current_steps": 60, "total_steps": 99, "loss": 0.6364, "lr": 5e-06, "epoch": 1.7679558011049723, "percentage": 60.61, "elapsed_time": "2:38:48", "remaining_time": "1:43:13"}
-{"current_steps": 67, "total_steps": 99, "eval_loss": 0.6614336371421814, "epoch": 1.9742173112338857, "percentage": 67.68, "elapsed_time": "3:00:57", "remaining_time": "1:26:25"}
+{"current_steps": 99, "total_steps": 99, "epoch": 2.9171270718232045, "percentage": 100.0, "elapsed_time": "0:00:00", "remaining_time": "0:00:00"}

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d483190452ae25dcf73c9b935732cb7dc0d1f92e9a0b828d40d82f486f4aec24
-size 7288
+oid sha256:93a3b073be09da47238f51566db09bc964363ac179d07f86d7b02f7440b20f07
+size 7352