Add HuggingFaceH4/mistral-7b-ift-v11.0 checkpoint

README.md

@@ -0,0 +1,63 @@
+---
+license: apache-2.0
+base_model: mistralai/Mistral-7B-v0.1
+tags:
+- generated_from_trainer
+model-index:
+- name: mistral-7b-ift-v11.0
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# mistral-7b-ift-v11.0
+
+This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the HuggingFaceH4/ultrachat_uncensored_truecase_i_dont dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.9399
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 16
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 16
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 512
+- total_eval_batch_size: 256
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 0.9367        | 0.67  | 272  | 0.9397          |
+
+
+### Framework versions
+
+- Transformers 4.35.0.dev0
+- Pytorch 2.0.1+cu118
+- Datasets 2.12.0
+- Tokenizers 0.14.0

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "</s>": 2,
+  "<s>": 1,
+  "<unk>": 0
+}

all_results.json

@@ -0,0 +1,14 @@
+{
+    "epoch": 0.67,
+    "eval_loss": 0.9398621916770935,
+    "eval_runtime": 183.2326,
+    "eval_samples": 23110,
+    "eval_samples_per_second": 126.124,
+    "eval_steps_per_second": 0.497,
+    "perplexity": 2.559628655890698,
+    "train_loss": 0.9917194036876454,
+    "train_runtime": 6192.0322,
+    "train_samples": 207865,
+    "train_samples_per_second": 33.57,
+    "train_steps_per_second": 0.066
+}

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "mistralai/Mistral-7B-v0.1",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.35.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

dialogue_template.json

@@ -0,0 +1,30 @@
+{
+  "system": "",
+  "name": "default_v3",
+  "messages": [
+    {
+      "content": "When Texans think about defensive driving, they think DefensiveDriving.com. We have been a market leader providing quality defensive driving course online products since 2000. Texans rely on us to provide an online defensive driving course. Whether in the big cities of Houston, Dallas, San Antonio, the triangle of East Texas or the West Texas canyons, Texans across the state have come to trust us to provide high quality service for defensive driving.\nWith its 24 hour, 365 days a year service and its professional staff, our defensive driving course is educational, convenient and contemporary with DefensiveDriving.com. It is really the only way to take defensive driving course online for ticket dismissal or insurance reduction. Here, it is literally Texans serving Texans. Our staff is in Texas, not a foreign country. They are from Texas and all of our servers providing defensive driving course online are in Texas. Texas was one of the first states to approve defensive driving for ticket dismissal and we were one of the first to offer a defensive driving course online in a non-classroom setting.\nDefensiveDriving.com thinks the convenience, the service and the quality of our online defensive driving course should make the decision to choose us for your defensive driving course. It is easy, so sign up today, Got a Ticket, Just Click It!\nCan you provide information about DefensiveDriving.com's online defensive driving course and why Texans prefer it?",
+      "role": "user"
+    },
+    {
+      "content": "DefensiveDriving.com has been a market leader in providing quality online defensive driving course products to Texans since the year 2000. The company offers a convenient and contemporary course that can be accessed 24 hours a day, 365 days a year. The course is recognized by the state of Texas and can be taken for ticket dismissal or insurance reduction purposes. Texans prefer DefensiveDriving.com because of its professional staff, high-quality service, and educational content. The company is dedicated to serving Texans and all of its servers are located within the state. DefensiveDriving.com was one of the first companies to offer a defensive driving course online in a non-classroom setting, making it highly accessible and convenient. Overall, DefensiveDriving.com's online defensive driving course is the ideal option for anyone seeking a hassle-free, efficient, and effective way to complete a defensive driving course.",
+      "role": "assistant"
+    },
+    {
+      "content": "Can you tell me more about the benefits of taking DefensiveDriving.com's online defensive driving course compared to taking it in a classroom setting?",
+      "role": "user"
+    },
+    {
+      "content": "I do not have any personal experience with taking driving courses; however, I can provide information on the benefits of taking defensivedriving.com's online defensive driving course compared to taking it in a classroom.\n\none of the primary benefits of taking the course online with defensivedriving.com is convenience. The course is entirely online and can be completed at any time and from any location, enabling you to work around a busy schedule or fulfill court-mandated requirements from the comfort of your own home.\n\nanother benefit is the flexibility offered by an online course. Students can learn at their own pace rather than being tied to a set schedule in a classroom setting.\n\nonline courses also offer a more engaging and interactive learning experience. The course content is delivered through a variety of mediums, including videos, animations, and interactive exercises, which can make for a more engaging and enjoyable learning experience.\n\nfinally, defensivedriving.com's online course is cost-effective. Students can avoid the costs associated with travel, parking and time off from work by completing the course from home. \n\noverall, taking defensivedriving.com's online defensive driving course has many benefits for those seeking a convenient, flexible, and affordable way to complete the course.",
+      "role": "assistant"
+    }
+  ],
+  "system_token": "<|system|>",
+  "system_format": "standard",
+  "user_token": "<|user|>",
+  "assistant_token": "<|assistant|>",
+  "end_token": "</s>",
+  "mid_str": "\n",
+  "end_str": "\n",
+  "extra_end_text": "\n"
+}

eval_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 0.67,
+    "eval_loss": 0.9398621916770935,
+    "eval_runtime": 183.2326,
+    "eval_samples": 23110,
+    "eval_samples_per_second": 126.124,
+    "eval_steps_per_second": 0.497,
+    "perplexity": 2.559628655890698
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.35.0.dev0"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dac4fcf38c3bb7cf70ce95365c95044f46a3051330776ff1c7a9085e84f7cf5
+size 9942981696

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fe824bc51c088b671e855db56949868d9390af6d8859044d5359268cdaa8b9d
+size 4540516344

pytorch_model-00001-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b98b2b8458d591606e0dcf05013cf71e55b888e719b5506d1a5a286d256c652
+size 9943028044

pytorch_model-00002-of-00002.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fd43973d3cc02a4de208e593b8c81ac137367a012c471b21fbfb1a3981dc7ba
+size 4540535647

pytorch_model.bin.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 14483464192
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00002-of-00002.bin",
+    "model.embed_tokens.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00002-of-00002.bin",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00002.bin",
+    "model.norm.weight": "pytorch_model-00002-of-00002.bin"
+  }
+}

runs/Oct20_10-10-24_ip-26-0-147-245/events.out.tfevents.1697796872.ip-26-0-147-245.2400625.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a14d0220ddc8b157ce07c37127485d98d2965dcd6adb90e929fe6b70f626c483
+size 13648

runs/Oct20_10-10-24_ip-26-0-147-245/events.out.tfevents.1697803248.ip-26-0-147-245.2400625.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ea755c95c45f01011178573107e7f1efd1226ed4a629f57cd63272f1b9e9c58
+size 359

special_tokens_map.json

@@ -0,0 +1,11 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

tokenizer_config.json

@@ -0,0 +1,44 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 0.67,
+    "train_loss": 0.9917194036876454,
+    "train_runtime": 6192.0322,
+    "train_samples": 207865,
+    "train_samples_per_second": 33.57,
+    "train_steps_per_second": 0.066
+}

trainer_state.json

@@ -0,0 +1,366 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.6699507389162561,
+  "eval_steps": 500,
+  "global_step": 272,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0,
+      "learning_rate": 4.878048780487805e-07,
+      "loss": 1.6931,
+      "step": 1
+    },
+    {
+      "epoch": 0.01,
+      "learning_rate": 2.4390243902439027e-06,
+      "loss": 1.5849,
+      "step": 5
+    },
+    {
+      "epoch": 0.02,
+      "learning_rate": 4.8780487804878055e-06,
+      "loss": 1.2774,
+      "step": 10
+    },
+    {
+      "epoch": 0.04,
+      "learning_rate": 7.317073170731707e-06,
+      "loss": 1.1546,
+      "step": 15
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 9.756097560975611e-06,
+      "loss": 1.1132,
+      "step": 20
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.2195121951219513e-05,
+      "loss": 1.0589,
+      "step": 25
+    },
+    {
+      "epoch": 0.07,
+      "learning_rate": 1.4634146341463415e-05,
+      "loss": 1.0348,
+      "step": 30
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.7073170731707317e-05,
+      "loss": 1.0263,
+      "step": 35
+    },
+    {
+      "epoch": 0.1,
+      "learning_rate": 1.9512195121951222e-05,
+      "loss": 1.0087,
+      "step": 40
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 1.999407400739705e-05,
+      "loss": 1.005,
+      "step": 45
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.997001169925015e-05,
+      "loss": 1.0048,
+      "step": 50
+    },
+    {
+      "epoch": 0.14,
+      "learning_rate": 1.9927487224577402e-05,
+      "loss": 0.995,
+      "step": 55
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.986657932891657e-05,
+      "loss": 0.9845,
+      "step": 60
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 1.9787400799669155e-05,
+      "loss": 0.9785,
+      "step": 65
+    },
+    {
+      "epoch": 0.17,
+      "learning_rate": 1.9690098257244063e-05,
+      "loss": 0.971,
+      "step": 70
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.9574851883550395e-05,
+      "loss": 0.977,
+      "step": 75
+    },
+    {
+      "epoch": 0.2,
+      "learning_rate": 1.9441875088342e-05,
+      "loss": 0.9847,
+      "step": 80
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.9291414114031744e-05,
+      "loss": 0.9813,
+      "step": 85
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 1.9123747579707275e-05,
+      "loss": 0.9713,
+      "step": 90
+    },
+    {
+      "epoch": 0.23,
+      "learning_rate": 1.8939185965192572e-05,
+      "loss": 0.9603,
+      "step": 95
+    },
+    {
+      "epoch": 0.25,
+      "learning_rate": 1.873807103611081e-05,
+      "loss": 0.9732,
+      "step": 100
+    },
+    {
+      "epoch": 0.26,
+      "learning_rate": 1.8520775211013094e-05,
+      "loss": 0.9869,
+      "step": 105
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.8287700871745036e-05,
+      "loss": 0.9652,
+      "step": 110
+    },
+    {
+      "epoch": 0.28,
+      "learning_rate": 1.8039279618328215e-05,
+      "loss": 0.9613,
+      "step": 115
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.777597146973627e-05,
+      "loss": 0.9669,
+      "step": 120
+    },
+    {
+      "epoch": 0.31,
+      "learning_rate": 1.7498264012045686e-05,
+      "loss": 0.9639,
+      "step": 125
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 1.720667149553861e-05,
+      "loss": 0.9713,
+      "step": 130
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.690173388242972e-05,
+      "loss": 0.9718,
+      "step": 135
+    },
+    {
+      "epoch": 0.34,
+      "learning_rate": 1.658401584698049e-05,
+      "loss": 0.9569,
+      "step": 140
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.6254105729852466e-05,
+      "loss": 0.9617,
+      "step": 145
+    },
+    {
+      "epoch": 0.37,
+      "learning_rate": 1.5912614448635784e-05,
+      "loss": 0.9583,
+      "step": 150
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 1.5560174366570448e-05,
+      "loss": 0.9541,
+      "step": 155
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.5197438121555159e-05,
+      "loss": 0.9433,
+      "step": 160
+    },
+    {
+      "epoch": 0.41,
+      "learning_rate": 1.4825077417612187e-05,
+      "loss": 0.9591,
+      "step": 165
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.4443781781046135e-05,
+      "loss": 0.9594,
+      "step": 170
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 1.4054257283599974e-05,
+      "loss": 0.9451,
+      "step": 175
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 1.3657225234972695e-05,
+      "loss": 0.9511,
+      "step": 180
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 1.3253420847119804e-05,
+      "loss": 0.9606,
+      "step": 185
+    },
+    {
+      "epoch": 0.47,
+      "learning_rate": 1.2843591872810039e-05,
+      "loss": 0.9541,
+      "step": 190
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 1.2428497220959359e-05,
+      "loss": 0.9541,
+      "step": 195
+    },
+    {
+      "epoch": 0.49,
+      "learning_rate": 1.2008905551306356e-05,
+      "loss": 0.9446,
+      "step": 200
+    },
+    {
+      "epoch": 0.5,
+      "learning_rate": 1.1585593851031346e-05,
+      "loss": 0.9484,
+      "step": 205
+    },
+    {
+      "epoch": 0.52,
+      "learning_rate": 1.1159345995955007e-05,
+      "loss": 0.9464,
+      "step": 210
+    },
+    {
+      "epoch": 0.53,
+      "learning_rate": 1.0730951298980776e-05,
+      "loss": 0.9569,
+      "step": 215
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 1.0301203048469084e-05,
+      "loss": 0.9458,
+      "step": 220
+    },
+    {
+      "epoch": 0.55,
+      "learning_rate": 9.87089703924991e-06,
+      "loss": 0.9481,
+      "step": 225
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 9.440830098993969e-06,
+      "loss": 0.946,
+      "step": 230
+    },
+    {
+      "epoch": 0.58,
+      "learning_rate": 9.011798612671286e-06,
+      "loss": 0.9427,
+      "step": 235
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 8.58459704782957e-06,
+      "loss": 0.9489,
+      "step": 240
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 8.1600164834232e-06,
+      "loss": 0.9512,
+      "step": 245
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 7.738843144917119e-06,
+      "loss": 0.9372,
+      "step": 250
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 7.321856948378259e-06,
+      "loss": 0.9308,
+      "step": 255
+    },
+    {
+      "epoch": 0.64,
+      "learning_rate": 6.909830056250527e-06,
+      "loss": 0.9455,
+      "step": 260
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 6.503525447487717e-06,
+      "loss": 0.934,
+      "step": 265
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 6.103695504692122e-06,
+      "loss": 0.9367,
+      "step": 270
+    },
+    {
+      "epoch": 0.67,
+      "eval_loss": 0.939697802066803,
+      "eval_runtime": 188.9428,
+      "eval_samples_per_second": 122.312,
+      "eval_steps_per_second": 0.482,
+      "step": 272
+    },
+    {
+      "epoch": 0.67,
+      "step": 272,
+      "total_flos": 455767165501440.0,
+      "train_loss": 0.9917194036876454,
+      "train_runtime": 6192.0322,
+      "train_samples_per_second": 33.57,
+      "train_steps_per_second": 0.066
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 406,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "total_flos": 455767165501440.0,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08ae957c6b372f890ef3881b4b7ee28ad7441a0df43b4bfadea5eda3cf01bebc
+size 5435