Training in progress, step 304, checkpoint

checkpoint-304/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/Qwen2.5-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.2

checkpoint-304/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "v_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": true
+}

checkpoint-304/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b4288b4444b8fdb5ee20ca5f28c4546da413e74364c6564b9ff3e79ae649803
+size 323014168

checkpoint-304/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-304/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-304/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a0a13fc4dda0002de7ecddcc6ff8633bb7360b302cf6ee06b817558468b3dc24
+size 164465413

checkpoint-304/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2581c486c2f0bf87e01082642970aa1b1009c41975d0a49b24b0a9781781052c
+size 14581

checkpoint-304/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8001c10d624922cafadc7ca6dfc8fe1dc0bd8e7a352621c2cdb5ff8521fadc75
+size 1465

checkpoint-304/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-304/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-304/tokenizer_config.json

@@ -0,0 +1,208 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-304/trainer_state.json

@@ -0,0 +1,2210 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 50,
+  "global_step": 304,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032921810699588477,
+      "grad_norm": 10.407278060913086,
+      "learning_rate": 0.0,
+      "loss": 1.082,
+      "step": 1
+    },
+    {
+      "epoch": 0.006584362139917695,
+      "grad_norm": 10.749567985534668,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 1.2311,
+      "step": 2
+    },
+    {
+      "epoch": 0.009876543209876543,
+      "grad_norm": 10.224628448486328,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 1.0955,
+      "step": 3
+    },
+    {
+      "epoch": 0.01316872427983539,
+      "grad_norm": 12.155407905578613,
+      "learning_rate": 6e-06,
+      "loss": 1.3251,
+      "step": 4
+    },
+    {
+      "epoch": 0.01646090534979424,
+      "grad_norm": 10.422830581665039,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.9511,
+      "step": 5
+    },
+    {
+      "epoch": 0.019753086419753086,
+      "grad_norm": 13.046320915222168,
+      "learning_rate": 1e-05,
+      "loss": 1.0949,
+      "step": 6
+    },
+    {
+      "epoch": 0.023045267489711935,
+      "grad_norm": 11.915493965148926,
+      "learning_rate": 9.966555183946488e-06,
+      "loss": 1.0078,
+      "step": 7
+    },
+    {
+      "epoch": 0.02633744855967078,
+      "grad_norm": 8.029522895812988,
+      "learning_rate": 9.933110367892978e-06,
+      "loss": 0.6369,
+      "step": 8
+    },
+    {
+      "epoch": 0.02962962962962963,
+      "grad_norm": 7.477173805236816,
+      "learning_rate": 9.899665551839465e-06,
+      "loss": 0.541,
+      "step": 9
+    },
+    {
+      "epoch": 0.03292181069958848,
+      "grad_norm": 6.052078723907471,
+      "learning_rate": 9.866220735785954e-06,
+      "loss": 0.5369,
+      "step": 10
+    },
+    {
+      "epoch": 0.03621399176954732,
+      "grad_norm": 6.789449214935303,
+      "learning_rate": 9.832775919732442e-06,
+      "loss": 0.5544,
+      "step": 11
+    },
+    {
+      "epoch": 0.03950617283950617,
+      "grad_norm": 7.142099857330322,
+      "learning_rate": 9.799331103678931e-06,
+      "loss": 0.3411,
+      "step": 12
+    },
+    {
+      "epoch": 0.04279835390946502,
+      "grad_norm": 4.035250186920166,
+      "learning_rate": 9.765886287625419e-06,
+      "loss": 0.3193,
+      "step": 13
+    },
+    {
+      "epoch": 0.04609053497942387,
+      "grad_norm": 8.023409843444824,
+      "learning_rate": 9.732441471571908e-06,
+      "loss": 0.2348,
+      "step": 14
+    },
+    {
+      "epoch": 0.04938271604938271,
+      "grad_norm": 4.609869003295898,
+      "learning_rate": 9.698996655518395e-06,
+      "loss": 0.3047,
+      "step": 15
+    },
+    {
+      "epoch": 0.05267489711934156,
+      "grad_norm": 2.2414326667785645,
+      "learning_rate": 9.665551839464884e-06,
+      "loss": 0.2832,
+      "step": 16
+    },
+    {
+      "epoch": 0.05596707818930041,
+      "grad_norm": 2.9218909740448,
+      "learning_rate": 9.632107023411372e-06,
+      "loss": 0.3209,
+      "step": 17
+    },
+    {
+      "epoch": 0.05925925925925926,
+      "grad_norm": 2.327075242996216,
+      "learning_rate": 9.598662207357861e-06,
+      "loss": 0.4692,
+      "step": 18
+    },
+    {
+      "epoch": 0.06255144032921811,
+      "grad_norm": 1.6720025539398193,
+      "learning_rate": 9.565217391304349e-06,
+      "loss": 0.3791,
+      "step": 19
+    },
+    {
+      "epoch": 0.06584362139917696,
+      "grad_norm": 1.5597883462905884,
+      "learning_rate": 9.531772575250838e-06,
+      "loss": 0.2174,
+      "step": 20
+    },
+    {
+      "epoch": 0.0691358024691358,
+      "grad_norm": 1.6142524480819702,
+      "learning_rate": 9.498327759197325e-06,
+      "loss": 0.246,
+      "step": 21
+    },
+    {
+      "epoch": 0.07242798353909465,
+      "grad_norm": 1.313617467880249,
+      "learning_rate": 9.464882943143815e-06,
+      "loss": 0.254,
+      "step": 22
+    },
+    {
+      "epoch": 0.0757201646090535,
+      "grad_norm": 1.499535322189331,
+      "learning_rate": 9.431438127090302e-06,
+      "loss": 0.3472,
+      "step": 23
+    },
+    {
+      "epoch": 0.07901234567901234,
+      "grad_norm": 1.5803781747817993,
+      "learning_rate": 9.39799331103679e-06,
+      "loss": 0.322,
+      "step": 24
+    },
+    {
+      "epoch": 0.0823045267489712,
+      "grad_norm": 1.3434972763061523,
+      "learning_rate": 9.364548494983279e-06,
+      "loss": 0.2204,
+      "step": 25
+    },
+    {
+      "epoch": 0.08559670781893004,
+      "grad_norm": 1.6280096769332886,
+      "learning_rate": 9.331103678929766e-06,
+      "loss": 0.3198,
+      "step": 26
+    },
+    {
+      "epoch": 0.08888888888888889,
+      "grad_norm": 1.1826003789901733,
+      "learning_rate": 9.297658862876256e-06,
+      "loss": 0.2687,
+      "step": 27
+    },
+    {
+      "epoch": 0.09218106995884774,
+      "grad_norm": 1.116934895515442,
+      "learning_rate": 9.264214046822743e-06,
+      "loss": 0.2469,
+      "step": 28
+    },
+    {
+      "epoch": 0.09547325102880659,
+      "grad_norm": 1.7356836795806885,
+      "learning_rate": 9.230769230769232e-06,
+      "loss": 0.2841,
+      "step": 29
+    },
+    {
+      "epoch": 0.09876543209876543,
+      "grad_norm": 1.2330541610717773,
+      "learning_rate": 9.19732441471572e-06,
+      "loss": 0.244,
+      "step": 30
+    },
+    {
+      "epoch": 0.10205761316872428,
+      "grad_norm": 0.9743934273719788,
+      "learning_rate": 9.163879598662207e-06,
+      "loss": 0.1321,
+      "step": 31
+    },
+    {
+      "epoch": 0.10534979423868313,
+      "grad_norm": 0.884885311126709,
+      "learning_rate": 9.130434782608697e-06,
+      "loss": 0.1349,
+      "step": 32
+    },
+    {
+      "epoch": 0.10864197530864197,
+      "grad_norm": 1.0773906707763672,
+      "learning_rate": 9.096989966555184e-06,
+      "loss": 0.1741,
+      "step": 33
+    },
+    {
+      "epoch": 0.11193415637860082,
+      "grad_norm": 1.1486691236495972,
+      "learning_rate": 9.063545150501673e-06,
+      "loss": 0.2394,
+      "step": 34
+    },
+    {
+      "epoch": 0.11522633744855967,
+      "grad_norm": 1.2714940309524536,
+      "learning_rate": 9.03010033444816e-06,
+      "loss": 0.2211,
+      "step": 35
+    },
+    {
+      "epoch": 0.11851851851851852,
+      "grad_norm": 1.30379056930542,
+      "learning_rate": 8.996655518394648e-06,
+      "loss": 0.2437,
+      "step": 36
+    },
+    {
+      "epoch": 0.12181069958847737,
+      "grad_norm": 1.4799457788467407,
+      "learning_rate": 8.963210702341138e-06,
+      "loss": 0.3251,
+      "step": 37
+    },
+    {
+      "epoch": 0.12510288065843622,
+      "grad_norm": 1.110399603843689,
+      "learning_rate": 8.929765886287625e-06,
+      "loss": 0.2332,
+      "step": 38
+    },
+    {
+      "epoch": 0.12839506172839507,
+      "grad_norm": 1.3540667295455933,
+      "learning_rate": 8.896321070234114e-06,
+      "loss": 0.2599,
+      "step": 39
+    },
+    {
+      "epoch": 0.13168724279835392,
+      "grad_norm": 1.2311596870422363,
+      "learning_rate": 8.862876254180602e-06,
+      "loss": 0.1963,
+      "step": 40
+    },
+    {
+      "epoch": 0.13497942386831277,
+      "grad_norm": 1.2690569162368774,
+      "learning_rate": 8.829431438127091e-06,
+      "loss": 0.173,
+      "step": 41
+    },
+    {
+      "epoch": 0.1382716049382716,
+      "grad_norm": 1.114758849143982,
+      "learning_rate": 8.795986622073578e-06,
+      "loss": 0.2128,
+      "step": 42
+    },
+    {
+      "epoch": 0.14156378600823044,
+      "grad_norm": 1.4038128852844238,
+      "learning_rate": 8.762541806020068e-06,
+      "loss": 0.3228,
+      "step": 43
+    },
+    {
+      "epoch": 0.1448559670781893,
+      "grad_norm": 0.9311850666999817,
+      "learning_rate": 8.729096989966555e-06,
+      "loss": 0.1835,
+      "step": 44
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "grad_norm": 1.3008413314819336,
+      "learning_rate": 8.695652173913044e-06,
+      "loss": 0.24,
+      "step": 45
+    },
+    {
+      "epoch": 0.151440329218107,
+      "grad_norm": 1.2420703172683716,
+      "learning_rate": 8.662207357859532e-06,
+      "loss": 0.2392,
+      "step": 46
+    },
+    {
+      "epoch": 0.15473251028806584,
+      "grad_norm": 1.244697093963623,
+      "learning_rate": 8.628762541806021e-06,
+      "loss": 0.1978,
+      "step": 47
+    },
+    {
+      "epoch": 0.1580246913580247,
+      "grad_norm": 1.030407428741455,
+      "learning_rate": 8.595317725752509e-06,
+      "loss": 0.174,
+      "step": 48
+    },
+    {
+      "epoch": 0.16131687242798354,
+      "grad_norm": 1.130656361579895,
+      "learning_rate": 8.561872909698998e-06,
+      "loss": 0.2082,
+      "step": 49
+    },
+    {
+      "epoch": 0.1646090534979424,
+      "grad_norm": 1.228980541229248,
+      "learning_rate": 8.528428093645485e-06,
+      "loss": 0.1929,
+      "step": 50
+    },
+    {
+      "epoch": 0.1646090534979424,
+      "eval_loss": 0.2575027346611023,
+      "eval_runtime": 21.2346,
+      "eval_samples_per_second": 25.43,
+      "eval_steps_per_second": 12.715,
+      "step": 50
+    },
+    {
+      "epoch": 0.16790123456790124,
+      "grad_norm": 1.3388785123825073,
+      "learning_rate": 8.494983277591975e-06,
+      "loss": 0.1798,
+      "step": 51
+    },
+    {
+      "epoch": 0.17119341563786009,
+      "grad_norm": 1.02818763256073,
+      "learning_rate": 8.461538461538462e-06,
+      "loss": 0.1663,
+      "step": 52
+    },
+    {
+      "epoch": 0.17448559670781894,
+      "grad_norm": 1.8590612411499023,
+      "learning_rate": 8.42809364548495e-06,
+      "loss": 0.182,
+      "step": 53
+    },
+    {
+      "epoch": 0.17777777777777778,
+      "grad_norm": 1.138355016708374,
+      "learning_rate": 8.394648829431439e-06,
+      "loss": 0.1687,
+      "step": 54
+    },
+    {
+      "epoch": 0.18106995884773663,
+      "grad_norm": 1.0137665271759033,
+      "learning_rate": 8.361204013377926e-06,
+      "loss": 0.1587,
+      "step": 55
+    },
+    {
+      "epoch": 0.18436213991769548,
+      "grad_norm": 1.2493849992752075,
+      "learning_rate": 8.327759197324416e-06,
+      "loss": 0.2544,
+      "step": 56
+    },
+    {
+      "epoch": 0.18765432098765433,
+      "grad_norm": 1.207789421081543,
+      "learning_rate": 8.294314381270903e-06,
+      "loss": 0.1917,
+      "step": 57
+    },
+    {
+      "epoch": 0.19094650205761318,
+      "grad_norm": 0.9713874459266663,
+      "learning_rate": 8.260869565217392e-06,
+      "loss": 0.1037,
+      "step": 58
+    },
+    {
+      "epoch": 0.194238683127572,
+      "grad_norm": 1.151794672012329,
+      "learning_rate": 8.22742474916388e-06,
+      "loss": 0.1605,
+      "step": 59
+    },
+    {
+      "epoch": 0.19753086419753085,
+      "grad_norm": 1.229486346244812,
+      "learning_rate": 8.193979933110369e-06,
+      "loss": 0.2045,
+      "step": 60
+    },
+    {
+      "epoch": 0.2008230452674897,
+      "grad_norm": 1.2773410081863403,
+      "learning_rate": 8.160535117056857e-06,
+      "loss": 0.3142,
+      "step": 61
+    },
+    {
+      "epoch": 0.20411522633744855,
+      "grad_norm": 1.1839615106582642,
+      "learning_rate": 8.127090301003346e-06,
+      "loss": 0.1706,
+      "step": 62
+    },
+    {
+      "epoch": 0.2074074074074074,
+      "grad_norm": 1.253584861755371,
+      "learning_rate": 8.093645484949833e-06,
+      "loss": 0.1947,
+      "step": 63
+    },
+    {
+      "epoch": 0.21069958847736625,
+      "grad_norm": 1.0032668113708496,
+      "learning_rate": 8.060200668896322e-06,
+      "loss": 0.1662,
+      "step": 64
+    },
+    {
+      "epoch": 0.2139917695473251,
+      "grad_norm": 1.1352976560592651,
+      "learning_rate": 8.02675585284281e-06,
+      "loss": 0.1034,
+      "step": 65
+    },
+    {
+      "epoch": 0.21728395061728395,
+      "grad_norm": 1.1658047437667847,
+      "learning_rate": 7.9933110367893e-06,
+      "loss": 0.1948,
+      "step": 66
+    },
+    {
+      "epoch": 0.2205761316872428,
+      "grad_norm": 1.2854225635528564,
+      "learning_rate": 7.959866220735787e-06,
+      "loss": 0.1935,
+      "step": 67
+    },
+    {
+      "epoch": 0.22386831275720165,
+      "grad_norm": 0.884494423866272,
+      "learning_rate": 7.926421404682276e-06,
+      "loss": 0.1087,
+      "step": 68
+    },
+    {
+      "epoch": 0.2271604938271605,
+      "grad_norm": 1.3082432746887207,
+      "learning_rate": 7.892976588628763e-06,
+      "loss": 0.2341,
+      "step": 69
+    },
+    {
+      "epoch": 0.23045267489711935,
+      "grad_norm": 1.410191297531128,
+      "learning_rate": 7.859531772575253e-06,
+      "loss": 0.2985,
+      "step": 70
+    },
+    {
+      "epoch": 0.2337448559670782,
+      "grad_norm": 1.2850663661956787,
+      "learning_rate": 7.82608695652174e-06,
+      "loss": 0.2076,
+      "step": 71
+    },
+    {
+      "epoch": 0.23703703703703705,
+      "grad_norm": 1.4123177528381348,
+      "learning_rate": 7.792642140468228e-06,
+      "loss": 0.2442,
+      "step": 72
+    },
+    {
+      "epoch": 0.2403292181069959,
+      "grad_norm": 1.158871054649353,
+      "learning_rate": 7.759197324414717e-06,
+      "loss": 0.1767,
+      "step": 73
+    },
+    {
+      "epoch": 0.24362139917695474,
+      "grad_norm": 1.233568787574768,
+      "learning_rate": 7.725752508361204e-06,
+      "loss": 0.1454,
+      "step": 74
+    },
+    {
+      "epoch": 0.24691358024691357,
+      "grad_norm": 1.1103307008743286,
+      "learning_rate": 7.692307692307694e-06,
+      "loss": 0.2064,
+      "step": 75
+    },
+    {
+      "epoch": 0.25020576131687244,
+      "grad_norm": 1.687068223953247,
+      "learning_rate": 7.658862876254181e-06,
+      "loss": 0.2991,
+      "step": 76
+    },
+    {
+      "epoch": 0.25349794238683127,
+      "grad_norm": 1.1935592889785767,
+      "learning_rate": 7.62541806020067e-06,
+      "loss": 0.1805,
+      "step": 77
+    },
+    {
+      "epoch": 0.25679012345679014,
+      "grad_norm": 1.1262483596801758,
+      "learning_rate": 7.591973244147159e-06,
+      "loss": 0.1657,
+      "step": 78
+    },
+    {
+      "epoch": 0.26008230452674896,
+      "grad_norm": 1.4275016784667969,
+      "learning_rate": 7.558528428093647e-06,
+      "loss": 0.2439,
+      "step": 79
+    },
+    {
+      "epoch": 0.26337448559670784,
+      "grad_norm": 1.284867763519287,
+      "learning_rate": 7.5250836120401346e-06,
+      "loss": 0.1983,
+      "step": 80
+    },
+    {
+      "epoch": 0.26666666666666666,
+      "grad_norm": 1.3254787921905518,
+      "learning_rate": 7.491638795986622e-06,
+      "loss": 0.2161,
+      "step": 81
+    },
+    {
+      "epoch": 0.26995884773662554,
+      "grad_norm": 1.468256950378418,
+      "learning_rate": 7.4581939799331104e-06,
+      "loss": 0.2021,
+      "step": 82
+    },
+    {
+      "epoch": 0.27325102880658436,
+      "grad_norm": 1.1076890230178833,
+      "learning_rate": 7.424749163879599e-06,
+      "loss": 0.1348,
+      "step": 83
+    },
+    {
+      "epoch": 0.2765432098765432,
+      "grad_norm": 1.5892263650894165,
+      "learning_rate": 7.391304347826087e-06,
+      "loss": 0.2191,
+      "step": 84
+    },
+    {
+      "epoch": 0.27983539094650206,
+      "grad_norm": 1.0485304594039917,
+      "learning_rate": 7.3578595317725755e-06,
+      "loss": 0.0915,
+      "step": 85
+    },
+    {
+      "epoch": 0.2831275720164609,
+      "grad_norm": 0.9797968864440918,
+      "learning_rate": 7.324414715719064e-06,
+      "loss": 0.1427,
+      "step": 86
+    },
+    {
+      "epoch": 0.28641975308641976,
+      "grad_norm": 1.0853939056396484,
+      "learning_rate": 7.290969899665552e-06,
+      "loss": 0.1428,
+      "step": 87
+    },
+    {
+      "epoch": 0.2897119341563786,
+      "grad_norm": 1.3073103427886963,
+      "learning_rate": 7.257525083612041e-06,
+      "loss": 0.2187,
+      "step": 88
+    },
+    {
+      "epoch": 0.29300411522633746,
+      "grad_norm": 1.190982699394226,
+      "learning_rate": 7.224080267558529e-06,
+      "loss": 0.2245,
+      "step": 89
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 1.0931715965270996,
+      "learning_rate": 7.190635451505017e-06,
+      "loss": 0.1896,
+      "step": 90
+    },
+    {
+      "epoch": 0.29958847736625516,
+      "grad_norm": 0.9769377708435059,
+      "learning_rate": 7.157190635451506e-06,
+      "loss": 0.1369,
+      "step": 91
+    },
+    {
+      "epoch": 0.302880658436214,
+      "grad_norm": 1.213437557220459,
+      "learning_rate": 7.123745819397993e-06,
+      "loss": 0.2299,
+      "step": 92
+    },
+    {
+      "epoch": 0.30617283950617286,
+      "grad_norm": 1.043562650680542,
+      "learning_rate": 7.0903010033444816e-06,
+      "loss": 0.1541,
+      "step": 93
+    },
+    {
+      "epoch": 0.3094650205761317,
+      "grad_norm": 1.1540846824645996,
+      "learning_rate": 7.05685618729097e-06,
+      "loss": 0.2291,
+      "step": 94
+    },
+    {
+      "epoch": 0.31275720164609055,
+      "grad_norm": 1.1542479991912842,
+      "learning_rate": 7.023411371237458e-06,
+      "loss": 0.1819,
+      "step": 95
+    },
+    {
+      "epoch": 0.3160493827160494,
+      "grad_norm": 1.0938620567321777,
+      "learning_rate": 6.989966555183947e-06,
+      "loss": 0.2015,
+      "step": 96
+    },
+    {
+      "epoch": 0.31934156378600825,
+      "grad_norm": 1.2111386060714722,
+      "learning_rate": 6.956521739130435e-06,
+      "loss": 0.2172,
+      "step": 97
+    },
+    {
+      "epoch": 0.3226337448559671,
+      "grad_norm": 1.5160588026046753,
+      "learning_rate": 6.923076923076923e-06,
+      "loss": 0.2161,
+      "step": 98
+    },
+    {
+      "epoch": 0.32592592592592595,
+      "grad_norm": 1.1146801710128784,
+      "learning_rate": 6.889632107023412e-06,
+      "loss": 0.2172,
+      "step": 99
+    },
+    {
+      "epoch": 0.3292181069958848,
+      "grad_norm": 1.0889993906021118,
+      "learning_rate": 6.8561872909699e-06,
+      "loss": 0.1135,
+      "step": 100
+    },
+    {
+      "epoch": 0.3292181069958848,
+      "eval_loss": 0.23604722321033478,
+      "eval_runtime": 20.1337,
+      "eval_samples_per_second": 26.821,
+      "eval_steps_per_second": 13.41,
+      "step": 100
+    },
+    {
+      "epoch": 0.3325102880658436,
+      "grad_norm": 1.293273687362671,
+      "learning_rate": 6.8227424749163885e-06,
+      "loss": 0.2341,
+      "step": 101
+    },
+    {
+      "epoch": 0.3358024691358025,
+      "grad_norm": 1.2798124551773071,
+      "learning_rate": 6.789297658862877e-06,
+      "loss": 0.2153,
+      "step": 102
+    },
+    {
+      "epoch": 0.3390946502057613,
+      "grad_norm": 1.5441484451293945,
+      "learning_rate": 6.755852842809365e-06,
+      "loss": 0.1846,
+      "step": 103
+    },
+    {
+      "epoch": 0.34238683127572017,
+      "grad_norm": 1.6105937957763672,
+      "learning_rate": 6.7224080267558536e-06,
+      "loss": 0.2818,
+      "step": 104
+    },
+    {
+      "epoch": 0.345679012345679,
+      "grad_norm": 1.2185057401657104,
+      "learning_rate": 6.688963210702342e-06,
+      "loss": 0.1769,
+      "step": 105
+    },
+    {
+      "epoch": 0.34897119341563787,
+      "grad_norm": 1.155174732208252,
+      "learning_rate": 6.65551839464883e-06,
+      "loss": 0.1572,
+      "step": 106
+    },
+    {
+      "epoch": 0.3522633744855967,
+      "grad_norm": 1.010528326034546,
+      "learning_rate": 6.622073578595319e-06,
+      "loss": 0.185,
+      "step": 107
+    },
+    {
+      "epoch": 0.35555555555555557,
+      "grad_norm": 1.3826285600662231,
+      "learning_rate": 6.588628762541807e-06,
+      "loss": 0.2785,
+      "step": 108
+    },
+    {
+      "epoch": 0.3588477366255144,
+      "grad_norm": 1.0385446548461914,
+      "learning_rate": 6.5551839464882945e-06,
+      "loss": 0.1726,
+      "step": 109
+    },
+    {
+      "epoch": 0.36213991769547327,
+      "grad_norm": 1.0369651317596436,
+      "learning_rate": 6.521739130434783e-06,
+      "loss": 0.1476,
+      "step": 110
+    },
+    {
+      "epoch": 0.3654320987654321,
+      "grad_norm": 1.4338607788085938,
+      "learning_rate": 6.488294314381271e-06,
+      "loss": 0.2828,
+      "step": 111
+    },
+    {
+      "epoch": 0.36872427983539097,
+      "grad_norm": 1.228732705116272,
+      "learning_rate": 6.45484949832776e-06,
+      "loss": 0.1857,
+      "step": 112
+    },
+    {
+      "epoch": 0.3720164609053498,
+      "grad_norm": 1.3671075105667114,
+      "learning_rate": 6.421404682274248e-06,
+      "loss": 0.2108,
+      "step": 113
+    },
+    {
+      "epoch": 0.37530864197530867,
+      "grad_norm": 1.2480144500732422,
+      "learning_rate": 6.387959866220736e-06,
+      "loss": 0.2125,
+      "step": 114
+    },
+    {
+      "epoch": 0.3786008230452675,
+      "grad_norm": 1.0772844552993774,
+      "learning_rate": 6.354515050167225e-06,
+      "loss": 0.2212,
+      "step": 115
+    },
+    {
+      "epoch": 0.38189300411522636,
+      "grad_norm": 1.4869307279586792,
+      "learning_rate": 6.321070234113713e-06,
+      "loss": 0.2464,
+      "step": 116
+    },
+    {
+      "epoch": 0.3851851851851852,
+      "grad_norm": 0.9682499766349792,
+      "learning_rate": 6.287625418060201e-06,
+      "loss": 0.1874,
+      "step": 117
+    },
+    {
+      "epoch": 0.388477366255144,
+      "grad_norm": 1.9302343130111694,
+      "learning_rate": 6.25418060200669e-06,
+      "loss": 0.2241,
+      "step": 118
+    },
+    {
+      "epoch": 0.3917695473251029,
+      "grad_norm": 1.3866771459579468,
+      "learning_rate": 6.220735785953178e-06,
+      "loss": 0.2276,
+      "step": 119
+    },
+    {
+      "epoch": 0.3950617283950617,
+      "grad_norm": 1.2504560947418213,
+      "learning_rate": 6.1872909698996665e-06,
+      "loss": 0.1888,
+      "step": 120
+    },
+    {
+      "epoch": 0.3983539094650206,
+      "grad_norm": 0.8729090094566345,
+      "learning_rate": 6.153846153846155e-06,
+      "loss": 0.1164,
+      "step": 121
+    },
+    {
+      "epoch": 0.4016460905349794,
+      "grad_norm": 1.3598899841308594,
+      "learning_rate": 6.120401337792643e-06,
+      "loss": 0.268,
+      "step": 122
+    },
+    {
+      "epoch": 0.4049382716049383,
+      "grad_norm": 1.0118461847305298,
+      "learning_rate": 6.086956521739132e-06,
+      "loss": 0.1658,
+      "step": 123
+    },
+    {
+      "epoch": 0.4082304526748971,
+      "grad_norm": 1.2561231851577759,
+      "learning_rate": 6.05351170568562e-06,
+      "loss": 0.1619,
+      "step": 124
+    },
+    {
+      "epoch": 0.411522633744856,
+      "grad_norm": 1.8596264123916626,
+      "learning_rate": 6.020066889632108e-06,
+      "loss": 0.3384,
+      "step": 125
+    },
+    {
+      "epoch": 0.4148148148148148,
+      "grad_norm": 1.094607949256897,
+      "learning_rate": 5.986622073578597e-06,
+      "loss": 0.1289,
+      "step": 126
+    },
+    {
+      "epoch": 0.4181069958847737,
+      "grad_norm": 1.2113516330718994,
+      "learning_rate": 5.953177257525084e-06,
+      "loss": 0.2228,
+      "step": 127
+    },
+    {
+      "epoch": 0.4213991769547325,
+      "grad_norm": 1.285474419593811,
+      "learning_rate": 5.9197324414715726e-06,
+      "loss": 0.217,
+      "step": 128
+    },
+    {
+      "epoch": 0.4246913580246914,
+      "grad_norm": 1.0305473804473877,
+      "learning_rate": 5.886287625418061e-06,
+      "loss": 0.1431,
+      "step": 129
+    },
+    {
+      "epoch": 0.4279835390946502,
+      "grad_norm": 1.2470332384109497,
+      "learning_rate": 5.852842809364549e-06,
+      "loss": 0.2083,
+      "step": 130
+    },
+    {
+      "epoch": 0.4312757201646091,
+      "grad_norm": 1.0445364713668823,
+      "learning_rate": 5.819397993311037e-06,
+      "loss": 0.2121,
+      "step": 131
+    },
+    {
+      "epoch": 0.4345679012345679,
+      "grad_norm": 1.2339346408843994,
+      "learning_rate": 5.785953177257525e-06,
+      "loss": 0.2085,
+      "step": 132
+    },
+    {
+      "epoch": 0.4378600823045268,
+      "grad_norm": 0.7741423845291138,
+      "learning_rate": 5.7525083612040135e-06,
+      "loss": 0.0984,
+      "step": 133
+    },
+    {
+      "epoch": 0.4411522633744856,
+      "grad_norm": 1.237925410270691,
+      "learning_rate": 5.719063545150502e-06,
+      "loss": 0.1901,
+      "step": 134
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.3307265043258667,
+      "learning_rate": 5.68561872909699e-06,
+      "loss": 0.147,
+      "step": 135
+    },
+    {
+      "epoch": 0.4477366255144033,
+      "grad_norm": 1.5230034589767456,
+      "learning_rate": 5.652173913043479e-06,
+      "loss": 0.2344,
+      "step": 136
+    },
+    {
+      "epoch": 0.4510288065843621,
+      "grad_norm": 1.0349246263504028,
+      "learning_rate": 5.618729096989967e-06,
+      "loss": 0.1341,
+      "step": 137
+    },
+    {
+      "epoch": 0.454320987654321,
+      "grad_norm": 1.2906243801116943,
+      "learning_rate": 5.585284280936455e-06,
+      "loss": 0.1609,
+      "step": 138
+    },
+    {
+      "epoch": 0.4576131687242798,
+      "grad_norm": 1.155429720878601,
+      "learning_rate": 5.551839464882943e-06,
+      "loss": 0.1391,
+      "step": 139
+    },
+    {
+      "epoch": 0.4609053497942387,
+      "grad_norm": 1.1577730178833008,
+      "learning_rate": 5.518394648829431e-06,
+      "loss": 0.1877,
+      "step": 140
+    },
+    {
+      "epoch": 0.4641975308641975,
+      "grad_norm": 1.194811463356018,
+      "learning_rate": 5.48494983277592e-06,
+      "loss": 0.1616,
+      "step": 141
+    },
+    {
+      "epoch": 0.4674897119341564,
+      "grad_norm": 1.3541065454483032,
+      "learning_rate": 5.451505016722408e-06,
+      "loss": 0.2122,
+      "step": 142
+    },
+    {
+      "epoch": 0.4707818930041152,
+      "grad_norm": 1.3565137386322021,
+      "learning_rate": 5.418060200668896e-06,
+      "loss": 0.1949,
+      "step": 143
+    },
+    {
+      "epoch": 0.4740740740740741,
+      "grad_norm": 1.089268684387207,
+      "learning_rate": 5.384615384615385e-06,
+      "loss": 0.1685,
+      "step": 144
+    },
+    {
+      "epoch": 0.4773662551440329,
+      "grad_norm": 1.4452251195907593,
+      "learning_rate": 5.351170568561873e-06,
+      "loss": 0.3263,
+      "step": 145
+    },
+    {
+      "epoch": 0.4806584362139918,
+      "grad_norm": 0.8950945138931274,
+      "learning_rate": 5.317725752508361e-06,
+      "loss": 0.1,
+      "step": 146
+    },
+    {
+      "epoch": 0.4839506172839506,
+      "grad_norm": 1.0361438989639282,
+      "learning_rate": 5.28428093645485e-06,
+      "loss": 0.1543,
+      "step": 147
+    },
+    {
+      "epoch": 0.4872427983539095,
+      "grad_norm": 0.9762063026428223,
+      "learning_rate": 5.250836120401338e-06,
+      "loss": 0.1633,
+      "step": 148
+    },
+    {
+      "epoch": 0.4905349794238683,
+      "grad_norm": 1.5362917184829712,
+      "learning_rate": 5.2173913043478265e-06,
+      "loss": 0.2806,
+      "step": 149
+    },
+    {
+      "epoch": 0.49382716049382713,
+      "grad_norm": 1.0566084384918213,
+      "learning_rate": 5.183946488294315e-06,
+      "loss": 0.1133,
+      "step": 150
+    },
+    {
+      "epoch": 0.49382716049382713,
+      "eval_loss": 0.22752724587917328,
+      "eval_runtime": 20.1975,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 13.368,
+      "step": 150
+    },
+    {
+      "epoch": 0.497119341563786,
+      "grad_norm": 1.178045392036438,
+      "learning_rate": 5.150501672240803e-06,
+      "loss": 0.1659,
+      "step": 151
+    },
+    {
+      "epoch": 0.5004115226337449,
+      "grad_norm": 1.1872235536575317,
+      "learning_rate": 5.1170568561872916e-06,
+      "loss": 0.2207,
+      "step": 152
+    },
+    {
+      "epoch": 0.5037037037037037,
+      "grad_norm": 1.1316269636154175,
+      "learning_rate": 5.08361204013378e-06,
+      "loss": 0.2055,
+      "step": 153
+    },
+    {
+      "epoch": 0.5069958847736625,
+      "grad_norm": 1.1316063404083252,
+      "learning_rate": 5.050167224080268e-06,
+      "loss": 0.1819,
+      "step": 154
+    },
+    {
+      "epoch": 0.5102880658436214,
+      "grad_norm": 1.0036729574203491,
+      "learning_rate": 5.016722408026757e-06,
+      "loss": 0.1675,
+      "step": 155
+    },
+    {
+      "epoch": 0.5135802469135803,
+      "grad_norm": 1.0103334188461304,
+      "learning_rate": 4.983277591973244e-06,
+      "loss": 0.1262,
+      "step": 156
+    },
+    {
+      "epoch": 0.516872427983539,
+      "grad_norm": 1.1434346437454224,
+      "learning_rate": 4.9498327759197325e-06,
+      "loss": 0.2188,
+      "step": 157
+    },
+    {
+      "epoch": 0.5201646090534979,
+      "grad_norm": 1.0806591510772705,
+      "learning_rate": 4.916387959866221e-06,
+      "loss": 0.1634,
+      "step": 158
+    },
+    {
+      "epoch": 0.5234567901234568,
+      "grad_norm": 0.8558173179626465,
+      "learning_rate": 4.882943143812709e-06,
+      "loss": 0.1089,
+      "step": 159
+    },
+    {
+      "epoch": 0.5267489711934157,
+      "grad_norm": 1.1523737907409668,
+      "learning_rate": 4.849498327759198e-06,
+      "loss": 0.1676,
+      "step": 160
+    },
+    {
+      "epoch": 0.5300411522633744,
+      "grad_norm": 1.069319486618042,
+      "learning_rate": 4.816053511705686e-06,
+      "loss": 0.1795,
+      "step": 161
+    },
+    {
+      "epoch": 0.5333333333333333,
+      "grad_norm": 1.1196931600570679,
+      "learning_rate": 4.782608695652174e-06,
+      "loss": 0.2341,
+      "step": 162
+    },
+    {
+      "epoch": 0.5366255144032922,
+      "grad_norm": 1.1405786275863647,
+      "learning_rate": 4.749163879598663e-06,
+      "loss": 0.1515,
+      "step": 163
+    },
+    {
+      "epoch": 0.5399176954732511,
+      "grad_norm": 1.1543610095977783,
+      "learning_rate": 4.715719063545151e-06,
+      "loss": 0.2153,
+      "step": 164
+    },
+    {
+      "epoch": 0.5432098765432098,
+      "grad_norm": 1.1050238609313965,
+      "learning_rate": 4.6822742474916394e-06,
+      "loss": 0.1611,
+      "step": 165
+    },
+    {
+      "epoch": 0.5465020576131687,
+      "grad_norm": 1.2301816940307617,
+      "learning_rate": 4.648829431438128e-06,
+      "loss": 0.16,
+      "step": 166
+    },
+    {
+      "epoch": 0.5497942386831276,
+      "grad_norm": 1.2778856754302979,
+      "learning_rate": 4.615384615384616e-06,
+      "loss": 0.2035,
+      "step": 167
+    },
+    {
+      "epoch": 0.5530864197530864,
+      "grad_norm": 0.8663844466209412,
+      "learning_rate": 4.581939799331104e-06,
+      "loss": 0.1446,
+      "step": 168
+    },
+    {
+      "epoch": 0.5563786008230452,
+      "grad_norm": 1.1689271926879883,
+      "learning_rate": 4.548494983277592e-06,
+      "loss": 0.1997,
+      "step": 169
+    },
+    {
+      "epoch": 0.5596707818930041,
+      "grad_norm": 2.0356605052948,
+      "learning_rate": 4.51505016722408e-06,
+      "loss": 0.3198,
+      "step": 170
+    },
+    {
+      "epoch": 0.562962962962963,
+      "grad_norm": 0.8193195462226868,
+      "learning_rate": 4.481605351170569e-06,
+      "loss": 0.0912,
+      "step": 171
+    },
+    {
+      "epoch": 0.5662551440329218,
+      "grad_norm": 1.179722785949707,
+      "learning_rate": 4.448160535117057e-06,
+      "loss": 0.113,
+      "step": 172
+    },
+    {
+      "epoch": 0.5695473251028806,
+      "grad_norm": 1.7245903015136719,
+      "learning_rate": 4.4147157190635455e-06,
+      "loss": 0.3279,
+      "step": 173
+    },
+    {
+      "epoch": 0.5728395061728395,
+      "grad_norm": 1.0297194719314575,
+      "learning_rate": 4.381270903010034e-06,
+      "loss": 0.1475,
+      "step": 174
+    },
+    {
+      "epoch": 0.5761316872427984,
+      "grad_norm": 1.527592658996582,
+      "learning_rate": 4.347826086956522e-06,
+      "loss": 0.1731,
+      "step": 175
+    },
+    {
+      "epoch": 0.5794238683127572,
+      "grad_norm": 1.31918203830719,
+      "learning_rate": 4.3143812709030106e-06,
+      "loss": 0.2278,
+      "step": 176
+    },
+    {
+      "epoch": 0.582716049382716,
+      "grad_norm": 1.2321281433105469,
+      "learning_rate": 4.280936454849499e-06,
+      "loss": 0.1816,
+      "step": 177
+    },
+    {
+      "epoch": 0.5860082304526749,
+      "grad_norm": 1.1424757242202759,
+      "learning_rate": 4.247491638795987e-06,
+      "loss": 0.1435,
+      "step": 178
+    },
+    {
+      "epoch": 0.5893004115226338,
+      "grad_norm": 0.8312201499938965,
+      "learning_rate": 4.214046822742475e-06,
+      "loss": 0.0953,
+      "step": 179
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "grad_norm": 1.071226716041565,
+      "learning_rate": 4.180602006688963e-06,
+      "loss": 0.2014,
+      "step": 180
+    },
+    {
+      "epoch": 0.5958847736625514,
+      "grad_norm": 1.217869520187378,
+      "learning_rate": 4.1471571906354515e-06,
+      "loss": 0.1701,
+      "step": 181
+    },
+    {
+      "epoch": 0.5991769547325103,
+      "grad_norm": 1.2251555919647217,
+      "learning_rate": 4.11371237458194e-06,
+      "loss": 0.1529,
+      "step": 182
+    },
+    {
+      "epoch": 0.6024691358024692,
+      "grad_norm": 1.8037480115890503,
+      "learning_rate": 4.080267558528428e-06,
+      "loss": 0.2995,
+      "step": 183
+    },
+    {
+      "epoch": 0.605761316872428,
+      "grad_norm": 1.0678819417953491,
+      "learning_rate": 4.046822742474917e-06,
+      "loss": 0.1691,
+      "step": 184
+    },
+    {
+      "epoch": 0.6090534979423868,
+      "grad_norm": 1.264555811882019,
+      "learning_rate": 4.013377926421405e-06,
+      "loss": 0.1985,
+      "step": 185
+    },
+    {
+      "epoch": 0.6123456790123457,
+      "grad_norm": 1.1602282524108887,
+      "learning_rate": 3.979933110367893e-06,
+      "loss": 0.2087,
+      "step": 186
+    },
+    {
+      "epoch": 0.6156378600823045,
+      "grad_norm": 1.1501871347427368,
+      "learning_rate": 3.946488294314382e-06,
+      "loss": 0.1374,
+      "step": 187
+    },
+    {
+      "epoch": 0.6189300411522634,
+      "grad_norm": 1.1230063438415527,
+      "learning_rate": 3.91304347826087e-06,
+      "loss": 0.1872,
+      "step": 188
+    },
+    {
+      "epoch": 0.6222222222222222,
+      "grad_norm": 1.1128565073013306,
+      "learning_rate": 3.8795986622073584e-06,
+      "loss": 0.1984,
+      "step": 189
+    },
+    {
+      "epoch": 0.6255144032921811,
+      "grad_norm": 1.3014814853668213,
+      "learning_rate": 3.846153846153847e-06,
+      "loss": 0.1629,
+      "step": 190
+    },
+    {
+      "epoch": 0.6288065843621399,
+      "grad_norm": 1.1544214487075806,
+      "learning_rate": 3.812709030100335e-06,
+      "loss": 0.212,
+      "step": 191
+    },
+    {
+      "epoch": 0.6320987654320988,
+      "grad_norm": 0.9426103830337524,
+      "learning_rate": 3.7792642140468235e-06,
+      "loss": 0.1384,
+      "step": 192
+    },
+    {
+      "epoch": 0.6353909465020576,
+      "grad_norm": 1.261274814605713,
+      "learning_rate": 3.745819397993311e-06,
+      "loss": 0.1159,
+      "step": 193
+    },
+    {
+      "epoch": 0.6386831275720165,
+      "grad_norm": 1.39129638671875,
+      "learning_rate": 3.7123745819397994e-06,
+      "loss": 0.1685,
+      "step": 194
+    },
+    {
+      "epoch": 0.6419753086419753,
+      "grad_norm": 1.3119721412658691,
+      "learning_rate": 3.6789297658862878e-06,
+      "loss": 0.2694,
+      "step": 195
+    },
+    {
+      "epoch": 0.6452674897119342,
+      "grad_norm": 1.122768759727478,
+      "learning_rate": 3.645484949832776e-06,
+      "loss": 0.1752,
+      "step": 196
+    },
+    {
+      "epoch": 0.648559670781893,
+      "grad_norm": 1.369023323059082,
+      "learning_rate": 3.6120401337792645e-06,
+      "loss": 0.2265,
+      "step": 197
+    },
+    {
+      "epoch": 0.6518518518518519,
+      "grad_norm": 1.1043347120285034,
+      "learning_rate": 3.578595317725753e-06,
+      "loss": 0.2232,
+      "step": 198
+    },
+    {
+      "epoch": 0.6551440329218107,
+      "grad_norm": 0.9642878174781799,
+      "learning_rate": 3.5451505016722408e-06,
+      "loss": 0.1767,
+      "step": 199
+    },
+    {
+      "epoch": 0.6584362139917695,
+      "grad_norm": 1.0870318412780762,
+      "learning_rate": 3.511705685618729e-06,
+      "loss": 0.1908,
+      "step": 200
+    },
+    {
+      "epoch": 0.6584362139917695,
+      "eval_loss": 0.2218295931816101,
+      "eval_runtime": 20.1388,
+      "eval_samples_per_second": 26.814,
+      "eval_steps_per_second": 13.407,
+      "step": 200
+    },
+    {
+      "epoch": 0.6617283950617284,
+      "grad_norm": 0.9222021698951721,
+      "learning_rate": 3.4782608695652175e-06,
+      "loss": 0.1448,
+      "step": 201
+    },
+    {
+      "epoch": 0.6650205761316872,
+      "grad_norm": 1.0164117813110352,
+      "learning_rate": 3.444816053511706e-06,
+      "loss": 0.153,
+      "step": 202
+    },
+    {
+      "epoch": 0.6683127572016461,
+      "grad_norm": 1.217971682548523,
+      "learning_rate": 3.4113712374581942e-06,
+      "loss": 0.1528,
+      "step": 203
+    },
+    {
+      "epoch": 0.671604938271605,
+      "grad_norm": 1.0905036926269531,
+      "learning_rate": 3.3779264214046826e-06,
+      "loss": 0.2251,
+      "step": 204
+    },
+    {
+      "epoch": 0.6748971193415638,
+      "grad_norm": 1.2327961921691895,
+      "learning_rate": 3.344481605351171e-06,
+      "loss": 0.1973,
+      "step": 205
+    },
+    {
+      "epoch": 0.6781893004115226,
+      "grad_norm": 0.8670262098312378,
+      "learning_rate": 3.3110367892976593e-06,
+      "loss": 0.1234,
+      "step": 206
+    },
+    {
+      "epoch": 0.6814814814814815,
+      "grad_norm": 1.0915966033935547,
+      "learning_rate": 3.2775919732441473e-06,
+      "loss": 0.1821,
+      "step": 207
+    },
+    {
+      "epoch": 0.6847736625514403,
+      "grad_norm": 0.9655203223228455,
+      "learning_rate": 3.2441471571906356e-06,
+      "loss": 0.145,
+      "step": 208
+    },
+    {
+      "epoch": 0.6880658436213992,
+      "grad_norm": 1.004298448562622,
+      "learning_rate": 3.210702341137124e-06,
+      "loss": 0.167,
+      "step": 209
+    },
+    {
+      "epoch": 0.691358024691358,
+      "grad_norm": 1.0791229009628296,
+      "learning_rate": 3.1772575250836123e-06,
+      "loss": 0.1421,
+      "step": 210
+    },
+    {
+      "epoch": 0.6946502057613169,
+      "grad_norm": 1.015527606010437,
+      "learning_rate": 3.1438127090301007e-06,
+      "loss": 0.1414,
+      "step": 211
+    },
+    {
+      "epoch": 0.6979423868312757,
+      "grad_norm": 1.3114148378372192,
+      "learning_rate": 3.110367892976589e-06,
+      "loss": 0.1937,
+      "step": 212
+    },
+    {
+      "epoch": 0.7012345679012346,
+      "grad_norm": 1.076414942741394,
+      "learning_rate": 3.0769230769230774e-06,
+      "loss": 0.1639,
+      "step": 213
+    },
+    {
+      "epoch": 0.7045267489711934,
+      "grad_norm": 1.2151762247085571,
+      "learning_rate": 3.043478260869566e-06,
+      "loss": 0.2207,
+      "step": 214
+    },
+    {
+      "epoch": 0.7078189300411523,
+      "grad_norm": 0.9043106436729431,
+      "learning_rate": 3.010033444816054e-06,
+      "loss": 0.1082,
+      "step": 215
+    },
+    {
+      "epoch": 0.7111111111111111,
+      "grad_norm": 1.1242438554763794,
+      "learning_rate": 2.976588628762542e-06,
+      "loss": 0.1559,
+      "step": 216
+    },
+    {
+      "epoch": 0.7144032921810699,
+      "grad_norm": 0.7654614448547363,
+      "learning_rate": 2.9431438127090305e-06,
+      "loss": 0.0792,
+      "step": 217
+    },
+    {
+      "epoch": 0.7176954732510288,
+      "grad_norm": 1.1293987035751343,
+      "learning_rate": 2.9096989966555184e-06,
+      "loss": 0.1881,
+      "step": 218
+    },
+    {
+      "epoch": 0.7209876543209877,
+      "grad_norm": 1.1171600818634033,
+      "learning_rate": 2.8762541806020068e-06,
+      "loss": 0.1815,
+      "step": 219
+    },
+    {
+      "epoch": 0.7242798353909465,
+      "grad_norm": 1.091762661933899,
+      "learning_rate": 2.842809364548495e-06,
+      "loss": 0.1778,
+      "step": 220
+    },
+    {
+      "epoch": 0.7275720164609053,
+      "grad_norm": 1.3154752254486084,
+      "learning_rate": 2.8093645484949835e-06,
+      "loss": 0.1756,
+      "step": 221
+    },
+    {
+      "epoch": 0.7308641975308642,
+      "grad_norm": 0.9100964665412903,
+      "learning_rate": 2.7759197324414714e-06,
+      "loss": 0.1013,
+      "step": 222
+    },
+    {
+      "epoch": 0.7341563786008231,
+      "grad_norm": 1.0970630645751953,
+      "learning_rate": 2.74247491638796e-06,
+      "loss": 0.1617,
+      "step": 223
+    },
+    {
+      "epoch": 0.7374485596707819,
+      "grad_norm": 0.9748005867004395,
+      "learning_rate": 2.709030100334448e-06,
+      "loss": 0.1405,
+      "step": 224
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 1.1842244863510132,
+      "learning_rate": 2.6755852842809365e-06,
+      "loss": 0.1881,
+      "step": 225
+    },
+    {
+      "epoch": 0.7440329218106996,
+      "grad_norm": 1.2400459051132202,
+      "learning_rate": 2.642140468227425e-06,
+      "loss": 0.1856,
+      "step": 226
+    },
+    {
+      "epoch": 0.7473251028806585,
+      "grad_norm": 0.8171914219856262,
+      "learning_rate": 2.6086956521739132e-06,
+      "loss": 0.0819,
+      "step": 227
+    },
+    {
+      "epoch": 0.7506172839506173,
+      "grad_norm": 1.512604832649231,
+      "learning_rate": 2.5752508361204016e-06,
+      "loss": 0.2865,
+      "step": 228
+    },
+    {
+      "epoch": 0.7539094650205761,
+      "grad_norm": 1.378312587738037,
+      "learning_rate": 2.54180602006689e-06,
+      "loss": 0.1725,
+      "step": 229
+    },
+    {
+      "epoch": 0.757201646090535,
+      "grad_norm": 1.0168036222457886,
+      "learning_rate": 2.5083612040133783e-06,
+      "loss": 0.1755,
+      "step": 230
+    },
+    {
+      "epoch": 0.7604938271604939,
+      "grad_norm": 1.290043592453003,
+      "learning_rate": 2.4749163879598663e-06,
+      "loss": 0.22,
+      "step": 231
+    },
+    {
+      "epoch": 0.7637860082304527,
+      "grad_norm": 1.371999979019165,
+      "learning_rate": 2.4414715719063546e-06,
+      "loss": 0.2688,
+      "step": 232
+    },
+    {
+      "epoch": 0.7670781893004115,
+      "grad_norm": 1.280881404876709,
+      "learning_rate": 2.408026755852843e-06,
+      "loss": 0.1796,
+      "step": 233
+    },
+    {
+      "epoch": 0.7703703703703704,
+      "grad_norm": 1.0171372890472412,
+      "learning_rate": 2.3745819397993314e-06,
+      "loss": 0.1535,
+      "step": 234
+    },
+    {
+      "epoch": 0.7736625514403292,
+      "grad_norm": 0.9772494435310364,
+      "learning_rate": 2.3411371237458197e-06,
+      "loss": 0.1508,
+      "step": 235
+    },
+    {
+      "epoch": 0.776954732510288,
+      "grad_norm": 1.149998664855957,
+      "learning_rate": 2.307692307692308e-06,
+      "loss": 0.2154,
+      "step": 236
+    },
+    {
+      "epoch": 0.7802469135802469,
+      "grad_norm": 1.3714057207107544,
+      "learning_rate": 2.274247491638796e-06,
+      "loss": 0.2856,
+      "step": 237
+    },
+    {
+      "epoch": 0.7835390946502058,
+      "grad_norm": 1.485761046409607,
+      "learning_rate": 2.2408026755852844e-06,
+      "loss": 0.3031,
+      "step": 238
+    },
+    {
+      "epoch": 0.7868312757201646,
+      "grad_norm": 1.1520341634750366,
+      "learning_rate": 2.2073578595317727e-06,
+      "loss": 0.1835,
+      "step": 239
+    },
+    {
+      "epoch": 0.7901234567901234,
+      "grad_norm": 1.3876756429672241,
+      "learning_rate": 2.173913043478261e-06,
+      "loss": 0.2038,
+      "step": 240
+    },
+    {
+      "epoch": 0.7934156378600823,
+      "grad_norm": 0.9016226530075073,
+      "learning_rate": 2.1404682274247495e-06,
+      "loss": 0.1042,
+      "step": 241
+    },
+    {
+      "epoch": 0.7967078189300412,
+      "grad_norm": 1.3661850690841675,
+      "learning_rate": 2.1070234113712374e-06,
+      "loss": 0.1934,
+      "step": 242
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 1.5856467485427856,
+      "learning_rate": 2.0735785953177258e-06,
+      "loss": 0.2653,
+      "step": 243
+    },
+    {
+      "epoch": 0.8032921810699588,
+      "grad_norm": 2.052778959274292,
+      "learning_rate": 2.040133779264214e-06,
+      "loss": 0.1748,
+      "step": 244
+    },
+    {
+      "epoch": 0.8065843621399177,
+      "grad_norm": 1.6269681453704834,
+      "learning_rate": 2.0066889632107025e-06,
+      "loss": 0.2878,
+      "step": 245
+    },
+    {
+      "epoch": 0.8098765432098766,
+      "grad_norm": 1.1611472368240356,
+      "learning_rate": 1.973244147157191e-06,
+      "loss": 0.1783,
+      "step": 246
+    },
+    {
+      "epoch": 0.8131687242798354,
+      "grad_norm": 1.4677059650421143,
+      "learning_rate": 1.9397993311036792e-06,
+      "loss": 0.2698,
+      "step": 247
+    },
+    {
+      "epoch": 0.8164609053497942,
+      "grad_norm": 1.347899079322815,
+      "learning_rate": 1.9063545150501676e-06,
+      "loss": 0.217,
+      "step": 248
+    },
+    {
+      "epoch": 0.8197530864197531,
+      "grad_norm": 0.8131496906280518,
+      "learning_rate": 1.8729096989966555e-06,
+      "loss": 0.0778,
+      "step": 249
+    },
+    {
+      "epoch": 0.823045267489712,
+      "grad_norm": 1.221699833869934,
+      "learning_rate": 1.8394648829431439e-06,
+      "loss": 0.1853,
+      "step": 250
+    },
+    {
+      "epoch": 0.823045267489712,
+      "eval_loss": 0.2179623693227768,
+      "eval_runtime": 20.14,
+      "eval_samples_per_second": 26.812,
+      "eval_steps_per_second": 13.406,
+      "step": 250
+    },
+    {
+      "epoch": 0.8263374485596707,
+      "grad_norm": 1.3362127542495728,
+      "learning_rate": 1.8060200668896322e-06,
+      "loss": 0.248,
+      "step": 251
+    },
+    {
+      "epoch": 0.8296296296296296,
+      "grad_norm": 1.2720140218734741,
+      "learning_rate": 1.7725752508361204e-06,
+      "loss": 0.2058,
+      "step": 252
+    },
+    {
+      "epoch": 0.8329218106995885,
+      "grad_norm": 1.0819056034088135,
+      "learning_rate": 1.7391304347826088e-06,
+      "loss": 0.1347,
+      "step": 253
+    },
+    {
+      "epoch": 0.8362139917695474,
+      "grad_norm": 1.6707384586334229,
+      "learning_rate": 1.7056856187290971e-06,
+      "loss": 0.2105,
+      "step": 254
+    },
+    {
+      "epoch": 0.8395061728395061,
+      "grad_norm": 1.070344090461731,
+      "learning_rate": 1.6722408026755855e-06,
+      "loss": 0.2051,
+      "step": 255
+    },
+    {
+      "epoch": 0.842798353909465,
+      "grad_norm": 1.5513736009597778,
+      "learning_rate": 1.6387959866220736e-06,
+      "loss": 0.2652,
+      "step": 256
+    },
+    {
+      "epoch": 0.8460905349794239,
+      "grad_norm": 0.9962535500526428,
+      "learning_rate": 1.605351170568562e-06,
+      "loss": 0.1588,
+      "step": 257
+    },
+    {
+      "epoch": 0.8493827160493828,
+      "grad_norm": 1.1429444551467896,
+      "learning_rate": 1.5719063545150504e-06,
+      "loss": 0.152,
+      "step": 258
+    },
+    {
+      "epoch": 0.8526748971193415,
+      "grad_norm": 1.0112603902816772,
+      "learning_rate": 1.5384615384615387e-06,
+      "loss": 0.1706,
+      "step": 259
+    },
+    {
+      "epoch": 0.8559670781893004,
+      "grad_norm": 1.1977404356002808,
+      "learning_rate": 1.505016722408027e-06,
+      "loss": 0.1781,
+      "step": 260
+    },
+    {
+      "epoch": 0.8592592592592593,
+      "grad_norm": 1.0919908285140991,
+      "learning_rate": 1.4715719063545152e-06,
+      "loss": 0.1433,
+      "step": 261
+    },
+    {
+      "epoch": 0.8625514403292182,
+      "grad_norm": 0.9938503503799438,
+      "learning_rate": 1.4381270903010034e-06,
+      "loss": 0.1375,
+      "step": 262
+    },
+    {
+      "epoch": 0.8658436213991769,
+      "grad_norm": 1.2814228534698486,
+      "learning_rate": 1.4046822742474917e-06,
+      "loss": 0.173,
+      "step": 263
+    },
+    {
+      "epoch": 0.8691358024691358,
+      "grad_norm": 1.1976473331451416,
+      "learning_rate": 1.37123745819398e-06,
+      "loss": 0.1807,
+      "step": 264
+    },
+    {
+      "epoch": 0.8724279835390947,
+      "grad_norm": 1.1962367296218872,
+      "learning_rate": 1.3377926421404683e-06,
+      "loss": 0.1823,
+      "step": 265
+    },
+    {
+      "epoch": 0.8757201646090536,
+      "grad_norm": 1.3826391696929932,
+      "learning_rate": 1.3043478260869566e-06,
+      "loss": 0.1871,
+      "step": 266
+    },
+    {
+      "epoch": 0.8790123456790123,
+      "grad_norm": 1.5050283670425415,
+      "learning_rate": 1.270903010033445e-06,
+      "loss": 0.2696,
+      "step": 267
+    },
+    {
+      "epoch": 0.8823045267489712,
+      "grad_norm": 1.0602251291275024,
+      "learning_rate": 1.2374581939799331e-06,
+      "loss": 0.1667,
+      "step": 268
+    },
+    {
+      "epoch": 0.8855967078189301,
+      "grad_norm": 1.2634330987930298,
+      "learning_rate": 1.2040133779264215e-06,
+      "loss": 0.1787,
+      "step": 269
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.529909372329712,
+      "learning_rate": 1.1705685618729099e-06,
+      "loss": 0.1901,
+      "step": 270
+    },
+    {
+      "epoch": 0.8921810699588477,
+      "grad_norm": 0.9806025624275208,
+      "learning_rate": 1.137123745819398e-06,
+      "loss": 0.1549,
+      "step": 271
+    },
+    {
+      "epoch": 0.8954732510288066,
+      "grad_norm": 1.1857572793960571,
+      "learning_rate": 1.1036789297658864e-06,
+      "loss": 0.1962,
+      "step": 272
+    },
+    {
+      "epoch": 0.8987654320987655,
+      "grad_norm": 1.510190725326538,
+      "learning_rate": 1.0702341137123747e-06,
+      "loss": 0.1577,
+      "step": 273
+    },
+    {
+      "epoch": 0.9020576131687242,
+      "grad_norm": 0.9441434144973755,
+      "learning_rate": 1.0367892976588629e-06,
+      "loss": 0.1057,
+      "step": 274
+    },
+    {
+      "epoch": 0.9053497942386831,
+      "grad_norm": 1.194615364074707,
+      "learning_rate": 1.0033444816053512e-06,
+      "loss": 0.1867,
+      "step": 275
+    },
+    {
+      "epoch": 0.908641975308642,
+      "grad_norm": 1.3393813371658325,
+      "learning_rate": 9.698996655518396e-07,
+      "loss": 0.203,
+      "step": 276
+    },
+    {
+      "epoch": 0.9119341563786009,
+      "grad_norm": 0.9781934022903442,
+      "learning_rate": 9.364548494983278e-07,
+      "loss": 0.1467,
+      "step": 277
+    },
+    {
+      "epoch": 0.9152263374485596,
+      "grad_norm": 1.4591021537780762,
+      "learning_rate": 9.030100334448161e-07,
+      "loss": 0.2241,
+      "step": 278
+    },
+    {
+      "epoch": 0.9185185185185185,
+      "grad_norm": 1.198244571685791,
+      "learning_rate": 8.695652173913044e-07,
+      "loss": 0.1557,
+      "step": 279
+    },
+    {
+      "epoch": 0.9218106995884774,
+      "grad_norm": 1.5261183977127075,
+      "learning_rate": 8.361204013377927e-07,
+      "loss": 0.2491,
+      "step": 280
+    },
+    {
+      "epoch": 0.9251028806584363,
+      "grad_norm": 1.4149837493896484,
+      "learning_rate": 8.02675585284281e-07,
+      "loss": 0.2155,
+      "step": 281
+    },
+    {
+      "epoch": 0.928395061728395,
+      "grad_norm": 1.1661046743392944,
+      "learning_rate": 7.692307692307694e-07,
+      "loss": 0.2183,
+      "step": 282
+    },
+    {
+      "epoch": 0.9316872427983539,
+      "grad_norm": 1.4822607040405273,
+      "learning_rate": 7.357859531772576e-07,
+      "loss": 0.1765,
+      "step": 283
+    },
+    {
+      "epoch": 0.9349794238683128,
+      "grad_norm": 0.9975975155830383,
+      "learning_rate": 7.023411371237459e-07,
+      "loss": 0.1129,
+      "step": 284
+    },
+    {
+      "epoch": 0.9382716049382716,
+      "grad_norm": 1.1125397682189941,
+      "learning_rate": 6.688963210702341e-07,
+      "loss": 0.1965,
+      "step": 285
+    },
+    {
+      "epoch": 0.9415637860082304,
+      "grad_norm": 1.3804467916488647,
+      "learning_rate": 6.354515050167225e-07,
+      "loss": 0.1594,
+      "step": 286
+    },
+    {
+      "epoch": 0.9448559670781893,
+      "grad_norm": 0.9886622428894043,
+      "learning_rate": 6.020066889632107e-07,
+      "loss": 0.1509,
+      "step": 287
+    },
+    {
+      "epoch": 0.9481481481481482,
+      "grad_norm": 1.1645519733428955,
+      "learning_rate": 5.68561872909699e-07,
+      "loss": 0.1871,
+      "step": 288
+    },
+    {
+      "epoch": 0.951440329218107,
+      "grad_norm": 0.9140220284461975,
+      "learning_rate": 5.351170568561874e-07,
+      "loss": 0.1471,
+      "step": 289
+    },
+    {
+      "epoch": 0.9547325102880658,
+      "grad_norm": 0.782092809677124,
+      "learning_rate": 5.016722408026756e-07,
+      "loss": 0.0831,
+      "step": 290
+    },
+    {
+      "epoch": 0.9580246913580247,
+      "grad_norm": 1.168330430984497,
+      "learning_rate": 4.682274247491639e-07,
+      "loss": 0.1713,
+      "step": 291
+    },
+    {
+      "epoch": 0.9613168724279836,
+      "grad_norm": 1.4891828298568726,
+      "learning_rate": 4.347826086956522e-07,
+      "loss": 0.2148,
+      "step": 292
+    },
+    {
+      "epoch": 0.9646090534979423,
+      "grad_norm": 1.2476691007614136,
+      "learning_rate": 4.013377926421405e-07,
+      "loss": 0.1475,
+      "step": 293
+    },
+    {
+      "epoch": 0.9679012345679012,
+      "grad_norm": 1.1673272848129272,
+      "learning_rate": 3.678929765886288e-07,
+      "loss": 0.2309,
+      "step": 294
+    },
+    {
+      "epoch": 0.9711934156378601,
+      "grad_norm": 1.1678235530853271,
+      "learning_rate": 3.3444816053511706e-07,
+      "loss": 0.176,
+      "step": 295
+    },
+    {
+      "epoch": 0.974485596707819,
+      "grad_norm": 1.061506748199463,
+      "learning_rate": 3.010033444816054e-07,
+      "loss": 0.1186,
+      "step": 296
+    },
+    {
+      "epoch": 0.9777777777777777,
+      "grad_norm": 0.9109651446342468,
+      "learning_rate": 2.675585284280937e-07,
+      "loss": 0.1308,
+      "step": 297
+    },
+    {
+      "epoch": 0.9810699588477366,
+      "grad_norm": 2.290402889251709,
+      "learning_rate": 2.3411371237458194e-07,
+      "loss": 0.4006,
+      "step": 298
+    },
+    {
+      "epoch": 0.9843621399176955,
+      "grad_norm": 1.1297715902328491,
+      "learning_rate": 2.0066889632107025e-07,
+      "loss": 0.1994,
+      "step": 299
+    },
+    {
+      "epoch": 0.9876543209876543,
+      "grad_norm": 1.2302424907684326,
+      "learning_rate": 1.6722408026755853e-07,
+      "loss": 0.1684,
+      "step": 300
+    },
+    {
+      "epoch": 0.9876543209876543,
+      "eval_loss": 0.2165330946445465,
+      "eval_runtime": 20.1905,
+      "eval_samples_per_second": 26.745,
+      "eval_steps_per_second": 13.373,
+      "step": 300
+    },
+    {
+      "epoch": 0.9909465020576131,
+      "grad_norm": 1.0293028354644775,
+      "learning_rate": 1.3377926421404684e-07,
+      "loss": 0.1268,
+      "step": 301
+    },
+    {
+      "epoch": 0.994238683127572,
+      "grad_norm": 0.9462326169013977,
+      "learning_rate": 1.0033444816053512e-07,
+      "loss": 0.1329,
+      "step": 302
+    },
+    {
+      "epoch": 0.9975308641975309,
+      "grad_norm": 1.3390073776245117,
+      "learning_rate": 6.688963210702342e-08,
+      "loss": 0.2647,
+      "step": 303
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 1.3430871963500977,
+      "learning_rate": 3.344481605351171e-08,
+      "loss": 0.1524,
+      "step": 304
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 304,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 10000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.334762911328051e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-304/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cb5038846c67b52fa7b8206423c50333cef29651d2daf20092d279512aef444
+size 6033