Upload folder using huggingface_hub

README.md

@@ -1,3 +1,202 @@
----
-license: apache-2.0
----
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:edfbb43ba9f8a339b92db1f1963eb57bf5e57a7b1f415ad5c0b8752d497c5a23
+size 35668592

checkpoint-123/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-123/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-123/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c78ca7fd5d71b674a3fd674fd228d6fbc46cedffff4b0c0aadcdb482c1e65858
+size 35668592

checkpoint-123/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dbe0cea0da3c0beef13327c8ab3a18a88d90b22cd7576f30f426e5ecafe717
+size 71379898

checkpoint-123/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0812bceec28a7a800dbade85da79768153f7586bb9ce87716e2bf7cc4be7eec
+size 14244

checkpoint-123/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:12651c3f5b5fbe89532cbed840df6c4bbfba47058ad4639230bd195dcc56d17c
+size 1064

checkpoint-123/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-123/tokenizer_config.json

@@ -0,0 +1,131 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": true,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "32000": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32002": {
+      "content": "<|placeholder1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32003": {
+      "content": "<|placeholder2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32004": {
+      "content": "<|placeholder3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32005": {
+      "content": "<|placeholder4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32006": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "<|placeholder5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "<|placeholder6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "legacy": false,
+  "model_max_length": 4096,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-123/trainer_state.json

@@ -0,0 +1,1878 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 4.96969696969697,
+  "eval_steps": 1,
+  "global_step": 123,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04040404040404041,
+      "grad_norm": 2.088834762573242,
+      "learning_rate": 8.333333333333333e-07,
+      "loss": 5.4688,
+      "step": 1
+    },
+    {
+      "epoch": 0.04040404040404041,
+      "eval_loss": 5.470384120941162,
+      "eval_runtime": 4.8066,
+      "eval_samples_per_second": 34.536,
+      "eval_steps_per_second": 0.624,
+      "step": 1
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "grad_norm": 2.0961971282958984,
+      "learning_rate": 1.6666666666666667e-06,
+      "loss": 5.4731,
+      "step": 2
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "eval_loss": 5.4689249992370605,
+      "eval_runtime": 4.8593,
+      "eval_samples_per_second": 34.161,
+      "eval_steps_per_second": 0.617,
+      "step": 2
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "grad_norm": 2.0940349102020264,
+      "learning_rate": 2.5e-06,
+      "loss": 5.4803,
+      "step": 3
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "eval_loss": 5.467825412750244,
+      "eval_runtime": 4.9081,
+      "eval_samples_per_second": 33.822,
+      "eval_steps_per_second": 0.611,
+      "step": 3
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "grad_norm": 2.095174551010132,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 5.5119,
+      "step": 4
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "eval_loss": 5.465516567230225,
+      "eval_runtime": 4.8615,
+      "eval_samples_per_second": 34.146,
+      "eval_steps_per_second": 0.617,
+      "step": 4
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "grad_norm": 2.1109118461608887,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 5.4662,
+      "step": 5
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "eval_loss": 5.4626851081848145,
+      "eval_runtime": 4.844,
+      "eval_samples_per_second": 34.27,
+      "eval_steps_per_second": 0.619,
+      "step": 5
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "grad_norm": 2.1030592918395996,
+      "learning_rate": 5e-06,
+      "loss": 5.4691,
+      "step": 6
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "eval_loss": 5.45806884765625,
+      "eval_runtime": 4.8388,
+      "eval_samples_per_second": 34.306,
+      "eval_steps_per_second": 0.62,
+      "step": 6
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "grad_norm": 2.1202447414398193,
+      "learning_rate": 5.833333333333334e-06,
+      "loss": 5.4851,
+      "step": 7
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "eval_loss": 5.449892520904541,
+      "eval_runtime": 4.8431,
+      "eval_samples_per_second": 34.276,
+      "eval_steps_per_second": 0.619,
+      "step": 7
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "grad_norm": 2.090963363647461,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 5.4228,
+      "step": 8
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "eval_loss": 5.442515850067139,
+      "eval_runtime": 4.8605,
+      "eval_samples_per_second": 34.153,
+      "eval_steps_per_second": 0.617,
+      "step": 8
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "grad_norm": 2.112323760986328,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 5.4715,
+      "step": 9
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "eval_loss": 5.4345855712890625,
+      "eval_runtime": 4.8796,
+      "eval_samples_per_second": 34.019,
+      "eval_steps_per_second": 0.615,
+      "step": 9
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "grad_norm": 2.1456995010375977,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 5.4211,
+      "step": 10
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "eval_loss": 5.424546718597412,
+      "eval_runtime": 4.9274,
+      "eval_samples_per_second": 33.689,
+      "eval_steps_per_second": 0.609,
+      "step": 10
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 2.1602349281311035,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 5.4301,
+      "step": 11
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 5.41035795211792,
+      "eval_runtime": 4.9819,
+      "eval_samples_per_second": 33.32,
+      "eval_steps_per_second": 0.602,
+      "step": 11
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "grad_norm": 2.1762421131134033,
+      "learning_rate": 1e-05,
+      "loss": 5.4158,
+      "step": 12
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "eval_loss": 5.400452613830566,
+      "eval_runtime": 5.0737,
+      "eval_samples_per_second": 32.718,
+      "eval_steps_per_second": 0.591,
+      "step": 12
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "grad_norm": 2.2415969371795654,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 5.3986,
+      "step": 13
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "eval_loss": 5.382190227508545,
+      "eval_runtime": 4.9555,
+      "eval_samples_per_second": 33.498,
+      "eval_steps_per_second": 0.605,
+      "step": 13
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "grad_norm": 2.2669475078582764,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 5.428,
+      "step": 14
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "eval_loss": 5.3634443283081055,
+      "eval_runtime": 4.9029,
+      "eval_samples_per_second": 33.857,
+      "eval_steps_per_second": 0.612,
+      "step": 14
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "grad_norm": 2.262896776199341,
+      "learning_rate": 1.25e-05,
+      "loss": 5.3741,
+      "step": 15
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "eval_loss": 5.347131729125977,
+      "eval_runtime": 4.8934,
+      "eval_samples_per_second": 33.923,
+      "eval_steps_per_second": 0.613,
+      "step": 15
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "grad_norm": 2.3363702297210693,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 5.4134,
+      "step": 16
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "eval_loss": 5.321357250213623,
+      "eval_runtime": 4.9027,
+      "eval_samples_per_second": 33.859,
+      "eval_steps_per_second": 0.612,
+      "step": 16
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "grad_norm": 2.3906755447387695,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 5.3305,
+      "step": 17
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "eval_loss": 5.292910099029541,
+      "eval_runtime": 4.9268,
+      "eval_samples_per_second": 33.693,
+      "eval_steps_per_second": 0.609,
+      "step": 17
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "grad_norm": 2.44368839263916,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 5.3406,
+      "step": 18
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "eval_loss": 5.257543563842773,
+      "eval_runtime": 4.9839,
+      "eval_samples_per_second": 33.308,
+      "eval_steps_per_second": 0.602,
+      "step": 18
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "grad_norm": 2.4937825202941895,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 5.3134,
+      "step": 19
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "eval_loss": 5.225679874420166,
+      "eval_runtime": 5.1205,
+      "eval_samples_per_second": 32.419,
+      "eval_steps_per_second": 0.586,
+      "step": 19
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "grad_norm": 2.547194480895996,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 5.2303,
+      "step": 20
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "eval_loss": 5.183907985687256,
+      "eval_runtime": 5.1208,
+      "eval_samples_per_second": 32.417,
+      "eval_steps_per_second": 0.586,
+      "step": 20
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "grad_norm": 2.5578083992004395,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 5.1255,
+      "step": 21
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "eval_loss": 5.136340141296387,
+      "eval_runtime": 5.1161,
+      "eval_samples_per_second": 32.446,
+      "eval_steps_per_second": 0.586,
+      "step": 21
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 2.6553032398223877,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 5.1376,
+      "step": 22
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 5.085525035858154,
+      "eval_runtime": 5.1496,
+      "eval_samples_per_second": 32.236,
+      "eval_steps_per_second": 0.583,
+      "step": 22
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "grad_norm": 2.733492374420166,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 5.1051,
+      "step": 23
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "eval_loss": 5.0193634033203125,
+      "eval_runtime": 5.0473,
+      "eval_samples_per_second": 32.889,
+      "eval_steps_per_second": 0.594,
+      "step": 23
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "grad_norm": 2.765582799911499,
+      "learning_rate": 2e-05,
+      "loss": 5.0082,
+      "step": 24
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "eval_loss": 4.950736999511719,
+      "eval_runtime": 4.989,
+      "eval_samples_per_second": 33.273,
+      "eval_steps_per_second": 0.601,
+      "step": 24
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "grad_norm": 2.8943891525268555,
+      "learning_rate": 1.990740740740741e-05,
+      "loss": 4.9736,
+      "step": 25
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "eval_loss": 4.87933349609375,
+      "eval_runtime": 4.991,
+      "eval_samples_per_second": 33.26,
+      "eval_steps_per_second": 0.601,
+      "step": 25
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "grad_norm": 2.9802019596099854,
+      "learning_rate": 1.9814814814814816e-05,
+      "loss": 4.8728,
+      "step": 26
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "eval_loss": 4.796956539154053,
+      "eval_runtime": 5.0224,
+      "eval_samples_per_second": 33.052,
+      "eval_steps_per_second": 0.597,
+      "step": 26
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "grad_norm": 3.068443536758423,
+      "learning_rate": 1.9722222222222224e-05,
+      "loss": 4.8132,
+      "step": 27
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "eval_loss": 4.7025275230407715,
+      "eval_runtime": 5.1226,
+      "eval_samples_per_second": 32.406,
+      "eval_steps_per_second": 0.586,
+      "step": 27
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "grad_norm": 3.2245917320251465,
+      "learning_rate": 1.962962962962963e-05,
+      "loss": 4.7175,
+      "step": 28
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "eval_loss": 4.607891082763672,
+      "eval_runtime": 5.0927,
+      "eval_samples_per_second": 32.596,
+      "eval_steps_per_second": 0.589,
+      "step": 28
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "grad_norm": 3.3136157989501953,
+      "learning_rate": 1.9537037037037038e-05,
+      "loss": 4.6429,
+      "step": 29
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "eval_loss": 4.510694980621338,
+      "eval_runtime": 5.0136,
+      "eval_samples_per_second": 33.11,
+      "eval_steps_per_second": 0.598,
+      "step": 29
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "grad_norm": 3.329773426055908,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 4.4686,
+      "step": 30
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "eval_loss": 4.406045913696289,
+      "eval_runtime": 4.9753,
+      "eval_samples_per_second": 33.365,
+      "eval_steps_per_second": 0.603,
+      "step": 30
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "grad_norm": 3.5462117195129395,
+      "learning_rate": 1.9351851851851853e-05,
+      "loss": 4.427,
+      "step": 31
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "eval_loss": 4.309689998626709,
+      "eval_runtime": 4.9974,
+      "eval_samples_per_second": 33.217,
+      "eval_steps_per_second": 0.6,
+      "step": 31
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "grad_norm": 3.6190805435180664,
+      "learning_rate": 1.925925925925926e-05,
+      "loss": 4.2899,
+      "step": 32
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "eval_loss": 4.215596675872803,
+      "eval_runtime": 5.0637,
+      "eval_samples_per_second": 32.782,
+      "eval_steps_per_second": 0.592,
+      "step": 32
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 3.698317289352417,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 4.1752,
+      "step": 33
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 4.106834411621094,
+      "eval_runtime": 5.1651,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 33
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "grad_norm": 3.4807214736938477,
+      "learning_rate": 1.9074074074074075e-05,
+      "loss": 4.1277,
+      "step": 34
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "eval_loss": 4.001583099365234,
+      "eval_runtime": 5.0567,
+      "eval_samples_per_second": 32.828,
+      "eval_steps_per_second": 0.593,
+      "step": 34
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "grad_norm": 3.1380534172058105,
+      "learning_rate": 1.8981481481481482e-05,
+      "loss": 4.0069,
+      "step": 35
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "eval_loss": 3.9105026721954346,
+      "eval_runtime": 5.07,
+      "eval_samples_per_second": 32.741,
+      "eval_steps_per_second": 0.592,
+      "step": 35
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "grad_norm": 2.9076974391937256,
+      "learning_rate": 1.888888888888889e-05,
+      "loss": 3.8911,
+      "step": 36
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "eval_loss": 3.838275194168091,
+      "eval_runtime": 5.1402,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 36
+    },
+    {
+      "epoch": 1.494949494949495,
+      "grad_norm": 2.4755232334136963,
+      "learning_rate": 1.8796296296296296e-05,
+      "loss": 3.8522,
+      "step": 37
+    },
+    {
+      "epoch": 1.494949494949495,
+      "eval_loss": 3.776526689529419,
+      "eval_runtime": 5.1768,
+      "eval_samples_per_second": 32.066,
+      "eval_steps_per_second": 0.58,
+      "step": 37
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "grad_norm": 2.1607069969177246,
+      "learning_rate": 1.8703703703703707e-05,
+      "loss": 3.7828,
+      "step": 38
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "eval_loss": 3.7257139682769775,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 38
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "grad_norm": 1.9251853227615356,
+      "learning_rate": 1.8611111111111114e-05,
+      "loss": 3.7511,
+      "step": 39
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "eval_loss": 3.680651903152466,
+      "eval_runtime": 5.139,
+      "eval_samples_per_second": 32.302,
+      "eval_steps_per_second": 0.584,
+      "step": 39
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "grad_norm": 1.8553980588912964,
+      "learning_rate": 1.851851851851852e-05,
+      "loss": 3.7108,
+      "step": 40
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "eval_loss": 3.633517026901245,
+      "eval_runtime": 5.1036,
+      "eval_samples_per_second": 32.526,
+      "eval_steps_per_second": 0.588,
+      "step": 40
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "grad_norm": 1.831110954284668,
+      "learning_rate": 1.8425925925925926e-05,
+      "loss": 3.6165,
+      "step": 41
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "eval_loss": 3.5877223014831543,
+      "eval_runtime": 5.1287,
+      "eval_samples_per_second": 32.367,
+      "eval_steps_per_second": 0.585,
+      "step": 41
+    },
+    {
+      "epoch": 1.696969696969697,
+      "grad_norm": 1.8035303354263306,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 3.5555,
+      "step": 42
+    },
+    {
+      "epoch": 1.696969696969697,
+      "eval_loss": 3.5403287410736084,
+      "eval_runtime": 5.1001,
+      "eval_samples_per_second": 32.549,
+      "eval_steps_per_second": 0.588,
+      "step": 42
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "grad_norm": 1.7953453063964844,
+      "learning_rate": 1.8240740740740744e-05,
+      "loss": 3.5437,
+      "step": 43
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "eval_loss": 3.492953300476074,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 43
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 1.7807087898254395,
+      "learning_rate": 1.814814814814815e-05,
+      "loss": 3.4676,
+      "step": 44
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 3.443131446838379,
+      "eval_runtime": 5.1066,
+      "eval_samples_per_second": 32.507,
+      "eval_steps_per_second": 0.587,
+      "step": 44
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 1.8204630613327026,
+      "learning_rate": 1.8055555555555558e-05,
+      "loss": 3.4517,
+      "step": 45
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "eval_loss": 3.396712303161621,
+      "eval_runtime": 5.1475,
+      "eval_samples_per_second": 32.249,
+      "eval_steps_per_second": 0.583,
+      "step": 45
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "grad_norm": 1.8122252225875854,
+      "learning_rate": 1.7962962962962965e-05,
+      "loss": 3.4189,
+      "step": 46
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "eval_loss": 3.3434314727783203,
+      "eval_runtime": 5.0828,
+      "eval_samples_per_second": 32.659,
+      "eval_steps_per_second": 0.59,
+      "step": 46
+    },
+    {
+      "epoch": 1.898989898989899,
+      "grad_norm": 1.8433269262313843,
+      "learning_rate": 1.7870370370370373e-05,
+      "loss": 3.3537,
+      "step": 47
+    },
+    {
+      "epoch": 1.898989898989899,
+      "eval_loss": 3.294283628463745,
+      "eval_runtime": 5.1195,
+      "eval_samples_per_second": 32.425,
+      "eval_steps_per_second": 0.586,
+      "step": 47
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "grad_norm": 1.833533763885498,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 3.3034,
+      "step": 48
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "eval_loss": 3.2412631511688232,
+      "eval_runtime": 5.1404,
+      "eval_samples_per_second": 32.293,
+      "eval_steps_per_second": 0.584,
+      "step": 48
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "grad_norm": 1.823169469833374,
+      "learning_rate": 1.7685185185185187e-05,
+      "loss": 3.2457,
+      "step": 49
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "eval_loss": 3.1890854835510254,
+      "eval_runtime": 5.1319,
+      "eval_samples_per_second": 32.347,
+      "eval_steps_per_second": 0.585,
+      "step": 49
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "grad_norm": 1.796790361404419,
+      "learning_rate": 1.7592592592592595e-05,
+      "loss": 3.1915,
+      "step": 50
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "eval_loss": 3.1396946907043457,
+      "eval_runtime": 5.1262,
+      "eval_samples_per_second": 32.383,
+      "eval_steps_per_second": 0.585,
+      "step": 50
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "grad_norm": 1.7133023738861084,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 3.1475,
+      "step": 51
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "eval_loss": 3.093197822570801,
+      "eval_runtime": 5.141,
+      "eval_samples_per_second": 32.29,
+      "eval_steps_per_second": 0.584,
+      "step": 51
+    },
+    {
+      "epoch": 2.101010101010101,
+      "grad_norm": 1.637424111366272,
+      "learning_rate": 1.740740740740741e-05,
+      "loss": 3.0912,
+      "step": 52
+    },
+    {
+      "epoch": 2.101010101010101,
+      "eval_loss": 3.0486488342285156,
+      "eval_runtime": 5.1578,
+      "eval_samples_per_second": 32.185,
+      "eval_steps_per_second": 0.582,
+      "step": 52
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "grad_norm": 1.5733733177185059,
+      "learning_rate": 1.7314814814814816e-05,
+      "loss": 3.0493,
+      "step": 53
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "eval_loss": 3.007200002670288,
+      "eval_runtime": 5.1715,
+      "eval_samples_per_second": 32.099,
+      "eval_steps_per_second": 0.58,
+      "step": 53
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "grad_norm": 1.579100251197815,
+      "learning_rate": 1.7222222222222224e-05,
+      "loss": 3.0007,
+      "step": 54
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "eval_loss": 2.9646313190460205,
+      "eval_runtime": 5.1737,
+      "eval_samples_per_second": 32.085,
+      "eval_steps_per_second": 0.58,
+      "step": 54
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.5558359622955322,
+      "learning_rate": 1.712962962962963e-05,
+      "loss": 2.955,
+      "step": 55
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 2.9225423336029053,
+      "eval_runtime": 5.1649,
+      "eval_samples_per_second": 32.14,
+      "eval_steps_per_second": 0.581,
+      "step": 55
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "grad_norm": 1.5695600509643555,
+      "learning_rate": 1.7037037037037038e-05,
+      "loss": 2.9242,
+      "step": 56
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "eval_loss": 2.883842706680298,
+      "eval_runtime": 5.1565,
+      "eval_samples_per_second": 32.192,
+      "eval_steps_per_second": 0.582,
+      "step": 56
+    },
+    {
+      "epoch": 2.303030303030303,
+      "grad_norm": 1.5856674909591675,
+      "learning_rate": 1.6944444444444446e-05,
+      "loss": 2.8626,
+      "step": 57
+    },
+    {
+      "epoch": 2.303030303030303,
+      "eval_loss": 2.8435447216033936,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 57
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "grad_norm": 1.5780515670776367,
+      "learning_rate": 1.6851851851851853e-05,
+      "loss": 2.8522,
+      "step": 58
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "eval_loss": 2.8028392791748047,
+      "eval_runtime": 5.1666,
+      "eval_samples_per_second": 32.129,
+      "eval_steps_per_second": 0.581,
+      "step": 58
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "grad_norm": 1.5944206714630127,
+      "learning_rate": 1.675925925925926e-05,
+      "loss": 2.8222,
+      "step": 59
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "eval_loss": 2.7607803344726562,
+      "eval_runtime": 5.1662,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 59
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "grad_norm": 1.6033095121383667,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.7291,
+      "step": 60
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "eval_loss": 2.7191247940063477,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 60
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "grad_norm": 1.621768593788147,
+      "learning_rate": 1.6574074074074075e-05,
+      "loss": 2.7046,
+      "step": 61
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "eval_loss": 2.6788933277130127,
+      "eval_runtime": 5.1728,
+      "eval_samples_per_second": 32.091,
+      "eval_steps_per_second": 0.58,
+      "step": 61
+    },
+    {
+      "epoch": 2.505050505050505,
+      "grad_norm": 1.6213834285736084,
+      "learning_rate": 1.6481481481481482e-05,
+      "loss": 2.6548,
+      "step": 62
+    },
+    {
+      "epoch": 2.505050505050505,
+      "eval_loss": 2.6340527534484863,
+      "eval_runtime": 5.1628,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 62
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "grad_norm": 1.607869267463684,
+      "learning_rate": 1.638888888888889e-05,
+      "loss": 2.6232,
+      "step": 63
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "eval_loss": 2.594001531600952,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 63
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "grad_norm": 1.583896279335022,
+      "learning_rate": 1.6296296296296297e-05,
+      "loss": 2.6068,
+      "step": 64
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "eval_loss": 2.5528736114501953,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 64
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "grad_norm": 1.5254372358322144,
+      "learning_rate": 1.6203703703703707e-05,
+      "loss": 2.5629,
+      "step": 65
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "eval_loss": 2.5128839015960693,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 65
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 1.4735966920852661,
+      "learning_rate": 1.6111111111111115e-05,
+      "loss": 2.523,
+      "step": 66
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 2.4759180545806885,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 66
+    },
+    {
+      "epoch": 2.707070707070707,
+      "grad_norm": 1.4135326147079468,
+      "learning_rate": 1.601851851851852e-05,
+      "loss": 2.5069,
+      "step": 67
+    },
+    {
+      "epoch": 2.707070707070707,
+      "eval_loss": 2.4382143020629883,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.274,
+      "eval_steps_per_second": 0.583,
+      "step": 67
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "grad_norm": 1.3859189748764038,
+      "learning_rate": 1.5925925925925926e-05,
+      "loss": 2.449,
+      "step": 68
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "eval_loss": 2.3999476432800293,
+      "eval_runtime": 5.0261,
+      "eval_samples_per_second": 33.028,
+      "eval_steps_per_second": 0.597,
+      "step": 68
+    },
+    {
+      "epoch": 2.787878787878788,
+      "grad_norm": 1.3678818941116333,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 2.3977,
+      "step": 69
+    },
+    {
+      "epoch": 2.787878787878788,
+      "eval_loss": 2.365516424179077,
+      "eval_runtime": 4.9945,
+      "eval_samples_per_second": 33.237,
+      "eval_steps_per_second": 0.601,
+      "step": 69
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "grad_norm": 1.372074842453003,
+      "learning_rate": 1.5740740740740744e-05,
+      "loss": 2.3556,
+      "step": 70
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "eval_loss": 2.3320348262786865,
+      "eval_runtime": 5.0457,
+      "eval_samples_per_second": 32.899,
+      "eval_steps_per_second": 0.595,
+      "step": 70
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "grad_norm": 1.355190634727478,
+      "learning_rate": 1.564814814814815e-05,
+      "loss": 2.3269,
+      "step": 71
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "eval_loss": 2.2980406284332275,
+      "eval_runtime": 5.1448,
+      "eval_samples_per_second": 32.265,
+      "eval_steps_per_second": 0.583,
+      "step": 71
+    },
+    {
+      "epoch": 2.909090909090909,
+      "grad_norm": 1.34852135181427,
+      "learning_rate": 1.555555555555556e-05,
+      "loss": 2.2962,
+      "step": 72
+    },
+    {
+      "epoch": 2.909090909090909,
+      "eval_loss": 2.262707233428955,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 72
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "grad_norm": 1.3559976816177368,
+      "learning_rate": 1.5462962962962966e-05,
+      "loss": 2.2534,
+      "step": 73
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "eval_loss": 2.2291126251220703,
+      "eval_runtime": 5.0794,
+      "eval_samples_per_second": 32.681,
+      "eval_steps_per_second": 0.591,
+      "step": 73
+    },
+    {
+      "epoch": 2.98989898989899,
+      "grad_norm": 1.3566159009933472,
+      "learning_rate": 1.537037037037037e-05,
+      "loss": 2.2061,
+      "step": 74
+    },
+    {
+      "epoch": 2.98989898989899,
+      "eval_loss": 2.193363904953003,
+      "eval_runtime": 5.0616,
+      "eval_samples_per_second": 32.796,
+      "eval_steps_per_second": 0.593,
+      "step": 74
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "grad_norm": 1.3466676473617554,
+      "learning_rate": 1.5277777777777777e-05,
+      "loss": 2.1815,
+      "step": 75
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "eval_loss": 2.159188985824585,
+      "eval_runtime": 5.0978,
+      "eval_samples_per_second": 32.563,
+      "eval_steps_per_second": 0.588,
+      "step": 75
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "grad_norm": 1.354913592338562,
+      "learning_rate": 1.5185185185185187e-05,
+      "loss": 2.1622,
+      "step": 76
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "eval_loss": 2.124568462371826,
+      "eval_runtime": 5.1439,
+      "eval_samples_per_second": 32.271,
+      "eval_steps_per_second": 0.583,
+      "step": 76
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 1.360551357269287,
+      "learning_rate": 1.5092592592592595e-05,
+      "loss": 2.1079,
+      "step": 77
+    },
+    {
+      "epoch": 3.111111111111111,
+      "eval_loss": 2.092242956161499,
+      "eval_runtime": 5.1333,
+      "eval_samples_per_second": 32.338,
+      "eval_steps_per_second": 0.584,
+      "step": 77
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "grad_norm": 1.3474143743515015,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 2.1036,
+      "step": 78
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "eval_loss": 2.0584089756011963,
+      "eval_runtime": 5.1757,
+      "eval_samples_per_second": 32.073,
+      "eval_steps_per_second": 0.58,
+      "step": 78
+    },
+    {
+      "epoch": 3.191919191919192,
+      "grad_norm": 1.368388295173645,
+      "learning_rate": 1.4907407407407408e-05,
+      "loss": 2.0527,
+      "step": 79
+    },
+    {
+      "epoch": 3.191919191919192,
+      "eval_loss": 2.0241854190826416,
+      "eval_runtime": 5.1747,
+      "eval_samples_per_second": 32.079,
+      "eval_steps_per_second": 0.58,
+      "step": 79
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "grad_norm": 1.3582555055618286,
+      "learning_rate": 1.4814814814814815e-05,
+      "loss": 1.9958,
+      "step": 80
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "eval_loss": 1.990015983581543,
+      "eval_runtime": 5.1542,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 80
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "grad_norm": 1.3820135593414307,
+      "learning_rate": 1.4722222222222224e-05,
+      "loss": 1.9878,
+      "step": 81
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "eval_loss": 1.9560723304748535,
+      "eval_runtime": 5.176,
+      "eval_samples_per_second": 32.071,
+      "eval_steps_per_second": 0.58,
+      "step": 81
+    },
+    {
+      "epoch": 3.313131313131313,
+      "grad_norm": 1.4004285335540771,
+      "learning_rate": 1.4629629629629631e-05,
+      "loss": 1.9533,
+      "step": 82
+    },
+    {
+      "epoch": 3.313131313131313,
+      "eval_loss": 1.9231067895889282,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 82
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "grad_norm": 1.4260685443878174,
+      "learning_rate": 1.4537037037037039e-05,
+      "loss": 1.9066,
+      "step": 83
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "eval_loss": 1.8882259130477905,
+      "eval_runtime": 5.1573,
+      "eval_samples_per_second": 32.187,
+      "eval_steps_per_second": 0.582,
+      "step": 83
+    },
+    {
+      "epoch": 3.393939393939394,
+      "grad_norm": 1.4351316690444946,
+      "learning_rate": 1.4444444444444446e-05,
+      "loss": 1.8803,
+      "step": 84
+    },
+    {
+      "epoch": 3.393939393939394,
+      "eval_loss": 1.8546966314315796,
+      "eval_runtime": 5.1748,
+      "eval_samples_per_second": 32.078,
+      "eval_steps_per_second": 0.58,
+      "step": 84
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "grad_norm": 1.4154078960418701,
+      "learning_rate": 1.4351851851851853e-05,
+      "loss": 1.8668,
+      "step": 85
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "eval_loss": 1.820953369140625,
+      "eval_runtime": 5.1325,
+      "eval_samples_per_second": 32.343,
+      "eval_steps_per_second": 0.585,
+      "step": 85
+    },
+    {
+      "epoch": 3.474747474747475,
+      "grad_norm": 1.4371109008789062,
+      "learning_rate": 1.4259259259259259e-05,
+      "loss": 1.822,
+      "step": 86
+    },
+    {
+      "epoch": 3.474747474747475,
+      "eval_loss": 1.7877326011657715,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 86
+    },
+    {
+      "epoch": 3.515151515151515,
+      "grad_norm": 1.4090356826782227,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 1.7796,
+      "step": 87
+    },
+    {
+      "epoch": 3.515151515151515,
+      "eval_loss": 1.7542084455490112,
+      "eval_runtime": 5.1712,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 87
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 1.382804036140442,
+      "learning_rate": 1.4074074074074075e-05,
+      "loss": 1.7457,
+      "step": 88
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "eval_loss": 1.7219429016113281,
+      "eval_runtime": 5.165,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 88
+    },
+    {
+      "epoch": 3.595959595959596,
+      "grad_norm": 1.4041632413864136,
+      "learning_rate": 1.3981481481481482e-05,
+      "loss": 1.7043,
+      "step": 89
+    },
+    {
+      "epoch": 3.595959595959596,
+      "eval_loss": 1.689993977546692,
+      "eval_runtime": 5.1401,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 89
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.3862537145614624,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 1.6742,
+      "step": 90
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "eval_loss": 1.659338116645813,
+      "eval_runtime": 5.1347,
+      "eval_samples_per_second": 32.329,
+      "eval_steps_per_second": 0.584,
+      "step": 90
+    },
+    {
+      "epoch": 3.676767676767677,
+      "grad_norm": 1.3727890253067017,
+      "learning_rate": 1.3796296296296297e-05,
+      "loss": 1.6502,
+      "step": 91
+    },
+    {
+      "epoch": 3.676767676767677,
+      "eval_loss": 1.628204345703125,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 91
+    },
+    {
+      "epoch": 3.717171717171717,
+      "grad_norm": 1.400058388710022,
+      "learning_rate": 1.3703703703703706e-05,
+      "loss": 1.6396,
+      "step": 92
+    },
+    {
+      "epoch": 3.717171717171717,
+      "eval_loss": 1.5992398262023926,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 92
+    },
+    {
+      "epoch": 3.757575757575758,
+      "grad_norm": 1.4236724376678467,
+      "learning_rate": 1.3611111111111113e-05,
+      "loss": 1.6123,
+      "step": 93
+    },
+    {
+      "epoch": 3.757575757575758,
+      "eval_loss": 1.5681532621383667,
+      "eval_runtime": 5.157,
+      "eval_samples_per_second": 32.189,
+      "eval_steps_per_second": 0.582,
+      "step": 93
+    },
+    {
+      "epoch": 3.797979797979798,
+      "grad_norm": 1.4351081848144531,
+      "learning_rate": 1.351851851851852e-05,
+      "loss": 1.5512,
+      "step": 94
+    },
+    {
+      "epoch": 3.797979797979798,
+      "eval_loss": 1.5355143547058105,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 94
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "grad_norm": 1.4533002376556396,
+      "learning_rate": 1.3425925925925926e-05,
+      "loss": 1.5697,
+      "step": 95
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "eval_loss": 1.506730318069458,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 95
+    },
+    {
+      "epoch": 3.878787878787879,
+      "grad_norm": 1.4752202033996582,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.495,
+      "step": 96
+    },
+    {
+      "epoch": 3.878787878787879,
+      "eval_loss": 1.4759498834609985,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 96
+    },
+    {
+      "epoch": 3.919191919191919,
+      "grad_norm": 1.4492502212524414,
+      "learning_rate": 1.3240740740740742e-05,
+      "loss": 1.4681,
+      "step": 97
+    },
+    {
+      "epoch": 3.919191919191919,
+      "eval_loss": 1.4438410997390747,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 97
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "grad_norm": 1.4566094875335693,
+      "learning_rate": 1.314814814814815e-05,
+      "loss": 1.4542,
+      "step": 98
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "eval_loss": 1.4141173362731934,
+      "eval_runtime": 5.1691,
+      "eval_samples_per_second": 32.114,
+      "eval_steps_per_second": 0.58,
+      "step": 98
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 1.47097647190094,
+      "learning_rate": 1.3055555555555557e-05,
+      "loss": 1.4428,
+      "step": 99
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.381595492362976,
+      "eval_runtime": 5.1554,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 99
+    },
+    {
+      "epoch": 4.040404040404041,
+      "grad_norm": 1.4813333749771118,
+      "learning_rate": 1.2962962962962964e-05,
+      "loss": 1.403,
+      "step": 100
+    },
+    {
+      "epoch": 4.040404040404041,
+      "eval_loss": 1.3522053956985474,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 100
+    },
+    {
+      "epoch": 4.08080808080808,
+      "grad_norm": 1.500218391418457,
+      "learning_rate": 1.2870370370370371e-05,
+      "loss": 1.3289,
+      "step": 101
+    },
+    {
+      "epoch": 4.08080808080808,
+      "eval_loss": 1.3220492601394653,
+      "eval_runtime": 5.1719,
+      "eval_samples_per_second": 32.096,
+      "eval_steps_per_second": 0.58,
+      "step": 101
+    },
+    {
+      "epoch": 4.121212121212121,
+      "grad_norm": 1.4614801406860352,
+      "learning_rate": 1.2777777777777777e-05,
+      "loss": 1.3151,
+      "step": 102
+    },
+    {
+      "epoch": 4.121212121212121,
+      "eval_loss": 1.2928389310836792,
+      "eval_runtime": 5.1718,
+      "eval_samples_per_second": 32.097,
+      "eval_steps_per_second": 0.58,
+      "step": 102
+    },
+    {
+      "epoch": 4.161616161616162,
+      "grad_norm": 1.4137208461761475,
+      "learning_rate": 1.2685185185185188e-05,
+      "loss": 1.2993,
+      "step": 103
+    },
+    {
+      "epoch": 4.161616161616162,
+      "eval_loss": 1.2663666009902954,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 103
+    },
+    {
+      "epoch": 4.202020202020202,
+      "grad_norm": 1.347014307975769,
+      "learning_rate": 1.2592592592592593e-05,
+      "loss": 1.2506,
+      "step": 104
+    },
+    {
+      "epoch": 4.202020202020202,
+      "eval_loss": 1.242059588432312,
+      "eval_runtime": 5.1713,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 104
+    },
+    {
+      "epoch": 4.242424242424242,
+      "grad_norm": 1.3283647298812866,
+      "learning_rate": 1.25e-05,
+      "loss": 1.2453,
+      "step": 105
+    },
+    {
+      "epoch": 4.242424242424242,
+      "eval_loss": 1.2196069955825806,
+      "eval_runtime": 5.1174,
+      "eval_samples_per_second": 32.438,
+      "eval_steps_per_second": 0.586,
+      "step": 105
+    },
+    {
+      "epoch": 4.282828282828283,
+      "grad_norm": 1.2796555757522583,
+      "learning_rate": 1.2407407407407408e-05,
+      "loss": 1.1805,
+      "step": 106
+    },
+    {
+      "epoch": 4.282828282828283,
+      "eval_loss": 1.1993300914764404,
+      "eval_runtime": 5.125,
+      "eval_samples_per_second": 32.39,
+      "eval_steps_per_second": 0.585,
+      "step": 106
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "grad_norm": 1.264662742614746,
+      "learning_rate": 1.2314814814814815e-05,
+      "loss": 1.1939,
+      "step": 107
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "eval_loss": 1.1796401739120483,
+      "eval_runtime": 5.154,
+      "eval_samples_per_second": 32.208,
+      "eval_steps_per_second": 0.582,
+      "step": 107
+    },
+    {
+      "epoch": 4.363636363636363,
+      "grad_norm": 1.2420271635055542,
+      "learning_rate": 1.2222222222222224e-05,
+      "loss": 1.1684,
+      "step": 108
+    },
+    {
+      "epoch": 4.363636363636363,
+      "eval_loss": 1.161789894104004,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 108
+    },
+    {
+      "epoch": 4.404040404040404,
+      "grad_norm": 1.2369439601898193,
+      "learning_rate": 1.2129629629629631e-05,
+      "loss": 1.1893,
+      "step": 109
+    },
+    {
+      "epoch": 4.404040404040404,
+      "eval_loss": 1.1432855129241943,
+      "eval_runtime": 5.1704,
+      "eval_samples_per_second": 32.106,
+      "eval_steps_per_second": 0.58,
+      "step": 109
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1795873641967773,
+      "learning_rate": 1.2037037037037039e-05,
+      "loss": 1.1554,
+      "step": 110
+    },
+    {
+      "epoch": 4.444444444444445,
+      "eval_loss": 1.126727819442749,
+      "eval_runtime": 5.1629,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 110
+    },
+    {
+      "epoch": 4.484848484848484,
+      "grad_norm": 1.1451513767242432,
+      "learning_rate": 1.1944444444444444e-05,
+      "loss": 1.1276,
+      "step": 111
+    },
+    {
+      "epoch": 4.484848484848484,
+      "eval_loss": 1.1112148761749268,
+      "eval_runtime": 5.1514,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 111
+    },
+    {
+      "epoch": 4.525252525252525,
+      "grad_norm": 1.124099612236023,
+      "learning_rate": 1.1851851851851852e-05,
+      "loss": 1.0972,
+      "step": 112
+    },
+    {
+      "epoch": 4.525252525252525,
+      "eval_loss": 1.0960901975631714,
+      "eval_runtime": 5.092,
+      "eval_samples_per_second": 32.6,
+      "eval_steps_per_second": 0.589,
+      "step": 112
+    },
+    {
+      "epoch": 4.565656565656566,
+      "grad_norm": 1.090491533279419,
+      "learning_rate": 1.1759259259259259e-05,
+      "loss": 1.0821,
+      "step": 113
+    },
+    {
+      "epoch": 4.565656565656566,
+      "eval_loss": 1.0813062191009521,
+      "eval_runtime": 5.0997,
+      "eval_samples_per_second": 32.551,
+      "eval_steps_per_second": 0.588,
+      "step": 113
+    },
+    {
+      "epoch": 4.606060606060606,
+      "grad_norm": 1.0799589157104492,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 1.038,
+      "step": 114
+    },
+    {
+      "epoch": 4.606060606060606,
+      "eval_loss": 1.0665714740753174,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.273,
+      "eval_steps_per_second": 0.583,
+      "step": 114
+    },
+    {
+      "epoch": 4.646464646464646,
+      "grad_norm": 1.0503891706466675,
+      "learning_rate": 1.1574074074074075e-05,
+      "loss": 1.0584,
+      "step": 115
+    },
+    {
+      "epoch": 4.646464646464646,
+      "eval_loss": 1.0545704364776611,
+      "eval_runtime": 5.1517,
+      "eval_samples_per_second": 32.222,
+      "eval_steps_per_second": 0.582,
+      "step": 115
+    },
+    {
+      "epoch": 4.686868686868687,
+      "grad_norm": 1.043385624885559,
+      "learning_rate": 1.1481481481481482e-05,
+      "loss": 1.0329,
+      "step": 116
+    },
+    {
+      "epoch": 4.686868686868687,
+      "eval_loss": 1.0427477359771729,
+      "eval_runtime": 5.1504,
+      "eval_samples_per_second": 32.23,
+      "eval_steps_per_second": 0.582,
+      "step": 116
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "grad_norm": 1.0548285245895386,
+      "learning_rate": 1.138888888888889e-05,
+      "loss": 1.0256,
+      "step": 117
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "eval_loss": 1.0287798643112183,
+      "eval_runtime": 5.1603,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 117
+    },
+    {
+      "epoch": 4.767676767676767,
+      "grad_norm": 1.0389068126678467,
+      "learning_rate": 1.1296296296296297e-05,
+      "loss": 1.0192,
+      "step": 118
+    },
+    {
+      "epoch": 4.767676767676767,
+      "eval_loss": 1.0174580812454224,
+      "eval_runtime": 5.1433,
+      "eval_samples_per_second": 32.275,
+      "eval_steps_per_second": 0.583,
+      "step": 118
+    },
+    {
+      "epoch": 4.808080808080808,
+      "grad_norm": 1.035844087600708,
+      "learning_rate": 1.1203703703703706e-05,
+      "loss": 1.0101,
+      "step": 119
+    },
+    {
+      "epoch": 4.808080808080808,
+      "eval_loss": 1.006226658821106,
+      "eval_runtime": 5.1693,
+      "eval_samples_per_second": 32.113,
+      "eval_steps_per_second": 0.58,
+      "step": 119
+    },
+    {
+      "epoch": 4.848484848484849,
+      "grad_norm": 1.050945520401001,
+      "learning_rate": 1.1111111111111113e-05,
+      "loss": 1.0403,
+      "step": 120
+    },
+    {
+      "epoch": 4.848484848484849,
+      "eval_loss": 0.9945307374000549,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.182,
+      "eval_steps_per_second": 0.582,
+      "step": 120
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 1.027295708656311,
+      "learning_rate": 1.1018518518518519e-05,
+      "loss": 1.0038,
+      "step": 121
+    },
+    {
+      "epoch": 4.888888888888889,
+      "eval_loss": 0.9830649495124817,
+      "eval_runtime": 5.1555,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 121
+    },
+    {
+      "epoch": 4.929292929292929,
+      "grad_norm": 1.035843014717102,
+      "learning_rate": 1.0925925925925926e-05,
+      "loss": 0.9544,
+      "step": 122
+    },
+    {
+      "epoch": 4.929292929292929,
+      "eval_loss": 0.9728271961212158,
+      "eval_runtime": 5.1521,
+      "eval_samples_per_second": 32.22,
+      "eval_steps_per_second": 0.582,
+      "step": 122
+    },
+    {
+      "epoch": 4.96969696969697,
+      "grad_norm": 1.050779938697815,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 0.9393,
+      "step": 123
+    },
+    {
+      "epoch": 4.96969696969697,
+      "eval_loss": 0.9609534740447998,
+      "eval_runtime": 5.1551,
+      "eval_samples_per_second": 32.201,
+      "eval_steps_per_second": 0.582,
+      "step": 123
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.191088236150784e+16,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-123/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dafca05b7a6d25266ace8b0db88af88ff18a0ac29cc5d191b665dd79638206
+size 5368

checkpoint-148/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-148/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-148/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f8dc66d4fa25487a2bcac74fc0e6d1fe72bb3c30c08b2f2fa1274ca1c36eb89
+size 35668592

checkpoint-148/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:012b87fba5e233f262ed9e5cbf622700e966107ab6c21aca356fcbe1903720b7
+size 71379898

checkpoint-148/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72204598f502259cdcc0f6db3763a4a1093332d0dd62b686146b4978f8f8f03a
+size 14244

checkpoint-148/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d426840a6b8aa6a2a44e733b299421c27ef1deb8f0f7f4ba3b3a39150980210
+size 1064

checkpoint-148/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-148/tokenizer_config.json

@@ -0,0 +1,131 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": true,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "32000": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32002": {
+      "content": "<|placeholder1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32003": {
+      "content": "<|placeholder2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32004": {
+      "content": "<|placeholder3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32005": {
+      "content": "<|placeholder4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32006": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "<|placeholder5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "<|placeholder6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "legacy": false,
+  "model_max_length": 4096,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-148/trainer_state.json

@@ -0,0 +1,2253 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 5.97979797979798,
+  "eval_steps": 1,
+  "global_step": 148,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04040404040404041,
+      "grad_norm": 2.088834762573242,
+      "learning_rate": 8.333333333333333e-07,
+      "loss": 5.4688,
+      "step": 1
+    },
+    {
+      "epoch": 0.04040404040404041,
+      "eval_loss": 5.470384120941162,
+      "eval_runtime": 4.8066,
+      "eval_samples_per_second": 34.536,
+      "eval_steps_per_second": 0.624,
+      "step": 1
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "grad_norm": 2.0961971282958984,
+      "learning_rate": 1.6666666666666667e-06,
+      "loss": 5.4731,
+      "step": 2
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "eval_loss": 5.4689249992370605,
+      "eval_runtime": 4.8593,
+      "eval_samples_per_second": 34.161,
+      "eval_steps_per_second": 0.617,
+      "step": 2
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "grad_norm": 2.0940349102020264,
+      "learning_rate": 2.5e-06,
+      "loss": 5.4803,
+      "step": 3
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "eval_loss": 5.467825412750244,
+      "eval_runtime": 4.9081,
+      "eval_samples_per_second": 33.822,
+      "eval_steps_per_second": 0.611,
+      "step": 3
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "grad_norm": 2.095174551010132,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 5.5119,
+      "step": 4
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "eval_loss": 5.465516567230225,
+      "eval_runtime": 4.8615,
+      "eval_samples_per_second": 34.146,
+      "eval_steps_per_second": 0.617,
+      "step": 4
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "grad_norm": 2.1109118461608887,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 5.4662,
+      "step": 5
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "eval_loss": 5.4626851081848145,
+      "eval_runtime": 4.844,
+      "eval_samples_per_second": 34.27,
+      "eval_steps_per_second": 0.619,
+      "step": 5
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "grad_norm": 2.1030592918395996,
+      "learning_rate": 5e-06,
+      "loss": 5.4691,
+      "step": 6
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "eval_loss": 5.45806884765625,
+      "eval_runtime": 4.8388,
+      "eval_samples_per_second": 34.306,
+      "eval_steps_per_second": 0.62,
+      "step": 6
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "grad_norm": 2.1202447414398193,
+      "learning_rate": 5.833333333333334e-06,
+      "loss": 5.4851,
+      "step": 7
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "eval_loss": 5.449892520904541,
+      "eval_runtime": 4.8431,
+      "eval_samples_per_second": 34.276,
+      "eval_steps_per_second": 0.619,
+      "step": 7
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "grad_norm": 2.090963363647461,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 5.4228,
+      "step": 8
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "eval_loss": 5.442515850067139,
+      "eval_runtime": 4.8605,
+      "eval_samples_per_second": 34.153,
+      "eval_steps_per_second": 0.617,
+      "step": 8
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "grad_norm": 2.112323760986328,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 5.4715,
+      "step": 9
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "eval_loss": 5.4345855712890625,
+      "eval_runtime": 4.8796,
+      "eval_samples_per_second": 34.019,
+      "eval_steps_per_second": 0.615,
+      "step": 9
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "grad_norm": 2.1456995010375977,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 5.4211,
+      "step": 10
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "eval_loss": 5.424546718597412,
+      "eval_runtime": 4.9274,
+      "eval_samples_per_second": 33.689,
+      "eval_steps_per_second": 0.609,
+      "step": 10
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 2.1602349281311035,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 5.4301,
+      "step": 11
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 5.41035795211792,
+      "eval_runtime": 4.9819,
+      "eval_samples_per_second": 33.32,
+      "eval_steps_per_second": 0.602,
+      "step": 11
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "grad_norm": 2.1762421131134033,
+      "learning_rate": 1e-05,
+      "loss": 5.4158,
+      "step": 12
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "eval_loss": 5.400452613830566,
+      "eval_runtime": 5.0737,
+      "eval_samples_per_second": 32.718,
+      "eval_steps_per_second": 0.591,
+      "step": 12
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "grad_norm": 2.2415969371795654,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 5.3986,
+      "step": 13
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "eval_loss": 5.382190227508545,
+      "eval_runtime": 4.9555,
+      "eval_samples_per_second": 33.498,
+      "eval_steps_per_second": 0.605,
+      "step": 13
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "grad_norm": 2.2669475078582764,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 5.428,
+      "step": 14
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "eval_loss": 5.3634443283081055,
+      "eval_runtime": 4.9029,
+      "eval_samples_per_second": 33.857,
+      "eval_steps_per_second": 0.612,
+      "step": 14
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "grad_norm": 2.262896776199341,
+      "learning_rate": 1.25e-05,
+      "loss": 5.3741,
+      "step": 15
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "eval_loss": 5.347131729125977,
+      "eval_runtime": 4.8934,
+      "eval_samples_per_second": 33.923,
+      "eval_steps_per_second": 0.613,
+      "step": 15
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "grad_norm": 2.3363702297210693,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 5.4134,
+      "step": 16
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "eval_loss": 5.321357250213623,
+      "eval_runtime": 4.9027,
+      "eval_samples_per_second": 33.859,
+      "eval_steps_per_second": 0.612,
+      "step": 16
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "grad_norm": 2.3906755447387695,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 5.3305,
+      "step": 17
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "eval_loss": 5.292910099029541,
+      "eval_runtime": 4.9268,
+      "eval_samples_per_second": 33.693,
+      "eval_steps_per_second": 0.609,
+      "step": 17
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "grad_norm": 2.44368839263916,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 5.3406,
+      "step": 18
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "eval_loss": 5.257543563842773,
+      "eval_runtime": 4.9839,
+      "eval_samples_per_second": 33.308,
+      "eval_steps_per_second": 0.602,
+      "step": 18
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "grad_norm": 2.4937825202941895,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 5.3134,
+      "step": 19
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "eval_loss": 5.225679874420166,
+      "eval_runtime": 5.1205,
+      "eval_samples_per_second": 32.419,
+      "eval_steps_per_second": 0.586,
+      "step": 19
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "grad_norm": 2.547194480895996,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 5.2303,
+      "step": 20
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "eval_loss": 5.183907985687256,
+      "eval_runtime": 5.1208,
+      "eval_samples_per_second": 32.417,
+      "eval_steps_per_second": 0.586,
+      "step": 20
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "grad_norm": 2.5578083992004395,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 5.1255,
+      "step": 21
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "eval_loss": 5.136340141296387,
+      "eval_runtime": 5.1161,
+      "eval_samples_per_second": 32.446,
+      "eval_steps_per_second": 0.586,
+      "step": 21
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 2.6553032398223877,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 5.1376,
+      "step": 22
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 5.085525035858154,
+      "eval_runtime": 5.1496,
+      "eval_samples_per_second": 32.236,
+      "eval_steps_per_second": 0.583,
+      "step": 22
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "grad_norm": 2.733492374420166,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 5.1051,
+      "step": 23
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "eval_loss": 5.0193634033203125,
+      "eval_runtime": 5.0473,
+      "eval_samples_per_second": 32.889,
+      "eval_steps_per_second": 0.594,
+      "step": 23
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "grad_norm": 2.765582799911499,
+      "learning_rate": 2e-05,
+      "loss": 5.0082,
+      "step": 24
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "eval_loss": 4.950736999511719,
+      "eval_runtime": 4.989,
+      "eval_samples_per_second": 33.273,
+      "eval_steps_per_second": 0.601,
+      "step": 24
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "grad_norm": 2.8943891525268555,
+      "learning_rate": 1.990740740740741e-05,
+      "loss": 4.9736,
+      "step": 25
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "eval_loss": 4.87933349609375,
+      "eval_runtime": 4.991,
+      "eval_samples_per_second": 33.26,
+      "eval_steps_per_second": 0.601,
+      "step": 25
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "grad_norm": 2.9802019596099854,
+      "learning_rate": 1.9814814814814816e-05,
+      "loss": 4.8728,
+      "step": 26
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "eval_loss": 4.796956539154053,
+      "eval_runtime": 5.0224,
+      "eval_samples_per_second": 33.052,
+      "eval_steps_per_second": 0.597,
+      "step": 26
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "grad_norm": 3.068443536758423,
+      "learning_rate": 1.9722222222222224e-05,
+      "loss": 4.8132,
+      "step": 27
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "eval_loss": 4.7025275230407715,
+      "eval_runtime": 5.1226,
+      "eval_samples_per_second": 32.406,
+      "eval_steps_per_second": 0.586,
+      "step": 27
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "grad_norm": 3.2245917320251465,
+      "learning_rate": 1.962962962962963e-05,
+      "loss": 4.7175,
+      "step": 28
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "eval_loss": 4.607891082763672,
+      "eval_runtime": 5.0927,
+      "eval_samples_per_second": 32.596,
+      "eval_steps_per_second": 0.589,
+      "step": 28
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "grad_norm": 3.3136157989501953,
+      "learning_rate": 1.9537037037037038e-05,
+      "loss": 4.6429,
+      "step": 29
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "eval_loss": 4.510694980621338,
+      "eval_runtime": 5.0136,
+      "eval_samples_per_second": 33.11,
+      "eval_steps_per_second": 0.598,
+      "step": 29
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "grad_norm": 3.329773426055908,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 4.4686,
+      "step": 30
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "eval_loss": 4.406045913696289,
+      "eval_runtime": 4.9753,
+      "eval_samples_per_second": 33.365,
+      "eval_steps_per_second": 0.603,
+      "step": 30
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "grad_norm": 3.5462117195129395,
+      "learning_rate": 1.9351851851851853e-05,
+      "loss": 4.427,
+      "step": 31
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "eval_loss": 4.309689998626709,
+      "eval_runtime": 4.9974,
+      "eval_samples_per_second": 33.217,
+      "eval_steps_per_second": 0.6,
+      "step": 31
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "grad_norm": 3.6190805435180664,
+      "learning_rate": 1.925925925925926e-05,
+      "loss": 4.2899,
+      "step": 32
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "eval_loss": 4.215596675872803,
+      "eval_runtime": 5.0637,
+      "eval_samples_per_second": 32.782,
+      "eval_steps_per_second": 0.592,
+      "step": 32
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 3.698317289352417,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 4.1752,
+      "step": 33
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 4.106834411621094,
+      "eval_runtime": 5.1651,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 33
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "grad_norm": 3.4807214736938477,
+      "learning_rate": 1.9074074074074075e-05,
+      "loss": 4.1277,
+      "step": 34
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "eval_loss": 4.001583099365234,
+      "eval_runtime": 5.0567,
+      "eval_samples_per_second": 32.828,
+      "eval_steps_per_second": 0.593,
+      "step": 34
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "grad_norm": 3.1380534172058105,
+      "learning_rate": 1.8981481481481482e-05,
+      "loss": 4.0069,
+      "step": 35
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "eval_loss": 3.9105026721954346,
+      "eval_runtime": 5.07,
+      "eval_samples_per_second": 32.741,
+      "eval_steps_per_second": 0.592,
+      "step": 35
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "grad_norm": 2.9076974391937256,
+      "learning_rate": 1.888888888888889e-05,
+      "loss": 3.8911,
+      "step": 36
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "eval_loss": 3.838275194168091,
+      "eval_runtime": 5.1402,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 36
+    },
+    {
+      "epoch": 1.494949494949495,
+      "grad_norm": 2.4755232334136963,
+      "learning_rate": 1.8796296296296296e-05,
+      "loss": 3.8522,
+      "step": 37
+    },
+    {
+      "epoch": 1.494949494949495,
+      "eval_loss": 3.776526689529419,
+      "eval_runtime": 5.1768,
+      "eval_samples_per_second": 32.066,
+      "eval_steps_per_second": 0.58,
+      "step": 37
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "grad_norm": 2.1607069969177246,
+      "learning_rate": 1.8703703703703707e-05,
+      "loss": 3.7828,
+      "step": 38
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "eval_loss": 3.7257139682769775,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 38
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "grad_norm": 1.9251853227615356,
+      "learning_rate": 1.8611111111111114e-05,
+      "loss": 3.7511,
+      "step": 39
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "eval_loss": 3.680651903152466,
+      "eval_runtime": 5.139,
+      "eval_samples_per_second": 32.302,
+      "eval_steps_per_second": 0.584,
+      "step": 39
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "grad_norm": 1.8553980588912964,
+      "learning_rate": 1.851851851851852e-05,
+      "loss": 3.7108,
+      "step": 40
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "eval_loss": 3.633517026901245,
+      "eval_runtime": 5.1036,
+      "eval_samples_per_second": 32.526,
+      "eval_steps_per_second": 0.588,
+      "step": 40
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "grad_norm": 1.831110954284668,
+      "learning_rate": 1.8425925925925926e-05,
+      "loss": 3.6165,
+      "step": 41
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "eval_loss": 3.5877223014831543,
+      "eval_runtime": 5.1287,
+      "eval_samples_per_second": 32.367,
+      "eval_steps_per_second": 0.585,
+      "step": 41
+    },
+    {
+      "epoch": 1.696969696969697,
+      "grad_norm": 1.8035303354263306,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 3.5555,
+      "step": 42
+    },
+    {
+      "epoch": 1.696969696969697,
+      "eval_loss": 3.5403287410736084,
+      "eval_runtime": 5.1001,
+      "eval_samples_per_second": 32.549,
+      "eval_steps_per_second": 0.588,
+      "step": 42
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "grad_norm": 1.7953453063964844,
+      "learning_rate": 1.8240740740740744e-05,
+      "loss": 3.5437,
+      "step": 43
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "eval_loss": 3.492953300476074,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 43
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 1.7807087898254395,
+      "learning_rate": 1.814814814814815e-05,
+      "loss": 3.4676,
+      "step": 44
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 3.443131446838379,
+      "eval_runtime": 5.1066,
+      "eval_samples_per_second": 32.507,
+      "eval_steps_per_second": 0.587,
+      "step": 44
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 1.8204630613327026,
+      "learning_rate": 1.8055555555555558e-05,
+      "loss": 3.4517,
+      "step": 45
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "eval_loss": 3.396712303161621,
+      "eval_runtime": 5.1475,
+      "eval_samples_per_second": 32.249,
+      "eval_steps_per_second": 0.583,
+      "step": 45
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "grad_norm": 1.8122252225875854,
+      "learning_rate": 1.7962962962962965e-05,
+      "loss": 3.4189,
+      "step": 46
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "eval_loss": 3.3434314727783203,
+      "eval_runtime": 5.0828,
+      "eval_samples_per_second": 32.659,
+      "eval_steps_per_second": 0.59,
+      "step": 46
+    },
+    {
+      "epoch": 1.898989898989899,
+      "grad_norm": 1.8433269262313843,
+      "learning_rate": 1.7870370370370373e-05,
+      "loss": 3.3537,
+      "step": 47
+    },
+    {
+      "epoch": 1.898989898989899,
+      "eval_loss": 3.294283628463745,
+      "eval_runtime": 5.1195,
+      "eval_samples_per_second": 32.425,
+      "eval_steps_per_second": 0.586,
+      "step": 47
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "grad_norm": 1.833533763885498,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 3.3034,
+      "step": 48
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "eval_loss": 3.2412631511688232,
+      "eval_runtime": 5.1404,
+      "eval_samples_per_second": 32.293,
+      "eval_steps_per_second": 0.584,
+      "step": 48
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "grad_norm": 1.823169469833374,
+      "learning_rate": 1.7685185185185187e-05,
+      "loss": 3.2457,
+      "step": 49
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "eval_loss": 3.1890854835510254,
+      "eval_runtime": 5.1319,
+      "eval_samples_per_second": 32.347,
+      "eval_steps_per_second": 0.585,
+      "step": 49
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "grad_norm": 1.796790361404419,
+      "learning_rate": 1.7592592592592595e-05,
+      "loss": 3.1915,
+      "step": 50
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "eval_loss": 3.1396946907043457,
+      "eval_runtime": 5.1262,
+      "eval_samples_per_second": 32.383,
+      "eval_steps_per_second": 0.585,
+      "step": 50
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "grad_norm": 1.7133023738861084,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 3.1475,
+      "step": 51
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "eval_loss": 3.093197822570801,
+      "eval_runtime": 5.141,
+      "eval_samples_per_second": 32.29,
+      "eval_steps_per_second": 0.584,
+      "step": 51
+    },
+    {
+      "epoch": 2.101010101010101,
+      "grad_norm": 1.637424111366272,
+      "learning_rate": 1.740740740740741e-05,
+      "loss": 3.0912,
+      "step": 52
+    },
+    {
+      "epoch": 2.101010101010101,
+      "eval_loss": 3.0486488342285156,
+      "eval_runtime": 5.1578,
+      "eval_samples_per_second": 32.185,
+      "eval_steps_per_second": 0.582,
+      "step": 52
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "grad_norm": 1.5733733177185059,
+      "learning_rate": 1.7314814814814816e-05,
+      "loss": 3.0493,
+      "step": 53
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "eval_loss": 3.007200002670288,
+      "eval_runtime": 5.1715,
+      "eval_samples_per_second": 32.099,
+      "eval_steps_per_second": 0.58,
+      "step": 53
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "grad_norm": 1.579100251197815,
+      "learning_rate": 1.7222222222222224e-05,
+      "loss": 3.0007,
+      "step": 54
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "eval_loss": 2.9646313190460205,
+      "eval_runtime": 5.1737,
+      "eval_samples_per_second": 32.085,
+      "eval_steps_per_second": 0.58,
+      "step": 54
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.5558359622955322,
+      "learning_rate": 1.712962962962963e-05,
+      "loss": 2.955,
+      "step": 55
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 2.9225423336029053,
+      "eval_runtime": 5.1649,
+      "eval_samples_per_second": 32.14,
+      "eval_steps_per_second": 0.581,
+      "step": 55
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "grad_norm": 1.5695600509643555,
+      "learning_rate": 1.7037037037037038e-05,
+      "loss": 2.9242,
+      "step": 56
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "eval_loss": 2.883842706680298,
+      "eval_runtime": 5.1565,
+      "eval_samples_per_second": 32.192,
+      "eval_steps_per_second": 0.582,
+      "step": 56
+    },
+    {
+      "epoch": 2.303030303030303,
+      "grad_norm": 1.5856674909591675,
+      "learning_rate": 1.6944444444444446e-05,
+      "loss": 2.8626,
+      "step": 57
+    },
+    {
+      "epoch": 2.303030303030303,
+      "eval_loss": 2.8435447216033936,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 57
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "grad_norm": 1.5780515670776367,
+      "learning_rate": 1.6851851851851853e-05,
+      "loss": 2.8522,
+      "step": 58
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "eval_loss": 2.8028392791748047,
+      "eval_runtime": 5.1666,
+      "eval_samples_per_second": 32.129,
+      "eval_steps_per_second": 0.581,
+      "step": 58
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "grad_norm": 1.5944206714630127,
+      "learning_rate": 1.675925925925926e-05,
+      "loss": 2.8222,
+      "step": 59
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "eval_loss": 2.7607803344726562,
+      "eval_runtime": 5.1662,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 59
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "grad_norm": 1.6033095121383667,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.7291,
+      "step": 60
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "eval_loss": 2.7191247940063477,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 60
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "grad_norm": 1.621768593788147,
+      "learning_rate": 1.6574074074074075e-05,
+      "loss": 2.7046,
+      "step": 61
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "eval_loss": 2.6788933277130127,
+      "eval_runtime": 5.1728,
+      "eval_samples_per_second": 32.091,
+      "eval_steps_per_second": 0.58,
+      "step": 61
+    },
+    {
+      "epoch": 2.505050505050505,
+      "grad_norm": 1.6213834285736084,
+      "learning_rate": 1.6481481481481482e-05,
+      "loss": 2.6548,
+      "step": 62
+    },
+    {
+      "epoch": 2.505050505050505,
+      "eval_loss": 2.6340527534484863,
+      "eval_runtime": 5.1628,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 62
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "grad_norm": 1.607869267463684,
+      "learning_rate": 1.638888888888889e-05,
+      "loss": 2.6232,
+      "step": 63
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "eval_loss": 2.594001531600952,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 63
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "grad_norm": 1.583896279335022,
+      "learning_rate": 1.6296296296296297e-05,
+      "loss": 2.6068,
+      "step": 64
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "eval_loss": 2.5528736114501953,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 64
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "grad_norm": 1.5254372358322144,
+      "learning_rate": 1.6203703703703707e-05,
+      "loss": 2.5629,
+      "step": 65
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "eval_loss": 2.5128839015960693,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 65
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 1.4735966920852661,
+      "learning_rate": 1.6111111111111115e-05,
+      "loss": 2.523,
+      "step": 66
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 2.4759180545806885,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 66
+    },
+    {
+      "epoch": 2.707070707070707,
+      "grad_norm": 1.4135326147079468,
+      "learning_rate": 1.601851851851852e-05,
+      "loss": 2.5069,
+      "step": 67
+    },
+    {
+      "epoch": 2.707070707070707,
+      "eval_loss": 2.4382143020629883,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.274,
+      "eval_steps_per_second": 0.583,
+      "step": 67
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "grad_norm": 1.3859189748764038,
+      "learning_rate": 1.5925925925925926e-05,
+      "loss": 2.449,
+      "step": 68
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "eval_loss": 2.3999476432800293,
+      "eval_runtime": 5.0261,
+      "eval_samples_per_second": 33.028,
+      "eval_steps_per_second": 0.597,
+      "step": 68
+    },
+    {
+      "epoch": 2.787878787878788,
+      "grad_norm": 1.3678818941116333,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 2.3977,
+      "step": 69
+    },
+    {
+      "epoch": 2.787878787878788,
+      "eval_loss": 2.365516424179077,
+      "eval_runtime": 4.9945,
+      "eval_samples_per_second": 33.237,
+      "eval_steps_per_second": 0.601,
+      "step": 69
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "grad_norm": 1.372074842453003,
+      "learning_rate": 1.5740740740740744e-05,
+      "loss": 2.3556,
+      "step": 70
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "eval_loss": 2.3320348262786865,
+      "eval_runtime": 5.0457,
+      "eval_samples_per_second": 32.899,
+      "eval_steps_per_second": 0.595,
+      "step": 70
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "grad_norm": 1.355190634727478,
+      "learning_rate": 1.564814814814815e-05,
+      "loss": 2.3269,
+      "step": 71
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "eval_loss": 2.2980406284332275,
+      "eval_runtime": 5.1448,
+      "eval_samples_per_second": 32.265,
+      "eval_steps_per_second": 0.583,
+      "step": 71
+    },
+    {
+      "epoch": 2.909090909090909,
+      "grad_norm": 1.34852135181427,
+      "learning_rate": 1.555555555555556e-05,
+      "loss": 2.2962,
+      "step": 72
+    },
+    {
+      "epoch": 2.909090909090909,
+      "eval_loss": 2.262707233428955,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 72
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "grad_norm": 1.3559976816177368,
+      "learning_rate": 1.5462962962962966e-05,
+      "loss": 2.2534,
+      "step": 73
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "eval_loss": 2.2291126251220703,
+      "eval_runtime": 5.0794,
+      "eval_samples_per_second": 32.681,
+      "eval_steps_per_second": 0.591,
+      "step": 73
+    },
+    {
+      "epoch": 2.98989898989899,
+      "grad_norm": 1.3566159009933472,
+      "learning_rate": 1.537037037037037e-05,
+      "loss": 2.2061,
+      "step": 74
+    },
+    {
+      "epoch": 2.98989898989899,
+      "eval_loss": 2.193363904953003,
+      "eval_runtime": 5.0616,
+      "eval_samples_per_second": 32.796,
+      "eval_steps_per_second": 0.593,
+      "step": 74
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "grad_norm": 1.3466676473617554,
+      "learning_rate": 1.5277777777777777e-05,
+      "loss": 2.1815,
+      "step": 75
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "eval_loss": 2.159188985824585,
+      "eval_runtime": 5.0978,
+      "eval_samples_per_second": 32.563,
+      "eval_steps_per_second": 0.588,
+      "step": 75
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "grad_norm": 1.354913592338562,
+      "learning_rate": 1.5185185185185187e-05,
+      "loss": 2.1622,
+      "step": 76
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "eval_loss": 2.124568462371826,
+      "eval_runtime": 5.1439,
+      "eval_samples_per_second": 32.271,
+      "eval_steps_per_second": 0.583,
+      "step": 76
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 1.360551357269287,
+      "learning_rate": 1.5092592592592595e-05,
+      "loss": 2.1079,
+      "step": 77
+    },
+    {
+      "epoch": 3.111111111111111,
+      "eval_loss": 2.092242956161499,
+      "eval_runtime": 5.1333,
+      "eval_samples_per_second": 32.338,
+      "eval_steps_per_second": 0.584,
+      "step": 77
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "grad_norm": 1.3474143743515015,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 2.1036,
+      "step": 78
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "eval_loss": 2.0584089756011963,
+      "eval_runtime": 5.1757,
+      "eval_samples_per_second": 32.073,
+      "eval_steps_per_second": 0.58,
+      "step": 78
+    },
+    {
+      "epoch": 3.191919191919192,
+      "grad_norm": 1.368388295173645,
+      "learning_rate": 1.4907407407407408e-05,
+      "loss": 2.0527,
+      "step": 79
+    },
+    {
+      "epoch": 3.191919191919192,
+      "eval_loss": 2.0241854190826416,
+      "eval_runtime": 5.1747,
+      "eval_samples_per_second": 32.079,
+      "eval_steps_per_second": 0.58,
+      "step": 79
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "grad_norm": 1.3582555055618286,
+      "learning_rate": 1.4814814814814815e-05,
+      "loss": 1.9958,
+      "step": 80
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "eval_loss": 1.990015983581543,
+      "eval_runtime": 5.1542,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 80
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "grad_norm": 1.3820135593414307,
+      "learning_rate": 1.4722222222222224e-05,
+      "loss": 1.9878,
+      "step": 81
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "eval_loss": 1.9560723304748535,
+      "eval_runtime": 5.176,
+      "eval_samples_per_second": 32.071,
+      "eval_steps_per_second": 0.58,
+      "step": 81
+    },
+    {
+      "epoch": 3.313131313131313,
+      "grad_norm": 1.4004285335540771,
+      "learning_rate": 1.4629629629629631e-05,
+      "loss": 1.9533,
+      "step": 82
+    },
+    {
+      "epoch": 3.313131313131313,
+      "eval_loss": 1.9231067895889282,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 82
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "grad_norm": 1.4260685443878174,
+      "learning_rate": 1.4537037037037039e-05,
+      "loss": 1.9066,
+      "step": 83
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "eval_loss": 1.8882259130477905,
+      "eval_runtime": 5.1573,
+      "eval_samples_per_second": 32.187,
+      "eval_steps_per_second": 0.582,
+      "step": 83
+    },
+    {
+      "epoch": 3.393939393939394,
+      "grad_norm": 1.4351316690444946,
+      "learning_rate": 1.4444444444444446e-05,
+      "loss": 1.8803,
+      "step": 84
+    },
+    {
+      "epoch": 3.393939393939394,
+      "eval_loss": 1.8546966314315796,
+      "eval_runtime": 5.1748,
+      "eval_samples_per_second": 32.078,
+      "eval_steps_per_second": 0.58,
+      "step": 84
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "grad_norm": 1.4154078960418701,
+      "learning_rate": 1.4351851851851853e-05,
+      "loss": 1.8668,
+      "step": 85
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "eval_loss": 1.820953369140625,
+      "eval_runtime": 5.1325,
+      "eval_samples_per_second": 32.343,
+      "eval_steps_per_second": 0.585,
+      "step": 85
+    },
+    {
+      "epoch": 3.474747474747475,
+      "grad_norm": 1.4371109008789062,
+      "learning_rate": 1.4259259259259259e-05,
+      "loss": 1.822,
+      "step": 86
+    },
+    {
+      "epoch": 3.474747474747475,
+      "eval_loss": 1.7877326011657715,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 86
+    },
+    {
+      "epoch": 3.515151515151515,
+      "grad_norm": 1.4090356826782227,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 1.7796,
+      "step": 87
+    },
+    {
+      "epoch": 3.515151515151515,
+      "eval_loss": 1.7542084455490112,
+      "eval_runtime": 5.1712,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 87
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 1.382804036140442,
+      "learning_rate": 1.4074074074074075e-05,
+      "loss": 1.7457,
+      "step": 88
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "eval_loss": 1.7219429016113281,
+      "eval_runtime": 5.165,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 88
+    },
+    {
+      "epoch": 3.595959595959596,
+      "grad_norm": 1.4041632413864136,
+      "learning_rate": 1.3981481481481482e-05,
+      "loss": 1.7043,
+      "step": 89
+    },
+    {
+      "epoch": 3.595959595959596,
+      "eval_loss": 1.689993977546692,
+      "eval_runtime": 5.1401,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 89
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.3862537145614624,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 1.6742,
+      "step": 90
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "eval_loss": 1.659338116645813,
+      "eval_runtime": 5.1347,
+      "eval_samples_per_second": 32.329,
+      "eval_steps_per_second": 0.584,
+      "step": 90
+    },
+    {
+      "epoch": 3.676767676767677,
+      "grad_norm": 1.3727890253067017,
+      "learning_rate": 1.3796296296296297e-05,
+      "loss": 1.6502,
+      "step": 91
+    },
+    {
+      "epoch": 3.676767676767677,
+      "eval_loss": 1.628204345703125,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 91
+    },
+    {
+      "epoch": 3.717171717171717,
+      "grad_norm": 1.400058388710022,
+      "learning_rate": 1.3703703703703706e-05,
+      "loss": 1.6396,
+      "step": 92
+    },
+    {
+      "epoch": 3.717171717171717,
+      "eval_loss": 1.5992398262023926,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 92
+    },
+    {
+      "epoch": 3.757575757575758,
+      "grad_norm": 1.4236724376678467,
+      "learning_rate": 1.3611111111111113e-05,
+      "loss": 1.6123,
+      "step": 93
+    },
+    {
+      "epoch": 3.757575757575758,
+      "eval_loss": 1.5681532621383667,
+      "eval_runtime": 5.157,
+      "eval_samples_per_second": 32.189,
+      "eval_steps_per_second": 0.582,
+      "step": 93
+    },
+    {
+      "epoch": 3.797979797979798,
+      "grad_norm": 1.4351081848144531,
+      "learning_rate": 1.351851851851852e-05,
+      "loss": 1.5512,
+      "step": 94
+    },
+    {
+      "epoch": 3.797979797979798,
+      "eval_loss": 1.5355143547058105,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 94
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "grad_norm": 1.4533002376556396,
+      "learning_rate": 1.3425925925925926e-05,
+      "loss": 1.5697,
+      "step": 95
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "eval_loss": 1.506730318069458,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 95
+    },
+    {
+      "epoch": 3.878787878787879,
+      "grad_norm": 1.4752202033996582,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.495,
+      "step": 96
+    },
+    {
+      "epoch": 3.878787878787879,
+      "eval_loss": 1.4759498834609985,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 96
+    },
+    {
+      "epoch": 3.919191919191919,
+      "grad_norm": 1.4492502212524414,
+      "learning_rate": 1.3240740740740742e-05,
+      "loss": 1.4681,
+      "step": 97
+    },
+    {
+      "epoch": 3.919191919191919,
+      "eval_loss": 1.4438410997390747,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 97
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "grad_norm": 1.4566094875335693,
+      "learning_rate": 1.314814814814815e-05,
+      "loss": 1.4542,
+      "step": 98
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "eval_loss": 1.4141173362731934,
+      "eval_runtime": 5.1691,
+      "eval_samples_per_second": 32.114,
+      "eval_steps_per_second": 0.58,
+      "step": 98
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 1.47097647190094,
+      "learning_rate": 1.3055555555555557e-05,
+      "loss": 1.4428,
+      "step": 99
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.381595492362976,
+      "eval_runtime": 5.1554,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 99
+    },
+    {
+      "epoch": 4.040404040404041,
+      "grad_norm": 1.4813333749771118,
+      "learning_rate": 1.2962962962962964e-05,
+      "loss": 1.403,
+      "step": 100
+    },
+    {
+      "epoch": 4.040404040404041,
+      "eval_loss": 1.3522053956985474,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 100
+    },
+    {
+      "epoch": 4.08080808080808,
+      "grad_norm": 1.500218391418457,
+      "learning_rate": 1.2870370370370371e-05,
+      "loss": 1.3289,
+      "step": 101
+    },
+    {
+      "epoch": 4.08080808080808,
+      "eval_loss": 1.3220492601394653,
+      "eval_runtime": 5.1719,
+      "eval_samples_per_second": 32.096,
+      "eval_steps_per_second": 0.58,
+      "step": 101
+    },
+    {
+      "epoch": 4.121212121212121,
+      "grad_norm": 1.4614801406860352,
+      "learning_rate": 1.2777777777777777e-05,
+      "loss": 1.3151,
+      "step": 102
+    },
+    {
+      "epoch": 4.121212121212121,
+      "eval_loss": 1.2928389310836792,
+      "eval_runtime": 5.1718,
+      "eval_samples_per_second": 32.097,
+      "eval_steps_per_second": 0.58,
+      "step": 102
+    },
+    {
+      "epoch": 4.161616161616162,
+      "grad_norm": 1.4137208461761475,
+      "learning_rate": 1.2685185185185188e-05,
+      "loss": 1.2993,
+      "step": 103
+    },
+    {
+      "epoch": 4.161616161616162,
+      "eval_loss": 1.2663666009902954,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 103
+    },
+    {
+      "epoch": 4.202020202020202,
+      "grad_norm": 1.347014307975769,
+      "learning_rate": 1.2592592592592593e-05,
+      "loss": 1.2506,
+      "step": 104
+    },
+    {
+      "epoch": 4.202020202020202,
+      "eval_loss": 1.242059588432312,
+      "eval_runtime": 5.1713,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 104
+    },
+    {
+      "epoch": 4.242424242424242,
+      "grad_norm": 1.3283647298812866,
+      "learning_rate": 1.25e-05,
+      "loss": 1.2453,
+      "step": 105
+    },
+    {
+      "epoch": 4.242424242424242,
+      "eval_loss": 1.2196069955825806,
+      "eval_runtime": 5.1174,
+      "eval_samples_per_second": 32.438,
+      "eval_steps_per_second": 0.586,
+      "step": 105
+    },
+    {
+      "epoch": 4.282828282828283,
+      "grad_norm": 1.2796555757522583,
+      "learning_rate": 1.2407407407407408e-05,
+      "loss": 1.1805,
+      "step": 106
+    },
+    {
+      "epoch": 4.282828282828283,
+      "eval_loss": 1.1993300914764404,
+      "eval_runtime": 5.125,
+      "eval_samples_per_second": 32.39,
+      "eval_steps_per_second": 0.585,
+      "step": 106
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "grad_norm": 1.264662742614746,
+      "learning_rate": 1.2314814814814815e-05,
+      "loss": 1.1939,
+      "step": 107
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "eval_loss": 1.1796401739120483,
+      "eval_runtime": 5.154,
+      "eval_samples_per_second": 32.208,
+      "eval_steps_per_second": 0.582,
+      "step": 107
+    },
+    {
+      "epoch": 4.363636363636363,
+      "grad_norm": 1.2420271635055542,
+      "learning_rate": 1.2222222222222224e-05,
+      "loss": 1.1684,
+      "step": 108
+    },
+    {
+      "epoch": 4.363636363636363,
+      "eval_loss": 1.161789894104004,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 108
+    },
+    {
+      "epoch": 4.404040404040404,
+      "grad_norm": 1.2369439601898193,
+      "learning_rate": 1.2129629629629631e-05,
+      "loss": 1.1893,
+      "step": 109
+    },
+    {
+      "epoch": 4.404040404040404,
+      "eval_loss": 1.1432855129241943,
+      "eval_runtime": 5.1704,
+      "eval_samples_per_second": 32.106,
+      "eval_steps_per_second": 0.58,
+      "step": 109
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1795873641967773,
+      "learning_rate": 1.2037037037037039e-05,
+      "loss": 1.1554,
+      "step": 110
+    },
+    {
+      "epoch": 4.444444444444445,
+      "eval_loss": 1.126727819442749,
+      "eval_runtime": 5.1629,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 110
+    },
+    {
+      "epoch": 4.484848484848484,
+      "grad_norm": 1.1451513767242432,
+      "learning_rate": 1.1944444444444444e-05,
+      "loss": 1.1276,
+      "step": 111
+    },
+    {
+      "epoch": 4.484848484848484,
+      "eval_loss": 1.1112148761749268,
+      "eval_runtime": 5.1514,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 111
+    },
+    {
+      "epoch": 4.525252525252525,
+      "grad_norm": 1.124099612236023,
+      "learning_rate": 1.1851851851851852e-05,
+      "loss": 1.0972,
+      "step": 112
+    },
+    {
+      "epoch": 4.525252525252525,
+      "eval_loss": 1.0960901975631714,
+      "eval_runtime": 5.092,
+      "eval_samples_per_second": 32.6,
+      "eval_steps_per_second": 0.589,
+      "step": 112
+    },
+    {
+      "epoch": 4.565656565656566,
+      "grad_norm": 1.090491533279419,
+      "learning_rate": 1.1759259259259259e-05,
+      "loss": 1.0821,
+      "step": 113
+    },
+    {
+      "epoch": 4.565656565656566,
+      "eval_loss": 1.0813062191009521,
+      "eval_runtime": 5.0997,
+      "eval_samples_per_second": 32.551,
+      "eval_steps_per_second": 0.588,
+      "step": 113
+    },
+    {
+      "epoch": 4.606060606060606,
+      "grad_norm": 1.0799589157104492,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 1.038,
+      "step": 114
+    },
+    {
+      "epoch": 4.606060606060606,
+      "eval_loss": 1.0665714740753174,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.273,
+      "eval_steps_per_second": 0.583,
+      "step": 114
+    },
+    {
+      "epoch": 4.646464646464646,
+      "grad_norm": 1.0503891706466675,
+      "learning_rate": 1.1574074074074075e-05,
+      "loss": 1.0584,
+      "step": 115
+    },
+    {
+      "epoch": 4.646464646464646,
+      "eval_loss": 1.0545704364776611,
+      "eval_runtime": 5.1517,
+      "eval_samples_per_second": 32.222,
+      "eval_steps_per_second": 0.582,
+      "step": 115
+    },
+    {
+      "epoch": 4.686868686868687,
+      "grad_norm": 1.043385624885559,
+      "learning_rate": 1.1481481481481482e-05,
+      "loss": 1.0329,
+      "step": 116
+    },
+    {
+      "epoch": 4.686868686868687,
+      "eval_loss": 1.0427477359771729,
+      "eval_runtime": 5.1504,
+      "eval_samples_per_second": 32.23,
+      "eval_steps_per_second": 0.582,
+      "step": 116
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "grad_norm": 1.0548285245895386,
+      "learning_rate": 1.138888888888889e-05,
+      "loss": 1.0256,
+      "step": 117
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "eval_loss": 1.0287798643112183,
+      "eval_runtime": 5.1603,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 117
+    },
+    {
+      "epoch": 4.767676767676767,
+      "grad_norm": 1.0389068126678467,
+      "learning_rate": 1.1296296296296297e-05,
+      "loss": 1.0192,
+      "step": 118
+    },
+    {
+      "epoch": 4.767676767676767,
+      "eval_loss": 1.0174580812454224,
+      "eval_runtime": 5.1433,
+      "eval_samples_per_second": 32.275,
+      "eval_steps_per_second": 0.583,
+      "step": 118
+    },
+    {
+      "epoch": 4.808080808080808,
+      "grad_norm": 1.035844087600708,
+      "learning_rate": 1.1203703703703706e-05,
+      "loss": 1.0101,
+      "step": 119
+    },
+    {
+      "epoch": 4.808080808080808,
+      "eval_loss": 1.006226658821106,
+      "eval_runtime": 5.1693,
+      "eval_samples_per_second": 32.113,
+      "eval_steps_per_second": 0.58,
+      "step": 119
+    },
+    {
+      "epoch": 4.848484848484849,
+      "grad_norm": 1.050945520401001,
+      "learning_rate": 1.1111111111111113e-05,
+      "loss": 1.0403,
+      "step": 120
+    },
+    {
+      "epoch": 4.848484848484849,
+      "eval_loss": 0.9945307374000549,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.182,
+      "eval_steps_per_second": 0.582,
+      "step": 120
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 1.027295708656311,
+      "learning_rate": 1.1018518518518519e-05,
+      "loss": 1.0038,
+      "step": 121
+    },
+    {
+      "epoch": 4.888888888888889,
+      "eval_loss": 0.9830649495124817,
+      "eval_runtime": 5.1555,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 121
+    },
+    {
+      "epoch": 4.929292929292929,
+      "grad_norm": 1.035843014717102,
+      "learning_rate": 1.0925925925925926e-05,
+      "loss": 0.9544,
+      "step": 122
+    },
+    {
+      "epoch": 4.929292929292929,
+      "eval_loss": 0.9728271961212158,
+      "eval_runtime": 5.1521,
+      "eval_samples_per_second": 32.22,
+      "eval_steps_per_second": 0.582,
+      "step": 122
+    },
+    {
+      "epoch": 4.96969696969697,
+      "grad_norm": 1.050779938697815,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 0.9393,
+      "step": 123
+    },
+    {
+      "epoch": 4.96969696969697,
+      "eval_loss": 0.9609534740447998,
+      "eval_runtime": 5.1551,
+      "eval_samples_per_second": 32.201,
+      "eval_steps_per_second": 0.582,
+      "step": 123
+    },
+    {
+      "epoch": 5.01010101010101,
+      "grad_norm": 1.0694254636764526,
+      "learning_rate": 1.0740740740740742e-05,
+      "loss": 0.9578,
+      "step": 124
+    },
+    {
+      "epoch": 5.01010101010101,
+      "eval_loss": 0.9495998620986938,
+      "eval_runtime": 5.1502,
+      "eval_samples_per_second": 32.232,
+      "eval_steps_per_second": 0.583,
+      "step": 124
+    },
+    {
+      "epoch": 5.05050505050505,
+      "grad_norm": 1.0505897998809814,
+      "learning_rate": 1.064814814814815e-05,
+      "loss": 0.9228,
+      "step": 125
+    },
+    {
+      "epoch": 5.05050505050505,
+      "eval_loss": 0.9396913051605225,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 125
+    },
+    {
+      "epoch": 5.090909090909091,
+      "grad_norm": 1.082509160041809,
+      "learning_rate": 1.0555555555555557e-05,
+      "loss": 0.9583,
+      "step": 126
+    },
+    {
+      "epoch": 5.090909090909091,
+      "eval_loss": 0.9291126132011414,
+      "eval_runtime": 5.1661,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 126
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "grad_norm": 1.0709625482559204,
+      "learning_rate": 1.0462962962962964e-05,
+      "loss": 0.9163,
+      "step": 127
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "eval_loss": 0.9183955192565918,
+      "eval_runtime": 5.159,
+      "eval_samples_per_second": 32.177,
+      "eval_steps_per_second": 0.582,
+      "step": 127
+    },
+    {
+      "epoch": 5.171717171717171,
+      "grad_norm": 1.042419195175171,
+      "learning_rate": 1.037037037037037e-05,
+      "loss": 0.923,
+      "step": 128
+    },
+    {
+      "epoch": 5.171717171717171,
+      "eval_loss": 0.9072820544242859,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 128
+    },
+    {
+      "epoch": 5.212121212121212,
+      "grad_norm": 1.0583133697509766,
+      "learning_rate": 1.0277777777777777e-05,
+      "loss": 0.9328,
+      "step": 129
+    },
+    {
+      "epoch": 5.212121212121212,
+      "eval_loss": 0.897919237613678,
+      "eval_runtime": 5.15,
+      "eval_samples_per_second": 32.233,
+      "eval_steps_per_second": 0.583,
+      "step": 129
+    },
+    {
+      "epoch": 5.252525252525253,
+      "grad_norm": 1.0396648645401,
+      "learning_rate": 1.0185185185185186e-05,
+      "loss": 0.9001,
+      "step": 130
+    },
+    {
+      "epoch": 5.252525252525253,
+      "eval_loss": 0.8888950347900391,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 130
+    },
+    {
+      "epoch": 5.292929292929293,
+      "grad_norm": 1.0212734937667847,
+      "learning_rate": 1.0092592592592594e-05,
+      "loss": 0.8731,
+      "step": 131
+    },
+    {
+      "epoch": 5.292929292929293,
+      "eval_loss": 0.8788275718688965,
+      "eval_runtime": 5.1742,
+      "eval_samples_per_second": 32.082,
+      "eval_steps_per_second": 0.58,
+      "step": 131
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 1.0188337564468384,
+      "learning_rate": 1e-05,
+      "loss": 0.8328,
+      "step": 132
+    },
+    {
+      "epoch": 5.333333333333333,
+      "eval_loss": 0.8719209432601929,
+      "eval_runtime": 5.1515,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 132
+    },
+    {
+      "epoch": 5.373737373737374,
+      "grad_norm": 1.010650873184204,
+      "learning_rate": 9.907407407407408e-06,
+      "loss": 0.8401,
+      "step": 133
+    },
+    {
+      "epoch": 5.373737373737374,
+      "eval_loss": 0.8616573810577393,
+      "eval_runtime": 5.1695,
+      "eval_samples_per_second": 32.111,
+      "eval_steps_per_second": 0.58,
+      "step": 133
+    },
+    {
+      "epoch": 5.414141414141414,
+      "grad_norm": 0.9998605847358704,
+      "learning_rate": 9.814814814814815e-06,
+      "loss": 0.8664,
+      "step": 134
+    },
+    {
+      "epoch": 5.414141414141414,
+      "eval_loss": 0.8543541431427002,
+      "eval_runtime": 5.1642,
+      "eval_samples_per_second": 32.144,
+      "eval_steps_per_second": 0.581,
+      "step": 134
+    },
+    {
+      "epoch": 5.454545454545454,
+      "grad_norm": 0.9973117113113403,
+      "learning_rate": 9.722222222222223e-06,
+      "loss": 0.8414,
+      "step": 135
+    },
+    {
+      "epoch": 5.454545454545454,
+      "eval_loss": 0.8471528887748718,
+      "eval_runtime": 5.1595,
+      "eval_samples_per_second": 32.173,
+      "eval_steps_per_second": 0.581,
+      "step": 135
+    },
+    {
+      "epoch": 5.494949494949495,
+      "grad_norm": 1.0174371004104614,
+      "learning_rate": 9.62962962962963e-06,
+      "loss": 0.863,
+      "step": 136
+    },
+    {
+      "epoch": 5.494949494949495,
+      "eval_loss": 0.839015543460846,
+      "eval_runtime": 5.1632,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 136
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "grad_norm": 1.0131112337112427,
+      "learning_rate": 9.537037037037037e-06,
+      "loss": 0.8296,
+      "step": 137
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "eval_loss": 0.831695556640625,
+      "eval_runtime": 5.1598,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 137
+    },
+    {
+      "epoch": 5.575757575757576,
+      "grad_norm": 1.0262633562088013,
+      "learning_rate": 9.444444444444445e-06,
+      "loss": 0.854,
+      "step": 138
+    },
+    {
+      "epoch": 5.575757575757576,
+      "eval_loss": 0.8235994577407837,
+      "eval_runtime": 5.1621,
+      "eval_samples_per_second": 32.158,
+      "eval_steps_per_second": 0.581,
+      "step": 138
+    },
+    {
+      "epoch": 5.616161616161616,
+      "grad_norm": 1.0295422077178955,
+      "learning_rate": 9.351851851851854e-06,
+      "loss": 0.8194,
+      "step": 139
+    },
+    {
+      "epoch": 5.616161616161616,
+      "eval_loss": 0.8169118762016296,
+      "eval_runtime": 5.168,
+      "eval_samples_per_second": 32.121,
+      "eval_steps_per_second": 0.58,
+      "step": 139
+    },
+    {
+      "epoch": 5.656565656565657,
+      "grad_norm": 1.0281344652175903,
+      "learning_rate": 9.25925925925926e-06,
+      "loss": 0.7902,
+      "step": 140
+    },
+    {
+      "epoch": 5.656565656565657,
+      "eval_loss": 0.8098058104515076,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.183,
+      "eval_steps_per_second": 0.582,
+      "step": 140
+    },
+    {
+      "epoch": 5.696969696969697,
+      "grad_norm": 1.0528678894042969,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 0.7939,
+      "step": 141
+    },
+    {
+      "epoch": 5.696969696969697,
+      "eval_loss": 0.8031502366065979,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 141
+    },
+    {
+      "epoch": 5.737373737373737,
+      "grad_norm": 1.0525215864181519,
+      "learning_rate": 9.074074074074075e-06,
+      "loss": 0.8127,
+      "step": 142
+    },
+    {
+      "epoch": 5.737373737373737,
+      "eval_loss": 0.7957438230514526,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 142
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 1.0634738206863403,
+      "learning_rate": 8.981481481481483e-06,
+      "loss": 0.7831,
+      "step": 143
+    },
+    {
+      "epoch": 5.777777777777778,
+      "eval_loss": 0.7881508469581604,
+      "eval_runtime": 5.1489,
+      "eval_samples_per_second": 32.24,
+      "eval_steps_per_second": 0.583,
+      "step": 143
+    },
+    {
+      "epoch": 5.818181818181818,
+      "grad_norm": 1.0633102655410767,
+      "learning_rate": 8.888888888888888e-06,
+      "loss": 0.7511,
+      "step": 144
+    },
+    {
+      "epoch": 5.818181818181818,
+      "eval_loss": 0.7818319201469421,
+      "eval_runtime": 5.169,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 144
+    },
+    {
+      "epoch": 5.858585858585858,
+      "grad_norm": 1.053615689277649,
+      "learning_rate": 8.796296296296297e-06,
+      "loss": 0.75,
+      "step": 145
+    },
+    {
+      "epoch": 5.858585858585858,
+      "eval_loss": 0.7745649814605713,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 145
+    },
+    {
+      "epoch": 5.898989898989899,
+      "grad_norm": 1.0772610902786255,
+      "learning_rate": 8.703703703703705e-06,
+      "loss": 0.7929,
+      "step": 146
+    },
+    {
+      "epoch": 5.898989898989899,
+      "eval_loss": 0.7681305408477783,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.16,
+      "eval_steps_per_second": 0.581,
+      "step": 146
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "grad_norm": 1.0765084028244019,
+      "learning_rate": 8.611111111111112e-06,
+      "loss": 0.7642,
+      "step": 147
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "eval_loss": 0.7618209719657898,
+      "eval_runtime": 5.1689,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 147
+    },
+    {
+      "epoch": 5.97979797979798,
+      "grad_norm": 1.0824408531188965,
+      "learning_rate": 8.518518518518519e-06,
+      "loss": 0.7387,
+      "step": 148
+    },
+    {
+      "epoch": 5.97979797979798,
+      "eval_loss": 0.755460798740387,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.206,
+      "eval_steps_per_second": 0.582,
+      "step": 148
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.82851062784e+16,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-148/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dafca05b7a6d25266ace8b0db88af88ff18a0ac29cc5d191b665dd79638206
+size 5368

checkpoint-173/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-173/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-173/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:007988d6eb4bffa620ea75c5ce124ad07f927a19c3f1f9bf13c1d21b651865af
+size 35668592

checkpoint-173/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65dd1f0140947d73e2b9374108c573931d4cb4b6d16c2f3a6acf596a189d4f2b
+size 71379898

checkpoint-173/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccb9ee5cceebae985e71a16cb82773afb19733c17615f697c9c3531c13da9a83
+size 14244

checkpoint-173/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0622223831e2994229e748cdd0c50126a2ccae239757a3ed4079ddf8a85c0317
+size 1064

checkpoint-173/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-173/tokenizer_config.json

@@ -0,0 +1,131 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": true,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "32000": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32002": {
+      "content": "<|placeholder1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32003": {
+      "content": "<|placeholder2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32004": {
+      "content": "<|placeholder3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32005": {
+      "content": "<|placeholder4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32006": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "<|placeholder5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "<|placeholder6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "legacy": false,
+  "model_max_length": 4096,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-173/trainer_state.json

@@ -0,0 +1,2628 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.98989898989899,
+  "eval_steps": 1,
+  "global_step": 173,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04040404040404041,
+      "grad_norm": 2.088834762573242,
+      "learning_rate": 8.333333333333333e-07,
+      "loss": 5.4688,
+      "step": 1
+    },
+    {
+      "epoch": 0.04040404040404041,
+      "eval_loss": 5.470384120941162,
+      "eval_runtime": 4.8066,
+      "eval_samples_per_second": 34.536,
+      "eval_steps_per_second": 0.624,
+      "step": 1
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "grad_norm": 2.0961971282958984,
+      "learning_rate": 1.6666666666666667e-06,
+      "loss": 5.4731,
+      "step": 2
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "eval_loss": 5.4689249992370605,
+      "eval_runtime": 4.8593,
+      "eval_samples_per_second": 34.161,
+      "eval_steps_per_second": 0.617,
+      "step": 2
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "grad_norm": 2.0940349102020264,
+      "learning_rate": 2.5e-06,
+      "loss": 5.4803,
+      "step": 3
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "eval_loss": 5.467825412750244,
+      "eval_runtime": 4.9081,
+      "eval_samples_per_second": 33.822,
+      "eval_steps_per_second": 0.611,
+      "step": 3
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "grad_norm": 2.095174551010132,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 5.5119,
+      "step": 4
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "eval_loss": 5.465516567230225,
+      "eval_runtime": 4.8615,
+      "eval_samples_per_second": 34.146,
+      "eval_steps_per_second": 0.617,
+      "step": 4
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "grad_norm": 2.1109118461608887,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 5.4662,
+      "step": 5
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "eval_loss": 5.4626851081848145,
+      "eval_runtime": 4.844,
+      "eval_samples_per_second": 34.27,
+      "eval_steps_per_second": 0.619,
+      "step": 5
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "grad_norm": 2.1030592918395996,
+      "learning_rate": 5e-06,
+      "loss": 5.4691,
+      "step": 6
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "eval_loss": 5.45806884765625,
+      "eval_runtime": 4.8388,
+      "eval_samples_per_second": 34.306,
+      "eval_steps_per_second": 0.62,
+      "step": 6
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "grad_norm": 2.1202447414398193,
+      "learning_rate": 5.833333333333334e-06,
+      "loss": 5.4851,
+      "step": 7
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "eval_loss": 5.449892520904541,
+      "eval_runtime": 4.8431,
+      "eval_samples_per_second": 34.276,
+      "eval_steps_per_second": 0.619,
+      "step": 7
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "grad_norm": 2.090963363647461,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 5.4228,
+      "step": 8
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "eval_loss": 5.442515850067139,
+      "eval_runtime": 4.8605,
+      "eval_samples_per_second": 34.153,
+      "eval_steps_per_second": 0.617,
+      "step": 8
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "grad_norm": 2.112323760986328,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 5.4715,
+      "step": 9
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "eval_loss": 5.4345855712890625,
+      "eval_runtime": 4.8796,
+      "eval_samples_per_second": 34.019,
+      "eval_steps_per_second": 0.615,
+      "step": 9
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "grad_norm": 2.1456995010375977,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 5.4211,
+      "step": 10
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "eval_loss": 5.424546718597412,
+      "eval_runtime": 4.9274,
+      "eval_samples_per_second": 33.689,
+      "eval_steps_per_second": 0.609,
+      "step": 10
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 2.1602349281311035,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 5.4301,
+      "step": 11
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 5.41035795211792,
+      "eval_runtime": 4.9819,
+      "eval_samples_per_second": 33.32,
+      "eval_steps_per_second": 0.602,
+      "step": 11
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "grad_norm": 2.1762421131134033,
+      "learning_rate": 1e-05,
+      "loss": 5.4158,
+      "step": 12
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "eval_loss": 5.400452613830566,
+      "eval_runtime": 5.0737,
+      "eval_samples_per_second": 32.718,
+      "eval_steps_per_second": 0.591,
+      "step": 12
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "grad_norm": 2.2415969371795654,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 5.3986,
+      "step": 13
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "eval_loss": 5.382190227508545,
+      "eval_runtime": 4.9555,
+      "eval_samples_per_second": 33.498,
+      "eval_steps_per_second": 0.605,
+      "step": 13
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "grad_norm": 2.2669475078582764,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 5.428,
+      "step": 14
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "eval_loss": 5.3634443283081055,
+      "eval_runtime": 4.9029,
+      "eval_samples_per_second": 33.857,
+      "eval_steps_per_second": 0.612,
+      "step": 14
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "grad_norm": 2.262896776199341,
+      "learning_rate": 1.25e-05,
+      "loss": 5.3741,
+      "step": 15
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "eval_loss": 5.347131729125977,
+      "eval_runtime": 4.8934,
+      "eval_samples_per_second": 33.923,
+      "eval_steps_per_second": 0.613,
+      "step": 15
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "grad_norm": 2.3363702297210693,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 5.4134,
+      "step": 16
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "eval_loss": 5.321357250213623,
+      "eval_runtime": 4.9027,
+      "eval_samples_per_second": 33.859,
+      "eval_steps_per_second": 0.612,
+      "step": 16
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "grad_norm": 2.3906755447387695,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 5.3305,
+      "step": 17
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "eval_loss": 5.292910099029541,
+      "eval_runtime": 4.9268,
+      "eval_samples_per_second": 33.693,
+      "eval_steps_per_second": 0.609,
+      "step": 17
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "grad_norm": 2.44368839263916,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 5.3406,
+      "step": 18
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "eval_loss": 5.257543563842773,
+      "eval_runtime": 4.9839,
+      "eval_samples_per_second": 33.308,
+      "eval_steps_per_second": 0.602,
+      "step": 18
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "grad_norm": 2.4937825202941895,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 5.3134,
+      "step": 19
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "eval_loss": 5.225679874420166,
+      "eval_runtime": 5.1205,
+      "eval_samples_per_second": 32.419,
+      "eval_steps_per_second": 0.586,
+      "step": 19
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "grad_norm": 2.547194480895996,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 5.2303,
+      "step": 20
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "eval_loss": 5.183907985687256,
+      "eval_runtime": 5.1208,
+      "eval_samples_per_second": 32.417,
+      "eval_steps_per_second": 0.586,
+      "step": 20
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "grad_norm": 2.5578083992004395,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 5.1255,
+      "step": 21
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "eval_loss": 5.136340141296387,
+      "eval_runtime": 5.1161,
+      "eval_samples_per_second": 32.446,
+      "eval_steps_per_second": 0.586,
+      "step": 21
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 2.6553032398223877,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 5.1376,
+      "step": 22
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 5.085525035858154,
+      "eval_runtime": 5.1496,
+      "eval_samples_per_second": 32.236,
+      "eval_steps_per_second": 0.583,
+      "step": 22
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "grad_norm": 2.733492374420166,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 5.1051,
+      "step": 23
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "eval_loss": 5.0193634033203125,
+      "eval_runtime": 5.0473,
+      "eval_samples_per_second": 32.889,
+      "eval_steps_per_second": 0.594,
+      "step": 23
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "grad_norm": 2.765582799911499,
+      "learning_rate": 2e-05,
+      "loss": 5.0082,
+      "step": 24
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "eval_loss": 4.950736999511719,
+      "eval_runtime": 4.989,
+      "eval_samples_per_second": 33.273,
+      "eval_steps_per_second": 0.601,
+      "step": 24
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "grad_norm": 2.8943891525268555,
+      "learning_rate": 1.990740740740741e-05,
+      "loss": 4.9736,
+      "step": 25
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "eval_loss": 4.87933349609375,
+      "eval_runtime": 4.991,
+      "eval_samples_per_second": 33.26,
+      "eval_steps_per_second": 0.601,
+      "step": 25
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "grad_norm": 2.9802019596099854,
+      "learning_rate": 1.9814814814814816e-05,
+      "loss": 4.8728,
+      "step": 26
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "eval_loss": 4.796956539154053,
+      "eval_runtime": 5.0224,
+      "eval_samples_per_second": 33.052,
+      "eval_steps_per_second": 0.597,
+      "step": 26
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "grad_norm": 3.068443536758423,
+      "learning_rate": 1.9722222222222224e-05,
+      "loss": 4.8132,
+      "step": 27
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "eval_loss": 4.7025275230407715,
+      "eval_runtime": 5.1226,
+      "eval_samples_per_second": 32.406,
+      "eval_steps_per_second": 0.586,
+      "step": 27
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "grad_norm": 3.2245917320251465,
+      "learning_rate": 1.962962962962963e-05,
+      "loss": 4.7175,
+      "step": 28
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "eval_loss": 4.607891082763672,
+      "eval_runtime": 5.0927,
+      "eval_samples_per_second": 32.596,
+      "eval_steps_per_second": 0.589,
+      "step": 28
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "grad_norm": 3.3136157989501953,
+      "learning_rate": 1.9537037037037038e-05,
+      "loss": 4.6429,
+      "step": 29
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "eval_loss": 4.510694980621338,
+      "eval_runtime": 5.0136,
+      "eval_samples_per_second": 33.11,
+      "eval_steps_per_second": 0.598,
+      "step": 29
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "grad_norm": 3.329773426055908,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 4.4686,
+      "step": 30
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "eval_loss": 4.406045913696289,
+      "eval_runtime": 4.9753,
+      "eval_samples_per_second": 33.365,
+      "eval_steps_per_second": 0.603,
+      "step": 30
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "grad_norm": 3.5462117195129395,
+      "learning_rate": 1.9351851851851853e-05,
+      "loss": 4.427,
+      "step": 31
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "eval_loss": 4.309689998626709,
+      "eval_runtime": 4.9974,
+      "eval_samples_per_second": 33.217,
+      "eval_steps_per_second": 0.6,
+      "step": 31
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "grad_norm": 3.6190805435180664,
+      "learning_rate": 1.925925925925926e-05,
+      "loss": 4.2899,
+      "step": 32
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "eval_loss": 4.215596675872803,
+      "eval_runtime": 5.0637,
+      "eval_samples_per_second": 32.782,
+      "eval_steps_per_second": 0.592,
+      "step": 32
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 3.698317289352417,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 4.1752,
+      "step": 33
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 4.106834411621094,
+      "eval_runtime": 5.1651,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 33
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "grad_norm": 3.4807214736938477,
+      "learning_rate": 1.9074074074074075e-05,
+      "loss": 4.1277,
+      "step": 34
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "eval_loss": 4.001583099365234,
+      "eval_runtime": 5.0567,
+      "eval_samples_per_second": 32.828,
+      "eval_steps_per_second": 0.593,
+      "step": 34
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "grad_norm": 3.1380534172058105,
+      "learning_rate": 1.8981481481481482e-05,
+      "loss": 4.0069,
+      "step": 35
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "eval_loss": 3.9105026721954346,
+      "eval_runtime": 5.07,
+      "eval_samples_per_second": 32.741,
+      "eval_steps_per_second": 0.592,
+      "step": 35
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "grad_norm": 2.9076974391937256,
+      "learning_rate": 1.888888888888889e-05,
+      "loss": 3.8911,
+      "step": 36
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "eval_loss": 3.838275194168091,
+      "eval_runtime": 5.1402,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 36
+    },
+    {
+      "epoch": 1.494949494949495,
+      "grad_norm": 2.4755232334136963,
+      "learning_rate": 1.8796296296296296e-05,
+      "loss": 3.8522,
+      "step": 37
+    },
+    {
+      "epoch": 1.494949494949495,
+      "eval_loss": 3.776526689529419,
+      "eval_runtime": 5.1768,
+      "eval_samples_per_second": 32.066,
+      "eval_steps_per_second": 0.58,
+      "step": 37
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "grad_norm": 2.1607069969177246,
+      "learning_rate": 1.8703703703703707e-05,
+      "loss": 3.7828,
+      "step": 38
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "eval_loss": 3.7257139682769775,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 38
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "grad_norm": 1.9251853227615356,
+      "learning_rate": 1.8611111111111114e-05,
+      "loss": 3.7511,
+      "step": 39
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "eval_loss": 3.680651903152466,
+      "eval_runtime": 5.139,
+      "eval_samples_per_second": 32.302,
+      "eval_steps_per_second": 0.584,
+      "step": 39
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "grad_norm": 1.8553980588912964,
+      "learning_rate": 1.851851851851852e-05,
+      "loss": 3.7108,
+      "step": 40
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "eval_loss": 3.633517026901245,
+      "eval_runtime": 5.1036,
+      "eval_samples_per_second": 32.526,
+      "eval_steps_per_second": 0.588,
+      "step": 40
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "grad_norm": 1.831110954284668,
+      "learning_rate": 1.8425925925925926e-05,
+      "loss": 3.6165,
+      "step": 41
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "eval_loss": 3.5877223014831543,
+      "eval_runtime": 5.1287,
+      "eval_samples_per_second": 32.367,
+      "eval_steps_per_second": 0.585,
+      "step": 41
+    },
+    {
+      "epoch": 1.696969696969697,
+      "grad_norm": 1.8035303354263306,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 3.5555,
+      "step": 42
+    },
+    {
+      "epoch": 1.696969696969697,
+      "eval_loss": 3.5403287410736084,
+      "eval_runtime": 5.1001,
+      "eval_samples_per_second": 32.549,
+      "eval_steps_per_second": 0.588,
+      "step": 42
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "grad_norm": 1.7953453063964844,
+      "learning_rate": 1.8240740740740744e-05,
+      "loss": 3.5437,
+      "step": 43
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "eval_loss": 3.492953300476074,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 43
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 1.7807087898254395,
+      "learning_rate": 1.814814814814815e-05,
+      "loss": 3.4676,
+      "step": 44
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 3.443131446838379,
+      "eval_runtime": 5.1066,
+      "eval_samples_per_second": 32.507,
+      "eval_steps_per_second": 0.587,
+      "step": 44
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 1.8204630613327026,
+      "learning_rate": 1.8055555555555558e-05,
+      "loss": 3.4517,
+      "step": 45
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "eval_loss": 3.396712303161621,
+      "eval_runtime": 5.1475,
+      "eval_samples_per_second": 32.249,
+      "eval_steps_per_second": 0.583,
+      "step": 45
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "grad_norm": 1.8122252225875854,
+      "learning_rate": 1.7962962962962965e-05,
+      "loss": 3.4189,
+      "step": 46
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "eval_loss": 3.3434314727783203,
+      "eval_runtime": 5.0828,
+      "eval_samples_per_second": 32.659,
+      "eval_steps_per_second": 0.59,
+      "step": 46
+    },
+    {
+      "epoch": 1.898989898989899,
+      "grad_norm": 1.8433269262313843,
+      "learning_rate": 1.7870370370370373e-05,
+      "loss": 3.3537,
+      "step": 47
+    },
+    {
+      "epoch": 1.898989898989899,
+      "eval_loss": 3.294283628463745,
+      "eval_runtime": 5.1195,
+      "eval_samples_per_second": 32.425,
+      "eval_steps_per_second": 0.586,
+      "step": 47
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "grad_norm": 1.833533763885498,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 3.3034,
+      "step": 48
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "eval_loss": 3.2412631511688232,
+      "eval_runtime": 5.1404,
+      "eval_samples_per_second": 32.293,
+      "eval_steps_per_second": 0.584,
+      "step": 48
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "grad_norm": 1.823169469833374,
+      "learning_rate": 1.7685185185185187e-05,
+      "loss": 3.2457,
+      "step": 49
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "eval_loss": 3.1890854835510254,
+      "eval_runtime": 5.1319,
+      "eval_samples_per_second": 32.347,
+      "eval_steps_per_second": 0.585,
+      "step": 49
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "grad_norm": 1.796790361404419,
+      "learning_rate": 1.7592592592592595e-05,
+      "loss": 3.1915,
+      "step": 50
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "eval_loss": 3.1396946907043457,
+      "eval_runtime": 5.1262,
+      "eval_samples_per_second": 32.383,
+      "eval_steps_per_second": 0.585,
+      "step": 50
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "grad_norm": 1.7133023738861084,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 3.1475,
+      "step": 51
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "eval_loss": 3.093197822570801,
+      "eval_runtime": 5.141,
+      "eval_samples_per_second": 32.29,
+      "eval_steps_per_second": 0.584,
+      "step": 51
+    },
+    {
+      "epoch": 2.101010101010101,
+      "grad_norm": 1.637424111366272,
+      "learning_rate": 1.740740740740741e-05,
+      "loss": 3.0912,
+      "step": 52
+    },
+    {
+      "epoch": 2.101010101010101,
+      "eval_loss": 3.0486488342285156,
+      "eval_runtime": 5.1578,
+      "eval_samples_per_second": 32.185,
+      "eval_steps_per_second": 0.582,
+      "step": 52
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "grad_norm": 1.5733733177185059,
+      "learning_rate": 1.7314814814814816e-05,
+      "loss": 3.0493,
+      "step": 53
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "eval_loss": 3.007200002670288,
+      "eval_runtime": 5.1715,
+      "eval_samples_per_second": 32.099,
+      "eval_steps_per_second": 0.58,
+      "step": 53
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "grad_norm": 1.579100251197815,
+      "learning_rate": 1.7222222222222224e-05,
+      "loss": 3.0007,
+      "step": 54
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "eval_loss": 2.9646313190460205,
+      "eval_runtime": 5.1737,
+      "eval_samples_per_second": 32.085,
+      "eval_steps_per_second": 0.58,
+      "step": 54
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.5558359622955322,
+      "learning_rate": 1.712962962962963e-05,
+      "loss": 2.955,
+      "step": 55
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 2.9225423336029053,
+      "eval_runtime": 5.1649,
+      "eval_samples_per_second": 32.14,
+      "eval_steps_per_second": 0.581,
+      "step": 55
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "grad_norm": 1.5695600509643555,
+      "learning_rate": 1.7037037037037038e-05,
+      "loss": 2.9242,
+      "step": 56
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "eval_loss": 2.883842706680298,
+      "eval_runtime": 5.1565,
+      "eval_samples_per_second": 32.192,
+      "eval_steps_per_second": 0.582,
+      "step": 56
+    },
+    {
+      "epoch": 2.303030303030303,
+      "grad_norm": 1.5856674909591675,
+      "learning_rate": 1.6944444444444446e-05,
+      "loss": 2.8626,
+      "step": 57
+    },
+    {
+      "epoch": 2.303030303030303,
+      "eval_loss": 2.8435447216033936,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 57
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "grad_norm": 1.5780515670776367,
+      "learning_rate": 1.6851851851851853e-05,
+      "loss": 2.8522,
+      "step": 58
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "eval_loss": 2.8028392791748047,
+      "eval_runtime": 5.1666,
+      "eval_samples_per_second": 32.129,
+      "eval_steps_per_second": 0.581,
+      "step": 58
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "grad_norm": 1.5944206714630127,
+      "learning_rate": 1.675925925925926e-05,
+      "loss": 2.8222,
+      "step": 59
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "eval_loss": 2.7607803344726562,
+      "eval_runtime": 5.1662,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 59
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "grad_norm": 1.6033095121383667,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.7291,
+      "step": 60
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "eval_loss": 2.7191247940063477,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 60
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "grad_norm": 1.621768593788147,
+      "learning_rate": 1.6574074074074075e-05,
+      "loss": 2.7046,
+      "step": 61
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "eval_loss": 2.6788933277130127,
+      "eval_runtime": 5.1728,
+      "eval_samples_per_second": 32.091,
+      "eval_steps_per_second": 0.58,
+      "step": 61
+    },
+    {
+      "epoch": 2.505050505050505,
+      "grad_norm": 1.6213834285736084,
+      "learning_rate": 1.6481481481481482e-05,
+      "loss": 2.6548,
+      "step": 62
+    },
+    {
+      "epoch": 2.505050505050505,
+      "eval_loss": 2.6340527534484863,
+      "eval_runtime": 5.1628,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 62
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "grad_norm": 1.607869267463684,
+      "learning_rate": 1.638888888888889e-05,
+      "loss": 2.6232,
+      "step": 63
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "eval_loss": 2.594001531600952,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 63
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "grad_norm": 1.583896279335022,
+      "learning_rate": 1.6296296296296297e-05,
+      "loss": 2.6068,
+      "step": 64
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "eval_loss": 2.5528736114501953,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 64
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "grad_norm": 1.5254372358322144,
+      "learning_rate": 1.6203703703703707e-05,
+      "loss": 2.5629,
+      "step": 65
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "eval_loss": 2.5128839015960693,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 65
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 1.4735966920852661,
+      "learning_rate": 1.6111111111111115e-05,
+      "loss": 2.523,
+      "step": 66
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 2.4759180545806885,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 66
+    },
+    {
+      "epoch": 2.707070707070707,
+      "grad_norm": 1.4135326147079468,
+      "learning_rate": 1.601851851851852e-05,
+      "loss": 2.5069,
+      "step": 67
+    },
+    {
+      "epoch": 2.707070707070707,
+      "eval_loss": 2.4382143020629883,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.274,
+      "eval_steps_per_second": 0.583,
+      "step": 67
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "grad_norm": 1.3859189748764038,
+      "learning_rate": 1.5925925925925926e-05,
+      "loss": 2.449,
+      "step": 68
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "eval_loss": 2.3999476432800293,
+      "eval_runtime": 5.0261,
+      "eval_samples_per_second": 33.028,
+      "eval_steps_per_second": 0.597,
+      "step": 68
+    },
+    {
+      "epoch": 2.787878787878788,
+      "grad_norm": 1.3678818941116333,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 2.3977,
+      "step": 69
+    },
+    {
+      "epoch": 2.787878787878788,
+      "eval_loss": 2.365516424179077,
+      "eval_runtime": 4.9945,
+      "eval_samples_per_second": 33.237,
+      "eval_steps_per_second": 0.601,
+      "step": 69
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "grad_norm": 1.372074842453003,
+      "learning_rate": 1.5740740740740744e-05,
+      "loss": 2.3556,
+      "step": 70
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "eval_loss": 2.3320348262786865,
+      "eval_runtime": 5.0457,
+      "eval_samples_per_second": 32.899,
+      "eval_steps_per_second": 0.595,
+      "step": 70
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "grad_norm": 1.355190634727478,
+      "learning_rate": 1.564814814814815e-05,
+      "loss": 2.3269,
+      "step": 71
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "eval_loss": 2.2980406284332275,
+      "eval_runtime": 5.1448,
+      "eval_samples_per_second": 32.265,
+      "eval_steps_per_second": 0.583,
+      "step": 71
+    },
+    {
+      "epoch": 2.909090909090909,
+      "grad_norm": 1.34852135181427,
+      "learning_rate": 1.555555555555556e-05,
+      "loss": 2.2962,
+      "step": 72
+    },
+    {
+      "epoch": 2.909090909090909,
+      "eval_loss": 2.262707233428955,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 72
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "grad_norm": 1.3559976816177368,
+      "learning_rate": 1.5462962962962966e-05,
+      "loss": 2.2534,
+      "step": 73
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "eval_loss": 2.2291126251220703,
+      "eval_runtime": 5.0794,
+      "eval_samples_per_second": 32.681,
+      "eval_steps_per_second": 0.591,
+      "step": 73
+    },
+    {
+      "epoch": 2.98989898989899,
+      "grad_norm": 1.3566159009933472,
+      "learning_rate": 1.537037037037037e-05,
+      "loss": 2.2061,
+      "step": 74
+    },
+    {
+      "epoch": 2.98989898989899,
+      "eval_loss": 2.193363904953003,
+      "eval_runtime": 5.0616,
+      "eval_samples_per_second": 32.796,
+      "eval_steps_per_second": 0.593,
+      "step": 74
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "grad_norm": 1.3466676473617554,
+      "learning_rate": 1.5277777777777777e-05,
+      "loss": 2.1815,
+      "step": 75
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "eval_loss": 2.159188985824585,
+      "eval_runtime": 5.0978,
+      "eval_samples_per_second": 32.563,
+      "eval_steps_per_second": 0.588,
+      "step": 75
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "grad_norm": 1.354913592338562,
+      "learning_rate": 1.5185185185185187e-05,
+      "loss": 2.1622,
+      "step": 76
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "eval_loss": 2.124568462371826,
+      "eval_runtime": 5.1439,
+      "eval_samples_per_second": 32.271,
+      "eval_steps_per_second": 0.583,
+      "step": 76
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 1.360551357269287,
+      "learning_rate": 1.5092592592592595e-05,
+      "loss": 2.1079,
+      "step": 77
+    },
+    {
+      "epoch": 3.111111111111111,
+      "eval_loss": 2.092242956161499,
+      "eval_runtime": 5.1333,
+      "eval_samples_per_second": 32.338,
+      "eval_steps_per_second": 0.584,
+      "step": 77
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "grad_norm": 1.3474143743515015,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 2.1036,
+      "step": 78
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "eval_loss": 2.0584089756011963,
+      "eval_runtime": 5.1757,
+      "eval_samples_per_second": 32.073,
+      "eval_steps_per_second": 0.58,
+      "step": 78
+    },
+    {
+      "epoch": 3.191919191919192,
+      "grad_norm": 1.368388295173645,
+      "learning_rate": 1.4907407407407408e-05,
+      "loss": 2.0527,
+      "step": 79
+    },
+    {
+      "epoch": 3.191919191919192,
+      "eval_loss": 2.0241854190826416,
+      "eval_runtime": 5.1747,
+      "eval_samples_per_second": 32.079,
+      "eval_steps_per_second": 0.58,
+      "step": 79
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "grad_norm": 1.3582555055618286,
+      "learning_rate": 1.4814814814814815e-05,
+      "loss": 1.9958,
+      "step": 80
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "eval_loss": 1.990015983581543,
+      "eval_runtime": 5.1542,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 80
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "grad_norm": 1.3820135593414307,
+      "learning_rate": 1.4722222222222224e-05,
+      "loss": 1.9878,
+      "step": 81
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "eval_loss": 1.9560723304748535,
+      "eval_runtime": 5.176,
+      "eval_samples_per_second": 32.071,
+      "eval_steps_per_second": 0.58,
+      "step": 81
+    },
+    {
+      "epoch": 3.313131313131313,
+      "grad_norm": 1.4004285335540771,
+      "learning_rate": 1.4629629629629631e-05,
+      "loss": 1.9533,
+      "step": 82
+    },
+    {
+      "epoch": 3.313131313131313,
+      "eval_loss": 1.9231067895889282,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 82
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "grad_norm": 1.4260685443878174,
+      "learning_rate": 1.4537037037037039e-05,
+      "loss": 1.9066,
+      "step": 83
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "eval_loss": 1.8882259130477905,
+      "eval_runtime": 5.1573,
+      "eval_samples_per_second": 32.187,
+      "eval_steps_per_second": 0.582,
+      "step": 83
+    },
+    {
+      "epoch": 3.393939393939394,
+      "grad_norm": 1.4351316690444946,
+      "learning_rate": 1.4444444444444446e-05,
+      "loss": 1.8803,
+      "step": 84
+    },
+    {
+      "epoch": 3.393939393939394,
+      "eval_loss": 1.8546966314315796,
+      "eval_runtime": 5.1748,
+      "eval_samples_per_second": 32.078,
+      "eval_steps_per_second": 0.58,
+      "step": 84
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "grad_norm": 1.4154078960418701,
+      "learning_rate": 1.4351851851851853e-05,
+      "loss": 1.8668,
+      "step": 85
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "eval_loss": 1.820953369140625,
+      "eval_runtime": 5.1325,
+      "eval_samples_per_second": 32.343,
+      "eval_steps_per_second": 0.585,
+      "step": 85
+    },
+    {
+      "epoch": 3.474747474747475,
+      "grad_norm": 1.4371109008789062,
+      "learning_rate": 1.4259259259259259e-05,
+      "loss": 1.822,
+      "step": 86
+    },
+    {
+      "epoch": 3.474747474747475,
+      "eval_loss": 1.7877326011657715,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 86
+    },
+    {
+      "epoch": 3.515151515151515,
+      "grad_norm": 1.4090356826782227,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 1.7796,
+      "step": 87
+    },
+    {
+      "epoch": 3.515151515151515,
+      "eval_loss": 1.7542084455490112,
+      "eval_runtime": 5.1712,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 87
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 1.382804036140442,
+      "learning_rate": 1.4074074074074075e-05,
+      "loss": 1.7457,
+      "step": 88
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "eval_loss": 1.7219429016113281,
+      "eval_runtime": 5.165,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 88
+    },
+    {
+      "epoch": 3.595959595959596,
+      "grad_norm": 1.4041632413864136,
+      "learning_rate": 1.3981481481481482e-05,
+      "loss": 1.7043,
+      "step": 89
+    },
+    {
+      "epoch": 3.595959595959596,
+      "eval_loss": 1.689993977546692,
+      "eval_runtime": 5.1401,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 89
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.3862537145614624,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 1.6742,
+      "step": 90
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "eval_loss": 1.659338116645813,
+      "eval_runtime": 5.1347,
+      "eval_samples_per_second": 32.329,
+      "eval_steps_per_second": 0.584,
+      "step": 90
+    },
+    {
+      "epoch": 3.676767676767677,
+      "grad_norm": 1.3727890253067017,
+      "learning_rate": 1.3796296296296297e-05,
+      "loss": 1.6502,
+      "step": 91
+    },
+    {
+      "epoch": 3.676767676767677,
+      "eval_loss": 1.628204345703125,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 91
+    },
+    {
+      "epoch": 3.717171717171717,
+      "grad_norm": 1.400058388710022,
+      "learning_rate": 1.3703703703703706e-05,
+      "loss": 1.6396,
+      "step": 92
+    },
+    {
+      "epoch": 3.717171717171717,
+      "eval_loss": 1.5992398262023926,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 92
+    },
+    {
+      "epoch": 3.757575757575758,
+      "grad_norm": 1.4236724376678467,
+      "learning_rate": 1.3611111111111113e-05,
+      "loss": 1.6123,
+      "step": 93
+    },
+    {
+      "epoch": 3.757575757575758,
+      "eval_loss": 1.5681532621383667,
+      "eval_runtime": 5.157,
+      "eval_samples_per_second": 32.189,
+      "eval_steps_per_second": 0.582,
+      "step": 93
+    },
+    {
+      "epoch": 3.797979797979798,
+      "grad_norm": 1.4351081848144531,
+      "learning_rate": 1.351851851851852e-05,
+      "loss": 1.5512,
+      "step": 94
+    },
+    {
+      "epoch": 3.797979797979798,
+      "eval_loss": 1.5355143547058105,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 94
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "grad_norm": 1.4533002376556396,
+      "learning_rate": 1.3425925925925926e-05,
+      "loss": 1.5697,
+      "step": 95
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "eval_loss": 1.506730318069458,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 95
+    },
+    {
+      "epoch": 3.878787878787879,
+      "grad_norm": 1.4752202033996582,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.495,
+      "step": 96
+    },
+    {
+      "epoch": 3.878787878787879,
+      "eval_loss": 1.4759498834609985,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 96
+    },
+    {
+      "epoch": 3.919191919191919,
+      "grad_norm": 1.4492502212524414,
+      "learning_rate": 1.3240740740740742e-05,
+      "loss": 1.4681,
+      "step": 97
+    },
+    {
+      "epoch": 3.919191919191919,
+      "eval_loss": 1.4438410997390747,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 97
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "grad_norm": 1.4566094875335693,
+      "learning_rate": 1.314814814814815e-05,
+      "loss": 1.4542,
+      "step": 98
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "eval_loss": 1.4141173362731934,
+      "eval_runtime": 5.1691,
+      "eval_samples_per_second": 32.114,
+      "eval_steps_per_second": 0.58,
+      "step": 98
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 1.47097647190094,
+      "learning_rate": 1.3055555555555557e-05,
+      "loss": 1.4428,
+      "step": 99
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.381595492362976,
+      "eval_runtime": 5.1554,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 99
+    },
+    {
+      "epoch": 4.040404040404041,
+      "grad_norm": 1.4813333749771118,
+      "learning_rate": 1.2962962962962964e-05,
+      "loss": 1.403,
+      "step": 100
+    },
+    {
+      "epoch": 4.040404040404041,
+      "eval_loss": 1.3522053956985474,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 100
+    },
+    {
+      "epoch": 4.08080808080808,
+      "grad_norm": 1.500218391418457,
+      "learning_rate": 1.2870370370370371e-05,
+      "loss": 1.3289,
+      "step": 101
+    },
+    {
+      "epoch": 4.08080808080808,
+      "eval_loss": 1.3220492601394653,
+      "eval_runtime": 5.1719,
+      "eval_samples_per_second": 32.096,
+      "eval_steps_per_second": 0.58,
+      "step": 101
+    },
+    {
+      "epoch": 4.121212121212121,
+      "grad_norm": 1.4614801406860352,
+      "learning_rate": 1.2777777777777777e-05,
+      "loss": 1.3151,
+      "step": 102
+    },
+    {
+      "epoch": 4.121212121212121,
+      "eval_loss": 1.2928389310836792,
+      "eval_runtime": 5.1718,
+      "eval_samples_per_second": 32.097,
+      "eval_steps_per_second": 0.58,
+      "step": 102
+    },
+    {
+      "epoch": 4.161616161616162,
+      "grad_norm": 1.4137208461761475,
+      "learning_rate": 1.2685185185185188e-05,
+      "loss": 1.2993,
+      "step": 103
+    },
+    {
+      "epoch": 4.161616161616162,
+      "eval_loss": 1.2663666009902954,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 103
+    },
+    {
+      "epoch": 4.202020202020202,
+      "grad_norm": 1.347014307975769,
+      "learning_rate": 1.2592592592592593e-05,
+      "loss": 1.2506,
+      "step": 104
+    },
+    {
+      "epoch": 4.202020202020202,
+      "eval_loss": 1.242059588432312,
+      "eval_runtime": 5.1713,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 104
+    },
+    {
+      "epoch": 4.242424242424242,
+      "grad_norm": 1.3283647298812866,
+      "learning_rate": 1.25e-05,
+      "loss": 1.2453,
+      "step": 105
+    },
+    {
+      "epoch": 4.242424242424242,
+      "eval_loss": 1.2196069955825806,
+      "eval_runtime": 5.1174,
+      "eval_samples_per_second": 32.438,
+      "eval_steps_per_second": 0.586,
+      "step": 105
+    },
+    {
+      "epoch": 4.282828282828283,
+      "grad_norm": 1.2796555757522583,
+      "learning_rate": 1.2407407407407408e-05,
+      "loss": 1.1805,
+      "step": 106
+    },
+    {
+      "epoch": 4.282828282828283,
+      "eval_loss": 1.1993300914764404,
+      "eval_runtime": 5.125,
+      "eval_samples_per_second": 32.39,
+      "eval_steps_per_second": 0.585,
+      "step": 106
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "grad_norm": 1.264662742614746,
+      "learning_rate": 1.2314814814814815e-05,
+      "loss": 1.1939,
+      "step": 107
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "eval_loss": 1.1796401739120483,
+      "eval_runtime": 5.154,
+      "eval_samples_per_second": 32.208,
+      "eval_steps_per_second": 0.582,
+      "step": 107
+    },
+    {
+      "epoch": 4.363636363636363,
+      "grad_norm": 1.2420271635055542,
+      "learning_rate": 1.2222222222222224e-05,
+      "loss": 1.1684,
+      "step": 108
+    },
+    {
+      "epoch": 4.363636363636363,
+      "eval_loss": 1.161789894104004,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 108
+    },
+    {
+      "epoch": 4.404040404040404,
+      "grad_norm": 1.2369439601898193,
+      "learning_rate": 1.2129629629629631e-05,
+      "loss": 1.1893,
+      "step": 109
+    },
+    {
+      "epoch": 4.404040404040404,
+      "eval_loss": 1.1432855129241943,
+      "eval_runtime": 5.1704,
+      "eval_samples_per_second": 32.106,
+      "eval_steps_per_second": 0.58,
+      "step": 109
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1795873641967773,
+      "learning_rate": 1.2037037037037039e-05,
+      "loss": 1.1554,
+      "step": 110
+    },
+    {
+      "epoch": 4.444444444444445,
+      "eval_loss": 1.126727819442749,
+      "eval_runtime": 5.1629,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 110
+    },
+    {
+      "epoch": 4.484848484848484,
+      "grad_norm": 1.1451513767242432,
+      "learning_rate": 1.1944444444444444e-05,
+      "loss": 1.1276,
+      "step": 111
+    },
+    {
+      "epoch": 4.484848484848484,
+      "eval_loss": 1.1112148761749268,
+      "eval_runtime": 5.1514,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 111
+    },
+    {
+      "epoch": 4.525252525252525,
+      "grad_norm": 1.124099612236023,
+      "learning_rate": 1.1851851851851852e-05,
+      "loss": 1.0972,
+      "step": 112
+    },
+    {
+      "epoch": 4.525252525252525,
+      "eval_loss": 1.0960901975631714,
+      "eval_runtime": 5.092,
+      "eval_samples_per_second": 32.6,
+      "eval_steps_per_second": 0.589,
+      "step": 112
+    },
+    {
+      "epoch": 4.565656565656566,
+      "grad_norm": 1.090491533279419,
+      "learning_rate": 1.1759259259259259e-05,
+      "loss": 1.0821,
+      "step": 113
+    },
+    {
+      "epoch": 4.565656565656566,
+      "eval_loss": 1.0813062191009521,
+      "eval_runtime": 5.0997,
+      "eval_samples_per_second": 32.551,
+      "eval_steps_per_second": 0.588,
+      "step": 113
+    },
+    {
+      "epoch": 4.606060606060606,
+      "grad_norm": 1.0799589157104492,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 1.038,
+      "step": 114
+    },
+    {
+      "epoch": 4.606060606060606,
+      "eval_loss": 1.0665714740753174,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.273,
+      "eval_steps_per_second": 0.583,
+      "step": 114
+    },
+    {
+      "epoch": 4.646464646464646,
+      "grad_norm": 1.0503891706466675,
+      "learning_rate": 1.1574074074074075e-05,
+      "loss": 1.0584,
+      "step": 115
+    },
+    {
+      "epoch": 4.646464646464646,
+      "eval_loss": 1.0545704364776611,
+      "eval_runtime": 5.1517,
+      "eval_samples_per_second": 32.222,
+      "eval_steps_per_second": 0.582,
+      "step": 115
+    },
+    {
+      "epoch": 4.686868686868687,
+      "grad_norm": 1.043385624885559,
+      "learning_rate": 1.1481481481481482e-05,
+      "loss": 1.0329,
+      "step": 116
+    },
+    {
+      "epoch": 4.686868686868687,
+      "eval_loss": 1.0427477359771729,
+      "eval_runtime": 5.1504,
+      "eval_samples_per_second": 32.23,
+      "eval_steps_per_second": 0.582,
+      "step": 116
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "grad_norm": 1.0548285245895386,
+      "learning_rate": 1.138888888888889e-05,
+      "loss": 1.0256,
+      "step": 117
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "eval_loss": 1.0287798643112183,
+      "eval_runtime": 5.1603,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 117
+    },
+    {
+      "epoch": 4.767676767676767,
+      "grad_norm": 1.0389068126678467,
+      "learning_rate": 1.1296296296296297e-05,
+      "loss": 1.0192,
+      "step": 118
+    },
+    {
+      "epoch": 4.767676767676767,
+      "eval_loss": 1.0174580812454224,
+      "eval_runtime": 5.1433,
+      "eval_samples_per_second": 32.275,
+      "eval_steps_per_second": 0.583,
+      "step": 118
+    },
+    {
+      "epoch": 4.808080808080808,
+      "grad_norm": 1.035844087600708,
+      "learning_rate": 1.1203703703703706e-05,
+      "loss": 1.0101,
+      "step": 119
+    },
+    {
+      "epoch": 4.808080808080808,
+      "eval_loss": 1.006226658821106,
+      "eval_runtime": 5.1693,
+      "eval_samples_per_second": 32.113,
+      "eval_steps_per_second": 0.58,
+      "step": 119
+    },
+    {
+      "epoch": 4.848484848484849,
+      "grad_norm": 1.050945520401001,
+      "learning_rate": 1.1111111111111113e-05,
+      "loss": 1.0403,
+      "step": 120
+    },
+    {
+      "epoch": 4.848484848484849,
+      "eval_loss": 0.9945307374000549,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.182,
+      "eval_steps_per_second": 0.582,
+      "step": 120
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 1.027295708656311,
+      "learning_rate": 1.1018518518518519e-05,
+      "loss": 1.0038,
+      "step": 121
+    },
+    {
+      "epoch": 4.888888888888889,
+      "eval_loss": 0.9830649495124817,
+      "eval_runtime": 5.1555,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 121
+    },
+    {
+      "epoch": 4.929292929292929,
+      "grad_norm": 1.035843014717102,
+      "learning_rate": 1.0925925925925926e-05,
+      "loss": 0.9544,
+      "step": 122
+    },
+    {
+      "epoch": 4.929292929292929,
+      "eval_loss": 0.9728271961212158,
+      "eval_runtime": 5.1521,
+      "eval_samples_per_second": 32.22,
+      "eval_steps_per_second": 0.582,
+      "step": 122
+    },
+    {
+      "epoch": 4.96969696969697,
+      "grad_norm": 1.050779938697815,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 0.9393,
+      "step": 123
+    },
+    {
+      "epoch": 4.96969696969697,
+      "eval_loss": 0.9609534740447998,
+      "eval_runtime": 5.1551,
+      "eval_samples_per_second": 32.201,
+      "eval_steps_per_second": 0.582,
+      "step": 123
+    },
+    {
+      "epoch": 5.01010101010101,
+      "grad_norm": 1.0694254636764526,
+      "learning_rate": 1.0740740740740742e-05,
+      "loss": 0.9578,
+      "step": 124
+    },
+    {
+      "epoch": 5.01010101010101,
+      "eval_loss": 0.9495998620986938,
+      "eval_runtime": 5.1502,
+      "eval_samples_per_second": 32.232,
+      "eval_steps_per_second": 0.583,
+      "step": 124
+    },
+    {
+      "epoch": 5.05050505050505,
+      "grad_norm": 1.0505897998809814,
+      "learning_rate": 1.064814814814815e-05,
+      "loss": 0.9228,
+      "step": 125
+    },
+    {
+      "epoch": 5.05050505050505,
+      "eval_loss": 0.9396913051605225,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 125
+    },
+    {
+      "epoch": 5.090909090909091,
+      "grad_norm": 1.082509160041809,
+      "learning_rate": 1.0555555555555557e-05,
+      "loss": 0.9583,
+      "step": 126
+    },
+    {
+      "epoch": 5.090909090909091,
+      "eval_loss": 0.9291126132011414,
+      "eval_runtime": 5.1661,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 126
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "grad_norm": 1.0709625482559204,
+      "learning_rate": 1.0462962962962964e-05,
+      "loss": 0.9163,
+      "step": 127
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "eval_loss": 0.9183955192565918,
+      "eval_runtime": 5.159,
+      "eval_samples_per_second": 32.177,
+      "eval_steps_per_second": 0.582,
+      "step": 127
+    },
+    {
+      "epoch": 5.171717171717171,
+      "grad_norm": 1.042419195175171,
+      "learning_rate": 1.037037037037037e-05,
+      "loss": 0.923,
+      "step": 128
+    },
+    {
+      "epoch": 5.171717171717171,
+      "eval_loss": 0.9072820544242859,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 128
+    },
+    {
+      "epoch": 5.212121212121212,
+      "grad_norm": 1.0583133697509766,
+      "learning_rate": 1.0277777777777777e-05,
+      "loss": 0.9328,
+      "step": 129
+    },
+    {
+      "epoch": 5.212121212121212,
+      "eval_loss": 0.897919237613678,
+      "eval_runtime": 5.15,
+      "eval_samples_per_second": 32.233,
+      "eval_steps_per_second": 0.583,
+      "step": 129
+    },
+    {
+      "epoch": 5.252525252525253,
+      "grad_norm": 1.0396648645401,
+      "learning_rate": 1.0185185185185186e-05,
+      "loss": 0.9001,
+      "step": 130
+    },
+    {
+      "epoch": 5.252525252525253,
+      "eval_loss": 0.8888950347900391,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 130
+    },
+    {
+      "epoch": 5.292929292929293,
+      "grad_norm": 1.0212734937667847,
+      "learning_rate": 1.0092592592592594e-05,
+      "loss": 0.8731,
+      "step": 131
+    },
+    {
+      "epoch": 5.292929292929293,
+      "eval_loss": 0.8788275718688965,
+      "eval_runtime": 5.1742,
+      "eval_samples_per_second": 32.082,
+      "eval_steps_per_second": 0.58,
+      "step": 131
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 1.0188337564468384,
+      "learning_rate": 1e-05,
+      "loss": 0.8328,
+      "step": 132
+    },
+    {
+      "epoch": 5.333333333333333,
+      "eval_loss": 0.8719209432601929,
+      "eval_runtime": 5.1515,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 132
+    },
+    {
+      "epoch": 5.373737373737374,
+      "grad_norm": 1.010650873184204,
+      "learning_rate": 9.907407407407408e-06,
+      "loss": 0.8401,
+      "step": 133
+    },
+    {
+      "epoch": 5.373737373737374,
+      "eval_loss": 0.8616573810577393,
+      "eval_runtime": 5.1695,
+      "eval_samples_per_second": 32.111,
+      "eval_steps_per_second": 0.58,
+      "step": 133
+    },
+    {
+      "epoch": 5.414141414141414,
+      "grad_norm": 0.9998605847358704,
+      "learning_rate": 9.814814814814815e-06,
+      "loss": 0.8664,
+      "step": 134
+    },
+    {
+      "epoch": 5.414141414141414,
+      "eval_loss": 0.8543541431427002,
+      "eval_runtime": 5.1642,
+      "eval_samples_per_second": 32.144,
+      "eval_steps_per_second": 0.581,
+      "step": 134
+    },
+    {
+      "epoch": 5.454545454545454,
+      "grad_norm": 0.9973117113113403,
+      "learning_rate": 9.722222222222223e-06,
+      "loss": 0.8414,
+      "step": 135
+    },
+    {
+      "epoch": 5.454545454545454,
+      "eval_loss": 0.8471528887748718,
+      "eval_runtime": 5.1595,
+      "eval_samples_per_second": 32.173,
+      "eval_steps_per_second": 0.581,
+      "step": 135
+    },
+    {
+      "epoch": 5.494949494949495,
+      "grad_norm": 1.0174371004104614,
+      "learning_rate": 9.62962962962963e-06,
+      "loss": 0.863,
+      "step": 136
+    },
+    {
+      "epoch": 5.494949494949495,
+      "eval_loss": 0.839015543460846,
+      "eval_runtime": 5.1632,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 136
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "grad_norm": 1.0131112337112427,
+      "learning_rate": 9.537037037037037e-06,
+      "loss": 0.8296,
+      "step": 137
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "eval_loss": 0.831695556640625,
+      "eval_runtime": 5.1598,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 137
+    },
+    {
+      "epoch": 5.575757575757576,
+      "grad_norm": 1.0262633562088013,
+      "learning_rate": 9.444444444444445e-06,
+      "loss": 0.854,
+      "step": 138
+    },
+    {
+      "epoch": 5.575757575757576,
+      "eval_loss": 0.8235994577407837,
+      "eval_runtime": 5.1621,
+      "eval_samples_per_second": 32.158,
+      "eval_steps_per_second": 0.581,
+      "step": 138
+    },
+    {
+      "epoch": 5.616161616161616,
+      "grad_norm": 1.0295422077178955,
+      "learning_rate": 9.351851851851854e-06,
+      "loss": 0.8194,
+      "step": 139
+    },
+    {
+      "epoch": 5.616161616161616,
+      "eval_loss": 0.8169118762016296,
+      "eval_runtime": 5.168,
+      "eval_samples_per_second": 32.121,
+      "eval_steps_per_second": 0.58,
+      "step": 139
+    },
+    {
+      "epoch": 5.656565656565657,
+      "grad_norm": 1.0281344652175903,
+      "learning_rate": 9.25925925925926e-06,
+      "loss": 0.7902,
+      "step": 140
+    },
+    {
+      "epoch": 5.656565656565657,
+      "eval_loss": 0.8098058104515076,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.183,
+      "eval_steps_per_second": 0.582,
+      "step": 140
+    },
+    {
+      "epoch": 5.696969696969697,
+      "grad_norm": 1.0528678894042969,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 0.7939,
+      "step": 141
+    },
+    {
+      "epoch": 5.696969696969697,
+      "eval_loss": 0.8031502366065979,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 141
+    },
+    {
+      "epoch": 5.737373737373737,
+      "grad_norm": 1.0525215864181519,
+      "learning_rate": 9.074074074074075e-06,
+      "loss": 0.8127,
+      "step": 142
+    },
+    {
+      "epoch": 5.737373737373737,
+      "eval_loss": 0.7957438230514526,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 142
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 1.0634738206863403,
+      "learning_rate": 8.981481481481483e-06,
+      "loss": 0.7831,
+      "step": 143
+    },
+    {
+      "epoch": 5.777777777777778,
+      "eval_loss": 0.7881508469581604,
+      "eval_runtime": 5.1489,
+      "eval_samples_per_second": 32.24,
+      "eval_steps_per_second": 0.583,
+      "step": 143
+    },
+    {
+      "epoch": 5.818181818181818,
+      "grad_norm": 1.0633102655410767,
+      "learning_rate": 8.888888888888888e-06,
+      "loss": 0.7511,
+      "step": 144
+    },
+    {
+      "epoch": 5.818181818181818,
+      "eval_loss": 0.7818319201469421,
+      "eval_runtime": 5.169,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 144
+    },
+    {
+      "epoch": 5.858585858585858,
+      "grad_norm": 1.053615689277649,
+      "learning_rate": 8.796296296296297e-06,
+      "loss": 0.75,
+      "step": 145
+    },
+    {
+      "epoch": 5.858585858585858,
+      "eval_loss": 0.7745649814605713,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 145
+    },
+    {
+      "epoch": 5.898989898989899,
+      "grad_norm": 1.0772610902786255,
+      "learning_rate": 8.703703703703705e-06,
+      "loss": 0.7929,
+      "step": 146
+    },
+    {
+      "epoch": 5.898989898989899,
+      "eval_loss": 0.7681305408477783,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.16,
+      "eval_steps_per_second": 0.581,
+      "step": 146
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "grad_norm": 1.0765084028244019,
+      "learning_rate": 8.611111111111112e-06,
+      "loss": 0.7642,
+      "step": 147
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "eval_loss": 0.7618209719657898,
+      "eval_runtime": 5.1689,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 147
+    },
+    {
+      "epoch": 5.97979797979798,
+      "grad_norm": 1.0824408531188965,
+      "learning_rate": 8.518518518518519e-06,
+      "loss": 0.7387,
+      "step": 148
+    },
+    {
+      "epoch": 5.97979797979798,
+      "eval_loss": 0.755460798740387,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.206,
+      "eval_steps_per_second": 0.582,
+      "step": 148
+    },
+    {
+      "epoch": 6.02020202020202,
+      "grad_norm": 1.0975412130355835,
+      "learning_rate": 8.425925925925926e-06,
+      "loss": 0.7218,
+      "step": 149
+    },
+    {
+      "epoch": 6.02020202020202,
+      "eval_loss": 0.749002993106842,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 149
+    },
+    {
+      "epoch": 6.0606060606060606,
+      "grad_norm": 1.1096184253692627,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 0.712,
+      "step": 150
+    },
+    {
+      "epoch": 6.0606060606060606,
+      "eval_loss": 0.7422470450401306,
+      "eval_runtime": 5.1674,
+      "eval_samples_per_second": 32.124,
+      "eval_steps_per_second": 0.581,
+      "step": 150
+    },
+    {
+      "epoch": 6.101010101010101,
+      "grad_norm": 1.1530879735946655,
+      "learning_rate": 8.240740740740741e-06,
+      "loss": 0.7268,
+      "step": 151
+    },
+    {
+      "epoch": 6.101010101010101,
+      "eval_loss": 0.7364265322685242,
+      "eval_runtime": 5.1645,
+      "eval_samples_per_second": 32.142,
+      "eval_steps_per_second": 0.581,
+      "step": 151
+    },
+    {
+      "epoch": 6.141414141414141,
+      "grad_norm": 1.1592988967895508,
+      "learning_rate": 8.148148148148148e-06,
+      "loss": 0.7258,
+      "step": 152
+    },
+    {
+      "epoch": 6.141414141414141,
+      "eval_loss": 0.7302441596984863,
+      "eval_runtime": 5.1636,
+      "eval_samples_per_second": 32.148,
+      "eval_steps_per_second": 0.581,
+      "step": 152
+    },
+    {
+      "epoch": 6.181818181818182,
+      "grad_norm": 1.1833539009094238,
+      "learning_rate": 8.055555555555557e-06,
+      "loss": 0.7224,
+      "step": 153
+    },
+    {
+      "epoch": 6.181818181818182,
+      "eval_loss": 0.7228173017501831,
+      "eval_runtime": 5.1641,
+      "eval_samples_per_second": 32.145,
+      "eval_steps_per_second": 0.581,
+      "step": 153
+    },
+    {
+      "epoch": 6.222222222222222,
+      "grad_norm": 1.1936936378479004,
+      "learning_rate": 7.962962962962963e-06,
+      "loss": 0.6966,
+      "step": 154
+    },
+    {
+      "epoch": 6.222222222222222,
+      "eval_loss": 0.716472327709198,
+      "eval_runtime": 5.1605,
+      "eval_samples_per_second": 32.167,
+      "eval_steps_per_second": 0.581,
+      "step": 154
+    },
+    {
+      "epoch": 6.262626262626263,
+      "grad_norm": 1.245132565498352,
+      "learning_rate": 7.870370370370372e-06,
+      "loss": 0.7028,
+      "step": 155
+    },
+    {
+      "epoch": 6.262626262626263,
+      "eval_loss": 0.7087136507034302,
+      "eval_runtime": 5.1583,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 155
+    },
+    {
+      "epoch": 6.303030303030303,
+      "grad_norm": 1.2659085988998413,
+      "learning_rate": 7.77777777777778e-06,
+      "loss": 0.7051,
+      "step": 156
+    },
+    {
+      "epoch": 6.303030303030303,
+      "eval_loss": 0.7009304165840149,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 156
+    },
+    {
+      "epoch": 6.343434343434343,
+      "grad_norm": 1.3588238954544067,
+      "learning_rate": 7.685185185185185e-06,
+      "loss": 0.7004,
+      "step": 157
+    },
+    {
+      "epoch": 6.343434343434343,
+      "eval_loss": 0.6932580471038818,
+      "eval_runtime": 5.1604,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 157
+    },
+    {
+      "epoch": 6.383838383838384,
+      "grad_norm": 1.3982634544372559,
+      "learning_rate": 7.592592592592594e-06,
+      "loss": 0.68,
+      "step": 158
+    },
+    {
+      "epoch": 6.383838383838384,
+      "eval_loss": 0.6845574378967285,
+      "eval_runtime": 5.1575,
+      "eval_samples_per_second": 32.186,
+      "eval_steps_per_second": 0.582,
+      "step": 158
+    },
+    {
+      "epoch": 6.424242424242424,
+      "grad_norm": 1.426499605178833,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 0.6641,
+      "step": 159
+    },
+    {
+      "epoch": 6.424242424242424,
+      "eval_loss": 0.6760482788085938,
+      "eval_runtime": 5.1632,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 159
+    },
+    {
+      "epoch": 6.4646464646464645,
+      "grad_norm": 1.4880263805389404,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 0.695,
+      "step": 160
+    },
+    {
+      "epoch": 6.4646464646464645,
+      "eval_loss": 0.6670196652412415,
+      "eval_runtime": 5.1633,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 160
+    },
+    {
+      "epoch": 6.505050505050505,
+      "grad_norm": 1.524919033050537,
+      "learning_rate": 7.314814814814816e-06,
+      "loss": 0.6316,
+      "step": 161
+    },
+    {
+      "epoch": 6.505050505050505,
+      "eval_loss": 0.6562865972518921,
+      "eval_runtime": 5.1726,
+      "eval_samples_per_second": 32.092,
+      "eval_steps_per_second": 0.58,
+      "step": 161
+    },
+    {
+      "epoch": 6.545454545454545,
+      "grad_norm": 1.542138934135437,
+      "learning_rate": 7.222222222222223e-06,
+      "loss": 0.6491,
+      "step": 162
+    },
+    {
+      "epoch": 6.545454545454545,
+      "eval_loss": 0.6472378969192505,
+      "eval_runtime": 5.1671,
+      "eval_samples_per_second": 32.126,
+      "eval_steps_per_second": 0.581,
+      "step": 162
+    },
+    {
+      "epoch": 6.585858585858586,
+      "grad_norm": 1.5400753021240234,
+      "learning_rate": 7.129629629629629e-06,
+      "loss": 0.6324,
+      "step": 163
+    },
+    {
+      "epoch": 6.585858585858586,
+      "eval_loss": 0.6379138231277466,
+      "eval_runtime": 5.1556,
+      "eval_samples_per_second": 32.198,
+      "eval_steps_per_second": 0.582,
+      "step": 163
+    },
+    {
+      "epoch": 6.626262626262626,
+      "grad_norm": 1.4753589630126953,
+      "learning_rate": 7.0370370370370375e-06,
+      "loss": 0.6473,
+      "step": 164
+    },
+    {
+      "epoch": 6.626262626262626,
+      "eval_loss": 0.6301048994064331,
+      "eval_runtime": 5.1423,
+      "eval_samples_per_second": 32.281,
+      "eval_steps_per_second": 0.583,
+      "step": 164
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 1.370703935623169,
+      "learning_rate": 6.944444444444445e-06,
+      "loss": 0.6255,
+      "step": 165
+    },
+    {
+      "epoch": 6.666666666666667,
+      "eval_loss": 0.6244276762008667,
+      "eval_runtime": 5.1569,
+      "eval_samples_per_second": 32.19,
+      "eval_steps_per_second": 0.582,
+      "step": 165
+    },
+    {
+      "epoch": 6.707070707070707,
+      "grad_norm": 1.2907015085220337,
+      "learning_rate": 6.851851851851853e-06,
+      "loss": 0.6029,
+      "step": 166
+    },
+    {
+      "epoch": 6.707070707070707,
+      "eval_loss": 0.6191242933273315,
+      "eval_runtime": 5.1685,
+      "eval_samples_per_second": 32.118,
+      "eval_steps_per_second": 0.58,
+      "step": 166
+    },
+    {
+      "epoch": 6.747474747474747,
+      "grad_norm": 1.206007480621338,
+      "learning_rate": 6.75925925925926e-06,
+      "loss": 0.5831,
+      "step": 167
+    },
+    {
+      "epoch": 6.747474747474747,
+      "eval_loss": 0.6155738830566406,
+      "eval_runtime": 5.1689,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 167
+    },
+    {
+      "epoch": 6.787878787878788,
+      "grad_norm": 1.14729642868042,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 0.6167,
+      "step": 168
+    },
+    {
+      "epoch": 6.787878787878788,
+      "eval_loss": 0.6123696565628052,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 168
+    },
+    {
+      "epoch": 6.828282828282829,
+      "grad_norm": 1.1298456192016602,
+      "learning_rate": 6.574074074074075e-06,
+      "loss": 0.597,
+      "step": 169
+    },
+    {
+      "epoch": 6.828282828282829,
+      "eval_loss": 0.6096829771995544,
+      "eval_runtime": 5.1532,
+      "eval_samples_per_second": 32.213,
+      "eval_steps_per_second": 0.582,
+      "step": 169
+    },
+    {
+      "epoch": 6.8686868686868685,
+      "grad_norm": 1.129176378250122,
+      "learning_rate": 6.481481481481482e-06,
+      "loss": 0.6278,
+      "step": 170
+    },
+    {
+      "epoch": 6.8686868686868685,
+      "eval_loss": 0.6058310270309448,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 170
+    },
+    {
+      "epoch": 6.909090909090909,
+      "grad_norm": 1.1008386611938477,
+      "learning_rate": 6.3888888888888885e-06,
+      "loss": 0.5974,
+      "step": 171
+    },
+    {
+      "epoch": 6.909090909090909,
+      "eval_loss": 0.6041752696037292,
+      "eval_runtime": 5.1564,
+      "eval_samples_per_second": 32.193,
+      "eval_steps_per_second": 0.582,
+      "step": 171
+    },
+    {
+      "epoch": 6.94949494949495,
+      "grad_norm": 1.0789906978607178,
+      "learning_rate": 6.296296296296297e-06,
+      "loss": 0.5983,
+      "step": 172
+    },
+    {
+      "epoch": 6.94949494949495,
+      "eval_loss": 0.6021236181259155,
+      "eval_runtime": 5.1613,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 172
+    },
+    {
+      "epoch": 6.98989898989899,
+      "grad_norm": 1.1015055179595947,
+      "learning_rate": 6.203703703703704e-06,
+      "loss": 0.6138,
+      "step": 173
+    },
+    {
+      "epoch": 6.98989898989899,
+      "eval_loss": 0.6002500653266907,
+      "eval_runtime": 5.1625,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 173
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.466076308815872e+16,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-173/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dafca05b7a6d25266ace8b0db88af88ff18a0ac29cc5d191b665dd79638206
+size 5368

checkpoint-198/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-198/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-198/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc9b6cfde7abaa742d12bcca23fcb6fdab4002620e75490bbb17849ebe8f58e1
+size 35668592

checkpoint-198/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b1ad92b33d8557ee8e871220ccc139f1aefcee1dbb3f56f57145cac82867d6a
+size 71379898

checkpoint-198/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bc0d95fc16a011c5e1faa7623e21777122b47375c4bb1d1c0a3654250498761
+size 14244

checkpoint-198/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d64de680071e3bc783171702ec49fc5ac2d62256c3c10449a88cf3104346f56
+size 1064

checkpoint-198/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-198/tokenizer_config.json

@@ -0,0 +1,131 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": true,
+  "add_prefix_space": null,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": false
+    },
+    "32000": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32001": {
+      "content": "<|assistant|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32002": {
+      "content": "<|placeholder1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32003": {
+      "content": "<|placeholder2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32004": {
+      "content": "<|placeholder3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32005": {
+      "content": "<|placeholder4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32006": {
+      "content": "<|system|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32007": {
+      "content": "<|end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32008": {
+      "content": "<|placeholder5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32009": {
+      "content": "<|placeholder6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "32010": {
+      "content": "<|user|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + '\n' + message['content'] + '<|end|>' + '\n' + '<|assistant|>' + '\n'}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + '\n'}}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "legacy": false,
+  "model_max_length": 4096,
+  "pad_token": "<unk>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-198/trainer_state.json

@@ -0,0 +1,3003 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 8.0,
+  "eval_steps": 1,
+  "global_step": 198,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04040404040404041,
+      "grad_norm": 2.088834762573242,
+      "learning_rate": 8.333333333333333e-07,
+      "loss": 5.4688,
+      "step": 1
+    },
+    {
+      "epoch": 0.04040404040404041,
+      "eval_loss": 5.470384120941162,
+      "eval_runtime": 4.8066,
+      "eval_samples_per_second": 34.536,
+      "eval_steps_per_second": 0.624,
+      "step": 1
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "grad_norm": 2.0961971282958984,
+      "learning_rate": 1.6666666666666667e-06,
+      "loss": 5.4731,
+      "step": 2
+    },
+    {
+      "epoch": 0.08080808080808081,
+      "eval_loss": 5.4689249992370605,
+      "eval_runtime": 4.8593,
+      "eval_samples_per_second": 34.161,
+      "eval_steps_per_second": 0.617,
+      "step": 2
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "grad_norm": 2.0940349102020264,
+      "learning_rate": 2.5e-06,
+      "loss": 5.4803,
+      "step": 3
+    },
+    {
+      "epoch": 0.12121212121212122,
+      "eval_loss": 5.467825412750244,
+      "eval_runtime": 4.9081,
+      "eval_samples_per_second": 33.822,
+      "eval_steps_per_second": 0.611,
+      "step": 3
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "grad_norm": 2.095174551010132,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 5.5119,
+      "step": 4
+    },
+    {
+      "epoch": 0.16161616161616163,
+      "eval_loss": 5.465516567230225,
+      "eval_runtime": 4.8615,
+      "eval_samples_per_second": 34.146,
+      "eval_steps_per_second": 0.617,
+      "step": 4
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "grad_norm": 2.1109118461608887,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 5.4662,
+      "step": 5
+    },
+    {
+      "epoch": 0.20202020202020202,
+      "eval_loss": 5.4626851081848145,
+      "eval_runtime": 4.844,
+      "eval_samples_per_second": 34.27,
+      "eval_steps_per_second": 0.619,
+      "step": 5
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "grad_norm": 2.1030592918395996,
+      "learning_rate": 5e-06,
+      "loss": 5.4691,
+      "step": 6
+    },
+    {
+      "epoch": 0.24242424242424243,
+      "eval_loss": 5.45806884765625,
+      "eval_runtime": 4.8388,
+      "eval_samples_per_second": 34.306,
+      "eval_steps_per_second": 0.62,
+      "step": 6
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "grad_norm": 2.1202447414398193,
+      "learning_rate": 5.833333333333334e-06,
+      "loss": 5.4851,
+      "step": 7
+    },
+    {
+      "epoch": 0.2828282828282828,
+      "eval_loss": 5.449892520904541,
+      "eval_runtime": 4.8431,
+      "eval_samples_per_second": 34.276,
+      "eval_steps_per_second": 0.619,
+      "step": 7
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "grad_norm": 2.090963363647461,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 5.4228,
+      "step": 8
+    },
+    {
+      "epoch": 0.32323232323232326,
+      "eval_loss": 5.442515850067139,
+      "eval_runtime": 4.8605,
+      "eval_samples_per_second": 34.153,
+      "eval_steps_per_second": 0.617,
+      "step": 8
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "grad_norm": 2.112323760986328,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 5.4715,
+      "step": 9
+    },
+    {
+      "epoch": 0.36363636363636365,
+      "eval_loss": 5.4345855712890625,
+      "eval_runtime": 4.8796,
+      "eval_samples_per_second": 34.019,
+      "eval_steps_per_second": 0.615,
+      "step": 9
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "grad_norm": 2.1456995010375977,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 5.4211,
+      "step": 10
+    },
+    {
+      "epoch": 0.40404040404040403,
+      "eval_loss": 5.424546718597412,
+      "eval_runtime": 4.9274,
+      "eval_samples_per_second": 33.689,
+      "eval_steps_per_second": 0.609,
+      "step": 10
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 2.1602349281311035,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 5.4301,
+      "step": 11
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "eval_loss": 5.41035795211792,
+      "eval_runtime": 4.9819,
+      "eval_samples_per_second": 33.32,
+      "eval_steps_per_second": 0.602,
+      "step": 11
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "grad_norm": 2.1762421131134033,
+      "learning_rate": 1e-05,
+      "loss": 5.4158,
+      "step": 12
+    },
+    {
+      "epoch": 0.48484848484848486,
+      "eval_loss": 5.400452613830566,
+      "eval_runtime": 5.0737,
+      "eval_samples_per_second": 32.718,
+      "eval_steps_per_second": 0.591,
+      "step": 12
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "grad_norm": 2.2415969371795654,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 5.3986,
+      "step": 13
+    },
+    {
+      "epoch": 0.5252525252525253,
+      "eval_loss": 5.382190227508545,
+      "eval_runtime": 4.9555,
+      "eval_samples_per_second": 33.498,
+      "eval_steps_per_second": 0.605,
+      "step": 13
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "grad_norm": 2.2669475078582764,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 5.428,
+      "step": 14
+    },
+    {
+      "epoch": 0.5656565656565656,
+      "eval_loss": 5.3634443283081055,
+      "eval_runtime": 4.9029,
+      "eval_samples_per_second": 33.857,
+      "eval_steps_per_second": 0.612,
+      "step": 14
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "grad_norm": 2.262896776199341,
+      "learning_rate": 1.25e-05,
+      "loss": 5.3741,
+      "step": 15
+    },
+    {
+      "epoch": 0.6060606060606061,
+      "eval_loss": 5.347131729125977,
+      "eval_runtime": 4.8934,
+      "eval_samples_per_second": 33.923,
+      "eval_steps_per_second": 0.613,
+      "step": 15
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "grad_norm": 2.3363702297210693,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 5.4134,
+      "step": 16
+    },
+    {
+      "epoch": 0.6464646464646465,
+      "eval_loss": 5.321357250213623,
+      "eval_runtime": 4.9027,
+      "eval_samples_per_second": 33.859,
+      "eval_steps_per_second": 0.612,
+      "step": 16
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "grad_norm": 2.3906755447387695,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 5.3305,
+      "step": 17
+    },
+    {
+      "epoch": 0.6868686868686869,
+      "eval_loss": 5.292910099029541,
+      "eval_runtime": 4.9268,
+      "eval_samples_per_second": 33.693,
+      "eval_steps_per_second": 0.609,
+      "step": 17
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "grad_norm": 2.44368839263916,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 5.3406,
+      "step": 18
+    },
+    {
+      "epoch": 0.7272727272727273,
+      "eval_loss": 5.257543563842773,
+      "eval_runtime": 4.9839,
+      "eval_samples_per_second": 33.308,
+      "eval_steps_per_second": 0.602,
+      "step": 18
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "grad_norm": 2.4937825202941895,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 5.3134,
+      "step": 19
+    },
+    {
+      "epoch": 0.7676767676767676,
+      "eval_loss": 5.225679874420166,
+      "eval_runtime": 5.1205,
+      "eval_samples_per_second": 32.419,
+      "eval_steps_per_second": 0.586,
+      "step": 19
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "grad_norm": 2.547194480895996,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 5.2303,
+      "step": 20
+    },
+    {
+      "epoch": 0.8080808080808081,
+      "eval_loss": 5.183907985687256,
+      "eval_runtime": 5.1208,
+      "eval_samples_per_second": 32.417,
+      "eval_steps_per_second": 0.586,
+      "step": 20
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "grad_norm": 2.5578083992004395,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 5.1255,
+      "step": 21
+    },
+    {
+      "epoch": 0.8484848484848485,
+      "eval_loss": 5.136340141296387,
+      "eval_runtime": 5.1161,
+      "eval_samples_per_second": 32.446,
+      "eval_steps_per_second": 0.586,
+      "step": 21
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 2.6553032398223877,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 5.1376,
+      "step": 22
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "eval_loss": 5.085525035858154,
+      "eval_runtime": 5.1496,
+      "eval_samples_per_second": 32.236,
+      "eval_steps_per_second": 0.583,
+      "step": 22
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "grad_norm": 2.733492374420166,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 5.1051,
+      "step": 23
+    },
+    {
+      "epoch": 0.9292929292929293,
+      "eval_loss": 5.0193634033203125,
+      "eval_runtime": 5.0473,
+      "eval_samples_per_second": 32.889,
+      "eval_steps_per_second": 0.594,
+      "step": 23
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "grad_norm": 2.765582799911499,
+      "learning_rate": 2e-05,
+      "loss": 5.0082,
+      "step": 24
+    },
+    {
+      "epoch": 0.9696969696969697,
+      "eval_loss": 4.950736999511719,
+      "eval_runtime": 4.989,
+      "eval_samples_per_second": 33.273,
+      "eval_steps_per_second": 0.601,
+      "step": 24
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "grad_norm": 2.8943891525268555,
+      "learning_rate": 1.990740740740741e-05,
+      "loss": 4.9736,
+      "step": 25
+    },
+    {
+      "epoch": 1.0101010101010102,
+      "eval_loss": 4.87933349609375,
+      "eval_runtime": 4.991,
+      "eval_samples_per_second": 33.26,
+      "eval_steps_per_second": 0.601,
+      "step": 25
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "grad_norm": 2.9802019596099854,
+      "learning_rate": 1.9814814814814816e-05,
+      "loss": 4.8728,
+      "step": 26
+    },
+    {
+      "epoch": 1.0505050505050506,
+      "eval_loss": 4.796956539154053,
+      "eval_runtime": 5.0224,
+      "eval_samples_per_second": 33.052,
+      "eval_steps_per_second": 0.597,
+      "step": 26
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "grad_norm": 3.068443536758423,
+      "learning_rate": 1.9722222222222224e-05,
+      "loss": 4.8132,
+      "step": 27
+    },
+    {
+      "epoch": 1.0909090909090908,
+      "eval_loss": 4.7025275230407715,
+      "eval_runtime": 5.1226,
+      "eval_samples_per_second": 32.406,
+      "eval_steps_per_second": 0.586,
+      "step": 27
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "grad_norm": 3.2245917320251465,
+      "learning_rate": 1.962962962962963e-05,
+      "loss": 4.7175,
+      "step": 28
+    },
+    {
+      "epoch": 1.1313131313131313,
+      "eval_loss": 4.607891082763672,
+      "eval_runtime": 5.0927,
+      "eval_samples_per_second": 32.596,
+      "eval_steps_per_second": 0.589,
+      "step": 28
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "grad_norm": 3.3136157989501953,
+      "learning_rate": 1.9537037037037038e-05,
+      "loss": 4.6429,
+      "step": 29
+    },
+    {
+      "epoch": 1.1717171717171717,
+      "eval_loss": 4.510694980621338,
+      "eval_runtime": 5.0136,
+      "eval_samples_per_second": 33.11,
+      "eval_steps_per_second": 0.598,
+      "step": 29
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "grad_norm": 3.329773426055908,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 4.4686,
+      "step": 30
+    },
+    {
+      "epoch": 1.2121212121212122,
+      "eval_loss": 4.406045913696289,
+      "eval_runtime": 4.9753,
+      "eval_samples_per_second": 33.365,
+      "eval_steps_per_second": 0.603,
+      "step": 30
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "grad_norm": 3.5462117195129395,
+      "learning_rate": 1.9351851851851853e-05,
+      "loss": 4.427,
+      "step": 31
+    },
+    {
+      "epoch": 1.2525252525252526,
+      "eval_loss": 4.309689998626709,
+      "eval_runtime": 4.9974,
+      "eval_samples_per_second": 33.217,
+      "eval_steps_per_second": 0.6,
+      "step": 31
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "grad_norm": 3.6190805435180664,
+      "learning_rate": 1.925925925925926e-05,
+      "loss": 4.2899,
+      "step": 32
+    },
+    {
+      "epoch": 1.2929292929292928,
+      "eval_loss": 4.215596675872803,
+      "eval_runtime": 5.0637,
+      "eval_samples_per_second": 32.782,
+      "eval_steps_per_second": 0.592,
+      "step": 32
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 3.698317289352417,
+      "learning_rate": 1.916666666666667e-05,
+      "loss": 4.1752,
+      "step": 33
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "eval_loss": 4.106834411621094,
+      "eval_runtime": 5.1651,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 33
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "grad_norm": 3.4807214736938477,
+      "learning_rate": 1.9074074074074075e-05,
+      "loss": 4.1277,
+      "step": 34
+    },
+    {
+      "epoch": 1.3737373737373737,
+      "eval_loss": 4.001583099365234,
+      "eval_runtime": 5.0567,
+      "eval_samples_per_second": 32.828,
+      "eval_steps_per_second": 0.593,
+      "step": 34
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "grad_norm": 3.1380534172058105,
+      "learning_rate": 1.8981481481481482e-05,
+      "loss": 4.0069,
+      "step": 35
+    },
+    {
+      "epoch": 1.4141414141414141,
+      "eval_loss": 3.9105026721954346,
+      "eval_runtime": 5.07,
+      "eval_samples_per_second": 32.741,
+      "eval_steps_per_second": 0.592,
+      "step": 35
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "grad_norm": 2.9076974391937256,
+      "learning_rate": 1.888888888888889e-05,
+      "loss": 3.8911,
+      "step": 36
+    },
+    {
+      "epoch": 1.4545454545454546,
+      "eval_loss": 3.838275194168091,
+      "eval_runtime": 5.1402,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 36
+    },
+    {
+      "epoch": 1.494949494949495,
+      "grad_norm": 2.4755232334136963,
+      "learning_rate": 1.8796296296296296e-05,
+      "loss": 3.8522,
+      "step": 37
+    },
+    {
+      "epoch": 1.494949494949495,
+      "eval_loss": 3.776526689529419,
+      "eval_runtime": 5.1768,
+      "eval_samples_per_second": 32.066,
+      "eval_steps_per_second": 0.58,
+      "step": 37
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "grad_norm": 2.1607069969177246,
+      "learning_rate": 1.8703703703703707e-05,
+      "loss": 3.7828,
+      "step": 38
+    },
+    {
+      "epoch": 1.5353535353535355,
+      "eval_loss": 3.7257139682769775,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 38
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "grad_norm": 1.9251853227615356,
+      "learning_rate": 1.8611111111111114e-05,
+      "loss": 3.7511,
+      "step": 39
+    },
+    {
+      "epoch": 1.5757575757575757,
+      "eval_loss": 3.680651903152466,
+      "eval_runtime": 5.139,
+      "eval_samples_per_second": 32.302,
+      "eval_steps_per_second": 0.584,
+      "step": 39
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "grad_norm": 1.8553980588912964,
+      "learning_rate": 1.851851851851852e-05,
+      "loss": 3.7108,
+      "step": 40
+    },
+    {
+      "epoch": 1.6161616161616161,
+      "eval_loss": 3.633517026901245,
+      "eval_runtime": 5.1036,
+      "eval_samples_per_second": 32.526,
+      "eval_steps_per_second": 0.588,
+      "step": 40
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "grad_norm": 1.831110954284668,
+      "learning_rate": 1.8425925925925926e-05,
+      "loss": 3.6165,
+      "step": 41
+    },
+    {
+      "epoch": 1.6565656565656566,
+      "eval_loss": 3.5877223014831543,
+      "eval_runtime": 5.1287,
+      "eval_samples_per_second": 32.367,
+      "eval_steps_per_second": 0.585,
+      "step": 41
+    },
+    {
+      "epoch": 1.696969696969697,
+      "grad_norm": 1.8035303354263306,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 3.5555,
+      "step": 42
+    },
+    {
+      "epoch": 1.696969696969697,
+      "eval_loss": 3.5403287410736084,
+      "eval_runtime": 5.1001,
+      "eval_samples_per_second": 32.549,
+      "eval_steps_per_second": 0.588,
+      "step": 42
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "grad_norm": 1.7953453063964844,
+      "learning_rate": 1.8240740740740744e-05,
+      "loss": 3.5437,
+      "step": 43
+    },
+    {
+      "epoch": 1.7373737373737375,
+      "eval_loss": 3.492953300476074,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 43
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 1.7807087898254395,
+      "learning_rate": 1.814814814814815e-05,
+      "loss": 3.4676,
+      "step": 44
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "eval_loss": 3.443131446838379,
+      "eval_runtime": 5.1066,
+      "eval_samples_per_second": 32.507,
+      "eval_steps_per_second": 0.587,
+      "step": 44
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "grad_norm": 1.8204630613327026,
+      "learning_rate": 1.8055555555555558e-05,
+      "loss": 3.4517,
+      "step": 45
+    },
+    {
+      "epoch": 1.8181818181818183,
+      "eval_loss": 3.396712303161621,
+      "eval_runtime": 5.1475,
+      "eval_samples_per_second": 32.249,
+      "eval_steps_per_second": 0.583,
+      "step": 45
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "grad_norm": 1.8122252225875854,
+      "learning_rate": 1.7962962962962965e-05,
+      "loss": 3.4189,
+      "step": 46
+    },
+    {
+      "epoch": 1.8585858585858586,
+      "eval_loss": 3.3434314727783203,
+      "eval_runtime": 5.0828,
+      "eval_samples_per_second": 32.659,
+      "eval_steps_per_second": 0.59,
+      "step": 46
+    },
+    {
+      "epoch": 1.898989898989899,
+      "grad_norm": 1.8433269262313843,
+      "learning_rate": 1.7870370370370373e-05,
+      "loss": 3.3537,
+      "step": 47
+    },
+    {
+      "epoch": 1.898989898989899,
+      "eval_loss": 3.294283628463745,
+      "eval_runtime": 5.1195,
+      "eval_samples_per_second": 32.425,
+      "eval_steps_per_second": 0.586,
+      "step": 47
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "grad_norm": 1.833533763885498,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 3.3034,
+      "step": 48
+    },
+    {
+      "epoch": 1.9393939393939394,
+      "eval_loss": 3.2412631511688232,
+      "eval_runtime": 5.1404,
+      "eval_samples_per_second": 32.293,
+      "eval_steps_per_second": 0.584,
+      "step": 48
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "grad_norm": 1.823169469833374,
+      "learning_rate": 1.7685185185185187e-05,
+      "loss": 3.2457,
+      "step": 49
+    },
+    {
+      "epoch": 1.9797979797979797,
+      "eval_loss": 3.1890854835510254,
+      "eval_runtime": 5.1319,
+      "eval_samples_per_second": 32.347,
+      "eval_steps_per_second": 0.585,
+      "step": 49
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "grad_norm": 1.796790361404419,
+      "learning_rate": 1.7592592592592595e-05,
+      "loss": 3.1915,
+      "step": 50
+    },
+    {
+      "epoch": 2.0202020202020203,
+      "eval_loss": 3.1396946907043457,
+      "eval_runtime": 5.1262,
+      "eval_samples_per_second": 32.383,
+      "eval_steps_per_second": 0.585,
+      "step": 50
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "grad_norm": 1.7133023738861084,
+      "learning_rate": 1.7500000000000002e-05,
+      "loss": 3.1475,
+      "step": 51
+    },
+    {
+      "epoch": 2.0606060606060606,
+      "eval_loss": 3.093197822570801,
+      "eval_runtime": 5.141,
+      "eval_samples_per_second": 32.29,
+      "eval_steps_per_second": 0.584,
+      "step": 51
+    },
+    {
+      "epoch": 2.101010101010101,
+      "grad_norm": 1.637424111366272,
+      "learning_rate": 1.740740740740741e-05,
+      "loss": 3.0912,
+      "step": 52
+    },
+    {
+      "epoch": 2.101010101010101,
+      "eval_loss": 3.0486488342285156,
+      "eval_runtime": 5.1578,
+      "eval_samples_per_second": 32.185,
+      "eval_steps_per_second": 0.582,
+      "step": 52
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "grad_norm": 1.5733733177185059,
+      "learning_rate": 1.7314814814814816e-05,
+      "loss": 3.0493,
+      "step": 53
+    },
+    {
+      "epoch": 2.1414141414141414,
+      "eval_loss": 3.007200002670288,
+      "eval_runtime": 5.1715,
+      "eval_samples_per_second": 32.099,
+      "eval_steps_per_second": 0.58,
+      "step": 53
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "grad_norm": 1.579100251197815,
+      "learning_rate": 1.7222222222222224e-05,
+      "loss": 3.0007,
+      "step": 54
+    },
+    {
+      "epoch": 2.1818181818181817,
+      "eval_loss": 2.9646313190460205,
+      "eval_runtime": 5.1737,
+      "eval_samples_per_second": 32.085,
+      "eval_steps_per_second": 0.58,
+      "step": 54
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.5558359622955322,
+      "learning_rate": 1.712962962962963e-05,
+      "loss": 2.955,
+      "step": 55
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_loss": 2.9225423336029053,
+      "eval_runtime": 5.1649,
+      "eval_samples_per_second": 32.14,
+      "eval_steps_per_second": 0.581,
+      "step": 55
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "grad_norm": 1.5695600509643555,
+      "learning_rate": 1.7037037037037038e-05,
+      "loss": 2.9242,
+      "step": 56
+    },
+    {
+      "epoch": 2.2626262626262625,
+      "eval_loss": 2.883842706680298,
+      "eval_runtime": 5.1565,
+      "eval_samples_per_second": 32.192,
+      "eval_steps_per_second": 0.582,
+      "step": 56
+    },
+    {
+      "epoch": 2.303030303030303,
+      "grad_norm": 1.5856674909591675,
+      "learning_rate": 1.6944444444444446e-05,
+      "loss": 2.8626,
+      "step": 57
+    },
+    {
+      "epoch": 2.303030303030303,
+      "eval_loss": 2.8435447216033936,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 57
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "grad_norm": 1.5780515670776367,
+      "learning_rate": 1.6851851851851853e-05,
+      "loss": 2.8522,
+      "step": 58
+    },
+    {
+      "epoch": 2.3434343434343434,
+      "eval_loss": 2.8028392791748047,
+      "eval_runtime": 5.1666,
+      "eval_samples_per_second": 32.129,
+      "eval_steps_per_second": 0.581,
+      "step": 58
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "grad_norm": 1.5944206714630127,
+      "learning_rate": 1.675925925925926e-05,
+      "loss": 2.8222,
+      "step": 59
+    },
+    {
+      "epoch": 2.3838383838383836,
+      "eval_loss": 2.7607803344726562,
+      "eval_runtime": 5.1662,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 59
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "grad_norm": 1.6033095121383667,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.7291,
+      "step": 60
+    },
+    {
+      "epoch": 2.4242424242424243,
+      "eval_loss": 2.7191247940063477,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 60
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "grad_norm": 1.621768593788147,
+      "learning_rate": 1.6574074074074075e-05,
+      "loss": 2.7046,
+      "step": 61
+    },
+    {
+      "epoch": 2.4646464646464645,
+      "eval_loss": 2.6788933277130127,
+      "eval_runtime": 5.1728,
+      "eval_samples_per_second": 32.091,
+      "eval_steps_per_second": 0.58,
+      "step": 61
+    },
+    {
+      "epoch": 2.505050505050505,
+      "grad_norm": 1.6213834285736084,
+      "learning_rate": 1.6481481481481482e-05,
+      "loss": 2.6548,
+      "step": 62
+    },
+    {
+      "epoch": 2.505050505050505,
+      "eval_loss": 2.6340527534484863,
+      "eval_runtime": 5.1628,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 62
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "grad_norm": 1.607869267463684,
+      "learning_rate": 1.638888888888889e-05,
+      "loss": 2.6232,
+      "step": 63
+    },
+    {
+      "epoch": 2.5454545454545454,
+      "eval_loss": 2.594001531600952,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 63
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "grad_norm": 1.583896279335022,
+      "learning_rate": 1.6296296296296297e-05,
+      "loss": 2.6068,
+      "step": 64
+    },
+    {
+      "epoch": 2.5858585858585856,
+      "eval_loss": 2.5528736114501953,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 64
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "grad_norm": 1.5254372358322144,
+      "learning_rate": 1.6203703703703707e-05,
+      "loss": 2.5629,
+      "step": 65
+    },
+    {
+      "epoch": 2.6262626262626263,
+      "eval_loss": 2.5128839015960693,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 65
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 1.4735966920852661,
+      "learning_rate": 1.6111111111111115e-05,
+      "loss": 2.523,
+      "step": 66
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "eval_loss": 2.4759180545806885,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 66
+    },
+    {
+      "epoch": 2.707070707070707,
+      "grad_norm": 1.4135326147079468,
+      "learning_rate": 1.601851851851852e-05,
+      "loss": 2.5069,
+      "step": 67
+    },
+    {
+      "epoch": 2.707070707070707,
+      "eval_loss": 2.4382143020629883,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.274,
+      "eval_steps_per_second": 0.583,
+      "step": 67
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "grad_norm": 1.3859189748764038,
+      "learning_rate": 1.5925925925925926e-05,
+      "loss": 2.449,
+      "step": 68
+    },
+    {
+      "epoch": 2.7474747474747474,
+      "eval_loss": 2.3999476432800293,
+      "eval_runtime": 5.0261,
+      "eval_samples_per_second": 33.028,
+      "eval_steps_per_second": 0.597,
+      "step": 68
+    },
+    {
+      "epoch": 2.787878787878788,
+      "grad_norm": 1.3678818941116333,
+      "learning_rate": 1.5833333333333333e-05,
+      "loss": 2.3977,
+      "step": 69
+    },
+    {
+      "epoch": 2.787878787878788,
+      "eval_loss": 2.365516424179077,
+      "eval_runtime": 4.9945,
+      "eval_samples_per_second": 33.237,
+      "eval_steps_per_second": 0.601,
+      "step": 69
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "grad_norm": 1.372074842453003,
+      "learning_rate": 1.5740740740740744e-05,
+      "loss": 2.3556,
+      "step": 70
+    },
+    {
+      "epoch": 2.8282828282828283,
+      "eval_loss": 2.3320348262786865,
+      "eval_runtime": 5.0457,
+      "eval_samples_per_second": 32.899,
+      "eval_steps_per_second": 0.595,
+      "step": 70
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "grad_norm": 1.355190634727478,
+      "learning_rate": 1.564814814814815e-05,
+      "loss": 2.3269,
+      "step": 71
+    },
+    {
+      "epoch": 2.8686868686868685,
+      "eval_loss": 2.2980406284332275,
+      "eval_runtime": 5.1448,
+      "eval_samples_per_second": 32.265,
+      "eval_steps_per_second": 0.583,
+      "step": 71
+    },
+    {
+      "epoch": 2.909090909090909,
+      "grad_norm": 1.34852135181427,
+      "learning_rate": 1.555555555555556e-05,
+      "loss": 2.2962,
+      "step": 72
+    },
+    {
+      "epoch": 2.909090909090909,
+      "eval_loss": 2.262707233428955,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 72
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "grad_norm": 1.3559976816177368,
+      "learning_rate": 1.5462962962962966e-05,
+      "loss": 2.2534,
+      "step": 73
+    },
+    {
+      "epoch": 2.9494949494949494,
+      "eval_loss": 2.2291126251220703,
+      "eval_runtime": 5.0794,
+      "eval_samples_per_second": 32.681,
+      "eval_steps_per_second": 0.591,
+      "step": 73
+    },
+    {
+      "epoch": 2.98989898989899,
+      "grad_norm": 1.3566159009933472,
+      "learning_rate": 1.537037037037037e-05,
+      "loss": 2.2061,
+      "step": 74
+    },
+    {
+      "epoch": 2.98989898989899,
+      "eval_loss": 2.193363904953003,
+      "eval_runtime": 5.0616,
+      "eval_samples_per_second": 32.796,
+      "eval_steps_per_second": 0.593,
+      "step": 74
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "grad_norm": 1.3466676473617554,
+      "learning_rate": 1.5277777777777777e-05,
+      "loss": 2.1815,
+      "step": 75
+    },
+    {
+      "epoch": 3.0303030303030303,
+      "eval_loss": 2.159188985824585,
+      "eval_runtime": 5.0978,
+      "eval_samples_per_second": 32.563,
+      "eval_steps_per_second": 0.588,
+      "step": 75
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "grad_norm": 1.354913592338562,
+      "learning_rate": 1.5185185185185187e-05,
+      "loss": 2.1622,
+      "step": 76
+    },
+    {
+      "epoch": 3.0707070707070705,
+      "eval_loss": 2.124568462371826,
+      "eval_runtime": 5.1439,
+      "eval_samples_per_second": 32.271,
+      "eval_steps_per_second": 0.583,
+      "step": 76
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 1.360551357269287,
+      "learning_rate": 1.5092592592592595e-05,
+      "loss": 2.1079,
+      "step": 77
+    },
+    {
+      "epoch": 3.111111111111111,
+      "eval_loss": 2.092242956161499,
+      "eval_runtime": 5.1333,
+      "eval_samples_per_second": 32.338,
+      "eval_steps_per_second": 0.584,
+      "step": 77
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "grad_norm": 1.3474143743515015,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 2.1036,
+      "step": 78
+    },
+    {
+      "epoch": 3.1515151515151514,
+      "eval_loss": 2.0584089756011963,
+      "eval_runtime": 5.1757,
+      "eval_samples_per_second": 32.073,
+      "eval_steps_per_second": 0.58,
+      "step": 78
+    },
+    {
+      "epoch": 3.191919191919192,
+      "grad_norm": 1.368388295173645,
+      "learning_rate": 1.4907407407407408e-05,
+      "loss": 2.0527,
+      "step": 79
+    },
+    {
+      "epoch": 3.191919191919192,
+      "eval_loss": 2.0241854190826416,
+      "eval_runtime": 5.1747,
+      "eval_samples_per_second": 32.079,
+      "eval_steps_per_second": 0.58,
+      "step": 79
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "grad_norm": 1.3582555055618286,
+      "learning_rate": 1.4814814814814815e-05,
+      "loss": 1.9958,
+      "step": 80
+    },
+    {
+      "epoch": 3.2323232323232323,
+      "eval_loss": 1.990015983581543,
+      "eval_runtime": 5.1542,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 80
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "grad_norm": 1.3820135593414307,
+      "learning_rate": 1.4722222222222224e-05,
+      "loss": 1.9878,
+      "step": 81
+    },
+    {
+      "epoch": 3.2727272727272725,
+      "eval_loss": 1.9560723304748535,
+      "eval_runtime": 5.176,
+      "eval_samples_per_second": 32.071,
+      "eval_steps_per_second": 0.58,
+      "step": 81
+    },
+    {
+      "epoch": 3.313131313131313,
+      "grad_norm": 1.4004285335540771,
+      "learning_rate": 1.4629629629629631e-05,
+      "loss": 1.9533,
+      "step": 82
+    },
+    {
+      "epoch": 3.313131313131313,
+      "eval_loss": 1.9231067895889282,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 82
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "grad_norm": 1.4260685443878174,
+      "learning_rate": 1.4537037037037039e-05,
+      "loss": 1.9066,
+      "step": 83
+    },
+    {
+      "epoch": 3.3535353535353534,
+      "eval_loss": 1.8882259130477905,
+      "eval_runtime": 5.1573,
+      "eval_samples_per_second": 32.187,
+      "eval_steps_per_second": 0.582,
+      "step": 83
+    },
+    {
+      "epoch": 3.393939393939394,
+      "grad_norm": 1.4351316690444946,
+      "learning_rate": 1.4444444444444446e-05,
+      "loss": 1.8803,
+      "step": 84
+    },
+    {
+      "epoch": 3.393939393939394,
+      "eval_loss": 1.8546966314315796,
+      "eval_runtime": 5.1748,
+      "eval_samples_per_second": 32.078,
+      "eval_steps_per_second": 0.58,
+      "step": 84
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "grad_norm": 1.4154078960418701,
+      "learning_rate": 1.4351851851851853e-05,
+      "loss": 1.8668,
+      "step": 85
+    },
+    {
+      "epoch": 3.4343434343434343,
+      "eval_loss": 1.820953369140625,
+      "eval_runtime": 5.1325,
+      "eval_samples_per_second": 32.343,
+      "eval_steps_per_second": 0.585,
+      "step": 85
+    },
+    {
+      "epoch": 3.474747474747475,
+      "grad_norm": 1.4371109008789062,
+      "learning_rate": 1.4259259259259259e-05,
+      "loss": 1.822,
+      "step": 86
+    },
+    {
+      "epoch": 3.474747474747475,
+      "eval_loss": 1.7877326011657715,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 86
+    },
+    {
+      "epoch": 3.515151515151515,
+      "grad_norm": 1.4090356826782227,
+      "learning_rate": 1.416666666666667e-05,
+      "loss": 1.7796,
+      "step": 87
+    },
+    {
+      "epoch": 3.515151515151515,
+      "eval_loss": 1.7542084455490112,
+      "eval_runtime": 5.1712,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 87
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 1.382804036140442,
+      "learning_rate": 1.4074074074074075e-05,
+      "loss": 1.7457,
+      "step": 88
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "eval_loss": 1.7219429016113281,
+      "eval_runtime": 5.165,
+      "eval_samples_per_second": 32.139,
+      "eval_steps_per_second": 0.581,
+      "step": 88
+    },
+    {
+      "epoch": 3.595959595959596,
+      "grad_norm": 1.4041632413864136,
+      "learning_rate": 1.3981481481481482e-05,
+      "loss": 1.7043,
+      "step": 89
+    },
+    {
+      "epoch": 3.595959595959596,
+      "eval_loss": 1.689993977546692,
+      "eval_runtime": 5.1401,
+      "eval_samples_per_second": 32.295,
+      "eval_steps_per_second": 0.584,
+      "step": 89
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "grad_norm": 1.3862537145614624,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 1.6742,
+      "step": 90
+    },
+    {
+      "epoch": 3.6363636363636362,
+      "eval_loss": 1.659338116645813,
+      "eval_runtime": 5.1347,
+      "eval_samples_per_second": 32.329,
+      "eval_steps_per_second": 0.584,
+      "step": 90
+    },
+    {
+      "epoch": 3.676767676767677,
+      "grad_norm": 1.3727890253067017,
+      "learning_rate": 1.3796296296296297e-05,
+      "loss": 1.6502,
+      "step": 91
+    },
+    {
+      "epoch": 3.676767676767677,
+      "eval_loss": 1.628204345703125,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 91
+    },
+    {
+      "epoch": 3.717171717171717,
+      "grad_norm": 1.400058388710022,
+      "learning_rate": 1.3703703703703706e-05,
+      "loss": 1.6396,
+      "step": 92
+    },
+    {
+      "epoch": 3.717171717171717,
+      "eval_loss": 1.5992398262023926,
+      "eval_runtime": 5.1584,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 92
+    },
+    {
+      "epoch": 3.757575757575758,
+      "grad_norm": 1.4236724376678467,
+      "learning_rate": 1.3611111111111113e-05,
+      "loss": 1.6123,
+      "step": 93
+    },
+    {
+      "epoch": 3.757575757575758,
+      "eval_loss": 1.5681532621383667,
+      "eval_runtime": 5.157,
+      "eval_samples_per_second": 32.189,
+      "eval_steps_per_second": 0.582,
+      "step": 93
+    },
+    {
+      "epoch": 3.797979797979798,
+      "grad_norm": 1.4351081848144531,
+      "learning_rate": 1.351851851851852e-05,
+      "loss": 1.5512,
+      "step": 94
+    },
+    {
+      "epoch": 3.797979797979798,
+      "eval_loss": 1.5355143547058105,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 94
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "grad_norm": 1.4533002376556396,
+      "learning_rate": 1.3425925925925926e-05,
+      "loss": 1.5697,
+      "step": 95
+    },
+    {
+      "epoch": 3.8383838383838382,
+      "eval_loss": 1.506730318069458,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 95
+    },
+    {
+      "epoch": 3.878787878787879,
+      "grad_norm": 1.4752202033996582,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 1.495,
+      "step": 96
+    },
+    {
+      "epoch": 3.878787878787879,
+      "eval_loss": 1.4759498834609985,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 96
+    },
+    {
+      "epoch": 3.919191919191919,
+      "grad_norm": 1.4492502212524414,
+      "learning_rate": 1.3240740740740742e-05,
+      "loss": 1.4681,
+      "step": 97
+    },
+    {
+      "epoch": 3.919191919191919,
+      "eval_loss": 1.4438410997390747,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.159,
+      "eval_steps_per_second": 0.581,
+      "step": 97
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "grad_norm": 1.4566094875335693,
+      "learning_rate": 1.314814814814815e-05,
+      "loss": 1.4542,
+      "step": 98
+    },
+    {
+      "epoch": 3.9595959595959593,
+      "eval_loss": 1.4141173362731934,
+      "eval_runtime": 5.1691,
+      "eval_samples_per_second": 32.114,
+      "eval_steps_per_second": 0.58,
+      "step": 98
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 1.47097647190094,
+      "learning_rate": 1.3055555555555557e-05,
+      "loss": 1.4428,
+      "step": 99
+    },
+    {
+      "epoch": 4.0,
+      "eval_loss": 1.381595492362976,
+      "eval_runtime": 5.1554,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 99
+    },
+    {
+      "epoch": 4.040404040404041,
+      "grad_norm": 1.4813333749771118,
+      "learning_rate": 1.2962962962962964e-05,
+      "loss": 1.403,
+      "step": 100
+    },
+    {
+      "epoch": 4.040404040404041,
+      "eval_loss": 1.3522053956985474,
+      "eval_runtime": 5.1659,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 100
+    },
+    {
+      "epoch": 4.08080808080808,
+      "grad_norm": 1.500218391418457,
+      "learning_rate": 1.2870370370370371e-05,
+      "loss": 1.3289,
+      "step": 101
+    },
+    {
+      "epoch": 4.08080808080808,
+      "eval_loss": 1.3220492601394653,
+      "eval_runtime": 5.1719,
+      "eval_samples_per_second": 32.096,
+      "eval_steps_per_second": 0.58,
+      "step": 101
+    },
+    {
+      "epoch": 4.121212121212121,
+      "grad_norm": 1.4614801406860352,
+      "learning_rate": 1.2777777777777777e-05,
+      "loss": 1.3151,
+      "step": 102
+    },
+    {
+      "epoch": 4.121212121212121,
+      "eval_loss": 1.2928389310836792,
+      "eval_runtime": 5.1718,
+      "eval_samples_per_second": 32.097,
+      "eval_steps_per_second": 0.58,
+      "step": 102
+    },
+    {
+      "epoch": 4.161616161616162,
+      "grad_norm": 1.4137208461761475,
+      "learning_rate": 1.2685185185185188e-05,
+      "loss": 1.2993,
+      "step": 103
+    },
+    {
+      "epoch": 4.161616161616162,
+      "eval_loss": 1.2663666009902954,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 103
+    },
+    {
+      "epoch": 4.202020202020202,
+      "grad_norm": 1.347014307975769,
+      "learning_rate": 1.2592592592592593e-05,
+      "loss": 1.2506,
+      "step": 104
+    },
+    {
+      "epoch": 4.202020202020202,
+      "eval_loss": 1.242059588432312,
+      "eval_runtime": 5.1713,
+      "eval_samples_per_second": 32.101,
+      "eval_steps_per_second": 0.58,
+      "step": 104
+    },
+    {
+      "epoch": 4.242424242424242,
+      "grad_norm": 1.3283647298812866,
+      "learning_rate": 1.25e-05,
+      "loss": 1.2453,
+      "step": 105
+    },
+    {
+      "epoch": 4.242424242424242,
+      "eval_loss": 1.2196069955825806,
+      "eval_runtime": 5.1174,
+      "eval_samples_per_second": 32.438,
+      "eval_steps_per_second": 0.586,
+      "step": 105
+    },
+    {
+      "epoch": 4.282828282828283,
+      "grad_norm": 1.2796555757522583,
+      "learning_rate": 1.2407407407407408e-05,
+      "loss": 1.1805,
+      "step": 106
+    },
+    {
+      "epoch": 4.282828282828283,
+      "eval_loss": 1.1993300914764404,
+      "eval_runtime": 5.125,
+      "eval_samples_per_second": 32.39,
+      "eval_steps_per_second": 0.585,
+      "step": 106
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "grad_norm": 1.264662742614746,
+      "learning_rate": 1.2314814814814815e-05,
+      "loss": 1.1939,
+      "step": 107
+    },
+    {
+      "epoch": 4.3232323232323235,
+      "eval_loss": 1.1796401739120483,
+      "eval_runtime": 5.154,
+      "eval_samples_per_second": 32.208,
+      "eval_steps_per_second": 0.582,
+      "step": 107
+    },
+    {
+      "epoch": 4.363636363636363,
+      "grad_norm": 1.2420271635055542,
+      "learning_rate": 1.2222222222222224e-05,
+      "loss": 1.1684,
+      "step": 108
+    },
+    {
+      "epoch": 4.363636363636363,
+      "eval_loss": 1.161789894104004,
+      "eval_runtime": 5.1599,
+      "eval_samples_per_second": 32.171,
+      "eval_steps_per_second": 0.581,
+      "step": 108
+    },
+    {
+      "epoch": 4.404040404040404,
+      "grad_norm": 1.2369439601898193,
+      "learning_rate": 1.2129629629629631e-05,
+      "loss": 1.1893,
+      "step": 109
+    },
+    {
+      "epoch": 4.404040404040404,
+      "eval_loss": 1.1432855129241943,
+      "eval_runtime": 5.1704,
+      "eval_samples_per_second": 32.106,
+      "eval_steps_per_second": 0.58,
+      "step": 109
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.1795873641967773,
+      "learning_rate": 1.2037037037037039e-05,
+      "loss": 1.1554,
+      "step": 110
+    },
+    {
+      "epoch": 4.444444444444445,
+      "eval_loss": 1.126727819442749,
+      "eval_runtime": 5.1629,
+      "eval_samples_per_second": 32.153,
+      "eval_steps_per_second": 0.581,
+      "step": 110
+    },
+    {
+      "epoch": 4.484848484848484,
+      "grad_norm": 1.1451513767242432,
+      "learning_rate": 1.1944444444444444e-05,
+      "loss": 1.1276,
+      "step": 111
+    },
+    {
+      "epoch": 4.484848484848484,
+      "eval_loss": 1.1112148761749268,
+      "eval_runtime": 5.1514,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 111
+    },
+    {
+      "epoch": 4.525252525252525,
+      "grad_norm": 1.124099612236023,
+      "learning_rate": 1.1851851851851852e-05,
+      "loss": 1.0972,
+      "step": 112
+    },
+    {
+      "epoch": 4.525252525252525,
+      "eval_loss": 1.0960901975631714,
+      "eval_runtime": 5.092,
+      "eval_samples_per_second": 32.6,
+      "eval_steps_per_second": 0.589,
+      "step": 112
+    },
+    {
+      "epoch": 4.565656565656566,
+      "grad_norm": 1.090491533279419,
+      "learning_rate": 1.1759259259259259e-05,
+      "loss": 1.0821,
+      "step": 113
+    },
+    {
+      "epoch": 4.565656565656566,
+      "eval_loss": 1.0813062191009521,
+      "eval_runtime": 5.0997,
+      "eval_samples_per_second": 32.551,
+      "eval_steps_per_second": 0.588,
+      "step": 113
+    },
+    {
+      "epoch": 4.606060606060606,
+      "grad_norm": 1.0799589157104492,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 1.038,
+      "step": 114
+    },
+    {
+      "epoch": 4.606060606060606,
+      "eval_loss": 1.0665714740753174,
+      "eval_runtime": 5.1435,
+      "eval_samples_per_second": 32.273,
+      "eval_steps_per_second": 0.583,
+      "step": 114
+    },
+    {
+      "epoch": 4.646464646464646,
+      "grad_norm": 1.0503891706466675,
+      "learning_rate": 1.1574074074074075e-05,
+      "loss": 1.0584,
+      "step": 115
+    },
+    {
+      "epoch": 4.646464646464646,
+      "eval_loss": 1.0545704364776611,
+      "eval_runtime": 5.1517,
+      "eval_samples_per_second": 32.222,
+      "eval_steps_per_second": 0.582,
+      "step": 115
+    },
+    {
+      "epoch": 4.686868686868687,
+      "grad_norm": 1.043385624885559,
+      "learning_rate": 1.1481481481481482e-05,
+      "loss": 1.0329,
+      "step": 116
+    },
+    {
+      "epoch": 4.686868686868687,
+      "eval_loss": 1.0427477359771729,
+      "eval_runtime": 5.1504,
+      "eval_samples_per_second": 32.23,
+      "eval_steps_per_second": 0.582,
+      "step": 116
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "grad_norm": 1.0548285245895386,
+      "learning_rate": 1.138888888888889e-05,
+      "loss": 1.0256,
+      "step": 117
+    },
+    {
+      "epoch": 4.7272727272727275,
+      "eval_loss": 1.0287798643112183,
+      "eval_runtime": 5.1603,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 117
+    },
+    {
+      "epoch": 4.767676767676767,
+      "grad_norm": 1.0389068126678467,
+      "learning_rate": 1.1296296296296297e-05,
+      "loss": 1.0192,
+      "step": 118
+    },
+    {
+      "epoch": 4.767676767676767,
+      "eval_loss": 1.0174580812454224,
+      "eval_runtime": 5.1433,
+      "eval_samples_per_second": 32.275,
+      "eval_steps_per_second": 0.583,
+      "step": 118
+    },
+    {
+      "epoch": 4.808080808080808,
+      "grad_norm": 1.035844087600708,
+      "learning_rate": 1.1203703703703706e-05,
+      "loss": 1.0101,
+      "step": 119
+    },
+    {
+      "epoch": 4.808080808080808,
+      "eval_loss": 1.006226658821106,
+      "eval_runtime": 5.1693,
+      "eval_samples_per_second": 32.113,
+      "eval_steps_per_second": 0.58,
+      "step": 119
+    },
+    {
+      "epoch": 4.848484848484849,
+      "grad_norm": 1.050945520401001,
+      "learning_rate": 1.1111111111111113e-05,
+      "loss": 1.0403,
+      "step": 120
+    },
+    {
+      "epoch": 4.848484848484849,
+      "eval_loss": 0.9945307374000549,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.182,
+      "eval_steps_per_second": 0.582,
+      "step": 120
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 1.027295708656311,
+      "learning_rate": 1.1018518518518519e-05,
+      "loss": 1.0038,
+      "step": 121
+    },
+    {
+      "epoch": 4.888888888888889,
+      "eval_loss": 0.9830649495124817,
+      "eval_runtime": 5.1555,
+      "eval_samples_per_second": 32.199,
+      "eval_steps_per_second": 0.582,
+      "step": 121
+    },
+    {
+      "epoch": 4.929292929292929,
+      "grad_norm": 1.035843014717102,
+      "learning_rate": 1.0925925925925926e-05,
+      "loss": 0.9544,
+      "step": 122
+    },
+    {
+      "epoch": 4.929292929292929,
+      "eval_loss": 0.9728271961212158,
+      "eval_runtime": 5.1521,
+      "eval_samples_per_second": 32.22,
+      "eval_steps_per_second": 0.582,
+      "step": 122
+    },
+    {
+      "epoch": 4.96969696969697,
+      "grad_norm": 1.050779938697815,
+      "learning_rate": 1.0833333333333334e-05,
+      "loss": 0.9393,
+      "step": 123
+    },
+    {
+      "epoch": 4.96969696969697,
+      "eval_loss": 0.9609534740447998,
+      "eval_runtime": 5.1551,
+      "eval_samples_per_second": 32.201,
+      "eval_steps_per_second": 0.582,
+      "step": 123
+    },
+    {
+      "epoch": 5.01010101010101,
+      "grad_norm": 1.0694254636764526,
+      "learning_rate": 1.0740740740740742e-05,
+      "loss": 0.9578,
+      "step": 124
+    },
+    {
+      "epoch": 5.01010101010101,
+      "eval_loss": 0.9495998620986938,
+      "eval_runtime": 5.1502,
+      "eval_samples_per_second": 32.232,
+      "eval_steps_per_second": 0.583,
+      "step": 124
+    },
+    {
+      "epoch": 5.05050505050505,
+      "grad_norm": 1.0505897998809814,
+      "learning_rate": 1.064814814814815e-05,
+      "loss": 0.9228,
+      "step": 125
+    },
+    {
+      "epoch": 5.05050505050505,
+      "eval_loss": 0.9396913051605225,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 125
+    },
+    {
+      "epoch": 5.090909090909091,
+      "grad_norm": 1.082509160041809,
+      "learning_rate": 1.0555555555555557e-05,
+      "loss": 0.9583,
+      "step": 126
+    },
+    {
+      "epoch": 5.090909090909091,
+      "eval_loss": 0.9291126132011414,
+      "eval_runtime": 5.1661,
+      "eval_samples_per_second": 32.132,
+      "eval_steps_per_second": 0.581,
+      "step": 126
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "grad_norm": 1.0709625482559204,
+      "learning_rate": 1.0462962962962964e-05,
+      "loss": 0.9163,
+      "step": 127
+    },
+    {
+      "epoch": 5.1313131313131315,
+      "eval_loss": 0.9183955192565918,
+      "eval_runtime": 5.159,
+      "eval_samples_per_second": 32.177,
+      "eval_steps_per_second": 0.582,
+      "step": 127
+    },
+    {
+      "epoch": 5.171717171717171,
+      "grad_norm": 1.042419195175171,
+      "learning_rate": 1.037037037037037e-05,
+      "loss": 0.923,
+      "step": 128
+    },
+    {
+      "epoch": 5.171717171717171,
+      "eval_loss": 0.9072820544242859,
+      "eval_runtime": 5.1597,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 128
+    },
+    {
+      "epoch": 5.212121212121212,
+      "grad_norm": 1.0583133697509766,
+      "learning_rate": 1.0277777777777777e-05,
+      "loss": 0.9328,
+      "step": 129
+    },
+    {
+      "epoch": 5.212121212121212,
+      "eval_loss": 0.897919237613678,
+      "eval_runtime": 5.15,
+      "eval_samples_per_second": 32.233,
+      "eval_steps_per_second": 0.583,
+      "step": 129
+    },
+    {
+      "epoch": 5.252525252525253,
+      "grad_norm": 1.0396648645401,
+      "learning_rate": 1.0185185185185186e-05,
+      "loss": 0.9001,
+      "step": 130
+    },
+    {
+      "epoch": 5.252525252525253,
+      "eval_loss": 0.8888950347900391,
+      "eval_runtime": 5.1631,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 130
+    },
+    {
+      "epoch": 5.292929292929293,
+      "grad_norm": 1.0212734937667847,
+      "learning_rate": 1.0092592592592594e-05,
+      "loss": 0.8731,
+      "step": 131
+    },
+    {
+      "epoch": 5.292929292929293,
+      "eval_loss": 0.8788275718688965,
+      "eval_runtime": 5.1742,
+      "eval_samples_per_second": 32.082,
+      "eval_steps_per_second": 0.58,
+      "step": 131
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 1.0188337564468384,
+      "learning_rate": 1e-05,
+      "loss": 0.8328,
+      "step": 132
+    },
+    {
+      "epoch": 5.333333333333333,
+      "eval_loss": 0.8719209432601929,
+      "eval_runtime": 5.1515,
+      "eval_samples_per_second": 32.224,
+      "eval_steps_per_second": 0.582,
+      "step": 132
+    },
+    {
+      "epoch": 5.373737373737374,
+      "grad_norm": 1.010650873184204,
+      "learning_rate": 9.907407407407408e-06,
+      "loss": 0.8401,
+      "step": 133
+    },
+    {
+      "epoch": 5.373737373737374,
+      "eval_loss": 0.8616573810577393,
+      "eval_runtime": 5.1695,
+      "eval_samples_per_second": 32.111,
+      "eval_steps_per_second": 0.58,
+      "step": 133
+    },
+    {
+      "epoch": 5.414141414141414,
+      "grad_norm": 0.9998605847358704,
+      "learning_rate": 9.814814814814815e-06,
+      "loss": 0.8664,
+      "step": 134
+    },
+    {
+      "epoch": 5.414141414141414,
+      "eval_loss": 0.8543541431427002,
+      "eval_runtime": 5.1642,
+      "eval_samples_per_second": 32.144,
+      "eval_steps_per_second": 0.581,
+      "step": 134
+    },
+    {
+      "epoch": 5.454545454545454,
+      "grad_norm": 0.9973117113113403,
+      "learning_rate": 9.722222222222223e-06,
+      "loss": 0.8414,
+      "step": 135
+    },
+    {
+      "epoch": 5.454545454545454,
+      "eval_loss": 0.8471528887748718,
+      "eval_runtime": 5.1595,
+      "eval_samples_per_second": 32.173,
+      "eval_steps_per_second": 0.581,
+      "step": 135
+    },
+    {
+      "epoch": 5.494949494949495,
+      "grad_norm": 1.0174371004104614,
+      "learning_rate": 9.62962962962963e-06,
+      "loss": 0.863,
+      "step": 136
+    },
+    {
+      "epoch": 5.494949494949495,
+      "eval_loss": 0.839015543460846,
+      "eval_runtime": 5.1632,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 136
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "grad_norm": 1.0131112337112427,
+      "learning_rate": 9.537037037037037e-06,
+      "loss": 0.8296,
+      "step": 137
+    },
+    {
+      "epoch": 5.5353535353535355,
+      "eval_loss": 0.831695556640625,
+      "eval_runtime": 5.1598,
+      "eval_samples_per_second": 32.172,
+      "eval_steps_per_second": 0.581,
+      "step": 137
+    },
+    {
+      "epoch": 5.575757575757576,
+      "grad_norm": 1.0262633562088013,
+      "learning_rate": 9.444444444444445e-06,
+      "loss": 0.854,
+      "step": 138
+    },
+    {
+      "epoch": 5.575757575757576,
+      "eval_loss": 0.8235994577407837,
+      "eval_runtime": 5.1621,
+      "eval_samples_per_second": 32.158,
+      "eval_steps_per_second": 0.581,
+      "step": 138
+    },
+    {
+      "epoch": 5.616161616161616,
+      "grad_norm": 1.0295422077178955,
+      "learning_rate": 9.351851851851854e-06,
+      "loss": 0.8194,
+      "step": 139
+    },
+    {
+      "epoch": 5.616161616161616,
+      "eval_loss": 0.8169118762016296,
+      "eval_runtime": 5.168,
+      "eval_samples_per_second": 32.121,
+      "eval_steps_per_second": 0.58,
+      "step": 139
+    },
+    {
+      "epoch": 5.656565656565657,
+      "grad_norm": 1.0281344652175903,
+      "learning_rate": 9.25925925925926e-06,
+      "loss": 0.7902,
+      "step": 140
+    },
+    {
+      "epoch": 5.656565656565657,
+      "eval_loss": 0.8098058104515076,
+      "eval_runtime": 5.1581,
+      "eval_samples_per_second": 32.183,
+      "eval_steps_per_second": 0.582,
+      "step": 140
+    },
+    {
+      "epoch": 5.696969696969697,
+      "grad_norm": 1.0528678894042969,
+      "learning_rate": 9.166666666666666e-06,
+      "loss": 0.7939,
+      "step": 141
+    },
+    {
+      "epoch": 5.696969696969697,
+      "eval_loss": 0.8031502366065979,
+      "eval_runtime": 5.1541,
+      "eval_samples_per_second": 32.207,
+      "eval_steps_per_second": 0.582,
+      "step": 141
+    },
+    {
+      "epoch": 5.737373737373737,
+      "grad_norm": 1.0525215864181519,
+      "learning_rate": 9.074074074074075e-06,
+      "loss": 0.8127,
+      "step": 142
+    },
+    {
+      "epoch": 5.737373737373737,
+      "eval_loss": 0.7957438230514526,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.205,
+      "eval_steps_per_second": 0.582,
+      "step": 142
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 1.0634738206863403,
+      "learning_rate": 8.981481481481483e-06,
+      "loss": 0.7831,
+      "step": 143
+    },
+    {
+      "epoch": 5.777777777777778,
+      "eval_loss": 0.7881508469581604,
+      "eval_runtime": 5.1489,
+      "eval_samples_per_second": 32.24,
+      "eval_steps_per_second": 0.583,
+      "step": 143
+    },
+    {
+      "epoch": 5.818181818181818,
+      "grad_norm": 1.0633102655410767,
+      "learning_rate": 8.888888888888888e-06,
+      "loss": 0.7511,
+      "step": 144
+    },
+    {
+      "epoch": 5.818181818181818,
+      "eval_loss": 0.7818319201469421,
+      "eval_runtime": 5.169,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 144
+    },
+    {
+      "epoch": 5.858585858585858,
+      "grad_norm": 1.053615689277649,
+      "learning_rate": 8.796296296296297e-06,
+      "loss": 0.75,
+      "step": 145
+    },
+    {
+      "epoch": 5.858585858585858,
+      "eval_loss": 0.7745649814605713,
+      "eval_runtime": 5.1694,
+      "eval_samples_per_second": 32.112,
+      "eval_steps_per_second": 0.58,
+      "step": 145
+    },
+    {
+      "epoch": 5.898989898989899,
+      "grad_norm": 1.0772610902786255,
+      "learning_rate": 8.703703703703705e-06,
+      "loss": 0.7929,
+      "step": 146
+    },
+    {
+      "epoch": 5.898989898989899,
+      "eval_loss": 0.7681305408477783,
+      "eval_runtime": 5.1618,
+      "eval_samples_per_second": 32.16,
+      "eval_steps_per_second": 0.581,
+      "step": 146
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "grad_norm": 1.0765084028244019,
+      "learning_rate": 8.611111111111112e-06,
+      "loss": 0.7642,
+      "step": 147
+    },
+    {
+      "epoch": 5.9393939393939394,
+      "eval_loss": 0.7618209719657898,
+      "eval_runtime": 5.1689,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 147
+    },
+    {
+      "epoch": 5.97979797979798,
+      "grad_norm": 1.0824408531188965,
+      "learning_rate": 8.518518518518519e-06,
+      "loss": 0.7387,
+      "step": 148
+    },
+    {
+      "epoch": 5.97979797979798,
+      "eval_loss": 0.755460798740387,
+      "eval_runtime": 5.1544,
+      "eval_samples_per_second": 32.206,
+      "eval_steps_per_second": 0.582,
+      "step": 148
+    },
+    {
+      "epoch": 6.02020202020202,
+      "grad_norm": 1.0975412130355835,
+      "learning_rate": 8.425925925925926e-06,
+      "loss": 0.7218,
+      "step": 149
+    },
+    {
+      "epoch": 6.02020202020202,
+      "eval_loss": 0.749002993106842,
+      "eval_runtime": 5.1624,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 149
+    },
+    {
+      "epoch": 6.0606060606060606,
+      "grad_norm": 1.1096184253692627,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 0.712,
+      "step": 150
+    },
+    {
+      "epoch": 6.0606060606060606,
+      "eval_loss": 0.7422470450401306,
+      "eval_runtime": 5.1674,
+      "eval_samples_per_second": 32.124,
+      "eval_steps_per_second": 0.581,
+      "step": 150
+    },
+    {
+      "epoch": 6.101010101010101,
+      "grad_norm": 1.1530879735946655,
+      "learning_rate": 8.240740740740741e-06,
+      "loss": 0.7268,
+      "step": 151
+    },
+    {
+      "epoch": 6.101010101010101,
+      "eval_loss": 0.7364265322685242,
+      "eval_runtime": 5.1645,
+      "eval_samples_per_second": 32.142,
+      "eval_steps_per_second": 0.581,
+      "step": 151
+    },
+    {
+      "epoch": 6.141414141414141,
+      "grad_norm": 1.1592988967895508,
+      "learning_rate": 8.148148148148148e-06,
+      "loss": 0.7258,
+      "step": 152
+    },
+    {
+      "epoch": 6.141414141414141,
+      "eval_loss": 0.7302441596984863,
+      "eval_runtime": 5.1636,
+      "eval_samples_per_second": 32.148,
+      "eval_steps_per_second": 0.581,
+      "step": 152
+    },
+    {
+      "epoch": 6.181818181818182,
+      "grad_norm": 1.1833539009094238,
+      "learning_rate": 8.055555555555557e-06,
+      "loss": 0.7224,
+      "step": 153
+    },
+    {
+      "epoch": 6.181818181818182,
+      "eval_loss": 0.7228173017501831,
+      "eval_runtime": 5.1641,
+      "eval_samples_per_second": 32.145,
+      "eval_steps_per_second": 0.581,
+      "step": 153
+    },
+    {
+      "epoch": 6.222222222222222,
+      "grad_norm": 1.1936936378479004,
+      "learning_rate": 7.962962962962963e-06,
+      "loss": 0.6966,
+      "step": 154
+    },
+    {
+      "epoch": 6.222222222222222,
+      "eval_loss": 0.716472327709198,
+      "eval_runtime": 5.1605,
+      "eval_samples_per_second": 32.167,
+      "eval_steps_per_second": 0.581,
+      "step": 154
+    },
+    {
+      "epoch": 6.262626262626263,
+      "grad_norm": 1.245132565498352,
+      "learning_rate": 7.870370370370372e-06,
+      "loss": 0.7028,
+      "step": 155
+    },
+    {
+      "epoch": 6.262626262626263,
+      "eval_loss": 0.7087136507034302,
+      "eval_runtime": 5.1583,
+      "eval_samples_per_second": 32.181,
+      "eval_steps_per_second": 0.582,
+      "step": 155
+    },
+    {
+      "epoch": 6.303030303030303,
+      "grad_norm": 1.2659085988998413,
+      "learning_rate": 7.77777777777778e-06,
+      "loss": 0.7051,
+      "step": 156
+    },
+    {
+      "epoch": 6.303030303030303,
+      "eval_loss": 0.7009304165840149,
+      "eval_runtime": 5.1665,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 156
+    },
+    {
+      "epoch": 6.343434343434343,
+      "grad_norm": 1.3588238954544067,
+      "learning_rate": 7.685185185185185e-06,
+      "loss": 0.7004,
+      "step": 157
+    },
+    {
+      "epoch": 6.343434343434343,
+      "eval_loss": 0.6932580471038818,
+      "eval_runtime": 5.1604,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 157
+    },
+    {
+      "epoch": 6.383838383838384,
+      "grad_norm": 1.3982634544372559,
+      "learning_rate": 7.592592592592594e-06,
+      "loss": 0.68,
+      "step": 158
+    },
+    {
+      "epoch": 6.383838383838384,
+      "eval_loss": 0.6845574378967285,
+      "eval_runtime": 5.1575,
+      "eval_samples_per_second": 32.186,
+      "eval_steps_per_second": 0.582,
+      "step": 158
+    },
+    {
+      "epoch": 6.424242424242424,
+      "grad_norm": 1.426499605178833,
+      "learning_rate": 7.500000000000001e-06,
+      "loss": 0.6641,
+      "step": 159
+    },
+    {
+      "epoch": 6.424242424242424,
+      "eval_loss": 0.6760482788085938,
+      "eval_runtime": 5.1632,
+      "eval_samples_per_second": 32.151,
+      "eval_steps_per_second": 0.581,
+      "step": 159
+    },
+    {
+      "epoch": 6.4646464646464645,
+      "grad_norm": 1.4880263805389404,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 0.695,
+      "step": 160
+    },
+    {
+      "epoch": 6.4646464646464645,
+      "eval_loss": 0.6670196652412415,
+      "eval_runtime": 5.1633,
+      "eval_samples_per_second": 32.15,
+      "eval_steps_per_second": 0.581,
+      "step": 160
+    },
+    {
+      "epoch": 6.505050505050505,
+      "grad_norm": 1.524919033050537,
+      "learning_rate": 7.314814814814816e-06,
+      "loss": 0.6316,
+      "step": 161
+    },
+    {
+      "epoch": 6.505050505050505,
+      "eval_loss": 0.6562865972518921,
+      "eval_runtime": 5.1726,
+      "eval_samples_per_second": 32.092,
+      "eval_steps_per_second": 0.58,
+      "step": 161
+    },
+    {
+      "epoch": 6.545454545454545,
+      "grad_norm": 1.542138934135437,
+      "learning_rate": 7.222222222222223e-06,
+      "loss": 0.6491,
+      "step": 162
+    },
+    {
+      "epoch": 6.545454545454545,
+      "eval_loss": 0.6472378969192505,
+      "eval_runtime": 5.1671,
+      "eval_samples_per_second": 32.126,
+      "eval_steps_per_second": 0.581,
+      "step": 162
+    },
+    {
+      "epoch": 6.585858585858586,
+      "grad_norm": 1.5400753021240234,
+      "learning_rate": 7.129629629629629e-06,
+      "loss": 0.6324,
+      "step": 163
+    },
+    {
+      "epoch": 6.585858585858586,
+      "eval_loss": 0.6379138231277466,
+      "eval_runtime": 5.1556,
+      "eval_samples_per_second": 32.198,
+      "eval_steps_per_second": 0.582,
+      "step": 163
+    },
+    {
+      "epoch": 6.626262626262626,
+      "grad_norm": 1.4753589630126953,
+      "learning_rate": 7.0370370370370375e-06,
+      "loss": 0.6473,
+      "step": 164
+    },
+    {
+      "epoch": 6.626262626262626,
+      "eval_loss": 0.6301048994064331,
+      "eval_runtime": 5.1423,
+      "eval_samples_per_second": 32.281,
+      "eval_steps_per_second": 0.583,
+      "step": 164
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 1.370703935623169,
+      "learning_rate": 6.944444444444445e-06,
+      "loss": 0.6255,
+      "step": 165
+    },
+    {
+      "epoch": 6.666666666666667,
+      "eval_loss": 0.6244276762008667,
+      "eval_runtime": 5.1569,
+      "eval_samples_per_second": 32.19,
+      "eval_steps_per_second": 0.582,
+      "step": 165
+    },
+    {
+      "epoch": 6.707070707070707,
+      "grad_norm": 1.2907015085220337,
+      "learning_rate": 6.851851851851853e-06,
+      "loss": 0.6029,
+      "step": 166
+    },
+    {
+      "epoch": 6.707070707070707,
+      "eval_loss": 0.6191242933273315,
+      "eval_runtime": 5.1685,
+      "eval_samples_per_second": 32.118,
+      "eval_steps_per_second": 0.58,
+      "step": 166
+    },
+    {
+      "epoch": 6.747474747474747,
+      "grad_norm": 1.206007480621338,
+      "learning_rate": 6.75925925925926e-06,
+      "loss": 0.5831,
+      "step": 167
+    },
+    {
+      "epoch": 6.747474747474747,
+      "eval_loss": 0.6155738830566406,
+      "eval_runtime": 5.1689,
+      "eval_samples_per_second": 32.115,
+      "eval_steps_per_second": 0.58,
+      "step": 167
+    },
+    {
+      "epoch": 6.787878787878788,
+      "grad_norm": 1.14729642868042,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 0.6167,
+      "step": 168
+    },
+    {
+      "epoch": 6.787878787878788,
+      "eval_loss": 0.6123696565628052,
+      "eval_runtime": 5.1612,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 168
+    },
+    {
+      "epoch": 6.828282828282829,
+      "grad_norm": 1.1298456192016602,
+      "learning_rate": 6.574074074074075e-06,
+      "loss": 0.597,
+      "step": 169
+    },
+    {
+      "epoch": 6.828282828282829,
+      "eval_loss": 0.6096829771995544,
+      "eval_runtime": 5.1532,
+      "eval_samples_per_second": 32.213,
+      "eval_steps_per_second": 0.582,
+      "step": 169
+    },
+    {
+      "epoch": 6.8686868686868685,
+      "grad_norm": 1.129176378250122,
+      "learning_rate": 6.481481481481482e-06,
+      "loss": 0.6278,
+      "step": 170
+    },
+    {
+      "epoch": 6.8686868686868685,
+      "eval_loss": 0.6058310270309448,
+      "eval_runtime": 5.1654,
+      "eval_samples_per_second": 32.137,
+      "eval_steps_per_second": 0.581,
+      "step": 170
+    },
+    {
+      "epoch": 6.909090909090909,
+      "grad_norm": 1.1008386611938477,
+      "learning_rate": 6.3888888888888885e-06,
+      "loss": 0.5974,
+      "step": 171
+    },
+    {
+      "epoch": 6.909090909090909,
+      "eval_loss": 0.6041752696037292,
+      "eval_runtime": 5.1564,
+      "eval_samples_per_second": 32.193,
+      "eval_steps_per_second": 0.582,
+      "step": 171
+    },
+    {
+      "epoch": 6.94949494949495,
+      "grad_norm": 1.0789906978607178,
+      "learning_rate": 6.296296296296297e-06,
+      "loss": 0.5983,
+      "step": 172
+    },
+    {
+      "epoch": 6.94949494949495,
+      "eval_loss": 0.6021236181259155,
+      "eval_runtime": 5.1613,
+      "eval_samples_per_second": 32.163,
+      "eval_steps_per_second": 0.581,
+      "step": 172
+    },
+    {
+      "epoch": 6.98989898989899,
+      "grad_norm": 1.1015055179595947,
+      "learning_rate": 6.203703703703704e-06,
+      "loss": 0.6138,
+      "step": 173
+    },
+    {
+      "epoch": 6.98989898989899,
+      "eval_loss": 0.6002500653266907,
+      "eval_runtime": 5.1625,
+      "eval_samples_per_second": 32.155,
+      "eval_steps_per_second": 0.581,
+      "step": 173
+    },
+    {
+      "epoch": 7.03030303030303,
+      "grad_norm": 1.0980877876281738,
+      "learning_rate": 6.111111111111112e-06,
+      "loss": 0.5882,
+      "step": 174
+    },
+    {
+      "epoch": 7.03030303030303,
+      "eval_loss": 0.5983395576477051,
+      "eval_runtime": 5.1587,
+      "eval_samples_per_second": 32.179,
+      "eval_steps_per_second": 0.582,
+      "step": 174
+    },
+    {
+      "epoch": 7.070707070707071,
+      "grad_norm": 1.1048911809921265,
+      "learning_rate": 6.018518518518519e-06,
+      "loss": 0.5746,
+      "step": 175
+    },
+    {
+      "epoch": 7.070707070707071,
+      "eval_loss": 0.5960097908973694,
+      "eval_runtime": 5.1657,
+      "eval_samples_per_second": 32.135,
+      "eval_steps_per_second": 0.581,
+      "step": 175
+    },
+    {
+      "epoch": 7.111111111111111,
+      "grad_norm": 1.1066044569015503,
+      "learning_rate": 5.925925925925926e-06,
+      "loss": 0.5812,
+      "step": 176
+    },
+    {
+      "epoch": 7.111111111111111,
+      "eval_loss": 0.5940980315208435,
+      "eval_runtime": 5.1696,
+      "eval_samples_per_second": 32.111,
+      "eval_steps_per_second": 0.58,
+      "step": 176
+    },
+    {
+      "epoch": 7.151515151515151,
+      "grad_norm": 1.1311701536178589,
+      "learning_rate": 5.833333333333334e-06,
+      "loss": 0.6367,
+      "step": 177
+    },
+    {
+      "epoch": 7.151515151515151,
+      "eval_loss": 0.5925431847572327,
+      "eval_runtime": 5.1587,
+      "eval_samples_per_second": 32.178,
+      "eval_steps_per_second": 0.582,
+      "step": 177
+    },
+    {
+      "epoch": 7.191919191919192,
+      "grad_norm": 1.1191725730895996,
+      "learning_rate": 5.740740740740741e-06,
+      "loss": 0.5794,
+      "step": 178
+    },
+    {
+      "epoch": 7.191919191919192,
+      "eval_loss": 0.5903013944625854,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 178
+    },
+    {
+      "epoch": 7.232323232323233,
+      "grad_norm": 1.1181325912475586,
+      "learning_rate": 5.6481481481481485e-06,
+      "loss": 0.5759,
+      "step": 179
+    },
+    {
+      "epoch": 7.232323232323233,
+      "eval_loss": 0.5883893370628357,
+      "eval_runtime": 5.1687,
+      "eval_samples_per_second": 32.117,
+      "eval_steps_per_second": 0.58,
+      "step": 179
+    },
+    {
+      "epoch": 7.2727272727272725,
+      "grad_norm": 1.1254656314849854,
+      "learning_rate": 5.555555555555557e-06,
+      "loss": 0.5733,
+      "step": 180
+    },
+    {
+      "epoch": 7.2727272727272725,
+      "eval_loss": 0.5864142775535583,
+      "eval_runtime": 5.1679,
+      "eval_samples_per_second": 32.121,
+      "eval_steps_per_second": 0.581,
+      "step": 180
+    },
+    {
+      "epoch": 7.313131313131313,
+      "grad_norm": 1.1285914182662964,
+      "learning_rate": 5.462962962962963e-06,
+      "loss": 0.5763,
+      "step": 181
+    },
+    {
+      "epoch": 7.313131313131313,
+      "eval_loss": 0.5846003293991089,
+      "eval_runtime": 5.15,
+      "eval_samples_per_second": 32.233,
+      "eval_steps_per_second": 0.583,
+      "step": 181
+    },
+    {
+      "epoch": 7.353535353535354,
+      "grad_norm": 1.1444602012634277,
+      "learning_rate": 5.370370370370371e-06,
+      "loss": 0.5624,
+      "step": 182
+    },
+    {
+      "epoch": 7.353535353535354,
+      "eval_loss": 0.582878589630127,
+      "eval_runtime": 5.1666,
+      "eval_samples_per_second": 32.13,
+      "eval_steps_per_second": 0.581,
+      "step": 182
+    },
+    {
+      "epoch": 7.393939393939394,
+      "grad_norm": 1.1658940315246582,
+      "learning_rate": 5.2777777777777785e-06,
+      "loss": 0.5963,
+      "step": 183
+    },
+    {
+      "epoch": 7.393939393939394,
+      "eval_loss": 0.581150472164154,
+      "eval_runtime": 5.1617,
+      "eval_samples_per_second": 32.16,
+      "eval_steps_per_second": 0.581,
+      "step": 183
+    },
+    {
+      "epoch": 7.434343434343434,
+      "grad_norm": 1.1656270027160645,
+      "learning_rate": 5.185185185185185e-06,
+      "loss": 0.5669,
+      "step": 184
+    },
+    {
+      "epoch": 7.434343434343434,
+      "eval_loss": 0.5794562101364136,
+      "eval_runtime": 5.1731,
+      "eval_samples_per_second": 32.089,
+      "eval_steps_per_second": 0.58,
+      "step": 184
+    },
+    {
+      "epoch": 7.474747474747475,
+      "grad_norm": 1.1592910289764404,
+      "learning_rate": 5.092592592592593e-06,
+      "loss": 0.5797,
+      "step": 185
+    },
+    {
+      "epoch": 7.474747474747475,
+      "eval_loss": 0.5789611339569092,
+      "eval_runtime": 5.1589,
+      "eval_samples_per_second": 32.177,
+      "eval_steps_per_second": 0.582,
+      "step": 185
+    },
+    {
+      "epoch": 7.515151515151516,
+      "grad_norm": 1.1633867025375366,
+      "learning_rate": 5e-06,
+      "loss": 0.5483,
+      "step": 186
+    },
+    {
+      "epoch": 7.515151515151516,
+      "eval_loss": 0.577148973941803,
+      "eval_runtime": 5.1615,
+      "eval_samples_per_second": 32.161,
+      "eval_steps_per_second": 0.581,
+      "step": 186
+    },
+    {
+      "epoch": 7.555555555555555,
+      "grad_norm": 1.1686466932296753,
+      "learning_rate": 4.907407407407408e-06,
+      "loss": 0.568,
+      "step": 187
+    },
+    {
+      "epoch": 7.555555555555555,
+      "eval_loss": 0.5747854709625244,
+      "eval_runtime": 5.1623,
+      "eval_samples_per_second": 32.156,
+      "eval_steps_per_second": 0.581,
+      "step": 187
+    },
+    {
+      "epoch": 7.595959595959596,
+      "grad_norm": 1.1744942665100098,
+      "learning_rate": 4.814814814814815e-06,
+      "loss": 0.5616,
+      "step": 188
+    },
+    {
+      "epoch": 7.595959595959596,
+      "eval_loss": 0.5750288367271423,
+      "eval_runtime": 5.1635,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 188
+    },
+    {
+      "epoch": 7.636363636363637,
+      "grad_norm": 1.1775929927825928,
+      "learning_rate": 4.722222222222222e-06,
+      "loss": 0.5502,
+      "step": 189
+    },
+    {
+      "epoch": 7.636363636363637,
+      "eval_loss": 0.5720074772834778,
+      "eval_runtime": 5.154,
+      "eval_samples_per_second": 32.208,
+      "eval_steps_per_second": 0.582,
+      "step": 189
+    },
+    {
+      "epoch": 7.6767676767676765,
+      "grad_norm": 1.195663332939148,
+      "learning_rate": 4.62962962962963e-06,
+      "loss": 0.5578,
+      "step": 190
+    },
+    {
+      "epoch": 7.6767676767676765,
+      "eval_loss": 0.5705359578132629,
+      "eval_runtime": 5.1634,
+      "eval_samples_per_second": 32.149,
+      "eval_steps_per_second": 0.581,
+      "step": 190
+    },
+    {
+      "epoch": 7.717171717171717,
+      "grad_norm": 1.208178162574768,
+      "learning_rate": 4.537037037037038e-06,
+      "loss": 0.5488,
+      "step": 191
+    },
+    {
+      "epoch": 7.717171717171717,
+      "eval_loss": 0.5704072117805481,
+      "eval_runtime": 5.1672,
+      "eval_samples_per_second": 32.126,
+      "eval_steps_per_second": 0.581,
+      "step": 191
+    },
+    {
+      "epoch": 7.757575757575758,
+      "grad_norm": 1.2050676345825195,
+      "learning_rate": 4.444444444444444e-06,
+      "loss": 0.5755,
+      "step": 192
+    },
+    {
+      "epoch": 7.757575757575758,
+      "eval_loss": 0.5683225989341736,
+      "eval_runtime": 5.1672,
+      "eval_samples_per_second": 32.126,
+      "eval_steps_per_second": 0.581,
+      "step": 192
+    },
+    {
+      "epoch": 7.797979797979798,
+      "grad_norm": 1.20169997215271,
+      "learning_rate": 4.351851851851852e-06,
+      "loss": 0.5456,
+      "step": 193
+    },
+    {
+      "epoch": 7.797979797979798,
+      "eval_loss": 0.5680342316627502,
+      "eval_runtime": 5.1647,
+      "eval_samples_per_second": 32.141,
+      "eval_steps_per_second": 0.581,
+      "step": 193
+    },
+    {
+      "epoch": 7.838383838383838,
+      "grad_norm": 1.2138899564743042,
+      "learning_rate": 4.2592592592592596e-06,
+      "loss": 0.5416,
+      "step": 194
+    },
+    {
+      "epoch": 7.838383838383838,
+      "eval_loss": 0.5660241842269897,
+      "eval_runtime": 5.1658,
+      "eval_samples_per_second": 32.134,
+      "eval_steps_per_second": 0.581,
+      "step": 194
+    },
+    {
+      "epoch": 7.878787878787879,
+      "grad_norm": 1.242883563041687,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 0.5491,
+      "step": 195
+    },
+    {
+      "epoch": 7.878787878787879,
+      "eval_loss": 0.5644729733467102,
+      "eval_runtime": 5.1604,
+      "eval_samples_per_second": 32.168,
+      "eval_steps_per_second": 0.581,
+      "step": 195
+    },
+    {
+      "epoch": 7.91919191919192,
+      "grad_norm": 1.2428083419799805,
+      "learning_rate": 4.074074074074074e-06,
+      "loss": 0.5876,
+      "step": 196
+    },
+    {
+      "epoch": 7.91919191919192,
+      "eval_loss": 0.5630542039871216,
+      "eval_runtime": 5.1536,
+      "eval_samples_per_second": 32.211,
+      "eval_steps_per_second": 0.582,
+      "step": 196
+    },
+    {
+      "epoch": 7.959595959595959,
+      "grad_norm": 1.246326208114624,
+      "learning_rate": 3.9814814814814814e-06,
+      "loss": 0.5774,
+      "step": 197
+    },
+    {
+      "epoch": 7.959595959595959,
+      "eval_loss": 0.5617477297782898,
+      "eval_runtime": 5.1536,
+      "eval_samples_per_second": 32.211,
+      "eval_steps_per_second": 0.582,
+      "step": 197
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 1.2619061470031738,
+      "learning_rate": 3.88888888888889e-06,
+      "loss": 0.5141,
+      "step": 198
+    },
+    {
+      "epoch": 8.0,
+      "eval_loss": 0.5603601932525635,
+      "eval_runtime": 5.1553,
+      "eval_samples_per_second": 32.2,
+      "eval_steps_per_second": 0.582,
+      "step": 198
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.10342705586176e+16,
+  "train_batch_size": 32,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-198/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7dafca05b7a6d25266ace8b0db88af88ff18a0ac29cc5d191b665dd79638206
+size 5368

checkpoint-24/README.md

@@ -0,0 +1,202 @@
+---
+base_model: microsoft/Phi-3-mini-4k-instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.11.1

checkpoint-24/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "microsoft/Phi-3-mini-4k-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-24/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb916389790f38282d53a09febd64eaa64a2eb35e622dbc10d33f1c7066af7ab
+size 35668592