Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,202 @@
+---
+base_model: togethercomputer/Meta-Llama-3.3-70B-Instruct-Reference
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "togethercomputer/Meta-Llama-3.3-70B-Instruct-Reference",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": [],
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.10000000149011612,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a5cef83d1edf849a139fcbd3435700976fbfcc97e40d39fa770a4be8c2570ba
+size 78643200

chat_template.jinja

@@ -0,0 +1,109 @@
+{{- bos_token }}
+{%- if custom_tools is defined %}
+    {%- set tools = custom_tools %}
+{%- endif %}
+{%- if not tools_in_user_message is defined %}
+    {%- set tools_in_user_message = true %}
+{%- endif %}
+{%- if not date_string is defined %}
+    {%- set date_string = "26 Jul 2024" %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+
+{#- System message + builtin tools #}
+{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
+{%- if builtin_tools is defined or tools is not none %}
+    {{- "Environment: ipython\n" }}
+{%- endif %}
+{%- if builtin_tools is defined %}
+    {{- "Tools: " + builtin_tools | reject('equalto', 'code_interpreter') | join(", ") + "\n\n"}}
+{%- endif %}
+{{- "Cutting Knowledge Date: December 2023\n" }}
+{{- "Today Date: " + date_string + "\n\n" }}
+{%- if tools is not none and not tools_in_user_message %}
+    {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+{%- endif %}
+{{- system_message }}
+{{- "<|eot_id|>" }}
+
+{#- Custom tools are passed in a user message with some extra guidance #}
+{%- if tools_in_user_message and not tools is none %}
+    {#- Extract the first user message so we can plug it in here #}
+    {%- if messages | length != 0 %}
+        {%- set first_user_message = messages[0]['content']|trim %}
+        {%- set messages = messages[1:] %}
+    {%- else %}
+        {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
+{%- endif %}
+    {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
+    {{- "Given the following functions, please respond with a JSON for a function call " }}
+    {{- "with its proper arguments that best answers the given prompt.\n\n" }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+    {{- first_user_message + "<|eot_id|>"}}
+{%- endif %}
+
+{%- for message in messages %}
+    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
+    {%- elif 'tool_calls' in message %}
+        {%- if not message.tool_calls|length == 1 %}
+            {{- raise_exception("This model only supports single tool-calls at once!") }}
+        {%- endif %}
+        {%- set tool_call = message.tool_calls[0].function %}
+        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}
+            {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+            {{- "<|python_tag|>" + tool_call.name + ".call(" }}
+            {%- for arg_name, arg_val in tool_call.arguments | items %}
+                {{- arg_name + '="' + arg_val + '"' }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- endif %}
+                {%- endfor %}
+            {{- ")" }}
+        {%- else  %}
+            {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+            {{- '{"name": "' + tool_call.name + '", ' }}
+            {{- '"parameters": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- "}" }}
+        {%- endif %}
+        {%- if builtin_tools is defined %}
+            {#- This means we're in ipython mode #}
+            {{- "<|eom_id|>" }}
+        {%- else %}
+            {{- "<|eot_id|>" }}
+        {%- endif %}
+    {%- elif message.role == "tool" or message.role == "ipython" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
+        {%- if message.content is mapping or message.content is iterable %}
+            {{- message.content | tojson }}
+        {%- else %}
+            {{- message.content }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,37 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": 128009,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "pad_token_id": 128009,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_theta": 500000.0,
+    "rope_type": "llama3"
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.10.1",
+  "use_cache": false,
+  "vocab_size": 128256,
+  "torch_dtype": "bfloat16"
+}

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|begin_of_text|>",
+  "eos_token": "<|eot_id|>",
+  "pad_token": "<|eot_id|>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|begin_of_text|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "is_local": false,
+  "local_files_only": true,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "tokenizer_class": "TokenizersBackend"
+}

trainer_state.json

@@ -0,0 +1,1362 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 36,
+  "global_step": 184,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010869565217391304,
+      "grad_norm": 0.18588238954544067,
+      "learning_rate": 0.0,
+      "loss": 1.4736328125,
+      "step": 1
+    },
+    {
+      "epoch": 0.021739130434782608,
+      "grad_norm": 0.1947048008441925,
+      "learning_rate": 1e-05,
+      "loss": 1.5341796875,
+      "step": 2
+    },
+    {
+      "epoch": 0.03260869565217391,
+      "grad_norm": 0.18918348848819733,
+      "learning_rate": 2e-05,
+      "loss": 1.4052734375,
+      "step": 3
+    },
+    {
+      "epoch": 0.043478260869565216,
+      "grad_norm": 0.18988992273807526,
+      "learning_rate": 3e-05,
+      "loss": 1.30517578125,
+      "step": 4
+    },
+    {
+      "epoch": 0.05434782608695652,
+      "grad_norm": 0.1465907096862793,
+      "learning_rate": 4e-05,
+      "loss": 1.349609375,
+      "step": 5
+    },
+    {
+      "epoch": 0.06521739130434782,
+      "grad_norm": 0.12894085049629211,
+      "learning_rate": 5e-05,
+      "loss": 1.33447265625,
+      "step": 6
+    },
+    {
+      "epoch": 0.07608695652173914,
+      "grad_norm": 0.10944489389657974,
+      "learning_rate": 6e-05,
+      "loss": 1.25830078125,
+      "step": 7
+    },
+    {
+      "epoch": 0.08695652173913043,
+      "grad_norm": 0.11167462170124054,
+      "learning_rate": 7e-05,
+      "loss": 1.306640625,
+      "step": 8
+    },
+    {
+      "epoch": 0.09782608695652174,
+      "grad_norm": 0.0900021567940712,
+      "learning_rate": 8e-05,
+      "loss": 1.23828125,
+      "step": 9
+    },
+    {
+      "epoch": 0.10869565217391304,
+      "grad_norm": 0.13176031410694122,
+      "learning_rate": 9e-05,
+      "loss": 1.3193359375,
+      "step": 10
+    },
+    {
+      "epoch": 0.11956521739130435,
+      "grad_norm": 0.1317533254623413,
+      "learning_rate": 0.0001,
+      "loss": 1.14404296875,
+      "step": 11
+    },
+    {
+      "epoch": 0.13043478260869565,
+      "grad_norm": 0.13434647023677826,
+      "learning_rate": 9.999266547540887e-05,
+      "loss": 1.197265625,
+      "step": 12
+    },
+    {
+      "epoch": 0.14130434782608695,
+      "grad_norm": 0.0930171012878418,
+      "learning_rate": 9.997066429253547e-05,
+      "loss": 1.22216796875,
+      "step": 13
+    },
+    {
+      "epoch": 0.15217391304347827,
+      "grad_norm": 0.05628405511379242,
+      "learning_rate": 9.993400362330058e-05,
+      "loss": 1.05419921875,
+      "step": 14
+    },
+    {
+      "epoch": 0.16304347826086957,
+      "grad_norm": 0.04639851301908493,
+      "learning_rate": 9.988269541830775e-05,
+      "loss": 1.1318359375,
+      "step": 15
+    },
+    {
+      "epoch": 0.17391304347826086,
+      "grad_norm": 0.053449347615242004,
+      "learning_rate": 9.98167564029477e-05,
+      "loss": 1.04931640625,
+      "step": 16
+    },
+    {
+      "epoch": 0.18478260869565216,
+      "grad_norm": 0.0613674633204937,
+      "learning_rate": 9.97362080719462e-05,
+      "loss": 1.17919921875,
+      "step": 17
+    },
+    {
+      "epoch": 0.1956521739130435,
+      "grad_norm": 0.0489179864525795,
+      "learning_rate": 9.964107668235719e-05,
+      "loss": 0.9716796875,
+      "step": 18
+    },
+    {
+      "epoch": 0.20652173913043478,
+      "grad_norm": 0.0503905713558197,
+      "learning_rate": 9.953139324500369e-05,
+      "loss": 1.1591796875,
+      "step": 19
+    },
+    {
+      "epoch": 0.21739130434782608,
+      "grad_norm": 0.05160759389400482,
+      "learning_rate": 9.940719351436869e-05,
+      "loss": 0.9921875,
+      "step": 20
+    },
+    {
+      "epoch": 0.22826086956521738,
+      "grad_norm": 0.05076571926474571,
+      "learning_rate": 9.926851797694012e-05,
+      "loss": 1.14013671875,
+      "step": 21
+    },
+    {
+      "epoch": 0.2391304347826087,
+      "grad_norm": 0.050418201833963394,
+      "learning_rate": 9.911541183801312e-05,
+      "loss": 1.10205078125,
+      "step": 22
+    },
+    {
+      "epoch": 0.25,
+      "grad_norm": 0.044996920973062515,
+      "learning_rate": 9.89479250069539e-05,
+      "loss": 1.06689453125,
+      "step": 23
+    },
+    {
+      "epoch": 0.2608695652173913,
+      "grad_norm": 0.0451265349984169,
+      "learning_rate": 9.876611208093055e-05,
+      "loss": 1.12060546875,
+      "step": 24
+    },
+    {
+      "epoch": 0.2717391304347826,
+      "grad_norm": 0.04035104438662529,
+      "learning_rate": 9.857003232711536e-05,
+      "loss": 1.0703125,
+      "step": 25
+    },
+    {
+      "epoch": 0.2826086956521739,
+      "grad_norm": 0.03495940566062927,
+      "learning_rate": 9.835974966336503e-05,
+      "loss": 0.88525390625,
+      "step": 26
+    },
+    {
+      "epoch": 0.29347826086956524,
+      "grad_norm": 0.05166083201766014,
+      "learning_rate": 9.813533263738486e-05,
+      "loss": 1.189453125,
+      "step": 27
+    },
+    {
+      "epoch": 0.30434782608695654,
+      "grad_norm": 0.038823310285806656,
+      "learning_rate": 9.789685440438353e-05,
+      "loss": 0.98779296875,
+      "step": 28
+    },
+    {
+      "epoch": 0.31521739130434784,
+      "grad_norm": 0.03691998869180679,
+      "learning_rate": 9.764439270322611e-05,
+      "loss": 1.1640625,
+      "step": 29
+    },
+    {
+      "epoch": 0.32608695652173914,
+      "grad_norm": 0.03796624019742012,
+      "learning_rate": 9.737802983109287e-05,
+      "loss": 1.02978515625,
+      "step": 30
+    },
+    {
+      "epoch": 0.33695652173913043,
+      "grad_norm": 0.04263104870915413,
+      "learning_rate": 9.709785261665205e-05,
+      "loss": 0.98974609375,
+      "step": 31
+    },
+    {
+      "epoch": 0.34782608695652173,
+      "grad_norm": 0.044888075441122055,
+      "learning_rate": 9.680395239175563e-05,
+      "loss": 1.07470703125,
+      "step": 32
+    },
+    {
+      "epoch": 0.358695652173913,
+      "grad_norm": 0.04229896143078804,
+      "learning_rate": 9.6496424961667e-05,
+      "loss": 1.1181640625,
+      "step": 33
+    },
+    {
+      "epoch": 0.3695652173913043,
+      "grad_norm": 0.04039580374956131,
+      "learning_rate": 9.617537057383055e-05,
+      "loss": 1.10107421875,
+      "step": 34
+    },
+    {
+      "epoch": 0.3804347826086957,
+      "grad_norm": 0.03618437424302101,
+      "learning_rate": 9.584089388519307e-05,
+      "loss": 1.01708984375,
+      "step": 35
+    },
+    {
+      "epoch": 0.391304347826087,
+      "grad_norm": 0.03835980221629143,
+      "learning_rate": 9.549310392808782e-05,
+      "loss": 1.1201171875,
+      "step": 36
+    },
+    {
+      "epoch": 0.40217391304347827,
+      "grad_norm": 0.03548717126250267,
+      "learning_rate": 9.513211407469221e-05,
+      "loss": 0.94830322265625,
+      "step": 37
+    },
+    {
+      "epoch": 0.41304347826086957,
+      "grad_norm": 0.03912119194865227,
+      "learning_rate": 9.475804200007104e-05,
+      "loss": 1.11767578125,
+      "step": 38
+    },
+    {
+      "epoch": 0.42391304347826086,
+      "grad_norm": 0.04173147305846214,
+      "learning_rate": 9.437100964381675e-05,
+      "loss": 1.0810546875,
+      "step": 39
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "grad_norm": 0.04037207365036011,
+      "learning_rate": 9.397114317029975e-05,
+      "loss": 1.0634765625,
+      "step": 40
+    },
+    {
+      "epoch": 0.43478260869565216,
+      "eval_loss": 1.04931640625,
+      "eval_runtime": 6.3585,
+      "eval_samples_per_second": 1.258,
+      "eval_steps_per_second": 0.157,
+      "step": 40
+    },
+    {
+      "epoch": 0.44565217391304346,
+      "grad_norm": 0.04123736172914505,
+      "learning_rate": 9.355857292754152e-05,
+      "loss": 1.04931640625,
+      "step": 41
+    },
+    {
+      "epoch": 0.45652173913043476,
+      "grad_norm": 0.039305102080106735,
+      "learning_rate": 9.313343340472388e-05,
+      "loss": 1.0908203125,
+      "step": 42
+    },
+    {
+      "epoch": 0.4673913043478261,
+      "grad_norm": 0.03992549329996109,
+      "learning_rate": 9.269586318834841e-05,
+      "loss": 1.05810546875,
+      "step": 43
+    },
+    {
+      "epoch": 0.4782608695652174,
+      "grad_norm": 0.03971557691693306,
+      "learning_rate": 9.224600491706008e-05,
+      "loss": 1.154296875,
+      "step": 44
+    },
+    {
+      "epoch": 0.4891304347826087,
+      "grad_norm": 0.03945128247141838,
+      "learning_rate": 9.178400523515012e-05,
+      "loss": 1.03857421875,
+      "step": 45
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 0.036954931914806366,
+      "learning_rate": 9.131001474475317e-05,
+      "loss": 0.91845703125,
+      "step": 46
+    },
+    {
+      "epoch": 0.5108695652173914,
+      "grad_norm": 0.041776467114686966,
+      "learning_rate": 9.082418795675397e-05,
+      "loss": 1.01220703125,
+      "step": 47
+    },
+    {
+      "epoch": 0.5217391304347826,
+      "grad_norm": 0.038964807987213135,
+      "learning_rate": 9.032668324042026e-05,
+      "loss": 0.8963623046875,
+      "step": 48
+    },
+    {
+      "epoch": 0.532608695652174,
+      "grad_norm": 0.04493962973356247,
+      "learning_rate": 8.981766277177764e-05,
+      "loss": 0.8782958984375,
+      "step": 49
+    },
+    {
+      "epoch": 0.5434782608695652,
+      "grad_norm": 0.04111941531300545,
+      "learning_rate": 8.929729248074364e-05,
+      "loss": 1.05712890625,
+      "step": 50
+    },
+    {
+      "epoch": 0.5543478260869565,
+      "grad_norm": 0.045288294553756714,
+      "learning_rate": 8.87657419970381e-05,
+      "loss": 1.0810546875,
+      "step": 51
+    },
+    {
+      "epoch": 0.5652173913043478,
+      "grad_norm": 0.03733203932642937,
+      "learning_rate": 8.822318459488744e-05,
+      "loss": 0.77294921875,
+      "step": 52
+    },
+    {
+      "epoch": 0.5760869565217391,
+      "grad_norm": 0.04698137566447258,
+      "learning_rate": 8.766979713654089e-05,
+      "loss": 1.0654296875,
+      "step": 53
+    },
+    {
+      "epoch": 0.5869565217391305,
+      "grad_norm": 0.05637023597955704,
+      "learning_rate": 8.71057600146172e-05,
+      "loss": 1.06201171875,
+      "step": 54
+    },
+    {
+      "epoch": 0.5978260869565217,
+      "grad_norm": 0.04036635905504227,
+      "learning_rate": 8.653125709330041e-05,
+      "loss": 0.9677734375,
+      "step": 55
+    },
+    {
+      "epoch": 0.6086956521739131,
+      "grad_norm": 0.04818817973136902,
+      "learning_rate": 8.594647564840405e-05,
+      "loss": 1.0546875,
+      "step": 56
+    },
+    {
+      "epoch": 0.6195652173913043,
+      "grad_norm": 0.04563218355178833,
+      "learning_rate": 8.535160630632313e-05,
+      "loss": 1.015625,
+      "step": 57
+    },
+    {
+      "epoch": 0.6304347826086957,
+      "grad_norm": 0.04481424391269684,
+      "learning_rate": 8.474684298189402e-05,
+      "loss": 1.00439453125,
+      "step": 58
+    },
+    {
+      "epoch": 0.6413043478260869,
+      "grad_norm": 0.0573306679725647,
+      "learning_rate": 8.413238281518225e-05,
+      "loss": 0.99609375,
+      "step": 59
+    },
+    {
+      "epoch": 0.6521739130434783,
+      "grad_norm": 0.04850015044212341,
+      "learning_rate": 8.350842610721909e-05,
+      "loss": 1.01025390625,
+      "step": 60
+    },
+    {
+      "epoch": 0.6630434782608695,
+      "grad_norm": 0.044663649052381516,
+      "learning_rate": 8.287517625470755e-05,
+      "loss": 1.06005859375,
+      "step": 61
+    },
+    {
+      "epoch": 0.6739130434782609,
+      "grad_norm": 0.04311933368444443,
+      "learning_rate": 8.223283968371944e-05,
+      "loss": 0.916748046875,
+      "step": 62
+    },
+    {
+      "epoch": 0.6847826086956522,
+      "grad_norm": 0.04746570438146591,
+      "learning_rate": 8.15816257824048e-05,
+      "loss": 0.99072265625,
+      "step": 63
+    },
+    {
+      "epoch": 0.6956521739130435,
+      "grad_norm": 0.04210105538368225,
+      "learning_rate": 8.092174683273578e-05,
+      "loss": 0.91845703125,
+      "step": 64
+    },
+    {
+      "epoch": 0.7065217391304348,
+      "grad_norm": 0.04694285988807678,
+      "learning_rate": 8.025341794130722e-05,
+      "loss": 0.9931640625,
+      "step": 65
+    },
+    {
+      "epoch": 0.717391304347826,
+      "grad_norm": 0.06701935827732086,
+      "learning_rate": 7.957685696921636e-05,
+      "loss": 1.00048828125,
+      "step": 66
+    },
+    {
+      "epoch": 0.7282608695652174,
+      "grad_norm": 0.06168576329946518,
+      "learning_rate": 7.88922844610449e-05,
+      "loss": 1.046875,
+      "step": 67
+    },
+    {
+      "epoch": 0.7391304347826086,
+      "grad_norm": 0.046262163668870926,
+      "learning_rate": 7.819992357296598e-05,
+      "loss": 1.01611328125,
+      "step": 68
+    },
+    {
+      "epoch": 0.75,
+      "grad_norm": 0.04800650104880333,
+      "learning_rate": 7.75e-05,
+      "loss": 0.87109375,
+      "step": 69
+    },
+    {
+      "epoch": 0.7608695652173914,
+      "grad_norm": 0.04885740950703621,
+      "learning_rate": 7.679274190244287e-05,
+      "loss": 1.0087890625,
+      "step": 70
+    },
+    {
+      "epoch": 0.7717391304347826,
+      "grad_norm": 0.04720492660999298,
+      "learning_rate": 7.607837983149056e-05,
+      "loss": 1.02978515625,
+      "step": 71
+    },
+    {
+      "epoch": 0.782608695652174,
+      "grad_norm": 0.04430517181754112,
+      "learning_rate": 7.535714665408423e-05,
+      "loss": 0.848388671875,
+      "step": 72
+    },
+    {
+      "epoch": 0.7934782608695652,
+      "grad_norm": 0.04949856922030449,
+      "learning_rate": 7.462927747700053e-05,
+      "loss": 0.888671875,
+      "step": 73
+    },
+    {
+      "epoch": 0.8043478260869565,
+      "grad_norm": 0.053681328892707825,
+      "learning_rate": 7.389500957021192e-05,
+      "loss": 1.0283203125,
+      "step": 74
+    },
+    {
+      "epoch": 0.8152173913043478,
+      "grad_norm": 0.04809682071208954,
+      "learning_rate": 7.31545822895414e-05,
+      "loss": 1.02001953125,
+      "step": 75
+    },
+    {
+      "epoch": 0.8260869565217391,
+      "grad_norm": 0.052168428897857666,
+      "learning_rate": 7.240823699863777e-05,
+      "loss": 1.02197265625,
+      "step": 76
+    },
+    {
+      "epoch": 0.8260869565217391,
+      "eval_loss": 0.99267578125,
+      "eval_runtime": 6.384,
+      "eval_samples_per_second": 1.253,
+      "eval_steps_per_second": 0.157,
+      "step": 76
+    },
+    {
+      "epoch": 0.8369565217391305,
+      "grad_norm": 0.05126576125621796,
+      "learning_rate": 7.165621699029615e-05,
+      "loss": 0.95361328125,
+      "step": 77
+    },
+    {
+      "epoch": 0.8478260869565217,
+      "grad_norm": 0.0479954332113266,
+      "learning_rate": 7.089876740714961e-05,
+      "loss": 0.93212890625,
+      "step": 78
+    },
+    {
+      "epoch": 0.8586956521739131,
+      "grad_norm": 0.05192503705620766,
+      "learning_rate": 7.013613516175788e-05,
+      "loss": 0.9462890625,
+      "step": 79
+    },
+    {
+      "epoch": 0.8695652173913043,
+      "grad_norm": 0.05096974968910217,
+      "learning_rate": 6.93685688561191e-05,
+      "loss": 0.95166015625,
+      "step": 80
+    },
+    {
+      "epoch": 0.8804347826086957,
+      "grad_norm": 0.0476989820599556,
+      "learning_rate": 6.859631870063076e-05,
+      "loss": 0.916015625,
+      "step": 81
+    },
+    {
+      "epoch": 0.8913043478260869,
+      "grad_norm": 0.12171965837478638,
+      "learning_rate": 6.781963643252651e-05,
+      "loss": 1.0029296875,
+      "step": 82
+    },
+    {
+      "epoch": 0.9021739130434783,
+      "grad_norm": 0.050482314079999924,
+      "learning_rate": 6.703877523381496e-05,
+      "loss": 1.001953125,
+      "step": 83
+    },
+    {
+      "epoch": 0.9130434782608695,
+      "grad_norm": 0.04611290618777275,
+      "learning_rate": 6.625398964874784e-05,
+      "loss": 0.8406982421875,
+      "step": 84
+    },
+    {
+      "epoch": 0.9239130434782609,
+      "grad_norm": 0.04967767000198364,
+      "learning_rate": 6.546553550084398e-05,
+      "loss": 0.8896484375,
+      "step": 85
+    },
+    {
+      "epoch": 0.9347826086956522,
+      "grad_norm": 0.055717870593070984,
+      "learning_rate": 6.46736698094961e-05,
+      "loss": 1.00537109375,
+      "step": 86
+    },
+    {
+      "epoch": 0.9456521739130435,
+      "grad_norm": 0.052366528660058975,
+      "learning_rate": 6.3878650706188e-05,
+      "loss": 1.01904296875,
+      "step": 87
+    },
+    {
+      "epoch": 0.9565217391304348,
+      "grad_norm": 0.07001721113920212,
+      "learning_rate": 6.308073735034923e-05,
+      "loss": 1.02001953125,
+      "step": 88
+    },
+    {
+      "epoch": 0.967391304347826,
+      "grad_norm": 0.04996788129210472,
+      "learning_rate": 6.228018984487442e-05,
+      "loss": 0.903076171875,
+      "step": 89
+    },
+    {
+      "epoch": 0.9782608695652174,
+      "grad_norm": 0.05138871446251869,
+      "learning_rate": 6.147726915133536e-05,
+      "loss": 0.96533203125,
+      "step": 90
+    },
+    {
+      "epoch": 0.9891304347826086,
+      "grad_norm": 0.05665658041834831,
+      "learning_rate": 6.067223700491302e-05,
+      "loss": 1.015625,
+      "step": 91
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.05422813072800636,
+      "learning_rate": 5.986535582907739e-05,
+      "loss": 1.00537109375,
+      "step": 92
+    },
+    {
+      "epoch": 1.0108695652173914,
+      "grad_norm": 0.05152361840009689,
+      "learning_rate": 5.9056888650042954e-05,
+      "loss": 1.0166015625,
+      "step": 93
+    },
+    {
+      "epoch": 1.0217391304347827,
+      "grad_norm": 0.060649726539850235,
+      "learning_rate": 5.8247099011027736e-05,
+      "loss": 1.056640625,
+      "step": 94
+    },
+    {
+      "epoch": 1.0326086956521738,
+      "grad_norm": 0.046568043529987335,
+      "learning_rate": 5.74362508863438e-05,
+      "loss": 0.9560546875,
+      "step": 95
+    },
+    {
+      "epoch": 1.0434782608695652,
+      "grad_norm": 0.05016551539301872,
+      "learning_rate": 5.662460859534714e-05,
+      "loss": 0.882568359375,
+      "step": 96
+    },
+    {
+      "epoch": 1.0543478260869565,
+      "grad_norm": 0.05591215193271637,
+      "learning_rate": 5.581243671627522e-05,
+      "loss": 0.97021484375,
+      "step": 97
+    },
+    {
+      "epoch": 1.065217391304348,
+      "grad_norm": 0.05546316131949425,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 0.958984375,
+      "step": 98
+    },
+    {
+      "epoch": 1.0760869565217392,
+      "grad_norm": 0.060860466212034225,
+      "learning_rate": 5.418756328372477e-05,
+      "loss": 0.93408203125,
+      "step": 99
+    },
+    {
+      "epoch": 1.0869565217391304,
+      "grad_norm": 0.06850528717041016,
+      "learning_rate": 5.337539140465287e-05,
+      "loss": 0.9853515625,
+      "step": 100
+    },
+    {
+      "epoch": 1.0978260869565217,
+      "grad_norm": 0.05219709500670433,
+      "learning_rate": 5.2563749113656216e-05,
+      "loss": 0.966796875,
+      "step": 101
+    },
+    {
+      "epoch": 1.108695652173913,
+      "grad_norm": 0.059890855103731155,
+      "learning_rate": 5.175290098897229e-05,
+      "loss": 1.03076171875,
+      "step": 102
+    },
+    {
+      "epoch": 1.1195652173913044,
+      "grad_norm": 0.061198536306619644,
+      "learning_rate": 5.094311134995708e-05,
+      "loss": 0.8857421875,
+      "step": 103
+    },
+    {
+      "epoch": 1.1304347826086956,
+      "grad_norm": 0.05207240954041481,
+      "learning_rate": 5.013464417092263e-05,
+      "loss": 0.9482421875,
+      "step": 104
+    },
+    {
+      "epoch": 1.141304347826087,
+      "grad_norm": 0.06015055254101753,
+      "learning_rate": 4.932776299508699e-05,
+      "loss": 0.98779296875,
+      "step": 105
+    },
+    {
+      "epoch": 1.1521739130434783,
+      "grad_norm": 0.06881439685821533,
+      "learning_rate": 4.852273084866464e-05,
+      "loss": 0.86865234375,
+      "step": 106
+    },
+    {
+      "epoch": 1.1630434782608696,
+      "grad_norm": 0.05143669992685318,
+      "learning_rate": 4.771981015512559e-05,
+      "loss": 0.93212890625,
+      "step": 107
+    },
+    {
+      "epoch": 1.1739130434782608,
+      "grad_norm": 0.05013814941048622,
+      "learning_rate": 4.6919262649650776e-05,
+      "loss": 0.87841796875,
+      "step": 108
+    },
+    {
+      "epoch": 1.184782608695652,
+      "grad_norm": 0.06504126638174057,
+      "learning_rate": 4.612134929381201e-05,
+      "loss": 0.9814453125,
+      "step": 109
+    },
+    {
+      "epoch": 1.1956521739130435,
+      "grad_norm": 0.04764424264431,
+      "learning_rate": 4.532633019050392e-05,
+      "loss": 0.811279296875,
+      "step": 110
+    },
+    {
+      "epoch": 1.2065217391304348,
+      "grad_norm": 0.05300690978765488,
+      "learning_rate": 4.453446449915605e-05,
+      "loss": 0.97021484375,
+      "step": 111
+    },
+    {
+      "epoch": 1.2173913043478262,
+      "grad_norm": 0.056520976126194,
+      "learning_rate": 4.3746010351252174e-05,
+      "loss": 0.835693359375,
+      "step": 112
+    },
+    {
+      "epoch": 1.2173913043478262,
+      "eval_loss": 0.9716796875,
+      "eval_runtime": 6.3847,
+      "eval_samples_per_second": 1.253,
+      "eval_steps_per_second": 0.157,
+      "step": 112
+    },
+    {
+      "epoch": 1.2282608695652173,
+      "grad_norm": 0.05752632021903992,
+      "learning_rate": 4.296122476618507e-05,
+      "loss": 0.9580078125,
+      "step": 113
+    },
+    {
+      "epoch": 1.2391304347826086,
+      "grad_norm": 0.052798960357904434,
+      "learning_rate": 4.21803635674735e-05,
+      "loss": 0.9326171875,
+      "step": 114
+    },
+    {
+      "epoch": 1.25,
+      "grad_norm": 0.0545145682990551,
+      "learning_rate": 4.1403681299369226e-05,
+      "loss": 0.90185546875,
+      "step": 115
+    },
+    {
+      "epoch": 1.2608695652173914,
+      "grad_norm": 0.12075046449899673,
+      "learning_rate": 4.0631431143880916e-05,
+      "loss": 0.9541015625,
+      "step": 116
+    },
+    {
+      "epoch": 1.2717391304347827,
+      "grad_norm": 0.05467211827635765,
+      "learning_rate": 3.986386483824212e-05,
+      "loss": 0.92041015625,
+      "step": 117
+    },
+    {
+      "epoch": 1.2826086956521738,
+      "grad_norm": 0.058259688317775726,
+      "learning_rate": 3.91012325928504e-05,
+      "loss": 0.755859375,
+      "step": 118
+    },
+    {
+      "epoch": 1.2934782608695652,
+      "grad_norm": 0.057315390557050705,
+      "learning_rate": 3.834378300970385e-05,
+      "loss": 1.02392578125,
+      "step": 119
+    },
+    {
+      "epoch": 1.3043478260869565,
+      "grad_norm": 0.052486542612314224,
+      "learning_rate": 3.7591763001362246e-05,
+      "loss": 0.84228515625,
+      "step": 120
+    },
+    {
+      "epoch": 1.315217391304348,
+      "grad_norm": 0.052620261907577515,
+      "learning_rate": 3.684541771045862e-05,
+      "loss": 1.00146484375,
+      "step": 121
+    },
+    {
+      "epoch": 1.3260869565217392,
+      "grad_norm": 0.05407518520951271,
+      "learning_rate": 3.61049904297881e-05,
+      "loss": 0.8876953125,
+      "step": 122
+    },
+    {
+      "epoch": 1.3369565217391304,
+      "grad_norm": 0.061481185257434845,
+      "learning_rate": 3.537072252299947e-05,
+      "loss": 0.85791015625,
+      "step": 123
+    },
+    {
+      "epoch": 1.3478260869565217,
+      "grad_norm": 0.053316932171583176,
+      "learning_rate": 3.464285334591579e-05,
+      "loss": 0.9296875,
+      "step": 124
+    },
+    {
+      "epoch": 1.358695652173913,
+      "grad_norm": 0.053633544594049454,
+      "learning_rate": 3.392162016850945e-05,
+      "loss": 0.96484375,
+      "step": 125
+    },
+    {
+      "epoch": 1.3695652173913042,
+      "grad_norm": 0.054563552141189575,
+      "learning_rate": 3.3207258097557136e-05,
+      "loss": 0.95703125,
+      "step": 126
+    },
+    {
+      "epoch": 1.3804347826086958,
+      "grad_norm": 0.055144086480140686,
+      "learning_rate": 3.250000000000001e-05,
+      "loss": 0.886474609375,
+      "step": 127
+    },
+    {
+      "epoch": 1.391304347826087,
+      "grad_norm": 0.05462967976927757,
+      "learning_rate": 3.180007642703402e-05,
+      "loss": 0.986328125,
+      "step": 128
+    },
+    {
+      "epoch": 1.4021739130434783,
+      "grad_norm": 0.051473114639520645,
+      "learning_rate": 3.1107715538955106e-05,
+      "loss": 0.82843017578125,
+      "step": 129
+    },
+    {
+      "epoch": 1.4130434782608696,
+      "grad_norm": 0.06261783838272095,
+      "learning_rate": 3.042314303078364e-05,
+      "loss": 0.97509765625,
+      "step": 130
+    },
+    {
+      "epoch": 1.4239130434782608,
+      "grad_norm": 0.05514520779252052,
+      "learning_rate": 2.97465820586928e-05,
+      "loss": 0.9443359375,
+      "step": 131
+    },
+    {
+      "epoch": 1.434782608695652,
+      "grad_norm": 0.05541987344622612,
+      "learning_rate": 2.9078253167264223e-05,
+      "loss": 0.92919921875,
+      "step": 132
+    },
+    {
+      "epoch": 1.4456521739130435,
+      "grad_norm": 0.05705087631940842,
+      "learning_rate": 2.8418374217595213e-05,
+      "loss": 0.91845703125,
+      "step": 133
+    },
+    {
+      "epoch": 1.4565217391304348,
+      "grad_norm": 0.05676661804318428,
+      "learning_rate": 2.7767160316280583e-05,
+      "loss": 0.95703125,
+      "step": 134
+    },
+    {
+      "epoch": 1.4673913043478262,
+      "grad_norm": 0.056343358010053635,
+      "learning_rate": 2.712482374529247e-05,
+      "loss": 0.92138671875,
+      "step": 135
+    },
+    {
+      "epoch": 1.4782608695652173,
+      "grad_norm": 0.05837247893214226,
+      "learning_rate": 2.649157389278094e-05,
+      "loss": 1.01513671875,
+      "step": 136
+    },
+    {
+      "epoch": 1.4891304347826086,
+      "grad_norm": 0.05657912418246269,
+      "learning_rate": 2.586761718481776e-05,
+      "loss": 0.9130859375,
+      "step": 137
+    },
+    {
+      "epoch": 1.5,
+      "grad_norm": 0.10456497222185135,
+      "learning_rate": 2.5253157018105993e-05,
+      "loss": 0.803466796875,
+      "step": 138
+    },
+    {
+      "epoch": 1.5108695652173914,
+      "grad_norm": 0.06464536488056183,
+      "learning_rate": 2.464839369367688e-05,
+      "loss": 0.890625,
+      "step": 139
+    },
+    {
+      "epoch": 1.5217391304347827,
+      "grad_norm": 0.05322624742984772,
+      "learning_rate": 2.4053524351595948e-05,
+      "loss": 0.790924072265625,
+      "step": 140
+    },
+    {
+      "epoch": 1.5326086956521738,
+      "grad_norm": 0.05579627305269241,
+      "learning_rate": 2.3468742906699597e-05,
+      "loss": 0.77099609375,
+      "step": 141
+    },
+    {
+      "epoch": 1.5434782608695652,
+      "grad_norm": 0.06104826182126999,
+      "learning_rate": 2.28942399853828e-05,
+      "loss": 0.9365234375,
+      "step": 142
+    },
+    {
+      "epoch": 1.5543478260869565,
+      "grad_norm": 0.06145164743065834,
+      "learning_rate": 2.2330202863459122e-05,
+      "loss": 0.958984375,
+      "step": 143
+    },
+    {
+      "epoch": 1.5652173913043477,
+      "grad_norm": 0.05294163525104523,
+      "learning_rate": 2.1776815405112568e-05,
+      "loss": 0.6845703125,
+      "step": 144
+    },
+    {
+      "epoch": 1.5760869565217392,
+      "grad_norm": 0.0592268668115139,
+      "learning_rate": 2.1234258002961897e-05,
+      "loss": 0.9482421875,
+      "step": 145
+    },
+    {
+      "epoch": 1.5869565217391304,
+      "grad_norm": 0.06355844438076019,
+      "learning_rate": 2.0702707519256364e-05,
+      "loss": 0.9443359375,
+      "step": 146
+    },
+    {
+      "epoch": 1.5978260869565217,
+      "grad_norm": 0.05757714435458183,
+      "learning_rate": 2.0182337228222365e-05,
+      "loss": 0.859375,
+      "step": 147
+    },
+    {
+      "epoch": 1.608695652173913,
+      "grad_norm": 0.06119934096932411,
+      "learning_rate": 1.967331675957975e-05,
+      "loss": 0.94140625,
+      "step": 148
+    },
+    {
+      "epoch": 1.608695652173913,
+      "eval_loss": 0.9599609375,
+      "eval_runtime": 6.387,
+      "eval_samples_per_second": 1.253,
+      "eval_steps_per_second": 0.157,
+      "step": 148
+    },
+    {
+      "epoch": 1.6195652173913042,
+      "grad_norm": 1.742512583732605,
+      "learning_rate": 1.9175812043246033e-05,
+      "loss": 0.9052734375,
+      "step": 149
+    },
+    {
+      "epoch": 1.6304347826086958,
+      "grad_norm": 0.06572595238685608,
+      "learning_rate": 1.8689985255246834e-05,
+      "loss": 0.8955078125,
+      "step": 150
+    },
+    {
+      "epoch": 1.641304347826087,
+      "grad_norm": 0.06171270087361336,
+      "learning_rate": 1.821599476484987e-05,
+      "loss": 0.888671875,
+      "step": 151
+    },
+    {
+      "epoch": 1.6521739130434783,
+      "grad_norm": 0.06164025515317917,
+      "learning_rate": 1.775399508293993e-05,
+      "loss": 0.9052734375,
+      "step": 152
+    },
+    {
+      "epoch": 1.6630434782608696,
+      "grad_norm": 0.06409742683172226,
+      "learning_rate": 1.7304136811651596e-05,
+      "loss": 0.947265625,
+      "step": 153
+    },
+    {
+      "epoch": 1.6739130434782608,
+      "grad_norm": 0.056571125984191895,
+      "learning_rate": 1.6866566595276106e-05,
+      "loss": 0.82403564453125,
+      "step": 154
+    },
+    {
+      "epoch": 1.6847826086956523,
+      "grad_norm": 0.07560455054044724,
+      "learning_rate": 1.6441427072458494e-05,
+      "loss": 0.888671875,
+      "step": 155
+    },
+    {
+      "epoch": 1.6956521739130435,
+      "grad_norm": 0.0648086741566658,
+      "learning_rate": 1.602885682970026e-05,
+      "loss": 0.82421875,
+      "step": 156
+    },
+    {
+      "epoch": 1.7065217391304348,
+      "grad_norm": 0.067837655544281,
+      "learning_rate": 1.5628990356183268e-05,
+      "loss": 0.890625,
+      "step": 157
+    },
+    {
+      "epoch": 1.7173913043478262,
+      "grad_norm": 0.0631956160068512,
+      "learning_rate": 1.5241957999928974e-05,
+      "loss": 0.89599609375,
+      "step": 158
+    },
+    {
+      "epoch": 1.7282608695652173,
+      "grad_norm": 0.07191090285778046,
+      "learning_rate": 1.48678859253078e-05,
+      "loss": 0.9482421875,
+      "step": 159
+    },
+    {
+      "epoch": 1.7391304347826086,
+      "grad_norm": 0.062349699437618256,
+      "learning_rate": 1.4506896071912207e-05,
+      "loss": 0.91552734375,
+      "step": 160
+    },
+    {
+      "epoch": 1.75,
+      "grad_norm": 0.06104462221264839,
+      "learning_rate": 1.4159106114806942e-05,
+      "loss": 0.7841796875,
+      "step": 161
+    },
+    {
+      "epoch": 1.7608695652173914,
+      "grad_norm": 0.060603346675634384,
+      "learning_rate": 1.3824629426169453e-05,
+      "loss": 0.91015625,
+      "step": 162
+    },
+    {
+      "epoch": 1.7717391304347827,
+      "grad_norm": 0.06206117570400238,
+      "learning_rate": 1.350357503833301e-05,
+      "loss": 0.93115234375,
+      "step": 163
+    },
+    {
+      "epoch": 1.7826086956521738,
+      "grad_norm": 0.06806284934282303,
+      "learning_rate": 1.3196047608244389e-05,
+      "loss": 0.7666015625,
+      "step": 164
+    },
+    {
+      "epoch": 1.7934782608695652,
+      "grad_norm": 0.05813819169998169,
+      "learning_rate": 1.2902147383347959e-05,
+      "loss": 0.80029296875,
+      "step": 165
+    },
+    {
+      "epoch": 1.8043478260869565,
+      "grad_norm": 0.06218123435974121,
+      "learning_rate": 1.262197016890714e-05,
+      "loss": 0.93408203125,
+      "step": 166
+    },
+    {
+      "epoch": 1.8152173913043477,
+      "grad_norm": 0.06234293431043625,
+      "learning_rate": 1.2355607296773895e-05,
+      "loss": 0.93017578125,
+      "step": 167
+    },
+    {
+      "epoch": 1.8260869565217392,
+      "grad_norm": 0.06383553892374039,
+      "learning_rate": 1.2103145595616483e-05,
+      "loss": 0.92626953125,
+      "step": 168
+    },
+    {
+      "epoch": 1.8369565217391304,
+      "grad_norm": 0.061867013573646545,
+      "learning_rate": 1.1864667362615142e-05,
+      "loss": 0.86474609375,
+      "step": 169
+    },
+    {
+      "epoch": 1.8478260869565217,
+      "grad_norm": 0.06146484985947609,
+      "learning_rate": 1.164025033663497e-05,
+      "loss": 0.84765625,
+      "step": 170
+    },
+    {
+      "epoch": 1.858695652173913,
+      "grad_norm": 0.06298917531967163,
+      "learning_rate": 1.1429967672884653e-05,
+      "loss": 0.865234375,
+      "step": 171
+    },
+    {
+      "epoch": 1.8695652173913042,
+      "grad_norm": 0.0625348687171936,
+      "learning_rate": 1.1233887919069461e-05,
+      "loss": 0.86865234375,
+      "step": 172
+    },
+    {
+      "epoch": 1.8804347826086958,
+      "grad_norm": 0.06479020416736603,
+      "learning_rate": 1.1052074993046102e-05,
+      "loss": 0.84033203125,
+      "step": 173
+    },
+    {
+      "epoch": 1.891304347826087,
+      "grad_norm": 0.06634494662284851,
+      "learning_rate": 1.0884588161986892e-05,
+      "loss": 0.91845703125,
+      "step": 174
+    },
+    {
+      "epoch": 1.9021739130434783,
+      "grad_norm": 0.06477183103561401,
+      "learning_rate": 1.073148202305988e-05,
+      "loss": 0.916015625,
+      "step": 175
+    },
+    {
+      "epoch": 1.9130434782608696,
+      "grad_norm": 0.060632795095443726,
+      "learning_rate": 1.0592806485631325e-05,
+      "loss": 0.773193359375,
+      "step": 176
+    },
+    {
+      "epoch": 1.9239130434782608,
+      "grad_norm": 0.06851464509963989,
+      "learning_rate": 1.0468606754996326e-05,
+      "loss": 0.81787109375,
+      "step": 177
+    },
+    {
+      "epoch": 1.9347826086956523,
+      "grad_norm": 0.06455637514591217,
+      "learning_rate": 1.0358923317642806e-05,
+      "loss": 0.923828125,
+      "step": 178
+    },
+    {
+      "epoch": 1.9456521739130435,
+      "grad_norm": 0.06511721760034561,
+      "learning_rate": 1.0263791928053819e-05,
+      "loss": 0.94091796875,
+      "step": 179
+    },
+    {
+      "epoch": 1.9565217391304348,
+      "grad_norm": 0.06577809900045395,
+      "learning_rate": 1.0183243597052312e-05,
+      "loss": 0.93994140625,
+      "step": 180
+    },
+    {
+      "epoch": 1.9673913043478262,
+      "grad_norm": 0.06064470857381821,
+      "learning_rate": 1.0117304581692261e-05,
+      "loss": 0.83453369140625,
+      "step": 181
+    },
+    {
+      "epoch": 1.9782608695652173,
+      "grad_norm": 0.06249881535768509,
+      "learning_rate": 1.006599637669943e-05,
+      "loss": 0.89453125,
+      "step": 182
+    },
+    {
+      "epoch": 1.9891304347826086,
+      "grad_norm": 0.0665738582611084,
+      "learning_rate": 1.0029335707464538e-05,
+      "loss": 0.9384765625,
+      "step": 183
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.06674587726593018,
+      "learning_rate": 1.0007334524591144e-05,
+      "loss": 0.9150390625,
+      "step": 184
+    },
+    {
+      "epoch": 2.0,
+      "eval_loss": 0.95654296875,
+      "eval_runtime": 6.4018,
+      "eval_samples_per_second": 1.25,
+      "eval_steps_per_second": 0.156,
+      "step": 184
+    }
+  ],
+  "logging_steps": 1.0,
+  "max_steps": 184,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 0,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5122809836124766e+19,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}