Add files using upload-large-folder tool

README.md

@@ -0,0 +1,62 @@
+---
+library_name: peft
+license: other
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+tags:
+- llama-factory
+- lora
+- generated_from_trainer
+model-index:
+- name: test
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# test
+
+This model is a fine-tuned version of [Qwen/Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct) on the parsed_data dataset.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 1
+- eval_batch_size: 8
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 2
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 8
+- total_eval_batch_size: 16
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 30
+
+### Training results
+
+
+
+### Framework versions
+
+- PEFT 0.15.0
+- Transformers 4.50.0
+- Pytorch 2.6.0+cu124
+- Datasets 3.4.1
+- Tokenizers 0.21.0

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

all_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 29.812698412698413,
+    "total_flos": 1.011586335464489e+17,
+    "train_loss": 0.4251327198149039,
+    "train_runtime": 14594.4914,
+    "train_samples_per_second": 2.59,
+    "train_steps_per_second": 0.323
+}

checkpoint-1099/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-1099/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1099/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-1099/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1099/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1099/trainer_state.json

@@ -0,0 +1,1567 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 6.958730158730159,
+  "eval_steps": 500,
+  "global_step": 1099,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 0.5545095205307007,
+      "learning_rate": 5.307855626326963e-07,
+      "loss": 3.7162,
+      "step": 5
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 0.6163601279258728,
+      "learning_rate": 1.0615711252653927e-06,
+      "loss": 3.9388,
+      "step": 10
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.5541530847549438,
+      "learning_rate": 1.5923566878980892e-06,
+      "loss": 3.9165,
+      "step": 15
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 0.457332044839859,
+      "learning_rate": 2.1231422505307854e-06,
+      "loss": 3.7326,
+      "step": 20
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 0.5335279107093811,
+      "learning_rate": 2.653927813163482e-06,
+      "loss": 3.8251,
+      "step": 25
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.7080379724502563,
+      "learning_rate": 3.1847133757961785e-06,
+      "loss": 3.7534,
+      "step": 30
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.520993709564209,
+      "learning_rate": 3.715498938428875e-06,
+      "loss": 3.898,
+      "step": 35
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 0.5451405644416809,
+      "learning_rate": 4.246284501061571e-06,
+      "loss": 3.8951,
+      "step": 40
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.6205154657363892,
+      "learning_rate": 4.777070063694268e-06,
+      "loss": 3.7666,
+      "step": 45
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 0.7404439449310303,
+      "learning_rate": 5.307855626326964e-06,
+      "loss": 4.0258,
+      "step": 50
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 0.6272220015525818,
+      "learning_rate": 5.838641188959661e-06,
+      "loss": 3.8464,
+      "step": 55
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 0.7744691967964172,
+      "learning_rate": 6.369426751592357e-06,
+      "loss": 3.7299,
+      "step": 60
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 0.8805738687515259,
+      "learning_rate": 6.900212314225053e-06,
+      "loss": 3.5008,
+      "step": 65
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.0740723609924316,
+      "learning_rate": 7.43099787685775e-06,
+      "loss": 3.7552,
+      "step": 70
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.965708315372467,
+      "learning_rate": 7.961783439490445e-06,
+      "loss": 3.5516,
+      "step": 75
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 0.9812778234481812,
+      "learning_rate": 8.492569002123141e-06,
+      "loss": 3.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 0.8831024169921875,
+      "learning_rate": 9.023354564755838e-06,
+      "loss": 3.613,
+      "step": 85
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.8358364105224609,
+      "learning_rate": 9.554140127388536e-06,
+      "loss": 3.1858,
+      "step": 90
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 1.0740444660186768,
+      "learning_rate": 1.0084925690021232e-05,
+      "loss": 3.0937,
+      "step": 95
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 1.0987530946731567,
+      "learning_rate": 1.0615711252653929e-05,
+      "loss": 3.154,
+      "step": 100
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 1.2300925254821777,
+      "learning_rate": 1.1146496815286625e-05,
+      "loss": 2.9414,
+      "step": 105
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 1.2214170694351196,
+      "learning_rate": 1.1677282377919321e-05,
+      "loss": 2.9464,
+      "step": 110
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 1.2803975343704224,
+      "learning_rate": 1.2208067940552018e-05,
+      "loss": 2.8921,
+      "step": 115
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 1.2232719659805298,
+      "learning_rate": 1.2738853503184714e-05,
+      "loss": 2.5252,
+      "step": 120
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 1.204835295677185,
+      "learning_rate": 1.326963906581741e-05,
+      "loss": 2.5215,
+      "step": 125
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 1.4095579385757446,
+      "learning_rate": 1.3800424628450107e-05,
+      "loss": 2.136,
+      "step": 130
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 1.4166598320007324,
+      "learning_rate": 1.4331210191082803e-05,
+      "loss": 2.2653,
+      "step": 135
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.3040446043014526,
+      "learning_rate": 1.48619957537155e-05,
+      "loss": 2.0193,
+      "step": 140
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 1.4114688634872437,
+      "learning_rate": 1.5392781316348196e-05,
+      "loss": 1.7935,
+      "step": 145
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 1.8066726922988892,
+      "learning_rate": 1.592356687898089e-05,
+      "loss": 1.5731,
+      "step": 150
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 1.4303158521652222,
+      "learning_rate": 1.6454352441613588e-05,
+      "loss": 1.6552,
+      "step": 155
+    },
+    {
+      "epoch": 1.0126984126984127,
+      "grad_norm": 1.6671762466430664,
+      "learning_rate": 1.6985138004246283e-05,
+      "loss": 1.6973,
+      "step": 160
+    },
+    {
+      "epoch": 1.0444444444444445,
+      "grad_norm": 1.5719650983810425,
+      "learning_rate": 1.751592356687898e-05,
+      "loss": 1.312,
+      "step": 165
+    },
+    {
+      "epoch": 1.0761904761904761,
+      "grad_norm": 1.4845054149627686,
+      "learning_rate": 1.8046709129511676e-05,
+      "loss": 1.3601,
+      "step": 170
+    },
+    {
+      "epoch": 1.107936507936508,
+      "grad_norm": 1.1172235012054443,
+      "learning_rate": 1.8577494692144374e-05,
+      "loss": 1.3137,
+      "step": 175
+    },
+    {
+      "epoch": 1.1396825396825396,
+      "grad_norm": 1.9621731042861938,
+      "learning_rate": 1.910828025477707e-05,
+      "loss": 1.1778,
+      "step": 180
+    },
+    {
+      "epoch": 1.1714285714285715,
+      "grad_norm": 1.7722721099853516,
+      "learning_rate": 1.963906581740977e-05,
+      "loss": 1.4534,
+      "step": 185
+    },
+    {
+      "epoch": 1.2031746031746031,
+      "grad_norm": 1.3677467107772827,
+      "learning_rate": 2.0169851380042464e-05,
+      "loss": 1.3356,
+      "step": 190
+    },
+    {
+      "epoch": 1.234920634920635,
+      "grad_norm": 1.3260482549667358,
+      "learning_rate": 2.0700636942675162e-05,
+      "loss": 1.0876,
+      "step": 195
+    },
+    {
+      "epoch": 1.2666666666666666,
+      "grad_norm": 1.5176818370819092,
+      "learning_rate": 2.1231422505307857e-05,
+      "loss": 1.1602,
+      "step": 200
+    },
+    {
+      "epoch": 1.2984126984126985,
+      "grad_norm": 1.2793077230453491,
+      "learning_rate": 2.1762208067940555e-05,
+      "loss": 1.1505,
+      "step": 205
+    },
+    {
+      "epoch": 1.33015873015873,
+      "grad_norm": 1.196784257888794,
+      "learning_rate": 2.229299363057325e-05,
+      "loss": 1.0664,
+      "step": 210
+    },
+    {
+      "epoch": 1.361904761904762,
+      "grad_norm": 1.303207516670227,
+      "learning_rate": 2.2823779193205948e-05,
+      "loss": 1.2557,
+      "step": 215
+    },
+    {
+      "epoch": 1.3936507936507936,
+      "grad_norm": 1.2853388786315918,
+      "learning_rate": 2.3354564755838642e-05,
+      "loss": 1.0704,
+      "step": 220
+    },
+    {
+      "epoch": 1.4253968253968254,
+      "grad_norm": 1.381369948387146,
+      "learning_rate": 2.388535031847134e-05,
+      "loss": 1.1371,
+      "step": 225
+    },
+    {
+      "epoch": 1.457142857142857,
+      "grad_norm": 1.8012712001800537,
+      "learning_rate": 2.4416135881104035e-05,
+      "loss": 1.248,
+      "step": 230
+    },
+    {
+      "epoch": 1.488888888888889,
+      "grad_norm": 1.7397032976150513,
+      "learning_rate": 2.4946921443736733e-05,
+      "loss": 1.2782,
+      "step": 235
+    },
+    {
+      "epoch": 1.5206349206349206,
+      "grad_norm": 1.4026210308074951,
+      "learning_rate": 2.5477707006369428e-05,
+      "loss": 1.154,
+      "step": 240
+    },
+    {
+      "epoch": 1.5523809523809524,
+      "grad_norm": 1.2906067371368408,
+      "learning_rate": 2.6008492569002126e-05,
+      "loss": 0.9141,
+      "step": 245
+    },
+    {
+      "epoch": 1.5841269841269843,
+      "grad_norm": 1.265598177909851,
+      "learning_rate": 2.653927813163482e-05,
+      "loss": 1.0625,
+      "step": 250
+    },
+    {
+      "epoch": 1.615873015873016,
+      "grad_norm": 1.6044715642929077,
+      "learning_rate": 2.707006369426752e-05,
+      "loss": 0.9624,
+      "step": 255
+    },
+    {
+      "epoch": 1.6476190476190475,
+      "grad_norm": 1.4612747430801392,
+      "learning_rate": 2.7600849256900213e-05,
+      "loss": 1.0413,
+      "step": 260
+    },
+    {
+      "epoch": 1.6793650793650794,
+      "grad_norm": 1.6222745180130005,
+      "learning_rate": 2.8131634819532908e-05,
+      "loss": 1.0929,
+      "step": 265
+    },
+    {
+      "epoch": 1.7111111111111112,
+      "grad_norm": 1.1456222534179688,
+      "learning_rate": 2.8662420382165606e-05,
+      "loss": 0.9957,
+      "step": 270
+    },
+    {
+      "epoch": 1.7428571428571429,
+      "grad_norm": 1.5746041536331177,
+      "learning_rate": 2.91932059447983e-05,
+      "loss": 1.0274,
+      "step": 275
+    },
+    {
+      "epoch": 1.7746031746031745,
+      "grad_norm": 1.3407832384109497,
+      "learning_rate": 2.9723991507431e-05,
+      "loss": 0.9487,
+      "step": 280
+    },
+    {
+      "epoch": 1.8063492063492064,
+      "grad_norm": 1.6232194900512695,
+      "learning_rate": 3.0254777070063693e-05,
+      "loss": 1.0966,
+      "step": 285
+    },
+    {
+      "epoch": 1.8380952380952382,
+      "grad_norm": 1.4920552968978882,
+      "learning_rate": 3.078556263269639e-05,
+      "loss": 0.9099,
+      "step": 290
+    },
+    {
+      "epoch": 1.8698412698412699,
+      "grad_norm": 1.2123301029205322,
+      "learning_rate": 3.1316348195329086e-05,
+      "loss": 1.0902,
+      "step": 295
+    },
+    {
+      "epoch": 1.9015873015873015,
+      "grad_norm": 1.2080968618392944,
+      "learning_rate": 3.184713375796178e-05,
+      "loss": 0.943,
+      "step": 300
+    },
+    {
+      "epoch": 1.9333333333333333,
+      "grad_norm": 1.190319299697876,
+      "learning_rate": 3.237791932059448e-05,
+      "loss": 0.7893,
+      "step": 305
+    },
+    {
+      "epoch": 1.9650793650793652,
+      "grad_norm": 1.5929204225540161,
+      "learning_rate": 3.2908704883227177e-05,
+      "loss": 1.0232,
+      "step": 310
+    },
+    {
+      "epoch": 1.9968253968253968,
+      "grad_norm": 1.0138347148895264,
+      "learning_rate": 3.343949044585987e-05,
+      "loss": 0.6693,
+      "step": 315
+    },
+    {
+      "epoch": 2.0253968253968253,
+      "grad_norm": 1.3012847900390625,
+      "learning_rate": 3.3970276008492566e-05,
+      "loss": 0.8355,
+      "step": 320
+    },
+    {
+      "epoch": 2.057142857142857,
+      "grad_norm": 1.2264782190322876,
+      "learning_rate": 3.450106157112527e-05,
+      "loss": 0.9872,
+      "step": 325
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 1.139275312423706,
+      "learning_rate": 3.503184713375796e-05,
+      "loss": 0.8662,
+      "step": 330
+    },
+    {
+      "epoch": 2.1206349206349207,
+      "grad_norm": 1.3836581707000732,
+      "learning_rate": 3.5562632696390657e-05,
+      "loss": 0.9549,
+      "step": 335
+    },
+    {
+      "epoch": 2.1523809523809523,
+      "grad_norm": 1.368600845336914,
+      "learning_rate": 3.609341825902335e-05,
+      "loss": 0.9195,
+      "step": 340
+    },
+    {
+      "epoch": 2.1841269841269844,
+      "grad_norm": 1.8793011903762817,
+      "learning_rate": 3.662420382165605e-05,
+      "loss": 0.8505,
+      "step": 345
+    },
+    {
+      "epoch": 2.215873015873016,
+      "grad_norm": 1.305284023284912,
+      "learning_rate": 3.715498938428875e-05,
+      "loss": 0.7755,
+      "step": 350
+    },
+    {
+      "epoch": 2.2476190476190476,
+      "grad_norm": 1.7851749658584595,
+      "learning_rate": 3.768577494692145e-05,
+      "loss": 0.9242,
+      "step": 355
+    },
+    {
+      "epoch": 2.2793650793650793,
+      "grad_norm": 1.4341535568237305,
+      "learning_rate": 3.821656050955414e-05,
+      "loss": 0.8221,
+      "step": 360
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 1.39107346534729,
+      "learning_rate": 3.874734607218684e-05,
+      "loss": 0.6999,
+      "step": 365
+    },
+    {
+      "epoch": 2.342857142857143,
+      "grad_norm": 1.2304264307022095,
+      "learning_rate": 3.927813163481954e-05,
+      "loss": 0.8362,
+      "step": 370
+    },
+    {
+      "epoch": 2.3746031746031746,
+      "grad_norm": 1.8470840454101562,
+      "learning_rate": 3.9808917197452234e-05,
+      "loss": 0.9398,
+      "step": 375
+    },
+    {
+      "epoch": 2.4063492063492062,
+      "grad_norm": 1.2533882856369019,
+      "learning_rate": 4.033970276008493e-05,
+      "loss": 0.7754,
+      "step": 380
+    },
+    {
+      "epoch": 2.4380952380952383,
+      "grad_norm": 1.5335006713867188,
+      "learning_rate": 4.087048832271762e-05,
+      "loss": 1.1124,
+      "step": 385
+    },
+    {
+      "epoch": 2.46984126984127,
+      "grad_norm": 1.5298357009887695,
+      "learning_rate": 4.1401273885350325e-05,
+      "loss": 1.017,
+      "step": 390
+    },
+    {
+      "epoch": 2.5015873015873016,
+      "grad_norm": 1.4403260946273804,
+      "learning_rate": 4.193205944798302e-05,
+      "loss": 0.8831,
+      "step": 395
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 1.1528433561325073,
+      "learning_rate": 4.2462845010615714e-05,
+      "loss": 0.801,
+      "step": 400
+    },
+    {
+      "epoch": 2.565079365079365,
+      "grad_norm": 1.3371326923370361,
+      "learning_rate": 4.299363057324841e-05,
+      "loss": 0.8692,
+      "step": 405
+    },
+    {
+      "epoch": 2.596825396825397,
+      "grad_norm": 1.4064775705337524,
+      "learning_rate": 4.352441613588111e-05,
+      "loss": 0.9059,
+      "step": 410
+    },
+    {
+      "epoch": 2.6285714285714286,
+      "grad_norm": 1.4531422853469849,
+      "learning_rate": 4.4055201698513805e-05,
+      "loss": 0.7344,
+      "step": 415
+    },
+    {
+      "epoch": 2.66031746031746,
+      "grad_norm": 1.7043890953063965,
+      "learning_rate": 4.45859872611465e-05,
+      "loss": 0.8298,
+      "step": 420
+    },
+    {
+      "epoch": 2.6920634920634923,
+      "grad_norm": 1.5105586051940918,
+      "learning_rate": 4.5116772823779194e-05,
+      "loss": 0.7768,
+      "step": 425
+    },
+    {
+      "epoch": 2.723809523809524,
+      "grad_norm": 1.8101528882980347,
+      "learning_rate": 4.5647558386411895e-05,
+      "loss": 0.733,
+      "step": 430
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 1.6365174055099487,
+      "learning_rate": 4.617834394904459e-05,
+      "loss": 0.8061,
+      "step": 435
+    },
+    {
+      "epoch": 2.787301587301587,
+      "grad_norm": 1.7808202505111694,
+      "learning_rate": 4.6709129511677285e-05,
+      "loss": 0.8333,
+      "step": 440
+    },
+    {
+      "epoch": 2.819047619047619,
+      "grad_norm": 1.5223265886306763,
+      "learning_rate": 4.723991507430998e-05,
+      "loss": 0.7557,
+      "step": 445
+    },
+    {
+      "epoch": 2.850793650793651,
+      "grad_norm": 1.3064416646957397,
+      "learning_rate": 4.777070063694268e-05,
+      "loss": 0.8041,
+      "step": 450
+    },
+    {
+      "epoch": 2.8825396825396825,
+      "grad_norm": 1.8025637865066528,
+      "learning_rate": 4.8301486199575375e-05,
+      "loss": 0.9534,
+      "step": 455
+    },
+    {
+      "epoch": 2.914285714285714,
+      "grad_norm": 1.924846887588501,
+      "learning_rate": 4.883227176220807e-05,
+      "loss": 0.9066,
+      "step": 460
+    },
+    {
+      "epoch": 2.9460317460317462,
+      "grad_norm": 1.9862899780273438,
+      "learning_rate": 4.9363057324840765e-05,
+      "loss": 0.7994,
+      "step": 465
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 1.9615916013717651,
+      "learning_rate": 4.9893842887473466e-05,
+      "loss": 0.7045,
+      "step": 470
+    },
+    {
+      "epoch": 3.0063492063492063,
+      "grad_norm": 1.519852876663208,
+      "learning_rate": 4.999989014936042e-05,
+      "loss": 0.7212,
+      "step": 475
+    },
+    {
+      "epoch": 3.038095238095238,
+      "grad_norm": 1.9328887462615967,
+      "learning_rate": 4.999944388279162e-05,
+      "loss": 0.6598,
+      "step": 480
+    },
+    {
+      "epoch": 3.06984126984127,
+      "grad_norm": 2.0340709686279297,
+      "learning_rate": 4.999865434075176e-05,
+      "loss": 0.6829,
+      "step": 485
+    },
+    {
+      "epoch": 3.1015873015873017,
+      "grad_norm": 1.8775280714035034,
+      "learning_rate": 4.999752153408229e-05,
+      "loss": 0.6664,
+      "step": 490
+    },
+    {
+      "epoch": 3.1333333333333333,
+      "grad_norm": 2.385218381881714,
+      "learning_rate": 4.999604547833814e-05,
+      "loss": 0.6836,
+      "step": 495
+    },
+    {
+      "epoch": 3.165079365079365,
+      "grad_norm": 2.1743783950805664,
+      "learning_rate": 4.999422619378752e-05,
+      "loss": 0.7,
+      "step": 500
+    },
+    {
+      "epoch": 3.196825396825397,
+      "grad_norm": 2.20786452293396,
+      "learning_rate": 4.999206370541162e-05,
+      "loss": 0.7253,
+      "step": 505
+    },
+    {
+      "epoch": 3.2285714285714286,
+      "grad_norm": 1.8182263374328613,
+      "learning_rate": 4.998955804290425e-05,
+      "loss": 0.6824,
+      "step": 510
+    },
+    {
+      "epoch": 3.2603174603174603,
+      "grad_norm": 2.2959372997283936,
+      "learning_rate": 4.9986709240671495e-05,
+      "loss": 0.601,
+      "step": 515
+    },
+    {
+      "epoch": 3.292063492063492,
+      "grad_norm": 2.385838031768799,
+      "learning_rate": 4.998351733783116e-05,
+      "loss": 0.7417,
+      "step": 520
+    },
+    {
+      "epoch": 3.323809523809524,
+      "grad_norm": 2.0416879653930664,
+      "learning_rate": 4.997998237821233e-05,
+      "loss": 0.6463,
+      "step": 525
+    },
+    {
+      "epoch": 3.3555555555555556,
+      "grad_norm": 2.2781031131744385,
+      "learning_rate": 4.9976104410354654e-05,
+      "loss": 0.6998,
+      "step": 530
+    },
+    {
+      "epoch": 3.3873015873015873,
+      "grad_norm": 2.146778106689453,
+      "learning_rate": 4.9971883487507775e-05,
+      "loss": 0.7694,
+      "step": 535
+    },
+    {
+      "epoch": 3.419047619047619,
+      "grad_norm": 2.1369104385375977,
+      "learning_rate": 4.9967319667630567e-05,
+      "loss": 0.6615,
+      "step": 540
+    },
+    {
+      "epoch": 3.450793650793651,
+      "grad_norm": 2.4529733657836914,
+      "learning_rate": 4.996241301339029e-05,
+      "loss": 0.6109,
+      "step": 545
+    },
+    {
+      "epoch": 3.4825396825396826,
+      "grad_norm": 2.07030987739563,
+      "learning_rate": 4.995716359216183e-05,
+      "loss": 0.7611,
+      "step": 550
+    },
+    {
+      "epoch": 3.5142857142857142,
+      "grad_norm": 2.4329919815063477,
+      "learning_rate": 4.995157147602669e-05,
+      "loss": 0.7515,
+      "step": 555
+    },
+    {
+      "epoch": 3.546031746031746,
+      "grad_norm": 2.056351900100708,
+      "learning_rate": 4.994563674177202e-05,
+      "loss": 0.6885,
+      "step": 560
+    },
+    {
+      "epoch": 3.5777777777777775,
+      "grad_norm": 2.3665318489074707,
+      "learning_rate": 4.993935947088958e-05,
+      "loss": 0.6271,
+      "step": 565
+    },
+    {
+      "epoch": 3.6095238095238096,
+      "grad_norm": 2.677706480026245,
+      "learning_rate": 4.993273974957463e-05,
+      "loss": 0.5586,
+      "step": 570
+    },
+    {
+      "epoch": 3.641269841269841,
+      "grad_norm": 3.422136068344116,
+      "learning_rate": 4.9925777668724685e-05,
+      "loss": 0.7552,
+      "step": 575
+    },
+    {
+      "epoch": 3.6730158730158733,
+      "grad_norm": 2.4525184631347656,
+      "learning_rate": 4.991847332393835e-05,
+      "loss": 0.7367,
+      "step": 580
+    },
+    {
+      "epoch": 3.704761904761905,
+      "grad_norm": 2.4242067337036133,
+      "learning_rate": 4.991082681551396e-05,
+      "loss": 0.7044,
+      "step": 585
+    },
+    {
+      "epoch": 3.7365079365079366,
+      "grad_norm": 1.8419867753982544,
+      "learning_rate": 4.9902838248448184e-05,
+      "loss": 0.5966,
+      "step": 590
+    },
+    {
+      "epoch": 3.768253968253968,
+      "grad_norm": 2.1394360065460205,
+      "learning_rate": 4.989450773243463e-05,
+      "loss": 0.6736,
+      "step": 595
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 1.285447597503662,
+      "learning_rate": 4.9885835381862326e-05,
+      "loss": 0.5021,
+      "step": 600
+    },
+    {
+      "epoch": 3.831746031746032,
+      "grad_norm": 2.724978446960449,
+      "learning_rate": 4.987682131581413e-05,
+      "loss": 0.6128,
+      "step": 605
+    },
+    {
+      "epoch": 3.8634920634920635,
+      "grad_norm": 2.239682912826538,
+      "learning_rate": 4.986746565806508e-05,
+      "loss": 0.5457,
+      "step": 610
+    },
+    {
+      "epoch": 3.895238095238095,
+      "grad_norm": 2.48944091796875,
+      "learning_rate": 4.9857768537080784e-05,
+      "loss": 0.6927,
+      "step": 615
+    },
+    {
+      "epoch": 3.9269841269841272,
+      "grad_norm": 2.4086852073669434,
+      "learning_rate": 4.9847730086015534e-05,
+      "loss": 0.5963,
+      "step": 620
+    },
+    {
+      "epoch": 3.958730158730159,
+      "grad_norm": 2.0070106983184814,
+      "learning_rate": 4.9837350442710553e-05,
+      "loss": 0.5856,
+      "step": 625
+    },
+    {
+      "epoch": 3.9904761904761905,
+      "grad_norm": 1.9726545810699463,
+      "learning_rate": 4.98266297496921e-05,
+      "loss": 0.6208,
+      "step": 630
+    },
+    {
+      "epoch": 4.019047619047619,
+      "grad_norm": 2.6137828826904297,
+      "learning_rate": 4.981556815416948e-05,
+      "loss": 0.6319,
+      "step": 635
+    },
+    {
+      "epoch": 4.050793650793651,
+      "grad_norm": 2.3489890098571777,
+      "learning_rate": 4.9804165808033054e-05,
+      "loss": 0.5887,
+      "step": 640
+    },
+    {
+      "epoch": 4.082539682539682,
+      "grad_norm": 2.8010590076446533,
+      "learning_rate": 4.979242286785214e-05,
+      "loss": 0.5257,
+      "step": 645
+    },
+    {
+      "epoch": 4.114285714285714,
+      "grad_norm": 2.993411064147949,
+      "learning_rate": 4.978033949487284e-05,
+      "loss": 0.4545,
+      "step": 650
+    },
+    {
+      "epoch": 4.146031746031746,
+      "grad_norm": 2.669935703277588,
+      "learning_rate": 4.976791585501588e-05,
+      "loss": 0.5989,
+      "step": 655
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 3.084409236907959,
+      "learning_rate": 4.9755152118874294e-05,
+      "loss": 0.528,
+      "step": 660
+    },
+    {
+      "epoch": 4.20952380952381,
+      "grad_norm": 2.797873020172119,
+      "learning_rate": 4.974204846171106e-05,
+      "loss": 0.5249,
+      "step": 665
+    },
+    {
+      "epoch": 4.241269841269841,
+      "grad_norm": 3.667867422103882,
+      "learning_rate": 4.9728605063456765e-05,
+      "loss": 0.5838,
+      "step": 670
+    },
+    {
+      "epoch": 4.273015873015873,
+      "grad_norm": 2.6918869018554688,
+      "learning_rate": 4.971482210870706e-05,
+      "loss": 0.5143,
+      "step": 675
+    },
+    {
+      "epoch": 4.304761904761905,
+      "grad_norm": 2.1545379161834717,
+      "learning_rate": 4.970069978672017e-05,
+      "loss": 0.5317,
+      "step": 680
+    },
+    {
+      "epoch": 4.336507936507936,
+      "grad_norm": 2.1043529510498047,
+      "learning_rate": 4.9686238291414275e-05,
+      "loss": 0.4815,
+      "step": 685
+    },
+    {
+      "epoch": 4.368253968253969,
+      "grad_norm": 2.1359753608703613,
+      "learning_rate": 4.9671437821364855e-05,
+      "loss": 0.4935,
+      "step": 690
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 3.092057228088379,
+      "learning_rate": 4.965629857980197e-05,
+      "loss": 0.6831,
+      "step": 695
+    },
+    {
+      "epoch": 4.431746031746032,
+      "grad_norm": 2.5296835899353027,
+      "learning_rate": 4.964082077460745e-05,
+      "loss": 0.5323,
+      "step": 700
+    },
+    {
+      "epoch": 4.463492063492064,
+      "grad_norm": 1.6655627489089966,
+      "learning_rate": 4.962500461831207e-05,
+      "loss": 0.4553,
+      "step": 705
+    },
+    {
+      "epoch": 4.495238095238095,
+      "grad_norm": 2.6663475036621094,
+      "learning_rate": 4.9608850328092576e-05,
+      "loss": 0.463,
+      "step": 710
+    },
+    {
+      "epoch": 4.526984126984127,
+      "grad_norm": 2.3763060569763184,
+      "learning_rate": 4.959235812576879e-05,
+      "loss": 0.4861,
+      "step": 715
+    },
+    {
+      "epoch": 4.5587301587301585,
+      "grad_norm": 2.2217962741851807,
+      "learning_rate": 4.957552823780047e-05,
+      "loss": 0.468,
+      "step": 720
+    },
+    {
+      "epoch": 4.59047619047619,
+      "grad_norm": 2.8885600566864014,
+      "learning_rate": 4.9558360895284295e-05,
+      "loss": 0.4588,
+      "step": 725
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 2.5661261081695557,
+      "learning_rate": 4.954085633395058e-05,
+      "loss": 0.4926,
+      "step": 730
+    },
+    {
+      "epoch": 4.653968253968254,
+      "grad_norm": 2.304365396499634,
+      "learning_rate": 4.952301479416015e-05,
+      "loss": 0.494,
+      "step": 735
+    },
+    {
+      "epoch": 4.685714285714286,
+      "grad_norm": 2.690577983856201,
+      "learning_rate": 4.9504836520900976e-05,
+      "loss": 0.5814,
+      "step": 740
+    },
+    {
+      "epoch": 4.717460317460318,
+      "grad_norm": 2.7180025577545166,
+      "learning_rate": 4.948632176378481e-05,
+      "loss": 0.5329,
+      "step": 745
+    },
+    {
+      "epoch": 4.749206349206349,
+      "grad_norm": 2.716587543487549,
+      "learning_rate": 4.9467470777043806e-05,
+      "loss": 0.5264,
+      "step": 750
+    },
+    {
+      "epoch": 4.780952380952381,
+      "grad_norm": 2.315419912338257,
+      "learning_rate": 4.9448283819526954e-05,
+      "loss": 0.4756,
+      "step": 755
+    },
+    {
+      "epoch": 4.8126984126984125,
+      "grad_norm": 2.1679515838623047,
+      "learning_rate": 4.9428761154696605e-05,
+      "loss": 0.4819,
+      "step": 760
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 3.389266014099121,
+      "learning_rate": 4.9408903050624796e-05,
+      "loss": 0.5121,
+      "step": 765
+    },
+    {
+      "epoch": 4.876190476190477,
+      "grad_norm": 3.4317383766174316,
+      "learning_rate": 4.938870977998959e-05,
+      "loss": 0.4535,
+      "step": 770
+    },
+    {
+      "epoch": 4.907936507936508,
+      "grad_norm": 2.9491918087005615,
+      "learning_rate": 4.9368181620071344e-05,
+      "loss": 0.5333,
+      "step": 775
+    },
+    {
+      "epoch": 4.93968253968254,
+      "grad_norm": 2.516798496246338,
+      "learning_rate": 4.934731885274887e-05,
+      "loss": 0.5367,
+      "step": 780
+    },
+    {
+      "epoch": 4.9714285714285715,
+      "grad_norm": 3.0031046867370605,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.4957,
+      "step": 785
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.334085702896118,
+      "learning_rate": 4.9304590646375614e-05,
+      "loss": 0.5287,
+      "step": 790
+    },
+    {
+      "epoch": 5.031746031746032,
+      "grad_norm": 1.9608453512191772,
+      "learning_rate": 4.928272579403969e-05,
+      "loss": 0.36,
+      "step": 795
+    },
+    {
+      "epoch": 5.063492063492063,
+      "grad_norm": 2.328850746154785,
+      "learning_rate": 4.92605275077212e-05,
+      "loss": 0.3628,
+      "step": 800
+    },
+    {
+      "epoch": 5.095238095238095,
+      "grad_norm": 2.3446412086486816,
+      "learning_rate": 4.923799609223202e-05,
+      "loss": 0.3327,
+      "step": 805
+    },
+    {
+      "epoch": 5.1269841269841265,
+      "grad_norm": 2.476181745529175,
+      "learning_rate": 4.921513185695831e-05,
+      "loss": 0.4246,
+      "step": 810
+    },
+    {
+      "epoch": 5.158730158730159,
+      "grad_norm": 3.1026763916015625,
+      "learning_rate": 4.91919351158563e-05,
+      "loss": 0.5048,
+      "step": 815
+    },
+    {
+      "epoch": 5.190476190476191,
+      "grad_norm": 2.8165297508239746,
+      "learning_rate": 4.916840618744798e-05,
+      "loss": 0.4361,
+      "step": 820
+    },
+    {
+      "epoch": 5.222222222222222,
+      "grad_norm": 1.8732138872146606,
+      "learning_rate": 4.9144545394816687e-05,
+      "loss": 0.4693,
+      "step": 825
+    },
+    {
+      "epoch": 5.253968253968254,
+      "grad_norm": 1.7250264883041382,
+      "learning_rate": 4.91203530656027e-05,
+      "loss": 0.4076,
+      "step": 830
+    },
+    {
+      "epoch": 5.285714285714286,
+      "grad_norm": 2.105459690093994,
+      "learning_rate": 4.9095829531998725e-05,
+      "loss": 0.3589,
+      "step": 835
+    },
+    {
+      "epoch": 5.317460317460317,
+      "grad_norm": 3.6825687885284424,
+      "learning_rate": 4.9070975130745387e-05,
+      "loss": 0.5263,
+      "step": 840
+    },
+    {
+      "epoch": 5.349206349206349,
+      "grad_norm": 2.947052001953125,
+      "learning_rate": 4.90457902031265e-05,
+      "loss": 0.4632,
+      "step": 845
+    },
+    {
+      "epoch": 5.380952380952381,
+      "grad_norm": 1.9546104669570923,
+      "learning_rate": 4.902027509496448e-05,
+      "loss": 0.4348,
+      "step": 850
+    },
+    {
+      "epoch": 5.412698412698413,
+      "grad_norm": 2.4471983909606934,
+      "learning_rate": 4.899443015661557e-05,
+      "loss": 0.4209,
+      "step": 855
+    },
+    {
+      "epoch": 5.444444444444445,
+      "grad_norm": 1.827124834060669,
+      "learning_rate": 4.8968255742964975e-05,
+      "loss": 0.413,
+      "step": 860
+    },
+    {
+      "epoch": 5.476190476190476,
+      "grad_norm": 2.654707431793213,
+      "learning_rate": 4.894175221342207e-05,
+      "loss": 0.432,
+      "step": 865
+    },
+    {
+      "epoch": 5.507936507936508,
+      "grad_norm": 2.648967981338501,
+      "learning_rate": 4.8914919931915407e-05,
+      "loss": 0.4339,
+      "step": 870
+    },
+    {
+      "epoch": 5.5396825396825395,
+      "grad_norm": 2.874075412750244,
+      "learning_rate": 4.888775926688775e-05,
+      "loss": 0.4392,
+      "step": 875
+    },
+    {
+      "epoch": 5.571428571428571,
+      "grad_norm": 2.9674830436706543,
+      "learning_rate": 4.8860270591291e-05,
+      "loss": 0.4459,
+      "step": 880
+    },
+    {
+      "epoch": 5.603174603174603,
+      "grad_norm": 2.054748296737671,
+      "learning_rate": 4.883245428258107e-05,
+      "loss": 0.4313,
+      "step": 885
+    },
+    {
+      "epoch": 5.634920634920634,
+      "grad_norm": 1.9174392223358154,
+      "learning_rate": 4.880431072271272e-05,
+      "loss": 0.3906,
+      "step": 890
+    },
+    {
+      "epoch": 5.666666666666667,
+      "grad_norm": 2.5257787704467773,
+      "learning_rate": 4.87758402981343e-05,
+      "loss": 0.4219,
+      "step": 895
+    },
+    {
+      "epoch": 5.698412698412699,
+      "grad_norm": 2.6365532875061035,
+      "learning_rate": 4.8747043399782424e-05,
+      "loss": 0.3978,
+      "step": 900
+    },
+    {
+      "epoch": 5.73015873015873,
+      "grad_norm": 2.0583746433258057,
+      "learning_rate": 4.871792042307667e-05,
+      "loss": 0.4847,
+      "step": 905
+    },
+    {
+      "epoch": 5.761904761904762,
+      "grad_norm": 2.035872459411621,
+      "learning_rate": 4.868847176791406e-05,
+      "loss": 0.4675,
+      "step": 910
+    },
+    {
+      "epoch": 5.7936507936507935,
+      "grad_norm": 2.3722939491271973,
+      "learning_rate": 4.8658697838663625e-05,
+      "loss": 0.4586,
+      "step": 915
+    },
+    {
+      "epoch": 5.825396825396825,
+      "grad_norm": 1.2609732151031494,
+      "learning_rate": 4.862859904416085e-05,
+      "loss": 0.3274,
+      "step": 920
+    },
+    {
+      "epoch": 5.857142857142857,
+      "grad_norm": 2.3673977851867676,
+      "learning_rate": 4.8598175797702036e-05,
+      "loss": 0.4685,
+      "step": 925
+    },
+    {
+      "epoch": 5.888888888888889,
+      "grad_norm": 2.8414175510406494,
+      "learning_rate": 4.856742851703866e-05,
+      "loss": 0.4762,
+      "step": 930
+    },
+    {
+      "epoch": 5.920634920634921,
+      "grad_norm": 2.4126765727996826,
+      "learning_rate": 4.853635762437159e-05,
+      "loss": 0.4075,
+      "step": 935
+    },
+    {
+      "epoch": 5.9523809523809526,
+      "grad_norm": 1.8691045045852661,
+      "learning_rate": 4.8504963546345334e-05,
+      "loss": 0.4865,
+      "step": 940
+    },
+    {
+      "epoch": 5.984126984126984,
+      "grad_norm": 3.5297420024871826,
+      "learning_rate": 4.8473246714042155e-05,
+      "loss": 0.4623,
+      "step": 945
+    },
+    {
+      "epoch": 6.012698412698413,
+      "grad_norm": 2.059169054031372,
+      "learning_rate": 4.844120756297617e-05,
+      "loss": 0.4164,
+      "step": 950
+    },
+    {
+      "epoch": 6.044444444444444,
+      "grad_norm": 2.4746127128601074,
+      "learning_rate": 4.840884653308735e-05,
+      "loss": 0.3552,
+      "step": 955
+    },
+    {
+      "epoch": 6.076190476190476,
+      "grad_norm": 2.504425287246704,
+      "learning_rate": 4.8376164068735485e-05,
+      "loss": 0.3368,
+      "step": 960
+    },
+    {
+      "epoch": 6.1079365079365076,
+      "grad_norm": 2.062577486038208,
+      "learning_rate": 4.83431606186941e-05,
+      "loss": 0.3139,
+      "step": 965
+    },
+    {
+      "epoch": 6.13968253968254,
+      "grad_norm": 2.4934544563293457,
+      "learning_rate": 4.830983663614427e-05,
+      "loss": 0.3777,
+      "step": 970
+    },
+    {
+      "epoch": 6.171428571428572,
+      "grad_norm": 2.5747485160827637,
+      "learning_rate": 4.827619257866839e-05,
+      "loss": 0.373,
+      "step": 975
+    },
+    {
+      "epoch": 6.203174603174603,
+      "grad_norm": 2.449357271194458,
+      "learning_rate": 4.8242228908243946e-05,
+      "loss": 0.3936,
+      "step": 980
+    },
+    {
+      "epoch": 6.234920634920635,
+      "grad_norm": 2.952680826187134,
+      "learning_rate": 4.82079460912371e-05,
+      "loss": 0.407,
+      "step": 985
+    },
+    {
+      "epoch": 6.266666666666667,
+      "grad_norm": 2.1754496097564697,
+      "learning_rate": 4.817334459839633e-05,
+      "loss": 0.3189,
+      "step": 990
+    },
+    {
+      "epoch": 6.298412698412698,
+      "grad_norm": 2.8406214714050293,
+      "learning_rate": 4.8138424904845947e-05,
+      "loss": 0.3883,
+      "step": 995
+    },
+    {
+      "epoch": 6.33015873015873,
+      "grad_norm": 1.7533257007598877,
+      "learning_rate": 4.8103187490079604e-05,
+      "loss": 0.3131,
+      "step": 1000
+    },
+    {
+      "epoch": 6.3619047619047615,
+      "grad_norm": 2.4574601650238037,
+      "learning_rate": 4.806763283795366e-05,
+      "loss": 0.3606,
+      "step": 1005
+    },
+    {
+      "epoch": 6.393650793650794,
+      "grad_norm": 2.002281427383423,
+      "learning_rate": 4.8031761436680575e-05,
+      "loss": 0.37,
+      "step": 1010
+    },
+    {
+      "epoch": 6.425396825396826,
+      "grad_norm": 2.823315143585205,
+      "learning_rate": 4.79955737788222e-05,
+      "loss": 0.3791,
+      "step": 1015
+    },
+    {
+      "epoch": 6.457142857142857,
+      "grad_norm": 2.7891204357147217,
+      "learning_rate": 4.795907036128299e-05,
+      "loss": 0.3556,
+      "step": 1020
+    },
+    {
+      "epoch": 6.488888888888889,
+      "grad_norm": 2.2387146949768066,
+      "learning_rate": 4.7922251685303213e-05,
+      "loss": 0.3929,
+      "step": 1025
+    },
+    {
+      "epoch": 6.520634920634921,
+      "grad_norm": 2.5023891925811768,
+      "learning_rate": 4.788511825645205e-05,
+      "loss": 0.379,
+      "step": 1030
+    },
+    {
+      "epoch": 6.552380952380952,
+      "grad_norm": 2.2654805183410645,
+      "learning_rate": 4.7847670584620653e-05,
+      "loss": 0.3435,
+      "step": 1035
+    },
+    {
+      "epoch": 6.584126984126984,
+      "grad_norm": 3.3823065757751465,
+      "learning_rate": 4.7809909184015146e-05,
+      "loss": 0.4109,
+      "step": 1040
+    },
+    {
+      "epoch": 6.6158730158730155,
+      "grad_norm": 2.6096551418304443,
+      "learning_rate": 4.7771834573149576e-05,
+      "loss": 0.4233,
+      "step": 1045
+    },
+    {
+      "epoch": 6.647619047619048,
+      "grad_norm": 2.3933897018432617,
+      "learning_rate": 4.773344727483876e-05,
+      "loss": 0.3709,
+      "step": 1050
+    },
+    {
+      "epoch": 6.67936507936508,
+      "grad_norm": 2.189544916152954,
+      "learning_rate": 4.769474781619114e-05,
+      "loss": 0.3287,
+      "step": 1055
+    },
+    {
+      "epoch": 6.711111111111111,
+      "grad_norm": 2.450892686843872,
+      "learning_rate": 4.765573672860154e-05,
+      "loss": 0.4022,
+      "step": 1060
+    },
+    {
+      "epoch": 6.742857142857143,
+      "grad_norm": 2.4342429637908936,
+      "learning_rate": 4.761641454774386e-05,
+      "loss": 0.4029,
+      "step": 1065
+    },
+    {
+      "epoch": 6.7746031746031745,
+      "grad_norm": 2.2122364044189453,
+      "learning_rate": 4.75767818135637e-05,
+      "loss": 0.3322,
+      "step": 1070
+    },
+    {
+      "epoch": 6.806349206349206,
+      "grad_norm": 3.968445301055908,
+      "learning_rate": 4.7536839070271e-05,
+      "loss": 0.3836,
+      "step": 1075
+    },
+    {
+      "epoch": 6.838095238095238,
+      "grad_norm": 3.529158353805542,
+      "learning_rate": 4.749658686633251e-05,
+      "loss": 0.4745,
+      "step": 1080
+    },
+    {
+      "epoch": 6.86984126984127,
+      "grad_norm": 2.430727243423462,
+      "learning_rate": 4.7456025754464304e-05,
+      "loss": 0.3664,
+      "step": 1085
+    },
+    {
+      "epoch": 6.901587301587302,
+      "grad_norm": 2.6552302837371826,
+      "learning_rate": 4.7415156291624166e-05,
+      "loss": 0.4359,
+      "step": 1090
+    },
+    {
+      "epoch": 6.933333333333334,
+      "grad_norm": 2.134822130203247,
+      "learning_rate": 4.737397903900393e-05,
+      "loss": 0.3969,
+      "step": 1095
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4710,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 30,
+  "save_steps": 157,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.361193959469875e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2355/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-2355/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-2355/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2355/trainer_state.json

@@ -0,0 +1,3331 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 14.907936507936508,
+  "eval_steps": 500,
+  "global_step": 2355,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 0.5545095205307007,
+      "learning_rate": 5.307855626326963e-07,
+      "loss": 3.7162,
+      "step": 5
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 0.6163601279258728,
+      "learning_rate": 1.0615711252653927e-06,
+      "loss": 3.9388,
+      "step": 10
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.5541530847549438,
+      "learning_rate": 1.5923566878980892e-06,
+      "loss": 3.9165,
+      "step": 15
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 0.457332044839859,
+      "learning_rate": 2.1231422505307854e-06,
+      "loss": 3.7326,
+      "step": 20
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 0.5335279107093811,
+      "learning_rate": 2.653927813163482e-06,
+      "loss": 3.8251,
+      "step": 25
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.7080379724502563,
+      "learning_rate": 3.1847133757961785e-06,
+      "loss": 3.7534,
+      "step": 30
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.520993709564209,
+      "learning_rate": 3.715498938428875e-06,
+      "loss": 3.898,
+      "step": 35
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 0.5451405644416809,
+      "learning_rate": 4.246284501061571e-06,
+      "loss": 3.8951,
+      "step": 40
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.6205154657363892,
+      "learning_rate": 4.777070063694268e-06,
+      "loss": 3.7666,
+      "step": 45
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 0.7404439449310303,
+      "learning_rate": 5.307855626326964e-06,
+      "loss": 4.0258,
+      "step": 50
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 0.6272220015525818,
+      "learning_rate": 5.838641188959661e-06,
+      "loss": 3.8464,
+      "step": 55
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 0.7744691967964172,
+      "learning_rate": 6.369426751592357e-06,
+      "loss": 3.7299,
+      "step": 60
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 0.8805738687515259,
+      "learning_rate": 6.900212314225053e-06,
+      "loss": 3.5008,
+      "step": 65
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.0740723609924316,
+      "learning_rate": 7.43099787685775e-06,
+      "loss": 3.7552,
+      "step": 70
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.965708315372467,
+      "learning_rate": 7.961783439490445e-06,
+      "loss": 3.5516,
+      "step": 75
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 0.9812778234481812,
+      "learning_rate": 8.492569002123141e-06,
+      "loss": 3.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 0.8831024169921875,
+      "learning_rate": 9.023354564755838e-06,
+      "loss": 3.613,
+      "step": 85
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.8358364105224609,
+      "learning_rate": 9.554140127388536e-06,
+      "loss": 3.1858,
+      "step": 90
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 1.0740444660186768,
+      "learning_rate": 1.0084925690021232e-05,
+      "loss": 3.0937,
+      "step": 95
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 1.0987530946731567,
+      "learning_rate": 1.0615711252653929e-05,
+      "loss": 3.154,
+      "step": 100
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 1.2300925254821777,
+      "learning_rate": 1.1146496815286625e-05,
+      "loss": 2.9414,
+      "step": 105
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 1.2214170694351196,
+      "learning_rate": 1.1677282377919321e-05,
+      "loss": 2.9464,
+      "step": 110
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 1.2803975343704224,
+      "learning_rate": 1.2208067940552018e-05,
+      "loss": 2.8921,
+      "step": 115
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 1.2232719659805298,
+      "learning_rate": 1.2738853503184714e-05,
+      "loss": 2.5252,
+      "step": 120
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 1.204835295677185,
+      "learning_rate": 1.326963906581741e-05,
+      "loss": 2.5215,
+      "step": 125
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 1.4095579385757446,
+      "learning_rate": 1.3800424628450107e-05,
+      "loss": 2.136,
+      "step": 130
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 1.4166598320007324,
+      "learning_rate": 1.4331210191082803e-05,
+      "loss": 2.2653,
+      "step": 135
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.3040446043014526,
+      "learning_rate": 1.48619957537155e-05,
+      "loss": 2.0193,
+      "step": 140
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 1.4114688634872437,
+      "learning_rate": 1.5392781316348196e-05,
+      "loss": 1.7935,
+      "step": 145
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 1.8066726922988892,
+      "learning_rate": 1.592356687898089e-05,
+      "loss": 1.5731,
+      "step": 150
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 1.4303158521652222,
+      "learning_rate": 1.6454352441613588e-05,
+      "loss": 1.6552,
+      "step": 155
+    },
+    {
+      "epoch": 1.0126984126984127,
+      "grad_norm": 1.6671762466430664,
+      "learning_rate": 1.6985138004246283e-05,
+      "loss": 1.6973,
+      "step": 160
+    },
+    {
+      "epoch": 1.0444444444444445,
+      "grad_norm": 1.5719650983810425,
+      "learning_rate": 1.751592356687898e-05,
+      "loss": 1.312,
+      "step": 165
+    },
+    {
+      "epoch": 1.0761904761904761,
+      "grad_norm": 1.4845054149627686,
+      "learning_rate": 1.8046709129511676e-05,
+      "loss": 1.3601,
+      "step": 170
+    },
+    {
+      "epoch": 1.107936507936508,
+      "grad_norm": 1.1172235012054443,
+      "learning_rate": 1.8577494692144374e-05,
+      "loss": 1.3137,
+      "step": 175
+    },
+    {
+      "epoch": 1.1396825396825396,
+      "grad_norm": 1.9621731042861938,
+      "learning_rate": 1.910828025477707e-05,
+      "loss": 1.1778,
+      "step": 180
+    },
+    {
+      "epoch": 1.1714285714285715,
+      "grad_norm": 1.7722721099853516,
+      "learning_rate": 1.963906581740977e-05,
+      "loss": 1.4534,
+      "step": 185
+    },
+    {
+      "epoch": 1.2031746031746031,
+      "grad_norm": 1.3677467107772827,
+      "learning_rate": 2.0169851380042464e-05,
+      "loss": 1.3356,
+      "step": 190
+    },
+    {
+      "epoch": 1.234920634920635,
+      "grad_norm": 1.3260482549667358,
+      "learning_rate": 2.0700636942675162e-05,
+      "loss": 1.0876,
+      "step": 195
+    },
+    {
+      "epoch": 1.2666666666666666,
+      "grad_norm": 1.5176818370819092,
+      "learning_rate": 2.1231422505307857e-05,
+      "loss": 1.1602,
+      "step": 200
+    },
+    {
+      "epoch": 1.2984126984126985,
+      "grad_norm": 1.2793077230453491,
+      "learning_rate": 2.1762208067940555e-05,
+      "loss": 1.1505,
+      "step": 205
+    },
+    {
+      "epoch": 1.33015873015873,
+      "grad_norm": 1.196784257888794,
+      "learning_rate": 2.229299363057325e-05,
+      "loss": 1.0664,
+      "step": 210
+    },
+    {
+      "epoch": 1.361904761904762,
+      "grad_norm": 1.303207516670227,
+      "learning_rate": 2.2823779193205948e-05,
+      "loss": 1.2557,
+      "step": 215
+    },
+    {
+      "epoch": 1.3936507936507936,
+      "grad_norm": 1.2853388786315918,
+      "learning_rate": 2.3354564755838642e-05,
+      "loss": 1.0704,
+      "step": 220
+    },
+    {
+      "epoch": 1.4253968253968254,
+      "grad_norm": 1.381369948387146,
+      "learning_rate": 2.388535031847134e-05,
+      "loss": 1.1371,
+      "step": 225
+    },
+    {
+      "epoch": 1.457142857142857,
+      "grad_norm": 1.8012712001800537,
+      "learning_rate": 2.4416135881104035e-05,
+      "loss": 1.248,
+      "step": 230
+    },
+    {
+      "epoch": 1.488888888888889,
+      "grad_norm": 1.7397032976150513,
+      "learning_rate": 2.4946921443736733e-05,
+      "loss": 1.2782,
+      "step": 235
+    },
+    {
+      "epoch": 1.5206349206349206,
+      "grad_norm": 1.4026210308074951,
+      "learning_rate": 2.5477707006369428e-05,
+      "loss": 1.154,
+      "step": 240
+    },
+    {
+      "epoch": 1.5523809523809524,
+      "grad_norm": 1.2906067371368408,
+      "learning_rate": 2.6008492569002126e-05,
+      "loss": 0.9141,
+      "step": 245
+    },
+    {
+      "epoch": 1.5841269841269843,
+      "grad_norm": 1.265598177909851,
+      "learning_rate": 2.653927813163482e-05,
+      "loss": 1.0625,
+      "step": 250
+    },
+    {
+      "epoch": 1.615873015873016,
+      "grad_norm": 1.6044715642929077,
+      "learning_rate": 2.707006369426752e-05,
+      "loss": 0.9624,
+      "step": 255
+    },
+    {
+      "epoch": 1.6476190476190475,
+      "grad_norm": 1.4612747430801392,
+      "learning_rate": 2.7600849256900213e-05,
+      "loss": 1.0413,
+      "step": 260
+    },
+    {
+      "epoch": 1.6793650793650794,
+      "grad_norm": 1.6222745180130005,
+      "learning_rate": 2.8131634819532908e-05,
+      "loss": 1.0929,
+      "step": 265
+    },
+    {
+      "epoch": 1.7111111111111112,
+      "grad_norm": 1.1456222534179688,
+      "learning_rate": 2.8662420382165606e-05,
+      "loss": 0.9957,
+      "step": 270
+    },
+    {
+      "epoch": 1.7428571428571429,
+      "grad_norm": 1.5746041536331177,
+      "learning_rate": 2.91932059447983e-05,
+      "loss": 1.0274,
+      "step": 275
+    },
+    {
+      "epoch": 1.7746031746031745,
+      "grad_norm": 1.3407832384109497,
+      "learning_rate": 2.9723991507431e-05,
+      "loss": 0.9487,
+      "step": 280
+    },
+    {
+      "epoch": 1.8063492063492064,
+      "grad_norm": 1.6232194900512695,
+      "learning_rate": 3.0254777070063693e-05,
+      "loss": 1.0966,
+      "step": 285
+    },
+    {
+      "epoch": 1.8380952380952382,
+      "grad_norm": 1.4920552968978882,
+      "learning_rate": 3.078556263269639e-05,
+      "loss": 0.9099,
+      "step": 290
+    },
+    {
+      "epoch": 1.8698412698412699,
+      "grad_norm": 1.2123301029205322,
+      "learning_rate": 3.1316348195329086e-05,
+      "loss": 1.0902,
+      "step": 295
+    },
+    {
+      "epoch": 1.9015873015873015,
+      "grad_norm": 1.2080968618392944,
+      "learning_rate": 3.184713375796178e-05,
+      "loss": 0.943,
+      "step": 300
+    },
+    {
+      "epoch": 1.9333333333333333,
+      "grad_norm": 1.190319299697876,
+      "learning_rate": 3.237791932059448e-05,
+      "loss": 0.7893,
+      "step": 305
+    },
+    {
+      "epoch": 1.9650793650793652,
+      "grad_norm": 1.5929204225540161,
+      "learning_rate": 3.2908704883227177e-05,
+      "loss": 1.0232,
+      "step": 310
+    },
+    {
+      "epoch": 1.9968253968253968,
+      "grad_norm": 1.0138347148895264,
+      "learning_rate": 3.343949044585987e-05,
+      "loss": 0.6693,
+      "step": 315
+    },
+    {
+      "epoch": 2.0253968253968253,
+      "grad_norm": 1.3012847900390625,
+      "learning_rate": 3.3970276008492566e-05,
+      "loss": 0.8355,
+      "step": 320
+    },
+    {
+      "epoch": 2.057142857142857,
+      "grad_norm": 1.2264782190322876,
+      "learning_rate": 3.450106157112527e-05,
+      "loss": 0.9872,
+      "step": 325
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 1.139275312423706,
+      "learning_rate": 3.503184713375796e-05,
+      "loss": 0.8662,
+      "step": 330
+    },
+    {
+      "epoch": 2.1206349206349207,
+      "grad_norm": 1.3836581707000732,
+      "learning_rate": 3.5562632696390657e-05,
+      "loss": 0.9549,
+      "step": 335
+    },
+    {
+      "epoch": 2.1523809523809523,
+      "grad_norm": 1.368600845336914,
+      "learning_rate": 3.609341825902335e-05,
+      "loss": 0.9195,
+      "step": 340
+    },
+    {
+      "epoch": 2.1841269841269844,
+      "grad_norm": 1.8793011903762817,
+      "learning_rate": 3.662420382165605e-05,
+      "loss": 0.8505,
+      "step": 345
+    },
+    {
+      "epoch": 2.215873015873016,
+      "grad_norm": 1.305284023284912,
+      "learning_rate": 3.715498938428875e-05,
+      "loss": 0.7755,
+      "step": 350
+    },
+    {
+      "epoch": 2.2476190476190476,
+      "grad_norm": 1.7851749658584595,
+      "learning_rate": 3.768577494692145e-05,
+      "loss": 0.9242,
+      "step": 355
+    },
+    {
+      "epoch": 2.2793650793650793,
+      "grad_norm": 1.4341535568237305,
+      "learning_rate": 3.821656050955414e-05,
+      "loss": 0.8221,
+      "step": 360
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 1.39107346534729,
+      "learning_rate": 3.874734607218684e-05,
+      "loss": 0.6999,
+      "step": 365
+    },
+    {
+      "epoch": 2.342857142857143,
+      "grad_norm": 1.2304264307022095,
+      "learning_rate": 3.927813163481954e-05,
+      "loss": 0.8362,
+      "step": 370
+    },
+    {
+      "epoch": 2.3746031746031746,
+      "grad_norm": 1.8470840454101562,
+      "learning_rate": 3.9808917197452234e-05,
+      "loss": 0.9398,
+      "step": 375
+    },
+    {
+      "epoch": 2.4063492063492062,
+      "grad_norm": 1.2533882856369019,
+      "learning_rate": 4.033970276008493e-05,
+      "loss": 0.7754,
+      "step": 380
+    },
+    {
+      "epoch": 2.4380952380952383,
+      "grad_norm": 1.5335006713867188,
+      "learning_rate": 4.087048832271762e-05,
+      "loss": 1.1124,
+      "step": 385
+    },
+    {
+      "epoch": 2.46984126984127,
+      "grad_norm": 1.5298357009887695,
+      "learning_rate": 4.1401273885350325e-05,
+      "loss": 1.017,
+      "step": 390
+    },
+    {
+      "epoch": 2.5015873015873016,
+      "grad_norm": 1.4403260946273804,
+      "learning_rate": 4.193205944798302e-05,
+      "loss": 0.8831,
+      "step": 395
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 1.1528433561325073,
+      "learning_rate": 4.2462845010615714e-05,
+      "loss": 0.801,
+      "step": 400
+    },
+    {
+      "epoch": 2.565079365079365,
+      "grad_norm": 1.3371326923370361,
+      "learning_rate": 4.299363057324841e-05,
+      "loss": 0.8692,
+      "step": 405
+    },
+    {
+      "epoch": 2.596825396825397,
+      "grad_norm": 1.4064775705337524,
+      "learning_rate": 4.352441613588111e-05,
+      "loss": 0.9059,
+      "step": 410
+    },
+    {
+      "epoch": 2.6285714285714286,
+      "grad_norm": 1.4531422853469849,
+      "learning_rate": 4.4055201698513805e-05,
+      "loss": 0.7344,
+      "step": 415
+    },
+    {
+      "epoch": 2.66031746031746,
+      "grad_norm": 1.7043890953063965,
+      "learning_rate": 4.45859872611465e-05,
+      "loss": 0.8298,
+      "step": 420
+    },
+    {
+      "epoch": 2.6920634920634923,
+      "grad_norm": 1.5105586051940918,
+      "learning_rate": 4.5116772823779194e-05,
+      "loss": 0.7768,
+      "step": 425
+    },
+    {
+      "epoch": 2.723809523809524,
+      "grad_norm": 1.8101528882980347,
+      "learning_rate": 4.5647558386411895e-05,
+      "loss": 0.733,
+      "step": 430
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 1.6365174055099487,
+      "learning_rate": 4.617834394904459e-05,
+      "loss": 0.8061,
+      "step": 435
+    },
+    {
+      "epoch": 2.787301587301587,
+      "grad_norm": 1.7808202505111694,
+      "learning_rate": 4.6709129511677285e-05,
+      "loss": 0.8333,
+      "step": 440
+    },
+    {
+      "epoch": 2.819047619047619,
+      "grad_norm": 1.5223265886306763,
+      "learning_rate": 4.723991507430998e-05,
+      "loss": 0.7557,
+      "step": 445
+    },
+    {
+      "epoch": 2.850793650793651,
+      "grad_norm": 1.3064416646957397,
+      "learning_rate": 4.777070063694268e-05,
+      "loss": 0.8041,
+      "step": 450
+    },
+    {
+      "epoch": 2.8825396825396825,
+      "grad_norm": 1.8025637865066528,
+      "learning_rate": 4.8301486199575375e-05,
+      "loss": 0.9534,
+      "step": 455
+    },
+    {
+      "epoch": 2.914285714285714,
+      "grad_norm": 1.924846887588501,
+      "learning_rate": 4.883227176220807e-05,
+      "loss": 0.9066,
+      "step": 460
+    },
+    {
+      "epoch": 2.9460317460317462,
+      "grad_norm": 1.9862899780273438,
+      "learning_rate": 4.9363057324840765e-05,
+      "loss": 0.7994,
+      "step": 465
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 1.9615916013717651,
+      "learning_rate": 4.9893842887473466e-05,
+      "loss": 0.7045,
+      "step": 470
+    },
+    {
+      "epoch": 3.0063492063492063,
+      "grad_norm": 1.519852876663208,
+      "learning_rate": 4.999989014936042e-05,
+      "loss": 0.7212,
+      "step": 475
+    },
+    {
+      "epoch": 3.038095238095238,
+      "grad_norm": 1.9328887462615967,
+      "learning_rate": 4.999944388279162e-05,
+      "loss": 0.6598,
+      "step": 480
+    },
+    {
+      "epoch": 3.06984126984127,
+      "grad_norm": 2.0340709686279297,
+      "learning_rate": 4.999865434075176e-05,
+      "loss": 0.6829,
+      "step": 485
+    },
+    {
+      "epoch": 3.1015873015873017,
+      "grad_norm": 1.8775280714035034,
+      "learning_rate": 4.999752153408229e-05,
+      "loss": 0.6664,
+      "step": 490
+    },
+    {
+      "epoch": 3.1333333333333333,
+      "grad_norm": 2.385218381881714,
+      "learning_rate": 4.999604547833814e-05,
+      "loss": 0.6836,
+      "step": 495
+    },
+    {
+      "epoch": 3.165079365079365,
+      "grad_norm": 2.1743783950805664,
+      "learning_rate": 4.999422619378752e-05,
+      "loss": 0.7,
+      "step": 500
+    },
+    {
+      "epoch": 3.196825396825397,
+      "grad_norm": 2.20786452293396,
+      "learning_rate": 4.999206370541162e-05,
+      "loss": 0.7253,
+      "step": 505
+    },
+    {
+      "epoch": 3.2285714285714286,
+      "grad_norm": 1.8182263374328613,
+      "learning_rate": 4.998955804290425e-05,
+      "loss": 0.6824,
+      "step": 510
+    },
+    {
+      "epoch": 3.2603174603174603,
+      "grad_norm": 2.2959372997283936,
+      "learning_rate": 4.9986709240671495e-05,
+      "loss": 0.601,
+      "step": 515
+    },
+    {
+      "epoch": 3.292063492063492,
+      "grad_norm": 2.385838031768799,
+      "learning_rate": 4.998351733783116e-05,
+      "loss": 0.7417,
+      "step": 520
+    },
+    {
+      "epoch": 3.323809523809524,
+      "grad_norm": 2.0416879653930664,
+      "learning_rate": 4.997998237821233e-05,
+      "loss": 0.6463,
+      "step": 525
+    },
+    {
+      "epoch": 3.3555555555555556,
+      "grad_norm": 2.2781031131744385,
+      "learning_rate": 4.9976104410354654e-05,
+      "loss": 0.6998,
+      "step": 530
+    },
+    {
+      "epoch": 3.3873015873015873,
+      "grad_norm": 2.146778106689453,
+      "learning_rate": 4.9971883487507775e-05,
+      "loss": 0.7694,
+      "step": 535
+    },
+    {
+      "epoch": 3.419047619047619,
+      "grad_norm": 2.1369104385375977,
+      "learning_rate": 4.9967319667630567e-05,
+      "loss": 0.6615,
+      "step": 540
+    },
+    {
+      "epoch": 3.450793650793651,
+      "grad_norm": 2.4529733657836914,
+      "learning_rate": 4.996241301339029e-05,
+      "loss": 0.6109,
+      "step": 545
+    },
+    {
+      "epoch": 3.4825396825396826,
+      "grad_norm": 2.07030987739563,
+      "learning_rate": 4.995716359216183e-05,
+      "loss": 0.7611,
+      "step": 550
+    },
+    {
+      "epoch": 3.5142857142857142,
+      "grad_norm": 2.4329919815063477,
+      "learning_rate": 4.995157147602669e-05,
+      "loss": 0.7515,
+      "step": 555
+    },
+    {
+      "epoch": 3.546031746031746,
+      "grad_norm": 2.056351900100708,
+      "learning_rate": 4.994563674177202e-05,
+      "loss": 0.6885,
+      "step": 560
+    },
+    {
+      "epoch": 3.5777777777777775,
+      "grad_norm": 2.3665318489074707,
+      "learning_rate": 4.993935947088958e-05,
+      "loss": 0.6271,
+      "step": 565
+    },
+    {
+      "epoch": 3.6095238095238096,
+      "grad_norm": 2.677706480026245,
+      "learning_rate": 4.993273974957463e-05,
+      "loss": 0.5586,
+      "step": 570
+    },
+    {
+      "epoch": 3.641269841269841,
+      "grad_norm": 3.422136068344116,
+      "learning_rate": 4.9925777668724685e-05,
+      "loss": 0.7552,
+      "step": 575
+    },
+    {
+      "epoch": 3.6730158730158733,
+      "grad_norm": 2.4525184631347656,
+      "learning_rate": 4.991847332393835e-05,
+      "loss": 0.7367,
+      "step": 580
+    },
+    {
+      "epoch": 3.704761904761905,
+      "grad_norm": 2.4242067337036133,
+      "learning_rate": 4.991082681551396e-05,
+      "loss": 0.7044,
+      "step": 585
+    },
+    {
+      "epoch": 3.7365079365079366,
+      "grad_norm": 1.8419867753982544,
+      "learning_rate": 4.9902838248448184e-05,
+      "loss": 0.5966,
+      "step": 590
+    },
+    {
+      "epoch": 3.768253968253968,
+      "grad_norm": 2.1394360065460205,
+      "learning_rate": 4.989450773243463e-05,
+      "loss": 0.6736,
+      "step": 595
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 1.285447597503662,
+      "learning_rate": 4.9885835381862326e-05,
+      "loss": 0.5021,
+      "step": 600
+    },
+    {
+      "epoch": 3.831746031746032,
+      "grad_norm": 2.724978446960449,
+      "learning_rate": 4.987682131581413e-05,
+      "loss": 0.6128,
+      "step": 605
+    },
+    {
+      "epoch": 3.8634920634920635,
+      "grad_norm": 2.239682912826538,
+      "learning_rate": 4.986746565806508e-05,
+      "loss": 0.5457,
+      "step": 610
+    },
+    {
+      "epoch": 3.895238095238095,
+      "grad_norm": 2.48944091796875,
+      "learning_rate": 4.9857768537080784e-05,
+      "loss": 0.6927,
+      "step": 615
+    },
+    {
+      "epoch": 3.9269841269841272,
+      "grad_norm": 2.4086852073669434,
+      "learning_rate": 4.9847730086015534e-05,
+      "loss": 0.5963,
+      "step": 620
+    },
+    {
+      "epoch": 3.958730158730159,
+      "grad_norm": 2.0070106983184814,
+      "learning_rate": 4.9837350442710553e-05,
+      "loss": 0.5856,
+      "step": 625
+    },
+    {
+      "epoch": 3.9904761904761905,
+      "grad_norm": 1.9726545810699463,
+      "learning_rate": 4.98266297496921e-05,
+      "loss": 0.6208,
+      "step": 630
+    },
+    {
+      "epoch": 4.019047619047619,
+      "grad_norm": 2.6137828826904297,
+      "learning_rate": 4.981556815416948e-05,
+      "loss": 0.6319,
+      "step": 635
+    },
+    {
+      "epoch": 4.050793650793651,
+      "grad_norm": 2.3489890098571777,
+      "learning_rate": 4.9804165808033054e-05,
+      "loss": 0.5887,
+      "step": 640
+    },
+    {
+      "epoch": 4.082539682539682,
+      "grad_norm": 2.8010590076446533,
+      "learning_rate": 4.979242286785214e-05,
+      "loss": 0.5257,
+      "step": 645
+    },
+    {
+      "epoch": 4.114285714285714,
+      "grad_norm": 2.993411064147949,
+      "learning_rate": 4.978033949487284e-05,
+      "loss": 0.4545,
+      "step": 650
+    },
+    {
+      "epoch": 4.146031746031746,
+      "grad_norm": 2.669935703277588,
+      "learning_rate": 4.976791585501588e-05,
+      "loss": 0.5989,
+      "step": 655
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 3.084409236907959,
+      "learning_rate": 4.9755152118874294e-05,
+      "loss": 0.528,
+      "step": 660
+    },
+    {
+      "epoch": 4.20952380952381,
+      "grad_norm": 2.797873020172119,
+      "learning_rate": 4.974204846171106e-05,
+      "loss": 0.5249,
+      "step": 665
+    },
+    {
+      "epoch": 4.241269841269841,
+      "grad_norm": 3.667867422103882,
+      "learning_rate": 4.9728605063456765e-05,
+      "loss": 0.5838,
+      "step": 670
+    },
+    {
+      "epoch": 4.273015873015873,
+      "grad_norm": 2.6918869018554688,
+      "learning_rate": 4.971482210870706e-05,
+      "loss": 0.5143,
+      "step": 675
+    },
+    {
+      "epoch": 4.304761904761905,
+      "grad_norm": 2.1545379161834717,
+      "learning_rate": 4.970069978672017e-05,
+      "loss": 0.5317,
+      "step": 680
+    },
+    {
+      "epoch": 4.336507936507936,
+      "grad_norm": 2.1043529510498047,
+      "learning_rate": 4.9686238291414275e-05,
+      "loss": 0.4815,
+      "step": 685
+    },
+    {
+      "epoch": 4.368253968253969,
+      "grad_norm": 2.1359753608703613,
+      "learning_rate": 4.9671437821364855e-05,
+      "loss": 0.4935,
+      "step": 690
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 3.092057228088379,
+      "learning_rate": 4.965629857980197e-05,
+      "loss": 0.6831,
+      "step": 695
+    },
+    {
+      "epoch": 4.431746031746032,
+      "grad_norm": 2.5296835899353027,
+      "learning_rate": 4.964082077460745e-05,
+      "loss": 0.5323,
+      "step": 700
+    },
+    {
+      "epoch": 4.463492063492064,
+      "grad_norm": 1.6655627489089966,
+      "learning_rate": 4.962500461831207e-05,
+      "loss": 0.4553,
+      "step": 705
+    },
+    {
+      "epoch": 4.495238095238095,
+      "grad_norm": 2.6663475036621094,
+      "learning_rate": 4.9608850328092576e-05,
+      "loss": 0.463,
+      "step": 710
+    },
+    {
+      "epoch": 4.526984126984127,
+      "grad_norm": 2.3763060569763184,
+      "learning_rate": 4.959235812576879e-05,
+      "loss": 0.4861,
+      "step": 715
+    },
+    {
+      "epoch": 4.5587301587301585,
+      "grad_norm": 2.2217962741851807,
+      "learning_rate": 4.957552823780047e-05,
+      "loss": 0.468,
+      "step": 720
+    },
+    {
+      "epoch": 4.59047619047619,
+      "grad_norm": 2.8885600566864014,
+      "learning_rate": 4.9558360895284295e-05,
+      "loss": 0.4588,
+      "step": 725
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 2.5661261081695557,
+      "learning_rate": 4.954085633395058e-05,
+      "loss": 0.4926,
+      "step": 730
+    },
+    {
+      "epoch": 4.653968253968254,
+      "grad_norm": 2.304365396499634,
+      "learning_rate": 4.952301479416015e-05,
+      "loss": 0.494,
+      "step": 735
+    },
+    {
+      "epoch": 4.685714285714286,
+      "grad_norm": 2.690577983856201,
+      "learning_rate": 4.9504836520900976e-05,
+      "loss": 0.5814,
+      "step": 740
+    },
+    {
+      "epoch": 4.717460317460318,
+      "grad_norm": 2.7180025577545166,
+      "learning_rate": 4.948632176378481e-05,
+      "loss": 0.5329,
+      "step": 745
+    },
+    {
+      "epoch": 4.749206349206349,
+      "grad_norm": 2.716587543487549,
+      "learning_rate": 4.9467470777043806e-05,
+      "loss": 0.5264,
+      "step": 750
+    },
+    {
+      "epoch": 4.780952380952381,
+      "grad_norm": 2.315419912338257,
+      "learning_rate": 4.9448283819526954e-05,
+      "loss": 0.4756,
+      "step": 755
+    },
+    {
+      "epoch": 4.8126984126984125,
+      "grad_norm": 2.1679515838623047,
+      "learning_rate": 4.9428761154696605e-05,
+      "loss": 0.4819,
+      "step": 760
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 3.389266014099121,
+      "learning_rate": 4.9408903050624796e-05,
+      "loss": 0.5121,
+      "step": 765
+    },
+    {
+      "epoch": 4.876190476190477,
+      "grad_norm": 3.4317383766174316,
+      "learning_rate": 4.938870977998959e-05,
+      "loss": 0.4535,
+      "step": 770
+    },
+    {
+      "epoch": 4.907936507936508,
+      "grad_norm": 2.9491918087005615,
+      "learning_rate": 4.9368181620071344e-05,
+      "loss": 0.5333,
+      "step": 775
+    },
+    {
+      "epoch": 4.93968253968254,
+      "grad_norm": 2.516798496246338,
+      "learning_rate": 4.934731885274887e-05,
+      "loss": 0.5367,
+      "step": 780
+    },
+    {
+      "epoch": 4.9714285714285715,
+      "grad_norm": 3.0031046867370605,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.4957,
+      "step": 785
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.334085702896118,
+      "learning_rate": 4.9304590646375614e-05,
+      "loss": 0.5287,
+      "step": 790
+    },
+    {
+      "epoch": 5.031746031746032,
+      "grad_norm": 1.9608453512191772,
+      "learning_rate": 4.928272579403969e-05,
+      "loss": 0.36,
+      "step": 795
+    },
+    {
+      "epoch": 5.063492063492063,
+      "grad_norm": 2.328850746154785,
+      "learning_rate": 4.92605275077212e-05,
+      "loss": 0.3628,
+      "step": 800
+    },
+    {
+      "epoch": 5.095238095238095,
+      "grad_norm": 2.3446412086486816,
+      "learning_rate": 4.923799609223202e-05,
+      "loss": 0.3327,
+      "step": 805
+    },
+    {
+      "epoch": 5.1269841269841265,
+      "grad_norm": 2.476181745529175,
+      "learning_rate": 4.921513185695831e-05,
+      "loss": 0.4246,
+      "step": 810
+    },
+    {
+      "epoch": 5.158730158730159,
+      "grad_norm": 3.1026763916015625,
+      "learning_rate": 4.91919351158563e-05,
+      "loss": 0.5048,
+      "step": 815
+    },
+    {
+      "epoch": 5.190476190476191,
+      "grad_norm": 2.8165297508239746,
+      "learning_rate": 4.916840618744798e-05,
+      "loss": 0.4361,
+      "step": 820
+    },
+    {
+      "epoch": 5.222222222222222,
+      "grad_norm": 1.8732138872146606,
+      "learning_rate": 4.9144545394816687e-05,
+      "loss": 0.4693,
+      "step": 825
+    },
+    {
+      "epoch": 5.253968253968254,
+      "grad_norm": 1.7250264883041382,
+      "learning_rate": 4.91203530656027e-05,
+      "loss": 0.4076,
+      "step": 830
+    },
+    {
+      "epoch": 5.285714285714286,
+      "grad_norm": 2.105459690093994,
+      "learning_rate": 4.9095829531998725e-05,
+      "loss": 0.3589,
+      "step": 835
+    },
+    {
+      "epoch": 5.317460317460317,
+      "grad_norm": 3.6825687885284424,
+      "learning_rate": 4.9070975130745387e-05,
+      "loss": 0.5263,
+      "step": 840
+    },
+    {
+      "epoch": 5.349206349206349,
+      "grad_norm": 2.947052001953125,
+      "learning_rate": 4.90457902031265e-05,
+      "loss": 0.4632,
+      "step": 845
+    },
+    {
+      "epoch": 5.380952380952381,
+      "grad_norm": 1.9546104669570923,
+      "learning_rate": 4.902027509496448e-05,
+      "loss": 0.4348,
+      "step": 850
+    },
+    {
+      "epoch": 5.412698412698413,
+      "grad_norm": 2.4471983909606934,
+      "learning_rate": 4.899443015661557e-05,
+      "loss": 0.4209,
+      "step": 855
+    },
+    {
+      "epoch": 5.444444444444445,
+      "grad_norm": 1.827124834060669,
+      "learning_rate": 4.8968255742964975e-05,
+      "loss": 0.413,
+      "step": 860
+    },
+    {
+      "epoch": 5.476190476190476,
+      "grad_norm": 2.654707431793213,
+      "learning_rate": 4.894175221342207e-05,
+      "loss": 0.432,
+      "step": 865
+    },
+    {
+      "epoch": 5.507936507936508,
+      "grad_norm": 2.648967981338501,
+      "learning_rate": 4.8914919931915407e-05,
+      "loss": 0.4339,
+      "step": 870
+    },
+    {
+      "epoch": 5.5396825396825395,
+      "grad_norm": 2.874075412750244,
+      "learning_rate": 4.888775926688775e-05,
+      "loss": 0.4392,
+      "step": 875
+    },
+    {
+      "epoch": 5.571428571428571,
+      "grad_norm": 2.9674830436706543,
+      "learning_rate": 4.8860270591291e-05,
+      "loss": 0.4459,
+      "step": 880
+    },
+    {
+      "epoch": 5.603174603174603,
+      "grad_norm": 2.054748296737671,
+      "learning_rate": 4.883245428258107e-05,
+      "loss": 0.4313,
+      "step": 885
+    },
+    {
+      "epoch": 5.634920634920634,
+      "grad_norm": 1.9174392223358154,
+      "learning_rate": 4.880431072271272e-05,
+      "loss": 0.3906,
+      "step": 890
+    },
+    {
+      "epoch": 5.666666666666667,
+      "grad_norm": 2.5257787704467773,
+      "learning_rate": 4.87758402981343e-05,
+      "loss": 0.4219,
+      "step": 895
+    },
+    {
+      "epoch": 5.698412698412699,
+      "grad_norm": 2.6365532875061035,
+      "learning_rate": 4.8747043399782424e-05,
+      "loss": 0.3978,
+      "step": 900
+    },
+    {
+      "epoch": 5.73015873015873,
+      "grad_norm": 2.0583746433258057,
+      "learning_rate": 4.871792042307667e-05,
+      "loss": 0.4847,
+      "step": 905
+    },
+    {
+      "epoch": 5.761904761904762,
+      "grad_norm": 2.035872459411621,
+      "learning_rate": 4.868847176791406e-05,
+      "loss": 0.4675,
+      "step": 910
+    },
+    {
+      "epoch": 5.7936507936507935,
+      "grad_norm": 2.3722939491271973,
+      "learning_rate": 4.8658697838663625e-05,
+      "loss": 0.4586,
+      "step": 915
+    },
+    {
+      "epoch": 5.825396825396825,
+      "grad_norm": 1.2609732151031494,
+      "learning_rate": 4.862859904416085e-05,
+      "loss": 0.3274,
+      "step": 920
+    },
+    {
+      "epoch": 5.857142857142857,
+      "grad_norm": 2.3673977851867676,
+      "learning_rate": 4.8598175797702036e-05,
+      "loss": 0.4685,
+      "step": 925
+    },
+    {
+      "epoch": 5.888888888888889,
+      "grad_norm": 2.8414175510406494,
+      "learning_rate": 4.856742851703866e-05,
+      "loss": 0.4762,
+      "step": 930
+    },
+    {
+      "epoch": 5.920634920634921,
+      "grad_norm": 2.4126765727996826,
+      "learning_rate": 4.853635762437159e-05,
+      "loss": 0.4075,
+      "step": 935
+    },
+    {
+      "epoch": 5.9523809523809526,
+      "grad_norm": 1.8691045045852661,
+      "learning_rate": 4.8504963546345334e-05,
+      "loss": 0.4865,
+      "step": 940
+    },
+    {
+      "epoch": 5.984126984126984,
+      "grad_norm": 3.5297420024871826,
+      "learning_rate": 4.8473246714042155e-05,
+      "loss": 0.4623,
+      "step": 945
+    },
+    {
+      "epoch": 6.012698412698413,
+      "grad_norm": 2.059169054031372,
+      "learning_rate": 4.844120756297617e-05,
+      "loss": 0.4164,
+      "step": 950
+    },
+    {
+      "epoch": 6.044444444444444,
+      "grad_norm": 2.4746127128601074,
+      "learning_rate": 4.840884653308735e-05,
+      "loss": 0.3552,
+      "step": 955
+    },
+    {
+      "epoch": 6.076190476190476,
+      "grad_norm": 2.504425287246704,
+      "learning_rate": 4.8376164068735485e-05,
+      "loss": 0.3368,
+      "step": 960
+    },
+    {
+      "epoch": 6.1079365079365076,
+      "grad_norm": 2.062577486038208,
+      "learning_rate": 4.83431606186941e-05,
+      "loss": 0.3139,
+      "step": 965
+    },
+    {
+      "epoch": 6.13968253968254,
+      "grad_norm": 2.4934544563293457,
+      "learning_rate": 4.830983663614427e-05,
+      "loss": 0.3777,
+      "step": 970
+    },
+    {
+      "epoch": 6.171428571428572,
+      "grad_norm": 2.5747485160827637,
+      "learning_rate": 4.827619257866839e-05,
+      "loss": 0.373,
+      "step": 975
+    },
+    {
+      "epoch": 6.203174603174603,
+      "grad_norm": 2.449357271194458,
+      "learning_rate": 4.8242228908243946e-05,
+      "loss": 0.3936,
+      "step": 980
+    },
+    {
+      "epoch": 6.234920634920635,
+      "grad_norm": 2.952680826187134,
+      "learning_rate": 4.82079460912371e-05,
+      "loss": 0.407,
+      "step": 985
+    },
+    {
+      "epoch": 6.266666666666667,
+      "grad_norm": 2.1754496097564697,
+      "learning_rate": 4.817334459839633e-05,
+      "loss": 0.3189,
+      "step": 990
+    },
+    {
+      "epoch": 6.298412698412698,
+      "grad_norm": 2.8406214714050293,
+      "learning_rate": 4.8138424904845947e-05,
+      "loss": 0.3883,
+      "step": 995
+    },
+    {
+      "epoch": 6.33015873015873,
+      "grad_norm": 1.7533257007598877,
+      "learning_rate": 4.8103187490079604e-05,
+      "loss": 0.3131,
+      "step": 1000
+    },
+    {
+      "epoch": 6.3619047619047615,
+      "grad_norm": 2.4574601650238037,
+      "learning_rate": 4.806763283795366e-05,
+      "loss": 0.3606,
+      "step": 1005
+    },
+    {
+      "epoch": 6.393650793650794,
+      "grad_norm": 2.002281427383423,
+      "learning_rate": 4.8031761436680575e-05,
+      "loss": 0.37,
+      "step": 1010
+    },
+    {
+      "epoch": 6.425396825396826,
+      "grad_norm": 2.823315143585205,
+      "learning_rate": 4.79955737788222e-05,
+      "loss": 0.3791,
+      "step": 1015
+    },
+    {
+      "epoch": 6.457142857142857,
+      "grad_norm": 2.7891204357147217,
+      "learning_rate": 4.795907036128299e-05,
+      "loss": 0.3556,
+      "step": 1020
+    },
+    {
+      "epoch": 6.488888888888889,
+      "grad_norm": 2.2387146949768066,
+      "learning_rate": 4.7922251685303213e-05,
+      "loss": 0.3929,
+      "step": 1025
+    },
+    {
+      "epoch": 6.520634920634921,
+      "grad_norm": 2.5023891925811768,
+      "learning_rate": 4.788511825645205e-05,
+      "loss": 0.379,
+      "step": 1030
+    },
+    {
+      "epoch": 6.552380952380952,
+      "grad_norm": 2.2654805183410645,
+      "learning_rate": 4.7847670584620653e-05,
+      "loss": 0.3435,
+      "step": 1035
+    },
+    {
+      "epoch": 6.584126984126984,
+      "grad_norm": 3.3823065757751465,
+      "learning_rate": 4.7809909184015146e-05,
+      "loss": 0.4109,
+      "step": 1040
+    },
+    {
+      "epoch": 6.6158730158730155,
+      "grad_norm": 2.6096551418304443,
+      "learning_rate": 4.7771834573149576e-05,
+      "loss": 0.4233,
+      "step": 1045
+    },
+    {
+      "epoch": 6.647619047619048,
+      "grad_norm": 2.3933897018432617,
+      "learning_rate": 4.773344727483876e-05,
+      "loss": 0.3709,
+      "step": 1050
+    },
+    {
+      "epoch": 6.67936507936508,
+      "grad_norm": 2.189544916152954,
+      "learning_rate": 4.769474781619114e-05,
+      "loss": 0.3287,
+      "step": 1055
+    },
+    {
+      "epoch": 6.711111111111111,
+      "grad_norm": 2.450892686843872,
+      "learning_rate": 4.765573672860154e-05,
+      "loss": 0.4022,
+      "step": 1060
+    },
+    {
+      "epoch": 6.742857142857143,
+      "grad_norm": 2.4342429637908936,
+      "learning_rate": 4.761641454774386e-05,
+      "loss": 0.4029,
+      "step": 1065
+    },
+    {
+      "epoch": 6.7746031746031745,
+      "grad_norm": 2.2122364044189453,
+      "learning_rate": 4.75767818135637e-05,
+      "loss": 0.3322,
+      "step": 1070
+    },
+    {
+      "epoch": 6.806349206349206,
+      "grad_norm": 3.968445301055908,
+      "learning_rate": 4.7536839070271e-05,
+      "loss": 0.3836,
+      "step": 1075
+    },
+    {
+      "epoch": 6.838095238095238,
+      "grad_norm": 3.529158353805542,
+      "learning_rate": 4.749658686633251e-05,
+      "loss": 0.4745,
+      "step": 1080
+    },
+    {
+      "epoch": 6.86984126984127,
+      "grad_norm": 2.430727243423462,
+      "learning_rate": 4.7456025754464304e-05,
+      "loss": 0.3664,
+      "step": 1085
+    },
+    {
+      "epoch": 6.901587301587302,
+      "grad_norm": 2.6552302837371826,
+      "learning_rate": 4.7415156291624166e-05,
+      "loss": 0.4359,
+      "step": 1090
+    },
+    {
+      "epoch": 6.933333333333334,
+      "grad_norm": 2.134822130203247,
+      "learning_rate": 4.737397903900393e-05,
+      "loss": 0.3969,
+      "step": 1095
+    },
+    {
+      "epoch": 6.965079365079365,
+      "grad_norm": 2.5052947998046875,
+      "learning_rate": 4.7332494562021815e-05,
+      "loss": 0.4069,
+      "step": 1100
+    },
+    {
+      "epoch": 6.996825396825397,
+      "grad_norm": 2.1377065181732178,
+      "learning_rate": 4.729070343031463e-05,
+      "loss": 0.3853,
+      "step": 1105
+    },
+    {
+      "epoch": 7.025396825396825,
+      "grad_norm": 1.9704042673110962,
+      "learning_rate": 4.724860621772995e-05,
+      "loss": 0.3283,
+      "step": 1110
+    },
+    {
+      "epoch": 7.057142857142857,
+      "grad_norm": 2.476968765258789,
+      "learning_rate": 4.7206203502318256e-05,
+      "loss": 0.3325,
+      "step": 1115
+    },
+    {
+      "epoch": 7.088888888888889,
+      "grad_norm": 1.9231969118118286,
+      "learning_rate": 4.716349586632499e-05,
+      "loss": 0.2876,
+      "step": 1120
+    },
+    {
+      "epoch": 7.12063492063492,
+      "grad_norm": 2.6444814205169678,
+      "learning_rate": 4.712048389618254e-05,
+      "loss": 0.3005,
+      "step": 1125
+    },
+    {
+      "epoch": 7.152380952380953,
+      "grad_norm": 3.2589964866638184,
+      "learning_rate": 4.7077168182502216e-05,
+      "loss": 0.4023,
+      "step": 1130
+    },
+    {
+      "epoch": 7.184126984126984,
+      "grad_norm": 2.5481936931610107,
+      "learning_rate": 4.703354932006615e-05,
+      "loss": 0.3302,
+      "step": 1135
+    },
+    {
+      "epoch": 7.215873015873016,
+      "grad_norm": 1.7125908136367798,
+      "learning_rate": 4.698962790781906e-05,
+      "loss": 0.3329,
+      "step": 1140
+    },
+    {
+      "epoch": 7.247619047619048,
+      "grad_norm": 2.2756667137145996,
+      "learning_rate": 4.6945404548860115e-05,
+      "loss": 0.3369,
+      "step": 1145
+    },
+    {
+      "epoch": 7.279365079365079,
+      "grad_norm": 2.9158453941345215,
+      "learning_rate": 4.6900879850434604e-05,
+      "loss": 0.3339,
+      "step": 1150
+    },
+    {
+      "epoch": 7.311111111111111,
+      "grad_norm": 2.3047537803649902,
+      "learning_rate": 4.685605442392559e-05,
+      "loss": 0.3915,
+      "step": 1155
+    },
+    {
+      "epoch": 7.3428571428571425,
+      "grad_norm": 2.7815029621124268,
+      "learning_rate": 4.681092888484554e-05,
+      "loss": 0.3317,
+      "step": 1160
+    },
+    {
+      "epoch": 7.374603174603175,
+      "grad_norm": 2.2644097805023193,
+      "learning_rate": 4.676550385282787e-05,
+      "loss": 0.3314,
+      "step": 1165
+    },
+    {
+      "epoch": 7.406349206349207,
+      "grad_norm": 2.5144474506378174,
+      "learning_rate": 4.671977995161843e-05,
+      "loss": 0.3188,
+      "step": 1170
+    },
+    {
+      "epoch": 7.438095238095238,
+      "grad_norm": 3.120821714401245,
+      "learning_rate": 4.667375780906693e-05,
+      "loss": 0.3523,
+      "step": 1175
+    },
+    {
+      "epoch": 7.46984126984127,
+      "grad_norm": 4.47842264175415,
+      "learning_rate": 4.662743805711832e-05,
+      "loss": 0.3611,
+      "step": 1180
+    },
+    {
+      "epoch": 7.501587301587302,
+      "grad_norm": 1.9228928089141846,
+      "learning_rate": 4.658082133180416e-05,
+      "loss": 0.3612,
+      "step": 1185
+    },
+    {
+      "epoch": 7.533333333333333,
+      "grad_norm": 2.1507537364959717,
+      "learning_rate": 4.6533908273233815e-05,
+      "loss": 0.3321,
+      "step": 1190
+    },
+    {
+      "epoch": 7.565079365079365,
+      "grad_norm": 2.1849119663238525,
+      "learning_rate": 4.64866995255857e-05,
+      "loss": 0.2943,
+      "step": 1195
+    },
+    {
+      "epoch": 7.5968253968253965,
+      "grad_norm": 2.1777775287628174,
+      "learning_rate": 4.643919573709843e-05,
+      "loss": 0.353,
+      "step": 1200
+    },
+    {
+      "epoch": 7.628571428571428,
+      "grad_norm": 2.5231118202209473,
+      "learning_rate": 4.639139756006195e-05,
+      "loss": 0.3571,
+      "step": 1205
+    },
+    {
+      "epoch": 7.660317460317461,
+      "grad_norm": 1.8409479856491089,
+      "learning_rate": 4.6343305650808516e-05,
+      "loss": 0.3691,
+      "step": 1210
+    },
+    {
+      "epoch": 7.692063492063492,
+      "grad_norm": 1.7940895557403564,
+      "learning_rate": 4.629492066970373e-05,
+      "loss": 0.3738,
+      "step": 1215
+    },
+    {
+      "epoch": 7.723809523809524,
+      "grad_norm": 2.014902114868164,
+      "learning_rate": 4.6246243281137474e-05,
+      "loss": 0.361,
+      "step": 1220
+    },
+    {
+      "epoch": 7.7555555555555555,
+      "grad_norm": 3.4182560443878174,
+      "learning_rate": 4.6197274153514735e-05,
+      "loss": 0.3663,
+      "step": 1225
+    },
+    {
+      "epoch": 7.787301587301587,
+      "grad_norm": 2.518728256225586,
+      "learning_rate": 4.614801395924649e-05,
+      "loss": 0.3646,
+      "step": 1230
+    },
+    {
+      "epoch": 7.819047619047619,
+      "grad_norm": 2.154189109802246,
+      "learning_rate": 4.6098463374740466e-05,
+      "loss": 0.3331,
+      "step": 1235
+    },
+    {
+      "epoch": 7.85079365079365,
+      "grad_norm": 2.536081075668335,
+      "learning_rate": 4.604862308039177e-05,
+      "loss": 0.3742,
+      "step": 1240
+    },
+    {
+      "epoch": 7.882539682539683,
+      "grad_norm": 2.340764045715332,
+      "learning_rate": 4.599849376057366e-05,
+      "loss": 0.3352,
+      "step": 1245
+    },
+    {
+      "epoch": 7.914285714285715,
+      "grad_norm": 3.5488364696502686,
+      "learning_rate": 4.5948076103628094e-05,
+      "loss": 0.3663,
+      "step": 1250
+    },
+    {
+      "epoch": 7.946031746031746,
+      "grad_norm": 2.779360294342041,
+      "learning_rate": 4.589737080185625e-05,
+      "loss": 0.3362,
+      "step": 1255
+    },
+    {
+      "epoch": 7.977777777777778,
+      "grad_norm": 1.8792667388916016,
+      "learning_rate": 4.5846378551509097e-05,
+      "loss": 0.346,
+      "step": 1260
+    },
+    {
+      "epoch": 8.006349206349206,
+      "grad_norm": 2.453295946121216,
+      "learning_rate": 4.579510005277774e-05,
+      "loss": 0.3509,
+      "step": 1265
+    },
+    {
+      "epoch": 8.038095238095238,
+      "grad_norm": 1.9493130445480347,
+      "learning_rate": 4.574353600978388e-05,
+      "loss": 0.3062,
+      "step": 1270
+    },
+    {
+      "epoch": 8.06984126984127,
+      "grad_norm": 1.9360930919647217,
+      "learning_rate": 4.56916871305701e-05,
+      "loss": 0.3056,
+      "step": 1275
+    },
+    {
+      "epoch": 8.101587301587301,
+      "grad_norm": 1.5592070817947388,
+      "learning_rate": 4.563955412709021e-05,
+      "loss": 0.2785,
+      "step": 1280
+    },
+    {
+      "epoch": 8.133333333333333,
+      "grad_norm": 1.8093425035476685,
+      "learning_rate": 4.5587137715199354e-05,
+      "loss": 0.308,
+      "step": 1285
+    },
+    {
+      "epoch": 8.165079365079364,
+      "grad_norm": 2.2939181327819824,
+      "learning_rate": 4.5534438614644294e-05,
+      "loss": 0.3038,
+      "step": 1290
+    },
+    {
+      "epoch": 8.196825396825396,
+      "grad_norm": 2.4204866886138916,
+      "learning_rate": 4.548145754905346e-05,
+      "loss": 0.3375,
+      "step": 1295
+    },
+    {
+      "epoch": 8.228571428571428,
+      "grad_norm": 1.725534439086914,
+      "learning_rate": 4.5428195245927064e-05,
+      "loss": 0.3101,
+      "step": 1300
+    },
+    {
+      "epoch": 8.260317460317461,
+      "grad_norm": 1.637730360031128,
+      "learning_rate": 4.537465243662704e-05,
+      "loss": 0.2931,
+      "step": 1305
+    },
+    {
+      "epoch": 8.292063492063493,
+      "grad_norm": 1.3372169733047485,
+      "learning_rate": 4.532082985636709e-05,
+      "loss": 0.2763,
+      "step": 1310
+    },
+    {
+      "epoch": 8.323809523809524,
+      "grad_norm": 2.5993168354034424,
+      "learning_rate": 4.5266728244202494e-05,
+      "loss": 0.3458,
+      "step": 1315
+    },
+    {
+      "epoch": 8.355555555555556,
+      "grad_norm": 2.461862564086914,
+      "learning_rate": 4.521234834302006e-05,
+      "loss": 0.3693,
+      "step": 1320
+    },
+    {
+      "epoch": 8.387301587301588,
+      "grad_norm": 1.8519413471221924,
+      "learning_rate": 4.5157690899527816e-05,
+      "loss": 0.3327,
+      "step": 1325
+    },
+    {
+      "epoch": 8.41904761904762,
+      "grad_norm": 2.1535580158233643,
+      "learning_rate": 4.510275666424487e-05,
+      "loss": 0.3229,
+      "step": 1330
+    },
+    {
+      "epoch": 8.450793650793651,
+      "grad_norm": 1.6819690465927124,
+      "learning_rate": 4.5047546391491e-05,
+      "loss": 0.2925,
+      "step": 1335
+    },
+    {
+      "epoch": 8.482539682539683,
+      "grad_norm": 1.6538281440734863,
+      "learning_rate": 4.499206083937638e-05,
+      "loss": 0.3218,
+      "step": 1340
+    },
+    {
+      "epoch": 8.514285714285714,
+      "grad_norm": 1.8956862688064575,
+      "learning_rate": 4.493630076979112e-05,
+      "loss": 0.3423,
+      "step": 1345
+    },
+    {
+      "epoch": 8.546031746031746,
+      "grad_norm": 2.274681806564331,
+      "learning_rate": 4.48802669483948e-05,
+      "loss": 0.3152,
+      "step": 1350
+    },
+    {
+      "epoch": 8.577777777777778,
+      "grad_norm": 2.2956337928771973,
+      "learning_rate": 4.4823960144606014e-05,
+      "loss": 0.3417,
+      "step": 1355
+    },
+    {
+      "epoch": 8.60952380952381,
+      "grad_norm": 1.8650286197662354,
+      "learning_rate": 4.4767381131591734e-05,
+      "loss": 0.2896,
+      "step": 1360
+    },
+    {
+      "epoch": 8.64126984126984,
+      "grad_norm": 1.3998652696609497,
+      "learning_rate": 4.471053068625674e-05,
+      "loss": 0.3372,
+      "step": 1365
+    },
+    {
+      "epoch": 8.673015873015872,
+      "grad_norm": 2.855074167251587,
+      "learning_rate": 4.465340958923293e-05,
+      "loss": 0.332,
+      "step": 1370
+    },
+    {
+      "epoch": 8.704761904761904,
+      "grad_norm": 1.6865357160568237,
+      "learning_rate": 4.459601862486862e-05,
+      "loss": 0.3053,
+      "step": 1375
+    },
+    {
+      "epoch": 8.736507936507937,
+      "grad_norm": 2.501856803894043,
+      "learning_rate": 4.453835858121773e-05,
+      "loss": 0.3119,
+      "step": 1380
+    },
+    {
+      "epoch": 8.768253968253969,
+      "grad_norm": 2.4325456619262695,
+      "learning_rate": 4.4480430250029046e-05,
+      "loss": 0.3395,
+      "step": 1385
+    },
+    {
+      "epoch": 8.8,
+      "grad_norm": 1.4845948219299316,
+      "learning_rate": 4.4422234426735256e-05,
+      "loss": 0.3237,
+      "step": 1390
+    },
+    {
+      "epoch": 8.831746031746032,
+      "grad_norm": 1.3553249835968018,
+      "learning_rate": 4.436377191044208e-05,
+      "loss": 0.3387,
+      "step": 1395
+    },
+    {
+      "epoch": 8.863492063492064,
+      "grad_norm": 1.8338890075683594,
+      "learning_rate": 4.430504350391729e-05,
+      "loss": 0.3618,
+      "step": 1400
+    },
+    {
+      "epoch": 8.895238095238096,
+      "grad_norm": 2.291538953781128,
+      "learning_rate": 4.4246050013579686e-05,
+      "loss": 0.3608,
+      "step": 1405
+    },
+    {
+      "epoch": 8.926984126984127,
+      "grad_norm": 1.3809788227081299,
+      "learning_rate": 4.4186792249488005e-05,
+      "loss": 0.3077,
+      "step": 1410
+    },
+    {
+      "epoch": 8.958730158730159,
+      "grad_norm": 1.5944230556488037,
+      "learning_rate": 4.412727102532983e-05,
+      "loss": 0.3307,
+      "step": 1415
+    },
+    {
+      "epoch": 8.99047619047619,
+      "grad_norm": 2.2244362831115723,
+      "learning_rate": 4.4067487158410396e-05,
+      "loss": 0.3469,
+      "step": 1420
+    },
+    {
+      "epoch": 9.019047619047619,
+      "grad_norm": 1.444221019744873,
+      "learning_rate": 4.400744146964136e-05,
+      "loss": 0.3049,
+      "step": 1425
+    },
+    {
+      "epoch": 9.05079365079365,
+      "grad_norm": 1.5847752094268799,
+      "learning_rate": 4.394713478352955e-05,
+      "loss": 0.2715,
+      "step": 1430
+    },
+    {
+      "epoch": 9.082539682539682,
+      "grad_norm": 1.6062681674957275,
+      "learning_rate": 4.388656792816562e-05,
+      "loss": 0.2487,
+      "step": 1435
+    },
+    {
+      "epoch": 9.114285714285714,
+      "grad_norm": 2.099787712097168,
+      "learning_rate": 4.382574173521272e-05,
+      "loss": 0.2866,
+      "step": 1440
+    },
+    {
+      "epoch": 9.146031746031746,
+      "grad_norm": 1.0997334718704224,
+      "learning_rate": 4.376465703989502e-05,
+      "loss": 0.3052,
+      "step": 1445
+    },
+    {
+      "epoch": 9.177777777777777,
+      "grad_norm": 2.4327454566955566,
+      "learning_rate": 4.370331468098628e-05,
+      "loss": 0.3212,
+      "step": 1450
+    },
+    {
+      "epoch": 9.209523809523809,
+      "grad_norm": 1.4816385507583618,
+      "learning_rate": 4.364171550079833e-05,
+      "loss": 0.3046,
+      "step": 1455
+    },
+    {
+      "epoch": 9.24126984126984,
+      "grad_norm": 2.039186716079712,
+      "learning_rate": 4.357986034516947e-05,
+      "loss": 0.3165,
+      "step": 1460
+    },
+    {
+      "epoch": 9.273015873015874,
+      "grad_norm": 1.437852382659912,
+      "learning_rate": 4.3517750063452934e-05,
+      "loss": 0.3037,
+      "step": 1465
+    },
+    {
+      "epoch": 9.304761904761905,
+      "grad_norm": 1.818982720375061,
+      "learning_rate": 4.345538550850512e-05,
+      "loss": 0.3122,
+      "step": 1470
+    },
+    {
+      "epoch": 9.336507936507937,
+      "grad_norm": 1.12025785446167,
+      "learning_rate": 4.339276753667395e-05,
+      "loss": 0.2909,
+      "step": 1475
+    },
+    {
+      "epoch": 9.368253968253969,
+      "grad_norm": 1.6094844341278076,
+      "learning_rate": 4.3329897007787125e-05,
+      "loss": 0.2823,
+      "step": 1480
+    },
+    {
+      "epoch": 9.4,
+      "grad_norm": 1.916200041770935,
+      "learning_rate": 4.326677478514024e-05,
+      "loss": 0.2939,
+      "step": 1485
+    },
+    {
+      "epoch": 9.431746031746032,
+      "grad_norm": 1.97919499874115,
+      "learning_rate": 4.320340173548503e-05,
+      "loss": 0.2826,
+      "step": 1490
+    },
+    {
+      "epoch": 9.463492063492064,
+      "grad_norm": 2.0238938331604004,
+      "learning_rate": 4.313977872901737e-05,
+      "loss": 0.3273,
+      "step": 1495
+    },
+    {
+      "epoch": 9.495238095238095,
+      "grad_norm": 2.5840957164764404,
+      "learning_rate": 4.307590663936541e-05,
+      "loss": 0.2889,
+      "step": 1500
+    },
+    {
+      "epoch": 9.526984126984127,
+      "grad_norm": 2.3503904342651367,
+      "learning_rate": 4.30117863435775e-05,
+      "loss": 0.3012,
+      "step": 1505
+    },
+    {
+      "epoch": 9.558730158730159,
+      "grad_norm": 2.019792318344116,
+      "learning_rate": 4.294741872211024e-05,
+      "loss": 0.3267,
+      "step": 1510
+    },
+    {
+      "epoch": 9.59047619047619,
+      "grad_norm": 2.2713353633880615,
+      "learning_rate": 4.288280465881632e-05,
+      "loss": 0.3096,
+      "step": 1515
+    },
+    {
+      "epoch": 9.622222222222222,
+      "grad_norm": 2.4236693382263184,
+      "learning_rate": 4.281794504093237e-05,
+      "loss": 0.3291,
+      "step": 1520
+    },
+    {
+      "epoch": 9.653968253968253,
+      "grad_norm": 1.772703766822815,
+      "learning_rate": 4.275284075906686e-05,
+      "loss": 0.3117,
+      "step": 1525
+    },
+    {
+      "epoch": 9.685714285714285,
+      "grad_norm": 1.9665186405181885,
+      "learning_rate": 4.268749270718778e-05,
+      "loss": 0.326,
+      "step": 1530
+    },
+    {
+      "epoch": 9.717460317460317,
+      "grad_norm": 1.9472782611846924,
+      "learning_rate": 4.262190178261044e-05,
+      "loss": 0.2683,
+      "step": 1535
+    },
+    {
+      "epoch": 9.74920634920635,
+      "grad_norm": 2.0638089179992676,
+      "learning_rate": 4.255606888598508e-05,
+      "loss": 0.314,
+      "step": 1540
+    },
+    {
+      "epoch": 9.780952380952382,
+      "grad_norm": 2.1349925994873047,
+      "learning_rate": 4.248999492128456e-05,
+      "loss": 0.2897,
+      "step": 1545
+    },
+    {
+      "epoch": 9.812698412698413,
+      "grad_norm": 2.112536907196045,
+      "learning_rate": 4.242368079579192e-05,
+      "loss": 0.31,
+      "step": 1550
+    },
+    {
+      "epoch": 9.844444444444445,
+      "grad_norm": 1.6859878301620483,
+      "learning_rate": 4.2357127420087917e-05,
+      "loss": 0.3412,
+      "step": 1555
+    },
+    {
+      "epoch": 9.876190476190477,
+      "grad_norm": 1.9178651571273804,
+      "learning_rate": 4.229033570803853e-05,
+      "loss": 0.334,
+      "step": 1560
+    },
+    {
+      "epoch": 9.907936507936508,
+      "grad_norm": 2.562436103820801,
+      "learning_rate": 4.2223306576782426e-05,
+      "loss": 0.3379,
+      "step": 1565
+    },
+    {
+      "epoch": 9.93968253968254,
+      "grad_norm": 1.8472412824630737,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.3415,
+      "step": 1570
+    },
+    {
+      "epoch": 9.971428571428572,
+      "grad_norm": 1.9416279792785645,
+      "learning_rate": 4.208853974149246e-05,
+      "loss": 0.3085,
+      "step": 1575
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 2.0056397914886475,
+      "learning_rate": 4.202080388798571e-05,
+      "loss": 0.3263,
+      "step": 1580
+    },
+    {
+      "epoch": 10.031746031746032,
+      "grad_norm": 2.195781946182251,
+      "learning_rate": 4.1952834316301065e-05,
+      "loss": 0.2867,
+      "step": 1585
+    },
+    {
+      "epoch": 10.063492063492063,
+      "grad_norm": 1.7489805221557617,
+      "learning_rate": 4.1884631959750766e-05,
+      "loss": 0.2589,
+      "step": 1590
+    },
+    {
+      "epoch": 10.095238095238095,
+      "grad_norm": 1.9361369609832764,
+      "learning_rate": 4.181619775484348e-05,
+      "loss": 0.2722,
+      "step": 1595
+    },
+    {
+      "epoch": 10.126984126984127,
+      "grad_norm": 2.24322509765625,
+      "learning_rate": 4.174753264127147e-05,
+      "loss": 0.2534,
+      "step": 1600
+    },
+    {
+      "epoch": 10.158730158730158,
+      "grad_norm": 2.4550466537475586,
+      "learning_rate": 4.167863756189767e-05,
+      "loss": 0.2777,
+      "step": 1605
+    },
+    {
+      "epoch": 10.19047619047619,
+      "grad_norm": 1.9439811706542969,
+      "learning_rate": 4.160951346274278e-05,
+      "loss": 0.2864,
+      "step": 1610
+    },
+    {
+      "epoch": 10.222222222222221,
+      "grad_norm": 1.633494257926941,
+      "learning_rate": 4.154016129297219e-05,
+      "loss": 0.2858,
+      "step": 1615
+    },
+    {
+      "epoch": 10.253968253968253,
+      "grad_norm": 1.69782292842865,
+      "learning_rate": 4.147058200488305e-05,
+      "loss": 0.2942,
+      "step": 1620
+    },
+    {
+      "epoch": 10.285714285714286,
+      "grad_norm": 1.613031268119812,
+      "learning_rate": 4.140077655389113e-05,
+      "loss": 0.2632,
+      "step": 1625
+    },
+    {
+      "epoch": 10.317460317460318,
+      "grad_norm": 2.0266177654266357,
+      "learning_rate": 4.1330745898517714e-05,
+      "loss": 0.3011,
+      "step": 1630
+    },
+    {
+      "epoch": 10.34920634920635,
+      "grad_norm": 1.8945387601852417,
+      "learning_rate": 4.1260491000376446e-05,
+      "loss": 0.2832,
+      "step": 1635
+    },
+    {
+      "epoch": 10.380952380952381,
+      "grad_norm": 1.7012510299682617,
+      "learning_rate": 4.119001282416009e-05,
+      "loss": 0.2718,
+      "step": 1640
+    },
+    {
+      "epoch": 10.412698412698413,
+      "grad_norm": 1.5538525581359863,
+      "learning_rate": 4.111931233762738e-05,
+      "loss": 0.3232,
+      "step": 1645
+    },
+    {
+      "epoch": 10.444444444444445,
+      "grad_norm": 2.3083150386810303,
+      "learning_rate": 4.1048390511589595e-05,
+      "loss": 0.3057,
+      "step": 1650
+    },
+    {
+      "epoch": 10.476190476190476,
+      "grad_norm": 1.293314814567566,
+      "learning_rate": 4.097724831989733e-05,
+      "loss": 0.2523,
+      "step": 1655
+    },
+    {
+      "epoch": 10.507936507936508,
+      "grad_norm": 2.517212152481079,
+      "learning_rate": 4.09058867394271e-05,
+      "loss": 0.3269,
+      "step": 1660
+    },
+    {
+      "epoch": 10.53968253968254,
+      "grad_norm": 2.057063102722168,
+      "learning_rate": 4.083430675006791e-05,
+      "loss": 0.2844,
+      "step": 1665
+    },
+    {
+      "epoch": 10.571428571428571,
+      "grad_norm": 1.5663833618164062,
+      "learning_rate": 4.0762509334707786e-05,
+      "loss": 0.3005,
+      "step": 1670
+    },
+    {
+      "epoch": 10.603174603174603,
+      "grad_norm": 2.5423505306243896,
+      "learning_rate": 4.069049547922035e-05,
+      "loss": 0.2802,
+      "step": 1675
+    },
+    {
+      "epoch": 10.634920634920634,
+      "grad_norm": 1.578316569328308,
+      "learning_rate": 4.061826617245119e-05,
+      "loss": 0.2667,
+      "step": 1680
+    },
+    {
+      "epoch": 10.666666666666666,
+      "grad_norm": 1.502928376197815,
+      "learning_rate": 4.0545822406204334e-05,
+      "loss": 0.3059,
+      "step": 1685
+    },
+    {
+      "epoch": 10.698412698412698,
+      "grad_norm": 1.2470905780792236,
+      "learning_rate": 4.047316517522864e-05,
+      "loss": 0.2879,
+      "step": 1690
+    },
+    {
+      "epoch": 10.73015873015873,
+      "grad_norm": 1.8238775730133057,
+      "learning_rate": 4.0400295477204105e-05,
+      "loss": 0.2923,
+      "step": 1695
+    },
+    {
+      "epoch": 10.761904761904763,
+      "grad_norm": 2.0516586303710938,
+      "learning_rate": 4.032721431272819e-05,
+      "loss": 0.3086,
+      "step": 1700
+    },
+    {
+      "epoch": 10.793650793650794,
+      "grad_norm": 1.3188791275024414,
+      "learning_rate": 4.0253922685302046e-05,
+      "loss": 0.2893,
+      "step": 1705
+    },
+    {
+      "epoch": 10.825396825396826,
+      "grad_norm": 1.7352266311645508,
+      "learning_rate": 4.01804216013168e-05,
+      "loss": 0.2981,
+      "step": 1710
+    },
+    {
+      "epoch": 10.857142857142858,
+      "grad_norm": 1.3449515104293823,
+      "learning_rate": 4.0106712070039656e-05,
+      "loss": 0.2841,
+      "step": 1715
+    },
+    {
+      "epoch": 10.88888888888889,
+      "grad_norm": 2.505431890487671,
+      "learning_rate": 4.00327951036001e-05,
+      "loss": 0.3034,
+      "step": 1720
+    },
+    {
+      "epoch": 10.920634920634921,
+      "grad_norm": 1.8870325088500977,
+      "learning_rate": 3.9958671716975966e-05,
+      "loss": 0.305,
+      "step": 1725
+    },
+    {
+      "epoch": 10.952380952380953,
+      "grad_norm": 2.913130044937134,
+      "learning_rate": 3.988434292797951e-05,
+      "loss": 0.3212,
+      "step": 1730
+    },
+    {
+      "epoch": 10.984126984126984,
+      "grad_norm": 1.7870115041732788,
+      "learning_rate": 3.980980975724344e-05,
+      "loss": 0.3108,
+      "step": 1735
+    },
+    {
+      "epoch": 11.012698412698413,
+      "grad_norm": 3.050985336303711,
+      "learning_rate": 3.9735073228206896e-05,
+      "loss": 0.3043,
+      "step": 1740
+    },
+    {
+      "epoch": 11.044444444444444,
+      "grad_norm": 1.5993611812591553,
+      "learning_rate": 3.96601343671014e-05,
+      "loss": 0.2465,
+      "step": 1745
+    },
+    {
+      "epoch": 11.076190476190476,
+      "grad_norm": 1.626888632774353,
+      "learning_rate": 3.9584994202936746e-05,
+      "loss": 0.2688,
+      "step": 1750
+    },
+    {
+      "epoch": 11.107936507936508,
+      "grad_norm": 1.7132880687713623,
+      "learning_rate": 3.950965376748689e-05,
+      "loss": 0.2458,
+      "step": 1755
+    },
+    {
+      "epoch": 11.13968253968254,
+      "grad_norm": 1.7764930725097656,
+      "learning_rate": 3.94341140952758e-05,
+      "loss": 0.2189,
+      "step": 1760
+    },
+    {
+      "epoch": 11.17142857142857,
+      "grad_norm": 2.5560712814331055,
+      "learning_rate": 3.9358376223563206e-05,
+      "loss": 0.2866,
+      "step": 1765
+    },
+    {
+      "epoch": 11.203174603174602,
+      "grad_norm": 1.1177359819412231,
+      "learning_rate": 3.928244119233038e-05,
+      "loss": 0.233,
+      "step": 1770
+    },
+    {
+      "epoch": 11.234920634920634,
+      "grad_norm": 1.584670901298523,
+      "learning_rate": 3.9206310044265866e-05,
+      "loss": 0.273,
+      "step": 1775
+    },
+    {
+      "epoch": 11.266666666666667,
+      "grad_norm": 1.6278687715530396,
+      "learning_rate": 3.912998382475115e-05,
+      "loss": 0.2746,
+      "step": 1780
+    },
+    {
+      "epoch": 11.2984126984127,
+      "grad_norm": 1.657038688659668,
+      "learning_rate": 3.905346358184629e-05,
+      "loss": 0.2885,
+      "step": 1785
+    },
+    {
+      "epoch": 11.33015873015873,
+      "grad_norm": 1.2840272188186646,
+      "learning_rate": 3.897675036627557e-05,
+      "loss": 0.2932,
+      "step": 1790
+    },
+    {
+      "epoch": 11.361904761904762,
+      "grad_norm": 1.5678766965866089,
+      "learning_rate": 3.8899845231413026e-05,
+      "loss": 0.2945,
+      "step": 1795
+    },
+    {
+      "epoch": 11.393650793650794,
+      "grad_norm": 1.788948655128479,
+      "learning_rate": 3.8822749233268006e-05,
+      "loss": 0.3013,
+      "step": 1800
+    },
+    {
+      "epoch": 11.425396825396826,
+      "grad_norm": 1.2259769439697266,
+      "learning_rate": 3.8745463430470664e-05,
+      "loss": 0.2582,
+      "step": 1805
+    },
+    {
+      "epoch": 11.457142857142857,
+      "grad_norm": 1.5430735349655151,
+      "learning_rate": 3.866798888425741e-05,
+      "loss": 0.275,
+      "step": 1810
+    },
+    {
+      "epoch": 11.488888888888889,
+      "grad_norm": 1.9102168083190918,
+      "learning_rate": 3.8590326658456376e-05,
+      "loss": 0.2909,
+      "step": 1815
+    },
+    {
+      "epoch": 11.52063492063492,
+      "grad_norm": 1.6118320226669312,
+      "learning_rate": 3.851247781947277e-05,
+      "loss": 0.2922,
+      "step": 1820
+    },
+    {
+      "epoch": 11.552380952380952,
+      "grad_norm": 1.393646478652954,
+      "learning_rate": 3.843444343627424e-05,
+      "loss": 0.2783,
+      "step": 1825
+    },
+    {
+      "epoch": 11.584126984126984,
+      "grad_norm": 2.522909641265869,
+      "learning_rate": 3.83562245803762e-05,
+      "loss": 0.2933,
+      "step": 1830
+    },
+    {
+      "epoch": 11.615873015873015,
+      "grad_norm": 2.2534332275390625,
+      "learning_rate": 3.827782232582714e-05,
+      "loss": 0.3081,
+      "step": 1835
+    },
+    {
+      "epoch": 11.647619047619047,
+      "grad_norm": 2.4088056087493896,
+      "learning_rate": 3.819923774919383e-05,
+      "loss": 0.276,
+      "step": 1840
+    },
+    {
+      "epoch": 11.679365079365079,
+      "grad_norm": 1.7626562118530273,
+      "learning_rate": 3.8120471929546576e-05,
+      "loss": 0.2697,
+      "step": 1845
+    },
+    {
+      "epoch": 11.71111111111111,
+      "grad_norm": 1.8656691312789917,
+      "learning_rate": 3.8041525948444414e-05,
+      "loss": 0.2979,
+      "step": 1850
+    },
+    {
+      "epoch": 11.742857142857144,
+      "grad_norm": 1.6537258625030518,
+      "learning_rate": 3.7962400889920185e-05,
+      "loss": 0.3042,
+      "step": 1855
+    },
+    {
+      "epoch": 11.774603174603175,
+      "grad_norm": 1.696975827217102,
+      "learning_rate": 3.788309784046574e-05,
+      "loss": 0.2984,
+      "step": 1860
+    },
+    {
+      "epoch": 11.806349206349207,
+      "grad_norm": 1.976236343383789,
+      "learning_rate": 3.780361788901696e-05,
+      "loss": 0.2711,
+      "step": 1865
+    },
+    {
+      "epoch": 11.838095238095239,
+      "grad_norm": 1.7781676054000854,
+      "learning_rate": 3.772396212693885e-05,
+      "loss": 0.3116,
+      "step": 1870
+    },
+    {
+      "epoch": 11.86984126984127,
+      "grad_norm": 1.6252080202102661,
+      "learning_rate": 3.7644131648010494e-05,
+      "loss": 0.2879,
+      "step": 1875
+    },
+    {
+      "epoch": 11.901587301587302,
+      "grad_norm": 1.7511012554168701,
+      "learning_rate": 3.75641275484101e-05,
+      "loss": 0.3106,
+      "step": 1880
+    },
+    {
+      "epoch": 11.933333333333334,
+      "grad_norm": 1.5861046314239502,
+      "learning_rate": 3.7483950926699885e-05,
+      "loss": 0.2703,
+      "step": 1885
+    },
+    {
+      "epoch": 11.965079365079365,
+      "grad_norm": 1.903902530670166,
+      "learning_rate": 3.740360288381105e-05,
+      "loss": 0.2808,
+      "step": 1890
+    },
+    {
+      "epoch": 11.996825396825397,
+      "grad_norm": 1.644740104675293,
+      "learning_rate": 3.732308452302864e-05,
+      "loss": 0.2883,
+      "step": 1895
+    },
+    {
+      "epoch": 12.025396825396825,
+      "grad_norm": 1.3671797513961792,
+      "learning_rate": 3.724239694997637e-05,
+      "loss": 0.2661,
+      "step": 1900
+    },
+    {
+      "epoch": 12.057142857142857,
+      "grad_norm": 1.1910501718521118,
+      "learning_rate": 3.716154127260147e-05,
+      "loss": 0.2352,
+      "step": 1905
+    },
+    {
+      "epoch": 12.088888888888889,
+      "grad_norm": 1.45137619972229,
+      "learning_rate": 3.708051860115947e-05,
+      "loss": 0.2703,
+      "step": 1910
+    },
+    {
+      "epoch": 12.12063492063492,
+      "grad_norm": 2.625089645385742,
+      "learning_rate": 3.699933004819895e-05,
+      "loss": 0.2524,
+      "step": 1915
+    },
+    {
+      "epoch": 12.152380952380952,
+      "grad_norm": 1.731602430343628,
+      "learning_rate": 3.691797672854625e-05,
+      "loss": 0.2533,
+      "step": 1920
+    },
+    {
+      "epoch": 12.184126984126983,
+      "grad_norm": 2.1266143321990967,
+      "learning_rate": 3.683645975929019e-05,
+      "loss": 0.2666,
+      "step": 1925
+    },
+    {
+      "epoch": 12.215873015873015,
+      "grad_norm": 1.7163398265838623,
+      "learning_rate": 3.675478025976671e-05,
+      "loss": 0.2838,
+      "step": 1930
+    },
+    {
+      "epoch": 12.247619047619047,
+      "grad_norm": 1.8726729154586792,
+      "learning_rate": 3.66729393515435e-05,
+      "loss": 0.2646,
+      "step": 1935
+    },
+    {
+      "epoch": 12.27936507936508,
+      "grad_norm": 1.2229722738265991,
+      "learning_rate": 3.659093815840462e-05,
+      "loss": 0.267,
+      "step": 1940
+    },
+    {
+      "epoch": 12.311111111111112,
+      "grad_norm": 1.341997504234314,
+      "learning_rate": 3.650877780633505e-05,
+      "loss": 0.2464,
+      "step": 1945
+    },
+    {
+      "epoch": 12.342857142857143,
+      "grad_norm": 1.3466174602508545,
+      "learning_rate": 3.6426459423505214e-05,
+      "loss": 0.2521,
+      "step": 1950
+    },
+    {
+      "epoch": 12.374603174603175,
+      "grad_norm": 1.7022134065628052,
+      "learning_rate": 3.6343984140255516e-05,
+      "loss": 0.2662,
+      "step": 1955
+    },
+    {
+      "epoch": 12.406349206349207,
+      "grad_norm": 2.5058987140655518,
+      "learning_rate": 3.626135308908084e-05,
+      "loss": 0.2745,
+      "step": 1960
+    },
+    {
+      "epoch": 12.438095238095238,
+      "grad_norm": 1.7667903900146484,
+      "learning_rate": 3.6178567404614936e-05,
+      "loss": 0.2752,
+      "step": 1965
+    },
+    {
+      "epoch": 12.46984126984127,
+      "grad_norm": 1.7334171533584595,
+      "learning_rate": 3.609562822361487e-05,
+      "loss": 0.2667,
+      "step": 1970
+    },
+    {
+      "epoch": 12.501587301587302,
+      "grad_norm": 1.7093929052352905,
+      "learning_rate": 3.601253668494546e-05,
+      "loss": 0.2829,
+      "step": 1975
+    },
+    {
+      "epoch": 12.533333333333333,
+      "grad_norm": 1.8272080421447754,
+      "learning_rate": 3.592929392956355e-05,
+      "loss": 0.2583,
+      "step": 1980
+    },
+    {
+      "epoch": 12.565079365079365,
+      "grad_norm": 1.5226343870162964,
+      "learning_rate": 3.584590110050241e-05,
+      "loss": 0.2652,
+      "step": 1985
+    },
+    {
+      "epoch": 12.596825396825396,
+      "grad_norm": 1.7395323514938354,
+      "learning_rate": 3.5762359342856036e-05,
+      "loss": 0.2585,
+      "step": 1990
+    },
+    {
+      "epoch": 12.628571428571428,
+      "grad_norm": 2.278144359588623,
+      "learning_rate": 3.567866980376337e-05,
+      "loss": 0.2862,
+      "step": 1995
+    },
+    {
+      "epoch": 12.66031746031746,
+      "grad_norm": 1.9335429668426514,
+      "learning_rate": 3.559483363239262e-05,
+      "loss": 0.2945,
+      "step": 2000
+    },
+    {
+      "epoch": 12.692063492063491,
+      "grad_norm": 1.3866902589797974,
+      "learning_rate": 3.551085197992545e-05,
+      "loss": 0.2675,
+      "step": 2005
+    },
+    {
+      "epoch": 12.723809523809523,
+      "grad_norm": 1.7110604047775269,
+      "learning_rate": 3.5426725999541174e-05,
+      "loss": 0.2705,
+      "step": 2010
+    },
+    {
+      "epoch": 12.755555555555556,
+      "grad_norm": 1.6411776542663574,
+      "learning_rate": 3.534245684640089e-05,
+      "loss": 0.2482,
+      "step": 2015
+    },
+    {
+      "epoch": 12.787301587301588,
+      "grad_norm": 1.4663981199264526,
+      "learning_rate": 3.525804567763167e-05,
+      "loss": 0.2838,
+      "step": 2020
+    },
+    {
+      "epoch": 12.81904761904762,
+      "grad_norm": 1.8676140308380127,
+      "learning_rate": 3.517349365231065e-05,
+      "loss": 0.2906,
+      "step": 2025
+    },
+    {
+      "epoch": 12.850793650793651,
+      "grad_norm": 1.6150639057159424,
+      "learning_rate": 3.508880193144911e-05,
+      "loss": 0.3011,
+      "step": 2030
+    },
+    {
+      "epoch": 12.882539682539683,
+      "grad_norm": 2.5578598976135254,
+      "learning_rate": 3.500397167797654e-05,
+      "loss": 0.2846,
+      "step": 2035
+    },
+    {
+      "epoch": 12.914285714285715,
+      "grad_norm": 1.3131941556930542,
+      "learning_rate": 3.491900405672466e-05,
+      "loss": 0.2509,
+      "step": 2040
+    },
+    {
+      "epoch": 12.946031746031746,
+      "grad_norm": 1.5713857412338257,
+      "learning_rate": 3.483390023441141e-05,
+      "loss": 0.2824,
+      "step": 2045
+    },
+    {
+      "epoch": 12.977777777777778,
+      "grad_norm": 1.788149356842041,
+      "learning_rate": 3.4748661379625e-05,
+      "loss": 0.2932,
+      "step": 2050
+    },
+    {
+      "epoch": 13.006349206349206,
+      "grad_norm": 1.080249547958374,
+      "learning_rate": 3.466328866280778e-05,
+      "loss": 0.2817,
+      "step": 2055
+    },
+    {
+      "epoch": 13.038095238095238,
+      "grad_norm": 1.1128981113433838,
+      "learning_rate": 3.45777832562402e-05,
+      "loss": 0.2141,
+      "step": 2060
+    },
+    {
+      "epoch": 13.06984126984127,
+      "grad_norm": 1.6394922733306885,
+      "learning_rate": 3.449214633402473e-05,
+      "loss": 0.216,
+      "step": 2065
+    },
+    {
+      "epoch": 13.101587301587301,
+      "grad_norm": 2.198416233062744,
+      "learning_rate": 3.440637907206973e-05,
+      "loss": 0.258,
+      "step": 2070
+    },
+    {
+      "epoch": 13.133333333333333,
+      "grad_norm": 1.5107051134109497,
+      "learning_rate": 3.4320482648073285e-05,
+      "loss": 0.2422,
+      "step": 2075
+    },
+    {
+      "epoch": 13.165079365079364,
+      "grad_norm": 1.7206802368164062,
+      "learning_rate": 3.4234458241507034e-05,
+      "loss": 0.2524,
+      "step": 2080
+    },
+    {
+      "epoch": 13.196825396825396,
+      "grad_norm": 1.4832013845443726,
+      "learning_rate": 3.4148307033600014e-05,
+      "loss": 0.27,
+      "step": 2085
+    },
+    {
+      "epoch": 13.228571428571428,
+      "grad_norm": 1.592344880104065,
+      "learning_rate": 3.4062030207322395e-05,
+      "loss": 0.2486,
+      "step": 2090
+    },
+    {
+      "epoch": 13.260317460317461,
+      "grad_norm": 1.7398722171783447,
+      "learning_rate": 3.397562894736926e-05,
+      "loss": 0.2605,
+      "step": 2095
+    },
+    {
+      "epoch": 13.292063492063493,
+      "grad_norm": 2.0896050930023193,
+      "learning_rate": 3.388910444014432e-05,
+      "loss": 0.2562,
+      "step": 2100
+    },
+    {
+      "epoch": 13.323809523809524,
+      "grad_norm": 2.7245311737060547,
+      "learning_rate": 3.380245787374365e-05,
+      "loss": 0.2573,
+      "step": 2105
+    },
+    {
+      "epoch": 13.355555555555556,
+      "grad_norm": 1.1951723098754883,
+      "learning_rate": 3.371569043793936e-05,
+      "loss": 0.2709,
+      "step": 2110
+    },
+    {
+      "epoch": 13.387301587301588,
+      "grad_norm": 1.697736144065857,
+      "learning_rate": 3.3628803324163236e-05,
+      "loss": 0.2413,
+      "step": 2115
+    },
+    {
+      "epoch": 13.41904761904762,
+      "grad_norm": 1.447302222251892,
+      "learning_rate": 3.354179772549042e-05,
+      "loss": 0.2685,
+      "step": 2120
+    },
+    {
+      "epoch": 13.450793650793651,
+      "grad_norm": 1.232428789138794,
+      "learning_rate": 3.3454674836623e-05,
+      "loss": 0.2546,
+      "step": 2125
+    },
+    {
+      "epoch": 13.482539682539683,
+      "grad_norm": 1.889103889465332,
+      "learning_rate": 3.336743585387362e-05,
+      "loss": 0.2412,
+      "step": 2130
+    },
+    {
+      "epoch": 13.514285714285714,
+      "grad_norm": 1.888352632522583,
+      "learning_rate": 3.328008197514903e-05,
+      "loss": 0.2737,
+      "step": 2135
+    },
+    {
+      "epoch": 13.546031746031746,
+      "grad_norm": 1.4205390214920044,
+      "learning_rate": 3.319261439993369e-05,
+      "loss": 0.2727,
+      "step": 2140
+    },
+    {
+      "epoch": 13.577777777777778,
+      "grad_norm": 1.442429542541504,
+      "learning_rate": 3.310503432927322e-05,
+      "loss": 0.2894,
+      "step": 2145
+    },
+    {
+      "epoch": 13.60952380952381,
+      "grad_norm": 1.413885474205017,
+      "learning_rate": 3.3017342965757986e-05,
+      "loss": 0.2928,
+      "step": 2150
+    },
+    {
+      "epoch": 13.64126984126984,
+      "grad_norm": 1.0464482307434082,
+      "learning_rate": 3.2929541513506524e-05,
+      "loss": 0.252,
+      "step": 2155
+    },
+    {
+      "epoch": 13.673015873015872,
+      "grad_norm": 1.6716376543045044,
+      "learning_rate": 3.284163117814906e-05,
+      "loss": 0.2614,
+      "step": 2160
+    },
+    {
+      "epoch": 13.704761904761904,
+      "grad_norm": 1.9366419315338135,
+      "learning_rate": 3.2753613166810914e-05,
+      "loss": 0.2963,
+      "step": 2165
+    },
+    {
+      "epoch": 13.736507936507937,
+      "grad_norm": 1.4641704559326172,
+      "learning_rate": 3.2665488688095944e-05,
+      "loss": 0.2654,
+      "step": 2170
+    },
+    {
+      "epoch": 13.768253968253969,
+      "grad_norm": 1.3404383659362793,
+      "learning_rate": 3.2577258952069934e-05,
+      "loss": 0.2601,
+      "step": 2175
+    },
+    {
+      "epoch": 13.8,
+      "grad_norm": 1.6471940279006958,
+      "learning_rate": 3.2488925170244015e-05,
+      "loss": 0.2775,
+      "step": 2180
+    },
+    {
+      "epoch": 13.831746031746032,
+      "grad_norm": 1.3665790557861328,
+      "learning_rate": 3.240048855555798e-05,
+      "loss": 0.2602,
+      "step": 2185
+    },
+    {
+      "epoch": 13.863492063492064,
+      "grad_norm": 1.1486542224884033,
+      "learning_rate": 3.2311950322363685e-05,
+      "loss": 0.2603,
+      "step": 2190
+    },
+    {
+      "epoch": 13.895238095238096,
+      "grad_norm": 1.45487642288208,
+      "learning_rate": 3.2223311686408304e-05,
+      "loss": 0.2794,
+      "step": 2195
+    },
+    {
+      "epoch": 13.926984126984127,
+      "grad_norm": 1.6884818077087402,
+      "learning_rate": 3.213457386481769e-05,
+      "loss": 0.2682,
+      "step": 2200
+    },
+    {
+      "epoch": 13.958730158730159,
+      "grad_norm": 1.9160594940185547,
+      "learning_rate": 3.204573807607967e-05,
+      "loss": 0.2619,
+      "step": 2205
+    },
+    {
+      "epoch": 13.99047619047619,
+      "grad_norm": 1.9444315433502197,
+      "learning_rate": 3.1956805540027244e-05,
+      "loss": 0.3029,
+      "step": 2210
+    },
+    {
+      "epoch": 14.019047619047619,
+      "grad_norm": 1.8739337921142578,
+      "learning_rate": 3.1867777477821915e-05,
+      "loss": 0.2298,
+      "step": 2215
+    },
+    {
+      "epoch": 14.05079365079365,
+      "grad_norm": 1.3971226215362549,
+      "learning_rate": 3.1778655111936866e-05,
+      "loss": 0.2351,
+      "step": 2220
+    },
+    {
+      "epoch": 14.082539682539682,
+      "grad_norm": 1.288387417793274,
+      "learning_rate": 3.168943966614022e-05,
+      "loss": 0.2331,
+      "step": 2225
+    },
+    {
+      "epoch": 14.114285714285714,
+      "grad_norm": 1.2136940956115723,
+      "learning_rate": 3.1600132365478186e-05,
+      "loss": 0.2406,
+      "step": 2230
+    },
+    {
+      "epoch": 14.146031746031746,
+      "grad_norm": 1.8233654499053955,
+      "learning_rate": 3.151073443625828e-05,
+      "loss": 0.2496,
+      "step": 2235
+    },
+    {
+      "epoch": 14.177777777777777,
+      "grad_norm": 1.7613590955734253,
+      "learning_rate": 3.142124710603245e-05,
+      "loss": 0.2523,
+      "step": 2240
+    },
+    {
+      "epoch": 14.209523809523809,
+      "grad_norm": 1.8933606147766113,
+      "learning_rate": 3.133167160358024e-05,
+      "loss": 0.2646,
+      "step": 2245
+    },
+    {
+      "epoch": 14.24126984126984,
+      "grad_norm": 2.7858712673187256,
+      "learning_rate": 3.124200915889195e-05,
+      "loss": 0.2818,
+      "step": 2250
+    },
+    {
+      "epoch": 14.273015873015874,
+      "grad_norm": 1.5656853914260864,
+      "learning_rate": 3.115226100315166e-05,
+      "loss": 0.2389,
+      "step": 2255
+    },
+    {
+      "epoch": 14.304761904761905,
+      "grad_norm": 1.606392741203308,
+      "learning_rate": 3.106242836872041e-05,
+      "loss": 0.2302,
+      "step": 2260
+    },
+    {
+      "epoch": 14.336507936507937,
+      "grad_norm": 2.1553008556365967,
+      "learning_rate": 3.097251248911922e-05,
+      "loss": 0.2484,
+      "step": 2265
+    },
+    {
+      "epoch": 14.368253968253969,
+      "grad_norm": 1.7176517248153687,
+      "learning_rate": 3.088251459901222e-05,
+      "loss": 0.268,
+      "step": 2270
+    },
+    {
+      "epoch": 14.4,
+      "grad_norm": 1.3635174036026,
+      "learning_rate": 3.0792435934189604e-05,
+      "loss": 0.2491,
+      "step": 2275
+    },
+    {
+      "epoch": 14.431746031746032,
+      "grad_norm": 1.4257878065109253,
+      "learning_rate": 3.070227773155074e-05,
+      "loss": 0.2884,
+      "step": 2280
+    },
+    {
+      "epoch": 14.463492063492064,
+      "grad_norm": 1.1561063528060913,
+      "learning_rate": 3.0612041229087145e-05,
+      "loss": 0.247,
+      "step": 2285
+    },
+    {
+      "epoch": 14.495238095238095,
+      "grad_norm": 1.6131694316864014,
+      "learning_rate": 3.0521727665865485e-05,
+      "loss": 0.2708,
+      "step": 2290
+    },
+    {
+      "epoch": 14.526984126984127,
+      "grad_norm": 1.7133935689926147,
+      "learning_rate": 3.0431338282010606e-05,
+      "loss": 0.2384,
+      "step": 2295
+    },
+    {
+      "epoch": 14.558730158730159,
+      "grad_norm": 1.3232437372207642,
+      "learning_rate": 3.0340874318688433e-05,
+      "loss": 0.2503,
+      "step": 2300
+    },
+    {
+      "epoch": 14.59047619047619,
+      "grad_norm": 1.5612751245498657,
+      "learning_rate": 3.0250337018088994e-05,
+      "loss": 0.2618,
+      "step": 2305
+    },
+    {
+      "epoch": 14.622222222222222,
+      "grad_norm": 1.279241681098938,
+      "learning_rate": 3.0159727623409313e-05,
+      "loss": 0.2535,
+      "step": 2310
+    },
+    {
+      "epoch": 14.653968253968253,
+      "grad_norm": 3.142040729522705,
+      "learning_rate": 3.0069047378836375e-05,
+      "loss": 0.2745,
+      "step": 2315
+    },
+    {
+      "epoch": 14.685714285714285,
+      "grad_norm": 1.3240234851837158,
+      "learning_rate": 2.9978297529530036e-05,
+      "loss": 0.2591,
+      "step": 2320
+    },
+    {
+      "epoch": 14.717460317460317,
+      "grad_norm": 1.121077537536621,
+      "learning_rate": 2.9887479321605895e-05,
+      "loss": 0.2453,
+      "step": 2325
+    },
+    {
+      "epoch": 14.74920634920635,
+      "grad_norm": 2.3831980228424072,
+      "learning_rate": 2.9796594002118218e-05,
+      "loss": 0.257,
+      "step": 2330
+    },
+    {
+      "epoch": 14.780952380952382,
+      "grad_norm": 2.175352096557617,
+      "learning_rate": 2.9705642819042805e-05,
+      "loss": 0.2434,
+      "step": 2335
+    },
+    {
+      "epoch": 14.812698412698413,
+      "grad_norm": 1.5452053546905518,
+      "learning_rate": 2.9614627021259846e-05,
+      "loss": 0.2683,
+      "step": 2340
+    },
+    {
+      "epoch": 14.844444444444445,
+      "grad_norm": 1.6765031814575195,
+      "learning_rate": 2.952354785853677e-05,
+      "loss": 0.2484,
+      "step": 2345
+    },
+    {
+      "epoch": 14.876190476190477,
+      "grad_norm": 1.3790463209152222,
+      "learning_rate": 2.9432406581511096e-05,
+      "loss": 0.2636,
+      "step": 2350
+    },
+    {
+      "epoch": 14.907936507936508,
+      "grad_norm": 1.2849820852279663,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.2734,
+      "step": 2355
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4710,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 30,
+  "save_steps": 157,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.058470270835098e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2512/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-2512/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-2512/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-2512/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2512/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2512/trainer_state.json

@@ -0,0 +1,3548 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 15.901587301587302,
+  "eval_steps": 500,
+  "global_step": 2512,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 0.5545095205307007,
+      "learning_rate": 5.307855626326963e-07,
+      "loss": 3.7162,
+      "step": 5
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 0.6163601279258728,
+      "learning_rate": 1.0615711252653927e-06,
+      "loss": 3.9388,
+      "step": 10
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.5541530847549438,
+      "learning_rate": 1.5923566878980892e-06,
+      "loss": 3.9165,
+      "step": 15
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 0.457332044839859,
+      "learning_rate": 2.1231422505307854e-06,
+      "loss": 3.7326,
+      "step": 20
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 0.5335279107093811,
+      "learning_rate": 2.653927813163482e-06,
+      "loss": 3.8251,
+      "step": 25
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.7080379724502563,
+      "learning_rate": 3.1847133757961785e-06,
+      "loss": 3.7534,
+      "step": 30
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.520993709564209,
+      "learning_rate": 3.715498938428875e-06,
+      "loss": 3.898,
+      "step": 35
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 0.5451405644416809,
+      "learning_rate": 4.246284501061571e-06,
+      "loss": 3.8951,
+      "step": 40
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.6205154657363892,
+      "learning_rate": 4.777070063694268e-06,
+      "loss": 3.7666,
+      "step": 45
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 0.7404439449310303,
+      "learning_rate": 5.307855626326964e-06,
+      "loss": 4.0258,
+      "step": 50
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 0.6272220015525818,
+      "learning_rate": 5.838641188959661e-06,
+      "loss": 3.8464,
+      "step": 55
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 0.7744691967964172,
+      "learning_rate": 6.369426751592357e-06,
+      "loss": 3.7299,
+      "step": 60
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 0.8805738687515259,
+      "learning_rate": 6.900212314225053e-06,
+      "loss": 3.5008,
+      "step": 65
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.0740723609924316,
+      "learning_rate": 7.43099787685775e-06,
+      "loss": 3.7552,
+      "step": 70
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.965708315372467,
+      "learning_rate": 7.961783439490445e-06,
+      "loss": 3.5516,
+      "step": 75
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 0.9812778234481812,
+      "learning_rate": 8.492569002123141e-06,
+      "loss": 3.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 0.8831024169921875,
+      "learning_rate": 9.023354564755838e-06,
+      "loss": 3.613,
+      "step": 85
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.8358364105224609,
+      "learning_rate": 9.554140127388536e-06,
+      "loss": 3.1858,
+      "step": 90
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 1.0740444660186768,
+      "learning_rate": 1.0084925690021232e-05,
+      "loss": 3.0937,
+      "step": 95
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 1.0987530946731567,
+      "learning_rate": 1.0615711252653929e-05,
+      "loss": 3.154,
+      "step": 100
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 1.2300925254821777,
+      "learning_rate": 1.1146496815286625e-05,
+      "loss": 2.9414,
+      "step": 105
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 1.2214170694351196,
+      "learning_rate": 1.1677282377919321e-05,
+      "loss": 2.9464,
+      "step": 110
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 1.2803975343704224,
+      "learning_rate": 1.2208067940552018e-05,
+      "loss": 2.8921,
+      "step": 115
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 1.2232719659805298,
+      "learning_rate": 1.2738853503184714e-05,
+      "loss": 2.5252,
+      "step": 120
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 1.204835295677185,
+      "learning_rate": 1.326963906581741e-05,
+      "loss": 2.5215,
+      "step": 125
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 1.4095579385757446,
+      "learning_rate": 1.3800424628450107e-05,
+      "loss": 2.136,
+      "step": 130
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 1.4166598320007324,
+      "learning_rate": 1.4331210191082803e-05,
+      "loss": 2.2653,
+      "step": 135
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.3040446043014526,
+      "learning_rate": 1.48619957537155e-05,
+      "loss": 2.0193,
+      "step": 140
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 1.4114688634872437,
+      "learning_rate": 1.5392781316348196e-05,
+      "loss": 1.7935,
+      "step": 145
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 1.8066726922988892,
+      "learning_rate": 1.592356687898089e-05,
+      "loss": 1.5731,
+      "step": 150
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 1.4303158521652222,
+      "learning_rate": 1.6454352441613588e-05,
+      "loss": 1.6552,
+      "step": 155
+    },
+    {
+      "epoch": 1.0126984126984127,
+      "grad_norm": 1.6671762466430664,
+      "learning_rate": 1.6985138004246283e-05,
+      "loss": 1.6973,
+      "step": 160
+    },
+    {
+      "epoch": 1.0444444444444445,
+      "grad_norm": 1.5719650983810425,
+      "learning_rate": 1.751592356687898e-05,
+      "loss": 1.312,
+      "step": 165
+    },
+    {
+      "epoch": 1.0761904761904761,
+      "grad_norm": 1.4845054149627686,
+      "learning_rate": 1.8046709129511676e-05,
+      "loss": 1.3601,
+      "step": 170
+    },
+    {
+      "epoch": 1.107936507936508,
+      "grad_norm": 1.1172235012054443,
+      "learning_rate": 1.8577494692144374e-05,
+      "loss": 1.3137,
+      "step": 175
+    },
+    {
+      "epoch": 1.1396825396825396,
+      "grad_norm": 1.9621731042861938,
+      "learning_rate": 1.910828025477707e-05,
+      "loss": 1.1778,
+      "step": 180
+    },
+    {
+      "epoch": 1.1714285714285715,
+      "grad_norm": 1.7722721099853516,
+      "learning_rate": 1.963906581740977e-05,
+      "loss": 1.4534,
+      "step": 185
+    },
+    {
+      "epoch": 1.2031746031746031,
+      "grad_norm": 1.3677467107772827,
+      "learning_rate": 2.0169851380042464e-05,
+      "loss": 1.3356,
+      "step": 190
+    },
+    {
+      "epoch": 1.234920634920635,
+      "grad_norm": 1.3260482549667358,
+      "learning_rate": 2.0700636942675162e-05,
+      "loss": 1.0876,
+      "step": 195
+    },
+    {
+      "epoch": 1.2666666666666666,
+      "grad_norm": 1.5176818370819092,
+      "learning_rate": 2.1231422505307857e-05,
+      "loss": 1.1602,
+      "step": 200
+    },
+    {
+      "epoch": 1.2984126984126985,
+      "grad_norm": 1.2793077230453491,
+      "learning_rate": 2.1762208067940555e-05,
+      "loss": 1.1505,
+      "step": 205
+    },
+    {
+      "epoch": 1.33015873015873,
+      "grad_norm": 1.196784257888794,
+      "learning_rate": 2.229299363057325e-05,
+      "loss": 1.0664,
+      "step": 210
+    },
+    {
+      "epoch": 1.361904761904762,
+      "grad_norm": 1.303207516670227,
+      "learning_rate": 2.2823779193205948e-05,
+      "loss": 1.2557,
+      "step": 215
+    },
+    {
+      "epoch": 1.3936507936507936,
+      "grad_norm": 1.2853388786315918,
+      "learning_rate": 2.3354564755838642e-05,
+      "loss": 1.0704,
+      "step": 220
+    },
+    {
+      "epoch": 1.4253968253968254,
+      "grad_norm": 1.381369948387146,
+      "learning_rate": 2.388535031847134e-05,
+      "loss": 1.1371,
+      "step": 225
+    },
+    {
+      "epoch": 1.457142857142857,
+      "grad_norm": 1.8012712001800537,
+      "learning_rate": 2.4416135881104035e-05,
+      "loss": 1.248,
+      "step": 230
+    },
+    {
+      "epoch": 1.488888888888889,
+      "grad_norm": 1.7397032976150513,
+      "learning_rate": 2.4946921443736733e-05,
+      "loss": 1.2782,
+      "step": 235
+    },
+    {
+      "epoch": 1.5206349206349206,
+      "grad_norm": 1.4026210308074951,
+      "learning_rate": 2.5477707006369428e-05,
+      "loss": 1.154,
+      "step": 240
+    },
+    {
+      "epoch": 1.5523809523809524,
+      "grad_norm": 1.2906067371368408,
+      "learning_rate": 2.6008492569002126e-05,
+      "loss": 0.9141,
+      "step": 245
+    },
+    {
+      "epoch": 1.5841269841269843,
+      "grad_norm": 1.265598177909851,
+      "learning_rate": 2.653927813163482e-05,
+      "loss": 1.0625,
+      "step": 250
+    },
+    {
+      "epoch": 1.615873015873016,
+      "grad_norm": 1.6044715642929077,
+      "learning_rate": 2.707006369426752e-05,
+      "loss": 0.9624,
+      "step": 255
+    },
+    {
+      "epoch": 1.6476190476190475,
+      "grad_norm": 1.4612747430801392,
+      "learning_rate": 2.7600849256900213e-05,
+      "loss": 1.0413,
+      "step": 260
+    },
+    {
+      "epoch": 1.6793650793650794,
+      "grad_norm": 1.6222745180130005,
+      "learning_rate": 2.8131634819532908e-05,
+      "loss": 1.0929,
+      "step": 265
+    },
+    {
+      "epoch": 1.7111111111111112,
+      "grad_norm": 1.1456222534179688,
+      "learning_rate": 2.8662420382165606e-05,
+      "loss": 0.9957,
+      "step": 270
+    },
+    {
+      "epoch": 1.7428571428571429,
+      "grad_norm": 1.5746041536331177,
+      "learning_rate": 2.91932059447983e-05,
+      "loss": 1.0274,
+      "step": 275
+    },
+    {
+      "epoch": 1.7746031746031745,
+      "grad_norm": 1.3407832384109497,
+      "learning_rate": 2.9723991507431e-05,
+      "loss": 0.9487,
+      "step": 280
+    },
+    {
+      "epoch": 1.8063492063492064,
+      "grad_norm": 1.6232194900512695,
+      "learning_rate": 3.0254777070063693e-05,
+      "loss": 1.0966,
+      "step": 285
+    },
+    {
+      "epoch": 1.8380952380952382,
+      "grad_norm": 1.4920552968978882,
+      "learning_rate": 3.078556263269639e-05,
+      "loss": 0.9099,
+      "step": 290
+    },
+    {
+      "epoch": 1.8698412698412699,
+      "grad_norm": 1.2123301029205322,
+      "learning_rate": 3.1316348195329086e-05,
+      "loss": 1.0902,
+      "step": 295
+    },
+    {
+      "epoch": 1.9015873015873015,
+      "grad_norm": 1.2080968618392944,
+      "learning_rate": 3.184713375796178e-05,
+      "loss": 0.943,
+      "step": 300
+    },
+    {
+      "epoch": 1.9333333333333333,
+      "grad_norm": 1.190319299697876,
+      "learning_rate": 3.237791932059448e-05,
+      "loss": 0.7893,
+      "step": 305
+    },
+    {
+      "epoch": 1.9650793650793652,
+      "grad_norm": 1.5929204225540161,
+      "learning_rate": 3.2908704883227177e-05,
+      "loss": 1.0232,
+      "step": 310
+    },
+    {
+      "epoch": 1.9968253968253968,
+      "grad_norm": 1.0138347148895264,
+      "learning_rate": 3.343949044585987e-05,
+      "loss": 0.6693,
+      "step": 315
+    },
+    {
+      "epoch": 2.0253968253968253,
+      "grad_norm": 1.3012847900390625,
+      "learning_rate": 3.3970276008492566e-05,
+      "loss": 0.8355,
+      "step": 320
+    },
+    {
+      "epoch": 2.057142857142857,
+      "grad_norm": 1.2264782190322876,
+      "learning_rate": 3.450106157112527e-05,
+      "loss": 0.9872,
+      "step": 325
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 1.139275312423706,
+      "learning_rate": 3.503184713375796e-05,
+      "loss": 0.8662,
+      "step": 330
+    },
+    {
+      "epoch": 2.1206349206349207,
+      "grad_norm": 1.3836581707000732,
+      "learning_rate": 3.5562632696390657e-05,
+      "loss": 0.9549,
+      "step": 335
+    },
+    {
+      "epoch": 2.1523809523809523,
+      "grad_norm": 1.368600845336914,
+      "learning_rate": 3.609341825902335e-05,
+      "loss": 0.9195,
+      "step": 340
+    },
+    {
+      "epoch": 2.1841269841269844,
+      "grad_norm": 1.8793011903762817,
+      "learning_rate": 3.662420382165605e-05,
+      "loss": 0.8505,
+      "step": 345
+    },
+    {
+      "epoch": 2.215873015873016,
+      "grad_norm": 1.305284023284912,
+      "learning_rate": 3.715498938428875e-05,
+      "loss": 0.7755,
+      "step": 350
+    },
+    {
+      "epoch": 2.2476190476190476,
+      "grad_norm": 1.7851749658584595,
+      "learning_rate": 3.768577494692145e-05,
+      "loss": 0.9242,
+      "step": 355
+    },
+    {
+      "epoch": 2.2793650793650793,
+      "grad_norm": 1.4341535568237305,
+      "learning_rate": 3.821656050955414e-05,
+      "loss": 0.8221,
+      "step": 360
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 1.39107346534729,
+      "learning_rate": 3.874734607218684e-05,
+      "loss": 0.6999,
+      "step": 365
+    },
+    {
+      "epoch": 2.342857142857143,
+      "grad_norm": 1.2304264307022095,
+      "learning_rate": 3.927813163481954e-05,
+      "loss": 0.8362,
+      "step": 370
+    },
+    {
+      "epoch": 2.3746031746031746,
+      "grad_norm": 1.8470840454101562,
+      "learning_rate": 3.9808917197452234e-05,
+      "loss": 0.9398,
+      "step": 375
+    },
+    {
+      "epoch": 2.4063492063492062,
+      "grad_norm": 1.2533882856369019,
+      "learning_rate": 4.033970276008493e-05,
+      "loss": 0.7754,
+      "step": 380
+    },
+    {
+      "epoch": 2.4380952380952383,
+      "grad_norm": 1.5335006713867188,
+      "learning_rate": 4.087048832271762e-05,
+      "loss": 1.1124,
+      "step": 385
+    },
+    {
+      "epoch": 2.46984126984127,
+      "grad_norm": 1.5298357009887695,
+      "learning_rate": 4.1401273885350325e-05,
+      "loss": 1.017,
+      "step": 390
+    },
+    {
+      "epoch": 2.5015873015873016,
+      "grad_norm": 1.4403260946273804,
+      "learning_rate": 4.193205944798302e-05,
+      "loss": 0.8831,
+      "step": 395
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 1.1528433561325073,
+      "learning_rate": 4.2462845010615714e-05,
+      "loss": 0.801,
+      "step": 400
+    },
+    {
+      "epoch": 2.565079365079365,
+      "grad_norm": 1.3371326923370361,
+      "learning_rate": 4.299363057324841e-05,
+      "loss": 0.8692,
+      "step": 405
+    },
+    {
+      "epoch": 2.596825396825397,
+      "grad_norm": 1.4064775705337524,
+      "learning_rate": 4.352441613588111e-05,
+      "loss": 0.9059,
+      "step": 410
+    },
+    {
+      "epoch": 2.6285714285714286,
+      "grad_norm": 1.4531422853469849,
+      "learning_rate": 4.4055201698513805e-05,
+      "loss": 0.7344,
+      "step": 415
+    },
+    {
+      "epoch": 2.66031746031746,
+      "grad_norm": 1.7043890953063965,
+      "learning_rate": 4.45859872611465e-05,
+      "loss": 0.8298,
+      "step": 420
+    },
+    {
+      "epoch": 2.6920634920634923,
+      "grad_norm": 1.5105586051940918,
+      "learning_rate": 4.5116772823779194e-05,
+      "loss": 0.7768,
+      "step": 425
+    },
+    {
+      "epoch": 2.723809523809524,
+      "grad_norm": 1.8101528882980347,
+      "learning_rate": 4.5647558386411895e-05,
+      "loss": 0.733,
+      "step": 430
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 1.6365174055099487,
+      "learning_rate": 4.617834394904459e-05,
+      "loss": 0.8061,
+      "step": 435
+    },
+    {
+      "epoch": 2.787301587301587,
+      "grad_norm": 1.7808202505111694,
+      "learning_rate": 4.6709129511677285e-05,
+      "loss": 0.8333,
+      "step": 440
+    },
+    {
+      "epoch": 2.819047619047619,
+      "grad_norm": 1.5223265886306763,
+      "learning_rate": 4.723991507430998e-05,
+      "loss": 0.7557,
+      "step": 445
+    },
+    {
+      "epoch": 2.850793650793651,
+      "grad_norm": 1.3064416646957397,
+      "learning_rate": 4.777070063694268e-05,
+      "loss": 0.8041,
+      "step": 450
+    },
+    {
+      "epoch": 2.8825396825396825,
+      "grad_norm": 1.8025637865066528,
+      "learning_rate": 4.8301486199575375e-05,
+      "loss": 0.9534,
+      "step": 455
+    },
+    {
+      "epoch": 2.914285714285714,
+      "grad_norm": 1.924846887588501,
+      "learning_rate": 4.883227176220807e-05,
+      "loss": 0.9066,
+      "step": 460
+    },
+    {
+      "epoch": 2.9460317460317462,
+      "grad_norm": 1.9862899780273438,
+      "learning_rate": 4.9363057324840765e-05,
+      "loss": 0.7994,
+      "step": 465
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 1.9615916013717651,
+      "learning_rate": 4.9893842887473466e-05,
+      "loss": 0.7045,
+      "step": 470
+    },
+    {
+      "epoch": 3.0063492063492063,
+      "grad_norm": 1.519852876663208,
+      "learning_rate": 4.999989014936042e-05,
+      "loss": 0.7212,
+      "step": 475
+    },
+    {
+      "epoch": 3.038095238095238,
+      "grad_norm": 1.9328887462615967,
+      "learning_rate": 4.999944388279162e-05,
+      "loss": 0.6598,
+      "step": 480
+    },
+    {
+      "epoch": 3.06984126984127,
+      "grad_norm": 2.0340709686279297,
+      "learning_rate": 4.999865434075176e-05,
+      "loss": 0.6829,
+      "step": 485
+    },
+    {
+      "epoch": 3.1015873015873017,
+      "grad_norm": 1.8775280714035034,
+      "learning_rate": 4.999752153408229e-05,
+      "loss": 0.6664,
+      "step": 490
+    },
+    {
+      "epoch": 3.1333333333333333,
+      "grad_norm": 2.385218381881714,
+      "learning_rate": 4.999604547833814e-05,
+      "loss": 0.6836,
+      "step": 495
+    },
+    {
+      "epoch": 3.165079365079365,
+      "grad_norm": 2.1743783950805664,
+      "learning_rate": 4.999422619378752e-05,
+      "loss": 0.7,
+      "step": 500
+    },
+    {
+      "epoch": 3.196825396825397,
+      "grad_norm": 2.20786452293396,
+      "learning_rate": 4.999206370541162e-05,
+      "loss": 0.7253,
+      "step": 505
+    },
+    {
+      "epoch": 3.2285714285714286,
+      "grad_norm": 1.8182263374328613,
+      "learning_rate": 4.998955804290425e-05,
+      "loss": 0.6824,
+      "step": 510
+    },
+    {
+      "epoch": 3.2603174603174603,
+      "grad_norm": 2.2959372997283936,
+      "learning_rate": 4.9986709240671495e-05,
+      "loss": 0.601,
+      "step": 515
+    },
+    {
+      "epoch": 3.292063492063492,
+      "grad_norm": 2.385838031768799,
+      "learning_rate": 4.998351733783116e-05,
+      "loss": 0.7417,
+      "step": 520
+    },
+    {
+      "epoch": 3.323809523809524,
+      "grad_norm": 2.0416879653930664,
+      "learning_rate": 4.997998237821233e-05,
+      "loss": 0.6463,
+      "step": 525
+    },
+    {
+      "epoch": 3.3555555555555556,
+      "grad_norm": 2.2781031131744385,
+      "learning_rate": 4.9976104410354654e-05,
+      "loss": 0.6998,
+      "step": 530
+    },
+    {
+      "epoch": 3.3873015873015873,
+      "grad_norm": 2.146778106689453,
+      "learning_rate": 4.9971883487507775e-05,
+      "loss": 0.7694,
+      "step": 535
+    },
+    {
+      "epoch": 3.419047619047619,
+      "grad_norm": 2.1369104385375977,
+      "learning_rate": 4.9967319667630567e-05,
+      "loss": 0.6615,
+      "step": 540
+    },
+    {
+      "epoch": 3.450793650793651,
+      "grad_norm": 2.4529733657836914,
+      "learning_rate": 4.996241301339029e-05,
+      "loss": 0.6109,
+      "step": 545
+    },
+    {
+      "epoch": 3.4825396825396826,
+      "grad_norm": 2.07030987739563,
+      "learning_rate": 4.995716359216183e-05,
+      "loss": 0.7611,
+      "step": 550
+    },
+    {
+      "epoch": 3.5142857142857142,
+      "grad_norm": 2.4329919815063477,
+      "learning_rate": 4.995157147602669e-05,
+      "loss": 0.7515,
+      "step": 555
+    },
+    {
+      "epoch": 3.546031746031746,
+      "grad_norm": 2.056351900100708,
+      "learning_rate": 4.994563674177202e-05,
+      "loss": 0.6885,
+      "step": 560
+    },
+    {
+      "epoch": 3.5777777777777775,
+      "grad_norm": 2.3665318489074707,
+      "learning_rate": 4.993935947088958e-05,
+      "loss": 0.6271,
+      "step": 565
+    },
+    {
+      "epoch": 3.6095238095238096,
+      "grad_norm": 2.677706480026245,
+      "learning_rate": 4.993273974957463e-05,
+      "loss": 0.5586,
+      "step": 570
+    },
+    {
+      "epoch": 3.641269841269841,
+      "grad_norm": 3.422136068344116,
+      "learning_rate": 4.9925777668724685e-05,
+      "loss": 0.7552,
+      "step": 575
+    },
+    {
+      "epoch": 3.6730158730158733,
+      "grad_norm": 2.4525184631347656,
+      "learning_rate": 4.991847332393835e-05,
+      "loss": 0.7367,
+      "step": 580
+    },
+    {
+      "epoch": 3.704761904761905,
+      "grad_norm": 2.4242067337036133,
+      "learning_rate": 4.991082681551396e-05,
+      "loss": 0.7044,
+      "step": 585
+    },
+    {
+      "epoch": 3.7365079365079366,
+      "grad_norm": 1.8419867753982544,
+      "learning_rate": 4.9902838248448184e-05,
+      "loss": 0.5966,
+      "step": 590
+    },
+    {
+      "epoch": 3.768253968253968,
+      "grad_norm": 2.1394360065460205,
+      "learning_rate": 4.989450773243463e-05,
+      "loss": 0.6736,
+      "step": 595
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 1.285447597503662,
+      "learning_rate": 4.9885835381862326e-05,
+      "loss": 0.5021,
+      "step": 600
+    },
+    {
+      "epoch": 3.831746031746032,
+      "grad_norm": 2.724978446960449,
+      "learning_rate": 4.987682131581413e-05,
+      "loss": 0.6128,
+      "step": 605
+    },
+    {
+      "epoch": 3.8634920634920635,
+      "grad_norm": 2.239682912826538,
+      "learning_rate": 4.986746565806508e-05,
+      "loss": 0.5457,
+      "step": 610
+    },
+    {
+      "epoch": 3.895238095238095,
+      "grad_norm": 2.48944091796875,
+      "learning_rate": 4.9857768537080784e-05,
+      "loss": 0.6927,
+      "step": 615
+    },
+    {
+      "epoch": 3.9269841269841272,
+      "grad_norm": 2.4086852073669434,
+      "learning_rate": 4.9847730086015534e-05,
+      "loss": 0.5963,
+      "step": 620
+    },
+    {
+      "epoch": 3.958730158730159,
+      "grad_norm": 2.0070106983184814,
+      "learning_rate": 4.9837350442710553e-05,
+      "loss": 0.5856,
+      "step": 625
+    },
+    {
+      "epoch": 3.9904761904761905,
+      "grad_norm": 1.9726545810699463,
+      "learning_rate": 4.98266297496921e-05,
+      "loss": 0.6208,
+      "step": 630
+    },
+    {
+      "epoch": 4.019047619047619,
+      "grad_norm": 2.6137828826904297,
+      "learning_rate": 4.981556815416948e-05,
+      "loss": 0.6319,
+      "step": 635
+    },
+    {
+      "epoch": 4.050793650793651,
+      "grad_norm": 2.3489890098571777,
+      "learning_rate": 4.9804165808033054e-05,
+      "loss": 0.5887,
+      "step": 640
+    },
+    {
+      "epoch": 4.082539682539682,
+      "grad_norm": 2.8010590076446533,
+      "learning_rate": 4.979242286785214e-05,
+      "loss": 0.5257,
+      "step": 645
+    },
+    {
+      "epoch": 4.114285714285714,
+      "grad_norm": 2.993411064147949,
+      "learning_rate": 4.978033949487284e-05,
+      "loss": 0.4545,
+      "step": 650
+    },
+    {
+      "epoch": 4.146031746031746,
+      "grad_norm": 2.669935703277588,
+      "learning_rate": 4.976791585501588e-05,
+      "loss": 0.5989,
+      "step": 655
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 3.084409236907959,
+      "learning_rate": 4.9755152118874294e-05,
+      "loss": 0.528,
+      "step": 660
+    },
+    {
+      "epoch": 4.20952380952381,
+      "grad_norm": 2.797873020172119,
+      "learning_rate": 4.974204846171106e-05,
+      "loss": 0.5249,
+      "step": 665
+    },
+    {
+      "epoch": 4.241269841269841,
+      "grad_norm": 3.667867422103882,
+      "learning_rate": 4.9728605063456765e-05,
+      "loss": 0.5838,
+      "step": 670
+    },
+    {
+      "epoch": 4.273015873015873,
+      "grad_norm": 2.6918869018554688,
+      "learning_rate": 4.971482210870706e-05,
+      "loss": 0.5143,
+      "step": 675
+    },
+    {
+      "epoch": 4.304761904761905,
+      "grad_norm": 2.1545379161834717,
+      "learning_rate": 4.970069978672017e-05,
+      "loss": 0.5317,
+      "step": 680
+    },
+    {
+      "epoch": 4.336507936507936,
+      "grad_norm": 2.1043529510498047,
+      "learning_rate": 4.9686238291414275e-05,
+      "loss": 0.4815,
+      "step": 685
+    },
+    {
+      "epoch": 4.368253968253969,
+      "grad_norm": 2.1359753608703613,
+      "learning_rate": 4.9671437821364855e-05,
+      "loss": 0.4935,
+      "step": 690
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 3.092057228088379,
+      "learning_rate": 4.965629857980197e-05,
+      "loss": 0.6831,
+      "step": 695
+    },
+    {
+      "epoch": 4.431746031746032,
+      "grad_norm": 2.5296835899353027,
+      "learning_rate": 4.964082077460745e-05,
+      "loss": 0.5323,
+      "step": 700
+    },
+    {
+      "epoch": 4.463492063492064,
+      "grad_norm": 1.6655627489089966,
+      "learning_rate": 4.962500461831207e-05,
+      "loss": 0.4553,
+      "step": 705
+    },
+    {
+      "epoch": 4.495238095238095,
+      "grad_norm": 2.6663475036621094,
+      "learning_rate": 4.9608850328092576e-05,
+      "loss": 0.463,
+      "step": 710
+    },
+    {
+      "epoch": 4.526984126984127,
+      "grad_norm": 2.3763060569763184,
+      "learning_rate": 4.959235812576879e-05,
+      "loss": 0.4861,
+      "step": 715
+    },
+    {
+      "epoch": 4.5587301587301585,
+      "grad_norm": 2.2217962741851807,
+      "learning_rate": 4.957552823780047e-05,
+      "loss": 0.468,
+      "step": 720
+    },
+    {
+      "epoch": 4.59047619047619,
+      "grad_norm": 2.8885600566864014,
+      "learning_rate": 4.9558360895284295e-05,
+      "loss": 0.4588,
+      "step": 725
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 2.5661261081695557,
+      "learning_rate": 4.954085633395058e-05,
+      "loss": 0.4926,
+      "step": 730
+    },
+    {
+      "epoch": 4.653968253968254,
+      "grad_norm": 2.304365396499634,
+      "learning_rate": 4.952301479416015e-05,
+      "loss": 0.494,
+      "step": 735
+    },
+    {
+      "epoch": 4.685714285714286,
+      "grad_norm": 2.690577983856201,
+      "learning_rate": 4.9504836520900976e-05,
+      "loss": 0.5814,
+      "step": 740
+    },
+    {
+      "epoch": 4.717460317460318,
+      "grad_norm": 2.7180025577545166,
+      "learning_rate": 4.948632176378481e-05,
+      "loss": 0.5329,
+      "step": 745
+    },
+    {
+      "epoch": 4.749206349206349,
+      "grad_norm": 2.716587543487549,
+      "learning_rate": 4.9467470777043806e-05,
+      "loss": 0.5264,
+      "step": 750
+    },
+    {
+      "epoch": 4.780952380952381,
+      "grad_norm": 2.315419912338257,
+      "learning_rate": 4.9448283819526954e-05,
+      "loss": 0.4756,
+      "step": 755
+    },
+    {
+      "epoch": 4.8126984126984125,
+      "grad_norm": 2.1679515838623047,
+      "learning_rate": 4.9428761154696605e-05,
+      "loss": 0.4819,
+      "step": 760
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 3.389266014099121,
+      "learning_rate": 4.9408903050624796e-05,
+      "loss": 0.5121,
+      "step": 765
+    },
+    {
+      "epoch": 4.876190476190477,
+      "grad_norm": 3.4317383766174316,
+      "learning_rate": 4.938870977998959e-05,
+      "loss": 0.4535,
+      "step": 770
+    },
+    {
+      "epoch": 4.907936507936508,
+      "grad_norm": 2.9491918087005615,
+      "learning_rate": 4.9368181620071344e-05,
+      "loss": 0.5333,
+      "step": 775
+    },
+    {
+      "epoch": 4.93968253968254,
+      "grad_norm": 2.516798496246338,
+      "learning_rate": 4.934731885274887e-05,
+      "loss": 0.5367,
+      "step": 780
+    },
+    {
+      "epoch": 4.9714285714285715,
+      "grad_norm": 3.0031046867370605,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.4957,
+      "step": 785
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.334085702896118,
+      "learning_rate": 4.9304590646375614e-05,
+      "loss": 0.5287,
+      "step": 790
+    },
+    {
+      "epoch": 5.031746031746032,
+      "grad_norm": 1.9608453512191772,
+      "learning_rate": 4.928272579403969e-05,
+      "loss": 0.36,
+      "step": 795
+    },
+    {
+      "epoch": 5.063492063492063,
+      "grad_norm": 2.328850746154785,
+      "learning_rate": 4.92605275077212e-05,
+      "loss": 0.3628,
+      "step": 800
+    },
+    {
+      "epoch": 5.095238095238095,
+      "grad_norm": 2.3446412086486816,
+      "learning_rate": 4.923799609223202e-05,
+      "loss": 0.3327,
+      "step": 805
+    },
+    {
+      "epoch": 5.1269841269841265,
+      "grad_norm": 2.476181745529175,
+      "learning_rate": 4.921513185695831e-05,
+      "loss": 0.4246,
+      "step": 810
+    },
+    {
+      "epoch": 5.158730158730159,
+      "grad_norm": 3.1026763916015625,
+      "learning_rate": 4.91919351158563e-05,
+      "loss": 0.5048,
+      "step": 815
+    },
+    {
+      "epoch": 5.190476190476191,
+      "grad_norm": 2.8165297508239746,
+      "learning_rate": 4.916840618744798e-05,
+      "loss": 0.4361,
+      "step": 820
+    },
+    {
+      "epoch": 5.222222222222222,
+      "grad_norm": 1.8732138872146606,
+      "learning_rate": 4.9144545394816687e-05,
+      "loss": 0.4693,
+      "step": 825
+    },
+    {
+      "epoch": 5.253968253968254,
+      "grad_norm": 1.7250264883041382,
+      "learning_rate": 4.91203530656027e-05,
+      "loss": 0.4076,
+      "step": 830
+    },
+    {
+      "epoch": 5.285714285714286,
+      "grad_norm": 2.105459690093994,
+      "learning_rate": 4.9095829531998725e-05,
+      "loss": 0.3589,
+      "step": 835
+    },
+    {
+      "epoch": 5.317460317460317,
+      "grad_norm": 3.6825687885284424,
+      "learning_rate": 4.9070975130745387e-05,
+      "loss": 0.5263,
+      "step": 840
+    },
+    {
+      "epoch": 5.349206349206349,
+      "grad_norm": 2.947052001953125,
+      "learning_rate": 4.90457902031265e-05,
+      "loss": 0.4632,
+      "step": 845
+    },
+    {
+      "epoch": 5.380952380952381,
+      "grad_norm": 1.9546104669570923,
+      "learning_rate": 4.902027509496448e-05,
+      "loss": 0.4348,
+      "step": 850
+    },
+    {
+      "epoch": 5.412698412698413,
+      "grad_norm": 2.4471983909606934,
+      "learning_rate": 4.899443015661557e-05,
+      "loss": 0.4209,
+      "step": 855
+    },
+    {
+      "epoch": 5.444444444444445,
+      "grad_norm": 1.827124834060669,
+      "learning_rate": 4.8968255742964975e-05,
+      "loss": 0.413,
+      "step": 860
+    },
+    {
+      "epoch": 5.476190476190476,
+      "grad_norm": 2.654707431793213,
+      "learning_rate": 4.894175221342207e-05,
+      "loss": 0.432,
+      "step": 865
+    },
+    {
+      "epoch": 5.507936507936508,
+      "grad_norm": 2.648967981338501,
+      "learning_rate": 4.8914919931915407e-05,
+      "loss": 0.4339,
+      "step": 870
+    },
+    {
+      "epoch": 5.5396825396825395,
+      "grad_norm": 2.874075412750244,
+      "learning_rate": 4.888775926688775e-05,
+      "loss": 0.4392,
+      "step": 875
+    },
+    {
+      "epoch": 5.571428571428571,
+      "grad_norm": 2.9674830436706543,
+      "learning_rate": 4.8860270591291e-05,
+      "loss": 0.4459,
+      "step": 880
+    },
+    {
+      "epoch": 5.603174603174603,
+      "grad_norm": 2.054748296737671,
+      "learning_rate": 4.883245428258107e-05,
+      "loss": 0.4313,
+      "step": 885
+    },
+    {
+      "epoch": 5.634920634920634,
+      "grad_norm": 1.9174392223358154,
+      "learning_rate": 4.880431072271272e-05,
+      "loss": 0.3906,
+      "step": 890
+    },
+    {
+      "epoch": 5.666666666666667,
+      "grad_norm": 2.5257787704467773,
+      "learning_rate": 4.87758402981343e-05,
+      "loss": 0.4219,
+      "step": 895
+    },
+    {
+      "epoch": 5.698412698412699,
+      "grad_norm": 2.6365532875061035,
+      "learning_rate": 4.8747043399782424e-05,
+      "loss": 0.3978,
+      "step": 900
+    },
+    {
+      "epoch": 5.73015873015873,
+      "grad_norm": 2.0583746433258057,
+      "learning_rate": 4.871792042307667e-05,
+      "loss": 0.4847,
+      "step": 905
+    },
+    {
+      "epoch": 5.761904761904762,
+      "grad_norm": 2.035872459411621,
+      "learning_rate": 4.868847176791406e-05,
+      "loss": 0.4675,
+      "step": 910
+    },
+    {
+      "epoch": 5.7936507936507935,
+      "grad_norm": 2.3722939491271973,
+      "learning_rate": 4.8658697838663625e-05,
+      "loss": 0.4586,
+      "step": 915
+    },
+    {
+      "epoch": 5.825396825396825,
+      "grad_norm": 1.2609732151031494,
+      "learning_rate": 4.862859904416085e-05,
+      "loss": 0.3274,
+      "step": 920
+    },
+    {
+      "epoch": 5.857142857142857,
+      "grad_norm": 2.3673977851867676,
+      "learning_rate": 4.8598175797702036e-05,
+      "loss": 0.4685,
+      "step": 925
+    },
+    {
+      "epoch": 5.888888888888889,
+      "grad_norm": 2.8414175510406494,
+      "learning_rate": 4.856742851703866e-05,
+      "loss": 0.4762,
+      "step": 930
+    },
+    {
+      "epoch": 5.920634920634921,
+      "grad_norm": 2.4126765727996826,
+      "learning_rate": 4.853635762437159e-05,
+      "loss": 0.4075,
+      "step": 935
+    },
+    {
+      "epoch": 5.9523809523809526,
+      "grad_norm": 1.8691045045852661,
+      "learning_rate": 4.8504963546345334e-05,
+      "loss": 0.4865,
+      "step": 940
+    },
+    {
+      "epoch": 5.984126984126984,
+      "grad_norm": 3.5297420024871826,
+      "learning_rate": 4.8473246714042155e-05,
+      "loss": 0.4623,
+      "step": 945
+    },
+    {
+      "epoch": 6.012698412698413,
+      "grad_norm": 2.059169054031372,
+      "learning_rate": 4.844120756297617e-05,
+      "loss": 0.4164,
+      "step": 950
+    },
+    {
+      "epoch": 6.044444444444444,
+      "grad_norm": 2.4746127128601074,
+      "learning_rate": 4.840884653308735e-05,
+      "loss": 0.3552,
+      "step": 955
+    },
+    {
+      "epoch": 6.076190476190476,
+      "grad_norm": 2.504425287246704,
+      "learning_rate": 4.8376164068735485e-05,
+      "loss": 0.3368,
+      "step": 960
+    },
+    {
+      "epoch": 6.1079365079365076,
+      "grad_norm": 2.062577486038208,
+      "learning_rate": 4.83431606186941e-05,
+      "loss": 0.3139,
+      "step": 965
+    },
+    {
+      "epoch": 6.13968253968254,
+      "grad_norm": 2.4934544563293457,
+      "learning_rate": 4.830983663614427e-05,
+      "loss": 0.3777,
+      "step": 970
+    },
+    {
+      "epoch": 6.171428571428572,
+      "grad_norm": 2.5747485160827637,
+      "learning_rate": 4.827619257866839e-05,
+      "loss": 0.373,
+      "step": 975
+    },
+    {
+      "epoch": 6.203174603174603,
+      "grad_norm": 2.449357271194458,
+      "learning_rate": 4.8242228908243946e-05,
+      "loss": 0.3936,
+      "step": 980
+    },
+    {
+      "epoch": 6.234920634920635,
+      "grad_norm": 2.952680826187134,
+      "learning_rate": 4.82079460912371e-05,
+      "loss": 0.407,
+      "step": 985
+    },
+    {
+      "epoch": 6.266666666666667,
+      "grad_norm": 2.1754496097564697,
+      "learning_rate": 4.817334459839633e-05,
+      "loss": 0.3189,
+      "step": 990
+    },
+    {
+      "epoch": 6.298412698412698,
+      "grad_norm": 2.8406214714050293,
+      "learning_rate": 4.8138424904845947e-05,
+      "loss": 0.3883,
+      "step": 995
+    },
+    {
+      "epoch": 6.33015873015873,
+      "grad_norm": 1.7533257007598877,
+      "learning_rate": 4.8103187490079604e-05,
+      "loss": 0.3131,
+      "step": 1000
+    },
+    {
+      "epoch": 6.3619047619047615,
+      "grad_norm": 2.4574601650238037,
+      "learning_rate": 4.806763283795366e-05,
+      "loss": 0.3606,
+      "step": 1005
+    },
+    {
+      "epoch": 6.393650793650794,
+      "grad_norm": 2.002281427383423,
+      "learning_rate": 4.8031761436680575e-05,
+      "loss": 0.37,
+      "step": 1010
+    },
+    {
+      "epoch": 6.425396825396826,
+      "grad_norm": 2.823315143585205,
+      "learning_rate": 4.79955737788222e-05,
+      "loss": 0.3791,
+      "step": 1015
+    },
+    {
+      "epoch": 6.457142857142857,
+      "grad_norm": 2.7891204357147217,
+      "learning_rate": 4.795907036128299e-05,
+      "loss": 0.3556,
+      "step": 1020
+    },
+    {
+      "epoch": 6.488888888888889,
+      "grad_norm": 2.2387146949768066,
+      "learning_rate": 4.7922251685303213e-05,
+      "loss": 0.3929,
+      "step": 1025
+    },
+    {
+      "epoch": 6.520634920634921,
+      "grad_norm": 2.5023891925811768,
+      "learning_rate": 4.788511825645205e-05,
+      "loss": 0.379,
+      "step": 1030
+    },
+    {
+      "epoch": 6.552380952380952,
+      "grad_norm": 2.2654805183410645,
+      "learning_rate": 4.7847670584620653e-05,
+      "loss": 0.3435,
+      "step": 1035
+    },
+    {
+      "epoch": 6.584126984126984,
+      "grad_norm": 3.3823065757751465,
+      "learning_rate": 4.7809909184015146e-05,
+      "loss": 0.4109,
+      "step": 1040
+    },
+    {
+      "epoch": 6.6158730158730155,
+      "grad_norm": 2.6096551418304443,
+      "learning_rate": 4.7771834573149576e-05,
+      "loss": 0.4233,
+      "step": 1045
+    },
+    {
+      "epoch": 6.647619047619048,
+      "grad_norm": 2.3933897018432617,
+      "learning_rate": 4.773344727483876e-05,
+      "loss": 0.3709,
+      "step": 1050
+    },
+    {
+      "epoch": 6.67936507936508,
+      "grad_norm": 2.189544916152954,
+      "learning_rate": 4.769474781619114e-05,
+      "loss": 0.3287,
+      "step": 1055
+    },
+    {
+      "epoch": 6.711111111111111,
+      "grad_norm": 2.450892686843872,
+      "learning_rate": 4.765573672860154e-05,
+      "loss": 0.4022,
+      "step": 1060
+    },
+    {
+      "epoch": 6.742857142857143,
+      "grad_norm": 2.4342429637908936,
+      "learning_rate": 4.761641454774386e-05,
+      "loss": 0.4029,
+      "step": 1065
+    },
+    {
+      "epoch": 6.7746031746031745,
+      "grad_norm": 2.2122364044189453,
+      "learning_rate": 4.75767818135637e-05,
+      "loss": 0.3322,
+      "step": 1070
+    },
+    {
+      "epoch": 6.806349206349206,
+      "grad_norm": 3.968445301055908,
+      "learning_rate": 4.7536839070271e-05,
+      "loss": 0.3836,
+      "step": 1075
+    },
+    {
+      "epoch": 6.838095238095238,
+      "grad_norm": 3.529158353805542,
+      "learning_rate": 4.749658686633251e-05,
+      "loss": 0.4745,
+      "step": 1080
+    },
+    {
+      "epoch": 6.86984126984127,
+      "grad_norm": 2.430727243423462,
+      "learning_rate": 4.7456025754464304e-05,
+      "loss": 0.3664,
+      "step": 1085
+    },
+    {
+      "epoch": 6.901587301587302,
+      "grad_norm": 2.6552302837371826,
+      "learning_rate": 4.7415156291624166e-05,
+      "loss": 0.4359,
+      "step": 1090
+    },
+    {
+      "epoch": 6.933333333333334,
+      "grad_norm": 2.134822130203247,
+      "learning_rate": 4.737397903900393e-05,
+      "loss": 0.3969,
+      "step": 1095
+    },
+    {
+      "epoch": 6.965079365079365,
+      "grad_norm": 2.5052947998046875,
+      "learning_rate": 4.7332494562021815e-05,
+      "loss": 0.4069,
+      "step": 1100
+    },
+    {
+      "epoch": 6.996825396825397,
+      "grad_norm": 2.1377065181732178,
+      "learning_rate": 4.729070343031463e-05,
+      "loss": 0.3853,
+      "step": 1105
+    },
+    {
+      "epoch": 7.025396825396825,
+      "grad_norm": 1.9704042673110962,
+      "learning_rate": 4.724860621772995e-05,
+      "loss": 0.3283,
+      "step": 1110
+    },
+    {
+      "epoch": 7.057142857142857,
+      "grad_norm": 2.476968765258789,
+      "learning_rate": 4.7206203502318256e-05,
+      "loss": 0.3325,
+      "step": 1115
+    },
+    {
+      "epoch": 7.088888888888889,
+      "grad_norm": 1.9231969118118286,
+      "learning_rate": 4.716349586632499e-05,
+      "loss": 0.2876,
+      "step": 1120
+    },
+    {
+      "epoch": 7.12063492063492,
+      "grad_norm": 2.6444814205169678,
+      "learning_rate": 4.712048389618254e-05,
+      "loss": 0.3005,
+      "step": 1125
+    },
+    {
+      "epoch": 7.152380952380953,
+      "grad_norm": 3.2589964866638184,
+      "learning_rate": 4.7077168182502216e-05,
+      "loss": 0.4023,
+      "step": 1130
+    },
+    {
+      "epoch": 7.184126984126984,
+      "grad_norm": 2.5481936931610107,
+      "learning_rate": 4.703354932006615e-05,
+      "loss": 0.3302,
+      "step": 1135
+    },
+    {
+      "epoch": 7.215873015873016,
+      "grad_norm": 1.7125908136367798,
+      "learning_rate": 4.698962790781906e-05,
+      "loss": 0.3329,
+      "step": 1140
+    },
+    {
+      "epoch": 7.247619047619048,
+      "grad_norm": 2.2756667137145996,
+      "learning_rate": 4.6945404548860115e-05,
+      "loss": 0.3369,
+      "step": 1145
+    },
+    {
+      "epoch": 7.279365079365079,
+      "grad_norm": 2.9158453941345215,
+      "learning_rate": 4.6900879850434604e-05,
+      "loss": 0.3339,
+      "step": 1150
+    },
+    {
+      "epoch": 7.311111111111111,
+      "grad_norm": 2.3047537803649902,
+      "learning_rate": 4.685605442392559e-05,
+      "loss": 0.3915,
+      "step": 1155
+    },
+    {
+      "epoch": 7.3428571428571425,
+      "grad_norm": 2.7815029621124268,
+      "learning_rate": 4.681092888484554e-05,
+      "loss": 0.3317,
+      "step": 1160
+    },
+    {
+      "epoch": 7.374603174603175,
+      "grad_norm": 2.2644097805023193,
+      "learning_rate": 4.676550385282787e-05,
+      "loss": 0.3314,
+      "step": 1165
+    },
+    {
+      "epoch": 7.406349206349207,
+      "grad_norm": 2.5144474506378174,
+      "learning_rate": 4.671977995161843e-05,
+      "loss": 0.3188,
+      "step": 1170
+    },
+    {
+      "epoch": 7.438095238095238,
+      "grad_norm": 3.120821714401245,
+      "learning_rate": 4.667375780906693e-05,
+      "loss": 0.3523,
+      "step": 1175
+    },
+    {
+      "epoch": 7.46984126984127,
+      "grad_norm": 4.47842264175415,
+      "learning_rate": 4.662743805711832e-05,
+      "loss": 0.3611,
+      "step": 1180
+    },
+    {
+      "epoch": 7.501587301587302,
+      "grad_norm": 1.9228928089141846,
+      "learning_rate": 4.658082133180416e-05,
+      "loss": 0.3612,
+      "step": 1185
+    },
+    {
+      "epoch": 7.533333333333333,
+      "grad_norm": 2.1507537364959717,
+      "learning_rate": 4.6533908273233815e-05,
+      "loss": 0.3321,
+      "step": 1190
+    },
+    {
+      "epoch": 7.565079365079365,
+      "grad_norm": 2.1849119663238525,
+      "learning_rate": 4.64866995255857e-05,
+      "loss": 0.2943,
+      "step": 1195
+    },
+    {
+      "epoch": 7.5968253968253965,
+      "grad_norm": 2.1777775287628174,
+      "learning_rate": 4.643919573709843e-05,
+      "loss": 0.353,
+      "step": 1200
+    },
+    {
+      "epoch": 7.628571428571428,
+      "grad_norm": 2.5231118202209473,
+      "learning_rate": 4.639139756006195e-05,
+      "loss": 0.3571,
+      "step": 1205
+    },
+    {
+      "epoch": 7.660317460317461,
+      "grad_norm": 1.8409479856491089,
+      "learning_rate": 4.6343305650808516e-05,
+      "loss": 0.3691,
+      "step": 1210
+    },
+    {
+      "epoch": 7.692063492063492,
+      "grad_norm": 1.7940895557403564,
+      "learning_rate": 4.629492066970373e-05,
+      "loss": 0.3738,
+      "step": 1215
+    },
+    {
+      "epoch": 7.723809523809524,
+      "grad_norm": 2.014902114868164,
+      "learning_rate": 4.6246243281137474e-05,
+      "loss": 0.361,
+      "step": 1220
+    },
+    {
+      "epoch": 7.7555555555555555,
+      "grad_norm": 3.4182560443878174,
+      "learning_rate": 4.6197274153514735e-05,
+      "loss": 0.3663,
+      "step": 1225
+    },
+    {
+      "epoch": 7.787301587301587,
+      "grad_norm": 2.518728256225586,
+      "learning_rate": 4.614801395924649e-05,
+      "loss": 0.3646,
+      "step": 1230
+    },
+    {
+      "epoch": 7.819047619047619,
+      "grad_norm": 2.154189109802246,
+      "learning_rate": 4.6098463374740466e-05,
+      "loss": 0.3331,
+      "step": 1235
+    },
+    {
+      "epoch": 7.85079365079365,
+      "grad_norm": 2.536081075668335,
+      "learning_rate": 4.604862308039177e-05,
+      "loss": 0.3742,
+      "step": 1240
+    },
+    {
+      "epoch": 7.882539682539683,
+      "grad_norm": 2.340764045715332,
+      "learning_rate": 4.599849376057366e-05,
+      "loss": 0.3352,
+      "step": 1245
+    },
+    {
+      "epoch": 7.914285714285715,
+      "grad_norm": 3.5488364696502686,
+      "learning_rate": 4.5948076103628094e-05,
+      "loss": 0.3663,
+      "step": 1250
+    },
+    {
+      "epoch": 7.946031746031746,
+      "grad_norm": 2.779360294342041,
+      "learning_rate": 4.589737080185625e-05,
+      "loss": 0.3362,
+      "step": 1255
+    },
+    {
+      "epoch": 7.977777777777778,
+      "grad_norm": 1.8792667388916016,
+      "learning_rate": 4.5846378551509097e-05,
+      "loss": 0.346,
+      "step": 1260
+    },
+    {
+      "epoch": 8.006349206349206,
+      "grad_norm": 2.453295946121216,
+      "learning_rate": 4.579510005277774e-05,
+      "loss": 0.3509,
+      "step": 1265
+    },
+    {
+      "epoch": 8.038095238095238,
+      "grad_norm": 1.9493130445480347,
+      "learning_rate": 4.574353600978388e-05,
+      "loss": 0.3062,
+      "step": 1270
+    },
+    {
+      "epoch": 8.06984126984127,
+      "grad_norm": 1.9360930919647217,
+      "learning_rate": 4.56916871305701e-05,
+      "loss": 0.3056,
+      "step": 1275
+    },
+    {
+      "epoch": 8.101587301587301,
+      "grad_norm": 1.5592070817947388,
+      "learning_rate": 4.563955412709021e-05,
+      "loss": 0.2785,
+      "step": 1280
+    },
+    {
+      "epoch": 8.133333333333333,
+      "grad_norm": 1.8093425035476685,
+      "learning_rate": 4.5587137715199354e-05,
+      "loss": 0.308,
+      "step": 1285
+    },
+    {
+      "epoch": 8.165079365079364,
+      "grad_norm": 2.2939181327819824,
+      "learning_rate": 4.5534438614644294e-05,
+      "loss": 0.3038,
+      "step": 1290
+    },
+    {
+      "epoch": 8.196825396825396,
+      "grad_norm": 2.4204866886138916,
+      "learning_rate": 4.548145754905346e-05,
+      "loss": 0.3375,
+      "step": 1295
+    },
+    {
+      "epoch": 8.228571428571428,
+      "grad_norm": 1.725534439086914,
+      "learning_rate": 4.5428195245927064e-05,
+      "loss": 0.3101,
+      "step": 1300
+    },
+    {
+      "epoch": 8.260317460317461,
+      "grad_norm": 1.637730360031128,
+      "learning_rate": 4.537465243662704e-05,
+      "loss": 0.2931,
+      "step": 1305
+    },
+    {
+      "epoch": 8.292063492063493,
+      "grad_norm": 1.3372169733047485,
+      "learning_rate": 4.532082985636709e-05,
+      "loss": 0.2763,
+      "step": 1310
+    },
+    {
+      "epoch": 8.323809523809524,
+      "grad_norm": 2.5993168354034424,
+      "learning_rate": 4.5266728244202494e-05,
+      "loss": 0.3458,
+      "step": 1315
+    },
+    {
+      "epoch": 8.355555555555556,
+      "grad_norm": 2.461862564086914,
+      "learning_rate": 4.521234834302006e-05,
+      "loss": 0.3693,
+      "step": 1320
+    },
+    {
+      "epoch": 8.387301587301588,
+      "grad_norm": 1.8519413471221924,
+      "learning_rate": 4.5157690899527816e-05,
+      "loss": 0.3327,
+      "step": 1325
+    },
+    {
+      "epoch": 8.41904761904762,
+      "grad_norm": 2.1535580158233643,
+      "learning_rate": 4.510275666424487e-05,
+      "loss": 0.3229,
+      "step": 1330
+    },
+    {
+      "epoch": 8.450793650793651,
+      "grad_norm": 1.6819690465927124,
+      "learning_rate": 4.5047546391491e-05,
+      "loss": 0.2925,
+      "step": 1335
+    },
+    {
+      "epoch": 8.482539682539683,
+      "grad_norm": 1.6538281440734863,
+      "learning_rate": 4.499206083937638e-05,
+      "loss": 0.3218,
+      "step": 1340
+    },
+    {
+      "epoch": 8.514285714285714,
+      "grad_norm": 1.8956862688064575,
+      "learning_rate": 4.493630076979112e-05,
+      "loss": 0.3423,
+      "step": 1345
+    },
+    {
+      "epoch": 8.546031746031746,
+      "grad_norm": 2.274681806564331,
+      "learning_rate": 4.48802669483948e-05,
+      "loss": 0.3152,
+      "step": 1350
+    },
+    {
+      "epoch": 8.577777777777778,
+      "grad_norm": 2.2956337928771973,
+      "learning_rate": 4.4823960144606014e-05,
+      "loss": 0.3417,
+      "step": 1355
+    },
+    {
+      "epoch": 8.60952380952381,
+      "grad_norm": 1.8650286197662354,
+      "learning_rate": 4.4767381131591734e-05,
+      "loss": 0.2896,
+      "step": 1360
+    },
+    {
+      "epoch": 8.64126984126984,
+      "grad_norm": 1.3998652696609497,
+      "learning_rate": 4.471053068625674e-05,
+      "loss": 0.3372,
+      "step": 1365
+    },
+    {
+      "epoch": 8.673015873015872,
+      "grad_norm": 2.855074167251587,
+      "learning_rate": 4.465340958923293e-05,
+      "loss": 0.332,
+      "step": 1370
+    },
+    {
+      "epoch": 8.704761904761904,
+      "grad_norm": 1.6865357160568237,
+      "learning_rate": 4.459601862486862e-05,
+      "loss": 0.3053,
+      "step": 1375
+    },
+    {
+      "epoch": 8.736507936507937,
+      "grad_norm": 2.501856803894043,
+      "learning_rate": 4.453835858121773e-05,
+      "loss": 0.3119,
+      "step": 1380
+    },
+    {
+      "epoch": 8.768253968253969,
+      "grad_norm": 2.4325456619262695,
+      "learning_rate": 4.4480430250029046e-05,
+      "loss": 0.3395,
+      "step": 1385
+    },
+    {
+      "epoch": 8.8,
+      "grad_norm": 1.4845948219299316,
+      "learning_rate": 4.4422234426735256e-05,
+      "loss": 0.3237,
+      "step": 1390
+    },
+    {
+      "epoch": 8.831746031746032,
+      "grad_norm": 1.3553249835968018,
+      "learning_rate": 4.436377191044208e-05,
+      "loss": 0.3387,
+      "step": 1395
+    },
+    {
+      "epoch": 8.863492063492064,
+      "grad_norm": 1.8338890075683594,
+      "learning_rate": 4.430504350391729e-05,
+      "loss": 0.3618,
+      "step": 1400
+    },
+    {
+      "epoch": 8.895238095238096,
+      "grad_norm": 2.291538953781128,
+      "learning_rate": 4.4246050013579686e-05,
+      "loss": 0.3608,
+      "step": 1405
+    },
+    {
+      "epoch": 8.926984126984127,
+      "grad_norm": 1.3809788227081299,
+      "learning_rate": 4.4186792249488005e-05,
+      "loss": 0.3077,
+      "step": 1410
+    },
+    {
+      "epoch": 8.958730158730159,
+      "grad_norm": 1.5944230556488037,
+      "learning_rate": 4.412727102532983e-05,
+      "loss": 0.3307,
+      "step": 1415
+    },
+    {
+      "epoch": 8.99047619047619,
+      "grad_norm": 2.2244362831115723,
+      "learning_rate": 4.4067487158410396e-05,
+      "loss": 0.3469,
+      "step": 1420
+    },
+    {
+      "epoch": 9.019047619047619,
+      "grad_norm": 1.444221019744873,
+      "learning_rate": 4.400744146964136e-05,
+      "loss": 0.3049,
+      "step": 1425
+    },
+    {
+      "epoch": 9.05079365079365,
+      "grad_norm": 1.5847752094268799,
+      "learning_rate": 4.394713478352955e-05,
+      "loss": 0.2715,
+      "step": 1430
+    },
+    {
+      "epoch": 9.082539682539682,
+      "grad_norm": 1.6062681674957275,
+      "learning_rate": 4.388656792816562e-05,
+      "loss": 0.2487,
+      "step": 1435
+    },
+    {
+      "epoch": 9.114285714285714,
+      "grad_norm": 2.099787712097168,
+      "learning_rate": 4.382574173521272e-05,
+      "loss": 0.2866,
+      "step": 1440
+    },
+    {
+      "epoch": 9.146031746031746,
+      "grad_norm": 1.0997334718704224,
+      "learning_rate": 4.376465703989502e-05,
+      "loss": 0.3052,
+      "step": 1445
+    },
+    {
+      "epoch": 9.177777777777777,
+      "grad_norm": 2.4327454566955566,
+      "learning_rate": 4.370331468098628e-05,
+      "loss": 0.3212,
+      "step": 1450
+    },
+    {
+      "epoch": 9.209523809523809,
+      "grad_norm": 1.4816385507583618,
+      "learning_rate": 4.364171550079833e-05,
+      "loss": 0.3046,
+      "step": 1455
+    },
+    {
+      "epoch": 9.24126984126984,
+      "grad_norm": 2.039186716079712,
+      "learning_rate": 4.357986034516947e-05,
+      "loss": 0.3165,
+      "step": 1460
+    },
+    {
+      "epoch": 9.273015873015874,
+      "grad_norm": 1.437852382659912,
+      "learning_rate": 4.3517750063452934e-05,
+      "loss": 0.3037,
+      "step": 1465
+    },
+    {
+      "epoch": 9.304761904761905,
+      "grad_norm": 1.818982720375061,
+      "learning_rate": 4.345538550850512e-05,
+      "loss": 0.3122,
+      "step": 1470
+    },
+    {
+      "epoch": 9.336507936507937,
+      "grad_norm": 1.12025785446167,
+      "learning_rate": 4.339276753667395e-05,
+      "loss": 0.2909,
+      "step": 1475
+    },
+    {
+      "epoch": 9.368253968253969,
+      "grad_norm": 1.6094844341278076,
+      "learning_rate": 4.3329897007787125e-05,
+      "loss": 0.2823,
+      "step": 1480
+    },
+    {
+      "epoch": 9.4,
+      "grad_norm": 1.916200041770935,
+      "learning_rate": 4.326677478514024e-05,
+      "loss": 0.2939,
+      "step": 1485
+    },
+    {
+      "epoch": 9.431746031746032,
+      "grad_norm": 1.97919499874115,
+      "learning_rate": 4.320340173548503e-05,
+      "loss": 0.2826,
+      "step": 1490
+    },
+    {
+      "epoch": 9.463492063492064,
+      "grad_norm": 2.0238938331604004,
+      "learning_rate": 4.313977872901737e-05,
+      "loss": 0.3273,
+      "step": 1495
+    },
+    {
+      "epoch": 9.495238095238095,
+      "grad_norm": 2.5840957164764404,
+      "learning_rate": 4.307590663936541e-05,
+      "loss": 0.2889,
+      "step": 1500
+    },
+    {
+      "epoch": 9.526984126984127,
+      "grad_norm": 2.3503904342651367,
+      "learning_rate": 4.30117863435775e-05,
+      "loss": 0.3012,
+      "step": 1505
+    },
+    {
+      "epoch": 9.558730158730159,
+      "grad_norm": 2.019792318344116,
+      "learning_rate": 4.294741872211024e-05,
+      "loss": 0.3267,
+      "step": 1510
+    },
+    {
+      "epoch": 9.59047619047619,
+      "grad_norm": 2.2713353633880615,
+      "learning_rate": 4.288280465881632e-05,
+      "loss": 0.3096,
+      "step": 1515
+    },
+    {
+      "epoch": 9.622222222222222,
+      "grad_norm": 2.4236693382263184,
+      "learning_rate": 4.281794504093237e-05,
+      "loss": 0.3291,
+      "step": 1520
+    },
+    {
+      "epoch": 9.653968253968253,
+      "grad_norm": 1.772703766822815,
+      "learning_rate": 4.275284075906686e-05,
+      "loss": 0.3117,
+      "step": 1525
+    },
+    {
+      "epoch": 9.685714285714285,
+      "grad_norm": 1.9665186405181885,
+      "learning_rate": 4.268749270718778e-05,
+      "loss": 0.326,
+      "step": 1530
+    },
+    {
+      "epoch": 9.717460317460317,
+      "grad_norm": 1.9472782611846924,
+      "learning_rate": 4.262190178261044e-05,
+      "loss": 0.2683,
+      "step": 1535
+    },
+    {
+      "epoch": 9.74920634920635,
+      "grad_norm": 2.0638089179992676,
+      "learning_rate": 4.255606888598508e-05,
+      "loss": 0.314,
+      "step": 1540
+    },
+    {
+      "epoch": 9.780952380952382,
+      "grad_norm": 2.1349925994873047,
+      "learning_rate": 4.248999492128456e-05,
+      "loss": 0.2897,
+      "step": 1545
+    },
+    {
+      "epoch": 9.812698412698413,
+      "grad_norm": 2.112536907196045,
+      "learning_rate": 4.242368079579192e-05,
+      "loss": 0.31,
+      "step": 1550
+    },
+    {
+      "epoch": 9.844444444444445,
+      "grad_norm": 1.6859878301620483,
+      "learning_rate": 4.2357127420087917e-05,
+      "loss": 0.3412,
+      "step": 1555
+    },
+    {
+      "epoch": 9.876190476190477,
+      "grad_norm": 1.9178651571273804,
+      "learning_rate": 4.229033570803853e-05,
+      "loss": 0.334,
+      "step": 1560
+    },
+    {
+      "epoch": 9.907936507936508,
+      "grad_norm": 2.562436103820801,
+      "learning_rate": 4.2223306576782426e-05,
+      "loss": 0.3379,
+      "step": 1565
+    },
+    {
+      "epoch": 9.93968253968254,
+      "grad_norm": 1.8472412824630737,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.3415,
+      "step": 1570
+    },
+    {
+      "epoch": 9.971428571428572,
+      "grad_norm": 1.9416279792785645,
+      "learning_rate": 4.208853974149246e-05,
+      "loss": 0.3085,
+      "step": 1575
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 2.0056397914886475,
+      "learning_rate": 4.202080388798571e-05,
+      "loss": 0.3263,
+      "step": 1580
+    },
+    {
+      "epoch": 10.031746031746032,
+      "grad_norm": 2.195781946182251,
+      "learning_rate": 4.1952834316301065e-05,
+      "loss": 0.2867,
+      "step": 1585
+    },
+    {
+      "epoch": 10.063492063492063,
+      "grad_norm": 1.7489805221557617,
+      "learning_rate": 4.1884631959750766e-05,
+      "loss": 0.2589,
+      "step": 1590
+    },
+    {
+      "epoch": 10.095238095238095,
+      "grad_norm": 1.9361369609832764,
+      "learning_rate": 4.181619775484348e-05,
+      "loss": 0.2722,
+      "step": 1595
+    },
+    {
+      "epoch": 10.126984126984127,
+      "grad_norm": 2.24322509765625,
+      "learning_rate": 4.174753264127147e-05,
+      "loss": 0.2534,
+      "step": 1600
+    },
+    {
+      "epoch": 10.158730158730158,
+      "grad_norm": 2.4550466537475586,
+      "learning_rate": 4.167863756189767e-05,
+      "loss": 0.2777,
+      "step": 1605
+    },
+    {
+      "epoch": 10.19047619047619,
+      "grad_norm": 1.9439811706542969,
+      "learning_rate": 4.160951346274278e-05,
+      "loss": 0.2864,
+      "step": 1610
+    },
+    {
+      "epoch": 10.222222222222221,
+      "grad_norm": 1.633494257926941,
+      "learning_rate": 4.154016129297219e-05,
+      "loss": 0.2858,
+      "step": 1615
+    },
+    {
+      "epoch": 10.253968253968253,
+      "grad_norm": 1.69782292842865,
+      "learning_rate": 4.147058200488305e-05,
+      "loss": 0.2942,
+      "step": 1620
+    },
+    {
+      "epoch": 10.285714285714286,
+      "grad_norm": 1.613031268119812,
+      "learning_rate": 4.140077655389113e-05,
+      "loss": 0.2632,
+      "step": 1625
+    },
+    {
+      "epoch": 10.317460317460318,
+      "grad_norm": 2.0266177654266357,
+      "learning_rate": 4.1330745898517714e-05,
+      "loss": 0.3011,
+      "step": 1630
+    },
+    {
+      "epoch": 10.34920634920635,
+      "grad_norm": 1.8945387601852417,
+      "learning_rate": 4.1260491000376446e-05,
+      "loss": 0.2832,
+      "step": 1635
+    },
+    {
+      "epoch": 10.380952380952381,
+      "grad_norm": 1.7012510299682617,
+      "learning_rate": 4.119001282416009e-05,
+      "loss": 0.2718,
+      "step": 1640
+    },
+    {
+      "epoch": 10.412698412698413,
+      "grad_norm": 1.5538525581359863,
+      "learning_rate": 4.111931233762738e-05,
+      "loss": 0.3232,
+      "step": 1645
+    },
+    {
+      "epoch": 10.444444444444445,
+      "grad_norm": 2.3083150386810303,
+      "learning_rate": 4.1048390511589595e-05,
+      "loss": 0.3057,
+      "step": 1650
+    },
+    {
+      "epoch": 10.476190476190476,
+      "grad_norm": 1.293314814567566,
+      "learning_rate": 4.097724831989733e-05,
+      "loss": 0.2523,
+      "step": 1655
+    },
+    {
+      "epoch": 10.507936507936508,
+      "grad_norm": 2.517212152481079,
+      "learning_rate": 4.09058867394271e-05,
+      "loss": 0.3269,
+      "step": 1660
+    },
+    {
+      "epoch": 10.53968253968254,
+      "grad_norm": 2.057063102722168,
+      "learning_rate": 4.083430675006791e-05,
+      "loss": 0.2844,
+      "step": 1665
+    },
+    {
+      "epoch": 10.571428571428571,
+      "grad_norm": 1.5663833618164062,
+      "learning_rate": 4.0762509334707786e-05,
+      "loss": 0.3005,
+      "step": 1670
+    },
+    {
+      "epoch": 10.603174603174603,
+      "grad_norm": 2.5423505306243896,
+      "learning_rate": 4.069049547922035e-05,
+      "loss": 0.2802,
+      "step": 1675
+    },
+    {
+      "epoch": 10.634920634920634,
+      "grad_norm": 1.578316569328308,
+      "learning_rate": 4.061826617245119e-05,
+      "loss": 0.2667,
+      "step": 1680
+    },
+    {
+      "epoch": 10.666666666666666,
+      "grad_norm": 1.502928376197815,
+      "learning_rate": 4.0545822406204334e-05,
+      "loss": 0.3059,
+      "step": 1685
+    },
+    {
+      "epoch": 10.698412698412698,
+      "grad_norm": 1.2470905780792236,
+      "learning_rate": 4.047316517522864e-05,
+      "loss": 0.2879,
+      "step": 1690
+    },
+    {
+      "epoch": 10.73015873015873,
+      "grad_norm": 1.8238775730133057,
+      "learning_rate": 4.0400295477204105e-05,
+      "loss": 0.2923,
+      "step": 1695
+    },
+    {
+      "epoch": 10.761904761904763,
+      "grad_norm": 2.0516586303710938,
+      "learning_rate": 4.032721431272819e-05,
+      "loss": 0.3086,
+      "step": 1700
+    },
+    {
+      "epoch": 10.793650793650794,
+      "grad_norm": 1.3188791275024414,
+      "learning_rate": 4.0253922685302046e-05,
+      "loss": 0.2893,
+      "step": 1705
+    },
+    {
+      "epoch": 10.825396825396826,
+      "grad_norm": 1.7352266311645508,
+      "learning_rate": 4.01804216013168e-05,
+      "loss": 0.2981,
+      "step": 1710
+    },
+    {
+      "epoch": 10.857142857142858,
+      "grad_norm": 1.3449515104293823,
+      "learning_rate": 4.0106712070039656e-05,
+      "loss": 0.2841,
+      "step": 1715
+    },
+    {
+      "epoch": 10.88888888888889,
+      "grad_norm": 2.505431890487671,
+      "learning_rate": 4.00327951036001e-05,
+      "loss": 0.3034,
+      "step": 1720
+    },
+    {
+      "epoch": 10.920634920634921,
+      "grad_norm": 1.8870325088500977,
+      "learning_rate": 3.9958671716975966e-05,
+      "loss": 0.305,
+      "step": 1725
+    },
+    {
+      "epoch": 10.952380952380953,
+      "grad_norm": 2.913130044937134,
+      "learning_rate": 3.988434292797951e-05,
+      "loss": 0.3212,
+      "step": 1730
+    },
+    {
+      "epoch": 10.984126984126984,
+      "grad_norm": 1.7870115041732788,
+      "learning_rate": 3.980980975724344e-05,
+      "loss": 0.3108,
+      "step": 1735
+    },
+    {
+      "epoch": 11.012698412698413,
+      "grad_norm": 3.050985336303711,
+      "learning_rate": 3.9735073228206896e-05,
+      "loss": 0.3043,
+      "step": 1740
+    },
+    {
+      "epoch": 11.044444444444444,
+      "grad_norm": 1.5993611812591553,
+      "learning_rate": 3.96601343671014e-05,
+      "loss": 0.2465,
+      "step": 1745
+    },
+    {
+      "epoch": 11.076190476190476,
+      "grad_norm": 1.626888632774353,
+      "learning_rate": 3.9584994202936746e-05,
+      "loss": 0.2688,
+      "step": 1750
+    },
+    {
+      "epoch": 11.107936507936508,
+      "grad_norm": 1.7132880687713623,
+      "learning_rate": 3.950965376748689e-05,
+      "loss": 0.2458,
+      "step": 1755
+    },
+    {
+      "epoch": 11.13968253968254,
+      "grad_norm": 1.7764930725097656,
+      "learning_rate": 3.94341140952758e-05,
+      "loss": 0.2189,
+      "step": 1760
+    },
+    {
+      "epoch": 11.17142857142857,
+      "grad_norm": 2.5560712814331055,
+      "learning_rate": 3.9358376223563206e-05,
+      "loss": 0.2866,
+      "step": 1765
+    },
+    {
+      "epoch": 11.203174603174602,
+      "grad_norm": 1.1177359819412231,
+      "learning_rate": 3.928244119233038e-05,
+      "loss": 0.233,
+      "step": 1770
+    },
+    {
+      "epoch": 11.234920634920634,
+      "grad_norm": 1.584670901298523,
+      "learning_rate": 3.9206310044265866e-05,
+      "loss": 0.273,
+      "step": 1775
+    },
+    {
+      "epoch": 11.266666666666667,
+      "grad_norm": 1.6278687715530396,
+      "learning_rate": 3.912998382475115e-05,
+      "loss": 0.2746,
+      "step": 1780
+    },
+    {
+      "epoch": 11.2984126984127,
+      "grad_norm": 1.657038688659668,
+      "learning_rate": 3.905346358184629e-05,
+      "loss": 0.2885,
+      "step": 1785
+    },
+    {
+      "epoch": 11.33015873015873,
+      "grad_norm": 1.2840272188186646,
+      "learning_rate": 3.897675036627557e-05,
+      "loss": 0.2932,
+      "step": 1790
+    },
+    {
+      "epoch": 11.361904761904762,
+      "grad_norm": 1.5678766965866089,
+      "learning_rate": 3.8899845231413026e-05,
+      "loss": 0.2945,
+      "step": 1795
+    },
+    {
+      "epoch": 11.393650793650794,
+      "grad_norm": 1.788948655128479,
+      "learning_rate": 3.8822749233268006e-05,
+      "loss": 0.3013,
+      "step": 1800
+    },
+    {
+      "epoch": 11.425396825396826,
+      "grad_norm": 1.2259769439697266,
+      "learning_rate": 3.8745463430470664e-05,
+      "loss": 0.2582,
+      "step": 1805
+    },
+    {
+      "epoch": 11.457142857142857,
+      "grad_norm": 1.5430735349655151,
+      "learning_rate": 3.866798888425741e-05,
+      "loss": 0.275,
+      "step": 1810
+    },
+    {
+      "epoch": 11.488888888888889,
+      "grad_norm": 1.9102168083190918,
+      "learning_rate": 3.8590326658456376e-05,
+      "loss": 0.2909,
+      "step": 1815
+    },
+    {
+      "epoch": 11.52063492063492,
+      "grad_norm": 1.6118320226669312,
+      "learning_rate": 3.851247781947277e-05,
+      "loss": 0.2922,
+      "step": 1820
+    },
+    {
+      "epoch": 11.552380952380952,
+      "grad_norm": 1.393646478652954,
+      "learning_rate": 3.843444343627424e-05,
+      "loss": 0.2783,
+      "step": 1825
+    },
+    {
+      "epoch": 11.584126984126984,
+      "grad_norm": 2.522909641265869,
+      "learning_rate": 3.83562245803762e-05,
+      "loss": 0.2933,
+      "step": 1830
+    },
+    {
+      "epoch": 11.615873015873015,
+      "grad_norm": 2.2534332275390625,
+      "learning_rate": 3.827782232582714e-05,
+      "loss": 0.3081,
+      "step": 1835
+    },
+    {
+      "epoch": 11.647619047619047,
+      "grad_norm": 2.4088056087493896,
+      "learning_rate": 3.819923774919383e-05,
+      "loss": 0.276,
+      "step": 1840
+    },
+    {
+      "epoch": 11.679365079365079,
+      "grad_norm": 1.7626562118530273,
+      "learning_rate": 3.8120471929546576e-05,
+      "loss": 0.2697,
+      "step": 1845
+    },
+    {
+      "epoch": 11.71111111111111,
+      "grad_norm": 1.8656691312789917,
+      "learning_rate": 3.8041525948444414e-05,
+      "loss": 0.2979,
+      "step": 1850
+    },
+    {
+      "epoch": 11.742857142857144,
+      "grad_norm": 1.6537258625030518,
+      "learning_rate": 3.7962400889920185e-05,
+      "loss": 0.3042,
+      "step": 1855
+    },
+    {
+      "epoch": 11.774603174603175,
+      "grad_norm": 1.696975827217102,
+      "learning_rate": 3.788309784046574e-05,
+      "loss": 0.2984,
+      "step": 1860
+    },
+    {
+      "epoch": 11.806349206349207,
+      "grad_norm": 1.976236343383789,
+      "learning_rate": 3.780361788901696e-05,
+      "loss": 0.2711,
+      "step": 1865
+    },
+    {
+      "epoch": 11.838095238095239,
+      "grad_norm": 1.7781676054000854,
+      "learning_rate": 3.772396212693885e-05,
+      "loss": 0.3116,
+      "step": 1870
+    },
+    {
+      "epoch": 11.86984126984127,
+      "grad_norm": 1.6252080202102661,
+      "learning_rate": 3.7644131648010494e-05,
+      "loss": 0.2879,
+      "step": 1875
+    },
+    {
+      "epoch": 11.901587301587302,
+      "grad_norm": 1.7511012554168701,
+      "learning_rate": 3.75641275484101e-05,
+      "loss": 0.3106,
+      "step": 1880
+    },
+    {
+      "epoch": 11.933333333333334,
+      "grad_norm": 1.5861046314239502,
+      "learning_rate": 3.7483950926699885e-05,
+      "loss": 0.2703,
+      "step": 1885
+    },
+    {
+      "epoch": 11.965079365079365,
+      "grad_norm": 1.903902530670166,
+      "learning_rate": 3.740360288381105e-05,
+      "loss": 0.2808,
+      "step": 1890
+    },
+    {
+      "epoch": 11.996825396825397,
+      "grad_norm": 1.644740104675293,
+      "learning_rate": 3.732308452302864e-05,
+      "loss": 0.2883,
+      "step": 1895
+    },
+    {
+      "epoch": 12.025396825396825,
+      "grad_norm": 1.3671797513961792,
+      "learning_rate": 3.724239694997637e-05,
+      "loss": 0.2661,
+      "step": 1900
+    },
+    {
+      "epoch": 12.057142857142857,
+      "grad_norm": 1.1910501718521118,
+      "learning_rate": 3.716154127260147e-05,
+      "loss": 0.2352,
+      "step": 1905
+    },
+    {
+      "epoch": 12.088888888888889,
+      "grad_norm": 1.45137619972229,
+      "learning_rate": 3.708051860115947e-05,
+      "loss": 0.2703,
+      "step": 1910
+    },
+    {
+      "epoch": 12.12063492063492,
+      "grad_norm": 2.625089645385742,
+      "learning_rate": 3.699933004819895e-05,
+      "loss": 0.2524,
+      "step": 1915
+    },
+    {
+      "epoch": 12.152380952380952,
+      "grad_norm": 1.731602430343628,
+      "learning_rate": 3.691797672854625e-05,
+      "loss": 0.2533,
+      "step": 1920
+    },
+    {
+      "epoch": 12.184126984126983,
+      "grad_norm": 2.1266143321990967,
+      "learning_rate": 3.683645975929019e-05,
+      "loss": 0.2666,
+      "step": 1925
+    },
+    {
+      "epoch": 12.215873015873015,
+      "grad_norm": 1.7163398265838623,
+      "learning_rate": 3.675478025976671e-05,
+      "loss": 0.2838,
+      "step": 1930
+    },
+    {
+      "epoch": 12.247619047619047,
+      "grad_norm": 1.8726729154586792,
+      "learning_rate": 3.66729393515435e-05,
+      "loss": 0.2646,
+      "step": 1935
+    },
+    {
+      "epoch": 12.27936507936508,
+      "grad_norm": 1.2229722738265991,
+      "learning_rate": 3.659093815840462e-05,
+      "loss": 0.267,
+      "step": 1940
+    },
+    {
+      "epoch": 12.311111111111112,
+      "grad_norm": 1.341997504234314,
+      "learning_rate": 3.650877780633505e-05,
+      "loss": 0.2464,
+      "step": 1945
+    },
+    {
+      "epoch": 12.342857142857143,
+      "grad_norm": 1.3466174602508545,
+      "learning_rate": 3.6426459423505214e-05,
+      "loss": 0.2521,
+      "step": 1950
+    },
+    {
+      "epoch": 12.374603174603175,
+      "grad_norm": 1.7022134065628052,
+      "learning_rate": 3.6343984140255516e-05,
+      "loss": 0.2662,
+      "step": 1955
+    },
+    {
+      "epoch": 12.406349206349207,
+      "grad_norm": 2.5058987140655518,
+      "learning_rate": 3.626135308908084e-05,
+      "loss": 0.2745,
+      "step": 1960
+    },
+    {
+      "epoch": 12.438095238095238,
+      "grad_norm": 1.7667903900146484,
+      "learning_rate": 3.6178567404614936e-05,
+      "loss": 0.2752,
+      "step": 1965
+    },
+    {
+      "epoch": 12.46984126984127,
+      "grad_norm": 1.7334171533584595,
+      "learning_rate": 3.609562822361487e-05,
+      "loss": 0.2667,
+      "step": 1970
+    },
+    {
+      "epoch": 12.501587301587302,
+      "grad_norm": 1.7093929052352905,
+      "learning_rate": 3.601253668494546e-05,
+      "loss": 0.2829,
+      "step": 1975
+    },
+    {
+      "epoch": 12.533333333333333,
+      "grad_norm": 1.8272080421447754,
+      "learning_rate": 3.592929392956355e-05,
+      "loss": 0.2583,
+      "step": 1980
+    },
+    {
+      "epoch": 12.565079365079365,
+      "grad_norm": 1.5226343870162964,
+      "learning_rate": 3.584590110050241e-05,
+      "loss": 0.2652,
+      "step": 1985
+    },
+    {
+      "epoch": 12.596825396825396,
+      "grad_norm": 1.7395323514938354,
+      "learning_rate": 3.5762359342856036e-05,
+      "loss": 0.2585,
+      "step": 1990
+    },
+    {
+      "epoch": 12.628571428571428,
+      "grad_norm": 2.278144359588623,
+      "learning_rate": 3.567866980376337e-05,
+      "loss": 0.2862,
+      "step": 1995
+    },
+    {
+      "epoch": 12.66031746031746,
+      "grad_norm": 1.9335429668426514,
+      "learning_rate": 3.559483363239262e-05,
+      "loss": 0.2945,
+      "step": 2000
+    },
+    {
+      "epoch": 12.692063492063491,
+      "grad_norm": 1.3866902589797974,
+      "learning_rate": 3.551085197992545e-05,
+      "loss": 0.2675,
+      "step": 2005
+    },
+    {
+      "epoch": 12.723809523809523,
+      "grad_norm": 1.7110604047775269,
+      "learning_rate": 3.5426725999541174e-05,
+      "loss": 0.2705,
+      "step": 2010
+    },
+    {
+      "epoch": 12.755555555555556,
+      "grad_norm": 1.6411776542663574,
+      "learning_rate": 3.534245684640089e-05,
+      "loss": 0.2482,
+      "step": 2015
+    },
+    {
+      "epoch": 12.787301587301588,
+      "grad_norm": 1.4663981199264526,
+      "learning_rate": 3.525804567763167e-05,
+      "loss": 0.2838,
+      "step": 2020
+    },
+    {
+      "epoch": 12.81904761904762,
+      "grad_norm": 1.8676140308380127,
+      "learning_rate": 3.517349365231065e-05,
+      "loss": 0.2906,
+      "step": 2025
+    },
+    {
+      "epoch": 12.850793650793651,
+      "grad_norm": 1.6150639057159424,
+      "learning_rate": 3.508880193144911e-05,
+      "loss": 0.3011,
+      "step": 2030
+    },
+    {
+      "epoch": 12.882539682539683,
+      "grad_norm": 2.5578598976135254,
+      "learning_rate": 3.500397167797654e-05,
+      "loss": 0.2846,
+      "step": 2035
+    },
+    {
+      "epoch": 12.914285714285715,
+      "grad_norm": 1.3131941556930542,
+      "learning_rate": 3.491900405672466e-05,
+      "loss": 0.2509,
+      "step": 2040
+    },
+    {
+      "epoch": 12.946031746031746,
+      "grad_norm": 1.5713857412338257,
+      "learning_rate": 3.483390023441141e-05,
+      "loss": 0.2824,
+      "step": 2045
+    },
+    {
+      "epoch": 12.977777777777778,
+      "grad_norm": 1.788149356842041,
+      "learning_rate": 3.4748661379625e-05,
+      "loss": 0.2932,
+      "step": 2050
+    },
+    {
+      "epoch": 13.006349206349206,
+      "grad_norm": 1.080249547958374,
+      "learning_rate": 3.466328866280778e-05,
+      "loss": 0.2817,
+      "step": 2055
+    },
+    {
+      "epoch": 13.038095238095238,
+      "grad_norm": 1.1128981113433838,
+      "learning_rate": 3.45777832562402e-05,
+      "loss": 0.2141,
+      "step": 2060
+    },
+    {
+      "epoch": 13.06984126984127,
+      "grad_norm": 1.6394922733306885,
+      "learning_rate": 3.449214633402473e-05,
+      "loss": 0.216,
+      "step": 2065
+    },
+    {
+      "epoch": 13.101587301587301,
+      "grad_norm": 2.198416233062744,
+      "learning_rate": 3.440637907206973e-05,
+      "loss": 0.258,
+      "step": 2070
+    },
+    {
+      "epoch": 13.133333333333333,
+      "grad_norm": 1.5107051134109497,
+      "learning_rate": 3.4320482648073285e-05,
+      "loss": 0.2422,
+      "step": 2075
+    },
+    {
+      "epoch": 13.165079365079364,
+      "grad_norm": 1.7206802368164062,
+      "learning_rate": 3.4234458241507034e-05,
+      "loss": 0.2524,
+      "step": 2080
+    },
+    {
+      "epoch": 13.196825396825396,
+      "grad_norm": 1.4832013845443726,
+      "learning_rate": 3.4148307033600014e-05,
+      "loss": 0.27,
+      "step": 2085
+    },
+    {
+      "epoch": 13.228571428571428,
+      "grad_norm": 1.592344880104065,
+      "learning_rate": 3.4062030207322395e-05,
+      "loss": 0.2486,
+      "step": 2090
+    },
+    {
+      "epoch": 13.260317460317461,
+      "grad_norm": 1.7398722171783447,
+      "learning_rate": 3.397562894736926e-05,
+      "loss": 0.2605,
+      "step": 2095
+    },
+    {
+      "epoch": 13.292063492063493,
+      "grad_norm": 2.0896050930023193,
+      "learning_rate": 3.388910444014432e-05,
+      "loss": 0.2562,
+      "step": 2100
+    },
+    {
+      "epoch": 13.323809523809524,
+      "grad_norm": 2.7245311737060547,
+      "learning_rate": 3.380245787374365e-05,
+      "loss": 0.2573,
+      "step": 2105
+    },
+    {
+      "epoch": 13.355555555555556,
+      "grad_norm": 1.1951723098754883,
+      "learning_rate": 3.371569043793936e-05,
+      "loss": 0.2709,
+      "step": 2110
+    },
+    {
+      "epoch": 13.387301587301588,
+      "grad_norm": 1.697736144065857,
+      "learning_rate": 3.3628803324163236e-05,
+      "loss": 0.2413,
+      "step": 2115
+    },
+    {
+      "epoch": 13.41904761904762,
+      "grad_norm": 1.447302222251892,
+      "learning_rate": 3.354179772549042e-05,
+      "loss": 0.2685,
+      "step": 2120
+    },
+    {
+      "epoch": 13.450793650793651,
+      "grad_norm": 1.232428789138794,
+      "learning_rate": 3.3454674836623e-05,
+      "loss": 0.2546,
+      "step": 2125
+    },
+    {
+      "epoch": 13.482539682539683,
+      "grad_norm": 1.889103889465332,
+      "learning_rate": 3.336743585387362e-05,
+      "loss": 0.2412,
+      "step": 2130
+    },
+    {
+      "epoch": 13.514285714285714,
+      "grad_norm": 1.888352632522583,
+      "learning_rate": 3.328008197514903e-05,
+      "loss": 0.2737,
+      "step": 2135
+    },
+    {
+      "epoch": 13.546031746031746,
+      "grad_norm": 1.4205390214920044,
+      "learning_rate": 3.319261439993369e-05,
+      "loss": 0.2727,
+      "step": 2140
+    },
+    {
+      "epoch": 13.577777777777778,
+      "grad_norm": 1.442429542541504,
+      "learning_rate": 3.310503432927322e-05,
+      "loss": 0.2894,
+      "step": 2145
+    },
+    {
+      "epoch": 13.60952380952381,
+      "grad_norm": 1.413885474205017,
+      "learning_rate": 3.3017342965757986e-05,
+      "loss": 0.2928,
+      "step": 2150
+    },
+    {
+      "epoch": 13.64126984126984,
+      "grad_norm": 1.0464482307434082,
+      "learning_rate": 3.2929541513506524e-05,
+      "loss": 0.252,
+      "step": 2155
+    },
+    {
+      "epoch": 13.673015873015872,
+      "grad_norm": 1.6716376543045044,
+      "learning_rate": 3.284163117814906e-05,
+      "loss": 0.2614,
+      "step": 2160
+    },
+    {
+      "epoch": 13.704761904761904,
+      "grad_norm": 1.9366419315338135,
+      "learning_rate": 3.2753613166810914e-05,
+      "loss": 0.2963,
+      "step": 2165
+    },
+    {
+      "epoch": 13.736507936507937,
+      "grad_norm": 1.4641704559326172,
+      "learning_rate": 3.2665488688095944e-05,
+      "loss": 0.2654,
+      "step": 2170
+    },
+    {
+      "epoch": 13.768253968253969,
+      "grad_norm": 1.3404383659362793,
+      "learning_rate": 3.2577258952069934e-05,
+      "loss": 0.2601,
+      "step": 2175
+    },
+    {
+      "epoch": 13.8,
+      "grad_norm": 1.6471940279006958,
+      "learning_rate": 3.2488925170244015e-05,
+      "loss": 0.2775,
+      "step": 2180
+    },
+    {
+      "epoch": 13.831746031746032,
+      "grad_norm": 1.3665790557861328,
+      "learning_rate": 3.240048855555798e-05,
+      "loss": 0.2602,
+      "step": 2185
+    },
+    {
+      "epoch": 13.863492063492064,
+      "grad_norm": 1.1486542224884033,
+      "learning_rate": 3.2311950322363685e-05,
+      "loss": 0.2603,
+      "step": 2190
+    },
+    {
+      "epoch": 13.895238095238096,
+      "grad_norm": 1.45487642288208,
+      "learning_rate": 3.2223311686408304e-05,
+      "loss": 0.2794,
+      "step": 2195
+    },
+    {
+      "epoch": 13.926984126984127,
+      "grad_norm": 1.6884818077087402,
+      "learning_rate": 3.213457386481769e-05,
+      "loss": 0.2682,
+      "step": 2200
+    },
+    {
+      "epoch": 13.958730158730159,
+      "grad_norm": 1.9160594940185547,
+      "learning_rate": 3.204573807607967e-05,
+      "loss": 0.2619,
+      "step": 2205
+    },
+    {
+      "epoch": 13.99047619047619,
+      "grad_norm": 1.9444315433502197,
+      "learning_rate": 3.1956805540027244e-05,
+      "loss": 0.3029,
+      "step": 2210
+    },
+    {
+      "epoch": 14.019047619047619,
+      "grad_norm": 1.8739337921142578,
+      "learning_rate": 3.1867777477821915e-05,
+      "loss": 0.2298,
+      "step": 2215
+    },
+    {
+      "epoch": 14.05079365079365,
+      "grad_norm": 1.3971226215362549,
+      "learning_rate": 3.1778655111936866e-05,
+      "loss": 0.2351,
+      "step": 2220
+    },
+    {
+      "epoch": 14.082539682539682,
+      "grad_norm": 1.288387417793274,
+      "learning_rate": 3.168943966614022e-05,
+      "loss": 0.2331,
+      "step": 2225
+    },
+    {
+      "epoch": 14.114285714285714,
+      "grad_norm": 1.2136940956115723,
+      "learning_rate": 3.1600132365478186e-05,
+      "loss": 0.2406,
+      "step": 2230
+    },
+    {
+      "epoch": 14.146031746031746,
+      "grad_norm": 1.8233654499053955,
+      "learning_rate": 3.151073443625828e-05,
+      "loss": 0.2496,
+      "step": 2235
+    },
+    {
+      "epoch": 14.177777777777777,
+      "grad_norm": 1.7613590955734253,
+      "learning_rate": 3.142124710603245e-05,
+      "loss": 0.2523,
+      "step": 2240
+    },
+    {
+      "epoch": 14.209523809523809,
+      "grad_norm": 1.8933606147766113,
+      "learning_rate": 3.133167160358024e-05,
+      "loss": 0.2646,
+      "step": 2245
+    },
+    {
+      "epoch": 14.24126984126984,
+      "grad_norm": 2.7858712673187256,
+      "learning_rate": 3.124200915889195e-05,
+      "loss": 0.2818,
+      "step": 2250
+    },
+    {
+      "epoch": 14.273015873015874,
+      "grad_norm": 1.5656853914260864,
+      "learning_rate": 3.115226100315166e-05,
+      "loss": 0.2389,
+      "step": 2255
+    },
+    {
+      "epoch": 14.304761904761905,
+      "grad_norm": 1.606392741203308,
+      "learning_rate": 3.106242836872041e-05,
+      "loss": 0.2302,
+      "step": 2260
+    },
+    {
+      "epoch": 14.336507936507937,
+      "grad_norm": 2.1553008556365967,
+      "learning_rate": 3.097251248911922e-05,
+      "loss": 0.2484,
+      "step": 2265
+    },
+    {
+      "epoch": 14.368253968253969,
+      "grad_norm": 1.7176517248153687,
+      "learning_rate": 3.088251459901222e-05,
+      "loss": 0.268,
+      "step": 2270
+    },
+    {
+      "epoch": 14.4,
+      "grad_norm": 1.3635174036026,
+      "learning_rate": 3.0792435934189604e-05,
+      "loss": 0.2491,
+      "step": 2275
+    },
+    {
+      "epoch": 14.431746031746032,
+      "grad_norm": 1.4257878065109253,
+      "learning_rate": 3.070227773155074e-05,
+      "loss": 0.2884,
+      "step": 2280
+    },
+    {
+      "epoch": 14.463492063492064,
+      "grad_norm": 1.1561063528060913,
+      "learning_rate": 3.0612041229087145e-05,
+      "loss": 0.247,
+      "step": 2285
+    },
+    {
+      "epoch": 14.495238095238095,
+      "grad_norm": 1.6131694316864014,
+      "learning_rate": 3.0521727665865485e-05,
+      "loss": 0.2708,
+      "step": 2290
+    },
+    {
+      "epoch": 14.526984126984127,
+      "grad_norm": 1.7133935689926147,
+      "learning_rate": 3.0431338282010606e-05,
+      "loss": 0.2384,
+      "step": 2295
+    },
+    {
+      "epoch": 14.558730158730159,
+      "grad_norm": 1.3232437372207642,
+      "learning_rate": 3.0340874318688433e-05,
+      "loss": 0.2503,
+      "step": 2300
+    },
+    {
+      "epoch": 14.59047619047619,
+      "grad_norm": 1.5612751245498657,
+      "learning_rate": 3.0250337018088994e-05,
+      "loss": 0.2618,
+      "step": 2305
+    },
+    {
+      "epoch": 14.622222222222222,
+      "grad_norm": 1.279241681098938,
+      "learning_rate": 3.0159727623409313e-05,
+      "loss": 0.2535,
+      "step": 2310
+    },
+    {
+      "epoch": 14.653968253968253,
+      "grad_norm": 3.142040729522705,
+      "learning_rate": 3.0069047378836375e-05,
+      "loss": 0.2745,
+      "step": 2315
+    },
+    {
+      "epoch": 14.685714285714285,
+      "grad_norm": 1.3240234851837158,
+      "learning_rate": 2.9978297529530036e-05,
+      "loss": 0.2591,
+      "step": 2320
+    },
+    {
+      "epoch": 14.717460317460317,
+      "grad_norm": 1.121077537536621,
+      "learning_rate": 2.9887479321605895e-05,
+      "loss": 0.2453,
+      "step": 2325
+    },
+    {
+      "epoch": 14.74920634920635,
+      "grad_norm": 2.3831980228424072,
+      "learning_rate": 2.9796594002118218e-05,
+      "loss": 0.257,
+      "step": 2330
+    },
+    {
+      "epoch": 14.780952380952382,
+      "grad_norm": 2.175352096557617,
+      "learning_rate": 2.9705642819042805e-05,
+      "loss": 0.2434,
+      "step": 2335
+    },
+    {
+      "epoch": 14.812698412698413,
+      "grad_norm": 1.5452053546905518,
+      "learning_rate": 2.9614627021259846e-05,
+      "loss": 0.2683,
+      "step": 2340
+    },
+    {
+      "epoch": 14.844444444444445,
+      "grad_norm": 1.6765031814575195,
+      "learning_rate": 2.952354785853677e-05,
+      "loss": 0.2484,
+      "step": 2345
+    },
+    {
+      "epoch": 14.876190476190477,
+      "grad_norm": 1.3790463209152222,
+      "learning_rate": 2.9432406581511096e-05,
+      "loss": 0.2636,
+      "step": 2350
+    },
+    {
+      "epoch": 14.907936507936508,
+      "grad_norm": 1.2849820852279663,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.2734,
+      "step": 2355
+    },
+    {
+      "epoch": 14.93968253968254,
+      "grad_norm": 1.5512354373931885,
+      "learning_rate": 2.924994269134942e-05,
+      "loss": 0.2627,
+      "step": 2360
+    },
+    {
+      "epoch": 14.971428571428572,
+      "grad_norm": 1.2973589897155762,
+      "learning_rate": 2.9158622583684264e-05,
+      "loss": 0.2464,
+      "step": 2365
+    },
+    {
+      "epoch": 15.0,
+      "grad_norm": 2.749181032180786,
+      "learning_rate": 2.906724537262381e-05,
+      "loss": 0.2622,
+      "step": 2370
+    },
+    {
+      "epoch": 15.031746031746032,
+      "grad_norm": 1.4415572881698608,
+      "learning_rate": 2.8975812312898152e-05,
+      "loss": 0.2395,
+      "step": 2375
+    },
+    {
+      "epoch": 15.063492063492063,
+      "grad_norm": 1.41774320602417,
+      "learning_rate": 2.8884324660004325e-05,
+      "loss": 0.2213,
+      "step": 2380
+    },
+    {
+      "epoch": 15.095238095238095,
+      "grad_norm": 1.375585675239563,
+      "learning_rate": 2.8792783670188927e-05,
+      "loss": 0.2319,
+      "step": 2385
+    },
+    {
+      "epoch": 15.126984126984127,
+      "grad_norm": 1.2285529375076294,
+      "learning_rate": 2.870119060043099e-05,
+      "loss": 0.2317,
+      "step": 2390
+    },
+    {
+      "epoch": 15.158730158730158,
+      "grad_norm": 2.5581822395324707,
+      "learning_rate": 2.8609546708424645e-05,
+      "loss": 0.2567,
+      "step": 2395
+    },
+    {
+      "epoch": 15.19047619047619,
+      "grad_norm": 2.0000274181365967,
+      "learning_rate": 2.8517853252561906e-05,
+      "loss": 0.2265,
+      "step": 2400
+    },
+    {
+      "epoch": 15.222222222222221,
+      "grad_norm": 1.6797828674316406,
+      "learning_rate": 2.8426111491915342e-05,
+      "loss": 0.243,
+      "step": 2405
+    },
+    {
+      "epoch": 15.253968253968253,
+      "grad_norm": 1.8280093669891357,
+      "learning_rate": 2.8334322686220826e-05,
+      "loss": 0.2442,
+      "step": 2410
+    },
+    {
+      "epoch": 15.285714285714286,
+      "grad_norm": 1.6335142850875854,
+      "learning_rate": 2.824248809586021e-05,
+      "loss": 0.229,
+      "step": 2415
+    },
+    {
+      "epoch": 15.317460317460318,
+      "grad_norm": 2.1388628482818604,
+      "learning_rate": 2.8150608981844034e-05,
+      "loss": 0.247,
+      "step": 2420
+    },
+    {
+      "epoch": 15.34920634920635,
+      "grad_norm": 1.5680638551712036,
+      "learning_rate": 2.805868660579422e-05,
+      "loss": 0.2269,
+      "step": 2425
+    },
+    {
+      "epoch": 15.380952380952381,
+      "grad_norm": 1.6333919763565063,
+      "learning_rate": 2.7966722229926712e-05,
+      "loss": 0.2426,
+      "step": 2430
+    },
+    {
+      "epoch": 15.412698412698413,
+      "grad_norm": 1.4023231267929077,
+      "learning_rate": 2.787471711703419e-05,
+      "loss": 0.2466,
+      "step": 2435
+    },
+    {
+      "epoch": 15.444444444444445,
+      "grad_norm": 1.5598480701446533,
+      "learning_rate": 2.7782672530468672e-05,
+      "loss": 0.2504,
+      "step": 2440
+    },
+    {
+      "epoch": 15.476190476190476,
+      "grad_norm": 1.3230295181274414,
+      "learning_rate": 2.769058973412424e-05,
+      "loss": 0.2478,
+      "step": 2445
+    },
+    {
+      "epoch": 15.507936507936508,
+      "grad_norm": 2.0796375274658203,
+      "learning_rate": 2.7598469992419646e-05,
+      "loss": 0.2382,
+      "step": 2450
+    },
+    {
+      "epoch": 15.53968253968254,
+      "grad_norm": 1.6772873401641846,
+      "learning_rate": 2.7506314570280916e-05,
+      "loss": 0.2716,
+      "step": 2455
+    },
+    {
+      "epoch": 15.571428571428571,
+      "grad_norm": 1.9949897527694702,
+      "learning_rate": 2.7414124733124046e-05,
+      "loss": 0.2541,
+      "step": 2460
+    },
+    {
+      "epoch": 15.603174603174603,
+      "grad_norm": 1.339775562286377,
+      "learning_rate": 2.7321901746837585e-05,
+      "loss": 0.238,
+      "step": 2465
+    },
+    {
+      "epoch": 15.634920634920634,
+      "grad_norm": 1.3870205879211426,
+      "learning_rate": 2.7229646877765264e-05,
+      "loss": 0.2377,
+      "step": 2470
+    },
+    {
+      "epoch": 15.666666666666666,
+      "grad_norm": 2.1485989093780518,
+      "learning_rate": 2.7137361392688613e-05,
+      "loss": 0.2748,
+      "step": 2475
+    },
+    {
+      "epoch": 15.698412698412698,
+      "grad_norm": 1.3165838718414307,
+      "learning_rate": 2.704504655880955e-05,
+      "loss": 0.2592,
+      "step": 2480
+    },
+    {
+      "epoch": 15.73015873015873,
+      "grad_norm": 1.5217639207839966,
+      "learning_rate": 2.6952703643732983e-05,
+      "loss": 0.2359,
+      "step": 2485
+    },
+    {
+      "epoch": 15.761904761904763,
+      "grad_norm": 1.6285779476165771,
+      "learning_rate": 2.686033391544945e-05,
+      "loss": 0.2653,
+      "step": 2490
+    },
+    {
+      "epoch": 15.793650793650794,
+      "grad_norm": 1.6619691848754883,
+      "learning_rate": 2.676793864231763e-05,
+      "loss": 0.2572,
+      "step": 2495
+    },
+    {
+      "epoch": 15.825396825396826,
+      "grad_norm": 1.6636338233947754,
+      "learning_rate": 2.6675519093046987e-05,
+      "loss": 0.2702,
+      "step": 2500
+    },
+    {
+      "epoch": 15.857142857142858,
+      "grad_norm": 1.6794034242630005,
+      "learning_rate": 2.6583076536680323e-05,
+      "loss": 0.2508,
+      "step": 2505
+    },
+    {
+      "epoch": 15.88888888888889,
+      "grad_norm": 1.669232964515686,
+      "learning_rate": 2.6490612242576344e-05,
+      "loss": 0.2601,
+      "step": 2510
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4710,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 30,
+  "save_steps": 157,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.39562980975575e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2669/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-2669/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2669/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2669/trainer_state.json

@@ -0,0 +1,3765 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 16.895238095238096,
+  "eval_steps": 500,
+  "global_step": 2669,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 0.5545095205307007,
+      "learning_rate": 5.307855626326963e-07,
+      "loss": 3.7162,
+      "step": 5
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 0.6163601279258728,
+      "learning_rate": 1.0615711252653927e-06,
+      "loss": 3.9388,
+      "step": 10
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 0.5541530847549438,
+      "learning_rate": 1.5923566878980892e-06,
+      "loss": 3.9165,
+      "step": 15
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 0.457332044839859,
+      "learning_rate": 2.1231422505307854e-06,
+      "loss": 3.7326,
+      "step": 20
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 0.5335279107093811,
+      "learning_rate": 2.653927813163482e-06,
+      "loss": 3.8251,
+      "step": 25
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 0.7080379724502563,
+      "learning_rate": 3.1847133757961785e-06,
+      "loss": 3.7534,
+      "step": 30
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 0.520993709564209,
+      "learning_rate": 3.715498938428875e-06,
+      "loss": 3.898,
+      "step": 35
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 0.5451405644416809,
+      "learning_rate": 4.246284501061571e-06,
+      "loss": 3.8951,
+      "step": 40
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.6205154657363892,
+      "learning_rate": 4.777070063694268e-06,
+      "loss": 3.7666,
+      "step": 45
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 0.7404439449310303,
+      "learning_rate": 5.307855626326964e-06,
+      "loss": 4.0258,
+      "step": 50
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 0.6272220015525818,
+      "learning_rate": 5.838641188959661e-06,
+      "loss": 3.8464,
+      "step": 55
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 0.7744691967964172,
+      "learning_rate": 6.369426751592357e-06,
+      "loss": 3.7299,
+      "step": 60
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 0.8805738687515259,
+      "learning_rate": 6.900212314225053e-06,
+      "loss": 3.5008,
+      "step": 65
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 1.0740723609924316,
+      "learning_rate": 7.43099787685775e-06,
+      "loss": 3.7552,
+      "step": 70
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.965708315372467,
+      "learning_rate": 7.961783439490445e-06,
+      "loss": 3.5516,
+      "step": 75
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 0.9812778234481812,
+      "learning_rate": 8.492569002123141e-06,
+      "loss": 3.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 0.8831024169921875,
+      "learning_rate": 9.023354564755838e-06,
+      "loss": 3.613,
+      "step": 85
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.8358364105224609,
+      "learning_rate": 9.554140127388536e-06,
+      "loss": 3.1858,
+      "step": 90
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 1.0740444660186768,
+      "learning_rate": 1.0084925690021232e-05,
+      "loss": 3.0937,
+      "step": 95
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 1.0987530946731567,
+      "learning_rate": 1.0615711252653929e-05,
+      "loss": 3.154,
+      "step": 100
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 1.2300925254821777,
+      "learning_rate": 1.1146496815286625e-05,
+      "loss": 2.9414,
+      "step": 105
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 1.2214170694351196,
+      "learning_rate": 1.1677282377919321e-05,
+      "loss": 2.9464,
+      "step": 110
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 1.2803975343704224,
+      "learning_rate": 1.2208067940552018e-05,
+      "loss": 2.8921,
+      "step": 115
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 1.2232719659805298,
+      "learning_rate": 1.2738853503184714e-05,
+      "loss": 2.5252,
+      "step": 120
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 1.204835295677185,
+      "learning_rate": 1.326963906581741e-05,
+      "loss": 2.5215,
+      "step": 125
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 1.4095579385757446,
+      "learning_rate": 1.3800424628450107e-05,
+      "loss": 2.136,
+      "step": 130
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 1.4166598320007324,
+      "learning_rate": 1.4331210191082803e-05,
+      "loss": 2.2653,
+      "step": 135
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 1.3040446043014526,
+      "learning_rate": 1.48619957537155e-05,
+      "loss": 2.0193,
+      "step": 140
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 1.4114688634872437,
+      "learning_rate": 1.5392781316348196e-05,
+      "loss": 1.7935,
+      "step": 145
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 1.8066726922988892,
+      "learning_rate": 1.592356687898089e-05,
+      "loss": 1.5731,
+      "step": 150
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 1.4303158521652222,
+      "learning_rate": 1.6454352441613588e-05,
+      "loss": 1.6552,
+      "step": 155
+    },
+    {
+      "epoch": 1.0126984126984127,
+      "grad_norm": 1.6671762466430664,
+      "learning_rate": 1.6985138004246283e-05,
+      "loss": 1.6973,
+      "step": 160
+    },
+    {
+      "epoch": 1.0444444444444445,
+      "grad_norm": 1.5719650983810425,
+      "learning_rate": 1.751592356687898e-05,
+      "loss": 1.312,
+      "step": 165
+    },
+    {
+      "epoch": 1.0761904761904761,
+      "grad_norm": 1.4845054149627686,
+      "learning_rate": 1.8046709129511676e-05,
+      "loss": 1.3601,
+      "step": 170
+    },
+    {
+      "epoch": 1.107936507936508,
+      "grad_norm": 1.1172235012054443,
+      "learning_rate": 1.8577494692144374e-05,
+      "loss": 1.3137,
+      "step": 175
+    },
+    {
+      "epoch": 1.1396825396825396,
+      "grad_norm": 1.9621731042861938,
+      "learning_rate": 1.910828025477707e-05,
+      "loss": 1.1778,
+      "step": 180
+    },
+    {
+      "epoch": 1.1714285714285715,
+      "grad_norm": 1.7722721099853516,
+      "learning_rate": 1.963906581740977e-05,
+      "loss": 1.4534,
+      "step": 185
+    },
+    {
+      "epoch": 1.2031746031746031,
+      "grad_norm": 1.3677467107772827,
+      "learning_rate": 2.0169851380042464e-05,
+      "loss": 1.3356,
+      "step": 190
+    },
+    {
+      "epoch": 1.234920634920635,
+      "grad_norm": 1.3260482549667358,
+      "learning_rate": 2.0700636942675162e-05,
+      "loss": 1.0876,
+      "step": 195
+    },
+    {
+      "epoch": 1.2666666666666666,
+      "grad_norm": 1.5176818370819092,
+      "learning_rate": 2.1231422505307857e-05,
+      "loss": 1.1602,
+      "step": 200
+    },
+    {
+      "epoch": 1.2984126984126985,
+      "grad_norm": 1.2793077230453491,
+      "learning_rate": 2.1762208067940555e-05,
+      "loss": 1.1505,
+      "step": 205
+    },
+    {
+      "epoch": 1.33015873015873,
+      "grad_norm": 1.196784257888794,
+      "learning_rate": 2.229299363057325e-05,
+      "loss": 1.0664,
+      "step": 210
+    },
+    {
+      "epoch": 1.361904761904762,
+      "grad_norm": 1.303207516670227,
+      "learning_rate": 2.2823779193205948e-05,
+      "loss": 1.2557,
+      "step": 215
+    },
+    {
+      "epoch": 1.3936507936507936,
+      "grad_norm": 1.2853388786315918,
+      "learning_rate": 2.3354564755838642e-05,
+      "loss": 1.0704,
+      "step": 220
+    },
+    {
+      "epoch": 1.4253968253968254,
+      "grad_norm": 1.381369948387146,
+      "learning_rate": 2.388535031847134e-05,
+      "loss": 1.1371,
+      "step": 225
+    },
+    {
+      "epoch": 1.457142857142857,
+      "grad_norm": 1.8012712001800537,
+      "learning_rate": 2.4416135881104035e-05,
+      "loss": 1.248,
+      "step": 230
+    },
+    {
+      "epoch": 1.488888888888889,
+      "grad_norm": 1.7397032976150513,
+      "learning_rate": 2.4946921443736733e-05,
+      "loss": 1.2782,
+      "step": 235
+    },
+    {
+      "epoch": 1.5206349206349206,
+      "grad_norm": 1.4026210308074951,
+      "learning_rate": 2.5477707006369428e-05,
+      "loss": 1.154,
+      "step": 240
+    },
+    {
+      "epoch": 1.5523809523809524,
+      "grad_norm": 1.2906067371368408,
+      "learning_rate": 2.6008492569002126e-05,
+      "loss": 0.9141,
+      "step": 245
+    },
+    {
+      "epoch": 1.5841269841269843,
+      "grad_norm": 1.265598177909851,
+      "learning_rate": 2.653927813163482e-05,
+      "loss": 1.0625,
+      "step": 250
+    },
+    {
+      "epoch": 1.615873015873016,
+      "grad_norm": 1.6044715642929077,
+      "learning_rate": 2.707006369426752e-05,
+      "loss": 0.9624,
+      "step": 255
+    },
+    {
+      "epoch": 1.6476190476190475,
+      "grad_norm": 1.4612747430801392,
+      "learning_rate": 2.7600849256900213e-05,
+      "loss": 1.0413,
+      "step": 260
+    },
+    {
+      "epoch": 1.6793650793650794,
+      "grad_norm": 1.6222745180130005,
+      "learning_rate": 2.8131634819532908e-05,
+      "loss": 1.0929,
+      "step": 265
+    },
+    {
+      "epoch": 1.7111111111111112,
+      "grad_norm": 1.1456222534179688,
+      "learning_rate": 2.8662420382165606e-05,
+      "loss": 0.9957,
+      "step": 270
+    },
+    {
+      "epoch": 1.7428571428571429,
+      "grad_norm": 1.5746041536331177,
+      "learning_rate": 2.91932059447983e-05,
+      "loss": 1.0274,
+      "step": 275
+    },
+    {
+      "epoch": 1.7746031746031745,
+      "grad_norm": 1.3407832384109497,
+      "learning_rate": 2.9723991507431e-05,
+      "loss": 0.9487,
+      "step": 280
+    },
+    {
+      "epoch": 1.8063492063492064,
+      "grad_norm": 1.6232194900512695,
+      "learning_rate": 3.0254777070063693e-05,
+      "loss": 1.0966,
+      "step": 285
+    },
+    {
+      "epoch": 1.8380952380952382,
+      "grad_norm": 1.4920552968978882,
+      "learning_rate": 3.078556263269639e-05,
+      "loss": 0.9099,
+      "step": 290
+    },
+    {
+      "epoch": 1.8698412698412699,
+      "grad_norm": 1.2123301029205322,
+      "learning_rate": 3.1316348195329086e-05,
+      "loss": 1.0902,
+      "step": 295
+    },
+    {
+      "epoch": 1.9015873015873015,
+      "grad_norm": 1.2080968618392944,
+      "learning_rate": 3.184713375796178e-05,
+      "loss": 0.943,
+      "step": 300
+    },
+    {
+      "epoch": 1.9333333333333333,
+      "grad_norm": 1.190319299697876,
+      "learning_rate": 3.237791932059448e-05,
+      "loss": 0.7893,
+      "step": 305
+    },
+    {
+      "epoch": 1.9650793650793652,
+      "grad_norm": 1.5929204225540161,
+      "learning_rate": 3.2908704883227177e-05,
+      "loss": 1.0232,
+      "step": 310
+    },
+    {
+      "epoch": 1.9968253968253968,
+      "grad_norm": 1.0138347148895264,
+      "learning_rate": 3.343949044585987e-05,
+      "loss": 0.6693,
+      "step": 315
+    },
+    {
+      "epoch": 2.0253968253968253,
+      "grad_norm": 1.3012847900390625,
+      "learning_rate": 3.3970276008492566e-05,
+      "loss": 0.8355,
+      "step": 320
+    },
+    {
+      "epoch": 2.057142857142857,
+      "grad_norm": 1.2264782190322876,
+      "learning_rate": 3.450106157112527e-05,
+      "loss": 0.9872,
+      "step": 325
+    },
+    {
+      "epoch": 2.088888888888889,
+      "grad_norm": 1.139275312423706,
+      "learning_rate": 3.503184713375796e-05,
+      "loss": 0.8662,
+      "step": 330
+    },
+    {
+      "epoch": 2.1206349206349207,
+      "grad_norm": 1.3836581707000732,
+      "learning_rate": 3.5562632696390657e-05,
+      "loss": 0.9549,
+      "step": 335
+    },
+    {
+      "epoch": 2.1523809523809523,
+      "grad_norm": 1.368600845336914,
+      "learning_rate": 3.609341825902335e-05,
+      "loss": 0.9195,
+      "step": 340
+    },
+    {
+      "epoch": 2.1841269841269844,
+      "grad_norm": 1.8793011903762817,
+      "learning_rate": 3.662420382165605e-05,
+      "loss": 0.8505,
+      "step": 345
+    },
+    {
+      "epoch": 2.215873015873016,
+      "grad_norm": 1.305284023284912,
+      "learning_rate": 3.715498938428875e-05,
+      "loss": 0.7755,
+      "step": 350
+    },
+    {
+      "epoch": 2.2476190476190476,
+      "grad_norm": 1.7851749658584595,
+      "learning_rate": 3.768577494692145e-05,
+      "loss": 0.9242,
+      "step": 355
+    },
+    {
+      "epoch": 2.2793650793650793,
+      "grad_norm": 1.4341535568237305,
+      "learning_rate": 3.821656050955414e-05,
+      "loss": 0.8221,
+      "step": 360
+    },
+    {
+      "epoch": 2.311111111111111,
+      "grad_norm": 1.39107346534729,
+      "learning_rate": 3.874734607218684e-05,
+      "loss": 0.6999,
+      "step": 365
+    },
+    {
+      "epoch": 2.342857142857143,
+      "grad_norm": 1.2304264307022095,
+      "learning_rate": 3.927813163481954e-05,
+      "loss": 0.8362,
+      "step": 370
+    },
+    {
+      "epoch": 2.3746031746031746,
+      "grad_norm": 1.8470840454101562,
+      "learning_rate": 3.9808917197452234e-05,
+      "loss": 0.9398,
+      "step": 375
+    },
+    {
+      "epoch": 2.4063492063492062,
+      "grad_norm": 1.2533882856369019,
+      "learning_rate": 4.033970276008493e-05,
+      "loss": 0.7754,
+      "step": 380
+    },
+    {
+      "epoch": 2.4380952380952383,
+      "grad_norm": 1.5335006713867188,
+      "learning_rate": 4.087048832271762e-05,
+      "loss": 1.1124,
+      "step": 385
+    },
+    {
+      "epoch": 2.46984126984127,
+      "grad_norm": 1.5298357009887695,
+      "learning_rate": 4.1401273885350325e-05,
+      "loss": 1.017,
+      "step": 390
+    },
+    {
+      "epoch": 2.5015873015873016,
+      "grad_norm": 1.4403260946273804,
+      "learning_rate": 4.193205944798302e-05,
+      "loss": 0.8831,
+      "step": 395
+    },
+    {
+      "epoch": 2.533333333333333,
+      "grad_norm": 1.1528433561325073,
+      "learning_rate": 4.2462845010615714e-05,
+      "loss": 0.801,
+      "step": 400
+    },
+    {
+      "epoch": 2.565079365079365,
+      "grad_norm": 1.3371326923370361,
+      "learning_rate": 4.299363057324841e-05,
+      "loss": 0.8692,
+      "step": 405
+    },
+    {
+      "epoch": 2.596825396825397,
+      "grad_norm": 1.4064775705337524,
+      "learning_rate": 4.352441613588111e-05,
+      "loss": 0.9059,
+      "step": 410
+    },
+    {
+      "epoch": 2.6285714285714286,
+      "grad_norm": 1.4531422853469849,
+      "learning_rate": 4.4055201698513805e-05,
+      "loss": 0.7344,
+      "step": 415
+    },
+    {
+      "epoch": 2.66031746031746,
+      "grad_norm": 1.7043890953063965,
+      "learning_rate": 4.45859872611465e-05,
+      "loss": 0.8298,
+      "step": 420
+    },
+    {
+      "epoch": 2.6920634920634923,
+      "grad_norm": 1.5105586051940918,
+      "learning_rate": 4.5116772823779194e-05,
+      "loss": 0.7768,
+      "step": 425
+    },
+    {
+      "epoch": 2.723809523809524,
+      "grad_norm": 1.8101528882980347,
+      "learning_rate": 4.5647558386411895e-05,
+      "loss": 0.733,
+      "step": 430
+    },
+    {
+      "epoch": 2.7555555555555555,
+      "grad_norm": 1.6365174055099487,
+      "learning_rate": 4.617834394904459e-05,
+      "loss": 0.8061,
+      "step": 435
+    },
+    {
+      "epoch": 2.787301587301587,
+      "grad_norm": 1.7808202505111694,
+      "learning_rate": 4.6709129511677285e-05,
+      "loss": 0.8333,
+      "step": 440
+    },
+    {
+      "epoch": 2.819047619047619,
+      "grad_norm": 1.5223265886306763,
+      "learning_rate": 4.723991507430998e-05,
+      "loss": 0.7557,
+      "step": 445
+    },
+    {
+      "epoch": 2.850793650793651,
+      "grad_norm": 1.3064416646957397,
+      "learning_rate": 4.777070063694268e-05,
+      "loss": 0.8041,
+      "step": 450
+    },
+    {
+      "epoch": 2.8825396825396825,
+      "grad_norm": 1.8025637865066528,
+      "learning_rate": 4.8301486199575375e-05,
+      "loss": 0.9534,
+      "step": 455
+    },
+    {
+      "epoch": 2.914285714285714,
+      "grad_norm": 1.924846887588501,
+      "learning_rate": 4.883227176220807e-05,
+      "loss": 0.9066,
+      "step": 460
+    },
+    {
+      "epoch": 2.9460317460317462,
+      "grad_norm": 1.9862899780273438,
+      "learning_rate": 4.9363057324840765e-05,
+      "loss": 0.7994,
+      "step": 465
+    },
+    {
+      "epoch": 2.977777777777778,
+      "grad_norm": 1.9615916013717651,
+      "learning_rate": 4.9893842887473466e-05,
+      "loss": 0.7045,
+      "step": 470
+    },
+    {
+      "epoch": 3.0063492063492063,
+      "grad_norm": 1.519852876663208,
+      "learning_rate": 4.999989014936042e-05,
+      "loss": 0.7212,
+      "step": 475
+    },
+    {
+      "epoch": 3.038095238095238,
+      "grad_norm": 1.9328887462615967,
+      "learning_rate": 4.999944388279162e-05,
+      "loss": 0.6598,
+      "step": 480
+    },
+    {
+      "epoch": 3.06984126984127,
+      "grad_norm": 2.0340709686279297,
+      "learning_rate": 4.999865434075176e-05,
+      "loss": 0.6829,
+      "step": 485
+    },
+    {
+      "epoch": 3.1015873015873017,
+      "grad_norm": 1.8775280714035034,
+      "learning_rate": 4.999752153408229e-05,
+      "loss": 0.6664,
+      "step": 490
+    },
+    {
+      "epoch": 3.1333333333333333,
+      "grad_norm": 2.385218381881714,
+      "learning_rate": 4.999604547833814e-05,
+      "loss": 0.6836,
+      "step": 495
+    },
+    {
+      "epoch": 3.165079365079365,
+      "grad_norm": 2.1743783950805664,
+      "learning_rate": 4.999422619378752e-05,
+      "loss": 0.7,
+      "step": 500
+    },
+    {
+      "epoch": 3.196825396825397,
+      "grad_norm": 2.20786452293396,
+      "learning_rate": 4.999206370541162e-05,
+      "loss": 0.7253,
+      "step": 505
+    },
+    {
+      "epoch": 3.2285714285714286,
+      "grad_norm": 1.8182263374328613,
+      "learning_rate": 4.998955804290425e-05,
+      "loss": 0.6824,
+      "step": 510
+    },
+    {
+      "epoch": 3.2603174603174603,
+      "grad_norm": 2.2959372997283936,
+      "learning_rate": 4.9986709240671495e-05,
+      "loss": 0.601,
+      "step": 515
+    },
+    {
+      "epoch": 3.292063492063492,
+      "grad_norm": 2.385838031768799,
+      "learning_rate": 4.998351733783116e-05,
+      "loss": 0.7417,
+      "step": 520
+    },
+    {
+      "epoch": 3.323809523809524,
+      "grad_norm": 2.0416879653930664,
+      "learning_rate": 4.997998237821233e-05,
+      "loss": 0.6463,
+      "step": 525
+    },
+    {
+      "epoch": 3.3555555555555556,
+      "grad_norm": 2.2781031131744385,
+      "learning_rate": 4.9976104410354654e-05,
+      "loss": 0.6998,
+      "step": 530
+    },
+    {
+      "epoch": 3.3873015873015873,
+      "grad_norm": 2.146778106689453,
+      "learning_rate": 4.9971883487507775e-05,
+      "loss": 0.7694,
+      "step": 535
+    },
+    {
+      "epoch": 3.419047619047619,
+      "grad_norm": 2.1369104385375977,
+      "learning_rate": 4.9967319667630567e-05,
+      "loss": 0.6615,
+      "step": 540
+    },
+    {
+      "epoch": 3.450793650793651,
+      "grad_norm": 2.4529733657836914,
+      "learning_rate": 4.996241301339029e-05,
+      "loss": 0.6109,
+      "step": 545
+    },
+    {
+      "epoch": 3.4825396825396826,
+      "grad_norm": 2.07030987739563,
+      "learning_rate": 4.995716359216183e-05,
+      "loss": 0.7611,
+      "step": 550
+    },
+    {
+      "epoch": 3.5142857142857142,
+      "grad_norm": 2.4329919815063477,
+      "learning_rate": 4.995157147602669e-05,
+      "loss": 0.7515,
+      "step": 555
+    },
+    {
+      "epoch": 3.546031746031746,
+      "grad_norm": 2.056351900100708,
+      "learning_rate": 4.994563674177202e-05,
+      "loss": 0.6885,
+      "step": 560
+    },
+    {
+      "epoch": 3.5777777777777775,
+      "grad_norm": 2.3665318489074707,
+      "learning_rate": 4.993935947088958e-05,
+      "loss": 0.6271,
+      "step": 565
+    },
+    {
+      "epoch": 3.6095238095238096,
+      "grad_norm": 2.677706480026245,
+      "learning_rate": 4.993273974957463e-05,
+      "loss": 0.5586,
+      "step": 570
+    },
+    {
+      "epoch": 3.641269841269841,
+      "grad_norm": 3.422136068344116,
+      "learning_rate": 4.9925777668724685e-05,
+      "loss": 0.7552,
+      "step": 575
+    },
+    {
+      "epoch": 3.6730158730158733,
+      "grad_norm": 2.4525184631347656,
+      "learning_rate": 4.991847332393835e-05,
+      "loss": 0.7367,
+      "step": 580
+    },
+    {
+      "epoch": 3.704761904761905,
+      "grad_norm": 2.4242067337036133,
+      "learning_rate": 4.991082681551396e-05,
+      "loss": 0.7044,
+      "step": 585
+    },
+    {
+      "epoch": 3.7365079365079366,
+      "grad_norm": 1.8419867753982544,
+      "learning_rate": 4.9902838248448184e-05,
+      "loss": 0.5966,
+      "step": 590
+    },
+    {
+      "epoch": 3.768253968253968,
+      "grad_norm": 2.1394360065460205,
+      "learning_rate": 4.989450773243463e-05,
+      "loss": 0.6736,
+      "step": 595
+    },
+    {
+      "epoch": 3.8,
+      "grad_norm": 1.285447597503662,
+      "learning_rate": 4.9885835381862326e-05,
+      "loss": 0.5021,
+      "step": 600
+    },
+    {
+      "epoch": 3.831746031746032,
+      "grad_norm": 2.724978446960449,
+      "learning_rate": 4.987682131581413e-05,
+      "loss": 0.6128,
+      "step": 605
+    },
+    {
+      "epoch": 3.8634920634920635,
+      "grad_norm": 2.239682912826538,
+      "learning_rate": 4.986746565806508e-05,
+      "loss": 0.5457,
+      "step": 610
+    },
+    {
+      "epoch": 3.895238095238095,
+      "grad_norm": 2.48944091796875,
+      "learning_rate": 4.9857768537080784e-05,
+      "loss": 0.6927,
+      "step": 615
+    },
+    {
+      "epoch": 3.9269841269841272,
+      "grad_norm": 2.4086852073669434,
+      "learning_rate": 4.9847730086015534e-05,
+      "loss": 0.5963,
+      "step": 620
+    },
+    {
+      "epoch": 3.958730158730159,
+      "grad_norm": 2.0070106983184814,
+      "learning_rate": 4.9837350442710553e-05,
+      "loss": 0.5856,
+      "step": 625
+    },
+    {
+      "epoch": 3.9904761904761905,
+      "grad_norm": 1.9726545810699463,
+      "learning_rate": 4.98266297496921e-05,
+      "loss": 0.6208,
+      "step": 630
+    },
+    {
+      "epoch": 4.019047619047619,
+      "grad_norm": 2.6137828826904297,
+      "learning_rate": 4.981556815416948e-05,
+      "loss": 0.6319,
+      "step": 635
+    },
+    {
+      "epoch": 4.050793650793651,
+      "grad_norm": 2.3489890098571777,
+      "learning_rate": 4.9804165808033054e-05,
+      "loss": 0.5887,
+      "step": 640
+    },
+    {
+      "epoch": 4.082539682539682,
+      "grad_norm": 2.8010590076446533,
+      "learning_rate": 4.979242286785214e-05,
+      "loss": 0.5257,
+      "step": 645
+    },
+    {
+      "epoch": 4.114285714285714,
+      "grad_norm": 2.993411064147949,
+      "learning_rate": 4.978033949487284e-05,
+      "loss": 0.4545,
+      "step": 650
+    },
+    {
+      "epoch": 4.146031746031746,
+      "grad_norm": 2.669935703277588,
+      "learning_rate": 4.976791585501588e-05,
+      "loss": 0.5989,
+      "step": 655
+    },
+    {
+      "epoch": 4.177777777777778,
+      "grad_norm": 3.084409236907959,
+      "learning_rate": 4.9755152118874294e-05,
+      "loss": 0.528,
+      "step": 660
+    },
+    {
+      "epoch": 4.20952380952381,
+      "grad_norm": 2.797873020172119,
+      "learning_rate": 4.974204846171106e-05,
+      "loss": 0.5249,
+      "step": 665
+    },
+    {
+      "epoch": 4.241269841269841,
+      "grad_norm": 3.667867422103882,
+      "learning_rate": 4.9728605063456765e-05,
+      "loss": 0.5838,
+      "step": 670
+    },
+    {
+      "epoch": 4.273015873015873,
+      "grad_norm": 2.6918869018554688,
+      "learning_rate": 4.971482210870706e-05,
+      "loss": 0.5143,
+      "step": 675
+    },
+    {
+      "epoch": 4.304761904761905,
+      "grad_norm": 2.1545379161834717,
+      "learning_rate": 4.970069978672017e-05,
+      "loss": 0.5317,
+      "step": 680
+    },
+    {
+      "epoch": 4.336507936507936,
+      "grad_norm": 2.1043529510498047,
+      "learning_rate": 4.9686238291414275e-05,
+      "loss": 0.4815,
+      "step": 685
+    },
+    {
+      "epoch": 4.368253968253969,
+      "grad_norm": 2.1359753608703613,
+      "learning_rate": 4.9671437821364855e-05,
+      "loss": 0.4935,
+      "step": 690
+    },
+    {
+      "epoch": 4.4,
+      "grad_norm": 3.092057228088379,
+      "learning_rate": 4.965629857980197e-05,
+      "loss": 0.6831,
+      "step": 695
+    },
+    {
+      "epoch": 4.431746031746032,
+      "grad_norm": 2.5296835899353027,
+      "learning_rate": 4.964082077460745e-05,
+      "loss": 0.5323,
+      "step": 700
+    },
+    {
+      "epoch": 4.463492063492064,
+      "grad_norm": 1.6655627489089966,
+      "learning_rate": 4.962500461831207e-05,
+      "loss": 0.4553,
+      "step": 705
+    },
+    {
+      "epoch": 4.495238095238095,
+      "grad_norm": 2.6663475036621094,
+      "learning_rate": 4.9608850328092576e-05,
+      "loss": 0.463,
+      "step": 710
+    },
+    {
+      "epoch": 4.526984126984127,
+      "grad_norm": 2.3763060569763184,
+      "learning_rate": 4.959235812576879e-05,
+      "loss": 0.4861,
+      "step": 715
+    },
+    {
+      "epoch": 4.5587301587301585,
+      "grad_norm": 2.2217962741851807,
+      "learning_rate": 4.957552823780047e-05,
+      "loss": 0.468,
+      "step": 720
+    },
+    {
+      "epoch": 4.59047619047619,
+      "grad_norm": 2.8885600566864014,
+      "learning_rate": 4.9558360895284295e-05,
+      "loss": 0.4588,
+      "step": 725
+    },
+    {
+      "epoch": 4.622222222222222,
+      "grad_norm": 2.5661261081695557,
+      "learning_rate": 4.954085633395058e-05,
+      "loss": 0.4926,
+      "step": 730
+    },
+    {
+      "epoch": 4.653968253968254,
+      "grad_norm": 2.304365396499634,
+      "learning_rate": 4.952301479416015e-05,
+      "loss": 0.494,
+      "step": 735
+    },
+    {
+      "epoch": 4.685714285714286,
+      "grad_norm": 2.690577983856201,
+      "learning_rate": 4.9504836520900976e-05,
+      "loss": 0.5814,
+      "step": 740
+    },
+    {
+      "epoch": 4.717460317460318,
+      "grad_norm": 2.7180025577545166,
+      "learning_rate": 4.948632176378481e-05,
+      "loss": 0.5329,
+      "step": 745
+    },
+    {
+      "epoch": 4.749206349206349,
+      "grad_norm": 2.716587543487549,
+      "learning_rate": 4.9467470777043806e-05,
+      "loss": 0.5264,
+      "step": 750
+    },
+    {
+      "epoch": 4.780952380952381,
+      "grad_norm": 2.315419912338257,
+      "learning_rate": 4.9448283819526954e-05,
+      "loss": 0.4756,
+      "step": 755
+    },
+    {
+      "epoch": 4.8126984126984125,
+      "grad_norm": 2.1679515838623047,
+      "learning_rate": 4.9428761154696605e-05,
+      "loss": 0.4819,
+      "step": 760
+    },
+    {
+      "epoch": 4.844444444444444,
+      "grad_norm": 3.389266014099121,
+      "learning_rate": 4.9408903050624796e-05,
+      "loss": 0.5121,
+      "step": 765
+    },
+    {
+      "epoch": 4.876190476190477,
+      "grad_norm": 3.4317383766174316,
+      "learning_rate": 4.938870977998959e-05,
+      "loss": 0.4535,
+      "step": 770
+    },
+    {
+      "epoch": 4.907936507936508,
+      "grad_norm": 2.9491918087005615,
+      "learning_rate": 4.9368181620071344e-05,
+      "loss": 0.5333,
+      "step": 775
+    },
+    {
+      "epoch": 4.93968253968254,
+      "grad_norm": 2.516798496246338,
+      "learning_rate": 4.934731885274887e-05,
+      "loss": 0.5367,
+      "step": 780
+    },
+    {
+      "epoch": 4.9714285714285715,
+      "grad_norm": 3.0031046867370605,
+      "learning_rate": 4.9326121764495596e-05,
+      "loss": 0.4957,
+      "step": 785
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.334085702896118,
+      "learning_rate": 4.9304590646375614e-05,
+      "loss": 0.5287,
+      "step": 790
+    },
+    {
+      "epoch": 5.031746031746032,
+      "grad_norm": 1.9608453512191772,
+      "learning_rate": 4.928272579403969e-05,
+      "loss": 0.36,
+      "step": 795
+    },
+    {
+      "epoch": 5.063492063492063,
+      "grad_norm": 2.328850746154785,
+      "learning_rate": 4.92605275077212e-05,
+      "loss": 0.3628,
+      "step": 800
+    },
+    {
+      "epoch": 5.095238095238095,
+      "grad_norm": 2.3446412086486816,
+      "learning_rate": 4.923799609223202e-05,
+      "loss": 0.3327,
+      "step": 805
+    },
+    {
+      "epoch": 5.1269841269841265,
+      "grad_norm": 2.476181745529175,
+      "learning_rate": 4.921513185695831e-05,
+      "loss": 0.4246,
+      "step": 810
+    },
+    {
+      "epoch": 5.158730158730159,
+      "grad_norm": 3.1026763916015625,
+      "learning_rate": 4.91919351158563e-05,
+      "loss": 0.5048,
+      "step": 815
+    },
+    {
+      "epoch": 5.190476190476191,
+      "grad_norm": 2.8165297508239746,
+      "learning_rate": 4.916840618744798e-05,
+      "loss": 0.4361,
+      "step": 820
+    },
+    {
+      "epoch": 5.222222222222222,
+      "grad_norm": 1.8732138872146606,
+      "learning_rate": 4.9144545394816687e-05,
+      "loss": 0.4693,
+      "step": 825
+    },
+    {
+      "epoch": 5.253968253968254,
+      "grad_norm": 1.7250264883041382,
+      "learning_rate": 4.91203530656027e-05,
+      "loss": 0.4076,
+      "step": 830
+    },
+    {
+      "epoch": 5.285714285714286,
+      "grad_norm": 2.105459690093994,
+      "learning_rate": 4.9095829531998725e-05,
+      "loss": 0.3589,
+      "step": 835
+    },
+    {
+      "epoch": 5.317460317460317,
+      "grad_norm": 3.6825687885284424,
+      "learning_rate": 4.9070975130745387e-05,
+      "loss": 0.5263,
+      "step": 840
+    },
+    {
+      "epoch": 5.349206349206349,
+      "grad_norm": 2.947052001953125,
+      "learning_rate": 4.90457902031265e-05,
+      "loss": 0.4632,
+      "step": 845
+    },
+    {
+      "epoch": 5.380952380952381,
+      "grad_norm": 1.9546104669570923,
+      "learning_rate": 4.902027509496448e-05,
+      "loss": 0.4348,
+      "step": 850
+    },
+    {
+      "epoch": 5.412698412698413,
+      "grad_norm": 2.4471983909606934,
+      "learning_rate": 4.899443015661557e-05,
+      "loss": 0.4209,
+      "step": 855
+    },
+    {
+      "epoch": 5.444444444444445,
+      "grad_norm": 1.827124834060669,
+      "learning_rate": 4.8968255742964975e-05,
+      "loss": 0.413,
+      "step": 860
+    },
+    {
+      "epoch": 5.476190476190476,
+      "grad_norm": 2.654707431793213,
+      "learning_rate": 4.894175221342207e-05,
+      "loss": 0.432,
+      "step": 865
+    },
+    {
+      "epoch": 5.507936507936508,
+      "grad_norm": 2.648967981338501,
+      "learning_rate": 4.8914919931915407e-05,
+      "loss": 0.4339,
+      "step": 870
+    },
+    {
+      "epoch": 5.5396825396825395,
+      "grad_norm": 2.874075412750244,
+      "learning_rate": 4.888775926688775e-05,
+      "loss": 0.4392,
+      "step": 875
+    },
+    {
+      "epoch": 5.571428571428571,
+      "grad_norm": 2.9674830436706543,
+      "learning_rate": 4.8860270591291e-05,
+      "loss": 0.4459,
+      "step": 880
+    },
+    {
+      "epoch": 5.603174603174603,
+      "grad_norm": 2.054748296737671,
+      "learning_rate": 4.883245428258107e-05,
+      "loss": 0.4313,
+      "step": 885
+    },
+    {
+      "epoch": 5.634920634920634,
+      "grad_norm": 1.9174392223358154,
+      "learning_rate": 4.880431072271272e-05,
+      "loss": 0.3906,
+      "step": 890
+    },
+    {
+      "epoch": 5.666666666666667,
+      "grad_norm": 2.5257787704467773,
+      "learning_rate": 4.87758402981343e-05,
+      "loss": 0.4219,
+      "step": 895
+    },
+    {
+      "epoch": 5.698412698412699,
+      "grad_norm": 2.6365532875061035,
+      "learning_rate": 4.8747043399782424e-05,
+      "loss": 0.3978,
+      "step": 900
+    },
+    {
+      "epoch": 5.73015873015873,
+      "grad_norm": 2.0583746433258057,
+      "learning_rate": 4.871792042307667e-05,
+      "loss": 0.4847,
+      "step": 905
+    },
+    {
+      "epoch": 5.761904761904762,
+      "grad_norm": 2.035872459411621,
+      "learning_rate": 4.868847176791406e-05,
+      "loss": 0.4675,
+      "step": 910
+    },
+    {
+      "epoch": 5.7936507936507935,
+      "grad_norm": 2.3722939491271973,
+      "learning_rate": 4.8658697838663625e-05,
+      "loss": 0.4586,
+      "step": 915
+    },
+    {
+      "epoch": 5.825396825396825,
+      "grad_norm": 1.2609732151031494,
+      "learning_rate": 4.862859904416085e-05,
+      "loss": 0.3274,
+      "step": 920
+    },
+    {
+      "epoch": 5.857142857142857,
+      "grad_norm": 2.3673977851867676,
+      "learning_rate": 4.8598175797702036e-05,
+      "loss": 0.4685,
+      "step": 925
+    },
+    {
+      "epoch": 5.888888888888889,
+      "grad_norm": 2.8414175510406494,
+      "learning_rate": 4.856742851703866e-05,
+      "loss": 0.4762,
+      "step": 930
+    },
+    {
+      "epoch": 5.920634920634921,
+      "grad_norm": 2.4126765727996826,
+      "learning_rate": 4.853635762437159e-05,
+      "loss": 0.4075,
+      "step": 935
+    },
+    {
+      "epoch": 5.9523809523809526,
+      "grad_norm": 1.8691045045852661,
+      "learning_rate": 4.8504963546345334e-05,
+      "loss": 0.4865,
+      "step": 940
+    },
+    {
+      "epoch": 5.984126984126984,
+      "grad_norm": 3.5297420024871826,
+      "learning_rate": 4.8473246714042155e-05,
+      "loss": 0.4623,
+      "step": 945
+    },
+    {
+      "epoch": 6.012698412698413,
+      "grad_norm": 2.059169054031372,
+      "learning_rate": 4.844120756297617e-05,
+      "loss": 0.4164,
+      "step": 950
+    },
+    {
+      "epoch": 6.044444444444444,
+      "grad_norm": 2.4746127128601074,
+      "learning_rate": 4.840884653308735e-05,
+      "loss": 0.3552,
+      "step": 955
+    },
+    {
+      "epoch": 6.076190476190476,
+      "grad_norm": 2.504425287246704,
+      "learning_rate": 4.8376164068735485e-05,
+      "loss": 0.3368,
+      "step": 960
+    },
+    {
+      "epoch": 6.1079365079365076,
+      "grad_norm": 2.062577486038208,
+      "learning_rate": 4.83431606186941e-05,
+      "loss": 0.3139,
+      "step": 965
+    },
+    {
+      "epoch": 6.13968253968254,
+      "grad_norm": 2.4934544563293457,
+      "learning_rate": 4.830983663614427e-05,
+      "loss": 0.3777,
+      "step": 970
+    },
+    {
+      "epoch": 6.171428571428572,
+      "grad_norm": 2.5747485160827637,
+      "learning_rate": 4.827619257866839e-05,
+      "loss": 0.373,
+      "step": 975
+    },
+    {
+      "epoch": 6.203174603174603,
+      "grad_norm": 2.449357271194458,
+      "learning_rate": 4.8242228908243946e-05,
+      "loss": 0.3936,
+      "step": 980
+    },
+    {
+      "epoch": 6.234920634920635,
+      "grad_norm": 2.952680826187134,
+      "learning_rate": 4.82079460912371e-05,
+      "loss": 0.407,
+      "step": 985
+    },
+    {
+      "epoch": 6.266666666666667,
+      "grad_norm": 2.1754496097564697,
+      "learning_rate": 4.817334459839633e-05,
+      "loss": 0.3189,
+      "step": 990
+    },
+    {
+      "epoch": 6.298412698412698,
+      "grad_norm": 2.8406214714050293,
+      "learning_rate": 4.8138424904845947e-05,
+      "loss": 0.3883,
+      "step": 995
+    },
+    {
+      "epoch": 6.33015873015873,
+      "grad_norm": 1.7533257007598877,
+      "learning_rate": 4.8103187490079604e-05,
+      "loss": 0.3131,
+      "step": 1000
+    },
+    {
+      "epoch": 6.3619047619047615,
+      "grad_norm": 2.4574601650238037,
+      "learning_rate": 4.806763283795366e-05,
+      "loss": 0.3606,
+      "step": 1005
+    },
+    {
+      "epoch": 6.393650793650794,
+      "grad_norm": 2.002281427383423,
+      "learning_rate": 4.8031761436680575e-05,
+      "loss": 0.37,
+      "step": 1010
+    },
+    {
+      "epoch": 6.425396825396826,
+      "grad_norm": 2.823315143585205,
+      "learning_rate": 4.79955737788222e-05,
+      "loss": 0.3791,
+      "step": 1015
+    },
+    {
+      "epoch": 6.457142857142857,
+      "grad_norm": 2.7891204357147217,
+      "learning_rate": 4.795907036128299e-05,
+      "loss": 0.3556,
+      "step": 1020
+    },
+    {
+      "epoch": 6.488888888888889,
+      "grad_norm": 2.2387146949768066,
+      "learning_rate": 4.7922251685303213e-05,
+      "loss": 0.3929,
+      "step": 1025
+    },
+    {
+      "epoch": 6.520634920634921,
+      "grad_norm": 2.5023891925811768,
+      "learning_rate": 4.788511825645205e-05,
+      "loss": 0.379,
+      "step": 1030
+    },
+    {
+      "epoch": 6.552380952380952,
+      "grad_norm": 2.2654805183410645,
+      "learning_rate": 4.7847670584620653e-05,
+      "loss": 0.3435,
+      "step": 1035
+    },
+    {
+      "epoch": 6.584126984126984,
+      "grad_norm": 3.3823065757751465,
+      "learning_rate": 4.7809909184015146e-05,
+      "loss": 0.4109,
+      "step": 1040
+    },
+    {
+      "epoch": 6.6158730158730155,
+      "grad_norm": 2.6096551418304443,
+      "learning_rate": 4.7771834573149576e-05,
+      "loss": 0.4233,
+      "step": 1045
+    },
+    {
+      "epoch": 6.647619047619048,
+      "grad_norm": 2.3933897018432617,
+      "learning_rate": 4.773344727483876e-05,
+      "loss": 0.3709,
+      "step": 1050
+    },
+    {
+      "epoch": 6.67936507936508,
+      "grad_norm": 2.189544916152954,
+      "learning_rate": 4.769474781619114e-05,
+      "loss": 0.3287,
+      "step": 1055
+    },
+    {
+      "epoch": 6.711111111111111,
+      "grad_norm": 2.450892686843872,
+      "learning_rate": 4.765573672860154e-05,
+      "loss": 0.4022,
+      "step": 1060
+    },
+    {
+      "epoch": 6.742857142857143,
+      "grad_norm": 2.4342429637908936,
+      "learning_rate": 4.761641454774386e-05,
+      "loss": 0.4029,
+      "step": 1065
+    },
+    {
+      "epoch": 6.7746031746031745,
+      "grad_norm": 2.2122364044189453,
+      "learning_rate": 4.75767818135637e-05,
+      "loss": 0.3322,
+      "step": 1070
+    },
+    {
+      "epoch": 6.806349206349206,
+      "grad_norm": 3.968445301055908,
+      "learning_rate": 4.7536839070271e-05,
+      "loss": 0.3836,
+      "step": 1075
+    },
+    {
+      "epoch": 6.838095238095238,
+      "grad_norm": 3.529158353805542,
+      "learning_rate": 4.749658686633251e-05,
+      "loss": 0.4745,
+      "step": 1080
+    },
+    {
+      "epoch": 6.86984126984127,
+      "grad_norm": 2.430727243423462,
+      "learning_rate": 4.7456025754464304e-05,
+      "loss": 0.3664,
+      "step": 1085
+    },
+    {
+      "epoch": 6.901587301587302,
+      "grad_norm": 2.6552302837371826,
+      "learning_rate": 4.7415156291624166e-05,
+      "loss": 0.4359,
+      "step": 1090
+    },
+    {
+      "epoch": 6.933333333333334,
+      "grad_norm": 2.134822130203247,
+      "learning_rate": 4.737397903900393e-05,
+      "loss": 0.3969,
+      "step": 1095
+    },
+    {
+      "epoch": 6.965079365079365,
+      "grad_norm": 2.5052947998046875,
+      "learning_rate": 4.7332494562021815e-05,
+      "loss": 0.4069,
+      "step": 1100
+    },
+    {
+      "epoch": 6.996825396825397,
+      "grad_norm": 2.1377065181732178,
+      "learning_rate": 4.729070343031463e-05,
+      "loss": 0.3853,
+      "step": 1105
+    },
+    {
+      "epoch": 7.025396825396825,
+      "grad_norm": 1.9704042673110962,
+      "learning_rate": 4.724860621772995e-05,
+      "loss": 0.3283,
+      "step": 1110
+    },
+    {
+      "epoch": 7.057142857142857,
+      "grad_norm": 2.476968765258789,
+      "learning_rate": 4.7206203502318256e-05,
+      "loss": 0.3325,
+      "step": 1115
+    },
+    {
+      "epoch": 7.088888888888889,
+      "grad_norm": 1.9231969118118286,
+      "learning_rate": 4.716349586632499e-05,
+      "loss": 0.2876,
+      "step": 1120
+    },
+    {
+      "epoch": 7.12063492063492,
+      "grad_norm": 2.6444814205169678,
+      "learning_rate": 4.712048389618254e-05,
+      "loss": 0.3005,
+      "step": 1125
+    },
+    {
+      "epoch": 7.152380952380953,
+      "grad_norm": 3.2589964866638184,
+      "learning_rate": 4.7077168182502216e-05,
+      "loss": 0.4023,
+      "step": 1130
+    },
+    {
+      "epoch": 7.184126984126984,
+      "grad_norm": 2.5481936931610107,
+      "learning_rate": 4.703354932006615e-05,
+      "loss": 0.3302,
+      "step": 1135
+    },
+    {
+      "epoch": 7.215873015873016,
+      "grad_norm": 1.7125908136367798,
+      "learning_rate": 4.698962790781906e-05,
+      "loss": 0.3329,
+      "step": 1140
+    },
+    {
+      "epoch": 7.247619047619048,
+      "grad_norm": 2.2756667137145996,
+      "learning_rate": 4.6945404548860115e-05,
+      "loss": 0.3369,
+      "step": 1145
+    },
+    {
+      "epoch": 7.279365079365079,
+      "grad_norm": 2.9158453941345215,
+      "learning_rate": 4.6900879850434604e-05,
+      "loss": 0.3339,
+      "step": 1150
+    },
+    {
+      "epoch": 7.311111111111111,
+      "grad_norm": 2.3047537803649902,
+      "learning_rate": 4.685605442392559e-05,
+      "loss": 0.3915,
+      "step": 1155
+    },
+    {
+      "epoch": 7.3428571428571425,
+      "grad_norm": 2.7815029621124268,
+      "learning_rate": 4.681092888484554e-05,
+      "loss": 0.3317,
+      "step": 1160
+    },
+    {
+      "epoch": 7.374603174603175,
+      "grad_norm": 2.2644097805023193,
+      "learning_rate": 4.676550385282787e-05,
+      "loss": 0.3314,
+      "step": 1165
+    },
+    {
+      "epoch": 7.406349206349207,
+      "grad_norm": 2.5144474506378174,
+      "learning_rate": 4.671977995161843e-05,
+      "loss": 0.3188,
+      "step": 1170
+    },
+    {
+      "epoch": 7.438095238095238,
+      "grad_norm": 3.120821714401245,
+      "learning_rate": 4.667375780906693e-05,
+      "loss": 0.3523,
+      "step": 1175
+    },
+    {
+      "epoch": 7.46984126984127,
+      "grad_norm": 4.47842264175415,
+      "learning_rate": 4.662743805711832e-05,
+      "loss": 0.3611,
+      "step": 1180
+    },
+    {
+      "epoch": 7.501587301587302,
+      "grad_norm": 1.9228928089141846,
+      "learning_rate": 4.658082133180416e-05,
+      "loss": 0.3612,
+      "step": 1185
+    },
+    {
+      "epoch": 7.533333333333333,
+      "grad_norm": 2.1507537364959717,
+      "learning_rate": 4.6533908273233815e-05,
+      "loss": 0.3321,
+      "step": 1190
+    },
+    {
+      "epoch": 7.565079365079365,
+      "grad_norm": 2.1849119663238525,
+      "learning_rate": 4.64866995255857e-05,
+      "loss": 0.2943,
+      "step": 1195
+    },
+    {
+      "epoch": 7.5968253968253965,
+      "grad_norm": 2.1777775287628174,
+      "learning_rate": 4.643919573709843e-05,
+      "loss": 0.353,
+      "step": 1200
+    },
+    {
+      "epoch": 7.628571428571428,
+      "grad_norm": 2.5231118202209473,
+      "learning_rate": 4.639139756006195e-05,
+      "loss": 0.3571,
+      "step": 1205
+    },
+    {
+      "epoch": 7.660317460317461,
+      "grad_norm": 1.8409479856491089,
+      "learning_rate": 4.6343305650808516e-05,
+      "loss": 0.3691,
+      "step": 1210
+    },
+    {
+      "epoch": 7.692063492063492,
+      "grad_norm": 1.7940895557403564,
+      "learning_rate": 4.629492066970373e-05,
+      "loss": 0.3738,
+      "step": 1215
+    },
+    {
+      "epoch": 7.723809523809524,
+      "grad_norm": 2.014902114868164,
+      "learning_rate": 4.6246243281137474e-05,
+      "loss": 0.361,
+      "step": 1220
+    },
+    {
+      "epoch": 7.7555555555555555,
+      "grad_norm": 3.4182560443878174,
+      "learning_rate": 4.6197274153514735e-05,
+      "loss": 0.3663,
+      "step": 1225
+    },
+    {
+      "epoch": 7.787301587301587,
+      "grad_norm": 2.518728256225586,
+      "learning_rate": 4.614801395924649e-05,
+      "loss": 0.3646,
+      "step": 1230
+    },
+    {
+      "epoch": 7.819047619047619,
+      "grad_norm": 2.154189109802246,
+      "learning_rate": 4.6098463374740466e-05,
+      "loss": 0.3331,
+      "step": 1235
+    },
+    {
+      "epoch": 7.85079365079365,
+      "grad_norm": 2.536081075668335,
+      "learning_rate": 4.604862308039177e-05,
+      "loss": 0.3742,
+      "step": 1240
+    },
+    {
+      "epoch": 7.882539682539683,
+      "grad_norm": 2.340764045715332,
+      "learning_rate": 4.599849376057366e-05,
+      "loss": 0.3352,
+      "step": 1245
+    },
+    {
+      "epoch": 7.914285714285715,
+      "grad_norm": 3.5488364696502686,
+      "learning_rate": 4.5948076103628094e-05,
+      "loss": 0.3663,
+      "step": 1250
+    },
+    {
+      "epoch": 7.946031746031746,
+      "grad_norm": 2.779360294342041,
+      "learning_rate": 4.589737080185625e-05,
+      "loss": 0.3362,
+      "step": 1255
+    },
+    {
+      "epoch": 7.977777777777778,
+      "grad_norm": 1.8792667388916016,
+      "learning_rate": 4.5846378551509097e-05,
+      "loss": 0.346,
+      "step": 1260
+    },
+    {
+      "epoch": 8.006349206349206,
+      "grad_norm": 2.453295946121216,
+      "learning_rate": 4.579510005277774e-05,
+      "loss": 0.3509,
+      "step": 1265
+    },
+    {
+      "epoch": 8.038095238095238,
+      "grad_norm": 1.9493130445480347,
+      "learning_rate": 4.574353600978388e-05,
+      "loss": 0.3062,
+      "step": 1270
+    },
+    {
+      "epoch": 8.06984126984127,
+      "grad_norm": 1.9360930919647217,
+      "learning_rate": 4.56916871305701e-05,
+      "loss": 0.3056,
+      "step": 1275
+    },
+    {
+      "epoch": 8.101587301587301,
+      "grad_norm": 1.5592070817947388,
+      "learning_rate": 4.563955412709021e-05,
+      "loss": 0.2785,
+      "step": 1280
+    },
+    {
+      "epoch": 8.133333333333333,
+      "grad_norm": 1.8093425035476685,
+      "learning_rate": 4.5587137715199354e-05,
+      "loss": 0.308,
+      "step": 1285
+    },
+    {
+      "epoch": 8.165079365079364,
+      "grad_norm": 2.2939181327819824,
+      "learning_rate": 4.5534438614644294e-05,
+      "loss": 0.3038,
+      "step": 1290
+    },
+    {
+      "epoch": 8.196825396825396,
+      "grad_norm": 2.4204866886138916,
+      "learning_rate": 4.548145754905346e-05,
+      "loss": 0.3375,
+      "step": 1295
+    },
+    {
+      "epoch": 8.228571428571428,
+      "grad_norm": 1.725534439086914,
+      "learning_rate": 4.5428195245927064e-05,
+      "loss": 0.3101,
+      "step": 1300
+    },
+    {
+      "epoch": 8.260317460317461,
+      "grad_norm": 1.637730360031128,
+      "learning_rate": 4.537465243662704e-05,
+      "loss": 0.2931,
+      "step": 1305
+    },
+    {
+      "epoch": 8.292063492063493,
+      "grad_norm": 1.3372169733047485,
+      "learning_rate": 4.532082985636709e-05,
+      "loss": 0.2763,
+      "step": 1310
+    },
+    {
+      "epoch": 8.323809523809524,
+      "grad_norm": 2.5993168354034424,
+      "learning_rate": 4.5266728244202494e-05,
+      "loss": 0.3458,
+      "step": 1315
+    },
+    {
+      "epoch": 8.355555555555556,
+      "grad_norm": 2.461862564086914,
+      "learning_rate": 4.521234834302006e-05,
+      "loss": 0.3693,
+      "step": 1320
+    },
+    {
+      "epoch": 8.387301587301588,
+      "grad_norm": 1.8519413471221924,
+      "learning_rate": 4.5157690899527816e-05,
+      "loss": 0.3327,
+      "step": 1325
+    },
+    {
+      "epoch": 8.41904761904762,
+      "grad_norm": 2.1535580158233643,
+      "learning_rate": 4.510275666424487e-05,
+      "loss": 0.3229,
+      "step": 1330
+    },
+    {
+      "epoch": 8.450793650793651,
+      "grad_norm": 1.6819690465927124,
+      "learning_rate": 4.5047546391491e-05,
+      "loss": 0.2925,
+      "step": 1335
+    },
+    {
+      "epoch": 8.482539682539683,
+      "grad_norm": 1.6538281440734863,
+      "learning_rate": 4.499206083937638e-05,
+      "loss": 0.3218,
+      "step": 1340
+    },
+    {
+      "epoch": 8.514285714285714,
+      "grad_norm": 1.8956862688064575,
+      "learning_rate": 4.493630076979112e-05,
+      "loss": 0.3423,
+      "step": 1345
+    },
+    {
+      "epoch": 8.546031746031746,
+      "grad_norm": 2.274681806564331,
+      "learning_rate": 4.48802669483948e-05,
+      "loss": 0.3152,
+      "step": 1350
+    },
+    {
+      "epoch": 8.577777777777778,
+      "grad_norm": 2.2956337928771973,
+      "learning_rate": 4.4823960144606014e-05,
+      "loss": 0.3417,
+      "step": 1355
+    },
+    {
+      "epoch": 8.60952380952381,
+      "grad_norm": 1.8650286197662354,
+      "learning_rate": 4.4767381131591734e-05,
+      "loss": 0.2896,
+      "step": 1360
+    },
+    {
+      "epoch": 8.64126984126984,
+      "grad_norm": 1.3998652696609497,
+      "learning_rate": 4.471053068625674e-05,
+      "loss": 0.3372,
+      "step": 1365
+    },
+    {
+      "epoch": 8.673015873015872,
+      "grad_norm": 2.855074167251587,
+      "learning_rate": 4.465340958923293e-05,
+      "loss": 0.332,
+      "step": 1370
+    },
+    {
+      "epoch": 8.704761904761904,
+      "grad_norm": 1.6865357160568237,
+      "learning_rate": 4.459601862486862e-05,
+      "loss": 0.3053,
+      "step": 1375
+    },
+    {
+      "epoch": 8.736507936507937,
+      "grad_norm": 2.501856803894043,
+      "learning_rate": 4.453835858121773e-05,
+      "loss": 0.3119,
+      "step": 1380
+    },
+    {
+      "epoch": 8.768253968253969,
+      "grad_norm": 2.4325456619262695,
+      "learning_rate": 4.4480430250029046e-05,
+      "loss": 0.3395,
+      "step": 1385
+    },
+    {
+      "epoch": 8.8,
+      "grad_norm": 1.4845948219299316,
+      "learning_rate": 4.4422234426735256e-05,
+      "loss": 0.3237,
+      "step": 1390
+    },
+    {
+      "epoch": 8.831746031746032,
+      "grad_norm": 1.3553249835968018,
+      "learning_rate": 4.436377191044208e-05,
+      "loss": 0.3387,
+      "step": 1395
+    },
+    {
+      "epoch": 8.863492063492064,
+      "grad_norm": 1.8338890075683594,
+      "learning_rate": 4.430504350391729e-05,
+      "loss": 0.3618,
+      "step": 1400
+    },
+    {
+      "epoch": 8.895238095238096,
+      "grad_norm": 2.291538953781128,
+      "learning_rate": 4.4246050013579686e-05,
+      "loss": 0.3608,
+      "step": 1405
+    },
+    {
+      "epoch": 8.926984126984127,
+      "grad_norm": 1.3809788227081299,
+      "learning_rate": 4.4186792249488005e-05,
+      "loss": 0.3077,
+      "step": 1410
+    },
+    {
+      "epoch": 8.958730158730159,
+      "grad_norm": 1.5944230556488037,
+      "learning_rate": 4.412727102532983e-05,
+      "loss": 0.3307,
+      "step": 1415
+    },
+    {
+      "epoch": 8.99047619047619,
+      "grad_norm": 2.2244362831115723,
+      "learning_rate": 4.4067487158410396e-05,
+      "loss": 0.3469,
+      "step": 1420
+    },
+    {
+      "epoch": 9.019047619047619,
+      "grad_norm": 1.444221019744873,
+      "learning_rate": 4.400744146964136e-05,
+      "loss": 0.3049,
+      "step": 1425
+    },
+    {
+      "epoch": 9.05079365079365,
+      "grad_norm": 1.5847752094268799,
+      "learning_rate": 4.394713478352955e-05,
+      "loss": 0.2715,
+      "step": 1430
+    },
+    {
+      "epoch": 9.082539682539682,
+      "grad_norm": 1.6062681674957275,
+      "learning_rate": 4.388656792816562e-05,
+      "loss": 0.2487,
+      "step": 1435
+    },
+    {
+      "epoch": 9.114285714285714,
+      "grad_norm": 2.099787712097168,
+      "learning_rate": 4.382574173521272e-05,
+      "loss": 0.2866,
+      "step": 1440
+    },
+    {
+      "epoch": 9.146031746031746,
+      "grad_norm": 1.0997334718704224,
+      "learning_rate": 4.376465703989502e-05,
+      "loss": 0.3052,
+      "step": 1445
+    },
+    {
+      "epoch": 9.177777777777777,
+      "grad_norm": 2.4327454566955566,
+      "learning_rate": 4.370331468098628e-05,
+      "loss": 0.3212,
+      "step": 1450
+    },
+    {
+      "epoch": 9.209523809523809,
+      "grad_norm": 1.4816385507583618,
+      "learning_rate": 4.364171550079833e-05,
+      "loss": 0.3046,
+      "step": 1455
+    },
+    {
+      "epoch": 9.24126984126984,
+      "grad_norm": 2.039186716079712,
+      "learning_rate": 4.357986034516947e-05,
+      "loss": 0.3165,
+      "step": 1460
+    },
+    {
+      "epoch": 9.273015873015874,
+      "grad_norm": 1.437852382659912,
+      "learning_rate": 4.3517750063452934e-05,
+      "loss": 0.3037,
+      "step": 1465
+    },
+    {
+      "epoch": 9.304761904761905,
+      "grad_norm": 1.818982720375061,
+      "learning_rate": 4.345538550850512e-05,
+      "loss": 0.3122,
+      "step": 1470
+    },
+    {
+      "epoch": 9.336507936507937,
+      "grad_norm": 1.12025785446167,
+      "learning_rate": 4.339276753667395e-05,
+      "loss": 0.2909,
+      "step": 1475
+    },
+    {
+      "epoch": 9.368253968253969,
+      "grad_norm": 1.6094844341278076,
+      "learning_rate": 4.3329897007787125e-05,
+      "loss": 0.2823,
+      "step": 1480
+    },
+    {
+      "epoch": 9.4,
+      "grad_norm": 1.916200041770935,
+      "learning_rate": 4.326677478514024e-05,
+      "loss": 0.2939,
+      "step": 1485
+    },
+    {
+      "epoch": 9.431746031746032,
+      "grad_norm": 1.97919499874115,
+      "learning_rate": 4.320340173548503e-05,
+      "loss": 0.2826,
+      "step": 1490
+    },
+    {
+      "epoch": 9.463492063492064,
+      "grad_norm": 2.0238938331604004,
+      "learning_rate": 4.313977872901737e-05,
+      "loss": 0.3273,
+      "step": 1495
+    },
+    {
+      "epoch": 9.495238095238095,
+      "grad_norm": 2.5840957164764404,
+      "learning_rate": 4.307590663936541e-05,
+      "loss": 0.2889,
+      "step": 1500
+    },
+    {
+      "epoch": 9.526984126984127,
+      "grad_norm": 2.3503904342651367,
+      "learning_rate": 4.30117863435775e-05,
+      "loss": 0.3012,
+      "step": 1505
+    },
+    {
+      "epoch": 9.558730158730159,
+      "grad_norm": 2.019792318344116,
+      "learning_rate": 4.294741872211024e-05,
+      "loss": 0.3267,
+      "step": 1510
+    },
+    {
+      "epoch": 9.59047619047619,
+      "grad_norm": 2.2713353633880615,
+      "learning_rate": 4.288280465881632e-05,
+      "loss": 0.3096,
+      "step": 1515
+    },
+    {
+      "epoch": 9.622222222222222,
+      "grad_norm": 2.4236693382263184,
+      "learning_rate": 4.281794504093237e-05,
+      "loss": 0.3291,
+      "step": 1520
+    },
+    {
+      "epoch": 9.653968253968253,
+      "grad_norm": 1.772703766822815,
+      "learning_rate": 4.275284075906686e-05,
+      "loss": 0.3117,
+      "step": 1525
+    },
+    {
+      "epoch": 9.685714285714285,
+      "grad_norm": 1.9665186405181885,
+      "learning_rate": 4.268749270718778e-05,
+      "loss": 0.326,
+      "step": 1530
+    },
+    {
+      "epoch": 9.717460317460317,
+      "grad_norm": 1.9472782611846924,
+      "learning_rate": 4.262190178261044e-05,
+      "loss": 0.2683,
+      "step": 1535
+    },
+    {
+      "epoch": 9.74920634920635,
+      "grad_norm": 2.0638089179992676,
+      "learning_rate": 4.255606888598508e-05,
+      "loss": 0.314,
+      "step": 1540
+    },
+    {
+      "epoch": 9.780952380952382,
+      "grad_norm": 2.1349925994873047,
+      "learning_rate": 4.248999492128456e-05,
+      "loss": 0.2897,
+      "step": 1545
+    },
+    {
+      "epoch": 9.812698412698413,
+      "grad_norm": 2.112536907196045,
+      "learning_rate": 4.242368079579192e-05,
+      "loss": 0.31,
+      "step": 1550
+    },
+    {
+      "epoch": 9.844444444444445,
+      "grad_norm": 1.6859878301620483,
+      "learning_rate": 4.2357127420087917e-05,
+      "loss": 0.3412,
+      "step": 1555
+    },
+    {
+      "epoch": 9.876190476190477,
+      "grad_norm": 1.9178651571273804,
+      "learning_rate": 4.229033570803853e-05,
+      "loss": 0.334,
+      "step": 1560
+    },
+    {
+      "epoch": 9.907936507936508,
+      "grad_norm": 2.562436103820801,
+      "learning_rate": 4.2223306576782426e-05,
+      "loss": 0.3379,
+      "step": 1565
+    },
+    {
+      "epoch": 9.93968253968254,
+      "grad_norm": 1.8472412824630737,
+      "learning_rate": 4.215604094671835e-05,
+      "loss": 0.3415,
+      "step": 1570
+    },
+    {
+      "epoch": 9.971428571428572,
+      "grad_norm": 1.9416279792785645,
+      "learning_rate": 4.208853974149246e-05,
+      "loss": 0.3085,
+      "step": 1575
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 2.0056397914886475,
+      "learning_rate": 4.202080388798571e-05,
+      "loss": 0.3263,
+      "step": 1580
+    },
+    {
+      "epoch": 10.031746031746032,
+      "grad_norm": 2.195781946182251,
+      "learning_rate": 4.1952834316301065e-05,
+      "loss": 0.2867,
+      "step": 1585
+    },
+    {
+      "epoch": 10.063492063492063,
+      "grad_norm": 1.7489805221557617,
+      "learning_rate": 4.1884631959750766e-05,
+      "loss": 0.2589,
+      "step": 1590
+    },
+    {
+      "epoch": 10.095238095238095,
+      "grad_norm": 1.9361369609832764,
+      "learning_rate": 4.181619775484348e-05,
+      "loss": 0.2722,
+      "step": 1595
+    },
+    {
+      "epoch": 10.126984126984127,
+      "grad_norm": 2.24322509765625,
+      "learning_rate": 4.174753264127147e-05,
+      "loss": 0.2534,
+      "step": 1600
+    },
+    {
+      "epoch": 10.158730158730158,
+      "grad_norm": 2.4550466537475586,
+      "learning_rate": 4.167863756189767e-05,
+      "loss": 0.2777,
+      "step": 1605
+    },
+    {
+      "epoch": 10.19047619047619,
+      "grad_norm": 1.9439811706542969,
+      "learning_rate": 4.160951346274278e-05,
+      "loss": 0.2864,
+      "step": 1610
+    },
+    {
+      "epoch": 10.222222222222221,
+      "grad_norm": 1.633494257926941,
+      "learning_rate": 4.154016129297219e-05,
+      "loss": 0.2858,
+      "step": 1615
+    },
+    {
+      "epoch": 10.253968253968253,
+      "grad_norm": 1.69782292842865,
+      "learning_rate": 4.147058200488305e-05,
+      "loss": 0.2942,
+      "step": 1620
+    },
+    {
+      "epoch": 10.285714285714286,
+      "grad_norm": 1.613031268119812,
+      "learning_rate": 4.140077655389113e-05,
+      "loss": 0.2632,
+      "step": 1625
+    },
+    {
+      "epoch": 10.317460317460318,
+      "grad_norm": 2.0266177654266357,
+      "learning_rate": 4.1330745898517714e-05,
+      "loss": 0.3011,
+      "step": 1630
+    },
+    {
+      "epoch": 10.34920634920635,
+      "grad_norm": 1.8945387601852417,
+      "learning_rate": 4.1260491000376446e-05,
+      "loss": 0.2832,
+      "step": 1635
+    },
+    {
+      "epoch": 10.380952380952381,
+      "grad_norm": 1.7012510299682617,
+      "learning_rate": 4.119001282416009e-05,
+      "loss": 0.2718,
+      "step": 1640
+    },
+    {
+      "epoch": 10.412698412698413,
+      "grad_norm": 1.5538525581359863,
+      "learning_rate": 4.111931233762738e-05,
+      "loss": 0.3232,
+      "step": 1645
+    },
+    {
+      "epoch": 10.444444444444445,
+      "grad_norm": 2.3083150386810303,
+      "learning_rate": 4.1048390511589595e-05,
+      "loss": 0.3057,
+      "step": 1650
+    },
+    {
+      "epoch": 10.476190476190476,
+      "grad_norm": 1.293314814567566,
+      "learning_rate": 4.097724831989733e-05,
+      "loss": 0.2523,
+      "step": 1655
+    },
+    {
+      "epoch": 10.507936507936508,
+      "grad_norm": 2.517212152481079,
+      "learning_rate": 4.09058867394271e-05,
+      "loss": 0.3269,
+      "step": 1660
+    },
+    {
+      "epoch": 10.53968253968254,
+      "grad_norm": 2.057063102722168,
+      "learning_rate": 4.083430675006791e-05,
+      "loss": 0.2844,
+      "step": 1665
+    },
+    {
+      "epoch": 10.571428571428571,
+      "grad_norm": 1.5663833618164062,
+      "learning_rate": 4.0762509334707786e-05,
+      "loss": 0.3005,
+      "step": 1670
+    },
+    {
+      "epoch": 10.603174603174603,
+      "grad_norm": 2.5423505306243896,
+      "learning_rate": 4.069049547922035e-05,
+      "loss": 0.2802,
+      "step": 1675
+    },
+    {
+      "epoch": 10.634920634920634,
+      "grad_norm": 1.578316569328308,
+      "learning_rate": 4.061826617245119e-05,
+      "loss": 0.2667,
+      "step": 1680
+    },
+    {
+      "epoch": 10.666666666666666,
+      "grad_norm": 1.502928376197815,
+      "learning_rate": 4.0545822406204334e-05,
+      "loss": 0.3059,
+      "step": 1685
+    },
+    {
+      "epoch": 10.698412698412698,
+      "grad_norm": 1.2470905780792236,
+      "learning_rate": 4.047316517522864e-05,
+      "loss": 0.2879,
+      "step": 1690
+    },
+    {
+      "epoch": 10.73015873015873,
+      "grad_norm": 1.8238775730133057,
+      "learning_rate": 4.0400295477204105e-05,
+      "loss": 0.2923,
+      "step": 1695
+    },
+    {
+      "epoch": 10.761904761904763,
+      "grad_norm": 2.0516586303710938,
+      "learning_rate": 4.032721431272819e-05,
+      "loss": 0.3086,
+      "step": 1700
+    },
+    {
+      "epoch": 10.793650793650794,
+      "grad_norm": 1.3188791275024414,
+      "learning_rate": 4.0253922685302046e-05,
+      "loss": 0.2893,
+      "step": 1705
+    },
+    {
+      "epoch": 10.825396825396826,
+      "grad_norm": 1.7352266311645508,
+      "learning_rate": 4.01804216013168e-05,
+      "loss": 0.2981,
+      "step": 1710
+    },
+    {
+      "epoch": 10.857142857142858,
+      "grad_norm": 1.3449515104293823,
+      "learning_rate": 4.0106712070039656e-05,
+      "loss": 0.2841,
+      "step": 1715
+    },
+    {
+      "epoch": 10.88888888888889,
+      "grad_norm": 2.505431890487671,
+      "learning_rate": 4.00327951036001e-05,
+      "loss": 0.3034,
+      "step": 1720
+    },
+    {
+      "epoch": 10.920634920634921,
+      "grad_norm": 1.8870325088500977,
+      "learning_rate": 3.9958671716975966e-05,
+      "loss": 0.305,
+      "step": 1725
+    },
+    {
+      "epoch": 10.952380952380953,
+      "grad_norm": 2.913130044937134,
+      "learning_rate": 3.988434292797951e-05,
+      "loss": 0.3212,
+      "step": 1730
+    },
+    {
+      "epoch": 10.984126984126984,
+      "grad_norm": 1.7870115041732788,
+      "learning_rate": 3.980980975724344e-05,
+      "loss": 0.3108,
+      "step": 1735
+    },
+    {
+      "epoch": 11.012698412698413,
+      "grad_norm": 3.050985336303711,
+      "learning_rate": 3.9735073228206896e-05,
+      "loss": 0.3043,
+      "step": 1740
+    },
+    {
+      "epoch": 11.044444444444444,
+      "grad_norm": 1.5993611812591553,
+      "learning_rate": 3.96601343671014e-05,
+      "loss": 0.2465,
+      "step": 1745
+    },
+    {
+      "epoch": 11.076190476190476,
+      "grad_norm": 1.626888632774353,
+      "learning_rate": 3.9584994202936746e-05,
+      "loss": 0.2688,
+      "step": 1750
+    },
+    {
+      "epoch": 11.107936507936508,
+      "grad_norm": 1.7132880687713623,
+      "learning_rate": 3.950965376748689e-05,
+      "loss": 0.2458,
+      "step": 1755
+    },
+    {
+      "epoch": 11.13968253968254,
+      "grad_norm": 1.7764930725097656,
+      "learning_rate": 3.94341140952758e-05,
+      "loss": 0.2189,
+      "step": 1760
+    },
+    {
+      "epoch": 11.17142857142857,
+      "grad_norm": 2.5560712814331055,
+      "learning_rate": 3.9358376223563206e-05,
+      "loss": 0.2866,
+      "step": 1765
+    },
+    {
+      "epoch": 11.203174603174602,
+      "grad_norm": 1.1177359819412231,
+      "learning_rate": 3.928244119233038e-05,
+      "loss": 0.233,
+      "step": 1770
+    },
+    {
+      "epoch": 11.234920634920634,
+      "grad_norm": 1.584670901298523,
+      "learning_rate": 3.9206310044265866e-05,
+      "loss": 0.273,
+      "step": 1775
+    },
+    {
+      "epoch": 11.266666666666667,
+      "grad_norm": 1.6278687715530396,
+      "learning_rate": 3.912998382475115e-05,
+      "loss": 0.2746,
+      "step": 1780
+    },
+    {
+      "epoch": 11.2984126984127,
+      "grad_norm": 1.657038688659668,
+      "learning_rate": 3.905346358184629e-05,
+      "loss": 0.2885,
+      "step": 1785
+    },
+    {
+      "epoch": 11.33015873015873,
+      "grad_norm": 1.2840272188186646,
+      "learning_rate": 3.897675036627557e-05,
+      "loss": 0.2932,
+      "step": 1790
+    },
+    {
+      "epoch": 11.361904761904762,
+      "grad_norm": 1.5678766965866089,
+      "learning_rate": 3.8899845231413026e-05,
+      "loss": 0.2945,
+      "step": 1795
+    },
+    {
+      "epoch": 11.393650793650794,
+      "grad_norm": 1.788948655128479,
+      "learning_rate": 3.8822749233268006e-05,
+      "loss": 0.3013,
+      "step": 1800
+    },
+    {
+      "epoch": 11.425396825396826,
+      "grad_norm": 1.2259769439697266,
+      "learning_rate": 3.8745463430470664e-05,
+      "loss": 0.2582,
+      "step": 1805
+    },
+    {
+      "epoch": 11.457142857142857,
+      "grad_norm": 1.5430735349655151,
+      "learning_rate": 3.866798888425741e-05,
+      "loss": 0.275,
+      "step": 1810
+    },
+    {
+      "epoch": 11.488888888888889,
+      "grad_norm": 1.9102168083190918,
+      "learning_rate": 3.8590326658456376e-05,
+      "loss": 0.2909,
+      "step": 1815
+    },
+    {
+      "epoch": 11.52063492063492,
+      "grad_norm": 1.6118320226669312,
+      "learning_rate": 3.851247781947277e-05,
+      "loss": 0.2922,
+      "step": 1820
+    },
+    {
+      "epoch": 11.552380952380952,
+      "grad_norm": 1.393646478652954,
+      "learning_rate": 3.843444343627424e-05,
+      "loss": 0.2783,
+      "step": 1825
+    },
+    {
+      "epoch": 11.584126984126984,
+      "grad_norm": 2.522909641265869,
+      "learning_rate": 3.83562245803762e-05,
+      "loss": 0.2933,
+      "step": 1830
+    },
+    {
+      "epoch": 11.615873015873015,
+      "grad_norm": 2.2534332275390625,
+      "learning_rate": 3.827782232582714e-05,
+      "loss": 0.3081,
+      "step": 1835
+    },
+    {
+      "epoch": 11.647619047619047,
+      "grad_norm": 2.4088056087493896,
+      "learning_rate": 3.819923774919383e-05,
+      "loss": 0.276,
+      "step": 1840
+    },
+    {
+      "epoch": 11.679365079365079,
+      "grad_norm": 1.7626562118530273,
+      "learning_rate": 3.8120471929546576e-05,
+      "loss": 0.2697,
+      "step": 1845
+    },
+    {
+      "epoch": 11.71111111111111,
+      "grad_norm": 1.8656691312789917,
+      "learning_rate": 3.8041525948444414e-05,
+      "loss": 0.2979,
+      "step": 1850
+    },
+    {
+      "epoch": 11.742857142857144,
+      "grad_norm": 1.6537258625030518,
+      "learning_rate": 3.7962400889920185e-05,
+      "loss": 0.3042,
+      "step": 1855
+    },
+    {
+      "epoch": 11.774603174603175,
+      "grad_norm": 1.696975827217102,
+      "learning_rate": 3.788309784046574e-05,
+      "loss": 0.2984,
+      "step": 1860
+    },
+    {
+      "epoch": 11.806349206349207,
+      "grad_norm": 1.976236343383789,
+      "learning_rate": 3.780361788901696e-05,
+      "loss": 0.2711,
+      "step": 1865
+    },
+    {
+      "epoch": 11.838095238095239,
+      "grad_norm": 1.7781676054000854,
+      "learning_rate": 3.772396212693885e-05,
+      "loss": 0.3116,
+      "step": 1870
+    },
+    {
+      "epoch": 11.86984126984127,
+      "grad_norm": 1.6252080202102661,
+      "learning_rate": 3.7644131648010494e-05,
+      "loss": 0.2879,
+      "step": 1875
+    },
+    {
+      "epoch": 11.901587301587302,
+      "grad_norm": 1.7511012554168701,
+      "learning_rate": 3.75641275484101e-05,
+      "loss": 0.3106,
+      "step": 1880
+    },
+    {
+      "epoch": 11.933333333333334,
+      "grad_norm": 1.5861046314239502,
+      "learning_rate": 3.7483950926699885e-05,
+      "loss": 0.2703,
+      "step": 1885
+    },
+    {
+      "epoch": 11.965079365079365,
+      "grad_norm": 1.903902530670166,
+      "learning_rate": 3.740360288381105e-05,
+      "loss": 0.2808,
+      "step": 1890
+    },
+    {
+      "epoch": 11.996825396825397,
+      "grad_norm": 1.644740104675293,
+      "learning_rate": 3.732308452302864e-05,
+      "loss": 0.2883,
+      "step": 1895
+    },
+    {
+      "epoch": 12.025396825396825,
+      "grad_norm": 1.3671797513961792,
+      "learning_rate": 3.724239694997637e-05,
+      "loss": 0.2661,
+      "step": 1900
+    },
+    {
+      "epoch": 12.057142857142857,
+      "grad_norm": 1.1910501718521118,
+      "learning_rate": 3.716154127260147e-05,
+      "loss": 0.2352,
+      "step": 1905
+    },
+    {
+      "epoch": 12.088888888888889,
+      "grad_norm": 1.45137619972229,
+      "learning_rate": 3.708051860115947e-05,
+      "loss": 0.2703,
+      "step": 1910
+    },
+    {
+      "epoch": 12.12063492063492,
+      "grad_norm": 2.625089645385742,
+      "learning_rate": 3.699933004819895e-05,
+      "loss": 0.2524,
+      "step": 1915
+    },
+    {
+      "epoch": 12.152380952380952,
+      "grad_norm": 1.731602430343628,
+      "learning_rate": 3.691797672854625e-05,
+      "loss": 0.2533,
+      "step": 1920
+    },
+    {
+      "epoch": 12.184126984126983,
+      "grad_norm": 2.1266143321990967,
+      "learning_rate": 3.683645975929019e-05,
+      "loss": 0.2666,
+      "step": 1925
+    },
+    {
+      "epoch": 12.215873015873015,
+      "grad_norm": 1.7163398265838623,
+      "learning_rate": 3.675478025976671e-05,
+      "loss": 0.2838,
+      "step": 1930
+    },
+    {
+      "epoch": 12.247619047619047,
+      "grad_norm": 1.8726729154586792,
+      "learning_rate": 3.66729393515435e-05,
+      "loss": 0.2646,
+      "step": 1935
+    },
+    {
+      "epoch": 12.27936507936508,
+      "grad_norm": 1.2229722738265991,
+      "learning_rate": 3.659093815840462e-05,
+      "loss": 0.267,
+      "step": 1940
+    },
+    {
+      "epoch": 12.311111111111112,
+      "grad_norm": 1.341997504234314,
+      "learning_rate": 3.650877780633505e-05,
+      "loss": 0.2464,
+      "step": 1945
+    },
+    {
+      "epoch": 12.342857142857143,
+      "grad_norm": 1.3466174602508545,
+      "learning_rate": 3.6426459423505214e-05,
+      "loss": 0.2521,
+      "step": 1950
+    },
+    {
+      "epoch": 12.374603174603175,
+      "grad_norm": 1.7022134065628052,
+      "learning_rate": 3.6343984140255516e-05,
+      "loss": 0.2662,
+      "step": 1955
+    },
+    {
+      "epoch": 12.406349206349207,
+      "grad_norm": 2.5058987140655518,
+      "learning_rate": 3.626135308908084e-05,
+      "loss": 0.2745,
+      "step": 1960
+    },
+    {
+      "epoch": 12.438095238095238,
+      "grad_norm": 1.7667903900146484,
+      "learning_rate": 3.6178567404614936e-05,
+      "loss": 0.2752,
+      "step": 1965
+    },
+    {
+      "epoch": 12.46984126984127,
+      "grad_norm": 1.7334171533584595,
+      "learning_rate": 3.609562822361487e-05,
+      "loss": 0.2667,
+      "step": 1970
+    },
+    {
+      "epoch": 12.501587301587302,
+      "grad_norm": 1.7093929052352905,
+      "learning_rate": 3.601253668494546e-05,
+      "loss": 0.2829,
+      "step": 1975
+    },
+    {
+      "epoch": 12.533333333333333,
+      "grad_norm": 1.8272080421447754,
+      "learning_rate": 3.592929392956355e-05,
+      "loss": 0.2583,
+      "step": 1980
+    },
+    {
+      "epoch": 12.565079365079365,
+      "grad_norm": 1.5226343870162964,
+      "learning_rate": 3.584590110050241e-05,
+      "loss": 0.2652,
+      "step": 1985
+    },
+    {
+      "epoch": 12.596825396825396,
+      "grad_norm": 1.7395323514938354,
+      "learning_rate": 3.5762359342856036e-05,
+      "loss": 0.2585,
+      "step": 1990
+    },
+    {
+      "epoch": 12.628571428571428,
+      "grad_norm": 2.278144359588623,
+      "learning_rate": 3.567866980376337e-05,
+      "loss": 0.2862,
+      "step": 1995
+    },
+    {
+      "epoch": 12.66031746031746,
+      "grad_norm": 1.9335429668426514,
+      "learning_rate": 3.559483363239262e-05,
+      "loss": 0.2945,
+      "step": 2000
+    },
+    {
+      "epoch": 12.692063492063491,
+      "grad_norm": 1.3866902589797974,
+      "learning_rate": 3.551085197992545e-05,
+      "loss": 0.2675,
+      "step": 2005
+    },
+    {
+      "epoch": 12.723809523809523,
+      "grad_norm": 1.7110604047775269,
+      "learning_rate": 3.5426725999541174e-05,
+      "loss": 0.2705,
+      "step": 2010
+    },
+    {
+      "epoch": 12.755555555555556,
+      "grad_norm": 1.6411776542663574,
+      "learning_rate": 3.534245684640089e-05,
+      "loss": 0.2482,
+      "step": 2015
+    },
+    {
+      "epoch": 12.787301587301588,
+      "grad_norm": 1.4663981199264526,
+      "learning_rate": 3.525804567763167e-05,
+      "loss": 0.2838,
+      "step": 2020
+    },
+    {
+      "epoch": 12.81904761904762,
+      "grad_norm": 1.8676140308380127,
+      "learning_rate": 3.517349365231065e-05,
+      "loss": 0.2906,
+      "step": 2025
+    },
+    {
+      "epoch": 12.850793650793651,
+      "grad_norm": 1.6150639057159424,
+      "learning_rate": 3.508880193144911e-05,
+      "loss": 0.3011,
+      "step": 2030
+    },
+    {
+      "epoch": 12.882539682539683,
+      "grad_norm": 2.5578598976135254,
+      "learning_rate": 3.500397167797654e-05,
+      "loss": 0.2846,
+      "step": 2035
+    },
+    {
+      "epoch": 12.914285714285715,
+      "grad_norm": 1.3131941556930542,
+      "learning_rate": 3.491900405672466e-05,
+      "loss": 0.2509,
+      "step": 2040
+    },
+    {
+      "epoch": 12.946031746031746,
+      "grad_norm": 1.5713857412338257,
+      "learning_rate": 3.483390023441141e-05,
+      "loss": 0.2824,
+      "step": 2045
+    },
+    {
+      "epoch": 12.977777777777778,
+      "grad_norm": 1.788149356842041,
+      "learning_rate": 3.4748661379625e-05,
+      "loss": 0.2932,
+      "step": 2050
+    },
+    {
+      "epoch": 13.006349206349206,
+      "grad_norm": 1.080249547958374,
+      "learning_rate": 3.466328866280778e-05,
+      "loss": 0.2817,
+      "step": 2055
+    },
+    {
+      "epoch": 13.038095238095238,
+      "grad_norm": 1.1128981113433838,
+      "learning_rate": 3.45777832562402e-05,
+      "loss": 0.2141,
+      "step": 2060
+    },
+    {
+      "epoch": 13.06984126984127,
+      "grad_norm": 1.6394922733306885,
+      "learning_rate": 3.449214633402473e-05,
+      "loss": 0.216,
+      "step": 2065
+    },
+    {
+      "epoch": 13.101587301587301,
+      "grad_norm": 2.198416233062744,
+      "learning_rate": 3.440637907206973e-05,
+      "loss": 0.258,
+      "step": 2070
+    },
+    {
+      "epoch": 13.133333333333333,
+      "grad_norm": 1.5107051134109497,
+      "learning_rate": 3.4320482648073285e-05,
+      "loss": 0.2422,
+      "step": 2075
+    },
+    {
+      "epoch": 13.165079365079364,
+      "grad_norm": 1.7206802368164062,
+      "learning_rate": 3.4234458241507034e-05,
+      "loss": 0.2524,
+      "step": 2080
+    },
+    {
+      "epoch": 13.196825396825396,
+      "grad_norm": 1.4832013845443726,
+      "learning_rate": 3.4148307033600014e-05,
+      "loss": 0.27,
+      "step": 2085
+    },
+    {
+      "epoch": 13.228571428571428,
+      "grad_norm": 1.592344880104065,
+      "learning_rate": 3.4062030207322395e-05,
+      "loss": 0.2486,
+      "step": 2090
+    },
+    {
+      "epoch": 13.260317460317461,
+      "grad_norm": 1.7398722171783447,
+      "learning_rate": 3.397562894736926e-05,
+      "loss": 0.2605,
+      "step": 2095
+    },
+    {
+      "epoch": 13.292063492063493,
+      "grad_norm": 2.0896050930023193,
+      "learning_rate": 3.388910444014432e-05,
+      "loss": 0.2562,
+      "step": 2100
+    },
+    {
+      "epoch": 13.323809523809524,
+      "grad_norm": 2.7245311737060547,
+      "learning_rate": 3.380245787374365e-05,
+      "loss": 0.2573,
+      "step": 2105
+    },
+    {
+      "epoch": 13.355555555555556,
+      "grad_norm": 1.1951723098754883,
+      "learning_rate": 3.371569043793936e-05,
+      "loss": 0.2709,
+      "step": 2110
+    },
+    {
+      "epoch": 13.387301587301588,
+      "grad_norm": 1.697736144065857,
+      "learning_rate": 3.3628803324163236e-05,
+      "loss": 0.2413,
+      "step": 2115
+    },
+    {
+      "epoch": 13.41904761904762,
+      "grad_norm": 1.447302222251892,
+      "learning_rate": 3.354179772549042e-05,
+      "loss": 0.2685,
+      "step": 2120
+    },
+    {
+      "epoch": 13.450793650793651,
+      "grad_norm": 1.232428789138794,
+      "learning_rate": 3.3454674836623e-05,
+      "loss": 0.2546,
+      "step": 2125
+    },
+    {
+      "epoch": 13.482539682539683,
+      "grad_norm": 1.889103889465332,
+      "learning_rate": 3.336743585387362e-05,
+      "loss": 0.2412,
+      "step": 2130
+    },
+    {
+      "epoch": 13.514285714285714,
+      "grad_norm": 1.888352632522583,
+      "learning_rate": 3.328008197514903e-05,
+      "loss": 0.2737,
+      "step": 2135
+    },
+    {
+      "epoch": 13.546031746031746,
+      "grad_norm": 1.4205390214920044,
+      "learning_rate": 3.319261439993369e-05,
+      "loss": 0.2727,
+      "step": 2140
+    },
+    {
+      "epoch": 13.577777777777778,
+      "grad_norm": 1.442429542541504,
+      "learning_rate": 3.310503432927322e-05,
+      "loss": 0.2894,
+      "step": 2145
+    },
+    {
+      "epoch": 13.60952380952381,
+      "grad_norm": 1.413885474205017,
+      "learning_rate": 3.3017342965757986e-05,
+      "loss": 0.2928,
+      "step": 2150
+    },
+    {
+      "epoch": 13.64126984126984,
+      "grad_norm": 1.0464482307434082,
+      "learning_rate": 3.2929541513506524e-05,
+      "loss": 0.252,
+      "step": 2155
+    },
+    {
+      "epoch": 13.673015873015872,
+      "grad_norm": 1.6716376543045044,
+      "learning_rate": 3.284163117814906e-05,
+      "loss": 0.2614,
+      "step": 2160
+    },
+    {
+      "epoch": 13.704761904761904,
+      "grad_norm": 1.9366419315338135,
+      "learning_rate": 3.2753613166810914e-05,
+      "loss": 0.2963,
+      "step": 2165
+    },
+    {
+      "epoch": 13.736507936507937,
+      "grad_norm": 1.4641704559326172,
+      "learning_rate": 3.2665488688095944e-05,
+      "loss": 0.2654,
+      "step": 2170
+    },
+    {
+      "epoch": 13.768253968253969,
+      "grad_norm": 1.3404383659362793,
+      "learning_rate": 3.2577258952069934e-05,
+      "loss": 0.2601,
+      "step": 2175
+    },
+    {
+      "epoch": 13.8,
+      "grad_norm": 1.6471940279006958,
+      "learning_rate": 3.2488925170244015e-05,
+      "loss": 0.2775,
+      "step": 2180
+    },
+    {
+      "epoch": 13.831746031746032,
+      "grad_norm": 1.3665790557861328,
+      "learning_rate": 3.240048855555798e-05,
+      "loss": 0.2602,
+      "step": 2185
+    },
+    {
+      "epoch": 13.863492063492064,
+      "grad_norm": 1.1486542224884033,
+      "learning_rate": 3.2311950322363685e-05,
+      "loss": 0.2603,
+      "step": 2190
+    },
+    {
+      "epoch": 13.895238095238096,
+      "grad_norm": 1.45487642288208,
+      "learning_rate": 3.2223311686408304e-05,
+      "loss": 0.2794,
+      "step": 2195
+    },
+    {
+      "epoch": 13.926984126984127,
+      "grad_norm": 1.6884818077087402,
+      "learning_rate": 3.213457386481769e-05,
+      "loss": 0.2682,
+      "step": 2200
+    },
+    {
+      "epoch": 13.958730158730159,
+      "grad_norm": 1.9160594940185547,
+      "learning_rate": 3.204573807607967e-05,
+      "loss": 0.2619,
+      "step": 2205
+    },
+    {
+      "epoch": 13.99047619047619,
+      "grad_norm": 1.9444315433502197,
+      "learning_rate": 3.1956805540027244e-05,
+      "loss": 0.3029,
+      "step": 2210
+    },
+    {
+      "epoch": 14.019047619047619,
+      "grad_norm": 1.8739337921142578,
+      "learning_rate": 3.1867777477821915e-05,
+      "loss": 0.2298,
+      "step": 2215
+    },
+    {
+      "epoch": 14.05079365079365,
+      "grad_norm": 1.3971226215362549,
+      "learning_rate": 3.1778655111936866e-05,
+      "loss": 0.2351,
+      "step": 2220
+    },
+    {
+      "epoch": 14.082539682539682,
+      "grad_norm": 1.288387417793274,
+      "learning_rate": 3.168943966614022e-05,
+      "loss": 0.2331,
+      "step": 2225
+    },
+    {
+      "epoch": 14.114285714285714,
+      "grad_norm": 1.2136940956115723,
+      "learning_rate": 3.1600132365478186e-05,
+      "loss": 0.2406,
+      "step": 2230
+    },
+    {
+      "epoch": 14.146031746031746,
+      "grad_norm": 1.8233654499053955,
+      "learning_rate": 3.151073443625828e-05,
+      "loss": 0.2496,
+      "step": 2235
+    },
+    {
+      "epoch": 14.177777777777777,
+      "grad_norm": 1.7613590955734253,
+      "learning_rate": 3.142124710603245e-05,
+      "loss": 0.2523,
+      "step": 2240
+    },
+    {
+      "epoch": 14.209523809523809,
+      "grad_norm": 1.8933606147766113,
+      "learning_rate": 3.133167160358024e-05,
+      "loss": 0.2646,
+      "step": 2245
+    },
+    {
+      "epoch": 14.24126984126984,
+      "grad_norm": 2.7858712673187256,
+      "learning_rate": 3.124200915889195e-05,
+      "loss": 0.2818,
+      "step": 2250
+    },
+    {
+      "epoch": 14.273015873015874,
+      "grad_norm": 1.5656853914260864,
+      "learning_rate": 3.115226100315166e-05,
+      "loss": 0.2389,
+      "step": 2255
+    },
+    {
+      "epoch": 14.304761904761905,
+      "grad_norm": 1.606392741203308,
+      "learning_rate": 3.106242836872041e-05,
+      "loss": 0.2302,
+      "step": 2260
+    },
+    {
+      "epoch": 14.336507936507937,
+      "grad_norm": 2.1553008556365967,
+      "learning_rate": 3.097251248911922e-05,
+      "loss": 0.2484,
+      "step": 2265
+    },
+    {
+      "epoch": 14.368253968253969,
+      "grad_norm": 1.7176517248153687,
+      "learning_rate": 3.088251459901222e-05,
+      "loss": 0.268,
+      "step": 2270
+    },
+    {
+      "epoch": 14.4,
+      "grad_norm": 1.3635174036026,
+      "learning_rate": 3.0792435934189604e-05,
+      "loss": 0.2491,
+      "step": 2275
+    },
+    {
+      "epoch": 14.431746031746032,
+      "grad_norm": 1.4257878065109253,
+      "learning_rate": 3.070227773155074e-05,
+      "loss": 0.2884,
+      "step": 2280
+    },
+    {
+      "epoch": 14.463492063492064,
+      "grad_norm": 1.1561063528060913,
+      "learning_rate": 3.0612041229087145e-05,
+      "loss": 0.247,
+      "step": 2285
+    },
+    {
+      "epoch": 14.495238095238095,
+      "grad_norm": 1.6131694316864014,
+      "learning_rate": 3.0521727665865485e-05,
+      "loss": 0.2708,
+      "step": 2290
+    },
+    {
+      "epoch": 14.526984126984127,
+      "grad_norm": 1.7133935689926147,
+      "learning_rate": 3.0431338282010606e-05,
+      "loss": 0.2384,
+      "step": 2295
+    },
+    {
+      "epoch": 14.558730158730159,
+      "grad_norm": 1.3232437372207642,
+      "learning_rate": 3.0340874318688433e-05,
+      "loss": 0.2503,
+      "step": 2300
+    },
+    {
+      "epoch": 14.59047619047619,
+      "grad_norm": 1.5612751245498657,
+      "learning_rate": 3.0250337018088994e-05,
+      "loss": 0.2618,
+      "step": 2305
+    },
+    {
+      "epoch": 14.622222222222222,
+      "grad_norm": 1.279241681098938,
+      "learning_rate": 3.0159727623409313e-05,
+      "loss": 0.2535,
+      "step": 2310
+    },
+    {
+      "epoch": 14.653968253968253,
+      "grad_norm": 3.142040729522705,
+      "learning_rate": 3.0069047378836375e-05,
+      "loss": 0.2745,
+      "step": 2315
+    },
+    {
+      "epoch": 14.685714285714285,
+      "grad_norm": 1.3240234851837158,
+      "learning_rate": 2.9978297529530036e-05,
+      "loss": 0.2591,
+      "step": 2320
+    },
+    {
+      "epoch": 14.717460317460317,
+      "grad_norm": 1.121077537536621,
+      "learning_rate": 2.9887479321605895e-05,
+      "loss": 0.2453,
+      "step": 2325
+    },
+    {
+      "epoch": 14.74920634920635,
+      "grad_norm": 2.3831980228424072,
+      "learning_rate": 2.9796594002118218e-05,
+      "loss": 0.257,
+      "step": 2330
+    },
+    {
+      "epoch": 14.780952380952382,
+      "grad_norm": 2.175352096557617,
+      "learning_rate": 2.9705642819042805e-05,
+      "loss": 0.2434,
+      "step": 2335
+    },
+    {
+      "epoch": 14.812698412698413,
+      "grad_norm": 1.5452053546905518,
+      "learning_rate": 2.9614627021259846e-05,
+      "loss": 0.2683,
+      "step": 2340
+    },
+    {
+      "epoch": 14.844444444444445,
+      "grad_norm": 1.6765031814575195,
+      "learning_rate": 2.952354785853677e-05,
+      "loss": 0.2484,
+      "step": 2345
+    },
+    {
+      "epoch": 14.876190476190477,
+      "grad_norm": 1.3790463209152222,
+      "learning_rate": 2.9432406581511096e-05,
+      "loss": 0.2636,
+      "step": 2350
+    },
+    {
+      "epoch": 14.907936507936508,
+      "grad_norm": 1.2849820852279663,
+      "learning_rate": 2.9341204441673266e-05,
+      "loss": 0.2734,
+      "step": 2355
+    },
+    {
+      "epoch": 14.93968253968254,
+      "grad_norm": 1.5512354373931885,
+      "learning_rate": 2.924994269134942e-05,
+      "loss": 0.2627,
+      "step": 2360
+    },
+    {
+      "epoch": 14.971428571428572,
+      "grad_norm": 1.2973589897155762,
+      "learning_rate": 2.9158622583684264e-05,
+      "loss": 0.2464,
+      "step": 2365
+    },
+    {
+      "epoch": 15.0,
+      "grad_norm": 2.749181032180786,
+      "learning_rate": 2.906724537262381e-05,
+      "loss": 0.2622,
+      "step": 2370
+    },
+    {
+      "epoch": 15.031746031746032,
+      "grad_norm": 1.4415572881698608,
+      "learning_rate": 2.8975812312898152e-05,
+      "loss": 0.2395,
+      "step": 2375
+    },
+    {
+      "epoch": 15.063492063492063,
+      "grad_norm": 1.41774320602417,
+      "learning_rate": 2.8884324660004325e-05,
+      "loss": 0.2213,
+      "step": 2380
+    },
+    {
+      "epoch": 15.095238095238095,
+      "grad_norm": 1.375585675239563,
+      "learning_rate": 2.8792783670188927e-05,
+      "loss": 0.2319,
+      "step": 2385
+    },
+    {
+      "epoch": 15.126984126984127,
+      "grad_norm": 1.2285529375076294,
+      "learning_rate": 2.870119060043099e-05,
+      "loss": 0.2317,
+      "step": 2390
+    },
+    {
+      "epoch": 15.158730158730158,
+      "grad_norm": 2.5581822395324707,
+      "learning_rate": 2.8609546708424645e-05,
+      "loss": 0.2567,
+      "step": 2395
+    },
+    {
+      "epoch": 15.19047619047619,
+      "grad_norm": 2.0000274181365967,
+      "learning_rate": 2.8517853252561906e-05,
+      "loss": 0.2265,
+      "step": 2400
+    },
+    {
+      "epoch": 15.222222222222221,
+      "grad_norm": 1.6797828674316406,
+      "learning_rate": 2.8426111491915342e-05,
+      "loss": 0.243,
+      "step": 2405
+    },
+    {
+      "epoch": 15.253968253968253,
+      "grad_norm": 1.8280093669891357,
+      "learning_rate": 2.8334322686220826e-05,
+      "loss": 0.2442,
+      "step": 2410
+    },
+    {
+      "epoch": 15.285714285714286,
+      "grad_norm": 1.6335142850875854,
+      "learning_rate": 2.824248809586021e-05,
+      "loss": 0.229,
+      "step": 2415
+    },
+    {
+      "epoch": 15.317460317460318,
+      "grad_norm": 2.1388628482818604,
+      "learning_rate": 2.8150608981844034e-05,
+      "loss": 0.247,
+      "step": 2420
+    },
+    {
+      "epoch": 15.34920634920635,
+      "grad_norm": 1.5680638551712036,
+      "learning_rate": 2.805868660579422e-05,
+      "loss": 0.2269,
+      "step": 2425
+    },
+    {
+      "epoch": 15.380952380952381,
+      "grad_norm": 1.6333919763565063,
+      "learning_rate": 2.7966722229926712e-05,
+      "loss": 0.2426,
+      "step": 2430
+    },
+    {
+      "epoch": 15.412698412698413,
+      "grad_norm": 1.4023231267929077,
+      "learning_rate": 2.787471711703419e-05,
+      "loss": 0.2466,
+      "step": 2435
+    },
+    {
+      "epoch": 15.444444444444445,
+      "grad_norm": 1.5598480701446533,
+      "learning_rate": 2.7782672530468672e-05,
+      "loss": 0.2504,
+      "step": 2440
+    },
+    {
+      "epoch": 15.476190476190476,
+      "grad_norm": 1.3230295181274414,
+      "learning_rate": 2.769058973412424e-05,
+      "loss": 0.2478,
+      "step": 2445
+    },
+    {
+      "epoch": 15.507936507936508,
+      "grad_norm": 2.0796375274658203,
+      "learning_rate": 2.7598469992419646e-05,
+      "loss": 0.2382,
+      "step": 2450
+    },
+    {
+      "epoch": 15.53968253968254,
+      "grad_norm": 1.6772873401641846,
+      "learning_rate": 2.7506314570280916e-05,
+      "loss": 0.2716,
+      "step": 2455
+    },
+    {
+      "epoch": 15.571428571428571,
+      "grad_norm": 1.9949897527694702,
+      "learning_rate": 2.7414124733124046e-05,
+      "loss": 0.2541,
+      "step": 2460
+    },
+    {
+      "epoch": 15.603174603174603,
+      "grad_norm": 1.339775562286377,
+      "learning_rate": 2.7321901746837585e-05,
+      "loss": 0.238,
+      "step": 2465
+    },
+    {
+      "epoch": 15.634920634920634,
+      "grad_norm": 1.3870205879211426,
+      "learning_rate": 2.7229646877765264e-05,
+      "loss": 0.2377,
+      "step": 2470
+    },
+    {
+      "epoch": 15.666666666666666,
+      "grad_norm": 2.1485989093780518,
+      "learning_rate": 2.7137361392688613e-05,
+      "loss": 0.2748,
+      "step": 2475
+    },
+    {
+      "epoch": 15.698412698412698,
+      "grad_norm": 1.3165838718414307,
+      "learning_rate": 2.704504655880955e-05,
+      "loss": 0.2592,
+      "step": 2480
+    },
+    {
+      "epoch": 15.73015873015873,
+      "grad_norm": 1.5217639207839966,
+      "learning_rate": 2.6952703643732983e-05,
+      "loss": 0.2359,
+      "step": 2485
+    },
+    {
+      "epoch": 15.761904761904763,
+      "grad_norm": 1.6285779476165771,
+      "learning_rate": 2.686033391544945e-05,
+      "loss": 0.2653,
+      "step": 2490
+    },
+    {
+      "epoch": 15.793650793650794,
+      "grad_norm": 1.6619691848754883,
+      "learning_rate": 2.676793864231763e-05,
+      "loss": 0.2572,
+      "step": 2495
+    },
+    {
+      "epoch": 15.825396825396826,
+      "grad_norm": 1.6636338233947754,
+      "learning_rate": 2.6675519093046987e-05,
+      "loss": 0.2702,
+      "step": 2500
+    },
+    {
+      "epoch": 15.857142857142858,
+      "grad_norm": 1.6794034242630005,
+      "learning_rate": 2.6583076536680323e-05,
+      "loss": 0.2508,
+      "step": 2505
+    },
+    {
+      "epoch": 15.88888888888889,
+      "grad_norm": 1.669232964515686,
+      "learning_rate": 2.6490612242576344e-05,
+      "loss": 0.2601,
+      "step": 2510
+    },
+    {
+      "epoch": 15.920634920634921,
+      "grad_norm": 1.4551194906234741,
+      "learning_rate": 2.639812748039228e-05,
+      "loss": 0.2514,
+      "step": 2515
+    },
+    {
+      "epoch": 15.952380952380953,
+      "grad_norm": 1.8457551002502441,
+      "learning_rate": 2.6305623520066382e-05,
+      "loss": 0.2682,
+      "step": 2520
+    },
+    {
+      "epoch": 15.984126984126984,
+      "grad_norm": 1.318781852722168,
+      "learning_rate": 2.6213101631800524e-05,
+      "loss": 0.2452,
+      "step": 2525
+    },
+    {
+      "epoch": 16.012698412698413,
+      "grad_norm": 1.4368094205856323,
+      "learning_rate": 2.6120563086042753e-05,
+      "loss": 0.2398,
+      "step": 2530
+    },
+    {
+      "epoch": 16.044444444444444,
+      "grad_norm": 1.314927101135254,
+      "learning_rate": 2.602800915346986e-05,
+      "loss": 0.2321,
+      "step": 2535
+    },
+    {
+      "epoch": 16.076190476190476,
+      "grad_norm": 1.501521110534668,
+      "learning_rate": 2.5935441104969914e-05,
+      "loss": 0.2329,
+      "step": 2540
+    },
+    {
+      "epoch": 16.107936507936508,
+      "grad_norm": 1.3708770275115967,
+      "learning_rate": 2.5842860211624802e-05,
+      "loss": 0.2272,
+      "step": 2545
+    },
+    {
+      "epoch": 16.13968253968254,
+      "grad_norm": 1.6570358276367188,
+      "learning_rate": 2.5750267744692786e-05,
+      "loss": 0.2323,
+      "step": 2550
+    },
+    {
+      "epoch": 16.17142857142857,
+      "grad_norm": 1.4252843856811523,
+      "learning_rate": 2.5657664975591063e-05,
+      "loss": 0.2308,
+      "step": 2555
+    },
+    {
+      "epoch": 16.203174603174602,
+      "grad_norm": 1.9588584899902344,
+      "learning_rate": 2.5565053175878285e-05,
+      "loss": 0.243,
+      "step": 2560
+    },
+    {
+      "epoch": 16.234920634920634,
+      "grad_norm": 1.7678033113479614,
+      "learning_rate": 2.5472433617237107e-05,
+      "loss": 0.2304,
+      "step": 2565
+    },
+    {
+      "epoch": 16.266666666666666,
+      "grad_norm": 1.45453679561615,
+      "learning_rate": 2.5379807571456727e-05,
+      "loss": 0.231,
+      "step": 2570
+    },
+    {
+      "epoch": 16.298412698412697,
+      "grad_norm": 1.313704490661621,
+      "learning_rate": 2.52871763104154e-05,
+      "loss": 0.2434,
+      "step": 2575
+    },
+    {
+      "epoch": 16.33015873015873,
+      "grad_norm": 1.331613302230835,
+      "learning_rate": 2.519454110606304e-05,
+      "loss": 0.2477,
+      "step": 2580
+    },
+    {
+      "epoch": 16.36190476190476,
+      "grad_norm": 1.5135482549667358,
+      "learning_rate": 2.510190323040366e-05,
+      "loss": 0.2302,
+      "step": 2585
+    },
+    {
+      "epoch": 16.393650793650792,
+      "grad_norm": 1.025403618812561,
+      "learning_rate": 2.5009263955477986e-05,
+      "loss": 0.2145,
+      "step": 2590
+    },
+    {
+      "epoch": 16.425396825396824,
+      "grad_norm": 1.3128396272659302,
+      "learning_rate": 2.491662455334592e-05,
+      "loss": 0.237,
+      "step": 2595
+    },
+    {
+      "epoch": 16.457142857142856,
+      "grad_norm": 1.9846177101135254,
+      "learning_rate": 2.482398629606916e-05,
+      "loss": 0.2227,
+      "step": 2600
+    },
+    {
+      "epoch": 16.488888888888887,
+      "grad_norm": 1.3919912576675415,
+      "learning_rate": 2.473135045569365e-05,
+      "loss": 0.2542,
+      "step": 2605
+    },
+    {
+      "epoch": 16.520634920634922,
+      "grad_norm": 1.4371297359466553,
+      "learning_rate": 2.463871830423215e-05,
+      "loss": 0.2533,
+      "step": 2610
+    },
+    {
+      "epoch": 16.552380952380954,
+      "grad_norm": 1.2560791969299316,
+      "learning_rate": 2.4546091113646774e-05,
+      "loss": 0.2425,
+      "step": 2615
+    },
+    {
+      "epoch": 16.584126984126986,
+      "grad_norm": 1.1686952114105225,
+      "learning_rate": 2.445347015583151e-05,
+      "loss": 0.2196,
+      "step": 2620
+    },
+    {
+      "epoch": 16.615873015873017,
+      "grad_norm": 1.4156274795532227,
+      "learning_rate": 2.436085670259479e-05,
+      "loss": 0.2514,
+      "step": 2625
+    },
+    {
+      "epoch": 16.64761904761905,
+      "grad_norm": 1.7230234146118164,
+      "learning_rate": 2.426825202564194e-05,
+      "loss": 0.24,
+      "step": 2630
+    },
+    {
+      "epoch": 16.67936507936508,
+      "grad_norm": 1.6709225177764893,
+      "learning_rate": 2.417565739655783e-05,
+      "loss": 0.2523,
+      "step": 2635
+    },
+    {
+      "epoch": 16.711111111111112,
+      "grad_norm": 1.4194560050964355,
+      "learning_rate": 2.4083074086789332e-05,
+      "loss": 0.2464,
+      "step": 2640
+    },
+    {
+      "epoch": 16.742857142857144,
+      "grad_norm": 1.3121566772460938,
+      "learning_rate": 2.3990503367627896e-05,
+      "loss": 0.234,
+      "step": 2645
+    },
+    {
+      "epoch": 16.774603174603175,
+      "grad_norm": 1.841577410697937,
+      "learning_rate": 2.3897946510192104e-05,
+      "loss": 0.2494,
+      "step": 2650
+    },
+    {
+      "epoch": 16.806349206349207,
+      "grad_norm": 1.262974739074707,
+      "learning_rate": 2.3805404785410157e-05,
+      "loss": 0.2648,
+      "step": 2655
+    },
+    {
+      "epoch": 16.83809523809524,
+      "grad_norm": 2.1865732669830322,
+      "learning_rate": 2.3712879464002504e-05,
+      "loss": 0.2486,
+      "step": 2660
+    },
+    {
+      "epoch": 16.86984126984127,
+      "grad_norm": 1.3114222288131714,
+      "learning_rate": 2.362037181646433e-05,
+      "loss": 0.2617,
+      "step": 2665
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 4710,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 30,
+  "save_steps": 157,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.732789348676403e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2826/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-2826/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-2826/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3611/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-3611/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-3611/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-3611/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ee23e4dcb29c1acafa71e412ed23177784ccbba1d925adea06c128efe5e320e
+size 14512

checkpoint-3611/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3611/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-3925/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

checkpoint-3925/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Coder-14B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-3925/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}