Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.2-11B-Vision-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/data/LLM/Llama-3.2-11B-Vision-Instruct",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": "(?:.*?(?:language|text).*?(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense).*?(?:q_proj|k_proj|v_proj|o_proj|fc1|fc2|gate_proj|up_proj|down_proj).*?)|(?:\\bmodel\\.layers\\.[\\d]{1,}\\.(?:self_attn|attention|attn|mlp|feed_forward|ffn|dense)\\.(?:(?:q_proj|k_proj|v_proj|o_proj|fc1|fc2|gate_proj|up_proj|down_proj)))",
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba9e42cf6ac08dfd6653a4924cb358c1d6a2ecd531ebb24f606bab5f9e87a378
+size 209798936

chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% for message in messages %}{% if loop.index0 == 0 %}{{ bos_token }}{% endif %}{{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n' }}{% if message['content'] is string %}{{ message['content'] }}{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' %}{{ '<|image|>' }}{% elif content['type'] == 'text' %}{{ content['text'] }}{% endif %}{% endfor %}{% endif %}{{ '<|eot_id|>' }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}"
+}

preprocessor_config.json

@@ -0,0 +1,26 @@
+{
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_pad": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "MllamaImageProcessor",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_image_tiles": 4,
+  "processor_class": "MllamaProcessor",
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 560,
+    "width": 560
+  }
+}

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d82bb8083bf36ff93a8a46a42c4519fe47f7264153c1c384bbab0980fd6fa2f0
+size 14244

special_tokens_map.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|finetune_right_pad_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc5f21c9c4c3b2e7521cc798bd2c3bd2a6625f8421601a5952b7581fb286ad0f
+size 17209159

tokenizer_config.json

@@ -0,0 +1,2074 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|step_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128256": {
+      "content": "<|image|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- if strftime_now is defined %}\n        {%- set date_string = strftime_now(\"%d %b %Y\") %}\n    {%- else %}\n        {%- set date_string = \"26 Jul 2024\" %}\n    {%- endif %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- Find out if there are any images #}\n{% set image_ns = namespace(has_images=false) %}      \n{%- for message in messages %}\n    {%- for content in message['content'] %}\n        {%- if content['type'] == 'image' %}\n            {%- set image_ns.has_images = true %}\n        {%- endif %}\n    {%- endfor %}\n{%- endfor %}\n\n{#- Error out if there are images and system message #}\n{%- if image_ns.has_images and not system_message == \"\" %}\n    {{- raise_exception(\"Prompting with images is incompatible with system messages.\") }}\n{%- endif %}\n\n{#- System message if there are no images #}\n{%- if not image_ns.has_images %}\n    {{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n    {%- if tools is not none %}\n        {{- \"Environment: ipython\\n\" }}\n    {%- endif %}\n    {{- \"Cutting Knowledge Date: December 2023\\n\" }}\n    {{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n    {%- if tools is not none and not tools_in_user_message %}\n        {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n        {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n        {{- \"Do not use variables.\\n\\n\" }}\n        {%- for t in tools %}\n            {{- t | tojson(indent=4) }}\n            {{- \"\\n\\n\" }}\n        {%- endfor %}\n    {%- endif %}\n    {{- system_message }}\n    {{- \"<|eot_id|>\" }}\n{%- endif %}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n    {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n' }}\n        {%- if message['content'] is string %}\n            {{- message['content'] }}\n        {%- else %}\n            {%- for content in message['content'] %}\n                {%- if content['type'] == 'image' %}\n                    {{- '<|image|>' }}\n                {%- elif content['type'] == 'text' %}\n                    {{- content['text'] }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n        {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n        {{- '\"parameters\": ' }}\n        {{- tool_call.arguments | tojson }}\n        {{- \"}\" }}\n        {{- \"<|eot_id|>\" }}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|finetune_right_pad_id|>",
+  "padding_side": "right",
+  "processor_class": "MllamaProcessor",
+  "tokenizer_class": "PreTrainedTokenizer"
+}

trainer_state.json

@@ -0,0 +1,2834 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.094488188976378,
+  "eval_steps": 500,
+  "global_step": 400,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.005249343832020997,
+      "grad_norm": 4.7173380851745605,
+      "learning_rate": 0.0,
+      "loss": 2.0177,
+      "step": 1
+    },
+    {
+      "epoch": 0.010498687664041995,
+      "grad_norm": 4.437033653259277,
+      "learning_rate": 3.5087719298245615e-06,
+      "loss": 1.7211,
+      "step": 2
+    },
+    {
+      "epoch": 0.015748031496062992,
+      "grad_norm": 4.145833969116211,
+      "learning_rate": 7.017543859649123e-06,
+      "loss": 1.6452,
+      "step": 3
+    },
+    {
+      "epoch": 0.02099737532808399,
+      "grad_norm": 2.83351731300354,
+      "learning_rate": 1.0526315789473684e-05,
+      "loss": 1.4565,
+      "step": 4
+    },
+    {
+      "epoch": 0.026246719160104987,
+      "grad_norm": 4.269916534423828,
+      "learning_rate": 1.4035087719298246e-05,
+      "loss": 1.8712,
+      "step": 5
+    },
+    {
+      "epoch": 0.031496062992125984,
+      "grad_norm": 3.0349326133728027,
+      "learning_rate": 1.7543859649122806e-05,
+      "loss": 1.9409,
+      "step": 6
+    },
+    {
+      "epoch": 0.03674540682414698,
+      "grad_norm": 4.1207780838012695,
+      "learning_rate": 2.105263157894737e-05,
+      "loss": 1.7171,
+      "step": 7
+    },
+    {
+      "epoch": 0.04199475065616798,
+      "grad_norm": 3.790914297103882,
+      "learning_rate": 2.456140350877193e-05,
+      "loss": 1.7406,
+      "step": 8
+    },
+    {
+      "epoch": 0.047244094488188976,
+      "grad_norm": 3.1181528568267822,
+      "learning_rate": 2.8070175438596492e-05,
+      "loss": 1.9785,
+      "step": 9
+    },
+    {
+      "epoch": 0.05249343832020997,
+      "grad_norm": 3.166116952896118,
+      "learning_rate": 3.157894736842105e-05,
+      "loss": 1.3592,
+      "step": 10
+    },
+    {
+      "epoch": 0.05774278215223097,
+      "grad_norm": 3.287881851196289,
+      "learning_rate": 3.508771929824561e-05,
+      "loss": 1.8857,
+      "step": 11
+    },
+    {
+      "epoch": 0.06299212598425197,
+      "grad_norm": 3.340050458908081,
+      "learning_rate": 3.859649122807018e-05,
+      "loss": 1.6225,
+      "step": 12
+    },
+    {
+      "epoch": 0.06824146981627296,
+      "grad_norm": 3.426025152206421,
+      "learning_rate": 4.210526315789474e-05,
+      "loss": 1.5674,
+      "step": 13
+    },
+    {
+      "epoch": 0.07349081364829396,
+      "grad_norm": 3.3121087551116943,
+      "learning_rate": 4.56140350877193e-05,
+      "loss": 1.4491,
+      "step": 14
+    },
+    {
+      "epoch": 0.07874015748031496,
+      "grad_norm": 3.173163890838623,
+      "learning_rate": 4.912280701754386e-05,
+      "loss": 1.4164,
+      "step": 15
+    },
+    {
+      "epoch": 0.08398950131233596,
+      "grad_norm": 3.2732391357421875,
+      "learning_rate": 5.2631578947368424e-05,
+      "loss": 1.5818,
+      "step": 16
+    },
+    {
+      "epoch": 0.08923884514435695,
+      "grad_norm": 4.392367362976074,
+      "learning_rate": 5.6140350877192984e-05,
+      "loss": 1.5491,
+      "step": 17
+    },
+    {
+      "epoch": 0.09448818897637795,
+      "grad_norm": 4.179324626922607,
+      "learning_rate": 5.9649122807017544e-05,
+      "loss": 1.6327,
+      "step": 18
+    },
+    {
+      "epoch": 0.09973753280839895,
+      "grad_norm": 3.5691120624542236,
+      "learning_rate": 6.31578947368421e-05,
+      "loss": 1.6418,
+      "step": 19
+    },
+    {
+      "epoch": 0.10498687664041995,
+      "grad_norm": 2.7537472248077393,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 1.3472,
+      "step": 20
+    },
+    {
+      "epoch": 0.11023622047244094,
+      "grad_norm": 4.199432373046875,
+      "learning_rate": 7.017543859649122e-05,
+      "loss": 1.3632,
+      "step": 21
+    },
+    {
+      "epoch": 0.11548556430446194,
+      "grad_norm": 4.307034969329834,
+      "learning_rate": 7.368421052631579e-05,
+      "loss": 1.3628,
+      "step": 22
+    },
+    {
+      "epoch": 0.12073490813648294,
+      "grad_norm": 3.5497403144836426,
+      "learning_rate": 7.719298245614036e-05,
+      "loss": 1.4816,
+      "step": 23
+    },
+    {
+      "epoch": 0.12598425196850394,
+      "grad_norm": 3.31937575340271,
+      "learning_rate": 8.070175438596491e-05,
+      "loss": 1.3599,
+      "step": 24
+    },
+    {
+      "epoch": 0.13123359580052493,
+      "grad_norm": 2.710407257080078,
+      "learning_rate": 8.421052631578948e-05,
+      "loss": 1.0672,
+      "step": 25
+    },
+    {
+      "epoch": 0.13648293963254593,
+      "grad_norm": 3.317232847213745,
+      "learning_rate": 8.771929824561403e-05,
+      "loss": 1.3396,
+      "step": 26
+    },
+    {
+      "epoch": 0.14173228346456693,
+      "grad_norm": 4.023993492126465,
+      "learning_rate": 9.12280701754386e-05,
+      "loss": 1.6528,
+      "step": 27
+    },
+    {
+      "epoch": 0.14698162729658792,
+      "grad_norm": 3.469362735748291,
+      "learning_rate": 9.473684210526316e-05,
+      "loss": 1.4297,
+      "step": 28
+    },
+    {
+      "epoch": 0.15223097112860892,
+      "grad_norm": 4.806138515472412,
+      "learning_rate": 9.824561403508771e-05,
+      "loss": 1.0343,
+      "step": 29
+    },
+    {
+      "epoch": 0.15748031496062992,
+      "grad_norm": 3.6015982627868652,
+      "learning_rate": 0.0001017543859649123,
+      "loss": 1.6401,
+      "step": 30
+    },
+    {
+      "epoch": 0.16272965879265092,
+      "grad_norm": 3.79402232170105,
+      "learning_rate": 0.00010526315789473685,
+      "loss": 1.6519,
+      "step": 31
+    },
+    {
+      "epoch": 0.1679790026246719,
+      "grad_norm": 4.123412132263184,
+      "learning_rate": 0.00010877192982456141,
+      "loss": 1.4302,
+      "step": 32
+    },
+    {
+      "epoch": 0.1732283464566929,
+      "grad_norm": 2.3286683559417725,
+      "learning_rate": 0.00011228070175438597,
+      "loss": 1.2029,
+      "step": 33
+    },
+    {
+      "epoch": 0.1784776902887139,
+      "grad_norm": 2.6430580615997314,
+      "learning_rate": 0.00011578947368421053,
+      "loss": 1.4423,
+      "step": 34
+    },
+    {
+      "epoch": 0.1837270341207349,
+      "grad_norm": 3.4652087688446045,
+      "learning_rate": 0.00011929824561403509,
+      "loss": 1.1598,
+      "step": 35
+    },
+    {
+      "epoch": 0.1889763779527559,
+      "grad_norm": 4.649867534637451,
+      "learning_rate": 0.00012280701754385965,
+      "loss": 1.3339,
+      "step": 36
+    },
+    {
+      "epoch": 0.1942257217847769,
+      "grad_norm": 2.450688123703003,
+      "learning_rate": 0.0001263157894736842,
+      "loss": 1.2083,
+      "step": 37
+    },
+    {
+      "epoch": 0.1994750656167979,
+      "grad_norm": 2.2536394596099854,
+      "learning_rate": 0.0001298245614035088,
+      "loss": 1.2621,
+      "step": 38
+    },
+    {
+      "epoch": 0.2047244094488189,
+      "grad_norm": 2.371548652648926,
+      "learning_rate": 0.00013333333333333334,
+      "loss": 1.198,
+      "step": 39
+    },
+    {
+      "epoch": 0.2099737532808399,
+      "grad_norm": 2.5427513122558594,
+      "learning_rate": 0.0001368421052631579,
+      "loss": 1.3013,
+      "step": 40
+    },
+    {
+      "epoch": 0.2152230971128609,
+      "grad_norm": 2.5822794437408447,
+      "learning_rate": 0.00014035087719298245,
+      "loss": 1.432,
+      "step": 41
+    },
+    {
+      "epoch": 0.2204724409448819,
+      "grad_norm": 2.5134873390197754,
+      "learning_rate": 0.00014385964912280703,
+      "loss": 1.2649,
+      "step": 42
+    },
+    {
+      "epoch": 0.22572178477690288,
+      "grad_norm": 2.4449403285980225,
+      "learning_rate": 0.00014736842105263158,
+      "loss": 1.2715,
+      "step": 43
+    },
+    {
+      "epoch": 0.23097112860892388,
+      "grad_norm": 2.940073013305664,
+      "learning_rate": 0.00015087719298245616,
+      "loss": 1.1609,
+      "step": 44
+    },
+    {
+      "epoch": 0.23622047244094488,
+      "grad_norm": 2.619502544403076,
+      "learning_rate": 0.0001543859649122807,
+      "loss": 1.5399,
+      "step": 45
+    },
+    {
+      "epoch": 0.24146981627296588,
+      "grad_norm": 2.3362364768981934,
+      "learning_rate": 0.00015789473684210527,
+      "loss": 1.3032,
+      "step": 46
+    },
+    {
+      "epoch": 0.24671916010498687,
+      "grad_norm": 2.1487205028533936,
+      "learning_rate": 0.00016140350877192982,
+      "loss": 1.3317,
+      "step": 47
+    },
+    {
+      "epoch": 0.25196850393700787,
+      "grad_norm": 3.1290791034698486,
+      "learning_rate": 0.0001649122807017544,
+      "loss": 1.5526,
+      "step": 48
+    },
+    {
+      "epoch": 0.2572178477690289,
+      "grad_norm": 2.4382166862487793,
+      "learning_rate": 0.00016842105263157895,
+      "loss": 1.2371,
+      "step": 49
+    },
+    {
+      "epoch": 0.26246719160104987,
+      "grad_norm": 2.218122959136963,
+      "learning_rate": 0.00017192982456140353,
+      "loss": 1.2424,
+      "step": 50
+    },
+    {
+      "epoch": 0.2677165354330709,
+      "grad_norm": 2.6388792991638184,
+      "learning_rate": 0.00017543859649122806,
+      "loss": 1.2306,
+      "step": 51
+    },
+    {
+      "epoch": 0.27296587926509186,
+      "grad_norm": 2.736288547515869,
+      "learning_rate": 0.00017894736842105264,
+      "loss": 1.3486,
+      "step": 52
+    },
+    {
+      "epoch": 0.2782152230971129,
+      "grad_norm": 2.465235948562622,
+      "learning_rate": 0.0001824561403508772,
+      "loss": 1.4959,
+      "step": 53
+    },
+    {
+      "epoch": 0.28346456692913385,
+      "grad_norm": 2.587756872177124,
+      "learning_rate": 0.00018596491228070177,
+      "loss": 1.6561,
+      "step": 54
+    },
+    {
+      "epoch": 0.2887139107611549,
+      "grad_norm": 2.873962163925171,
+      "learning_rate": 0.00018947368421052632,
+      "loss": 1.154,
+      "step": 55
+    },
+    {
+      "epoch": 0.29396325459317585,
+      "grad_norm": 2.857832431793213,
+      "learning_rate": 0.00019298245614035088,
+      "loss": 1.4398,
+      "step": 56
+    },
+    {
+      "epoch": 0.2992125984251969,
+      "grad_norm": 2.450753688812256,
+      "learning_rate": 0.00019649122807017543,
+      "loss": 1.3935,
+      "step": 57
+    },
+    {
+      "epoch": 0.30446194225721784,
+      "grad_norm": 3.0704116821289062,
+      "learning_rate": 0.0002,
+      "loss": 1.4604,
+      "step": 58
+    },
+    {
+      "epoch": 0.30971128608923887,
+      "grad_norm": 2.362713098526001,
+      "learning_rate": 0.00019999812486015523,
+      "loss": 1.5173,
+      "step": 59
+    },
+    {
+      "epoch": 0.31496062992125984,
+      "grad_norm": 2.8491404056549072,
+      "learning_rate": 0.00019999249951094388,
+      "loss": 1.5408,
+      "step": 60
+    },
+    {
+      "epoch": 0.32020997375328086,
+      "grad_norm": 2.454577684402466,
+      "learning_rate": 0.00019998312416333227,
+      "loss": 1.4086,
+      "step": 61
+    },
+    {
+      "epoch": 0.32545931758530183,
+      "grad_norm": 2.2537734508514404,
+      "learning_rate": 0.0001999699991689222,
+      "loss": 1.1157,
+      "step": 62
+    },
+    {
+      "epoch": 0.33070866141732286,
+      "grad_norm": 2.300297260284424,
+      "learning_rate": 0.00019995312501993765,
+      "loss": 1.1927,
+      "step": 63
+    },
+    {
+      "epoch": 0.3359580052493438,
+      "grad_norm": 2.0972819328308105,
+      "learning_rate": 0.00019993250234920636,
+      "loss": 1.1443,
+      "step": 64
+    },
+    {
+      "epoch": 0.34120734908136485,
+      "grad_norm": 2.440847396850586,
+      "learning_rate": 0.00019990813193013625,
+      "loss": 1.7633,
+      "step": 65
+    },
+    {
+      "epoch": 0.3464566929133858,
+      "grad_norm": 2.1753475666046143,
+      "learning_rate": 0.0001998800146766861,
+      "loss": 1.2409,
+      "step": 66
+    },
+    {
+      "epoch": 0.35170603674540685,
+      "grad_norm": 2.196859836578369,
+      "learning_rate": 0.00019984815164333163,
+      "loss": 1.1315,
+      "step": 67
+    },
+    {
+      "epoch": 0.3569553805774278,
+      "grad_norm": 2.589327335357666,
+      "learning_rate": 0.00019981254402502566,
+      "loss": 1.4154,
+      "step": 68
+    },
+    {
+      "epoch": 0.36220472440944884,
+      "grad_norm": 2.040648937225342,
+      "learning_rate": 0.0001997731931571535,
+      "loss": 1.1035,
+      "step": 69
+    },
+    {
+      "epoch": 0.3674540682414698,
+      "grad_norm": 3.925663709640503,
+      "learning_rate": 0.00019973010051548275,
+      "loss": 1.562,
+      "step": 70
+    },
+    {
+      "epoch": 0.37270341207349084,
+      "grad_norm": 2.7915618419647217,
+      "learning_rate": 0.00019968326771610797,
+      "loss": 1.1201,
+      "step": 71
+    },
+    {
+      "epoch": 0.3779527559055118,
+      "grad_norm": 2.5829081535339355,
+      "learning_rate": 0.00019963269651539017,
+      "loss": 1.3134,
+      "step": 72
+    },
+    {
+      "epoch": 0.38320209973753283,
+      "grad_norm": 3.7920446395874023,
+      "learning_rate": 0.00019957838880989078,
+      "loss": 1.4021,
+      "step": 73
+    },
+    {
+      "epoch": 0.3884514435695538,
+      "grad_norm": 2.2051122188568115,
+      "learning_rate": 0.00019952034663630062,
+      "loss": 1.1945,
+      "step": 74
+    },
+    {
+      "epoch": 0.3937007874015748,
+      "grad_norm": 2.5238990783691406,
+      "learning_rate": 0.00019945857217136363,
+      "loss": 1.3494,
+      "step": 75
+    },
+    {
+      "epoch": 0.3989501312335958,
+      "grad_norm": 2.7515692710876465,
+      "learning_rate": 0.00019939306773179497,
+      "loss": 1.3467,
+      "step": 76
+    },
+    {
+      "epoch": 0.4041994750656168,
+      "grad_norm": 2.8522536754608154,
+      "learning_rate": 0.00019932383577419432,
+      "loss": 1.57,
+      "step": 77
+    },
+    {
+      "epoch": 0.4094488188976378,
+      "grad_norm": 1.9163004159927368,
+      "learning_rate": 0.00019925087889495374,
+      "loss": 1.3051,
+      "step": 78
+    },
+    {
+      "epoch": 0.4146981627296588,
+      "grad_norm": 1.9789259433746338,
+      "learning_rate": 0.00019917419983016025,
+      "loss": 0.9476,
+      "step": 79
+    },
+    {
+      "epoch": 0.4199475065616798,
+      "grad_norm": 2.6120073795318604,
+      "learning_rate": 0.00019909380145549324,
+      "loss": 1.3412,
+      "step": 80
+    },
+    {
+      "epoch": 0.4251968503937008,
+      "grad_norm": 2.627711772918701,
+      "learning_rate": 0.00019900968678611666,
+      "loss": 1.424,
+      "step": 81
+    },
+    {
+      "epoch": 0.4304461942257218,
+      "grad_norm": 2.3083677291870117,
+      "learning_rate": 0.00019892185897656578,
+      "loss": 1.137,
+      "step": 82
+    },
+    {
+      "epoch": 0.4356955380577428,
+      "grad_norm": 2.250309705734253,
+      "learning_rate": 0.00019883032132062925,
+      "loss": 1.0466,
+      "step": 83
+    },
+    {
+      "epoch": 0.4409448818897638,
+      "grad_norm": 2.2926323413848877,
+      "learning_rate": 0.00019873507725122504,
+      "loss": 1.4918,
+      "step": 84
+    },
+    {
+      "epoch": 0.4461942257217848,
+      "grad_norm": 2.8048651218414307,
+      "learning_rate": 0.00019863613034027224,
+      "loss": 1.2737,
+      "step": 85
+    },
+    {
+      "epoch": 0.45144356955380577,
+      "grad_norm": 2.108257532119751,
+      "learning_rate": 0.00019853348429855672,
+      "loss": 1.0448,
+      "step": 86
+    },
+    {
+      "epoch": 0.4566929133858268,
+      "grad_norm": 2.1626484394073486,
+      "learning_rate": 0.00019842714297559213,
+      "loss": 1.1422,
+      "step": 87
+    },
+    {
+      "epoch": 0.46194225721784776,
+      "grad_norm": 2.1590943336486816,
+      "learning_rate": 0.0001983171103594755,
+      "loss": 1.073,
+      "step": 88
+    },
+    {
+      "epoch": 0.4671916010498688,
+      "grad_norm": 2.6183314323425293,
+      "learning_rate": 0.0001982033905767377,
+      "loss": 1.1683,
+      "step": 89
+    },
+    {
+      "epoch": 0.47244094488188976,
+      "grad_norm": 2.6423206329345703,
+      "learning_rate": 0.00019808598789218865,
+      "loss": 1.284,
+      "step": 90
+    },
+    {
+      "epoch": 0.4776902887139108,
+      "grad_norm": 2.9666941165924072,
+      "learning_rate": 0.0001979649067087574,
+      "loss": 1.4623,
+      "step": 91
+    },
+    {
+      "epoch": 0.48293963254593175,
+      "grad_norm": 2.3356740474700928,
+      "learning_rate": 0.00019784015156732693,
+      "loss": 1.4087,
+      "step": 92
+    },
+    {
+      "epoch": 0.4881889763779528,
+      "grad_norm": 2.55161190032959,
+      "learning_rate": 0.000197711727146564,
+      "loss": 1.2149,
+      "step": 93
+    },
+    {
+      "epoch": 0.49343832020997375,
+      "grad_norm": 2.503931760787964,
+      "learning_rate": 0.00019757963826274357,
+      "loss": 1.0572,
+      "step": 94
+    },
+    {
+      "epoch": 0.49868766404199477,
+      "grad_norm": 2.113316297531128,
+      "learning_rate": 0.00019744388986956822,
+      "loss": 1.3952,
+      "step": 95
+    },
+    {
+      "epoch": 0.5039370078740157,
+      "grad_norm": 2.413565158843994,
+      "learning_rate": 0.00019730448705798239,
+      "loss": 1.0974,
+      "step": 96
+    },
+    {
+      "epoch": 0.5091863517060368,
+      "grad_norm": 2.6888630390167236,
+      "learning_rate": 0.0001971614350559814,
+      "loss": 1.5181,
+      "step": 97
+    },
+    {
+      "epoch": 0.5144356955380578,
+      "grad_norm": 2.208359718322754,
+      "learning_rate": 0.0001970147392284154,
+      "loss": 1.1508,
+      "step": 98
+    },
+    {
+      "epoch": 0.5196850393700787,
+      "grad_norm": 2.3736019134521484,
+      "learning_rate": 0.00019686440507678824,
+      "loss": 1.4067,
+      "step": 99
+    },
+    {
+      "epoch": 0.5249343832020997,
+      "grad_norm": 2.40409255027771,
+      "learning_rate": 0.0001967104382390511,
+      "loss": 1.1971,
+      "step": 100
+    },
+    {
+      "epoch": 0.5301837270341208,
+      "grad_norm": 2.203507661819458,
+      "learning_rate": 0.00019655284448939094,
+      "loss": 1.3298,
+      "step": 101
+    },
+    {
+      "epoch": 0.5354330708661418,
+      "grad_norm": 2.1737711429595947,
+      "learning_rate": 0.00019639162973801426,
+      "loss": 1.4618,
+      "step": 102
+    },
+    {
+      "epoch": 0.5406824146981627,
+      "grad_norm": 3.172531843185425,
+      "learning_rate": 0.00019622680003092503,
+      "loss": 1.6789,
+      "step": 103
+    },
+    {
+      "epoch": 0.5459317585301837,
+      "grad_norm": 2.5439305305480957,
+      "learning_rate": 0.0001960583615496984,
+      "loss": 1.6033,
+      "step": 104
+    },
+    {
+      "epoch": 0.5511811023622047,
+      "grad_norm": 2.414440870285034,
+      "learning_rate": 0.00019588632061124837,
+      "loss": 1.1086,
+      "step": 105
+    },
+    {
+      "epoch": 0.5564304461942258,
+      "grad_norm": 2.828094244003296,
+      "learning_rate": 0.00019571068366759143,
+      "loss": 1.2589,
+      "step": 106
+    },
+    {
+      "epoch": 0.5616797900262467,
+      "grad_norm": 2.0109658241271973,
+      "learning_rate": 0.00019553145730560415,
+      "loss": 0.8927,
+      "step": 107
+    },
+    {
+      "epoch": 0.5669291338582677,
+      "grad_norm": 2.842677116394043,
+      "learning_rate": 0.0001953486482467764,
+      "loss": 1.4432,
+      "step": 108
+    },
+    {
+      "epoch": 0.5721784776902887,
+      "grad_norm": 2.2482569217681885,
+      "learning_rate": 0.0001951622633469592,
+      "loss": 1.1888,
+      "step": 109
+    },
+    {
+      "epoch": 0.5774278215223098,
+      "grad_norm": 2.6185362339019775,
+      "learning_rate": 0.00019497230959610756,
+      "loss": 1.1135,
+      "step": 110
+    },
+    {
+      "epoch": 0.5826771653543307,
+      "grad_norm": 2.3009023666381836,
+      "learning_rate": 0.00019477879411801844,
+      "loss": 1.1068,
+      "step": 111
+    },
+    {
+      "epoch": 0.5879265091863517,
+      "grad_norm": 2.750929832458496,
+      "learning_rate": 0.00019458172417006347,
+      "loss": 1.0964,
+      "step": 112
+    },
+    {
+      "epoch": 0.5931758530183727,
+      "grad_norm": 2.6601922512054443,
+      "learning_rate": 0.00019438110714291694,
+      "loss": 1.4321,
+      "step": 113
+    },
+    {
+      "epoch": 0.5984251968503937,
+      "grad_norm": 2.598353624343872,
+      "learning_rate": 0.00019417695056027844,
+      "loss": 1.1159,
+      "step": 114
+    },
+    {
+      "epoch": 0.6036745406824147,
+      "grad_norm": 2.5931055545806885,
+      "learning_rate": 0.00019396926207859084,
+      "loss": 1.5328,
+      "step": 115
+    },
+    {
+      "epoch": 0.6089238845144357,
+      "grad_norm": 2.612820625305176,
+      "learning_rate": 0.00019375804948675306,
+      "loss": 1.5006,
+      "step": 116
+    },
+    {
+      "epoch": 0.6141732283464567,
+      "grad_norm": 2.5003819465637207,
+      "learning_rate": 0.0001935433207058281,
+      "loss": 1.5221,
+      "step": 117
+    },
+    {
+      "epoch": 0.6194225721784777,
+      "grad_norm": 2.438344955444336,
+      "learning_rate": 0.0001933250837887457,
+      "loss": 1.1499,
+      "step": 118
+    },
+    {
+      "epoch": 0.6246719160104987,
+      "grad_norm": 2.2599291801452637,
+      "learning_rate": 0.00019310334692000075,
+      "loss": 1.1137,
+      "step": 119
+    },
+    {
+      "epoch": 0.6299212598425197,
+      "grad_norm": 2.45780086517334,
+      "learning_rate": 0.00019287811841534595,
+      "loss": 1.363,
+      "step": 120
+    },
+    {
+      "epoch": 0.6351706036745407,
+      "grad_norm": 2.098870038986206,
+      "learning_rate": 0.00019264940672148018,
+      "loss": 1.2201,
+      "step": 121
+    },
+    {
+      "epoch": 0.6404199475065617,
+      "grad_norm": 1.8870902061462402,
+      "learning_rate": 0.00019241722041573166,
+      "loss": 1.3615,
+      "step": 122
+    },
+    {
+      "epoch": 0.6456692913385826,
+      "grad_norm": 1.971105694770813,
+      "learning_rate": 0.0001921815682057362,
+      "loss": 1.1174,
+      "step": 123
+    },
+    {
+      "epoch": 0.6509186351706037,
+      "grad_norm": 2.195482015609741,
+      "learning_rate": 0.0001919424589291108,
+      "loss": 1.2562,
+      "step": 124
+    },
+    {
+      "epoch": 0.6561679790026247,
+      "grad_norm": 2.429697036743164,
+      "learning_rate": 0.0001916999015531221,
+      "loss": 1.3243,
+      "step": 125
+    },
+    {
+      "epoch": 0.6614173228346457,
+      "grad_norm": 2.208289623260498,
+      "learning_rate": 0.00019145390517435012,
+      "loss": 1.0649,
+      "step": 126
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 2.1772406101226807,
+      "learning_rate": 0.00019120447901834706,
+      "loss": 1.1999,
+      "step": 127
+    },
+    {
+      "epoch": 0.6719160104986877,
+      "grad_norm": 2.6156198978424072,
+      "learning_rate": 0.00019095163243929142,
+      "loss": 1.2446,
+      "step": 128
+    },
+    {
+      "epoch": 0.6771653543307087,
+      "grad_norm": 2.6300153732299805,
+      "learning_rate": 0.0001906953749196371,
+      "loss": 1.3671,
+      "step": 129
+    },
+    {
+      "epoch": 0.6824146981627297,
+      "grad_norm": 2.3485264778137207,
+      "learning_rate": 0.00019043571606975777,
+      "loss": 0.9951,
+      "step": 130
+    },
+    {
+      "epoch": 0.6876640419947506,
+      "grad_norm": 2.8363163471221924,
+      "learning_rate": 0.00019017266562758659,
+      "loss": 1.3425,
+      "step": 131
+    },
+    {
+      "epoch": 0.6929133858267716,
+      "grad_norm": 2.223158121109009,
+      "learning_rate": 0.00018990623345825083,
+      "loss": 1.3456,
+      "step": 132
+    },
+    {
+      "epoch": 0.6981627296587927,
+      "grad_norm": 2.4961116313934326,
+      "learning_rate": 0.00018963642955370201,
+      "loss": 1.4691,
+      "step": 133
+    },
+    {
+      "epoch": 0.7034120734908137,
+      "grad_norm": 2.3813393115997314,
+      "learning_rate": 0.00018936326403234125,
+      "loss": 1.2141,
+      "step": 134
+    },
+    {
+      "epoch": 0.7086614173228346,
+      "grad_norm": 2.0415079593658447,
+      "learning_rate": 0.00018908674713863952,
+      "loss": 1.1995,
+      "step": 135
+    },
+    {
+      "epoch": 0.7139107611548556,
+      "grad_norm": 2.443024158477783,
+      "learning_rate": 0.00018880688924275378,
+      "loss": 1.5299,
+      "step": 136
+    },
+    {
+      "epoch": 0.7191601049868767,
+      "grad_norm": 2.8351380825042725,
+      "learning_rate": 0.0001885237008401378,
+      "loss": 1.2206,
+      "step": 137
+    },
+    {
+      "epoch": 0.7244094488188977,
+      "grad_norm": 2.696510076522827,
+      "learning_rate": 0.0001882371925511488,
+      "loss": 1.0794,
+      "step": 138
+    },
+    {
+      "epoch": 0.7296587926509186,
+      "grad_norm": 2.9665584564208984,
+      "learning_rate": 0.0001879473751206489,
+      "loss": 1.575,
+      "step": 139
+    },
+    {
+      "epoch": 0.7349081364829396,
+      "grad_norm": 2.1545960903167725,
+      "learning_rate": 0.00018765425941760238,
+      "loss": 1.0347,
+      "step": 140
+    },
+    {
+      "epoch": 0.7401574803149606,
+      "grad_norm": 2.3312506675720215,
+      "learning_rate": 0.00018735785643466784,
+      "loss": 1.3836,
+      "step": 141
+    },
+    {
+      "epoch": 0.7454068241469817,
+      "grad_norm": 2.3382620811462402,
+      "learning_rate": 0.00018705817728778624,
+      "loss": 1.1358,
+      "step": 142
+    },
+    {
+      "epoch": 0.7506561679790026,
+      "grad_norm": 2.4452474117279053,
+      "learning_rate": 0.00018675523321576371,
+      "loss": 1.1749,
+      "step": 143
+    },
+    {
+      "epoch": 0.7559055118110236,
+      "grad_norm": 2.0249741077423096,
+      "learning_rate": 0.00018644903557985025,
+      "loss": 0.9222,
+      "step": 144
+    },
+    {
+      "epoch": 0.7611548556430446,
+      "grad_norm": 2.624455451965332,
+      "learning_rate": 0.00018613959586331362,
+      "loss": 1.0862,
+      "step": 145
+    },
+    {
+      "epoch": 0.7664041994750657,
+      "grad_norm": 2.175856351852417,
+      "learning_rate": 0.00018582692567100867,
+      "loss": 1.1019,
+      "step": 146
+    },
+    {
+      "epoch": 0.7716535433070866,
+      "grad_norm": 2.5203380584716797,
+      "learning_rate": 0.00018551103672894206,
+      "loss": 1.5635,
+      "step": 147
+    },
+    {
+      "epoch": 0.7769028871391076,
+      "grad_norm": 2.405241012573242,
+      "learning_rate": 0.00018519194088383273,
+      "loss": 1.2331,
+      "step": 148
+    },
+    {
+      "epoch": 0.7821522309711286,
+      "grad_norm": 2.598151206970215,
+      "learning_rate": 0.00018486965010266725,
+      "loss": 1.7277,
+      "step": 149
+    },
+    {
+      "epoch": 0.7874015748031497,
+      "grad_norm": 2.171656370162964,
+      "learning_rate": 0.0001845441764722514,
+      "loss": 1.2162,
+      "step": 150
+    },
+    {
+      "epoch": 0.7926509186351706,
+      "grad_norm": 2.485642433166504,
+      "learning_rate": 0.00018421553219875658,
+      "loss": 1.5379,
+      "step": 151
+    },
+    {
+      "epoch": 0.7979002624671916,
+      "grad_norm": 2.297564744949341,
+      "learning_rate": 0.00018388372960726228,
+      "loss": 1.168,
+      "step": 152
+    },
+    {
+      "epoch": 0.8031496062992126,
+      "grad_norm": 1.9809898138046265,
+      "learning_rate": 0.00018354878114129367,
+      "loss": 1.1138,
+      "step": 153
+    },
+    {
+      "epoch": 0.8083989501312336,
+      "grad_norm": 2.277679920196533,
+      "learning_rate": 0.00018321069936235503,
+      "loss": 0.9873,
+      "step": 154
+    },
+    {
+      "epoch": 0.8136482939632546,
+      "grad_norm": 2.324462890625,
+      "learning_rate": 0.00018286949694945866,
+      "loss": 1.2728,
+      "step": 155
+    },
+    {
+      "epoch": 0.8188976377952756,
+      "grad_norm": 2.318861722946167,
+      "learning_rate": 0.00018252518669864936,
+      "loss": 1.2905,
+      "step": 156
+    },
+    {
+      "epoch": 0.8241469816272966,
+      "grad_norm": 2.5826478004455566,
+      "learning_rate": 0.0001821777815225245,
+      "loss": 1.2632,
+      "step": 157
+    },
+    {
+      "epoch": 0.8293963254593176,
+      "grad_norm": 1.9897890090942383,
+      "learning_rate": 0.00018182729444974992,
+      "loss": 1.1829,
+      "step": 158
+    },
+    {
+      "epoch": 0.8346456692913385,
+      "grad_norm": 2.1801273822784424,
+      "learning_rate": 0.00018147373862457107,
+      "loss": 1.1657,
+      "step": 159
+    },
+    {
+      "epoch": 0.8398950131233596,
+      "grad_norm": 2.332425117492676,
+      "learning_rate": 0.00018111712730632022,
+      "loss": 1.1415,
+      "step": 160
+    },
+    {
+      "epoch": 0.8451443569553806,
+      "grad_norm": 2.0971760749816895,
+      "learning_rate": 0.0001807574738689193,
+      "loss": 1.3374,
+      "step": 161
+    },
+    {
+      "epoch": 0.8503937007874016,
+      "grad_norm": 2.436295509338379,
+      "learning_rate": 0.000180394791800378,
+      "loss": 1.2387,
+      "step": 162
+    },
+    {
+      "epoch": 0.8556430446194225,
+      "grad_norm": 2.6877710819244385,
+      "learning_rate": 0.00018002909470228842,
+      "loss": 1.5381,
+      "step": 163
+    },
+    {
+      "epoch": 0.8608923884514436,
+      "grad_norm": 2.4646594524383545,
+      "learning_rate": 0.00017966039628931446,
+      "loss": 1.4384,
+      "step": 164
+    },
+    {
+      "epoch": 0.8661417322834646,
+      "grad_norm": 2.020935297012329,
+      "learning_rate": 0.00017928871038867784,
+      "loss": 1.2144,
+      "step": 165
+    },
+    {
+      "epoch": 0.8713910761154856,
+      "grad_norm": 2.4887311458587646,
+      "learning_rate": 0.00017891405093963938,
+      "loss": 1.3464,
+      "step": 166
+    },
+    {
+      "epoch": 0.8766404199475065,
+      "grad_norm": 2.3591365814208984,
+      "learning_rate": 0.00017853643199297633,
+      "loss": 1.1275,
+      "step": 167
+    },
+    {
+      "epoch": 0.8818897637795275,
+      "grad_norm": 1.8852261304855347,
+      "learning_rate": 0.00017815586771045535,
+      "loss": 1.1718,
+      "step": 168
+    },
+    {
+      "epoch": 0.8871391076115486,
+      "grad_norm": 2.501147747039795,
+      "learning_rate": 0.0001777723723643014,
+      "loss": 1.5136,
+      "step": 169
+    },
+    {
+      "epoch": 0.8923884514435696,
+      "grad_norm": 2.3164873123168945,
+      "learning_rate": 0.0001773859603366626,
+      "loss": 1.2471,
+      "step": 170
+    },
+    {
+      "epoch": 0.8976377952755905,
+      "grad_norm": 2.3213908672332764,
+      "learning_rate": 0.00017699664611907072,
+      "loss": 1.5152,
+      "step": 171
+    },
+    {
+      "epoch": 0.9028871391076115,
+      "grad_norm": 2.225059986114502,
+      "learning_rate": 0.0001766044443118978,
+      "loss": 0.9982,
+      "step": 172
+    },
+    {
+      "epoch": 0.9081364829396326,
+      "grad_norm": 2.617565631866455,
+      "learning_rate": 0.00017620936962380856,
+      "loss": 1.1797,
+      "step": 173
+    },
+    {
+      "epoch": 0.9133858267716536,
+      "grad_norm": 2.1258366107940674,
+      "learning_rate": 0.00017581143687120875,
+      "loss": 1.2575,
+      "step": 174
+    },
+    {
+      "epoch": 0.9186351706036745,
+      "grad_norm": 2.1051361560821533,
+      "learning_rate": 0.00017541066097768963,
+      "loss": 0.8311,
+      "step": 175
+    },
+    {
+      "epoch": 0.9238845144356955,
+      "grad_norm": 2.1657958030700684,
+      "learning_rate": 0.0001750070569734681,
+      "loss": 1.093,
+      "step": 176
+    },
+    {
+      "epoch": 0.9291338582677166,
+      "grad_norm": 2.4127931594848633,
+      "learning_rate": 0.00017460063999482316,
+      "loss": 1.0833,
+      "step": 177
+    },
+    {
+      "epoch": 0.9343832020997376,
+      "grad_norm": 2.753276824951172,
+      "learning_rate": 0.00017419142528352817,
+      "loss": 1.178,
+      "step": 178
+    },
+    {
+      "epoch": 0.9396325459317585,
+      "grad_norm": 2.67309832572937,
+      "learning_rate": 0.00017377942818627942,
+      "loss": 1.303,
+      "step": 179
+    },
+    {
+      "epoch": 0.9448818897637795,
+      "grad_norm": 2.5962181091308594,
+      "learning_rate": 0.00017336466415412028,
+      "loss": 1.5608,
+      "step": 180
+    },
+    {
+      "epoch": 0.9501312335958005,
+      "grad_norm": 3.0257773399353027,
+      "learning_rate": 0.0001729471487418621,
+      "loss": 1.3775,
+      "step": 181
+    },
+    {
+      "epoch": 0.9553805774278216,
+      "grad_norm": 1.9228885173797607,
+      "learning_rate": 0.0001725268976075005,
+      "loss": 1.0905,
+      "step": 182
+    },
+    {
+      "epoch": 0.9606299212598425,
+      "grad_norm": 3.0711910724639893,
+      "learning_rate": 0.0001721039265116285,
+      "loss": 1.1671,
+      "step": 183
+    },
+    {
+      "epoch": 0.9658792650918635,
+      "grad_norm": 2.53415846824646,
+      "learning_rate": 0.00017167825131684513,
+      "loss": 1.458,
+      "step": 184
+    },
+    {
+      "epoch": 0.9711286089238845,
+      "grad_norm": 2.325265407562256,
+      "learning_rate": 0.00017124988798716083,
+      "loss": 1.3166,
+      "step": 185
+    },
+    {
+      "epoch": 0.9763779527559056,
+      "grad_norm": 1.972551941871643,
+      "learning_rate": 0.00017081885258739846,
+      "loss": 1.1434,
+      "step": 186
+    },
+    {
+      "epoch": 0.9816272965879265,
+      "grad_norm": 2.308837890625,
+      "learning_rate": 0.00017038516128259115,
+      "loss": 1.25,
+      "step": 187
+    },
+    {
+      "epoch": 0.9868766404199475,
+      "grad_norm": 2.6584460735321045,
+      "learning_rate": 0.00016994883033737582,
+      "loss": 1.3522,
+      "step": 188
+    },
+    {
+      "epoch": 0.9921259842519685,
+      "grad_norm": 2.0960097312927246,
+      "learning_rate": 0.00016950987611538324,
+      "loss": 1.0266,
+      "step": 189
+    },
+    {
+      "epoch": 0.9973753280839895,
+      "grad_norm": 2.1752665042877197,
+      "learning_rate": 0.00016906831507862443,
+      "loss": 1.3627,
+      "step": 190
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 2.6606860160827637,
+      "learning_rate": 0.0001686241637868734,
+      "loss": 0.8187,
+      "step": 191
+    },
+    {
+      "epoch": 1.005249343832021,
+      "grad_norm": 2.049098491668701,
+      "learning_rate": 0.00016817743889704565,
+      "loss": 1.0819,
+      "step": 192
+    },
+    {
+      "epoch": 1.010498687664042,
+      "grad_norm": 1.9727576971054077,
+      "learning_rate": 0.00016772815716257412,
+      "loss": 1.0079,
+      "step": 193
+    },
+    {
+      "epoch": 1.015748031496063,
+      "grad_norm": 1.9842270612716675,
+      "learning_rate": 0.0001672763354327804,
+      "loss": 0.8639,
+      "step": 194
+    },
+    {
+      "epoch": 1.020997375328084,
+      "grad_norm": 1.639641523361206,
+      "learning_rate": 0.00016682199065224307,
+      "loss": 0.6213,
+      "step": 195
+    },
+    {
+      "epoch": 1.026246719160105,
+      "grad_norm": 2.2830586433410645,
+      "learning_rate": 0.00016636513986016213,
+      "loss": 0.7467,
+      "step": 196
+    },
+    {
+      "epoch": 1.031496062992126,
+      "grad_norm": 3.062472105026245,
+      "learning_rate": 0.0001659058001897201,
+      "loss": 0.9837,
+      "step": 197
+    },
+    {
+      "epoch": 1.036745406824147,
+      "grad_norm": 2.6546707153320312,
+      "learning_rate": 0.00016544398886743933,
+      "loss": 1.0166,
+      "step": 198
+    },
+    {
+      "epoch": 1.041994750656168,
+      "grad_norm": 2.1115078926086426,
+      "learning_rate": 0.000164979723212536,
+      "loss": 0.8896,
+      "step": 199
+    },
+    {
+      "epoch": 1.047244094488189,
+      "grad_norm": 2.4062306880950928,
+      "learning_rate": 0.00016451302063627066,
+      "loss": 0.9431,
+      "step": 200
+    },
+    {
+      "epoch": 1.05249343832021,
+      "grad_norm": 2.4070167541503906,
+      "learning_rate": 0.00016404389864129533,
+      "loss": 0.7534,
+      "step": 201
+    },
+    {
+      "epoch": 1.057742782152231,
+      "grad_norm": 2.7025625705718994,
+      "learning_rate": 0.00016357237482099684,
+      "loss": 0.6826,
+      "step": 202
+    },
+    {
+      "epoch": 1.0629921259842519,
+      "grad_norm": 2.4860000610351562,
+      "learning_rate": 0.00016309846685883726,
+      "loss": 0.7801,
+      "step": 203
+    },
+    {
+      "epoch": 1.068241469816273,
+      "grad_norm": 2.442067861557007,
+      "learning_rate": 0.00016262219252769064,
+      "loss": 0.8237,
+      "step": 204
+    },
+    {
+      "epoch": 1.073490813648294,
+      "grad_norm": 2.510655164718628,
+      "learning_rate": 0.00016214356968917648,
+      "loss": 0.8001,
+      "step": 205
+    },
+    {
+      "epoch": 1.078740157480315,
+      "grad_norm": 3.1672093868255615,
+      "learning_rate": 0.00016166261629298995,
+      "loss": 0.9451,
+      "step": 206
+    },
+    {
+      "epoch": 1.083989501312336,
+      "grad_norm": 3.2933852672576904,
+      "learning_rate": 0.0001611793503762285,
+      "loss": 0.9388,
+      "step": 207
+    },
+    {
+      "epoch": 1.0892388451443569,
+      "grad_norm": 3.1465041637420654,
+      "learning_rate": 0.00016069379006271566,
+      "loss": 1.0714,
+      "step": 208
+    },
+    {
+      "epoch": 1.094488188976378,
+      "grad_norm": 2.6741809844970703,
+      "learning_rate": 0.00016020595356232135,
+      "loss": 0.7402,
+      "step": 209
+    },
+    {
+      "epoch": 1.099737532808399,
+      "grad_norm": 2.4382576942443848,
+      "learning_rate": 0.00015971585917027862,
+      "loss": 0.6643,
+      "step": 210
+    },
+    {
+      "epoch": 1.10498687664042,
+      "grad_norm": 2.5925192832946777,
+      "learning_rate": 0.00015922352526649803,
+      "loss": 0.8338,
+      "step": 211
+    },
+    {
+      "epoch": 1.110236220472441,
+      "grad_norm": 2.8813819885253906,
+      "learning_rate": 0.00015872897031487791,
+      "loss": 0.9946,
+      "step": 212
+    },
+    {
+      "epoch": 1.1154855643044619,
+      "grad_norm": 2.19948673248291,
+      "learning_rate": 0.00015823221286261215,
+      "loss": 0.916,
+      "step": 213
+    },
+    {
+      "epoch": 1.120734908136483,
+      "grad_norm": 2.330357551574707,
+      "learning_rate": 0.00015773327153949465,
+      "loss": 0.9889,
+      "step": 214
+    },
+    {
+      "epoch": 1.125984251968504,
+      "grad_norm": 2.321606397628784,
+      "learning_rate": 0.0001572321650572205,
+      "loss": 0.9533,
+      "step": 215
+    },
+    {
+      "epoch": 1.1312335958005248,
+      "grad_norm": 2.219377040863037,
+      "learning_rate": 0.00015672891220868432,
+      "loss": 0.6138,
+      "step": 216
+    },
+    {
+      "epoch": 1.136482939632546,
+      "grad_norm": 2.7651844024658203,
+      "learning_rate": 0.00015622353186727544,
+      "loss": 0.8008,
+      "step": 217
+    },
+    {
+      "epoch": 1.141732283464567,
+      "grad_norm": 2.5043694972991943,
+      "learning_rate": 0.0001557160429861702,
+      "loss": 0.9846,
+      "step": 218
+    },
+    {
+      "epoch": 1.1469816272965878,
+      "grad_norm": 2.6043448448181152,
+      "learning_rate": 0.000155206464597621,
+      "loss": 0.932,
+      "step": 219
+    },
+    {
+      "epoch": 1.152230971128609,
+      "grad_norm": 2.5369505882263184,
+      "learning_rate": 0.00015469481581224272,
+      "loss": 0.8897,
+      "step": 220
+    },
+    {
+      "epoch": 1.1574803149606299,
+      "grad_norm": 2.6622097492218018,
+      "learning_rate": 0.00015418111581829574,
+      "loss": 0.952,
+      "step": 221
+    },
+    {
+      "epoch": 1.162729658792651,
+      "grad_norm": 2.4234066009521484,
+      "learning_rate": 0.0001536653838809667,
+      "loss": 0.8107,
+      "step": 222
+    },
+    {
+      "epoch": 1.167979002624672,
+      "grad_norm": 2.463465452194214,
+      "learning_rate": 0.0001531476393416456,
+      "loss": 0.7944,
+      "step": 223
+    },
+    {
+      "epoch": 1.1732283464566928,
+      "grad_norm": 2.748023271560669,
+      "learning_rate": 0.0001526279016172008,
+      "loss": 0.8127,
+      "step": 224
+    },
+    {
+      "epoch": 1.178477690288714,
+      "grad_norm": 2.4598255157470703,
+      "learning_rate": 0.00015210619019925066,
+      "loss": 0.7711,
+      "step": 225
+    },
+    {
+      "epoch": 1.1837270341207349,
+      "grad_norm": 2.580259323120117,
+      "learning_rate": 0.00015158252465343242,
+      "loss": 0.8544,
+      "step": 226
+    },
+    {
+      "epoch": 1.188976377952756,
+      "grad_norm": 2.5548136234283447,
+      "learning_rate": 0.00015105692461866874,
+      "loss": 0.7007,
+      "step": 227
+    },
+    {
+      "epoch": 1.194225721784777,
+      "grad_norm": 2.474823236465454,
+      "learning_rate": 0.000150529409806431,
+      "loss": 0.7501,
+      "step": 228
+    },
+    {
+      "epoch": 1.1994750656167978,
+      "grad_norm": 2.46974515914917,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 0.6326,
+      "step": 229
+    },
+    {
+      "epoch": 1.204724409448819,
+      "grad_norm": 2.542745590209961,
+      "learning_rate": 0.00014946871505372425,
+      "loss": 0.63,
+      "step": 230
+    },
+    {
+      "epoch": 1.20997375328084,
+      "grad_norm": 2.988248109817505,
+      "learning_rate": 0.00014893557489227517,
+      "loss": 1.0908,
+      "step": 231
+    },
+    {
+      "epoch": 1.2152230971128608,
+      "grad_norm": 3.3131489753723145,
+      "learning_rate": 0.0001484005995098999,
+      "loss": 1.2089,
+      "step": 232
+    },
+    {
+      "epoch": 1.220472440944882,
+      "grad_norm": 3.060359477996826,
+      "learning_rate": 0.0001478638089696716,
+      "loss": 0.7317,
+      "step": 233
+    },
+    {
+      "epoch": 1.2257217847769029,
+      "grad_norm": 3.0129809379577637,
+      "learning_rate": 0.00014732522340273684,
+      "loss": 0.6776,
+      "step": 234
+    },
+    {
+      "epoch": 1.2309711286089238,
+      "grad_norm": 2.892338752746582,
+      "learning_rate": 0.0001467848630075608,
+      "loss": 0.731,
+      "step": 235
+    },
+    {
+      "epoch": 1.236220472440945,
+      "grad_norm": 3.0470635890960693,
+      "learning_rate": 0.00014624274804916958,
+      "loss": 0.9027,
+      "step": 236
+    },
+    {
+      "epoch": 1.2414698162729658,
+      "grad_norm": 2.4284868240356445,
+      "learning_rate": 0.00014569889885839037,
+      "loss": 0.7888,
+      "step": 237
+    },
+    {
+      "epoch": 1.246719160104987,
+      "grad_norm": 2.565948486328125,
+      "learning_rate": 0.00014515333583108896,
+      "loss": 0.6491,
+      "step": 238
+    },
+    {
+      "epoch": 1.2519685039370079,
+      "grad_norm": 2.1786141395568848,
+      "learning_rate": 0.00014460607942740468,
+      "loss": 0.7064,
+      "step": 239
+    },
+    {
+      "epoch": 1.257217847769029,
+      "grad_norm": 2.6910786628723145,
+      "learning_rate": 0.00014405715017098335,
+      "loss": 0.6241,
+      "step": 240
+    },
+    {
+      "epoch": 1.26246719160105,
+      "grad_norm": 2.9195802211761475,
+      "learning_rate": 0.00014350656864820733,
+      "loss": 0.815,
+      "step": 241
+    },
+    {
+      "epoch": 1.2677165354330708,
+      "grad_norm": 2.84618878364563,
+      "learning_rate": 0.0001429543555074237,
+      "loss": 0.8676,
+      "step": 242
+    },
+    {
+      "epoch": 1.272965879265092,
+      "grad_norm": 2.5274624824523926,
+      "learning_rate": 0.00014240053145816967,
+      "loss": 0.8162,
+      "step": 243
+    },
+    {
+      "epoch": 1.2782152230971129,
+      "grad_norm": 2.549936294555664,
+      "learning_rate": 0.00014184511727039612,
+      "loss": 0.7318,
+      "step": 244
+    },
+    {
+      "epoch": 1.2834645669291338,
+      "grad_norm": 2.6315500736236572,
+      "learning_rate": 0.0001412881337736885,
+      "loss": 0.828,
+      "step": 245
+    },
+    {
+      "epoch": 1.288713910761155,
+      "grad_norm": 2.9797654151916504,
+      "learning_rate": 0.00014072960185648577,
+      "loss": 0.8592,
+      "step": 246
+    },
+    {
+      "epoch": 1.2939632545931758,
+      "grad_norm": 3.380535364151001,
+      "learning_rate": 0.00014016954246529696,
+      "loss": 1.2413,
+      "step": 247
+    },
+    {
+      "epoch": 1.2992125984251968,
+      "grad_norm": 2.557527780532837,
+      "learning_rate": 0.0001396079766039157,
+      "loss": 0.7001,
+      "step": 248
+    },
+    {
+      "epoch": 1.304461942257218,
+      "grad_norm": 2.8971340656280518,
+      "learning_rate": 0.00013904492533263244,
+      "loss": 0.7283,
+      "step": 249
+    },
+    {
+      "epoch": 1.3097112860892388,
+      "grad_norm": 2.7849743366241455,
+      "learning_rate": 0.00013848040976744457,
+      "loss": 0.7282,
+      "step": 250
+    },
+    {
+      "epoch": 1.3149606299212597,
+      "grad_norm": 2.8634750843048096,
+      "learning_rate": 0.00013791445107926478,
+      "loss": 0.8419,
+      "step": 251
+    },
+    {
+      "epoch": 1.3202099737532809,
+      "grad_norm": 2.310309410095215,
+      "learning_rate": 0.00013734707049312673,
+      "loss": 0.5901,
+      "step": 252
+    },
+    {
+      "epoch": 1.3254593175853018,
+      "grad_norm": 2.7387428283691406,
+      "learning_rate": 0.00013677828928738934,
+      "loss": 0.7358,
+      "step": 253
+    },
+    {
+      "epoch": 1.330708661417323,
+      "grad_norm": 2.7614364624023438,
+      "learning_rate": 0.00013620812879293863,
+      "loss": 0.792,
+      "step": 254
+    },
+    {
+      "epoch": 1.3359580052493438,
+      "grad_norm": 2.8637635707855225,
+      "learning_rate": 0.00013563661039238785,
+      "loss": 0.8998,
+      "step": 255
+    },
+    {
+      "epoch": 1.341207349081365,
+      "grad_norm": 2.455495595932007,
+      "learning_rate": 0.00013506375551927547,
+      "loss": 0.6392,
+      "step": 256
+    },
+    {
+      "epoch": 1.3464566929133859,
+      "grad_norm": 3.1629257202148438,
+      "learning_rate": 0.00013448958565726144,
+      "loss": 0.8548,
+      "step": 257
+    },
+    {
+      "epoch": 1.3517060367454068,
+      "grad_norm": 2.6103389263153076,
+      "learning_rate": 0.00013391412233932149,
+      "loss": 0.6498,
+      "step": 258
+    },
+    {
+      "epoch": 1.356955380577428,
+      "grad_norm": 2.9760587215423584,
+      "learning_rate": 0.00013333738714693956,
+      "loss": 0.7394,
+      "step": 259
+    },
+    {
+      "epoch": 1.3622047244094488,
+      "grad_norm": 2.5022411346435547,
+      "learning_rate": 0.00013275940170929843,
+      "loss": 0.6005,
+      "step": 260
+    },
+    {
+      "epoch": 1.3674540682414698,
+      "grad_norm": 2.8095810413360596,
+      "learning_rate": 0.00013218018770246858,
+      "loss": 0.6823,
+      "step": 261
+    },
+    {
+      "epoch": 1.372703412073491,
+      "grad_norm": 2.7416610717773438,
+      "learning_rate": 0.00013159976684859527,
+      "loss": 0.7922,
+      "step": 262
+    },
+    {
+      "epoch": 1.3779527559055118,
+      "grad_norm": 2.2850701808929443,
+      "learning_rate": 0.00013101816091508388,
+      "loss": 0.7069,
+      "step": 263
+    },
+    {
+      "epoch": 1.3832020997375327,
+      "grad_norm": 2.835042953491211,
+      "learning_rate": 0.0001304353917137836,
+      "loss": 0.7351,
+      "step": 264
+    },
+    {
+      "epoch": 1.3884514435695539,
+      "grad_norm": 2.9981067180633545,
+      "learning_rate": 0.00012985148110016947,
+      "loss": 0.8191,
+      "step": 265
+    },
+    {
+      "epoch": 1.3937007874015748,
+      "grad_norm": 2.754315137863159,
+      "learning_rate": 0.0001292664509725226,
+      "loss": 0.6671,
+      "step": 266
+    },
+    {
+      "epoch": 1.3989501312335957,
+      "grad_norm": 2.780714750289917,
+      "learning_rate": 0.00012868032327110904,
+      "loss": 0.6961,
+      "step": 267
+    },
+    {
+      "epoch": 1.4041994750656168,
+      "grad_norm": 2.5875256061553955,
+      "learning_rate": 0.00012809311997735696,
+      "loss": 0.5871,
+      "step": 268
+    },
+    {
+      "epoch": 1.4094488188976377,
+      "grad_norm": 2.6932876110076904,
+      "learning_rate": 0.00012750486311303218,
+      "loss": 0.7657,
+      "step": 269
+    },
+    {
+      "epoch": 1.4146981627296589,
+      "grad_norm": 2.3105850219726562,
+      "learning_rate": 0.00012691557473941243,
+      "loss": 0.4689,
+      "step": 270
+    },
+    {
+      "epoch": 1.4199475065616798,
+      "grad_norm": 2.749490261077881,
+      "learning_rate": 0.00012632527695645993,
+      "loss": 0.9078,
+      "step": 271
+    },
+    {
+      "epoch": 1.425196850393701,
+      "grad_norm": 2.5074801445007324,
+      "learning_rate": 0.0001257339919019925,
+      "loss": 0.6867,
+      "step": 272
+    },
+    {
+      "epoch": 1.4304461942257218,
+      "grad_norm": 2.0776736736297607,
+      "learning_rate": 0.00012514174175085345,
+      "loss": 0.6132,
+      "step": 273
+    },
+    {
+      "epoch": 1.4356955380577427,
+      "grad_norm": 2.6735877990722656,
+      "learning_rate": 0.00012454854871407994,
+      "loss": 0.7334,
+      "step": 274
+    },
+    {
+      "epoch": 1.4409448818897639,
+      "grad_norm": 2.643948554992676,
+      "learning_rate": 0.0001239544350380699,
+      "loss": 0.6834,
+      "step": 275
+    },
+    {
+      "epoch": 1.4461942257217848,
+      "grad_norm": 3.092463254928589,
+      "learning_rate": 0.00012335942300374788,
+      "loss": 1.0018,
+      "step": 276
+    },
+    {
+      "epoch": 1.4514435695538057,
+      "grad_norm": 2.8050155639648438,
+      "learning_rate": 0.00012276353492572935,
+      "loss": 0.8914,
+      "step": 277
+    },
+    {
+      "epoch": 1.4566929133858268,
+      "grad_norm": 2.794189214706421,
+      "learning_rate": 0.00012216679315148386,
+      "loss": 0.8057,
+      "step": 278
+    },
+    {
+      "epoch": 1.4619422572178478,
+      "grad_norm": 2.8576409816741943,
+      "learning_rate": 0.00012156922006049702,
+      "loss": 0.9259,
+      "step": 279
+    },
+    {
+      "epoch": 1.4671916010498687,
+      "grad_norm": 3.4657177925109863,
+      "learning_rate": 0.00012097083806343103,
+      "loss": 1.0509,
+      "step": 280
+    },
+    {
+      "epoch": 1.4724409448818898,
+      "grad_norm": 2.971736431121826,
+      "learning_rate": 0.00012037166960128443,
+      "loss": 1.2074,
+      "step": 281
+    },
+    {
+      "epoch": 1.4776902887139107,
+      "grad_norm": 2.635852336883545,
+      "learning_rate": 0.00011977173714455034,
+      "loss": 0.739,
+      "step": 282
+    },
+    {
+      "epoch": 1.4829396325459316,
+      "grad_norm": 2.4697046279907227,
+      "learning_rate": 0.00011917106319237386,
+      "loss": 0.8156,
+      "step": 283
+    },
+    {
+      "epoch": 1.4881889763779528,
+      "grad_norm": 2.8160102367401123,
+      "learning_rate": 0.00011856967027170818,
+      "loss": 0.7248,
+      "step": 284
+    },
+    {
+      "epoch": 1.4934383202099737,
+      "grad_norm": 2.7745792865753174,
+      "learning_rate": 0.00011796758093646989,
+      "loss": 0.7826,
+      "step": 285
+    },
+    {
+      "epoch": 1.4986876640419948,
+      "grad_norm": 2.684284210205078,
+      "learning_rate": 0.00011736481776669306,
+      "loss": 0.6299,
+      "step": 286
+    },
+    {
+      "epoch": 1.5039370078740157,
+      "grad_norm": 2.7349674701690674,
+      "learning_rate": 0.00011676140336768236,
+      "loss": 0.8248,
+      "step": 287
+    },
+    {
+      "epoch": 1.5091863517060369,
+      "grad_norm": 3.207608938217163,
+      "learning_rate": 0.00011615736036916549,
+      "loss": 1.0109,
+      "step": 288
+    },
+    {
+      "epoch": 1.5144356955380578,
+      "grad_norm": 2.8411524295806885,
+      "learning_rate": 0.00011555271142444433,
+      "loss": 0.8515,
+      "step": 289
+    },
+    {
+      "epoch": 1.5196850393700787,
+      "grad_norm": 2.936953544616699,
+      "learning_rate": 0.00011494747920954545,
+      "loss": 0.7542,
+      "step": 290
+    },
+    {
+      "epoch": 1.5249343832020998,
+      "grad_norm": 2.1223931312561035,
+      "learning_rate": 0.00011434168642236964,
+      "loss": 0.5709,
+      "step": 291
+    },
+    {
+      "epoch": 1.5301837270341208,
+      "grad_norm": 3.362982988357544,
+      "learning_rate": 0.00011373535578184082,
+      "loss": 0.9206,
+      "step": 292
+    },
+    {
+      "epoch": 1.5354330708661417,
+      "grad_norm": 3.006317138671875,
+      "learning_rate": 0.00011312851002705383,
+      "loss": 1.017,
+      "step": 293
+    },
+    {
+      "epoch": 1.5406824146981628,
+      "grad_norm": 3.088223934173584,
+      "learning_rate": 0.00011252117191642175,
+      "loss": 1.025,
+      "step": 294
+    },
+    {
+      "epoch": 1.5459317585301837,
+      "grad_norm": 2.5200142860412598,
+      "learning_rate": 0.00011191336422682237,
+      "loss": 0.5996,
+      "step": 295
+    },
+    {
+      "epoch": 1.5511811023622046,
+      "grad_norm": 2.9995408058166504,
+      "learning_rate": 0.00011130510975274409,
+      "loss": 0.6813,
+      "step": 296
+    },
+    {
+      "epoch": 1.5564304461942258,
+      "grad_norm": 2.932790994644165,
+      "learning_rate": 0.00011069643130543084,
+      "loss": 0.9218,
+      "step": 297
+    },
+    {
+      "epoch": 1.5616797900262467,
+      "grad_norm": 2.309981346130371,
+      "learning_rate": 0.00011008735171202684,
+      "loss": 0.7003,
+      "step": 298
+    },
+    {
+      "epoch": 1.5669291338582676,
+      "grad_norm": 2.602471113204956,
+      "learning_rate": 0.00010947789381472035,
+      "loss": 0.7051,
+      "step": 299
+    },
+    {
+      "epoch": 1.5721784776902887,
+      "grad_norm": 2.633842945098877,
+      "learning_rate": 0.00010886808046988717,
+      "loss": 0.6228,
+      "step": 300
+    },
+    {
+      "epoch": 1.5774278215223099,
+      "grad_norm": 3.1037299633026123,
+      "learning_rate": 0.00010825793454723325,
+      "loss": 0.7738,
+      "step": 301
+    },
+    {
+      "epoch": 1.5826771653543306,
+      "grad_norm": 2.92518949508667,
+      "learning_rate": 0.00010764747892893723,
+      "loss": 1.1082,
+      "step": 302
+    },
+    {
+      "epoch": 1.5879265091863517,
+      "grad_norm": 2.7783684730529785,
+      "learning_rate": 0.00010703673650879218,
+      "loss": 0.7418,
+      "step": 303
+    },
+    {
+      "epoch": 1.5931758530183728,
+      "grad_norm": 2.6031315326690674,
+      "learning_rate": 0.00010642573019134703,
+      "loss": 0.6829,
+      "step": 304
+    },
+    {
+      "epoch": 1.5984251968503937,
+      "grad_norm": 2.9734861850738525,
+      "learning_rate": 0.00010581448289104758,
+      "loss": 0.7531,
+      "step": 305
+    },
+    {
+      "epoch": 1.6036745406824147,
+      "grad_norm": 2.526393175125122,
+      "learning_rate": 0.00010520301753137724,
+      "loss": 0.7303,
+      "step": 306
+    },
+    {
+      "epoch": 1.6089238845144358,
+      "grad_norm": 3.016721487045288,
+      "learning_rate": 0.00010459135704399718,
+      "loss": 0.9378,
+      "step": 307
+    },
+    {
+      "epoch": 1.6141732283464567,
+      "grad_norm": 2.5446736812591553,
+      "learning_rate": 0.00010397952436788642,
+      "loss": 0.6229,
+      "step": 308
+    },
+    {
+      "epoch": 1.6194225721784776,
+      "grad_norm": 2.3816916942596436,
+      "learning_rate": 0.00010336754244848157,
+      "loss": 0.6167,
+      "step": 309
+    },
+    {
+      "epoch": 1.6246719160104988,
+      "grad_norm": 2.27657413482666,
+      "learning_rate": 0.00010275543423681621,
+      "loss": 0.5536,
+      "step": 310
+    },
+    {
+      "epoch": 1.6299212598425197,
+      "grad_norm": 2.585496664047241,
+      "learning_rate": 0.00010214322268866032,
+      "loss": 0.5935,
+      "step": 311
+    },
+    {
+      "epoch": 1.6351706036745406,
+      "grad_norm": 2.376311779022217,
+      "learning_rate": 0.00010153093076365923,
+      "loss": 0.6658,
+      "step": 312
+    },
+    {
+      "epoch": 1.6404199475065617,
+      "grad_norm": 2.830921173095703,
+      "learning_rate": 0.00010091858142447265,
+      "loss": 0.7428,
+      "step": 313
+    },
+    {
+      "epoch": 1.6456692913385826,
+      "grad_norm": 2.712291717529297,
+      "learning_rate": 0.00010030619763591347,
+      "loss": 0.7036,
+      "step": 314
+    },
+    {
+      "epoch": 1.6509186351706036,
+      "grad_norm": 3.999314546585083,
+      "learning_rate": 9.969380236408656e-05,
+      "loss": 1.0524,
+      "step": 315
+    },
+    {
+      "epoch": 1.6561679790026247,
+      "grad_norm": 3.0849058628082275,
+      "learning_rate": 9.908141857552737e-05,
+      "loss": 0.6273,
+      "step": 316
+    },
+    {
+      "epoch": 1.6614173228346458,
+      "grad_norm": 2.9424805641174316,
+      "learning_rate": 9.846906923634079e-05,
+      "loss": 0.6581,
+      "step": 317
+    },
+    {
+      "epoch": 1.6666666666666665,
+      "grad_norm": 2.7930872440338135,
+      "learning_rate": 9.78567773113397e-05,
+      "loss": 0.7108,
+      "step": 318
+    },
+    {
+      "epoch": 1.6719160104986877,
+      "grad_norm": 2.6098508834838867,
+      "learning_rate": 9.724456576318381e-05,
+      "loss": 0.7179,
+      "step": 319
+    },
+    {
+      "epoch": 1.6771653543307088,
+      "grad_norm": 3.0556881427764893,
+      "learning_rate": 9.663245755151846e-05,
+      "loss": 0.7449,
+      "step": 320
+    },
+    {
+      "epoch": 1.6824146981627297,
+      "grad_norm": 2.536299705505371,
+      "learning_rate": 9.602047563211359e-05,
+      "loss": 0.5627,
+      "step": 321
+    },
+    {
+      "epoch": 1.6876640419947506,
+      "grad_norm": 2.9197168350219727,
+      "learning_rate": 9.540864295600283e-05,
+      "loss": 0.6701,
+      "step": 322
+    },
+    {
+      "epoch": 1.6929133858267718,
+      "grad_norm": 2.77664852142334,
+      "learning_rate": 9.479698246862276e-05,
+      "loss": 0.8544,
+      "step": 323
+    },
+    {
+      "epoch": 1.6981627296587927,
+      "grad_norm": 2.833603858947754,
+      "learning_rate": 9.418551710895243e-05,
+      "loss": 0.7597,
+      "step": 324
+    },
+    {
+      "epoch": 1.7034120734908136,
+      "grad_norm": 2.527200937271118,
+      "learning_rate": 9.357426980865301e-05,
+      "loss": 0.7198,
+      "step": 325
+    },
+    {
+      "epoch": 1.7086614173228347,
+      "grad_norm": 2.7248785495758057,
+      "learning_rate": 9.296326349120785e-05,
+      "loss": 0.7121,
+      "step": 326
+    },
+    {
+      "epoch": 1.7139107611548556,
+      "grad_norm": 2.6824331283569336,
+      "learning_rate": 9.235252107106279e-05,
+      "loss": 0.884,
+      "step": 327
+    },
+    {
+      "epoch": 1.7191601049868765,
+      "grad_norm": 2.580566167831421,
+      "learning_rate": 9.174206545276677e-05,
+      "loss": 0.6406,
+      "step": 328
+    },
+    {
+      "epoch": 1.7244094488188977,
+      "grad_norm": 2.741162061691284,
+      "learning_rate": 9.113191953011287e-05,
+      "loss": 0.8105,
+      "step": 329
+    },
+    {
+      "epoch": 1.7296587926509186,
+      "grad_norm": 2.2699320316314697,
+      "learning_rate": 9.052210618527966e-05,
+      "loss": 0.598,
+      "step": 330
+    },
+    {
+      "epoch": 1.7349081364829395,
+      "grad_norm": 2.4851629734039307,
+      "learning_rate": 8.991264828797319e-05,
+      "loss": 0.7384,
+      "step": 331
+    },
+    {
+      "epoch": 1.7401574803149606,
+      "grad_norm": 2.476698637008667,
+      "learning_rate": 8.930356869456919e-05,
+      "loss": 0.6595,
+      "step": 332
+    },
+    {
+      "epoch": 1.7454068241469818,
+      "grad_norm": 2.465303897857666,
+      "learning_rate": 8.869489024725595e-05,
+      "loss": 0.7266,
+      "step": 333
+    },
+    {
+      "epoch": 1.7506561679790025,
+      "grad_norm": 2.6506006717681885,
+      "learning_rate": 8.808663577317764e-05,
+      "loss": 0.8465,
+      "step": 334
+    },
+    {
+      "epoch": 1.7559055118110236,
+      "grad_norm": 2.6736481189727783,
+      "learning_rate": 8.747882808357828e-05,
+      "loss": 0.7328,
+      "step": 335
+    },
+    {
+      "epoch": 1.7611548556430447,
+      "grad_norm": 2.2621066570281982,
+      "learning_rate": 8.687148997294621e-05,
+      "loss": 0.5939,
+      "step": 336
+    },
+    {
+      "epoch": 1.7664041994750657,
+      "grad_norm": 2.6498119831085205,
+      "learning_rate": 8.626464421815919e-05,
+      "loss": 0.6821,
+      "step": 337
+    },
+    {
+      "epoch": 1.7716535433070866,
+      "grad_norm": 2.5098793506622314,
+      "learning_rate": 8.565831357763039e-05,
+      "loss": 0.6737,
+      "step": 338
+    },
+    {
+      "epoch": 1.7769028871391077,
+      "grad_norm": 2.26190447807312,
+      "learning_rate": 8.505252079045458e-05,
+      "loss": 0.6358,
+      "step": 339
+    },
+    {
+      "epoch": 1.7821522309711286,
+      "grad_norm": 2.6021599769592285,
+      "learning_rate": 8.444728857555572e-05,
+      "loss": 0.764,
+      "step": 340
+    },
+    {
+      "epoch": 1.7874015748031495,
+      "grad_norm": 2.805920362472534,
+      "learning_rate": 8.384263963083453e-05,
+      "loss": 0.7641,
+      "step": 341
+    },
+    {
+      "epoch": 1.7926509186351707,
+      "grad_norm": 3.371159791946411,
+      "learning_rate": 8.323859663231768e-05,
+      "loss": 0.8458,
+      "step": 342
+    },
+    {
+      "epoch": 1.7979002624671916,
+      "grad_norm": 3.0499484539031982,
+      "learning_rate": 8.263518223330697e-05,
+      "loss": 0.7696,
+      "step": 343
+    },
+    {
+      "epoch": 1.8031496062992125,
+      "grad_norm": 3.3315682411193848,
+      "learning_rate": 8.203241906353014e-05,
+      "loss": 0.8452,
+      "step": 344
+    },
+    {
+      "epoch": 1.8083989501312336,
+      "grad_norm": 2.8910834789276123,
+      "learning_rate": 8.143032972829183e-05,
+      "loss": 0.7281,
+      "step": 345
+    },
+    {
+      "epoch": 1.8136482939632546,
+      "grad_norm": 2.424577236175537,
+      "learning_rate": 8.082893680762619e-05,
+      "loss": 0.5982,
+      "step": 346
+    },
+    {
+      "epoch": 1.8188976377952755,
+      "grad_norm": 2.4802539348602295,
+      "learning_rate": 8.022826285544968e-05,
+      "loss": 0.4913,
+      "step": 347
+    },
+    {
+      "epoch": 1.8241469816272966,
+      "grad_norm": 3.608314037322998,
+      "learning_rate": 7.96283303987156e-05,
+      "loss": 0.965,
+      "step": 348
+    },
+    {
+      "epoch": 1.8293963254593177,
+      "grad_norm": 2.9640915393829346,
+      "learning_rate": 7.902916193656898e-05,
+      "loss": 0.8494,
+      "step": 349
+    },
+    {
+      "epoch": 1.8346456692913384,
+      "grad_norm": 3.1008963584899902,
+      "learning_rate": 7.843077993950302e-05,
+      "loss": 0.9431,
+      "step": 350
+    },
+    {
+      "epoch": 1.8398950131233596,
+      "grad_norm": 3.1320395469665527,
+      "learning_rate": 7.783320684851614e-05,
+      "loss": 1.0122,
+      "step": 351
+    },
+    {
+      "epoch": 1.8451443569553807,
+      "grad_norm": 2.527387857437134,
+      "learning_rate": 7.72364650742707e-05,
+      "loss": 0.7018,
+      "step": 352
+    },
+    {
+      "epoch": 1.8503937007874016,
+      "grad_norm": 2.8600051403045654,
+      "learning_rate": 7.664057699625214e-05,
+      "loss": 0.7482,
+      "step": 353
+    },
+    {
+      "epoch": 1.8556430446194225,
+      "grad_norm": 2.6719634532928467,
+      "learning_rate": 7.604556496193015e-05,
+      "loss": 0.7669,
+      "step": 354
+    },
+    {
+      "epoch": 1.8608923884514437,
+      "grad_norm": 2.4626736640930176,
+      "learning_rate": 7.54514512859201e-05,
+      "loss": 0.6617,
+      "step": 355
+    },
+    {
+      "epoch": 1.8661417322834646,
+      "grad_norm": 3.1328530311584473,
+      "learning_rate": 7.485825824914659e-05,
+      "loss": 0.9866,
+      "step": 356
+    },
+    {
+      "epoch": 1.8713910761154855,
+      "grad_norm": 2.8781819343566895,
+      "learning_rate": 7.426600809800752e-05,
+      "loss": 0.8311,
+      "step": 357
+    },
+    {
+      "epoch": 1.8766404199475066,
+      "grad_norm": 2.889805793762207,
+      "learning_rate": 7.36747230435401e-05,
+      "loss": 0.6937,
+      "step": 358
+    },
+    {
+      "epoch": 1.8818897637795275,
+      "grad_norm": 3.0408389568328857,
+      "learning_rate": 7.308442526058756e-05,
+      "loss": 0.6836,
+      "step": 359
+    },
+    {
+      "epoch": 1.8871391076115485,
+      "grad_norm": 3.536181926727295,
+      "learning_rate": 7.249513688696786e-05,
+      "loss": 1.0847,
+      "step": 360
+    },
+    {
+      "epoch": 1.8923884514435696,
+      "grad_norm": 3.20278263092041,
+      "learning_rate": 7.190688002264308e-05,
+      "loss": 1.0111,
+      "step": 361
+    },
+    {
+      "epoch": 1.8976377952755905,
+      "grad_norm": 2.5212113857269287,
+      "learning_rate": 7.131967672889101e-05,
+      "loss": 0.5622,
+      "step": 362
+    },
+    {
+      "epoch": 1.9028871391076114,
+      "grad_norm": 2.54811429977417,
+      "learning_rate": 7.073354902747741e-05,
+      "loss": 0.6633,
+      "step": 363
+    },
+    {
+      "epoch": 1.9081364829396326,
+      "grad_norm": 2.259267568588257,
+      "learning_rate": 7.014851889983057e-05,
+      "loss": 0.5843,
+      "step": 364
+    },
+    {
+      "epoch": 1.9133858267716537,
+      "grad_norm": 2.4579734802246094,
+      "learning_rate": 6.95646082862164e-05,
+      "loss": 0.573,
+      "step": 365
+    },
+    {
+      "epoch": 1.9186351706036744,
+      "grad_norm": 3.317549228668213,
+      "learning_rate": 6.898183908491617e-05,
+      "loss": 0.9837,
+      "step": 366
+    },
+    {
+      "epoch": 1.9238845144356955,
+      "grad_norm": 2.377362012863159,
+      "learning_rate": 6.840023315140475e-05,
+      "loss": 0.7062,
+      "step": 367
+    },
+    {
+      "epoch": 1.9291338582677167,
+      "grad_norm": 2.8858208656311035,
+      "learning_rate": 6.781981229753145e-05,
+      "loss": 0.7657,
+      "step": 368
+    },
+    {
+      "epoch": 1.9343832020997376,
+      "grad_norm": 2.407649278640747,
+      "learning_rate": 6.724059829070158e-05,
+      "loss": 0.6103,
+      "step": 369
+    },
+    {
+      "epoch": 1.9396325459317585,
+      "grad_norm": 2.943190813064575,
+      "learning_rate": 6.666261285306047e-05,
+      "loss": 0.9676,
+      "step": 370
+    },
+    {
+      "epoch": 1.9448818897637796,
+      "grad_norm": 2.791794538497925,
+      "learning_rate": 6.608587766067852e-05,
+      "loss": 0.7191,
+      "step": 371
+    },
+    {
+      "epoch": 1.9501312335958005,
+      "grad_norm": 2.5075485706329346,
+      "learning_rate": 6.551041434273861e-05,
+      "loss": 0.634,
+      "step": 372
+    },
+    {
+      "epoch": 1.9553805774278215,
+      "grad_norm": 2.816528797149658,
+      "learning_rate": 6.493624448072457e-05,
+      "loss": 0.7335,
+      "step": 373
+    },
+    {
+      "epoch": 1.9606299212598426,
+      "grad_norm": 2.5737648010253906,
+      "learning_rate": 6.43633896076122e-05,
+      "loss": 0.6795,
+      "step": 374
+    },
+    {
+      "epoch": 1.9658792650918635,
+      "grad_norm": 2.7753782272338867,
+      "learning_rate": 6.379187120706138e-05,
+      "loss": 0.721,
+      "step": 375
+    },
+    {
+      "epoch": 1.9711286089238844,
+      "grad_norm": 2.586721181869507,
+      "learning_rate": 6.322171071261071e-05,
+      "loss": 0.6044,
+      "step": 376
+    },
+    {
+      "epoch": 1.9763779527559056,
+      "grad_norm": 2.821720838546753,
+      "learning_rate": 6.26529295068733e-05,
+      "loss": 0.687,
+      "step": 377
+    },
+    {
+      "epoch": 1.9816272965879265,
+      "grad_norm": 2.5189385414123535,
+      "learning_rate": 6.208554892073528e-05,
+      "loss": 0.6973,
+      "step": 378
+    },
+    {
+      "epoch": 1.9868766404199474,
+      "grad_norm": 2.7033824920654297,
+      "learning_rate": 6.151959023255545e-05,
+      "loss": 0.5675,
+      "step": 379
+    },
+    {
+      "epoch": 1.9921259842519685,
+      "grad_norm": 2.864698886871338,
+      "learning_rate": 6.095507466736763e-05,
+      "loss": 0.8994,
+      "step": 380
+    },
+    {
+      "epoch": 1.9973753280839897,
+      "grad_norm": 2.771103858947754,
+      "learning_rate": 6.039202339608432e-05,
+      "loss": 0.7414,
+      "step": 381
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 4.474254131317139,
+      "learning_rate": 5.983045753470308e-05,
+      "loss": 0.8273,
+      "step": 382
+    },
+    {
+      "epoch": 2.005249343832021,
+      "grad_norm": 1.8631569147109985,
+      "learning_rate": 5.927039814351426e-05,
+      "loss": 0.3455,
+      "step": 383
+    },
+    {
+      "epoch": 2.010498687664042,
+      "grad_norm": 1.857398271560669,
+      "learning_rate": 5.8711866226311553e-05,
+      "loss": 0.3454,
+      "step": 384
+    },
+    {
+      "epoch": 2.015748031496063,
+      "grad_norm": 2.315157651901245,
+      "learning_rate": 5.8154882729603876e-05,
+      "loss": 0.4925,
+      "step": 385
+    },
+    {
+      "epoch": 2.020997375328084,
+      "grad_norm": 2.0346195697784424,
+      "learning_rate": 5.7599468541830356e-05,
+      "loss": 0.3541,
+      "step": 386
+    },
+    {
+      "epoch": 2.026246719160105,
+      "grad_norm": 2.2926833629608154,
+      "learning_rate": 5.7045644492576346e-05,
+      "loss": 0.3705,
+      "step": 387
+    },
+    {
+      "epoch": 2.031496062992126,
+      "grad_norm": 1.8985930681228638,
+      "learning_rate": 5.64934313517927e-05,
+      "loss": 0.3665,
+      "step": 388
+    },
+    {
+      "epoch": 2.036745406824147,
+      "grad_norm": 1.6224257946014404,
+      "learning_rate": 5.5942849829016695e-05,
+      "loss": 0.2719,
+      "step": 389
+    },
+    {
+      "epoch": 2.041994750656168,
+      "grad_norm": 1.9239685535430908,
+      "learning_rate": 5.5393920572595356e-05,
+      "loss": 0.2669,
+      "step": 390
+    },
+    {
+      "epoch": 2.047244094488189,
+      "grad_norm": 1.7690950632095337,
+      "learning_rate": 5.484666416891109e-05,
+      "loss": 0.2509,
+      "step": 391
+    },
+    {
+      "epoch": 2.05249343832021,
+      "grad_norm": 2.04697322845459,
+      "learning_rate": 5.430110114160964e-05,
+      "loss": 0.2676,
+      "step": 392
+    },
+    {
+      "epoch": 2.057742782152231,
+      "grad_norm": 2.4531140327453613,
+      "learning_rate": 5.375725195083046e-05,
+      "loss": 0.3482,
+      "step": 393
+    },
+    {
+      "epoch": 2.062992125984252,
+      "grad_norm": 2.427957057952881,
+      "learning_rate": 5.321513699243924e-05,
+      "loss": 0.3182,
+      "step": 394
+    },
+    {
+      "epoch": 2.068241469816273,
+      "grad_norm": 2.123382091522217,
+      "learning_rate": 5.2674776597263186e-05,
+      "loss": 0.2888,
+      "step": 395
+    },
+    {
+      "epoch": 2.073490813648294,
+      "grad_norm": 3.2138147354125977,
+      "learning_rate": 5.2136191030328455e-05,
+      "loss": 0.3561,
+      "step": 396
+    },
+    {
+      "epoch": 2.078740157480315,
+      "grad_norm": 2.332437753677368,
+      "learning_rate": 5.159940049010015e-05,
+      "loss": 0.3039,
+      "step": 397
+    },
+    {
+      "epoch": 2.083989501312336,
+      "grad_norm": 2.8219337463378906,
+      "learning_rate": 5.106442510772489e-05,
+      "loss": 0.3129,
+      "step": 398
+    },
+    {
+      "epoch": 2.089238845144357,
+      "grad_norm": 2.9364748001098633,
+      "learning_rate": 5.0531284946275784e-05,
+      "loss": 0.2937,
+      "step": 399
+    },
+    {
+      "epoch": 2.094488188976378,
+      "grad_norm": 2.8431618213653564,
+      "learning_rate": 5.000000000000002e-05,
+      "loss": 0.3048,
+      "step": 400
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 570,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.535375299019562e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84be915ac356453be8155258c27d9140e543c98c46b31c533caab17efad8a66f
+size 5624