Initial model upload

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-568/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,202 @@
----
-license: apache-2.0
----
+---
+base_model: unsloth/qwq-32b-preview-bnb-4bit
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/qwq-32b-preview-bnb-4bit",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": true
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cdb6133af159aecdfd884d89b887938f2de9fcd125838697f1697d34a4aecab
+size 2147605960

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-500/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/qwq-32b-preview-bnb-4bit
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-500/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/qwq-32b-preview-bnb-4bit",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": true
+}

checkpoint-500/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cdb6133af159aecdfd884d89b887938f2de9fcd125838697f1697d34a4aecab
+size 2147605960

checkpoint-500/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-500/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a901a9a5d0b82d24fd529f22334f48775a959ca301e59768510ecc03599210d
+size 1091573332

checkpoint-500/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1c0fb523914a5a4fd99843319c531ac1b76196088779c16816fd1dd6a05915
+size 14244

checkpoint-500/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:183b296603218453f024f97991de116afc302207a9c1d93500eb201e53da7e60
+size 1064

checkpoint-500/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-500/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-500/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-500/trainer_state.json

@@ -0,0 +1,3573 @@
+{
+  "best_metric": 0.03437558189034462,
+  "best_model_checkpoint": "./models/QwQ-32B-Preview_All-Label-Avengers/checkpoint-500",
+  "epoch": 1.7557117750439368,
+  "eval_steps": 100,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0035149384885764497,
+      "grad_norm": 2.4306211471557617,
+      "learning_rate": 2.325581395348837e-06,
+      "loss": 1.7264,
+      "step": 1
+    },
+    {
+      "epoch": 0.007029876977152899,
+      "grad_norm": 2.4965059757232666,
+      "learning_rate": 4.651162790697674e-06,
+      "loss": 1.7642,
+      "step": 2
+    },
+    {
+      "epoch": 0.01054481546572935,
+      "grad_norm": 2.1308212280273438,
+      "learning_rate": 6.976744186046512e-06,
+      "loss": 1.7182,
+      "step": 3
+    },
+    {
+      "epoch": 0.014059753954305799,
+      "grad_norm": 1.6225489377975464,
+      "learning_rate": 9.302325581395349e-06,
+      "loss": 1.7998,
+      "step": 4
+    },
+    {
+      "epoch": 0.01757469244288225,
+      "grad_norm": 1.573882818222046,
+      "learning_rate": 1.1627906976744187e-05,
+      "loss": 1.769,
+      "step": 5
+    },
+    {
+      "epoch": 0.0210896309314587,
+      "grad_norm": 1.4130934476852417,
+      "learning_rate": 1.3953488372093024e-05,
+      "loss": 1.6766,
+      "step": 6
+    },
+    {
+      "epoch": 0.02460456942003515,
+      "grad_norm": 1.290319800376892,
+      "learning_rate": 1.6279069767441862e-05,
+      "loss": 1.6666,
+      "step": 7
+    },
+    {
+      "epoch": 0.028119507908611598,
+      "grad_norm": 1.12405526638031,
+      "learning_rate": 1.8604651162790697e-05,
+      "loss": 1.5581,
+      "step": 8
+    },
+    {
+      "epoch": 0.03163444639718805,
+      "grad_norm": 1.1816198825836182,
+      "learning_rate": 2.0930232558139536e-05,
+      "loss": 1.4704,
+      "step": 9
+    },
+    {
+      "epoch": 0.0351493848857645,
+      "grad_norm": 1.293810486793518,
+      "learning_rate": 2.3255813953488374e-05,
+      "loss": 1.4473,
+      "step": 10
+    },
+    {
+      "epoch": 0.03866432337434095,
+      "grad_norm": 1.3082643747329712,
+      "learning_rate": 2.5581395348837212e-05,
+      "loss": 1.1881,
+      "step": 11
+    },
+    {
+      "epoch": 0.0421792618629174,
+      "grad_norm": 1.2528855800628662,
+      "learning_rate": 2.7906976744186048e-05,
+      "loss": 1.0205,
+      "step": 12
+    },
+    {
+      "epoch": 0.04569420035149385,
+      "grad_norm": 1.2669192552566528,
+      "learning_rate": 3.0232558139534883e-05,
+      "loss": 0.8347,
+      "step": 13
+    },
+    {
+      "epoch": 0.0492091388400703,
+      "grad_norm": 1.3900455236434937,
+      "learning_rate": 3.2558139534883724e-05,
+      "loss": 0.6902,
+      "step": 14
+    },
+    {
+      "epoch": 0.05272407732864675,
+      "grad_norm": 1.1759059429168701,
+      "learning_rate": 3.488372093023256e-05,
+      "loss": 0.5728,
+      "step": 15
+    },
+    {
+      "epoch": 0.056239015817223195,
+      "grad_norm": 1.1620585918426514,
+      "learning_rate": 3.7209302325581394e-05,
+      "loss": 0.3972,
+      "step": 16
+    },
+    {
+      "epoch": 0.05975395430579965,
+      "grad_norm": 0.7304664254188538,
+      "learning_rate": 3.953488372093023e-05,
+      "loss": 0.3352,
+      "step": 17
+    },
+    {
+      "epoch": 0.0632688927943761,
+      "grad_norm": 0.46442729234695435,
+      "learning_rate": 4.186046511627907e-05,
+      "loss": 0.2574,
+      "step": 18
+    },
+    {
+      "epoch": 0.06678383128295255,
+      "grad_norm": 0.30402088165283203,
+      "learning_rate": 4.418604651162791e-05,
+      "loss": 0.206,
+      "step": 19
+    },
+    {
+      "epoch": 0.070298769771529,
+      "grad_norm": 0.32389989495277405,
+      "learning_rate": 4.651162790697675e-05,
+      "loss": 0.1669,
+      "step": 20
+    },
+    {
+      "epoch": 0.07381370826010544,
+      "grad_norm": 0.20291510224342346,
+      "learning_rate": 4.883720930232558e-05,
+      "loss": 0.2767,
+      "step": 21
+    },
+    {
+      "epoch": 0.0773286467486819,
+      "grad_norm": 0.19848257303237915,
+      "learning_rate": 5.1162790697674425e-05,
+      "loss": 0.1934,
+      "step": 22
+    },
+    {
+      "epoch": 0.08084358523725835,
+      "grad_norm": 0.26607176661491394,
+      "learning_rate": 5.348837209302326e-05,
+      "loss": 0.2551,
+      "step": 23
+    },
+    {
+      "epoch": 0.0843585237258348,
+      "grad_norm": 0.24385203421115875,
+      "learning_rate": 5.5813953488372095e-05,
+      "loss": 0.2151,
+      "step": 24
+    },
+    {
+      "epoch": 0.08787346221441125,
+      "grad_norm": 0.2142629325389862,
+      "learning_rate": 5.8139534883720933e-05,
+      "loss": 0.2451,
+      "step": 25
+    },
+    {
+      "epoch": 0.0913884007029877,
+      "grad_norm": 0.18275344371795654,
+      "learning_rate": 6.0465116279069765e-05,
+      "loss": 0.2199,
+      "step": 26
+    },
+    {
+      "epoch": 0.09490333919156414,
+      "grad_norm": 0.18780425190925598,
+      "learning_rate": 6.27906976744186e-05,
+      "loss": 0.1993,
+      "step": 27
+    },
+    {
+      "epoch": 0.0984182776801406,
+      "grad_norm": 0.15310600399971008,
+      "learning_rate": 6.511627906976745e-05,
+      "loss": 0.1828,
+      "step": 28
+    },
+    {
+      "epoch": 0.10193321616871705,
+      "grad_norm": 0.18113793432712555,
+      "learning_rate": 6.744186046511628e-05,
+      "loss": 0.2343,
+      "step": 29
+    },
+    {
+      "epoch": 0.1054481546572935,
+      "grad_norm": 0.1769862323999405,
+      "learning_rate": 6.976744186046513e-05,
+      "loss": 0.2475,
+      "step": 30
+    },
+    {
+      "epoch": 0.10896309314586995,
+      "grad_norm": 0.15000343322753906,
+      "learning_rate": 7.209302325581396e-05,
+      "loss": 0.1947,
+      "step": 31
+    },
+    {
+      "epoch": 0.11247803163444639,
+      "grad_norm": 0.13723662495613098,
+      "learning_rate": 7.441860465116279e-05,
+      "loss": 0.1622,
+      "step": 32
+    },
+    {
+      "epoch": 0.11599297012302284,
+      "grad_norm": 0.3588716983795166,
+      "learning_rate": 7.674418604651163e-05,
+      "loss": 0.2407,
+      "step": 33
+    },
+    {
+      "epoch": 0.1195079086115993,
+      "grad_norm": 0.13395386934280396,
+      "learning_rate": 7.906976744186047e-05,
+      "loss": 0.1658,
+      "step": 34
+    },
+    {
+      "epoch": 0.12302284710017575,
+      "grad_norm": 0.14775735139846802,
+      "learning_rate": 8.139534883720931e-05,
+      "loss": 0.1826,
+      "step": 35
+    },
+    {
+      "epoch": 0.1265377855887522,
+      "grad_norm": 0.1646275818347931,
+      "learning_rate": 8.372093023255814e-05,
+      "loss": 0.248,
+      "step": 36
+    },
+    {
+      "epoch": 0.13005272407732865,
+      "grad_norm": 0.1475340873003006,
+      "learning_rate": 8.604651162790697e-05,
+      "loss": 0.1973,
+      "step": 37
+    },
+    {
+      "epoch": 0.1335676625659051,
+      "grad_norm": 0.1472110003232956,
+      "learning_rate": 8.837209302325582e-05,
+      "loss": 0.1749,
+      "step": 38
+    },
+    {
+      "epoch": 0.13708260105448156,
+      "grad_norm": 0.175991490483284,
+      "learning_rate": 9.069767441860465e-05,
+      "loss": 0.2015,
+      "step": 39
+    },
+    {
+      "epoch": 0.140597539543058,
+      "grad_norm": 0.15504464507102966,
+      "learning_rate": 9.30232558139535e-05,
+      "loss": 0.1953,
+      "step": 40
+    },
+    {
+      "epoch": 0.14411247803163443,
+      "grad_norm": 0.1356693059206009,
+      "learning_rate": 9.534883720930233e-05,
+      "loss": 0.1514,
+      "step": 41
+    },
+    {
+      "epoch": 0.14762741652021089,
+      "grad_norm": 0.1488504558801651,
+      "learning_rate": 9.767441860465116e-05,
+      "loss": 0.2224,
+      "step": 42
+    },
+    {
+      "epoch": 0.15114235500878734,
+      "grad_norm": 0.15970350801944733,
+      "learning_rate": 0.0001,
+      "loss": 0.204,
+      "step": 43
+    },
+    {
+      "epoch": 0.1546572934973638,
+      "grad_norm": 0.15022525191307068,
+      "learning_rate": 0.00010232558139534885,
+      "loss": 0.1918,
+      "step": 44
+    },
+    {
+      "epoch": 0.15817223198594024,
+      "grad_norm": 0.15593580901622772,
+      "learning_rate": 0.00010465116279069768,
+      "loss": 0.2139,
+      "step": 45
+    },
+    {
+      "epoch": 0.1616871704745167,
+      "grad_norm": 0.18487761914730072,
+      "learning_rate": 0.00010697674418604651,
+      "loss": 0.221,
+      "step": 46
+    },
+    {
+      "epoch": 0.16520210896309315,
+      "grad_norm": 0.14810211956501007,
+      "learning_rate": 0.00010930232558139534,
+      "loss": 0.1725,
+      "step": 47
+    },
+    {
+      "epoch": 0.1687170474516696,
+      "grad_norm": 0.14104707539081573,
+      "learning_rate": 0.00011162790697674419,
+      "loss": 0.1917,
+      "step": 48
+    },
+    {
+      "epoch": 0.17223198594024605,
+      "grad_norm": 0.14195498824119568,
+      "learning_rate": 0.00011395348837209304,
+      "loss": 0.1238,
+      "step": 49
+    },
+    {
+      "epoch": 0.1757469244288225,
+      "grad_norm": 0.17054127156734467,
+      "learning_rate": 0.00011627906976744187,
+      "loss": 0.1907,
+      "step": 50
+    },
+    {
+      "epoch": 0.17926186291739896,
+      "grad_norm": 0.14932486414909363,
+      "learning_rate": 0.00011860465116279071,
+      "loss": 0.2198,
+      "step": 51
+    },
+    {
+      "epoch": 0.1827768014059754,
+      "grad_norm": 0.1489359587430954,
+      "learning_rate": 0.00012093023255813953,
+      "loss": 0.15,
+      "step": 52
+    },
+    {
+      "epoch": 0.18629173989455183,
+      "grad_norm": 0.18924278020858765,
+      "learning_rate": 0.00012325581395348836,
+      "loss": 0.2287,
+      "step": 53
+    },
+    {
+      "epoch": 0.18980667838312829,
+      "grad_norm": 0.16668835282325745,
+      "learning_rate": 0.0001255813953488372,
+      "loss": 0.17,
+      "step": 54
+    },
+    {
+      "epoch": 0.19332161687170474,
+      "grad_norm": 0.16550259292125702,
+      "learning_rate": 0.00012790697674418605,
+      "loss": 0.2501,
+      "step": 55
+    },
+    {
+      "epoch": 0.1968365553602812,
+      "grad_norm": 0.18516315519809723,
+      "learning_rate": 0.0001302325581395349,
+      "loss": 0.221,
+      "step": 56
+    },
+    {
+      "epoch": 0.20035149384885764,
+      "grad_norm": 0.1466783583164215,
+      "learning_rate": 0.00013255813953488372,
+      "loss": 0.165,
+      "step": 57
+    },
+    {
+      "epoch": 0.2038664323374341,
+      "grad_norm": 0.16772057116031647,
+      "learning_rate": 0.00013488372093023256,
+      "loss": 0.173,
+      "step": 58
+    },
+    {
+      "epoch": 0.20738137082601055,
+      "grad_norm": 0.13877983391284943,
+      "learning_rate": 0.0001372093023255814,
+      "loss": 0.1387,
+      "step": 59
+    },
+    {
+      "epoch": 0.210896309314587,
+      "grad_norm": 0.1386060118675232,
+      "learning_rate": 0.00013953488372093025,
+      "loss": 0.1249,
+      "step": 60
+    },
+    {
+      "epoch": 0.21441124780316345,
+      "grad_norm": 0.14806397259235382,
+      "learning_rate": 0.0001418604651162791,
+      "loss": 0.1471,
+      "step": 61
+    },
+    {
+      "epoch": 0.2179261862917399,
+      "grad_norm": 0.14199328422546387,
+      "learning_rate": 0.00014418604651162791,
+      "loss": 0.1218,
+      "step": 62
+    },
+    {
+      "epoch": 0.22144112478031636,
+      "grad_norm": 0.2156495600938797,
+      "learning_rate": 0.00014651162790697673,
+      "loss": 0.1707,
+      "step": 63
+    },
+    {
+      "epoch": 0.22495606326889278,
+      "grad_norm": 0.1671278327703476,
+      "learning_rate": 0.00014883720930232558,
+      "loss": 0.176,
+      "step": 64
+    },
+    {
+      "epoch": 0.22847100175746923,
+      "grad_norm": 0.23461486399173737,
+      "learning_rate": 0.00015116279069767442,
+      "loss": 0.1741,
+      "step": 65
+    },
+    {
+      "epoch": 0.23198594024604569,
+      "grad_norm": 0.234146386384964,
+      "learning_rate": 0.00015348837209302327,
+      "loss": 0.2728,
+      "step": 66
+    },
+    {
+      "epoch": 0.23550087873462214,
+      "grad_norm": 0.17772763967514038,
+      "learning_rate": 0.0001558139534883721,
+      "loss": 0.1268,
+      "step": 67
+    },
+    {
+      "epoch": 0.2390158172231986,
+      "grad_norm": 0.17169110476970673,
+      "learning_rate": 0.00015813953488372093,
+      "loss": 0.1236,
+      "step": 68
+    },
+    {
+      "epoch": 0.24253075571177504,
+      "grad_norm": 0.208359956741333,
+      "learning_rate": 0.00016046511627906978,
+      "loss": 0.1933,
+      "step": 69
+    },
+    {
+      "epoch": 0.2460456942003515,
+      "grad_norm": 0.2004474252462387,
+      "learning_rate": 0.00016279069767441862,
+      "loss": 0.1925,
+      "step": 70
+    },
+    {
+      "epoch": 0.24956063268892795,
+      "grad_norm": 0.22541354596614838,
+      "learning_rate": 0.00016511627906976747,
+      "loss": 0.1802,
+      "step": 71
+    },
+    {
+      "epoch": 0.2530755711775044,
+      "grad_norm": 0.23796068131923676,
+      "learning_rate": 0.00016744186046511629,
+      "loss": 0.203,
+      "step": 72
+    },
+    {
+      "epoch": 0.2565905096660808,
+      "grad_norm": 0.1686377376317978,
+      "learning_rate": 0.0001697674418604651,
+      "loss": 0.1693,
+      "step": 73
+    },
+    {
+      "epoch": 0.2601054481546573,
+      "grad_norm": 0.14461028575897217,
+      "learning_rate": 0.00017209302325581395,
+      "loss": 0.1203,
+      "step": 74
+    },
+    {
+      "epoch": 0.26362038664323373,
+      "grad_norm": 0.1455416977405548,
+      "learning_rate": 0.0001744186046511628,
+      "loss": 0.0993,
+      "step": 75
+    },
+    {
+      "epoch": 0.2671353251318102,
+      "grad_norm": 0.15028974413871765,
+      "learning_rate": 0.00017674418604651164,
+      "loss": 0.1035,
+      "step": 76
+    },
+    {
+      "epoch": 0.27065026362038663,
+      "grad_norm": 0.2451164275407791,
+      "learning_rate": 0.00017906976744186048,
+      "loss": 0.1948,
+      "step": 77
+    },
+    {
+      "epoch": 0.2741652021089631,
+      "grad_norm": 0.159187912940979,
+      "learning_rate": 0.0001813953488372093,
+      "loss": 0.1251,
+      "step": 78
+    },
+    {
+      "epoch": 0.27768014059753954,
+      "grad_norm": 0.25709378719329834,
+      "learning_rate": 0.00018372093023255815,
+      "loss": 0.0994,
+      "step": 79
+    },
+    {
+      "epoch": 0.281195079086116,
+      "grad_norm": 0.1876499205827713,
+      "learning_rate": 0.000186046511627907,
+      "loss": 0.1344,
+      "step": 80
+    },
+    {
+      "epoch": 0.28471001757469244,
+      "grad_norm": 0.17002429068088531,
+      "learning_rate": 0.00018837209302325584,
+      "loss": 0.1226,
+      "step": 81
+    },
+    {
+      "epoch": 0.28822495606326887,
+      "grad_norm": 0.19951826333999634,
+      "learning_rate": 0.00019069767441860466,
+      "loss": 0.181,
+      "step": 82
+    },
+    {
+      "epoch": 0.29173989455184535,
+      "grad_norm": 0.2241663634777069,
+      "learning_rate": 0.0001930232558139535,
+      "loss": 0.1659,
+      "step": 83
+    },
+    {
+      "epoch": 0.29525483304042177,
+      "grad_norm": 0.19991622865200043,
+      "learning_rate": 0.00019534883720930232,
+      "loss": 0.1638,
+      "step": 84
+    },
+    {
+      "epoch": 0.29876977152899825,
+      "grad_norm": 0.17116616666316986,
+      "learning_rate": 0.00019767441860465116,
+      "loss": 0.0865,
+      "step": 85
+    },
+    {
+      "epoch": 0.3022847100175747,
+      "grad_norm": 0.2004881054162979,
+      "learning_rate": 0.0002,
+      "loss": 0.1426,
+      "step": 86
+    },
+    {
+      "epoch": 0.30579964850615116,
+      "grad_norm": 0.1808985322713852,
+      "learning_rate": 0.0001995850622406639,
+      "loss": 0.1523,
+      "step": 87
+    },
+    {
+      "epoch": 0.3093145869947276,
+      "grad_norm": 0.25665614008903503,
+      "learning_rate": 0.0001991701244813278,
+      "loss": 0.1296,
+      "step": 88
+    },
+    {
+      "epoch": 0.31282952548330406,
+      "grad_norm": 0.1651581972837448,
+      "learning_rate": 0.0001987551867219917,
+      "loss": 0.1194,
+      "step": 89
+    },
+    {
+      "epoch": 0.3163444639718805,
+      "grad_norm": 0.19588658213615417,
+      "learning_rate": 0.00019834024896265561,
+      "loss": 0.167,
+      "step": 90
+    },
+    {
+      "epoch": 0.31985940246045697,
+      "grad_norm": 0.23157978057861328,
+      "learning_rate": 0.00019792531120331952,
+      "loss": 0.1484,
+      "step": 91
+    },
+    {
+      "epoch": 0.3233743409490334,
+      "grad_norm": 0.19277921319007874,
+      "learning_rate": 0.00019751037344398342,
+      "loss": 0.1622,
+      "step": 92
+    },
+    {
+      "epoch": 0.3268892794376098,
+      "grad_norm": 0.2064240425825119,
+      "learning_rate": 0.00019709543568464732,
+      "loss": 0.1529,
+      "step": 93
+    },
+    {
+      "epoch": 0.3304042179261863,
+      "grad_norm": 0.3631608784198761,
+      "learning_rate": 0.0001966804979253112,
+      "loss": 0.12,
+      "step": 94
+    },
+    {
+      "epoch": 0.3339191564147627,
+      "grad_norm": 0.3188731372356415,
+      "learning_rate": 0.0001962655601659751,
+      "loss": 0.1911,
+      "step": 95
+    },
+    {
+      "epoch": 0.3374340949033392,
+      "grad_norm": 0.18162627518177032,
+      "learning_rate": 0.000195850622406639,
+      "loss": 0.1048,
+      "step": 96
+    },
+    {
+      "epoch": 0.3409490333919156,
+      "grad_norm": 0.13265462219715118,
+      "learning_rate": 0.0001954356846473029,
+      "loss": 0.0898,
+      "step": 97
+    },
+    {
+      "epoch": 0.3444639718804921,
+      "grad_norm": 0.18490265309810638,
+      "learning_rate": 0.0001950207468879668,
+      "loss": 0.1274,
+      "step": 98
+    },
+    {
+      "epoch": 0.34797891036906853,
+      "grad_norm": 0.13423575460910797,
+      "learning_rate": 0.0001946058091286307,
+      "loss": 0.0941,
+      "step": 99
+    },
+    {
+      "epoch": 0.351493848857645,
+      "grad_norm": 0.16704542934894562,
+      "learning_rate": 0.00019419087136929463,
+      "loss": 0.1202,
+      "step": 100
+    },
+    {
+      "epoch": 0.351493848857645,
+      "eval_loss": 0.12555669248104095,
+      "eval_runtime": 178.7564,
+      "eval_samples_per_second": 2.831,
+      "eval_steps_per_second": 0.358,
+      "step": 100
+    },
+    {
+      "epoch": 0.35500878734622143,
+      "grad_norm": 0.16820575296878815,
+      "learning_rate": 0.00019377593360995853,
+      "loss": 0.107,
+      "step": 101
+    },
+    {
+      "epoch": 0.3585237258347979,
+      "grad_norm": 0.30647799372673035,
+      "learning_rate": 0.00019336099585062243,
+      "loss": 0.1739,
+      "step": 102
+    },
+    {
+      "epoch": 0.36203866432337434,
+      "grad_norm": 0.24310201406478882,
+      "learning_rate": 0.00019294605809128633,
+      "loss": 0.136,
+      "step": 103
+    },
+    {
+      "epoch": 0.3655536028119508,
+      "grad_norm": 0.22280918061733246,
+      "learning_rate": 0.00019253112033195023,
+      "loss": 0.1606,
+      "step": 104
+    },
+    {
+      "epoch": 0.36906854130052724,
+      "grad_norm": 0.17549653351306915,
+      "learning_rate": 0.00019211618257261413,
+      "loss": 0.0676,
+      "step": 105
+    },
+    {
+      "epoch": 0.37258347978910367,
+      "grad_norm": 0.15003140270709991,
+      "learning_rate": 0.00019170124481327803,
+      "loss": 0.1123,
+      "step": 106
+    },
+    {
+      "epoch": 0.37609841827768015,
+      "grad_norm": 0.17405907809734344,
+      "learning_rate": 0.00019128630705394194,
+      "loss": 0.1145,
+      "step": 107
+    },
+    {
+      "epoch": 0.37961335676625657,
+      "grad_norm": 0.22116738557815552,
+      "learning_rate": 0.00019087136929460584,
+      "loss": 0.2581,
+      "step": 108
+    },
+    {
+      "epoch": 0.38312829525483305,
+      "grad_norm": 0.17248253524303436,
+      "learning_rate": 0.0001904564315352697,
+      "loss": 0.116,
+      "step": 109
+    },
+    {
+      "epoch": 0.3866432337434095,
+      "grad_norm": 0.1622823178768158,
+      "learning_rate": 0.0001900414937759336,
+      "loss": 0.1112,
+      "step": 110
+    },
+    {
+      "epoch": 0.39015817223198596,
+      "grad_norm": 0.19119834899902344,
+      "learning_rate": 0.0001896265560165975,
+      "loss": 0.1545,
+      "step": 111
+    },
+    {
+      "epoch": 0.3936731107205624,
+      "grad_norm": 0.21934115886688232,
+      "learning_rate": 0.00018921161825726141,
+      "loss": 0.1259,
+      "step": 112
+    },
+    {
+      "epoch": 0.39718804920913886,
+      "grad_norm": 0.2224389910697937,
+      "learning_rate": 0.00018879668049792532,
+      "loss": 0.1125,
+      "step": 113
+    },
+    {
+      "epoch": 0.4007029876977153,
+      "grad_norm": 0.4230864346027374,
+      "learning_rate": 0.00018838174273858922,
+      "loss": 0.1372,
+      "step": 114
+    },
+    {
+      "epoch": 0.40421792618629176,
+      "grad_norm": 0.22001534700393677,
+      "learning_rate": 0.00018796680497925312,
+      "loss": 0.1052,
+      "step": 115
+    },
+    {
+      "epoch": 0.4077328646748682,
+      "grad_norm": 0.18826138973236084,
+      "learning_rate": 0.00018755186721991702,
+      "loss": 0.109,
+      "step": 116
+    },
+    {
+      "epoch": 0.4112478031634446,
+      "grad_norm": 0.18136151134967804,
+      "learning_rate": 0.00018713692946058092,
+      "loss": 0.1017,
+      "step": 117
+    },
+    {
+      "epoch": 0.4147627416520211,
+      "grad_norm": 0.1845865249633789,
+      "learning_rate": 0.00018672199170124482,
+      "loss": 0.1208,
+      "step": 118
+    },
+    {
+      "epoch": 0.4182776801405975,
+      "grad_norm": 0.2266140878200531,
+      "learning_rate": 0.00018630705394190872,
+      "loss": 0.1312,
+      "step": 119
+    },
+    {
+      "epoch": 0.421792618629174,
+      "grad_norm": 0.19567352533340454,
+      "learning_rate": 0.00018589211618257262,
+      "loss": 0.1188,
+      "step": 120
+    },
+    {
+      "epoch": 0.4253075571177504,
+      "grad_norm": 0.19902321696281433,
+      "learning_rate": 0.00018547717842323653,
+      "loss": 0.1031,
+      "step": 121
+    },
+    {
+      "epoch": 0.4288224956063269,
+      "grad_norm": 0.17255137860774994,
+      "learning_rate": 0.00018506224066390043,
+      "loss": 0.1095,
+      "step": 122
+    },
+    {
+      "epoch": 0.43233743409490333,
+      "grad_norm": 0.20506910979747772,
+      "learning_rate": 0.00018464730290456433,
+      "loss": 0.1176,
+      "step": 123
+    },
+    {
+      "epoch": 0.4358523725834798,
+      "grad_norm": 0.128911554813385,
+      "learning_rate": 0.0001842323651452282,
+      "loss": 0.0805,
+      "step": 124
+    },
+    {
+      "epoch": 0.43936731107205623,
+      "grad_norm": 0.21368758380413055,
+      "learning_rate": 0.0001838174273858921,
+      "loss": 0.1099,
+      "step": 125
+    },
+    {
+      "epoch": 0.4428822495606327,
+      "grad_norm": 0.19906243681907654,
+      "learning_rate": 0.000183402489626556,
+      "loss": 0.13,
+      "step": 126
+    },
+    {
+      "epoch": 0.44639718804920914,
+      "grad_norm": 0.2857668101787567,
+      "learning_rate": 0.0001829875518672199,
+      "loss": 0.1051,
+      "step": 127
+    },
+    {
+      "epoch": 0.44991212653778556,
+      "grad_norm": 0.18914097547531128,
+      "learning_rate": 0.00018257261410788383,
+      "loss": 0.1271,
+      "step": 128
+    },
+    {
+      "epoch": 0.45342706502636204,
+      "grad_norm": 0.17243094742298126,
+      "learning_rate": 0.00018215767634854774,
+      "loss": 0.087,
+      "step": 129
+    },
+    {
+      "epoch": 0.45694200351493847,
+      "grad_norm": 0.1896534413099289,
+      "learning_rate": 0.00018174273858921164,
+      "loss": 0.0635,
+      "step": 130
+    },
+    {
+      "epoch": 0.46045694200351495,
+      "grad_norm": 0.2188025712966919,
+      "learning_rate": 0.00018132780082987554,
+      "loss": 0.123,
+      "step": 131
+    },
+    {
+      "epoch": 0.46397188049209137,
+      "grad_norm": 0.14551542699337006,
+      "learning_rate": 0.00018091286307053944,
+      "loss": 0.0932,
+      "step": 132
+    },
+    {
+      "epoch": 0.46748681898066785,
+      "grad_norm": 0.17265719175338745,
+      "learning_rate": 0.00018049792531120334,
+      "loss": 0.0855,
+      "step": 133
+    },
+    {
+      "epoch": 0.4710017574692443,
+      "grad_norm": 0.15373249351978302,
+      "learning_rate": 0.00018008298755186724,
+      "loss": 0.0797,
+      "step": 134
+    },
+    {
+      "epoch": 0.47451669595782076,
+      "grad_norm": 0.2197037786245346,
+      "learning_rate": 0.00017966804979253114,
+      "loss": 0.1079,
+      "step": 135
+    },
+    {
+      "epoch": 0.4780316344463972,
+      "grad_norm": 0.13843557238578796,
+      "learning_rate": 0.00017925311203319504,
+      "loss": 0.0893,
+      "step": 136
+    },
+    {
+      "epoch": 0.48154657293497366,
+      "grad_norm": 0.1315803974866867,
+      "learning_rate": 0.00017883817427385895,
+      "loss": 0.0798,
+      "step": 137
+    },
+    {
+      "epoch": 0.4850615114235501,
+      "grad_norm": 0.16765527427196503,
+      "learning_rate": 0.00017842323651452285,
+      "loss": 0.1129,
+      "step": 138
+    },
+    {
+      "epoch": 0.48857644991212656,
+      "grad_norm": 0.1512288898229599,
+      "learning_rate": 0.00017800829875518672,
+      "loss": 0.1077,
+      "step": 139
+    },
+    {
+      "epoch": 0.492091388400703,
+      "grad_norm": 0.12558338046073914,
+      "learning_rate": 0.00017759336099585062,
+      "loss": 0.0757,
+      "step": 140
+    },
+    {
+      "epoch": 0.4956063268892794,
+      "grad_norm": 0.148188054561615,
+      "learning_rate": 0.00017717842323651452,
+      "loss": 0.0838,
+      "step": 141
+    },
+    {
+      "epoch": 0.4991212653778559,
+      "grad_norm": 0.11713378876447678,
+      "learning_rate": 0.00017676348547717842,
+      "loss": 0.0671,
+      "step": 142
+    },
+    {
+      "epoch": 0.5026362038664324,
+      "grad_norm": 0.1130850538611412,
+      "learning_rate": 0.00017634854771784233,
+      "loss": 0.0651,
+      "step": 143
+    },
+    {
+      "epoch": 0.5061511423550088,
+      "grad_norm": 0.2193579077720642,
+      "learning_rate": 0.00017593360995850623,
+      "loss": 0.1009,
+      "step": 144
+    },
+    {
+      "epoch": 0.5096660808435852,
+      "grad_norm": 0.16929981112480164,
+      "learning_rate": 0.00017551867219917013,
+      "loss": 0.0726,
+      "step": 145
+    },
+    {
+      "epoch": 0.5131810193321616,
+      "grad_norm": 0.250143438577652,
+      "learning_rate": 0.00017510373443983403,
+      "loss": 0.0946,
+      "step": 146
+    },
+    {
+      "epoch": 0.5166959578207382,
+      "grad_norm": 0.18434955179691315,
+      "learning_rate": 0.00017468879668049793,
+      "loss": 0.0928,
+      "step": 147
+    },
+    {
+      "epoch": 0.5202108963093146,
+      "grad_norm": 0.24621042609214783,
+      "learning_rate": 0.00017427385892116183,
+      "loss": 0.0915,
+      "step": 148
+    },
+    {
+      "epoch": 0.523725834797891,
+      "grad_norm": 0.14161203801631927,
+      "learning_rate": 0.00017385892116182573,
+      "loss": 0.0754,
+      "step": 149
+    },
+    {
+      "epoch": 0.5272407732864675,
+      "grad_norm": 0.22755476832389832,
+      "learning_rate": 0.00017344398340248963,
+      "loss": 0.1233,
+      "step": 150
+    },
+    {
+      "epoch": 0.5307557117750439,
+      "grad_norm": 0.14949020743370056,
+      "learning_rate": 0.00017302904564315354,
+      "loss": 0.102,
+      "step": 151
+    },
+    {
+      "epoch": 0.5342706502636204,
+      "grad_norm": 0.14558053016662598,
+      "learning_rate": 0.00017261410788381744,
+      "loss": 0.0649,
+      "step": 152
+    },
+    {
+      "epoch": 0.5377855887521968,
+      "grad_norm": 0.13403621315956116,
+      "learning_rate": 0.00017219917012448134,
+      "loss": 0.0769,
+      "step": 153
+    },
+    {
+      "epoch": 0.5413005272407733,
+      "grad_norm": 0.1492697149515152,
+      "learning_rate": 0.0001717842323651452,
+      "loss": 0.0695,
+      "step": 154
+    },
+    {
+      "epoch": 0.5448154657293497,
+      "grad_norm": 0.176810160279274,
+      "learning_rate": 0.00017136929460580914,
+      "loss": 0.0914,
+      "step": 155
+    },
+    {
+      "epoch": 0.5483304042179262,
+      "grad_norm": 0.1693807691335678,
+      "learning_rate": 0.00017095435684647304,
+      "loss": 0.1186,
+      "step": 156
+    },
+    {
+      "epoch": 0.5518453427065027,
+      "grad_norm": 0.16816498339176178,
+      "learning_rate": 0.00017053941908713694,
+      "loss": 0.0891,
+      "step": 157
+    },
+    {
+      "epoch": 0.5553602811950791,
+      "grad_norm": 0.1630387306213379,
+      "learning_rate": 0.00017012448132780084,
+      "loss": 0.0817,
+      "step": 158
+    },
+    {
+      "epoch": 0.5588752196836555,
+      "grad_norm": 0.2269333004951477,
+      "learning_rate": 0.00016970954356846475,
+      "loss": 0.1311,
+      "step": 159
+    },
+    {
+      "epoch": 0.562390158172232,
+      "grad_norm": 0.16005942225456238,
+      "learning_rate": 0.00016929460580912865,
+      "loss": 0.077,
+      "step": 160
+    },
+    {
+      "epoch": 0.5659050966608085,
+      "grad_norm": 0.22128961980342865,
+      "learning_rate": 0.00016887966804979255,
+      "loss": 0.0663,
+      "step": 161
+    },
+    {
+      "epoch": 0.5694200351493849,
+      "grad_norm": 0.24515265226364136,
+      "learning_rate": 0.00016846473029045645,
+      "loss": 0.0891,
+      "step": 162
+    },
+    {
+      "epoch": 0.5729349736379613,
+      "grad_norm": 0.15797489881515503,
+      "learning_rate": 0.00016804979253112035,
+      "loss": 0.0604,
+      "step": 163
+    },
+    {
+      "epoch": 0.5764499121265377,
+      "grad_norm": 0.11231053620576859,
+      "learning_rate": 0.00016763485477178425,
+      "loss": 0.0637,
+      "step": 164
+    },
+    {
+      "epoch": 0.5799648506151143,
+      "grad_norm": 0.1319078505039215,
+      "learning_rate": 0.00016721991701244815,
+      "loss": 0.061,
+      "step": 165
+    },
+    {
+      "epoch": 0.5834797891036907,
+      "grad_norm": 0.15379250049591064,
+      "learning_rate": 0.00016680497925311205,
+      "loss": 0.0727,
+      "step": 166
+    },
+    {
+      "epoch": 0.5869947275922671,
+      "grad_norm": 0.12770119309425354,
+      "learning_rate": 0.00016639004149377596,
+      "loss": 0.0749,
+      "step": 167
+    },
+    {
+      "epoch": 0.5905096660808435,
+      "grad_norm": 0.17939890921115875,
+      "learning_rate": 0.00016597510373443986,
+      "loss": 0.0904,
+      "step": 168
+    },
+    {
+      "epoch": 0.5940246045694201,
+      "grad_norm": 0.15956297516822815,
+      "learning_rate": 0.00016556016597510373,
+      "loss": 0.0892,
+      "step": 169
+    },
+    {
+      "epoch": 0.5975395430579965,
+      "grad_norm": 0.17730525135993958,
+      "learning_rate": 0.00016514522821576763,
+      "loss": 0.0783,
+      "step": 170
+    },
+    {
+      "epoch": 0.6010544815465729,
+      "grad_norm": 0.32799553871154785,
+      "learning_rate": 0.00016473029045643153,
+      "loss": 0.082,
+      "step": 171
+    },
+    {
+      "epoch": 0.6045694200351494,
+      "grad_norm": 0.1100563257932663,
+      "learning_rate": 0.00016431535269709543,
+      "loss": 0.0633,
+      "step": 172
+    },
+    {
+      "epoch": 0.6080843585237259,
+      "grad_norm": 0.12468238919973373,
+      "learning_rate": 0.00016390041493775934,
+      "loss": 0.0506,
+      "step": 173
+    },
+    {
+      "epoch": 0.6115992970123023,
+      "grad_norm": 0.11538554728031158,
+      "learning_rate": 0.00016348547717842324,
+      "loss": 0.0614,
+      "step": 174
+    },
+    {
+      "epoch": 0.6151142355008787,
+      "grad_norm": 0.15802019834518433,
+      "learning_rate": 0.00016307053941908714,
+      "loss": 0.0806,
+      "step": 175
+    },
+    {
+      "epoch": 0.6186291739894552,
+      "grad_norm": 0.17224474251270294,
+      "learning_rate": 0.00016265560165975104,
+      "loss": 0.0588,
+      "step": 176
+    },
+    {
+      "epoch": 0.6221441124780316,
+      "grad_norm": 0.2495221644639969,
+      "learning_rate": 0.00016224066390041494,
+      "loss": 0.1109,
+      "step": 177
+    },
+    {
+      "epoch": 0.6256590509666081,
+      "grad_norm": 0.1328081488609314,
+      "learning_rate": 0.00016182572614107884,
+      "loss": 0.0572,
+      "step": 178
+    },
+    {
+      "epoch": 0.6291739894551845,
+      "grad_norm": 0.18458999693393707,
+      "learning_rate": 0.00016141078838174274,
+      "loss": 0.0975,
+      "step": 179
+    },
+    {
+      "epoch": 0.632688927943761,
+      "grad_norm": 0.1648002564907074,
+      "learning_rate": 0.00016099585062240664,
+      "loss": 0.065,
+      "step": 180
+    },
+    {
+      "epoch": 0.6362038664323374,
+      "grad_norm": 0.20207300782203674,
+      "learning_rate": 0.00016058091286307055,
+      "loss": 0.1027,
+      "step": 181
+    },
+    {
+      "epoch": 0.6397188049209139,
+      "grad_norm": 0.17972172796726227,
+      "learning_rate": 0.00016016597510373445,
+      "loss": 0.1069,
+      "step": 182
+    },
+    {
+      "epoch": 0.6432337434094904,
+      "grad_norm": 0.15856166183948517,
+      "learning_rate": 0.00015975103734439835,
+      "loss": 0.0948,
+      "step": 183
+    },
+    {
+      "epoch": 0.6467486818980668,
+      "grad_norm": 0.21730534732341766,
+      "learning_rate": 0.00015933609958506225,
+      "loss": 0.1264,
+      "step": 184
+    },
+    {
+      "epoch": 0.6502636203866432,
+      "grad_norm": 0.10988626629114151,
+      "learning_rate": 0.00015892116182572615,
+      "loss": 0.0439,
+      "step": 185
+    },
+    {
+      "epoch": 0.6537785588752196,
+      "grad_norm": 0.17182111740112305,
+      "learning_rate": 0.00015850622406639005,
+      "loss": 0.1122,
+      "step": 186
+    },
+    {
+      "epoch": 0.6572934973637962,
+      "grad_norm": 0.13410013914108276,
+      "learning_rate": 0.00015809128630705395,
+      "loss": 0.0551,
+      "step": 187
+    },
+    {
+      "epoch": 0.6608084358523726,
+      "grad_norm": 0.11806394904851913,
+      "learning_rate": 0.00015767634854771785,
+      "loss": 0.0782,
+      "step": 188
+    },
+    {
+      "epoch": 0.664323374340949,
+      "grad_norm": 0.18627063930034637,
+      "learning_rate": 0.00015726141078838176,
+      "loss": 0.0787,
+      "step": 189
+    },
+    {
+      "epoch": 0.6678383128295254,
+      "grad_norm": 0.26391828060150146,
+      "learning_rate": 0.00015684647302904566,
+      "loss": 0.0614,
+      "step": 190
+    },
+    {
+      "epoch": 0.671353251318102,
+      "grad_norm": 0.1657516062259674,
+      "learning_rate": 0.00015643153526970956,
+      "loss": 0.0798,
+      "step": 191
+    },
+    {
+      "epoch": 0.6748681898066784,
+      "grad_norm": 0.15479570627212524,
+      "learning_rate": 0.00015601659751037346,
+      "loss": 0.0811,
+      "step": 192
+    },
+    {
+      "epoch": 0.6783831282952548,
+      "grad_norm": 0.16939130425453186,
+      "learning_rate": 0.00015560165975103736,
+      "loss": 0.0749,
+      "step": 193
+    },
+    {
+      "epoch": 0.6818980667838312,
+      "grad_norm": 0.17529913783073425,
+      "learning_rate": 0.00015518672199170126,
+      "loss": 0.0517,
+      "step": 194
+    },
+    {
+      "epoch": 0.6854130052724078,
+      "grad_norm": 0.1807660162448883,
+      "learning_rate": 0.00015477178423236516,
+      "loss": 0.0928,
+      "step": 195
+    },
+    {
+      "epoch": 0.6889279437609842,
+      "grad_norm": 0.13066497445106506,
+      "learning_rate": 0.00015435684647302906,
+      "loss": 0.0525,
+      "step": 196
+    },
+    {
+      "epoch": 0.6924428822495606,
+      "grad_norm": 0.17132331430912018,
+      "learning_rate": 0.00015394190871369297,
+      "loss": 0.1139,
+      "step": 197
+    },
+    {
+      "epoch": 0.6959578207381371,
+      "grad_norm": 0.1522899866104126,
+      "learning_rate": 0.00015352697095435687,
+      "loss": 0.0818,
+      "step": 198
+    },
+    {
+      "epoch": 0.6994727592267135,
+      "grad_norm": 0.18056978285312653,
+      "learning_rate": 0.00015311203319502074,
+      "loss": 0.0698,
+      "step": 199
+    },
+    {
+      "epoch": 0.70298769771529,
+      "grad_norm": 0.1712590903043747,
+      "learning_rate": 0.00015269709543568464,
+      "loss": 0.0743,
+      "step": 200
+    },
+    {
+      "epoch": 0.70298769771529,
+      "eval_loss": 0.07312215119600296,
+      "eval_runtime": 178.6909,
+      "eval_samples_per_second": 2.832,
+      "eval_steps_per_second": 0.358,
+      "step": 200
+    },
+    {
+      "epoch": 0.7065026362038664,
+      "grad_norm": 0.14959929883480072,
+      "learning_rate": 0.00015228215767634854,
+      "loss": 0.0838,
+      "step": 201
+    },
+    {
+      "epoch": 0.7100175746924429,
+      "grad_norm": 0.18013989925384521,
+      "learning_rate": 0.00015186721991701244,
+      "loss": 0.0768,
+      "step": 202
+    },
+    {
+      "epoch": 0.7135325131810193,
+      "grad_norm": 0.1600325107574463,
+      "learning_rate": 0.00015145228215767635,
+      "loss": 0.0934,
+      "step": 203
+    },
+    {
+      "epoch": 0.7170474516695958,
+      "grad_norm": 0.1611802875995636,
+      "learning_rate": 0.00015103734439834025,
+      "loss": 0.0675,
+      "step": 204
+    },
+    {
+      "epoch": 0.7205623901581723,
+      "grad_norm": 0.1379013955593109,
+      "learning_rate": 0.00015062240663900415,
+      "loss": 0.0883,
+      "step": 205
+    },
+    {
+      "epoch": 0.7240773286467487,
+      "grad_norm": 0.1827850192785263,
+      "learning_rate": 0.00015020746887966805,
+      "loss": 0.0686,
+      "step": 206
+    },
+    {
+      "epoch": 0.7275922671353251,
+      "grad_norm": 0.14171870052814484,
+      "learning_rate": 0.00014979253112033195,
+      "loss": 0.0624,
+      "step": 207
+    },
+    {
+      "epoch": 0.7311072056239016,
+      "grad_norm": 0.15266627073287964,
+      "learning_rate": 0.00014937759336099585,
+      "loss": 0.0797,
+      "step": 208
+    },
+    {
+      "epoch": 0.7346221441124781,
+      "grad_norm": 0.14795458316802979,
+      "learning_rate": 0.00014896265560165975,
+      "loss": 0.0713,
+      "step": 209
+    },
+    {
+      "epoch": 0.7381370826010545,
+      "grad_norm": 0.12821845710277557,
+      "learning_rate": 0.00014854771784232365,
+      "loss": 0.0592,
+      "step": 210
+    },
+    {
+      "epoch": 0.7416520210896309,
+      "grad_norm": 0.12268483638763428,
+      "learning_rate": 0.00014813278008298756,
+      "loss": 0.0484,
+      "step": 211
+    },
+    {
+      "epoch": 0.7451669595782073,
+      "grad_norm": 0.12236201018095016,
+      "learning_rate": 0.00014771784232365146,
+      "loss": 0.0642,
+      "step": 212
+    },
+    {
+      "epoch": 0.7486818980667839,
+      "grad_norm": 0.1301528811454773,
+      "learning_rate": 0.00014730290456431536,
+      "loss": 0.0696,
+      "step": 213
+    },
+    {
+      "epoch": 0.7521968365553603,
+      "grad_norm": 0.1912907063961029,
+      "learning_rate": 0.00014688796680497926,
+      "loss": 0.1043,
+      "step": 214
+    },
+    {
+      "epoch": 0.7557117750439367,
+      "grad_norm": 0.11695796251296997,
+      "learning_rate": 0.00014647302904564316,
+      "loss": 0.0463,
+      "step": 215
+    },
+    {
+      "epoch": 0.7592267135325131,
+      "grad_norm": 0.18601001799106598,
+      "learning_rate": 0.00014605809128630706,
+      "loss": 0.0643,
+      "step": 216
+    },
+    {
+      "epoch": 0.7627416520210897,
+      "grad_norm": 0.1472391039133072,
+      "learning_rate": 0.00014564315352697096,
+      "loss": 0.0691,
+      "step": 217
+    },
+    {
+      "epoch": 0.7662565905096661,
+      "grad_norm": 0.1806488037109375,
+      "learning_rate": 0.00014522821576763486,
+      "loss": 0.0429,
+      "step": 218
+    },
+    {
+      "epoch": 0.7697715289982425,
+      "grad_norm": 0.11694241315126419,
+      "learning_rate": 0.00014481327800829877,
+      "loss": 0.0604,
+      "step": 219
+    },
+    {
+      "epoch": 0.773286467486819,
+      "grad_norm": 0.21174874901771545,
+      "learning_rate": 0.00014439834024896267,
+      "loss": 0.0691,
+      "step": 220
+    },
+    {
+      "epoch": 0.7768014059753954,
+      "grad_norm": 0.16953590512275696,
+      "learning_rate": 0.00014398340248962657,
+      "loss": 0.0712,
+      "step": 221
+    },
+    {
+      "epoch": 0.7803163444639719,
+      "grad_norm": 0.14451764523983002,
+      "learning_rate": 0.00014356846473029047,
+      "loss": 0.0423,
+      "step": 222
+    },
+    {
+      "epoch": 0.7838312829525483,
+      "grad_norm": 0.20399795472621918,
+      "learning_rate": 0.00014315352697095437,
+      "loss": 0.0685,
+      "step": 223
+    },
+    {
+      "epoch": 0.7873462214411248,
+      "grad_norm": 0.14048655331134796,
+      "learning_rate": 0.00014273858921161827,
+      "loss": 0.0675,
+      "step": 224
+    },
+    {
+      "epoch": 0.7908611599297012,
+      "grad_norm": 0.2338792383670807,
+      "learning_rate": 0.00014232365145228217,
+      "loss": 0.063,
+      "step": 225
+    },
+    {
+      "epoch": 0.7943760984182777,
+      "grad_norm": 0.14961326122283936,
+      "learning_rate": 0.00014190871369294607,
+      "loss": 0.0651,
+      "step": 226
+    },
+    {
+      "epoch": 0.7978910369068541,
+      "grad_norm": 0.15257729589939117,
+      "learning_rate": 0.00014149377593360998,
+      "loss": 0.0428,
+      "step": 227
+    },
+    {
+      "epoch": 0.8014059753954306,
+      "grad_norm": 0.24161341786384583,
+      "learning_rate": 0.00014107883817427388,
+      "loss": 0.0764,
+      "step": 228
+    },
+    {
+      "epoch": 0.804920913884007,
+      "grad_norm": 0.1564214676618576,
+      "learning_rate": 0.00014066390041493778,
+      "loss": 0.0528,
+      "step": 229
+    },
+    {
+      "epoch": 0.8084358523725835,
+      "grad_norm": 0.184456467628479,
+      "learning_rate": 0.00014024896265560165,
+      "loss": 0.0553,
+      "step": 230
+    },
+    {
+      "epoch": 0.81195079086116,
+      "grad_norm": 0.1617126762866974,
+      "learning_rate": 0.00013983402489626555,
+      "loss": 0.0632,
+      "step": 231
+    },
+    {
+      "epoch": 0.8154657293497364,
+      "grad_norm": 0.13109862804412842,
+      "learning_rate": 0.00013941908713692945,
+      "loss": 0.0475,
+      "step": 232
+    },
+    {
+      "epoch": 0.8189806678383128,
+      "grad_norm": 0.13069158792495728,
+      "learning_rate": 0.00013900414937759336,
+      "loss": 0.0388,
+      "step": 233
+    },
+    {
+      "epoch": 0.8224956063268892,
+      "grad_norm": 0.13061407208442688,
+      "learning_rate": 0.00013858921161825726,
+      "loss": 0.0472,
+      "step": 234
+    },
+    {
+      "epoch": 0.8260105448154658,
+      "grad_norm": 0.27340853214263916,
+      "learning_rate": 0.00013817427385892116,
+      "loss": 0.1715,
+      "step": 235
+    },
+    {
+      "epoch": 0.8295254833040422,
+      "grad_norm": 0.11120892316102982,
+      "learning_rate": 0.00013775933609958506,
+      "loss": 0.0393,
+      "step": 236
+    },
+    {
+      "epoch": 0.8330404217926186,
+      "grad_norm": 0.1893947571516037,
+      "learning_rate": 0.00013734439834024896,
+      "loss": 0.0897,
+      "step": 237
+    },
+    {
+      "epoch": 0.836555360281195,
+      "grad_norm": 0.11574364453554153,
+      "learning_rate": 0.00013692946058091286,
+      "loss": 0.053,
+      "step": 238
+    },
+    {
+      "epoch": 0.8400702987697716,
+      "grad_norm": 0.13650314509868622,
+      "learning_rate": 0.00013651452282157676,
+      "loss": 0.0718,
+      "step": 239
+    },
+    {
+      "epoch": 0.843585237258348,
+      "grad_norm": 0.19434796273708344,
+      "learning_rate": 0.00013609958506224066,
+      "loss": 0.0606,
+      "step": 240
+    },
+    {
+      "epoch": 0.8471001757469244,
+      "grad_norm": 0.17645402252674103,
+      "learning_rate": 0.00013568464730290457,
+      "loss": 0.0995,
+      "step": 241
+    },
+    {
+      "epoch": 0.8506151142355008,
+      "grad_norm": 0.13868428766727448,
+      "learning_rate": 0.00013526970954356847,
+      "loss": 0.0768,
+      "step": 242
+    },
+    {
+      "epoch": 0.8541300527240774,
+      "grad_norm": 0.1283743679523468,
+      "learning_rate": 0.00013485477178423237,
+      "loss": 0.0587,
+      "step": 243
+    },
+    {
+      "epoch": 0.8576449912126538,
+      "grad_norm": 0.1996167004108429,
+      "learning_rate": 0.00013443983402489627,
+      "loss": 0.0562,
+      "step": 244
+    },
+    {
+      "epoch": 0.8611599297012302,
+      "grad_norm": 0.16913233697414398,
+      "learning_rate": 0.00013402489626556017,
+      "loss": 0.1163,
+      "step": 245
+    },
+    {
+      "epoch": 0.8646748681898067,
+      "grad_norm": 0.18035849928855896,
+      "learning_rate": 0.00013360995850622407,
+      "loss": 0.063,
+      "step": 246
+    },
+    {
+      "epoch": 0.8681898066783831,
+      "grad_norm": 0.13519661128520966,
+      "learning_rate": 0.00013319502074688797,
+      "loss": 0.082,
+      "step": 247
+    },
+    {
+      "epoch": 0.8717047451669596,
+      "grad_norm": 0.12751896679401398,
+      "learning_rate": 0.00013278008298755187,
+      "loss": 0.0594,
+      "step": 248
+    },
+    {
+      "epoch": 0.875219683655536,
+      "grad_norm": 0.10623247176408768,
+      "learning_rate": 0.00013236514522821578,
+      "loss": 0.0635,
+      "step": 249
+    },
+    {
+      "epoch": 0.8787346221441125,
+      "grad_norm": 0.16478785872459412,
+      "learning_rate": 0.00013195020746887968,
+      "loss": 0.0646,
+      "step": 250
+    },
+    {
+      "epoch": 0.8822495606326889,
+      "grad_norm": 0.14821447432041168,
+      "learning_rate": 0.00013153526970954358,
+      "loss": 0.0574,
+      "step": 251
+    },
+    {
+      "epoch": 0.8857644991212654,
+      "grad_norm": 0.14830711483955383,
+      "learning_rate": 0.00013112033195020748,
+      "loss": 0.0569,
+      "step": 252
+    },
+    {
+      "epoch": 0.8892794376098418,
+      "grad_norm": 0.15182024240493774,
+      "learning_rate": 0.00013070539419087138,
+      "loss": 0.0486,
+      "step": 253
+    },
+    {
+      "epoch": 0.8927943760984183,
+      "grad_norm": 0.13804112374782562,
+      "learning_rate": 0.00013029045643153528,
+      "loss": 0.0341,
+      "step": 254
+    },
+    {
+      "epoch": 0.8963093145869947,
+      "grad_norm": 0.23451949656009674,
+      "learning_rate": 0.00012987551867219918,
+      "loss": 0.104,
+      "step": 255
+    },
+    {
+      "epoch": 0.8998242530755711,
+      "grad_norm": 0.19352230429649353,
+      "learning_rate": 0.00012946058091286308,
+      "loss": 0.1153,
+      "step": 256
+    },
+    {
+      "epoch": 0.9033391915641477,
+      "grad_norm": 0.1840514838695526,
+      "learning_rate": 0.00012904564315352699,
+      "loss": 0.0661,
+      "step": 257
+    },
+    {
+      "epoch": 0.9068541300527241,
+      "grad_norm": 0.17361341416835785,
+      "learning_rate": 0.0001286307053941909,
+      "loss": 0.0473,
+      "step": 258
+    },
+    {
+      "epoch": 0.9103690685413005,
+      "grad_norm": 0.24270959198474884,
+      "learning_rate": 0.0001282157676348548,
+      "loss": 0.0578,
+      "step": 259
+    },
+    {
+      "epoch": 0.9138840070298769,
+      "grad_norm": 0.2252267301082611,
+      "learning_rate": 0.00012780082987551866,
+      "loss": 0.0804,
+      "step": 260
+    },
+    {
+      "epoch": 0.9173989455184535,
+      "grad_norm": 0.13661521673202515,
+      "learning_rate": 0.00012738589211618256,
+      "loss": 0.0433,
+      "step": 261
+    },
+    {
+      "epoch": 0.9209138840070299,
+      "grad_norm": 0.14729367196559906,
+      "learning_rate": 0.00012697095435684646,
+      "loss": 0.0506,
+      "step": 262
+    },
+    {
+      "epoch": 0.9244288224956063,
+      "grad_norm": 0.12483599036931992,
+      "learning_rate": 0.00012655601659751037,
+      "loss": 0.0361,
+      "step": 263
+    },
+    {
+      "epoch": 0.9279437609841827,
+      "grad_norm": 0.14307105541229248,
+      "learning_rate": 0.00012614107883817427,
+      "loss": 0.0423,
+      "step": 264
+    },
+    {
+      "epoch": 0.9314586994727593,
+      "grad_norm": 0.18808116018772125,
+      "learning_rate": 0.00012572614107883817,
+      "loss": 0.0585,
+      "step": 265
+    },
+    {
+      "epoch": 0.9349736379613357,
+      "grad_norm": 0.37760576605796814,
+      "learning_rate": 0.00012531120331950207,
+      "loss": 0.0689,
+      "step": 266
+    },
+    {
+      "epoch": 0.9384885764499121,
+      "grad_norm": 0.12569907307624817,
+      "learning_rate": 0.00012489626556016597,
+      "loss": 0.0376,
+      "step": 267
+    },
+    {
+      "epoch": 0.9420035149384886,
+      "grad_norm": 0.13358008861541748,
+      "learning_rate": 0.00012448132780082987,
+      "loss": 0.045,
+      "step": 268
+    },
+    {
+      "epoch": 0.945518453427065,
+      "grad_norm": 0.2513868808746338,
+      "learning_rate": 0.00012406639004149377,
+      "loss": 0.0848,
+      "step": 269
+    },
+    {
+      "epoch": 0.9490333919156415,
+      "grad_norm": 0.13180561363697052,
+      "learning_rate": 0.00012365145228215767,
+      "loss": 0.068,
+      "step": 270
+    },
+    {
+      "epoch": 0.9525483304042179,
+      "grad_norm": 0.18591004610061646,
+      "learning_rate": 0.00012323651452282158,
+      "loss": 0.0657,
+      "step": 271
+    },
+    {
+      "epoch": 0.9560632688927944,
+      "grad_norm": 0.14035774767398834,
+      "learning_rate": 0.00012282157676348548,
+      "loss": 0.0483,
+      "step": 272
+    },
+    {
+      "epoch": 0.9595782073813708,
+      "grad_norm": 0.160274475812912,
+      "learning_rate": 0.0001224066390041494,
+      "loss": 0.0361,
+      "step": 273
+    },
+    {
+      "epoch": 0.9630931458699473,
+      "grad_norm": 0.11012961715459824,
+      "learning_rate": 0.0001219917012448133,
+      "loss": 0.0394,
+      "step": 274
+    },
+    {
+      "epoch": 0.9666080843585237,
+      "grad_norm": 0.14294235408306122,
+      "learning_rate": 0.00012157676348547717,
+      "loss": 0.0677,
+      "step": 275
+    },
+    {
+      "epoch": 0.9701230228471002,
+      "grad_norm": 0.18198156356811523,
+      "learning_rate": 0.00012116182572614108,
+      "loss": 0.0617,
+      "step": 276
+    },
+    {
+      "epoch": 0.9736379613356766,
+      "grad_norm": 0.13320007920265198,
+      "learning_rate": 0.00012074688796680498,
+      "loss": 0.0432,
+      "step": 277
+    },
+    {
+      "epoch": 0.9771528998242531,
+      "grad_norm": 0.1330510377883911,
+      "learning_rate": 0.00012033195020746888,
+      "loss": 0.0443,
+      "step": 278
+    },
+    {
+      "epoch": 0.9806678383128296,
+      "grad_norm": 0.16604888439178467,
+      "learning_rate": 0.00011991701244813279,
+      "loss": 0.0614,
+      "step": 279
+    },
+    {
+      "epoch": 0.984182776801406,
+      "grad_norm": 0.13113927841186523,
+      "learning_rate": 0.00011950207468879669,
+      "loss": 0.0593,
+      "step": 280
+    },
+    {
+      "epoch": 0.9876977152899824,
+      "grad_norm": 0.14142243564128876,
+      "learning_rate": 0.00011908713692946059,
+      "loss": 0.0473,
+      "step": 281
+    },
+    {
+      "epoch": 0.9912126537785588,
+      "grad_norm": 0.1621742993593216,
+      "learning_rate": 0.00011867219917012449,
+      "loss": 0.0555,
+      "step": 282
+    },
+    {
+      "epoch": 0.9947275922671354,
+      "grad_norm": 0.16650210320949554,
+      "learning_rate": 0.00011825726141078839,
+      "loss": 0.0678,
+      "step": 283
+    },
+    {
+      "epoch": 0.9982425307557118,
+      "grad_norm": 0.1241079717874527,
+      "learning_rate": 0.00011784232365145229,
+      "loss": 0.0773,
+      "step": 284
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.5624362826347351,
+      "learning_rate": 0.0001174273858921162,
+      "loss": 0.0893,
+      "step": 285
+    },
+    {
+      "epoch": 1.0035149384885764,
+      "grad_norm": 0.1310419738292694,
+      "learning_rate": 0.0001170124481327801,
+      "loss": 0.0323,
+      "step": 286
+    },
+    {
+      "epoch": 1.0070298769771528,
+      "grad_norm": 0.2076292186975479,
+      "learning_rate": 0.000116597510373444,
+      "loss": 0.0864,
+      "step": 287
+    },
+    {
+      "epoch": 1.0105448154657293,
+      "grad_norm": 0.16203393042087555,
+      "learning_rate": 0.0001161825726141079,
+      "loss": 0.0411,
+      "step": 288
+    },
+    {
+      "epoch": 1.0140597539543057,
+      "grad_norm": 0.18476437032222748,
+      "learning_rate": 0.0001157676348547718,
+      "loss": 0.0596,
+      "step": 289
+    },
+    {
+      "epoch": 1.0175746924428823,
+      "grad_norm": 0.1175275593996048,
+      "learning_rate": 0.00011535269709543569,
+      "loss": 0.0335,
+      "step": 290
+    },
+    {
+      "epoch": 1.0210896309314588,
+      "grad_norm": 0.14564698934555054,
+      "learning_rate": 0.00011493775933609959,
+      "loss": 0.0372,
+      "step": 291
+    },
+    {
+      "epoch": 1.0246045694200352,
+      "grad_norm": 0.11464350670576096,
+      "learning_rate": 0.00011452282157676349,
+      "loss": 0.0361,
+      "step": 292
+    },
+    {
+      "epoch": 1.0281195079086116,
+      "grad_norm": 0.17061354219913483,
+      "learning_rate": 0.00011410788381742739,
+      "loss": 0.0319,
+      "step": 293
+    },
+    {
+      "epoch": 1.031634446397188,
+      "grad_norm": 0.12375995516777039,
+      "learning_rate": 0.00011369294605809129,
+      "loss": 0.0295,
+      "step": 294
+    },
+    {
+      "epoch": 1.0351493848857645,
+      "grad_norm": 0.1310289353132248,
+      "learning_rate": 0.00011327800829875519,
+      "loss": 0.0335,
+      "step": 295
+    },
+    {
+      "epoch": 1.038664323374341,
+      "grad_norm": 0.11135981231927872,
+      "learning_rate": 0.0001128630705394191,
+      "loss": 0.0249,
+      "step": 296
+    },
+    {
+      "epoch": 1.0421792618629173,
+      "grad_norm": 0.15976910293102264,
+      "learning_rate": 0.000112448132780083,
+      "loss": 0.0341,
+      "step": 297
+    },
+    {
+      "epoch": 1.0456942003514937,
+      "grad_norm": 0.13744553923606873,
+      "learning_rate": 0.0001120331950207469,
+      "loss": 0.0314,
+      "step": 298
+    },
+    {
+      "epoch": 1.0492091388400704,
+      "grad_norm": 0.13137714564800262,
+      "learning_rate": 0.0001116182572614108,
+      "loss": 0.0337,
+      "step": 299
+    },
+    {
+      "epoch": 1.0527240773286468,
+      "grad_norm": 0.16147193312644958,
+      "learning_rate": 0.0001112033195020747,
+      "loss": 0.0437,
+      "step": 300
+    },
+    {
+      "epoch": 1.0527240773286468,
+      "eval_loss": 0.05115514248609543,
+      "eval_runtime": 178.6438,
+      "eval_samples_per_second": 2.832,
+      "eval_steps_per_second": 0.358,
+      "step": 300
+    },
+    {
+      "epoch": 1.0562390158172232,
+      "grad_norm": 0.12608520686626434,
+      "learning_rate": 0.0001107883817427386,
+      "loss": 0.0458,
+      "step": 301
+    },
+    {
+      "epoch": 1.0597539543057997,
+      "grad_norm": 0.1479218602180481,
+      "learning_rate": 0.0001103734439834025,
+      "loss": 0.0324,
+      "step": 302
+    },
+    {
+      "epoch": 1.063268892794376,
+      "grad_norm": 0.11473022401332855,
+      "learning_rate": 0.0001099585062240664,
+      "loss": 0.0313,
+      "step": 303
+    },
+    {
+      "epoch": 1.0667838312829525,
+      "grad_norm": 0.11279737949371338,
+      "learning_rate": 0.0001095435684647303,
+      "loss": 0.0334,
+      "step": 304
+    },
+    {
+      "epoch": 1.070298769771529,
+      "grad_norm": 0.11454727500677109,
+      "learning_rate": 0.00010912863070539419,
+      "loss": 0.0346,
+      "step": 305
+    },
+    {
+      "epoch": 1.0738137082601054,
+      "grad_norm": 0.16063125431537628,
+      "learning_rate": 0.00010871369294605809,
+      "loss": 0.0445,
+      "step": 306
+    },
+    {
+      "epoch": 1.0773286467486818,
+      "grad_norm": 0.08446641266345978,
+      "learning_rate": 0.000108298755186722,
+      "loss": 0.0233,
+      "step": 307
+    },
+    {
+      "epoch": 1.0808435852372584,
+      "grad_norm": 0.12478985637426376,
+      "learning_rate": 0.0001078838174273859,
+      "loss": 0.0454,
+      "step": 308
+    },
+    {
+      "epoch": 1.0843585237258349,
+      "grad_norm": 0.10559240728616714,
+      "learning_rate": 0.0001074688796680498,
+      "loss": 0.0286,
+      "step": 309
+    },
+    {
+      "epoch": 1.0878734622144113,
+      "grad_norm": 0.08633790910243988,
+      "learning_rate": 0.0001070539419087137,
+      "loss": 0.0286,
+      "step": 310
+    },
+    {
+      "epoch": 1.0913884007029877,
+      "grad_norm": 0.1345328986644745,
+      "learning_rate": 0.0001066390041493776,
+      "loss": 0.0387,
+      "step": 311
+    },
+    {
+      "epoch": 1.0949033391915641,
+      "grad_norm": 0.1071149930357933,
+      "learning_rate": 0.0001062240663900415,
+      "loss": 0.03,
+      "step": 312
+    },
+    {
+      "epoch": 1.0984182776801406,
+      "grad_norm": 0.1703241467475891,
+      "learning_rate": 0.0001058091286307054,
+      "loss": 0.0527,
+      "step": 313
+    },
+    {
+      "epoch": 1.101933216168717,
+      "grad_norm": 0.09247270226478577,
+      "learning_rate": 0.0001053941908713693,
+      "loss": 0.0323,
+      "step": 314
+    },
+    {
+      "epoch": 1.1054481546572934,
+      "grad_norm": 0.15563809871673584,
+      "learning_rate": 0.0001049792531120332,
+      "loss": 0.0377,
+      "step": 315
+    },
+    {
+      "epoch": 1.10896309314587,
+      "grad_norm": 0.09453997761011124,
+      "learning_rate": 0.0001045643153526971,
+      "loss": 0.0245,
+      "step": 316
+    },
+    {
+      "epoch": 1.1124780316344465,
+      "grad_norm": 0.10154609382152557,
+      "learning_rate": 0.000104149377593361,
+      "loss": 0.0366,
+      "step": 317
+    },
+    {
+      "epoch": 1.115992970123023,
+      "grad_norm": 0.09299053996801376,
+      "learning_rate": 0.00010373443983402491,
+      "loss": 0.0255,
+      "step": 318
+    },
+    {
+      "epoch": 1.1195079086115993,
+      "grad_norm": 0.11212610453367233,
+      "learning_rate": 0.00010331950207468881,
+      "loss": 0.0244,
+      "step": 319
+    },
+    {
+      "epoch": 1.1230228471001757,
+      "grad_norm": 0.08826769888401031,
+      "learning_rate": 0.0001029045643153527,
+      "loss": 0.025,
+      "step": 320
+    },
+    {
+      "epoch": 1.1265377855887522,
+      "grad_norm": 0.16080456972122192,
+      "learning_rate": 0.0001024896265560166,
+      "loss": 0.0333,
+      "step": 321
+    },
+    {
+      "epoch": 1.1300527240773286,
+      "grad_norm": 0.11305706948041916,
+      "learning_rate": 0.0001020746887966805,
+      "loss": 0.0262,
+      "step": 322
+    },
+    {
+      "epoch": 1.133567662565905,
+      "grad_norm": 0.1509881168603897,
+      "learning_rate": 0.0001016597510373444,
+      "loss": 0.0389,
+      "step": 323
+    },
+    {
+      "epoch": 1.1370826010544817,
+      "grad_norm": 0.11454585194587708,
+      "learning_rate": 0.0001012448132780083,
+      "loss": 0.0323,
+      "step": 324
+    },
+    {
+      "epoch": 1.140597539543058,
+      "grad_norm": 0.16684749722480774,
+      "learning_rate": 0.0001008298755186722,
+      "loss": 0.0441,
+      "step": 325
+    },
+    {
+      "epoch": 1.1441124780316345,
+      "grad_norm": 0.09474905580282211,
+      "learning_rate": 0.0001004149377593361,
+      "loss": 0.0307,
+      "step": 326
+    },
+    {
+      "epoch": 1.147627416520211,
+      "grad_norm": 0.1257026344537735,
+      "learning_rate": 0.0001,
+      "loss": 0.0382,
+      "step": 327
+    },
+    {
+      "epoch": 1.1511423550087874,
+      "grad_norm": 0.16471615433692932,
+      "learning_rate": 9.95850622406639e-05,
+      "loss": 0.0472,
+      "step": 328
+    },
+    {
+      "epoch": 1.1546572934973638,
+      "grad_norm": 0.16340135037899017,
+      "learning_rate": 9.917012448132781e-05,
+      "loss": 0.0502,
+      "step": 329
+    },
+    {
+      "epoch": 1.1581722319859402,
+      "grad_norm": 0.09674856066703796,
+      "learning_rate": 9.875518672199171e-05,
+      "loss": 0.029,
+      "step": 330
+    },
+    {
+      "epoch": 1.1616871704745166,
+      "grad_norm": 0.11328493803739548,
+      "learning_rate": 9.83402489626556e-05,
+      "loss": 0.0313,
+      "step": 331
+    },
+    {
+      "epoch": 1.165202108963093,
+      "grad_norm": 0.14702650904655457,
+      "learning_rate": 9.79253112033195e-05,
+      "loss": 0.0563,
+      "step": 332
+    },
+    {
+      "epoch": 1.1687170474516697,
+      "grad_norm": 0.1175096407532692,
+      "learning_rate": 9.75103734439834e-05,
+      "loss": 0.0324,
+      "step": 333
+    },
+    {
+      "epoch": 1.1722319859402461,
+      "grad_norm": 0.1501559168100357,
+      "learning_rate": 9.709543568464731e-05,
+      "loss": 0.0433,
+      "step": 334
+    },
+    {
+      "epoch": 1.1757469244288226,
+      "grad_norm": 0.1217423677444458,
+      "learning_rate": 9.668049792531121e-05,
+      "loss": 0.0325,
+      "step": 335
+    },
+    {
+      "epoch": 1.179261862917399,
+      "grad_norm": 0.21164333820343018,
+      "learning_rate": 9.626556016597512e-05,
+      "loss": 0.0331,
+      "step": 336
+    },
+    {
+      "epoch": 1.1827768014059754,
+      "grad_norm": 0.08103303611278534,
+      "learning_rate": 9.585062240663902e-05,
+      "loss": 0.0242,
+      "step": 337
+    },
+    {
+      "epoch": 1.1862917398945518,
+      "grad_norm": 0.09643135219812393,
+      "learning_rate": 9.543568464730292e-05,
+      "loss": 0.0301,
+      "step": 338
+    },
+    {
+      "epoch": 1.1898066783831283,
+      "grad_norm": 0.11211568117141724,
+      "learning_rate": 9.50207468879668e-05,
+      "loss": 0.0355,
+      "step": 339
+    },
+    {
+      "epoch": 1.1933216168717047,
+      "grad_norm": 0.08113870769739151,
+      "learning_rate": 9.460580912863071e-05,
+      "loss": 0.0255,
+      "step": 340
+    },
+    {
+      "epoch": 1.196836555360281,
+      "grad_norm": 0.10726438462734222,
+      "learning_rate": 9.419087136929461e-05,
+      "loss": 0.035,
+      "step": 341
+    },
+    {
+      "epoch": 1.2003514938488578,
+      "grad_norm": 0.09150339663028717,
+      "learning_rate": 9.377593360995851e-05,
+      "loss": 0.0335,
+      "step": 342
+    },
+    {
+      "epoch": 1.2038664323374342,
+      "grad_norm": 0.10595715790987015,
+      "learning_rate": 9.336099585062241e-05,
+      "loss": 0.0379,
+      "step": 343
+    },
+    {
+      "epoch": 1.2073813708260106,
+      "grad_norm": 0.0731506273150444,
+      "learning_rate": 9.294605809128631e-05,
+      "loss": 0.0232,
+      "step": 344
+    },
+    {
+      "epoch": 1.210896309314587,
+      "grad_norm": 0.08540864288806915,
+      "learning_rate": 9.253112033195021e-05,
+      "loss": 0.0223,
+      "step": 345
+    },
+    {
+      "epoch": 1.2144112478031635,
+      "grad_norm": 0.07775469869375229,
+      "learning_rate": 9.21161825726141e-05,
+      "loss": 0.0204,
+      "step": 346
+    },
+    {
+      "epoch": 1.2179261862917399,
+      "grad_norm": 0.10598568618297577,
+      "learning_rate": 9.1701244813278e-05,
+      "loss": 0.0312,
+      "step": 347
+    },
+    {
+      "epoch": 1.2214411247803163,
+      "grad_norm": 0.1299615204334259,
+      "learning_rate": 9.128630705394192e-05,
+      "loss": 0.0308,
+      "step": 348
+    },
+    {
+      "epoch": 1.2249560632688927,
+      "grad_norm": 0.10963129252195358,
+      "learning_rate": 9.087136929460582e-05,
+      "loss": 0.0267,
+      "step": 349
+    },
+    {
+      "epoch": 1.2284710017574691,
+      "grad_norm": 0.24104978144168854,
+      "learning_rate": 9.045643153526972e-05,
+      "loss": 0.0614,
+      "step": 350
+    },
+    {
+      "epoch": 1.2319859402460458,
+      "grad_norm": 0.060414571315050125,
+      "learning_rate": 9.004149377593362e-05,
+      "loss": 0.0197,
+      "step": 351
+    },
+    {
+      "epoch": 1.2355008787346222,
+      "grad_norm": 0.10535873472690582,
+      "learning_rate": 8.962655601659752e-05,
+      "loss": 0.0348,
+      "step": 352
+    },
+    {
+      "epoch": 1.2390158172231986,
+      "grad_norm": 0.11544504016637802,
+      "learning_rate": 8.921161825726142e-05,
+      "loss": 0.0266,
+      "step": 353
+    },
+    {
+      "epoch": 1.242530755711775,
+      "grad_norm": 0.10720725357532501,
+      "learning_rate": 8.879668049792531e-05,
+      "loss": 0.0345,
+      "step": 354
+    },
+    {
+      "epoch": 1.2460456942003515,
+      "grad_norm": 0.1170039027929306,
+      "learning_rate": 8.838174273858921e-05,
+      "loss": 0.043,
+      "step": 355
+    },
+    {
+      "epoch": 1.249560632688928,
+      "grad_norm": 0.08738044649362564,
+      "learning_rate": 8.796680497925311e-05,
+      "loss": 0.0277,
+      "step": 356
+    },
+    {
+      "epoch": 1.2530755711775043,
+      "grad_norm": 0.11182297021150589,
+      "learning_rate": 8.755186721991701e-05,
+      "loss": 0.0273,
+      "step": 357
+    },
+    {
+      "epoch": 1.2565905096660808,
+      "grad_norm": 0.09249255061149597,
+      "learning_rate": 8.713692946058092e-05,
+      "loss": 0.0337,
+      "step": 358
+    },
+    {
+      "epoch": 1.2601054481546572,
+      "grad_norm": 0.10308247059583664,
+      "learning_rate": 8.672199170124482e-05,
+      "loss": 0.034,
+      "step": 359
+    },
+    {
+      "epoch": 1.2636203866432338,
+      "grad_norm": 0.0934724286198616,
+      "learning_rate": 8.630705394190872e-05,
+      "loss": 0.0274,
+      "step": 360
+    },
+    {
+      "epoch": 1.2671353251318103,
+      "grad_norm": 0.08682486414909363,
+      "learning_rate": 8.58921161825726e-05,
+      "loss": 0.034,
+      "step": 361
+    },
+    {
+      "epoch": 1.2706502636203867,
+      "grad_norm": 0.17702794075012207,
+      "learning_rate": 8.547717842323652e-05,
+      "loss": 0.0846,
+      "step": 362
+    },
+    {
+      "epoch": 1.2741652021089631,
+      "grad_norm": 0.13790243864059448,
+      "learning_rate": 8.506224066390042e-05,
+      "loss": 0.0346,
+      "step": 363
+    },
+    {
+      "epoch": 1.2776801405975395,
+      "grad_norm": 0.08539144694805145,
+      "learning_rate": 8.464730290456432e-05,
+      "loss": 0.0283,
+      "step": 364
+    },
+    {
+      "epoch": 1.281195079086116,
+      "grad_norm": 0.09345310926437378,
+      "learning_rate": 8.423236514522822e-05,
+      "loss": 0.0283,
+      "step": 365
+    },
+    {
+      "epoch": 1.2847100175746924,
+      "grad_norm": 0.1070915013551712,
+      "learning_rate": 8.381742738589213e-05,
+      "loss": 0.0334,
+      "step": 366
+    },
+    {
+      "epoch": 1.2882249560632688,
+      "grad_norm": 0.1502905786037445,
+      "learning_rate": 8.340248962655603e-05,
+      "loss": 0.0511,
+      "step": 367
+    },
+    {
+      "epoch": 1.2917398945518452,
+      "grad_norm": 0.09904105961322784,
+      "learning_rate": 8.298755186721993e-05,
+      "loss": 0.0291,
+      "step": 368
+    },
+    {
+      "epoch": 1.2952548330404219,
+      "grad_norm": 0.1428239345550537,
+      "learning_rate": 8.257261410788382e-05,
+      "loss": 0.0339,
+      "step": 369
+    },
+    {
+      "epoch": 1.2987697715289983,
+      "grad_norm": 0.08671260625123978,
+      "learning_rate": 8.215767634854772e-05,
+      "loss": 0.0288,
+      "step": 370
+    },
+    {
+      "epoch": 1.3022847100175747,
+      "grad_norm": 0.1576804518699646,
+      "learning_rate": 8.174273858921162e-05,
+      "loss": 0.0707,
+      "step": 371
+    },
+    {
+      "epoch": 1.3057996485061512,
+      "grad_norm": 0.10015079379081726,
+      "learning_rate": 8.132780082987552e-05,
+      "loss": 0.0354,
+      "step": 372
+    },
+    {
+      "epoch": 1.3093145869947276,
+      "grad_norm": 0.11030863225460052,
+      "learning_rate": 8.091286307053942e-05,
+      "loss": 0.0349,
+      "step": 373
+    },
+    {
+      "epoch": 1.312829525483304,
+      "grad_norm": 0.09071624279022217,
+      "learning_rate": 8.049792531120332e-05,
+      "loss": 0.0315,
+      "step": 374
+    },
+    {
+      "epoch": 1.3163444639718804,
+      "grad_norm": 0.12986351549625397,
+      "learning_rate": 8.008298755186722e-05,
+      "loss": 0.0415,
+      "step": 375
+    },
+    {
+      "epoch": 1.319859402460457,
+      "grad_norm": 0.09092254191637039,
+      "learning_rate": 7.966804979253112e-05,
+      "loss": 0.0327,
+      "step": 376
+    },
+    {
+      "epoch": 1.3233743409490333,
+      "grad_norm": 0.10521815717220306,
+      "learning_rate": 7.925311203319503e-05,
+      "loss": 0.0288,
+      "step": 377
+    },
+    {
+      "epoch": 1.32688927943761,
+      "grad_norm": 0.1093856543302536,
+      "learning_rate": 7.883817427385893e-05,
+      "loss": 0.0323,
+      "step": 378
+    },
+    {
+      "epoch": 1.3304042179261863,
+      "grad_norm": 0.08131673187017441,
+      "learning_rate": 7.842323651452283e-05,
+      "loss": 0.031,
+      "step": 379
+    },
+    {
+      "epoch": 1.3339191564147628,
+      "grad_norm": 0.11505994945764542,
+      "learning_rate": 7.800829875518673e-05,
+      "loss": 0.0389,
+      "step": 380
+    },
+    {
+      "epoch": 1.3374340949033392,
+      "grad_norm": 0.11734279245138168,
+      "learning_rate": 7.759336099585063e-05,
+      "loss": 0.0347,
+      "step": 381
+    },
+    {
+      "epoch": 1.3409490333919156,
+      "grad_norm": 0.1097937524318695,
+      "learning_rate": 7.717842323651453e-05,
+      "loss": 0.0345,
+      "step": 382
+    },
+    {
+      "epoch": 1.344463971880492,
+      "grad_norm": 0.08438119292259216,
+      "learning_rate": 7.676348547717843e-05,
+      "loss": 0.0214,
+      "step": 383
+    },
+    {
+      "epoch": 1.3479789103690685,
+      "grad_norm": 0.08465591818094254,
+      "learning_rate": 7.634854771784232e-05,
+      "loss": 0.0306,
+      "step": 384
+    },
+    {
+      "epoch": 1.3514938488576451,
+      "grad_norm": 0.08609631657600403,
+      "learning_rate": 7.593360995850622e-05,
+      "loss": 0.0214,
+      "step": 385
+    },
+    {
+      "epoch": 1.3550087873462213,
+      "grad_norm": 0.10714844614267349,
+      "learning_rate": 7.551867219917012e-05,
+      "loss": 0.0272,
+      "step": 386
+    },
+    {
+      "epoch": 1.358523725834798,
+      "grad_norm": 0.0996679738163948,
+      "learning_rate": 7.510373443983402e-05,
+      "loss": 0.0151,
+      "step": 387
+    },
+    {
+      "epoch": 1.3620386643233744,
+      "grad_norm": 0.1393178105354309,
+      "learning_rate": 7.468879668049793e-05,
+      "loss": 0.0475,
+      "step": 388
+    },
+    {
+      "epoch": 1.3655536028119508,
+      "grad_norm": 0.1370774358510971,
+      "learning_rate": 7.427385892116183e-05,
+      "loss": 0.0249,
+      "step": 389
+    },
+    {
+      "epoch": 1.3690685413005272,
+      "grad_norm": 0.08210641145706177,
+      "learning_rate": 7.385892116182573e-05,
+      "loss": 0.0256,
+      "step": 390
+    },
+    {
+      "epoch": 1.3725834797891037,
+      "grad_norm": 0.10344867408275604,
+      "learning_rate": 7.344398340248963e-05,
+      "loss": 0.0368,
+      "step": 391
+    },
+    {
+      "epoch": 1.37609841827768,
+      "grad_norm": 0.11747119575738907,
+      "learning_rate": 7.302904564315353e-05,
+      "loss": 0.0438,
+      "step": 392
+    },
+    {
+      "epoch": 1.3796133567662565,
+      "grad_norm": 0.0993000715970993,
+      "learning_rate": 7.261410788381743e-05,
+      "loss": 0.026,
+      "step": 393
+    },
+    {
+      "epoch": 1.3831282952548332,
+      "grad_norm": 0.104093037545681,
+      "learning_rate": 7.219917012448133e-05,
+      "loss": 0.0276,
+      "step": 394
+    },
+    {
+      "epoch": 1.3866432337434094,
+      "grad_norm": 0.11269046366214752,
+      "learning_rate": 7.178423236514523e-05,
+      "loss": 0.0242,
+      "step": 395
+    },
+    {
+      "epoch": 1.390158172231986,
+      "grad_norm": 0.10255884379148483,
+      "learning_rate": 7.136929460580914e-05,
+      "loss": 0.0256,
+      "step": 396
+    },
+    {
+      "epoch": 1.3936731107205624,
+      "grad_norm": 0.1722956746816635,
+      "learning_rate": 7.095435684647304e-05,
+      "loss": 0.0498,
+      "step": 397
+    },
+    {
+      "epoch": 1.3971880492091389,
+      "grad_norm": 0.11679506301879883,
+      "learning_rate": 7.053941908713694e-05,
+      "loss": 0.038,
+      "step": 398
+    },
+    {
+      "epoch": 1.4007029876977153,
+      "grad_norm": 0.08600778132677078,
+      "learning_rate": 7.012448132780083e-05,
+      "loss": 0.0279,
+      "step": 399
+    },
+    {
+      "epoch": 1.4042179261862917,
+      "grad_norm": 0.07060538977384567,
+      "learning_rate": 6.970954356846473e-05,
+      "loss": 0.0233,
+      "step": 400
+    },
+    {
+      "epoch": 1.4042179261862917,
+      "eval_loss": 0.0394415408372879,
+      "eval_runtime": 178.4848,
+      "eval_samples_per_second": 2.835,
+      "eval_steps_per_second": 0.359,
+      "step": 400
+    },
+    {
+      "epoch": 1.4077328646748681,
+      "grad_norm": 0.09761733561754227,
+      "learning_rate": 6.929460580912863e-05,
+      "loss": 0.0322,
+      "step": 401
+    },
+    {
+      "epoch": 1.4112478031634446,
+      "grad_norm": 0.08879027515649796,
+      "learning_rate": 6.887966804979253e-05,
+      "loss": 0.0288,
+      "step": 402
+    },
+    {
+      "epoch": 1.4147627416520212,
+      "grad_norm": 0.11523272097110748,
+      "learning_rate": 6.846473029045643e-05,
+      "loss": 0.0291,
+      "step": 403
+    },
+    {
+      "epoch": 1.4182776801405974,
+      "grad_norm": 0.09919934719800949,
+      "learning_rate": 6.804979253112033e-05,
+      "loss": 0.0381,
+      "step": 404
+    },
+    {
+      "epoch": 1.421792618629174,
+      "grad_norm": 0.07385391741991043,
+      "learning_rate": 6.763485477178423e-05,
+      "loss": 0.0272,
+      "step": 405
+    },
+    {
+      "epoch": 1.4253075571177505,
+      "grad_norm": 0.07892516255378723,
+      "learning_rate": 6.721991701244813e-05,
+      "loss": 0.0254,
+      "step": 406
+    },
+    {
+      "epoch": 1.428822495606327,
+      "grad_norm": 0.1717812418937683,
+      "learning_rate": 6.680497925311204e-05,
+      "loss": 0.0598,
+      "step": 407
+    },
+    {
+      "epoch": 1.4323374340949033,
+      "grad_norm": 0.10333540290594101,
+      "learning_rate": 6.639004149377594e-05,
+      "loss": 0.0321,
+      "step": 408
+    },
+    {
+      "epoch": 1.4358523725834798,
+      "grad_norm": 0.10540790855884552,
+      "learning_rate": 6.597510373443984e-05,
+      "loss": 0.0316,
+      "step": 409
+    },
+    {
+      "epoch": 1.4393673110720562,
+      "grad_norm": 0.09513910859823227,
+      "learning_rate": 6.556016597510374e-05,
+      "loss": 0.0378,
+      "step": 410
+    },
+    {
+      "epoch": 1.4428822495606326,
+      "grad_norm": 0.13851504027843475,
+      "learning_rate": 6.514522821576764e-05,
+      "loss": 0.0336,
+      "step": 411
+    },
+    {
+      "epoch": 1.4463971880492092,
+      "grad_norm": 0.09247998148202896,
+      "learning_rate": 6.473029045643154e-05,
+      "loss": 0.0362,
+      "step": 412
+    },
+    {
+      "epoch": 1.4499121265377855,
+      "grad_norm": 0.087465301156044,
+      "learning_rate": 6.431535269709544e-05,
+      "loss": 0.0301,
+      "step": 413
+    },
+    {
+      "epoch": 1.453427065026362,
+      "grad_norm": 0.06486547738313675,
+      "learning_rate": 6.390041493775933e-05,
+      "loss": 0.0218,
+      "step": 414
+    },
+    {
+      "epoch": 1.4569420035149385,
+      "grad_norm": 0.07497644424438477,
+      "learning_rate": 6.348547717842323e-05,
+      "loss": 0.0279,
+      "step": 415
+    },
+    {
+      "epoch": 1.460456942003515,
+      "grad_norm": 0.09684702008962631,
+      "learning_rate": 6.307053941908713e-05,
+      "loss": 0.0382,
+      "step": 416
+    },
+    {
+      "epoch": 1.4639718804920914,
+      "grad_norm": 0.10948526114225388,
+      "learning_rate": 6.265560165975103e-05,
+      "loss": 0.0336,
+      "step": 417
+    },
+    {
+      "epoch": 1.4674868189806678,
+      "grad_norm": 0.08034105598926544,
+      "learning_rate": 6.224066390041494e-05,
+      "loss": 0.024,
+      "step": 418
+    },
+    {
+      "epoch": 1.4710017574692442,
+      "grad_norm": 0.04268701747059822,
+      "learning_rate": 6.182572614107884e-05,
+      "loss": 0.0157,
+      "step": 419
+    },
+    {
+      "epoch": 1.4745166959578206,
+      "grad_norm": 0.0965370312333107,
+      "learning_rate": 6.141078838174274e-05,
+      "loss": 0.0287,
+      "step": 420
+    },
+    {
+      "epoch": 1.4780316344463973,
+      "grad_norm": 0.08190871775150299,
+      "learning_rate": 6.099585062240665e-05,
+      "loss": 0.0298,
+      "step": 421
+    },
+    {
+      "epoch": 1.4815465729349737,
+      "grad_norm": 0.08082269132137299,
+      "learning_rate": 6.058091286307054e-05,
+      "loss": 0.0262,
+      "step": 422
+    },
+    {
+      "epoch": 1.4850615114235501,
+      "grad_norm": 0.09048648923635483,
+      "learning_rate": 6.016597510373444e-05,
+      "loss": 0.0256,
+      "step": 423
+    },
+    {
+      "epoch": 1.4885764499121266,
+      "grad_norm": 0.07813292741775513,
+      "learning_rate": 5.9751037344398344e-05,
+      "loss": 0.0329,
+      "step": 424
+    },
+    {
+      "epoch": 1.492091388400703,
+      "grad_norm": 0.09163984656333923,
+      "learning_rate": 5.9336099585062245e-05,
+      "loss": 0.025,
+      "step": 425
+    },
+    {
+      "epoch": 1.4956063268892794,
+      "grad_norm": 0.08940015733242035,
+      "learning_rate": 5.8921161825726146e-05,
+      "loss": 0.0238,
+      "step": 426
+    },
+    {
+      "epoch": 1.4991212653778558,
+      "grad_norm": 0.11653712391853333,
+      "learning_rate": 5.850622406639005e-05,
+      "loss": 0.0277,
+      "step": 427
+    },
+    {
+      "epoch": 1.5026362038664325,
+      "grad_norm": 0.10306468605995178,
+      "learning_rate": 5.809128630705395e-05,
+      "loss": 0.0293,
+      "step": 428
+    },
+    {
+      "epoch": 1.5061511423550087,
+      "grad_norm": 0.10990747809410095,
+      "learning_rate": 5.767634854771784e-05,
+      "loss": 0.0283,
+      "step": 429
+    },
+    {
+      "epoch": 1.5096660808435853,
+      "grad_norm": 0.08972524851560593,
+      "learning_rate": 5.7261410788381744e-05,
+      "loss": 0.0221,
+      "step": 430
+    },
+    {
+      "epoch": 1.5131810193321615,
+      "grad_norm": 0.08488809317350388,
+      "learning_rate": 5.6846473029045646e-05,
+      "loss": 0.0225,
+      "step": 431
+    },
+    {
+      "epoch": 1.5166959578207382,
+      "grad_norm": 0.10268191993236542,
+      "learning_rate": 5.643153526970955e-05,
+      "loss": 0.0259,
+      "step": 432
+    },
+    {
+      "epoch": 1.5202108963093146,
+      "grad_norm": 0.0738985687494278,
+      "learning_rate": 5.601659751037345e-05,
+      "loss": 0.0246,
+      "step": 433
+    },
+    {
+      "epoch": 1.523725834797891,
+      "grad_norm": 0.09742774814367294,
+      "learning_rate": 5.560165975103735e-05,
+      "loss": 0.0262,
+      "step": 434
+    },
+    {
+      "epoch": 1.5272407732864675,
+      "grad_norm": 0.18726953864097595,
+      "learning_rate": 5.518672199170125e-05,
+      "loss": 0.0661,
+      "step": 435
+    },
+    {
+      "epoch": 1.5307557117750439,
+      "grad_norm": 0.08196069300174713,
+      "learning_rate": 5.477178423236515e-05,
+      "loss": 0.0251,
+      "step": 436
+    },
+    {
+      "epoch": 1.5342706502636205,
+      "grad_norm": 0.11763795465230942,
+      "learning_rate": 5.4356846473029046e-05,
+      "loss": 0.0365,
+      "step": 437
+    },
+    {
+      "epoch": 1.5377855887521967,
+      "grad_norm": 0.08862517774105072,
+      "learning_rate": 5.394190871369295e-05,
+      "loss": 0.0281,
+      "step": 438
+    },
+    {
+      "epoch": 1.5413005272407734,
+      "grad_norm": 0.08350022882223129,
+      "learning_rate": 5.352697095435685e-05,
+      "loss": 0.025,
+      "step": 439
+    },
+    {
+      "epoch": 1.5448154657293496,
+      "grad_norm": 0.07232296466827393,
+      "learning_rate": 5.311203319502075e-05,
+      "loss": 0.0229,
+      "step": 440
+    },
+    {
+      "epoch": 1.5483304042179262,
+      "grad_norm": 0.07918655872344971,
+      "learning_rate": 5.269709543568465e-05,
+      "loss": 0.0222,
+      "step": 441
+    },
+    {
+      "epoch": 1.5518453427065027,
+      "grad_norm": 0.1173887774348259,
+      "learning_rate": 5.228215767634855e-05,
+      "loss": 0.0313,
+      "step": 442
+    },
+    {
+      "epoch": 1.555360281195079,
+      "grad_norm": 0.08697450160980225,
+      "learning_rate": 5.1867219917012454e-05,
+      "loss": 0.0237,
+      "step": 443
+    },
+    {
+      "epoch": 1.5588752196836555,
+      "grad_norm": 0.10145950317382812,
+      "learning_rate": 5.145228215767635e-05,
+      "loss": 0.027,
+      "step": 444
+    },
+    {
+      "epoch": 1.562390158172232,
+      "grad_norm": 0.13053980469703674,
+      "learning_rate": 5.103734439834025e-05,
+      "loss": 0.0372,
+      "step": 445
+    },
+    {
+      "epoch": 1.5659050966608086,
+      "grad_norm": 0.11387127637863159,
+      "learning_rate": 5.062240663900415e-05,
+      "loss": 0.037,
+      "step": 446
+    },
+    {
+      "epoch": 1.5694200351493848,
+      "grad_norm": 0.09816371649503708,
+      "learning_rate": 5.020746887966805e-05,
+      "loss": 0.0319,
+      "step": 447
+    },
+    {
+      "epoch": 1.5729349736379614,
+      "grad_norm": 0.07378123700618744,
+      "learning_rate": 4.979253112033195e-05,
+      "loss": 0.0258,
+      "step": 448
+    },
+    {
+      "epoch": 1.5764499121265376,
+      "grad_norm": 0.0998382717370987,
+      "learning_rate": 4.9377593360995854e-05,
+      "loss": 0.0402,
+      "step": 449
+    },
+    {
+      "epoch": 1.5799648506151143,
+      "grad_norm": 0.053047847002744675,
+      "learning_rate": 4.896265560165975e-05,
+      "loss": 0.0193,
+      "step": 450
+    },
+    {
+      "epoch": 1.5834797891036907,
+      "grad_norm": 0.12353768944740295,
+      "learning_rate": 4.854771784232366e-05,
+      "loss": 0.038,
+      "step": 451
+    },
+    {
+      "epoch": 1.5869947275922671,
+      "grad_norm": 0.10450875759124756,
+      "learning_rate": 4.813278008298756e-05,
+      "loss": 0.0353,
+      "step": 452
+    },
+    {
+      "epoch": 1.5905096660808435,
+      "grad_norm": 0.11111406981945038,
+      "learning_rate": 4.771784232365146e-05,
+      "loss": 0.0361,
+      "step": 453
+    },
+    {
+      "epoch": 1.59402460456942,
+      "grad_norm": 0.08408033102750778,
+      "learning_rate": 4.7302904564315354e-05,
+      "loss": 0.0239,
+      "step": 454
+    },
+    {
+      "epoch": 1.5975395430579966,
+      "grad_norm": 0.08122161030769348,
+      "learning_rate": 4.6887966804979255e-05,
+      "loss": 0.0271,
+      "step": 455
+    },
+    {
+      "epoch": 1.6010544815465728,
+      "grad_norm": 0.1017850786447525,
+      "learning_rate": 4.6473029045643156e-05,
+      "loss": 0.0358,
+      "step": 456
+    },
+    {
+      "epoch": 1.6045694200351495,
+      "grad_norm": 0.09576894342899323,
+      "learning_rate": 4.605809128630705e-05,
+      "loss": 0.0307,
+      "step": 457
+    },
+    {
+      "epoch": 1.6080843585237259,
+      "grad_norm": 0.1270962357521057,
+      "learning_rate": 4.564315352697096e-05,
+      "loss": 0.0323,
+      "step": 458
+    },
+    {
+      "epoch": 1.6115992970123023,
+      "grad_norm": 0.09580868482589722,
+      "learning_rate": 4.522821576763486e-05,
+      "loss": 0.0379,
+      "step": 459
+    },
+    {
+      "epoch": 1.6151142355008787,
+      "grad_norm": 0.10440625250339508,
+      "learning_rate": 4.481327800829876e-05,
+      "loss": 0.041,
+      "step": 460
+    },
+    {
+      "epoch": 1.6186291739894552,
+      "grad_norm": 0.12036912888288498,
+      "learning_rate": 4.4398340248962656e-05,
+      "loss": 0.0365,
+      "step": 461
+    },
+    {
+      "epoch": 1.6221441124780316,
+      "grad_norm": 0.06830941140651703,
+      "learning_rate": 4.398340248962656e-05,
+      "loss": 0.0224,
+      "step": 462
+    },
+    {
+      "epoch": 1.625659050966608,
+      "grad_norm": 0.08766771107912064,
+      "learning_rate": 4.356846473029046e-05,
+      "loss": 0.0349,
+      "step": 463
+    },
+    {
+      "epoch": 1.6291739894551847,
+      "grad_norm": 0.12244472652673721,
+      "learning_rate": 4.315352697095436e-05,
+      "loss": 0.0305,
+      "step": 464
+    },
+    {
+      "epoch": 1.6326889279437609,
+      "grad_norm": 0.06699530780315399,
+      "learning_rate": 4.273858921161826e-05,
+      "loss": 0.0213,
+      "step": 465
+    },
+    {
+      "epoch": 1.6362038664323375,
+      "grad_norm": 0.04710953310132027,
+      "learning_rate": 4.232365145228216e-05,
+      "loss": 0.0172,
+      "step": 466
+    },
+    {
+      "epoch": 1.639718804920914,
+      "grad_norm": 0.03541887179017067,
+      "learning_rate": 4.190871369294606e-05,
+      "loss": 0.0145,
+      "step": 467
+    },
+    {
+      "epoch": 1.6432337434094904,
+      "grad_norm": 0.08300907164812088,
+      "learning_rate": 4.1493775933609964e-05,
+      "loss": 0.0186,
+      "step": 468
+    },
+    {
+      "epoch": 1.6467486818980668,
+      "grad_norm": 0.08546904474496841,
+      "learning_rate": 4.107883817427386e-05,
+      "loss": 0.0263,
+      "step": 469
+    },
+    {
+      "epoch": 1.6502636203866432,
+      "grad_norm": 0.0735994502902031,
+      "learning_rate": 4.066390041493776e-05,
+      "loss": 0.0216,
+      "step": 470
+    },
+    {
+      "epoch": 1.6537785588752196,
+      "grad_norm": 0.0836733877658844,
+      "learning_rate": 4.024896265560166e-05,
+      "loss": 0.0274,
+      "step": 471
+    },
+    {
+      "epoch": 1.657293497363796,
+      "grad_norm": 0.07491277158260345,
+      "learning_rate": 3.983402489626556e-05,
+      "loss": 0.0225,
+      "step": 472
+    },
+    {
+      "epoch": 1.6608084358523727,
+      "grad_norm": 0.10139133036136627,
+      "learning_rate": 3.9419087136929464e-05,
+      "loss": 0.0344,
+      "step": 473
+    },
+    {
+      "epoch": 1.664323374340949,
+      "grad_norm": 0.06060206517577171,
+      "learning_rate": 3.9004149377593365e-05,
+      "loss": 0.0222,
+      "step": 474
+    },
+    {
+      "epoch": 1.6678383128295255,
+      "grad_norm": 0.08016759902238846,
+      "learning_rate": 3.8589211618257266e-05,
+      "loss": 0.0199,
+      "step": 475
+    },
+    {
+      "epoch": 1.671353251318102,
+      "grad_norm": 0.06554888933897018,
+      "learning_rate": 3.817427385892116e-05,
+      "loss": 0.0233,
+      "step": 476
+    },
+    {
+      "epoch": 1.6748681898066784,
+      "grad_norm": 0.08745160698890686,
+      "learning_rate": 3.775933609958506e-05,
+      "loss": 0.0287,
+      "step": 477
+    },
+    {
+      "epoch": 1.6783831282952548,
+      "grad_norm": 0.0867169126868248,
+      "learning_rate": 3.734439834024896e-05,
+      "loss": 0.0315,
+      "step": 478
+    },
+    {
+      "epoch": 1.6818980667838312,
+      "grad_norm": 0.07407302409410477,
+      "learning_rate": 3.6929460580912864e-05,
+      "loss": 0.0253,
+      "step": 479
+    },
+    {
+      "epoch": 1.685413005272408,
+      "grad_norm": 0.05431722104549408,
+      "learning_rate": 3.6514522821576766e-05,
+      "loss": 0.0167,
+      "step": 480
+    },
+    {
+      "epoch": 1.688927943760984,
+      "grad_norm": 0.08641435205936432,
+      "learning_rate": 3.609958506224067e-05,
+      "loss": 0.0284,
+      "step": 481
+    },
+    {
+      "epoch": 1.6924428822495607,
+      "grad_norm": 0.05755087360739708,
+      "learning_rate": 3.568464730290457e-05,
+      "loss": 0.0177,
+      "step": 482
+    },
+    {
+      "epoch": 1.695957820738137,
+      "grad_norm": 0.057126596570014954,
+      "learning_rate": 3.526970954356847e-05,
+      "loss": 0.0187,
+      "step": 483
+    },
+    {
+      "epoch": 1.6994727592267136,
+      "grad_norm": 0.13521793484687805,
+      "learning_rate": 3.4854771784232364e-05,
+      "loss": 0.0423,
+      "step": 484
+    },
+    {
+      "epoch": 1.70298769771529,
+      "grad_norm": 0.06800301373004913,
+      "learning_rate": 3.4439834024896265e-05,
+      "loss": 0.0218,
+      "step": 485
+    },
+    {
+      "epoch": 1.7065026362038664,
+      "grad_norm": 0.11221302300691605,
+      "learning_rate": 3.4024896265560166e-05,
+      "loss": 0.0274,
+      "step": 486
+    },
+    {
+      "epoch": 1.7100175746924429,
+      "grad_norm": 0.07233195751905441,
+      "learning_rate": 3.360995850622407e-05,
+      "loss": 0.0251,
+      "step": 487
+    },
+    {
+      "epoch": 1.7135325131810193,
+      "grad_norm": 0.08482959866523743,
+      "learning_rate": 3.319502074688797e-05,
+      "loss": 0.0183,
+      "step": 488
+    },
+    {
+      "epoch": 1.717047451669596,
+      "grad_norm": 0.05164050683379173,
+      "learning_rate": 3.278008298755187e-05,
+      "loss": 0.0172,
+      "step": 489
+    },
+    {
+      "epoch": 1.7205623901581721,
+      "grad_norm": 0.09971071034669876,
+      "learning_rate": 3.236514522821577e-05,
+      "loss": 0.0263,
+      "step": 490
+    },
+    {
+      "epoch": 1.7240773286467488,
+      "grad_norm": 0.0788310170173645,
+      "learning_rate": 3.1950207468879666e-05,
+      "loss": 0.0256,
+      "step": 491
+    },
+    {
+      "epoch": 1.727592267135325,
+      "grad_norm": 0.08644759654998779,
+      "learning_rate": 3.153526970954357e-05,
+      "loss": 0.0253,
+      "step": 492
+    },
+    {
+      "epoch": 1.7311072056239016,
+      "grad_norm": 0.0552949421107769,
+      "learning_rate": 3.112033195020747e-05,
+      "loss": 0.0201,
+      "step": 493
+    },
+    {
+      "epoch": 1.734622144112478,
+      "grad_norm": 0.12068801373243332,
+      "learning_rate": 3.070539419087137e-05,
+      "loss": 0.0324,
+      "step": 494
+    },
+    {
+      "epoch": 1.7381370826010545,
+      "grad_norm": 0.0852043628692627,
+      "learning_rate": 3.029045643153527e-05,
+      "loss": 0.0251,
+      "step": 495
+    },
+    {
+      "epoch": 1.741652021089631,
+      "grad_norm": 0.08057104051113129,
+      "learning_rate": 2.9875518672199172e-05,
+      "loss": 0.0225,
+      "step": 496
+    },
+    {
+      "epoch": 1.7451669595782073,
+      "grad_norm": 0.06668426841497421,
+      "learning_rate": 2.9460580912863073e-05,
+      "loss": 0.0235,
+      "step": 497
+    },
+    {
+      "epoch": 1.748681898066784,
+      "grad_norm": 0.046700526028871536,
+      "learning_rate": 2.9045643153526974e-05,
+      "loss": 0.0169,
+      "step": 498
+    },
+    {
+      "epoch": 1.7521968365553602,
+      "grad_norm": 0.10106682032346725,
+      "learning_rate": 2.8630705394190872e-05,
+      "loss": 0.0271,
+      "step": 499
+    },
+    {
+      "epoch": 1.7557117750439368,
+      "grad_norm": 0.0915568470954895,
+      "learning_rate": 2.8215767634854773e-05,
+      "loss": 0.0275,
+      "step": 500
+    },
+    {
+      "epoch": 1.7557117750439368,
+      "eval_loss": 0.03437558189034462,
+      "eval_runtime": 178.6138,
+      "eval_samples_per_second": 2.833,
+      "eval_steps_per_second": 0.358,
+      "step": 500
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 568,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.5333795292618875e+18,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5317ae7c8fd1fdde83f11b34f1e8a0bbac31dca36c4ce0659dee88523cebf08b
+size 5624

checkpoint-568/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/qwq-32b-preview-bnb-4bit
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-568/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/qwq-32b-preview-bnb-4bit",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "v_proj",
+    "gate_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": true
+}

checkpoint-568/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a89415edce0bc8e5e1d7061c79b5016468b445c5df4f6a0e4854de6b4abdb566
+size 2147605960

checkpoint-568/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-568/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74b59b0f81060a6bcb2d995e4705440aaaf41ab896964087829f1dadbf1ffa4b
+size 1091573332

checkpoint-568/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e1c0fb523914a5a4fd99843319c531ac1b76196088779c16816fd1dd6a05915
+size 14244

checkpoint-568/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c46b8151cf74e639931a65476d42e59be513c3e428f58e4cc322c87abf981835
+size 1064

checkpoint-568/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-568/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-568/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-568/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5317ae7c8fd1fdde83f11b34f1e8a0bbac31dca36c4ce0659dee88523cebf08b
+size 5624

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}