Add files using upload-large-folder tool

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1200/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1200/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1200/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1200/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/\u4e0d\u51bb\u7ed3Qwen\u8bad\u7ec3/models/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-1500/trainer_state.json

@@ -0,0 +1,2141 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 375.0,
+  "eval_steps": 500,
+  "global_step": 1500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.32,
+      "grad_norm": 17.919572830200195,
+      "learning_rate": 0.0,
+      "loss": 3.1974,
+      "step": 1
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 15.281089782714844,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 3.1689,
+      "step": 5
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 10.974032402038574,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.6104,
+      "step": 10
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 6.320477485656738,
+      "learning_rate": 2.5925925925925925e-05,
+      "loss": 1.5232,
+      "step": 15
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.79914927482605,
+      "learning_rate": 3.518518518518519e-05,
+      "loss": 0.6024,
+      "step": 20
+    },
+    {
+      "epoch": 6.32,
+      "grad_norm": 1.5797882080078125,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 0.3262,
+      "step": 25
+    },
+    {
+      "epoch": 7.64,
+      "grad_norm": 1.0163109302520752,
+      "learning_rate": 5.370370370370371e-05,
+      "loss": 0.3026,
+      "step": 30
+    },
+    {
+      "epoch": 8.96,
+      "grad_norm": 0.6873035430908203,
+      "learning_rate": 6.296296296296296e-05,
+      "loss": 0.2911,
+      "step": 35
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.2708097696304321,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.2855,
+      "step": 40
+    },
+    {
+      "epoch": 11.32,
+      "grad_norm": 0.8784252405166626,
+      "learning_rate": 8.148148148148148e-05,
+      "loss": 0.2598,
+      "step": 45
+    },
+    {
+      "epoch": 12.64,
+      "grad_norm": 0.7586076855659485,
+      "learning_rate": 9.074074074074075e-05,
+      "loss": 0.2475,
+      "step": 50
+    },
+    {
+      "epoch": 13.96,
+      "grad_norm": 0.8225407600402832,
+      "learning_rate": 0.0001,
+      "loss": 0.2336,
+      "step": 55
+    },
+    {
+      "epoch": 15.0,
+      "grad_norm": 1.7115800380706787,
+      "learning_rate": 9.971363115693013e-05,
+      "loss": 0.2326,
+      "step": 60
+    },
+    {
+      "epoch": 16.32,
+      "grad_norm": 0.5084587335586548,
+      "learning_rate": 9.942726231386026e-05,
+      "loss": 0.1953,
+      "step": 65
+    },
+    {
+      "epoch": 17.64,
+      "grad_norm": 0.4642345905303955,
+      "learning_rate": 9.914089347079038e-05,
+      "loss": 0.1768,
+      "step": 70
+    },
+    {
+      "epoch": 18.96,
+      "grad_norm": 0.6053332090377808,
+      "learning_rate": 9.885452462772051e-05,
+      "loss": 0.1529,
+      "step": 75
+    },
+    {
+      "epoch": 20.0,
+      "grad_norm": 0.8135235905647278,
+      "learning_rate": 9.856815578465064e-05,
+      "loss": 0.1353,
+      "step": 80
+    },
+    {
+      "epoch": 21.32,
+      "grad_norm": 0.35404855012893677,
+      "learning_rate": 9.828178694158075e-05,
+      "loss": 0.1295,
+      "step": 85
+    },
+    {
+      "epoch": 22.64,
+      "grad_norm": 0.6411961913108826,
+      "learning_rate": 9.799541809851088e-05,
+      "loss": 0.1212,
+      "step": 90
+    },
+    {
+      "epoch": 23.96,
+      "grad_norm": 0.4661860764026642,
+      "learning_rate": 9.7709049255441e-05,
+      "loss": 0.1202,
+      "step": 95
+    },
+    {
+      "epoch": 25.0,
+      "grad_norm": 1.4080708026885986,
+      "learning_rate": 9.742268041237114e-05,
+      "loss": 0.1196,
+      "step": 100
+    },
+    {
+      "epoch": 26.32,
+      "grad_norm": 0.3422178626060486,
+      "learning_rate": 9.713631156930127e-05,
+      "loss": 0.1204,
+      "step": 105
+    },
+    {
+      "epoch": 27.64,
+      "grad_norm": 0.4003280699253082,
+      "learning_rate": 9.68499427262314e-05,
+      "loss": 0.1189,
+      "step": 110
+    },
+    {
+      "epoch": 28.96,
+      "grad_norm": 0.324015736579895,
+      "learning_rate": 9.656357388316152e-05,
+      "loss": 0.1166,
+      "step": 115
+    },
+    {
+      "epoch": 30.0,
+      "grad_norm": 1.315653920173645,
+      "learning_rate": 9.627720504009165e-05,
+      "loss": 0.1167,
+      "step": 120
+    },
+    {
+      "epoch": 31.32,
+      "grad_norm": 0.2823339104652405,
+      "learning_rate": 9.599083619702178e-05,
+      "loss": 0.1192,
+      "step": 125
+    },
+    {
+      "epoch": 32.64,
+      "grad_norm": 0.2731497883796692,
+      "learning_rate": 9.57044673539519e-05,
+      "loss": 0.1309,
+      "step": 130
+    },
+    {
+      "epoch": 33.96,
+      "grad_norm": 0.784600019454956,
+      "learning_rate": 9.541809851088203e-05,
+      "loss": 0.1134,
+      "step": 135
+    },
+    {
+      "epoch": 35.0,
+      "grad_norm": 1.0997117757797241,
+      "learning_rate": 9.513172966781214e-05,
+      "loss": 0.1157,
+      "step": 140
+    },
+    {
+      "epoch": 36.32,
+      "grad_norm": 0.2136397361755371,
+      "learning_rate": 9.484536082474227e-05,
+      "loss": 0.1099,
+      "step": 145
+    },
+    {
+      "epoch": 37.64,
+      "grad_norm": 0.19623281061649323,
+      "learning_rate": 9.45589919816724e-05,
+      "loss": 0.109,
+      "step": 150
+    },
+    {
+      "epoch": 38.96,
+      "grad_norm": 0.19020365178585052,
+      "learning_rate": 9.427262313860252e-05,
+      "loss": 0.1048,
+      "step": 155
+    },
+    {
+      "epoch": 40.0,
+      "grad_norm": 0.3295031487941742,
+      "learning_rate": 9.398625429553265e-05,
+      "loss": 0.1073,
+      "step": 160
+    },
+    {
+      "epoch": 41.32,
+      "grad_norm": 0.25549960136413574,
+      "learning_rate": 9.369988545246277e-05,
+      "loss": 0.1135,
+      "step": 165
+    },
+    {
+      "epoch": 42.64,
+      "grad_norm": 0.24988926947116852,
+      "learning_rate": 9.34135166093929e-05,
+      "loss": 0.1047,
+      "step": 170
+    },
+    {
+      "epoch": 43.96,
+      "grad_norm": 0.44344109296798706,
+      "learning_rate": 9.312714776632303e-05,
+      "loss": 0.1069,
+      "step": 175
+    },
+    {
+      "epoch": 45.0,
+      "grad_norm": 0.7739079594612122,
+      "learning_rate": 9.284077892325315e-05,
+      "loss": 0.1074,
+      "step": 180
+    },
+    {
+      "epoch": 46.32,
+      "grad_norm": 0.16808579862117767,
+      "learning_rate": 9.255441008018328e-05,
+      "loss": 0.1063,
+      "step": 185
+    },
+    {
+      "epoch": 47.64,
+      "grad_norm": 0.22390314936637878,
+      "learning_rate": 9.22680412371134e-05,
+      "loss": 0.107,
+      "step": 190
+    },
+    {
+      "epoch": 48.96,
+      "grad_norm": 0.24022641777992249,
+      "learning_rate": 9.198167239404353e-05,
+      "loss": 0.1062,
+      "step": 195
+    },
+    {
+      "epoch": 50.0,
+      "grad_norm": 1.2908257246017456,
+      "learning_rate": 9.169530355097366e-05,
+      "loss": 0.1098,
+      "step": 200
+    },
+    {
+      "epoch": 51.32,
+      "grad_norm": 0.24903152883052826,
+      "learning_rate": 9.140893470790379e-05,
+      "loss": 0.1301,
+      "step": 205
+    },
+    {
+      "epoch": 52.64,
+      "grad_norm": 0.16907045245170593,
+      "learning_rate": 9.112256586483391e-05,
+      "loss": 0.1119,
+      "step": 210
+    },
+    {
+      "epoch": 53.96,
+      "grad_norm": 0.19608496129512787,
+      "learning_rate": 9.083619702176404e-05,
+      "loss": 0.1115,
+      "step": 215
+    },
+    {
+      "epoch": 55.0,
+      "grad_norm": 0.681024968624115,
+      "learning_rate": 9.054982817869416e-05,
+      "loss": 0.107,
+      "step": 220
+    },
+    {
+      "epoch": 56.32,
+      "grad_norm": 0.2148016393184662,
+      "learning_rate": 9.026345933562429e-05,
+      "loss": 0.1064,
+      "step": 225
+    },
+    {
+      "epoch": 57.64,
+      "grad_norm": 0.4002823531627655,
+      "learning_rate": 8.997709049255442e-05,
+      "loss": 0.11,
+      "step": 230
+    },
+    {
+      "epoch": 58.96,
+      "grad_norm": 0.17194221913814545,
+      "learning_rate": 8.969072164948454e-05,
+      "loss": 0.1075,
+      "step": 235
+    },
+    {
+      "epoch": 60.0,
+      "grad_norm": 0.5104140043258667,
+      "learning_rate": 8.940435280641467e-05,
+      "loss": 0.1032,
+      "step": 240
+    },
+    {
+      "epoch": 61.32,
+      "grad_norm": 0.1683240681886673,
+      "learning_rate": 8.91179839633448e-05,
+      "loss": 0.1044,
+      "step": 245
+    },
+    {
+      "epoch": 62.64,
+      "grad_norm": 0.14366662502288818,
+      "learning_rate": 8.883161512027491e-05,
+      "loss": 0.105,
+      "step": 250
+    },
+    {
+      "epoch": 63.96,
+      "grad_norm": 0.11408836394548416,
+      "learning_rate": 8.854524627720504e-05,
+      "loss": 0.1058,
+      "step": 255
+    },
+    {
+      "epoch": 65.0,
+      "grad_norm": 0.3783295750617981,
+      "learning_rate": 8.825887743413516e-05,
+      "loss": 0.1035,
+      "step": 260
+    },
+    {
+      "epoch": 66.32,
+      "grad_norm": 0.15358224511146545,
+      "learning_rate": 8.797250859106529e-05,
+      "loss": 0.1038,
+      "step": 265
+    },
+    {
+      "epoch": 67.64,
+      "grad_norm": 0.18252335488796234,
+      "learning_rate": 8.768613974799542e-05,
+      "loss": 0.1002,
+      "step": 270
+    },
+    {
+      "epoch": 68.96,
+      "grad_norm": 0.15634660422801971,
+      "learning_rate": 8.739977090492554e-05,
+      "loss": 0.1062,
+      "step": 275
+    },
+    {
+      "epoch": 70.0,
+      "grad_norm": 0.6345776319503784,
+      "learning_rate": 8.711340206185567e-05,
+      "loss": 0.1066,
+      "step": 280
+    },
+    {
+      "epoch": 71.32,
+      "grad_norm": 0.18226927518844604,
+      "learning_rate": 8.682703321878581e-05,
+      "loss": 0.1039,
+      "step": 285
+    },
+    {
+      "epoch": 72.64,
+      "grad_norm": 0.10437572747468948,
+      "learning_rate": 8.654066437571594e-05,
+      "loss": 0.1064,
+      "step": 290
+    },
+    {
+      "epoch": 73.96,
+      "grad_norm": 0.14951160550117493,
+      "learning_rate": 8.625429553264606e-05,
+      "loss": 0.1045,
+      "step": 295
+    },
+    {
+      "epoch": 75.0,
+      "grad_norm": 0.4717479646205902,
+      "learning_rate": 8.596792668957619e-05,
+      "loss": 0.1035,
+      "step": 300
+    },
+    {
+      "epoch": 76.32,
+      "grad_norm": 0.1413341760635376,
+      "learning_rate": 8.56815578465063e-05,
+      "loss": 0.1009,
+      "step": 305
+    },
+    {
+      "epoch": 77.64,
+      "grad_norm": 0.1581067591905594,
+      "learning_rate": 8.539518900343643e-05,
+      "loss": 0.1029,
+      "step": 310
+    },
+    {
+      "epoch": 78.96,
+      "grad_norm": 0.12125775218009949,
+      "learning_rate": 8.510882016036655e-05,
+      "loss": 0.102,
+      "step": 315
+    },
+    {
+      "epoch": 80.0,
+      "grad_norm": 0.309116005897522,
+      "learning_rate": 8.482245131729668e-05,
+      "loss": 0.102,
+      "step": 320
+    },
+    {
+      "epoch": 81.32,
+      "grad_norm": 0.12490837275981903,
+      "learning_rate": 8.453608247422681e-05,
+      "loss": 0.1036,
+      "step": 325
+    },
+    {
+      "epoch": 82.64,
+      "grad_norm": 0.15004275739192963,
+      "learning_rate": 8.424971363115693e-05,
+      "loss": 0.1005,
+      "step": 330
+    },
+    {
+      "epoch": 83.96,
+      "grad_norm": 0.10462955385446548,
+      "learning_rate": 8.396334478808706e-05,
+      "loss": 0.1012,
+      "step": 335
+    },
+    {
+      "epoch": 85.0,
+      "grad_norm": 1.216468095779419,
+      "learning_rate": 8.367697594501719e-05,
+      "loss": 0.1158,
+      "step": 340
+    },
+    {
+      "epoch": 86.32,
+      "grad_norm": 0.12770289182662964,
+      "learning_rate": 8.339060710194731e-05,
+      "loss": 0.103,
+      "step": 345
+    },
+    {
+      "epoch": 87.64,
+      "grad_norm": 0.11857649683952332,
+      "learning_rate": 8.310423825887744e-05,
+      "loss": 0.1024,
+      "step": 350
+    },
+    {
+      "epoch": 88.96,
+      "grad_norm": 0.15543484687805176,
+      "learning_rate": 8.281786941580757e-05,
+      "loss": 0.104,
+      "step": 355
+    },
+    {
+      "epoch": 90.0,
+      "grad_norm": 0.5385985970497131,
+      "learning_rate": 8.253150057273768e-05,
+      "loss": 0.1013,
+      "step": 360
+    },
+    {
+      "epoch": 91.32,
+      "grad_norm": 0.12159006297588348,
+      "learning_rate": 8.224513172966782e-05,
+      "loss": 0.1049,
+      "step": 365
+    },
+    {
+      "epoch": 92.64,
+      "grad_norm": 0.13037694990634918,
+      "learning_rate": 8.195876288659795e-05,
+      "loss": 0.1018,
+      "step": 370
+    },
+    {
+      "epoch": 93.96,
+      "grad_norm": 0.1268284171819687,
+      "learning_rate": 8.167239404352807e-05,
+      "loss": 0.1054,
+      "step": 375
+    },
+    {
+      "epoch": 95.0,
+      "grad_norm": 0.292937695980072,
+      "learning_rate": 8.13860252004582e-05,
+      "loss": 0.1023,
+      "step": 380
+    },
+    {
+      "epoch": 96.32,
+      "grad_norm": 0.13367846608161926,
+      "learning_rate": 8.109965635738833e-05,
+      "loss": 0.1025,
+      "step": 385
+    },
+    {
+      "epoch": 97.64,
+      "grad_norm": 0.09405620396137238,
+      "learning_rate": 8.081328751431845e-05,
+      "loss": 0.103,
+      "step": 390
+    },
+    {
+      "epoch": 98.96,
+      "grad_norm": 0.10122726112604141,
+      "learning_rate": 8.052691867124858e-05,
+      "loss": 0.106,
+      "step": 395
+    },
+    {
+      "epoch": 100.0,
+      "grad_norm": 0.3003501892089844,
+      "learning_rate": 8.02405498281787e-05,
+      "loss": 0.101,
+      "step": 400
+    },
+    {
+      "epoch": 101.32,
+      "grad_norm": 0.08715490251779556,
+      "learning_rate": 7.995418098510883e-05,
+      "loss": 0.1004,
+      "step": 405
+    },
+    {
+      "epoch": 102.64,
+      "grad_norm": 0.1160975769162178,
+      "learning_rate": 7.966781214203894e-05,
+      "loss": 0.1029,
+      "step": 410
+    },
+    {
+      "epoch": 103.96,
+      "grad_norm": 0.12515407800674438,
+      "learning_rate": 7.938144329896907e-05,
+      "loss": 0.1019,
+      "step": 415
+    },
+    {
+      "epoch": 105.0,
+      "grad_norm": 0.43526867032051086,
+      "learning_rate": 7.90950744558992e-05,
+      "loss": 0.1037,
+      "step": 420
+    },
+    {
+      "epoch": 106.32,
+      "grad_norm": 0.1083005964756012,
+      "learning_rate": 7.880870561282932e-05,
+      "loss": 0.1048,
+      "step": 425
+    },
+    {
+      "epoch": 107.64,
+      "grad_norm": 0.12727200984954834,
+      "learning_rate": 7.852233676975945e-05,
+      "loss": 0.1013,
+      "step": 430
+    },
+    {
+      "epoch": 108.96,
+      "grad_norm": 0.11788661777973175,
+      "learning_rate": 7.823596792668958e-05,
+      "loss": 0.1002,
+      "step": 435
+    },
+    {
+      "epoch": 110.0,
+      "grad_norm": 0.5509673357009888,
+      "learning_rate": 7.79495990836197e-05,
+      "loss": 0.1065,
+      "step": 440
+    },
+    {
+      "epoch": 111.32,
+      "grad_norm": 0.12785306572914124,
+      "learning_rate": 7.766323024054983e-05,
+      "loss": 0.1013,
+      "step": 445
+    },
+    {
+      "epoch": 112.64,
+      "grad_norm": 0.13988617062568665,
+      "learning_rate": 7.737686139747996e-05,
+      "loss": 0.1014,
+      "step": 450
+    },
+    {
+      "epoch": 113.96,
+      "grad_norm": 0.11280432343482971,
+      "learning_rate": 7.709049255441008e-05,
+      "loss": 0.1017,
+      "step": 455
+    },
+    {
+      "epoch": 115.0,
+      "grad_norm": 0.33109962940216064,
+      "learning_rate": 7.680412371134021e-05,
+      "loss": 0.0993,
+      "step": 460
+    },
+    {
+      "epoch": 116.32,
+      "grad_norm": 0.10653392225503922,
+      "learning_rate": 7.651775486827034e-05,
+      "loss": 0.1016,
+      "step": 465
+    },
+    {
+      "epoch": 117.64,
+      "grad_norm": 0.09670863300561905,
+      "learning_rate": 7.623138602520046e-05,
+      "loss": 0.1003,
+      "step": 470
+    },
+    {
+      "epoch": 118.96,
+      "grad_norm": 0.11432068794965744,
+      "learning_rate": 7.594501718213059e-05,
+      "loss": 0.1027,
+      "step": 475
+    },
+    {
+      "epoch": 120.0,
+      "grad_norm": 0.6067518591880798,
+      "learning_rate": 7.565864833906071e-05,
+      "loss": 0.1054,
+      "step": 480
+    },
+    {
+      "epoch": 121.32,
+      "grad_norm": 0.08575757592916489,
+      "learning_rate": 7.537227949599084e-05,
+      "loss": 0.0993,
+      "step": 485
+    },
+    {
+      "epoch": 122.64,
+      "grad_norm": 0.1070568859577179,
+      "learning_rate": 7.508591065292097e-05,
+      "loss": 0.0991,
+      "step": 490
+    },
+    {
+      "epoch": 123.96,
+      "grad_norm": 0.09047789871692657,
+      "learning_rate": 7.47995418098511e-05,
+      "loss": 0.1032,
+      "step": 495
+    },
+    {
+      "epoch": 125.0,
+      "grad_norm": 0.2450919896364212,
+      "learning_rate": 7.451317296678122e-05,
+      "loss": 0.1008,
+      "step": 500
+    },
+    {
+      "epoch": 126.32,
+      "grad_norm": 0.11366956681013107,
+      "learning_rate": 7.422680412371135e-05,
+      "loss": 0.099,
+      "step": 505
+    },
+    {
+      "epoch": 127.64,
+      "grad_norm": 0.10186628252267838,
+      "learning_rate": 7.394043528064147e-05,
+      "loss": 0.1012,
+      "step": 510
+    },
+    {
+      "epoch": 128.96,
+      "grad_norm": 0.1052548959851265,
+      "learning_rate": 7.36540664375716e-05,
+      "loss": 0.1013,
+      "step": 515
+    },
+    {
+      "epoch": 130.0,
+      "grad_norm": 0.3205568492412567,
+      "learning_rate": 7.336769759450171e-05,
+      "loss": 0.1006,
+      "step": 520
+    },
+    {
+      "epoch": 131.32,
+      "grad_norm": 0.07446248829364777,
+      "learning_rate": 7.308132875143184e-05,
+      "loss": 0.1012,
+      "step": 525
+    },
+    {
+      "epoch": 132.64,
+      "grad_norm": 0.08148901909589767,
+      "learning_rate": 7.279495990836197e-05,
+      "loss": 0.1012,
+      "step": 530
+    },
+    {
+      "epoch": 133.96,
+      "grad_norm": 0.11023813486099243,
+      "learning_rate": 7.250859106529209e-05,
+      "loss": 0.1001,
+      "step": 535
+    },
+    {
+      "epoch": 135.0,
+      "grad_norm": 0.32420921325683594,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.1017,
+      "step": 540
+    },
+    {
+      "epoch": 136.32,
+      "grad_norm": 0.08981231600046158,
+      "learning_rate": 7.193585337915235e-05,
+      "loss": 0.1019,
+      "step": 545
+    },
+    {
+      "epoch": 137.64,
+      "grad_norm": 0.1241590604186058,
+      "learning_rate": 7.164948453608247e-05,
+      "loss": 0.0994,
+      "step": 550
+    },
+    {
+      "epoch": 138.96,
+      "grad_norm": 0.12690334022045135,
+      "learning_rate": 7.136311569301261e-05,
+      "loss": 0.0998,
+      "step": 555
+    },
+    {
+      "epoch": 140.0,
+      "grad_norm": 0.26669779419898987,
+      "learning_rate": 7.107674684994274e-05,
+      "loss": 0.0995,
+      "step": 560
+    },
+    {
+      "epoch": 141.32,
+      "grad_norm": 0.10272305458784103,
+      "learning_rate": 7.079037800687286e-05,
+      "loss": 0.1032,
+      "step": 565
+    },
+    {
+      "epoch": 142.64,
+      "grad_norm": 0.10494570434093475,
+      "learning_rate": 7.050400916380299e-05,
+      "loss": 0.1015,
+      "step": 570
+    },
+    {
+      "epoch": 143.96,
+      "grad_norm": 0.11733629554510117,
+      "learning_rate": 7.02176403207331e-05,
+      "loss": 0.1026,
+      "step": 575
+    },
+    {
+      "epoch": 145.0,
+      "grad_norm": 0.18178793787956238,
+      "learning_rate": 6.993127147766323e-05,
+      "loss": 0.1009,
+      "step": 580
+    },
+    {
+      "epoch": 146.32,
+      "grad_norm": 0.07104074954986572,
+      "learning_rate": 6.964490263459336e-05,
+      "loss": 0.099,
+      "step": 585
+    },
+    {
+      "epoch": 147.64,
+      "grad_norm": 0.0909457802772522,
+      "learning_rate": 6.935853379152348e-05,
+      "loss": 0.1015,
+      "step": 590
+    },
+    {
+      "epoch": 148.96,
+      "grad_norm": 0.09944809228181839,
+      "learning_rate": 6.907216494845361e-05,
+      "loss": 0.1001,
+      "step": 595
+    },
+    {
+      "epoch": 150.0,
+      "grad_norm": 0.23125752806663513,
+      "learning_rate": 6.878579610538374e-05,
+      "loss": 0.0984,
+      "step": 600
+    },
+    {
+      "epoch": 151.32,
+      "grad_norm": 0.0829208642244339,
+      "learning_rate": 6.849942726231386e-05,
+      "loss": 0.0995,
+      "step": 605
+    },
+    {
+      "epoch": 152.64,
+      "grad_norm": 0.09419254958629608,
+      "learning_rate": 6.821305841924399e-05,
+      "loss": 0.1008,
+      "step": 610
+    },
+    {
+      "epoch": 153.96,
+      "grad_norm": 0.0779041051864624,
+      "learning_rate": 6.792668957617412e-05,
+      "loss": 0.0982,
+      "step": 615
+    },
+    {
+      "epoch": 155.0,
+      "grad_norm": 0.2598252594470978,
+      "learning_rate": 6.764032073310424e-05,
+      "loss": 0.1,
+      "step": 620
+    },
+    {
+      "epoch": 156.32,
+      "grad_norm": 0.08832815289497375,
+      "learning_rate": 6.735395189003437e-05,
+      "loss": 0.0989,
+      "step": 625
+    },
+    {
+      "epoch": 157.64,
+      "grad_norm": 0.07834926247596741,
+      "learning_rate": 6.706758304696448e-05,
+      "loss": 0.1016,
+      "step": 630
+    },
+    {
+      "epoch": 158.96,
+      "grad_norm": 0.08721348643302917,
+      "learning_rate": 6.678121420389462e-05,
+      "loss": 0.1002,
+      "step": 635
+    },
+    {
+      "epoch": 160.0,
+      "grad_norm": 0.22782059013843536,
+      "learning_rate": 6.649484536082475e-05,
+      "loss": 0.1003,
+      "step": 640
+    },
+    {
+      "epoch": 161.32,
+      "grad_norm": 0.07970811426639557,
+      "learning_rate": 6.620847651775487e-05,
+      "loss": 0.1,
+      "step": 645
+    },
+    {
+      "epoch": 162.64,
+      "grad_norm": 0.09135862439870834,
+      "learning_rate": 6.5922107674685e-05,
+      "loss": 0.1015,
+      "step": 650
+    },
+    {
+      "epoch": 163.96,
+      "grad_norm": 0.10493220388889313,
+      "learning_rate": 6.563573883161513e-05,
+      "loss": 0.1004,
+      "step": 655
+    },
+    {
+      "epoch": 165.0,
+      "grad_norm": 0.3030238747596741,
+      "learning_rate": 6.534936998854525e-05,
+      "loss": 0.1023,
+      "step": 660
+    },
+    {
+      "epoch": 166.32,
+      "grad_norm": 0.09718652069568634,
+      "learning_rate": 6.506300114547538e-05,
+      "loss": 0.101,
+      "step": 665
+    },
+    {
+      "epoch": 167.64,
+      "grad_norm": 0.06749033182859421,
+      "learning_rate": 6.477663230240551e-05,
+      "loss": 0.1008,
+      "step": 670
+    },
+    {
+      "epoch": 168.96,
+      "grad_norm": 0.08545351028442383,
+      "learning_rate": 6.449026345933563e-05,
+      "loss": 0.1021,
+      "step": 675
+    },
+    {
+      "epoch": 170.0,
+      "grad_norm": 0.25453537702560425,
+      "learning_rate": 6.420389461626576e-05,
+      "loss": 0.0976,
+      "step": 680
+    },
+    {
+      "epoch": 171.32,
+      "grad_norm": 0.07819271087646484,
+      "learning_rate": 6.391752577319587e-05,
+      "loss": 0.1014,
+      "step": 685
+    },
+    {
+      "epoch": 172.64,
+      "grad_norm": 0.07297082990407944,
+      "learning_rate": 6.3631156930126e-05,
+      "loss": 0.1023,
+      "step": 690
+    },
+    {
+      "epoch": 173.96,
+      "grad_norm": 0.08446590602397919,
+      "learning_rate": 6.334478808705613e-05,
+      "loss": 0.1002,
+      "step": 695
+    },
+    {
+      "epoch": 175.0,
+      "grad_norm": 0.21633073687553406,
+      "learning_rate": 6.305841924398625e-05,
+      "loss": 0.0992,
+      "step": 700
+    },
+    {
+      "epoch": 176.32,
+      "grad_norm": 0.07673562318086624,
+      "learning_rate": 6.277205040091638e-05,
+      "loss": 0.0995,
+      "step": 705
+    },
+    {
+      "epoch": 177.64,
+      "grad_norm": 0.09338875859975815,
+      "learning_rate": 6.24856815578465e-05,
+      "loss": 0.1014,
+      "step": 710
+    },
+    {
+      "epoch": 178.96,
+      "grad_norm": 0.09649301320314407,
+      "learning_rate": 6.219931271477663e-05,
+      "loss": 0.1006,
+      "step": 715
+    },
+    {
+      "epoch": 180.0,
+      "grad_norm": 0.2125590741634369,
+      "learning_rate": 6.191294387170676e-05,
+      "loss": 0.0996,
+      "step": 720
+    },
+    {
+      "epoch": 181.32,
+      "grad_norm": 0.07358326762914658,
+      "learning_rate": 6.162657502863689e-05,
+      "loss": 0.1006,
+      "step": 725
+    },
+    {
+      "epoch": 182.64,
+      "grad_norm": 0.07445739209651947,
+      "learning_rate": 6.134020618556701e-05,
+      "loss": 0.0984,
+      "step": 730
+    },
+    {
+      "epoch": 183.96,
+      "grad_norm": 0.07530005276203156,
+      "learning_rate": 6.105383734249714e-05,
+      "loss": 0.0999,
+      "step": 735
+    },
+    {
+      "epoch": 185.0,
+      "grad_norm": 0.29669082164764404,
+      "learning_rate": 6.076746849942726e-05,
+      "loss": 0.102,
+      "step": 740
+    },
+    {
+      "epoch": 186.32,
+      "grad_norm": 0.09269877523183823,
+      "learning_rate": 6.0481099656357384e-05,
+      "loss": 0.0997,
+      "step": 745
+    },
+    {
+      "epoch": 187.64,
+      "grad_norm": 0.0804276242852211,
+      "learning_rate": 6.019473081328752e-05,
+      "loss": 0.1014,
+      "step": 750
+    },
+    {
+      "epoch": 188.96,
+      "grad_norm": 0.098207488656044,
+      "learning_rate": 5.9908361970217644e-05,
+      "loss": 0.0988,
+      "step": 755
+    },
+    {
+      "epoch": 190.0,
+      "grad_norm": 0.19817565381526947,
+      "learning_rate": 5.962199312714777e-05,
+      "loss": 0.1001,
+      "step": 760
+    },
+    {
+      "epoch": 191.32,
+      "grad_norm": 0.06558432430028915,
+      "learning_rate": 5.93356242840779e-05,
+      "loss": 0.1006,
+      "step": 765
+    },
+    {
+      "epoch": 192.64,
+      "grad_norm": 0.0970512330532074,
+      "learning_rate": 5.904925544100802e-05,
+      "loss": 0.1011,
+      "step": 770
+    },
+    {
+      "epoch": 193.96,
+      "grad_norm": 0.07481536269187927,
+      "learning_rate": 5.876288659793815e-05,
+      "loss": 0.0988,
+      "step": 775
+    },
+    {
+      "epoch": 195.0,
+      "grad_norm": 0.2697769105434418,
+      "learning_rate": 5.8476517754868276e-05,
+      "loss": 0.1008,
+      "step": 780
+    },
+    {
+      "epoch": 196.32,
+      "grad_norm": 0.08228090405464172,
+      "learning_rate": 5.81901489117984e-05,
+      "loss": 0.0999,
+      "step": 785
+    },
+    {
+      "epoch": 197.64,
+      "grad_norm": 0.08324407786130905,
+      "learning_rate": 5.790378006872853e-05,
+      "loss": 0.0999,
+      "step": 790
+    },
+    {
+      "epoch": 198.96,
+      "grad_norm": 0.0824645459651947,
+      "learning_rate": 5.761741122565865e-05,
+      "loss": 0.1011,
+      "step": 795
+    },
+    {
+      "epoch": 200.0,
+      "grad_norm": 0.2783829867839813,
+      "learning_rate": 5.7331042382588775e-05,
+      "loss": 0.101,
+      "step": 800
+    },
+    {
+      "epoch": 201.32,
+      "grad_norm": 0.07741741836071014,
+      "learning_rate": 5.70446735395189e-05,
+      "loss": 0.1011,
+      "step": 805
+    },
+    {
+      "epoch": 202.64,
+      "grad_norm": 0.06530506908893585,
+      "learning_rate": 5.675830469644903e-05,
+      "loss": 0.0997,
+      "step": 810
+    },
+    {
+      "epoch": 203.96,
+      "grad_norm": 0.06250391900539398,
+      "learning_rate": 5.6471935853379155e-05,
+      "loss": 0.1013,
+      "step": 815
+    },
+    {
+      "epoch": 205.0,
+      "grad_norm": 0.3014598488807678,
+      "learning_rate": 5.618556701030928e-05,
+      "loss": 0.0994,
+      "step": 820
+    },
+    {
+      "epoch": 206.32,
+      "grad_norm": 0.06866947561502457,
+      "learning_rate": 5.589919816723941e-05,
+      "loss": 0.0971,
+      "step": 825
+    },
+    {
+      "epoch": 207.64,
+      "grad_norm": 0.08845752477645874,
+      "learning_rate": 5.5612829324169534e-05,
+      "loss": 0.1009,
+      "step": 830
+    },
+    {
+      "epoch": 208.96,
+      "grad_norm": 0.08089124411344528,
+      "learning_rate": 5.532646048109966e-05,
+      "loss": 0.0993,
+      "step": 835
+    },
+    {
+      "epoch": 210.0,
+      "grad_norm": 0.30784446001052856,
+      "learning_rate": 5.504009163802979e-05,
+      "loss": 0.0992,
+      "step": 840
+    },
+    {
+      "epoch": 211.32,
+      "grad_norm": 0.06972301006317139,
+      "learning_rate": 5.4753722794959914e-05,
+      "loss": 0.1,
+      "step": 845
+    },
+    {
+      "epoch": 212.64,
+      "grad_norm": 0.07285023480653763,
+      "learning_rate": 5.4467353951890033e-05,
+      "loss": 0.0993,
+      "step": 850
+    },
+    {
+      "epoch": 213.96,
+      "grad_norm": 0.060395676642656326,
+      "learning_rate": 5.418098510882016e-05,
+      "loss": 0.0989,
+      "step": 855
+    },
+    {
+      "epoch": 215.0,
+      "grad_norm": 0.17777375876903534,
+      "learning_rate": 5.3894616265750286e-05,
+      "loss": 0.1002,
+      "step": 860
+    },
+    {
+      "epoch": 216.32,
+      "grad_norm": 0.07913000881671906,
+      "learning_rate": 5.360824742268041e-05,
+      "loss": 0.0985,
+      "step": 865
+    },
+    {
+      "epoch": 217.64,
+      "grad_norm": 0.08352327346801758,
+      "learning_rate": 5.332187857961054e-05,
+      "loss": 0.0994,
+      "step": 870
+    },
+    {
+      "epoch": 218.96,
+      "grad_norm": 0.07089956849813461,
+      "learning_rate": 5.3035509736540666e-05,
+      "loss": 0.0993,
+      "step": 875
+    },
+    {
+      "epoch": 220.0,
+      "grad_norm": 0.27649030089378357,
+      "learning_rate": 5.274914089347079e-05,
+      "loss": 0.1013,
+      "step": 880
+    },
+    {
+      "epoch": 221.32,
+      "grad_norm": 0.08725525438785553,
+      "learning_rate": 5.246277205040092e-05,
+      "loss": 0.0998,
+      "step": 885
+    },
+    {
+      "epoch": 222.64,
+      "grad_norm": 0.06686926633119583,
+      "learning_rate": 5.2176403207331045e-05,
+      "loss": 0.1004,
+      "step": 890
+    },
+    {
+      "epoch": 223.96,
+      "grad_norm": 0.0881044864654541,
+      "learning_rate": 5.189003436426118e-05,
+      "loss": 0.0993,
+      "step": 895
+    },
+    {
+      "epoch": 225.0,
+      "grad_norm": 0.2503332793712616,
+      "learning_rate": 5.1603665521191305e-05,
+      "loss": 0.1002,
+      "step": 900
+    },
+    {
+      "epoch": 226.32,
+      "grad_norm": 0.0743650272488594,
+      "learning_rate": 5.131729667812142e-05,
+      "loss": 0.1,
+      "step": 905
+    },
+    {
+      "epoch": 227.64,
+      "grad_norm": 0.06994261592626572,
+      "learning_rate": 5.1030927835051544e-05,
+      "loss": 0.099,
+      "step": 910
+    },
+    {
+      "epoch": 228.96,
+      "grad_norm": 0.07234133034944534,
+      "learning_rate": 5.074455899198167e-05,
+      "loss": 0.0993,
+      "step": 915
+    },
+    {
+      "epoch": 230.0,
+      "grad_norm": 0.22859309613704681,
+      "learning_rate": 5.04581901489118e-05,
+      "loss": 0.1013,
+      "step": 920
+    },
+    {
+      "epoch": 231.32,
+      "grad_norm": 0.07225388288497925,
+      "learning_rate": 5.0171821305841924e-05,
+      "loss": 0.0984,
+      "step": 925
+    },
+    {
+      "epoch": 232.64,
+      "grad_norm": 0.08787540346384048,
+      "learning_rate": 4.988545246277205e-05,
+      "loss": 0.1013,
+      "step": 930
+    },
+    {
+      "epoch": 233.96,
+      "grad_norm": 0.07095170021057129,
+      "learning_rate": 4.9599083619702184e-05,
+      "loss": 0.0993,
+      "step": 935
+    },
+    {
+      "epoch": 235.0,
+      "grad_norm": 0.36082205176353455,
+      "learning_rate": 4.931271477663231e-05,
+      "loss": 0.1009,
+      "step": 940
+    },
+    {
+      "epoch": 236.32,
+      "grad_norm": 0.07094796001911163,
+      "learning_rate": 4.902634593356243e-05,
+      "loss": 0.0982,
+      "step": 945
+    },
+    {
+      "epoch": 237.64,
+      "grad_norm": 0.061540842056274414,
+      "learning_rate": 4.8739977090492556e-05,
+      "loss": 0.1005,
+      "step": 950
+    },
+    {
+      "epoch": 238.96,
+      "grad_norm": 0.0834394097328186,
+      "learning_rate": 4.845360824742268e-05,
+      "loss": 0.1003,
+      "step": 955
+    },
+    {
+      "epoch": 240.0,
+      "grad_norm": 0.24069315195083618,
+      "learning_rate": 4.816723940435281e-05,
+      "loss": 0.0989,
+      "step": 960
+    },
+    {
+      "epoch": 241.32,
+      "grad_norm": 0.0824229046702385,
+      "learning_rate": 4.7880870561282936e-05,
+      "loss": 0.0987,
+      "step": 965
+    },
+    {
+      "epoch": 242.64,
+      "grad_norm": 0.07148510217666626,
+      "learning_rate": 4.7594501718213055e-05,
+      "loss": 0.1003,
+      "step": 970
+    },
+    {
+      "epoch": 243.96,
+      "grad_norm": 0.07550371438264847,
+      "learning_rate": 4.730813287514318e-05,
+      "loss": 0.1011,
+      "step": 975
+    },
+    {
+      "epoch": 245.0,
+      "grad_norm": 0.24366816878318787,
+      "learning_rate": 4.7021764032073315e-05,
+      "loss": 0.1001,
+      "step": 980
+    },
+    {
+      "epoch": 246.32,
+      "grad_norm": 0.08431750535964966,
+      "learning_rate": 4.673539518900344e-05,
+      "loss": 0.0996,
+      "step": 985
+    },
+    {
+      "epoch": 247.64,
+      "grad_norm": 0.07557923346757889,
+      "learning_rate": 4.644902634593357e-05,
+      "loss": 0.0999,
+      "step": 990
+    },
+    {
+      "epoch": 248.96,
+      "grad_norm": 0.07260127365589142,
+      "learning_rate": 4.6162657502863694e-05,
+      "loss": 0.0994,
+      "step": 995
+    },
+    {
+      "epoch": 250.0,
+      "grad_norm": 0.226700559258461,
+      "learning_rate": 4.5876288659793814e-05,
+      "loss": 0.0997,
+      "step": 1000
+    },
+    {
+      "epoch": 251.32,
+      "grad_norm": 0.062034279108047485,
+      "learning_rate": 4.558991981672394e-05,
+      "loss": 0.0996,
+      "step": 1005
+    },
+    {
+      "epoch": 252.64,
+      "grad_norm": 0.07869929820299149,
+      "learning_rate": 4.530355097365407e-05,
+      "loss": 0.1016,
+      "step": 1010
+    },
+    {
+      "epoch": 253.96,
+      "grad_norm": 0.07183313369750977,
+      "learning_rate": 4.5017182130584194e-05,
+      "loss": 0.0984,
+      "step": 1015
+    },
+    {
+      "epoch": 255.0,
+      "grad_norm": 0.22644160687923431,
+      "learning_rate": 4.473081328751432e-05,
+      "loss": 0.0983,
+      "step": 1020
+    },
+    {
+      "epoch": 256.32,
+      "grad_norm": 0.06311376392841339,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 0.1001,
+      "step": 1025
+    },
+    {
+      "epoch": 257.64,
+      "grad_norm": 0.07967089116573334,
+      "learning_rate": 4.415807560137457e-05,
+      "loss": 0.098,
+      "step": 1030
+    },
+    {
+      "epoch": 258.96,
+      "grad_norm": 0.07578577101230621,
+      "learning_rate": 4.38717067583047e-05,
+      "loss": 0.0999,
+      "step": 1035
+    },
+    {
+      "epoch": 260.0,
+      "grad_norm": 0.22114117443561554,
+      "learning_rate": 4.3585337915234826e-05,
+      "loss": 0.0999,
+      "step": 1040
+    },
+    {
+      "epoch": 261.32,
+      "grad_norm": 0.07699496299028397,
+      "learning_rate": 4.329896907216495e-05,
+      "loss": 0.0988,
+      "step": 1045
+    },
+    {
+      "epoch": 262.64,
+      "grad_norm": 0.07236776500940323,
+      "learning_rate": 4.301260022909508e-05,
+      "loss": 0.0998,
+      "step": 1050
+    },
+    {
+      "epoch": 263.96,
+      "grad_norm": 0.0862107202410698,
+      "learning_rate": 4.27262313860252e-05,
+      "loss": 0.0997,
+      "step": 1055
+    },
+    {
+      "epoch": 265.0,
+      "grad_norm": 0.2598220109939575,
+      "learning_rate": 4.2439862542955325e-05,
+      "loss": 0.0988,
+      "step": 1060
+    },
+    {
+      "epoch": 266.32,
+      "grad_norm": 0.0842781811952591,
+      "learning_rate": 4.215349369988545e-05,
+      "loss": 0.0989,
+      "step": 1065
+    },
+    {
+      "epoch": 267.64,
+      "grad_norm": 0.06537719070911407,
+      "learning_rate": 4.1867124856815585e-05,
+      "loss": 0.0977,
+      "step": 1070
+    },
+    {
+      "epoch": 268.96,
+      "grad_norm": 0.06625049561262131,
+      "learning_rate": 4.158075601374571e-05,
+      "loss": 0.0994,
+      "step": 1075
+    },
+    {
+      "epoch": 270.0,
+      "grad_norm": 0.20921845734119415,
+      "learning_rate": 4.129438717067583e-05,
+      "loss": 0.0973,
+      "step": 1080
+    },
+    {
+      "epoch": 271.32,
+      "grad_norm": 0.06545858085155487,
+      "learning_rate": 4.100801832760596e-05,
+      "loss": 0.1,
+      "step": 1085
+    },
+    {
+      "epoch": 272.64,
+      "grad_norm": 0.08670192956924438,
+      "learning_rate": 4.0721649484536084e-05,
+      "loss": 0.0985,
+      "step": 1090
+    },
+    {
+      "epoch": 273.96,
+      "grad_norm": 0.07144106179475784,
+      "learning_rate": 4.043528064146621e-05,
+      "loss": 0.0986,
+      "step": 1095
+    },
+    {
+      "epoch": 275.0,
+      "grad_norm": 0.1721893697977066,
+      "learning_rate": 4.014891179839634e-05,
+      "loss": 0.0974,
+      "step": 1100
+    },
+    {
+      "epoch": 276.32,
+      "grad_norm": 0.07914085686206818,
+      "learning_rate": 3.9862542955326463e-05,
+      "loss": 0.0988,
+      "step": 1105
+    },
+    {
+      "epoch": 277.64,
+      "grad_norm": 0.07200506329536438,
+      "learning_rate": 3.957617411225659e-05,
+      "loss": 0.0996,
+      "step": 1110
+    },
+    {
+      "epoch": 278.96,
+      "grad_norm": 0.08419088274240494,
+      "learning_rate": 3.9289805269186716e-05,
+      "loss": 0.099,
+      "step": 1115
+    },
+    {
+      "epoch": 280.0,
+      "grad_norm": 0.22748175263404846,
+      "learning_rate": 3.900343642611684e-05,
+      "loss": 0.1005,
+      "step": 1120
+    },
+    {
+      "epoch": 281.32,
+      "grad_norm": 0.06296680122613907,
+      "learning_rate": 3.871706758304697e-05,
+      "loss": 0.0996,
+      "step": 1125
+    },
+    {
+      "epoch": 282.64,
+      "grad_norm": 0.0694783627986908,
+      "learning_rate": 3.8430698739977096e-05,
+      "loss": 0.0995,
+      "step": 1130
+    },
+    {
+      "epoch": 283.96,
+      "grad_norm": 0.06090915948152542,
+      "learning_rate": 3.8144329896907216e-05,
+      "loss": 0.0989,
+      "step": 1135
+    },
+    {
+      "epoch": 285.0,
+      "grad_norm": 0.22922737896442413,
+      "learning_rate": 3.785796105383734e-05,
+      "loss": 0.0996,
+      "step": 1140
+    },
+    {
+      "epoch": 286.32,
+      "grad_norm": 0.06500030308961868,
+      "learning_rate": 3.757159221076747e-05,
+      "loss": 0.0994,
+      "step": 1145
+    },
+    {
+      "epoch": 287.64,
+      "grad_norm": 0.06274694204330444,
+      "learning_rate": 3.7285223367697595e-05,
+      "loss": 0.0979,
+      "step": 1150
+    },
+    {
+      "epoch": 288.96,
+      "grad_norm": 0.07129135727882385,
+      "learning_rate": 3.699885452462772e-05,
+      "loss": 0.0981,
+      "step": 1155
+    },
+    {
+      "epoch": 290.0,
+      "grad_norm": 0.22372718155384064,
+      "learning_rate": 3.671248568155785e-05,
+      "loss": 0.1004,
+      "step": 1160
+    },
+    {
+      "epoch": 291.32,
+      "grad_norm": 0.062255196273326874,
+      "learning_rate": 3.6426116838487974e-05,
+      "loss": 0.0983,
+      "step": 1165
+    },
+    {
+      "epoch": 292.64,
+      "grad_norm": 0.052688661962747574,
+      "learning_rate": 3.61397479954181e-05,
+      "loss": 0.0997,
+      "step": 1170
+    },
+    {
+      "epoch": 293.96,
+      "grad_norm": 0.06491158157587051,
+      "learning_rate": 3.585337915234823e-05,
+      "loss": 0.1013,
+      "step": 1175
+    },
+    {
+      "epoch": 295.0,
+      "grad_norm": 0.19028101861476898,
+      "learning_rate": 3.5567010309278354e-05,
+      "loss": 0.1,
+      "step": 1180
+    },
+    {
+      "epoch": 296.32,
+      "grad_norm": 0.063275046646595,
+      "learning_rate": 3.528064146620848e-05,
+      "loss": 0.0991,
+      "step": 1185
+    },
+    {
+      "epoch": 297.64,
+      "grad_norm": 0.07221543043851852,
+      "learning_rate": 3.49942726231386e-05,
+      "loss": 0.0992,
+      "step": 1190
+    },
+    {
+      "epoch": 298.96,
+      "grad_norm": 0.06164846569299698,
+      "learning_rate": 3.4707903780068726e-05,
+      "loss": 0.0987,
+      "step": 1195
+    },
+    {
+      "epoch": 300.0,
+      "grad_norm": 0.2711406350135803,
+      "learning_rate": 3.442153493699885e-05,
+      "loss": 0.0997,
+      "step": 1200
+    },
+    {
+      "epoch": 301.32,
+      "grad_norm": 0.06757987290620804,
+      "learning_rate": 3.4135166093928986e-05,
+      "loss": 0.0988,
+      "step": 1205
+    },
+    {
+      "epoch": 302.64,
+      "grad_norm": 0.07149416208267212,
+      "learning_rate": 3.384879725085911e-05,
+      "loss": 0.0983,
+      "step": 1210
+    },
+    {
+      "epoch": 303.96,
+      "grad_norm": 0.06529082357883453,
+      "learning_rate": 3.356242840778923e-05,
+      "loss": 0.0991,
+      "step": 1215
+    },
+    {
+      "epoch": 305.0,
+      "grad_norm": 0.2772134840488434,
+      "learning_rate": 3.327605956471936e-05,
+      "loss": 0.0981,
+      "step": 1220
+    },
+    {
+      "epoch": 306.32,
+      "grad_norm": 0.07956410199403763,
+      "learning_rate": 3.2989690721649485e-05,
+      "loss": 0.1002,
+      "step": 1225
+    },
+    {
+      "epoch": 307.64,
+      "grad_norm": 0.061629608273506165,
+      "learning_rate": 3.270332187857961e-05,
+      "loss": 0.0989,
+      "step": 1230
+    },
+    {
+      "epoch": 308.96,
+      "grad_norm": 0.06648664176464081,
+      "learning_rate": 3.241695303550974e-05,
+      "loss": 0.0995,
+      "step": 1235
+    },
+    {
+      "epoch": 310.0,
+      "grad_norm": 0.2157808542251587,
+      "learning_rate": 3.2130584192439865e-05,
+      "loss": 0.0995,
+      "step": 1240
+    },
+    {
+      "epoch": 311.32,
+      "grad_norm": 0.057500071823596954,
+      "learning_rate": 3.184421534936999e-05,
+      "loss": 0.0995,
+      "step": 1245
+    },
+    {
+      "epoch": 312.64,
+      "grad_norm": 0.0696285292506218,
+      "learning_rate": 3.155784650630012e-05,
+      "loss": 0.0988,
+      "step": 1250
+    },
+    {
+      "epoch": 313.96,
+      "grad_norm": 0.06828924268484116,
+      "learning_rate": 3.1271477663230244e-05,
+      "loss": 0.0982,
+      "step": 1255
+    },
+    {
+      "epoch": 315.0,
+      "grad_norm": 0.22276295721530914,
+      "learning_rate": 3.098510882016037e-05,
+      "loss": 0.1008,
+      "step": 1260
+    },
+    {
+      "epoch": 316.32,
+      "grad_norm": 0.06957324594259262,
+      "learning_rate": 3.06987399770905e-05,
+      "loss": 0.0983,
+      "step": 1265
+    },
+    {
+      "epoch": 317.64,
+      "grad_norm": 0.07875330001115799,
+      "learning_rate": 3.0412371134020617e-05,
+      "loss": 0.0988,
+      "step": 1270
+    },
+    {
+      "epoch": 318.96,
+      "grad_norm": 0.06879551708698273,
+      "learning_rate": 3.0126002290950743e-05,
+      "loss": 0.0974,
+      "step": 1275
+    },
+    {
+      "epoch": 320.0,
+      "grad_norm": 0.18313516676425934,
+      "learning_rate": 2.983963344788087e-05,
+      "loss": 0.0981,
+      "step": 1280
+    },
+    {
+      "epoch": 321.32,
+      "grad_norm": 0.060802001506090164,
+      "learning_rate": 2.9553264604811e-05,
+      "loss": 0.0997,
+      "step": 1285
+    },
+    {
+      "epoch": 322.64,
+      "grad_norm": 0.06690306961536407,
+      "learning_rate": 2.9266895761741126e-05,
+      "loss": 0.0998,
+      "step": 1290
+    },
+    {
+      "epoch": 323.96,
+      "grad_norm": 0.0787249505519867,
+      "learning_rate": 2.8980526918671253e-05,
+      "loss": 0.0984,
+      "step": 1295
+    },
+    {
+      "epoch": 325.0,
+      "grad_norm": 0.21499867737293243,
+      "learning_rate": 2.8694158075601372e-05,
+      "loss": 0.0994,
+      "step": 1300
+    },
+    {
+      "epoch": 326.32,
+      "grad_norm": 0.06578684598207474,
+      "learning_rate": 2.8407789232531502e-05,
+      "loss": 0.0994,
+      "step": 1305
+    },
+    {
+      "epoch": 327.64,
+      "grad_norm": 0.07056637853384018,
+      "learning_rate": 2.812142038946163e-05,
+      "loss": 0.0998,
+      "step": 1310
+    },
+    {
+      "epoch": 328.96,
+      "grad_norm": 0.06579981744289398,
+      "learning_rate": 2.7835051546391755e-05,
+      "loss": 0.0989,
+      "step": 1315
+    },
+    {
+      "epoch": 330.0,
+      "grad_norm": 0.2260318398475647,
+      "learning_rate": 2.754868270332188e-05,
+      "loss": 0.0983,
+      "step": 1320
+    },
+    {
+      "epoch": 331.32,
+      "grad_norm": 0.06875937432050705,
+      "learning_rate": 2.7262313860252005e-05,
+      "loss": 0.0993,
+      "step": 1325
+    },
+    {
+      "epoch": 332.64,
+      "grad_norm": 0.06524664908647537,
+      "learning_rate": 2.697594501718213e-05,
+      "loss": 0.0992,
+      "step": 1330
+    },
+    {
+      "epoch": 333.96,
+      "grad_norm": 0.06697370857000351,
+      "learning_rate": 2.6689576174112258e-05,
+      "loss": 0.0992,
+      "step": 1335
+    },
+    {
+      "epoch": 335.0,
+      "grad_norm": 0.22555196285247803,
+      "learning_rate": 2.6403207331042384e-05,
+      "loss": 0.0996,
+      "step": 1340
+    },
+    {
+      "epoch": 336.32,
+      "grad_norm": 0.0852067843079567,
+      "learning_rate": 2.611683848797251e-05,
+      "loss": 0.0975,
+      "step": 1345
+    },
+    {
+      "epoch": 337.64,
+      "grad_norm": 0.07985348999500275,
+      "learning_rate": 2.5830469644902637e-05,
+      "loss": 0.0991,
+      "step": 1350
+    },
+    {
+      "epoch": 338.96,
+      "grad_norm": 0.05920860171318054,
+      "learning_rate": 2.554410080183276e-05,
+      "loss": 0.0995,
+      "step": 1355
+    },
+    {
+      "epoch": 340.0,
+      "grad_norm": 0.27031657099723816,
+      "learning_rate": 2.5257731958762887e-05,
+      "loss": 0.0987,
+      "step": 1360
+    },
+    {
+      "epoch": 341.32,
+      "grad_norm": 0.06413642317056656,
+      "learning_rate": 2.4971363115693013e-05,
+      "loss": 0.0987,
+      "step": 1365
+    },
+    {
+      "epoch": 342.64,
+      "grad_norm": 0.0787920281291008,
+      "learning_rate": 2.468499427262314e-05,
+      "loss": 0.0986,
+      "step": 1370
+    },
+    {
+      "epoch": 343.96,
+      "grad_norm": 0.0672549158334732,
+      "learning_rate": 2.4398625429553266e-05,
+      "loss": 0.0987,
+      "step": 1375
+    },
+    {
+      "epoch": 345.0,
+      "grad_norm": 0.27731525897979736,
+      "learning_rate": 2.4112256586483393e-05,
+      "loss": 0.0996,
+      "step": 1380
+    },
+    {
+      "epoch": 346.32,
+      "grad_norm": 0.06739719212055206,
+      "learning_rate": 2.3825887743413516e-05,
+      "loss": 0.0998,
+      "step": 1385
+    },
+    {
+      "epoch": 347.64,
+      "grad_norm": 0.07705069333314896,
+      "learning_rate": 2.3539518900343642e-05,
+      "loss": 0.0979,
+      "step": 1390
+    },
+    {
+      "epoch": 348.96,
+      "grad_norm": 0.07613033801317215,
+      "learning_rate": 2.3253150057273772e-05,
+      "loss": 0.0976,
+      "step": 1395
+    },
+    {
+      "epoch": 350.0,
+      "grad_norm": 0.26018571853637695,
+      "learning_rate": 2.2966781214203895e-05,
+      "loss": 0.0999,
+      "step": 1400
+    },
+    {
+      "epoch": 351.32,
+      "grad_norm": 0.06443151086568832,
+      "learning_rate": 2.268041237113402e-05,
+      "loss": 0.0984,
+      "step": 1405
+    },
+    {
+      "epoch": 352.64,
+      "grad_norm": 0.07143900543451309,
+      "learning_rate": 2.2394043528064148e-05,
+      "loss": 0.0987,
+      "step": 1410
+    },
+    {
+      "epoch": 353.96,
+      "grad_norm": 0.06685233861207962,
+      "learning_rate": 2.210767468499427e-05,
+      "loss": 0.099,
+      "step": 1415
+    },
+    {
+      "epoch": 355.0,
+      "grad_norm": 0.18464432656764984,
+      "learning_rate": 2.18213058419244e-05,
+      "loss": 0.1005,
+      "step": 1420
+    },
+    {
+      "epoch": 356.32,
+      "grad_norm": 0.06670142710208893,
+      "learning_rate": 2.1534936998854528e-05,
+      "loss": 0.0983,
+      "step": 1425
+    },
+    {
+      "epoch": 357.64,
+      "grad_norm": 0.08404285460710526,
+      "learning_rate": 2.124856815578465e-05,
+      "loss": 0.0995,
+      "step": 1430
+    },
+    {
+      "epoch": 358.96,
+      "grad_norm": 0.06260077655315399,
+      "learning_rate": 2.0962199312714777e-05,
+      "loss": 0.099,
+      "step": 1435
+    },
+    {
+      "epoch": 360.0,
+      "grad_norm": 0.22142066061496735,
+      "learning_rate": 2.0675830469644904e-05,
+      "loss": 0.0972,
+      "step": 1440
+    },
+    {
+      "epoch": 361.32,
+      "grad_norm": 0.0707300528883934,
+      "learning_rate": 2.038946162657503e-05,
+      "loss": 0.0987,
+      "step": 1445
+    },
+    {
+      "epoch": 362.64,
+      "grad_norm": 0.06083051487803459,
+      "learning_rate": 2.0103092783505157e-05,
+      "loss": 0.0988,
+      "step": 1450
+    },
+    {
+      "epoch": 363.96,
+      "grad_norm": 0.06447107344865799,
+      "learning_rate": 1.981672394043528e-05,
+      "loss": 0.0998,
+      "step": 1455
+    },
+    {
+      "epoch": 365.0,
+      "grad_norm": 0.2644057869911194,
+      "learning_rate": 1.9530355097365406e-05,
+      "loss": 0.0969,
+      "step": 1460
+    },
+    {
+      "epoch": 366.32,
+      "grad_norm": 0.06638660281896591,
+      "learning_rate": 1.9243986254295536e-05,
+      "loss": 0.098,
+      "step": 1465
+    },
+    {
+      "epoch": 367.64,
+      "grad_norm": 0.08120372146368027,
+      "learning_rate": 1.895761741122566e-05,
+      "loss": 0.0975,
+      "step": 1470
+    },
+    {
+      "epoch": 368.96,
+      "grad_norm": 0.07163622975349426,
+      "learning_rate": 1.8671248568155786e-05,
+      "loss": 0.0988,
+      "step": 1475
+    },
+    {
+      "epoch": 370.0,
+      "grad_norm": 0.268240749835968,
+      "learning_rate": 1.8384879725085912e-05,
+      "loss": 0.0952,
+      "step": 1480
+    },
+    {
+      "epoch": 371.32,
+      "grad_norm": 0.08139947801828384,
+      "learning_rate": 1.809851088201604e-05,
+      "loss": 0.0987,
+      "step": 1485
+    },
+    {
+      "epoch": 372.64,
+      "grad_norm": 0.0629943311214447,
+      "learning_rate": 1.7812142038946165e-05,
+      "loss": 0.0979,
+      "step": 1490
+    },
+    {
+      "epoch": 373.96,
+      "grad_norm": 0.06232031434774399,
+      "learning_rate": 1.7525773195876288e-05,
+      "loss": 0.1005,
+      "step": 1495
+    },
+    {
+      "epoch": 375.0,
+      "grad_norm": 0.24264411628246307,
+      "learning_rate": 1.7239404352806415e-05,
+      "loss": 0.0994,
+      "step": 1500
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 450,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.275678425088e+18,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-300/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-300/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-600/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/\u4e0d\u51bb\u7ed3Qwen\u8bad\u7ec3/models/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "gate_proj",
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_00/checkpoints/checkpoint-900/trainer_state.json

@@ -0,0 +1,1301 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 225.0,
+  "eval_steps": 500,
+  "global_step": 900,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.32,
+      "grad_norm": 17.919572830200195,
+      "learning_rate": 0.0,
+      "loss": 3.1974,
+      "step": 1
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 15.281089782714844,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 3.1689,
+      "step": 5
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 10.974032402038574,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.6104,
+      "step": 10
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 6.320477485656738,
+      "learning_rate": 2.5925925925925925e-05,
+      "loss": 1.5232,
+      "step": 15
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 3.79914927482605,
+      "learning_rate": 3.518518518518519e-05,
+      "loss": 0.6024,
+      "step": 20
+    },
+    {
+      "epoch": 6.32,
+      "grad_norm": 1.5797882080078125,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 0.3262,
+      "step": 25
+    },
+    {
+      "epoch": 7.64,
+      "grad_norm": 1.0163109302520752,
+      "learning_rate": 5.370370370370371e-05,
+      "loss": 0.3026,
+      "step": 30
+    },
+    {
+      "epoch": 8.96,
+      "grad_norm": 0.6873035430908203,
+      "learning_rate": 6.296296296296296e-05,
+      "loss": 0.2911,
+      "step": 35
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.2708097696304321,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.2855,
+      "step": 40
+    },
+    {
+      "epoch": 11.32,
+      "grad_norm": 0.8784252405166626,
+      "learning_rate": 8.148148148148148e-05,
+      "loss": 0.2598,
+      "step": 45
+    },
+    {
+      "epoch": 12.64,
+      "grad_norm": 0.7586076855659485,
+      "learning_rate": 9.074074074074075e-05,
+      "loss": 0.2475,
+      "step": 50
+    },
+    {
+      "epoch": 13.96,
+      "grad_norm": 0.8225407600402832,
+      "learning_rate": 0.0001,
+      "loss": 0.2336,
+      "step": 55
+    },
+    {
+      "epoch": 15.0,
+      "grad_norm": 1.7115800380706787,
+      "learning_rate": 9.971363115693013e-05,
+      "loss": 0.2326,
+      "step": 60
+    },
+    {
+      "epoch": 16.32,
+      "grad_norm": 0.5084587335586548,
+      "learning_rate": 9.942726231386026e-05,
+      "loss": 0.1953,
+      "step": 65
+    },
+    {
+      "epoch": 17.64,
+      "grad_norm": 0.4642345905303955,
+      "learning_rate": 9.914089347079038e-05,
+      "loss": 0.1768,
+      "step": 70
+    },
+    {
+      "epoch": 18.96,
+      "grad_norm": 0.6053332090377808,
+      "learning_rate": 9.885452462772051e-05,
+      "loss": 0.1529,
+      "step": 75
+    },
+    {
+      "epoch": 20.0,
+      "grad_norm": 0.8135235905647278,
+      "learning_rate": 9.856815578465064e-05,
+      "loss": 0.1353,
+      "step": 80
+    },
+    {
+      "epoch": 21.32,
+      "grad_norm": 0.35404855012893677,
+      "learning_rate": 9.828178694158075e-05,
+      "loss": 0.1295,
+      "step": 85
+    },
+    {
+      "epoch": 22.64,
+      "grad_norm": 0.6411961913108826,
+      "learning_rate": 9.799541809851088e-05,
+      "loss": 0.1212,
+      "step": 90
+    },
+    {
+      "epoch": 23.96,
+      "grad_norm": 0.4661860764026642,
+      "learning_rate": 9.7709049255441e-05,
+      "loss": 0.1202,
+      "step": 95
+    },
+    {
+      "epoch": 25.0,
+      "grad_norm": 1.4080708026885986,
+      "learning_rate": 9.742268041237114e-05,
+      "loss": 0.1196,
+      "step": 100
+    },
+    {
+      "epoch": 26.32,
+      "grad_norm": 0.3422178626060486,
+      "learning_rate": 9.713631156930127e-05,
+      "loss": 0.1204,
+      "step": 105
+    },
+    {
+      "epoch": 27.64,
+      "grad_norm": 0.4003280699253082,
+      "learning_rate": 9.68499427262314e-05,
+      "loss": 0.1189,
+      "step": 110
+    },
+    {
+      "epoch": 28.96,
+      "grad_norm": 0.324015736579895,
+      "learning_rate": 9.656357388316152e-05,
+      "loss": 0.1166,
+      "step": 115
+    },
+    {
+      "epoch": 30.0,
+      "grad_norm": 1.315653920173645,
+      "learning_rate": 9.627720504009165e-05,
+      "loss": 0.1167,
+      "step": 120
+    },
+    {
+      "epoch": 31.32,
+      "grad_norm": 0.2823339104652405,
+      "learning_rate": 9.599083619702178e-05,
+      "loss": 0.1192,
+      "step": 125
+    },
+    {
+      "epoch": 32.64,
+      "grad_norm": 0.2731497883796692,
+      "learning_rate": 9.57044673539519e-05,
+      "loss": 0.1309,
+      "step": 130
+    },
+    {
+      "epoch": 33.96,
+      "grad_norm": 0.784600019454956,
+      "learning_rate": 9.541809851088203e-05,
+      "loss": 0.1134,
+      "step": 135
+    },
+    {
+      "epoch": 35.0,
+      "grad_norm": 1.0997117757797241,
+      "learning_rate": 9.513172966781214e-05,
+      "loss": 0.1157,
+      "step": 140
+    },
+    {
+      "epoch": 36.32,
+      "grad_norm": 0.2136397361755371,
+      "learning_rate": 9.484536082474227e-05,
+      "loss": 0.1099,
+      "step": 145
+    },
+    {
+      "epoch": 37.64,
+      "grad_norm": 0.19623281061649323,
+      "learning_rate": 9.45589919816724e-05,
+      "loss": 0.109,
+      "step": 150
+    },
+    {
+      "epoch": 38.96,
+      "grad_norm": 0.19020365178585052,
+      "learning_rate": 9.427262313860252e-05,
+      "loss": 0.1048,
+      "step": 155
+    },
+    {
+      "epoch": 40.0,
+      "grad_norm": 0.3295031487941742,
+      "learning_rate": 9.398625429553265e-05,
+      "loss": 0.1073,
+      "step": 160
+    },
+    {
+      "epoch": 41.32,
+      "grad_norm": 0.25549960136413574,
+      "learning_rate": 9.369988545246277e-05,
+      "loss": 0.1135,
+      "step": 165
+    },
+    {
+      "epoch": 42.64,
+      "grad_norm": 0.24988926947116852,
+      "learning_rate": 9.34135166093929e-05,
+      "loss": 0.1047,
+      "step": 170
+    },
+    {
+      "epoch": 43.96,
+      "grad_norm": 0.44344109296798706,
+      "learning_rate": 9.312714776632303e-05,
+      "loss": 0.1069,
+      "step": 175
+    },
+    {
+      "epoch": 45.0,
+      "grad_norm": 0.7739079594612122,
+      "learning_rate": 9.284077892325315e-05,
+      "loss": 0.1074,
+      "step": 180
+    },
+    {
+      "epoch": 46.32,
+      "grad_norm": 0.16808579862117767,
+      "learning_rate": 9.255441008018328e-05,
+      "loss": 0.1063,
+      "step": 185
+    },
+    {
+      "epoch": 47.64,
+      "grad_norm": 0.22390314936637878,
+      "learning_rate": 9.22680412371134e-05,
+      "loss": 0.107,
+      "step": 190
+    },
+    {
+      "epoch": 48.96,
+      "grad_norm": 0.24022641777992249,
+      "learning_rate": 9.198167239404353e-05,
+      "loss": 0.1062,
+      "step": 195
+    },
+    {
+      "epoch": 50.0,
+      "grad_norm": 1.2908257246017456,
+      "learning_rate": 9.169530355097366e-05,
+      "loss": 0.1098,
+      "step": 200
+    },
+    {
+      "epoch": 51.32,
+      "grad_norm": 0.24903152883052826,
+      "learning_rate": 9.140893470790379e-05,
+      "loss": 0.1301,
+      "step": 205
+    },
+    {
+      "epoch": 52.64,
+      "grad_norm": 0.16907045245170593,
+      "learning_rate": 9.112256586483391e-05,
+      "loss": 0.1119,
+      "step": 210
+    },
+    {
+      "epoch": 53.96,
+      "grad_norm": 0.19608496129512787,
+      "learning_rate": 9.083619702176404e-05,
+      "loss": 0.1115,
+      "step": 215
+    },
+    {
+      "epoch": 55.0,
+      "grad_norm": 0.681024968624115,
+      "learning_rate": 9.054982817869416e-05,
+      "loss": 0.107,
+      "step": 220
+    },
+    {
+      "epoch": 56.32,
+      "grad_norm": 0.2148016393184662,
+      "learning_rate": 9.026345933562429e-05,
+      "loss": 0.1064,
+      "step": 225
+    },
+    {
+      "epoch": 57.64,
+      "grad_norm": 0.4002823531627655,
+      "learning_rate": 8.997709049255442e-05,
+      "loss": 0.11,
+      "step": 230
+    },
+    {
+      "epoch": 58.96,
+      "grad_norm": 0.17194221913814545,
+      "learning_rate": 8.969072164948454e-05,
+      "loss": 0.1075,
+      "step": 235
+    },
+    {
+      "epoch": 60.0,
+      "grad_norm": 0.5104140043258667,
+      "learning_rate": 8.940435280641467e-05,
+      "loss": 0.1032,
+      "step": 240
+    },
+    {
+      "epoch": 61.32,
+      "grad_norm": 0.1683240681886673,
+      "learning_rate": 8.91179839633448e-05,
+      "loss": 0.1044,
+      "step": 245
+    },
+    {
+      "epoch": 62.64,
+      "grad_norm": 0.14366662502288818,
+      "learning_rate": 8.883161512027491e-05,
+      "loss": 0.105,
+      "step": 250
+    },
+    {
+      "epoch": 63.96,
+      "grad_norm": 0.11408836394548416,
+      "learning_rate": 8.854524627720504e-05,
+      "loss": 0.1058,
+      "step": 255
+    },
+    {
+      "epoch": 65.0,
+      "grad_norm": 0.3783295750617981,
+      "learning_rate": 8.825887743413516e-05,
+      "loss": 0.1035,
+      "step": 260
+    },
+    {
+      "epoch": 66.32,
+      "grad_norm": 0.15358224511146545,
+      "learning_rate": 8.797250859106529e-05,
+      "loss": 0.1038,
+      "step": 265
+    },
+    {
+      "epoch": 67.64,
+      "grad_norm": 0.18252335488796234,
+      "learning_rate": 8.768613974799542e-05,
+      "loss": 0.1002,
+      "step": 270
+    },
+    {
+      "epoch": 68.96,
+      "grad_norm": 0.15634660422801971,
+      "learning_rate": 8.739977090492554e-05,
+      "loss": 0.1062,
+      "step": 275
+    },
+    {
+      "epoch": 70.0,
+      "grad_norm": 0.6345776319503784,
+      "learning_rate": 8.711340206185567e-05,
+      "loss": 0.1066,
+      "step": 280
+    },
+    {
+      "epoch": 71.32,
+      "grad_norm": 0.18226927518844604,
+      "learning_rate": 8.682703321878581e-05,
+      "loss": 0.1039,
+      "step": 285
+    },
+    {
+      "epoch": 72.64,
+      "grad_norm": 0.10437572747468948,
+      "learning_rate": 8.654066437571594e-05,
+      "loss": 0.1064,
+      "step": 290
+    },
+    {
+      "epoch": 73.96,
+      "grad_norm": 0.14951160550117493,
+      "learning_rate": 8.625429553264606e-05,
+      "loss": 0.1045,
+      "step": 295
+    },
+    {
+      "epoch": 75.0,
+      "grad_norm": 0.4717479646205902,
+      "learning_rate": 8.596792668957619e-05,
+      "loss": 0.1035,
+      "step": 300
+    },
+    {
+      "epoch": 76.32,
+      "grad_norm": 0.1413341760635376,
+      "learning_rate": 8.56815578465063e-05,
+      "loss": 0.1009,
+      "step": 305
+    },
+    {
+      "epoch": 77.64,
+      "grad_norm": 0.1581067591905594,
+      "learning_rate": 8.539518900343643e-05,
+      "loss": 0.1029,
+      "step": 310
+    },
+    {
+      "epoch": 78.96,
+      "grad_norm": 0.12125775218009949,
+      "learning_rate": 8.510882016036655e-05,
+      "loss": 0.102,
+      "step": 315
+    },
+    {
+      "epoch": 80.0,
+      "grad_norm": 0.309116005897522,
+      "learning_rate": 8.482245131729668e-05,
+      "loss": 0.102,
+      "step": 320
+    },
+    {
+      "epoch": 81.32,
+      "grad_norm": 0.12490837275981903,
+      "learning_rate": 8.453608247422681e-05,
+      "loss": 0.1036,
+      "step": 325
+    },
+    {
+      "epoch": 82.64,
+      "grad_norm": 0.15004275739192963,
+      "learning_rate": 8.424971363115693e-05,
+      "loss": 0.1005,
+      "step": 330
+    },
+    {
+      "epoch": 83.96,
+      "grad_norm": 0.10462955385446548,
+      "learning_rate": 8.396334478808706e-05,
+      "loss": 0.1012,
+      "step": 335
+    },
+    {
+      "epoch": 85.0,
+      "grad_norm": 1.216468095779419,
+      "learning_rate": 8.367697594501719e-05,
+      "loss": 0.1158,
+      "step": 340
+    },
+    {
+      "epoch": 86.32,
+      "grad_norm": 0.12770289182662964,
+      "learning_rate": 8.339060710194731e-05,
+      "loss": 0.103,
+      "step": 345
+    },
+    {
+      "epoch": 87.64,
+      "grad_norm": 0.11857649683952332,
+      "learning_rate": 8.310423825887744e-05,
+      "loss": 0.1024,
+      "step": 350
+    },
+    {
+      "epoch": 88.96,
+      "grad_norm": 0.15543484687805176,
+      "learning_rate": 8.281786941580757e-05,
+      "loss": 0.104,
+      "step": 355
+    },
+    {
+      "epoch": 90.0,
+      "grad_norm": 0.5385985970497131,
+      "learning_rate": 8.253150057273768e-05,
+      "loss": 0.1013,
+      "step": 360
+    },
+    {
+      "epoch": 91.32,
+      "grad_norm": 0.12159006297588348,
+      "learning_rate": 8.224513172966782e-05,
+      "loss": 0.1049,
+      "step": 365
+    },
+    {
+      "epoch": 92.64,
+      "grad_norm": 0.13037694990634918,
+      "learning_rate": 8.195876288659795e-05,
+      "loss": 0.1018,
+      "step": 370
+    },
+    {
+      "epoch": 93.96,
+      "grad_norm": 0.1268284171819687,
+      "learning_rate": 8.167239404352807e-05,
+      "loss": 0.1054,
+      "step": 375
+    },
+    {
+      "epoch": 95.0,
+      "grad_norm": 0.292937695980072,
+      "learning_rate": 8.13860252004582e-05,
+      "loss": 0.1023,
+      "step": 380
+    },
+    {
+      "epoch": 96.32,
+      "grad_norm": 0.13367846608161926,
+      "learning_rate": 8.109965635738833e-05,
+      "loss": 0.1025,
+      "step": 385
+    },
+    {
+      "epoch": 97.64,
+      "grad_norm": 0.09405620396137238,
+      "learning_rate": 8.081328751431845e-05,
+      "loss": 0.103,
+      "step": 390
+    },
+    {
+      "epoch": 98.96,
+      "grad_norm": 0.10122726112604141,
+      "learning_rate": 8.052691867124858e-05,
+      "loss": 0.106,
+      "step": 395
+    },
+    {
+      "epoch": 100.0,
+      "grad_norm": 0.3003501892089844,
+      "learning_rate": 8.02405498281787e-05,
+      "loss": 0.101,
+      "step": 400
+    },
+    {
+      "epoch": 101.32,
+      "grad_norm": 0.08715490251779556,
+      "learning_rate": 7.995418098510883e-05,
+      "loss": 0.1004,
+      "step": 405
+    },
+    {
+      "epoch": 102.64,
+      "grad_norm": 0.1160975769162178,
+      "learning_rate": 7.966781214203894e-05,
+      "loss": 0.1029,
+      "step": 410
+    },
+    {
+      "epoch": 103.96,
+      "grad_norm": 0.12515407800674438,
+      "learning_rate": 7.938144329896907e-05,
+      "loss": 0.1019,
+      "step": 415
+    },
+    {
+      "epoch": 105.0,
+      "grad_norm": 0.43526867032051086,
+      "learning_rate": 7.90950744558992e-05,
+      "loss": 0.1037,
+      "step": 420
+    },
+    {
+      "epoch": 106.32,
+      "grad_norm": 0.1083005964756012,
+      "learning_rate": 7.880870561282932e-05,
+      "loss": 0.1048,
+      "step": 425
+    },
+    {
+      "epoch": 107.64,
+      "grad_norm": 0.12727200984954834,
+      "learning_rate": 7.852233676975945e-05,
+      "loss": 0.1013,
+      "step": 430
+    },
+    {
+      "epoch": 108.96,
+      "grad_norm": 0.11788661777973175,
+      "learning_rate": 7.823596792668958e-05,
+      "loss": 0.1002,
+      "step": 435
+    },
+    {
+      "epoch": 110.0,
+      "grad_norm": 0.5509673357009888,
+      "learning_rate": 7.79495990836197e-05,
+      "loss": 0.1065,
+      "step": 440
+    },
+    {
+      "epoch": 111.32,
+      "grad_norm": 0.12785306572914124,
+      "learning_rate": 7.766323024054983e-05,
+      "loss": 0.1013,
+      "step": 445
+    },
+    {
+      "epoch": 112.64,
+      "grad_norm": 0.13988617062568665,
+      "learning_rate": 7.737686139747996e-05,
+      "loss": 0.1014,
+      "step": 450
+    },
+    {
+      "epoch": 113.96,
+      "grad_norm": 0.11280432343482971,
+      "learning_rate": 7.709049255441008e-05,
+      "loss": 0.1017,
+      "step": 455
+    },
+    {
+      "epoch": 115.0,
+      "grad_norm": 0.33109962940216064,
+      "learning_rate": 7.680412371134021e-05,
+      "loss": 0.0993,
+      "step": 460
+    },
+    {
+      "epoch": 116.32,
+      "grad_norm": 0.10653392225503922,
+      "learning_rate": 7.651775486827034e-05,
+      "loss": 0.1016,
+      "step": 465
+    },
+    {
+      "epoch": 117.64,
+      "grad_norm": 0.09670863300561905,
+      "learning_rate": 7.623138602520046e-05,
+      "loss": 0.1003,
+      "step": 470
+    },
+    {
+      "epoch": 118.96,
+      "grad_norm": 0.11432068794965744,
+      "learning_rate": 7.594501718213059e-05,
+      "loss": 0.1027,
+      "step": 475
+    },
+    {
+      "epoch": 120.0,
+      "grad_norm": 0.6067518591880798,
+      "learning_rate": 7.565864833906071e-05,
+      "loss": 0.1054,
+      "step": 480
+    },
+    {
+      "epoch": 121.32,
+      "grad_norm": 0.08575757592916489,
+      "learning_rate": 7.537227949599084e-05,
+      "loss": 0.0993,
+      "step": 485
+    },
+    {
+      "epoch": 122.64,
+      "grad_norm": 0.1070568859577179,
+      "learning_rate": 7.508591065292097e-05,
+      "loss": 0.0991,
+      "step": 490
+    },
+    {
+      "epoch": 123.96,
+      "grad_norm": 0.09047789871692657,
+      "learning_rate": 7.47995418098511e-05,
+      "loss": 0.1032,
+      "step": 495
+    },
+    {
+      "epoch": 125.0,
+      "grad_norm": 0.2450919896364212,
+      "learning_rate": 7.451317296678122e-05,
+      "loss": 0.1008,
+      "step": 500
+    },
+    {
+      "epoch": 126.32,
+      "grad_norm": 0.11366956681013107,
+      "learning_rate": 7.422680412371135e-05,
+      "loss": 0.099,
+      "step": 505
+    },
+    {
+      "epoch": 127.64,
+      "grad_norm": 0.10186628252267838,
+      "learning_rate": 7.394043528064147e-05,
+      "loss": 0.1012,
+      "step": 510
+    },
+    {
+      "epoch": 128.96,
+      "grad_norm": 0.1052548959851265,
+      "learning_rate": 7.36540664375716e-05,
+      "loss": 0.1013,
+      "step": 515
+    },
+    {
+      "epoch": 130.0,
+      "grad_norm": 0.3205568492412567,
+      "learning_rate": 7.336769759450171e-05,
+      "loss": 0.1006,
+      "step": 520
+    },
+    {
+      "epoch": 131.32,
+      "grad_norm": 0.07446248829364777,
+      "learning_rate": 7.308132875143184e-05,
+      "loss": 0.1012,
+      "step": 525
+    },
+    {
+      "epoch": 132.64,
+      "grad_norm": 0.08148901909589767,
+      "learning_rate": 7.279495990836197e-05,
+      "loss": 0.1012,
+      "step": 530
+    },
+    {
+      "epoch": 133.96,
+      "grad_norm": 0.11023813486099243,
+      "learning_rate": 7.250859106529209e-05,
+      "loss": 0.1001,
+      "step": 535
+    },
+    {
+      "epoch": 135.0,
+      "grad_norm": 0.32420921325683594,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.1017,
+      "step": 540
+    },
+    {
+      "epoch": 136.32,
+      "grad_norm": 0.08981231600046158,
+      "learning_rate": 7.193585337915235e-05,
+      "loss": 0.1019,
+      "step": 545
+    },
+    {
+      "epoch": 137.64,
+      "grad_norm": 0.1241590604186058,
+      "learning_rate": 7.164948453608247e-05,
+      "loss": 0.0994,
+      "step": 550
+    },
+    {
+      "epoch": 138.96,
+      "grad_norm": 0.12690334022045135,
+      "learning_rate": 7.136311569301261e-05,
+      "loss": 0.0998,
+      "step": 555
+    },
+    {
+      "epoch": 140.0,
+      "grad_norm": 0.26669779419898987,
+      "learning_rate": 7.107674684994274e-05,
+      "loss": 0.0995,
+      "step": 560
+    },
+    {
+      "epoch": 141.32,
+      "grad_norm": 0.10272305458784103,
+      "learning_rate": 7.079037800687286e-05,
+      "loss": 0.1032,
+      "step": 565
+    },
+    {
+      "epoch": 142.64,
+      "grad_norm": 0.10494570434093475,
+      "learning_rate": 7.050400916380299e-05,
+      "loss": 0.1015,
+      "step": 570
+    },
+    {
+      "epoch": 143.96,
+      "grad_norm": 0.11733629554510117,
+      "learning_rate": 7.02176403207331e-05,
+      "loss": 0.1026,
+      "step": 575
+    },
+    {
+      "epoch": 145.0,
+      "grad_norm": 0.18178793787956238,
+      "learning_rate": 6.993127147766323e-05,
+      "loss": 0.1009,
+      "step": 580
+    },
+    {
+      "epoch": 146.32,
+      "grad_norm": 0.07104074954986572,
+      "learning_rate": 6.964490263459336e-05,
+      "loss": 0.099,
+      "step": 585
+    },
+    {
+      "epoch": 147.64,
+      "grad_norm": 0.0909457802772522,
+      "learning_rate": 6.935853379152348e-05,
+      "loss": 0.1015,
+      "step": 590
+    },
+    {
+      "epoch": 148.96,
+      "grad_norm": 0.09944809228181839,
+      "learning_rate": 6.907216494845361e-05,
+      "loss": 0.1001,
+      "step": 595
+    },
+    {
+      "epoch": 150.0,
+      "grad_norm": 0.23125752806663513,
+      "learning_rate": 6.878579610538374e-05,
+      "loss": 0.0984,
+      "step": 600
+    },
+    {
+      "epoch": 151.32,
+      "grad_norm": 0.0829208642244339,
+      "learning_rate": 6.849942726231386e-05,
+      "loss": 0.0995,
+      "step": 605
+    },
+    {
+      "epoch": 152.64,
+      "grad_norm": 0.09419254958629608,
+      "learning_rate": 6.821305841924399e-05,
+      "loss": 0.1008,
+      "step": 610
+    },
+    {
+      "epoch": 153.96,
+      "grad_norm": 0.0779041051864624,
+      "learning_rate": 6.792668957617412e-05,
+      "loss": 0.0982,
+      "step": 615
+    },
+    {
+      "epoch": 155.0,
+      "grad_norm": 0.2598252594470978,
+      "learning_rate": 6.764032073310424e-05,
+      "loss": 0.1,
+      "step": 620
+    },
+    {
+      "epoch": 156.32,
+      "grad_norm": 0.08832815289497375,
+      "learning_rate": 6.735395189003437e-05,
+      "loss": 0.0989,
+      "step": 625
+    },
+    {
+      "epoch": 157.64,
+      "grad_norm": 0.07834926247596741,
+      "learning_rate": 6.706758304696448e-05,
+      "loss": 0.1016,
+      "step": 630
+    },
+    {
+      "epoch": 158.96,
+      "grad_norm": 0.08721348643302917,
+      "learning_rate": 6.678121420389462e-05,
+      "loss": 0.1002,
+      "step": 635
+    },
+    {
+      "epoch": 160.0,
+      "grad_norm": 0.22782059013843536,
+      "learning_rate": 6.649484536082475e-05,
+      "loss": 0.1003,
+      "step": 640
+    },
+    {
+      "epoch": 161.32,
+      "grad_norm": 0.07970811426639557,
+      "learning_rate": 6.620847651775487e-05,
+      "loss": 0.1,
+      "step": 645
+    },
+    {
+      "epoch": 162.64,
+      "grad_norm": 0.09135862439870834,
+      "learning_rate": 6.5922107674685e-05,
+      "loss": 0.1015,
+      "step": 650
+    },
+    {
+      "epoch": 163.96,
+      "grad_norm": 0.10493220388889313,
+      "learning_rate": 6.563573883161513e-05,
+      "loss": 0.1004,
+      "step": 655
+    },
+    {
+      "epoch": 165.0,
+      "grad_norm": 0.3030238747596741,
+      "learning_rate": 6.534936998854525e-05,
+      "loss": 0.1023,
+      "step": 660
+    },
+    {
+      "epoch": 166.32,
+      "grad_norm": 0.09718652069568634,
+      "learning_rate": 6.506300114547538e-05,
+      "loss": 0.101,
+      "step": 665
+    },
+    {
+      "epoch": 167.64,
+      "grad_norm": 0.06749033182859421,
+      "learning_rate": 6.477663230240551e-05,
+      "loss": 0.1008,
+      "step": 670
+    },
+    {
+      "epoch": 168.96,
+      "grad_norm": 0.08545351028442383,
+      "learning_rate": 6.449026345933563e-05,
+      "loss": 0.1021,
+      "step": 675
+    },
+    {
+      "epoch": 170.0,
+      "grad_norm": 0.25453537702560425,
+      "learning_rate": 6.420389461626576e-05,
+      "loss": 0.0976,
+      "step": 680
+    },
+    {
+      "epoch": 171.32,
+      "grad_norm": 0.07819271087646484,
+      "learning_rate": 6.391752577319587e-05,
+      "loss": 0.1014,
+      "step": 685
+    },
+    {
+      "epoch": 172.64,
+      "grad_norm": 0.07297082990407944,
+      "learning_rate": 6.3631156930126e-05,
+      "loss": 0.1023,
+      "step": 690
+    },
+    {
+      "epoch": 173.96,
+      "grad_norm": 0.08446590602397919,
+      "learning_rate": 6.334478808705613e-05,
+      "loss": 0.1002,
+      "step": 695
+    },
+    {
+      "epoch": 175.0,
+      "grad_norm": 0.21633073687553406,
+      "learning_rate": 6.305841924398625e-05,
+      "loss": 0.0992,
+      "step": 700
+    },
+    {
+      "epoch": 176.32,
+      "grad_norm": 0.07673562318086624,
+      "learning_rate": 6.277205040091638e-05,
+      "loss": 0.0995,
+      "step": 705
+    },
+    {
+      "epoch": 177.64,
+      "grad_norm": 0.09338875859975815,
+      "learning_rate": 6.24856815578465e-05,
+      "loss": 0.1014,
+      "step": 710
+    },
+    {
+      "epoch": 178.96,
+      "grad_norm": 0.09649301320314407,
+      "learning_rate": 6.219931271477663e-05,
+      "loss": 0.1006,
+      "step": 715
+    },
+    {
+      "epoch": 180.0,
+      "grad_norm": 0.2125590741634369,
+      "learning_rate": 6.191294387170676e-05,
+      "loss": 0.0996,
+      "step": 720
+    },
+    {
+      "epoch": 181.32,
+      "grad_norm": 0.07358326762914658,
+      "learning_rate": 6.162657502863689e-05,
+      "loss": 0.1006,
+      "step": 725
+    },
+    {
+      "epoch": 182.64,
+      "grad_norm": 0.07445739209651947,
+      "learning_rate": 6.134020618556701e-05,
+      "loss": 0.0984,
+      "step": 730
+    },
+    {
+      "epoch": 183.96,
+      "grad_norm": 0.07530005276203156,
+      "learning_rate": 6.105383734249714e-05,
+      "loss": 0.0999,
+      "step": 735
+    },
+    {
+      "epoch": 185.0,
+      "grad_norm": 0.29669082164764404,
+      "learning_rate": 6.076746849942726e-05,
+      "loss": 0.102,
+      "step": 740
+    },
+    {
+      "epoch": 186.32,
+      "grad_norm": 0.09269877523183823,
+      "learning_rate": 6.0481099656357384e-05,
+      "loss": 0.0997,
+      "step": 745
+    },
+    {
+      "epoch": 187.64,
+      "grad_norm": 0.0804276242852211,
+      "learning_rate": 6.019473081328752e-05,
+      "loss": 0.1014,
+      "step": 750
+    },
+    {
+      "epoch": 188.96,
+      "grad_norm": 0.098207488656044,
+      "learning_rate": 5.9908361970217644e-05,
+      "loss": 0.0988,
+      "step": 755
+    },
+    {
+      "epoch": 190.0,
+      "grad_norm": 0.19817565381526947,
+      "learning_rate": 5.962199312714777e-05,
+      "loss": 0.1001,
+      "step": 760
+    },
+    {
+      "epoch": 191.32,
+      "grad_norm": 0.06558432430028915,
+      "learning_rate": 5.93356242840779e-05,
+      "loss": 0.1006,
+      "step": 765
+    },
+    {
+      "epoch": 192.64,
+      "grad_norm": 0.0970512330532074,
+      "learning_rate": 5.904925544100802e-05,
+      "loss": 0.1011,
+      "step": 770
+    },
+    {
+      "epoch": 193.96,
+      "grad_norm": 0.07481536269187927,
+      "learning_rate": 5.876288659793815e-05,
+      "loss": 0.0988,
+      "step": 775
+    },
+    {
+      "epoch": 195.0,
+      "grad_norm": 0.2697769105434418,
+      "learning_rate": 5.8476517754868276e-05,
+      "loss": 0.1008,
+      "step": 780
+    },
+    {
+      "epoch": 196.32,
+      "grad_norm": 0.08228090405464172,
+      "learning_rate": 5.81901489117984e-05,
+      "loss": 0.0999,
+      "step": 785
+    },
+    {
+      "epoch": 197.64,
+      "grad_norm": 0.08324407786130905,
+      "learning_rate": 5.790378006872853e-05,
+      "loss": 0.0999,
+      "step": 790
+    },
+    {
+      "epoch": 198.96,
+      "grad_norm": 0.0824645459651947,
+      "learning_rate": 5.761741122565865e-05,
+      "loss": 0.1011,
+      "step": 795
+    },
+    {
+      "epoch": 200.0,
+      "grad_norm": 0.2783829867839813,
+      "learning_rate": 5.7331042382588775e-05,
+      "loss": 0.101,
+      "step": 800
+    },
+    {
+      "epoch": 201.32,
+      "grad_norm": 0.07741741836071014,
+      "learning_rate": 5.70446735395189e-05,
+      "loss": 0.1011,
+      "step": 805
+    },
+    {
+      "epoch": 202.64,
+      "grad_norm": 0.06530506908893585,
+      "learning_rate": 5.675830469644903e-05,
+      "loss": 0.0997,
+      "step": 810
+    },
+    {
+      "epoch": 203.96,
+      "grad_norm": 0.06250391900539398,
+      "learning_rate": 5.6471935853379155e-05,
+      "loss": 0.1013,
+      "step": 815
+    },
+    {
+      "epoch": 205.0,
+      "grad_norm": 0.3014598488807678,
+      "learning_rate": 5.618556701030928e-05,
+      "loss": 0.0994,
+      "step": 820
+    },
+    {
+      "epoch": 206.32,
+      "grad_norm": 0.06866947561502457,
+      "learning_rate": 5.589919816723941e-05,
+      "loss": 0.0971,
+      "step": 825
+    },
+    {
+      "epoch": 207.64,
+      "grad_norm": 0.08845752477645874,
+      "learning_rate": 5.5612829324169534e-05,
+      "loss": 0.1009,
+      "step": 830
+    },
+    {
+      "epoch": 208.96,
+      "grad_norm": 0.08089124411344528,
+      "learning_rate": 5.532646048109966e-05,
+      "loss": 0.0993,
+      "step": 835
+    },
+    {
+      "epoch": 210.0,
+      "grad_norm": 0.30784446001052856,
+      "learning_rate": 5.504009163802979e-05,
+      "loss": 0.0992,
+      "step": 840
+    },
+    {
+      "epoch": 211.32,
+      "grad_norm": 0.06972301006317139,
+      "learning_rate": 5.4753722794959914e-05,
+      "loss": 0.1,
+      "step": 845
+    },
+    {
+      "epoch": 212.64,
+      "grad_norm": 0.07285023480653763,
+      "learning_rate": 5.4467353951890033e-05,
+      "loss": 0.0993,
+      "step": 850
+    },
+    {
+      "epoch": 213.96,
+      "grad_norm": 0.060395676642656326,
+      "learning_rate": 5.418098510882016e-05,
+      "loss": 0.0989,
+      "step": 855
+    },
+    {
+      "epoch": 215.0,
+      "grad_norm": 0.17777375876903534,
+      "learning_rate": 5.3894616265750286e-05,
+      "loss": 0.1002,
+      "step": 860
+    },
+    {
+      "epoch": 216.32,
+      "grad_norm": 0.07913000881671906,
+      "learning_rate": 5.360824742268041e-05,
+      "loss": 0.0985,
+      "step": 865
+    },
+    {
+      "epoch": 217.64,
+      "grad_norm": 0.08352327346801758,
+      "learning_rate": 5.332187857961054e-05,
+      "loss": 0.0994,
+      "step": 870
+    },
+    {
+      "epoch": 218.96,
+      "grad_norm": 0.07089956849813461,
+      "learning_rate": 5.3035509736540666e-05,
+      "loss": 0.0993,
+      "step": 875
+    },
+    {
+      "epoch": 220.0,
+      "grad_norm": 0.27649030089378357,
+      "learning_rate": 5.274914089347079e-05,
+      "loss": 0.1013,
+      "step": 880
+    },
+    {
+      "epoch": 221.32,
+      "grad_norm": 0.08725525438785553,
+      "learning_rate": 5.246277205040092e-05,
+      "loss": 0.0998,
+      "step": 885
+    },
+    {
+      "epoch": 222.64,
+      "grad_norm": 0.06686926633119583,
+      "learning_rate": 5.2176403207331045e-05,
+      "loss": 0.1004,
+      "step": 890
+    },
+    {
+      "epoch": 223.96,
+      "grad_norm": 0.0881044864654541,
+      "learning_rate": 5.189003436426118e-05,
+      "loss": 0.0993,
+      "step": 895
+    },
+    {
+      "epoch": 225.0,
+      "grad_norm": 0.2503332793712616,
+      "learning_rate": 5.1603665521191305e-05,
+      "loss": 0.1002,
+      "step": 900
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 450,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.654070550528e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

Math-IIO-68K-Mini/group_01/adapter/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_01/adapter/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Math-IIO-68K-Mini/group_01/adapter/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_01/adapter/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_01/checkpoints/checkpoint-900/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_01/checkpoints/checkpoint-900/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Math-IIO-68K-Mini/group_03/adapter/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_03/adapter/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/\u4e0d\u51bb\u7ed3Qwen\u8bad\u7ec3/models/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Math-IIO-68K-Mini/group_03/adapter/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

Math-IIO-68K-Mini/group_03/adapter/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_03/adapter/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.12.0

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/\u4e0d\u51bb\u7ed3Qwen\u8bad\u7ec3/models/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

Math-IIO-68K-Mini/group_03/checkpoints/checkpoint-1200/trainer_state.json

@@ -0,0 +1,1721 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 300.0,
+  "eval_steps": 500,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.32,
+      "grad_norm": 17.34026336669922,
+      "learning_rate": 0.0,
+      "loss": 3.2051,
+      "step": 1
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 15.267053604125977,
+      "learning_rate": 7.4074074074074075e-06,
+      "loss": 3.2263,
+      "step": 5
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 10.574139595031738,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 2.6208,
+      "step": 10
+    },
+    {
+      "epoch": 3.96,
+      "grad_norm": 6.36294412612915,
+      "learning_rate": 2.5925925925925925e-05,
+      "loss": 1.5366,
+      "step": 15
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 2.573376417160034,
+      "learning_rate": 3.518518518518519e-05,
+      "loss": 0.6351,
+      "step": 20
+    },
+    {
+      "epoch": 6.32,
+      "grad_norm": 1.3061388731002808,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 0.3481,
+      "step": 25
+    },
+    {
+      "epoch": 7.64,
+      "grad_norm": 1.4732104539871216,
+      "learning_rate": 5.370370370370371e-05,
+      "loss": 0.3107,
+      "step": 30
+    },
+    {
+      "epoch": 8.96,
+      "grad_norm": 0.7342837452888489,
+      "learning_rate": 6.296296296296296e-05,
+      "loss": 0.3214,
+      "step": 35
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.623380422592163,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.3003,
+      "step": 40
+    },
+    {
+      "epoch": 11.32,
+      "grad_norm": 0.5047767162322998,
+      "learning_rate": 8.148148148148148e-05,
+      "loss": 0.2806,
+      "step": 45
+    },
+    {
+      "epoch": 12.64,
+      "grad_norm": 0.6891778111457825,
+      "learning_rate": 9.074074074074075e-05,
+      "loss": 0.2621,
+      "step": 50
+    },
+    {
+      "epoch": 13.96,
+      "grad_norm": 0.5145540833473206,
+      "learning_rate": 0.0001,
+      "loss": 0.2487,
+      "step": 55
+    },
+    {
+      "epoch": 15.0,
+      "grad_norm": 1.1229959726333618,
+      "learning_rate": 9.971363115693013e-05,
+      "loss": 0.2493,
+      "step": 60
+    },
+    {
+      "epoch": 16.32,
+      "grad_norm": 0.7204778790473938,
+      "learning_rate": 9.942726231386026e-05,
+      "loss": 0.2055,
+      "step": 65
+    },
+    {
+      "epoch": 17.64,
+      "grad_norm": 1.791100025177002,
+      "learning_rate": 9.914089347079038e-05,
+      "loss": 0.1917,
+      "step": 70
+    },
+    {
+      "epoch": 18.96,
+      "grad_norm": 0.5924006104469299,
+      "learning_rate": 9.885452462772051e-05,
+      "loss": 0.1728,
+      "step": 75
+    },
+    {
+      "epoch": 20.0,
+      "grad_norm": 1.136206030845642,
+      "learning_rate": 9.856815578465064e-05,
+      "loss": 0.1439,
+      "step": 80
+    },
+    {
+      "epoch": 21.32,
+      "grad_norm": 0.7784563899040222,
+      "learning_rate": 9.828178694158075e-05,
+      "loss": 0.1382,
+      "step": 85
+    },
+    {
+      "epoch": 22.64,
+      "grad_norm": 0.6235980987548828,
+      "learning_rate": 9.799541809851088e-05,
+      "loss": 0.1362,
+      "step": 90
+    },
+    {
+      "epoch": 23.96,
+      "grad_norm": 0.5678330659866333,
+      "learning_rate": 9.7709049255441e-05,
+      "loss": 0.1221,
+      "step": 95
+    },
+    {
+      "epoch": 25.0,
+      "grad_norm": 1.6904798746109009,
+      "learning_rate": 9.742268041237114e-05,
+      "loss": 0.1205,
+      "step": 100
+    },
+    {
+      "epoch": 26.32,
+      "grad_norm": 0.4402594268321991,
+      "learning_rate": 9.713631156930127e-05,
+      "loss": 0.1159,
+      "step": 105
+    },
+    {
+      "epoch": 27.64,
+      "grad_norm": 0.451010137796402,
+      "learning_rate": 9.68499427262314e-05,
+      "loss": 0.1151,
+      "step": 110
+    },
+    {
+      "epoch": 28.96,
+      "grad_norm": 0.36653900146484375,
+      "learning_rate": 9.656357388316152e-05,
+      "loss": 0.1129,
+      "step": 115
+    },
+    {
+      "epoch": 30.0,
+      "grad_norm": 1.1552420854568481,
+      "learning_rate": 9.627720504009165e-05,
+      "loss": 0.1159,
+      "step": 120
+    },
+    {
+      "epoch": 31.32,
+      "grad_norm": 0.2390529215335846,
+      "learning_rate": 9.599083619702178e-05,
+      "loss": 0.1124,
+      "step": 125
+    },
+    {
+      "epoch": 32.64,
+      "grad_norm": 0.21517720818519592,
+      "learning_rate": 9.57044673539519e-05,
+      "loss": 0.1079,
+      "step": 130
+    },
+    {
+      "epoch": 33.96,
+      "grad_norm": 0.28939270973205566,
+      "learning_rate": 9.541809851088203e-05,
+      "loss": 0.1111,
+      "step": 135
+    },
+    {
+      "epoch": 35.0,
+      "grad_norm": 0.7307063341140747,
+      "learning_rate": 9.513172966781214e-05,
+      "loss": 0.1137,
+      "step": 140
+    },
+    {
+      "epoch": 36.32,
+      "grad_norm": 0.20776157081127167,
+      "learning_rate": 9.484536082474227e-05,
+      "loss": 0.1124,
+      "step": 145
+    },
+    {
+      "epoch": 37.64,
+      "grad_norm": 0.2950611710548401,
+      "learning_rate": 9.45589919816724e-05,
+      "loss": 0.1089,
+      "step": 150
+    },
+    {
+      "epoch": 38.96,
+      "grad_norm": 0.26791685819625854,
+      "learning_rate": 9.427262313860252e-05,
+      "loss": 0.1082,
+      "step": 155
+    },
+    {
+      "epoch": 40.0,
+      "grad_norm": 0.4527050256729126,
+      "learning_rate": 9.398625429553265e-05,
+      "loss": 0.1052,
+      "step": 160
+    },
+    {
+      "epoch": 41.32,
+      "grad_norm": 0.2053116410970688,
+      "learning_rate": 9.369988545246277e-05,
+      "loss": 0.1055,
+      "step": 165
+    },
+    {
+      "epoch": 42.64,
+      "grad_norm": 0.18580006062984467,
+      "learning_rate": 9.34135166093929e-05,
+      "loss": 0.1065,
+      "step": 170
+    },
+    {
+      "epoch": 43.96,
+      "grad_norm": 0.17079514265060425,
+      "learning_rate": 9.312714776632303e-05,
+      "loss": 0.1039,
+      "step": 175
+    },
+    {
+      "epoch": 45.0,
+      "grad_norm": 0.7385118007659912,
+      "learning_rate": 9.284077892325315e-05,
+      "loss": 0.1059,
+      "step": 180
+    },
+    {
+      "epoch": 46.32,
+      "grad_norm": 0.23394139111042023,
+      "learning_rate": 9.255441008018328e-05,
+      "loss": 0.1043,
+      "step": 185
+    },
+    {
+      "epoch": 47.64,
+      "grad_norm": 0.2056766003370285,
+      "learning_rate": 9.22680412371134e-05,
+      "loss": 0.11,
+      "step": 190
+    },
+    {
+      "epoch": 48.96,
+      "grad_norm": 0.12725186347961426,
+      "learning_rate": 9.198167239404353e-05,
+      "loss": 0.1086,
+      "step": 195
+    },
+    {
+      "epoch": 50.0,
+      "grad_norm": 0.4424174129962921,
+      "learning_rate": 9.169530355097366e-05,
+      "loss": 0.1053,
+      "step": 200
+    },
+    {
+      "epoch": 51.32,
+      "grad_norm": 0.18803177773952484,
+      "learning_rate": 9.140893470790379e-05,
+      "loss": 0.1079,
+      "step": 205
+    },
+    {
+      "epoch": 52.64,
+      "grad_norm": 0.15884272754192352,
+      "learning_rate": 9.112256586483391e-05,
+      "loss": 0.1022,
+      "step": 210
+    },
+    {
+      "epoch": 53.96,
+      "grad_norm": 0.16245657205581665,
+      "learning_rate": 9.083619702176404e-05,
+      "loss": 0.1024,
+      "step": 215
+    },
+    {
+      "epoch": 55.0,
+      "grad_norm": 0.6643825769424438,
+      "learning_rate": 9.054982817869416e-05,
+      "loss": 0.1046,
+      "step": 220
+    },
+    {
+      "epoch": 56.32,
+      "grad_norm": 0.15928825736045837,
+      "learning_rate": 9.026345933562429e-05,
+      "loss": 0.1017,
+      "step": 225
+    },
+    {
+      "epoch": 57.64,
+      "grad_norm": 0.11277788877487183,
+      "learning_rate": 8.997709049255442e-05,
+      "loss": 0.1058,
+      "step": 230
+    },
+    {
+      "epoch": 58.96,
+      "grad_norm": 0.19445186853408813,
+      "learning_rate": 8.969072164948454e-05,
+      "loss": 0.1017,
+      "step": 235
+    },
+    {
+      "epoch": 60.0,
+      "grad_norm": 0.615966260433197,
+      "learning_rate": 8.940435280641467e-05,
+      "loss": 0.1039,
+      "step": 240
+    },
+    {
+      "epoch": 61.32,
+      "grad_norm": 0.18269860744476318,
+      "learning_rate": 8.91179839633448e-05,
+      "loss": 0.1026,
+      "step": 245
+    },
+    {
+      "epoch": 62.64,
+      "grad_norm": 0.13997894525527954,
+      "learning_rate": 8.883161512027491e-05,
+      "loss": 0.1039,
+      "step": 250
+    },
+    {
+      "epoch": 63.96,
+      "grad_norm": 0.12029420584440231,
+      "learning_rate": 8.854524627720504e-05,
+      "loss": 0.1042,
+      "step": 255
+    },
+    {
+      "epoch": 65.0,
+      "grad_norm": 0.27522212266921997,
+      "learning_rate": 8.825887743413516e-05,
+      "loss": 0.1044,
+      "step": 260
+    },
+    {
+      "epoch": 66.32,
+      "grad_norm": 0.11091982573270798,
+      "learning_rate": 8.797250859106529e-05,
+      "loss": 0.1027,
+      "step": 265
+    },
+    {
+      "epoch": 67.64,
+      "grad_norm": 0.12264374643564224,
+      "learning_rate": 8.768613974799542e-05,
+      "loss": 0.1008,
+      "step": 270
+    },
+    {
+      "epoch": 68.96,
+      "grad_norm": 0.12374477088451385,
+      "learning_rate": 8.739977090492554e-05,
+      "loss": 0.1076,
+      "step": 275
+    },
+    {
+      "epoch": 70.0,
+      "grad_norm": 0.5439356565475464,
+      "learning_rate": 8.711340206185567e-05,
+      "loss": 0.1065,
+      "step": 280
+    },
+    {
+      "epoch": 71.32,
+      "grad_norm": 0.1330622285604477,
+      "learning_rate": 8.682703321878581e-05,
+      "loss": 0.1035,
+      "step": 285
+    },
+    {
+      "epoch": 72.64,
+      "grad_norm": 0.1356915831565857,
+      "learning_rate": 8.654066437571594e-05,
+      "loss": 0.1042,
+      "step": 290
+    },
+    {
+      "epoch": 73.96,
+      "grad_norm": 0.18466618657112122,
+      "learning_rate": 8.625429553264606e-05,
+      "loss": 0.1064,
+      "step": 295
+    },
+    {
+      "epoch": 75.0,
+      "grad_norm": 0.35966020822525024,
+      "learning_rate": 8.596792668957619e-05,
+      "loss": 0.1027,
+      "step": 300
+    },
+    {
+      "epoch": 76.32,
+      "grad_norm": 0.11449939757585526,
+      "learning_rate": 8.56815578465063e-05,
+      "loss": 0.103,
+      "step": 305
+    },
+    {
+      "epoch": 77.64,
+      "grad_norm": 0.1334460973739624,
+      "learning_rate": 8.539518900343643e-05,
+      "loss": 0.102,
+      "step": 310
+    },
+    {
+      "epoch": 78.96,
+      "grad_norm": 0.11892566084861755,
+      "learning_rate": 8.510882016036655e-05,
+      "loss": 0.1031,
+      "step": 315
+    },
+    {
+      "epoch": 80.0,
+      "grad_norm": 0.4657507538795471,
+      "learning_rate": 8.482245131729668e-05,
+      "loss": 0.1028,
+      "step": 320
+    },
+    {
+      "epoch": 81.32,
+      "grad_norm": 0.15308040380477905,
+      "learning_rate": 8.453608247422681e-05,
+      "loss": 0.102,
+      "step": 325
+    },
+    {
+      "epoch": 82.64,
+      "grad_norm": 0.16250653564929962,
+      "learning_rate": 8.424971363115693e-05,
+      "loss": 0.1026,
+      "step": 330
+    },
+    {
+      "epoch": 83.96,
+      "grad_norm": 0.1413871794939041,
+      "learning_rate": 8.396334478808706e-05,
+      "loss": 0.0989,
+      "step": 335
+    },
+    {
+      "epoch": 85.0,
+      "grad_norm": 0.29202479124069214,
+      "learning_rate": 8.367697594501719e-05,
+      "loss": 0.1035,
+      "step": 340
+    },
+    {
+      "epoch": 86.32,
+      "grad_norm": 0.10527799278497696,
+      "learning_rate": 8.339060710194731e-05,
+      "loss": 0.101,
+      "step": 345
+    },
+    {
+      "epoch": 87.64,
+      "grad_norm": 0.11784271895885468,
+      "learning_rate": 8.310423825887744e-05,
+      "loss": 0.101,
+      "step": 350
+    },
+    {
+      "epoch": 88.96,
+      "grad_norm": 0.14686010777950287,
+      "learning_rate": 8.281786941580757e-05,
+      "loss": 0.1026,
+      "step": 355
+    },
+    {
+      "epoch": 90.0,
+      "grad_norm": 0.25344258546829224,
+      "learning_rate": 8.253150057273768e-05,
+      "loss": 0.1008,
+      "step": 360
+    },
+    {
+      "epoch": 91.32,
+      "grad_norm": 0.13996697962284088,
+      "learning_rate": 8.224513172966782e-05,
+      "loss": 0.1015,
+      "step": 365
+    },
+    {
+      "epoch": 92.64,
+      "grad_norm": 0.09638503938913345,
+      "learning_rate": 8.195876288659795e-05,
+      "loss": 0.1,
+      "step": 370
+    },
+    {
+      "epoch": 93.96,
+      "grad_norm": 0.08330681174993515,
+      "learning_rate": 8.167239404352807e-05,
+      "loss": 0.0999,
+      "step": 375
+    },
+    {
+      "epoch": 95.0,
+      "grad_norm": 0.38614600896835327,
+      "learning_rate": 8.13860252004582e-05,
+      "loss": 0.1021,
+      "step": 380
+    },
+    {
+      "epoch": 96.32,
+      "grad_norm": 0.13534197211265564,
+      "learning_rate": 8.109965635738833e-05,
+      "loss": 0.1021,
+      "step": 385
+    },
+    {
+      "epoch": 97.64,
+      "grad_norm": 0.12718380987644196,
+      "learning_rate": 8.081328751431845e-05,
+      "loss": 0.1064,
+      "step": 390
+    },
+    {
+      "epoch": 98.96,
+      "grad_norm": 0.12607838213443756,
+      "learning_rate": 8.052691867124858e-05,
+      "loss": 0.1016,
+      "step": 395
+    },
+    {
+      "epoch": 100.0,
+      "grad_norm": 0.26584750413894653,
+      "learning_rate": 8.02405498281787e-05,
+      "loss": 0.1019,
+      "step": 400
+    },
+    {
+      "epoch": 101.32,
+      "grad_norm": 0.08825451880693436,
+      "learning_rate": 7.995418098510883e-05,
+      "loss": 0.0994,
+      "step": 405
+    },
+    {
+      "epoch": 102.64,
+      "grad_norm": 0.10965154320001602,
+      "learning_rate": 7.966781214203894e-05,
+      "loss": 0.1025,
+      "step": 410
+    },
+    {
+      "epoch": 103.96,
+      "grad_norm": 0.12383153289556503,
+      "learning_rate": 7.938144329896907e-05,
+      "loss": 0.102,
+      "step": 415
+    },
+    {
+      "epoch": 105.0,
+      "grad_norm": 0.325641930103302,
+      "learning_rate": 7.90950744558992e-05,
+      "loss": 0.1014,
+      "step": 420
+    },
+    {
+      "epoch": 106.32,
+      "grad_norm": 0.09022090584039688,
+      "learning_rate": 7.880870561282932e-05,
+      "loss": 0.1005,
+      "step": 425
+    },
+    {
+      "epoch": 107.64,
+      "grad_norm": 0.1068163812160492,
+      "learning_rate": 7.852233676975945e-05,
+      "loss": 0.1037,
+      "step": 430
+    },
+    {
+      "epoch": 108.96,
+      "grad_norm": 0.1266889125108719,
+      "learning_rate": 7.823596792668958e-05,
+      "loss": 0.1011,
+      "step": 435
+    },
+    {
+      "epoch": 110.0,
+      "grad_norm": 0.2371605485677719,
+      "learning_rate": 7.79495990836197e-05,
+      "loss": 0.1009,
+      "step": 440
+    },
+    {
+      "epoch": 111.32,
+      "grad_norm": 0.1123933494091034,
+      "learning_rate": 7.766323024054983e-05,
+      "loss": 0.1,
+      "step": 445
+    },
+    {
+      "epoch": 112.64,
+      "grad_norm": 0.10072564333677292,
+      "learning_rate": 7.737686139747996e-05,
+      "loss": 0.1019,
+      "step": 450
+    },
+    {
+      "epoch": 113.96,
+      "grad_norm": 0.09094753116369247,
+      "learning_rate": 7.709049255441008e-05,
+      "loss": 0.0999,
+      "step": 455
+    },
+    {
+      "epoch": 115.0,
+      "grad_norm": 0.23755896091461182,
+      "learning_rate": 7.680412371134021e-05,
+      "loss": 0.1,
+      "step": 460
+    },
+    {
+      "epoch": 116.32,
+      "grad_norm": 0.09574108570814133,
+      "learning_rate": 7.651775486827034e-05,
+      "loss": 0.1001,
+      "step": 465
+    },
+    {
+      "epoch": 117.64,
+      "grad_norm": 0.10204888880252838,
+      "learning_rate": 7.623138602520046e-05,
+      "loss": 0.0985,
+      "step": 470
+    },
+    {
+      "epoch": 118.96,
+      "grad_norm": 0.12228886038064957,
+      "learning_rate": 7.594501718213059e-05,
+      "loss": 0.1008,
+      "step": 475
+    },
+    {
+      "epoch": 120.0,
+      "grad_norm": 0.3703334331512451,
+      "learning_rate": 7.565864833906071e-05,
+      "loss": 0.1006,
+      "step": 480
+    },
+    {
+      "epoch": 121.32,
+      "grad_norm": 0.09238343685865402,
+      "learning_rate": 7.537227949599084e-05,
+      "loss": 0.0996,
+      "step": 485
+    },
+    {
+      "epoch": 122.64,
+      "grad_norm": 0.08858782798051834,
+      "learning_rate": 7.508591065292097e-05,
+      "loss": 0.1021,
+      "step": 490
+    },
+    {
+      "epoch": 123.96,
+      "grad_norm": 0.07125329226255417,
+      "learning_rate": 7.47995418098511e-05,
+      "loss": 0.1008,
+      "step": 495
+    },
+    {
+      "epoch": 125.0,
+      "grad_norm": 0.186512753367424,
+      "learning_rate": 7.451317296678122e-05,
+      "loss": 0.099,
+      "step": 500
+    },
+    {
+      "epoch": 126.32,
+      "grad_norm": 0.08707877993583679,
+      "learning_rate": 7.422680412371135e-05,
+      "loss": 0.0979,
+      "step": 505
+    },
+    {
+      "epoch": 127.64,
+      "grad_norm": 0.07942412048578262,
+      "learning_rate": 7.394043528064147e-05,
+      "loss": 0.0998,
+      "step": 510
+    },
+    {
+      "epoch": 128.96,
+      "grad_norm": 0.08220551162958145,
+      "learning_rate": 7.36540664375716e-05,
+      "loss": 0.1009,
+      "step": 515
+    },
+    {
+      "epoch": 130.0,
+      "grad_norm": 0.3104532063007355,
+      "learning_rate": 7.336769759450171e-05,
+      "loss": 0.0993,
+      "step": 520
+    },
+    {
+      "epoch": 131.32,
+      "grad_norm": 0.06341040879487991,
+      "learning_rate": 7.308132875143184e-05,
+      "loss": 0.0997,
+      "step": 525
+    },
+    {
+      "epoch": 132.64,
+      "grad_norm": 0.07828553766012192,
+      "learning_rate": 7.279495990836197e-05,
+      "loss": 0.1017,
+      "step": 530
+    },
+    {
+      "epoch": 133.96,
+      "grad_norm": 0.07414849102497101,
+      "learning_rate": 7.250859106529209e-05,
+      "loss": 0.0999,
+      "step": 535
+    },
+    {
+      "epoch": 135.0,
+      "grad_norm": 0.34112414717674255,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 0.1025,
+      "step": 540
+    },
+    {
+      "epoch": 136.32,
+      "grad_norm": 0.07769013941287994,
+      "learning_rate": 7.193585337915235e-05,
+      "loss": 0.1013,
+      "step": 545
+    },
+    {
+      "epoch": 137.64,
+      "grad_norm": 0.06875228136777878,
+      "learning_rate": 7.164948453608247e-05,
+      "loss": 0.1051,
+      "step": 550
+    },
+    {
+      "epoch": 138.96,
+      "grad_norm": 0.07767084240913391,
+      "learning_rate": 7.136311569301261e-05,
+      "loss": 0.1005,
+      "step": 555
+    },
+    {
+      "epoch": 140.0,
+      "grad_norm": 0.23940159380435944,
+      "learning_rate": 7.107674684994274e-05,
+      "loss": 0.0992,
+      "step": 560
+    },
+    {
+      "epoch": 141.32,
+      "grad_norm": 0.08386550843715668,
+      "learning_rate": 7.079037800687286e-05,
+      "loss": 0.1005,
+      "step": 565
+    },
+    {
+      "epoch": 142.64,
+      "grad_norm": 0.08115385472774506,
+      "learning_rate": 7.050400916380299e-05,
+      "loss": 0.1001,
+      "step": 570
+    },
+    {
+      "epoch": 143.96,
+      "grad_norm": 0.07699214667081833,
+      "learning_rate": 7.02176403207331e-05,
+      "loss": 0.0999,
+      "step": 575
+    },
+    {
+      "epoch": 145.0,
+      "grad_norm": 0.24153083562850952,
+      "learning_rate": 6.993127147766323e-05,
+      "loss": 0.101,
+      "step": 580
+    },
+    {
+      "epoch": 146.32,
+      "grad_norm": 0.08120574802160263,
+      "learning_rate": 6.964490263459336e-05,
+      "loss": 0.099,
+      "step": 585
+    },
+    {
+      "epoch": 147.64,
+      "grad_norm": 0.07880707830190659,
+      "learning_rate": 6.935853379152348e-05,
+      "loss": 0.0998,
+      "step": 590
+    },
+    {
+      "epoch": 148.96,
+      "grad_norm": 0.08910374343395233,
+      "learning_rate": 6.907216494845361e-05,
+      "loss": 0.0995,
+      "step": 595
+    },
+    {
+      "epoch": 150.0,
+      "grad_norm": 0.20559290051460266,
+      "learning_rate": 6.878579610538374e-05,
+      "loss": 0.1003,
+      "step": 600
+    },
+    {
+      "epoch": 151.32,
+      "grad_norm": 0.08372558653354645,
+      "learning_rate": 6.849942726231386e-05,
+      "loss": 0.1007,
+      "step": 605
+    },
+    {
+      "epoch": 152.64,
+      "grad_norm": 0.08071479201316833,
+      "learning_rate": 6.821305841924399e-05,
+      "loss": 0.0994,
+      "step": 610
+    },
+    {
+      "epoch": 153.96,
+      "grad_norm": 0.07804345339536667,
+      "learning_rate": 6.792668957617412e-05,
+      "loss": 0.0982,
+      "step": 615
+    },
+    {
+      "epoch": 155.0,
+      "grad_norm": 0.25985032320022583,
+      "learning_rate": 6.764032073310424e-05,
+      "loss": 0.1,
+      "step": 620
+    },
+    {
+      "epoch": 156.32,
+      "grad_norm": 0.07368581742048264,
+      "learning_rate": 6.735395189003437e-05,
+      "loss": 0.0997,
+      "step": 625
+    },
+    {
+      "epoch": 157.64,
+      "grad_norm": 0.06839299947023392,
+      "learning_rate": 6.706758304696448e-05,
+      "loss": 0.0992,
+      "step": 630
+    },
+    {
+      "epoch": 158.96,
+      "grad_norm": 0.08271060138940811,
+      "learning_rate": 6.678121420389462e-05,
+      "loss": 0.098,
+      "step": 635
+    },
+    {
+      "epoch": 160.0,
+      "grad_norm": 0.22405998408794403,
+      "learning_rate": 6.649484536082475e-05,
+      "loss": 0.0995,
+      "step": 640
+    },
+    {
+      "epoch": 161.32,
+      "grad_norm": 0.08114491403102875,
+      "learning_rate": 6.620847651775487e-05,
+      "loss": 0.1002,
+      "step": 645
+    },
+    {
+      "epoch": 162.64,
+      "grad_norm": 0.0626523420214653,
+      "learning_rate": 6.5922107674685e-05,
+      "loss": 0.0984,
+      "step": 650
+    },
+    {
+      "epoch": 163.96,
+      "grad_norm": 0.08073456585407257,
+      "learning_rate": 6.563573883161513e-05,
+      "loss": 0.0997,
+      "step": 655
+    },
+    {
+      "epoch": 165.0,
+      "grad_norm": 0.17625287175178528,
+      "learning_rate": 6.534936998854525e-05,
+      "loss": 0.0978,
+      "step": 660
+    },
+    {
+      "epoch": 166.32,
+      "grad_norm": 0.0576891154050827,
+      "learning_rate": 6.506300114547538e-05,
+      "loss": 0.098,
+      "step": 665
+    },
+    {
+      "epoch": 167.64,
+      "grad_norm": 0.061594679951667786,
+      "learning_rate": 6.477663230240551e-05,
+      "loss": 0.0989,
+      "step": 670
+    },
+    {
+      "epoch": 168.96,
+      "grad_norm": 0.07297411561012268,
+      "learning_rate": 6.449026345933563e-05,
+      "loss": 0.0999,
+      "step": 675
+    },
+    {
+      "epoch": 170.0,
+      "grad_norm": 0.2650276720523834,
+      "learning_rate": 6.420389461626576e-05,
+      "loss": 0.0991,
+      "step": 680
+    },
+    {
+      "epoch": 171.32,
+      "grad_norm": 0.07988583296537399,
+      "learning_rate": 6.391752577319587e-05,
+      "loss": 0.0999,
+      "step": 685
+    },
+    {
+      "epoch": 172.64,
+      "grad_norm": 0.08536892384290695,
+      "learning_rate": 6.3631156930126e-05,
+      "loss": 0.0988,
+      "step": 690
+    },
+    {
+      "epoch": 173.96,
+      "grad_norm": 0.07444474846124649,
+      "learning_rate": 6.334478808705613e-05,
+      "loss": 0.0988,
+      "step": 695
+    },
+    {
+      "epoch": 175.0,
+      "grad_norm": 0.18621835112571716,
+      "learning_rate": 6.305841924398625e-05,
+      "loss": 0.0981,
+      "step": 700
+    },
+    {
+      "epoch": 176.32,
+      "grad_norm": 0.06464474648237228,
+      "learning_rate": 6.277205040091638e-05,
+      "loss": 0.0986,
+      "step": 705
+    },
+    {
+      "epoch": 177.64,
+      "grad_norm": 0.0667959451675415,
+      "learning_rate": 6.24856815578465e-05,
+      "loss": 0.0987,
+      "step": 710
+    },
+    {
+      "epoch": 178.96,
+      "grad_norm": 0.08776934444904327,
+      "learning_rate": 6.219931271477663e-05,
+      "loss": 0.0986,
+      "step": 715
+    },
+    {
+      "epoch": 180.0,
+      "grad_norm": 0.2339775264263153,
+      "learning_rate": 6.191294387170676e-05,
+      "loss": 0.1007,
+      "step": 720
+    },
+    {
+      "epoch": 181.32,
+      "grad_norm": 0.07515422999858856,
+      "learning_rate": 6.162657502863689e-05,
+      "loss": 0.0993,
+      "step": 725
+    },
+    {
+      "epoch": 182.64,
+      "grad_norm": 0.05487849563360214,
+      "learning_rate": 6.134020618556701e-05,
+      "loss": 0.0973,
+      "step": 730
+    },
+    {
+      "epoch": 183.96,
+      "grad_norm": 0.07211857289075851,
+      "learning_rate": 6.105383734249714e-05,
+      "loss": 0.0984,
+      "step": 735
+    },
+    {
+      "epoch": 185.0,
+      "grad_norm": 0.17548519372940063,
+      "learning_rate": 6.076746849942726e-05,
+      "loss": 0.0994,
+      "step": 740
+    },
+    {
+      "epoch": 186.32,
+      "grad_norm": 0.06068222224712372,
+      "learning_rate": 6.0481099656357384e-05,
+      "loss": 0.0979,
+      "step": 745
+    },
+    {
+      "epoch": 187.64,
+      "grad_norm": 0.05800328031182289,
+      "learning_rate": 6.019473081328752e-05,
+      "loss": 0.0994,
+      "step": 750
+    },
+    {
+      "epoch": 188.96,
+      "grad_norm": 0.07508601248264313,
+      "learning_rate": 5.9908361970217644e-05,
+      "loss": 0.0985,
+      "step": 755
+    },
+    {
+      "epoch": 190.0,
+      "grad_norm": 0.21594305336475372,
+      "learning_rate": 5.962199312714777e-05,
+      "loss": 0.0985,
+      "step": 760
+    },
+    {
+      "epoch": 191.32,
+      "grad_norm": 0.05984228849411011,
+      "learning_rate": 5.93356242840779e-05,
+      "loss": 0.0988,
+      "step": 765
+    },
+    {
+      "epoch": 192.64,
+      "grad_norm": 0.09342896938323975,
+      "learning_rate": 5.904925544100802e-05,
+      "loss": 0.0997,
+      "step": 770
+    },
+    {
+      "epoch": 193.96,
+      "grad_norm": 0.05633103474974632,
+      "learning_rate": 5.876288659793815e-05,
+      "loss": 0.0985,
+      "step": 775
+    },
+    {
+      "epoch": 195.0,
+      "grad_norm": 0.2114800661802292,
+      "learning_rate": 5.8476517754868276e-05,
+      "loss": 0.0994,
+      "step": 780
+    },
+    {
+      "epoch": 196.32,
+      "grad_norm": 0.06558126211166382,
+      "learning_rate": 5.81901489117984e-05,
+      "loss": 0.0987,
+      "step": 785
+    },
+    {
+      "epoch": 197.64,
+      "grad_norm": 0.053430985659360886,
+      "learning_rate": 5.790378006872853e-05,
+      "loss": 0.0987,
+      "step": 790
+    },
+    {
+      "epoch": 198.96,
+      "grad_norm": 0.10079631209373474,
+      "learning_rate": 5.761741122565865e-05,
+      "loss": 0.1008,
+      "step": 795
+    },
+    {
+      "epoch": 200.0,
+      "grad_norm": 0.224177747964859,
+      "learning_rate": 5.7331042382588775e-05,
+      "loss": 0.1014,
+      "step": 800
+    },
+    {
+      "epoch": 201.32,
+      "grad_norm": 0.06240411847829819,
+      "learning_rate": 5.70446735395189e-05,
+      "loss": 0.0986,
+      "step": 805
+    },
+    {
+      "epoch": 202.64,
+      "grad_norm": 0.05734168365597725,
+      "learning_rate": 5.675830469644903e-05,
+      "loss": 0.0979,
+      "step": 810
+    },
+    {
+      "epoch": 203.96,
+      "grad_norm": 0.06485852599143982,
+      "learning_rate": 5.6471935853379155e-05,
+      "loss": 0.0991,
+      "step": 815
+    },
+    {
+      "epoch": 205.0,
+      "grad_norm": 0.20834438502788544,
+      "learning_rate": 5.618556701030928e-05,
+      "loss": 0.1002,
+      "step": 820
+    },
+    {
+      "epoch": 206.32,
+      "grad_norm": 0.06883325427770615,
+      "learning_rate": 5.589919816723941e-05,
+      "loss": 0.0981,
+      "step": 825
+    },
+    {
+      "epoch": 207.64,
+      "grad_norm": 0.06779716163873672,
+      "learning_rate": 5.5612829324169534e-05,
+      "loss": 0.0987,
+      "step": 830
+    },
+    {
+      "epoch": 208.96,
+      "grad_norm": 0.07835811376571655,
+      "learning_rate": 5.532646048109966e-05,
+      "loss": 0.0991,
+      "step": 835
+    },
+    {
+      "epoch": 210.0,
+      "grad_norm": 0.22523128986358643,
+      "learning_rate": 5.504009163802979e-05,
+      "loss": 0.0985,
+      "step": 840
+    },
+    {
+      "epoch": 211.32,
+      "grad_norm": 0.06611000746488571,
+      "learning_rate": 5.4753722794959914e-05,
+      "loss": 0.0986,
+      "step": 845
+    },
+    {
+      "epoch": 212.64,
+      "grad_norm": 0.06890663504600525,
+      "learning_rate": 5.4467353951890033e-05,
+      "loss": 0.0986,
+      "step": 850
+    },
+    {
+      "epoch": 213.96,
+      "grad_norm": 0.06495572626590729,
+      "learning_rate": 5.418098510882016e-05,
+      "loss": 0.0994,
+      "step": 855
+    },
+    {
+      "epoch": 215.0,
+      "grad_norm": 0.22761495411396027,
+      "learning_rate": 5.3894616265750286e-05,
+      "loss": 0.0993,
+      "step": 860
+    },
+    {
+      "epoch": 216.32,
+      "grad_norm": 0.08932021260261536,
+      "learning_rate": 5.360824742268041e-05,
+      "loss": 0.0999,
+      "step": 865
+    },
+    {
+      "epoch": 217.64,
+      "grad_norm": 0.06438680738210678,
+      "learning_rate": 5.332187857961054e-05,
+      "loss": 0.0987,
+      "step": 870
+    },
+    {
+      "epoch": 218.96,
+      "grad_norm": 0.05768786370754242,
+      "learning_rate": 5.3035509736540666e-05,
+      "loss": 0.0976,
+      "step": 875
+    },
+    {
+      "epoch": 220.0,
+      "grad_norm": 0.1853507161140442,
+      "learning_rate": 5.274914089347079e-05,
+      "loss": 0.1006,
+      "step": 880
+    },
+    {
+      "epoch": 221.32,
+      "grad_norm": 0.05133757367730141,
+      "learning_rate": 5.246277205040092e-05,
+      "loss": 0.0981,
+      "step": 885
+    },
+    {
+      "epoch": 222.64,
+      "grad_norm": 0.05265878885984421,
+      "learning_rate": 5.2176403207331045e-05,
+      "loss": 0.099,
+      "step": 890
+    },
+    {
+      "epoch": 223.96,
+      "grad_norm": 0.06624596565961838,
+      "learning_rate": 5.189003436426118e-05,
+      "loss": 0.0983,
+      "step": 895
+    },
+    {
+      "epoch": 225.0,
+      "grad_norm": 0.20079456269741058,
+      "learning_rate": 5.1603665521191305e-05,
+      "loss": 0.0985,
+      "step": 900
+    },
+    {
+      "epoch": 226.32,
+      "grad_norm": 0.05243641883134842,
+      "learning_rate": 5.131729667812142e-05,
+      "loss": 0.0984,
+      "step": 905
+    },
+    {
+      "epoch": 227.64,
+      "grad_norm": 0.054776884615421295,
+      "learning_rate": 5.1030927835051544e-05,
+      "loss": 0.0977,
+      "step": 910
+    },
+    {
+      "epoch": 228.96,
+      "grad_norm": 0.08650607615709305,
+      "learning_rate": 5.074455899198167e-05,
+      "loss": 0.0997,
+      "step": 915
+    },
+    {
+      "epoch": 230.0,
+      "grad_norm": 0.22193726897239685,
+      "learning_rate": 5.04581901489118e-05,
+      "loss": 0.0995,
+      "step": 920
+    },
+    {
+      "epoch": 231.32,
+      "grad_norm": 0.05722770839929581,
+      "learning_rate": 5.0171821305841924e-05,
+      "loss": 0.0979,
+      "step": 925
+    },
+    {
+      "epoch": 232.64,
+      "grad_norm": 0.06476964801549911,
+      "learning_rate": 4.988545246277205e-05,
+      "loss": 0.0976,
+      "step": 930
+    },
+    {
+      "epoch": 233.96,
+      "grad_norm": 0.05505547672510147,
+      "learning_rate": 4.9599083619702184e-05,
+      "loss": 0.0989,
+      "step": 935
+    },
+    {
+      "epoch": 235.0,
+      "grad_norm": 0.19139918684959412,
+      "learning_rate": 4.931271477663231e-05,
+      "loss": 0.098,
+      "step": 940
+    },
+    {
+      "epoch": 236.32,
+      "grad_norm": 0.0507647879421711,
+      "learning_rate": 4.902634593356243e-05,
+      "loss": 0.0976,
+      "step": 945
+    },
+    {
+      "epoch": 237.64,
+      "grad_norm": 0.05280376970767975,
+      "learning_rate": 4.8739977090492556e-05,
+      "loss": 0.0988,
+      "step": 950
+    },
+    {
+      "epoch": 238.96,
+      "grad_norm": 0.060991156846284866,
+      "learning_rate": 4.845360824742268e-05,
+      "loss": 0.0982,
+      "step": 955
+    },
+    {
+      "epoch": 240.0,
+      "grad_norm": 0.20928047597408295,
+      "learning_rate": 4.816723940435281e-05,
+      "loss": 0.097,
+      "step": 960
+    },
+    {
+      "epoch": 241.32,
+      "grad_norm": 0.06290781497955322,
+      "learning_rate": 4.7880870561282936e-05,
+      "loss": 0.0983,
+      "step": 965
+    },
+    {
+      "epoch": 242.64,
+      "grad_norm": 0.05372194945812225,
+      "learning_rate": 4.7594501718213055e-05,
+      "loss": 0.0982,
+      "step": 970
+    },
+    {
+      "epoch": 243.96,
+      "grad_norm": 0.07552998512983322,
+      "learning_rate": 4.730813287514318e-05,
+      "loss": 0.0989,
+      "step": 975
+    },
+    {
+      "epoch": 245.0,
+      "grad_norm": 0.23483139276504517,
+      "learning_rate": 4.7021764032073315e-05,
+      "loss": 0.0978,
+      "step": 980
+    },
+    {
+      "epoch": 246.32,
+      "grad_norm": 0.05948429927229881,
+      "learning_rate": 4.673539518900344e-05,
+      "loss": 0.0986,
+      "step": 985
+    },
+    {
+      "epoch": 247.64,
+      "grad_norm": 0.061117224395275116,
+      "learning_rate": 4.644902634593357e-05,
+      "loss": 0.0984,
+      "step": 990
+    },
+    {
+      "epoch": 248.96,
+      "grad_norm": 0.055287718772888184,
+      "learning_rate": 4.6162657502863694e-05,
+      "loss": 0.1004,
+      "step": 995
+    },
+    {
+      "epoch": 250.0,
+      "grad_norm": 0.17993661761283875,
+      "learning_rate": 4.5876288659793814e-05,
+      "loss": 0.0973,
+      "step": 1000
+    },
+    {
+      "epoch": 251.32,
+      "grad_norm": 0.07195509970188141,
+      "learning_rate": 4.558991981672394e-05,
+      "loss": 0.098,
+      "step": 1005
+    },
+    {
+      "epoch": 252.64,
+      "grad_norm": 0.062129341065883636,
+      "learning_rate": 4.530355097365407e-05,
+      "loss": 0.0983,
+      "step": 1010
+    },
+    {
+      "epoch": 253.96,
+      "grad_norm": 0.05049443244934082,
+      "learning_rate": 4.5017182130584194e-05,
+      "loss": 0.0976,
+      "step": 1015
+    },
+    {
+      "epoch": 255.0,
+      "grad_norm": 0.20472285151481628,
+      "learning_rate": 4.473081328751432e-05,
+      "loss": 0.0995,
+      "step": 1020
+    },
+    {
+      "epoch": 256.32,
+      "grad_norm": 0.0742129236459732,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 0.0997,
+      "step": 1025
+    },
+    {
+      "epoch": 257.64,
+      "grad_norm": 0.08675533533096313,
+      "learning_rate": 4.415807560137457e-05,
+      "loss": 0.0987,
+      "step": 1030
+    },
+    {
+      "epoch": 258.96,
+      "grad_norm": 0.055044736713171005,
+      "learning_rate": 4.38717067583047e-05,
+      "loss": 0.099,
+      "step": 1035
+    },
+    {
+      "epoch": 260.0,
+      "grad_norm": 0.13128946721553802,
+      "learning_rate": 4.3585337915234826e-05,
+      "loss": 0.0971,
+      "step": 1040
+    },
+    {
+      "epoch": 261.32,
+      "grad_norm": 0.05028077960014343,
+      "learning_rate": 4.329896907216495e-05,
+      "loss": 0.0993,
+      "step": 1045
+    },
+    {
+      "epoch": 262.64,
+      "grad_norm": 0.05382866412401199,
+      "learning_rate": 4.301260022909508e-05,
+      "loss": 0.0977,
+      "step": 1050
+    },
+    {
+      "epoch": 263.96,
+      "grad_norm": 0.052796345204114914,
+      "learning_rate": 4.27262313860252e-05,
+      "loss": 0.0993,
+      "step": 1055
+    },
+    {
+      "epoch": 265.0,
+      "grad_norm": 0.15410888195037842,
+      "learning_rate": 4.2439862542955325e-05,
+      "loss": 0.0985,
+      "step": 1060
+    },
+    {
+      "epoch": 266.32,
+      "grad_norm": 0.04920932650566101,
+      "learning_rate": 4.215349369988545e-05,
+      "loss": 0.0969,
+      "step": 1065
+    },
+    {
+      "epoch": 267.64,
+      "grad_norm": 0.06144626811146736,
+      "learning_rate": 4.1867124856815585e-05,
+      "loss": 0.0988,
+      "step": 1070
+    },
+    {
+      "epoch": 268.96,
+      "grad_norm": 0.08298223465681076,
+      "learning_rate": 4.158075601374571e-05,
+      "loss": 0.0986,
+      "step": 1075
+    },
+    {
+      "epoch": 270.0,
+      "grad_norm": 0.15285271406173706,
+      "learning_rate": 4.129438717067583e-05,
+      "loss": 0.098,
+      "step": 1080
+    },
+    {
+      "epoch": 271.32,
+      "grad_norm": 0.05147601664066315,
+      "learning_rate": 4.100801832760596e-05,
+      "loss": 0.0994,
+      "step": 1085
+    },
+    {
+      "epoch": 272.64,
+      "grad_norm": 0.0584992878139019,
+      "learning_rate": 4.0721649484536084e-05,
+      "loss": 0.0972,
+      "step": 1090
+    },
+    {
+      "epoch": 273.96,
+      "grad_norm": 0.05973227322101593,
+      "learning_rate": 4.043528064146621e-05,
+      "loss": 0.0977,
+      "step": 1095
+    },
+    {
+      "epoch": 275.0,
+      "grad_norm": 0.13904881477355957,
+      "learning_rate": 4.014891179839634e-05,
+      "loss": 0.0962,
+      "step": 1100
+    },
+    {
+      "epoch": 276.32,
+      "grad_norm": 0.059971824288368225,
+      "learning_rate": 3.9862542955326463e-05,
+      "loss": 0.0982,
+      "step": 1105
+    },
+    {
+      "epoch": 277.64,
+      "grad_norm": 0.055378515273332596,
+      "learning_rate": 3.957617411225659e-05,
+      "loss": 0.0977,
+      "step": 1110
+    },
+    {
+      "epoch": 278.96,
+      "grad_norm": 0.06384849548339844,
+      "learning_rate": 3.9289805269186716e-05,
+      "loss": 0.099,
+      "step": 1115
+    },
+    {
+      "epoch": 280.0,
+      "grad_norm": 0.18498972058296204,
+      "learning_rate": 3.900343642611684e-05,
+      "loss": 0.1003,
+      "step": 1120
+    },
+    {
+      "epoch": 281.32,
+      "grad_norm": 0.06320986896753311,
+      "learning_rate": 3.871706758304697e-05,
+      "loss": 0.0972,
+      "step": 1125
+    },
+    {
+      "epoch": 282.64,
+      "grad_norm": 0.04780237749218941,
+      "learning_rate": 3.8430698739977096e-05,
+      "loss": 0.0985,
+      "step": 1130
+    },
+    {
+      "epoch": 283.96,
+      "grad_norm": 0.04820997267961502,
+      "learning_rate": 3.8144329896907216e-05,
+      "loss": 0.0994,
+      "step": 1135
+    },
+    {
+      "epoch": 285.0,
+      "grad_norm": 0.16919052600860596,
+      "learning_rate": 3.785796105383734e-05,
+      "loss": 0.0995,
+      "step": 1140
+    },
+    {
+      "epoch": 286.32,
+      "grad_norm": 0.05371937155723572,
+      "learning_rate": 3.757159221076747e-05,
+      "loss": 0.0992,
+      "step": 1145
+    },
+    {
+      "epoch": 287.64,
+      "grad_norm": 0.05926801636815071,
+      "learning_rate": 3.7285223367697595e-05,
+      "loss": 0.0987,
+      "step": 1150
+    },
+    {
+      "epoch": 288.96,
+      "grad_norm": 0.05819291993975639,
+      "learning_rate": 3.699885452462772e-05,
+      "loss": 0.0969,
+      "step": 1155
+    },
+    {
+      "epoch": 290.0,
+      "grad_norm": 0.17235444486141205,
+      "learning_rate": 3.671248568155785e-05,
+      "loss": 0.0961,
+      "step": 1160
+    },
+    {
+      "epoch": 291.32,
+      "grad_norm": 0.05417826771736145,
+      "learning_rate": 3.6426116838487974e-05,
+      "loss": 0.0974,
+      "step": 1165
+    },
+    {
+      "epoch": 292.64,
+      "grad_norm": 0.06401331722736359,
+      "learning_rate": 3.61397479954181e-05,
+      "loss": 0.0992,
+      "step": 1170
+    },
+    {
+      "epoch": 293.96,
+      "grad_norm": 0.05331256985664368,
+      "learning_rate": 3.585337915234823e-05,
+      "loss": 0.1009,
+      "step": 1175
+    },
+    {
+      "epoch": 295.0,
+      "grad_norm": 0.2026655226945877,
+      "learning_rate": 3.5567010309278354e-05,
+      "loss": 0.0986,
+      "step": 1180
+    },
+    {
+      "epoch": 296.32,
+      "grad_norm": 0.06182396784424782,
+      "learning_rate": 3.528064146620848e-05,
+      "loss": 0.0984,
+      "step": 1185
+    },
+    {
+      "epoch": 297.64,
+      "grad_norm": 0.054141268134117126,
+      "learning_rate": 3.49942726231386e-05,
+      "loss": 0.0976,
+      "step": 1190
+    },
+    {
+      "epoch": 298.96,
+      "grad_norm": 0.054891202598810196,
+      "learning_rate": 3.4707903780068726e-05,
+      "loss": 0.0993,
+      "step": 1195
+    },
+    {
+      "epoch": 300.0,
+      "grad_norm": 0.17108815908432007,
+      "learning_rate": 3.442153493699885e-05,
+      "loss": 0.0978,
+      "step": 1200
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 1800,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 450,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.0205427400704e+18,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}