Upload folder using huggingface_hub

README.md

@@ -1,199 +1,58 @@
 ---
+base_model: Qwen/Qwen3-1.7B
 library_name: transformers
-tags: []
+model_name: sft
+tags:
+- generated_from_trainer
+- trl
+- sft
+licence: license
 ---
 
-# Model Card for Model ID
+# Model Card for sft
 
-<!-- Provide a quick summary of what the model is/does. -->
+This model is a fine-tuned version of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B).
+It has been trained using [TRL](https://github.com/huggingface/trl).
 
+## Quick start
 
+```python
+from transformers import pipeline
 
-## Model Details
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
 
-### Model Description
+## Training procedure
 
-<!-- Provide a longer summary of what this model is. -->
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/duo_run/huggingface/runs/r0gc2y2j) 
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
 
-### Model Sources [optional]
+This model was trained with SFT.
 
-<!-- Provide the basic links for the model. -->
+### Framework versions
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- TRL: 1.3.0
+- Transformers: 5.7.0
+- Pytorch: 2.10.0+cu128
+- Datasets: 4.8.5
+- Tokenizers: 0.22.2
 
-## Uses
+## Citations
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

chat_template.jinja

@@ -1,3 +1,4 @@
+{%- set enable_thinking = true %}
 {%- if tools %}
     {{- '<|im_start|>system\n' }}
     {%- if messages[0].role == 'system' %}
@@ -83,4 +84,7 @@
 {%- endfor %}
 {%- if add_generation_prompt %}
     {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
 {%- endif %}

checkpoint-600/chat_template.jinja

@@ -0,0 +1,90 @@
+{%- set enable_thinking = true %}
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-600/config.json

@@ -0,0 +1,63 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 40960,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151645,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.7.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

checkpoint-600/generation_config.json

@@ -0,0 +1,12 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151645,
+  "temperature": 0.6,
+  "top_k": 20,
+  "top_p": 0.95,
+  "transformers_version": "5.7.0"
+}

checkpoint-600/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce94791b73aae1ca2649e5d3a4c2c413fa21ecaec0c03dbbdaf85bd46e622ebc
+size 3441185608

checkpoint-600/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d706d13236aa53a72d1d45f89f42b9960421acedbb4dac47a5a66a877686f31c
+size 6882572207

checkpoint-600/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645

checkpoint-600/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:20a96f9d642685b227368306bbebca0caa781c13ea0e60d952755dbc93f8c001
+size 1465

checkpoint-600/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

checkpoint-600/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-600/trainer_state.json

@@ -0,0 +1,634 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.8,
+  "eval_steps": 500,
+  "global_step": 600,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 0.4763989023864269,
+      "epoch": 0.013333333333333334,
+      "grad_norm": 39.5,
+      "learning_rate": 4.736842105263158e-06,
+      "loss": 2.0091196060180665,
+      "mean_token_accuracy": 0.6925897747278214,
+      "num_tokens": 42464.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.5551879126578569,
+      "epoch": 0.02666666666666667,
+      "grad_norm": 12.125,
+      "learning_rate": 1e-05,
+      "loss": 1.332655143737793,
+      "mean_token_accuracy": 0.7217346377670765,
+      "num_tokens": 96049.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.7102062199264765,
+      "epoch": 0.04,
+      "grad_norm": 4.9375,
+      "learning_rate": 1.5263157894736846e-05,
+      "loss": 0.8764092445373535,
+      "mean_token_accuracy": 0.7632560275495053,
+      "num_tokens": 140505.0,
+      "step": 30
+    },
+    {
+      "entropy": 0.7993051432073116,
+      "epoch": 0.05333333333333334,
+      "grad_norm": 4.5625,
+      "learning_rate": 1.9999902656029183e-05,
+      "loss": 0.7961350917816162,
+      "mean_token_accuracy": 0.7790301404893398,
+      "num_tokens": 186827.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.7229622758924961,
+      "epoch": 0.06666666666666667,
+      "grad_norm": 2.578125,
+      "learning_rate": 1.9988223672512097e-05,
+      "loss": 0.7293992042541504,
+      "mean_token_accuracy": 0.7856058329343796,
+      "num_tokens": 237512.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.706776586547494,
+      "epoch": 0.08,
+      "grad_norm": 3.125,
+      "learning_rate": 1.995710194507367e-05,
+      "loss": 0.7110776901245117,
+      "mean_token_accuracy": 0.7912242174148559,
+      "num_tokens": 285719.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.6807167779654264,
+      "epoch": 0.09333333333333334,
+      "grad_norm": 2.59375,
+      "learning_rate": 1.9906598054233197e-05,
+      "loss": 0.6580380439758301,
+      "mean_token_accuracy": 0.7998592600226402,
+      "num_tokens": 344554.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.7038780480623246,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 4.0,
+      "learning_rate": 1.9836810309184468e-05,
+      "loss": 0.6892334938049316,
+      "mean_token_accuracy": 0.7912278659641743,
+      "num_tokens": 398019.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.6895901430398226,
+      "epoch": 0.12,
+      "grad_norm": 2.84375,
+      "learning_rate": 1.9747874556430363e-05,
+      "loss": 0.6818968772888183,
+      "mean_token_accuracy": 0.7969276301562787,
+      "num_tokens": 447685.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7114015109837055,
+      "epoch": 0.13333333333333333,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.9639963915348547e-05,
+      "loss": 0.6934291362762451,
+      "mean_token_accuracy": 0.7938909940421581,
+      "num_tokens": 497899.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.7047938119620085,
+      "epoch": 0.14666666666666667,
+      "grad_norm": 3.484375,
+      "learning_rate": 1.9513288441203083e-05,
+      "loss": 0.6858394145965576,
+      "mean_token_accuracy": 0.7970494098961354,
+      "num_tokens": 548312.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.67772804684937,
+      "epoch": 0.16,
+      "grad_norm": 2.21875,
+      "learning_rate": 1.93680947162578e-05,
+      "loss": 0.661228609085083,
+      "mean_token_accuracy": 0.8077008336782455,
+      "num_tokens": 596458.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.6539881706237793,
+      "epoch": 0.17333333333333334,
+      "grad_norm": 3.140625,
+      "learning_rate": 1.9204665369787507e-05,
+      "loss": 0.6504672527313232,
+      "mean_token_accuracy": 0.8092843152582645,
+      "num_tokens": 649939.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.686749292910099,
+      "epoch": 0.18666666666666668,
+      "grad_norm": 2.953125,
+      "learning_rate": 1.9023318527921213e-05,
+      "loss": 0.6639729499816894,
+      "mean_token_accuracy": 0.7960164114832878,
+      "num_tokens": 696515.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.6799734767526389,
+      "epoch": 0.2,
+      "grad_norm": 2.625,
+      "learning_rate": 1.8824407194388408e-05,
+      "loss": 0.6643428802490234,
+      "mean_token_accuracy": 0.8016563065350055,
+      "num_tokens": 745199.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.65655233040452,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 3.15625,
+      "learning_rate": 1.8608318563373735e-05,
+      "loss": 0.6439759254455566,
+      "mean_token_accuracy": 0.8110810227692127,
+      "num_tokens": 793167.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.6463201232254505,
+      "epoch": 0.22666666666666666,
+      "grad_norm": 2.890625,
+      "learning_rate": 1.837547326581768e-05,
+      "loss": 0.6181783199310302,
+      "mean_token_accuracy": 0.8121586792171002,
+      "num_tokens": 844567.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.6676008347421885,
+      "epoch": 0.24,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.8126324550630364e-05,
+      "loss": 0.6397620677947998,
+      "mean_token_accuracy": 0.8043703198432922,
+      "num_tokens": 896377.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.6894530899822712,
+      "epoch": 0.25333333333333335,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.786135740241232e-05,
+      "loss": 0.6637216091156006,
+      "mean_token_accuracy": 0.8027763769030571,
+      "num_tokens": 946438.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.6975695930421353,
+      "epoch": 0.26666666666666666,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.7581087597399556e-05,
+      "loss": 0.6652172088623047,
+      "mean_token_accuracy": 0.7989616326987743,
+      "num_tokens": 995473.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.6387956503778696,
+      "epoch": 0.28,
+      "grad_norm": 2.75,
+      "learning_rate": 1.7286060699470746e-05,
+      "loss": 0.6128511905670166,
+      "mean_token_accuracy": 0.8127545997500419,
+      "num_tokens": 1045963.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.6565726511180401,
+      "epoch": 0.29333333333333333,
+      "grad_norm": 3.3125,
+      "learning_rate": 1.69768509981707e-05,
+      "loss": 0.6489222526550293,
+      "mean_token_accuracy": 0.8057824857532978,
+      "num_tokens": 1096633.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.6619438689202071,
+      "epoch": 0.30666666666666664,
+      "grad_norm": 2.734375,
+      "learning_rate": 1.6654060390817445e-05,
+      "loss": 0.6426784038543701,
+      "mean_token_accuracy": 0.809200967848301,
+      "num_tokens": 1149606.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.6764651168137789,
+      "epoch": 0.32,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.6318317210868936e-05,
+      "loss": 0.6661844730377198,
+      "mean_token_accuracy": 0.8004243083298206,
+      "num_tokens": 1198813.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.684938239119947,
+      "epoch": 0.3333333333333333,
+      "grad_norm": 3.015625,
+      "learning_rate": 1.5970275004830053e-05,
+      "loss": 0.649436616897583,
+      "mean_token_accuracy": 0.8030574709177017,
+      "num_tokens": 1245303.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.6592010721564293,
+      "epoch": 0.3466666666666667,
+      "grad_norm": 2.640625,
+      "learning_rate": 1.56106112600807e-05,
+      "loss": 0.6452194213867187,
+      "mean_token_accuracy": 0.8091069459915161,
+      "num_tokens": 1293082.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.6599753946065903,
+      "epoch": 0.36,
+      "grad_norm": 2.671875,
+      "learning_rate": 1.5240026086101425e-05,
+      "loss": 0.6302877902984619,
+      "mean_token_accuracy": 0.8083149395883084,
+      "num_tokens": 1345989.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.6778957467526198,
+      "epoch": 0.37333333333333335,
+      "grad_norm": 2.703125,
+      "learning_rate": 1.485924085166359e-05,
+      "loss": 0.653476333618164,
+      "mean_token_accuracy": 0.8066982626914978,
+      "num_tokens": 1395698.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.648598388582468,
+      "epoch": 0.38666666666666666,
+      "grad_norm": 2.953125,
+      "learning_rate": 1.4468996780636922e-05,
+      "loss": 0.6419031143188476,
+      "mean_token_accuracy": 0.8062063075602055,
+      "num_tokens": 1449209.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.6481347244232893,
+      "epoch": 0.4,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.4070053509147777e-05,
+      "loss": 0.6035490512847901,
+      "mean_token_accuracy": 0.8153042212128639,
+      "num_tokens": 1499225.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7093129184097051,
+      "epoch": 0.41333333333333333,
+      "grad_norm": 2.984375,
+      "learning_rate": 1.3663187606896733e-05,
+      "loss": 0.7133637428283691,
+      "mean_token_accuracy": 0.7937003694474697,
+      "num_tokens": 1542301.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.6543151158839464,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 3.234375,
+      "learning_rate": 1.3249191065513857e-05,
+      "loss": 0.6388350486755371,
+      "mean_token_accuracy": 0.8044970251619816,
+      "num_tokens": 1596800.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.6799626916646957,
+      "epoch": 0.44,
+      "grad_norm": 2.09375,
+      "learning_rate": 1.2828869756894122e-05,
+      "loss": 0.6594656944274903,
+      "mean_token_accuracy": 0.8026202499866486,
+      "num_tokens": 1648619.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.680319806188345,
+      "epoch": 0.4533333333333333,
+      "grad_norm": 2.4375,
+      "learning_rate": 1.2403041864514005e-05,
+      "loss": 0.6563356876373291,
+      "mean_token_accuracy": 0.8025765180587768,
+      "num_tokens": 1703363.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.6631284486502409,
+      "epoch": 0.4666666666666667,
+      "grad_norm": 2.84375,
+      "learning_rate": 1.1972536290782738e-05,
+      "loss": 0.6403236865997315,
+      "mean_token_accuracy": 0.8110226787626743,
+      "num_tokens": 1754906.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.7027981210500002,
+      "epoch": 0.48,
+      "grad_norm": 3.5,
+      "learning_rate": 1.153819104352846e-05,
+      "loss": 0.6743718147277832,
+      "mean_token_accuracy": 0.7984775453805923,
+      "num_tokens": 1796484.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.6734891567379236,
+      "epoch": 0.49333333333333335,
+      "grad_norm": 2.140625,
+      "learning_rate": 1.1100851604760057e-05,
+      "loss": 0.6835583686828614,
+      "mean_token_accuracy": 0.7995599187910557,
+      "num_tokens": 1845839.0,
+      "step": 370
+    },
+    {
+      "entropy": 0.6895672902464867,
+      "epoch": 0.5066666666666667,
+      "grad_norm": 3.171875,
+      "learning_rate": 1.0661369284880005e-05,
+      "loss": 0.675282096862793,
+      "mean_token_accuracy": 0.800595736503601,
+      "num_tokens": 1887111.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.6721615947782993,
+      "epoch": 0.52,
+      "grad_norm": 2.875,
+      "learning_rate": 1.0220599565551865e-05,
+      "loss": 0.6567937850952148,
+      "mean_token_accuracy": 0.8083706870675087,
+      "num_tokens": 1938813.0,
+      "step": 390
+    },
+    {
+      "entropy": 0.6703221883624793,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 2.765625,
+      "learning_rate": 9.779400434448137e-06,
+      "loss": 0.6485669612884521,
+      "mean_token_accuracy": 0.8101305291056633,
+      "num_tokens": 1989967.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.7154329728335143,
+      "epoch": 0.5466666666666666,
+      "grad_norm": 3.203125,
+      "learning_rate": 9.338630715119998e-06,
+      "loss": 0.7143341541290283,
+      "mean_token_accuracy": 0.7937317594885827,
+      "num_tokens": 2036598.0,
+      "step": 410
+    },
+    {
+      "entropy": 0.6946513410657644,
+      "epoch": 0.56,
+      "grad_norm": 2.828125,
+      "learning_rate": 8.899148395239946e-06,
+      "loss": 0.6960912227630616,
+      "mean_token_accuracy": 0.7990265294909478,
+      "num_tokens": 2084928.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.6808087941259146,
+      "epoch": 0.5733333333333334,
+      "grad_norm": 2.96875,
+      "learning_rate": 8.461808956471545e-06,
+      "loss": 0.6666933536529541,
+      "mean_token_accuracy": 0.8025331974029541,
+      "num_tokens": 2133851.0,
+      "step": 430
+    },
+    {
+      "entropy": 0.6575148230418563,
+      "epoch": 0.5866666666666667,
+      "grad_norm": 2.890625,
+      "learning_rate": 8.027463709217266e-06,
+      "loss": 0.6442523002624512,
+      "mean_token_accuracy": 0.811263907700777,
+      "num_tokens": 2179617.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.6445509284734726,
+      "epoch": 0.6,
+      "grad_norm": 3.140625,
+      "learning_rate": 7.5969581354859976e-06,
+      "loss": 0.6103779315948487,
+      "mean_token_accuracy": 0.8134750738739968,
+      "num_tokens": 2225093.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6816472139209508,
+      "epoch": 0.6133333333333333,
+      "grad_norm": 2.90625,
+      "learning_rate": 7.1711302431058795e-06,
+      "loss": 0.6601366519927978,
+      "mean_token_accuracy": 0.8032179318368435,
+      "num_tokens": 2278848.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.6660377617925406,
+      "epoch": 0.6266666666666667,
+      "grad_norm": 2.75,
+      "learning_rate": 6.750808934486146e-06,
+      "loss": 0.6550974369049072,
+      "mean_token_accuracy": 0.8068648695945739,
+      "num_tokens": 2331678.0,
+      "step": 470
+    },
+    {
+      "entropy": 0.652556081302464,
+      "epoch": 0.64,
+      "grad_norm": 2.859375,
+      "learning_rate": 6.33681239310327e-06,
+      "loss": 0.6436736583709717,
+      "mean_token_accuracy": 0.8105904631316662,
+      "num_tokens": 2382894.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.6687446342781186,
+      "epoch": 0.6533333333333333,
+      "grad_norm": 2.25,
+      "learning_rate": 5.92994649085223e-06,
+      "loss": 0.6521422863006592,
+      "mean_token_accuracy": 0.8062435537576675,
+      "num_tokens": 2429736.0,
+      "step": 490
+    },
+    {
+      "entropy": 0.6641898091882468,
+      "epoch": 0.6666666666666666,
+      "grad_norm": 3.015625,
+      "learning_rate": 5.531003219363079e-06,
+      "loss": 0.6382893085479736,
+      "mean_token_accuracy": 0.8112858273088932,
+      "num_tokens": 2483182.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6794864580035209,
+      "epoch": 0.68,
+      "grad_norm": 2.640625,
+      "learning_rate": 5.140759148336411e-06,
+      "loss": 0.6581945896148682,
+      "mean_token_accuracy": 0.8064373843371868,
+      "num_tokens": 2540079.0,
+      "step": 510
+    },
+    {
+      "entropy": 0.6667013496160508,
+      "epoch": 0.6933333333333334,
+      "grad_norm": 2.9375,
+      "learning_rate": 4.759973913898578e-06,
+      "loss": 0.643217945098877,
+      "mean_token_accuracy": 0.8076037019491196,
+      "num_tokens": 2592280.0,
+      "step": 520
+    },
+    {
+      "entropy": 0.6570233944803476,
+      "epoch": 0.7066666666666667,
+      "grad_norm": 3.078125,
+      "learning_rate": 4.389388739919303e-06,
+      "loss": 0.6433084964752197,
+      "mean_token_accuracy": 0.8076967656612396,
+      "num_tokens": 2644064.0,
+      "step": 530
+    },
+    {
+      "entropy": 0.6746390011161566,
+      "epoch": 0.72,
+      "grad_norm": 2.5625,
+      "learning_rate": 4.029724995169951e-06,
+      "loss": 0.6688372611999511,
+      "mean_token_accuracy": 0.8042960688471794,
+      "num_tokens": 2694721.0,
+      "step": 540
+    },
+    {
+      "entropy": 0.650193777307868,
+      "epoch": 0.7333333333333333,
+      "grad_norm": 3.625,
+      "learning_rate": 3.681682789131068e-06,
+      "loss": 0.6250029087066651,
+      "mean_token_accuracy": 0.8112101398408413,
+      "num_tokens": 2742327.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.682454134337604,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 3.140625,
+      "learning_rate": 3.345939609182558e-06,
+      "loss": 0.6473059177398681,
+      "mean_token_accuracy": 0.8062132216989994,
+      "num_tokens": 2787863.0,
+      "step": 560
+    },
+    {
+      "entropy": 0.6479893431067467,
+      "epoch": 0.76,
+      "grad_norm": 3.25,
+      "learning_rate": 3.0231490018293042e-06,
+      "loss": 0.6270086765289307,
+      "mean_token_accuracy": 0.811413899064064,
+      "num_tokens": 2837010.0,
+      "step": 570
+    },
+    {
+      "entropy": 0.659004195779562,
+      "epoch": 0.7733333333333333,
+      "grad_norm": 3.078125,
+      "learning_rate": 2.713939300529255e-06,
+      "loss": 0.6230898857116699,
+      "mean_token_accuracy": 0.8098825231194496,
+      "num_tokens": 2886153.0,
+      "step": 580
+    },
+    {
+      "entropy": 0.6754213474690914,
+      "epoch": 0.7866666666666666,
+      "grad_norm": 3.46875,
+      "learning_rate": 2.4189124026004474e-06,
+      "loss": 0.6536226272583008,
+      "mean_token_accuracy": 0.8024932295084,
+      "num_tokens": 2936592.0,
+      "step": 590
+    },
+    {
+      "entropy": 0.6474416930228472,
+      "epoch": 0.8,
+      "grad_norm": 3.03125,
+      "learning_rate": 2.138642597587686e-06,
+      "loss": 0.622972822189331,
+      "mean_token_accuracy": 0.8115987822413444,
+      "num_tokens": 2985720.0,
+      "step": 600
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 750,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.4473081709522944e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-600/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be830bdc5f60d908c726016e2883e3030a67ac3c95df8f0b3975ce517d68f4c6
+size 5777

checkpoint-750/chat_template.jinja

@@ -0,0 +1,90 @@
+{%- set enable_thinking = true %}
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-750/config.json

@@ -0,0 +1,63 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 40960,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151645,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 1000000,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.7.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

checkpoint-750/generation_config.json

@@ -0,0 +1,12 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151645,
+  "temperature": 0.6,
+  "top_k": 20,
+  "top_p": 0.95,
+  "transformers_version": "5.7.0"
+}

checkpoint-750/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a09c0e3c1c41d499df7523ef0c7b66c06311691531033ffdcbc0ee4288d73b2
+size 3441185608

checkpoint-750/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee7f243935ca6fcf488771e0823f82bc6cad2df328eb250040f87702dde959be
+size 6882572207

checkpoint-750/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
+size 14645

checkpoint-750/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:231e3cdc97ab6261dc9ee5d89d917c3b2308dfd806ce51ebc8b15c89ac997303
+size 1465

checkpoint-750/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

checkpoint-750/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|im_end|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-750/trainer_state.json

@@ -0,0 +1,784 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 750,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 0.4763989023864269,
+      "epoch": 0.013333333333333334,
+      "grad_norm": 39.5,
+      "learning_rate": 4.736842105263158e-06,
+      "loss": 2.0091196060180665,
+      "mean_token_accuracy": 0.6925897747278214,
+      "num_tokens": 42464.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.5551879126578569,
+      "epoch": 0.02666666666666667,
+      "grad_norm": 12.125,
+      "learning_rate": 1e-05,
+      "loss": 1.332655143737793,
+      "mean_token_accuracy": 0.7217346377670765,
+      "num_tokens": 96049.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.7102062199264765,
+      "epoch": 0.04,
+      "grad_norm": 4.9375,
+      "learning_rate": 1.5263157894736846e-05,
+      "loss": 0.8764092445373535,
+      "mean_token_accuracy": 0.7632560275495053,
+      "num_tokens": 140505.0,
+      "step": 30
+    },
+    {
+      "entropy": 0.7993051432073116,
+      "epoch": 0.05333333333333334,
+      "grad_norm": 4.5625,
+      "learning_rate": 1.9999902656029183e-05,
+      "loss": 0.7961350917816162,
+      "mean_token_accuracy": 0.7790301404893398,
+      "num_tokens": 186827.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.7229622758924961,
+      "epoch": 0.06666666666666667,
+      "grad_norm": 2.578125,
+      "learning_rate": 1.9988223672512097e-05,
+      "loss": 0.7293992042541504,
+      "mean_token_accuracy": 0.7856058329343796,
+      "num_tokens": 237512.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.706776586547494,
+      "epoch": 0.08,
+      "grad_norm": 3.125,
+      "learning_rate": 1.995710194507367e-05,
+      "loss": 0.7110776901245117,
+      "mean_token_accuracy": 0.7912242174148559,
+      "num_tokens": 285719.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.6807167779654264,
+      "epoch": 0.09333333333333334,
+      "grad_norm": 2.59375,
+      "learning_rate": 1.9906598054233197e-05,
+      "loss": 0.6580380439758301,
+      "mean_token_accuracy": 0.7998592600226402,
+      "num_tokens": 344554.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.7038780480623246,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 4.0,
+      "learning_rate": 1.9836810309184468e-05,
+      "loss": 0.6892334938049316,
+      "mean_token_accuracy": 0.7912278659641743,
+      "num_tokens": 398019.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.6895901430398226,
+      "epoch": 0.12,
+      "grad_norm": 2.84375,
+      "learning_rate": 1.9747874556430363e-05,
+      "loss": 0.6818968772888183,
+      "mean_token_accuracy": 0.7969276301562787,
+      "num_tokens": 447685.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.7114015109837055,
+      "epoch": 0.13333333333333333,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.9639963915348547e-05,
+      "loss": 0.6934291362762451,
+      "mean_token_accuracy": 0.7938909940421581,
+      "num_tokens": 497899.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.7047938119620085,
+      "epoch": 0.14666666666666667,
+      "grad_norm": 3.484375,
+      "learning_rate": 1.9513288441203083e-05,
+      "loss": 0.6858394145965576,
+      "mean_token_accuracy": 0.7970494098961354,
+      "num_tokens": 548312.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.67772804684937,
+      "epoch": 0.16,
+      "grad_norm": 2.21875,
+      "learning_rate": 1.93680947162578e-05,
+      "loss": 0.661228609085083,
+      "mean_token_accuracy": 0.8077008336782455,
+      "num_tokens": 596458.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.6539881706237793,
+      "epoch": 0.17333333333333334,
+      "grad_norm": 3.140625,
+      "learning_rate": 1.9204665369787507e-05,
+      "loss": 0.6504672527313232,
+      "mean_token_accuracy": 0.8092843152582645,
+      "num_tokens": 649939.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.686749292910099,
+      "epoch": 0.18666666666666668,
+      "grad_norm": 2.953125,
+      "learning_rate": 1.9023318527921213e-05,
+      "loss": 0.6639729499816894,
+      "mean_token_accuracy": 0.7960164114832878,
+      "num_tokens": 696515.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.6799734767526389,
+      "epoch": 0.2,
+      "grad_norm": 2.625,
+      "learning_rate": 1.8824407194388408e-05,
+      "loss": 0.6643428802490234,
+      "mean_token_accuracy": 0.8016563065350055,
+      "num_tokens": 745199.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.65655233040452,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 3.15625,
+      "learning_rate": 1.8608318563373735e-05,
+      "loss": 0.6439759254455566,
+      "mean_token_accuracy": 0.8110810227692127,
+      "num_tokens": 793167.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.6463201232254505,
+      "epoch": 0.22666666666666666,
+      "grad_norm": 2.890625,
+      "learning_rate": 1.837547326581768e-05,
+      "loss": 0.6181783199310302,
+      "mean_token_accuracy": 0.8121586792171002,
+      "num_tokens": 844567.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.6676008347421885,
+      "epoch": 0.24,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.8126324550630364e-05,
+      "loss": 0.6397620677947998,
+      "mean_token_accuracy": 0.8043703198432922,
+      "num_tokens": 896377.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.6894530899822712,
+      "epoch": 0.25333333333333335,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.786135740241232e-05,
+      "loss": 0.6637216091156006,
+      "mean_token_accuracy": 0.8027763769030571,
+      "num_tokens": 946438.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.6975695930421353,
+      "epoch": 0.26666666666666666,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.7581087597399556e-05,
+      "loss": 0.6652172088623047,
+      "mean_token_accuracy": 0.7989616326987743,
+      "num_tokens": 995473.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.6387956503778696,
+      "epoch": 0.28,
+      "grad_norm": 2.75,
+      "learning_rate": 1.7286060699470746e-05,
+      "loss": 0.6128511905670166,
+      "mean_token_accuracy": 0.8127545997500419,
+      "num_tokens": 1045963.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.6565726511180401,
+      "epoch": 0.29333333333333333,
+      "grad_norm": 3.3125,
+      "learning_rate": 1.69768509981707e-05,
+      "loss": 0.6489222526550293,
+      "mean_token_accuracy": 0.8057824857532978,
+      "num_tokens": 1096633.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.6619438689202071,
+      "epoch": 0.30666666666666664,
+      "grad_norm": 2.734375,
+      "learning_rate": 1.6654060390817445e-05,
+      "loss": 0.6426784038543701,
+      "mean_token_accuracy": 0.809200967848301,
+      "num_tokens": 1149606.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.6764651168137789,
+      "epoch": 0.32,
+      "grad_norm": 2.859375,
+      "learning_rate": 1.6318317210868936e-05,
+      "loss": 0.6661844730377198,
+      "mean_token_accuracy": 0.8004243083298206,
+      "num_tokens": 1198813.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.684938239119947,
+      "epoch": 0.3333333333333333,
+      "grad_norm": 3.015625,
+      "learning_rate": 1.5970275004830053e-05,
+      "loss": 0.649436616897583,
+      "mean_token_accuracy": 0.8030574709177017,
+      "num_tokens": 1245303.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.6592010721564293,
+      "epoch": 0.3466666666666667,
+      "grad_norm": 2.640625,
+      "learning_rate": 1.56106112600807e-05,
+      "loss": 0.6452194213867187,
+      "mean_token_accuracy": 0.8091069459915161,
+      "num_tokens": 1293082.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.6599753946065903,
+      "epoch": 0.36,
+      "grad_norm": 2.671875,
+      "learning_rate": 1.5240026086101425e-05,
+      "loss": 0.6302877902984619,
+      "mean_token_accuracy": 0.8083149395883084,
+      "num_tokens": 1345989.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.6778957467526198,
+      "epoch": 0.37333333333333335,
+      "grad_norm": 2.703125,
+      "learning_rate": 1.485924085166359e-05,
+      "loss": 0.653476333618164,
+      "mean_token_accuracy": 0.8066982626914978,
+      "num_tokens": 1395698.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.648598388582468,
+      "epoch": 0.38666666666666666,
+      "grad_norm": 2.953125,
+      "learning_rate": 1.4468996780636922e-05,
+      "loss": 0.6419031143188476,
+      "mean_token_accuracy": 0.8062063075602055,
+      "num_tokens": 1449209.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.6481347244232893,
+      "epoch": 0.4,
+      "grad_norm": 2.40625,
+      "learning_rate": 1.4070053509147777e-05,
+      "loss": 0.6035490512847901,
+      "mean_token_accuracy": 0.8153042212128639,
+      "num_tokens": 1499225.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.7093129184097051,
+      "epoch": 0.41333333333333333,
+      "grad_norm": 2.984375,
+      "learning_rate": 1.3663187606896733e-05,
+      "loss": 0.7133637428283691,
+      "mean_token_accuracy": 0.7937003694474697,
+      "num_tokens": 1542301.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.6543151158839464,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 3.234375,
+      "learning_rate": 1.3249191065513857e-05,
+      "loss": 0.6388350486755371,
+      "mean_token_accuracy": 0.8044970251619816,
+      "num_tokens": 1596800.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.6799626916646957,
+      "epoch": 0.44,
+      "grad_norm": 2.09375,
+      "learning_rate": 1.2828869756894122e-05,
+      "loss": 0.6594656944274903,
+      "mean_token_accuracy": 0.8026202499866486,
+      "num_tokens": 1648619.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.680319806188345,
+      "epoch": 0.4533333333333333,
+      "grad_norm": 2.4375,
+      "learning_rate": 1.2403041864514005e-05,
+      "loss": 0.6563356876373291,
+      "mean_token_accuracy": 0.8025765180587768,
+      "num_tokens": 1703363.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.6631284486502409,
+      "epoch": 0.4666666666666667,
+      "grad_norm": 2.84375,
+      "learning_rate": 1.1972536290782738e-05,
+      "loss": 0.6403236865997315,
+      "mean_token_accuracy": 0.8110226787626743,
+      "num_tokens": 1754906.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.7027981210500002,
+      "epoch": 0.48,
+      "grad_norm": 3.5,
+      "learning_rate": 1.153819104352846e-05,
+      "loss": 0.6743718147277832,
+      "mean_token_accuracy": 0.7984775453805923,
+      "num_tokens": 1796484.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.6734891567379236,
+      "epoch": 0.49333333333333335,
+      "grad_norm": 2.140625,
+      "learning_rate": 1.1100851604760057e-05,
+      "loss": 0.6835583686828614,
+      "mean_token_accuracy": 0.7995599187910557,
+      "num_tokens": 1845839.0,
+      "step": 370
+    },
+    {
+      "entropy": 0.6895672902464867,
+      "epoch": 0.5066666666666667,
+      "grad_norm": 3.171875,
+      "learning_rate": 1.0661369284880005e-05,
+      "loss": 0.675282096862793,
+      "mean_token_accuracy": 0.800595736503601,
+      "num_tokens": 1887111.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.6721615947782993,
+      "epoch": 0.52,
+      "grad_norm": 2.875,
+      "learning_rate": 1.0220599565551865e-05,
+      "loss": 0.6567937850952148,
+      "mean_token_accuracy": 0.8083706870675087,
+      "num_tokens": 1938813.0,
+      "step": 390
+    },
+    {
+      "entropy": 0.6703221883624793,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 2.765625,
+      "learning_rate": 9.779400434448137e-06,
+      "loss": 0.6485669612884521,
+      "mean_token_accuracy": 0.8101305291056633,
+      "num_tokens": 1989967.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.7154329728335143,
+      "epoch": 0.5466666666666666,
+      "grad_norm": 3.203125,
+      "learning_rate": 9.338630715119998e-06,
+      "loss": 0.7143341541290283,
+      "mean_token_accuracy": 0.7937317594885827,
+      "num_tokens": 2036598.0,
+      "step": 410
+    },
+    {
+      "entropy": 0.6946513410657644,
+      "epoch": 0.56,
+      "grad_norm": 2.828125,
+      "learning_rate": 8.899148395239946e-06,
+      "loss": 0.6960912227630616,
+      "mean_token_accuracy": 0.7990265294909478,
+      "num_tokens": 2084928.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.6808087941259146,
+      "epoch": 0.5733333333333334,
+      "grad_norm": 2.96875,
+      "learning_rate": 8.461808956471545e-06,
+      "loss": 0.6666933536529541,
+      "mean_token_accuracy": 0.8025331974029541,
+      "num_tokens": 2133851.0,
+      "step": 430
+    },
+    {
+      "entropy": 0.6575148230418563,
+      "epoch": 0.5866666666666667,
+      "grad_norm": 2.890625,
+      "learning_rate": 8.027463709217266e-06,
+      "loss": 0.6442523002624512,
+      "mean_token_accuracy": 0.811263907700777,
+      "num_tokens": 2179617.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.6445509284734726,
+      "epoch": 0.6,
+      "grad_norm": 3.140625,
+      "learning_rate": 7.5969581354859976e-06,
+      "loss": 0.6103779315948487,
+      "mean_token_accuracy": 0.8134750738739968,
+      "num_tokens": 2225093.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6816472139209508,
+      "epoch": 0.6133333333333333,
+      "grad_norm": 2.90625,
+      "learning_rate": 7.1711302431058795e-06,
+      "loss": 0.6601366519927978,
+      "mean_token_accuracy": 0.8032179318368435,
+      "num_tokens": 2278848.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.6660377617925406,
+      "epoch": 0.6266666666666667,
+      "grad_norm": 2.75,
+      "learning_rate": 6.750808934486146e-06,
+      "loss": 0.6550974369049072,
+      "mean_token_accuracy": 0.8068648695945739,
+      "num_tokens": 2331678.0,
+      "step": 470
+    },
+    {
+      "entropy": 0.652556081302464,
+      "epoch": 0.64,
+      "grad_norm": 2.859375,
+      "learning_rate": 6.33681239310327e-06,
+      "loss": 0.6436736583709717,
+      "mean_token_accuracy": 0.8105904631316662,
+      "num_tokens": 2382894.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.6687446342781186,
+      "epoch": 0.6533333333333333,
+      "grad_norm": 2.25,
+      "learning_rate": 5.92994649085223e-06,
+      "loss": 0.6521422863006592,
+      "mean_token_accuracy": 0.8062435537576675,
+      "num_tokens": 2429736.0,
+      "step": 490
+    },
+    {
+      "entropy": 0.6641898091882468,
+      "epoch": 0.6666666666666666,
+      "grad_norm": 3.015625,
+      "learning_rate": 5.531003219363079e-06,
+      "loss": 0.6382893085479736,
+      "mean_token_accuracy": 0.8112858273088932,
+      "num_tokens": 2483182.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.6794864580035209,
+      "epoch": 0.68,
+      "grad_norm": 2.640625,
+      "learning_rate": 5.140759148336411e-06,
+      "loss": 0.6581945896148682,
+      "mean_token_accuracy": 0.8064373843371868,
+      "num_tokens": 2540079.0,
+      "step": 510
+    },
+    {
+      "entropy": 0.6667013496160508,
+      "epoch": 0.6933333333333334,
+      "grad_norm": 2.9375,
+      "learning_rate": 4.759973913898578e-06,
+      "loss": 0.643217945098877,
+      "mean_token_accuracy": 0.8076037019491196,
+      "num_tokens": 2592280.0,
+      "step": 520
+    },
+    {
+      "entropy": 0.6570233944803476,
+      "epoch": 0.7066666666666667,
+      "grad_norm": 3.078125,
+      "learning_rate": 4.389388739919303e-06,
+      "loss": 0.6433084964752197,
+      "mean_token_accuracy": 0.8076967656612396,
+      "num_tokens": 2644064.0,
+      "step": 530
+    },
+    {
+      "entropy": 0.6746390011161566,
+      "epoch": 0.72,
+      "grad_norm": 2.5625,
+      "learning_rate": 4.029724995169951e-06,
+      "loss": 0.6688372611999511,
+      "mean_token_accuracy": 0.8042960688471794,
+      "num_tokens": 2694721.0,
+      "step": 540
+    },
+    {
+      "entropy": 0.650193777307868,
+      "epoch": 0.7333333333333333,
+      "grad_norm": 3.625,
+      "learning_rate": 3.681682789131068e-06,
+      "loss": 0.6250029087066651,
+      "mean_token_accuracy": 0.8112101398408413,
+      "num_tokens": 2742327.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.682454134337604,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 3.140625,
+      "learning_rate": 3.345939609182558e-06,
+      "loss": 0.6473059177398681,
+      "mean_token_accuracy": 0.8062132216989994,
+      "num_tokens": 2787863.0,
+      "step": 560
+    },
+    {
+      "entropy": 0.6479893431067467,
+      "epoch": 0.76,
+      "grad_norm": 3.25,
+      "learning_rate": 3.0231490018293042e-06,
+      "loss": 0.6270086765289307,
+      "mean_token_accuracy": 0.811413899064064,
+      "num_tokens": 2837010.0,
+      "step": 570
+    },
+    {
+      "entropy": 0.659004195779562,
+      "epoch": 0.7733333333333333,
+      "grad_norm": 3.078125,
+      "learning_rate": 2.713939300529255e-06,
+      "loss": 0.6230898857116699,
+      "mean_token_accuracy": 0.8098825231194496,
+      "num_tokens": 2886153.0,
+      "step": 580
+    },
+    {
+      "entropy": 0.6754213474690914,
+      "epoch": 0.7866666666666666,
+      "grad_norm": 3.46875,
+      "learning_rate": 2.4189124026004474e-06,
+      "loss": 0.6536226272583008,
+      "mean_token_accuracy": 0.8024932295084,
+      "num_tokens": 2936592.0,
+      "step": 590
+    },
+    {
+      "entropy": 0.6474416930228472,
+      "epoch": 0.8,
+      "grad_norm": 3.03125,
+      "learning_rate": 2.138642597587686e-06,
+      "loss": 0.622972822189331,
+      "mean_token_accuracy": 0.8115987822413444,
+      "num_tokens": 2985720.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6490513959899544,
+      "epoch": 0.8133333333333334,
+      "grad_norm": 4.0,
+      "learning_rate": 1.8736754493696352e-06,
+      "loss": 0.6318037033081054,
+      "mean_token_accuracy": 0.8130880683660507,
+      "num_tokens": 3034549.0,
+      "step": 610
+    },
+    {
+      "entropy": 0.6814188580960036,
+      "epoch": 0.8266666666666667,
+      "grad_norm": 2.671875,
+      "learning_rate": 1.6245267341823235e-06,
+      "loss": 0.6752352237701416,
+      "mean_token_accuracy": 0.8002629041671753,
+      "num_tokens": 3085447.0,
+      "step": 620
+    },
+    {
+      "entropy": 0.635303413681686,
+      "epoch": 0.84,
+      "grad_norm": 3.296875,
+      "learning_rate": 1.3916814366262677e-06,
+      "loss": 0.5970020771026612,
+      "mean_token_accuracy": 0.8152928575873375,
+      "num_tokens": 3135437.0,
+      "step": 630
+    },
+    {
+      "entropy": 0.6500982265919447,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 2.265625,
+      "learning_rate": 1.1755928056115951e-06,
+      "loss": 0.6392026901245117,
+      "mean_token_accuracy": 0.8090934410691262,
+      "num_tokens": 3183930.0,
+      "step": 640
+    },
+    {
+      "entropy": 0.6468437172472477,
+      "epoch": 0.8666666666666667,
+      "grad_norm": 2.890625,
+      "learning_rate": 9.766814720787887e-07,
+      "loss": 0.6033987045288086,
+      "mean_token_accuracy": 0.8132735326886177,
+      "num_tokens": 3231321.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6570335838943719,
+      "epoch": 0.88,
+      "grad_norm": 2.765625,
+      "learning_rate": 7.953346302124943e-07,
+      "loss": 0.6359596729278565,
+      "mean_token_accuracy": 0.8083353474736213,
+      "num_tokens": 3280050.0,
+      "step": 660
+    },
+    {
+      "entropy": 0.6886860881000757,
+      "epoch": 0.8933333333333333,
+      "grad_norm": 4.15625,
+      "learning_rate": 6.319052837422013e-07,
+      "loss": 0.6836255550384521,
+      "mean_token_accuracy": 0.7978803515434265,
+      "num_tokens": 3327947.0,
+      "step": 670
+    },
+    {
+      "entropy": 0.6405102459713816,
+      "epoch": 0.9066666666666666,
+      "grad_norm": 3.625,
+      "learning_rate": 4.867115587969218e-07,
+      "loss": 0.5980209350585938,
+      "mean_token_accuracy": 0.8176288396120072,
+      "num_tokens": 3378711.0,
+      "step": 680
+    },
+    {
+      "entropy": 0.636499953456223,
+      "epoch": 0.92,
+      "grad_norm": 2.609375,
+      "learning_rate": 3.600360846514539e-07,
+      "loss": 0.620827054977417,
+      "mean_token_accuracy": 0.8175054028630256,
+      "num_tokens": 3429400.0,
+      "step": 690
+    },
+    {
+      "entropy": 0.6172923348844052,
+      "epoch": 0.9333333333333333,
+      "grad_norm": 2.8125,
+      "learning_rate": 2.5212544356964073e-07,
+      "loss": 0.6034162521362305,
+      "mean_token_accuracy": 0.8199474096298218,
+      "num_tokens": 3483247.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.664278375543654,
+      "epoch": 0.9466666666666667,
+      "grad_norm": 2.734375,
+      "learning_rate": 1.6318969081553283e-07,
+      "loss": 0.663159704208374,
+      "mean_token_accuracy": 0.8069490611553192,
+      "num_tokens": 3531738.0,
+      "step": 710
+    },
+    {
+      "entropy": 0.6435161855071783,
+      "epoch": 0.96,
+      "grad_norm": 2.578125,
+      "learning_rate": 9.340194576680562e-08,
+      "loss": 0.5957161903381347,
+      "mean_token_accuracy": 0.8148073829710484,
+      "num_tokens": 3583252.0,
+      "step": 720
+    },
+    {
+      "entropy": 0.6636522088199854,
+      "epoch": 0.9733333333333334,
+      "grad_norm": 2.90625,
+      "learning_rate": 4.289805492633248e-08,
+      "loss": 0.6373569965362549,
+      "mean_token_accuracy": 0.8058297611773014,
+      "num_tokens": 3631660.0,
+      "step": 730
+    },
+    {
+      "entropy": 0.6847980171442032,
+      "epoch": 0.9866666666666667,
+      "grad_norm": 3.21875,
+      "learning_rate": 1.1776327487903472e-08,
+      "loss": 0.6613836765289307,
+      "mean_token_accuracy": 0.7997118130326271,
+      "num_tokens": 3677802.0,
+      "step": 740
+    },
+    {
+      "entropy": 0.6479170603677631,
+      "epoch": 1.0,
+      "grad_norm": 2.890625,
+      "learning_rate": 9.734397081961888e-11,
+      "loss": 0.6307967662811279,
+      "mean_token_accuracy": 0.8109949566423893,
+      "num_tokens": 3730419.0,
+      "step": 750
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 750,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.307411364260659e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-750/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be830bdc5f60d908c726016e2883e3030a67ac3c95df8f0b3975ce517d68f4c6
+size 5777

config.json

@@ -4,7 +4,7 @@
   ],
   "attention_bias": false,
   "attention_dropout": 0.0,
-  "bos_token_id": 151643,
+  "bos_token_id": null,
   "dtype": "bfloat16",
   "eos_token_id": 151645,
   "head_dim": 128,
@@ -48,7 +48,7 @@
   "num_attention_heads": 16,
   "num_hidden_layers": 28,
   "num_key_value_heads": 8,
-  "pad_token_id": null,
+  "pad_token_id": 151645,
   "rms_norm_eps": 1e-06,
   "rope_parameters": {
     "rope_theta": 1000000,
@@ -57,7 +57,7 @@
   "sliding_window": null,
   "tie_word_embeddings": true,
   "transformers_version": "5.7.0",
-  "use_cache": true,
+  "use_cache": false,
   "use_sliding_window": false,
   "vocab_size": 151936
 }

generation_config.json

@@ -1,12 +1,10 @@
 {
-  "bos_token_id": 151643,
   "do_sample": true,
   "eos_token_id": [
     151645,
     151643
   ],
-  "max_new_tokens": 32768,
-  "pad_token_id": 151643,
+  "pad_token_id": 151645,
   "temperature": 0.6,
   "top_k": 20,
   "top_p": 0.95,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:169ad53ec313c3a34b06c0809216e4fc072cce444a5d4ff2b59690d064130ed5
+oid sha256:8a09c0e3c1c41d499df7523ef0c7b66c06311691531033ffdcbc0ee4288d73b2
 size 3441185608

tokenizer_config.json

@@ -23,7 +23,7 @@
   "is_local": false,
   "local_files_only": false,
   "model_max_length": 131072,
-  "pad_token": "<|endoftext|>",
+  "pad_token": "<|im_end|>",
   "split_special_tokens": false,
   "tokenizer_class": "Qwen2Tokenizer",
   "unk_token": null

train.log

@@ -0,0 +1,112 @@
+
+
+[transformers] warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
+
+
+Training on 12000 examples
+Effective batch size: 16
+Max seq length: 4096
+[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151645}.
+wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
+wandb: Currently logged in as: soulio-haddad (duo_run) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
+wandb: setting up run r0gc2y2j
+wandb: Tracking run with wandb version 0.26.1
+wandb: Run data is saved locally in /scratch/repo/wandb/run-20260512_183722-r0gc2y2j
+wandb: Run `wandb offline` to turn off syncing.
+wandb: Syncing run math_model_sft
+wandb: ⭐️ View project at https://wandb.ai/duo_run/huggingface
+wandb: 🚀 View run at https://wandb.ai/duo_run/huggingface/runs/r0gc2y2j
+
  0%|          | 0/750 [00:00<?, ?it/s]
  0%|          | 1/750 [00:02<35:47,  2.87s/it]
  0%|          | 2/750 [00:04<29:00,  2.33s/it]
  0%|          | 3/750 [00:06<26:35,  2.14s/it]
  1%|          | 4/750 [00:09<27:36,  2.22s/it]
  1%|          | 5/750 [00:11<26:30,  2.14s/it]
  1%|          | 6/750 [00:13<25:58,  2.09s/it]
  1%|          | 7/750 [00:15<25:19,  2.05s/it]
  1%|          | 8/750 [00:16<24:59,  2.02s/it]
  1%|          | 9/750 [00:19<25:30,  2.07s/it]
  1%|▏         | 10/750 [00:21<25:01,  2.03s/it]
                                                
+
  1%|▏         | 10/750 [00:21<25:01,  2.03s/it]
  1%|▏         | 11/750 [00:23<24:58,  2.03s/it]
  2%|▏         | 12/750 [00:25<25:04,  2.04s/it]
  2%|▏         | 13/750 [00:27<25:45,  2.10s/it]
  2%|▏         | 14/750 [00:29<26:18,  2.15s/it]
  2%|▏         | 15/750 [00:31<25:59,  2.12s/it]
  2%|▏         | 16/750 [00:33<25:19,  2.07s/it]
  2%|▏         | 17/750 [00:36<26:35,  2.18s/it]
  2%|▏         | 18/750 [00:38<26:19,  2.16s/it]
  3%|▎         | 19/750 [00:40<25:38,  2.11s/it]
  3%|▎         | 20/750 [00:42<25:16,  2.08s/it]
                                                
+
  3%|▎         | 20/750 [00:42<25:16,  2.08s/it]
  3%|▎         | 21/750 [00:44<25:47,  2.12s/it]
  3%|▎         | 22/750 [00:46<25:22,  2.09s/it]
  3%|▎         | 23/750 [00:48<24:53,  2.05s/it]
  3%|▎         | 24/750 [00:50<24:47,  2.05s/it]
  3%|▎         | 25/750 [00:52<24:58,  2.07s/it]
  3%|▎         | 26/750 [00:54<24:33,  2.04s/it]
  4%|▎         | 27/750 [00:56<24:18,  2.02s/it]
  4%|▎         | 28/750 [00:58<24:25,  2.03s/it]
  4%|▍         | 29/750 [01:00<25:20,  2.11s/it]
  4%|▍         | 30/750 [01:02<25:13,  2.10s/it]
                                                
+
  4%|▍         | 30/750 [01:02<25:13,  2.10s/it]
  4%|▍         | 31/750 [01:05<25:02,  2.09s/it]
  4%|▍         | 32/750 [01:07<24:51,  2.08s/it]
  4%|▍         | 33/750 [01:09<24:31,  2.05s/it]
  5%|▍         | 34/750 [01:11<25:22,  2.13s/it]
  5%|▍         | 35/750 [01:13<24:36,  2.06s/it]
  5%|▍         | 36/750 [01:15<24:15,  2.04s/it]
  5%|▍         | 37/750 [01:17<24:24,  2.05s/it]
  5%|▌         | 38/750 [01:19<25:27,  2.15s/it]
  5%|▌         | 39/750 [01:21<24:50,  2.10s/it]
  5%|▌         | 40/750 [01:23<24:43,  2.09s/it]
                                                
+
  5%|▌         | 40/750 [01:23<24:43,  2.09s/it]
  5%|▌         | 41/750 [01:25<24:54,  2.11s/it]
  6%|▌         | 42/750 [01:28<25:26,  2.16s/it]
  6%|▌         | 43/750 [01:30<24:59,  2.12s/it]
  6%|▌         | 44/750 [01:32<25:01,  2.13s/it]
  6%|▌         | 45/750 [01:34<24:19,  2.07s/it]
  6%|▌         | 46/750 [01:36<25:20,  2.16s/it]
  6%|▋         | 47/750 [01:38<24:51,  2.12s/it]
  6%|▋         | 48/750 [01:40<24:51,  2.12s/it]
  7%|▋         | 49/750 [01:42<24:20,  2.08s/it]
  7%|▋         | 50/750 [01:45<24:54,  2.14s/it]
                                                
+
  7%|▋         | 50/750 [01:45<24:54,  2.14s/it]
  7%|▋         | 51/750 [01:47<25:13,  2.16s/it]
  7%|▋         | 52/750 [01:49<25:23,  2.18s/it]
  7%|▋         | 53/750 [01:51<24:58,  2.15s/it]
  7%|▋         | 54/750 [01:54<26:47,  2.31s/it]
  7%|▋         | 55/750 [01:56<26:37,  2.30s/it]
  7%|▋         | 56/750 [01:58<25:28,  2.20s/it]
  8%|▊         | 57/750 [02:00<24:33,  2.13s/it]
  8%|▊         | 58/750 [02:02<24:02,  2.09s/it]
  8%|▊         | 59/750 [02:04<24:55,  2.16s/it]
  8%|▊         | 60/750 [02:06<24:35,  2.14s/it]
                                                
+
  8%|▊         | 60/750 [02:06<24:35,  2.14s/it]
  8%|▊         | 61/750 [02:09<24:35,  2.14s/it]
  8%|▊         | 62/750 [02:11<24:33,  2.14s/it]
  8%|▊         | 63/750 [02:13<25:05,  2.19s/it]
  9%|▊         | 64/750 [02:15<25:09,  2.20s/it]
  9%|▊         | 65/750 [02:17<24:26,  2.14s/it]
  9%|▉         | 66/750 [02:19<24:23,  2.14s/it]
  9%|▉         | 67/750 [02:22<24:28,  2.15s/it]
  9%|▉         | 68/750 [02:24<25:29,  2.24s/it]
  9%|▉         | 69/750 [02:26<24:49,  2.19s/it]
  9%|▉         | 70/750 [02:28<24:24,  2.15s/it]
                                                
+
  9%|▉         | 70/750 [02:28<24:24,  2.15s/it]
  9%|▉         | 71/750 [02:30<24:33,  2.17s/it]
 10%|▉         | 72/750 [02:33<24:41,  2.19s/it]
 10%|▉         | 73/750 [02:35<24:15,  2.15s/it]
 10%|▉         | 74/750 [02:37<24:18,  2.16s/it]
 10%|█         | 75/750 [02:39<24:26,  2.17s/it]
 10%|█         | 76/750 [02:41<24:57,  2.22s/it]
 10%|█         | 77/750 [02:44<25:05,  2.24s/it]
 10%|█         | 78/750 [02:46<24:15,  2.17s/it]
 11%|█         | 79/750 [02:48<23:51,  2.13s/it]
 11%|█         | 80/750 [02:50<24:12,  2.17s/it]
                                                
+
 11%|█         | 80/750 [02:50<24:12,  2.17s/it]
 11%|█         | 81/750 [02:52<23:40,  2.12s/it]
 11%|█         | 82/750 [02:54<23:41,  2.13s/it]
 11%|█         | 83/750 [02:56<23:32,  2.12s/it]
 11%|█         | 84/750 [02:58<23:30,  2.12s/it]
 11%|█▏        | 85/750 [03:01<24:00,  2.17s/it]
 11%|█▏        | 86/750 [03:03<23:30,  2.12s/it]
 12%|█▏        | 87/750 [03:05<23:11,  2.10s/it]
 12%|█▏        | 88/750 [03:07<22:51,  2.07s/it]
 12%|█▏        | 89/750 [03:09<23:44,  2.16s/it]
 12%|█▏        | 90/750 [03:11<23:12,  2.11s/it]
                                                
+
 12%|█▏        | 90/750 [03:11<23:12,  2.11s/it]
 12%|█▏        | 91/750 [03:13<22:37,  2.06s/it]
 12%|█▏        | 92/750 [03:15<22:20,  2.04s/it]
 12%|█▏        | 93/750 [03:17<23:40,  2.16s/it]
 13%|█▎        | 94/750 [03:20<23:34,  2.16s/it]
 13%|█▎        | 95/750 [03:22<23:34,  2.16s/it]
 13%|█▎        | 96/750 [03:24<22:58,  2.11s/it]
 13%|█▎        | 97/750 [03:26<23:41,  2.18s/it]
 13%|█▎        | 98/750 [03:28<22:59,  2.12s/it]
 13%|█▎        | 99/750 [03:30<23:00,  2.12s/it]
 13%|█▎        | 100/750 [03:32<22:48,  2.10s/it]
                                                 
+
 13%|█▎        | 100/750 [03:32<22:48,  2.10s/it]
 13%|█▎        | 101/750 [03:35<23:32,  2.18s/it]
 14%|█▎        | 102/750 [03:37<23:02,  2.13s/it]
 14%|█▎        | 103/750 [03:39<23:41,  2.20s/it]
 14%|█▍        | 104/750 [03:41<23:20,  2.17s/it]
 14%|█▍        | 105/750 [03:43<23:39,  2.20s/it]
 14%|█▍        | 106/750 [03:45<22:57,  2.14s/it]
 14%|█▍        | 107/750 [03:47<22:34,  2.11s/it]
 14%|█▍        | 108/750 [03:49<22:26,  2.10s/it]
 15%|█▍        | 109/750 [03:51<22:09,  2.07s/it]
 15%|█▍        | 110/750 [03:54<22:42,  2.13s/it]
                                                 
+
 15%|█▍        | 110/750 [03:54<22:42,  2.13s/it]
 15%|█▍        | 111/750 [03:56<22:13,  2.09s/it]
 15%|█▍        | 112/750 [03:58<21:48,  2.05s/it]
 15%|█▌        | 113/750 [04:00<21:53,  2.06s/it]
 15%|█▌        | 114/750 [04:02<22:28,  2.12s/it]
 15%|█▌        | 115/750 [04:04<22:01,  2.08s/it]
 15%|█▌        | 116/750 [04:06<22:05,  2.09s/it]
 16%|█▌        | 117/750 [04:08<22:14,  2.11s/it]
 16%|█▌        | 118/750 [04:11<22:56,  2.18s/it]
 16%|█▌        | 119/750 [04:13<22:21,  2.13s/it]
 16%|█▌        | 120/750 [04:15<22:16,  2.12s/it]
                                                 
+
 16%|█▌        | 120/750 [04:15<22:16,  2.12s/it]
 16%|█▌        | 121/750 [04:17<22:16,  2.12s/it]
 16%|█▋        | 122/750 [04:19<23:01,  2.20s/it]
 16%|█▋        | 123/750 [04:21<22:07,  2.12s/it]
 17%|█▋        | 124/750 [04:23<21:58,  2.11s/it]
 17%|█▋        | 125/750 [04:25<21:53,  2.10s/it]
 17%|█▋        | 126/750 [04:28<22:39,  2.18s/it]
 17%|█▋        | 127/750 [04:30<22:35,  2.18s/it]
 17%|█▋        | 128/750 [04:32<22:21,  2.16s/it]
 17%|█▋        | 129/750 [04:34<22:03,  2.13s/it]
 17%|█▋        | 130/750 [04:36<21:46,  2.11s/it]
                                                 
+
 17%|█▋        | 130/750 [04:36<21:46,  2.11s/it]
 17%|█▋        | 131/750 [04:38<21:52,  2.12s/it]
 18%|█▊        | 132/750 [04:40<21:30,  2.09s/it]
 18%|█▊        | 133/750 [04:42<21:41,  2.11s/it]
 18%|█▊        | 134/750 [04:44<21:38,  2.11s/it]
 18%|█▊        | 135/750 [04:47<21:48,  2.13s/it]
 18%|█▊        | 136/750 [04:49<21:28,  2.10s/it]
 18%|█▊        | 137/750 [04:51<21:48,  2.13s/it]
 18%|█▊        | 138/750 [04:53<21:19,  2.09s/it]
 19%|█▊        | 139/750 [04:55<21:45,  2.14s/it]
 19%|█▊        | 140/750 [04:57<21:35,  2.12s/it]
                                                 
+
 19%|█▊        | 140/750 [04:57<21:35,  2.12s/it]
 19%|█▉        | 141/750 [04:59<21:27,  2.11s/it]
 19%|█▉        | 142/750 [05:01<21:05,  2.08s/it]
 19%|█▉        | 143/750 [05:04<22:03,  2.18s/it]
 19%|█▉        | 144/750 [05:06<21:35,  2.14s/it]
 19%|█▉        | 145/750 [05:08<21:04,  2.09s/it]
 19%|█▉        | 146/750 [05:10<20:55,  2.08s/it]
 20%|█▉        | 147/750 [05:12<21:28,  2.14s/it]
 20%|█▉        | 148/750 [05:14<21:12,  2.11s/it]
 20%|█▉        | 149/750 [05:16<20:56,  2.09s/it]
 20%|██        | 150/750 [05:18<21:09,  2.12s/it]
                                                 
+
 20%|██        | 150/750 [05:18<21:09,  2.12s/it]
 20%|██        | 151/750 [05:20<20:39,  2.07s/it]
 20%|██        | 152/750 [05:23<21:17,  2.14s/it]
 20%|██        | 153/750 [05:25<21:06,  2.12s/it]
 21%|██        | 154/750 [05:27<21:02,  2.12s/it]
 21%|██        | 155/750 [05:29<20:50,  2.10s/it]
 21%|██        | 156/750 [05:31<21:20,  2.16s/it]
 21%|██        | 157/750 [05:33<21:06,  2.14s/it]
 21%|██        | 158/750 [05:35<20:33,  2.08s/it]
 21%|██        | 159/750 [05:37<20:30,  2.08s/it]
 21%|██▏       | 160/750 [05:39<20:49,  2.12s/it]
                                                 
+
 21%|██▏       | 160/750 [05:39<20:49,  2.12s/it]
 21%|██▏       | 161/750 [05:41<20:17,  2.07s/it]
 22%|██▏       | 162/750 [05:43<20:04,  2.05s/it]
 22%|██▏       | 163/750 [05:46<20:12,  2.07s/it]
 22%|██▏       | 164/750 [05:48<21:10,  2.17s/it]
 22%|██▏       | 165/750 [05:50<20:53,  2.14s/it]
 22%|██▏       | 166/750 [05:52<21:01,  2.16s/it]
 22%|██▏       | 167/750 [05:54<20:46,  2.14s/it]
 22%|██▏       | 168/750 [05:57<21:06,  2.18s/it]
 23%|██▎       | 169/750 [05:59<20:25,  2.11s/it]
 23%|██▎       | 170/750 [06:01<20:28,  2.12s/it]
                                                 
+
 23%|██▎       | 170/750 [06:01<20:28,  2.12s/it]
 23%|██▎       | 171/750 [06:03<20:22,  2.11s/it]
 23%|██▎       | 172/750 [06:05<20:53,  2.17s/it]
 23%|██▎       | 173/750 [06:07<20:37,  2.14s/it]
 23%|██▎       | 174/750 [06:09<20:13,  2.11s/it]
 23%|██▎       | 175/750 [06:11<20:04,  2.09s/it]
 23%|██▎       | 176/750 [06:13<20:00,  2.09s/it]
 24%|██▎       | 177/750 [06:16<22:17,  2.33s/it]
 24%|██▎       | 178/750 [06:18<21:36,  2.27s/it]
 24%|██▍       | 179/750 [06:20<20:54,  2.20s/it]
 24%|██▍       | 180/750 [06:22<20:14,  2.13s/it]
                                                 
+
 24%|██▍       | 180/750 [06:22<20:14,  2.13s/it]
 24%|██▍       | 181/750 [06:25<20:26,  2.16s/it]
 24%|██▍       | 182/750 [06:27<19:54,  2.10s/it]
 24%|██▍       | 183/750 [06:29<19:40,  2.08s/it]
 25%|██▍       | 184/750 [06:31<19:28,  2.07s/it]
 25%|██▍       | 185/750 [06:33<20:16,  2.15s/it]
 25%|██▍       | 186/750 [06:35<19:46,  2.10s/it]
 25%|██▍       | 187/750 [06:37<19:38,  2.09s/it]
 25%|██▌       | 188/750 [06:39<19:36,  2.09s/it]
 25%|██▌       | 189/750 [06:41<19:58,  2.14s/it]
 25%|██▌       | 190/750 [06:44<20:20,  2.18s/it]
                                                 
+
 25%|██▌       | 190/750 [06:44<20:20,  2.18s/it]
 25%|██▌       | 191/750 [06:46<19:53,  2.13s/it]
 26%|██▌       | 192/750 [06:48<19:26,  2.09s/it]
 26%|██▌       | 193/750 [06:50<19:47,  2.13s/it]
 26%|██▌       | 194/750 [06:52<19:30,  2.11s/it]
 26%|██▌       | 195/750 [06:54<19:22,  2.10s/it]
 26%|██▌       | 196/750 [06:56<19:15,  2.09s/it]
 26%|██▋       | 197/750 [06:58<18:59,  2.06s/it]
 26%|██▋       | 198/750 [07:00<19:36,  2.13s/it]
 27%|██▋       | 199/750 [07:02<19:22,  2.11s/it]
 27%|██▋       | 200/750 [07:05<19:35,  2.14s/it]
                                                 
+
 27%|██▋       | 200/750 [07:05<19:35,  2.14s/it]
+
+
+
 27%|██▋       | 201/750 [07:30<1:22:34,  9.02s/it]
 27%|██▋       | 202/750 [07:32<1:04:05,  7.02s/it]
 27%|██▋       | 203/750 [07:34<50:23,  5.53s/it]  
 27%|██▋       | 204/750 [07:37<42:45,  4.70s/it]
 27%|██▋       | 205/750 [07:39<35:19,  3.89s/it]
 27%|██▋       | 206/750 [07:41<30:38,  3.38s/it]
 28%|██▊       | 207/750 [07:43<26:50,  2.97s/it]
 28%|██▊       | 208/750 [07:45<24:29,  2.71s/it]
 28%|██▊       | 209/750 [07:47<22:46,  2.53s/it]
 28%|██▊       | 210/750 [07:49<21:56,  2.44s/it]
                                                 
+
 28%|██▊       | 210/750 [07:50<21:56,  2.44s/it]
 28%|██▊       | 211/750 [07:52<21:40,  2.41s/it]
 28%|██▊       | 212/750 [07:54<20:33,  2.29s/it]
 28%|██▊       | 213/750 [07:56<20:00,  2.24s/it]
 29%|██▊       | 214/750 [07:58<19:55,  2.23s/it]
 29%|██▊       | 215/750 [08:00<19:55,  2.23s/it]
 29%|██▉       | 216/750 [08:02<19:22,  2.18s/it]
 29%|██▉       | 217/750 [08:05<19:03,  2.14s/it]
 29%|██▉       | 218/750 [08:07<18:47,  2.12s/it]
 29%|██▉       | 219/750 [08:09<19:01,  2.15s/it]
 29%|██▉       | 220/750 [08:11<18:59,  2.15s/it]
                                                 
+
 29%|██▉       | 220/750 [08:11<18:59,  2.15s/it]
 29%|██▉       | 221/750 [08:13<18:48,  2.13s/it]
 30%|██▉       | 222/750 [08:15<18:44,  2.13s/it]
 30%|██▉       | 223/750 [08:18<19:26,  2.21s/it]
 30%|██▉       | 224/750 [08:20<19:10,  2.19s/it]
 30%|███       | 225/750 [08:22<18:39,  2.13s/it]
 30%|███       | 226/750 [08:24<18:29,  2.12s/it]
 30%|███       | 227/750 [08:26<19:05,  2.19s/it]
 30%|███       | 228/750 [08:28<18:58,  2.18s/it]
 31%|███       | 229/750 [08:30<18:20,  2.11s/it]
 31%|███       | 230/750 [08:32<18:25,  2.13s/it]
                                                 
+
 31%|███       | 230/750 [08:32<18:25,  2.13s/it]
 31%|███       | 231/750 [08:35<18:20,  2.12s/it]
 31%|███       | 232/750 [08:37<18:31,  2.15s/it]
 31%|███       | 233/750 [08:39<18:13,  2.12s/it]
 31%|███       | 234/750 [08:41<17:52,  2.08s/it]
 31%|███▏      | 235/750 [08:43<17:57,  2.09s/it]
 31%|███▏      | 236/750 [08:45<18:15,  2.13s/it]
 32%|███▏      | 237/750 [08:47<17:51,  2.09s/it]
 32%|███▏      | 238/750 [08:49<18:13,  2.14s/it]
 32%|███▏      | 239/750 [08:51<18:01,  2.12s/it]
 32%|███▏      | 240/750 [08:54<18:26,  2.17s/it]
                                                 
+
 32%|███▏      | 240/750 [08:54<18:26,  2.17s/it]
 32%|███▏      | 241/750 [08:56<17:56,  2.11s/it]
 32%|███▏      | 242/750 [08:58<17:36,  2.08s/it]
 32%|███▏      | 243/750 [09:00<17:28,  2.07s/it]
 33%|███▎      | 244/750 [09:02<17:49,  2.11s/it]
 33%|███▎      | 245/750 [09:04<17:27,  2.07s/it]
 33%|███▎      | 246/750 [09:06<17:23,  2.07s/it]
 33%|███▎      | 247/750 [09:08<17:24,  2.08s/it]
 33%|███▎      | 248/750 [09:10<17:45,  2.12s/it]
 33%|███▎      | 249/750 [09:12<17:35,  2.11s/it]
 33%|███▎      | 250/750 [09:14<17:16,  2.07s/it]
                                                 
+
 33%|███▎      | 250/750 [09:14<17:16,  2.07s/it]
 33%|███▎      | 251/750 [09:16<17:16,  2.08s/it]
 34%|███▎      | 252/750 [09:19<17:32,  2.11s/it]
 34%|███▎      | 253/750 [09:21<17:48,  2.15s/it]
 34%|███▍      | 254/750 [09:23<17:35,  2.13s/it]
 34%|███▍      | 255/750 [09:25<17:08,  2.08s/it]
 34%|███▍      | 256/750 [09:27<16:51,  2.05s/it]
 34%|███▍      | 257/750 [09:29<17:50,  2.17s/it]
 34%|███▍      | 258/750 [09:31<17:36,  2.15s/it]
 35%|███▍      | 259/750 [09:33<17:08,  2.09s/it]
 35%|███▍      | 260/750 [09:36<17:14,  2.11s/it]
                                                 
+
 35%|███▍      | 260/750 [09:36<17:14,  2.11s/it]
 35%|███▍      | 261/750 [09:38<17:39,  2.17s/it]
 35%|███▍      | 262/750 [09:40<17:32,  2.16s/it]
 35%|███▌      | 263/750 [09:42<17:31,  2.16s/it]
 35%|███▌      | 264/750 [09:44<17:15,  2.13s/it]
 35%|███▌      | 265/750 [09:47<17:30,  2.17s/it]
 35%|███▌      | 266/750 [09:49<17:12,  2.13s/it]
 36%|███▌      | 267/750 [09:51<16:45,  2.08s/it]
 36%|███▌      | 268/750 [09:53<16:51,  2.10s/it]
 36%|███▌      | 269/750 [09:55<17:15,  2.15s/it]
 36%|███▌      | 270/750 [09:57<17:06,  2.14s/it]
                                                 
+
 36%|███▌      | 270/750 [09:57<17:06,  2.14s/it]
 36%|███▌      | 271/750 [09:59<16:39,  2.09s/it]
 36%|███▋      | 272/750 [10:01<16:22,  2.06s/it]
 36%|███▋      | 273/750 [10:03<16:23,  2.06s/it]
 37%|███▋      | 274/750 [10:05<16:53,  2.13s/it]
 37%|███▋      | 275/750 [10:07<16:41,  2.11s/it]
 37%|███▋      | 276/750 [10:10<16:40,  2.11s/it]
 37%|███▋      | 277/750 [10:12<16:36,  2.11s/it]
 37%|███▋      | 278/750 [10:14<16:50,  2.14s/it]
 37%|███▋      | 279/750 [10:16<16:27,  2.10s/it]
 37%|███▋      | 280/750 [10:18<16:40,  2.13s/it]
                                                 
+
 37%|███▋      | 280/750 [10:18<16:40,  2.13s/it]
 37%|███▋      | 281/750 [10:20<16:28,  2.11s/it]
 38%|███▊      | 282/750 [10:22<16:59,  2.18s/it]
 38%|███▊      | 283/750 [10:25<16:54,  2.17s/it]
 38%|███▊      | 284/750 [10:27<16:44,  2.15s/it]
 38%|███▊      | 285/750 [10:29<16:32,  2.13s/it]
 38%|███▊      | 286/750 [10:31<16:41,  2.16s/it]
 38%|███▊      | 287/750 [10:33<16:27,  2.13s/it]
 38%|███▊      | 288/750 [10:35<16:12,  2.11s/it]
 39%|███▊      | 289/750 [10:37<16:20,  2.13s/it]
 39%|███▊      | 290/750 [10:39<15:59,  2.09s/it]
                                                 
+
 39%|███▊      | 290/750 [10:39<15:59,  2.09s/it]
 39%|███▉      | 291/750 [10:42<16:40,  2.18s/it]
 39%|███▉      | 292/750 [10:44<16:07,  2.11s/it]
 39%|███▉      | 293/750 [10:46<16:07,  2.12s/it]
 39%|███▉      | 294/750 [10:48<15:54,  2.09s/it]
 39%|███▉      | 295/750 [10:50<16:34,  2.19s/it]
 39%|███▉      | 296/750 [10:52<16:11,  2.14s/it]
 40%|███▉      | 297/750 [10:54<16:08,  2.14s/it]
 40%|███▉      | 298/750 [10:56<15:41,  2.08s/it]
 40%|███▉      | 299/750 [10:59<15:55,  2.12s/it]
 40%|████      | 300/750 [11:01<15:41,  2.09s/it]
                                                 
+
 40%|████      | 300/750 [11:01<15:41,  2.09s/it]
 40%|████      | 301/750 [11:03<15:24,  2.06s/it]
 40%|████      | 302/750 [11:05<15:09,  2.03s/it]
 40%|████      | 303/750 [11:07<15:08,  2.03s/it]
 41%|████      | 304/750 [11:09<15:29,  2.08s/it]
 41%|████      | 305/750 [11:11<15:21,  2.07s/it]
 41%|████      | 306/750 [11:13<15:09,  2.05s/it]
 41%|████      | 307/750 [11:15<15:03,  2.04s/it]
 41%|████      | 308/750 [11:17<15:22,  2.09s/it]
 41%|███��      | 309/750 [11:19<15:03,  2.05s/it]
 41%|████▏     | 310/750 [11:21<15:06,  2.06s/it]
                                                 
+
 41%|████▏     | 310/750 [11:21<15:06,  2.06s/it]
 41%|████▏     | 311/750 [11:23<15:12,  2.08s/it]
 42%|████▏     | 312/750 [11:26<15:45,  2.16s/it]
 42%|████▏     | 313/750 [11:28<16:31,  2.27s/it]
 42%|████▏     | 314/750 [11:30<16:01,  2.20s/it]
 42%|████▏     | 315/750 [11:32<15:30,  2.14s/it]
 42%|████▏     | 316/750 [11:34<15:53,  2.20s/it]
 42%|████▏     | 317/750 [11:36<15:26,  2.14s/it]
 42%|████▏     | 318/750 [11:38<15:11,  2.11s/it]
 43%|████▎     | 319/750 [11:41<14:56,  2.08s/it]
 43%|████▎     | 320/750 [11:43<15:19,  2.14s/it]
                                                 
+
 43%|████▎     | 320/750 [11:43<15:19,  2.14s/it]
 43%|████▎     | 321/750 [11:45<15:17,  2.14s/it]
 43%|████▎     | 322/750 [11:47<14:54,  2.09s/it]
 43%|████▎     | 323/750 [11:49<14:48,  2.08s/it]
 43%|████▎     | 324/750 [11:51<14:32,  2.05s/it]
 43%|████▎     | 325/750 [11:53<14:47,  2.09s/it]
 43%|████▎     | 326/750 [11:55<14:42,  2.08s/it]
 44%|████▎     | 327/750 [11:57<14:29,  2.06s/it]
 44%|████▎     | 328/750 [11:59<14:44,  2.10s/it]
 44%|████▍     | 329/750 [12:02<15:12,  2.17s/it]
 44%|████▍     | 330/750 [12:04<15:43,  2.25s/it]
                                                 
+
 44%|████▍     | 330/750 [12:04<15:43,  2.25s/it]
 44%|████▍     | 331/750 [12:06<15:50,  2.27s/it]
 44%|████▍     | 332/750 [12:09<15:40,  2.25s/it]
 44%|████▍     | 333/750 [12:11<16:07,  2.32s/it]
 45%|████▍     | 334/750 [12:13<15:48,  2.28s/it]
 45%|████▍     | 335/750 [12:16<16:00,  2.31s/it]
 45%|████▍     | 336/750 [12:18<15:42,  2.28s/it]
 45%|████▍     | 337/750 [12:20<16:00,  2.33s/it]
 45%|████▌     | 338/750 [12:22<15:27,  2.25s/it]
 45%|████▌     | 339/750 [12:25<15:06,  2.21s/it]
 45%|████▌     | 340/750 [12:27<15:02,  2.20s/it]
                                                 
+
 45%|████▌     | 340/750 [12:27<15:02,  2.20s/it]
 45%|████▌     | 341/750 [12:29<14:52,  2.18s/it]
 46%|████▌     | 342/750 [12:31<14:35,  2.15s/it]
 46%|████▌     | 343/750 [12:33<14:23,  2.12s/it]
 46%|████▌     | 344/750 [12:35<14:04,  2.08s/it]
 46%|████▌     | 345/750 [12:37<14:33,  2.16s/it]
 46%|████▌     | 346/750 [12:40<15:01,  2.23s/it]
 46%|████▋     | 347/750 [12:42<14:33,  2.17s/it]
 46%|████▋     | 348/750 [12:44<14:34,  2.18s/it]
 47%|████▋     | 349/750 [12:46<14:23,  2.15s/it]
 47%|████▋     | 350/750 [12:48<14:29,  2.17s/it]
                                                 
+
 47%|████▋     | 350/750 [12:48<14:29,  2.17s/it]
 47%|████▋     | 351/750 [12:50<14:15,  2.14s/it]
 47%|████▋     | 352/750 [12:52<13:58,  2.11s/it]
 47%|████▋     | 353/750 [12:54<13:43,  2.08s/it]
 47%|████▋     | 354/750 [12:57<14:02,  2.13s/it]
 47%|████▋     | 355/750 [12:59<13:49,  2.10s/it]
 47%|████▋     | 356/750 [13:01<13:28,  2.05s/it]
 48%|████▊     | 357/750 [13:03<13:31,  2.07s/it]
 48%|████▊     | 358/750 [13:05<13:45,  2.11s/it]
 48%|████▊     | 359/750 [13:07<13:26,  2.06s/it]
 48%|████▊     | 360/750 [13:09<13:14,  2.04s/it]
                                                 
+
 48%|████▊     | 360/750 [13:09<13:14,  2.04s/it]
 48%|████▊     | 361/750 [13:11<13:18,  2.05s/it]
 48%|████▊     | 362/750 [13:13<13:48,  2.14s/it]
 48%|████▊     | 363/750 [13:15<13:54,  2.16s/it]
 49%|████▊     | 364/750 [13:17<13:28,  2.10s/it]
 49%|████▊     | 365/750 [13:19<13:20,  2.08s/it]
 49%|████▉     | 366/750 [13:22<13:38,  2.13s/it]
 49%|████▉     | 367/750 [13:24<13:28,  2.11s/it]
 49%|████▉     | 368/750 [13:26<13:16,  2.08s/it]
 49%|████▉     | 369/750 [13:28<13:03,  2.06s/it]
 49%|████▉     | 370/750 [13:30<13:36,  2.15s/it]
                                                 
+
 49%|████▉     | 370/750 [13:30<13:36,  2.15s/it]
 49%|████▉     | 371/750 [13:32<13:41,  2.17s/it]
 50%|████▉     | 372/750 [13:34<13:20,  2.12s/it]
 50%|████▉     | 373/750 [13:36<12:57,  2.06s/it]
 50%|████▉     | 374/750 [13:38<12:46,  2.04s/it]
 50%|█████     | 375/750 [13:40<13:10,  2.11s/it]
 50%|█████     | 376/750 [13:43<13:02,  2.09s/it]
 50%|█████     | 377/750 [13:45<13:01,  2.09s/it]
 50%|█████     | 378/750 [13:47<12:46,  2.06s/it]
 51%|█████     | 379/750 [13:49<13:02,  2.11s/it]
 51%|█████     | 380/750 [13:51<12:44,  2.07s/it]
                                                 
+
 51%|█████     | 380/750 [13:51<12:44,  2.07s/it]
 51%|█████     | 381/750 [13:53<12:30,  2.03s/it]
 51%|█████     | 382/750 [13:55<12:38,  2.06s/it]
 51%|█████     | 383/750 [13:57<13:09,  2.15s/it]
 51%|█████     | 384/750 [13:59<13:04,  2.14s/it]
 51%|█████▏    | 385/750 [14:01<12:54,  2.12s/it]
 51%|█████▏    | 386/750 [14:03<12:42,  2.10s/it]
 52%|█████▏    | 387/750 [14:06<13:20,  2.21s/it]
 52%|█████▏    | 388/750 [14:08<13:19,  2.21s/it]
 52%|█████▏    | 389/750 [14:10<12:58,  2.16s/it]
 52%|█████▏    | 390/750 [14:12<12:39,  2.11s/it]
                                                 
+
 52%|█████▏    | 390/750 [14:12<12:39,  2.11s/it]
 52%|█████▏    | 391/750 [14:14<12:31,  2.09s/it]
 52%|█████▏    | 392/750 [14:17<13:23,  2.24s/it]
 52%|█████▏    | 393/750 [14:19<12:58,  2.18s/it]
 53%|█████▎    | 394/750 [14:21<12:53,  2.17s/it]
 53%|█████▎    | 395/750 [14:23<12:32,  2.12s/it]
 53%|█████▎    | 396/750 [14:25<12:45,  2.16s/it]
 53%|█████▎    | 397/750 [14:27<12:24,  2.11s/it]
 53%|█████▎    | 398/750 [14:29<12:21,  2.11s/it]
 53%|█████▎    | 399/750 [14:31<12:10,  2.08s/it]
 53%|█████▎    | 400/750 [14:34<12:34,  2.16s/it]
                                                 
+
 53%|█████▎    | 400/750 [14:34<12:34,  2.16s/it]
+
+
+
 53%|█████▎    | 401/750 [14:58<50:15,  8.64s/it]
 54%|█████▎    | 402/750 [14:59<38:30,  6.64s/it]
 54%|█████▎    | 403/750 [15:02<30:29,  5.27s/it]
 54%|█████▍    | 404/750 [15:04<25:06,  4.35s/it]
 54%|█████▍    | 405/750 [15:06<21:24,  3.72s/it]
 54%|█████▍    | 406/750 [15:08<18:20,  3.20s/it]
 54%|█████▍    | 407/750 [15:10<16:17,  2.85s/it]
 54%|█████▍    | 408/750 [15:12<14:38,  2.57s/it]
 55%|█████▍    | 409/750 [15:14<14:14,  2.50s/it]
 55%|█████▍    | 410/750 [15:16<13:22,  2.36s/it]
                                                 
+
 55%|█████▍    | 410/750 [15:16<13:22,  2.36s/it]
 55%|█████▍    | 411/750 [15:18<12:44,  2.25s/it]
 55%|█████▍    | 412/750 [15:20<12:22,  2.20s/it]
 55%|█████▌    | 413/750 [15:23<12:29,  2.22s/it]
 55%|█████▌    | 414/750 [15:25<12:21,  2.21s/it]
 55%|█████▌    | 415/750 [15:27<12:02,  2.16s/it]
 55%|█████▌    | 416/750 [15:29<11:52,  2.13s/it]
 56%|█████▌    | 417/750 [15:31<12:00,  2.16s/it]
 56%|█████▌    | 418/750 [15:33<11:44,  2.12s/it]
 56%|█████▌    | 419/750 [15:35<11:32,  2.09s/it]
 56%|█████▌    | 420/750 [15:37<11:29,  2.09s/it]
                                                 
+
 56%|█████▌    | 420/750 [15:37<11:29,  2.09s/it]
 56%|█████▌    | 421/750 [15:40<11:48,  2.15s/it]
 56%|█████▋    | 422/750 [15:42<11:36,  2.12s/it]
 56%|█████▋    | 423/750 [15:44<11:24,  2.09s/it]
 57%|█████▋    | 424/750 [15:46<11:14,  2.07s/it]
 57%|█████▋    | 425/750 [15:48<11:24,  2.11s/it]
 57%|█████▋    | 426/750 [15:50<11:35,  2.15s/it]
 57%|█████▋    | 427/750 [15:52<11:19,  2.10s/it]
 57%|█████▋    | 428/750 [15:54<11:07,  2.07s/it]
 57%|█████▋    | 429/750 [15:56<11:16,  2.11s/it]
 57%|█████▋    | 430/750 [15:59<11:29,  2.16s/it]
                                                 
+
 57%|█████▋    | 430/750 [15:59<11:29,  2.16s/it]
 57%|█████▋    | 431/750 [16:01<11:20,  2.13s/it]
 58%|█████▊    | 432/750 [16:03<11:00,  2.08s/it]
 58%|█████▊    | 433/750 [16:05<11:01,  2.09s/it]
 58%|█████▊    | 434/750 [16:07<11:14,  2.14s/it]
 58%|█████▊    | 435/750 [16:09<11:10,  2.13s/it]
 58%|█████▊    | 436/750 [16:11<10:52,  2.08s/it]
 58%|█████▊    | 437/750 [16:13<11:02,  2.12s/it]
 58%|█████▊    | 438/750 [16:15<11:05,  2.13s/it]
 59%|█████▊    | 439/750 [16:17<10:47,  2.08s/it]
 59%|█████▊    | 440/750 [16:19<10:37,  2.06s/it]
                                                 
+
 59%|█████▊    | 440/750 [16:19<10:37,  2.06s/it]
 59%|█████▉    | 441/750 [16:21<10:26,  2.03s/it]
 59%|█████▉    | 442/750 [16:23<10:25,  2.03s/it]
 59%|█████▉    | 443/750 [16:26<10:36,  2.07s/it]
 59%|█████▉    | 444/750 [16:28<10:38,  2.09s/it]
 59%|█████▉    | 445/750 [16:30<10:30,  2.07s/it]
 59%|█████▉    | 446/750 [16:32<10:22,  2.05s/it]
 60%|█████▉    | 447/750 [16:34<10:31,  2.08s/it]
 60%|█████▉    | 448/750 [16:36<10:16,  2.04s/it]
 60%|█████▉    | 449/750 [16:38<10:27,  2.09s/it]
 60%|██████    | 450/750 [16:40<10:13,  2.05s/it]
                                                 
+
 60%|██████    | 450/750 [16:40<10:13,  2.05s/it]
 60%|██████    | 451/750 [16:42<10:28,  2.10s/it]
 60%|██████    | 452/750 [16:44<10:19,  2.08s/it]
 60%|██████    | 453/750 [16:46<10:19,  2.09s/it]
 61%|██████    | 454/750 [16:48<10:08,  2.06s/it]
 61%|██████    | 455/750 [16:51<10:27,  2.13s/it]
 61%|██████    | 456/750 [16:53<10:25,  2.13s/it]
 61%|██████    | 457/750 [16:55<10:24,  2.13s/it]
 61%|██████    | 458/750 [16:57<10:46,  2.21s/it]
 61%|██████    | 459/750 [16:59<10:23,  2.14s/it]
 61%|██████▏   | 460/750 [17:02<10:37,  2.20s/it]
                                                 
+
 61%|██████▏   | 460/750 [17:02<10:37,  2.20s/it]
 61%|██████▏   | 461/750 [17:04<10:14,  2.12s/it]
 62%|██████▏   | 462/750 [17:06<10:15,  2.14s/it]
 62%|██████▏   | 463/750 [17:08<10:09,  2.12s/it]
 62%|██████▏   | 464/750 [17:10<10:20,  2.17s/it]
 62%|██████▏   | 465/750 [17:12<10:05,  2.12s/it]
 62%|██████▏   | 466/750 [17:14<10:01,  2.12s/it]
 62%|██████▏   | 467/750 [17:16<09:49,  2.08s/it]
 62%|██████▏   | 468/750 [17:18<10:01,  2.13s/it]
 63%|██████▎   | 469/750 [17:20<09:48,  2.09s/it]
 63%|██████▎   | 470/750 [17:22<09:40,  2.07s/it]
                                                 
+
 63%|██████▎   | 470/750 [17:23<09:40,  2.07s/it]
 63%|██████▎   | 471/750 [17:25<09:38,  2.07s/it]
 63%|██████▎   | 472/750 [17:27<09:46,  2.11s/it]
 63%|██████▎   | 473/750 [17:29<09:36,  2.08s/it]
 63%|██████▎   | 474/750 [17:31<09:36,  2.09s/it]
 63%|██████▎   | 475/750 [17:33<09:28,  2.07s/it]
 63%|██████▎   | 476/750 [17:35<09:37,  2.11s/it]
 64%|█████��▎   | 477/750 [17:37<09:21,  2.06s/it]
 64%|██████▎   | 478/750 [17:39<09:38,  2.13s/it]
 64%|██████▍   | 479/750 [17:41<09:39,  2.14s/it]
 64%|██████▍   | 480/750 [17:44<09:31,  2.12s/it]
                                                 
+
 64%|██████▍   | 480/750 [17:44<09:31,  2.12s/it]
 64%|██████▍   | 481/750 [17:46<09:36,  2.14s/it]
 64%|██████▍   | 482/750 [17:48<09:22,  2.10s/it]
 64%|██████▍   | 483/750 [17:50<09:22,  2.11s/it]
 65%|██████▍   | 484/750 [17:52<09:18,  2.10s/it]
 65%|██████▍   | 485/750 [17:54<09:26,  2.14s/it]
 65%|██████▍   | 486/750 [17:56<09:18,  2.11s/it]
 65%|██████▍   | 487/750 [17:58<09:02,  2.06s/it]
 65%|██████▌   | 488/750 [18:00<09:01,  2.07s/it]
 65%|██████▌   | 489/750 [18:03<09:18,  2.14s/it]
 65%|██████▌   | 490/750 [18:05<09:10,  2.12s/it]
                                                 
+
 65%|██████▌   | 490/750 [18:05<09:10,  2.12s/it]
 65%|██████▌   | 491/750 [18:07<09:27,  2.19s/it]
 66%|██████▌   | 492/750 [18:09<09:15,  2.15s/it]
 66%|██████▌   | 493/750 [18:11<09:18,  2.17s/it]
 66%|██████▌   | 494/750 [18:13<09:11,  2.15s/it]
 66%|██████▌   | 495/750 [18:16<09:12,  2.17s/it]
 66%|██████▌   | 496/750 [18:18<08:57,  2.12s/it]
 66%|██████▋   | 497/750 [18:20<09:14,  2.19s/it]
 66%|██████▋   | 498/750 [18:22<09:10,  2.18s/it]
 67%|██████▋   | 499/750 [18:24<08:59,  2.15s/it]
 67%|██████▋   | 500/750 [18:26<08:51,  2.13s/it]
                                                 
+
 67%|██████▋   | 500/750 [18:26<08:51,  2.13s/it]
 67%|██████▋   | 501/750 [18:28<08:45,  2.11s/it]
 67%|██████▋   | 502/750 [18:31<08:55,  2.16s/it]
 67%|██████▋   | 503/750 [18:33<08:47,  2.14s/it]
 67%|██████▋   | 504/750 [18:35<08:48,  2.15s/it]
 67%|██████▋   | 505/750 [18:37<08:34,  2.10s/it]
 67%|██████▋   | 506/750 [18:39<08:56,  2.20s/it]
 68%|██████▊   | 507/750 [18:42<09:14,  2.28s/it]
 68%|██████▊   | 508/750 [18:44<08:59,  2.23s/it]
 68%|██████▊   | 509/750 [18:46<08:51,  2.20s/it]
 68%|██████▊   | 510/750 [18:48<08:59,  2.25s/it]
                                                 
+
 68%|██████▊   | 510/750 [18:48<08:59,  2.25s/it]
 68%|██████▊   | 511/750 [18:51<08:50,  2.22s/it]
 68%|██████▊   | 512/750 [18:53<08:33,  2.16s/it]
 68%|██████▊   | 513/750 [18:55<08:25,  2.13s/it]
 69%|██████▊   | 514/750 [18:57<08:43,  2.22s/it]
 69%|██████▊   | 515/750 [18:59<08:26,  2.15s/it]
 69%|██████▉   | 516/750 [19:01<08:16,  2.12s/it]
 69%|██████▉   | 517/750 [19:03<08:21,  2.15s/it]
 69%|██████▉   | 518/750 [19:06<08:34,  2.22s/it]
 69%|██████▉   | 519/750 [19:08<08:17,  2.15s/it]
 69%|██████▉   | 520/750 [19:10<08:06,  2.12s/it]
                                                 
+
 69%|██████▉   | 520/750 [19:10<08:06,  2.12s/it]
 69%|██████▉   | 521/750 [19:12<07:58,  2.09s/it]
 70%|██████▉   | 522/750 [19:14<08:11,  2.15s/it]
 70%|██████▉   | 523/750 [19:16<08:01,  2.12s/it]
 70%|██████▉   | 524/750 [19:18<07:58,  2.12s/it]
 70%|███████   | 525/750 [19:20<07:48,  2.08s/it]
 70%|███████   | 526/750 [19:22<07:48,  2.09s/it]
 70%|███████   | 527/750 [19:25<08:03,  2.17s/it]
 70%|███████   | 528/750 [19:27<08:10,  2.21s/it]
 71%|███████   | 529/750 [19:29<08:13,  2.23s/it]
 71%|███████   | 530/750 [19:32<08:13,  2.24s/it]
                                                 
+
 71%|███████   | 530/750 [19:32<08:13,  2.24s/it]
 71%|███████   | 531/750 [19:34<08:19,  2.28s/it]
 71%|███████   | 532/750 [19:36<08:02,  2.21s/it]
 71%|███████   | 533/750 [19:38<07:56,  2.19s/it]
 71%|███████   | 534/750 [19:40<07:45,  2.15s/it]
 71%|███████▏  | 535/750 [19:42<07:47,  2.18s/it]
 71%|███████▏  | 536/750 [19:44<07:33,  2.12s/it]
 72%|███████▏  | 537/750 [19:46<07:31,  2.12s/it]
 72%|███████▏  | 538/750 [19:48<07:21,  2.08s/it]
 72%|███████▏  | 539/750 [19:51<07:31,  2.14s/it]
 72%|███████▏  | 540/750 [19:53<07:24,  2.12s/it]
                                                 
+
 72%|███████▏  | 540/750 [19:53<07:24,  2.12s/it]
 72%|███████▏  | 541/750 [19:55<07:16,  2.09s/it]
 72%|███████▏  | 542/750 [19:57<07:13,  2.08s/it]
 72%|███████▏  | 543/750 [19:59<07:34,  2.20s/it]
 73%|███████▎  | 544/750 [20:01<07:21,  2.14s/it]
 73%|███████▎  | 545/750 [20:03<07:13,  2.11s/it]
 73%|███████▎  | 546/750 [20:05<07:05,  2.08s/it]
 73%|███████▎  | 547/750 [20:07<06:56,  2.05s/it]
 73%|███████▎  | 548/750 [20:10<07:07,  2.12s/it]
 73%|███████▎  | 549/750 [20:12<06:56,  2.07s/it]
 73%|███████▎  | 550/750 [20:14<06:50,  2.05s/it]
                                                 
+
 73%|███████▎  | 550/750 [20:14<06:50,  2.05s/it]
 73%|███████▎  | 551/750 [20:16<06:48,  2.05s/it]
 74%|███████▎  | 552/750 [20:18<07:02,  2.13s/it]
 74%|███████▎  | 553/750 [20:20<06:54,  2.11s/it]
 74%|███████▍  | 554/750 [20:22<06:48,  2.08s/it]
 74%|███████▍  | 555/750 [20:24<06:44,  2.07s/it]
 74%|███████▍  | 556/750 [20:26<06:50,  2.12s/it]
 74%|███████▍  | 557/750 [20:28<06:45,  2.10s/it]
 74%|███████▍  | 558/750 [20:31<06:49,  2.13s/it]
 75%|███████▍  | 559/750 [20:33<06:39,  2.09s/it]
 75%|███████▍  | 560/750 [20:35<06:42,  2.12s/it]
                                                 
+
 75%|███████▍  | 560/750 [20:35<06:42,  2.12s/it]
 75%|███████▍  | 561/750 [20:37<06:44,  2.14s/it]
 75%|███████▍  | 562/750 [20:39<06:32,  2.09s/it]
 75%|███████▌  | 563/750 [20:41<06:35,  2.11s/it]
 75%|███████▌  | 564/750 [20:43<06:42,  2.16s/it]
 75%|███████▌  | 565/750 [20:46<06:39,  2.16s/it]
 75%|███████▌  | 566/750 [20:48<06:33,  2.14s/it]
 76%|███████▌  | 567/750 [20:50<06:22,  2.09s/it]
 76%|███████▌  | 568/750 [20:52<06:20,  2.09s/it]
 76%|███████▌  | 569/750 [20:54<06:32,  2.17s/it]
 76%|███████▌  | 570/750 [20:56<06:19,  2.11s/it]
                                                 
+
 76%|███████▌  | 570/750 [20:56<06:19,  2.11s/it]
 76%|███████▌  | 571/750 [20:58<06:14,  2.09s/it]
 76%|███████▋  | 572/750 [21:00<06:14,  2.10s/it]
 76%|███████▋  | 573/750 [21:03<06:20,  2.15s/it]
 77%|███████▋  | 574/750 [21:05<06:32,  2.23s/it]
 77%|███████▋  | 575/750 [21:07<06:22,  2.19s/it]
 77%|███████▋  | 576/750 [21:09<06:14,  2.15s/it]
 77%|███████▋  | 577/750 [21:11<06:16,  2.18s/it]
 77%|███████▋  | 578/750 [21:13<06:11,  2.16s/it]
 77%|███████▋  | 579/750 [21:15<06:02,  2.12s/it]
 77%|███████▋  | 580/750 [21:17<05:54,  2.08s/it]
                                                 
+
 77%|███████▋  | 580/750 [21:17<05:54,  2.08s/it]
 77%|███████▋  | 581/750 [21:20<06:04,  2.15s/it]
 78%|███████▊  | 582/750 [21:22<05:53,  2.10s/it]
 78%|███████▊  | 583/750 [21:24<05:47,  2.08s/it]
 78%|███████▊  | 584/750 [21:26<05:39,  2.05s/it]
 78%|███████▊  | 585/750 [21:28<05:52,  2.14s/it]
 78%|███████▊  | 586/750 [21:30<05:47,  2.12s/it]
 78%|███████▊  | 587/750 [21:32<05:47,  2.13s/it]
 78%|███████▊  | 588/750 [21:34<05:38,  2.09s/it]
 79%|███████▊  | 589/750 [21:36<05:36,  2.09s/it]
 79%|███████▊  | 590/750 [21:39<05:39,  2.12s/it]
                                                 
+
 79%|███████▊  | 590/750 [21:39<05:39,  2.12s/it]
 79%|███████▉  | 591/750 [21:41<05:46,  2.18s/it]
 79%|███████▉  | 592/750 [21:43<05:35,  2.12s/it]
 79%|███████▉  | 593/750 [21:45<05:37,  2.15s/it]
 79%|███████▉  | 594/750 [21:47<05:38,  2.17s/it]
 79%|███████▉  | 595/750 [21:49<05:31,  2.14s/it]
 79%|███████▉  | 596/750 [21:51<05:20,  2.08s/it]
 80%|███████▉  | 597/750 [21:53<05:11,  2.04s/it]
 80%|███████▉  | 598/750 [21:56<05:19,  2.10s/it]
 80%|███████▉  | 599/750 [21:58<05:15,  2.09s/it]
 80%|████████  | 600/750 [22:00<05:10,  2.07s/it]
                                                 
+
 80%|████████  | 600/750 [22:00<05:10,  2.07s/it]
+
+
+
 80%|████████  | 601/750 [22:25<22:22,  9.01s/it]
 80%|████████  | 602/750 [22:28<17:34,  7.13s/it]
 80%|████████  | 603/750 [22:30<13:45,  5.61s/it]
 81%|████████  | 604/750 [22:32<11:07,  4.57s/it]
 81%|████████  | 605/750 [22:34<09:16,  3.84s/it]
 81%|████████  | 606/750 [22:36<08:00,  3.34s/it]
 81%|████████  | 607/750 [22:38<07:02,  2.96s/it]
 81%|████████  | 608/750 [22:40<06:17,  2.66s/it]
 81%|████████  | 609/750 [22:42<05:46,  2.46s/it]
 81%|████████▏ | 610/750 [22:44<05:22,  2.30s/it]
                                                 
+
 81%|████████▏ | 610/750 [22:44<05:22,  2.30s/it]
 81%|████████▏ | 611/750 [22:46<05:15,  2.27s/it]
 82%|████████▏ | 612/750 [22:48<05:12,  2.26s/it]
 82%|████████▏ | 613/750 [22:51<05:03,  2.22s/it]
 82%|████████▏ | 614/750 [22:53<04:54,  2.17s/it]
 82%|████████▏ | 615/750 [22:55<04:56,  2.20s/it]
 82%|████████▏ | 616/750 [22:57<04:45,  2.13s/it]
 82%|████████▏ | 617/750 [22:59<04:43,  2.13s/it]
 82%|████████▏ | 618/750 [23:01<04:37,  2.10s/it]
 83%|████████▎ | 619/750 [23:03<04:43,  2.16s/it]
 83%|████████▎ | 620/750 [23:05<04:39,  2.15s/it]
                                                 
+
 83%|████████▎ | 620/750 [23:06<04:39,  2.15s/it]
 83%|████████▎ | 621/750 [23:08<04:37,  2.15s/it]
 83%|████████▎ | 622/750 [23:10<04:32,  2.13s/it]
 83%|████████▎ | 623/750 [23:12<04:32,  2.15s/it]
 83%|████████▎ | 624/750 [23:14<04:24,  2.10s/it]
 83%|████████▎ | 625/750 [23:16<04:15,  2.05s/it]
 83%|████████▎ | 626/750 [23:18<04:16,  2.07s/it]
 84%|████████▎ | 627/750 [23:20<04:18,  2.10s/it]
 84%|████████▎ | 628/750 [23:22<04:20,  2.13s/it]
 84%|████████▍ | 629/750 [23:24<04:13,  2.10s/it]
 84%|████████▍ | 630/750 [23:26<04:07,  2.06s/it]
                                                 
+
 84%|████████▍ | 630/750 [23:26<04:07,  2.06s/it]
 84%|████████▍ | 631/750 [23:28<04:05,  2.06s/it]
 84%|████████▍ | 632/750 [23:31<04:09,  2.11s/it]
 84%|████████▍ | 633/750 [23:33<04:03,  2.08s/it]
 85%|████████▍ | 634/750 [23:35<04:13,  2.18s/it]
 85%|████████▍ | 635/750 [23:37<04:04,  2.12s/it]
 85%|████████▍ | 636/750 [23:39<04:05,  2.16s/it]
 85%|████████▍ | 637/750 [23:41<03:56,  2.09s/it]
 85%|████████▌ | 638/750 [23:43<03:49,  2.04s/it]
 85%|████████▌ | 639/750 [23:45<03:44,  2.02s/it]
 85%|████████▌ | 640/750 [23:47<03:53,  2.12s/it]
                                                 
+
 85%|████████▌ | 640/750 [23:47<03:53,  2.12s/it]
 85%|████████▌ | 641/750 [23:50<03:49,  2.10s/it]
 86%|████████▌ | 642/750 [23:52<03:45,  2.09s/it]
 86%|████████▌ | 643/750 [23:54<03:40,  2.06s/it]
 86%|████████▌ | 644/750 [23:56<03:37,  2.05s/it]
 86%|████████▌ | 645/750 [23:58<03:39,  2.09s/it]
 86%|████████▌ | 646/750 [24:00<03:38,  2.10s/it]
 86%|████████▋ | 647/750 [24:02<03:36,  2.10s/it]
 86%|████████▋ | 648/750 [24:04<03:30,  2.06s/it]
 87%|████████▋ | 649/750 [24:06<03:38,  2.16s/it]
 87%|████████▋ | 650/750 [24:09<03:35,  2.15s/it]
                                                 
+
 87%|████████▋ | 650/750 [24:09<03:35,  2.15s/it]
 87%|████████▋ | 651/750 [24:11<03:33,  2.16s/it]
 87%|████████▋ | 652/750 [24:13<03:28,  2.13s/it]
 87%|████████▋ | 653/750 [24:15<03:32,  2.19s/it]
 87%|████████▋ | 654/750 [24:17<03:27,  2.16s/it]
 87%|████████▋ | 655/750 [24:19<03:22,  2.13s/it]
 87%|████████▋ | 656/750 [24:21<03:19,  2.12s/it]
 88%|████████▊ | 657/750 [24:24<03:22,  2.17s/it]
 88%|████████▊ | 658/750 [24:26<03:19,  2.17s/it]
 88%|████████▊ | 659/750 [24:28<03:21,  2.22s/it]
 88%|████████▊ | 660/750 [24:30<03:15,  2.17s/it]
                                                 
+
 88%|████████▊ | 660/750 [24:30<03:15,  2.17s/it]
 88%|████████▊ | 661/750 [24:32<03:15,  2.19s/it]
 88%|████████▊ | 662/750 [24:34<03:08,  2.15s/it]
 88%|████████▊ | 663/750 [24:37<03:07,  2.16s/it]
 89%|████████▊ | 664/750 [24:39<03:02,  2.13s/it]
 89%|████████▊ | 665/750 [24:41<02:58,  2.10s/it]
 89%|████████▉ | 666/750 [24:43<02:59,  2.13s/it]
 89%|████████▉ | 667/750 [24:45<02:54,  2.10s/it]
 89%|████████▉ | 668/750 [24:47<02:51,  2.09s/it]
 89%|████████▉ | 669/750 [24:49<02:46,  2.06s/it]
 89%|████████▉ | 670/750 [24:51<02:46,  2.08s/it]
                                                 
+
 89%|████████▉ | 670/750 [24:51<02:46,  2.08s/it]
 89%|████████▉ | 671/750 [24:53<02:41,  2.04s/it]
 90%|████████▉ | 672/750 [24:55<02:43,  2.10s/it]
 90%|████████▉ | 673/750 [24:57<02:41,  2.10s/it]
 90%|████████▉ | 674/750 [25:00<02:42,  2.13s/it]
 90%|█████████ | 675/750 [25:02<02:40,  2.14s/it]
 90%|█████████ | 676/750 [25:04<02:33,  2.08s/it]
 90%|█████████ | 677/750 [25:06<02:31,  2.08s/it]
 90%|█████████ | 678/750 [25:08<02:33,  2.13s/it]
 91%|█████████ | 679/750 [25:10<02:36,  2.21s/it]
 91%|█████████ | 680/750 [25:12<02:31,  2.16s/it]
                                                 
+
 91%|█████████ | 680/750 [25:13<02:31,  2.16s/it]
 91%|█████████ | 681/750 [25:15<02:28,  2.16s/it]
 91%|█████████ | 682/750 [25:17<02:28,  2.18s/it]
 91%|█████████ | 683/750 [25:19<02:22,  2.13s/it]
 91%|█████████ | 684/750 [25:21<02:18,  2.10s/it]
 91%|█████████▏| 685/750 [25:23<02:15,  2.08s/it]
 91%|█████████▏| 686/750 [25:25<02:17,  2.15s/it]
 92%|█████████▏| 687/750 [25:27<02:12,  2.10s/it]
 92%|█████████▏| 688/750 [25:29<02:09,  2.09s/it]
 92%|█████████▏| 689/750 [25:32<02:08,  2.11s/it]
 92%|█████████▏| 690/750 [25:34<02:05,  2.08s/it]
                                                 
+
 92%|█████████▏| 690/750 [25:34<02:05,  2.08s/it]
 92%|█████████▏| 691/750 [25:36<02:07,  2.15s/it]
 92%|█████████▏| 692/750 [25:38<02:03,  2.14s/it]
 92%|█████████▏| 693/750 [25:40<02:05,  2.20s/it]
 93%|█████████▎| 694/750 [25:42<01:58,  2.12s/it]
 93%|█████████▎| 695/750 [25:44<01:59,  2.17s/it]
 93%|█████████▎| 696/750 [25:46<01:53,  2.11s/it]
 93%|█████████▎| 697/750 [25:49<01:52,  2.12s/it]
 93%|█████████▎| 698/750 [25:51<01:49,  2.11s/it]
 93%|█████████▎| 699/750 [25:53<01:48,  2.13s/it]
 93%|█████████▎| 700/750 [25:55<01:45,  2.10s/it]
                                                 
+
 93%|█████████▎| 700/750 [25:55<01:45,  2.10s/it]
 93%|█████████▎| 701/750 [25:57<01:42,  2.08s/it]
 94%|█████████▎| 702/750 [25:59<01:38,  2.05s/it]
 94%|█████████▎| 703/750 [26:01<01:39,  2.12s/it]
 94%|█████████▍| 704/750 [26:03<01:35,  2.07s/it]
 94%|█████████▍| 705/750 [26:05<01:32,  2.05s/it]
 94%|█████████▍| 706/750 [26:07<01:29,  2.04s/it]
 94%|█████████▍| 707/750 [26:09<01:26,  2.02s/it]
 94%|█████████▍| 708/750 [26:12<01:29,  2.13s/it]
 95%|█████████▍| 709/750 [26:14<01:27,  2.14s/it]
 95%|█████████▍| 710/750 [26:16<01:24,  2.12s/it]
                                                 
+
 95%|█████████▍| 710/750 [26:16<01:24,  2.12s/it]
 95%|█████████▍| 711/750 [26:18<01:20,  2.07s/it]
 95%|█████████▍| 712/750 [26:20<01:21,  2.14s/it]
 95%|█████████▌| 713/750 [26:22<01:17,  2.10s/it]
 95%|█████████▌| 714/750 [26:24<01:15,  2.10s/it]
 95%|█████████▌| 715/750 [26:26<01:13,  2.11s/it]
 95%|█████████▌| 716/750 [26:28<01:12,  2.14s/it]
 96%|█████████▌| 717/750 [26:31<01:10,  2.13s/it]
 96%|█████████▌| 718/750 [26:33<01:07,  2.12s/it]
 96%|█████████▌| 719/750 [26:35<01:04,  2.09s/it]
 96%|█████████▌| 720/750 [26:37<01:04,  2.14s/it]
                                                 
+
 96%|█████████▌| 720/750 [26:37<01:04,  2.14s/it]
 96%|█████████▌| 721/750 [26:39<01:01,  2.12s/it]
 96%|█████████▋| 722/750 [26:41<00:58,  2.10s/it]
 96%|█████████▋| 723/750 [26:43<00:56,  2.11s/it]
 97%|█████████▋| 724/750 [26:45<00:53,  2.07s/it]
 97%|█████████▋| 725/750 [26:47<00:53,  2.13s/it]
 97%|█████████▋| 726/750 [26:49<00:49,  2.07s/it]
 97%|█████████▋| 727/750 [26:51<00:47,  2.06s/it]
 97%|█████████▋| 728/750 [26:54<00:45,  2.09s/it]
 97%|█████████▋| 729/750 [26:56<00:44,  2.14s/it]
 97%|█████████▋| 730/750 [26:58<00:42,  2.11s/it]
                                                 
+
 97%|█████████▋| 730/750 [26:58<00:42,  2.11s/it]
 97%|█████████▋| 731/750 [27:00<00:39,  2.08s/it]
 98%|█████████▊| 732/750 [27:02<00:36,  2.05s/it]
 98%|█████████▊| 733/750 [27:04<00:35,  2.11s/it]
 98%|█████████▊| 734/750 [27:06<00:33,  2.08s/it]
 98%|█████████▊| 735/750 [27:08<00:30,  2.05s/it]
 98%|█████████▊| 736/750 [27:10<00:28,  2.06s/it]
 98%|█████████▊| 737/750 [27:12<00:27,  2.11s/it]
 98%|█████████▊| 738/750 [27:15<00:25,  2.11s/it]
 99%|█████████▊| 739/750 [27:17<00:22,  2.09s/it]
 99%|█████████▊| 740/750 [27:19<00:20,  2.07s/it]
                                                 
+
 99%|█████████▊| 740/750 [27:19<00:20,  2.07s/it]
 99%|█████████▉| 741/750 [27:21<00:18,  2.08s/it]
 99%|█████████▉| 742/750 [27:23<00:16,  2.12s/it]
 99%|█████████▉| 743/750 [27:25<00:14,  2.12s/it]
 99%|█████████▉| 744/750 [27:28<00:13,  2.23s/it]
 99%|█████████▉| 745/750 [27:29<00:10,  2.15s/it]
 99%|█████████▉| 746/750 [27:32<00:08,  2.15s/it]
                                                 
+
+
+
+
                                                 
+
+
+Saved to /scratch/sohaddad/checkpoints/sft
+wandb: 
+wandb: 🚀 View run math_model_sft at: https://wandb.ai/duo_run/huggingface/runs/r0gc2y2j
+wandb: Find logs at: wandb/run-20260512_183722-r0gc2y2j/logs

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be830bdc5f60d908c726016e2883e3030a67ac3c95df8f0b3975ce517d68f4c6
+size 5777