Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-2352/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,58 @@
+---
+base_model: Qwen/Qwen3-8B
+library_name: transformers
+model_name: complexities_12345
+tags:
+- generated_from_trainer
+- sft
+- trl
+licence: license
+---
+
+# Model Card for complexities_12345
+
+This model is a fine-tuned version of [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B).
+It has been trained using [TRL](https://github.com/huggingface/trl).
+
+## Quick start
+
+```python
+from transformers import pipeline
+
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="None", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
+
+## Training procedure
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/lion_epfl/esa/runs/xstrsw0t) 
+
+
+
+This model was trained with SFT.
+
+### Framework versions
+
+- TRL: 1.5.1
+- Transformers: 5.10.2
+- Pytorch: 2.11.0+cu130
+- Datasets: 5.0.0
+- Tokenizers: 0.22.2
+
+## Citations
+
+
+
+Cite TRL as:
+    
+```bibtex
+@software{vonwerra2020trl,
+  title   = {{TRL: Transformers Reinforcement Learning}},
+  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
+  license = {Apache-2.0},
+  url     = {https://github.com/huggingface/trl},
+  year    = {2020}
+}
+```

checkpoint-2352/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Qwen/Qwen3-8B
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen3-8B
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

checkpoint-2352/adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen3-8B",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 8,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2352/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f57c1a49d08ef14f20f20e35f50de05c4237cb7cbb0dd791e9e7d24626270a67
+size 30709192

checkpoint-2352/chat_template.jinja

@@ -0,0 +1,89 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

checkpoint-2352/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee57f7ee0b5f1485aa3f776d0df5c0ea7f51009282d2e8d2beae793f7fee125a
+size 61587787

checkpoint-2352/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4a9f217e852f439efa6bd32fde98d6867f11aa6ea13ddc021ba10af6a0b0934
+size 14645

checkpoint-2352/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee44d5d650acf7f902c30f4c76d9a3d3f3dad3c889460dd247b20eb4c82f74a0
+size 1465

checkpoint-2352/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

checkpoint-2352/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2352/trainer_state.json

@@ -0,0 +1,2406 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2352,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0,
+      "eval_entropy": 0.7441253011463252,
+      "eval_loss": 2.4717867374420166,
+      "eval_mean_token_accuracy": 0.6263054841347323,
+      "eval_num_tokens": 0.0,
+      "eval_runtime": 44.5378,
+      "eval_samples_per_second": 23.463,
+      "eval_steps_per_second": 5.883,
+      "step": 0
+    },
+    {
+      "entropy": 0.7895201802253723,
+      "epoch": 0.004251700680272109,
+      "grad_norm": 0.8038196563720703,
+      "learning_rate": 9.961734693877551e-05,
+      "loss": 2.4038867950439453,
+      "mean_token_accuracy": 0.6224425852298736,
+      "num_tokens": 9328.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.8704251646995544,
+      "epoch": 0.008503401360544218,
+      "grad_norm": 0.7324869632720947,
+      "learning_rate": 9.91921768707483e-05,
+      "loss": 1.9501838684082031,
+      "mean_token_accuracy": 0.6429464161396027,
+      "num_tokens": 19473.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.0007735908031463,
+      "epoch": 0.012755102040816327,
+      "grad_norm": 1.2104312181472778,
+      "learning_rate": 9.876700680272109e-05,
+      "loss": 1.489585781097412,
+      "mean_token_accuracy": 0.6886208653450012,
+      "num_tokens": 29449.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.2490371704101562,
+      "epoch": 0.017006802721088437,
+      "grad_norm": 0.35242822766304016,
+      "learning_rate": 9.834183673469389e-05,
+      "loss": 1.4031854629516602,
+      "mean_token_accuracy": 0.6893938302993774,
+      "num_tokens": 38885.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.156481719017029,
+      "epoch": 0.021258503401360544,
+      "grad_norm": 0.29233095049858093,
+      "learning_rate": 9.791666666666667e-05,
+      "loss": 1.13862886428833,
+      "mean_token_accuracy": 0.7356042921543121,
+      "num_tokens": 49658.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.0561675131320953,
+      "epoch": 0.025510204081632654,
+      "grad_norm": 0.44513002038002014,
+      "learning_rate": 9.749149659863947e-05,
+      "loss": 1.1001633644104003,
+      "mean_token_accuracy": 0.7346036016941071,
+      "num_tokens": 60560.0,
+      "step": 60
+    },
+    {
+      "entropy": 1.0129335343837738,
+      "epoch": 0.02976190476190476,
+      "grad_norm": 0.35273969173431396,
+      "learning_rate": 9.706632653061225e-05,
+      "loss": 1.0496091842651367,
+      "mean_token_accuracy": 0.7338723540306091,
+      "num_tokens": 73767.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.9904452860355377,
+      "epoch": 0.034013605442176874,
+      "grad_norm": 0.20354461669921875,
+      "learning_rate": 9.664115646258503e-05,
+      "loss": 0.9870148658752441,
+      "mean_token_accuracy": 0.7565846800804138,
+      "num_tokens": 85087.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.9738265931606293,
+      "epoch": 0.03826530612244898,
+      "grad_norm": 0.45790979266166687,
+      "learning_rate": 9.621598639455783e-05,
+      "loss": 0.9613595008850098,
+      "mean_token_accuracy": 0.7610337018966675,
+      "num_tokens": 95022.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.9440798819065094,
+      "epoch": 0.04251700680272109,
+      "grad_norm": 0.35626640915870667,
+      "learning_rate": 9.579081632653061e-05,
+      "loss": 0.9162805557250977,
+      "mean_token_accuracy": 0.7618738412857056,
+      "num_tokens": 106825.0,
+      "step": 100
+    },
+    {
+      "entropy": 0.9322093605995179,
+      "epoch": 0.0467687074829932,
+      "grad_norm": 0.41860198974609375,
+      "learning_rate": 9.536564625850341e-05,
+      "loss": 0.9208142280578613,
+      "mean_token_accuracy": 0.7668713092803955,
+      "num_tokens": 117627.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.8814590692520141,
+      "epoch": 0.05102040816326531,
+      "grad_norm": 0.5872869491577148,
+      "learning_rate": 9.494047619047619e-05,
+      "loss": 0.8783014297485352,
+      "mean_token_accuracy": 0.7709871232509613,
+      "num_tokens": 129618.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.86140576004982,
+      "epoch": 0.055272108843537414,
+      "grad_norm": 0.4151027798652649,
+      "learning_rate": 9.451530612244899e-05,
+      "loss": 0.8385090827941895,
+      "mean_token_accuracy": 0.7878557801246643,
+      "num_tokens": 140476.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.7823852598667145,
+      "epoch": 0.05952380952380952,
+      "grad_norm": 0.6097142696380615,
+      "learning_rate": 9.409013605442177e-05,
+      "loss": 0.7834642887115478,
+      "mean_token_accuracy": 0.7922966778278351,
+      "num_tokens": 152252.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.8885609090328217,
+      "epoch": 0.06377551020408163,
+      "grad_norm": 0.3831205368041992,
+      "learning_rate": 9.366496598639456e-05,
+      "loss": 0.9057139396667481,
+      "mean_token_accuracy": 0.7586827576160431,
+      "num_tokens": 162998.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.8506790935993195,
+      "epoch": 0.06802721088435375,
+      "grad_norm": 0.4612574279308319,
+      "learning_rate": 9.323979591836735e-05,
+      "loss": 0.8219076156616211,
+      "mean_token_accuracy": 0.7892741382122039,
+      "num_tokens": 173248.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.85696662068367,
+      "epoch": 0.07227891156462585,
+      "grad_norm": 0.6407354474067688,
+      "learning_rate": 9.281462585034014e-05,
+      "loss": 0.8771568298339844,
+      "mean_token_accuracy": 0.7746504783630371,
+      "num_tokens": 183540.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.8503131151199341,
+      "epoch": 0.07653061224489796,
+      "grad_norm": 0.4775673449039459,
+      "learning_rate": 9.238945578231293e-05,
+      "loss": 0.7829782962799072,
+      "mean_token_accuracy": 0.7903713524341583,
+      "num_tokens": 194744.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.7273110747337341,
+      "epoch": 0.08078231292517007,
+      "grad_norm": 0.28129827976226807,
+      "learning_rate": 9.196428571428572e-05,
+      "loss": 0.7294204235076904,
+      "mean_token_accuracy": 0.8032773494720459,
+      "num_tokens": 207174.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.7966374695301056,
+      "epoch": 0.08503401360544217,
+      "grad_norm": 0.4943501651287079,
+      "learning_rate": 9.153911564625851e-05,
+      "loss": 0.8117904663085938,
+      "mean_token_accuracy": 0.7718977212905884,
+      "num_tokens": 217341.0,
+      "step": 200
+    },
+    {
+      "entropy": 0.7621297895908355,
+      "epoch": 0.08928571428571429,
+      "grad_norm": 0.7795273661613464,
+      "learning_rate": 9.11139455782313e-05,
+      "loss": 0.7390175819396972,
+      "mean_token_accuracy": 0.7991273105144501,
+      "num_tokens": 227481.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.7846836626529694,
+      "epoch": 0.0935374149659864,
+      "grad_norm": 0.5696049928665161,
+      "learning_rate": 9.068877551020408e-05,
+      "loss": 0.7595390796661377,
+      "mean_token_accuracy": 0.788711428642273,
+      "num_tokens": 237612.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.7737141489982605,
+      "epoch": 0.0977891156462585,
+      "grad_norm": 0.5130123496055603,
+      "learning_rate": 9.026360544217687e-05,
+      "loss": 0.7576333522796631,
+      "mean_token_accuracy": 0.7758967220783234,
+      "num_tokens": 248218.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.6375986337661743,
+      "epoch": 0.10204081632653061,
+      "grad_norm": 0.8193577527999878,
+      "learning_rate": 8.983843537414967e-05,
+      "loss": 0.6211213111877442,
+      "mean_token_accuracy": 0.8208125472068787,
+      "num_tokens": 258441.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.6664006829261779,
+      "epoch": 0.10629251700680271,
+      "grad_norm": 0.6085699796676636,
+      "learning_rate": 8.941326530612245e-05,
+      "loss": 0.694005823135376,
+      "mean_token_accuracy": 0.8076555073261261,
+      "num_tokens": 268684.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7731355726718903,
+      "epoch": 0.11054421768707483,
+      "grad_norm": 0.669662356376648,
+      "learning_rate": 8.898809523809524e-05,
+      "loss": 0.7673550605773926,
+      "mean_token_accuracy": 0.7881227731704712,
+      "num_tokens": 279434.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.6918620884418487,
+      "epoch": 0.11479591836734694,
+      "grad_norm": 0.41331204771995544,
+      "learning_rate": 8.856292517006803e-05,
+      "loss": 0.6603828430175781,
+      "mean_token_accuracy": 0.7999509751796723,
+      "num_tokens": 292081.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.6284413754940033,
+      "epoch": 0.11904761904761904,
+      "grad_norm": 0.5364882946014404,
+      "learning_rate": 8.813775510204082e-05,
+      "loss": 0.6319595813751221,
+      "mean_token_accuracy": 0.8188855946063995,
+      "num_tokens": 302835.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.7235157012939453,
+      "epoch": 0.12329931972789115,
+      "grad_norm": 0.6309073567390442,
+      "learning_rate": 8.77125850340136e-05,
+      "loss": 0.7478713512420654,
+      "mean_token_accuracy": 0.7892998337745667,
+      "num_tokens": 312703.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.6613646924495697,
+      "epoch": 0.12755102040816327,
+      "grad_norm": 0.6847456693649292,
+      "learning_rate": 8.72874149659864e-05,
+      "loss": 0.6457613945007324,
+      "mean_token_accuracy": 0.8177429616451264,
+      "num_tokens": 322433.0,
+      "step": 300
+    },
+    {
+      "entropy": 0.6452407896518707,
+      "epoch": 0.13180272108843538,
+      "grad_norm": 0.846867561340332,
+      "learning_rate": 8.68622448979592e-05,
+      "loss": 0.646483039855957,
+      "mean_token_accuracy": 0.8097379863262176,
+      "num_tokens": 333425.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.6624458074569702,
+      "epoch": 0.1360544217687075,
+      "grad_norm": 0.5457995533943176,
+      "learning_rate": 8.643707482993198e-05,
+      "loss": 0.6448498249053956,
+      "mean_token_accuracy": 0.8157779753208161,
+      "num_tokens": 345275.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.653342092037201,
+      "epoch": 0.14030612244897958,
+      "grad_norm": 0.5937319397926331,
+      "learning_rate": 8.601190476190477e-05,
+      "loss": 0.657109546661377,
+      "mean_token_accuracy": 0.7984300255775452,
+      "num_tokens": 355488.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.7131676137447357,
+      "epoch": 0.1445578231292517,
+      "grad_norm": 0.8101398944854736,
+      "learning_rate": 8.558673469387756e-05,
+      "loss": 0.7153036594390869,
+      "mean_token_accuracy": 0.7985480785369873,
+      "num_tokens": 366696.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.708580482006073,
+      "epoch": 0.1488095238095238,
+      "grad_norm": 0.6513782143592834,
+      "learning_rate": 8.516156462585034e-05,
+      "loss": 0.7102705001831054,
+      "mean_token_accuracy": 0.8050340950489044,
+      "num_tokens": 378108.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.6968572676181793,
+      "epoch": 0.15306122448979592,
+      "grad_norm": 0.9672437310218811,
+      "learning_rate": 8.473639455782312e-05,
+      "loss": 0.6984944820404053,
+      "mean_token_accuracy": 0.8096450090408325,
+      "num_tokens": 387247.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.6053666353225708,
+      "epoch": 0.15731292517006804,
+      "grad_norm": 0.7467218637466431,
+      "learning_rate": 8.431122448979592e-05,
+      "loss": 0.6112328052520752,
+      "mean_token_accuracy": 0.8201241433620453,
+      "num_tokens": 398723.0,
+      "step": 370
+    },
+    {
+      "entropy": 0.6494611740112305,
+      "epoch": 0.16156462585034015,
+      "grad_norm": 0.7286034822463989,
+      "learning_rate": 8.388605442176872e-05,
+      "loss": 0.6477136135101318,
+      "mean_token_accuracy": 0.8122058808803558,
+      "num_tokens": 409886.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.6897552013397217,
+      "epoch": 0.16581632653061223,
+      "grad_norm": 0.6391761898994446,
+      "learning_rate": 8.34608843537415e-05,
+      "loss": 0.6935544013977051,
+      "mean_token_accuracy": 0.7966985881328583,
+      "num_tokens": 422738.0,
+      "step": 390
+    },
+    {
+      "entropy": 0.6955088675022125,
+      "epoch": 0.17006802721088435,
+      "grad_norm": 0.5648608803749084,
+      "learning_rate": 8.30357142857143e-05,
+      "loss": 0.715025520324707,
+      "mean_token_accuracy": 0.7990479111671448,
+      "num_tokens": 431499.0,
+      "step": 400
+    },
+    {
+      "entropy": 0.657172217965126,
+      "epoch": 0.17431972789115646,
+      "grad_norm": 0.5713154077529907,
+      "learning_rate": 8.261054421768708e-05,
+      "loss": 0.6249523162841797,
+      "mean_token_accuracy": 0.8170171797275543,
+      "num_tokens": 441604.0,
+      "step": 410
+    },
+    {
+      "entropy": 0.614434689283371,
+      "epoch": 0.17857142857142858,
+      "grad_norm": 0.5609672665596008,
+      "learning_rate": 8.218537414965986e-05,
+      "loss": 0.6232338428497315,
+      "mean_token_accuracy": 0.8169759333133697,
+      "num_tokens": 452819.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.6596544206142425,
+      "epoch": 0.1828231292517007,
+      "grad_norm": 0.8918700218200684,
+      "learning_rate": 8.176020408163265e-05,
+      "loss": 0.6472682952880859,
+      "mean_token_accuracy": 0.8114864528179169,
+      "num_tokens": 463985.0,
+      "step": 430
+    },
+    {
+      "entropy": 0.5568734735250473,
+      "epoch": 0.1870748299319728,
+      "grad_norm": 0.6551166772842407,
+      "learning_rate": 8.133503401360544e-05,
+      "loss": 0.5621543407440186,
+      "mean_token_accuracy": 0.8313680768013001,
+      "num_tokens": 473644.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.6327908158302307,
+      "epoch": 0.1913265306122449,
+      "grad_norm": 0.7210569977760315,
+      "learning_rate": 8.090986394557824e-05,
+      "loss": 0.6351128101348877,
+      "mean_token_accuracy": 0.8192271828651428,
+      "num_tokens": 483952.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6680390477180481,
+      "epoch": 0.195578231292517,
+      "grad_norm": 0.8066325783729553,
+      "learning_rate": 8.048469387755102e-05,
+      "loss": 0.6428803920745849,
+      "mean_token_accuracy": 0.8096719264984131,
+      "num_tokens": 495728.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.5830813407897949,
+      "epoch": 0.19982993197278912,
+      "grad_norm": 0.6207858324050903,
+      "learning_rate": 8.005952380952382e-05,
+      "loss": 0.5903921604156495,
+      "mean_token_accuracy": 0.832057636976242,
+      "num_tokens": 506225.0,
+      "step": 470
+    },
+    {
+      "entropy": 0.6163366287946701,
+      "epoch": 0.20408163265306123,
+      "grad_norm": 0.27546176314353943,
+      "learning_rate": 7.96343537414966e-05,
+      "loss": 0.6205612182617187,
+      "mean_token_accuracy": 0.8090631544589997,
+      "num_tokens": 518289.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.5763369917869567,
+      "epoch": 0.20833333333333334,
+      "grad_norm": 0.5931833982467651,
+      "learning_rate": 7.920918367346939e-05,
+      "loss": 0.5674782752990722,
+      "mean_token_accuracy": 0.8279130697250366,
+      "num_tokens": 531099.0,
+      "step": 490
+    },
+    {
+      "entropy": 0.6561572968959808,
+      "epoch": 0.21258503401360543,
+      "grad_norm": 0.7970412373542786,
+      "learning_rate": 7.878401360544218e-05,
+      "loss": 0.6701024055480957,
+      "mean_token_accuracy": 0.8162985026836396,
+      "num_tokens": 539953.0,
+      "step": 500
+    },
+    {
+      "entropy": 0.5853851974010468,
+      "epoch": 0.21683673469387754,
+      "grad_norm": 0.74663907289505,
+      "learning_rate": 7.835884353741498e-05,
+      "loss": 0.5840775966644287,
+      "mean_token_accuracy": 0.830883127450943,
+      "num_tokens": 550283.0,
+      "step": 510
+    },
+    {
+      "entropy": 0.6775606602430344,
+      "epoch": 0.22108843537414966,
+      "grad_norm": 0.5372825860977173,
+      "learning_rate": 7.793367346938776e-05,
+      "loss": 0.6854439735412597,
+      "mean_token_accuracy": 0.7958937227725983,
+      "num_tokens": 561951.0,
+      "step": 520
+    },
+    {
+      "entropy": 0.6704744577407837,
+      "epoch": 0.22534013605442177,
+      "grad_norm": 0.7710736393928528,
+      "learning_rate": 7.750850340136054e-05,
+      "loss": 0.6726304054260254,
+      "mean_token_accuracy": 0.8020202696323395,
+      "num_tokens": 572509.0,
+      "step": 530
+    },
+    {
+      "entropy": 0.635757040977478,
+      "epoch": 0.22959183673469388,
+      "grad_norm": 0.620157778263092,
+      "learning_rate": 7.708333333333334e-05,
+      "loss": 0.6136677742004395,
+      "mean_token_accuracy": 0.8304697453975678,
+      "num_tokens": 583522.0,
+      "step": 540
+    },
+    {
+      "entropy": 0.6270190805196763,
+      "epoch": 0.233843537414966,
+      "grad_norm": 0.3838287889957428,
+      "learning_rate": 7.665816326530612e-05,
+      "loss": 0.6416101455688477,
+      "mean_token_accuracy": 0.8076549887657165,
+      "num_tokens": 595086.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6419758051633835,
+      "epoch": 0.23809523809523808,
+      "grad_norm": 0.6037173867225647,
+      "learning_rate": 7.623299319727891e-05,
+      "loss": 0.6315017223358155,
+      "mean_token_accuracy": 0.8098805785179138,
+      "num_tokens": 606510.0,
+      "step": 560
+    },
+    {
+      "entropy": 0.5769951343536377,
+      "epoch": 0.2423469387755102,
+      "grad_norm": 1.013440489768982,
+      "learning_rate": 7.58078231292517e-05,
+      "loss": 0.5704180717468261,
+      "mean_token_accuracy": 0.8303274273872375,
+      "num_tokens": 617019.0,
+      "step": 570
+    },
+    {
+      "entropy": 0.6124611109495163,
+      "epoch": 0.2465986394557823,
+      "grad_norm": 0.7080560326576233,
+      "learning_rate": 7.53826530612245e-05,
+      "loss": 0.621029281616211,
+      "mean_token_accuracy": 0.8226043820381165,
+      "num_tokens": 627658.0,
+      "step": 580
+    },
+    {
+      "entropy": 0.6244533747434616,
+      "epoch": 0.2508503401360544,
+      "grad_norm": 0.7130760550498962,
+      "learning_rate": 7.495748299319728e-05,
+      "loss": 0.6250315189361573,
+      "mean_token_accuracy": 0.812971156835556,
+      "num_tokens": 636966.0,
+      "step": 590
+    },
+    {
+      "entropy": 0.61286960542202,
+      "epoch": 0.25510204081632654,
+      "grad_norm": 0.5573196411132812,
+      "learning_rate": 7.453231292517007e-05,
+      "loss": 0.6197751045227051,
+      "mean_token_accuracy": 0.8135783731937408,
+      "num_tokens": 648475.0,
+      "step": 600
+    },
+    {
+      "entropy": 0.6105983853340149,
+      "epoch": 0.2593537414965986,
+      "grad_norm": 0.8304420113563538,
+      "learning_rate": 7.410714285714286e-05,
+      "loss": 0.6049539566040039,
+      "mean_token_accuracy": 0.8161618828773498,
+      "num_tokens": 660048.0,
+      "step": 610
+    },
+    {
+      "entropy": 0.6327467203140259,
+      "epoch": 0.26360544217687076,
+      "grad_norm": 0.6550470590591431,
+      "learning_rate": 7.368197278911565e-05,
+      "loss": 0.6316221237182618,
+      "mean_token_accuracy": 0.8131963431835174,
+      "num_tokens": 671505.0,
+      "step": 620
+    },
+    {
+      "entropy": 0.6295043647289276,
+      "epoch": 0.26785714285714285,
+      "grad_norm": 0.6002435088157654,
+      "learning_rate": 7.325680272108843e-05,
+      "loss": 0.6159311771392822,
+      "mean_token_accuracy": 0.8182806372642517,
+      "num_tokens": 681911.0,
+      "step": 630
+    },
+    {
+      "entropy": 0.5830861747264862,
+      "epoch": 0.272108843537415,
+      "grad_norm": 0.7070937752723694,
+      "learning_rate": 7.283163265306123e-05,
+      "loss": 0.5752198219299316,
+      "mean_token_accuracy": 0.8236642122268677,
+      "num_tokens": 691175.0,
+      "step": 640
+    },
+    {
+      "entropy": 0.6063521772623062,
+      "epoch": 0.2763605442176871,
+      "grad_norm": 0.70107501745224,
+      "learning_rate": 7.240646258503402e-05,
+      "loss": 0.6175377845764161,
+      "mean_token_accuracy": 0.8269807457923889,
+      "num_tokens": 701381.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6279960155487061,
+      "epoch": 0.28061224489795916,
+      "grad_norm": 0.7800400257110596,
+      "learning_rate": 7.19812925170068e-05,
+      "loss": 0.6200803756713867,
+      "mean_token_accuracy": 0.8243706822395325,
+      "num_tokens": 711160.0,
+      "step": 660
+    },
+    {
+      "entropy": 0.6208126276731492,
+      "epoch": 0.2848639455782313,
+      "grad_norm": 0.5167033076286316,
+      "learning_rate": 7.155612244897959e-05,
+      "loss": 0.6058928012847901,
+      "mean_token_accuracy": 0.825822365283966,
+      "num_tokens": 721555.0,
+      "step": 670
+    },
+    {
+      "entropy": 0.609162974357605,
+      "epoch": 0.2891156462585034,
+      "grad_norm": 0.8961696624755859,
+      "learning_rate": 7.113095238095239e-05,
+      "loss": 0.6239143371582031,
+      "mean_token_accuracy": 0.8218865990638733,
+      "num_tokens": 731723.0,
+      "step": 680
+    },
+    {
+      "entropy": 0.6567182302474975,
+      "epoch": 0.29336734693877553,
+      "grad_norm": 0.3495732247829437,
+      "learning_rate": 7.070578231292517e-05,
+      "loss": 0.6583582878112793,
+      "mean_token_accuracy": 0.8035318732261658,
+      "num_tokens": 745338.0,
+      "step": 690
+    },
+    {
+      "entropy": 0.5773716390132904,
+      "epoch": 0.2976190476190476,
+      "grad_norm": 0.7330310344696045,
+      "learning_rate": 7.028061224489795e-05,
+      "loss": 0.5598068237304688,
+      "mean_token_accuracy": 0.8329594731330872,
+      "num_tokens": 755761.0,
+      "step": 700
+    },
+    {
+      "entropy": 0.6005543798208237,
+      "epoch": 0.3018707482993197,
+      "grad_norm": 1.0653876066207886,
+      "learning_rate": 6.985544217687075e-05,
+      "loss": 0.6206864356994629,
+      "mean_token_accuracy": 0.8177128791809082,
+      "num_tokens": 765987.0,
+      "step": 710
+    },
+    {
+      "entropy": 0.62425377368927,
+      "epoch": 0.30612244897959184,
+      "grad_norm": 0.8440945148468018,
+      "learning_rate": 6.943027210884355e-05,
+      "loss": 0.6362304210662841,
+      "mean_token_accuracy": 0.8133590519428253,
+      "num_tokens": 776707.0,
+      "step": 720
+    },
+    {
+      "entropy": 0.6137575387954712,
+      "epoch": 0.31037414965986393,
+      "grad_norm": 0.8290141224861145,
+      "learning_rate": 6.900510204081633e-05,
+      "loss": 0.602045726776123,
+      "mean_token_accuracy": 0.8269721269607544,
+      "num_tokens": 785808.0,
+      "step": 730
+    },
+    {
+      "entropy": 0.6052783876657486,
+      "epoch": 0.31462585034013607,
+      "grad_norm": 0.6991459131240845,
+      "learning_rate": 6.857993197278913e-05,
+      "loss": 0.6025216579437256,
+      "mean_token_accuracy": 0.8308734178543091,
+      "num_tokens": 795902.0,
+      "step": 740
+    },
+    {
+      "entropy": 0.6147994667291641,
+      "epoch": 0.31887755102040816,
+      "grad_norm": 0.7895340323448181,
+      "learning_rate": 6.815476190476191e-05,
+      "loss": 0.6181727409362793,
+      "mean_token_accuracy": 0.8263129591941833,
+      "num_tokens": 805319.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6230242431163788,
+      "epoch": 0.3231292517006803,
+      "grad_norm": 0.7686090469360352,
+      "learning_rate": 6.772959183673469e-05,
+      "loss": 0.620832872390747,
+      "mean_token_accuracy": 0.809659230709076,
+      "num_tokens": 817027.0,
+      "step": 760
+    },
+    {
+      "entropy": 0.5874412208795547,
+      "epoch": 0.3273809523809524,
+      "grad_norm": 0.9315619468688965,
+      "learning_rate": 6.730442176870749e-05,
+      "loss": 0.5822995185852051,
+      "mean_token_accuracy": 0.8226828098297119,
+      "num_tokens": 830032.0,
+      "step": 770
+    },
+    {
+      "entropy": 0.5659277379512787,
+      "epoch": 0.33163265306122447,
+      "grad_norm": 0.8107892870903015,
+      "learning_rate": 6.687925170068029e-05,
+      "loss": 0.5684999465942383,
+      "mean_token_accuracy": 0.8392240762710571,
+      "num_tokens": 840386.0,
+      "step": 780
+    },
+    {
+      "entropy": 0.5930269837379456,
+      "epoch": 0.3358843537414966,
+      "grad_norm": 0.7134554386138916,
+      "learning_rate": 6.645408163265307e-05,
+      "loss": 0.5747966766357422,
+      "mean_token_accuracy": 0.827442079782486,
+      "num_tokens": 851076.0,
+      "step": 790
+    },
+    {
+      "entropy": 0.6496313869953155,
+      "epoch": 0.3401360544217687,
+      "grad_norm": 0.87777179479599,
+      "learning_rate": 6.602891156462585e-05,
+      "loss": 0.6809831142425538,
+      "mean_token_accuracy": 0.785821259021759,
+      "num_tokens": 862678.0,
+      "step": 800
+    },
+    {
+      "entropy": 0.5907361328601837,
+      "epoch": 0.34438775510204084,
+      "grad_norm": 0.711604118347168,
+      "learning_rate": 6.560374149659865e-05,
+      "loss": 0.5807193756103516,
+      "mean_token_accuracy": 0.8352659344673157,
+      "num_tokens": 873136.0,
+      "step": 810
+    },
+    {
+      "entropy": 0.5893878310918808,
+      "epoch": 0.3486394557823129,
+      "grad_norm": 0.6100995540618896,
+      "learning_rate": 6.517857142857143e-05,
+      "loss": 0.5770053863525391,
+      "mean_token_accuracy": 0.8248592674732208,
+      "num_tokens": 884688.0,
+      "step": 820
+    },
+    {
+      "entropy": 0.5606941819190979,
+      "epoch": 0.352891156462585,
+      "grad_norm": 0.6781875491142273,
+      "learning_rate": 6.475340136054421e-05,
+      "loss": 0.5801912784576416,
+      "mean_token_accuracy": 0.8323125898838043,
+      "num_tokens": 896185.0,
+      "step": 830
+    },
+    {
+      "entropy": 0.5933171272277832,
+      "epoch": 0.35714285714285715,
+      "grad_norm": 0.5787054300308228,
+      "learning_rate": 6.432823129251701e-05,
+      "loss": 0.5779991149902344,
+      "mean_token_accuracy": 0.8351444482803345,
+      "num_tokens": 906197.0,
+      "step": 840
+    },
+    {
+      "entropy": 0.5471704989671707,
+      "epoch": 0.36139455782312924,
+      "grad_norm": 1.0463767051696777,
+      "learning_rate": 6.390306122448981e-05,
+      "loss": 0.5410798549652099,
+      "mean_token_accuracy": 0.8452448129653931,
+      "num_tokens": 915927.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.537025648355484,
+      "epoch": 0.3656462585034014,
+      "grad_norm": 0.725310742855072,
+      "learning_rate": 6.347789115646259e-05,
+      "loss": 0.5336166381835937,
+      "mean_token_accuracy": 0.8406780958175659,
+      "num_tokens": 925972.0,
+      "step": 860
+    },
+    {
+      "entropy": 0.6086225181818008,
+      "epoch": 0.36989795918367346,
+      "grad_norm": 0.6319683194160461,
+      "learning_rate": 6.305272108843537e-05,
+      "loss": 0.6223300933837891,
+      "mean_token_accuracy": 0.8144682884216309,
+      "num_tokens": 938614.0,
+      "step": 870
+    },
+    {
+      "entropy": 0.6386535853147507,
+      "epoch": 0.3741496598639456,
+      "grad_norm": 0.9547185897827148,
+      "learning_rate": 6.262755102040817e-05,
+      "loss": 0.636644697189331,
+      "mean_token_accuracy": 0.8114496529102325,
+      "num_tokens": 951270.0,
+      "step": 880
+    },
+    {
+      "entropy": 0.5482926398515702,
+      "epoch": 0.3784013605442177,
+      "grad_norm": 0.8478467464447021,
+      "learning_rate": 6.220238095238095e-05,
+      "loss": 0.5229831218719483,
+      "mean_token_accuracy": 0.8450385332107544,
+      "num_tokens": 962007.0,
+      "step": 890
+    },
+    {
+      "entropy": 0.5352211892604828,
+      "epoch": 0.3826530612244898,
+      "grad_norm": 1.1555845737457275,
+      "learning_rate": 6.177721088435374e-05,
+      "loss": 0.5529211044311524,
+      "mean_token_accuracy": 0.8393910169601441,
+      "num_tokens": 971221.0,
+      "step": 900
+    },
+    {
+      "entropy": 0.6504414319992066,
+      "epoch": 0.3869047619047619,
+      "grad_norm": 0.6787072420120239,
+      "learning_rate": 6.135204081632653e-05,
+      "loss": 0.6610816478729248,
+      "mean_token_accuracy": 0.7907391488552094,
+      "num_tokens": 982564.0,
+      "step": 910
+    },
+    {
+      "entropy": 0.5921294629573822,
+      "epoch": 0.391156462585034,
+      "grad_norm": 0.7555624842643738,
+      "learning_rate": 6.092687074829932e-05,
+      "loss": 0.5777195930480957,
+      "mean_token_accuracy": 0.8236903607845306,
+      "num_tokens": 992309.0,
+      "step": 920
+    },
+    {
+      "entropy": 0.578193974494934,
+      "epoch": 0.39540816326530615,
+      "grad_norm": 0.5832639336585999,
+      "learning_rate": 6.050170068027211e-05,
+      "loss": 0.5782378196716309,
+      "mean_token_accuracy": 0.825832200050354,
+      "num_tokens": 1004082.0,
+      "step": 930
+    },
+    {
+      "entropy": 0.5312168657779693,
+      "epoch": 0.39965986394557823,
+      "grad_norm": 0.6483455300331116,
+      "learning_rate": 6.0076530612244896e-05,
+      "loss": 0.5226385593414307,
+      "mean_token_accuracy": 0.837943822145462,
+      "num_tokens": 1014018.0,
+      "step": 940
+    },
+    {
+      "entropy": 0.5291696071624756,
+      "epoch": 0.4039115646258503,
+      "grad_norm": 0.8548340797424316,
+      "learning_rate": 5.965136054421769e-05,
+      "loss": 0.5349358558654785,
+      "mean_token_accuracy": 0.8440997183322907,
+      "num_tokens": 1024546.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.5704925835132599,
+      "epoch": 0.40816326530612246,
+      "grad_norm": 0.7204116582870483,
+      "learning_rate": 5.922619047619048e-05,
+      "loss": 0.5723382472991944,
+      "mean_token_accuracy": 0.8350829958915711,
+      "num_tokens": 1035324.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.5551350146532059,
+      "epoch": 0.41241496598639454,
+      "grad_norm": 0.8478047251701355,
+      "learning_rate": 5.8801020408163266e-05,
+      "loss": 0.5603343963623046,
+      "mean_token_accuracy": 0.8310107290744781,
+      "num_tokens": 1045459.0,
+      "step": 970
+    },
+    {
+      "entropy": 0.5924204260110855,
+      "epoch": 0.4166666666666667,
+      "grad_norm": 1.0831539630889893,
+      "learning_rate": 5.837585034013606e-05,
+      "loss": 0.5897612571716309,
+      "mean_token_accuracy": 0.8289736449718476,
+      "num_tokens": 1054399.0,
+      "step": 980
+    },
+    {
+      "entropy": 0.6486367821693421,
+      "epoch": 0.42091836734693877,
+      "grad_norm": 0.7564862966537476,
+      "learning_rate": 5.7950680272108846e-05,
+      "loss": 0.6640692234039307,
+      "mean_token_accuracy": 0.8002952694892883,
+      "num_tokens": 1068567.0,
+      "step": 990
+    },
+    {
+      "entropy": 0.5693342804908752,
+      "epoch": 0.42517006802721086,
+      "grad_norm": 0.9830188751220703,
+      "learning_rate": 5.7525510204081636e-05,
+      "loss": 0.5391183853149414,
+      "mean_token_accuracy": 0.8396909654140472,
+      "num_tokens": 1079595.0,
+      "step": 1000
+    },
+    {
+      "entropy": 0.6238065242767334,
+      "epoch": 0.429421768707483,
+      "grad_norm": 0.8739970326423645,
+      "learning_rate": 5.710034013605442e-05,
+      "loss": 0.6429170608520508,
+      "mean_token_accuracy": 0.816579920053482,
+      "num_tokens": 1090254.0,
+      "step": 1010
+    },
+    {
+      "entropy": 0.650809234380722,
+      "epoch": 0.4336734693877551,
+      "grad_norm": 0.6658247113227844,
+      "learning_rate": 5.6675170068027215e-05,
+      "loss": 0.633099365234375,
+      "mean_token_accuracy": 0.8052545428276062,
+      "num_tokens": 1100320.0,
+      "step": 1020
+    },
+    {
+      "entropy": 0.6291825830936432,
+      "epoch": 0.4379251700680272,
+      "grad_norm": 0.7465150356292725,
+      "learning_rate": 5.6250000000000005e-05,
+      "loss": 0.6484538078308105,
+      "mean_token_accuracy": 0.8210124790668487,
+      "num_tokens": 1109938.0,
+      "step": 1030
+    },
+    {
+      "entropy": 0.6182401776313782,
+      "epoch": 0.4421768707482993,
+      "grad_norm": 0.720923125743866,
+      "learning_rate": 5.582482993197279e-05,
+      "loss": 0.6223091602325439,
+      "mean_token_accuracy": 0.8170465350151062,
+      "num_tokens": 1121480.0,
+      "step": 1040
+    },
+    {
+      "entropy": 0.6288217008113861,
+      "epoch": 0.44642857142857145,
+      "grad_norm": 0.8048785328865051,
+      "learning_rate": 5.5399659863945585e-05,
+      "loss": 0.6142692565917969,
+      "mean_token_accuracy": 0.8101728081703186,
+      "num_tokens": 1134615.0,
+      "step": 1050
+    },
+    {
+      "entropy": 0.5737671077251434,
+      "epoch": 0.45068027210884354,
+      "grad_norm": 1.0010799169540405,
+      "learning_rate": 5.497448979591837e-05,
+      "loss": 0.578005313873291,
+      "mean_token_accuracy": 0.8241186082363129,
+      "num_tokens": 1148059.0,
+      "step": 1060
+    },
+    {
+      "entropy": 0.6049871623516083,
+      "epoch": 0.4549319727891156,
+      "grad_norm": 0.7644203901290894,
+      "learning_rate": 5.454931972789116e-05,
+      "loss": 0.6036805152893067,
+      "mean_token_accuracy": 0.8251873433589936,
+      "num_tokens": 1159077.0,
+      "step": 1070
+    },
+    {
+      "entropy": 0.5016378700733185,
+      "epoch": 0.45918367346938777,
+      "grad_norm": 0.8055089116096497,
+      "learning_rate": 5.412414965986394e-05,
+      "loss": 0.47589454650878904,
+      "mean_token_accuracy": 0.852514261007309,
+      "num_tokens": 1170338.0,
+      "step": 1080
+    },
+    {
+      "entropy": 0.5905508637428284,
+      "epoch": 0.46343537414965985,
+      "grad_norm": 0.8147892355918884,
+      "learning_rate": 5.369897959183674e-05,
+      "loss": 0.6135655403137207,
+      "mean_token_accuracy": 0.8159750401973724,
+      "num_tokens": 1181967.0,
+      "step": 1090
+    },
+    {
+      "entropy": 0.5504540711641311,
+      "epoch": 0.467687074829932,
+      "grad_norm": 0.7014718055725098,
+      "learning_rate": 5.327380952380953e-05,
+      "loss": 0.5393699169158935,
+      "mean_token_accuracy": 0.8400495707988739,
+      "num_tokens": 1191911.0,
+      "step": 1100
+    },
+    {
+      "entropy": 0.5911126136779785,
+      "epoch": 0.4719387755102041,
+      "grad_norm": 0.8102626800537109,
+      "learning_rate": 5.284863945578231e-05,
+      "loss": 0.5901049137115478,
+      "mean_token_accuracy": 0.8300528645515441,
+      "num_tokens": 1201620.0,
+      "step": 1110
+    },
+    {
+      "entropy": 0.5760516464710236,
+      "epoch": 0.47619047619047616,
+      "grad_norm": 0.5337172150611877,
+      "learning_rate": 5.242346938775511e-05,
+      "loss": 0.6015921115875245,
+      "mean_token_accuracy": 0.8279711067676544,
+      "num_tokens": 1212976.0,
+      "step": 1120
+    },
+    {
+      "entropy": 0.6197247624397277,
+      "epoch": 0.4804421768707483,
+      "grad_norm": 0.6187336444854736,
+      "learning_rate": 5.19982993197279e-05,
+      "loss": 0.5980201244354248,
+      "mean_token_accuracy": 0.8173838019371032,
+      "num_tokens": 1224500.0,
+      "step": 1130
+    },
+    {
+      "entropy": 0.5247576057910919,
+      "epoch": 0.4846938775510204,
+      "grad_norm": 0.38675031065940857,
+      "learning_rate": 5.157312925170068e-05,
+      "loss": 0.5128117084503174,
+      "mean_token_accuracy": 0.8427697956562042,
+      "num_tokens": 1236056.0,
+      "step": 1140
+    },
+    {
+      "entropy": 0.5545101583003997,
+      "epoch": 0.48894557823129253,
+      "grad_norm": 0.8362757563591003,
+      "learning_rate": 5.114795918367348e-05,
+      "loss": 0.5881862163543701,
+      "mean_token_accuracy": 0.8283910691738129,
+      "num_tokens": 1246454.0,
+      "step": 1150
+    },
+    {
+      "entropy": 0.5925249218940735,
+      "epoch": 0.4931972789115646,
+      "grad_norm": 0.6821584105491638,
+      "learning_rate": 5.072278911564626e-05,
+      "loss": 0.5613045215606689,
+      "mean_token_accuracy": 0.8208597123622894,
+      "num_tokens": 1258943.0,
+      "step": 1160
+    },
+    {
+      "entropy": 0.5415159404277802,
+      "epoch": 0.49744897959183676,
+      "grad_norm": 0.6268453001976013,
+      "learning_rate": 5.029761904761905e-05,
+      "loss": 0.5331630706787109,
+      "mean_token_accuracy": 0.8364126265048981,
+      "num_tokens": 1272811.0,
+      "step": 1170
+    },
+    {
+      "entropy": 0.5170255899429321,
+      "epoch": 0.5017006802721088,
+      "grad_norm": 0.9418641924858093,
+      "learning_rate": 4.987244897959184e-05,
+      "loss": 0.5485562324523926,
+      "mean_token_accuracy": 0.8380054891109466,
+      "num_tokens": 1284235.0,
+      "step": 1180
+    },
+    {
+      "entropy": 0.5575446605682373,
+      "epoch": 0.5059523809523809,
+      "grad_norm": 0.6781197190284729,
+      "learning_rate": 4.944727891156462e-05,
+      "loss": 0.5431806564331054,
+      "mean_token_accuracy": 0.8374831795692443,
+      "num_tokens": 1293691.0,
+      "step": 1190
+    },
+    {
+      "entropy": 0.5571623414754867,
+      "epoch": 0.5102040816326531,
+      "grad_norm": 0.6811428666114807,
+      "learning_rate": 4.902210884353742e-05,
+      "loss": 0.571410083770752,
+      "mean_token_accuracy": 0.8248052895069122,
+      "num_tokens": 1304900.0,
+      "step": 1200
+    },
+    {
+      "entropy": 0.6219375789165497,
+      "epoch": 0.5144557823129252,
+      "grad_norm": 0.8971446752548218,
+      "learning_rate": 4.859693877551021e-05,
+      "loss": 0.6108813285827637,
+      "mean_token_accuracy": 0.8282350063323974,
+      "num_tokens": 1315658.0,
+      "step": 1210
+    },
+    {
+      "entropy": 0.5790914714336395,
+      "epoch": 0.5187074829931972,
+      "grad_norm": 0.8718211054801941,
+      "learning_rate": 4.817176870748299e-05,
+      "loss": 0.5702257633209229,
+      "mean_token_accuracy": 0.8363608837127685,
+      "num_tokens": 1325007.0,
+      "step": 1220
+    },
+    {
+      "entropy": 0.553312674164772,
+      "epoch": 0.5229591836734694,
+      "grad_norm": 0.6368089914321899,
+      "learning_rate": 4.774659863945579e-05,
+      "loss": 0.5468079566955566,
+      "mean_token_accuracy": 0.8384977638721466,
+      "num_tokens": 1334178.0,
+      "step": 1230
+    },
+    {
+      "entropy": 0.5654692828655243,
+      "epoch": 0.5272108843537415,
+      "grad_norm": 0.28180381655693054,
+      "learning_rate": 4.732142857142857e-05,
+      "loss": 0.5865198135375976,
+      "mean_token_accuracy": 0.8254834353923798,
+      "num_tokens": 1346386.0,
+      "step": 1240
+    },
+    {
+      "entropy": 0.605426836013794,
+      "epoch": 0.5314625850340136,
+      "grad_norm": 0.7490566372871399,
+      "learning_rate": 4.689625850340136e-05,
+      "loss": 0.5977043628692627,
+      "mean_token_accuracy": 0.8090501189231872,
+      "num_tokens": 1355782.0,
+      "step": 1250
+    },
+    {
+      "entropy": 0.6523470669984818,
+      "epoch": 0.5357142857142857,
+      "grad_norm": 0.8416953086853027,
+      "learning_rate": 4.647108843537415e-05,
+      "loss": 0.6364702224731446,
+      "mean_token_accuracy": 0.8065241694450378,
+      "num_tokens": 1367425.0,
+      "step": 1260
+    },
+    {
+      "entropy": 0.6016772747039795,
+      "epoch": 0.5399659863945578,
+      "grad_norm": 0.723196804523468,
+      "learning_rate": 4.604591836734694e-05,
+      "loss": 0.5914052486419678,
+      "mean_token_accuracy": 0.8149433314800263,
+      "num_tokens": 1379306.0,
+      "step": 1270
+    },
+    {
+      "entropy": 0.538858824968338,
+      "epoch": 0.54421768707483,
+      "grad_norm": 0.7268431186676025,
+      "learning_rate": 4.562074829931973e-05,
+      "loss": 0.5699212074279785,
+      "mean_token_accuracy": 0.8373402774333953,
+      "num_tokens": 1388943.0,
+      "step": 1280
+    },
+    {
+      "entropy": 0.5116531759500503,
+      "epoch": 0.548469387755102,
+      "grad_norm": 0.6881728172302246,
+      "learning_rate": 4.5195578231292515e-05,
+      "loss": 0.501812219619751,
+      "mean_token_accuracy": 0.8508045136928558,
+      "num_tokens": 1398890.0,
+      "step": 1290
+    },
+    {
+      "entropy": 0.5898961424827576,
+      "epoch": 0.5527210884353742,
+      "grad_norm": 0.9265963435173035,
+      "learning_rate": 4.477040816326531e-05,
+      "loss": 0.5903323650360107,
+      "mean_token_accuracy": 0.8182829022407532,
+      "num_tokens": 1412402.0,
+      "step": 1300
+    },
+    {
+      "entropy": 0.5762919217348099,
+      "epoch": 0.5569727891156463,
+      "grad_norm": 0.6862229108810425,
+      "learning_rate": 4.4345238095238095e-05,
+      "loss": 0.5722722053527832,
+      "mean_token_accuracy": 0.8374342918395996,
+      "num_tokens": 1425081.0,
+      "step": 1310
+    },
+    {
+      "entropy": 0.5345231264829635,
+      "epoch": 0.5612244897959183,
+      "grad_norm": 0.7506856322288513,
+      "learning_rate": 4.3920068027210885e-05,
+      "loss": 0.5166549205780029,
+      "mean_token_accuracy": 0.8448654294013977,
+      "num_tokens": 1436487.0,
+      "step": 1320
+    },
+    {
+      "entropy": 0.5727064400911331,
+      "epoch": 0.5654761904761905,
+      "grad_norm": 0.7576751708984375,
+      "learning_rate": 4.3494897959183675e-05,
+      "loss": 0.5965213298797607,
+      "mean_token_accuracy": 0.8292974472045899,
+      "num_tokens": 1447115.0,
+      "step": 1330
+    },
+    {
+      "entropy": 0.6086050748825074,
+      "epoch": 0.5697278911564626,
+      "grad_norm": 1.0944355726242065,
+      "learning_rate": 4.3069727891156465e-05,
+      "loss": 0.6146334648132324,
+      "mean_token_accuracy": 0.8208171963691712,
+      "num_tokens": 1458109.0,
+      "step": 1340
+    },
+    {
+      "entropy": 0.548018091917038,
+      "epoch": 0.5739795918367347,
+      "grad_norm": 0.7556106448173523,
+      "learning_rate": 4.2644557823129255e-05,
+      "loss": 0.5399428367614746,
+      "mean_token_accuracy": 0.8316729366779327,
+      "num_tokens": 1468353.0,
+      "step": 1350
+    },
+    {
+      "entropy": 0.57362260222435,
+      "epoch": 0.5782312925170068,
+      "grad_norm": 0.958890974521637,
+      "learning_rate": 4.2219387755102045e-05,
+      "loss": 0.5638460159301758,
+      "mean_token_accuracy": 0.8328832030296326,
+      "num_tokens": 1478064.0,
+      "step": 1360
+    },
+    {
+      "entropy": 0.4699955850839615,
+      "epoch": 0.5824829931972789,
+      "grad_norm": 0.951663613319397,
+      "learning_rate": 4.1794217687074834e-05,
+      "loss": 0.4790843963623047,
+      "mean_token_accuracy": 0.8547844409942627,
+      "num_tokens": 1488401.0,
+      "step": 1370
+    },
+    {
+      "entropy": 0.5664187341928482,
+      "epoch": 0.5867346938775511,
+      "grad_norm": 0.8573746085166931,
+      "learning_rate": 4.136904761904762e-05,
+      "loss": 0.6015422821044922,
+      "mean_token_accuracy": 0.8200101733207703,
+      "num_tokens": 1498757.0,
+      "step": 1380
+    },
+    {
+      "entropy": 0.5411120235919953,
+      "epoch": 0.5909863945578231,
+      "grad_norm": 0.738762617111206,
+      "learning_rate": 4.094387755102041e-05,
+      "loss": 0.49553709030151366,
+      "mean_token_accuracy": 0.8463812232017517,
+      "num_tokens": 1509252.0,
+      "step": 1390
+    },
+    {
+      "entropy": 0.5284088224172592,
+      "epoch": 0.5952380952380952,
+      "grad_norm": 0.9987853765487671,
+      "learning_rate": 4.0518707482993204e-05,
+      "loss": 0.521688985824585,
+      "mean_token_accuracy": 0.8317105531692505,
+      "num_tokens": 1521575.0,
+      "step": 1400
+    },
+    {
+      "entropy": 0.5420585960149765,
+      "epoch": 0.5994897959183674,
+      "grad_norm": 0.5594030022621155,
+      "learning_rate": 4.009353741496599e-05,
+      "loss": 0.5624300003051758,
+      "mean_token_accuracy": 0.8328603029251098,
+      "num_tokens": 1533858.0,
+      "step": 1410
+    },
+    {
+      "entropy": 0.6201832562685012,
+      "epoch": 0.6037414965986394,
+      "grad_norm": 0.9479369521141052,
+      "learning_rate": 3.966836734693878e-05,
+      "loss": 0.5940685749053956,
+      "mean_token_accuracy": 0.8293437898159027,
+      "num_tokens": 1544902.0,
+      "step": 1420
+    },
+    {
+      "entropy": 0.6230043023824692,
+      "epoch": 0.6079931972789115,
+      "grad_norm": 0.8622655868530273,
+      "learning_rate": 3.924319727891157e-05,
+      "loss": 0.631603193283081,
+      "mean_token_accuracy": 0.8104620337486267,
+      "num_tokens": 1555779.0,
+      "step": 1430
+    },
+    {
+      "entropy": 0.5547834306955337,
+      "epoch": 0.6122448979591837,
+      "grad_norm": 0.8286665678024292,
+      "learning_rate": 3.881802721088436e-05,
+      "loss": 0.555398416519165,
+      "mean_token_accuracy": 0.8266190946102142,
+      "num_tokens": 1568157.0,
+      "step": 1440
+    },
+    {
+      "entropy": 0.5555658012628555,
+      "epoch": 0.6164965986394558,
+      "grad_norm": 1.2064694166183472,
+      "learning_rate": 3.839285714285715e-05,
+      "loss": 0.5589458465576171,
+      "mean_token_accuracy": 0.8308910727500916,
+      "num_tokens": 1578660.0,
+      "step": 1450
+    },
+    {
+      "entropy": 0.5215276837348938,
+      "epoch": 0.6207482993197279,
+      "grad_norm": 1.134692668914795,
+      "learning_rate": 3.796768707482993e-05,
+      "loss": 0.5223335266113281,
+      "mean_token_accuracy": 0.8505008935928344,
+      "num_tokens": 1588221.0,
+      "step": 1460
+    },
+    {
+      "entropy": 0.5241445243358612,
+      "epoch": 0.625,
+      "grad_norm": 0.32119959592819214,
+      "learning_rate": 3.7542517006802727e-05,
+      "loss": 0.5141430377960206,
+      "mean_token_accuracy": 0.8423964560031891,
+      "num_tokens": 1600810.0,
+      "step": 1470
+    },
+    {
+      "entropy": 0.5927034109830857,
+      "epoch": 0.6292517006802721,
+      "grad_norm": 0.8377068042755127,
+      "learning_rate": 3.711734693877551e-05,
+      "loss": 0.5961884975433349,
+      "mean_token_accuracy": 0.8162056863307953,
+      "num_tokens": 1611543.0,
+      "step": 1480
+    },
+    {
+      "entropy": 0.5436273574829101,
+      "epoch": 0.6335034013605442,
+      "grad_norm": 1.032684087753296,
+      "learning_rate": 3.66921768707483e-05,
+      "loss": 0.534994649887085,
+      "mean_token_accuracy": 0.8348295450210571,
+      "num_tokens": 1623564.0,
+      "step": 1490
+    },
+    {
+      "entropy": 0.5652735084295273,
+      "epoch": 0.6377551020408163,
+      "grad_norm": 0.9198411107063293,
+      "learning_rate": 3.626700680272109e-05,
+      "loss": 0.56695556640625,
+      "mean_token_accuracy": 0.8294547438621521,
+      "num_tokens": 1634706.0,
+      "step": 1500
+    },
+    {
+      "entropy": 0.5435687988996506,
+      "epoch": 0.6420068027210885,
+      "grad_norm": 1.0693076848983765,
+      "learning_rate": 3.584183673469388e-05,
+      "loss": 0.5479542255401612,
+      "mean_token_accuracy": 0.831254106760025,
+      "num_tokens": 1646948.0,
+      "step": 1510
+    },
+    {
+      "entropy": 0.5947155982255936,
+      "epoch": 0.6462585034013606,
+      "grad_norm": 0.8154687285423279,
+      "learning_rate": 3.541666666666667e-05,
+      "loss": 0.5963615417480469,
+      "mean_token_accuracy": 0.8205921828746796,
+      "num_tokens": 1659463.0,
+      "step": 1520
+    },
+    {
+      "entropy": 0.5729703456163406,
+      "epoch": 0.6505102040816326,
+      "grad_norm": 0.9937994480133057,
+      "learning_rate": 3.499149659863946e-05,
+      "loss": 0.5724692344665527,
+      "mean_token_accuracy": 0.8275785386562348,
+      "num_tokens": 1669020.0,
+      "step": 1530
+    },
+    {
+      "entropy": 0.5577893137931824,
+      "epoch": 0.6547619047619048,
+      "grad_norm": 0.9926438331604004,
+      "learning_rate": 3.456632653061225e-05,
+      "loss": 0.5496519088745118,
+      "mean_token_accuracy": 0.8388534963130951,
+      "num_tokens": 1679086.0,
+      "step": 1540
+    },
+    {
+      "entropy": 0.5564258128404618,
+      "epoch": 0.6590136054421769,
+      "grad_norm": 0.6609563827514648,
+      "learning_rate": 3.414115646258503e-05,
+      "loss": 0.5487947940826416,
+      "mean_token_accuracy": 0.835220330953598,
+      "num_tokens": 1688728.0,
+      "step": 1550
+    },
+    {
+      "entropy": 0.5531272172927857,
+      "epoch": 0.6632653061224489,
+      "grad_norm": 0.8581554293632507,
+      "learning_rate": 3.371598639455782e-05,
+      "loss": 0.5585684776306152,
+      "mean_token_accuracy": 0.8341455698013306,
+      "num_tokens": 1700307.0,
+      "step": 1560
+    },
+    {
+      "entropy": 0.5576629728078842,
+      "epoch": 0.6675170068027211,
+      "grad_norm": 1.0460259914398193,
+      "learning_rate": 3.329081632653062e-05,
+      "loss": 0.5379378318786621,
+      "mean_token_accuracy": 0.8366690516471863,
+      "num_tokens": 1710339.0,
+      "step": 1570
+    },
+    {
+      "entropy": 0.5780779421329498,
+      "epoch": 0.6717687074829932,
+      "grad_norm": 0.8212461471557617,
+      "learning_rate": 3.28656462585034e-05,
+      "loss": 0.5949641704559326,
+      "mean_token_accuracy": 0.8214079976081848,
+      "num_tokens": 1721506.0,
+      "step": 1580
+    },
+    {
+      "entropy": 0.5216499924659729,
+      "epoch": 0.6760204081632653,
+      "grad_norm": 0.8629688620567322,
+      "learning_rate": 3.244047619047619e-05,
+      "loss": 0.5170677661895752,
+      "mean_token_accuracy": 0.8427080929279327,
+      "num_tokens": 1732208.0,
+      "step": 1590
+    },
+    {
+      "entropy": 0.49583795964717864,
+      "epoch": 0.6802721088435374,
+      "grad_norm": 1.0993518829345703,
+      "learning_rate": 3.201530612244898e-05,
+      "loss": 0.5001798152923584,
+      "mean_token_accuracy": 0.8505501627922059,
+      "num_tokens": 1743050.0,
+      "step": 1600
+    },
+    {
+      "entropy": 0.5788540303707123,
+      "epoch": 0.6845238095238095,
+      "grad_norm": 0.7463700175285339,
+      "learning_rate": 3.159013605442177e-05,
+      "loss": 0.5734623432159424,
+      "mean_token_accuracy": 0.820825070142746,
+      "num_tokens": 1756862.0,
+      "step": 1610
+    },
+    {
+      "entropy": 0.5762177228927612,
+      "epoch": 0.6887755102040817,
+      "grad_norm": 0.9257715344429016,
+      "learning_rate": 3.116496598639456e-05,
+      "loss": 0.5818124294281006,
+      "mean_token_accuracy": 0.8196542203426361,
+      "num_tokens": 1768627.0,
+      "step": 1620
+    },
+    {
+      "entropy": 0.5790290325880051,
+      "epoch": 0.6930272108843537,
+      "grad_norm": 0.7693411707878113,
+      "learning_rate": 3.073979591836735e-05,
+      "loss": 0.5630362033843994,
+      "mean_token_accuracy": 0.8297848999500275,
+      "num_tokens": 1778631.0,
+      "step": 1630
+    },
+    {
+      "entropy": 0.5448883175849915,
+      "epoch": 0.6972789115646258,
+      "grad_norm": 0.8125811219215393,
+      "learning_rate": 3.0314625850340138e-05,
+      "loss": 0.5536840438842774,
+      "mean_token_accuracy": 0.8345063030719757,
+      "num_tokens": 1790461.0,
+      "step": 1640
+    },
+    {
+      "entropy": 0.6271291971206665,
+      "epoch": 0.701530612244898,
+      "grad_norm": 0.7942107915878296,
+      "learning_rate": 2.9889455782312924e-05,
+      "loss": 0.6389796257019043,
+      "mean_token_accuracy": 0.810248178243637,
+      "num_tokens": 1800913.0,
+      "step": 1650
+    },
+    {
+      "entropy": 0.5628238677978515,
+      "epoch": 0.70578231292517,
+      "grad_norm": 0.6301717162132263,
+      "learning_rate": 2.9464285714285718e-05,
+      "loss": 0.5345774173736573,
+      "mean_token_accuracy": 0.8370151519775391,
+      "num_tokens": 1811886.0,
+      "step": 1660
+    },
+    {
+      "entropy": 0.48286513090133665,
+      "epoch": 0.7100340136054422,
+      "grad_norm": 1.233020544052124,
+      "learning_rate": 2.9039115646258504e-05,
+      "loss": 0.5052211284637451,
+      "mean_token_accuracy": 0.8479276895523071,
+      "num_tokens": 1821468.0,
+      "step": 1670
+    },
+    {
+      "entropy": 0.6830842137336731,
+      "epoch": 0.7142857142857143,
+      "grad_norm": 1.1565057039260864,
+      "learning_rate": 2.8613945578231294e-05,
+      "loss": 0.7028451442718506,
+      "mean_token_accuracy": 0.7885268032550812,
+      "num_tokens": 1830341.0,
+      "step": 1680
+    },
+    {
+      "entropy": 0.6287564933300018,
+      "epoch": 0.7185374149659864,
+      "grad_norm": 0.5923300385475159,
+      "learning_rate": 2.8188775510204084e-05,
+      "loss": 0.6260773658752441,
+      "mean_token_accuracy": 0.80694420337677,
+      "num_tokens": 1840794.0,
+      "step": 1690
+    },
+    {
+      "entropy": 0.5761405915021897,
+      "epoch": 0.7227891156462585,
+      "grad_norm": 0.813294529914856,
+      "learning_rate": 2.776360544217687e-05,
+      "loss": 0.5476944923400879,
+      "mean_token_accuracy": 0.8244221389293671,
+      "num_tokens": 1851534.0,
+      "step": 1700
+    },
+    {
+      "entropy": 0.5528579622507095,
+      "epoch": 0.7270408163265306,
+      "grad_norm": 0.9476060271263123,
+      "learning_rate": 2.7338435374149664e-05,
+      "loss": 0.5636507987976074,
+      "mean_token_accuracy": 0.8234051108360291,
+      "num_tokens": 1863352.0,
+      "step": 1710
+    },
+    {
+      "entropy": 0.5006043136119842,
+      "epoch": 0.7312925170068028,
+      "grad_norm": 0.9661667943000793,
+      "learning_rate": 2.6913265306122447e-05,
+      "loss": 0.4998173236846924,
+      "mean_token_accuracy": 0.8556727051734925,
+      "num_tokens": 1873010.0,
+      "step": 1720
+    },
+    {
+      "entropy": 0.5668102502822876,
+      "epoch": 0.7355442176870748,
+      "grad_norm": 0.9204102754592896,
+      "learning_rate": 2.648809523809524e-05,
+      "loss": 0.5956824779510498,
+      "mean_token_accuracy": 0.8302618801593781,
+      "num_tokens": 1882296.0,
+      "step": 1730
+    },
+    {
+      "entropy": 0.5725611984729767,
+      "epoch": 0.7397959183673469,
+      "grad_norm": 0.9404184222221375,
+      "learning_rate": 2.606292517006803e-05,
+      "loss": 0.5676938056945801,
+      "mean_token_accuracy": 0.8323635518550873,
+      "num_tokens": 1894428.0,
+      "step": 1740
+    },
+    {
+      "entropy": 0.606747168302536,
+      "epoch": 0.7440476190476191,
+      "grad_norm": 0.8424393534660339,
+      "learning_rate": 2.5637755102040816e-05,
+      "loss": 0.5834713935852051,
+      "mean_token_accuracy": 0.8257153987884521,
+      "num_tokens": 1904617.0,
+      "step": 1750
+    },
+    {
+      "entropy": 0.5970776557922364,
+      "epoch": 0.7482993197278912,
+      "grad_norm": 1.1546332836151123,
+      "learning_rate": 2.521258503401361e-05,
+      "loss": 0.5974382877349853,
+      "mean_token_accuracy": 0.8301580667495727,
+      "num_tokens": 1914950.0,
+      "step": 1760
+    },
+    {
+      "entropy": 0.5316449493169785,
+      "epoch": 0.7525510204081632,
+      "grad_norm": 1.3904755115509033,
+      "learning_rate": 2.4787414965986396e-05,
+      "loss": 0.527727460861206,
+      "mean_token_accuracy": 0.8430564641952515,
+      "num_tokens": 1924291.0,
+      "step": 1770
+    },
+    {
+      "entropy": 0.5104436665773392,
+      "epoch": 0.7568027210884354,
+      "grad_norm": 0.9666491150856018,
+      "learning_rate": 2.4362244897959186e-05,
+      "loss": 0.5042972564697266,
+      "mean_token_accuracy": 0.841067123413086,
+      "num_tokens": 1935889.0,
+      "step": 1780
+    },
+    {
+      "entropy": 0.5277058184146881,
+      "epoch": 0.7610544217687075,
+      "grad_norm": 1.3077667951583862,
+      "learning_rate": 2.3937074829931973e-05,
+      "loss": 0.5233573913574219,
+      "mean_token_accuracy": 0.8365782618522644,
+      "num_tokens": 1946417.0,
+      "step": 1790
+    },
+    {
+      "entropy": 0.5900924652814865,
+      "epoch": 0.7653061224489796,
+      "grad_norm": 1.129841923713684,
+      "learning_rate": 2.3511904761904762e-05,
+      "loss": 0.6162432193756103,
+      "mean_token_accuracy": 0.8183229088783264,
+      "num_tokens": 1958665.0,
+      "step": 1800
+    },
+    {
+      "entropy": 0.5582915276288987,
+      "epoch": 0.7695578231292517,
+      "grad_norm": 1.0439772605895996,
+      "learning_rate": 2.3086734693877552e-05,
+      "loss": 0.5462551593780518,
+      "mean_token_accuracy": 0.8470064103603363,
+      "num_tokens": 1967612.0,
+      "step": 1810
+    },
+    {
+      "entropy": 0.5408319383859634,
+      "epoch": 0.7738095238095238,
+      "grad_norm": 0.9232698082923889,
+      "learning_rate": 2.266156462585034e-05,
+      "loss": 0.5256539344787597,
+      "mean_token_accuracy": 0.8365410387516021,
+      "num_tokens": 1977989.0,
+      "step": 1820
+    },
+    {
+      "entropy": 0.5403281807899475,
+      "epoch": 0.7780612244897959,
+      "grad_norm": 1.0451940298080444,
+      "learning_rate": 2.2236394557823132e-05,
+      "loss": 0.5474090576171875,
+      "mean_token_accuracy": 0.834468537569046,
+      "num_tokens": 1990698.0,
+      "step": 1830
+    },
+    {
+      "entropy": 0.5493248134851456,
+      "epoch": 0.782312925170068,
+      "grad_norm": 0.774145245552063,
+      "learning_rate": 2.181122448979592e-05,
+      "loss": 0.554120397567749,
+      "mean_token_accuracy": 0.8225033700466156,
+      "num_tokens": 2001822.0,
+      "step": 1840
+    },
+    {
+      "entropy": 0.5199731826782227,
+      "epoch": 0.7865646258503401,
+      "grad_norm": 1.0582787990570068,
+      "learning_rate": 2.138605442176871e-05,
+      "loss": 0.5166659355163574,
+      "mean_token_accuracy": 0.8476324081420898,
+      "num_tokens": 2012779.0,
+      "step": 1850
+    },
+    {
+      "entropy": 0.504674881696701,
+      "epoch": 0.7908163265306123,
+      "grad_norm": 0.8114542961120605,
+      "learning_rate": 2.09608843537415e-05,
+      "loss": 0.5136760711669922,
+      "mean_token_accuracy": 0.8496062397956848,
+      "num_tokens": 2023065.0,
+      "step": 1860
+    },
+    {
+      "entropy": 0.5434481918811798,
+      "epoch": 0.7950680272108843,
+      "grad_norm": 0.9178285002708435,
+      "learning_rate": 2.0535714285714285e-05,
+      "loss": 0.5489007472991944,
+      "mean_token_accuracy": 0.8227114081382751,
+      "num_tokens": 2037557.0,
+      "step": 1870
+    },
+    {
+      "entropy": 0.5326362639665604,
+      "epoch": 0.7993197278911565,
+      "grad_norm": 0.7624576091766357,
+      "learning_rate": 2.0110544217687075e-05,
+      "loss": 0.5364444732666016,
+      "mean_token_accuracy": 0.8336756289005279,
+      "num_tokens": 2048539.0,
+      "step": 1880
+    },
+    {
+      "entropy": 0.5348059087991714,
+      "epoch": 0.8035714285714286,
+      "grad_norm": 0.6548311710357666,
+      "learning_rate": 1.9685374149659865e-05,
+      "loss": 0.5343772411346436,
+      "mean_token_accuracy": 0.8385817587375641,
+      "num_tokens": 2059612.0,
+      "step": 1890
+    },
+    {
+      "entropy": 0.5933915734291076,
+      "epoch": 0.8078231292517006,
+      "grad_norm": 1.13102126121521,
+      "learning_rate": 1.9260204081632655e-05,
+      "loss": 0.6000364780426025,
+      "mean_token_accuracy": 0.8166829824447632,
+      "num_tokens": 2070418.0,
+      "step": 1900
+    },
+    {
+      "entropy": 0.606149223446846,
+      "epoch": 0.8120748299319728,
+      "grad_norm": 1.6694409847259521,
+      "learning_rate": 1.8835034013605444e-05,
+      "loss": 0.6033652782440185,
+      "mean_token_accuracy": 0.8095532536506653,
+      "num_tokens": 2082806.0,
+      "step": 1910
+    },
+    {
+      "entropy": 0.6159947037696838,
+      "epoch": 0.8163265306122449,
+      "grad_norm": 0.7934441566467285,
+      "learning_rate": 1.840986394557823e-05,
+      "loss": 0.6037148475646973,
+      "mean_token_accuracy": 0.8110804080963134,
+      "num_tokens": 2095279.0,
+      "step": 1920
+    },
+    {
+      "entropy": 0.587390485405922,
+      "epoch": 0.8205782312925171,
+      "grad_norm": 0.8202760219573975,
+      "learning_rate": 1.798469387755102e-05,
+      "loss": 0.586046838760376,
+      "mean_token_accuracy": 0.8243176221847535,
+      "num_tokens": 2105372.0,
+      "step": 1930
+    },
+    {
+      "entropy": 0.5912446022033692,
+      "epoch": 0.8248299319727891,
+      "grad_norm": 1.623152256011963,
+      "learning_rate": 1.755952380952381e-05,
+      "loss": 0.596843671798706,
+      "mean_token_accuracy": 0.8153438031673431,
+      "num_tokens": 2114729.0,
+      "step": 1940
+    },
+    {
+      "entropy": 0.5157216489315033,
+      "epoch": 0.8290816326530612,
+      "grad_norm": 0.9215971827507019,
+      "learning_rate": 1.71343537414966e-05,
+      "loss": 0.5145907402038574,
+      "mean_token_accuracy": 0.8426204800605774,
+      "num_tokens": 2125290.0,
+      "step": 1950
+    },
+    {
+      "entropy": 0.5829062938690186,
+      "epoch": 0.8333333333333334,
+      "grad_norm": 1.193843126296997,
+      "learning_rate": 1.670918367346939e-05,
+      "loss": 0.5695863723754883,
+      "mean_token_accuracy": 0.8192765057086945,
+      "num_tokens": 2136344.0,
+      "step": 1960
+    },
+    {
+      "entropy": 0.4996066093444824,
+      "epoch": 0.8375850340136054,
+      "grad_norm": 0.8927513360977173,
+      "learning_rate": 1.6284013605442177e-05,
+      "loss": 0.4900965213775635,
+      "mean_token_accuracy": 0.8509392082691193,
+      "num_tokens": 2146080.0,
+      "step": 1970
+    },
+    {
+      "entropy": 0.5360583394765854,
+      "epoch": 0.8418367346938775,
+      "grad_norm": 1.058348536491394,
+      "learning_rate": 1.5858843537414967e-05,
+      "loss": 0.5512252807617187,
+      "mean_token_accuracy": 0.8354442238807678,
+      "num_tokens": 2156011.0,
+      "step": 1980
+    },
+    {
+      "entropy": 0.6012335926294327,
+      "epoch": 0.8460884353741497,
+      "grad_norm": 0.7904354333877563,
+      "learning_rate": 1.5433673469387753e-05,
+      "loss": 0.6157900333404541,
+      "mean_token_accuracy": 0.8187953770160675,
+      "num_tokens": 2167057.0,
+      "step": 1990
+    },
+    {
+      "entropy": 0.6478571712970733,
+      "epoch": 0.8503401360544217,
+      "grad_norm": 1.0595366954803467,
+      "learning_rate": 1.5008503401360543e-05,
+      "loss": 0.6455856323242187,
+      "mean_token_accuracy": 0.7955973386764527,
+      "num_tokens": 2178418.0,
+      "step": 2000
+    },
+    {
+      "entropy": 0.5748985707759857,
+      "epoch": 0.8545918367346939,
+      "grad_norm": 1.1120526790618896,
+      "learning_rate": 1.4583333333333335e-05,
+      "loss": 0.5637976646423339,
+      "mean_token_accuracy": 0.8272497594356537,
+      "num_tokens": 2189919.0,
+      "step": 2010
+    },
+    {
+      "entropy": 0.5793386727571488,
+      "epoch": 0.858843537414966,
+      "grad_norm": 0.9996525645256042,
+      "learning_rate": 1.4158163265306123e-05,
+      "loss": 0.5585187911987305,
+      "mean_token_accuracy": 0.8226561129093171,
+      "num_tokens": 2202449.0,
+      "step": 2020
+    },
+    {
+      "entropy": 0.48502568900585175,
+      "epoch": 0.8630952380952381,
+      "grad_norm": 0.8485120534896851,
+      "learning_rate": 1.3732993197278913e-05,
+      "loss": 0.4754936218261719,
+      "mean_token_accuracy": 0.8551179468631744,
+      "num_tokens": 2212256.0,
+      "step": 2030
+    },
+    {
+      "entropy": 0.5177913874387741,
+      "epoch": 0.8673469387755102,
+      "grad_norm": 1.0269567966461182,
+      "learning_rate": 1.3307823129251701e-05,
+      "loss": 0.5295746326446533,
+      "mean_token_accuracy": 0.8426526129245758,
+      "num_tokens": 2222079.0,
+      "step": 2040
+    },
+    {
+      "entropy": 0.5418720453977585,
+      "epoch": 0.8715986394557823,
+      "grad_norm": 1.1140215396881104,
+      "learning_rate": 1.288265306122449e-05,
+      "loss": 0.5403457164764405,
+      "mean_token_accuracy": 0.8360271990299225,
+      "num_tokens": 2233010.0,
+      "step": 2050
+    },
+    {
+      "entropy": 0.5022894471883774,
+      "epoch": 0.8758503401360545,
+      "grad_norm": 0.9412987232208252,
+      "learning_rate": 1.245748299319728e-05,
+      "loss": 0.5068081855773926,
+      "mean_token_accuracy": 0.8397215008735657,
+      "num_tokens": 2242876.0,
+      "step": 2060
+    },
+    {
+      "entropy": 0.559627965092659,
+      "epoch": 0.8801020408163265,
+      "grad_norm": 1.1703239679336548,
+      "learning_rate": 1.2032312925170067e-05,
+      "loss": 0.5765266418457031,
+      "mean_token_accuracy": 0.8238355994224549,
+      "num_tokens": 2253375.0,
+      "step": 2070
+    },
+    {
+      "entropy": 0.5473792344331742,
+      "epoch": 0.8843537414965986,
+      "grad_norm": 1.2103474140167236,
+      "learning_rate": 1.1607142857142857e-05,
+      "loss": 0.5515032768249511,
+      "mean_token_accuracy": 0.8346924901008606,
+      "num_tokens": 2265217.0,
+      "step": 2080
+    },
+    {
+      "entropy": 0.5150429278612136,
+      "epoch": 0.8886054421768708,
+      "grad_norm": 0.8060351014137268,
+      "learning_rate": 1.1181972789115647e-05,
+      "loss": 0.5019734859466553,
+      "mean_token_accuracy": 0.8517545342445374,
+      "num_tokens": 2276492.0,
+      "step": 2090
+    },
+    {
+      "entropy": 0.5388852089643479,
+      "epoch": 0.8928571428571429,
+      "grad_norm": 1.129920244216919,
+      "learning_rate": 1.0756802721088435e-05,
+      "loss": 0.5413199424743652,
+      "mean_token_accuracy": 0.8384690225124359,
+      "num_tokens": 2286622.0,
+      "step": 2100
+    },
+    {
+      "entropy": 0.5830584645271302,
+      "epoch": 0.8971088435374149,
+      "grad_norm": 1.0372401475906372,
+      "learning_rate": 1.0331632653061225e-05,
+      "loss": 0.5750691890716553,
+      "mean_token_accuracy": 0.8245911538600922,
+      "num_tokens": 2298062.0,
+      "step": 2110
+    },
+    {
+      "entropy": 0.5636593252420425,
+      "epoch": 0.9013605442176871,
+      "grad_norm": 0.7910812497138977,
+      "learning_rate": 9.906462585034014e-06,
+      "loss": 0.568049955368042,
+      "mean_token_accuracy": 0.8189381241798401,
+      "num_tokens": 2310002.0,
+      "step": 2120
+    },
+    {
+      "entropy": 0.5329378515481948,
+      "epoch": 0.9056122448979592,
+      "grad_norm": 0.780502438545227,
+      "learning_rate": 9.481292517006803e-06,
+      "loss": 0.5344277381896972,
+      "mean_token_accuracy": 0.8355330467224121,
+      "num_tokens": 2321286.0,
+      "step": 2130
+    },
+    {
+      "entropy": 0.5965909093618393,
+      "epoch": 0.9098639455782312,
+      "grad_norm": 1.108359694480896,
+      "learning_rate": 9.056122448979593e-06,
+      "loss": 0.6025591373443604,
+      "mean_token_accuracy": 0.8153828561306,
+      "num_tokens": 2334075.0,
+      "step": 2140
+    },
+    {
+      "entropy": 0.551003360748291,
+      "epoch": 0.9141156462585034,
+      "grad_norm": 0.9165040850639343,
+      "learning_rate": 8.630952380952381e-06,
+      "loss": 0.5357895851135254,
+      "mean_token_accuracy": 0.8391867280006409,
+      "num_tokens": 2343342.0,
+      "step": 2150
+    },
+    {
+      "entropy": 0.6025931894779205,
+      "epoch": 0.9183673469387755,
+      "grad_norm": 0.9292078018188477,
+      "learning_rate": 8.20578231292517e-06,
+      "loss": 0.6112618923187256,
+      "mean_token_accuracy": 0.8223409533500672,
+      "num_tokens": 2353860.0,
+      "step": 2160
+    },
+    {
+      "entropy": 0.5607409834861755,
+      "epoch": 0.9226190476190477,
+      "grad_norm": 0.8626019358634949,
+      "learning_rate": 7.78061224489796e-06,
+      "loss": 0.5666313648223877,
+      "mean_token_accuracy": 0.8334515213966369,
+      "num_tokens": 2364432.0,
+      "step": 2170
+    },
+    {
+      "entropy": 0.6042696863412857,
+      "epoch": 0.9268707482993197,
+      "grad_norm": 1.1293621063232422,
+      "learning_rate": 7.355442176870749e-06,
+      "loss": 0.6011404037475586,
+      "mean_token_accuracy": 0.8190859973430633,
+      "num_tokens": 2374475.0,
+      "step": 2180
+    },
+    {
+      "entropy": 0.5735134840011596,
+      "epoch": 0.9311224489795918,
+      "grad_norm": 0.9066168069839478,
+      "learning_rate": 6.930272108843537e-06,
+      "loss": 0.5751115798950195,
+      "mean_token_accuracy": 0.825209504365921,
+      "num_tokens": 2386447.0,
+      "step": 2190
+    },
+    {
+      "entropy": 0.5244353115558624,
+      "epoch": 0.935374149659864,
+      "grad_norm": 1.072506308555603,
+      "learning_rate": 6.505102040816327e-06,
+      "loss": 0.5103401660919189,
+      "mean_token_accuracy": 0.8505155682563782,
+      "num_tokens": 2396159.0,
+      "step": 2200
+    },
+    {
+      "entropy": 0.5949863910675048,
+      "epoch": 0.939625850340136,
+      "grad_norm": 0.6773778200149536,
+      "learning_rate": 6.079931972789116e-06,
+      "loss": 0.5959713459014893,
+      "mean_token_accuracy": 0.8145836770534516,
+      "num_tokens": 2406947.0,
+      "step": 2210
+    },
+    {
+      "entropy": 0.551213338971138,
+      "epoch": 0.9438775510204082,
+      "grad_norm": 0.6725026369094849,
+      "learning_rate": 5.654761904761905e-06,
+      "loss": 0.5293383121490478,
+      "mean_token_accuracy": 0.8271439135074615,
+      "num_tokens": 2419071.0,
+      "step": 2220
+    },
+    {
+      "entropy": 0.5137664794921875,
+      "epoch": 0.9481292517006803,
+      "grad_norm": 0.8478580117225647,
+      "learning_rate": 5.229591836734695e-06,
+      "loss": 0.5074201583862304,
+      "mean_token_accuracy": 0.842332798242569,
+      "num_tokens": 2430620.0,
+      "step": 2230
+    },
+    {
+      "entropy": 0.5922556847333909,
+      "epoch": 0.9523809523809523,
+      "grad_norm": 0.3467719554901123,
+      "learning_rate": 4.804421768707483e-06,
+      "loss": 0.5839669227600097,
+      "mean_token_accuracy": 0.811210173368454,
+      "num_tokens": 2444057.0,
+      "step": 2240
+    },
+    {
+      "entropy": 0.5484513103961944,
+      "epoch": 0.9566326530612245,
+      "grad_norm": 0.760888397693634,
+      "learning_rate": 4.379251700680272e-06,
+      "loss": 0.5440735340118408,
+      "mean_token_accuracy": 0.8317016720771789,
+      "num_tokens": 2454080.0,
+      "step": 2250
+    },
+    {
+      "entropy": 0.5103359013795853,
+      "epoch": 0.9608843537414966,
+      "grad_norm": 0.9869306683540344,
+      "learning_rate": 3.954081632653062e-06,
+      "loss": 0.5173935890197754,
+      "mean_token_accuracy": 0.845134687423706,
+      "num_tokens": 2463311.0,
+      "step": 2260
+    },
+    {
+      "entropy": 0.5766151458024978,
+      "epoch": 0.9651360544217688,
+      "grad_norm": 1.1104836463928223,
+      "learning_rate": 3.5289115646258504e-06,
+      "loss": 0.5886892318725586,
+      "mean_token_accuracy": 0.8289117157459259,
+      "num_tokens": 2475264.0,
+      "step": 2270
+    },
+    {
+      "entropy": 0.5101310968399048,
+      "epoch": 0.9693877551020408,
+      "grad_norm": 3.386535406112671,
+      "learning_rate": 3.1037414965986395e-06,
+      "loss": 0.5177210807800293,
+      "mean_token_accuracy": 0.8472946226596832,
+      "num_tokens": 2484532.0,
+      "step": 2280
+    },
+    {
+      "entropy": 0.5026126235723496,
+      "epoch": 0.9736394557823129,
+      "grad_norm": 0.9637086987495422,
+      "learning_rate": 2.6785714285714285e-06,
+      "loss": 0.49527649879455565,
+      "mean_token_accuracy": 0.85142502784729,
+      "num_tokens": 2494982.0,
+      "step": 2290
+    },
+    {
+      "entropy": 0.5845519959926605,
+      "epoch": 0.9778911564625851,
+      "grad_norm": 0.9303038120269775,
+      "learning_rate": 2.253401360544218e-06,
+      "loss": 0.568077802658081,
+      "mean_token_accuracy": 0.8299287617206573,
+      "num_tokens": 2505785.0,
+      "step": 2300
+    },
+    {
+      "entropy": 0.5489324152469635,
+      "epoch": 0.9821428571428571,
+      "grad_norm": 1.0844236612319946,
+      "learning_rate": 1.8282312925170068e-06,
+      "loss": 0.5580832481384277,
+      "mean_token_accuracy": 0.8412139117717743,
+      "num_tokens": 2514540.0,
+      "step": 2310
+    },
+    {
+      "entropy": 0.5149367898702621,
+      "epoch": 0.9863945578231292,
+      "grad_norm": 1.0444934368133545,
+      "learning_rate": 1.403061224489796e-06,
+      "loss": 0.5115521907806396,
+      "mean_token_accuracy": 0.8455105900764466,
+      "num_tokens": 2525422.0,
+      "step": 2320
+    },
+    {
+      "entropy": 0.5144448757171631,
+      "epoch": 0.9906462585034014,
+      "grad_norm": 0.7299341559410095,
+      "learning_rate": 9.778911564625851e-07,
+      "loss": 0.5067722797393799,
+      "mean_token_accuracy": 0.8519455373287201,
+      "num_tokens": 2535049.0,
+      "step": 2330
+    },
+    {
+      "entropy": 0.5480362325906754,
+      "epoch": 0.9948979591836735,
+      "grad_norm": 1.154684066772461,
+      "learning_rate": 5.527210884353742e-07,
+      "loss": 0.5571912288665771,
+      "mean_token_accuracy": 0.8421735644340516,
+      "num_tokens": 2544974.0,
+      "step": 2340
+    },
+    {
+      "entropy": 0.5128873139619827,
+      "epoch": 0.9991496598639455,
+      "grad_norm": 0.5887725949287415,
+      "learning_rate": 1.2755102040816326e-07,
+      "loss": 0.5141651153564453,
+      "mean_token_accuracy": 0.8440634250640869,
+      "num_tokens": 2555761.0,
+      "step": 2350
+    },
+    {
+      "epoch": 1.0,
+      "eval_entropy": 0.5433352808005937,
+      "eval_loss": 0.5373454689979553,
+      "eval_mean_token_accuracy": 0.8366637416468322,
+      "eval_num_tokens": 2558089.0,
+      "eval_runtime": 35.2529,
+      "eval_samples_per_second": 29.643,
+      "eval_steps_per_second": 7.432,
+      "step": 2352
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2352,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.921285046068777e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2352/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be72912b0c23c5ff1e9832dab7ec1662bf093daae78147fc1b5c95c2f501666e
+size 5649