diff --git a/.gitattributes b/.gitattributes
index 37544a9fda905c62b1804ef7ada1f99b4f7d5054..51a059284988cf628ee891a753c959554b514b8d 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -95,3 +95,5 @@ mbpp/output_evaluation/scores-4-2.txt filter=lfs diff=lfs merge=lfs -text
 mbpp/output_evaluation/scores-5-0.txt filter=lfs diff=lfs merge=lfs -text
 mbpp/output_evaluation/scores-5-1.txt filter=lfs diff=lfs merge=lfs -text
 mbpp/output_evaluation/scores-5-2.txt filter=lfs diff=lfs merge=lfs -text
+math/finetuned/checkpoint/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+math/finetuned/merged/tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/math/finetuned/checkpoint/README.md b/math/finetuned/checkpoint/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..c0bcb93efbdbc6525ca54820d6dc5697bd0267bf
--- /dev/null
+++ b/math/finetuned/checkpoint/README.md
@@ -0,0 +1,208 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+tags:
+- base_model:adapter:meta-llama/Llama-3.1-8B-Instruct
+- dpo
+- lora
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.0
\ No newline at end of file
diff --git a/math/finetuned/checkpoint/adapter_config.json b/math/finetuned/checkpoint/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..8b9af8fb49c284f77e02485556810a9b34fdeea1
--- /dev/null
+++ b/math/finetuned/checkpoint/adapter_config.json
@@ -0,0 +1,40 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "LlamaForCausalLM",
+    "parent_library": "transformers.models.llama.modeling_llama"
+  },
+  "base_model_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.01,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/math/finetuned/checkpoint/adapter_model.safetensors b/math/finetuned/checkpoint/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b0fbc093c2f3857d5dc49d955efe6f298ad0083c
--- /dev/null
+++ b/math/finetuned/checkpoint/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28f82ac5cd7e7cfd1d9e7fd97ea679426c021b11f6013137c698b95067b1bbae
+size 13648432
diff --git a/math/finetuned/checkpoint/optimizer.pt b/math/finetuned/checkpoint/optimizer.pt
new file mode 100644
index 0000000000000000000000000000000000000000..6bff1e028e206a182eef9e8ef5add01807dd8488
--- /dev/null
+++ b/math/finetuned/checkpoint/optimizer.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b33c3aedc14f2b1a316e9ec11c1efdb452c2d2e7613c96dc14828c46c3da01
+size 27370618
diff --git a/math/finetuned/checkpoint/rng_state.pth b/math/finetuned/checkpoint/rng_state.pth
new file mode 100644
index 0000000000000000000000000000000000000000..93a52228ea16468eaab41b66555ced3eaaea002c
--- /dev/null
+++ b/math/finetuned/checkpoint/rng_state.pth
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:386fcc8cc1089aade9450d86fb239ea3483f455fd2d78d8378645feecfec9d69
+size 14244
diff --git a/math/finetuned/checkpoint/scheduler.pt b/math/finetuned/checkpoint/scheduler.pt
new file mode 100644
index 0000000000000000000000000000000000000000..4d0df3a646891395da9ffb64d0114c16cc00401e
--- /dev/null
+++ b/math/finetuned/checkpoint/scheduler.pt
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a2c20000a5dc5c87aafd6e26e31a23a9d4439fd01b8e50c10cd7c72c97ee11c
+size 1064
diff --git a/math/finetuned/checkpoint/special_tokens_map.json b/math/finetuned/checkpoint/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..b43be96621d147110fb8a18b5776ec6e38516127
--- /dev/null
+++ b/math/finetuned/checkpoint/special_tokens_map.json
@@ -0,0 +1,17 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}
diff --git a/math/finetuned/checkpoint/tokenizer.json b/math/finetuned/checkpoint/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..1c1d8d5c9024994f1d3b00f9662b8dd89ca13cf2
--- /dev/null
+++ b/math/finetuned/checkpoint/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920
diff --git a/math/finetuned/checkpoint/tokenizer_config.json b/math/finetuned/checkpoint/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..b6e4e4df0ecc9c7f724a657ad557fae021e2b830
--- /dev/null
+++ b/math/finetuned/checkpoint/tokenizer_config.json
@@ -0,0 +1,2063 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}
diff --git a/math/finetuned/checkpoint/trainer_state.json b/math/finetuned/checkpoint/trainer_state.json
new file mode 100644
index 0000000000000000000000000000000000000000..0e45d67c9f8f222d367dacd457d083f269ebd4b5
--- /dev/null
+++ b/math/finetuned/checkpoint/trainer_state.json
@@ -0,0 +1,3033 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.005,
+      "grad_norm": 0.12633441388607025,
+      "learning_rate": 9.95e-07,
+      "logits/chosen": 0.1572265625,
+      "logits/rejected": 0.16015625,
+      "logps/chosen": -23.5,
+      "logps/rejected": -26.0,
+      "loss": 0.6917,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.36166664958000183,
+      "rewards/margins": 0.012500007636845112,
+      "rewards/rejected": 0.3491666316986084,
+      "step": 10
+    },
+    {
+      "epoch": 0.01,
+      "grad_norm": 8.615042686462402,
+      "learning_rate": 9.9e-07,
+      "logits/chosen": 0.1669921875,
+      "logits/rejected": 0.181640625,
+      "logps/chosen": -25.5,
+      "logps/rejected": -26.375,
+      "loss": 0.6876,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.36750003695487976,
+      "rewards/margins": 0.014999995008111,
+      "rewards/rejected": 0.3525000512599945,
+      "step": 20
+    },
+    {
+      "epoch": 0.015,
+      "grad_norm": 7.2749199867248535,
+      "learning_rate": 9.849999999999999e-07,
+      "logits/chosen": 0.166015625,
+      "logits/rejected": 0.16015625,
+      "logps/chosen": -25.125,
+      "logps/rejected": -23.25,
+      "loss": 0.6495,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.3970833420753479,
+      "rewards/margins": 0.09375002980232239,
+      "rewards/rejected": 0.3033332824707031,
+      "step": 30
+    },
+    {
+      "epoch": 0.02,
+      "grad_norm": 10.585458755493164,
+      "learning_rate": 9.8e-07,
+      "logits/chosen": 0.1494140625,
+      "logits/rejected": 0.189453125,
+      "logps/chosen": -26.75,
+      "logps/rejected": -26.5,
+      "loss": 0.6768,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.37958335876464844,
+      "rewards/margins": 0.04250001907348633,
+      "rewards/rejected": 0.3370833098888397,
+      "step": 40
+    },
+    {
+      "epoch": 0.025,
+      "grad_norm": 7.547507286071777,
+      "learning_rate": 9.75e-07,
+      "logits/chosen": 0.15234375,
+      "logits/rejected": 0.1865234375,
+      "logps/chosen": -24.25,
+      "logps/rejected": -29.0,
+      "loss": 0.651,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.4070833623409271,
+      "rewards/margins": 0.09791669249534607,
+      "rewards/rejected": 0.30916666984558105,
+      "step": 50
+    },
+    {
+      "epoch": 0.03,
+      "grad_norm": 7.430334091186523,
+      "learning_rate": 9.7e-07,
+      "logits/chosen": 0.1484375,
+      "logits/rejected": 0.14453125,
+      "logps/chosen": -24.75,
+      "logps/rejected": -24.625,
+      "loss": 0.6988,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.42208337783813477,
+      "rewards/margins": 0.008333375677466393,
+      "rewards/rejected": 0.4137499928474426,
+      "step": 60
+    },
+    {
+      "epoch": 0.035,
+      "grad_norm": 12.350720405578613,
+      "learning_rate": 9.649999999999999e-07,
+      "logits/chosen": 0.1259765625,
+      "logits/rejected": 0.1455078125,
+      "logps/chosen": -25.625,
+      "logps/rejected": -27.0,
+      "loss": 0.6275,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.45666664838790894,
+      "rewards/margins": 0.14125004410743713,
+      "rewards/rejected": 0.3154166340827942,
+      "step": 70
+    },
+    {
+      "epoch": 0.04,
+      "grad_norm": 5.559193134307861,
+      "learning_rate": 9.6e-07,
+      "logits/chosen": 0.1572265625,
+      "logits/rejected": 0.171875,
+      "logps/chosen": -24.375,
+      "logps/rejected": -25.875,
+      "loss": 0.7133,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.35333335399627686,
+      "rewards/margins": -0.022083332762122154,
+      "rewards/rejected": 0.37541669607162476,
+      "step": 80
+    },
+    {
+      "epoch": 0.045,
+      "grad_norm": 10.201204299926758,
+      "learning_rate": 9.55e-07,
+      "logits/chosen": 0.1513671875,
+      "logits/rejected": 0.201171875,
+      "logps/chosen": -23.625,
+      "logps/rejected": -30.0,
+      "loss": 0.6804,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.36500003933906555,
+      "rewards/margins": 0.032500009983778,
+      "rewards/rejected": 0.33250004053115845,
+      "step": 90
+    },
+    {
+      "epoch": 0.05,
+      "grad_norm": 11.54755973815918,
+      "learning_rate": 9.499999999999999e-07,
+      "logits/chosen": 0.166015625,
+      "logits/rejected": 0.1875,
+      "logps/chosen": -28.0,
+      "logps/rejected": -26.75,
+      "loss": 0.6821,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.33666667342185974,
+      "rewards/margins": 0.027916723862290382,
+      "rewards/rejected": 0.3087499439716339,
+      "step": 100
+    },
+    {
+      "epoch": 0.055,
+      "grad_norm": 6.9829792976379395,
+      "learning_rate": 9.45e-07,
+      "logits/chosen": 0.15625,
+      "logits/rejected": 0.146484375,
+      "logps/chosen": -26.125,
+      "logps/rejected": -23.5,
+      "loss": 0.6818,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.40833336114883423,
+      "rewards/margins": 0.03875003382563591,
+      "rewards/rejected": 0.36958327889442444,
+      "step": 110
+    },
+    {
+      "epoch": 0.06,
+      "grad_norm": 19.138856887817383,
+      "learning_rate": 9.399999999999999e-07,
+      "logits/chosen": 0.15234375,
+      "logits/rejected": 0.1806640625,
+      "logps/chosen": -24.25,
+      "logps/rejected": -29.875,
+      "loss": 0.6597,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.43958336114883423,
+      "rewards/margins": 0.08500001579523087,
+      "rewards/rejected": 0.3545833230018616,
+      "step": 120
+    },
+    {
+      "epoch": 0.065,
+      "grad_norm": 11.872893333435059,
+      "learning_rate": 9.35e-07,
+      "logits/chosen": 0.1669921875,
+      "logits/rejected": 0.1767578125,
+      "logps/chosen": -28.0,
+      "logps/rejected": -27.875,
+      "loss": 0.669,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.47958335280418396,
+      "rewards/margins": 0.062499988824129105,
+      "rewards/rejected": 0.41708335280418396,
+      "step": 130
+    },
+    {
+      "epoch": 0.07,
+      "grad_norm": 0.14289496839046478,
+      "learning_rate": 9.3e-07,
+      "logits/chosen": 0.1484375,
+      "logits/rejected": 0.15625,
+      "logps/chosen": -30.875,
+      "logps/rejected": -25.75,
+      "loss": 0.7281,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4037500321865082,
+      "rewards/margins": -0.04916665703058243,
+      "rewards/rejected": 0.45291662216186523,
+      "step": 140
+    },
+    {
+      "epoch": 0.075,
+      "grad_norm": 10.220949172973633,
+      "learning_rate": 9.25e-07,
+      "logits/chosen": 0.1533203125,
+      "logits/rejected": 0.1513671875,
+      "logps/chosen": -23.75,
+      "logps/rejected": -24.0,
+      "loss": 0.6597,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.44833335280418396,
+      "rewards/margins": 0.07375001162290573,
+      "rewards/rejected": 0.37458333373069763,
+      "step": 150
+    },
+    {
+      "epoch": 0.08,
+      "grad_norm": 7.098750591278076,
+      "learning_rate": 9.2e-07,
+      "logits/chosen": 0.162109375,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -22.875,
+      "logps/rejected": -32.75,
+      "loss": 0.7058,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4412499964237213,
+      "rewards/margins": -0.004999992437660694,
+      "rewards/rejected": 0.4462500214576721,
+      "step": 160
+    },
+    {
+      "epoch": 0.085,
+      "grad_norm": 7.279146671295166,
+      "learning_rate": 9.15e-07,
+      "logits/chosen": 0.140625,
+      "logits/rejected": 0.1455078125,
+      "logps/chosen": -25.375,
+      "logps/rejected": -24.125,
+      "loss": 0.7107,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.42416661977767944,
+      "rewards/margins": -0.014999988488852978,
+      "rewards/rejected": 0.4391666352748871,
+      "step": 170
+    },
+    {
+      "epoch": 0.09,
+      "grad_norm": 7.507859706878662,
+      "learning_rate": 9.1e-07,
+      "logits/chosen": 0.15234375,
+      "logits/rejected": 0.146484375,
+      "logps/chosen": -22.625,
+      "logps/rejected": -22.125,
+      "loss": 0.6733,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.45916667580604553,
+      "rewards/margins": 0.05458337068557739,
+      "rewards/rejected": 0.40458330512046814,
+      "step": 180
+    },
+    {
+      "epoch": 0.095,
+      "grad_norm": 9.808679580688477,
+      "learning_rate": 9.05e-07,
+      "logits/chosen": 0.1572265625,
+      "logits/rejected": 0.158203125,
+      "logps/chosen": -29.125,
+      "logps/rejected": -23.625,
+      "loss": 0.689,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.37708336114883423,
+      "rewards/margins": 0.03166668862104416,
+      "rewards/rejected": 0.3454166650772095,
+      "step": 190
+    },
+    {
+      "epoch": 0.1,
+      "grad_norm": 5.743889331817627,
+      "learning_rate": 9e-07,
+      "logits/chosen": 0.1611328125,
+      "logits/rejected": 0.189453125,
+      "logps/chosen": -23.75,
+      "logps/rejected": -26.625,
+      "loss": 0.6298,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5375000238418579,
+      "rewards/margins": 0.14125001430511475,
+      "rewards/rejected": 0.39625000953674316,
+      "step": 200
+    },
+    {
+      "epoch": 0.105,
+      "grad_norm": 6.256867408752441,
+      "learning_rate": 8.95e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.1669921875,
+      "logps/chosen": -23.0,
+      "logps/rejected": -26.25,
+      "loss": 0.6833,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4383333623409271,
+      "rewards/margins": 0.038333337754011154,
+      "rewards/rejected": 0.3999999761581421,
+      "step": 210
+    },
+    {
+      "epoch": 0.11,
+      "grad_norm": 9.232077598571777,
+      "learning_rate": 8.9e-07,
+      "logits/chosen": 0.1513671875,
+      "logits/rejected": 0.16015625,
+      "logps/chosen": -26.375,
+      "logps/rejected": -24.625,
+      "loss": 0.7148,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.3566666543483734,
+      "rewards/margins": -0.02708331309258938,
+      "rewards/rejected": 0.38374999165534973,
+      "step": 220
+    },
+    {
+      "epoch": 0.115,
+      "grad_norm": 11.062021255493164,
+      "learning_rate": 8.85e-07,
+      "logits/chosen": 0.1552734375,
+      "logits/rejected": 0.1640625,
+      "logps/chosen": -24.875,
+      "logps/rejected": -23.25,
+      "loss": 0.6377,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.48416662216186523,
+      "rewards/margins": 0.12208335101604462,
+      "rewards/rejected": 0.3620833158493042,
+      "step": 230
+    },
+    {
+      "epoch": 0.12,
+      "grad_norm": 9.357806205749512,
+      "learning_rate": 8.799999999999999e-07,
+      "logits/chosen": 0.1533203125,
+      "logits/rejected": 0.15234375,
+      "logps/chosen": -23.5,
+      "logps/rejected": -23.75,
+      "loss": 0.6926,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.46041664481163025,
+      "rewards/margins": 0.013750016689300537,
+      "rewards/rejected": 0.4466666281223297,
+      "step": 240
+    },
+    {
+      "epoch": 0.125,
+      "grad_norm": 7.112157344818115,
+      "learning_rate": 8.75e-07,
+      "logits/chosen": 0.134765625,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -24.0,
+      "logps/rejected": -26.25,
+      "loss": 0.6339,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4816667139530182,
+      "rewards/margins": 0.13458336889743805,
+      "rewards/rejected": 0.34708333015441895,
+      "step": 250
+    },
+    {
+      "epoch": 0.13,
+      "grad_norm": 8.50083065032959,
+      "learning_rate": 8.699999999999999e-07,
+      "logits/chosen": 0.158203125,
+      "logits/rejected": 0.15625,
+      "logps/chosen": -23.75,
+      "logps/rejected": -24.75,
+      "loss": 0.7027,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.43166667222976685,
+      "rewards/margins": -0.008749982342123985,
+      "rewards/rejected": 0.4404166340827942,
+      "step": 260
+    },
+    {
+      "epoch": 0.135,
+      "grad_norm": 0.14750409126281738,
+      "learning_rate": 8.65e-07,
+      "logits/chosen": 0.150390625,
+      "logits/rejected": 0.154296875,
+      "logps/chosen": -24.25,
+      "logps/rejected": -24.125,
+      "loss": 0.6486,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5224999785423279,
+      "rewards/margins": 0.10208334028720856,
+      "rewards/rejected": 0.4204166829586029,
+      "step": 270
+    },
+    {
+      "epoch": 0.14,
+      "grad_norm": 8.060702323913574,
+      "learning_rate": 8.599999999999999e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -24.5,
+      "logps/rejected": -25.75,
+      "loss": 0.654,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.44749999046325684,
+      "rewards/margins": 0.08666671812534332,
+      "rewards/rejected": 0.3608332872390747,
+      "step": 280
+    },
+    {
+      "epoch": 0.145,
+      "grad_norm": 7.588810920715332,
+      "learning_rate": 8.55e-07,
+      "logits/chosen": 0.140625,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -24.0,
+      "logps/rejected": -22.0,
+      "loss": 0.6948,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4595833420753479,
+      "rewards/margins": 0.014583337120711803,
+      "rewards/rejected": 0.4449999928474426,
+      "step": 290
+    },
+    {
+      "epoch": 0.15,
+      "grad_norm": 8.901789665222168,
+      "learning_rate": 8.499999999999999e-07,
+      "logits/chosen": 0.15625,
+      "logits/rejected": 0.1552734375,
+      "logps/chosen": -24.375,
+      "logps/rejected": -24.5,
+      "loss": 0.648,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5116666555404663,
+      "rewards/margins": 0.10916668176651001,
+      "rewards/rejected": 0.4025000035762787,
+      "step": 300
+    },
+    {
+      "epoch": 0.155,
+      "grad_norm": 0.13474789261817932,
+      "learning_rate": 8.45e-07,
+      "logits/chosen": 0.140625,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -22.875,
+      "logps/rejected": -24.125,
+      "loss": 0.6781,
+      "rewards/accuracies": 0.30000001192092896,
+      "rewards/chosen": 0.5095833539962769,
+      "rewards/margins": 0.03833334892988205,
+      "rewards/rejected": 0.4712499976158142,
+      "step": 310
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 9.167498588562012,
+      "learning_rate": 8.399999999999999e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -23.75,
+      "logps/rejected": -22.25,
+      "loss": 0.7382,
+      "rewards/accuracies": 0.30000001192092896,
+      "rewards/chosen": 0.4150000214576721,
+      "rewards/margins": -0.07166668772697449,
+      "rewards/rejected": 0.4866666793823242,
+      "step": 320
+    },
+    {
+      "epoch": 0.165,
+      "grad_norm": 13.539304733276367,
+      "learning_rate": 8.349999999999999e-07,
+      "logits/chosen": 0.14453125,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -27.5,
+      "logps/rejected": -25.25,
+      "loss": 0.6682,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.5325000286102295,
+      "rewards/margins": 0.0779166966676712,
+      "rewards/rejected": 0.4545833170413971,
+      "step": 330
+    },
+    {
+      "epoch": 0.17,
+      "grad_norm": 5.054110050201416,
+      "learning_rate": 8.299999999999999e-07,
+      "logits/chosen": 0.1611328125,
+      "logits/rejected": 0.1689453125,
+      "logps/chosen": -23.625,
+      "logps/rejected": -24.25,
+      "loss": 0.673,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.49291667342185974,
+      "rewards/margins": 0.060000013560056686,
+      "rewards/rejected": 0.43291664123535156,
+      "step": 340
+    },
+    {
+      "epoch": 0.175,
+      "grad_norm": 10.040949821472168,
+      "learning_rate": 8.249999999999999e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.142578125,
+      "logps/chosen": -24.625,
+      "logps/rejected": -25.25,
+      "loss": 0.6925,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.4229167103767395,
+      "rewards/margins": 0.010000040754675865,
+      "rewards/rejected": 0.4129166603088379,
+      "step": 350
+    },
+    {
+      "epoch": 0.18,
+      "grad_norm": 7.361537933349609,
+      "learning_rate": 8.199999999999999e-07,
+      "logits/chosen": 0.158203125,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -25.25,
+      "logps/rejected": -22.625,
+      "loss": 0.6816,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4945833683013916,
+      "rewards/margins": 0.03750002011656761,
+      "rewards/rejected": 0.4570833742618561,
+      "step": 360
+    },
+    {
+      "epoch": 0.185,
+      "grad_norm": 7.821290016174316,
+      "learning_rate": 8.149999999999999e-07,
+      "logits/chosen": 0.123046875,
+      "logits/rejected": 0.13671875,
+      "logps/chosen": -26.75,
+      "logps/rejected": -23.5,
+      "loss": 0.6038,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.5879167318344116,
+      "rewards/margins": 0.19500002264976501,
+      "rewards/rejected": 0.39291661977767944,
+      "step": 370
+    },
+    {
+      "epoch": 0.19,
+      "grad_norm": 6.75149393081665,
+      "learning_rate": 8.1e-07,
+      "logits/chosen": 0.1376953125,
+      "logits/rejected": 0.1640625,
+      "logps/chosen": -26.25,
+      "logps/rejected": -23.625,
+      "loss": 0.6692,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4687500596046448,
+      "rewards/margins": 0.065000019967556,
+      "rewards/rejected": 0.4037500023841858,
+      "step": 380
+    },
+    {
+      "epoch": 0.195,
+      "grad_norm": 11.053235054016113,
+      "learning_rate": 8.05e-07,
+      "logits/chosen": 0.146484375,
+      "logits/rejected": 0.1484375,
+      "logps/chosen": -25.75,
+      "logps/rejected": -25.25,
+      "loss": 0.644,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4658333361148834,
+      "rewards/margins": 0.11500000953674316,
+      "rewards/rejected": 0.35083329677581787,
+      "step": 390
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 7.667977809906006,
+      "learning_rate": 8e-07,
+      "logits/chosen": 0.169921875,
+      "logits/rejected": 0.166015625,
+      "logps/chosen": -21.75,
+      "logps/rejected": -27.125,
+      "loss": 0.6815,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4308333396911621,
+      "rewards/margins": 0.04750003665685654,
+      "rewards/rejected": 0.383333295583725,
+      "step": 400
+    },
+    {
+      "epoch": 0.205,
+      "grad_norm": 4.176445960998535,
+      "learning_rate": 7.95e-07,
+      "logits/chosen": 0.154296875,
+      "logits/rejected": 0.1513671875,
+      "logps/chosen": -24.125,
+      "logps/rejected": -26.875,
+      "loss": 0.6778,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.48375004529953003,
+      "rewards/margins": 0.044166646897792816,
+      "rewards/rejected": 0.43958336114883423,
+      "step": 410
+    },
+    {
+      "epoch": 0.21,
+      "grad_norm": 7.142360210418701,
+      "learning_rate": 7.9e-07,
+      "logits/chosen": 0.1767578125,
+      "logits/rejected": 0.1787109375,
+      "logps/chosen": -25.25,
+      "logps/rejected": -25.375,
+      "loss": 0.7183,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.43916669487953186,
+      "rewards/margins": -0.02749999240040779,
+      "rewards/rejected": 0.46666663885116577,
+      "step": 420
+    },
+    {
+      "epoch": 0.215,
+      "grad_norm": 8.128364562988281,
+      "learning_rate": 7.85e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -23.75,
+      "logps/rejected": -27.25,
+      "loss": 0.641,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.45875000953674316,
+      "rewards/margins": 0.11333335936069489,
+      "rewards/rejected": 0.3454166650772095,
+      "step": 430
+    },
+    {
+      "epoch": 0.22,
+      "grad_norm": 6.533936023712158,
+      "learning_rate": 7.799999999999999e-07,
+      "logits/chosen": 0.1552734375,
+      "logits/rejected": 0.1396484375,
+      "logps/chosen": -23.75,
+      "logps/rejected": -29.5,
+      "loss": 0.6354,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.45916667580604553,
+      "rewards/margins": 0.13583336770534515,
+      "rewards/rejected": 0.3233332931995392,
+      "step": 440
+    },
+    {
+      "epoch": 0.225,
+      "grad_norm": 6.75587272644043,
+      "learning_rate": 7.75e-07,
+      "logits/chosen": 0.1494140625,
+      "logits/rejected": 0.1435546875,
+      "logps/chosen": -23.0,
+      "logps/rejected": -27.25,
+      "loss": 0.6403,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.559583306312561,
+      "rewards/margins": 0.1366666853427887,
+      "rewards/rejected": 0.4229167103767395,
+      "step": 450
+    },
+    {
+      "epoch": 0.23,
+      "grad_norm": 8.185866355895996,
+      "learning_rate": 7.699999999999999e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.1474609375,
+      "logps/chosen": -22.625,
+      "logps/rejected": -25.625,
+      "loss": 0.6932,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4766666293144226,
+      "rewards/margins": 0.023333333432674408,
+      "rewards/rejected": 0.45333331823349,
+      "step": 460
+    },
+    {
+      "epoch": 0.235,
+      "grad_norm": 9.489606857299805,
+      "learning_rate": 7.65e-07,
+      "logits/chosen": 0.130859375,
+      "logits/rejected": 0.1494140625,
+      "logps/chosen": -25.375,
+      "logps/rejected": -24.75,
+      "loss": 0.6606,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4895833432674408,
+      "rewards/margins": 0.08833334594964981,
+      "rewards/rejected": 0.4012499749660492,
+      "step": 470
+    },
+    {
+      "epoch": 0.24,
+      "grad_norm": 13.735665321350098,
+      "learning_rate": 7.599999999999999e-07,
+      "logits/chosen": 0.1455078125,
+      "logits/rejected": 0.1435546875,
+      "logps/chosen": -25.375,
+      "logps/rejected": -24.125,
+      "loss": 0.6963,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4545833170413971,
+      "rewards/margins": 0.01458333432674408,
+      "rewards/rejected": 0.4399999976158142,
+      "step": 480
+    },
+    {
+      "epoch": 0.245,
+      "grad_norm": 7.873671054840088,
+      "learning_rate": 7.55e-07,
+      "logits/chosen": 0.150390625,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -23.625,
+      "logps/rejected": -23.375,
+      "loss": 0.7072,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.48083335161209106,
+      "rewards/margins": -0.013333344832062721,
+      "rewards/rejected": 0.49416667222976685,
+      "step": 490
+    },
+    {
+      "epoch": 0.25,
+      "grad_norm": 8.703064918518066,
+      "learning_rate": 7.5e-07,
+      "logits/chosen": 0.14453125,
+      "logits/rejected": 0.1484375,
+      "logps/chosen": -24.5,
+      "logps/rejected": -23.875,
+      "loss": 0.6463,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.48624998331069946,
+      "rewards/margins": 0.12000002712011337,
+      "rewards/rejected": 0.36625000834465027,
+      "step": 500
+    },
+    {
+      "epoch": 0.255,
+      "grad_norm": 8.718635559082031,
+      "learning_rate": 7.45e-07,
+      "logits/chosen": 0.1591796875,
+      "logits/rejected": 0.1669921875,
+      "logps/chosen": -24.0,
+      "logps/rejected": -28.0,
+      "loss": 0.5839,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5375000238418579,
+      "rewards/margins": 0.2379167079925537,
+      "rewards/rejected": 0.2995833158493042,
+      "step": 510
+    },
+    {
+      "epoch": 0.26,
+      "grad_norm": 8.673093795776367,
+      "learning_rate": 7.4e-07,
+      "logits/chosen": 0.154296875,
+      "logits/rejected": 0.1630859375,
+      "logps/chosen": -21.5,
+      "logps/rejected": -27.75,
+      "loss": 0.6522,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5000000596046448,
+      "rewards/margins": 0.10500003397464752,
+      "rewards/rejected": 0.39500004053115845,
+      "step": 520
+    },
+    {
+      "epoch": 0.265,
+      "grad_norm": 8.47291088104248,
+      "learning_rate": 7.35e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.177734375,
+      "logps/chosen": -23.75,
+      "logps/rejected": -28.0,
+      "loss": 0.5914,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.54708331823349,
+      "rewards/margins": 0.21958334743976593,
+      "rewards/rejected": 0.32750001549720764,
+      "step": 530
+    },
+    {
+      "epoch": 0.27,
+      "grad_norm": 7.0806474685668945,
+      "learning_rate": 7.3e-07,
+      "logits/chosen": 0.1728515625,
+      "logits/rejected": 0.16015625,
+      "logps/chosen": -22.25,
+      "logps/rejected": -26.875,
+      "loss": 0.707,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.50041663646698,
+      "rewards/margins": -0.005833321716636419,
+      "rewards/rejected": 0.5062500238418579,
+      "step": 540
+    },
+    {
+      "epoch": 0.275,
+      "grad_norm": 8.11788558959961,
+      "learning_rate": 7.249999999999999e-07,
+      "logits/chosen": 0.1611328125,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -23.625,
+      "logps/rejected": -25.5,
+      "loss": 0.654,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5933333039283752,
+      "rewards/margins": 0.1041666716337204,
+      "rewards/rejected": 0.48916664719581604,
+      "step": 550
+    },
+    {
+      "epoch": 0.28,
+      "grad_norm": 9.295035362243652,
+      "learning_rate": 7.2e-07,
+      "logits/chosen": 0.150390625,
+      "logits/rejected": 0.158203125,
+      "logps/chosen": -23.0,
+      "logps/rejected": -27.0,
+      "loss": 0.6366,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4766666293144226,
+      "rewards/margins": 0.13458330929279327,
+      "rewards/rejected": 0.3420833647251129,
+      "step": 560
+    },
+    {
+      "epoch": 0.285,
+      "grad_norm": 10.309056282043457,
+      "learning_rate": 7.149999999999999e-07,
+      "logits/chosen": 0.1337890625,
+      "logits/rejected": 0.1396484375,
+      "logps/chosen": -23.125,
+      "logps/rejected": -26.75,
+      "loss": 0.7594,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.42083340883255005,
+      "rewards/margins": -0.09708327054977417,
+      "rewards/rejected": 0.5179166197776794,
+      "step": 570
+    },
+    {
+      "epoch": 0.29,
+      "grad_norm": 0.14241382479667664,
+      "learning_rate": 7.1e-07,
+      "logits/chosen": 0.1484375,
+      "logits/rejected": 0.14453125,
+      "logps/chosen": -23.625,
+      "logps/rejected": -24.25,
+      "loss": 0.6574,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5345833897590637,
+      "rewards/margins": 0.08291669934988022,
+      "rewards/rejected": 0.4516666829586029,
+      "step": 580
+    },
+    {
+      "epoch": 0.295,
+      "grad_norm": 10.084491729736328,
+      "learning_rate": 7.049999999999999e-07,
+      "logits/chosen": 0.125,
+      "logits/rejected": 0.1328125,
+      "logps/chosen": -25.5,
+      "logps/rejected": -22.5,
+      "loss": 0.611,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6287499666213989,
+      "rewards/margins": 0.18708336353302002,
+      "rewards/rejected": 0.4416666030883789,
+      "step": 590
+    },
+    {
+      "epoch": 0.3,
+      "grad_norm": 11.172276496887207,
+      "learning_rate": 7e-07,
+      "logits/chosen": 0.1513671875,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -25.25,
+      "logps/rejected": -27.375,
+      "loss": 0.7312,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.40791669487953186,
+      "rewards/margins": -0.04791667312383652,
+      "rewards/rejected": 0.455833375453949,
+      "step": 600
+    },
+    {
+      "epoch": 0.305,
+      "grad_norm": 8.67631721496582,
+      "learning_rate": 6.949999999999999e-07,
+      "logits/chosen": 0.14453125,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -23.0,
+      "logps/rejected": -22.625,
+      "loss": 0.667,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.596666693687439,
+      "rewards/margins": 0.06541667133569717,
+      "rewards/rejected": 0.53125,
+      "step": 610
+    },
+    {
+      "epoch": 0.31,
+      "grad_norm": 4.066090106964111,
+      "learning_rate": 6.9e-07,
+      "logits/chosen": 0.1357421875,
+      "logits/rejected": 0.1376953125,
+      "logps/chosen": -21.5,
+      "logps/rejected": -25.0,
+      "loss": 0.681,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.5116667151451111,
+      "rewards/margins": 0.036666709929704666,
+      "rewards/rejected": 0.47499996423721313,
+      "step": 620
+    },
+    {
+      "epoch": 0.315,
+      "grad_norm": 0.14283305406570435,
+      "learning_rate": 6.85e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.15234375,
+      "logps/chosen": -22.75,
+      "logps/rejected": -25.375,
+      "loss": 0.6609,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4908333420753479,
+      "rewards/margins": 0.08125002682209015,
+      "rewards/rejected": 0.40958333015441895,
+      "step": 630
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 9.351428985595703,
+      "learning_rate": 6.800000000000001e-07,
+      "logits/chosen": 0.154296875,
+      "logits/rejected": 0.166015625,
+      "logps/chosen": -22.375,
+      "logps/rejected": -26.625,
+      "loss": 0.6408,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5083333253860474,
+      "rewards/margins": 0.13458332419395447,
+      "rewards/rejected": 0.3737500309944153,
+      "step": 640
+    },
+    {
+      "epoch": 0.325,
+      "grad_norm": 4.95792818069458,
+      "learning_rate": 6.75e-07,
+      "logits/chosen": 0.146484375,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -26.125,
+      "logps/rejected": -24.125,
+      "loss": 0.6434,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5412499904632568,
+      "rewards/margins": 0.11500000953674316,
+      "rewards/rejected": 0.42624998092651367,
+      "step": 650
+    },
+    {
+      "epoch": 0.33,
+      "grad_norm": 4.454109191894531,
+      "learning_rate": 6.7e-07,
+      "logits/chosen": 0.150390625,
+      "logits/rejected": 0.1640625,
+      "logps/chosen": -23.125,
+      "logps/rejected": -25.25,
+      "loss": 0.6704,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4554166793823242,
+      "rewards/margins": 0.06125000864267349,
+      "rewards/rejected": 0.3941666781902313,
+      "step": 660
+    },
+    {
+      "epoch": 0.335,
+      "grad_norm": 21.211313247680664,
+      "learning_rate": 6.65e-07,
+      "logits/chosen": 0.1572265625,
+      "logits/rejected": 0.14453125,
+      "logps/chosen": -24.0,
+      "logps/rejected": -28.375,
+      "loss": 0.7321,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.42875003814697266,
+      "rewards/margins": -0.05583333224058151,
+      "rewards/rejected": 0.48458331823349,
+      "step": 670
+    },
+    {
+      "epoch": 0.34,
+      "grad_norm": 10.264230728149414,
+      "learning_rate": 6.6e-07,
+      "logits/chosen": 0.1533203125,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -23.375,
+      "logps/rejected": -24.5,
+      "loss": 0.6593,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.53125,
+      "rewards/margins": 0.09666665643453598,
+      "rewards/rejected": 0.4345833659172058,
+      "step": 680
+    },
+    {
+      "epoch": 0.345,
+      "grad_norm": 8.515113830566406,
+      "learning_rate": 6.55e-07,
+      "logits/chosen": 0.1318359375,
+      "logits/rejected": 0.1318359375,
+      "logps/chosen": -24.25,
+      "logps/rejected": -26.375,
+      "loss": 0.5939,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5245833992958069,
+      "rewards/margins": 0.22291669249534607,
+      "rewards/rejected": 0.3016666769981384,
+      "step": 690
+    },
+    {
+      "epoch": 0.35,
+      "grad_norm": 8.171746253967285,
+      "learning_rate": 6.5e-07,
+      "logits/chosen": 0.1396484375,
+      "logits/rejected": 0.1298828125,
+      "logps/chosen": -25.0,
+      "logps/rejected": -26.25,
+      "loss": 0.6543,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.4949999749660492,
+      "rewards/margins": 0.09958335757255554,
+      "rewards/rejected": 0.3954166769981384,
+      "step": 700
+    },
+    {
+      "epoch": 0.355,
+      "grad_norm": 9.327075958251953,
+      "learning_rate": 6.45e-07,
+      "logits/chosen": 0.140625,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -25.25,
+      "logps/rejected": -26.25,
+      "loss": 0.641,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5345832705497742,
+      "rewards/margins": 0.13500000536441803,
+      "rewards/rejected": 0.3995833396911621,
+      "step": 710
+    },
+    {
+      "epoch": 0.36,
+      "grad_norm": 17.877674102783203,
+      "learning_rate": 6.4e-07,
+      "logits/chosen": 0.126953125,
+      "logits/rejected": 0.1591796875,
+      "logps/chosen": -24.625,
+      "logps/rejected": -26.875,
+      "loss": 0.6205,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5750000476837158,
+      "rewards/margins": 0.16666670143604279,
+      "rewards/rejected": 0.40833336114883423,
+      "step": 720
+    },
+    {
+      "epoch": 0.365,
+      "grad_norm": 7.1101908683776855,
+      "learning_rate": 6.35e-07,
+      "logits/chosen": 0.1279296875,
+      "logits/rejected": 0.1708984375,
+      "logps/chosen": -27.625,
+      "logps/rejected": -25.375,
+      "loss": 0.569,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6341666579246521,
+      "rewards/margins": 0.2841666638851166,
+      "rewards/rejected": 0.3499999940395355,
+      "step": 730
+    },
+    {
+      "epoch": 0.37,
+      "grad_norm": 9.20040225982666,
+      "learning_rate": 6.3e-07,
+      "logits/chosen": 0.12890625,
+      "logits/rejected": 0.1728515625,
+      "logps/chosen": -26.625,
+      "logps/rejected": -25.25,
+      "loss": 0.6237,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6141666173934937,
+      "rewards/margins": 0.16708336770534515,
+      "rewards/rejected": 0.44708332419395447,
+      "step": 740
+    },
+    {
+      "epoch": 0.375,
+      "grad_norm": 11.740049362182617,
+      "learning_rate": 6.249999999999999e-07,
+      "logits/chosen": 0.1357421875,
+      "logits/rejected": 0.1474609375,
+      "logps/chosen": -22.75,
+      "logps/rejected": -25.375,
+      "loss": 0.666,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4658333659172058,
+      "rewards/margins": 0.07958336174488068,
+      "rewards/rejected": 0.38625001907348633,
+      "step": 750
+    },
+    {
+      "epoch": 0.38,
+      "grad_norm": 5.444693088531494,
+      "learning_rate": 6.2e-07,
+      "logits/chosen": 0.140625,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -24.25,
+      "logps/rejected": -29.375,
+      "loss": 0.6279,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5208333730697632,
+      "rewards/margins": 0.17874999344348907,
+      "rewards/rejected": 0.34208330512046814,
+      "step": 760
+    },
+    {
+      "epoch": 0.385,
+      "grad_norm": 8.790399551391602,
+      "learning_rate": 6.149999999999999e-07,
+      "logits/chosen": 0.1435546875,
+      "logits/rejected": 0.158203125,
+      "logps/chosen": -25.25,
+      "logps/rejected": -29.25,
+      "loss": 0.6257,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5674999952316284,
+      "rewards/margins": 0.16583338379859924,
+      "rewards/rejected": 0.40166664123535156,
+      "step": 770
+    },
+    {
+      "epoch": 0.39,
+      "grad_norm": 7.73515510559082,
+      "learning_rate": 6.1e-07,
+      "logits/chosen": 0.126953125,
+      "logits/rejected": 0.1484375,
+      "logps/chosen": -22.5,
+      "logps/rejected": -27.0,
+      "loss": 0.6446,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.49833330512046814,
+      "rewards/margins": 0.11666667461395264,
+      "rewards/rejected": 0.3816666305065155,
+      "step": 780
+    },
+    {
+      "epoch": 0.395,
+      "grad_norm": 8.817035675048828,
+      "learning_rate": 6.049999999999999e-07,
+      "logits/chosen": 0.1181640625,
+      "logits/rejected": 0.134765625,
+      "logps/chosen": -24.5,
+      "logps/rejected": -24.875,
+      "loss": 0.7179,
+      "rewards/accuracies": 0.30000001192092896,
+      "rewards/chosen": 0.5083333253860474,
+      "rewards/margins": -0.021250015124678612,
+      "rewards/rejected": 0.5295833349227905,
+      "step": 790
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 7.170980453491211,
+      "learning_rate": 6e-07,
+      "logits/chosen": 0.134765625,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -21.5,
+      "logps/rejected": -24.875,
+      "loss": 0.5959,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6200000047683716,
+      "rewards/margins": 0.23958337306976318,
+      "rewards/rejected": 0.3804166615009308,
+      "step": 800
+    },
+    {
+      "epoch": 0.405,
+      "grad_norm": 7.622363090515137,
+      "learning_rate": 5.949999999999999e-07,
+      "logits/chosen": 0.134765625,
+      "logits/rejected": 0.1591796875,
+      "logps/chosen": -24.0,
+      "logps/rejected": -23.125,
+      "loss": 0.7351,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4241666793823242,
+      "rewards/margins": -0.05791662260890007,
+      "rewards/rejected": 0.4820833206176758,
+      "step": 810
+    },
+    {
+      "epoch": 0.41,
+      "grad_norm": 8.68386459350586,
+      "learning_rate": 5.9e-07,
+      "logits/chosen": 0.1376953125,
+      "logits/rejected": 0.142578125,
+      "logps/chosen": -21.5,
+      "logps/rejected": -24.0,
+      "loss": 0.6807,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5554167032241821,
+      "rewards/margins": 0.06499999761581421,
+      "rewards/rejected": 0.4904167056083679,
+      "step": 820
+    },
+    {
+      "epoch": 0.415,
+      "grad_norm": 5.363429546356201,
+      "learning_rate": 5.849999999999999e-07,
+      "logits/chosen": 0.130859375,
+      "logits/rejected": 0.1455078125,
+      "logps/chosen": -22.5,
+      "logps/rejected": -25.5,
+      "loss": 0.5821,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6333333253860474,
+      "rewards/margins": 0.26083335280418396,
+      "rewards/rejected": 0.3725000023841858,
+      "step": 830
+    },
+    {
+      "epoch": 0.42,
+      "grad_norm": 14.140009880065918,
+      "learning_rate": 5.8e-07,
+      "logits/chosen": 0.1376953125,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -21.25,
+      "logps/rejected": -24.5,
+      "loss": 0.7382,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.4254167079925537,
+      "rewards/margins": -0.05708333104848862,
+      "rewards/rejected": 0.48249998688697815,
+      "step": 840
+    },
+    {
+      "epoch": 0.425,
+      "grad_norm": 9.647602081298828,
+      "learning_rate": 5.749999999999999e-07,
+      "logits/chosen": 0.1435546875,
+      "logits/rejected": 0.146484375,
+      "logps/chosen": -21.5,
+      "logps/rejected": -22.375,
+      "loss": 0.6062,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5841667056083679,
+      "rewards/margins": 0.20625002682209015,
+      "rewards/rejected": 0.3779166340827942,
+      "step": 850
+    },
+    {
+      "epoch": 0.43,
+      "grad_norm": 10.13477897644043,
+      "learning_rate": 5.699999999999999e-07,
+      "logits/chosen": 0.126953125,
+      "logits/rejected": 0.158203125,
+      "logps/chosen": -22.625,
+      "logps/rejected": -26.5,
+      "loss": 0.615,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.5879167318344116,
+      "rewards/margins": 0.17499999701976776,
+      "rewards/rejected": 0.41291671991348267,
+      "step": 860
+    },
+    {
+      "epoch": 0.435,
+      "grad_norm": 10.309593200683594,
+      "learning_rate": 5.649999999999999e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.1494140625,
+      "logps/chosen": -22.625,
+      "logps/rejected": -27.5,
+      "loss": 0.5851,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.5762499570846558,
+      "rewards/margins": 0.24666666984558105,
+      "rewards/rejected": 0.3295833468437195,
+      "step": 870
+    },
+    {
+      "epoch": 0.44,
+      "grad_norm": 0.1319269835948944,
+      "learning_rate": 5.6e-07,
+      "logits/chosen": 0.1201171875,
+      "logits/rejected": 0.1591796875,
+      "logps/chosen": -26.5,
+      "logps/rejected": -27.5,
+      "loss": 0.6642,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.4504167139530182,
+      "rewards/margins": 0.07875002920627594,
+      "rewards/rejected": 0.37166666984558105,
+      "step": 880
+    },
+    {
+      "epoch": 0.445,
+      "grad_norm": 9.670498847961426,
+      "learning_rate": 5.55e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.125,
+      "logps/chosen": -23.5,
+      "logps/rejected": -27.0,
+      "loss": 0.6028,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5729166269302368,
+      "rewards/margins": 0.21833336353302002,
+      "rewards/rejected": 0.3545833230018616,
+      "step": 890
+    },
+    {
+      "epoch": 0.45,
+      "grad_norm": 10.779263496398926,
+      "learning_rate": 5.5e-07,
+      "logits/chosen": 0.1240234375,
+      "logits/rejected": 0.142578125,
+      "logps/chosen": -25.25,
+      "logps/rejected": -25.625,
+      "loss": 0.6035,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5925000309944153,
+      "rewards/margins": 0.22166666388511658,
+      "rewards/rejected": 0.3708333373069763,
+      "step": 900
+    },
+    {
+      "epoch": 0.455,
+      "grad_norm": 7.300227165222168,
+      "learning_rate": 5.45e-07,
+      "logits/chosen": 0.13671875,
+      "logits/rejected": 0.150390625,
+      "logps/chosen": -22.625,
+      "logps/rejected": -24.0,
+      "loss": 0.5389,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.7162500619888306,
+      "rewards/margins": 0.35708335041999817,
+      "rewards/rejected": 0.35916668176651,
+      "step": 910
+    },
+    {
+      "epoch": 0.46,
+      "grad_norm": 9.367812156677246,
+      "learning_rate": 5.4e-07,
+      "logits/chosen": 0.1298828125,
+      "logits/rejected": 0.1533203125,
+      "logps/chosen": -27.0,
+      "logps/rejected": -25.0,
+      "loss": 0.556,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.659166693687439,
+      "rewards/margins": 0.30250006914138794,
+      "rewards/rejected": 0.356666624546051,
+      "step": 920
+    },
+    {
+      "epoch": 0.465,
+      "grad_norm": 26.304065704345703,
+      "learning_rate": 5.35e-07,
+      "logits/chosen": 0.1279296875,
+      "logits/rejected": 0.13671875,
+      "logps/chosen": -25.75,
+      "logps/rejected": -25.25,
+      "loss": 0.7087,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4362499713897705,
+      "rewards/margins": 0.016250004991889,
+      "rewards/rejected": 0.41999998688697815,
+      "step": 930
+    },
+    {
+      "epoch": 0.47,
+      "grad_norm": 13.385437965393066,
+      "learning_rate": 5.3e-07,
+      "logits/chosen": 0.13671875,
+      "logits/rejected": 0.1435546875,
+      "logps/chosen": -23.75,
+      "logps/rejected": -26.25,
+      "loss": 0.5893,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6487500071525574,
+      "rewards/margins": 0.24041669070720673,
+      "rewards/rejected": 0.40833330154418945,
+      "step": 940
+    },
+    {
+      "epoch": 0.475,
+      "grad_norm": 7.525325298309326,
+      "learning_rate": 5.25e-07,
+      "logits/chosen": 0.1318359375,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -24.625,
+      "logps/rejected": -25.625,
+      "loss": 0.6107,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5858333706855774,
+      "rewards/margins": 0.20375001430511475,
+      "rewards/rejected": 0.38208335638046265,
+      "step": 950
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 11.17664623260498,
+      "learning_rate": 5.2e-07,
+      "logits/chosen": 0.1396484375,
+      "logits/rejected": 0.1416015625,
+      "logps/chosen": -23.5,
+      "logps/rejected": -25.0,
+      "loss": 0.5759,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6720834374427795,
+      "rewards/margins": 0.27916669845581055,
+      "rewards/rejected": 0.3929166793823242,
+      "step": 960
+    },
+    {
+      "epoch": 0.485,
+      "grad_norm": 0.1770247519016266,
+      "learning_rate": 5.149999999999999e-07,
+      "logits/chosen": 0.1474609375,
+      "logits/rejected": 0.1474609375,
+      "logps/chosen": -21.0,
+      "logps/rejected": -25.75,
+      "loss": 0.6972,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.4833333492279053,
+      "rewards/margins": 0.05624999850988388,
+      "rewards/rejected": 0.4270833432674408,
+      "step": 970
+    },
+    {
+      "epoch": 0.49,
+      "grad_norm": 14.87246322631836,
+      "learning_rate": 5.1e-07,
+      "logits/chosen": 0.1376953125,
+      "logits/rejected": 0.16015625,
+      "logps/chosen": -25.875,
+      "logps/rejected": -25.875,
+      "loss": 0.5926,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.5495833158493042,
+      "rewards/margins": 0.23791666328907013,
+      "rewards/rejected": 0.31166666746139526,
+      "step": 980
+    },
+    {
+      "epoch": 0.495,
+      "grad_norm": 6.852414131164551,
+      "learning_rate": 5.049999999999999e-07,
+      "logits/chosen": 0.1259765625,
+      "logits/rejected": 0.12890625,
+      "logps/chosen": -23.75,
+      "logps/rejected": -26.125,
+      "loss": 0.6779,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5162500143051147,
+      "rewards/margins": 0.06833334267139435,
+      "rewards/rejected": 0.4479166567325592,
+      "step": 990
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 10.259302139282227,
+      "learning_rate": 5e-07,
+      "logits/chosen": 0.13671875,
+      "logits/rejected": 0.134765625,
+      "logps/chosen": -21.875,
+      "logps/rejected": -27.25,
+      "loss": 0.6163,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5020833015441895,
+      "rewards/margins": 0.17458339035511017,
+      "rewards/rejected": 0.3274999260902405,
+      "step": 1000
+    },
+    {
+      "epoch": 0.505,
+      "grad_norm": 9.82231330871582,
+      "learning_rate": 4.95e-07,
+      "logits/chosen": 0.109375,
+      "logits/rejected": 0.1259765625,
+      "logps/chosen": -27.75,
+      "logps/rejected": -25.75,
+      "loss": 0.5508,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6070833206176758,
+      "rewards/margins": 0.3604167103767395,
+      "rewards/rejected": 0.24666666984558105,
+      "step": 1010
+    },
+    {
+      "epoch": 0.51,
+      "grad_norm": 7.75898551940918,
+      "learning_rate": 4.9e-07,
+      "logits/chosen": 0.126953125,
+      "logits/rejected": 0.14453125,
+      "logps/chosen": -21.75,
+      "logps/rejected": -26.125,
+      "loss": 0.6278,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5449999570846558,
+      "rewards/margins": 0.16374999284744263,
+      "rewards/rejected": 0.3812499940395355,
+      "step": 1020
+    },
+    {
+      "epoch": 0.515,
+      "grad_norm": 8.985075950622559,
+      "learning_rate": 4.85e-07,
+      "logits/chosen": 0.1494140625,
+      "logits/rejected": 0.1494140625,
+      "logps/chosen": -20.75,
+      "logps/rejected": -25.0,
+      "loss": 0.5819,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6354166865348816,
+      "rewards/margins": 0.2629167139530182,
+      "rewards/rejected": 0.3724999725818634,
+      "step": 1030
+    },
+    {
+      "epoch": 0.52,
+      "grad_norm": 3.210916757583618,
+      "learning_rate": 4.8e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.13671875,
+      "logps/chosen": -20.75,
+      "logps/rejected": -28.625,
+      "loss": 0.559,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6966665983200073,
+      "rewards/margins": 0.32375001907348633,
+      "rewards/rejected": 0.37291663885116577,
+      "step": 1040
+    },
+    {
+      "epoch": 0.525,
+      "grad_norm": 9.324535369873047,
+      "learning_rate": 4.7499999999999995e-07,
+      "logits/chosen": 0.1396484375,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -24.5,
+      "logps/rejected": -26.375,
+      "loss": 0.6807,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.492499977350235,
+      "rewards/margins": 0.05583333969116211,
+      "rewards/rejected": 0.43666666746139526,
+      "step": 1050
+    },
+    {
+      "epoch": 0.53,
+      "grad_norm": 7.928718090057373,
+      "learning_rate": 4.6999999999999995e-07,
+      "logits/chosen": 0.1357421875,
+      "logits/rejected": 0.146484375,
+      "logps/chosen": -24.375,
+      "logps/rejected": -27.25,
+      "loss": 0.6088,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5270833373069763,
+      "rewards/margins": 0.24166671931743622,
+      "rewards/rejected": 0.2854166328907013,
+      "step": 1060
+    },
+    {
+      "epoch": 0.535,
+      "grad_norm": 11.371706008911133,
+      "learning_rate": 4.65e-07,
+      "logits/chosen": 0.1533203125,
+      "logits/rejected": 0.1552734375,
+      "logps/chosen": -23.875,
+      "logps/rejected": -25.0,
+      "loss": 0.6584,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5083333253860474,
+      "rewards/margins": 0.11083336174488068,
+      "rewards/rejected": 0.39750000834465027,
+      "step": 1070
+    },
+    {
+      "epoch": 0.54,
+      "grad_norm": 5.549586772918701,
+      "learning_rate": 4.6e-07,
+      "logits/chosen": 0.14453125,
+      "logits/rejected": 0.1591796875,
+      "logps/chosen": -26.875,
+      "logps/rejected": -25.5,
+      "loss": 0.6492,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5445833206176758,
+      "rewards/margins": 0.12416665256023407,
+      "rewards/rejected": 0.4204166531562805,
+      "step": 1080
+    },
+    {
+      "epoch": 0.545,
+      "grad_norm": 10.966238975524902,
+      "learning_rate": 4.55e-07,
+      "logits/chosen": 0.1513671875,
+      "logits/rejected": 0.146484375,
+      "logps/chosen": -21.75,
+      "logps/rejected": -28.125,
+      "loss": 0.5464,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.6316666603088379,
+      "rewards/margins": 0.3445833623409271,
+      "rewards/rejected": 0.28708332777023315,
+      "step": 1090
+    },
+    {
+      "epoch": 0.55,
+      "grad_norm": 8.734814643859863,
+      "learning_rate": 4.5e-07,
+      "logits/chosen": 0.12109375,
+      "logits/rejected": 0.1240234375,
+      "logps/chosen": -23.25,
+      "logps/rejected": -26.625,
+      "loss": 0.6714,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5304166674613953,
+      "rewards/margins": 0.09208334982395172,
+      "rewards/rejected": 0.4383333623409271,
+      "step": 1100
+    },
+    {
+      "epoch": 0.555,
+      "grad_norm": 11.257500648498535,
+      "learning_rate": 4.45e-07,
+      "logits/chosen": 0.130859375,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -20.875,
+      "logps/rejected": -29.5,
+      "loss": 0.6018,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5216666460037231,
+      "rewards/margins": 0.23916669189929962,
+      "rewards/rejected": 0.2824999988079071,
+      "step": 1110
+    },
+    {
+      "epoch": 0.56,
+      "grad_norm": 8.232009887695312,
+      "learning_rate": 4.3999999999999997e-07,
+      "logits/chosen": 0.12353515625,
+      "logits/rejected": 0.1435546875,
+      "logps/chosen": -22.875,
+      "logps/rejected": -27.875,
+      "loss": 0.5176,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.65625,
+      "rewards/margins": 0.4304167330265045,
+      "rewards/rejected": 0.22583332657814026,
+      "step": 1120
+    },
+    {
+      "epoch": 0.565,
+      "grad_norm": 0.21383002400398254,
+      "learning_rate": 4.3499999999999996e-07,
+      "logits/chosen": 0.123046875,
+      "logits/rejected": 0.1513671875,
+      "logps/chosen": -26.75,
+      "logps/rejected": -28.5,
+      "loss": 0.608,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5508333444595337,
+      "rewards/margins": 0.1966666728258133,
+      "rewards/rejected": 0.3541666865348816,
+      "step": 1130
+    },
+    {
+      "epoch": 0.57,
+      "grad_norm": 6.056092739105225,
+      "learning_rate": 4.2999999999999996e-07,
+      "logits/chosen": 0.1298828125,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -23.375,
+      "logps/rejected": -26.625,
+      "loss": 0.6301,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.49791669845581055,
+      "rewards/margins": 0.18666666746139526,
+      "rewards/rejected": 0.3112500011920929,
+      "step": 1140
+    },
+    {
+      "epoch": 0.575,
+      "grad_norm": 9.0851411819458,
+      "learning_rate": 4.2499999999999995e-07,
+      "logits/chosen": 0.1259765625,
+      "logits/rejected": 0.142578125,
+      "logps/chosen": -23.0,
+      "logps/rejected": -23.875,
+      "loss": 0.5912,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6379166841506958,
+      "rewards/margins": 0.25833332538604736,
+      "rewards/rejected": 0.37958329916000366,
+      "step": 1150
+    },
+    {
+      "epoch": 0.58,
+      "grad_norm": 14.7762451171875,
+      "learning_rate": 4.1999999999999995e-07,
+      "logits/chosen": 0.10888671875,
+      "logits/rejected": 0.1220703125,
+      "logps/chosen": -25.5,
+      "logps/rejected": -21.0,
+      "loss": 0.6191,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.54666668176651,
+      "rewards/margins": 0.17666670680046082,
+      "rewards/rejected": 0.3699999451637268,
+      "step": 1160
+    },
+    {
+      "epoch": 0.585,
+      "grad_norm": 0.1491793990135193,
+      "learning_rate": 4.1499999999999994e-07,
+      "logits/chosen": 0.119140625,
+      "logits/rejected": 0.130859375,
+      "logps/chosen": -23.625,
+      "logps/rejected": -28.5,
+      "loss": 0.5726,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5162500143051147,
+      "rewards/margins": 0.29625004529953003,
+      "rewards/rejected": 0.2199999839067459,
+      "step": 1170
+    },
+    {
+      "epoch": 0.59,
+      "grad_norm": 9.796224594116211,
+      "learning_rate": 4.0999999999999994e-07,
+      "logits/chosen": 0.12060546875,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -21.625,
+      "logps/rejected": -27.0,
+      "loss": 0.6273,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5412500500679016,
+      "rewards/margins": 0.182083398103714,
+      "rewards/rejected": 0.35916668176651,
+      "step": 1180
+    },
+    {
+      "epoch": 0.595,
+      "grad_norm": 15.499927520751953,
+      "learning_rate": 4.05e-07,
+      "logits/chosen": 0.1298828125,
+      "logits/rejected": 0.1435546875,
+      "logps/chosen": -22.375,
+      "logps/rejected": -23.875,
+      "loss": 0.6713,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5795833468437195,
+      "rewards/margins": 0.06291668117046356,
+      "rewards/rejected": 0.5166667103767395,
+      "step": 1190
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 7.725632667541504,
+      "learning_rate": 4e-07,
+      "logits/chosen": 0.11083984375,
+      "logits/rejected": 0.1240234375,
+      "logps/chosen": -23.375,
+      "logps/rejected": -26.625,
+      "loss": 0.5437,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6620832681655884,
+      "rewards/margins": 0.3866667151451111,
+      "rewards/rejected": 0.27541667222976685,
+      "step": 1200
+    },
+    {
+      "epoch": 0.605,
+      "grad_norm": 8.964713096618652,
+      "learning_rate": 3.95e-07,
+      "logits/chosen": 0.173828125,
+      "logits/rejected": 0.1484375,
+      "logps/chosen": -23.125,
+      "logps/rejected": -26.5,
+      "loss": 0.6108,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.5612499713897705,
+      "rewards/margins": 0.197500079870224,
+      "rewards/rejected": 0.3637499213218689,
+      "step": 1210
+    },
+    {
+      "epoch": 0.61,
+      "grad_norm": 20.50394058227539,
+      "learning_rate": 3.8999999999999997e-07,
+      "logits/chosen": 0.1103515625,
+      "logits/rejected": 0.1181640625,
+      "logps/chosen": -25.625,
+      "logps/rejected": -24.0,
+      "loss": 0.6494,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4987500309944153,
+      "rewards/margins": 0.1054166927933693,
+      "rewards/rejected": 0.3933333158493042,
+      "step": 1220
+    },
+    {
+      "epoch": 0.615,
+      "grad_norm": 6.254834175109863,
+      "learning_rate": 3.8499999999999997e-07,
+      "logits/chosen": 0.12158203125,
+      "logits/rejected": 0.1318359375,
+      "logps/chosen": -28.375,
+      "logps/rejected": -23.375,
+      "loss": 0.6895,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.45124998688697815,
+      "rewards/margins": 0.07499999552965164,
+      "rewards/rejected": 0.3762499690055847,
+      "step": 1230
+    },
+    {
+      "epoch": 0.62,
+      "grad_norm": 6.0932230949401855,
+      "learning_rate": 3.7999999999999996e-07,
+      "logits/chosen": 0.126953125,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -23.875,
+      "logps/rejected": -24.875,
+      "loss": 0.5932,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6037500500679016,
+      "rewards/margins": 0.2616666853427887,
+      "rewards/rejected": 0.34208330512046814,
+      "step": 1240
+    },
+    {
+      "epoch": 0.625,
+      "grad_norm": 10.014700889587402,
+      "learning_rate": 3.75e-07,
+      "logits/chosen": 0.1162109375,
+      "logits/rejected": 0.11328125,
+      "logps/chosen": -25.875,
+      "logps/rejected": -25.375,
+      "loss": 0.7547,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.44583335518836975,
+      "rewards/margins": -0.03041663207113743,
+      "rewards/rejected": 0.4762499928474426,
+      "step": 1250
+    },
+    {
+      "epoch": 0.63,
+      "grad_norm": 3.394895076751709,
+      "learning_rate": 3.7e-07,
+      "logits/chosen": 0.1513671875,
+      "logits/rejected": 0.11767578125,
+      "logps/chosen": -22.625,
+      "logps/rejected": -25.375,
+      "loss": 0.6522,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5591666102409363,
+      "rewards/margins": 0.14791670441627502,
+      "rewards/rejected": 0.4112500250339508,
+      "step": 1260
+    },
+    {
+      "epoch": 0.635,
+      "grad_norm": 26.48250389099121,
+      "learning_rate": 3.65e-07,
+      "logits/chosen": 0.1435546875,
+      "logits/rejected": 0.14453125,
+      "logps/chosen": -23.25,
+      "logps/rejected": -30.25,
+      "loss": 0.572,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5804166793823242,
+      "rewards/margins": 0.3162499964237213,
+      "rewards/rejected": 0.2641666829586029,
+      "step": 1270
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 16.003503799438477,
+      "learning_rate": 3.6e-07,
+      "logits/chosen": 0.1162109375,
+      "logits/rejected": 0.130859375,
+      "logps/chosen": -22.75,
+      "logps/rejected": -28.5,
+      "loss": 0.6133,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.45458340644836426,
+      "rewards/margins": 0.20916669070720673,
+      "rewards/rejected": 0.24541664123535156,
+      "step": 1280
+    },
+    {
+      "epoch": 0.645,
+      "grad_norm": 10.279973030090332,
+      "learning_rate": 3.55e-07,
+      "logits/chosen": 0.1123046875,
+      "logits/rejected": 0.1318359375,
+      "logps/chosen": -22.0,
+      "logps/rejected": -24.5,
+      "loss": 0.5077,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 0.7700001001358032,
+      "rewards/margins": 0.42500004172325134,
+      "rewards/rejected": 0.3449999988079071,
+      "step": 1290
+    },
+    {
+      "epoch": 0.65,
+      "grad_norm": 12.892542839050293,
+      "learning_rate": 3.5e-07,
+      "logits/chosen": 0.1337890625,
+      "logits/rejected": 0.1337890625,
+      "logps/chosen": -21.125,
+      "logps/rejected": -25.75,
+      "loss": 0.5661,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6541666388511658,
+      "rewards/margins": 0.3112500309944153,
+      "rewards/rejected": 0.34291666746139526,
+      "step": 1300
+    },
+    {
+      "epoch": 0.655,
+      "grad_norm": 11.426549911499023,
+      "learning_rate": 3.45e-07,
+      "logits/chosen": 0.1396484375,
+      "logits/rejected": 0.1494140625,
+      "logps/chosen": -22.75,
+      "logps/rejected": -27.625,
+      "loss": 0.5512,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6325000524520874,
+      "rewards/margins": 0.3691667318344116,
+      "rewards/rejected": 0.2633332908153534,
+      "step": 1310
+    },
+    {
+      "epoch": 0.66,
+      "grad_norm": 14.325943946838379,
+      "learning_rate": 3.4000000000000003e-07,
+      "logits/chosen": 0.107421875,
+      "logits/rejected": 0.12255859375,
+      "logps/chosen": -22.25,
+      "logps/rejected": -27.25,
+      "loss": 0.5381,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6320833563804626,
+      "rewards/margins": 0.37291669845581055,
+      "rewards/rejected": 0.2591666281223297,
+      "step": 1320
+    },
+    {
+      "epoch": 0.665,
+      "grad_norm": 10.824604988098145,
+      "learning_rate": 3.35e-07,
+      "logits/chosen": 0.13671875,
+      "logits/rejected": 0.1572265625,
+      "logps/chosen": -24.5,
+      "logps/rejected": -26.0,
+      "loss": 0.6308,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5329166650772095,
+      "rewards/margins": 0.1929166615009308,
+      "rewards/rejected": 0.3399999737739563,
+      "step": 1330
+    },
+    {
+      "epoch": 0.67,
+      "grad_norm": 16.04193687438965,
+      "learning_rate": 3.3e-07,
+      "logits/chosen": 0.125,
+      "logits/rejected": 0.130859375,
+      "logps/chosen": -22.375,
+      "logps/rejected": -29.25,
+      "loss": 0.5347,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6441666483879089,
+      "rewards/margins": 0.3795833885669708,
+      "rewards/rejected": 0.2645833194255829,
+      "step": 1340
+    },
+    {
+      "epoch": 0.675,
+      "grad_norm": 12.836771965026855,
+      "learning_rate": 3.25e-07,
+      "logits/chosen": 0.10546875,
+      "logits/rejected": 0.115234375,
+      "logps/chosen": -24.75,
+      "logps/rejected": -24.5,
+      "loss": 0.6179,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6162499189376831,
+      "rewards/margins": 0.20833337306976318,
+      "rewards/rejected": 0.4079166352748871,
+      "step": 1350
+    },
+    {
+      "epoch": 0.68,
+      "grad_norm": 18.027015686035156,
+      "learning_rate": 3.2e-07,
+      "logits/chosen": 0.1064453125,
+      "logits/rejected": 0.10986328125,
+      "logps/chosen": -25.375,
+      "logps/rejected": -24.375,
+      "loss": 0.7324,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.4612500071525574,
+      "rewards/margins": -0.027916649356484413,
+      "rewards/rejected": 0.48916664719581604,
+      "step": 1360
+    },
+    {
+      "epoch": 0.685,
+      "grad_norm": 11.233678817749023,
+      "learning_rate": 3.15e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -24.625,
+      "logps/rejected": -31.125,
+      "loss": 0.6677,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4462500214576721,
+      "rewards/margins": 0.0808333307504654,
+      "rewards/rejected": 0.36541667580604553,
+      "step": 1370
+    },
+    {
+      "epoch": 0.69,
+      "grad_norm": 16.14885139465332,
+      "learning_rate": 3.1e-07,
+      "logits/chosen": 0.1416015625,
+      "logits/rejected": 0.1328125,
+      "logps/chosen": -24.625,
+      "logps/rejected": -25.25,
+      "loss": 0.6476,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5849999785423279,
+      "rewards/margins": 0.11833332479000092,
+      "rewards/rejected": 0.46666663885116577,
+      "step": 1380
+    },
+    {
+      "epoch": 0.695,
+      "grad_norm": 10.956652641296387,
+      "learning_rate": 3.05e-07,
+      "logits/chosen": 0.12890625,
+      "logits/rejected": 0.12353515625,
+      "logps/chosen": -24.25,
+      "logps/rejected": -25.0,
+      "loss": 0.6605,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5516666769981384,
+      "rewards/margins": 0.10416670143604279,
+      "rewards/rejected": 0.44749999046325684,
+      "step": 1390
+    },
+    {
+      "epoch": 0.7,
+      "grad_norm": 13.609949111938477,
+      "learning_rate": 3e-07,
+      "logits/chosen": 0.1279296875,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -24.0,
+      "logps/rejected": -26.625,
+      "loss": 0.5317,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6195833086967468,
+      "rewards/margins": 0.37583333253860474,
+      "rewards/rejected": 0.24375002086162567,
+      "step": 1400
+    },
+    {
+      "epoch": 0.705,
+      "grad_norm": 7.815566062927246,
+      "learning_rate": 2.95e-07,
+      "logits/chosen": 0.125,
+      "logits/rejected": 0.1162109375,
+      "logps/chosen": -22.25,
+      "logps/rejected": -28.0,
+      "loss": 0.6546,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4570833742618561,
+      "rewards/margins": 0.1079166978597641,
+      "rewards/rejected": 0.3491666615009308,
+      "step": 1410
+    },
+    {
+      "epoch": 0.71,
+      "grad_norm": 0.23640184104442596,
+      "learning_rate": 2.9e-07,
+      "logits/chosen": 0.1455078125,
+      "logits/rejected": 0.142578125,
+      "logps/chosen": -24.875,
+      "logps/rejected": -24.625,
+      "loss": 0.7137,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.477083295583725,
+      "rewards/margins": 0.022916659712791443,
+      "rewards/rejected": 0.4541666507720947,
+      "step": 1420
+    },
+    {
+      "epoch": 0.715,
+      "grad_norm": 13.797260284423828,
+      "learning_rate": 2.8499999999999997e-07,
+      "logits/chosen": 0.11279296875,
+      "logits/rejected": 0.126953125,
+      "logps/chosen": -23.75,
+      "logps/rejected": -27.25,
+      "loss": 0.613,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6162500381469727,
+      "rewards/margins": 0.2625000476837158,
+      "rewards/rejected": 0.35374999046325684,
+      "step": 1430
+    },
+    {
+      "epoch": 0.72,
+      "grad_norm": 23.680862426757812,
+      "learning_rate": 2.8e-07,
+      "logits/chosen": 0.1201171875,
+      "logits/rejected": 0.11669921875,
+      "logps/chosen": -23.75,
+      "logps/rejected": -23.0,
+      "loss": 0.6882,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.65666663646698,
+      "rewards/margins": 0.047916680574417114,
+      "rewards/rejected": 0.6087499856948853,
+      "step": 1440
+    },
+    {
+      "epoch": 0.725,
+      "grad_norm": 14.350584030151367,
+      "learning_rate": 2.75e-07,
+      "logits/chosen": 0.1318359375,
+      "logits/rejected": 0.1259765625,
+      "logps/chosen": -22.75,
+      "logps/rejected": -25.375,
+      "loss": 0.6801,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5525000095367432,
+      "rewards/margins": 0.06541669368743896,
+      "rewards/rejected": 0.4870833456516266,
+      "step": 1450
+    },
+    {
+      "epoch": 0.73,
+      "grad_norm": 9.058170318603516,
+      "learning_rate": 2.7e-07,
+      "logits/chosen": 0.11572265625,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -22.375,
+      "logps/rejected": -24.25,
+      "loss": 0.6003,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5929166674613953,
+      "rewards/margins": 0.2562499940395355,
+      "rewards/rejected": 0.33666664361953735,
+      "step": 1460
+    },
+    {
+      "epoch": 0.735,
+      "grad_norm": 19.62697410583496,
+      "learning_rate": 2.65e-07,
+      "logits/chosen": 0.12060546875,
+      "logits/rejected": 0.11962890625,
+      "logps/chosen": -22.625,
+      "logps/rejected": -29.75,
+      "loss": 0.626,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5200000405311584,
+      "rewards/margins": 0.18458335101604462,
+      "rewards/rejected": 0.33541664481163025,
+      "step": 1470
+    },
+    {
+      "epoch": 0.74,
+      "grad_norm": 10.374255180358887,
+      "learning_rate": 2.6e-07,
+      "logits/chosen": 0.12109375,
+      "logits/rejected": 0.12890625,
+      "logps/chosen": -22.25,
+      "logps/rejected": -23.75,
+      "loss": 0.6601,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.659166693687439,
+      "rewards/margins": 0.16083335876464844,
+      "rewards/rejected": 0.49833330512046814,
+      "step": 1480
+    },
+    {
+      "epoch": 0.745,
+      "grad_norm": 9.207683563232422,
+      "learning_rate": 2.55e-07,
+      "logits/chosen": 0.1103515625,
+      "logits/rejected": 0.119140625,
+      "logps/chosen": -29.25,
+      "logps/rejected": -23.375,
+      "loss": 0.6965,
+      "rewards/accuracies": 0.4000000059604645,
+      "rewards/chosen": 0.4920833110809326,
+      "rewards/margins": 0.019999990239739418,
+      "rewards/rejected": 0.47208333015441895,
+      "step": 1490
+    },
+    {
+      "epoch": 0.75,
+      "grad_norm": 14.681502342224121,
+      "learning_rate": 2.5e-07,
+      "logits/chosen": 0.10986328125,
+      "logits/rejected": 0.1259765625,
+      "logps/chosen": -22.5,
+      "logps/rejected": -25.25,
+      "loss": 0.5205,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.7095833420753479,
+      "rewards/margins": 0.4108334183692932,
+      "rewards/rejected": 0.2987499535083771,
+      "step": 1500
+    },
+    {
+      "epoch": 0.755,
+      "grad_norm": 11.020295143127441,
+      "learning_rate": 2.45e-07,
+      "logits/chosen": 0.1337890625,
+      "logits/rejected": 0.1416015625,
+      "logps/chosen": -25.625,
+      "logps/rejected": -26.75,
+      "loss": 0.7036,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.34958332777023315,
+      "rewards/margins": 0.021666664630174637,
+      "rewards/rejected": 0.3279166519641876,
+      "step": 1510
+    },
+    {
+      "epoch": 0.76,
+      "grad_norm": 18.642732620239258,
+      "learning_rate": 2.4e-07,
+      "logits/chosen": 0.12353515625,
+      "logits/rejected": 0.1259765625,
+      "logps/chosen": -25.125,
+      "logps/rejected": -25.5,
+      "loss": 0.6732,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.49291664361953735,
+      "rewards/margins": 0.10500000417232513,
+      "rewards/rejected": 0.387916624546051,
+      "step": 1520
+    },
+    {
+      "epoch": 0.765,
+      "grad_norm": 9.671899795532227,
+      "learning_rate": 2.3499999999999997e-07,
+      "logits/chosen": 0.1025390625,
+      "logits/rejected": 0.1572265625,
+      "logps/chosen": -24.625,
+      "logps/rejected": -28.0,
+      "loss": 0.5545,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.7033333778381348,
+      "rewards/margins": 0.351250022649765,
+      "rewards/rejected": 0.35208332538604736,
+      "step": 1530
+    },
+    {
+      "epoch": 0.77,
+      "grad_norm": 9.261238098144531,
+      "learning_rate": 2.3e-07,
+      "logits/chosen": 0.13671875,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -22.5,
+      "logps/rejected": -24.25,
+      "loss": 0.6447,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5237500071525574,
+      "rewards/margins": 0.1524999588727951,
+      "rewards/rejected": 0.3712500035762787,
+      "step": 1540
+    },
+    {
+      "epoch": 0.775,
+      "grad_norm": 11.36690616607666,
+      "learning_rate": 2.25e-07,
+      "logits/chosen": 0.1435546875,
+      "logits/rejected": 0.1513671875,
+      "logps/chosen": -21.875,
+      "logps/rejected": -26.875,
+      "loss": 0.5553,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5991666913032532,
+      "rewards/margins": 0.3608333170413971,
+      "rewards/rejected": 0.2383333146572113,
+      "step": 1550
+    },
+    {
+      "epoch": 0.78,
+      "grad_norm": 11.070174217224121,
+      "learning_rate": 2.1999999999999998e-07,
+      "logits/chosen": 0.111328125,
+      "logits/rejected": 0.1494140625,
+      "logps/chosen": -23.625,
+      "logps/rejected": -29.25,
+      "loss": 0.6321,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5083333253860474,
+      "rewards/margins": 0.1962500363588333,
+      "rewards/rejected": 0.31208330392837524,
+      "step": 1560
+    },
+    {
+      "epoch": 0.785,
+      "grad_norm": 10.88608455657959,
+      "learning_rate": 2.1499999999999998e-07,
+      "logits/chosen": 0.11865234375,
+      "logits/rejected": 0.13671875,
+      "logps/chosen": -19.625,
+      "logps/rejected": -24.375,
+      "loss": 0.5845,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.6787500381469727,
+      "rewards/margins": 0.26958340406417847,
+      "rewards/rejected": 0.4091666340827942,
+      "step": 1570
+    },
+    {
+      "epoch": 0.79,
+      "grad_norm": 16.319265365600586,
+      "learning_rate": 2.0999999999999997e-07,
+      "logits/chosen": 0.1171875,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -21.125,
+      "logps/rejected": -27.75,
+      "loss": 0.5511,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.65583336353302,
+      "rewards/margins": 0.3550000488758087,
+      "rewards/rejected": 0.3008332848548889,
+      "step": 1580
+    },
+    {
+      "epoch": 0.795,
+      "grad_norm": 6.782731533050537,
+      "learning_rate": 2.0499999999999997e-07,
+      "logits/chosen": 0.1279296875,
+      "logits/rejected": 0.177734375,
+      "logps/chosen": -22.75,
+      "logps/rejected": -26.375,
+      "loss": 0.5027,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.752916693687439,
+      "rewards/margins": 0.4554166793823242,
+      "rewards/rejected": 0.29749998450279236,
+      "step": 1590
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 15.223215103149414,
+      "learning_rate": 2e-07,
+      "logits/chosen": 0.1533203125,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -28.5,
+      "logps/rejected": -23.875,
+      "loss": 0.7215,
+      "rewards/accuracies": 0.30000001192092896,
+      "rewards/chosen": 0.4324999749660492,
+      "rewards/margins": 0.0004166722355876118,
+      "rewards/rejected": 0.4320833086967468,
+      "step": 1600
+    },
+    {
+      "epoch": 0.805,
+      "grad_norm": 12.704072952270508,
+      "learning_rate": 1.9499999999999999e-07,
+      "logits/chosen": 0.0869140625,
+      "logits/rejected": 0.11376953125,
+      "logps/chosen": -23.625,
+      "logps/rejected": -24.375,
+      "loss": 0.5228,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.7858332991600037,
+      "rewards/margins": 0.4112500250339508,
+      "rewards/rejected": 0.37458330392837524,
+      "step": 1610
+    },
+    {
+      "epoch": 0.81,
+      "grad_norm": 18.95876693725586,
+      "learning_rate": 1.8999999999999998e-07,
+      "logits/chosen": 0.146484375,
+      "logits/rejected": 0.162109375,
+      "logps/chosen": -25.0,
+      "logps/rejected": -27.5,
+      "loss": 0.6073,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.47291669249534607,
+      "rewards/margins": 0.23583336174488068,
+      "rewards/rejected": 0.2370833456516266,
+      "step": 1620
+    },
+    {
+      "epoch": 0.815,
+      "grad_norm": 9.891932487487793,
+      "learning_rate": 1.85e-07,
+      "logits/chosen": 0.1298828125,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -25.0,
+      "logps/rejected": -25.125,
+      "loss": 0.6349,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4995834231376648,
+      "rewards/margins": 0.15458336472511292,
+      "rewards/rejected": 0.3449999690055847,
+      "step": 1630
+    },
+    {
+      "epoch": 0.82,
+      "grad_norm": 7.139009952545166,
+      "learning_rate": 1.8e-07,
+      "logits/chosen": 0.11767578125,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -25.625,
+      "logps/rejected": -26.125,
+      "loss": 0.5299,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6675000190734863,
+      "rewards/margins": 0.39500004053115845,
+      "rewards/rejected": 0.2724999785423279,
+      "step": 1640
+    },
+    {
+      "epoch": 0.825,
+      "grad_norm": 6.264760971069336,
+      "learning_rate": 1.75e-07,
+      "logits/chosen": 0.10205078125,
+      "logits/rejected": 0.1103515625,
+      "logps/chosen": -24.75,
+      "logps/rejected": -24.625,
+      "loss": 0.5481,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6441667079925537,
+      "rewards/margins": 0.34125006198883057,
+      "rewards/rejected": 0.3029167056083679,
+      "step": 1650
+    },
+    {
+      "epoch": 0.83,
+      "grad_norm": 14.040396690368652,
+      "learning_rate": 1.7000000000000001e-07,
+      "logits/chosen": 0.1328125,
+      "logits/rejected": 0.134765625,
+      "logps/chosen": -22.125,
+      "logps/rejected": -21.875,
+      "loss": 0.6255,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6404167413711548,
+      "rewards/margins": 0.17791667580604553,
+      "rewards/rejected": 0.4625000059604645,
+      "step": 1660
+    },
+    {
+      "epoch": 0.835,
+      "grad_norm": 6.089046478271484,
+      "learning_rate": 1.65e-07,
+      "logits/chosen": 0.12060546875,
+      "logits/rejected": 0.1298828125,
+      "logps/chosen": -23.125,
+      "logps/rejected": -26.25,
+      "loss": 0.5936,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6683333516120911,
+      "rewards/margins": 0.2762500047683716,
+      "rewards/rejected": 0.3920833468437195,
+      "step": 1670
+    },
+    {
+      "epoch": 0.84,
+      "grad_norm": 8.51190185546875,
+      "learning_rate": 1.6e-07,
+      "logits/chosen": 0.12255859375,
+      "logits/rejected": 0.125,
+      "logps/chosen": -22.5,
+      "logps/rejected": -24.5,
+      "loss": 0.5977,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6008332967758179,
+      "rewards/margins": 0.2875000536441803,
+      "rewards/rejected": 0.31333330273628235,
+      "step": 1680
+    },
+    {
+      "epoch": 0.845,
+      "grad_norm": 5.385739803314209,
+      "learning_rate": 1.55e-07,
+      "logits/chosen": 0.1083984375,
+      "logits/rejected": 0.1328125,
+      "logps/chosen": -26.25,
+      "logps/rejected": -28.375,
+      "loss": 0.5651,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.622083306312561,
+      "rewards/margins": 0.3345833420753479,
+      "rewards/rejected": 0.28749996423721313,
+      "step": 1690
+    },
+    {
+      "epoch": 0.85,
+      "grad_norm": 8.961523056030273,
+      "learning_rate": 1.5e-07,
+      "logits/chosen": 0.111328125,
+      "logits/rejected": 0.1513671875,
+      "logps/chosen": -23.375,
+      "logps/rejected": -26.625,
+      "loss": 0.5546,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.715833306312561,
+      "rewards/margins": 0.398749977350235,
+      "rewards/rejected": 0.31708332896232605,
+      "step": 1700
+    },
+    {
+      "epoch": 0.855,
+      "grad_norm": 12.527627944946289,
+      "learning_rate": 1.45e-07,
+      "logits/chosen": 0.11572265625,
+      "logits/rejected": 0.11767578125,
+      "logps/chosen": -23.125,
+      "logps/rejected": -22.25,
+      "loss": 0.6266,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5920833349227905,
+      "rewards/margins": 0.1704166829586029,
+      "rewards/rejected": 0.42166668176651,
+      "step": 1710
+    },
+    {
+      "epoch": 0.86,
+      "grad_norm": 0.17771057784557343,
+      "learning_rate": 1.4e-07,
+      "logits/chosen": 0.1259765625,
+      "logits/rejected": 0.1259765625,
+      "logps/chosen": -26.125,
+      "logps/rejected": -27.625,
+      "loss": 0.6534,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5520833730697632,
+      "rewards/margins": 0.11625002324581146,
+      "rewards/rejected": 0.43583330512046814,
+      "step": 1720
+    },
+    {
+      "epoch": 0.865,
+      "grad_norm": 6.410358905792236,
+      "learning_rate": 1.35e-07,
+      "logits/chosen": 0.10888671875,
+      "logits/rejected": 0.1357421875,
+      "logps/chosen": -20.25,
+      "logps/rejected": -28.375,
+      "loss": 0.5758,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.6495832800865173,
+      "rewards/margins": 0.28999999165534973,
+      "rewards/rejected": 0.3595833480358124,
+      "step": 1730
+    },
+    {
+      "epoch": 0.87,
+      "grad_norm": 6.186086177825928,
+      "learning_rate": 1.3e-07,
+      "logits/chosen": 0.1279296875,
+      "logits/rejected": 0.126953125,
+      "logps/chosen": -22.5,
+      "logps/rejected": -24.625,
+      "loss": 0.6591,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5766667127609253,
+      "rewards/margins": 0.13166671991348267,
+      "rewards/rejected": 0.4449999928474426,
+      "step": 1740
+    },
+    {
+      "epoch": 0.875,
+      "grad_norm": 0.17272135615348816,
+      "learning_rate": 1.25e-07,
+      "logits/chosen": 0.11669921875,
+      "logits/rejected": 0.10205078125,
+      "logps/chosen": -24.25,
+      "logps/rejected": -28.0,
+      "loss": 0.6849,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.44083333015441895,
+      "rewards/margins": 0.021250009536743164,
+      "rewards/rejected": 0.4195833206176758,
+      "step": 1750
+    },
+    {
+      "epoch": 0.88,
+      "grad_norm": 12.538702011108398,
+      "learning_rate": 1.2e-07,
+      "logits/chosen": 0.12353515625,
+      "logits/rejected": 0.1484375,
+      "logps/chosen": -25.5,
+      "logps/rejected": -29.75,
+      "loss": 0.5796,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5054166913032532,
+      "rewards/margins": 0.29083332419395447,
+      "rewards/rejected": 0.21458332240581512,
+      "step": 1760
+    },
+    {
+      "epoch": 0.885,
+      "grad_norm": 19.890647888183594,
+      "learning_rate": 1.15e-07,
+      "logits/chosen": 0.109375,
+      "logits/rejected": 0.126953125,
+      "logps/chosen": -22.875,
+      "logps/rejected": -24.0,
+      "loss": 0.5566,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.6408334374427795,
+      "rewards/margins": 0.320000022649765,
+      "rewards/rejected": 0.320833295583725,
+      "step": 1770
+    },
+    {
+      "epoch": 0.89,
+      "grad_norm": 6.63965368270874,
+      "learning_rate": 1.0999999999999999e-07,
+      "logits/chosen": 0.12060546875,
+      "logits/rejected": 0.140625,
+      "logps/chosen": -22.75,
+      "logps/rejected": -25.75,
+      "loss": 0.5556,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6916666626930237,
+      "rewards/margins": 0.3179166913032532,
+      "rewards/rejected": 0.3737499713897705,
+      "step": 1780
+    },
+    {
+      "epoch": 0.895,
+      "grad_norm": 10.550226211547852,
+      "learning_rate": 1.0499999999999999e-07,
+      "logits/chosen": 0.10400390625,
+      "logits/rejected": 0.1181640625,
+      "logps/chosen": -21.125,
+      "logps/rejected": -28.875,
+      "loss": 0.5649,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.7695833444595337,
+      "rewards/margins": 0.3279167115688324,
+      "rewards/rejected": 0.4416666626930237,
+      "step": 1790
+    },
+    {
+      "epoch": 0.9,
+      "grad_norm": 24.27086067199707,
+      "learning_rate": 1e-07,
+      "logits/chosen": 0.1259765625,
+      "logits/rejected": 0.130859375,
+      "logps/chosen": -24.875,
+      "logps/rejected": -30.375,
+      "loss": 0.6215,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.4112500250339508,
+      "rewards/margins": 0.16249996423721313,
+      "rewards/rejected": 0.2487500160932541,
+      "step": 1800
+    },
+    {
+      "epoch": 0.905,
+      "grad_norm": 11.343680381774902,
+      "learning_rate": 9.499999999999999e-08,
+      "logits/chosen": 0.10986328125,
+      "logits/rejected": 0.134765625,
+      "logps/chosen": -22.5,
+      "logps/rejected": -27.0,
+      "loss": 0.5433,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.7104166746139526,
+      "rewards/margins": 0.38458338379859924,
+      "rewards/rejected": 0.3258333206176758,
+      "step": 1810
+    },
+    {
+      "epoch": 0.91,
+      "grad_norm": 10.268211364746094,
+      "learning_rate": 9e-08,
+      "logits/chosen": 0.10107421875,
+      "logits/rejected": 0.1015625,
+      "logps/chosen": -20.25,
+      "logps/rejected": -25.0,
+      "loss": 0.6155,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.6162499785423279,
+      "rewards/margins": 0.2433333694934845,
+      "rewards/rejected": 0.37291666865348816,
+      "step": 1820
+    },
+    {
+      "epoch": 0.915,
+      "grad_norm": 15.951262474060059,
+      "learning_rate": 8.500000000000001e-08,
+      "logits/chosen": 0.11962890625,
+      "logits/rejected": 0.1337890625,
+      "logps/chosen": -24.25,
+      "logps/rejected": -25.25,
+      "loss": 0.6402,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.5320833325386047,
+      "rewards/margins": 0.17666670680046082,
+      "rewards/rejected": 0.3554166257381439,
+      "step": 1830
+    },
+    {
+      "epoch": 0.92,
+      "grad_norm": 11.495180130004883,
+      "learning_rate": 8e-08,
+      "logits/chosen": 0.0888671875,
+      "logits/rejected": 0.115234375,
+      "logps/chosen": -27.25,
+      "logps/rejected": -30.875,
+      "loss": 0.4958,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6766666769981384,
+      "rewards/margins": 0.4995833933353424,
+      "rewards/rejected": 0.17708329856395721,
+      "step": 1840
+    },
+    {
+      "epoch": 0.925,
+      "grad_norm": 9.108668327331543,
+      "learning_rate": 7.5e-08,
+      "logits/chosen": 0.11962890625,
+      "logits/rejected": 0.126953125,
+      "logps/chosen": -19.875,
+      "logps/rejected": -29.25,
+      "loss": 0.5253,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": 0.7145833373069763,
+      "rewards/margins": 0.424166738986969,
+      "rewards/rejected": 0.2904166579246521,
+      "step": 1850
+    },
+    {
+      "epoch": 0.93,
+      "grad_norm": 8.972005844116211,
+      "learning_rate": 7e-08,
+      "logits/chosen": 0.11865234375,
+      "logits/rejected": 0.1328125,
+      "logps/chosen": -25.875,
+      "logps/rejected": -24.75,
+      "loss": 0.6629,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.527916669845581,
+      "rewards/margins": 0.11875002086162567,
+      "rewards/rejected": 0.4091666340827942,
+      "step": 1860
+    },
+    {
+      "epoch": 0.935,
+      "grad_norm": 10.217000961303711,
+      "learning_rate": 6.5e-08,
+      "logits/chosen": 0.0869140625,
+      "logits/rejected": 0.119140625,
+      "logps/chosen": -22.25,
+      "logps/rejected": -23.25,
+      "loss": 0.6061,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6825000047683716,
+      "rewards/margins": 0.2320832908153534,
+      "rewards/rejected": 0.4504166543483734,
+      "step": 1870
+    },
+    {
+      "epoch": 0.94,
+      "grad_norm": 14.174996376037598,
+      "learning_rate": 6e-08,
+      "logits/chosen": 0.1123046875,
+      "logits/rejected": 0.1240234375,
+      "logps/chosen": -23.25,
+      "logps/rejected": -25.125,
+      "loss": 0.5783,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6787499785423279,
+      "rewards/margins": 0.299583375453949,
+      "rewards/rejected": 0.3791666626930237,
+      "step": 1880
+    },
+    {
+      "epoch": 0.945,
+      "grad_norm": 3.4117889404296875,
+      "learning_rate": 5.4999999999999996e-08,
+      "logits/chosen": 0.11328125,
+      "logits/rejected": 0.123046875,
+      "logps/chosen": -21.75,
+      "logps/rejected": -25.25,
+      "loss": 0.4953,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.7429167032241821,
+      "rewards/margins": 0.4724999964237213,
+      "rewards/rejected": 0.27041664719581604,
+      "step": 1890
+    },
+    {
+      "epoch": 0.95,
+      "grad_norm": 10.660746574401855,
+      "learning_rate": 5e-08,
+      "logits/chosen": 0.11279296875,
+      "logits/rejected": 0.11962890625,
+      "logps/chosen": -22.625,
+      "logps/rejected": -26.5,
+      "loss": 0.6275,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.6579166650772095,
+      "rewards/margins": 0.2579166889190674,
+      "rewards/rejected": 0.4000000059604645,
+      "step": 1900
+    },
+    {
+      "epoch": 0.955,
+      "grad_norm": 11.314635276794434,
+      "learning_rate": 4.5e-08,
+      "logits/chosen": 0.1318359375,
+      "logits/rejected": 0.1552734375,
+      "logps/chosen": -21.875,
+      "logps/rejected": -29.25,
+      "loss": 0.5534,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.5808333158493042,
+      "rewards/margins": 0.3237500488758087,
+      "rewards/rejected": 0.25708329677581787,
+      "step": 1910
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 16.68134307861328,
+      "learning_rate": 4e-08,
+      "logits/chosen": 0.10791015625,
+      "logits/rejected": 0.12255859375,
+      "logps/chosen": -20.125,
+      "logps/rejected": -30.25,
+      "loss": 0.5353,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6974999904632568,
+      "rewards/margins": 0.4570833742618561,
+      "rewards/rejected": 0.24041661620140076,
+      "step": 1920
+    },
+    {
+      "epoch": 0.965,
+      "grad_norm": 11.01811695098877,
+      "learning_rate": 3.5e-08,
+      "logits/chosen": 0.1103515625,
+      "logits/rejected": 0.10888671875,
+      "logps/chosen": -22.625,
+      "logps/rejected": -24.0,
+      "loss": 0.6438,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5449999570846558,
+      "rewards/margins": 0.15458333492279053,
+      "rewards/rejected": 0.39041668176651,
+      "step": 1930
+    },
+    {
+      "epoch": 0.97,
+      "grad_norm": 3.993349552154541,
+      "learning_rate": 3e-08,
+      "logits/chosen": 0.115234375,
+      "logits/rejected": 0.1181640625,
+      "logps/chosen": -23.75,
+      "logps/rejected": -24.875,
+      "loss": 0.6592,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5054166316986084,
+      "rewards/margins": 0.1574999988079071,
+      "rewards/rejected": 0.3479166626930237,
+      "step": 1940
+    },
+    {
+      "epoch": 0.975,
+      "grad_norm": 9.688010215759277,
+      "learning_rate": 2.5e-08,
+      "logits/chosen": 0.11669921875,
+      "logits/rejected": 0.12890625,
+      "logps/chosen": -23.0,
+      "logps/rejected": -26.25,
+      "loss": 0.5644,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6320832967758179,
+      "rewards/margins": 0.31291672587394714,
+      "rewards/rejected": 0.3191666603088379,
+      "step": 1950
+    },
+    {
+      "epoch": 0.98,
+      "grad_norm": 9.296296119689941,
+      "learning_rate": 2e-08,
+      "logits/chosen": 0.1240234375,
+      "logits/rejected": 0.130859375,
+      "logps/chosen": -21.625,
+      "logps/rejected": -27.375,
+      "loss": 0.5299,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6679166555404663,
+      "rewards/margins": 0.3858333230018616,
+      "rewards/rejected": 0.2820833623409271,
+      "step": 1960
+    },
+    {
+      "epoch": 0.985,
+      "grad_norm": 7.18031644821167,
+      "learning_rate": 1.5e-08,
+      "logits/chosen": 0.1298828125,
+      "logits/rejected": 0.11962890625,
+      "logps/chosen": -24.0,
+      "logps/rejected": -28.0,
+      "loss": 0.6122,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.4866666793823242,
+      "rewards/margins": 0.1991666853427887,
+      "rewards/rejected": 0.2874999940395355,
+      "step": 1970
+    },
+    {
+      "epoch": 0.99,
+      "grad_norm": 19.945270538330078,
+      "learning_rate": 1e-08,
+      "logits/chosen": 0.12109375,
+      "logits/rejected": 0.1279296875,
+      "logps/chosen": -23.75,
+      "logps/rejected": -23.5,
+      "loss": 0.6879,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": 0.5387499928474426,
+      "rewards/margins": 0.07541666179895401,
+      "rewards/rejected": 0.4633333086967468,
+      "step": 1980
+    },
+    {
+      "epoch": 0.995,
+      "grad_norm": 13.096246719360352,
+      "learning_rate": 5e-09,
+      "logits/chosen": 0.103515625,
+      "logits/rejected": 0.1171875,
+      "logps/chosen": -21.625,
+      "logps/rejected": -26.625,
+      "loss": 0.6005,
+      "rewards/accuracies": 0.699999988079071,
+      "rewards/chosen": 0.6704167127609253,
+      "rewards/margins": 0.22333335876464844,
+      "rewards/rejected": 0.4470832943916321,
+      "step": 1990
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 10.517897605895996,
+      "learning_rate": 0.0,
+      "logits/chosen": 0.115234375,
+      "logits/rejected": 0.1337890625,
+      "logps/chosen": -21.5,
+      "logps/rejected": -28.125,
+      "loss": 0.6165,
+      "rewards/accuracies": 0.6000000238418579,
+      "rewards/chosen": 0.5891666412353516,
+      "rewards/margins": 0.26250001788139343,
+      "rewards/rejected": 0.3266666531562805,
+      "step": 2000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 140000,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}
diff --git a/math/finetuned/checkpoint/training_args.bin b/math/finetuned/checkpoint/training_args.bin
new file mode 100644
index 0000000000000000000000000000000000000000..1bfa0d3d51c2be0c4d575e55532f3070d0a4208e
--- /dev/null
+++ b/math/finetuned/checkpoint/training_args.bin
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:423da21efd308b91595b6f1eeb4d256199f38df224437ab144eb513f2ed5792a
+size 6072
diff --git a/math/finetuned/merged/config.json b/math/finetuned/merged/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..0b0b35a359021051e71677b54a054cb96d544ed1
--- /dev/null
+++ b/math/finetuned/merged/config.json
@@ -0,0 +1,40 @@
+{
+  "_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.46.0",
+  "use_cache": true,
+  "vocab_size": 128256
+}
diff --git a/math/finetuned/merged/generation_config.json b/math/finetuned/merged/generation_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..85636bf06e4077943669fe30f92ec430c1d5b044
--- /dev/null
+++ b/math/finetuned/merged/generation_config.json
@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.46.0"
+}
diff --git a/math/finetuned/merged/model-00001-of-00004.safetensors b/math/finetuned/merged/model-00001-of-00004.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..2b21a7ae22d2ce51e2ba236c8bbea0a33533b529
--- /dev/null
+++ b/math/finetuned/merged/model-00001-of-00004.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4996b0e44cdc281f875028036b84bc6299383a52f73bdf96b8d0fd019fe03058
+size 4976698672
diff --git a/math/finetuned/merged/model-00002-of-00004.safetensors b/math/finetuned/merged/model-00002-of-00004.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..a4638f6d8deef097623b32ae95385077176af764
--- /dev/null
+++ b/math/finetuned/merged/model-00002-of-00004.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2a9cb2c0fe61c46252b5768207d61c11d1a020907a83b0a03ff87cd4a8323e2
+size 4999802720
diff --git a/math/finetuned/merged/model-00003-of-00004.safetensors b/math/finetuned/merged/model-00003-of-00004.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bf51e6caf29c324a35fea24859822950148a2f80
--- /dev/null
+++ b/math/finetuned/merged/model-00003-of-00004.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16c0a41f8c38b6266adebce28033c911f4a5b345ddcd4f10b3396ebe0fd17488
+size 4915916176
diff --git a/math/finetuned/merged/model-00004-of-00004.safetensors b/math/finetuned/merged/model-00004-of-00004.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b8babcaa5a888d6efd5c05206361b4685492ad13
--- /dev/null
+++ b/math/finetuned/merged/model-00004-of-00004.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92ecfe1a2414458b4821ac8c13cf8cb70aed66b5eea8dc5ad9eeb4ff309d6d7b
+size 1168138808
diff --git a/math/finetuned/merged/model.safetensors.index.json b/math/finetuned/merged/model.safetensors.index.json
new file mode 100644
index 0000000000000000000000000000000000000000..0fd8120f1c6acddc268ebc2583058efaf699a771
--- /dev/null
+++ b/math/finetuned/merged/model.safetensors.index.json
@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 16060522496
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00004-of-00004.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.norm.weight": "model-00004-of-00004.safetensors"
+  }
+}
diff --git a/math/finetuned/merged/special_tokens_map.json b/math/finetuned/merged/special_tokens_map.json
new file mode 100644
index 0000000000000000000000000000000000000000..02ee80b6196926a5ad790a004d9efd6ab1ba6542
--- /dev/null
+++ b/math/finetuned/merged/special_tokens_map.json
@@ -0,0 +1,16 @@
+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}
diff --git a/math/finetuned/merged/tokenizer.json b/math/finetuned/merged/tokenizer.json
new file mode 100644
index 0000000000000000000000000000000000000000..1c1d8d5c9024994f1d3b00f9662b8dd89ca13cf2
--- /dev/null
+++ b/math/finetuned/merged/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920
diff --git a/math/finetuned/merged/tokenizer_config.json b/math/finetuned/merged/tokenizer_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..db88166e2bc4c799fd5d1ae643b75e84d03ee70e
--- /dev/null
+++ b/math/finetuned/merged/tokenizer_config.json
@@ -0,0 +1,2062 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}
diff --git a/math/output_evaluation/scores-0-0.txt b/math/output_evaluation/scores-0-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..060aa31fcdc28cb0585da9a750f6239e78e4e3f6
Binary files /dev/null and b/math/output_evaluation/scores-0-0.txt differ
diff --git a/math/output_evaluation/scores-0-1.txt b/math/output_evaluation/scores-0-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..056189d64f4a7165f8d6ac111640e2da4306296e
Binary files /dev/null and b/math/output_evaluation/scores-0-1.txt differ
diff --git a/math/output_evaluation/scores-0-2.txt b/math/output_evaluation/scores-0-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..c77ed5a24a1d476c2ff983bd3f3464380ab048fc
Binary files /dev/null and b/math/output_evaluation/scores-0-2.txt differ
diff --git a/math/output_evaluation/scores-1-0.txt b/math/output_evaluation/scores-1-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..17d76711345b17d0d24691e1d3bef25cb77bb564
Binary files /dev/null and b/math/output_evaluation/scores-1-0.txt differ
diff --git a/math/output_evaluation/scores-1-1.txt b/math/output_evaluation/scores-1-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..80e63b051533f3c6420f2cb69673f544609a9531
Binary files /dev/null and b/math/output_evaluation/scores-1-1.txt differ
diff --git a/math/output_evaluation/scores-1-2.txt b/math/output_evaluation/scores-1-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..04883ad2937ebf846c051f5460f9e226351decdc
Binary files /dev/null and b/math/output_evaluation/scores-1-2.txt differ
diff --git a/math/output_evaluation/scores-2-0.txt b/math/output_evaluation/scores-2-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9d1f89b98c83055d1eefeaa1d320c53205b477b2
Binary files /dev/null and b/math/output_evaluation/scores-2-0.txt differ
diff --git a/math/output_evaluation/scores-2-1.txt b/math/output_evaluation/scores-2-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..bbe126a96733a91621b78b946c87a6e84a404f92
Binary files /dev/null and b/math/output_evaluation/scores-2-1.txt differ
diff --git a/math/output_evaluation/scores-2-2.txt b/math/output_evaluation/scores-2-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..78fc7cf36400acc9c88073f3a7127078507f8a36
Binary files /dev/null and b/math/output_evaluation/scores-2-2.txt differ
diff --git a/math/output_evaluation/scores-3-0.txt b/math/output_evaluation/scores-3-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..ec144fe064e8f61badbfd494e7d765c1119f79b4
Binary files /dev/null and b/math/output_evaluation/scores-3-0.txt differ
diff --git a/math/output_evaluation/scores-3-1.txt b/math/output_evaluation/scores-3-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..abb0b05b4d729131b91583ccd87c44f92517c86b
Binary files /dev/null and b/math/output_evaluation/scores-3-1.txt differ
diff --git a/math/output_evaluation/scores-3-2.txt b/math/output_evaluation/scores-3-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..8688447ea64e0c8fd60cc10150100b99da2eb74e
Binary files /dev/null and b/math/output_evaluation/scores-3-2.txt differ
diff --git a/math/output_evaluation/scores-4-0.txt b/math/output_evaluation/scores-4-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..df686383e10a9faf6f787c5434373acc14f7819f
Binary files /dev/null and b/math/output_evaluation/scores-4-0.txt differ
diff --git a/math/output_evaluation/scores-4-1.txt b/math/output_evaluation/scores-4-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..3cc95b73a3c8ef4483c8167142f3a56bbb5fd0f8
Binary files /dev/null and b/math/output_evaluation/scores-4-1.txt differ
diff --git a/math/output_evaluation/scores-4-2.txt b/math/output_evaluation/scores-4-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..262a7b1095aa247d534e8bcc0a6f40b733bcbed1
Binary files /dev/null and b/math/output_evaluation/scores-4-2.txt differ
diff --git a/math/output_evaluation/scores-5-0.txt b/math/output_evaluation/scores-5-0.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2d68d8b4c9099f858b799c6fd47f45b4d315e827
Binary files /dev/null and b/math/output_evaluation/scores-5-0.txt differ
diff --git a/math/output_evaluation/scores-5-1.txt b/math/output_evaluation/scores-5-1.txt
new file mode 100644
index 0000000000000000000000000000000000000000..0f41f530dcaad9f9e2cb82ddaf2c4231f2901821
Binary files /dev/null and b/math/output_evaluation/scores-5-1.txt differ
diff --git a/math/output_evaluation/scores-5-2.txt b/math/output_evaluation/scores-5-2.txt
new file mode 100644
index 0000000000000000000000000000000000000000..9955b3b9efe17c2b7c9efebc93014a72b82c0936
Binary files /dev/null and b/math/output_evaluation/scores-5-2.txt differ
diff --git a/math/output_preference_data/preference_data-0.pkl b/math/output_preference_data/preference_data-0.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..94f82c665c19804ca5b0d30ee1fa3aee9491a39b
--- /dev/null
+++ b/math/output_preference_data/preference_data-0.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9211d84844b1b6baf4be849a043fc4cc9fbd3c4a91110d364040d6c99b665fe7
+size 923072
diff --git a/math/output_preference_data/preference_data-1.pkl b/math/output_preference_data/preference_data-1.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..fc43e300048825fef30e2f4bbd59c06749dcf50d
--- /dev/null
+++ b/math/output_preference_data/preference_data-1.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4970c2e99b9b26b7350054f556fb41a8cad71823b299a8d1111ae7a11f9e6778
+size 880059
diff --git a/math/output_preference_data/preference_data-2.pkl b/math/output_preference_data/preference_data-2.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..b9b19f3443a522eff119f76328a62921fac53e9f
--- /dev/null
+++ b/math/output_preference_data/preference_data-2.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a10ef465b9944be3853ce4395ff4afe5174044f2b152ecdb4e7b6ecd52b4cd78
+size 814469
diff --git a/math/output_preference_data/preference_data-3.pkl b/math/output_preference_data/preference_data-3.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..8960eb19779b598bb27012d6bed6b7e57725ba84
--- /dev/null
+++ b/math/output_preference_data/preference_data-3.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc9adb764b7187d9016a1a8555bc3971541610d1fe3f2e4f8daa0beda04b8064
+size 880180
diff --git a/math/output_preference_data/preference_data-4.pkl b/math/output_preference_data/preference_data-4.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..a363efce248cd9e72eaf77ca847b0c2fabc6015d
--- /dev/null
+++ b/math/output_preference_data/preference_data-4.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40ed1ff3e718f7469ca64bf91ad4619d58f85c6ba64e864e200741d6b16be555
+size 807225
diff --git a/math/output_preference_data/preference_data-5.pkl b/math/output_preference_data/preference_data-5.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..0df9804decdd5c0fe832132c916801e090a7d355
--- /dev/null
+++ b/math/output_preference_data/preference_data-5.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd9b681890575f5751f4c2e5b38fa1f0666aaa7f7fa44d81e07359329ef45c66
+size 752590
diff --git a/math/output_workflow/dataset-0.pkl b/math/output_workflow/dataset-0.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..20a1e46366b850992c946664735114a5bf592d9b
--- /dev/null
+++ b/math/output_workflow/dataset-0.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85f16031724744c6caa96581874ffeef29ea00865c56ee114aa0b3881fc2e596
+size 1331388
diff --git a/math/output_workflow/dataset-1.pkl b/math/output_workflow/dataset-1.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..dc00a344a71ea2eba1abc89dee8d8d84c13a3418
--- /dev/null
+++ b/math/output_workflow/dataset-1.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f97beace59d402e31c0ac6730ee6c472823530b5392fbe06bd822c80d3355d33
+size 1337572
diff --git a/math/output_workflow/dataset-2.pkl b/math/output_workflow/dataset-2.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..9e2e66fbf440ab7488ba8e3e23a0a750984d72e4
--- /dev/null
+++ b/math/output_workflow/dataset-2.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf615aac0ad91d68c221346b3c3ec8b09a3031105d8ca51e87f032b4fc505fcc
+size 1377704
diff --git a/math/output_workflow/dataset-3.pkl b/math/output_workflow/dataset-3.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..8af400cad168ac5121b2001c6cc49182d9fcdc77
--- /dev/null
+++ b/math/output_workflow/dataset-3.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22972df425009538624c51acc094108c4380b5488ee72ca3cf998c5d56991001
+size 1386306
diff --git a/math/output_workflow/dataset-4.pkl b/math/output_workflow/dataset-4.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..4cfe19a413562d7a8675993a3276b7eb5e97024a
--- /dev/null
+++ b/math/output_workflow/dataset-4.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0814d732676e7069e89967a8d1950885855924913557b22a377efdc4ae1cccea
+size 1352015
diff --git a/math/output_workflow/dataset-5.pkl b/math/output_workflow/dataset-5.pkl
new file mode 100644
index 0000000000000000000000000000000000000000..a9ab55ae735836c7f1ea59579dc76032e9065585
--- /dev/null
+++ b/math/output_workflow/dataset-5.pkl
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8ce3ce4ee78eb374b174f6323c13b17dfd31c428ba9a16a8586e9fa68705f42
+size 1363241