Upload folder using huggingface_hub

checkpoint-300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/phi-4-unsloth-bnb-4bit
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-300/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/phi-4-unsloth-bnb-4bit",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b77e5bc4cf683732b9ef5cc90dc1640a0bf3719cfae3fcbddfc9a6ca4fbbe55c
+size 262219392

checkpoint-300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c03ee32f43be613c52e6a94b0ff968ae58a941c1fe73f0644d6d511c93d7caa2
+size 133778516

checkpoint-300/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd55c3b412ed63dda63d944e5148e12c72c1382a50017bf4d626548cc6ff19a6
+size 14244

checkpoint-300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac3679f3f7c7a8bc531d82f07028479b6652ee99d757c0486fdd296b767f9de5
+size 1064

checkpoint-300/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|dummy_87|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "�",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-300/tokenizer_config.json

@@ -0,0 +1,791 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "5809": {
+      "content": "�",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100256": {
+      "content": "<|dummy_0|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100261": {
+      "content": "<|dummy_1|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100262": {
+      "content": "<|dummy_2|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100263": {
+      "content": "<|dummy_3|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100266": {
+      "content": "<|im_sep|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100267": {
+      "content": "<|dummy_4|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100268": {
+      "content": "<|dummy_5|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100269": {
+      "content": "<|dummy_6|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100270": {
+      "content": "<|dummy_7|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100271": {
+      "content": "<|dummy_8|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100272": {
+      "content": "<|dummy_9|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100273": {
+      "content": "<|dummy_10|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100274": {
+      "content": "<|dummy_11|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100275": {
+      "content": "<|dummy_12|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100277": {
+      "content": "<|dummy_13|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100278": {
+      "content": "<|dummy_14|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100279": {
+      "content": "<|dummy_15|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100280": {
+      "content": "<|dummy_16|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100281": {
+      "content": "<|dummy_17|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100282": {
+      "content": "<|dummy_18|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100283": {
+      "content": "<|dummy_19|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100284": {
+      "content": "<|dummy_20|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100285": {
+      "content": "<|dummy_21|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100286": {
+      "content": "<|dummy_22|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100287": {
+      "content": "<|dummy_23|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100288": {
+      "content": "<|dummy_24|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100289": {
+      "content": "<|dummy_25|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100290": {
+      "content": "<|dummy_26|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100291": {
+      "content": "<|dummy_27|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100292": {
+      "content": "<|dummy_28|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100293": {
+      "content": "<|dummy_29|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100294": {
+      "content": "<|dummy_30|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100295": {
+      "content": "<|dummy_31|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100296": {
+      "content": "<|dummy_32|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100297": {
+      "content": "<|dummy_33|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100298": {
+      "content": "<|dummy_34|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100299": {
+      "content": "<|dummy_35|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100300": {
+      "content": "<|dummy_36|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100301": {
+      "content": "<|dummy_37|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100302": {
+      "content": "<|dummy_38|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100303": {
+      "content": "<|dummy_39|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100304": {
+      "content": "<|dummy_40|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100305": {
+      "content": "<|dummy_41|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100306": {
+      "content": "<|dummy_42|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100307": {
+      "content": "<|dummy_43|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100308": {
+      "content": "<|dummy_44|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100309": {
+      "content": "<|dummy_45|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100310": {
+      "content": "<|dummy_46|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100311": {
+      "content": "<|dummy_47|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100312": {
+      "content": "<|dummy_48|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100313": {
+      "content": "<|dummy_49|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100314": {
+      "content": "<|dummy_50|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100315": {
+      "content": "<|dummy_51|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100316": {
+      "content": "<|dummy_52|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100317": {
+      "content": "<|dummy_53|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100318": {
+      "content": "<|dummy_54|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100319": {
+      "content": "<|dummy_55|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100320": {
+      "content": "<|dummy_56|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100321": {
+      "content": "<|dummy_57|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100322": {
+      "content": "<|dummy_58|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100323": {
+      "content": "<|dummy_59|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100324": {
+      "content": "<|dummy_60|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100325": {
+      "content": "<|dummy_61|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100326": {
+      "content": "<|dummy_62|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100327": {
+      "content": "<|dummy_63|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100328": {
+      "content": "<|dummy_64|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100329": {
+      "content": "<|dummy_65|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100330": {
+      "content": "<|dummy_66|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100331": {
+      "content": "<|dummy_67|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100332": {
+      "content": "<|dummy_68|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100333": {
+      "content": "<|dummy_69|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100334": {
+      "content": "<|dummy_70|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100335": {
+      "content": "<|dummy_71|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100336": {
+      "content": "<|dummy_72|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100337": {
+      "content": "<|dummy_73|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100338": {
+      "content": "<|dummy_74|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100339": {
+      "content": "<|dummy_75|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100340": {
+      "content": "<|dummy_76|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100341": {
+      "content": "<|dummy_77|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100342": {
+      "content": "<|dummy_78|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100343": {
+      "content": "<|dummy_79|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100344": {
+      "content": "<|dummy_80|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100345": {
+      "content": "<|dummy_81|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100346": {
+      "content": "<|dummy_82|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100347": {
+      "content": "<|dummy_83|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100348": {
+      "content": "<|dummy_84|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100349": {
+      "content": "<|dummy_85|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100350": {
+      "content": "<|dummy_86|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100351": {
+      "content": "<|dummy_87|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "\n{% for message in messages %}\n{% if (message['role'] == 'system') %}\n{{'<|im_start|>system<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'user') %}\n{{'<|im_start|>user<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'assistant') %}\n{{'<|im_start|>assistant<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% endif %}\n{% endfor %}\n{% if add_generation_prompt %}\n{{ '<|im_start|>assistant<|im_sep|>' }}\n{% endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": {},
+  "model_max_length": 16384,
+  "pad_token": "<|dummy_87|>",
+  "padding_side": "right",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "�"
+}

checkpoint-300/trainer_state.json

@@ -0,0 +1,2157 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.8094117647058825,
+  "eval_steps": 96,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009411764705882352,
+      "grad_norm": 0.8642500638961792,
+      "learning_rate": 4e-05,
+      "loss": 0.9841,
+      "step": 1
+    },
+    {
+      "epoch": 0.018823529411764704,
+      "grad_norm": 0.8003044724464417,
+      "learning_rate": 8e-05,
+      "loss": 0.9727,
+      "step": 2
+    },
+    {
+      "epoch": 0.02823529411764706,
+      "grad_norm": 0.7982213497161865,
+      "learning_rate": 0.00012,
+      "loss": 1.0076,
+      "step": 3
+    },
+    {
+      "epoch": 0.03764705882352941,
+      "grad_norm": 0.4778502285480499,
+      "learning_rate": 0.00016,
+      "loss": 0.9179,
+      "step": 4
+    },
+    {
+      "epoch": 0.047058823529411764,
+      "grad_norm": 0.1798139065504074,
+      "learning_rate": 0.0002,
+      "loss": 0.8544,
+      "step": 5
+    },
+    {
+      "epoch": 0.05647058823529412,
+      "grad_norm": 0.10934074968099594,
+      "learning_rate": 0.00019936102236421725,
+      "loss": 0.8259,
+      "step": 6
+    },
+    {
+      "epoch": 0.06588235294117648,
+      "grad_norm": 0.451732873916626,
+      "learning_rate": 0.00019872204472843452,
+      "loss": 0.8176,
+      "step": 7
+    },
+    {
+      "epoch": 0.07529411764705882,
+      "grad_norm": 0.39763253927230835,
+      "learning_rate": 0.00019808306709265177,
+      "loss": 0.8091,
+      "step": 8
+    },
+    {
+      "epoch": 0.08470588235294117,
+      "grad_norm": 0.08421512693166733,
+      "learning_rate": 0.000197444089456869,
+      "loss": 0.8066,
+      "step": 9
+    },
+    {
+      "epoch": 0.09411764705882353,
+      "grad_norm": 0.0752546563744545,
+      "learning_rate": 0.00019680511182108628,
+      "loss": 0.788,
+      "step": 10
+    },
+    {
+      "epoch": 0.10352941176470588,
+      "grad_norm": 0.07105962187051773,
+      "learning_rate": 0.00019616613418530353,
+      "loss": 0.8142,
+      "step": 11
+    },
+    {
+      "epoch": 0.11294117647058824,
+      "grad_norm": 0.07268067449331284,
+      "learning_rate": 0.0001955271565495208,
+      "loss": 0.7427,
+      "step": 12
+    },
+    {
+      "epoch": 0.1223529411764706,
+      "grad_norm": 0.071601502597332,
+      "learning_rate": 0.00019488817891373804,
+      "loss": 0.7673,
+      "step": 13
+    },
+    {
+      "epoch": 0.13176470588235295,
+      "grad_norm": 0.06190289184451103,
+      "learning_rate": 0.00019424920127795528,
+      "loss": 0.7897,
+      "step": 14
+    },
+    {
+      "epoch": 0.1411764705882353,
+      "grad_norm": 0.062015291303396225,
+      "learning_rate": 0.00019361022364217253,
+      "loss": 0.7552,
+      "step": 15
+    },
+    {
+      "epoch": 0.15058823529411763,
+      "grad_norm": 0.06343454122543335,
+      "learning_rate": 0.00019297124600638977,
+      "loss": 0.7983,
+      "step": 16
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.0880446657538414,
+      "learning_rate": 0.00019233226837060702,
+      "loss": 0.7331,
+      "step": 17
+    },
+    {
+      "epoch": 0.16941176470588235,
+      "grad_norm": 0.06399823725223541,
+      "learning_rate": 0.00019169329073482429,
+      "loss": 0.7017,
+      "step": 18
+    },
+    {
+      "epoch": 0.17882352941176471,
+      "grad_norm": 0.06614473462104797,
+      "learning_rate": 0.00019105431309904153,
+      "loss": 0.7346,
+      "step": 19
+    },
+    {
+      "epoch": 0.18823529411764706,
+      "grad_norm": 0.05928972363471985,
+      "learning_rate": 0.0001904153354632588,
+      "loss": 0.7088,
+      "step": 20
+    },
+    {
+      "epoch": 0.1976470588235294,
+      "grad_norm": 0.05864707753062248,
+      "learning_rate": 0.00018977635782747604,
+      "loss": 0.7017,
+      "step": 21
+    },
+    {
+      "epoch": 0.20705882352941177,
+      "grad_norm": 0.0725838840007782,
+      "learning_rate": 0.0001891373801916933,
+      "loss": 0.7255,
+      "step": 22
+    },
+    {
+      "epoch": 0.2164705882352941,
+      "grad_norm": 0.07014375925064087,
+      "learning_rate": 0.00018849840255591056,
+      "loss": 0.6699,
+      "step": 23
+    },
+    {
+      "epoch": 0.22588235294117648,
+      "grad_norm": 0.08222078531980515,
+      "learning_rate": 0.0001878594249201278,
+      "loss": 0.6802,
+      "step": 24
+    },
+    {
+      "epoch": 0.23529411764705882,
+      "grad_norm": 0.07020223140716553,
+      "learning_rate": 0.00018722044728434505,
+      "loss": 0.7013,
+      "step": 25
+    },
+    {
+      "epoch": 0.2447058823529412,
+      "grad_norm": 0.05753432214260101,
+      "learning_rate": 0.00018658146964856232,
+      "loss": 0.6547,
+      "step": 26
+    },
+    {
+      "epoch": 0.2541176470588235,
+      "grad_norm": 0.06647774577140808,
+      "learning_rate": 0.00018594249201277956,
+      "loss": 0.7132,
+      "step": 27
+    },
+    {
+      "epoch": 0.2635294117647059,
+      "grad_norm": 0.061453238129615784,
+      "learning_rate": 0.00018530351437699683,
+      "loss": 0.6891,
+      "step": 28
+    },
+    {
+      "epoch": 0.27294117647058824,
+      "grad_norm": 0.055535465478897095,
+      "learning_rate": 0.00018466453674121408,
+      "loss": 0.6542,
+      "step": 29
+    },
+    {
+      "epoch": 0.2823529411764706,
+      "grad_norm": 0.05627722293138504,
+      "learning_rate": 0.00018402555910543132,
+      "loss": 0.677,
+      "step": 30
+    },
+    {
+      "epoch": 0.2917647058823529,
+      "grad_norm": 0.05428534746170044,
+      "learning_rate": 0.0001833865814696486,
+      "loss": 0.6592,
+      "step": 31
+    },
+    {
+      "epoch": 0.30117647058823527,
+      "grad_norm": 0.06012285500764847,
+      "learning_rate": 0.00018274760383386583,
+      "loss": 0.6662,
+      "step": 32
+    },
+    {
+      "epoch": 0.31058823529411766,
+      "grad_norm": 0.08650115132331848,
+      "learning_rate": 0.00018210862619808308,
+      "loss": 0.6546,
+      "step": 33
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.058147966861724854,
+      "learning_rate": 0.00018146964856230032,
+      "loss": 0.6505,
+      "step": 34
+    },
+    {
+      "epoch": 0.32941176470588235,
+      "grad_norm": 0.054181504994630814,
+      "learning_rate": 0.00018083067092651756,
+      "loss": 0.6428,
+      "step": 35
+    },
+    {
+      "epoch": 0.3388235294117647,
+      "grad_norm": 0.05629313737154007,
+      "learning_rate": 0.00018019169329073484,
+      "loss": 0.6916,
+      "step": 36
+    },
+    {
+      "epoch": 0.34823529411764703,
+      "grad_norm": 0.05400744453072548,
+      "learning_rate": 0.00017955271565495208,
+      "loss": 0.6646,
+      "step": 37
+    },
+    {
+      "epoch": 0.35764705882352943,
+      "grad_norm": 0.053580883890390396,
+      "learning_rate": 0.00017891373801916932,
+      "loss": 0.6297,
+      "step": 38
+    },
+    {
+      "epoch": 0.36705882352941177,
+      "grad_norm": 0.05560213327407837,
+      "learning_rate": 0.0001782747603833866,
+      "loss": 0.6284,
+      "step": 39
+    },
+    {
+      "epoch": 0.3764705882352941,
+      "grad_norm": 0.05281270295381546,
+      "learning_rate": 0.00017763578274760384,
+      "loss": 0.6392,
+      "step": 40
+    },
+    {
+      "epoch": 0.38588235294117645,
+      "grad_norm": 0.06260088086128235,
+      "learning_rate": 0.00017699680511182108,
+      "loss": 0.7056,
+      "step": 41
+    },
+    {
+      "epoch": 0.3952941176470588,
+      "grad_norm": 0.058404579758644104,
+      "learning_rate": 0.00017635782747603835,
+      "loss": 0.6138,
+      "step": 42
+    },
+    {
+      "epoch": 0.4047058823529412,
+      "grad_norm": 0.050437066704034805,
+      "learning_rate": 0.0001757188498402556,
+      "loss": 0.6569,
+      "step": 43
+    },
+    {
+      "epoch": 0.41411764705882353,
+      "grad_norm": 0.05336596816778183,
+      "learning_rate": 0.00017507987220447287,
+      "loss": 0.612,
+      "step": 44
+    },
+    {
+      "epoch": 0.4235294117647059,
+      "grad_norm": 0.05253816023468971,
+      "learning_rate": 0.0001744408945686901,
+      "loss": 0.6195,
+      "step": 45
+    },
+    {
+      "epoch": 0.4329411764705882,
+      "grad_norm": 0.05323391407728195,
+      "learning_rate": 0.00017380191693290735,
+      "loss": 0.624,
+      "step": 46
+    },
+    {
+      "epoch": 0.4423529411764706,
+      "grad_norm": 0.05454220622777939,
+      "learning_rate": 0.00017316293929712462,
+      "loss": 0.6654,
+      "step": 47
+    },
+    {
+      "epoch": 0.45176470588235296,
+      "grad_norm": 0.05840761214494705,
+      "learning_rate": 0.00017252396166134187,
+      "loss": 0.6791,
+      "step": 48
+    },
+    {
+      "epoch": 0.4611764705882353,
+      "grad_norm": 0.05933906137943268,
+      "learning_rate": 0.0001718849840255591,
+      "loss": 0.6501,
+      "step": 49
+    },
+    {
+      "epoch": 0.47058823529411764,
+      "grad_norm": 0.06150069832801819,
+      "learning_rate": 0.00017124600638977638,
+      "loss": 0.6332,
+      "step": 50
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.054139360785484314,
+      "learning_rate": 0.00017060702875399363,
+      "loss": 0.6268,
+      "step": 51
+    },
+    {
+      "epoch": 0.4894117647058824,
+      "grad_norm": 0.05578823760151863,
+      "learning_rate": 0.00016996805111821087,
+      "loss": 0.6259,
+      "step": 52
+    },
+    {
+      "epoch": 0.4988235294117647,
+      "grad_norm": 0.06352369487285614,
+      "learning_rate": 0.00016932907348242811,
+      "loss": 0.6271,
+      "step": 53
+    },
+    {
+      "epoch": 0.508235294117647,
+      "grad_norm": 0.06086834892630577,
+      "learning_rate": 0.00016869009584664536,
+      "loss": 0.631,
+      "step": 54
+    },
+    {
+      "epoch": 0.5176470588235295,
+      "grad_norm": 0.05695689842104912,
+      "learning_rate": 0.00016805111821086263,
+      "loss": 0.64,
+      "step": 55
+    },
+    {
+      "epoch": 0.5270588235294118,
+      "grad_norm": 0.056874532252550125,
+      "learning_rate": 0.00016741214057507987,
+      "loss": 0.6296,
+      "step": 56
+    },
+    {
+      "epoch": 0.5364705882352941,
+      "grad_norm": 0.061433881521224976,
+      "learning_rate": 0.00016677316293929712,
+      "loss": 0.683,
+      "step": 57
+    },
+    {
+      "epoch": 0.5458823529411765,
+      "grad_norm": 0.06481802463531494,
+      "learning_rate": 0.0001661341853035144,
+      "loss": 0.599,
+      "step": 58
+    },
+    {
+      "epoch": 0.5552941176470588,
+      "grad_norm": 0.06558655947446823,
+      "learning_rate": 0.00016549520766773163,
+      "loss": 0.6362,
+      "step": 59
+    },
+    {
+      "epoch": 0.5647058823529412,
+      "grad_norm": 0.057915814220905304,
+      "learning_rate": 0.0001648562300319489,
+      "loss": 0.6396,
+      "step": 60
+    },
+    {
+      "epoch": 0.5741176470588235,
+      "grad_norm": 0.06019767373800278,
+      "learning_rate": 0.00016421725239616614,
+      "loss": 0.6444,
+      "step": 61
+    },
+    {
+      "epoch": 0.5835294117647059,
+      "grad_norm": 0.06425336748361588,
+      "learning_rate": 0.0001635782747603834,
+      "loss": 0.6175,
+      "step": 62
+    },
+    {
+      "epoch": 0.5929411764705882,
+      "grad_norm": 0.062431298196315765,
+      "learning_rate": 0.00016293929712460066,
+      "loss": 0.6458,
+      "step": 63
+    },
+    {
+      "epoch": 0.6023529411764705,
+      "grad_norm": 0.057641636580228806,
+      "learning_rate": 0.0001623003194888179,
+      "loss": 0.6262,
+      "step": 64
+    },
+    {
+      "epoch": 0.611764705882353,
+      "grad_norm": 0.05894685909152031,
+      "learning_rate": 0.00016166134185303515,
+      "loss": 0.6036,
+      "step": 65
+    },
+    {
+      "epoch": 0.6211764705882353,
+      "grad_norm": 0.06207429617643356,
+      "learning_rate": 0.00016102236421725242,
+      "loss": 0.6367,
+      "step": 66
+    },
+    {
+      "epoch": 0.6305882352941177,
+      "grad_norm": 0.06500203162431717,
+      "learning_rate": 0.00016038338658146966,
+      "loss": 0.6443,
+      "step": 67
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.06465457379817963,
+      "learning_rate": 0.0001597444089456869,
+      "loss": 0.6105,
+      "step": 68
+    },
+    {
+      "epoch": 0.6494117647058824,
+      "grad_norm": 0.060090478509664536,
+      "learning_rate": 0.00015910543130990418,
+      "loss": 0.6447,
+      "step": 69
+    },
+    {
+      "epoch": 0.6588235294117647,
+      "grad_norm": 0.06724690645933151,
+      "learning_rate": 0.00015846645367412142,
+      "loss": 0.6062,
+      "step": 70
+    },
+    {
+      "epoch": 0.668235294117647,
+      "grad_norm": 0.06291332095861435,
+      "learning_rate": 0.00015782747603833866,
+      "loss": 0.6005,
+      "step": 71
+    },
+    {
+      "epoch": 0.6776470588235294,
+      "grad_norm": 0.06592954695224762,
+      "learning_rate": 0.0001571884984025559,
+      "loss": 0.6036,
+      "step": 72
+    },
+    {
+      "epoch": 0.6870588235294117,
+      "grad_norm": 0.06050164997577667,
+      "learning_rate": 0.00015654952076677315,
+      "loss": 0.5993,
+      "step": 73
+    },
+    {
+      "epoch": 0.6964705882352941,
+      "grad_norm": 0.061909306794404984,
+      "learning_rate": 0.00015591054313099042,
+      "loss": 0.6159,
+      "step": 74
+    },
+    {
+      "epoch": 0.7058823529411765,
+      "grad_norm": 0.06927211582660675,
+      "learning_rate": 0.00015527156549520767,
+      "loss": 0.6554,
+      "step": 75
+    },
+    {
+      "epoch": 0.7152941176470589,
+      "grad_norm": 0.06098336726427078,
+      "learning_rate": 0.00015463258785942494,
+      "loss": 0.5998,
+      "step": 76
+    },
+    {
+      "epoch": 0.7247058823529412,
+      "grad_norm": 0.06612212210893631,
+      "learning_rate": 0.00015399361022364218,
+      "loss": 0.6057,
+      "step": 77
+    },
+    {
+      "epoch": 0.7341176470588235,
+      "grad_norm": 0.06455227732658386,
+      "learning_rate": 0.00015335463258785942,
+      "loss": 0.64,
+      "step": 78
+    },
+    {
+      "epoch": 0.7435294117647059,
+      "grad_norm": 0.40494394302368164,
+      "learning_rate": 0.0001527156549520767,
+      "loss": 0.6397,
+      "step": 79
+    },
+    {
+      "epoch": 0.7529411764705882,
+      "grad_norm": 0.05910542979836464,
+      "learning_rate": 0.00015207667731629394,
+      "loss": 0.6087,
+      "step": 80
+    },
+    {
+      "epoch": 0.7623529411764706,
+      "grad_norm": 0.06125890091061592,
+      "learning_rate": 0.00015143769968051118,
+      "loss": 0.6133,
+      "step": 81
+    },
+    {
+      "epoch": 0.7717647058823529,
+      "grad_norm": 0.0716337338089943,
+      "learning_rate": 0.00015079872204472845,
+      "loss": 0.6136,
+      "step": 82
+    },
+    {
+      "epoch": 0.7811764705882352,
+      "grad_norm": 0.07003322243690491,
+      "learning_rate": 0.0001501597444089457,
+      "loss": 0.6236,
+      "step": 83
+    },
+    {
+      "epoch": 0.7905882352941176,
+      "grad_norm": 0.06324715912342072,
+      "learning_rate": 0.00014952076677316297,
+      "loss": 0.6282,
+      "step": 84
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.07135389745235443,
+      "learning_rate": 0.0001488817891373802,
+      "loss": 0.6443,
+      "step": 85
+    },
+    {
+      "epoch": 0.8094117647058824,
+      "grad_norm": 0.06127123162150383,
+      "learning_rate": 0.00014824281150159745,
+      "loss": 0.6396,
+      "step": 86
+    },
+    {
+      "epoch": 0.8188235294117647,
+      "grad_norm": 0.06172896921634674,
+      "learning_rate": 0.0001476038338658147,
+      "loss": 0.6377,
+      "step": 87
+    },
+    {
+      "epoch": 0.8282352941176471,
+      "grad_norm": 0.3543515205383301,
+      "learning_rate": 0.00014696485623003194,
+      "loss": 0.6454,
+      "step": 88
+    },
+    {
+      "epoch": 0.8376470588235294,
+      "grad_norm": 0.06527980417013168,
+      "learning_rate": 0.0001463258785942492,
+      "loss": 0.615,
+      "step": 89
+    },
+    {
+      "epoch": 0.8470588235294118,
+      "grad_norm": 0.06327975541353226,
+      "learning_rate": 0.00014568690095846646,
+      "loss": 0.6447,
+      "step": 90
+    },
+    {
+      "epoch": 0.8564705882352941,
+      "grad_norm": 0.06739926338195801,
+      "learning_rate": 0.0001450479233226837,
+      "loss": 0.6087,
+      "step": 91
+    },
+    {
+      "epoch": 0.8658823529411764,
+      "grad_norm": 0.06617800146341324,
+      "learning_rate": 0.00014440894568690097,
+      "loss": 0.6259,
+      "step": 92
+    },
+    {
+      "epoch": 0.8752941176470588,
+      "grad_norm": 0.06242835149168968,
+      "learning_rate": 0.00014376996805111821,
+      "loss": 0.6049,
+      "step": 93
+    },
+    {
+      "epoch": 0.8847058823529412,
+      "grad_norm": 0.06359254568815231,
+      "learning_rate": 0.00014313099041533546,
+      "loss": 0.6274,
+      "step": 94
+    },
+    {
+      "epoch": 0.8941176470588236,
+      "grad_norm": 0.06559597700834274,
+      "learning_rate": 0.00014249201277955273,
+      "loss": 0.5935,
+      "step": 95
+    },
+    {
+      "epoch": 0.9035294117647059,
+      "grad_norm": 0.07493000477552414,
+      "learning_rate": 0.00014185303514376997,
+      "loss": 0.5881,
+      "step": 96
+    },
+    {
+      "epoch": 0.9035294117647059,
+      "eval_loss": 0.6278424859046936,
+      "eval_runtime": 5.469,
+      "eval_samples_per_second": 8.045,
+      "eval_steps_per_second": 1.097,
+      "step": 96
+    },
+    {
+      "epoch": 0.9129411764705883,
+      "grad_norm": 0.06714215129613876,
+      "learning_rate": 0.00014121405750798722,
+      "loss": 0.6438,
+      "step": 97
+    },
+    {
+      "epoch": 0.9223529411764706,
+      "grad_norm": 0.06775335222482681,
+      "learning_rate": 0.0001405750798722045,
+      "loss": 0.6256,
+      "step": 98
+    },
+    {
+      "epoch": 0.9317647058823529,
+      "grad_norm": 0.07045792043209076,
+      "learning_rate": 0.00013993610223642173,
+      "loss": 0.6712,
+      "step": 99
+    },
+    {
+      "epoch": 0.9411764705882353,
+      "grad_norm": 0.06958217918872833,
+      "learning_rate": 0.000139297124600639,
+      "loss": 0.6278,
+      "step": 100
+    },
+    {
+      "epoch": 0.9505882352941176,
+      "grad_norm": 0.06508302688598633,
+      "learning_rate": 0.00013865814696485625,
+      "loss": 0.6415,
+      "step": 101
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.06876156479120255,
+      "learning_rate": 0.0001380191693290735,
+      "loss": 0.6211,
+      "step": 102
+    },
+    {
+      "epoch": 0.9694117647058823,
+      "grad_norm": 0.07631471753120422,
+      "learning_rate": 0.00013738019169329076,
+      "loss": 0.6276,
+      "step": 103
+    },
+    {
+      "epoch": 0.9788235294117648,
+      "grad_norm": 0.07122895866632462,
+      "learning_rate": 0.000136741214057508,
+      "loss": 0.5997,
+      "step": 104
+    },
+    {
+      "epoch": 0.9882352941176471,
+      "grad_norm": 0.06862416863441467,
+      "learning_rate": 0.00013610223642172525,
+      "loss": 0.6563,
+      "step": 105
+    },
+    {
+      "epoch": 0.9976470588235294,
+      "grad_norm": 0.06701315939426422,
+      "learning_rate": 0.0001354632587859425,
+      "loss": 0.61,
+      "step": 106
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.14547984302043915,
+      "learning_rate": 0.00013482428115015973,
+      "loss": 0.6382,
+      "step": 107
+    },
+    {
+      "epoch": 1.0094117647058825,
+      "grad_norm": 0.0676453560590744,
+      "learning_rate": 0.000134185303514377,
+      "loss": 0.5766,
+      "step": 108
+    },
+    {
+      "epoch": 1.0188235294117647,
+      "grad_norm": 0.06471403688192368,
+      "learning_rate": 0.00013354632587859425,
+      "loss": 0.5769,
+      "step": 109
+    },
+    {
+      "epoch": 1.0282352941176471,
+      "grad_norm": 0.07038411498069763,
+      "learning_rate": 0.0001329073482428115,
+      "loss": 0.5828,
+      "step": 110
+    },
+    {
+      "epoch": 1.0376470588235294,
+      "grad_norm": 0.0679129809141159,
+      "learning_rate": 0.00013226837060702876,
+      "loss": 0.5755,
+      "step": 111
+    },
+    {
+      "epoch": 1.0470588235294118,
+      "grad_norm": 0.07297012954950333,
+      "learning_rate": 0.000131629392971246,
+      "loss": 0.6114,
+      "step": 112
+    },
+    {
+      "epoch": 1.056470588235294,
+      "grad_norm": 0.07036986202001572,
+      "learning_rate": 0.00013099041533546325,
+      "loss": 0.6254,
+      "step": 113
+    },
+    {
+      "epoch": 1.0658823529411765,
+      "grad_norm": 0.08385265618562698,
+      "learning_rate": 0.00013035143769968052,
+      "loss": 0.6284,
+      "step": 114
+    },
+    {
+      "epoch": 1.0752941176470587,
+      "grad_norm": 0.08367498964071274,
+      "learning_rate": 0.00012971246006389777,
+      "loss": 0.6128,
+      "step": 115
+    },
+    {
+      "epoch": 1.0847058823529412,
+      "grad_norm": 0.082453154027462,
+      "learning_rate": 0.000129073482428115,
+      "loss": 0.6012,
+      "step": 116
+    },
+    {
+      "epoch": 1.0941176470588236,
+      "grad_norm": 0.0714535042643547,
+      "learning_rate": 0.00012843450479233228,
+      "loss": 0.599,
+      "step": 117
+    },
+    {
+      "epoch": 1.1035294117647059,
+      "grad_norm": 0.07391790300607681,
+      "learning_rate": 0.00012779552715654952,
+      "loss": 0.5897,
+      "step": 118
+    },
+    {
+      "epoch": 1.1129411764705883,
+      "grad_norm": 0.08383256196975708,
+      "learning_rate": 0.0001271565495207668,
+      "loss": 0.634,
+      "step": 119
+    },
+    {
+      "epoch": 1.1223529411764706,
+      "grad_norm": 0.08821647614240646,
+      "learning_rate": 0.00012651757188498404,
+      "loss": 0.5869,
+      "step": 120
+    },
+    {
+      "epoch": 1.131764705882353,
+      "grad_norm": 0.07556191831827164,
+      "learning_rate": 0.00012587859424920128,
+      "loss": 0.6275,
+      "step": 121
+    },
+    {
+      "epoch": 1.1411764705882352,
+      "grad_norm": 0.08794881403446198,
+      "learning_rate": 0.00012523961661341855,
+      "loss": 0.6115,
+      "step": 122
+    },
+    {
+      "epoch": 1.1505882352941177,
+      "grad_norm": 0.0785142183303833,
+      "learning_rate": 0.0001246006389776358,
+      "loss": 0.6156,
+      "step": 123
+    },
+    {
+      "epoch": 1.16,
+      "grad_norm": 0.07684168219566345,
+      "learning_rate": 0.00012396166134185304,
+      "loss": 0.6107,
+      "step": 124
+    },
+    {
+      "epoch": 1.1694117647058824,
+      "grad_norm": 0.07763410359621048,
+      "learning_rate": 0.00012332268370607028,
+      "loss": 0.5888,
+      "step": 125
+    },
+    {
+      "epoch": 1.1788235294117646,
+      "grad_norm": 0.09084579348564148,
+      "learning_rate": 0.00012268370607028753,
+      "loss": 0.5912,
+      "step": 126
+    },
+    {
+      "epoch": 1.188235294117647,
+      "grad_norm": 0.0838058665394783,
+      "learning_rate": 0.00012204472843450481,
+      "loss": 0.6152,
+      "step": 127
+    },
+    {
+      "epoch": 1.1976470588235295,
+      "grad_norm": 0.08908473700284958,
+      "learning_rate": 0.00012140575079872206,
+      "loss": 0.6049,
+      "step": 128
+    },
+    {
+      "epoch": 1.2070588235294117,
+      "grad_norm": 0.08239720016717911,
+      "learning_rate": 0.0001207667731629393,
+      "loss": 0.5852,
+      "step": 129
+    },
+    {
+      "epoch": 1.2164705882352942,
+      "grad_norm": 0.08228597790002823,
+      "learning_rate": 0.00012012779552715656,
+      "loss": 0.5642,
+      "step": 130
+    },
+    {
+      "epoch": 1.2258823529411764,
+      "grad_norm": 0.08196559548377991,
+      "learning_rate": 0.0001194888178913738,
+      "loss": 0.5971,
+      "step": 131
+    },
+    {
+      "epoch": 1.2352941176470589,
+      "grad_norm": 0.08850298076868057,
+      "learning_rate": 0.00011884984025559104,
+      "loss": 0.6149,
+      "step": 132
+    },
+    {
+      "epoch": 1.244705882352941,
+      "grad_norm": 0.08643686771392822,
+      "learning_rate": 0.00011821086261980832,
+      "loss": 0.6097,
+      "step": 133
+    },
+    {
+      "epoch": 1.2541176470588236,
+      "grad_norm": 0.08877662569284439,
+      "learning_rate": 0.00011757188498402556,
+      "loss": 0.6362,
+      "step": 134
+    },
+    {
+      "epoch": 1.263529411764706,
+      "grad_norm": 0.08452433347702026,
+      "learning_rate": 0.00011693290734824283,
+      "loss": 0.6331,
+      "step": 135
+    },
+    {
+      "epoch": 1.2729411764705882,
+      "grad_norm": 0.08155957609415054,
+      "learning_rate": 0.00011629392971246007,
+      "loss": 0.6214,
+      "step": 136
+    },
+    {
+      "epoch": 1.2823529411764705,
+      "grad_norm": 0.08229539543390274,
+      "learning_rate": 0.00011565495207667732,
+      "loss": 0.5981,
+      "step": 137
+    },
+    {
+      "epoch": 1.291764705882353,
+      "grad_norm": 0.09338800609111786,
+      "learning_rate": 0.00011501597444089457,
+      "loss": 0.6326,
+      "step": 138
+    },
+    {
+      "epoch": 1.3011764705882354,
+      "grad_norm": 0.0837632343173027,
+      "learning_rate": 0.00011437699680511182,
+      "loss": 0.5736,
+      "step": 139
+    },
+    {
+      "epoch": 1.3105882352941176,
+      "grad_norm": 0.08500789105892181,
+      "learning_rate": 0.00011373801916932908,
+      "loss": 0.6253,
+      "step": 140
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.0843970999121666,
+      "learning_rate": 0.00011309904153354633,
+      "loss": 0.6053,
+      "step": 141
+    },
+    {
+      "epoch": 1.3294117647058823,
+      "grad_norm": 0.08848956227302551,
+      "learning_rate": 0.00011246006389776358,
+      "loss": 0.6184,
+      "step": 142
+    },
+    {
+      "epoch": 1.3388235294117647,
+      "grad_norm": 0.10718461126089096,
+      "learning_rate": 0.00011182108626198085,
+      "loss": 0.583,
+      "step": 143
+    },
+    {
+      "epoch": 1.348235294117647,
+      "grad_norm": 0.08756856620311737,
+      "learning_rate": 0.00011118210862619809,
+      "loss": 0.5861,
+      "step": 144
+    },
+    {
+      "epoch": 1.3576470588235294,
+      "grad_norm": 0.08492905646562576,
+      "learning_rate": 0.00011054313099041533,
+      "loss": 0.5707,
+      "step": 145
+    },
+    {
+      "epoch": 1.3670588235294119,
+      "grad_norm": 0.08758281171321869,
+      "learning_rate": 0.0001099041533546326,
+      "loss": 0.555,
+      "step": 146
+    },
+    {
+      "epoch": 1.3764705882352941,
+      "grad_norm": 0.09751519560813904,
+      "learning_rate": 0.00010926517571884985,
+      "loss": 0.5976,
+      "step": 147
+    },
+    {
+      "epoch": 1.3858823529411763,
+      "grad_norm": 0.09590818732976913,
+      "learning_rate": 0.00010862619808306709,
+      "loss": 0.6015,
+      "step": 148
+    },
+    {
+      "epoch": 1.3952941176470588,
+      "grad_norm": 0.08812163025140762,
+      "learning_rate": 0.00010798722044728435,
+      "loss": 0.5905,
+      "step": 149
+    },
+    {
+      "epoch": 1.4047058823529412,
+      "grad_norm": 0.09162674099206924,
+      "learning_rate": 0.0001073482428115016,
+      "loss": 0.6063,
+      "step": 150
+    },
+    {
+      "epoch": 1.4141176470588235,
+      "grad_norm": 0.08727669715881348,
+      "learning_rate": 0.00010670926517571886,
+      "loss": 0.6181,
+      "step": 151
+    },
+    {
+      "epoch": 1.423529411764706,
+      "grad_norm": 0.0928676649928093,
+      "learning_rate": 0.00010607028753993611,
+      "loss": 0.6188,
+      "step": 152
+    },
+    {
+      "epoch": 1.4329411764705882,
+      "grad_norm": 0.09681576490402222,
+      "learning_rate": 0.00010543130990415335,
+      "loss": 0.5818,
+      "step": 153
+    },
+    {
+      "epoch": 1.4423529411764706,
+      "grad_norm": 0.08795556426048279,
+      "learning_rate": 0.00010479233226837062,
+      "loss": 0.6134,
+      "step": 154
+    },
+    {
+      "epoch": 1.4517647058823528,
+      "grad_norm": 0.0861482247710228,
+      "learning_rate": 0.00010415335463258787,
+      "loss": 0.6082,
+      "step": 155
+    },
+    {
+      "epoch": 1.4611764705882353,
+      "grad_norm": 0.09977812319993973,
+      "learning_rate": 0.00010351437699680511,
+      "loss": 0.5921,
+      "step": 156
+    },
+    {
+      "epoch": 1.4705882352941178,
+      "grad_norm": 0.10190846771001816,
+      "learning_rate": 0.00010287539936102237,
+      "loss": 0.6271,
+      "step": 157
+    },
+    {
+      "epoch": 1.48,
+      "grad_norm": 0.09232486039400101,
+      "learning_rate": 0.00010223642172523961,
+      "loss": 0.6067,
+      "step": 158
+    },
+    {
+      "epoch": 1.4894117647058824,
+      "grad_norm": 0.10211990028619766,
+      "learning_rate": 0.00010159744408945688,
+      "loss": 0.6238,
+      "step": 159
+    },
+    {
+      "epoch": 1.4988235294117647,
+      "grad_norm": 0.08955651521682739,
+      "learning_rate": 0.00010095846645367413,
+      "loss": 0.6002,
+      "step": 160
+    },
+    {
+      "epoch": 1.5082352941176471,
+      "grad_norm": 0.10033038258552551,
+      "learning_rate": 0.00010031948881789137,
+      "loss": 0.6129,
+      "step": 161
+    },
+    {
+      "epoch": 1.5176470588235293,
+      "grad_norm": 0.09701602160930634,
+      "learning_rate": 9.968051118210863e-05,
+      "loss": 0.5867,
+      "step": 162
+    },
+    {
+      "epoch": 1.5270588235294118,
+      "grad_norm": 0.0969882607460022,
+      "learning_rate": 9.904153354632588e-05,
+      "loss": 0.599,
+      "step": 163
+    },
+    {
+      "epoch": 1.5364705882352943,
+      "grad_norm": 0.09163233637809753,
+      "learning_rate": 9.840255591054314e-05,
+      "loss": 0.6078,
+      "step": 164
+    },
+    {
+      "epoch": 1.5458823529411765,
+      "grad_norm": 0.0925818681716919,
+      "learning_rate": 9.77635782747604e-05,
+      "loss": 0.6231,
+      "step": 165
+    },
+    {
+      "epoch": 1.5552941176470587,
+      "grad_norm": 0.09490446001291275,
+      "learning_rate": 9.712460063897764e-05,
+      "loss": 0.6099,
+      "step": 166
+    },
+    {
+      "epoch": 1.5647058823529412,
+      "grad_norm": 0.10370604693889618,
+      "learning_rate": 9.648562300319489e-05,
+      "loss": 0.6133,
+      "step": 167
+    },
+    {
+      "epoch": 1.5741176470588236,
+      "grad_norm": 0.10043718665838242,
+      "learning_rate": 9.584664536741214e-05,
+      "loss": 0.6002,
+      "step": 168
+    },
+    {
+      "epoch": 1.5835294117647059,
+      "grad_norm": 0.10102451592683792,
+      "learning_rate": 9.52076677316294e-05,
+      "loss": 0.6326,
+      "step": 169
+    },
+    {
+      "epoch": 1.592941176470588,
+      "grad_norm": 0.09170352667570114,
+      "learning_rate": 9.456869009584664e-05,
+      "loss": 0.5725,
+      "step": 170
+    },
+    {
+      "epoch": 1.6023529411764705,
+      "grad_norm": 0.09696916490793228,
+      "learning_rate": 9.39297124600639e-05,
+      "loss": 0.6162,
+      "step": 171
+    },
+    {
+      "epoch": 1.611764705882353,
+      "grad_norm": 0.1032591313123703,
+      "learning_rate": 9.329073482428116e-05,
+      "loss": 0.6022,
+      "step": 172
+    },
+    {
+      "epoch": 1.6211764705882352,
+      "grad_norm": 0.09963531792163849,
+      "learning_rate": 9.265175718849842e-05,
+      "loss": 0.5918,
+      "step": 173
+    },
+    {
+      "epoch": 1.6305882352941177,
+      "grad_norm": 0.10566572844982147,
+      "learning_rate": 9.201277955271566e-05,
+      "loss": 0.6248,
+      "step": 174
+    },
+    {
+      "epoch": 1.6400000000000001,
+      "grad_norm": 0.09654838591814041,
+      "learning_rate": 9.137380191693292e-05,
+      "loss": 0.5747,
+      "step": 175
+    },
+    {
+      "epoch": 1.6494117647058824,
+      "grad_norm": 0.09412531554698944,
+      "learning_rate": 9.073482428115016e-05,
+      "loss": 0.611,
+      "step": 176
+    },
+    {
+      "epoch": 1.6588235294117646,
+      "grad_norm": 0.10707147419452667,
+      "learning_rate": 9.009584664536742e-05,
+      "loss": 0.6371,
+      "step": 177
+    },
+    {
+      "epoch": 1.668235294117647,
+      "grad_norm": 0.0976637527346611,
+      "learning_rate": 8.945686900958466e-05,
+      "loss": 0.636,
+      "step": 178
+    },
+    {
+      "epoch": 1.6776470588235295,
+      "grad_norm": 0.09769190102815628,
+      "learning_rate": 8.881789137380192e-05,
+      "loss": 0.6379,
+      "step": 179
+    },
+    {
+      "epoch": 1.6870588235294117,
+      "grad_norm": 0.1029646173119545,
+      "learning_rate": 8.817891373801918e-05,
+      "loss": 0.6533,
+      "step": 180
+    },
+    {
+      "epoch": 1.696470588235294,
+      "grad_norm": 0.10120602697134018,
+      "learning_rate": 8.753993610223643e-05,
+      "loss": 0.6044,
+      "step": 181
+    },
+    {
+      "epoch": 1.7058823529411766,
+      "grad_norm": 0.0992206409573555,
+      "learning_rate": 8.690095846645368e-05,
+      "loss": 0.6072,
+      "step": 182
+    },
+    {
+      "epoch": 1.7152941176470589,
+      "grad_norm": 0.10124579071998596,
+      "learning_rate": 8.626198083067093e-05,
+      "loss": 0.5923,
+      "step": 183
+    },
+    {
+      "epoch": 1.724705882352941,
+      "grad_norm": 0.10673081874847412,
+      "learning_rate": 8.562300319488819e-05,
+      "loss": 0.5964,
+      "step": 184
+    },
+    {
+      "epoch": 1.7341176470588235,
+      "grad_norm": 0.10745410621166229,
+      "learning_rate": 8.498402555910544e-05,
+      "loss": 0.6052,
+      "step": 185
+    },
+    {
+      "epoch": 1.743529411764706,
+      "grad_norm": 0.10404065251350403,
+      "learning_rate": 8.434504792332268e-05,
+      "loss": 0.5975,
+      "step": 186
+    },
+    {
+      "epoch": 1.7529411764705882,
+      "grad_norm": 0.09927040338516235,
+      "learning_rate": 8.370607028753994e-05,
+      "loss": 0.5901,
+      "step": 187
+    },
+    {
+      "epoch": 1.7623529411764705,
+      "grad_norm": 0.10322125256061554,
+      "learning_rate": 8.30670926517572e-05,
+      "loss": 0.6154,
+      "step": 188
+    },
+    {
+      "epoch": 1.771764705882353,
+      "grad_norm": 0.10899791866540909,
+      "learning_rate": 8.242811501597445e-05,
+      "loss": 0.6255,
+      "step": 189
+    },
+    {
+      "epoch": 1.7811764705882354,
+      "grad_norm": 0.1041494831442833,
+      "learning_rate": 8.17891373801917e-05,
+      "loss": 0.5925,
+      "step": 190
+    },
+    {
+      "epoch": 1.7905882352941176,
+      "grad_norm": 0.09239751845598221,
+      "learning_rate": 8.115015974440895e-05,
+      "loss": 0.5713,
+      "step": 191
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.09495066851377487,
+      "learning_rate": 8.051118210862621e-05,
+      "loss": 0.5896,
+      "step": 192
+    },
+    {
+      "epoch": 1.8,
+      "eval_loss": 0.6196094155311584,
+      "eval_runtime": 5.4299,
+      "eval_samples_per_second": 8.103,
+      "eval_steps_per_second": 1.105,
+      "step": 192
+    },
+    {
+      "epoch": 1.8094117647058825,
+      "grad_norm": 0.09602215141057968,
+      "learning_rate": 7.987220447284345e-05,
+      "loss": 0.5979,
+      "step": 193
+    },
+    {
+      "epoch": 1.8188235294117647,
+      "grad_norm": 0.10422977060079575,
+      "learning_rate": 7.923322683706071e-05,
+      "loss": 0.5704,
+      "step": 194
+    },
+    {
+      "epoch": 1.828235294117647,
+      "grad_norm": 0.09678755700588226,
+      "learning_rate": 7.859424920127795e-05,
+      "loss": 0.5774,
+      "step": 195
+    },
+    {
+      "epoch": 1.8376470588235294,
+      "grad_norm": 0.09845128655433655,
+      "learning_rate": 7.795527156549521e-05,
+      "loss": 0.5816,
+      "step": 196
+    },
+    {
+      "epoch": 1.8470588235294119,
+      "grad_norm": 0.1065673753619194,
+      "learning_rate": 7.731629392971247e-05,
+      "loss": 0.6135,
+      "step": 197
+    },
+    {
+      "epoch": 1.856470588235294,
+      "grad_norm": 0.10570935904979706,
+      "learning_rate": 7.667731629392971e-05,
+      "loss": 0.5908,
+      "step": 198
+    },
+    {
+      "epoch": 1.8658823529411763,
+      "grad_norm": 0.09649720042943954,
+      "learning_rate": 7.603833865814697e-05,
+      "loss": 0.599,
+      "step": 199
+    },
+    {
+      "epoch": 1.8752941176470588,
+      "grad_norm": 0.10398146510124207,
+      "learning_rate": 7.539936102236423e-05,
+      "loss": 0.5853,
+      "step": 200
+    },
+    {
+      "epoch": 1.8847058823529412,
+      "grad_norm": 0.09633094817399979,
+      "learning_rate": 7.476038338658148e-05,
+      "loss": 0.5927,
+      "step": 201
+    },
+    {
+      "epoch": 1.8941176470588235,
+      "grad_norm": 0.0993979275226593,
+      "learning_rate": 7.412140575079873e-05,
+      "loss": 0.5928,
+      "step": 202
+    },
+    {
+      "epoch": 1.903529411764706,
+      "grad_norm": 0.10687059164047241,
+      "learning_rate": 7.348242811501597e-05,
+      "loss": 0.5882,
+      "step": 203
+    },
+    {
+      "epoch": 1.9129411764705884,
+      "grad_norm": 0.10128702223300934,
+      "learning_rate": 7.284345047923323e-05,
+      "loss": 0.6009,
+      "step": 204
+    },
+    {
+      "epoch": 1.9223529411764706,
+      "grad_norm": 0.10716967284679413,
+      "learning_rate": 7.220447284345049e-05,
+      "loss": 0.6198,
+      "step": 205
+    },
+    {
+      "epoch": 1.9317647058823528,
+      "grad_norm": 0.10603494942188263,
+      "learning_rate": 7.156549520766773e-05,
+      "loss": 0.6222,
+      "step": 206
+    },
+    {
+      "epoch": 1.9411764705882353,
+      "grad_norm": 0.10125169903039932,
+      "learning_rate": 7.092651757188499e-05,
+      "loss": 0.5938,
+      "step": 207
+    },
+    {
+      "epoch": 1.9505882352941177,
+      "grad_norm": 0.10542251914739609,
+      "learning_rate": 7.028753993610224e-05,
+      "loss": 0.6341,
+      "step": 208
+    },
+    {
+      "epoch": 1.96,
+      "grad_norm": 0.09659598022699356,
+      "learning_rate": 6.96485623003195e-05,
+      "loss": 0.59,
+      "step": 209
+    },
+    {
+      "epoch": 1.9694117647058822,
+      "grad_norm": 0.1122719794511795,
+      "learning_rate": 6.900958466453674e-05,
+      "loss": 0.6109,
+      "step": 210
+    },
+    {
+      "epoch": 1.9788235294117649,
+      "grad_norm": 0.11106933653354645,
+      "learning_rate": 6.8370607028754e-05,
+      "loss": 0.58,
+      "step": 211
+    },
+    {
+      "epoch": 1.988235294117647,
+      "grad_norm": 0.09962272644042969,
+      "learning_rate": 6.773162939297125e-05,
+      "loss": 0.6105,
+      "step": 212
+    },
+    {
+      "epoch": 1.9976470588235293,
+      "grad_norm": 0.09547895193099976,
+      "learning_rate": 6.70926517571885e-05,
+      "loss": 0.5709,
+      "step": 213
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.21181434392929077,
+      "learning_rate": 6.645367412140575e-05,
+      "loss": 0.6164,
+      "step": 214
+    },
+    {
+      "epoch": 2.0094117647058822,
+      "grad_norm": 0.11414559185504913,
+      "learning_rate": 6.5814696485623e-05,
+      "loss": 0.5642,
+      "step": 215
+    },
+    {
+      "epoch": 2.018823529411765,
+      "grad_norm": 0.11991499364376068,
+      "learning_rate": 6.517571884984026e-05,
+      "loss": 0.5741,
+      "step": 216
+    },
+    {
+      "epoch": 2.028235294117647,
+      "grad_norm": 0.10295461863279343,
+      "learning_rate": 6.45367412140575e-05,
+      "loss": 0.5604,
+      "step": 217
+    },
+    {
+      "epoch": 2.0376470588235294,
+      "grad_norm": 0.10355547070503235,
+      "learning_rate": 6.389776357827476e-05,
+      "loss": 0.5884,
+      "step": 218
+    },
+    {
+      "epoch": 2.0470588235294116,
+      "grad_norm": 0.12203363329172134,
+      "learning_rate": 6.325878594249202e-05,
+      "loss": 0.5903,
+      "step": 219
+    },
+    {
+      "epoch": 2.0564705882352943,
+      "grad_norm": 0.12483850866556168,
+      "learning_rate": 6.261980830670928e-05,
+      "loss": 0.5666,
+      "step": 220
+    },
+    {
+      "epoch": 2.0658823529411765,
+      "grad_norm": 0.12819188833236694,
+      "learning_rate": 6.198083067092652e-05,
+      "loss": 0.5493,
+      "step": 221
+    },
+    {
+      "epoch": 2.0752941176470587,
+      "grad_norm": 0.11613185703754425,
+      "learning_rate": 6.134185303514376e-05,
+      "loss": 0.6121,
+      "step": 222
+    },
+    {
+      "epoch": 2.084705882352941,
+      "grad_norm": 0.11060836911201477,
+      "learning_rate": 6.070287539936103e-05,
+      "loss": 0.5748,
+      "step": 223
+    },
+    {
+      "epoch": 2.0941176470588236,
+      "grad_norm": 0.13958008587360382,
+      "learning_rate": 6.006389776357828e-05,
+      "loss": 0.62,
+      "step": 224
+    },
+    {
+      "epoch": 2.103529411764706,
+      "grad_norm": 0.14533892273902893,
+      "learning_rate": 5.942492012779552e-05,
+      "loss": 0.5897,
+      "step": 225
+    },
+    {
+      "epoch": 2.112941176470588,
+      "grad_norm": 0.12220677733421326,
+      "learning_rate": 5.878594249201278e-05,
+      "loss": 0.5508,
+      "step": 226
+    },
+    {
+      "epoch": 2.1223529411764708,
+      "grad_norm": 0.11975925415754318,
+      "learning_rate": 5.814696485623004e-05,
+      "loss": 0.5483,
+      "step": 227
+    },
+    {
+      "epoch": 2.131764705882353,
+      "grad_norm": 0.12308664619922638,
+      "learning_rate": 5.750798722044729e-05,
+      "loss": 0.5921,
+      "step": 228
+    },
+    {
+      "epoch": 2.1411764705882352,
+      "grad_norm": 0.12570548057556152,
+      "learning_rate": 5.686900958466454e-05,
+      "loss": 0.6229,
+      "step": 229
+    },
+    {
+      "epoch": 2.1505882352941175,
+      "grad_norm": 0.12253190577030182,
+      "learning_rate": 5.623003194888179e-05,
+      "loss": 0.5758,
+      "step": 230
+    },
+    {
+      "epoch": 2.16,
+      "grad_norm": 0.1249895840883255,
+      "learning_rate": 5.5591054313099045e-05,
+      "loss": 0.5633,
+      "step": 231
+    },
+    {
+      "epoch": 2.1694117647058824,
+      "grad_norm": 0.1355949193239212,
+      "learning_rate": 5.49520766773163e-05,
+      "loss": 0.5486,
+      "step": 232
+    },
+    {
+      "epoch": 2.1788235294117646,
+      "grad_norm": 0.12255489826202393,
+      "learning_rate": 5.4313099041533546e-05,
+      "loss": 0.5906,
+      "step": 233
+    },
+    {
+      "epoch": 2.1882352941176473,
+      "grad_norm": 0.12850622832775116,
+      "learning_rate": 5.36741214057508e-05,
+      "loss": 0.5738,
+      "step": 234
+    },
+    {
+      "epoch": 2.1976470588235295,
+      "grad_norm": 0.18490703403949738,
+      "learning_rate": 5.3035143769968054e-05,
+      "loss": 0.5737,
+      "step": 235
+    },
+    {
+      "epoch": 2.2070588235294117,
+      "grad_norm": 0.135318785905838,
+      "learning_rate": 5.239616613418531e-05,
+      "loss": 0.5963,
+      "step": 236
+    },
+    {
+      "epoch": 2.216470588235294,
+      "grad_norm": 0.12884393334388733,
+      "learning_rate": 5.1757188498402555e-05,
+      "loss": 0.5399,
+      "step": 237
+    },
+    {
+      "epoch": 2.2258823529411766,
+      "grad_norm": 0.13065044581890106,
+      "learning_rate": 5.1118210862619806e-05,
+      "loss": 0.5611,
+      "step": 238
+    },
+    {
+      "epoch": 2.235294117647059,
+      "grad_norm": 0.12230806797742844,
+      "learning_rate": 5.047923322683706e-05,
+      "loss": 0.5446,
+      "step": 239
+    },
+    {
+      "epoch": 2.244705882352941,
+      "grad_norm": 0.13359951972961426,
+      "learning_rate": 4.984025559105431e-05,
+      "loss": 0.5652,
+      "step": 240
+    },
+    {
+      "epoch": 2.2541176470588233,
+      "grad_norm": 0.1409720480442047,
+      "learning_rate": 4.920127795527157e-05,
+      "loss": 0.5437,
+      "step": 241
+    },
+    {
+      "epoch": 2.263529411764706,
+      "grad_norm": 0.1343528777360916,
+      "learning_rate": 4.856230031948882e-05,
+      "loss": 0.5811,
+      "step": 242
+    },
+    {
+      "epoch": 2.2729411764705882,
+      "grad_norm": 0.1315775066614151,
+      "learning_rate": 4.792332268370607e-05,
+      "loss": 0.5718,
+      "step": 243
+    },
+    {
+      "epoch": 2.2823529411764705,
+      "grad_norm": 0.1300155520439148,
+      "learning_rate": 4.728434504792332e-05,
+      "loss": 0.5629,
+      "step": 244
+    },
+    {
+      "epoch": 2.291764705882353,
+      "grad_norm": 0.1364685744047165,
+      "learning_rate": 4.664536741214058e-05,
+      "loss": 0.5641,
+      "step": 245
+    },
+    {
+      "epoch": 2.3011764705882354,
+      "grad_norm": 0.13575628399848938,
+      "learning_rate": 4.600638977635783e-05,
+      "loss": 0.5861,
+      "step": 246
+    },
+    {
+      "epoch": 2.3105882352941176,
+      "grad_norm": 0.13364288210868835,
+      "learning_rate": 4.536741214057508e-05,
+      "loss": 0.5959,
+      "step": 247
+    },
+    {
+      "epoch": 2.32,
+      "grad_norm": 0.129843071103096,
+      "learning_rate": 4.472843450479233e-05,
+      "loss": 0.5631,
+      "step": 248
+    },
+    {
+      "epoch": 2.3294117647058825,
+      "grad_norm": 0.13373443484306335,
+      "learning_rate": 4.408945686900959e-05,
+      "loss": 0.557,
+      "step": 249
+    },
+    {
+      "epoch": 2.3388235294117647,
+      "grad_norm": 0.14091914892196655,
+      "learning_rate": 4.345047923322684e-05,
+      "loss": 0.577,
+      "step": 250
+    },
+    {
+      "epoch": 2.348235294117647,
+      "grad_norm": 0.13922256231307983,
+      "learning_rate": 4.2811501597444096e-05,
+      "loss": 0.5404,
+      "step": 251
+    },
+    {
+      "epoch": 2.357647058823529,
+      "grad_norm": 0.13278770446777344,
+      "learning_rate": 4.217252396166134e-05,
+      "loss": 0.5131,
+      "step": 252
+    },
+    {
+      "epoch": 2.367058823529412,
+      "grad_norm": 0.1421966701745987,
+      "learning_rate": 4.15335463258786e-05,
+      "loss": 0.5803,
+      "step": 253
+    },
+    {
+      "epoch": 2.376470588235294,
+      "grad_norm": 0.13415196537971497,
+      "learning_rate": 4.089456869009585e-05,
+      "loss": 0.5689,
+      "step": 254
+    },
+    {
+      "epoch": 2.3858823529411763,
+      "grad_norm": 0.13316908478736877,
+      "learning_rate": 4.0255591054313104e-05,
+      "loss": 0.5741,
+      "step": 255
+    },
+    {
+      "epoch": 2.395294117647059,
+      "grad_norm": 0.13901880383491516,
+      "learning_rate": 3.9616613418530355e-05,
+      "loss": 0.5365,
+      "step": 256
+    },
+    {
+      "epoch": 2.4047058823529412,
+      "grad_norm": 0.13510987162590027,
+      "learning_rate": 3.8977635782747605e-05,
+      "loss": 0.5511,
+      "step": 257
+    },
+    {
+      "epoch": 2.4141176470588235,
+      "grad_norm": 0.140598326921463,
+      "learning_rate": 3.8338658146964856e-05,
+      "loss": 0.5566,
+      "step": 258
+    },
+    {
+      "epoch": 2.4235294117647057,
+      "grad_norm": 0.13736282289028168,
+      "learning_rate": 3.769968051118211e-05,
+      "loss": 0.576,
+      "step": 259
+    },
+    {
+      "epoch": 2.4329411764705884,
+      "grad_norm": 0.13893315196037292,
+      "learning_rate": 3.7060702875399364e-05,
+      "loss": 0.5839,
+      "step": 260
+    },
+    {
+      "epoch": 2.4423529411764706,
+      "grad_norm": 0.13888853788375854,
+      "learning_rate": 3.6421725239616614e-05,
+      "loss": 0.5634,
+      "step": 261
+    },
+    {
+      "epoch": 2.451764705882353,
+      "grad_norm": 0.1429632157087326,
+      "learning_rate": 3.5782747603833865e-05,
+      "loss": 0.5627,
+      "step": 262
+    },
+    {
+      "epoch": 2.461176470588235,
+      "grad_norm": 0.13526563346385956,
+      "learning_rate": 3.514376996805112e-05,
+      "loss": 0.5538,
+      "step": 263
+    },
+    {
+      "epoch": 2.4705882352941178,
+      "grad_norm": 0.1388704627752304,
+      "learning_rate": 3.450479233226837e-05,
+      "loss": 0.5671,
+      "step": 264
+    },
+    {
+      "epoch": 2.48,
+      "grad_norm": 0.13550125062465668,
+      "learning_rate": 3.386581469648562e-05,
+      "loss": 0.5431,
+      "step": 265
+    },
+    {
+      "epoch": 2.489411764705882,
+      "grad_norm": 0.13878309726715088,
+      "learning_rate": 3.322683706070287e-05,
+      "loss": 0.5771,
+      "step": 266
+    },
+    {
+      "epoch": 2.498823529411765,
+      "grad_norm": 0.15442192554473877,
+      "learning_rate": 3.258785942492013e-05,
+      "loss": 0.5689,
+      "step": 267
+    },
+    {
+      "epoch": 2.508235294117647,
+      "grad_norm": 0.13992522656917572,
+      "learning_rate": 3.194888178913738e-05,
+      "loss": 0.5514,
+      "step": 268
+    },
+    {
+      "epoch": 2.5176470588235293,
+      "grad_norm": 0.13724872469902039,
+      "learning_rate": 3.130990415335464e-05,
+      "loss": 0.5537,
+      "step": 269
+    },
+    {
+      "epoch": 2.527058823529412,
+      "grad_norm": 0.13630978763103485,
+      "learning_rate": 3.067092651757188e-05,
+      "loss": 0.5616,
+      "step": 270
+    },
+    {
+      "epoch": 2.5364705882352943,
+      "grad_norm": 0.13853025436401367,
+      "learning_rate": 3.003194888178914e-05,
+      "loss": 0.5656,
+      "step": 271
+    },
+    {
+      "epoch": 2.5458823529411765,
+      "grad_norm": 0.1420363038778305,
+      "learning_rate": 2.939297124600639e-05,
+      "loss": 0.5826,
+      "step": 272
+    },
+    {
+      "epoch": 2.5552941176470587,
+      "grad_norm": 0.1408669352531433,
+      "learning_rate": 2.8753993610223644e-05,
+      "loss": 0.5583,
+      "step": 273
+    },
+    {
+      "epoch": 2.564705882352941,
+      "grad_norm": 0.1526196449995041,
+      "learning_rate": 2.8115015974440894e-05,
+      "loss": 0.5917,
+      "step": 274
+    },
+    {
+      "epoch": 2.5741176470588236,
+      "grad_norm": 0.1629597693681717,
+      "learning_rate": 2.747603833865815e-05,
+      "loss": 0.5624,
+      "step": 275
+    },
+    {
+      "epoch": 2.583529411764706,
+      "grad_norm": 0.14529789984226227,
+      "learning_rate": 2.68370607028754e-05,
+      "loss": 0.5328,
+      "step": 276
+    },
+    {
+      "epoch": 2.592941176470588,
+      "grad_norm": 0.1526152491569519,
+      "learning_rate": 2.6198083067092656e-05,
+      "loss": 0.588,
+      "step": 277
+    },
+    {
+      "epoch": 2.6023529411764708,
+      "grad_norm": 0.14647038280963898,
+      "learning_rate": 2.5559105431309903e-05,
+      "loss": 0.5696,
+      "step": 278
+    },
+    {
+      "epoch": 2.611764705882353,
+      "grad_norm": 0.13880112767219543,
+      "learning_rate": 2.4920127795527157e-05,
+      "loss": 0.5448,
+      "step": 279
+    },
+    {
+      "epoch": 2.621176470588235,
+      "grad_norm": 0.1403883844614029,
+      "learning_rate": 2.428115015974441e-05,
+      "loss": 0.5485,
+      "step": 280
+    },
+    {
+      "epoch": 2.630588235294118,
+      "grad_norm": 0.1447409838438034,
+      "learning_rate": 2.364217252396166e-05,
+      "loss": 0.5519,
+      "step": 281
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.14351636171340942,
+      "learning_rate": 2.3003194888178915e-05,
+      "loss": 0.5546,
+      "step": 282
+    },
+    {
+      "epoch": 2.6494117647058824,
+      "grad_norm": 0.13909754157066345,
+      "learning_rate": 2.2364217252396165e-05,
+      "loss": 0.5576,
+      "step": 283
+    },
+    {
+      "epoch": 2.6588235294117646,
+      "grad_norm": 0.14639312028884888,
+      "learning_rate": 2.172523961661342e-05,
+      "loss": 0.5786,
+      "step": 284
+    },
+    {
+      "epoch": 2.668235294117647,
+      "grad_norm": 0.1540895700454712,
+      "learning_rate": 2.108626198083067e-05,
+      "loss": 0.6019,
+      "step": 285
+    },
+    {
+      "epoch": 2.6776470588235295,
+      "grad_norm": 0.1561094969511032,
+      "learning_rate": 2.0447284345047924e-05,
+      "loss": 0.6138,
+      "step": 286
+    },
+    {
+      "epoch": 2.6870588235294117,
+      "grad_norm": 0.14837074279785156,
+      "learning_rate": 1.9808306709265177e-05,
+      "loss": 0.5633,
+      "step": 287
+    },
+    {
+      "epoch": 2.696470588235294,
+      "grad_norm": 0.1421661227941513,
+      "learning_rate": 1.9169329073482428e-05,
+      "loss": 0.5308,
+      "step": 288
+    },
+    {
+      "epoch": 2.696470588235294,
+      "eval_loss": 0.6246063709259033,
+      "eval_runtime": 5.4441,
+      "eval_samples_per_second": 8.082,
+      "eval_steps_per_second": 1.102,
+      "step": 288
+    },
+    {
+      "epoch": 2.7058823529411766,
+      "grad_norm": 0.14756028354167938,
+      "learning_rate": 1.8530351437699682e-05,
+      "loss": 0.5692,
+      "step": 289
+    },
+    {
+      "epoch": 2.715294117647059,
+      "grad_norm": 0.1475173979997635,
+      "learning_rate": 1.7891373801916932e-05,
+      "loss": 0.5383,
+      "step": 290
+    },
+    {
+      "epoch": 2.724705882352941,
+      "grad_norm": 0.14893798530101776,
+      "learning_rate": 1.7252396166134186e-05,
+      "loss": 0.5603,
+      "step": 291
+    },
+    {
+      "epoch": 2.7341176470588238,
+      "grad_norm": 0.14635878801345825,
+      "learning_rate": 1.6613418530351437e-05,
+      "loss": 0.573,
+      "step": 292
+    },
+    {
+      "epoch": 2.743529411764706,
+      "grad_norm": 0.14744271337985992,
+      "learning_rate": 1.597444089456869e-05,
+      "loss": 0.5646,
+      "step": 293
+    },
+    {
+      "epoch": 2.7529411764705882,
+      "grad_norm": 0.14066603779792786,
+      "learning_rate": 1.533546325878594e-05,
+      "loss": 0.5335,
+      "step": 294
+    },
+    {
+      "epoch": 2.7623529411764705,
+      "grad_norm": 0.14420145750045776,
+      "learning_rate": 1.4696485623003195e-05,
+      "loss": 0.5885,
+      "step": 295
+    },
+    {
+      "epoch": 2.7717647058823527,
+      "grad_norm": 0.1417757123708725,
+      "learning_rate": 1.4057507987220447e-05,
+      "loss": 0.525,
+      "step": 296
+    },
+    {
+      "epoch": 2.7811764705882354,
+      "grad_norm": 0.13913412392139435,
+      "learning_rate": 1.34185303514377e-05,
+      "loss": 0.5355,
+      "step": 297
+    },
+    {
+      "epoch": 2.7905882352941176,
+      "grad_norm": 0.14291714131832123,
+      "learning_rate": 1.2779552715654951e-05,
+      "loss": 0.5419,
+      "step": 298
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.15101145207881927,
+      "learning_rate": 1.2140575079872205e-05,
+      "loss": 0.5945,
+      "step": 299
+    },
+    {
+      "epoch": 2.8094117647058825,
+      "grad_norm": 0.15145310759544373,
+      "learning_rate": 1.1501597444089457e-05,
+      "loss": 0.5801,
+      "step": 300
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 318,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.1701101665255424e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5e1fbde269e87149778e7fdffe2535c52443b523b88667e085fbff81ba781c2
+size 5560

checkpoint-318/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/phi-4-unsloth-bnb-4bit
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-318/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/phi-4-unsloth-bnb-4bit",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "q_proj",
+    "up_proj",
+    "v_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-318/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c68c2b0294b6f6bd5cd30944deeb817e09035f75830bf5b4265a85e72d8db17
+size 262219392

checkpoint-318/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3edd3cbd4316858aa21ef52359d2c1a77c297cace97c4650362d7de41632562
+size 133778516

checkpoint-318/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd55c3b412ed63dda63d944e5148e12c72c1382a50017bf4d626548cc6ff19a6
+size 14244

checkpoint-318/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b8f975c49e3b488d0c5341bf1e121574fa4e39f977bbcdcaa0b828bb7cb4d44
+size 1064

checkpoint-318/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|dummy_87|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "�",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-318/tokenizer_config.json

@@ -0,0 +1,791 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "5809": {
+      "content": "�",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100256": {
+      "content": "<|dummy_0|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100261": {
+      "content": "<|dummy_1|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100262": {
+      "content": "<|dummy_2|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100263": {
+      "content": "<|dummy_3|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100266": {
+      "content": "<|im_sep|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100267": {
+      "content": "<|dummy_4|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100268": {
+      "content": "<|dummy_5|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100269": {
+      "content": "<|dummy_6|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100270": {
+      "content": "<|dummy_7|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100271": {
+      "content": "<|dummy_8|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100272": {
+      "content": "<|dummy_9|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100273": {
+      "content": "<|dummy_10|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100274": {
+      "content": "<|dummy_11|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100275": {
+      "content": "<|dummy_12|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100277": {
+      "content": "<|dummy_13|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100278": {
+      "content": "<|dummy_14|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100279": {
+      "content": "<|dummy_15|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100280": {
+      "content": "<|dummy_16|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100281": {
+      "content": "<|dummy_17|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100282": {
+      "content": "<|dummy_18|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100283": {
+      "content": "<|dummy_19|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100284": {
+      "content": "<|dummy_20|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100285": {
+      "content": "<|dummy_21|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100286": {
+      "content": "<|dummy_22|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100287": {
+      "content": "<|dummy_23|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100288": {
+      "content": "<|dummy_24|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100289": {
+      "content": "<|dummy_25|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100290": {
+      "content": "<|dummy_26|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100291": {
+      "content": "<|dummy_27|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100292": {
+      "content": "<|dummy_28|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100293": {
+      "content": "<|dummy_29|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100294": {
+      "content": "<|dummy_30|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100295": {
+      "content": "<|dummy_31|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100296": {
+      "content": "<|dummy_32|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100297": {
+      "content": "<|dummy_33|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100298": {
+      "content": "<|dummy_34|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100299": {
+      "content": "<|dummy_35|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100300": {
+      "content": "<|dummy_36|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100301": {
+      "content": "<|dummy_37|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100302": {
+      "content": "<|dummy_38|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100303": {
+      "content": "<|dummy_39|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100304": {
+      "content": "<|dummy_40|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100305": {
+      "content": "<|dummy_41|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100306": {
+      "content": "<|dummy_42|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100307": {
+      "content": "<|dummy_43|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100308": {
+      "content": "<|dummy_44|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100309": {
+      "content": "<|dummy_45|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100310": {
+      "content": "<|dummy_46|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100311": {
+      "content": "<|dummy_47|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100312": {
+      "content": "<|dummy_48|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100313": {
+      "content": "<|dummy_49|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100314": {
+      "content": "<|dummy_50|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100315": {
+      "content": "<|dummy_51|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100316": {
+      "content": "<|dummy_52|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100317": {
+      "content": "<|dummy_53|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100318": {
+      "content": "<|dummy_54|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100319": {
+      "content": "<|dummy_55|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100320": {
+      "content": "<|dummy_56|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100321": {
+      "content": "<|dummy_57|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100322": {
+      "content": "<|dummy_58|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100323": {
+      "content": "<|dummy_59|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100324": {
+      "content": "<|dummy_60|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100325": {
+      "content": "<|dummy_61|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100326": {
+      "content": "<|dummy_62|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100327": {
+      "content": "<|dummy_63|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100328": {
+      "content": "<|dummy_64|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100329": {
+      "content": "<|dummy_65|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100330": {
+      "content": "<|dummy_66|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100331": {
+      "content": "<|dummy_67|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100332": {
+      "content": "<|dummy_68|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100333": {
+      "content": "<|dummy_69|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100334": {
+      "content": "<|dummy_70|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100335": {
+      "content": "<|dummy_71|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100336": {
+      "content": "<|dummy_72|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100337": {
+      "content": "<|dummy_73|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100338": {
+      "content": "<|dummy_74|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100339": {
+      "content": "<|dummy_75|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100340": {
+      "content": "<|dummy_76|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100341": {
+      "content": "<|dummy_77|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100342": {
+      "content": "<|dummy_78|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100343": {
+      "content": "<|dummy_79|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100344": {
+      "content": "<|dummy_80|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100345": {
+      "content": "<|dummy_81|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100346": {
+      "content": "<|dummy_82|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100347": {
+      "content": "<|dummy_83|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100348": {
+      "content": "<|dummy_84|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100349": {
+      "content": "<|dummy_85|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100350": {
+      "content": "<|dummy_86|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100351": {
+      "content": "<|dummy_87|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "\n{% for message in messages %}\n{% if (message['role'] == 'system') %}\n{{'<|im_start|>system<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'user') %}\n{{'<|im_start|>user<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'assistant') %}\n{{'<|im_start|>assistant<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% endif %}\n{% endfor %}\n{% if add_generation_prompt %}\n{{ '<|im_start|>assistant<|im_sep|>' }}\n{% endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": {},
+  "model_max_length": 16384,
+  "pad_token": "<|dummy_87|>",
+  "padding_side": "right",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "�"
+}

checkpoint-318/trainer_state.json

@@ -0,0 +1,2283 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.978823529411765,
+  "eval_steps": 96,
+  "global_step": 318,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009411764705882352,
+      "grad_norm": 0.8642500638961792,
+      "learning_rate": 4e-05,
+      "loss": 0.9841,
+      "step": 1
+    },
+    {
+      "epoch": 0.018823529411764704,
+      "grad_norm": 0.8003044724464417,
+      "learning_rate": 8e-05,
+      "loss": 0.9727,
+      "step": 2
+    },
+    {
+      "epoch": 0.02823529411764706,
+      "grad_norm": 0.7982213497161865,
+      "learning_rate": 0.00012,
+      "loss": 1.0076,
+      "step": 3
+    },
+    {
+      "epoch": 0.03764705882352941,
+      "grad_norm": 0.4778502285480499,
+      "learning_rate": 0.00016,
+      "loss": 0.9179,
+      "step": 4
+    },
+    {
+      "epoch": 0.047058823529411764,
+      "grad_norm": 0.1798139065504074,
+      "learning_rate": 0.0002,
+      "loss": 0.8544,
+      "step": 5
+    },
+    {
+      "epoch": 0.05647058823529412,
+      "grad_norm": 0.10934074968099594,
+      "learning_rate": 0.00019936102236421725,
+      "loss": 0.8259,
+      "step": 6
+    },
+    {
+      "epoch": 0.06588235294117648,
+      "grad_norm": 0.451732873916626,
+      "learning_rate": 0.00019872204472843452,
+      "loss": 0.8176,
+      "step": 7
+    },
+    {
+      "epoch": 0.07529411764705882,
+      "grad_norm": 0.39763253927230835,
+      "learning_rate": 0.00019808306709265177,
+      "loss": 0.8091,
+      "step": 8
+    },
+    {
+      "epoch": 0.08470588235294117,
+      "grad_norm": 0.08421512693166733,
+      "learning_rate": 0.000197444089456869,
+      "loss": 0.8066,
+      "step": 9
+    },
+    {
+      "epoch": 0.09411764705882353,
+      "grad_norm": 0.0752546563744545,
+      "learning_rate": 0.00019680511182108628,
+      "loss": 0.788,
+      "step": 10
+    },
+    {
+      "epoch": 0.10352941176470588,
+      "grad_norm": 0.07105962187051773,
+      "learning_rate": 0.00019616613418530353,
+      "loss": 0.8142,
+      "step": 11
+    },
+    {
+      "epoch": 0.11294117647058824,
+      "grad_norm": 0.07268067449331284,
+      "learning_rate": 0.0001955271565495208,
+      "loss": 0.7427,
+      "step": 12
+    },
+    {
+      "epoch": 0.1223529411764706,
+      "grad_norm": 0.071601502597332,
+      "learning_rate": 0.00019488817891373804,
+      "loss": 0.7673,
+      "step": 13
+    },
+    {
+      "epoch": 0.13176470588235295,
+      "grad_norm": 0.06190289184451103,
+      "learning_rate": 0.00019424920127795528,
+      "loss": 0.7897,
+      "step": 14
+    },
+    {
+      "epoch": 0.1411764705882353,
+      "grad_norm": 0.062015291303396225,
+      "learning_rate": 0.00019361022364217253,
+      "loss": 0.7552,
+      "step": 15
+    },
+    {
+      "epoch": 0.15058823529411763,
+      "grad_norm": 0.06343454122543335,
+      "learning_rate": 0.00019297124600638977,
+      "loss": 0.7983,
+      "step": 16
+    },
+    {
+      "epoch": 0.16,
+      "grad_norm": 0.0880446657538414,
+      "learning_rate": 0.00019233226837060702,
+      "loss": 0.7331,
+      "step": 17
+    },
+    {
+      "epoch": 0.16941176470588235,
+      "grad_norm": 0.06399823725223541,
+      "learning_rate": 0.00019169329073482429,
+      "loss": 0.7017,
+      "step": 18
+    },
+    {
+      "epoch": 0.17882352941176471,
+      "grad_norm": 0.06614473462104797,
+      "learning_rate": 0.00019105431309904153,
+      "loss": 0.7346,
+      "step": 19
+    },
+    {
+      "epoch": 0.18823529411764706,
+      "grad_norm": 0.05928972363471985,
+      "learning_rate": 0.0001904153354632588,
+      "loss": 0.7088,
+      "step": 20
+    },
+    {
+      "epoch": 0.1976470588235294,
+      "grad_norm": 0.05864707753062248,
+      "learning_rate": 0.00018977635782747604,
+      "loss": 0.7017,
+      "step": 21
+    },
+    {
+      "epoch": 0.20705882352941177,
+      "grad_norm": 0.0725838840007782,
+      "learning_rate": 0.0001891373801916933,
+      "loss": 0.7255,
+      "step": 22
+    },
+    {
+      "epoch": 0.2164705882352941,
+      "grad_norm": 0.07014375925064087,
+      "learning_rate": 0.00018849840255591056,
+      "loss": 0.6699,
+      "step": 23
+    },
+    {
+      "epoch": 0.22588235294117648,
+      "grad_norm": 0.08222078531980515,
+      "learning_rate": 0.0001878594249201278,
+      "loss": 0.6802,
+      "step": 24
+    },
+    {
+      "epoch": 0.23529411764705882,
+      "grad_norm": 0.07020223140716553,
+      "learning_rate": 0.00018722044728434505,
+      "loss": 0.7013,
+      "step": 25
+    },
+    {
+      "epoch": 0.2447058823529412,
+      "grad_norm": 0.05753432214260101,
+      "learning_rate": 0.00018658146964856232,
+      "loss": 0.6547,
+      "step": 26
+    },
+    {
+      "epoch": 0.2541176470588235,
+      "grad_norm": 0.06647774577140808,
+      "learning_rate": 0.00018594249201277956,
+      "loss": 0.7132,
+      "step": 27
+    },
+    {
+      "epoch": 0.2635294117647059,
+      "grad_norm": 0.061453238129615784,
+      "learning_rate": 0.00018530351437699683,
+      "loss": 0.6891,
+      "step": 28
+    },
+    {
+      "epoch": 0.27294117647058824,
+      "grad_norm": 0.055535465478897095,
+      "learning_rate": 0.00018466453674121408,
+      "loss": 0.6542,
+      "step": 29
+    },
+    {
+      "epoch": 0.2823529411764706,
+      "grad_norm": 0.05627722293138504,
+      "learning_rate": 0.00018402555910543132,
+      "loss": 0.677,
+      "step": 30
+    },
+    {
+      "epoch": 0.2917647058823529,
+      "grad_norm": 0.05428534746170044,
+      "learning_rate": 0.0001833865814696486,
+      "loss": 0.6592,
+      "step": 31
+    },
+    {
+      "epoch": 0.30117647058823527,
+      "grad_norm": 0.06012285500764847,
+      "learning_rate": 0.00018274760383386583,
+      "loss": 0.6662,
+      "step": 32
+    },
+    {
+      "epoch": 0.31058823529411766,
+      "grad_norm": 0.08650115132331848,
+      "learning_rate": 0.00018210862619808308,
+      "loss": 0.6546,
+      "step": 33
+    },
+    {
+      "epoch": 0.32,
+      "grad_norm": 0.058147966861724854,
+      "learning_rate": 0.00018146964856230032,
+      "loss": 0.6505,
+      "step": 34
+    },
+    {
+      "epoch": 0.32941176470588235,
+      "grad_norm": 0.054181504994630814,
+      "learning_rate": 0.00018083067092651756,
+      "loss": 0.6428,
+      "step": 35
+    },
+    {
+      "epoch": 0.3388235294117647,
+      "grad_norm": 0.05629313737154007,
+      "learning_rate": 0.00018019169329073484,
+      "loss": 0.6916,
+      "step": 36
+    },
+    {
+      "epoch": 0.34823529411764703,
+      "grad_norm": 0.05400744453072548,
+      "learning_rate": 0.00017955271565495208,
+      "loss": 0.6646,
+      "step": 37
+    },
+    {
+      "epoch": 0.35764705882352943,
+      "grad_norm": 0.053580883890390396,
+      "learning_rate": 0.00017891373801916932,
+      "loss": 0.6297,
+      "step": 38
+    },
+    {
+      "epoch": 0.36705882352941177,
+      "grad_norm": 0.05560213327407837,
+      "learning_rate": 0.0001782747603833866,
+      "loss": 0.6284,
+      "step": 39
+    },
+    {
+      "epoch": 0.3764705882352941,
+      "grad_norm": 0.05281270295381546,
+      "learning_rate": 0.00017763578274760384,
+      "loss": 0.6392,
+      "step": 40
+    },
+    {
+      "epoch": 0.38588235294117645,
+      "grad_norm": 0.06260088086128235,
+      "learning_rate": 0.00017699680511182108,
+      "loss": 0.7056,
+      "step": 41
+    },
+    {
+      "epoch": 0.3952941176470588,
+      "grad_norm": 0.058404579758644104,
+      "learning_rate": 0.00017635782747603835,
+      "loss": 0.6138,
+      "step": 42
+    },
+    {
+      "epoch": 0.4047058823529412,
+      "grad_norm": 0.050437066704034805,
+      "learning_rate": 0.0001757188498402556,
+      "loss": 0.6569,
+      "step": 43
+    },
+    {
+      "epoch": 0.41411764705882353,
+      "grad_norm": 0.05336596816778183,
+      "learning_rate": 0.00017507987220447287,
+      "loss": 0.612,
+      "step": 44
+    },
+    {
+      "epoch": 0.4235294117647059,
+      "grad_norm": 0.05253816023468971,
+      "learning_rate": 0.0001744408945686901,
+      "loss": 0.6195,
+      "step": 45
+    },
+    {
+      "epoch": 0.4329411764705882,
+      "grad_norm": 0.05323391407728195,
+      "learning_rate": 0.00017380191693290735,
+      "loss": 0.624,
+      "step": 46
+    },
+    {
+      "epoch": 0.4423529411764706,
+      "grad_norm": 0.05454220622777939,
+      "learning_rate": 0.00017316293929712462,
+      "loss": 0.6654,
+      "step": 47
+    },
+    {
+      "epoch": 0.45176470588235296,
+      "grad_norm": 0.05840761214494705,
+      "learning_rate": 0.00017252396166134187,
+      "loss": 0.6791,
+      "step": 48
+    },
+    {
+      "epoch": 0.4611764705882353,
+      "grad_norm": 0.05933906137943268,
+      "learning_rate": 0.0001718849840255591,
+      "loss": 0.6501,
+      "step": 49
+    },
+    {
+      "epoch": 0.47058823529411764,
+      "grad_norm": 0.06150069832801819,
+      "learning_rate": 0.00017124600638977638,
+      "loss": 0.6332,
+      "step": 50
+    },
+    {
+      "epoch": 0.48,
+      "grad_norm": 0.054139360785484314,
+      "learning_rate": 0.00017060702875399363,
+      "loss": 0.6268,
+      "step": 51
+    },
+    {
+      "epoch": 0.4894117647058824,
+      "grad_norm": 0.05578823760151863,
+      "learning_rate": 0.00016996805111821087,
+      "loss": 0.6259,
+      "step": 52
+    },
+    {
+      "epoch": 0.4988235294117647,
+      "grad_norm": 0.06352369487285614,
+      "learning_rate": 0.00016932907348242811,
+      "loss": 0.6271,
+      "step": 53
+    },
+    {
+      "epoch": 0.508235294117647,
+      "grad_norm": 0.06086834892630577,
+      "learning_rate": 0.00016869009584664536,
+      "loss": 0.631,
+      "step": 54
+    },
+    {
+      "epoch": 0.5176470588235295,
+      "grad_norm": 0.05695689842104912,
+      "learning_rate": 0.00016805111821086263,
+      "loss": 0.64,
+      "step": 55
+    },
+    {
+      "epoch": 0.5270588235294118,
+      "grad_norm": 0.056874532252550125,
+      "learning_rate": 0.00016741214057507987,
+      "loss": 0.6296,
+      "step": 56
+    },
+    {
+      "epoch": 0.5364705882352941,
+      "grad_norm": 0.061433881521224976,
+      "learning_rate": 0.00016677316293929712,
+      "loss": 0.683,
+      "step": 57
+    },
+    {
+      "epoch": 0.5458823529411765,
+      "grad_norm": 0.06481802463531494,
+      "learning_rate": 0.0001661341853035144,
+      "loss": 0.599,
+      "step": 58
+    },
+    {
+      "epoch": 0.5552941176470588,
+      "grad_norm": 0.06558655947446823,
+      "learning_rate": 0.00016549520766773163,
+      "loss": 0.6362,
+      "step": 59
+    },
+    {
+      "epoch": 0.5647058823529412,
+      "grad_norm": 0.057915814220905304,
+      "learning_rate": 0.0001648562300319489,
+      "loss": 0.6396,
+      "step": 60
+    },
+    {
+      "epoch": 0.5741176470588235,
+      "grad_norm": 0.06019767373800278,
+      "learning_rate": 0.00016421725239616614,
+      "loss": 0.6444,
+      "step": 61
+    },
+    {
+      "epoch": 0.5835294117647059,
+      "grad_norm": 0.06425336748361588,
+      "learning_rate": 0.0001635782747603834,
+      "loss": 0.6175,
+      "step": 62
+    },
+    {
+      "epoch": 0.5929411764705882,
+      "grad_norm": 0.062431298196315765,
+      "learning_rate": 0.00016293929712460066,
+      "loss": 0.6458,
+      "step": 63
+    },
+    {
+      "epoch": 0.6023529411764705,
+      "grad_norm": 0.057641636580228806,
+      "learning_rate": 0.0001623003194888179,
+      "loss": 0.6262,
+      "step": 64
+    },
+    {
+      "epoch": 0.611764705882353,
+      "grad_norm": 0.05894685909152031,
+      "learning_rate": 0.00016166134185303515,
+      "loss": 0.6036,
+      "step": 65
+    },
+    {
+      "epoch": 0.6211764705882353,
+      "grad_norm": 0.06207429617643356,
+      "learning_rate": 0.00016102236421725242,
+      "loss": 0.6367,
+      "step": 66
+    },
+    {
+      "epoch": 0.6305882352941177,
+      "grad_norm": 0.06500203162431717,
+      "learning_rate": 0.00016038338658146966,
+      "loss": 0.6443,
+      "step": 67
+    },
+    {
+      "epoch": 0.64,
+      "grad_norm": 0.06465457379817963,
+      "learning_rate": 0.0001597444089456869,
+      "loss": 0.6105,
+      "step": 68
+    },
+    {
+      "epoch": 0.6494117647058824,
+      "grad_norm": 0.060090478509664536,
+      "learning_rate": 0.00015910543130990418,
+      "loss": 0.6447,
+      "step": 69
+    },
+    {
+      "epoch": 0.6588235294117647,
+      "grad_norm": 0.06724690645933151,
+      "learning_rate": 0.00015846645367412142,
+      "loss": 0.6062,
+      "step": 70
+    },
+    {
+      "epoch": 0.668235294117647,
+      "grad_norm": 0.06291332095861435,
+      "learning_rate": 0.00015782747603833866,
+      "loss": 0.6005,
+      "step": 71
+    },
+    {
+      "epoch": 0.6776470588235294,
+      "grad_norm": 0.06592954695224762,
+      "learning_rate": 0.0001571884984025559,
+      "loss": 0.6036,
+      "step": 72
+    },
+    {
+      "epoch": 0.6870588235294117,
+      "grad_norm": 0.06050164997577667,
+      "learning_rate": 0.00015654952076677315,
+      "loss": 0.5993,
+      "step": 73
+    },
+    {
+      "epoch": 0.6964705882352941,
+      "grad_norm": 0.061909306794404984,
+      "learning_rate": 0.00015591054313099042,
+      "loss": 0.6159,
+      "step": 74
+    },
+    {
+      "epoch": 0.7058823529411765,
+      "grad_norm": 0.06927211582660675,
+      "learning_rate": 0.00015527156549520767,
+      "loss": 0.6554,
+      "step": 75
+    },
+    {
+      "epoch": 0.7152941176470589,
+      "grad_norm": 0.06098336726427078,
+      "learning_rate": 0.00015463258785942494,
+      "loss": 0.5998,
+      "step": 76
+    },
+    {
+      "epoch": 0.7247058823529412,
+      "grad_norm": 0.06612212210893631,
+      "learning_rate": 0.00015399361022364218,
+      "loss": 0.6057,
+      "step": 77
+    },
+    {
+      "epoch": 0.7341176470588235,
+      "grad_norm": 0.06455227732658386,
+      "learning_rate": 0.00015335463258785942,
+      "loss": 0.64,
+      "step": 78
+    },
+    {
+      "epoch": 0.7435294117647059,
+      "grad_norm": 0.40494394302368164,
+      "learning_rate": 0.0001527156549520767,
+      "loss": 0.6397,
+      "step": 79
+    },
+    {
+      "epoch": 0.7529411764705882,
+      "grad_norm": 0.05910542979836464,
+      "learning_rate": 0.00015207667731629394,
+      "loss": 0.6087,
+      "step": 80
+    },
+    {
+      "epoch": 0.7623529411764706,
+      "grad_norm": 0.06125890091061592,
+      "learning_rate": 0.00015143769968051118,
+      "loss": 0.6133,
+      "step": 81
+    },
+    {
+      "epoch": 0.7717647058823529,
+      "grad_norm": 0.0716337338089943,
+      "learning_rate": 0.00015079872204472845,
+      "loss": 0.6136,
+      "step": 82
+    },
+    {
+      "epoch": 0.7811764705882352,
+      "grad_norm": 0.07003322243690491,
+      "learning_rate": 0.0001501597444089457,
+      "loss": 0.6236,
+      "step": 83
+    },
+    {
+      "epoch": 0.7905882352941176,
+      "grad_norm": 0.06324715912342072,
+      "learning_rate": 0.00014952076677316297,
+      "loss": 0.6282,
+      "step": 84
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.07135389745235443,
+      "learning_rate": 0.0001488817891373802,
+      "loss": 0.6443,
+      "step": 85
+    },
+    {
+      "epoch": 0.8094117647058824,
+      "grad_norm": 0.06127123162150383,
+      "learning_rate": 0.00014824281150159745,
+      "loss": 0.6396,
+      "step": 86
+    },
+    {
+      "epoch": 0.8188235294117647,
+      "grad_norm": 0.06172896921634674,
+      "learning_rate": 0.0001476038338658147,
+      "loss": 0.6377,
+      "step": 87
+    },
+    {
+      "epoch": 0.8282352941176471,
+      "grad_norm": 0.3543515205383301,
+      "learning_rate": 0.00014696485623003194,
+      "loss": 0.6454,
+      "step": 88
+    },
+    {
+      "epoch": 0.8376470588235294,
+      "grad_norm": 0.06527980417013168,
+      "learning_rate": 0.0001463258785942492,
+      "loss": 0.615,
+      "step": 89
+    },
+    {
+      "epoch": 0.8470588235294118,
+      "grad_norm": 0.06327975541353226,
+      "learning_rate": 0.00014568690095846646,
+      "loss": 0.6447,
+      "step": 90
+    },
+    {
+      "epoch": 0.8564705882352941,
+      "grad_norm": 0.06739926338195801,
+      "learning_rate": 0.0001450479233226837,
+      "loss": 0.6087,
+      "step": 91
+    },
+    {
+      "epoch": 0.8658823529411764,
+      "grad_norm": 0.06617800146341324,
+      "learning_rate": 0.00014440894568690097,
+      "loss": 0.6259,
+      "step": 92
+    },
+    {
+      "epoch": 0.8752941176470588,
+      "grad_norm": 0.06242835149168968,
+      "learning_rate": 0.00014376996805111821,
+      "loss": 0.6049,
+      "step": 93
+    },
+    {
+      "epoch": 0.8847058823529412,
+      "grad_norm": 0.06359254568815231,
+      "learning_rate": 0.00014313099041533546,
+      "loss": 0.6274,
+      "step": 94
+    },
+    {
+      "epoch": 0.8941176470588236,
+      "grad_norm": 0.06559597700834274,
+      "learning_rate": 0.00014249201277955273,
+      "loss": 0.5935,
+      "step": 95
+    },
+    {
+      "epoch": 0.9035294117647059,
+      "grad_norm": 0.07493000477552414,
+      "learning_rate": 0.00014185303514376997,
+      "loss": 0.5881,
+      "step": 96
+    },
+    {
+      "epoch": 0.9035294117647059,
+      "eval_loss": 0.6278424859046936,
+      "eval_runtime": 5.469,
+      "eval_samples_per_second": 8.045,
+      "eval_steps_per_second": 1.097,
+      "step": 96
+    },
+    {
+      "epoch": 0.9129411764705883,
+      "grad_norm": 0.06714215129613876,
+      "learning_rate": 0.00014121405750798722,
+      "loss": 0.6438,
+      "step": 97
+    },
+    {
+      "epoch": 0.9223529411764706,
+      "grad_norm": 0.06775335222482681,
+      "learning_rate": 0.0001405750798722045,
+      "loss": 0.6256,
+      "step": 98
+    },
+    {
+      "epoch": 0.9317647058823529,
+      "grad_norm": 0.07045792043209076,
+      "learning_rate": 0.00013993610223642173,
+      "loss": 0.6712,
+      "step": 99
+    },
+    {
+      "epoch": 0.9411764705882353,
+      "grad_norm": 0.06958217918872833,
+      "learning_rate": 0.000139297124600639,
+      "loss": 0.6278,
+      "step": 100
+    },
+    {
+      "epoch": 0.9505882352941176,
+      "grad_norm": 0.06508302688598633,
+      "learning_rate": 0.00013865814696485625,
+      "loss": 0.6415,
+      "step": 101
+    },
+    {
+      "epoch": 0.96,
+      "grad_norm": 0.06876156479120255,
+      "learning_rate": 0.0001380191693290735,
+      "loss": 0.6211,
+      "step": 102
+    },
+    {
+      "epoch": 0.9694117647058823,
+      "grad_norm": 0.07631471753120422,
+      "learning_rate": 0.00013738019169329076,
+      "loss": 0.6276,
+      "step": 103
+    },
+    {
+      "epoch": 0.9788235294117648,
+      "grad_norm": 0.07122895866632462,
+      "learning_rate": 0.000136741214057508,
+      "loss": 0.5997,
+      "step": 104
+    },
+    {
+      "epoch": 0.9882352941176471,
+      "grad_norm": 0.06862416863441467,
+      "learning_rate": 0.00013610223642172525,
+      "loss": 0.6563,
+      "step": 105
+    },
+    {
+      "epoch": 0.9976470588235294,
+      "grad_norm": 0.06701315939426422,
+      "learning_rate": 0.0001354632587859425,
+      "loss": 0.61,
+      "step": 106
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.14547984302043915,
+      "learning_rate": 0.00013482428115015973,
+      "loss": 0.6382,
+      "step": 107
+    },
+    {
+      "epoch": 1.0094117647058825,
+      "grad_norm": 0.0676453560590744,
+      "learning_rate": 0.000134185303514377,
+      "loss": 0.5766,
+      "step": 108
+    },
+    {
+      "epoch": 1.0188235294117647,
+      "grad_norm": 0.06471403688192368,
+      "learning_rate": 0.00013354632587859425,
+      "loss": 0.5769,
+      "step": 109
+    },
+    {
+      "epoch": 1.0282352941176471,
+      "grad_norm": 0.07038411498069763,
+      "learning_rate": 0.0001329073482428115,
+      "loss": 0.5828,
+      "step": 110
+    },
+    {
+      "epoch": 1.0376470588235294,
+      "grad_norm": 0.0679129809141159,
+      "learning_rate": 0.00013226837060702876,
+      "loss": 0.5755,
+      "step": 111
+    },
+    {
+      "epoch": 1.0470588235294118,
+      "grad_norm": 0.07297012954950333,
+      "learning_rate": 0.000131629392971246,
+      "loss": 0.6114,
+      "step": 112
+    },
+    {
+      "epoch": 1.056470588235294,
+      "grad_norm": 0.07036986202001572,
+      "learning_rate": 0.00013099041533546325,
+      "loss": 0.6254,
+      "step": 113
+    },
+    {
+      "epoch": 1.0658823529411765,
+      "grad_norm": 0.08385265618562698,
+      "learning_rate": 0.00013035143769968052,
+      "loss": 0.6284,
+      "step": 114
+    },
+    {
+      "epoch": 1.0752941176470587,
+      "grad_norm": 0.08367498964071274,
+      "learning_rate": 0.00012971246006389777,
+      "loss": 0.6128,
+      "step": 115
+    },
+    {
+      "epoch": 1.0847058823529412,
+      "grad_norm": 0.082453154027462,
+      "learning_rate": 0.000129073482428115,
+      "loss": 0.6012,
+      "step": 116
+    },
+    {
+      "epoch": 1.0941176470588236,
+      "grad_norm": 0.0714535042643547,
+      "learning_rate": 0.00012843450479233228,
+      "loss": 0.599,
+      "step": 117
+    },
+    {
+      "epoch": 1.1035294117647059,
+      "grad_norm": 0.07391790300607681,
+      "learning_rate": 0.00012779552715654952,
+      "loss": 0.5897,
+      "step": 118
+    },
+    {
+      "epoch": 1.1129411764705883,
+      "grad_norm": 0.08383256196975708,
+      "learning_rate": 0.0001271565495207668,
+      "loss": 0.634,
+      "step": 119
+    },
+    {
+      "epoch": 1.1223529411764706,
+      "grad_norm": 0.08821647614240646,
+      "learning_rate": 0.00012651757188498404,
+      "loss": 0.5869,
+      "step": 120
+    },
+    {
+      "epoch": 1.131764705882353,
+      "grad_norm": 0.07556191831827164,
+      "learning_rate": 0.00012587859424920128,
+      "loss": 0.6275,
+      "step": 121
+    },
+    {
+      "epoch": 1.1411764705882352,
+      "grad_norm": 0.08794881403446198,
+      "learning_rate": 0.00012523961661341855,
+      "loss": 0.6115,
+      "step": 122
+    },
+    {
+      "epoch": 1.1505882352941177,
+      "grad_norm": 0.0785142183303833,
+      "learning_rate": 0.0001246006389776358,
+      "loss": 0.6156,
+      "step": 123
+    },
+    {
+      "epoch": 1.16,
+      "grad_norm": 0.07684168219566345,
+      "learning_rate": 0.00012396166134185304,
+      "loss": 0.6107,
+      "step": 124
+    },
+    {
+      "epoch": 1.1694117647058824,
+      "grad_norm": 0.07763410359621048,
+      "learning_rate": 0.00012332268370607028,
+      "loss": 0.5888,
+      "step": 125
+    },
+    {
+      "epoch": 1.1788235294117646,
+      "grad_norm": 0.09084579348564148,
+      "learning_rate": 0.00012268370607028753,
+      "loss": 0.5912,
+      "step": 126
+    },
+    {
+      "epoch": 1.188235294117647,
+      "grad_norm": 0.0838058665394783,
+      "learning_rate": 0.00012204472843450481,
+      "loss": 0.6152,
+      "step": 127
+    },
+    {
+      "epoch": 1.1976470588235295,
+      "grad_norm": 0.08908473700284958,
+      "learning_rate": 0.00012140575079872206,
+      "loss": 0.6049,
+      "step": 128
+    },
+    {
+      "epoch": 1.2070588235294117,
+      "grad_norm": 0.08239720016717911,
+      "learning_rate": 0.0001207667731629393,
+      "loss": 0.5852,
+      "step": 129
+    },
+    {
+      "epoch": 1.2164705882352942,
+      "grad_norm": 0.08228597790002823,
+      "learning_rate": 0.00012012779552715656,
+      "loss": 0.5642,
+      "step": 130
+    },
+    {
+      "epoch": 1.2258823529411764,
+      "grad_norm": 0.08196559548377991,
+      "learning_rate": 0.0001194888178913738,
+      "loss": 0.5971,
+      "step": 131
+    },
+    {
+      "epoch": 1.2352941176470589,
+      "grad_norm": 0.08850298076868057,
+      "learning_rate": 0.00011884984025559104,
+      "loss": 0.6149,
+      "step": 132
+    },
+    {
+      "epoch": 1.244705882352941,
+      "grad_norm": 0.08643686771392822,
+      "learning_rate": 0.00011821086261980832,
+      "loss": 0.6097,
+      "step": 133
+    },
+    {
+      "epoch": 1.2541176470588236,
+      "grad_norm": 0.08877662569284439,
+      "learning_rate": 0.00011757188498402556,
+      "loss": 0.6362,
+      "step": 134
+    },
+    {
+      "epoch": 1.263529411764706,
+      "grad_norm": 0.08452433347702026,
+      "learning_rate": 0.00011693290734824283,
+      "loss": 0.6331,
+      "step": 135
+    },
+    {
+      "epoch": 1.2729411764705882,
+      "grad_norm": 0.08155957609415054,
+      "learning_rate": 0.00011629392971246007,
+      "loss": 0.6214,
+      "step": 136
+    },
+    {
+      "epoch": 1.2823529411764705,
+      "grad_norm": 0.08229539543390274,
+      "learning_rate": 0.00011565495207667732,
+      "loss": 0.5981,
+      "step": 137
+    },
+    {
+      "epoch": 1.291764705882353,
+      "grad_norm": 0.09338800609111786,
+      "learning_rate": 0.00011501597444089457,
+      "loss": 0.6326,
+      "step": 138
+    },
+    {
+      "epoch": 1.3011764705882354,
+      "grad_norm": 0.0837632343173027,
+      "learning_rate": 0.00011437699680511182,
+      "loss": 0.5736,
+      "step": 139
+    },
+    {
+      "epoch": 1.3105882352941176,
+      "grad_norm": 0.08500789105892181,
+      "learning_rate": 0.00011373801916932908,
+      "loss": 0.6253,
+      "step": 140
+    },
+    {
+      "epoch": 1.32,
+      "grad_norm": 0.0843970999121666,
+      "learning_rate": 0.00011309904153354633,
+      "loss": 0.6053,
+      "step": 141
+    },
+    {
+      "epoch": 1.3294117647058823,
+      "grad_norm": 0.08848956227302551,
+      "learning_rate": 0.00011246006389776358,
+      "loss": 0.6184,
+      "step": 142
+    },
+    {
+      "epoch": 1.3388235294117647,
+      "grad_norm": 0.10718461126089096,
+      "learning_rate": 0.00011182108626198085,
+      "loss": 0.583,
+      "step": 143
+    },
+    {
+      "epoch": 1.348235294117647,
+      "grad_norm": 0.08756856620311737,
+      "learning_rate": 0.00011118210862619809,
+      "loss": 0.5861,
+      "step": 144
+    },
+    {
+      "epoch": 1.3576470588235294,
+      "grad_norm": 0.08492905646562576,
+      "learning_rate": 0.00011054313099041533,
+      "loss": 0.5707,
+      "step": 145
+    },
+    {
+      "epoch": 1.3670588235294119,
+      "grad_norm": 0.08758281171321869,
+      "learning_rate": 0.0001099041533546326,
+      "loss": 0.555,
+      "step": 146
+    },
+    {
+      "epoch": 1.3764705882352941,
+      "grad_norm": 0.09751519560813904,
+      "learning_rate": 0.00010926517571884985,
+      "loss": 0.5976,
+      "step": 147
+    },
+    {
+      "epoch": 1.3858823529411763,
+      "grad_norm": 0.09590818732976913,
+      "learning_rate": 0.00010862619808306709,
+      "loss": 0.6015,
+      "step": 148
+    },
+    {
+      "epoch": 1.3952941176470588,
+      "grad_norm": 0.08812163025140762,
+      "learning_rate": 0.00010798722044728435,
+      "loss": 0.5905,
+      "step": 149
+    },
+    {
+      "epoch": 1.4047058823529412,
+      "grad_norm": 0.09162674099206924,
+      "learning_rate": 0.0001073482428115016,
+      "loss": 0.6063,
+      "step": 150
+    },
+    {
+      "epoch": 1.4141176470588235,
+      "grad_norm": 0.08727669715881348,
+      "learning_rate": 0.00010670926517571886,
+      "loss": 0.6181,
+      "step": 151
+    },
+    {
+      "epoch": 1.423529411764706,
+      "grad_norm": 0.0928676649928093,
+      "learning_rate": 0.00010607028753993611,
+      "loss": 0.6188,
+      "step": 152
+    },
+    {
+      "epoch": 1.4329411764705882,
+      "grad_norm": 0.09681576490402222,
+      "learning_rate": 0.00010543130990415335,
+      "loss": 0.5818,
+      "step": 153
+    },
+    {
+      "epoch": 1.4423529411764706,
+      "grad_norm": 0.08795556426048279,
+      "learning_rate": 0.00010479233226837062,
+      "loss": 0.6134,
+      "step": 154
+    },
+    {
+      "epoch": 1.4517647058823528,
+      "grad_norm": 0.0861482247710228,
+      "learning_rate": 0.00010415335463258787,
+      "loss": 0.6082,
+      "step": 155
+    },
+    {
+      "epoch": 1.4611764705882353,
+      "grad_norm": 0.09977812319993973,
+      "learning_rate": 0.00010351437699680511,
+      "loss": 0.5921,
+      "step": 156
+    },
+    {
+      "epoch": 1.4705882352941178,
+      "grad_norm": 0.10190846771001816,
+      "learning_rate": 0.00010287539936102237,
+      "loss": 0.6271,
+      "step": 157
+    },
+    {
+      "epoch": 1.48,
+      "grad_norm": 0.09232486039400101,
+      "learning_rate": 0.00010223642172523961,
+      "loss": 0.6067,
+      "step": 158
+    },
+    {
+      "epoch": 1.4894117647058824,
+      "grad_norm": 0.10211990028619766,
+      "learning_rate": 0.00010159744408945688,
+      "loss": 0.6238,
+      "step": 159
+    },
+    {
+      "epoch": 1.4988235294117647,
+      "grad_norm": 0.08955651521682739,
+      "learning_rate": 0.00010095846645367413,
+      "loss": 0.6002,
+      "step": 160
+    },
+    {
+      "epoch": 1.5082352941176471,
+      "grad_norm": 0.10033038258552551,
+      "learning_rate": 0.00010031948881789137,
+      "loss": 0.6129,
+      "step": 161
+    },
+    {
+      "epoch": 1.5176470588235293,
+      "grad_norm": 0.09701602160930634,
+      "learning_rate": 9.968051118210863e-05,
+      "loss": 0.5867,
+      "step": 162
+    },
+    {
+      "epoch": 1.5270588235294118,
+      "grad_norm": 0.0969882607460022,
+      "learning_rate": 9.904153354632588e-05,
+      "loss": 0.599,
+      "step": 163
+    },
+    {
+      "epoch": 1.5364705882352943,
+      "grad_norm": 0.09163233637809753,
+      "learning_rate": 9.840255591054314e-05,
+      "loss": 0.6078,
+      "step": 164
+    },
+    {
+      "epoch": 1.5458823529411765,
+      "grad_norm": 0.0925818681716919,
+      "learning_rate": 9.77635782747604e-05,
+      "loss": 0.6231,
+      "step": 165
+    },
+    {
+      "epoch": 1.5552941176470587,
+      "grad_norm": 0.09490446001291275,
+      "learning_rate": 9.712460063897764e-05,
+      "loss": 0.6099,
+      "step": 166
+    },
+    {
+      "epoch": 1.5647058823529412,
+      "grad_norm": 0.10370604693889618,
+      "learning_rate": 9.648562300319489e-05,
+      "loss": 0.6133,
+      "step": 167
+    },
+    {
+      "epoch": 1.5741176470588236,
+      "grad_norm": 0.10043718665838242,
+      "learning_rate": 9.584664536741214e-05,
+      "loss": 0.6002,
+      "step": 168
+    },
+    {
+      "epoch": 1.5835294117647059,
+      "grad_norm": 0.10102451592683792,
+      "learning_rate": 9.52076677316294e-05,
+      "loss": 0.6326,
+      "step": 169
+    },
+    {
+      "epoch": 1.592941176470588,
+      "grad_norm": 0.09170352667570114,
+      "learning_rate": 9.456869009584664e-05,
+      "loss": 0.5725,
+      "step": 170
+    },
+    {
+      "epoch": 1.6023529411764705,
+      "grad_norm": 0.09696916490793228,
+      "learning_rate": 9.39297124600639e-05,
+      "loss": 0.6162,
+      "step": 171
+    },
+    {
+      "epoch": 1.611764705882353,
+      "grad_norm": 0.1032591313123703,
+      "learning_rate": 9.329073482428116e-05,
+      "loss": 0.6022,
+      "step": 172
+    },
+    {
+      "epoch": 1.6211764705882352,
+      "grad_norm": 0.09963531792163849,
+      "learning_rate": 9.265175718849842e-05,
+      "loss": 0.5918,
+      "step": 173
+    },
+    {
+      "epoch": 1.6305882352941177,
+      "grad_norm": 0.10566572844982147,
+      "learning_rate": 9.201277955271566e-05,
+      "loss": 0.6248,
+      "step": 174
+    },
+    {
+      "epoch": 1.6400000000000001,
+      "grad_norm": 0.09654838591814041,
+      "learning_rate": 9.137380191693292e-05,
+      "loss": 0.5747,
+      "step": 175
+    },
+    {
+      "epoch": 1.6494117647058824,
+      "grad_norm": 0.09412531554698944,
+      "learning_rate": 9.073482428115016e-05,
+      "loss": 0.611,
+      "step": 176
+    },
+    {
+      "epoch": 1.6588235294117646,
+      "grad_norm": 0.10707147419452667,
+      "learning_rate": 9.009584664536742e-05,
+      "loss": 0.6371,
+      "step": 177
+    },
+    {
+      "epoch": 1.668235294117647,
+      "grad_norm": 0.0976637527346611,
+      "learning_rate": 8.945686900958466e-05,
+      "loss": 0.636,
+      "step": 178
+    },
+    {
+      "epoch": 1.6776470588235295,
+      "grad_norm": 0.09769190102815628,
+      "learning_rate": 8.881789137380192e-05,
+      "loss": 0.6379,
+      "step": 179
+    },
+    {
+      "epoch": 1.6870588235294117,
+      "grad_norm": 0.1029646173119545,
+      "learning_rate": 8.817891373801918e-05,
+      "loss": 0.6533,
+      "step": 180
+    },
+    {
+      "epoch": 1.696470588235294,
+      "grad_norm": 0.10120602697134018,
+      "learning_rate": 8.753993610223643e-05,
+      "loss": 0.6044,
+      "step": 181
+    },
+    {
+      "epoch": 1.7058823529411766,
+      "grad_norm": 0.0992206409573555,
+      "learning_rate": 8.690095846645368e-05,
+      "loss": 0.6072,
+      "step": 182
+    },
+    {
+      "epoch": 1.7152941176470589,
+      "grad_norm": 0.10124579071998596,
+      "learning_rate": 8.626198083067093e-05,
+      "loss": 0.5923,
+      "step": 183
+    },
+    {
+      "epoch": 1.724705882352941,
+      "grad_norm": 0.10673081874847412,
+      "learning_rate": 8.562300319488819e-05,
+      "loss": 0.5964,
+      "step": 184
+    },
+    {
+      "epoch": 1.7341176470588235,
+      "grad_norm": 0.10745410621166229,
+      "learning_rate": 8.498402555910544e-05,
+      "loss": 0.6052,
+      "step": 185
+    },
+    {
+      "epoch": 1.743529411764706,
+      "grad_norm": 0.10404065251350403,
+      "learning_rate": 8.434504792332268e-05,
+      "loss": 0.5975,
+      "step": 186
+    },
+    {
+      "epoch": 1.7529411764705882,
+      "grad_norm": 0.09927040338516235,
+      "learning_rate": 8.370607028753994e-05,
+      "loss": 0.5901,
+      "step": 187
+    },
+    {
+      "epoch": 1.7623529411764705,
+      "grad_norm": 0.10322125256061554,
+      "learning_rate": 8.30670926517572e-05,
+      "loss": 0.6154,
+      "step": 188
+    },
+    {
+      "epoch": 1.771764705882353,
+      "grad_norm": 0.10899791866540909,
+      "learning_rate": 8.242811501597445e-05,
+      "loss": 0.6255,
+      "step": 189
+    },
+    {
+      "epoch": 1.7811764705882354,
+      "grad_norm": 0.1041494831442833,
+      "learning_rate": 8.17891373801917e-05,
+      "loss": 0.5925,
+      "step": 190
+    },
+    {
+      "epoch": 1.7905882352941176,
+      "grad_norm": 0.09239751845598221,
+      "learning_rate": 8.115015974440895e-05,
+      "loss": 0.5713,
+      "step": 191
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.09495066851377487,
+      "learning_rate": 8.051118210862621e-05,
+      "loss": 0.5896,
+      "step": 192
+    },
+    {
+      "epoch": 1.8,
+      "eval_loss": 0.6196094155311584,
+      "eval_runtime": 5.4299,
+      "eval_samples_per_second": 8.103,
+      "eval_steps_per_second": 1.105,
+      "step": 192
+    },
+    {
+      "epoch": 1.8094117647058825,
+      "grad_norm": 0.09602215141057968,
+      "learning_rate": 7.987220447284345e-05,
+      "loss": 0.5979,
+      "step": 193
+    },
+    {
+      "epoch": 1.8188235294117647,
+      "grad_norm": 0.10422977060079575,
+      "learning_rate": 7.923322683706071e-05,
+      "loss": 0.5704,
+      "step": 194
+    },
+    {
+      "epoch": 1.828235294117647,
+      "grad_norm": 0.09678755700588226,
+      "learning_rate": 7.859424920127795e-05,
+      "loss": 0.5774,
+      "step": 195
+    },
+    {
+      "epoch": 1.8376470588235294,
+      "grad_norm": 0.09845128655433655,
+      "learning_rate": 7.795527156549521e-05,
+      "loss": 0.5816,
+      "step": 196
+    },
+    {
+      "epoch": 1.8470588235294119,
+      "grad_norm": 0.1065673753619194,
+      "learning_rate": 7.731629392971247e-05,
+      "loss": 0.6135,
+      "step": 197
+    },
+    {
+      "epoch": 1.856470588235294,
+      "grad_norm": 0.10570935904979706,
+      "learning_rate": 7.667731629392971e-05,
+      "loss": 0.5908,
+      "step": 198
+    },
+    {
+      "epoch": 1.8658823529411763,
+      "grad_norm": 0.09649720042943954,
+      "learning_rate": 7.603833865814697e-05,
+      "loss": 0.599,
+      "step": 199
+    },
+    {
+      "epoch": 1.8752941176470588,
+      "grad_norm": 0.10398146510124207,
+      "learning_rate": 7.539936102236423e-05,
+      "loss": 0.5853,
+      "step": 200
+    },
+    {
+      "epoch": 1.8847058823529412,
+      "grad_norm": 0.09633094817399979,
+      "learning_rate": 7.476038338658148e-05,
+      "loss": 0.5927,
+      "step": 201
+    },
+    {
+      "epoch": 1.8941176470588235,
+      "grad_norm": 0.0993979275226593,
+      "learning_rate": 7.412140575079873e-05,
+      "loss": 0.5928,
+      "step": 202
+    },
+    {
+      "epoch": 1.903529411764706,
+      "grad_norm": 0.10687059164047241,
+      "learning_rate": 7.348242811501597e-05,
+      "loss": 0.5882,
+      "step": 203
+    },
+    {
+      "epoch": 1.9129411764705884,
+      "grad_norm": 0.10128702223300934,
+      "learning_rate": 7.284345047923323e-05,
+      "loss": 0.6009,
+      "step": 204
+    },
+    {
+      "epoch": 1.9223529411764706,
+      "grad_norm": 0.10716967284679413,
+      "learning_rate": 7.220447284345049e-05,
+      "loss": 0.6198,
+      "step": 205
+    },
+    {
+      "epoch": 1.9317647058823528,
+      "grad_norm": 0.10603494942188263,
+      "learning_rate": 7.156549520766773e-05,
+      "loss": 0.6222,
+      "step": 206
+    },
+    {
+      "epoch": 1.9411764705882353,
+      "grad_norm": 0.10125169903039932,
+      "learning_rate": 7.092651757188499e-05,
+      "loss": 0.5938,
+      "step": 207
+    },
+    {
+      "epoch": 1.9505882352941177,
+      "grad_norm": 0.10542251914739609,
+      "learning_rate": 7.028753993610224e-05,
+      "loss": 0.6341,
+      "step": 208
+    },
+    {
+      "epoch": 1.96,
+      "grad_norm": 0.09659598022699356,
+      "learning_rate": 6.96485623003195e-05,
+      "loss": 0.59,
+      "step": 209
+    },
+    {
+      "epoch": 1.9694117647058822,
+      "grad_norm": 0.1122719794511795,
+      "learning_rate": 6.900958466453674e-05,
+      "loss": 0.6109,
+      "step": 210
+    },
+    {
+      "epoch": 1.9788235294117649,
+      "grad_norm": 0.11106933653354645,
+      "learning_rate": 6.8370607028754e-05,
+      "loss": 0.58,
+      "step": 211
+    },
+    {
+      "epoch": 1.988235294117647,
+      "grad_norm": 0.09962272644042969,
+      "learning_rate": 6.773162939297125e-05,
+      "loss": 0.6105,
+      "step": 212
+    },
+    {
+      "epoch": 1.9976470588235293,
+      "grad_norm": 0.09547895193099976,
+      "learning_rate": 6.70926517571885e-05,
+      "loss": 0.5709,
+      "step": 213
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.21181434392929077,
+      "learning_rate": 6.645367412140575e-05,
+      "loss": 0.6164,
+      "step": 214
+    },
+    {
+      "epoch": 2.0094117647058822,
+      "grad_norm": 0.11414559185504913,
+      "learning_rate": 6.5814696485623e-05,
+      "loss": 0.5642,
+      "step": 215
+    },
+    {
+      "epoch": 2.018823529411765,
+      "grad_norm": 0.11991499364376068,
+      "learning_rate": 6.517571884984026e-05,
+      "loss": 0.5741,
+      "step": 216
+    },
+    {
+      "epoch": 2.028235294117647,
+      "grad_norm": 0.10295461863279343,
+      "learning_rate": 6.45367412140575e-05,
+      "loss": 0.5604,
+      "step": 217
+    },
+    {
+      "epoch": 2.0376470588235294,
+      "grad_norm": 0.10355547070503235,
+      "learning_rate": 6.389776357827476e-05,
+      "loss": 0.5884,
+      "step": 218
+    },
+    {
+      "epoch": 2.0470588235294116,
+      "grad_norm": 0.12203363329172134,
+      "learning_rate": 6.325878594249202e-05,
+      "loss": 0.5903,
+      "step": 219
+    },
+    {
+      "epoch": 2.0564705882352943,
+      "grad_norm": 0.12483850866556168,
+      "learning_rate": 6.261980830670928e-05,
+      "loss": 0.5666,
+      "step": 220
+    },
+    {
+      "epoch": 2.0658823529411765,
+      "grad_norm": 0.12819188833236694,
+      "learning_rate": 6.198083067092652e-05,
+      "loss": 0.5493,
+      "step": 221
+    },
+    {
+      "epoch": 2.0752941176470587,
+      "grad_norm": 0.11613185703754425,
+      "learning_rate": 6.134185303514376e-05,
+      "loss": 0.6121,
+      "step": 222
+    },
+    {
+      "epoch": 2.084705882352941,
+      "grad_norm": 0.11060836911201477,
+      "learning_rate": 6.070287539936103e-05,
+      "loss": 0.5748,
+      "step": 223
+    },
+    {
+      "epoch": 2.0941176470588236,
+      "grad_norm": 0.13958008587360382,
+      "learning_rate": 6.006389776357828e-05,
+      "loss": 0.62,
+      "step": 224
+    },
+    {
+      "epoch": 2.103529411764706,
+      "grad_norm": 0.14533892273902893,
+      "learning_rate": 5.942492012779552e-05,
+      "loss": 0.5897,
+      "step": 225
+    },
+    {
+      "epoch": 2.112941176470588,
+      "grad_norm": 0.12220677733421326,
+      "learning_rate": 5.878594249201278e-05,
+      "loss": 0.5508,
+      "step": 226
+    },
+    {
+      "epoch": 2.1223529411764708,
+      "grad_norm": 0.11975925415754318,
+      "learning_rate": 5.814696485623004e-05,
+      "loss": 0.5483,
+      "step": 227
+    },
+    {
+      "epoch": 2.131764705882353,
+      "grad_norm": 0.12308664619922638,
+      "learning_rate": 5.750798722044729e-05,
+      "loss": 0.5921,
+      "step": 228
+    },
+    {
+      "epoch": 2.1411764705882352,
+      "grad_norm": 0.12570548057556152,
+      "learning_rate": 5.686900958466454e-05,
+      "loss": 0.6229,
+      "step": 229
+    },
+    {
+      "epoch": 2.1505882352941175,
+      "grad_norm": 0.12253190577030182,
+      "learning_rate": 5.623003194888179e-05,
+      "loss": 0.5758,
+      "step": 230
+    },
+    {
+      "epoch": 2.16,
+      "grad_norm": 0.1249895840883255,
+      "learning_rate": 5.5591054313099045e-05,
+      "loss": 0.5633,
+      "step": 231
+    },
+    {
+      "epoch": 2.1694117647058824,
+      "grad_norm": 0.1355949193239212,
+      "learning_rate": 5.49520766773163e-05,
+      "loss": 0.5486,
+      "step": 232
+    },
+    {
+      "epoch": 2.1788235294117646,
+      "grad_norm": 0.12255489826202393,
+      "learning_rate": 5.4313099041533546e-05,
+      "loss": 0.5906,
+      "step": 233
+    },
+    {
+      "epoch": 2.1882352941176473,
+      "grad_norm": 0.12850622832775116,
+      "learning_rate": 5.36741214057508e-05,
+      "loss": 0.5738,
+      "step": 234
+    },
+    {
+      "epoch": 2.1976470588235295,
+      "grad_norm": 0.18490703403949738,
+      "learning_rate": 5.3035143769968054e-05,
+      "loss": 0.5737,
+      "step": 235
+    },
+    {
+      "epoch": 2.2070588235294117,
+      "grad_norm": 0.135318785905838,
+      "learning_rate": 5.239616613418531e-05,
+      "loss": 0.5963,
+      "step": 236
+    },
+    {
+      "epoch": 2.216470588235294,
+      "grad_norm": 0.12884393334388733,
+      "learning_rate": 5.1757188498402555e-05,
+      "loss": 0.5399,
+      "step": 237
+    },
+    {
+      "epoch": 2.2258823529411766,
+      "grad_norm": 0.13065044581890106,
+      "learning_rate": 5.1118210862619806e-05,
+      "loss": 0.5611,
+      "step": 238
+    },
+    {
+      "epoch": 2.235294117647059,
+      "grad_norm": 0.12230806797742844,
+      "learning_rate": 5.047923322683706e-05,
+      "loss": 0.5446,
+      "step": 239
+    },
+    {
+      "epoch": 2.244705882352941,
+      "grad_norm": 0.13359951972961426,
+      "learning_rate": 4.984025559105431e-05,
+      "loss": 0.5652,
+      "step": 240
+    },
+    {
+      "epoch": 2.2541176470588233,
+      "grad_norm": 0.1409720480442047,
+      "learning_rate": 4.920127795527157e-05,
+      "loss": 0.5437,
+      "step": 241
+    },
+    {
+      "epoch": 2.263529411764706,
+      "grad_norm": 0.1343528777360916,
+      "learning_rate": 4.856230031948882e-05,
+      "loss": 0.5811,
+      "step": 242
+    },
+    {
+      "epoch": 2.2729411764705882,
+      "grad_norm": 0.1315775066614151,
+      "learning_rate": 4.792332268370607e-05,
+      "loss": 0.5718,
+      "step": 243
+    },
+    {
+      "epoch": 2.2823529411764705,
+      "grad_norm": 0.1300155520439148,
+      "learning_rate": 4.728434504792332e-05,
+      "loss": 0.5629,
+      "step": 244
+    },
+    {
+      "epoch": 2.291764705882353,
+      "grad_norm": 0.1364685744047165,
+      "learning_rate": 4.664536741214058e-05,
+      "loss": 0.5641,
+      "step": 245
+    },
+    {
+      "epoch": 2.3011764705882354,
+      "grad_norm": 0.13575628399848938,
+      "learning_rate": 4.600638977635783e-05,
+      "loss": 0.5861,
+      "step": 246
+    },
+    {
+      "epoch": 2.3105882352941176,
+      "grad_norm": 0.13364288210868835,
+      "learning_rate": 4.536741214057508e-05,
+      "loss": 0.5959,
+      "step": 247
+    },
+    {
+      "epoch": 2.32,
+      "grad_norm": 0.129843071103096,
+      "learning_rate": 4.472843450479233e-05,
+      "loss": 0.5631,
+      "step": 248
+    },
+    {
+      "epoch": 2.3294117647058825,
+      "grad_norm": 0.13373443484306335,
+      "learning_rate": 4.408945686900959e-05,
+      "loss": 0.557,
+      "step": 249
+    },
+    {
+      "epoch": 2.3388235294117647,
+      "grad_norm": 0.14091914892196655,
+      "learning_rate": 4.345047923322684e-05,
+      "loss": 0.577,
+      "step": 250
+    },
+    {
+      "epoch": 2.348235294117647,
+      "grad_norm": 0.13922256231307983,
+      "learning_rate": 4.2811501597444096e-05,
+      "loss": 0.5404,
+      "step": 251
+    },
+    {
+      "epoch": 2.357647058823529,
+      "grad_norm": 0.13278770446777344,
+      "learning_rate": 4.217252396166134e-05,
+      "loss": 0.5131,
+      "step": 252
+    },
+    {
+      "epoch": 2.367058823529412,
+      "grad_norm": 0.1421966701745987,
+      "learning_rate": 4.15335463258786e-05,
+      "loss": 0.5803,
+      "step": 253
+    },
+    {
+      "epoch": 2.376470588235294,
+      "grad_norm": 0.13415196537971497,
+      "learning_rate": 4.089456869009585e-05,
+      "loss": 0.5689,
+      "step": 254
+    },
+    {
+      "epoch": 2.3858823529411763,
+      "grad_norm": 0.13316908478736877,
+      "learning_rate": 4.0255591054313104e-05,
+      "loss": 0.5741,
+      "step": 255
+    },
+    {
+      "epoch": 2.395294117647059,
+      "grad_norm": 0.13901880383491516,
+      "learning_rate": 3.9616613418530355e-05,
+      "loss": 0.5365,
+      "step": 256
+    },
+    {
+      "epoch": 2.4047058823529412,
+      "grad_norm": 0.13510987162590027,
+      "learning_rate": 3.8977635782747605e-05,
+      "loss": 0.5511,
+      "step": 257
+    },
+    {
+      "epoch": 2.4141176470588235,
+      "grad_norm": 0.140598326921463,
+      "learning_rate": 3.8338658146964856e-05,
+      "loss": 0.5566,
+      "step": 258
+    },
+    {
+      "epoch": 2.4235294117647057,
+      "grad_norm": 0.13736282289028168,
+      "learning_rate": 3.769968051118211e-05,
+      "loss": 0.576,
+      "step": 259
+    },
+    {
+      "epoch": 2.4329411764705884,
+      "grad_norm": 0.13893315196037292,
+      "learning_rate": 3.7060702875399364e-05,
+      "loss": 0.5839,
+      "step": 260
+    },
+    {
+      "epoch": 2.4423529411764706,
+      "grad_norm": 0.13888853788375854,
+      "learning_rate": 3.6421725239616614e-05,
+      "loss": 0.5634,
+      "step": 261
+    },
+    {
+      "epoch": 2.451764705882353,
+      "grad_norm": 0.1429632157087326,
+      "learning_rate": 3.5782747603833865e-05,
+      "loss": 0.5627,
+      "step": 262
+    },
+    {
+      "epoch": 2.461176470588235,
+      "grad_norm": 0.13526563346385956,
+      "learning_rate": 3.514376996805112e-05,
+      "loss": 0.5538,
+      "step": 263
+    },
+    {
+      "epoch": 2.4705882352941178,
+      "grad_norm": 0.1388704627752304,
+      "learning_rate": 3.450479233226837e-05,
+      "loss": 0.5671,
+      "step": 264
+    },
+    {
+      "epoch": 2.48,
+      "grad_norm": 0.13550125062465668,
+      "learning_rate": 3.386581469648562e-05,
+      "loss": 0.5431,
+      "step": 265
+    },
+    {
+      "epoch": 2.489411764705882,
+      "grad_norm": 0.13878309726715088,
+      "learning_rate": 3.322683706070287e-05,
+      "loss": 0.5771,
+      "step": 266
+    },
+    {
+      "epoch": 2.498823529411765,
+      "grad_norm": 0.15442192554473877,
+      "learning_rate": 3.258785942492013e-05,
+      "loss": 0.5689,
+      "step": 267
+    },
+    {
+      "epoch": 2.508235294117647,
+      "grad_norm": 0.13992522656917572,
+      "learning_rate": 3.194888178913738e-05,
+      "loss": 0.5514,
+      "step": 268
+    },
+    {
+      "epoch": 2.5176470588235293,
+      "grad_norm": 0.13724872469902039,
+      "learning_rate": 3.130990415335464e-05,
+      "loss": 0.5537,
+      "step": 269
+    },
+    {
+      "epoch": 2.527058823529412,
+      "grad_norm": 0.13630978763103485,
+      "learning_rate": 3.067092651757188e-05,
+      "loss": 0.5616,
+      "step": 270
+    },
+    {
+      "epoch": 2.5364705882352943,
+      "grad_norm": 0.13853025436401367,
+      "learning_rate": 3.003194888178914e-05,
+      "loss": 0.5656,
+      "step": 271
+    },
+    {
+      "epoch": 2.5458823529411765,
+      "grad_norm": 0.1420363038778305,
+      "learning_rate": 2.939297124600639e-05,
+      "loss": 0.5826,
+      "step": 272
+    },
+    {
+      "epoch": 2.5552941176470587,
+      "grad_norm": 0.1408669352531433,
+      "learning_rate": 2.8753993610223644e-05,
+      "loss": 0.5583,
+      "step": 273
+    },
+    {
+      "epoch": 2.564705882352941,
+      "grad_norm": 0.1526196449995041,
+      "learning_rate": 2.8115015974440894e-05,
+      "loss": 0.5917,
+      "step": 274
+    },
+    {
+      "epoch": 2.5741176470588236,
+      "grad_norm": 0.1629597693681717,
+      "learning_rate": 2.747603833865815e-05,
+      "loss": 0.5624,
+      "step": 275
+    },
+    {
+      "epoch": 2.583529411764706,
+      "grad_norm": 0.14529789984226227,
+      "learning_rate": 2.68370607028754e-05,
+      "loss": 0.5328,
+      "step": 276
+    },
+    {
+      "epoch": 2.592941176470588,
+      "grad_norm": 0.1526152491569519,
+      "learning_rate": 2.6198083067092656e-05,
+      "loss": 0.588,
+      "step": 277
+    },
+    {
+      "epoch": 2.6023529411764708,
+      "grad_norm": 0.14647038280963898,
+      "learning_rate": 2.5559105431309903e-05,
+      "loss": 0.5696,
+      "step": 278
+    },
+    {
+      "epoch": 2.611764705882353,
+      "grad_norm": 0.13880112767219543,
+      "learning_rate": 2.4920127795527157e-05,
+      "loss": 0.5448,
+      "step": 279
+    },
+    {
+      "epoch": 2.621176470588235,
+      "grad_norm": 0.1403883844614029,
+      "learning_rate": 2.428115015974441e-05,
+      "loss": 0.5485,
+      "step": 280
+    },
+    {
+      "epoch": 2.630588235294118,
+      "grad_norm": 0.1447409838438034,
+      "learning_rate": 2.364217252396166e-05,
+      "loss": 0.5519,
+      "step": 281
+    },
+    {
+      "epoch": 2.64,
+      "grad_norm": 0.14351636171340942,
+      "learning_rate": 2.3003194888178915e-05,
+      "loss": 0.5546,
+      "step": 282
+    },
+    {
+      "epoch": 2.6494117647058824,
+      "grad_norm": 0.13909754157066345,
+      "learning_rate": 2.2364217252396165e-05,
+      "loss": 0.5576,
+      "step": 283
+    },
+    {
+      "epoch": 2.6588235294117646,
+      "grad_norm": 0.14639312028884888,
+      "learning_rate": 2.172523961661342e-05,
+      "loss": 0.5786,
+      "step": 284
+    },
+    {
+      "epoch": 2.668235294117647,
+      "grad_norm": 0.1540895700454712,
+      "learning_rate": 2.108626198083067e-05,
+      "loss": 0.6019,
+      "step": 285
+    },
+    {
+      "epoch": 2.6776470588235295,
+      "grad_norm": 0.1561094969511032,
+      "learning_rate": 2.0447284345047924e-05,
+      "loss": 0.6138,
+      "step": 286
+    },
+    {
+      "epoch": 2.6870588235294117,
+      "grad_norm": 0.14837074279785156,
+      "learning_rate": 1.9808306709265177e-05,
+      "loss": 0.5633,
+      "step": 287
+    },
+    {
+      "epoch": 2.696470588235294,
+      "grad_norm": 0.1421661227941513,
+      "learning_rate": 1.9169329073482428e-05,
+      "loss": 0.5308,
+      "step": 288
+    },
+    {
+      "epoch": 2.696470588235294,
+      "eval_loss": 0.6246063709259033,
+      "eval_runtime": 5.4441,
+      "eval_samples_per_second": 8.082,
+      "eval_steps_per_second": 1.102,
+      "step": 288
+    },
+    {
+      "epoch": 2.7058823529411766,
+      "grad_norm": 0.14756028354167938,
+      "learning_rate": 1.8530351437699682e-05,
+      "loss": 0.5692,
+      "step": 289
+    },
+    {
+      "epoch": 2.715294117647059,
+      "grad_norm": 0.1475173979997635,
+      "learning_rate": 1.7891373801916932e-05,
+      "loss": 0.5383,
+      "step": 290
+    },
+    {
+      "epoch": 2.724705882352941,
+      "grad_norm": 0.14893798530101776,
+      "learning_rate": 1.7252396166134186e-05,
+      "loss": 0.5603,
+      "step": 291
+    },
+    {
+      "epoch": 2.7341176470588238,
+      "grad_norm": 0.14635878801345825,
+      "learning_rate": 1.6613418530351437e-05,
+      "loss": 0.573,
+      "step": 292
+    },
+    {
+      "epoch": 2.743529411764706,
+      "grad_norm": 0.14744271337985992,
+      "learning_rate": 1.597444089456869e-05,
+      "loss": 0.5646,
+      "step": 293
+    },
+    {
+      "epoch": 2.7529411764705882,
+      "grad_norm": 0.14066603779792786,
+      "learning_rate": 1.533546325878594e-05,
+      "loss": 0.5335,
+      "step": 294
+    },
+    {
+      "epoch": 2.7623529411764705,
+      "grad_norm": 0.14420145750045776,
+      "learning_rate": 1.4696485623003195e-05,
+      "loss": 0.5885,
+      "step": 295
+    },
+    {
+      "epoch": 2.7717647058823527,
+      "grad_norm": 0.1417757123708725,
+      "learning_rate": 1.4057507987220447e-05,
+      "loss": 0.525,
+      "step": 296
+    },
+    {
+      "epoch": 2.7811764705882354,
+      "grad_norm": 0.13913412392139435,
+      "learning_rate": 1.34185303514377e-05,
+      "loss": 0.5355,
+      "step": 297
+    },
+    {
+      "epoch": 2.7905882352941176,
+      "grad_norm": 0.14291714131832123,
+      "learning_rate": 1.2779552715654951e-05,
+      "loss": 0.5419,
+      "step": 298
+    },
+    {
+      "epoch": 2.8,
+      "grad_norm": 0.15101145207881927,
+      "learning_rate": 1.2140575079872205e-05,
+      "loss": 0.5945,
+      "step": 299
+    },
+    {
+      "epoch": 2.8094117647058825,
+      "grad_norm": 0.15145310759544373,
+      "learning_rate": 1.1501597444089457e-05,
+      "loss": 0.5801,
+      "step": 300
+    },
+    {
+      "epoch": 2.8188235294117647,
+      "grad_norm": 0.14465861022472382,
+      "learning_rate": 1.086261980830671e-05,
+      "loss": 0.561,
+      "step": 301
+    },
+    {
+      "epoch": 2.828235294117647,
+      "grad_norm": 0.14341317117214203,
+      "learning_rate": 1.0223642172523962e-05,
+      "loss": 0.5524,
+      "step": 302
+    },
+    {
+      "epoch": 2.8376470588235296,
+      "grad_norm": 0.15162834525108337,
+      "learning_rate": 9.584664536741214e-06,
+      "loss": 0.568,
+      "step": 303
+    },
+    {
+      "epoch": 2.847058823529412,
+      "grad_norm": 0.14496251940727234,
+      "learning_rate": 8.945686900958466e-06,
+      "loss": 0.5336,
+      "step": 304
+    },
+    {
+      "epoch": 2.856470588235294,
+      "grad_norm": 0.14667992293834686,
+      "learning_rate": 8.306709265175718e-06,
+      "loss": 0.5613,
+      "step": 305
+    },
+    {
+      "epoch": 2.8658823529411763,
+      "grad_norm": 0.1467258483171463,
+      "learning_rate": 7.66773162939297e-06,
+      "loss": 0.5572,
+      "step": 306
+    },
+    {
+      "epoch": 2.8752941176470586,
+      "grad_norm": 0.1460108608007431,
+      "learning_rate": 7.0287539936102235e-06,
+      "loss": 0.5788,
+      "step": 307
+    },
+    {
+      "epoch": 2.8847058823529412,
+      "grad_norm": 0.14580175280570984,
+      "learning_rate": 6.389776357827476e-06,
+      "loss": 0.5501,
+      "step": 308
+    },
+    {
+      "epoch": 2.8941176470588235,
+      "grad_norm": 0.1488231122493744,
+      "learning_rate": 5.750798722044729e-06,
+      "loss": 0.5537,
+      "step": 309
+    },
+    {
+      "epoch": 2.9035294117647057,
+      "grad_norm": 0.14228259027004242,
+      "learning_rate": 5.111821086261981e-06,
+      "loss": 0.5383,
+      "step": 310
+    },
+    {
+      "epoch": 2.9129411764705884,
+      "grad_norm": 0.15516069531440735,
+      "learning_rate": 4.472843450479233e-06,
+      "loss": 0.5889,
+      "step": 311
+    },
+    {
+      "epoch": 2.9223529411764706,
+      "grad_norm": 0.14953361451625824,
+      "learning_rate": 3.833865814696485e-06,
+      "loss": 0.5448,
+      "step": 312
+    },
+    {
+      "epoch": 2.931764705882353,
+      "grad_norm": 0.15361154079437256,
+      "learning_rate": 3.194888178913738e-06,
+      "loss": 0.5687,
+      "step": 313
+    },
+    {
+      "epoch": 2.9411764705882355,
+      "grad_norm": 0.15242530405521393,
+      "learning_rate": 2.5559105431309904e-06,
+      "loss": 0.5819,
+      "step": 314
+    },
+    {
+      "epoch": 2.9505882352941177,
+      "grad_norm": 0.1486331820487976,
+      "learning_rate": 1.9169329073482426e-06,
+      "loss": 0.5652,
+      "step": 315
+    },
+    {
+      "epoch": 2.96,
+      "grad_norm": 0.15188926458358765,
+      "learning_rate": 1.2779552715654952e-06,
+      "loss": 0.5816,
+      "step": 316
+    },
+    {
+      "epoch": 2.969411764705882,
+      "grad_norm": 0.1547756791114807,
+      "learning_rate": 6.389776357827476e-07,
+      "loss": 0.5798,
+      "step": 317
+    },
+    {
+      "epoch": 2.978823529411765,
+      "grad_norm": 0.15288227796554565,
+      "learning_rate": 0.0,
+      "loss": 0.5555,
+      "step": 318
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 318,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 300,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.4215740961652736e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-318/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5e1fbde269e87149778e7fdffe2535c52443b523b88667e085fbff81ba781c2
+size 5560

config.json

@@ -0,0 +1,34 @@
+{
+  "_name_or_path": "unsloth/phi-4-unsloth-bnb-4bit",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 100257,
+  "eos_token_id": 100265,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 17920,
+  "max_position_embeddings": 16384,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 10,
+  "original_max_position_embeddings": 16384,
+  "pad_token_id": 100351,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 250000,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.47.1",
+  "unsloth_fixed": true,
+  "unsloth_version": "2025.1.5",
+  "use_cache": true,
+  "vocab_size": 100352
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 100257,
+  "eos_token_id": 100265,
+  "max_length": 16384,
+  "pad_token_id": 100351,
+  "transformers_version": "4.47.1"
+}

model-00001-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32150df732721a8c6a42b04de48693286c18b3b7ab1e0956f083bf44f678b247
+size 4933658528

model-00002-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:516049acdf42d3f4cb0d2dd3099aefa1900630153a0f58ffd978eb437231b6af
+size 4954693112

model-00003-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e375ee376f9537da92d8216f85a7a1692c1be75b74182199a44f270d7d291af
+size 4902243992

model-00004-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:116df38c08da5b4dcfa15c98d664181ba2f644c8bd319d84986305df333193df
+size 4954672440

model-00005-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8533a6e8aa2c51fffdd7af02599ed54090cc6354322b9e69eb4f1c8f25176e51
+size 4954672432

model-00006-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b46160645708d51e4e0926fd2e2883204db4e046bda78e9475df9624c98494b3
+size 4619116224

model.safetensors.index.json

@@ -0,0 +1,370 @@
+{
+  "metadata": {
+    "total_size": 29319014400
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00006-of-00006.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00006.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00004-of-00006.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00005-of-00006.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.36.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.37.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.38.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.input_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.mlp.down_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.mlp.gate_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.mlp.up_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.post_attention_layernorm.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.self_attn.k_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.self_attn.o_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.self_attn.q_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.39.self_attn.v_proj.weight": "model-00006-of-00006.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00006.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00006.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00006.safetensors",
+    "model.norm.weight": "model-00006-of-00006.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|dummy_87|>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": true,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "�",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,791 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "5809": {
+      "content": "�",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100256": {
+      "content": "<|dummy_0|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100261": {
+      "content": "<|dummy_1|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100262": {
+      "content": "<|dummy_2|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100263": {
+      "content": "<|dummy_3|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<|im_start|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<|im_end|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100266": {
+      "content": "<|im_sep|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100267": {
+      "content": "<|dummy_4|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100268": {
+      "content": "<|dummy_5|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100269": {
+      "content": "<|dummy_6|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100270": {
+      "content": "<|dummy_7|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100271": {
+      "content": "<|dummy_8|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100272": {
+      "content": "<|dummy_9|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100273": {
+      "content": "<|dummy_10|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100274": {
+      "content": "<|dummy_11|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100275": {
+      "content": "<|dummy_12|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100277": {
+      "content": "<|dummy_13|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100278": {
+      "content": "<|dummy_14|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100279": {
+      "content": "<|dummy_15|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100280": {
+      "content": "<|dummy_16|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100281": {
+      "content": "<|dummy_17|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100282": {
+      "content": "<|dummy_18|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100283": {
+      "content": "<|dummy_19|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100284": {
+      "content": "<|dummy_20|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100285": {
+      "content": "<|dummy_21|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100286": {
+      "content": "<|dummy_22|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100287": {
+      "content": "<|dummy_23|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100288": {
+      "content": "<|dummy_24|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100289": {
+      "content": "<|dummy_25|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100290": {
+      "content": "<|dummy_26|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100291": {
+      "content": "<|dummy_27|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100292": {
+      "content": "<|dummy_28|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100293": {
+      "content": "<|dummy_29|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100294": {
+      "content": "<|dummy_30|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100295": {
+      "content": "<|dummy_31|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100296": {
+      "content": "<|dummy_32|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100297": {
+      "content": "<|dummy_33|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100298": {
+      "content": "<|dummy_34|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100299": {
+      "content": "<|dummy_35|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100300": {
+      "content": "<|dummy_36|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100301": {
+      "content": "<|dummy_37|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100302": {
+      "content": "<|dummy_38|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100303": {
+      "content": "<|dummy_39|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100304": {
+      "content": "<|dummy_40|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100305": {
+      "content": "<|dummy_41|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100306": {
+      "content": "<|dummy_42|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100307": {
+      "content": "<|dummy_43|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100308": {
+      "content": "<|dummy_44|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100309": {
+      "content": "<|dummy_45|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100310": {
+      "content": "<|dummy_46|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100311": {
+      "content": "<|dummy_47|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100312": {
+      "content": "<|dummy_48|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100313": {
+      "content": "<|dummy_49|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100314": {
+      "content": "<|dummy_50|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100315": {
+      "content": "<|dummy_51|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100316": {
+      "content": "<|dummy_52|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100317": {
+      "content": "<|dummy_53|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100318": {
+      "content": "<|dummy_54|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100319": {
+      "content": "<|dummy_55|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100320": {
+      "content": "<|dummy_56|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100321": {
+      "content": "<|dummy_57|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100322": {
+      "content": "<|dummy_58|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100323": {
+      "content": "<|dummy_59|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100324": {
+      "content": "<|dummy_60|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100325": {
+      "content": "<|dummy_61|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100326": {
+      "content": "<|dummy_62|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100327": {
+      "content": "<|dummy_63|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100328": {
+      "content": "<|dummy_64|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100329": {
+      "content": "<|dummy_65|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100330": {
+      "content": "<|dummy_66|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100331": {
+      "content": "<|dummy_67|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100332": {
+      "content": "<|dummy_68|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100333": {
+      "content": "<|dummy_69|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100334": {
+      "content": "<|dummy_70|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100335": {
+      "content": "<|dummy_71|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100336": {
+      "content": "<|dummy_72|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100337": {
+      "content": "<|dummy_73|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100338": {
+      "content": "<|dummy_74|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100339": {
+      "content": "<|dummy_75|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100340": {
+      "content": "<|dummy_76|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100341": {
+      "content": "<|dummy_77|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100342": {
+      "content": "<|dummy_78|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100343": {
+      "content": "<|dummy_79|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100344": {
+      "content": "<|dummy_80|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100345": {
+      "content": "<|dummy_81|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100346": {
+      "content": "<|dummy_82|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100347": {
+      "content": "<|dummy_83|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100348": {
+      "content": "<|dummy_84|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100349": {
+      "content": "<|dummy_85|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100350": {
+      "content": "<|dummy_86|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    },
+    "100351": {
+      "content": "<|dummy_87|>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": true,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "\n{% for message in messages %}\n{% if (message['role'] == 'system') %}\n{{'<|im_start|>system<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'user') %}\n{{'<|im_start|>user<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% elif (message['role'] == 'assistant') %}\n{{'<|im_start|>assistant<|im_sep|>' + message['content'] + '<|im_end|>'}}\n{% endif %}\n{% endfor %}\n{% if add_generation_prompt %}\n{{ '<|im_start|>assistant<|im_sep|>' }}\n{% endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "extra_special_tokens": {},
+  "model_max_length": 16384,
+  "pad_token": "<|dummy_87|>",
+  "padding_side": "left",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "�"
+}