Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,208 @@
+---
+base_model: OpenOneRec/OneReason-0.8B-pretrain-competition
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/inspire/hdd/global_user/zhengkaipeng-240108120123/models/OneReason-0.8B-pretrain-competition
+- llama-factory
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

adapter_config.json

@@ -0,0 +1,48 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/inspire/hdd/global_user/zhengkaipeng-240108120123/models/OneReason-0.8B-pretrain-competition",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e42221f3258234af8f7ecf53a65a8a025b596c196ceacaff2a11069452f9b99
+size 40422168

chat_template.jinja

@@ -0,0 +1,89 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dac9cfd83495d7af6848eb82a981f7f4ad44d149d9e17c64ab1fcb7c82cd7c5d
+size 81070011

rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bff8a506b2a0d826393762df29a4938909ff99da7b3a34cf1f94528e75482bb
+size 15429

rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85463b3eb6d47f843366cf43f5a2fad84e06d85d498ace3a455e767649955b92
+size 15429

rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd871e116a632862902421fe5f77c988d54f59ee131bdae0543ca0754e35aefa
+size 15429

rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6afc5122bdb3fb6af4b4bdee1e66653694eb2115866dbafeffdda4a04748470b
+size 15429

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2cac2d929b1833a57939acddf74a5eee464fc119c5cf7fdc0649bbeb2a05e3df
+size 1465

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd4d15f596979aecbc11ba12668cb21c9fb8452ce1235cd9d7878cc950170421
+size 16016554

tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "is_local": true,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

trainer_state.json

@@ -0,0 +1,2876 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 2030,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0024630541871921183,
+      "grad_norm": 1.627393126487732,
+      "learning_rate": 9.836065573770491e-06,
+      "loss": 3.0176469802856447,
+      "step": 5
+    },
+    {
+      "epoch": 0.0049261083743842365,
+      "grad_norm": 0.8007542490959167,
+      "learning_rate": 2.2131147540983603e-05,
+      "loss": 2.913766860961914,
+      "step": 10
+    },
+    {
+      "epoch": 0.007389162561576354,
+      "grad_norm": 0.3084839880466461,
+      "learning_rate": 3.442622950819672e-05,
+      "loss": 2.6249759674072264,
+      "step": 15
+    },
+    {
+      "epoch": 0.009852216748768473,
+      "grad_norm": 0.3165278136730194,
+      "learning_rate": 4.672131147540983e-05,
+      "loss": 2.54384765625,
+      "step": 20
+    },
+    {
+      "epoch": 0.012315270935960592,
+      "grad_norm": 0.3195851743221283,
+      "learning_rate": 5.901639344262294e-05,
+      "loss": 2.4461576461791994,
+      "step": 25
+    },
+    {
+      "epoch": 0.014778325123152709,
+      "grad_norm": 0.20587186515331268,
+      "learning_rate": 7.131147540983606e-05,
+      "loss": 2.35736198425293,
+      "step": 30
+    },
+    {
+      "epoch": 0.017241379310344827,
+      "grad_norm": 0.23413971066474915,
+      "learning_rate": 8.360655737704916e-05,
+      "loss": 2.207070159912109,
+      "step": 35
+    },
+    {
+      "epoch": 0.019704433497536946,
+      "grad_norm": 0.21079401671886444,
+      "learning_rate": 9.59016393442623e-05,
+      "loss": 2.1630226135253907,
+      "step": 40
+    },
+    {
+      "epoch": 0.022167487684729065,
+      "grad_norm": 0.17173269391059875,
+      "learning_rate": 0.0001081967213114754,
+      "loss": 2.098428153991699,
+      "step": 45
+    },
+    {
+      "epoch": 0.024630541871921183,
+      "grad_norm": 0.16880469024181366,
+      "learning_rate": 0.00012049180327868852,
+      "loss": 2.018040084838867,
+      "step": 50
+    },
+    {
+      "epoch": 0.027093596059113302,
+      "grad_norm": 0.1860426664352417,
+      "learning_rate": 0.00013278688524590162,
+      "loss": 1.9394796371459961,
+      "step": 55
+    },
+    {
+      "epoch": 0.029556650246305417,
+      "grad_norm": 0.2036733627319336,
+      "learning_rate": 0.00014508196721311472,
+      "loss": 1.8991952896118165,
+      "step": 60
+    },
+    {
+      "epoch": 0.03201970443349754,
+      "grad_norm": 0.21620874106884003,
+      "learning_rate": 0.0001499991408257329,
+      "loss": 1.8576393127441406,
+      "step": 65
+    },
+    {
+      "epoch": 0.034482758620689655,
+      "grad_norm": 0.1799483746290207,
+      "learning_rate": 0.00014999389038760955,
+      "loss": 1.834630584716797,
+      "step": 70
+    },
+    {
+      "epoch": 0.03694581280788178,
+      "grad_norm": 0.18848225474357605,
+      "learning_rate": 0.00014998386716414503,
+      "loss": 1.7935523986816406,
+      "step": 75
+    },
+    {
+      "epoch": 0.03940886699507389,
+      "grad_norm": 0.21468029916286469,
+      "learning_rate": 0.00014996907179324066,
+      "loss": 1.8077653884887694,
+      "step": 80
+    },
+    {
+      "epoch": 0.04187192118226601,
+      "grad_norm": 0.20116886496543884,
+      "learning_rate": 0.00014994950521650814,
+      "loss": 1.7493778228759767,
+      "step": 85
+    },
+    {
+      "epoch": 0.04433497536945813,
+      "grad_norm": 0.18760953843593597,
+      "learning_rate": 0.0001499251686792099,
+      "loss": 1.7583953857421875,
+      "step": 90
+    },
+    {
+      "epoch": 0.046798029556650245,
+      "grad_norm": 0.22799813747406006,
+      "learning_rate": 0.00014989606373017966,
+      "loss": 1.7204261779785157,
+      "step": 95
+    },
+    {
+      "epoch": 0.04926108374384237,
+      "grad_norm": 0.2023308128118515,
+      "learning_rate": 0.00014986219222172398,
+      "loss": 1.6771041870117187,
+      "step": 100
+    },
+    {
+      "epoch": 0.05172413793103448,
+      "grad_norm": 0.2221589833498001,
+      "learning_rate": 0.00014982355630950431,
+      "loss": 1.723992156982422,
+      "step": 105
+    },
+    {
+      "epoch": 0.054187192118226604,
+      "grad_norm": 0.2168349176645279,
+      "learning_rate": 0.00014978015845239985,
+      "loss": 1.6915512084960938,
+      "step": 110
+    },
+    {
+      "epoch": 0.05665024630541872,
+      "grad_norm": 0.24630236625671387,
+      "learning_rate": 0.00014973200141235097,
+      "loss": 1.6741020202636718,
+      "step": 115
+    },
+    {
+      "epoch": 0.059113300492610835,
+      "grad_norm": 0.20771881937980652,
+      "learning_rate": 0.00014967908825418353,
+      "loss": 1.6523841857910155,
+      "step": 120
+    },
+    {
+      "epoch": 0.06157635467980296,
+      "grad_norm": 0.2257814109325409,
+      "learning_rate": 0.00014962142234541383,
+      "loss": 1.658163833618164,
+      "step": 125
+    },
+    {
+      "epoch": 0.06403940886699508,
+      "grad_norm": 0.19681569933891296,
+      "learning_rate": 0.00014955900735603425,
+      "loss": 1.6330482482910156,
+      "step": 130
+    },
+    {
+      "epoch": 0.0665024630541872,
+      "grad_norm": 0.22882717847824097,
+      "learning_rate": 0.00014949184725827956,
+      "loss": 1.697220230102539,
+      "step": 135
+    },
+    {
+      "epoch": 0.06896551724137931,
+      "grad_norm": 0.21855653822422028,
+      "learning_rate": 0.0001494199463263744,
+      "loss": 1.6352031707763672,
+      "step": 140
+    },
+    {
+      "epoch": 0.07142857142857142,
+      "grad_norm": 0.21962988376617432,
+      "learning_rate": 0.0001493433091362611,
+      "loss": 1.6244991302490235,
+      "step": 145
+    },
+    {
+      "epoch": 0.07389162561576355,
+      "grad_norm": 0.23113107681274414,
+      "learning_rate": 0.00014926194056530834,
+      "loss": 1.6402645111083984,
+      "step": 150
+    },
+    {
+      "epoch": 0.07635467980295567,
+      "grad_norm": 0.20645377039909363,
+      "learning_rate": 0.00014917584579200095,
+      "loss": 1.5941497802734375,
+      "step": 155
+    },
+    {
+      "epoch": 0.07881773399014778,
+      "grad_norm": 0.20156218111515045,
+      "learning_rate": 0.00014908503029561026,
+      "loss": 1.6078596115112305,
+      "step": 160
+    },
+    {
+      "epoch": 0.0812807881773399,
+      "grad_norm": 0.2265356332063675,
+      "learning_rate": 0.00014898949985584533,
+      "loss": 1.5930806159973145,
+      "step": 165
+    },
+    {
+      "epoch": 0.08374384236453201,
+      "grad_norm": 0.20574317872524261,
+      "learning_rate": 0.00014888926055248524,
+      "loss": 1.6218435287475585,
+      "step": 170
+    },
+    {
+      "epoch": 0.08620689655172414,
+      "grad_norm": 0.30682989954948425,
+      "learning_rate": 0.000148784318764992,
+      "loss": 1.5747103691101074,
+      "step": 175
+    },
+    {
+      "epoch": 0.08866995073891626,
+      "grad_norm": 0.21600587666034698,
+      "learning_rate": 0.00014867468117210468,
+      "loss": 1.5787104606628417,
+      "step": 180
+    },
+    {
+      "epoch": 0.09113300492610837,
+      "grad_norm": 0.24654413759708405,
+      "learning_rate": 0.00014856035475141433,
+      "loss": 1.5819267272949218,
+      "step": 185
+    },
+    {
+      "epoch": 0.09359605911330049,
+      "grad_norm": 0.2086670994758606,
+      "learning_rate": 0.0001484413467789198,
+      "loss": 1.5679956436157227,
+      "step": 190
+    },
+    {
+      "epoch": 0.0960591133004926,
+      "grad_norm": 0.21862246096134186,
+      "learning_rate": 0.0001483176648285649,
+      "loss": 1.580882740020752,
+      "step": 195
+    },
+    {
+      "epoch": 0.09852216748768473,
+      "grad_norm": 0.21744588017463684,
+      "learning_rate": 0.0001481893167717561,
+      "loss": 1.617708969116211,
+      "step": 200
+    },
+    {
+      "epoch": 0.10098522167487685,
+      "grad_norm": 0.22349102795124054,
+      "learning_rate": 0.00014805631077686185,
+      "loss": 1.5356934547424317,
+      "step": 205
+    },
+    {
+      "epoch": 0.10344827586206896,
+      "grad_norm": 0.20569421350955963,
+      "learning_rate": 0.00014791865530869254,
+      "loss": 1.5298048973083496,
+      "step": 210
+    },
+    {
+      "epoch": 0.10591133004926108,
+      "grad_norm": 0.19164322316646576,
+      "learning_rate": 0.00014777635912796181,
+      "loss": 1.532371711730957,
+      "step": 215
+    },
+    {
+      "epoch": 0.10837438423645321,
+      "grad_norm": 0.2381337434053421,
+      "learning_rate": 0.0001476294312907291,
+      "loss": 1.5280881881713868,
+      "step": 220
+    },
+    {
+      "epoch": 0.11083743842364532,
+      "grad_norm": 0.23151648044586182,
+      "learning_rate": 0.0001474778811478231,
+      "loss": 1.5589888572692872,
+      "step": 225
+    },
+    {
+      "epoch": 0.11330049261083744,
+      "grad_norm": 0.2313881665468216,
+      "learning_rate": 0.00014732171834424694,
+      "loss": 1.5487717628479003,
+      "step": 230
+    },
+    {
+      "epoch": 0.11576354679802955,
+      "grad_norm": 0.2612460255622864,
+      "learning_rate": 0.000147160952818564,
+      "loss": 1.5135647773742675,
+      "step": 235
+    },
+    {
+      "epoch": 0.11822660098522167,
+      "grad_norm": 0.20482192933559418,
+      "learning_rate": 0.0001469955948022658,
+      "loss": 1.5466550827026366,
+      "step": 240
+    },
+    {
+      "epoch": 0.1206896551724138,
+      "grad_norm": 0.19292517006397247,
+      "learning_rate": 0.00014682565481912037,
+      "loss": 1.5234511375427247,
+      "step": 245
+    },
+    {
+      "epoch": 0.12315270935960591,
+      "grad_norm": 0.21411260962486267,
+      "learning_rate": 0.00014665114368450294,
+      "loss": 1.5166451454162597,
+      "step": 250
+    },
+    {
+      "epoch": 0.12561576354679804,
+      "grad_norm": 0.25454139709472656,
+      "learning_rate": 0.0001464720725047074,
+      "loss": 1.5222977638244628,
+      "step": 255
+    },
+    {
+      "epoch": 0.12807881773399016,
+      "grad_norm": 0.21349242329597473,
+      "learning_rate": 0.0001462884526762395,
+      "loss": 1.523837947845459,
+      "step": 260
+    },
+    {
+      "epoch": 0.13054187192118227,
+      "grad_norm": 0.24694757163524628,
+      "learning_rate": 0.00014610029588509147,
+      "loss": 1.5198171615600586,
+      "step": 265
+    },
+    {
+      "epoch": 0.1330049261083744,
+      "grad_norm": 0.20886920392513275,
+      "learning_rate": 0.00014590761410599853,
+      "loss": 1.5158114433288574,
+      "step": 270
+    },
+    {
+      "epoch": 0.1354679802955665,
+      "grad_norm": 0.23610642552375793,
+      "learning_rate": 0.0001457104196016765,
+      "loss": 1.5281448364257812,
+      "step": 275
+    },
+    {
+      "epoch": 0.13793103448275862,
+      "grad_norm": 0.1951839178800583,
+      "learning_rate": 0.00014550872492204164,
+      "loss": 1.5356016159057617,
+      "step": 280
+    },
+    {
+      "epoch": 0.14039408866995073,
+      "grad_norm": 0.21323804557323456,
+      "learning_rate": 0.00014530254290341167,
+      "loss": 1.5305219650268556,
+      "step": 285
+    },
+    {
+      "epoch": 0.14285714285714285,
+      "grad_norm": 0.21984122693538666,
+      "learning_rate": 0.00014509188666768914,
+      "loss": 1.5287348747253418,
+      "step": 290
+    },
+    {
+      "epoch": 0.14532019704433496,
+      "grad_norm": 0.21187560260295868,
+      "learning_rate": 0.00014487676962152607,
+      "loss": 1.5206907272338868,
+      "step": 295
+    },
+    {
+      "epoch": 0.1477832512315271,
+      "grad_norm": 0.2447018027305603,
+      "learning_rate": 0.00014465720545547083,
+      "loss": 1.557987594604492,
+      "step": 300
+    },
+    {
+      "epoch": 0.15024630541871922,
+      "grad_norm": 0.2692071199417114,
+      "learning_rate": 0.00014443320814309685,
+      "loss": 1.4944952011108399,
+      "step": 305
+    },
+    {
+      "epoch": 0.15270935960591134,
+      "grad_norm": 0.21768179535865784,
+      "learning_rate": 0.00014420479194011325,
+      "loss": 1.502120590209961,
+      "step": 310
+    },
+    {
+      "epoch": 0.15517241379310345,
+      "grad_norm": 0.20890283584594727,
+      "learning_rate": 0.00014397197138345767,
+      "loss": 1.5136074066162108,
+      "step": 315
+    },
+    {
+      "epoch": 0.15763546798029557,
+      "grad_norm": 0.20394501090049744,
+      "learning_rate": 0.000143734761290371,
+      "loss": 1.4903701782226562,
+      "step": 320
+    },
+    {
+      "epoch": 0.16009852216748768,
+      "grad_norm": 0.2339993268251419,
+      "learning_rate": 0.00014349317675745442,
+      "loss": 1.4917552947998047,
+      "step": 325
+    },
+    {
+      "epoch": 0.1625615763546798,
+      "grad_norm": 0.19806212186813354,
+      "learning_rate": 0.00014324723315970866,
+      "loss": 1.5024358749389648,
+      "step": 330
+    },
+    {
+      "epoch": 0.16502463054187191,
+      "grad_norm": 0.20194905996322632,
+      "learning_rate": 0.00014299694614955538,
+      "loss": 1.4618555068969727,
+      "step": 335
+    },
+    {
+      "epoch": 0.16748768472906403,
+      "grad_norm": 0.2149001955986023,
+      "learning_rate": 0.0001427423316558412,
+      "loss": 1.5069053649902344,
+      "step": 340
+    },
+    {
+      "epoch": 0.16995073891625614,
+      "grad_norm": 0.21930049359798431,
+      "learning_rate": 0.0001424834058828237,
+      "loss": 1.4832588195800782,
+      "step": 345
+    },
+    {
+      "epoch": 0.1724137931034483,
+      "grad_norm": 0.23513002693653107,
+      "learning_rate": 0.00014222018530914049,
+      "loss": 1.5300941467285156,
+      "step": 350
+    },
+    {
+      "epoch": 0.1748768472906404,
+      "grad_norm": 0.24027951061725616,
+      "learning_rate": 0.00014195268668676007,
+      "loss": 1.4527239799499512,
+      "step": 355
+    },
+    {
+      "epoch": 0.17733990147783252,
+      "grad_norm": 0.22858044505119324,
+      "learning_rate": 0.000141680927039916,
+      "loss": 1.4938695907592774,
+      "step": 360
+    },
+    {
+      "epoch": 0.17980295566502463,
+      "grad_norm": 0.1923241913318634,
+      "learning_rate": 0.00014140492366402334,
+      "loss": 1.477768325805664,
+      "step": 365
+    },
+    {
+      "epoch": 0.18226600985221675,
+      "grad_norm": 0.20562614500522614,
+      "learning_rate": 0.00014112469412457782,
+      "loss": 1.4464492797851562,
+      "step": 370
+    },
+    {
+      "epoch": 0.18472906403940886,
+      "grad_norm": 0.21619316935539246,
+      "learning_rate": 0.00014084025625603813,
+      "loss": 1.5029674530029298,
+      "step": 375
+    },
+    {
+      "epoch": 0.18719211822660098,
+      "grad_norm": 0.2293156236410141,
+      "learning_rate": 0.00014055162816069077,
+      "loss": 1.4607781410217284,
+      "step": 380
+    },
+    {
+      "epoch": 0.1896551724137931,
+      "grad_norm": 0.22010239958763123,
+      "learning_rate": 0.000140258828207498,
+      "loss": 1.453076171875,
+      "step": 385
+    },
+    {
+      "epoch": 0.1921182266009852,
+      "grad_norm": 0.21647651493549347,
+      "learning_rate": 0.00013996187503092884,
+      "loss": 1.4681432723999024,
+      "step": 390
+    },
+    {
+      "epoch": 0.19458128078817735,
+      "grad_norm": 0.2153208702802658,
+      "learning_rate": 0.000139660787529773,
+      "loss": 1.5228609085083007,
+      "step": 395
+    },
+    {
+      "epoch": 0.19704433497536947,
+      "grad_norm": 0.19213175773620605,
+      "learning_rate": 0.0001393555848659383,
+      "loss": 1.471001434326172,
+      "step": 400
+    },
+    {
+      "epoch": 0.19950738916256158,
+      "grad_norm": 0.21129681169986725,
+      "learning_rate": 0.00013904628646323105,
+      "loss": 1.4540966033935547,
+      "step": 405
+    },
+    {
+      "epoch": 0.2019704433497537,
+      "grad_norm": 0.23157598078250885,
+      "learning_rate": 0.0001387329120061198,
+      "loss": 1.464762020111084,
+      "step": 410
+    },
+    {
+      "epoch": 0.2044334975369458,
+      "grad_norm": 0.24082177877426147,
+      "learning_rate": 0.00013841548143848283,
+      "loss": 1.4243175506591796,
+      "step": 415
+    },
+    {
+      "epoch": 0.20689655172413793,
+      "grad_norm": 0.21667946875095367,
+      "learning_rate": 0.0001380940149623386,
+      "loss": 1.4062715530395509,
+      "step": 420
+    },
+    {
+      "epoch": 0.20935960591133004,
+      "grad_norm": 0.2099546641111374,
+      "learning_rate": 0.00013776853303656012,
+      "loss": 1.4669032096862793,
+      "step": 425
+    },
+    {
+      "epoch": 0.21182266009852216,
+      "grad_norm": 0.23444649577140808,
+      "learning_rate": 0.0001374390563755731,
+      "loss": 1.4326099395751952,
+      "step": 430
+    },
+    {
+      "epoch": 0.21428571428571427,
+      "grad_norm": 0.19571436941623688,
+      "learning_rate": 0.00013710560594803735,
+      "loss": 1.43844575881958,
+      "step": 435
+    },
+    {
+      "epoch": 0.21674876847290642,
+      "grad_norm": 0.23410262167453766,
+      "learning_rate": 0.00013676820297551243,
+      "loss": 1.4372885704040528,
+      "step": 440
+    },
+    {
+      "epoch": 0.21921182266009853,
+      "grad_norm": 0.19299852848052979,
+      "learning_rate": 0.00013642686893110706,
+      "loss": 1.4768877029418945,
+      "step": 445
+    },
+    {
+      "epoch": 0.22167487684729065,
+      "grad_norm": 0.20944936573505402,
+      "learning_rate": 0.00013608162553811255,
+      "loss": 1.4158610343933105,
+      "step": 450
+    },
+    {
+      "epoch": 0.22413793103448276,
+      "grad_norm": 0.2328590452671051,
+      "learning_rate": 0.00013573249476862007,
+      "loss": 1.404058074951172,
+      "step": 455
+    },
+    {
+      "epoch": 0.22660098522167488,
+      "grad_norm": 0.219072163105011,
+      "learning_rate": 0.0001353794988421227,
+      "loss": 1.3862098693847655,
+      "step": 460
+    },
+    {
+      "epoch": 0.229064039408867,
+      "grad_norm": 0.20859263837337494,
+      "learning_rate": 0.00013502266022410084,
+      "loss": 1.4217020988464355,
+      "step": 465
+    },
+    {
+      "epoch": 0.2315270935960591,
+      "grad_norm": 0.21829617023468018,
+      "learning_rate": 0.00013466200162459292,
+      "loss": 1.40992431640625,
+      "step": 470
+    },
+    {
+      "epoch": 0.23399014778325122,
+      "grad_norm": 0.21700215339660645,
+      "learning_rate": 0.00013429754599674968,
+      "loss": 1.4436223983764649,
+      "step": 475
+    },
+    {
+      "epoch": 0.23645320197044334,
+      "grad_norm": 0.19958427548408508,
+      "learning_rate": 0.0001339293165353737,
+      "loss": 1.4243131637573243,
+      "step": 480
+    },
+    {
+      "epoch": 0.23891625615763548,
+      "grad_norm": 0.2509094178676605,
+      "learning_rate": 0.00013355733667544303,
+      "loss": 1.4158190727233886,
+      "step": 485
+    },
+    {
+      "epoch": 0.2413793103448276,
+      "grad_norm": 0.20713144540786743,
+      "learning_rate": 0.00013318163009061977,
+      "loss": 1.4425319671630858,
+      "step": 490
+    },
+    {
+      "epoch": 0.2438423645320197,
+      "grad_norm": 0.17888081073760986,
+      "learning_rate": 0.00013280222069174348,
+      "loss": 1.437204360961914,
+      "step": 495
+    },
+    {
+      "epoch": 0.24630541871921183,
+      "grad_norm": 0.22022056579589844,
+      "learning_rate": 0.0001324191326253094,
+      "loss": 1.4549561500549317,
+      "step": 500
+    },
+    {
+      "epoch": 0.24876847290640394,
+      "grad_norm": 0.21922940015792847,
+      "learning_rate": 0.00013203239027193174,
+      "loss": 1.4171875,
+      "step": 505
+    },
+    {
+      "epoch": 0.2512315270935961,
+      "grad_norm": 0.19908225536346436,
+      "learning_rate": 0.00013164201824479197,
+      "loss": 1.3846515655517577,
+      "step": 510
+    },
+    {
+      "epoch": 0.2536945812807882,
+      "grad_norm": 0.2016427367925644,
+      "learning_rate": 0.00013124804138807248,
+      "loss": 1.4141998291015625,
+      "step": 515
+    },
+    {
+      "epoch": 0.2561576354679803,
+      "grad_norm": 0.2238454669713974,
+      "learning_rate": 0.00013085048477537531,
+      "loss": 1.4454143524169922,
+      "step": 520
+    },
+    {
+      "epoch": 0.25862068965517243,
+      "grad_norm": 0.22041630744934082,
+      "learning_rate": 0.00013044937370812655,
+      "loss": 1.3954301834106446,
+      "step": 525
+    },
+    {
+      "epoch": 0.26108374384236455,
+      "grad_norm": 0.2228841781616211,
+      "learning_rate": 0.00013004473371396603,
+      "loss": 1.3917302131652831,
+      "step": 530
+    },
+    {
+      "epoch": 0.26354679802955666,
+      "grad_norm": 0.21002982556819916,
+      "learning_rate": 0.00012963659054512265,
+      "loss": 1.4145011901855469,
+      "step": 535
+    },
+    {
+      "epoch": 0.2660098522167488,
+      "grad_norm": 0.18161620199680328,
+      "learning_rate": 0.0001292249701767755,
+      "loss": 1.439390277862549,
+      "step": 540
+    },
+    {
+      "epoch": 0.2684729064039409,
+      "grad_norm": 0.189104825258255,
+      "learning_rate": 0.00012880989880540068,
+      "loss": 1.4274805068969727,
+      "step": 545
+    },
+    {
+      "epoch": 0.270935960591133,
+      "grad_norm": 0.21180245280265808,
+      "learning_rate": 0.00012839140284710425,
+      "loss": 1.4168146133422852,
+      "step": 550
+    },
+    {
+      "epoch": 0.2733990147783251,
+      "grad_norm": 0.21424278616905212,
+      "learning_rate": 0.00012796950893594083,
+      "loss": 1.4109338760375976,
+      "step": 555
+    },
+    {
+      "epoch": 0.27586206896551724,
+      "grad_norm": 0.21506839990615845,
+      "learning_rate": 0.00012754424392221866,
+      "loss": 1.3610061645507812,
+      "step": 560
+    },
+    {
+      "epoch": 0.27832512315270935,
+      "grad_norm": 0.23869743943214417,
+      "learning_rate": 0.0001271156348707908,
+      "loss": 1.4243642807006835,
+      "step": 565
+    },
+    {
+      "epoch": 0.28078817733990147,
+      "grad_norm": 0.2110007107257843,
+      "learning_rate": 0.0001266837090593327,
+      "loss": 1.4199881553649902,
+      "step": 570
+    },
+    {
+      "epoch": 0.2832512315270936,
+      "grad_norm": 0.1898839771747589,
+      "learning_rate": 0.00012624849397660596,
+      "loss": 1.403303337097168,
+      "step": 575
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 0.26501619815826416,
+      "learning_rate": 0.0001258100173207092,
+      "loss": 1.3995702743530274,
+      "step": 580
+    },
+    {
+      "epoch": 0.2881773399014778,
+      "grad_norm": 0.2318245768547058,
+      "learning_rate": 0.00012536830699731502,
+      "loss": 1.4003644943237306,
+      "step": 585
+    },
+    {
+      "epoch": 0.29064039408866993,
+      "grad_norm": 0.1923232227563858,
+      "learning_rate": 0.00012492339111789424,
+      "loss": 1.4000411987304688,
+      "step": 590
+    },
+    {
+      "epoch": 0.29310344827586204,
+      "grad_norm": 0.24175603687763214,
+      "learning_rate": 0.0001244752979979266,
+      "loss": 1.4027944564819337,
+      "step": 595
+    },
+    {
+      "epoch": 0.2955665024630542,
+      "grad_norm": 0.2166663110256195,
+      "learning_rate": 0.00012402405615509886,
+      "loss": 1.3940664291381837,
+      "step": 600
+    },
+    {
+      "epoch": 0.29802955665024633,
+      "grad_norm": 0.20402714610099792,
+      "learning_rate": 0.0001235696943074898,
+      "loss": 1.430446243286133,
+      "step": 605
+    },
+    {
+      "epoch": 0.30049261083743845,
+      "grad_norm": 0.21095804870128632,
+      "learning_rate": 0.00012311224137174256,
+      "loss": 1.4028675079345703,
+      "step": 610
+    },
+    {
+      "epoch": 0.30295566502463056,
+      "grad_norm": 0.24699363112449646,
+      "learning_rate": 0.0001226517264612243,
+      "loss": 1.3896512985229492,
+      "step": 615
+    },
+    {
+      "epoch": 0.3054187192118227,
+      "grad_norm": 0.23743489384651184,
+      "learning_rate": 0.00012218817888417341,
+      "loss": 1.3832998275756836,
+      "step": 620
+    },
+    {
+      "epoch": 0.3078817733990148,
+      "grad_norm": 0.22747831046581268,
+      "learning_rate": 0.00012172162814183414,
+      "loss": 1.4147226333618164,
+      "step": 625
+    },
+    {
+      "epoch": 0.3103448275862069,
+      "grad_norm": 0.21350650489330292,
+      "learning_rate": 0.0001212521039265792,
+      "loss": 1.4063333511352538,
+      "step": 630
+    },
+    {
+      "epoch": 0.312807881773399,
+      "grad_norm": 0.231597900390625,
+      "learning_rate": 0.00012077963612002003,
+      "loss": 1.4085102081298828,
+      "step": 635
+    },
+    {
+      "epoch": 0.31527093596059114,
+      "grad_norm": 0.22213512659072876,
+      "learning_rate": 0.00012030425479110504,
+      "loss": 1.379136562347412,
+      "step": 640
+    },
+    {
+      "epoch": 0.31773399014778325,
+      "grad_norm": 0.21056486666202545,
+      "learning_rate": 0.00011982599019420594,
+      "loss": 1.3508719444274901,
+      "step": 645
+    },
+    {
+      "epoch": 0.32019704433497537,
+      "grad_norm": 0.24325105547904968,
+      "learning_rate": 0.00011934487276719238,
+      "loss": 1.4178260803222655,
+      "step": 650
+    },
+    {
+      "epoch": 0.3226600985221675,
+      "grad_norm": 0.23807983100414276,
+      "learning_rate": 0.00011886093312949469,
+      "loss": 1.3915666580200194,
+      "step": 655
+    },
+    {
+      "epoch": 0.3251231527093596,
+      "grad_norm": 0.20901484787464142,
+      "learning_rate": 0.00011837420208015526,
+      "loss": 1.3760847091674804,
+      "step": 660
+    },
+    {
+      "epoch": 0.3275862068965517,
+      "grad_norm": 0.19761306047439575,
+      "learning_rate": 0.00011788471059586839,
+      "loss": 1.4036226272583008,
+      "step": 665
+    },
+    {
+      "epoch": 0.33004926108374383,
+      "grad_norm": 0.250301718711853,
+      "learning_rate": 0.00011739248982900887,
+      "loss": 1.3270477294921874,
+      "step": 670
+    },
+    {
+      "epoch": 0.33251231527093594,
+      "grad_norm": 0.19960308074951172,
+      "learning_rate": 0.0001168975711056494,
+      "loss": 1.3685549736022948,
+      "step": 675
+    },
+    {
+      "epoch": 0.33497536945812806,
+      "grad_norm": 0.19794125854969025,
+      "learning_rate": 0.00011639998592356684,
+      "loss": 1.3679879188537598,
+      "step": 680
+    },
+    {
+      "epoch": 0.3374384236453202,
+      "grad_norm": 0.236933633685112,
+      "learning_rate": 0.00011589976595023777,
+      "loss": 1.3485721588134765,
+      "step": 685
+    },
+    {
+      "epoch": 0.3399014778325123,
+      "grad_norm": 0.23574577271938324,
+      "learning_rate": 0.00011539694302082288,
+      "loss": 1.3585886001586913,
+      "step": 690
+    },
+    {
+      "epoch": 0.34236453201970446,
+      "grad_norm": 0.24639375507831573,
+      "learning_rate": 0.00011489154913614111,
+      "loss": 1.3602943420410156,
+      "step": 695
+    },
+    {
+      "epoch": 0.3448275862068966,
+      "grad_norm": 0.2161063253879547,
+      "learning_rate": 0.00011438361646063298,
+      "loss": 1.3920618057250977,
+      "step": 700
+    },
+    {
+      "epoch": 0.3472906403940887,
+      "grad_norm": 0.23967303335666656,
+      "learning_rate": 0.00011387317732031346,
+      "loss": 1.323176097869873,
+      "step": 705
+    },
+    {
+      "epoch": 0.3497536945812808,
+      "grad_norm": 0.22842799127101898,
+      "learning_rate": 0.00011336026420071491,
+      "loss": 1.3453125,
+      "step": 710
+    },
+    {
+      "epoch": 0.3522167487684729,
+      "grad_norm": 0.26447567343711853,
+      "learning_rate": 0.00011284490974481929,
+      "loss": 1.3231969833374024,
+      "step": 715
+    },
+    {
+      "epoch": 0.35467980295566504,
+      "grad_norm": 0.19098453223705292,
+      "learning_rate": 0.00011232714675098106,
+      "loss": 1.3952215194702149,
+      "step": 720
+    },
+    {
+      "epoch": 0.35714285714285715,
+      "grad_norm": 0.215730220079422,
+      "learning_rate": 0.00011180700817083949,
+      "loss": 1.4024740219116212,
+      "step": 725
+    },
+    {
+      "epoch": 0.35960591133004927,
+      "grad_norm": 0.20249366760253906,
+      "learning_rate": 0.00011128452710722171,
+      "loss": 1.3558260917663574,
+      "step": 730
+    },
+    {
+      "epoch": 0.3620689655172414,
+      "grad_norm": 0.21701344847679138,
+      "learning_rate": 0.00011075973681203601,
+      "loss": 1.4104610443115235,
+      "step": 735
+    },
+    {
+      "epoch": 0.3645320197044335,
+      "grad_norm": 0.20061460137367249,
+      "learning_rate": 0.00011023267068415545,
+      "loss": 1.3634204864501953,
+      "step": 740
+    },
+    {
+      "epoch": 0.3669950738916256,
+      "grad_norm": 0.2143600583076477,
+      "learning_rate": 0.00010970336226729242,
+      "loss": 1.3240375518798828,
+      "step": 745
+    },
+    {
+      "epoch": 0.3694581280788177,
+      "grad_norm": 0.20612865686416626,
+      "learning_rate": 0.00010917184524786377,
+      "loss": 1.3753711700439453,
+      "step": 750
+    },
+    {
+      "epoch": 0.37192118226600984,
+      "grad_norm": 0.2549595832824707,
+      "learning_rate": 0.0001086381534528469,
+      "loss": 1.3931709289550782,
+      "step": 755
+    },
+    {
+      "epoch": 0.37438423645320196,
+      "grad_norm": 0.21721267700195312,
+      "learning_rate": 0.0001081023208476271,
+      "loss": 1.3496780395507812,
+      "step": 760
+    },
+    {
+      "epoch": 0.3768472906403941,
+      "grad_norm": 0.20957210659980774,
+      "learning_rate": 0.00010756438153383568,
+      "loss": 1.3488155364990235,
+      "step": 765
+    },
+    {
+      "epoch": 0.3793103448275862,
+      "grad_norm": 0.21333415806293488,
+      "learning_rate": 0.00010702436974717984,
+      "loss": 1.314176368713379,
+      "step": 770
+    },
+    {
+      "epoch": 0.3817733990147783,
+      "grad_norm": 0.22144381701946259,
+      "learning_rate": 0.00010648231985526371,
+      "loss": 1.3665472984313964,
+      "step": 775
+    },
+    {
+      "epoch": 0.3842364532019704,
+      "grad_norm": 0.22947055101394653,
+      "learning_rate": 0.00010593826635540124,
+      "loss": 1.3796130180358888,
+      "step": 780
+    },
+    {
+      "epoch": 0.3866995073891626,
+      "grad_norm": 0.22103546559810638,
+      "learning_rate": 0.00010539224387242056,
+      "loss": 1.3775650024414063,
+      "step": 785
+    },
+    {
+      "epoch": 0.3891625615763547,
+      "grad_norm": 0.2005070149898529,
+      "learning_rate": 0.00010484428715646054,
+      "loss": 1.3010364532470704,
+      "step": 790
+    },
+    {
+      "epoch": 0.3916256157635468,
+      "grad_norm": 0.1832893341779709,
+      "learning_rate": 0.00010429443108075909,
+      "loss": 1.3438265800476075,
+      "step": 795
+    },
+    {
+      "epoch": 0.39408866995073893,
+      "grad_norm": 0.23130114376544952,
+      "learning_rate": 0.00010374271063943377,
+      "loss": 1.3167936325073242,
+      "step": 800
+    },
+    {
+      "epoch": 0.39655172413793105,
+      "grad_norm": 0.24008631706237793,
+      "learning_rate": 0.0001031891609452547,
+      "loss": 1.3509979248046875,
+      "step": 805
+    },
+    {
+      "epoch": 0.39901477832512317,
+      "grad_norm": 0.22518666088581085,
+      "learning_rate": 0.00010263381722740999,
+      "loss": 1.3557344436645509,
+      "step": 810
+    },
+    {
+      "epoch": 0.4014778325123153,
+      "grad_norm": 0.2309686541557312,
+      "learning_rate": 0.00010207671482926341,
+      "loss": 1.350716781616211,
+      "step": 815
+    },
+    {
+      "epoch": 0.4039408866995074,
+      "grad_norm": 0.19502563774585724,
+      "learning_rate": 0.00010151788920610539,
+      "loss": 1.3548830986022948,
+      "step": 820
+    },
+    {
+      "epoch": 0.4064039408866995,
+      "grad_norm": 0.21045354008674622,
+      "learning_rate": 0.00010095737592289636,
+      "loss": 1.3594703674316406,
+      "step": 825
+    },
+    {
+      "epoch": 0.4088669950738916,
+      "grad_norm": 0.22062793374061584,
+      "learning_rate": 0.00010039521065200324,
+      "loss": 1.338469886779785,
+      "step": 830
+    },
+    {
+      "epoch": 0.41133004926108374,
+      "grad_norm": 0.19777610898017883,
+      "learning_rate": 9.983142917092945e-05,
+      "loss": 1.3236379623413086,
+      "step": 835
+    },
+    {
+      "epoch": 0.41379310344827586,
+      "grad_norm": 0.22311320900917053,
+      "learning_rate": 9.926606736003767e-05,
+      "loss": 1.3418456077575684,
+      "step": 840
+    },
+    {
+      "epoch": 0.41625615763546797,
+      "grad_norm": 0.1984044313430786,
+      "learning_rate": 9.869916120026651e-05,
+      "loss": 1.3537532806396484,
+      "step": 845
+    },
+    {
+      "epoch": 0.4187192118226601,
+      "grad_norm": 0.21143019199371338,
+      "learning_rate": 9.813074677084046e-05,
+      "loss": 1.3383522033691406,
+      "step": 850
+    },
+    {
+      "epoch": 0.4211822660098522,
+      "grad_norm": 0.23575901985168457,
+      "learning_rate": 9.756086024697383e-05,
+      "loss": 1.3635971069335937,
+      "step": 855
+    },
+    {
+      "epoch": 0.4236453201970443,
+      "grad_norm": 0.2878618836402893,
+      "learning_rate": 9.698953789756844e-05,
+      "loss": 1.4005340576171874,
+      "step": 860
+    },
+    {
+      "epoch": 0.42610837438423643,
+      "grad_norm": 0.2632979154586792,
+      "learning_rate": 9.641681608290539e-05,
+      "loss": 1.3476665496826172,
+      "step": 865
+    },
+    {
+      "epoch": 0.42857142857142855,
+      "grad_norm": 0.2310360074043274,
+      "learning_rate": 9.584273125233097e-05,
+      "loss": 1.3294408798217774,
+      "step": 870
+    },
+    {
+      "epoch": 0.43103448275862066,
+      "grad_norm": 0.23508256673812866,
+      "learning_rate": 9.526731994193695e-05,
+      "loss": 1.34185733795166,
+      "step": 875
+    },
+    {
+      "epoch": 0.43349753694581283,
+      "grad_norm": 0.21447646617889404,
+      "learning_rate": 9.469061877223547e-05,
+      "loss": 1.3190247535705566,
+      "step": 880
+    },
+    {
+      "epoch": 0.43596059113300495,
+      "grad_norm": 0.24297568202018738,
+      "learning_rate": 9.411266444582817e-05,
+      "loss": 1.3310616493225098,
+      "step": 885
+    },
+    {
+      "epoch": 0.43842364532019706,
+      "grad_norm": 0.22872385382652283,
+      "learning_rate": 9.353349374507056e-05,
+      "loss": 1.3593338966369628,
+      "step": 890
+    },
+    {
+      "epoch": 0.4408866995073892,
+      "grad_norm": 0.22505290806293488,
+      "learning_rate": 9.29531435297311e-05,
+      "loss": 1.3543621063232423,
+      "step": 895
+    },
+    {
+      "epoch": 0.4433497536945813,
+      "grad_norm": 0.2278539389371872,
+      "learning_rate": 9.237165073464512e-05,
+      "loss": 1.3401315689086915,
+      "step": 900
+    },
+    {
+      "epoch": 0.4458128078817734,
+      "grad_norm": 0.2113586664199829,
+      "learning_rate": 9.178905236736457e-05,
+      "loss": 1.3289443969726562,
+      "step": 905
+    },
+    {
+      "epoch": 0.4482758620689655,
+      "grad_norm": 0.2925296127796173,
+      "learning_rate": 9.120538550580247e-05,
+      "loss": 1.3231324195861816,
+      "step": 910
+    },
+    {
+      "epoch": 0.45073891625615764,
+      "grad_norm": 0.19817602634429932,
+      "learning_rate": 9.062068729587331e-05,
+      "loss": 1.3819540023803711,
+      "step": 915
+    },
+    {
+      "epoch": 0.45320197044334976,
+      "grad_norm": 0.2078256458044052,
+      "learning_rate": 9.003499494912907e-05,
+      "loss": 1.3268487930297852,
+      "step": 920
+    },
+    {
+      "epoch": 0.45566502463054187,
+      "grad_norm": 0.23125478625297546,
+      "learning_rate": 8.944834574039077e-05,
+      "loss": 1.2976766586303712,
+      "step": 925
+    },
+    {
+      "epoch": 0.458128078817734,
+      "grad_norm": 0.2297288328409195,
+      "learning_rate": 8.886077700537644e-05,
+      "loss": 1.295766830444336,
+      "step": 930
+    },
+    {
+      "epoch": 0.4605911330049261,
+      "grad_norm": 0.20964743196964264,
+      "learning_rate": 8.827232613832487e-05,
+      "loss": 1.3288904190063477,
+      "step": 935
+    },
+    {
+      "epoch": 0.4630541871921182,
+      "grad_norm": 0.20685701072216034,
+      "learning_rate": 8.768303058961581e-05,
+      "loss": 1.2745248794555664,
+      "step": 940
+    },
+    {
+      "epoch": 0.46551724137931033,
+      "grad_norm": 0.23295199871063232,
+      "learning_rate": 8.709292786338648e-05,
+      "loss": 1.300822925567627,
+      "step": 945
+    },
+    {
+      "epoch": 0.46798029556650245,
+      "grad_norm": 0.2697843611240387,
+      "learning_rate": 8.650205551514475e-05,
+      "loss": 1.3437896728515626,
+      "step": 950
+    },
+    {
+      "epoch": 0.47044334975369456,
+      "grad_norm": 0.26048171520233154,
+      "learning_rate": 8.591045114937903e-05,
+      "loss": 1.2844138145446777,
+      "step": 955
+    },
+    {
+      "epoch": 0.4729064039408867,
+      "grad_norm": 0.19388702511787415,
+      "learning_rate": 8.531815241716503e-05,
+      "loss": 1.3405949592590332,
+      "step": 960
+    },
+    {
+      "epoch": 0.4753694581280788,
+      "grad_norm": 0.2128061205148697,
+      "learning_rate": 8.472519701376958e-05,
+      "loss": 1.3624204635620116,
+      "step": 965
+    },
+    {
+      "epoch": 0.47783251231527096,
+      "grad_norm": 0.2112760990858078,
+      "learning_rate": 8.413162267625154e-05,
+      "loss": 1.3031497955322267,
+      "step": 970
+    },
+    {
+      "epoch": 0.4802955665024631,
+      "grad_norm": 0.20936837792396545,
+      "learning_rate": 8.353746718106016e-05,
+      "loss": 1.3384126663208007,
+      "step": 975
+    },
+    {
+      "epoch": 0.4827586206896552,
+      "grad_norm": 0.2166673243045807,
+      "learning_rate": 8.294276834163105e-05,
+      "loss": 1.2886722564697266,
+      "step": 980
+    },
+    {
+      "epoch": 0.4852216748768473,
+      "grad_norm": 0.2544674575328827,
+      "learning_rate": 8.234756400597928e-05,
+      "loss": 1.309803581237793,
+      "step": 985
+    },
+    {
+      "epoch": 0.4876847290640394,
+      "grad_norm": 0.25606396794319153,
+      "learning_rate": 8.17518920542911e-05,
+      "loss": 1.3331369400024413,
+      "step": 990
+    },
+    {
+      "epoch": 0.49014778325123154,
+      "grad_norm": 0.25076860189437866,
+      "learning_rate": 8.11557903965128e-05,
+      "loss": 1.2805109024047852,
+      "step": 995
+    },
+    {
+      "epoch": 0.49261083743842365,
+      "grad_norm": 0.2974901795387268,
+      "learning_rate": 8.055929696993818e-05,
+      "loss": 1.2993100166320801,
+      "step": 1000
+    },
+    {
+      "epoch": 0.49507389162561577,
+      "grad_norm": 0.24903179705142975,
+      "learning_rate": 7.996244973679417e-05,
+      "loss": 1.3078741073608398,
+      "step": 1005
+    },
+    {
+      "epoch": 0.4975369458128079,
+      "grad_norm": 0.23043100535869598,
+      "learning_rate": 7.936528668182469e-05,
+      "loss": 1.2467409133911134,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 0.21764904260635376,
+      "learning_rate": 7.876784580987335e-05,
+      "loss": 1.3044242858886719,
+      "step": 1015
+    },
+    {
+      "epoch": 0.5024630541871922,
+      "grad_norm": 0.24756427109241486,
+      "learning_rate": 7.817016514346469e-05,
+      "loss": 1.2884490966796875,
+      "step": 1020
+    },
+    {
+      "epoch": 0.5049261083743842,
+      "grad_norm": 0.21987412869930267,
+      "learning_rate": 7.757228272038429e-05,
+      "loss": 1.2775875091552735,
+      "step": 1025
+    },
+    {
+      "epoch": 0.5073891625615764,
+      "grad_norm": 0.21271178126335144,
+      "learning_rate": 7.6974236591258e-05,
+      "loss": 1.253822135925293,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5098522167487685,
+      "grad_norm": 0.22218871116638184,
+      "learning_rate": 7.637606481713033e-05,
+      "loss": 1.3192794799804688,
+      "step": 1035
+    },
+    {
+      "epoch": 0.5123152709359606,
+      "grad_norm": 0.21748924255371094,
+      "learning_rate": 7.577780546704205e-05,
+      "loss": 1.2793066024780273,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5147783251231527,
+      "grad_norm": 0.27152219414711,
+      "learning_rate": 7.517949661560758e-05,
+      "loss": 1.3064208984375,
+      "step": 1045
+    },
+    {
+      "epoch": 0.5172413793103449,
+      "grad_norm": 0.2652308940887451,
+      "learning_rate": 7.458117634059167e-05,
+      "loss": 1.2768896102905274,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5197044334975369,
+      "grad_norm": 0.2394336760044098,
+      "learning_rate": 7.398288272048605e-05,
+      "loss": 1.3248939514160156,
+      "step": 1055
+    },
+    {
+      "epoch": 0.5221674876847291,
+      "grad_norm": 0.21198320388793945,
+      "learning_rate": 7.338465383208616e-05,
+      "loss": 1.330735206604004,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5246305418719212,
+      "grad_norm": 0.22435452044010162,
+      "learning_rate": 7.278652774806765e-05,
+      "loss": 1.2982617378234864,
+      "step": 1065
+    },
+    {
+      "epoch": 0.5270935960591133,
+      "grad_norm": 0.22276167571544647,
+      "learning_rate": 7.218854253456357e-05,
+      "loss": 1.3383667945861817,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5295566502463054,
+      "grad_norm": 0.21616998314857483,
+      "learning_rate": 7.159073624874156e-05,
+      "loss": 1.3443126678466797,
+      "step": 1075
+    },
+    {
+      "epoch": 0.5320197044334976,
+      "grad_norm": 0.25767794251441956,
+      "learning_rate": 7.099314693638197e-05,
+      "loss": 1.346919059753418,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5344827586206896,
+      "grad_norm": 0.21083039045333862,
+      "learning_rate": 7.039581262945641e-05,
+      "loss": 1.2509237289428712,
+      "step": 1085
+    },
+    {
+      "epoch": 0.5369458128078818,
+      "grad_norm": 0.2217107117176056,
+      "learning_rate": 6.979877134370729e-05,
+      "loss": 1.3491952896118165,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5394088669950738,
+      "grad_norm": 0.2276228666305542,
+      "learning_rate": 6.920206107622857e-05,
+      "loss": 1.3038724899291991,
+      "step": 1095
+    },
+    {
+      "epoch": 0.541871921182266,
+      "grad_norm": 0.20058490335941315,
+      "learning_rate": 6.860571980304739e-05,
+      "loss": 1.2769588470458983,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5443349753694581,
+      "grad_norm": 0.2430969476699829,
+      "learning_rate": 6.800978547670727e-05,
+      "loss": 1.297713279724121,
+      "step": 1105
+    },
+    {
+      "epoch": 0.5467980295566502,
+      "grad_norm": 0.20122674107551575,
+      "learning_rate": 6.741429602385268e-05,
+      "loss": 1.3468814849853517,
+      "step": 1110
+    },
+    {
+      "epoch": 0.5492610837438424,
+      "grad_norm": 0.22891351580619812,
+      "learning_rate": 6.681928934281521e-05,
+      "loss": 1.3072088241577149,
+      "step": 1115
+    },
+    {
+      "epoch": 0.5517241379310345,
+      "grad_norm": 0.22393646836280823,
+      "learning_rate": 6.622480330120186e-05,
+      "loss": 1.2663318634033203,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5541871921182266,
+      "grad_norm": 0.247694730758667,
+      "learning_rate": 6.563087573348494e-05,
+      "loss": 1.3064285278320313,
+      "step": 1125
+    },
+    {
+      "epoch": 0.5566502463054187,
+      "grad_norm": 0.26341551542282104,
+      "learning_rate": 6.503754443859403e-05,
+      "loss": 1.311113166809082,
+      "step": 1130
+    },
+    {
+      "epoch": 0.5591133004926109,
+      "grad_norm": 0.24816490709781647,
+      "learning_rate": 6.44448471775107e-05,
+      "loss": 1.2693605422973633,
+      "step": 1135
+    },
+    {
+      "epoch": 0.5615763546798029,
+      "grad_norm": 0.2166696935892105,
+      "learning_rate": 6.3852821670865e-05,
+      "loss": 1.2208733558654785,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5640394088669951,
+      "grad_norm": 0.20724032819271088,
+      "learning_rate": 6.326150559653502e-05,
+      "loss": 1.2915777206420898,
+      "step": 1145
+    },
+    {
+      "epoch": 0.5665024630541872,
+      "grad_norm": 0.23592734336853027,
+      "learning_rate": 6.267093658724898e-05,
+      "loss": 1.3022445678710937,
+      "step": 1150
+    },
+    {
+      "epoch": 0.5689655172413793,
+      "grad_norm": 0.21348340809345245,
+      "learning_rate": 6.208115222819005e-05,
+      "loss": 1.2936946868896484,
+      "step": 1155
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 0.22701257467269897,
+      "learning_rate": 6.149219005460453e-05,
+      "loss": 1.274067497253418,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5738916256157636,
+      "grad_norm": 0.2418663650751114,
+      "learning_rate": 6.090408754941285e-05,
+      "loss": 1.28143367767334,
+      "step": 1165
+    },
+    {
+      "epoch": 0.5763546798029556,
+      "grad_norm": 0.2417578399181366,
+      "learning_rate": 6.031688214082415e-05,
+      "loss": 1.28759765625,
+      "step": 1170
+    },
+    {
+      "epoch": 0.5788177339901478,
+      "grad_norm": 0.23263967037200928,
+      "learning_rate": 5.973061119995434e-05,
+      "loss": 1.3238054275512696,
+      "step": 1175
+    },
+    {
+      "epoch": 0.5812807881773399,
+      "grad_norm": 0.21644318103790283,
+      "learning_rate": 5.9145312038447575e-05,
+      "loss": 1.2848752975463866,
+      "step": 1180
+    },
+    {
+      "epoch": 0.583743842364532,
+      "grad_norm": 0.23622092604637146,
+      "learning_rate": 5.8561021906101726e-05,
+      "loss": 1.2801434516906738,
+      "step": 1185
+    },
+    {
+      "epoch": 0.5862068965517241,
+      "grad_norm": 0.21884071826934814,
+      "learning_rate": 5.7977777988497664e-05,
+      "loss": 1.2974514007568358,
+      "step": 1190
+    },
+    {
+      "epoch": 0.5886699507389163,
+      "grad_norm": 0.24541740119457245,
+      "learning_rate": 5.7395617404632805e-05,
+      "loss": 1.2843940734863282,
+      "step": 1195
+    },
+    {
+      "epoch": 0.5911330049261084,
+      "grad_norm": 0.24113544821739197,
+      "learning_rate": 5.68145772045586e-05,
+      "loss": 1.3175272941589355,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5935960591133005,
+      "grad_norm": 0.24389614164829254,
+      "learning_rate": 5.6234694367022773e-05,
+      "loss": 1.2755836486816405,
+      "step": 1205
+    },
+    {
+      "epoch": 0.5960591133004927,
+      "grad_norm": 0.23618771135807037,
+      "learning_rate": 5.565600579711574e-05,
+      "loss": 1.2963144302368164,
+      "step": 1210
+    },
+    {
+      "epoch": 0.5985221674876847,
+      "grad_norm": 0.23990461230278015,
+      "learning_rate": 5.507854832392192e-05,
+      "loss": 1.3165901184082032,
+      "step": 1215
+    },
+    {
+      "epoch": 0.6009852216748769,
+      "grad_norm": 0.2784763276576996,
+      "learning_rate": 5.450235869817596e-05,
+      "loss": 1.2695181846618653,
+      "step": 1220
+    },
+    {
+      "epoch": 0.603448275862069,
+      "grad_norm": 0.1964227259159088,
+      "learning_rate": 5.392747358992374e-05,
+      "loss": 1.2765348434448243,
+      "step": 1225
+    },
+    {
+      "epoch": 0.6059113300492611,
+      "grad_norm": 0.2433655858039856,
+      "learning_rate": 5.335392958618855e-05,
+      "loss": 1.3531946182250976,
+      "step": 1230
+    },
+    {
+      "epoch": 0.6083743842364532,
+      "grad_norm": 0.18715134263038635,
+      "learning_rate": 5.278176318864278e-05,
+      "loss": 1.287767505645752,
+      "step": 1235
+    },
+    {
+      "epoch": 0.6108374384236454,
+      "grad_norm": 0.2287338525056839,
+      "learning_rate": 5.221101081128463e-05,
+      "loss": 1.2737717628479004,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6133004926108374,
+      "grad_norm": 0.2689953148365021,
+      "learning_rate": 5.1641708778120867e-05,
+      "loss": 1.2990927696228027,
+      "step": 1245
+    },
+    {
+      "epoch": 0.6157635467980296,
+      "grad_norm": 0.23197118937969208,
+      "learning_rate": 5.1073893320854976e-05,
+      "loss": 1.2827943801879882,
+      "step": 1250
+    },
+    {
+      "epoch": 0.6182266009852216,
+      "grad_norm": 0.2240389734506607,
+      "learning_rate": 5.050760057658125e-05,
+      "loss": 1.3229182243347168,
+      "step": 1255
+    },
+    {
+      "epoch": 0.6206896551724138,
+      "grad_norm": 0.2288672775030136,
+      "learning_rate": 4.994286658548508e-05,
+      "loss": 1.252695083618164,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6231527093596059,
+      "grad_norm": 0.24411918222904205,
+      "learning_rate": 4.937972728854909e-05,
+      "loss": 1.2915426254272462,
+      "step": 1265
+    },
+    {
+      "epoch": 0.625615763546798,
+      "grad_norm": 0.24725782871246338,
+      "learning_rate": 4.8818218525265895e-05,
+      "loss": 1.317911720275879,
+      "step": 1270
+    },
+    {
+      "epoch": 0.6280788177339901,
+      "grad_norm": 0.2176976054906845,
+      "learning_rate": 4.825837603135718e-05,
+      "loss": 1.269662094116211,
+      "step": 1275
+    },
+    {
+      "epoch": 0.6305418719211823,
+      "grad_norm": 0.21912750601768494,
+      "learning_rate": 4.77002354364994e-05,
+      "loss": 1.2884700775146485,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6330049261083743,
+      "grad_norm": 0.26924338936805725,
+      "learning_rate": 4.7143832262056166e-05,
+      "loss": 1.2128746032714843,
+      "step": 1285
+    },
+    {
+      "epoch": 0.6354679802955665,
+      "grad_norm": 0.2496652603149414,
+      "learning_rate": 4.658920191881757e-05,
+      "loss": 1.363164520263672,
+      "step": 1290
+    },
+    {
+      "epoch": 0.6379310344827587,
+      "grad_norm": 0.2446102499961853,
+      "learning_rate": 4.603637970474671e-05,
+      "loss": 1.245234775543213,
+      "step": 1295
+    },
+    {
+      "epoch": 0.6403940886699507,
+      "grad_norm": 0.25118279457092285,
+      "learning_rate": 4.548540080273308e-05,
+      "loss": 1.2631161689758301,
+      "step": 1300
+    },
+    {
+      "epoch": 0.6428571428571429,
+      "grad_norm": 0.21729733049869537,
+      "learning_rate": 4.493630027835356e-05,
+      "loss": 1.2592344284057617,
+      "step": 1305
+    },
+    {
+      "epoch": 0.645320197044335,
+      "grad_norm": 0.26334258913993835,
+      "learning_rate": 4.4389113077640646e-05,
+      "loss": 1.2242905616760253,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6477832512315271,
+      "grad_norm": 0.227190300822258,
+      "learning_rate": 4.3843874024858524e-05,
+      "loss": 1.2664241790771484,
+      "step": 1315
+    },
+    {
+      "epoch": 0.6502463054187192,
+      "grad_norm": 0.23248690366744995,
+      "learning_rate": 4.330061782028671e-05,
+      "loss": 1.2610064506530763,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6527093596059114,
+      "grad_norm": 0.21380947530269623,
+      "learning_rate": 4.2759379038011686e-05,
+      "loss": 1.2617598533630372,
+      "step": 1325
+    },
+    {
+      "epoch": 0.6551724137931034,
+      "grad_norm": 0.2660558223724365,
+      "learning_rate": 4.222019212372644e-05,
+      "loss": 1.226475143432617,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6576354679802956,
+      "grad_norm": 0.22776232659816742,
+      "learning_rate": 4.168309139253834e-05,
+      "loss": 1.237477684020996,
+      "step": 1335
+    },
+    {
+      "epoch": 0.6600985221674877,
+      "grad_norm": 0.2977202832698822,
+      "learning_rate": 4.114811102678523e-05,
+      "loss": 1.2966550827026366,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6625615763546798,
+      "grad_norm": 0.23678140342235565,
+      "learning_rate": 4.061528507385995e-05,
+      "loss": 1.2590033531188964,
+      "step": 1345
+    },
+    {
+      "epoch": 0.6650246305418719,
+      "grad_norm": 0.27373895049095154,
+      "learning_rate": 4.0084647444043485e-05,
+      "loss": 1.2824597358703613,
+      "step": 1350
+    },
+    {
+      "epoch": 0.6674876847290641,
+      "grad_norm": 0.22791257500648499,
+      "learning_rate": 3.955623190834697e-05,
+      "loss": 1.1797836303710938,
+      "step": 1355
+    },
+    {
+      "epoch": 0.6699507389162561,
+      "grad_norm": 0.3305644094944,
+      "learning_rate": 3.903007209636215e-05,
+      "loss": 1.2767440795898437,
+      "step": 1360
+    },
+    {
+      "epoch": 0.6724137931034483,
+      "grad_norm": 0.24109555780887604,
+      "learning_rate": 3.850620149412135e-05,
+      "loss": 1.3337753295898438,
+      "step": 1365
+    },
+    {
+      "epoch": 0.6748768472906403,
+      "grad_norm": 0.2364107072353363,
+      "learning_rate": 3.798465344196631e-05,
+      "loss": 1.2710576057434082,
+      "step": 1370
+    },
+    {
+      "epoch": 0.6773399014778325,
+      "grad_norm": 0.271882027387619,
+      "learning_rate": 3.7465461132426245e-05,
+      "loss": 1.286097240447998,
+      "step": 1375
+    },
+    {
+      "epoch": 0.6798029556650246,
+      "grad_norm": 0.2673625349998474,
+      "learning_rate": 3.694865760810542e-05,
+      "loss": 1.259279155731201,
+      "step": 1380
+    },
+    {
+      "epoch": 0.6822660098522167,
+      "grad_norm": 0.262508749961853,
+      "learning_rate": 3.643427575958031e-05,
+      "loss": 1.2561509132385253,
+      "step": 1385
+    },
+    {
+      "epoch": 0.6847290640394089,
+      "grad_norm": 0.20407506823539734,
+      "learning_rate": 3.592234832330628e-05,
+      "loss": 1.2327983856201172,
+      "step": 1390
+    },
+    {
+      "epoch": 0.687192118226601,
+      "grad_norm": 0.2393566519021988,
+      "learning_rate": 3.54129078795342e-05,
+      "loss": 1.2643770217895507,
+      "step": 1395
+    },
+    {
+      "epoch": 0.6896551724137931,
+      "grad_norm": 0.2600564658641815,
+      "learning_rate": 3.490598685023701e-05,
+      "loss": 1.339667510986328,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6921182266009852,
+      "grad_norm": 0.23139071464538574,
+      "learning_rate": 3.440161749704623e-05,
+      "loss": 1.2579167366027832,
+      "step": 1405
+    },
+    {
+      "epoch": 0.6945812807881774,
+      "grad_norm": 0.24983708560466766,
+      "learning_rate": 3.389983191919872e-05,
+      "loss": 1.3015851974487305,
+      "step": 1410
+    },
+    {
+      "epoch": 0.6970443349753694,
+      "grad_norm": 0.21438054740428925,
+      "learning_rate": 3.340066205149401e-05,
+      "loss": 1.2945125579833985,
+      "step": 1415
+    },
+    {
+      "epoch": 0.6995073891625616,
+      "grad_norm": 0.2539718449115753,
+      "learning_rate": 3.2904139662261664e-05,
+      "loss": 1.2290349006652832,
+      "step": 1420
+    },
+    {
+      "epoch": 0.7019704433497537,
+      "grad_norm": 0.2936062216758728,
+      "learning_rate": 3.241029635133957e-05,
+      "loss": 1.2282495498657227,
+      "step": 1425
+    },
+    {
+      "epoch": 0.7044334975369458,
+      "grad_norm": 0.23874711990356445,
+      "learning_rate": 3.191916354806297e-05,
+      "loss": 1.2566828727722168,
+      "step": 1430
+    },
+    {
+      "epoch": 0.7068965517241379,
+      "grad_norm": 0.2245590090751648,
+      "learning_rate": 3.143077250926391e-05,
+      "loss": 1.2410771369934082,
+      "step": 1435
+    },
+    {
+      "epoch": 0.7093596059113301,
+      "grad_norm": 0.26783260703086853,
+      "learning_rate": 3.0945154317282297e-05,
+      "loss": 1.2264184951782227,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7118226600985221,
+      "grad_norm": 0.2439153492450714,
+      "learning_rate": 3.0462339877987666e-05,
+      "loss": 1.3033985137939452,
+      "step": 1445
+    },
+    {
+      "epoch": 0.7142857142857143,
+      "grad_norm": 0.25006458163261414,
+      "learning_rate": 2.9982359918812136e-05,
+      "loss": 1.2718771934509276,
+      "step": 1450
+    },
+    {
+      "epoch": 0.7167487684729064,
+      "grad_norm": 0.23204629123210907,
+      "learning_rate": 2.9505244986794946e-05,
+      "loss": 1.2503164291381836,
+      "step": 1455
+    },
+    {
+      "epoch": 0.7192118226600985,
+      "grad_norm": 0.26418495178222656,
+      "learning_rate": 2.9031025446638375e-05,
+      "loss": 1.2460610389709472,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7216748768472906,
+      "grad_norm": 0.22843700647354126,
+      "learning_rate": 2.8559731478775218e-05,
+      "loss": 1.2206365585327148,
+      "step": 1465
+    },
+    {
+      "epoch": 0.7241379310344828,
+      "grad_norm": 0.25052499771118164,
+      "learning_rate": 2.8091393077448082e-05,
+      "loss": 1.2625818252563477,
+      "step": 1470
+    },
+    {
+      "epoch": 0.7266009852216748,
+      "grad_norm": 0.23987461626529694,
+      "learning_rate": 2.7626040048800403e-05,
+      "loss": 1.2096717834472657,
+      "step": 1475
+    },
+    {
+      "epoch": 0.729064039408867,
+      "grad_norm": 0.21900923550128937,
+      "learning_rate": 2.716370200897969e-05,
+      "loss": 1.2399868011474608,
+      "step": 1480
+    },
+    {
+      "epoch": 0.7315270935960592,
+      "grad_norm": 0.27598026394844055,
+      "learning_rate": 2.6704408382252362e-05,
+      "loss": 1.2283302307128907,
+      "step": 1485
+    },
+    {
+      "epoch": 0.7339901477832512,
+      "grad_norm": 0.23923085629940033,
+      "learning_rate": 2.624818839913151e-05,
+      "loss": 1.341434860229492,
+      "step": 1490
+    },
+    {
+      "epoch": 0.7364532019704434,
+      "grad_norm": 0.23537442088127136,
+      "learning_rate": 2.579507109451632e-05,
+      "loss": 1.280338668823242,
+      "step": 1495
+    },
+    {
+      "epoch": 0.7389162561576355,
+      "grad_norm": 0.28500378131866455,
+      "learning_rate": 2.534508530584431e-05,
+      "loss": 1.305718994140625,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7413793103448276,
+      "grad_norm": 0.2542401850223541,
+      "learning_rate": 2.489825967125607e-05,
+      "loss": 1.1917360305786133,
+      "step": 1505
+    },
+    {
+      "epoch": 0.7438423645320197,
+      "grad_norm": 0.23925921320915222,
+      "learning_rate": 2.4454622627772636e-05,
+      "loss": 1.2214764595031737,
+      "step": 1510
+    },
+    {
+      "epoch": 0.7463054187192119,
+      "grad_norm": 0.2583833634853363,
+      "learning_rate": 2.401420240948571e-05,
+      "loss": 1.242533016204834,
+      "step": 1515
+    },
+    {
+      "epoch": 0.7487684729064039,
+      "grad_norm": 0.2212097942829132,
+      "learning_rate": 2.3577027045760727e-05,
+      "loss": 1.3234115600585938,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7512315270935961,
+      "grad_norm": 0.26380887627601624,
+      "learning_rate": 2.3143124359453137e-05,
+      "loss": 1.252053451538086,
+      "step": 1525
+    },
+    {
+      "epoch": 0.7536945812807881,
+      "grad_norm": 0.2283976823091507,
+      "learning_rate": 2.271252196513752e-05,
+      "loss": 1.249081802368164,
+      "step": 1530
+    },
+    {
+      "epoch": 0.7561576354679803,
+      "grad_norm": 0.2376411408185959,
+      "learning_rate": 2.2285247267350212e-05,
+      "loss": 1.2989398956298828,
+      "step": 1535
+    },
+    {
+      "epoch": 0.7586206896551724,
+      "grad_norm": 0.2347487211227417,
+      "learning_rate": 2.1861327458845253e-05,
+      "loss": 1.2198263168334962,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7610837438423645,
+      "grad_norm": 0.2686555087566376,
+      "learning_rate": 2.1440789518863706e-05,
+      "loss": 1.2571537017822265,
+      "step": 1545
+    },
+    {
+      "epoch": 0.7635467980295566,
+      "grad_norm": 0.24728627502918243,
+      "learning_rate": 2.1023660211416653e-05,
+      "loss": 1.2556602478027343,
+      "step": 1550
+    },
+    {
+      "epoch": 0.7660098522167488,
+      "grad_norm": 0.2575431168079376,
+      "learning_rate": 2.0609966083581987e-05,
+      "loss": 1.2640047073364258,
+      "step": 1555
+    },
+    {
+      "epoch": 0.7684729064039408,
+      "grad_norm": 0.21967598795890808,
+      "learning_rate": 2.019973346381464e-05,
+      "loss": 1.2949155807495116,
+      "step": 1560
+    },
+    {
+      "epoch": 0.770935960591133,
+      "grad_norm": 0.25700730085372925,
+      "learning_rate": 1.9792988460271214e-05,
+      "loss": 1.283220100402832,
+      "step": 1565
+    },
+    {
+      "epoch": 0.7733990147783252,
+      "grad_norm": 0.24390947818756104,
+      "learning_rate": 1.9389756959148357e-05,
+      "loss": 1.2551408767700196,
+      "step": 1570
+    },
+    {
+      "epoch": 0.7758620689655172,
+      "grad_norm": 0.27307766675949097,
+      "learning_rate": 1.8990064623035204e-05,
+      "loss": 1.2660499572753907,
+      "step": 1575
+    },
+    {
+      "epoch": 0.7783251231527094,
+      "grad_norm": 0.238468199968338,
+      "learning_rate": 1.85939368892802e-05,
+      "loss": 1.2559860229492188,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7807881773399015,
+      "grad_norm": 0.2351454347372055,
+      "learning_rate": 1.8201398968372256e-05,
+      "loss": 1.210586166381836,
+      "step": 1585
+    },
+    {
+      "epoch": 0.7832512315270936,
+      "grad_norm": 0.2293718159198761,
+      "learning_rate": 1.7812475842336242e-05,
+      "loss": 1.253939437866211,
+      "step": 1590
+    },
+    {
+      "epoch": 0.7857142857142857,
+      "grad_norm": 0.25299587845802307,
+      "learning_rate": 1.7427192263143056e-05,
+      "loss": 1.2384809494018554,
+      "step": 1595
+    },
+    {
+      "epoch": 0.7881773399014779,
+      "grad_norm": 0.19808216392993927,
+      "learning_rate": 1.7045572751134464e-05,
+      "loss": 1.2435653686523438,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7906403940886699,
+      "grad_norm": 0.24835017323493958,
+      "learning_rate": 1.6667641593462442e-05,
+      "loss": 1.2406715393066405,
+      "step": 1605
+    },
+    {
+      "epoch": 0.7931034482758621,
+      "grad_norm": 0.2713432312011719,
+      "learning_rate": 1.629342284254346e-05,
+      "loss": 1.2658330917358398,
+      "step": 1610
+    },
+    {
+      "epoch": 0.7955665024630542,
+      "grad_norm": 0.236631378531456,
+      "learning_rate": 1.5922940314527923e-05,
+      "loss": 1.239140510559082,
+      "step": 1615
+    },
+    {
+      "epoch": 0.7980295566502463,
+      "grad_norm": 0.2512910068035126,
+      "learning_rate": 1.5556217587784272e-05,
+      "loss": 1.2621894836425782,
+      "step": 1620
+    },
+    {
+      "epoch": 0.8004926108374384,
+      "grad_norm": 0.258465975522995,
+      "learning_rate": 1.5193278001398454e-05,
+      "loss": 1.2158693313598632,
+      "step": 1625
+    },
+    {
+      "epoch": 0.8029556650246306,
+      "grad_norm": 0.22228635847568512,
+      "learning_rate": 1.4834144653688601e-05,
+      "loss": 1.1974903106689454,
+      "step": 1630
+    },
+    {
+      "epoch": 0.8054187192118226,
+      "grad_norm": 0.23385341465473175,
+      "learning_rate": 1.4478840400734967e-05,
+      "loss": 1.1803974151611327,
+      "step": 1635
+    },
+    {
+      "epoch": 0.8078817733990148,
+      "grad_norm": 0.24148158729076385,
+      "learning_rate": 1.4127387854925315e-05,
+      "loss": 1.2340662002563476,
+      "step": 1640
+    },
+    {
+      "epoch": 0.8103448275862069,
+      "grad_norm": 0.26258596777915955,
+      "learning_rate": 1.3779809383515866e-05,
+      "loss": 1.2787314414978028,
+      "step": 1645
+    },
+    {
+      "epoch": 0.812807881773399,
+      "grad_norm": 0.23182962834835052,
+      "learning_rate": 1.3436127107207709e-05,
+      "loss": 1.2892789840698242,
+      "step": 1650
+    },
+    {
+      "epoch": 0.8152709359605911,
+      "grad_norm": 0.25138214230537415,
+      "learning_rate": 1.3096362898738988e-05,
+      "loss": 1.2018814086914062,
+      "step": 1655
+    },
+    {
+      "epoch": 0.8177339901477833,
+      "grad_norm": 0.2587434649467468,
+      "learning_rate": 1.2760538381493004e-05,
+      "loss": 1.2567611694335938,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8201970443349754,
+      "grad_norm": 0.24641695618629456,
+      "learning_rate": 1.2428674928121899e-05,
+      "loss": 1.2686907768249511,
+      "step": 1665
+    },
+    {
+      "epoch": 0.8226600985221675,
+      "grad_norm": 0.22543184459209442,
+      "learning_rate": 1.2100793659186527e-05,
+      "loss": 1.1918268203735352,
+      "step": 1670
+    },
+    {
+      "epoch": 0.8251231527093597,
+      "grad_norm": 0.23715642094612122,
+      "learning_rate": 1.177691544181227e-05,
+      "loss": 1.2081964492797852,
+      "step": 1675
+    },
+    {
+      "epoch": 0.8275862068965517,
+      "grad_norm": 0.20962002873420715,
+      "learning_rate": 1.1457060888361028e-05,
+      "loss": 1.2116300582885742,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8300492610837439,
+      "grad_norm": 0.21805016696453094,
+      "learning_rate": 1.1141250355119347e-05,
+      "loss": 1.2473871231079101,
+      "step": 1685
+    },
+    {
+      "epoch": 0.8325123152709359,
+      "grad_norm": 0.2535460293292999,
+      "learning_rate": 1.0829503941003001e-05,
+      "loss": 1.2010614395141601,
+      "step": 1690
+    },
+    {
+      "epoch": 0.8349753694581281,
+      "grad_norm": 0.2526600956916809,
+      "learning_rate": 1.0521841486277738e-05,
+      "loss": 1.2784499168395995,
+      "step": 1695
+    },
+    {
+      "epoch": 0.8374384236453202,
+      "grad_norm": 0.22173699736595154,
+      "learning_rate": 1.0218282571296637e-05,
+      "loss": 1.2362339973449707,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8399014778325123,
+      "grad_norm": 0.29041486978530884,
+      "learning_rate": 9.918846515254011e-06,
+      "loss": 1.2847709655761719,
+      "step": 1705
+    },
+    {
+      "epoch": 0.8423645320197044,
+      "grad_norm": 0.261730819940567,
+      "learning_rate": 9.623552374955842e-06,
+      "loss": 1.2652846336364747,
+      "step": 1710
+    },
+    {
+      "epoch": 0.8448275862068966,
+      "grad_norm": 0.30616867542266846,
+      "learning_rate": 9.332418943606965e-06,
+      "loss": 1.2111236572265625,
+      "step": 1715
+    },
+    {
+      "epoch": 0.8472906403940886,
+      "grad_norm": 0.22861535847187042,
+      "learning_rate": 9.045464749615027e-06,
+      "loss": 1.247592067718506,
+      "step": 1720
+    },
+    {
+      "epoch": 0.8497536945812808,
+      "grad_norm": 0.28312185406684875,
+      "learning_rate": 8.762708055411383e-06,
+      "loss": 1.231865406036377,
+      "step": 1725
+    },
+    {
+      "epoch": 0.8522167487684729,
+      "grad_norm": 0.23308123648166656,
+      "learning_rate": 8.484166856288635e-06,
+      "loss": 1.2434009552001952,
+      "step": 1730
+    },
+    {
+      "epoch": 0.854679802955665,
+      "grad_norm": 0.2353149652481079,
+      "learning_rate": 8.209858879255545e-06,
+      "loss": 1.2786510467529297,
+      "step": 1735
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 0.21151383221149445,
+      "learning_rate": 7.939801581908834e-06,
+      "loss": 1.2421545028686523,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8596059113300493,
+      "grad_norm": 0.2373751699924469,
+      "learning_rate": 7.67401215132202e-06,
+      "loss": 1.2191984176635742,
+      "step": 1745
+    },
+    {
+      "epoch": 0.8620689655172413,
+      "grad_norm": 0.21754534542560577,
+      "learning_rate": 7.412507502951717e-06,
+      "loss": 1.2775027275085449,
+      "step": 1750
+    },
+    {
+      "epoch": 0.8645320197044335,
+      "grad_norm": 0.25160351395606995,
+      "learning_rate": 7.155304279561033e-06,
+      "loss": 1.189585304260254,
+      "step": 1755
+    },
+    {
+      "epoch": 0.8669950738916257,
+      "grad_norm": 0.2518729567527771,
+      "learning_rate": 6.902418850160399e-06,
+      "loss": 1.1966890335083007,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8694581280788177,
+      "grad_norm": 0.21244123578071594,
+      "learning_rate": 6.653867308965799e-06,
+      "loss": 1.2111897468566895,
+      "step": 1765
+    },
+    {
+      "epoch": 0.8719211822660099,
+      "grad_norm": 0.2451592981815338,
+      "learning_rate": 6.4096654743745424e-06,
+      "loss": 1.2103561401367187,
+      "step": 1770
+    },
+    {
+      "epoch": 0.874384236453202,
+      "grad_norm": 0.27943119406700134,
+      "learning_rate": 6.1698288879585085e-06,
+      "loss": 1.2733654022216796,
+      "step": 1775
+    },
+    {
+      "epoch": 0.8768472906403941,
+      "grad_norm": 0.21996136009693146,
+      "learning_rate": 5.934372813474963e-06,
+      "loss": 1.2440560340881348,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8793103448275862,
+      "grad_norm": 0.2190336436033249,
+      "learning_rate": 5.70331223589533e-06,
+      "loss": 1.2366500854492188,
+      "step": 1785
+    },
+    {
+      "epoch": 0.8817733990147784,
+      "grad_norm": 0.22338411211967468,
+      "learning_rate": 5.4766618604513395e-06,
+      "loss": 1.2069709777832032,
+      "step": 1790
+    },
+    {
+      "epoch": 0.8842364532019704,
+      "grad_norm": 0.2502281963825226,
+      "learning_rate": 5.2544361116992175e-06,
+      "loss": 1.2337852478027345,
+      "step": 1795
+    },
+    {
+      "epoch": 0.8866995073891626,
+      "grad_norm": 0.2651473581790924,
+      "learning_rate": 5.036649132601722e-06,
+      "loss": 1.2154728889465332,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8891625615763546,
+      "grad_norm": 0.2430681437253952,
+      "learning_rate": 4.823314783627927e-06,
+      "loss": 1.2741308212280273,
+      "step": 1805
+    },
+    {
+      "epoch": 0.8916256157635468,
+      "grad_norm": 0.20495837926864624,
+      "learning_rate": 4.6144466418712305e-06,
+      "loss": 1.2369997024536132,
+      "step": 1810
+    },
+    {
+      "epoch": 0.8940886699507389,
+      "grad_norm": 0.25070708990097046,
+      "learning_rate": 4.410058000185243e-06,
+      "loss": 1.2604141235351562,
+      "step": 1815
+    },
+    {
+      "epoch": 0.896551724137931,
+      "grad_norm": 0.2330113798379898,
+      "learning_rate": 4.210161866337752e-06,
+      "loss": 1.2419015884399414,
+      "step": 1820
+    },
+    {
+      "epoch": 0.8990147783251231,
+      "grad_norm": 0.2585564851760864,
+      "learning_rate": 4.014770962182945e-06,
+      "loss": 1.2229095458984376,
+      "step": 1825
+    },
+    {
+      "epoch": 0.9014778325123153,
+      "grad_norm": 0.2556352913379669,
+      "learning_rate": 3.823897722851696e-06,
+      "loss": 1.2521157264709473,
+      "step": 1830
+    },
+    {
+      "epoch": 0.9039408866995073,
+      "grad_norm": 0.2235206812620163,
+      "learning_rate": 3.637554295960235e-06,
+      "loss": 1.2267001152038575,
+      "step": 1835
+    },
+    {
+      "epoch": 0.9064039408866995,
+      "grad_norm": 0.24141712486743927,
+      "learning_rate": 3.4557525408369645e-06,
+      "loss": 1.2922637939453125,
+      "step": 1840
+    },
+    {
+      "epoch": 0.9088669950738916,
+      "grad_norm": 0.24639971554279327,
+      "learning_rate": 3.278504027767814e-06,
+      "loss": 1.2198986053466796,
+      "step": 1845
+    },
+    {
+      "epoch": 0.9113300492610837,
+      "grad_norm": 0.30096888542175293,
+      "learning_rate": 3.1058200372597542e-06,
+      "loss": 1.260652732849121,
+      "step": 1850
+    },
+    {
+      "epoch": 0.9137931034482759,
+      "grad_norm": 0.19956085085868835,
+      "learning_rate": 2.937711559322953e-06,
+      "loss": 1.2672597885131835,
+      "step": 1855
+    },
+    {
+      "epoch": 0.916256157635468,
+      "grad_norm": 0.2193523645401001,
+      "learning_rate": 2.7741892927713617e-06,
+      "loss": 1.2428022384643556,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9187192118226601,
+      "grad_norm": 0.22981801629066467,
+      "learning_rate": 2.6152636445417513e-06,
+      "loss": 1.2323244094848633,
+      "step": 1865
+    },
+    {
+      "epoch": 0.9211822660098522,
+      "grad_norm": 0.20749390125274658,
+      "learning_rate": 2.46094472903145e-06,
+      "loss": 1.2612208366394042,
+      "step": 1870
+    },
+    {
+      "epoch": 0.9236453201970444,
+      "grad_norm": 0.2641940116882324,
+      "learning_rate": 2.3112423674546242e-06,
+      "loss": 1.167129898071289,
+      "step": 1875
+    },
+    {
+      "epoch": 0.9261083743842364,
+      "grad_norm": 0.24970325827598572,
+      "learning_rate": 2.166166087217197e-06,
+      "loss": 1.2519586563110352,
+      "step": 1880
+    },
+    {
+      "epoch": 0.9285714285714286,
+      "grad_norm": 0.250540167093277,
+      "learning_rate": 2.0257251213105645e-06,
+      "loss": 1.2368090629577637,
+      "step": 1885
+    },
+    {
+      "epoch": 0.9310344827586207,
+      "grad_norm": 0.25144103169441223,
+      "learning_rate": 1.8899284077239257e-06,
+      "loss": 1.2359929084777832,
+      "step": 1890
+    },
+    {
+      "epoch": 0.9334975369458128,
+      "grad_norm": 0.23775865137577057,
+      "learning_rate": 1.7587845888755032e-06,
+      "loss": 1.2116907119750977,
+      "step": 1895
+    },
+    {
+      "epoch": 0.9359605911330049,
+      "grad_norm": 0.3029141426086426,
+      "learning_rate": 1.6323020110624835e-06,
+      "loss": 1.2756942749023437,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9384236453201971,
+      "grad_norm": 0.23481281101703644,
+      "learning_rate": 1.5104887239298425e-06,
+      "loss": 1.274601364135742,
+      "step": 1905
+    },
+    {
+      "epoch": 0.9408866995073891,
+      "grad_norm": 0.2863881587982178,
+      "learning_rate": 1.3933524799580542e-06,
+      "loss": 1.2304944038391112,
+      "step": 1910
+    },
+    {
+      "epoch": 0.9433497536945813,
+      "grad_norm": 0.2509642541408539,
+      "learning_rate": 1.2809007339697114e-06,
+      "loss": 1.2000990867614747,
+      "step": 1915
+    },
+    {
+      "epoch": 0.9458128078817734,
+      "grad_norm": 0.24525979161262512,
+      "learning_rate": 1.1731406426550628e-06,
+      "loss": 1.242922878265381,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9482758620689655,
+      "grad_norm": 0.2376176118850708,
+      "learning_rate": 1.0700790641166023e-06,
+      "loss": 1.271295166015625,
+      "step": 1925
+    },
+    {
+      "epoch": 0.9507389162561576,
+      "grad_norm": 0.2372656762599945,
+      "learning_rate": 9.71722557432486e-07,
+      "loss": 1.243540382385254,
+      "step": 1930
+    },
+    {
+      "epoch": 0.9532019704433498,
+      "grad_norm": 0.23170247673988342,
+      "learning_rate": 8.780773822392229e-07,
+      "loss": 1.2256221771240234,
+      "step": 1935
+    },
+    {
+      "epoch": 0.9556650246305419,
+      "grad_norm": 0.23671060800552368,
+      "learning_rate": 7.891494983332525e-07,
+      "loss": 1.1968117713928224,
+      "step": 1940
+    },
+    {
+      "epoch": 0.958128078817734,
+      "grad_norm": 0.23950909078121185,
+      "learning_rate": 7.049445652915986e-07,
+      "loss": 1.2302292823791503,
+      "step": 1945
+    },
+    {
+      "epoch": 0.9605911330049262,
+      "grad_norm": 0.22995109856128693,
+      "learning_rate": 6.254679421117487e-07,
+      "loss": 1.2180012702941894,
+      "step": 1950
+    },
+    {
+      "epoch": 0.9630541871921182,
+      "grad_norm": 0.22918790578842163,
+      "learning_rate": 5.507246868705851e-07,
+      "loss": 1.2474028587341308,
+      "step": 1955
+    },
+    {
+      "epoch": 0.9655172413793104,
+      "grad_norm": 0.25687795877456665,
+      "learning_rate": 4.80719556402434e-07,
+      "loss": 1.2379139900207519,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9679802955665024,
+      "grad_norm": 0.22696813941001892,
+      "learning_rate": 4.154570059963752e-07,
+      "loss": 1.2781574249267578,
+      "step": 1965
+    },
+    {
+      "epoch": 0.9704433497536946,
+      "grad_norm": 0.2369939386844635,
+      "learning_rate": 3.5494118911266745e-07,
+      "loss": 1.277975082397461,
+      "step": 1970
+    },
+    {
+      "epoch": 0.9729064039408867,
+      "grad_norm": 0.21082457900047302,
+      "learning_rate": 2.99175957118436e-07,
+      "loss": 1.2537274360656738,
+      "step": 1975
+    },
+    {
+      "epoch": 0.9753694581280788,
+      "grad_norm": 0.24875393509864807,
+      "learning_rate": 2.4816485904255167e-07,
+      "loss": 1.2338659286499023,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9778325123152709,
+      "grad_norm": 0.21517492830753326,
+      "learning_rate": 2.0191114134975316e-07,
+      "loss": 1.2391672134399414,
+      "step": 1985
+    },
+    {
+      "epoch": 0.9802955665024631,
+      "grad_norm": 0.24385160207748413,
+      "learning_rate": 1.604177477340457e-07,
+      "loss": 1.2772870063781738,
+      "step": 1990
+    },
+    {
+      "epoch": 0.9827586206896551,
+      "grad_norm": 0.24616920948028564,
+      "learning_rate": 1.2368731893136752e-07,
+      "loss": 1.245918083190918,
+      "step": 1995
+    },
+    {
+      "epoch": 0.9852216748768473,
+      "grad_norm": 0.31482505798339844,
+      "learning_rate": 9.172219255149937e-08,
+      "loss": 1.2193574905395508,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9876847290640394,
+      "grad_norm": 0.20566366612911224,
+      "learning_rate": 6.452440292931682e-08,
+      "loss": 1.1942904472351075,
+      "step": 2005
+    },
+    {
+      "epoch": 0.9901477832512315,
+      "grad_norm": 0.3087645173072815,
+      "learning_rate": 4.2095680995310464e-08,
+      "loss": 1.190518569946289,
+      "step": 2010
+    },
+    {
+      "epoch": 0.9926108374384236,
+      "grad_norm": 0.27292242646217346,
+      "learning_rate": 2.443745416543241e-08,
+      "loss": 1.2792659759521485,
+      "step": 2015
+    },
+    {
+      "epoch": 0.9950738916256158,
+      "grad_norm": 0.21285434067249298,
+      "learning_rate": 1.1550846250252265e-08,
+      "loss": 1.2324320793151855,
+      "step": 2020
+    },
+    {
+      "epoch": 0.9975369458128078,
+      "grad_norm": 0.2532356381416321,
+      "learning_rate": 3.4366773834143547e-09,
+      "loss": 1.2427490234375,
+      "step": 2025
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.24805334210395813,
+      "learning_rate": 9.546396946280566e-11,
+      "loss": 1.2228047370910644,
+      "step": 2030
+    }
+  ],
+  "logging_steps": 5,
+  "max_steps": 2030,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.9519558621383885e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d8a957de182ebbfda687a47c9460145e0a0c0ea4888eb9f12cbd0081ea2b02b
+size 5905