Upload folder using huggingface_hub

checkpoint-2625/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-1.5B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-1.5B-Instruct
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.21.2

checkpoint-2625/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "o_proj",
+    "gate_proj",
+    "k_proj",
+    "down_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

checkpoint-2625/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:26f4fdbc97f8e891b0ef13bb3d4bc7b38685c7c8a6680c900d49e6dedcfb2b76
+size 73911112

checkpoint-2625/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

checkpoint-2625/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:459b3cd0a831fb7ac8f3458a4d04301e889b22700ff3f8356d67c66163175bdc
+size 148048187

checkpoint-2625/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80fc03bc750db5b4cbdbe0aa76fbee687a292adcdf016b1e64892b379a2e3155
+size 14645

checkpoint-2625/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92bc87758e953906f9944b2aa74413ce823681bd1ef256b86ca6a7aad5399b13
+size 1383

checkpoint-2625/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78320d3887317ad8bbad583146bf3b7a82cd7faa8f6eee912f58429e16305674
+size 1465

checkpoint-2625/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f55e63353d3d978b390d346bae531be8b83bc9532c0be500d62b7253aa4c595
+size 11421991

checkpoint-2625/tokenizer_config.json

@@ -0,0 +1,30 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-2625/trainer_state.json

@@ -0,0 +1,1868 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.0053627005027532,
+  "eval_steps": 500,
+  "global_step": 2625,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008119354511316351,
+      "grad_norm": 1.192345380783081,
+      "learning_rate": 2.7272727272727273e-05,
+      "loss": 1.3216888427734375,
+      "step": 10
+    },
+    {
+      "epoch": 0.016238709022632702,
+      "grad_norm": 1.2548257112503052,
+      "learning_rate": 5.757575757575758e-05,
+      "loss": 1.0566269874572753,
+      "step": 20
+    },
+    {
+      "epoch": 0.02435806353394905,
+      "grad_norm": 0.30806204676628113,
+      "learning_rate": 8.787878787878789e-05,
+      "loss": 0.6396953582763671,
+      "step": 30
+    },
+    {
+      "epoch": 0.032477418045265403,
+      "grad_norm": 0.25909167528152466,
+      "learning_rate": 0.0001181818181818182,
+      "loss": 0.5402174472808838,
+      "step": 40
+    },
+    {
+      "epoch": 0.04059677255658175,
+      "grad_norm": 0.21031738817691803,
+      "learning_rate": 0.00014848484848484849,
+      "loss": 0.5005859375,
+      "step": 50
+    },
+    {
+      "epoch": 0.0487161270678981,
+      "grad_norm": 0.18716108798980713,
+      "learning_rate": 0.0001787878787878788,
+      "loss": 0.537840461730957,
+      "step": 60
+    },
+    {
+      "epoch": 0.056835481579214454,
+      "grad_norm": 0.1792488396167755,
+      "learning_rate": 0.0001999990247348105,
+      "loss": 0.4450899600982666,
+      "step": 70
+    },
+    {
+      "epoch": 0.06495483609053081,
+      "grad_norm": 0.21492543816566467,
+      "learning_rate": 0.00019998168721618701,
+      "loss": 0.4202279090881348,
+      "step": 80
+    },
+    {
+      "epoch": 0.07307419060184715,
+      "grad_norm": 0.16661711037158966,
+      "learning_rate": 0.00019994268146276777,
+      "loss": 0.4485616207122803,
+      "step": 90
+    },
+    {
+      "epoch": 0.0811935451131635,
+      "grad_norm": 0.24056391417980194,
+      "learning_rate": 0.0001998820159279591,
+      "loss": 0.44878439903259276,
+      "step": 100
+    },
+    {
+      "epoch": 0.08931289962447986,
+      "grad_norm": 0.20612840354442596,
+      "learning_rate": 0.00019979970375931912,
+      "loss": 0.4974456787109375,
+      "step": 110
+    },
+    {
+      "epoch": 0.0974322541357962,
+      "grad_norm": 0.17296524345874786,
+      "learning_rate": 0.00019969576279570866,
+      "loss": 0.4231257438659668,
+      "step": 120
+    },
+    {
+      "epoch": 0.10555160864711255,
+      "grad_norm": 0.20717096328735352,
+      "learning_rate": 0.00019957021556342495,
+      "loss": 0.4275392532348633,
+      "step": 130
+    },
+    {
+      "epoch": 0.11367096315842891,
+      "grad_norm": 0.20023062825202942,
+      "learning_rate": 0.00019942308927131972,
+      "loss": 0.41733384132385254,
+      "step": 140
+    },
+    {
+      "epoch": 0.12179031766974525,
+      "grad_norm": 0.17242860794067383,
+      "learning_rate": 0.00019925441580490255,
+      "loss": 0.41766815185546874,
+      "step": 150
+    },
+    {
+      "epoch": 0.12990967218106161,
+      "grad_norm": 0.17718906700611115,
+      "learning_rate": 0.00019906423171943047,
+      "loss": 0.43944649696350097,
+      "step": 160
+    },
+    {
+      "epoch": 0.13802902669237796,
+      "grad_norm": 0.1699272096157074,
+      "learning_rate": 0.0001988525782319856,
+      "loss": 0.4246710777282715,
+      "step": 170
+    },
+    {
+      "epoch": 0.1461483812036943,
+      "grad_norm": 0.18154670298099518,
+      "learning_rate": 0.00019861950121254275,
+      "loss": 0.40392107963562013,
+      "step": 180
+    },
+    {
+      "epoch": 0.15426773571501065,
+      "grad_norm": 0.18973930180072784,
+      "learning_rate": 0.00019836505117402792,
+      "loss": 0.4155430316925049,
+      "step": 190
+    },
+    {
+      "epoch": 0.162387090226327,
+      "grad_norm": 0.15529297292232513,
+      "learning_rate": 0.00019808928326137156,
+      "loss": 0.430379581451416,
+      "step": 200
+    },
+    {
+      "epoch": 0.17050644473764337,
+      "grad_norm": 0.20729880034923553,
+      "learning_rate": 0.00019779225723955707,
+      "loss": 0.42681236267089845,
+      "step": 210
+    },
+    {
+      "epoch": 0.17862579924895972,
+      "grad_norm": 0.17007844150066376,
+      "learning_rate": 0.0001974740374806686,
+      "loss": 0.3926879644393921,
+      "step": 220
+    },
+    {
+      "epoch": 0.18674515376027606,
+      "grad_norm": 0.1643807291984558,
+      "learning_rate": 0.00019713469294994012,
+      "loss": 0.41318230628967284,
+      "step": 230
+    },
+    {
+      "epoch": 0.1948645082715924,
+      "grad_norm": 0.1504553258419037,
+      "learning_rate": 0.00019677429719080916,
+      "loss": 0.443330192565918,
+      "step": 240
+    },
+    {
+      "epoch": 0.20298386278290875,
+      "grad_norm": 0.17451228201389313,
+      "learning_rate": 0.00019639292830897837,
+      "loss": 0.4365452766418457,
+      "step": 250
+    },
+    {
+      "epoch": 0.2111032172942251,
+      "grad_norm": 0.16454367339611053,
+      "learning_rate": 0.0001959906689554881,
+      "loss": 0.45282588005065916,
+      "step": 260
+    },
+    {
+      "epoch": 0.21922257180554147,
+      "grad_norm": 0.15446598827838898,
+      "learning_rate": 0.0001955676063088045,
+      "loss": 0.46193952560424806,
+      "step": 270
+    },
+    {
+      "epoch": 0.22734192631685782,
+      "grad_norm": 0.17489995062351227,
+      "learning_rate": 0.00019512383205592552,
+      "loss": 0.46073436737060547,
+      "step": 280
+    },
+    {
+      "epoch": 0.23546128082817416,
+      "grad_norm": 0.1772591769695282,
+      "learning_rate": 0.00019465944237251071,
+      "loss": 0.38799960613250734,
+      "step": 290
+    },
+    {
+      "epoch": 0.2435806353394905,
+      "grad_norm": 0.11530005931854248,
+      "learning_rate": 0.00019417453790203772,
+      "loss": 0.4481969833374023,
+      "step": 300
+    },
+    {
+      "epoch": 0.2516999898508069,
+      "grad_norm": 0.1683167964220047,
+      "learning_rate": 0.0001936692237339905,
+      "loss": 0.41562438011169434,
+      "step": 310
+    },
+    {
+      "epoch": 0.25981934436212323,
+      "grad_norm": 0.16425232589244843,
+      "learning_rate": 0.00019314360938108425,
+      "loss": 0.40752553939819336,
+      "step": 320
+    },
+    {
+      "epoch": 0.2679386988734396,
+      "grad_norm": 0.16231301426887512,
+      "learning_rate": 0.0001925978087555316,
+      "loss": 0.3947896480560303,
+      "step": 330
+    },
+    {
+      "epoch": 0.2760580533847559,
+      "grad_norm": 0.17653369903564453,
+      "learning_rate": 0.00019203194014435525,
+      "loss": 0.4535045146942139,
+      "step": 340
+    },
+    {
+      "epoch": 0.28417740789607226,
+      "grad_norm": 0.15756867825984955,
+      "learning_rate": 0.0001914461261837526,
+      "loss": 0.4795684814453125,
+      "step": 350
+    },
+    {
+      "epoch": 0.2922967624073886,
+      "grad_norm": 0.1505475640296936,
+      "learning_rate": 0.00019084049383251783,
+      "loss": 0.4024923324584961,
+      "step": 360
+    },
+    {
+      "epoch": 0.30041611691870496,
+      "grad_norm": 0.16846773028373718,
+      "learning_rate": 0.00019021517434452702,
+      "loss": 0.42346930503845215,
+      "step": 370
+    },
+    {
+      "epoch": 0.3085354714300213,
+      "grad_norm": 0.17196664214134216,
+      "learning_rate": 0.0001895703032402926,
+      "loss": 0.38081505298614504,
+      "step": 380
+    },
+    {
+      "epoch": 0.31665482594133765,
+      "grad_norm": 0.18256904184818268,
+      "learning_rate": 0.00018890602027759313,
+      "loss": 0.37936599254608155,
+      "step": 390
+    },
+    {
+      "epoch": 0.324774180452654,
+      "grad_norm": 0.1385713666677475,
+      "learning_rate": 0.00018822246942118466,
+      "loss": 0.3991415977478027,
+      "step": 400
+    },
+    {
+      "epoch": 0.33289353496397034,
+      "grad_norm": 0.15789403021335602,
+      "learning_rate": 0.00018751979881160029,
+      "loss": 0.40332279205322263,
+      "step": 410
+    },
+    {
+      "epoch": 0.34101288947528674,
+      "grad_norm": 0.15686365962028503,
+      "learning_rate": 0.0001867981607330451,
+      "loss": 0.3878762722015381,
+      "step": 420
+    },
+    {
+      "epoch": 0.3491322439866031,
+      "grad_norm": 0.15789690613746643,
+      "learning_rate": 0.00018605771158039253,
+      "loss": 0.3754732131958008,
+      "step": 430
+    },
+    {
+      "epoch": 0.35725159849791943,
+      "grad_norm": 0.14583982527256012,
+      "learning_rate": 0.00018529861182529036,
+      "loss": 0.4119413375854492,
+      "step": 440
+    },
+    {
+      "epoch": 0.3653709530092358,
+      "grad_norm": 0.16271716356277466,
+      "learning_rate": 0.0001845210259813829,
+      "loss": 0.4371773719787598,
+      "step": 450
+    },
+    {
+      "epoch": 0.3734903075205521,
+      "grad_norm": 0.15252803266048431,
+      "learning_rate": 0.00018372512256865724,
+      "loss": 0.3985795021057129,
+      "step": 460
+    },
+    {
+      "epoch": 0.38160966203186847,
+      "grad_norm": 0.17694011330604553,
+      "learning_rate": 0.00018291107407692128,
+      "loss": 0.40482492446899415,
+      "step": 470
+    },
+    {
+      "epoch": 0.3897290165431848,
+      "grad_norm": 0.15677419304847717,
+      "learning_rate": 0.00018207905692842134,
+      "loss": 0.3925381898880005,
+      "step": 480
+    },
+    {
+      "epoch": 0.39784837105450116,
+      "grad_norm": 0.15828488767147064,
+      "learning_rate": 0.00018122925143960775,
+      "loss": 0.3869333028793335,
+      "step": 490
+    },
+    {
+      "epoch": 0.4059677255658175,
+      "grad_norm": 0.13660173118114471,
+      "learning_rate": 0.000180361841782056,
+      "loss": 0.4138071060180664,
+      "step": 500
+    },
+    {
+      "epoch": 0.41408708007713385,
+      "grad_norm": 0.164328932762146,
+      "learning_rate": 0.00017947701594255299,
+      "loss": 0.4766359806060791,
+      "step": 510
+    },
+    {
+      "epoch": 0.4222064345884502,
+      "grad_norm": 0.13619564473628998,
+      "learning_rate": 0.00017857496568235597,
+      "loss": 0.3740208625793457,
+      "step": 520
+    },
+    {
+      "epoch": 0.43032578909976654,
+      "grad_norm": 0.1389133632183075,
+      "learning_rate": 0.00017765588649563372,
+      "loss": 0.4194491386413574,
+      "step": 530
+    },
+    {
+      "epoch": 0.43844514361108294,
+      "grad_norm": 0.16105148196220398,
+      "learning_rate": 0.00017671997756709863,
+      "loss": 0.37844417095184324,
+      "step": 540
+    },
+    {
+      "epoch": 0.4465644981223993,
+      "grad_norm": 0.1176551878452301,
+      "learning_rate": 0.00017576744172883899,
+      "loss": 0.3884027719497681,
+      "step": 550
+    },
+    {
+      "epoch": 0.45468385263371563,
+      "grad_norm": 0.15062262117862701,
+      "learning_rate": 0.0001747984854163609,
+      "loss": 0.4128002643585205,
+      "step": 560
+    },
+    {
+      "epoch": 0.462803207145032,
+      "grad_norm": 0.14474593102931976,
+      "learning_rate": 0.00017381331862384894,
+      "loss": 0.37835040092468264,
+      "step": 570
+    },
+    {
+      "epoch": 0.4709225616563483,
+      "grad_norm": 0.11984218657016754,
+      "learning_rate": 0.00017281215485865603,
+      "loss": 0.3885404109954834,
+      "step": 580
+    },
+    {
+      "epoch": 0.47904191616766467,
+      "grad_norm": 0.11157386004924774,
+      "learning_rate": 0.00017179521109503146,
+      "loss": 0.41469721794128417,
+      "step": 590
+    },
+    {
+      "epoch": 0.487161270678981,
+      "grad_norm": 0.15861837565898895,
+      "learning_rate": 0.00017076270772709807,
+      "loss": 0.40916032791137696,
+      "step": 600
+    },
+    {
+      "epoch": 0.49528062519029736,
+      "grad_norm": 0.16919773817062378,
+      "learning_rate": 0.00016971486852108763,
+      "loss": 0.38907546997070314,
+      "step": 610
+    },
+    {
+      "epoch": 0.5033999797016138,
+      "grad_norm": 0.1534120887517929,
+      "learning_rate": 0.00016865192056684605,
+      "loss": 0.3835587978363037,
+      "step": 620
+    },
+    {
+      "epoch": 0.51151933421293,
+      "grad_norm": 0.1444127857685089,
+      "learning_rate": 0.00016757409422861802,
+      "loss": 0.41408615112304686,
+      "step": 630
+    },
+    {
+      "epoch": 0.5196386887242465,
+      "grad_norm": 0.16517969965934753,
+      "learning_rate": 0.00016648162309512173,
+      "loss": 0.3881347179412842,
+      "step": 640
+    },
+    {
+      "epoch": 0.5277580432355627,
+      "grad_norm": 0.16764643788337708,
+      "learning_rate": 0.00016537474392892528,
+      "loss": 0.36968770027160647,
+      "step": 650
+    },
+    {
+      "epoch": 0.5358773977468791,
+      "grad_norm": 0.15984776616096497,
+      "learning_rate": 0.0001642536966151349,
+      "loss": 0.4436800003051758,
+      "step": 660
+    },
+    {
+      "epoch": 0.5439967522581954,
+      "grad_norm": 0.15119516849517822,
+      "learning_rate": 0.00016311872410940676,
+      "loss": 0.36648738384246826,
+      "step": 670
+    },
+    {
+      "epoch": 0.5521161067695118,
+      "grad_norm": 0.17760701477527618,
+      "learning_rate": 0.00016197007238529277,
+      "loss": 0.43828520774841306,
+      "step": 680
+    },
+    {
+      "epoch": 0.5602354612808281,
+      "grad_norm": 0.15702351927757263,
+      "learning_rate": 0.00016080799038093311,
+      "loss": 0.39908370971679685,
+      "step": 690
+    },
+    {
+      "epoch": 0.5683548157921445,
+      "grad_norm": 0.16063162684440613,
+      "learning_rate": 0.0001596327299451054,
+      "loss": 0.3547415256500244,
+      "step": 700
+    },
+    {
+      "epoch": 0.5764741703034608,
+      "grad_norm": 0.15705077350139618,
+      "learning_rate": 0.000158444545782644,
+      "loss": 0.38064041137695315,
+      "step": 710
+    },
+    {
+      "epoch": 0.5845935248147772,
+      "grad_norm": 0.2802039086818695,
+      "learning_rate": 0.0001572436953992394,
+      "loss": 0.34356114864349363,
+      "step": 720
+    },
+    {
+      "epoch": 0.5927128793260936,
+      "grad_norm": 0.14802053570747375,
+      "learning_rate": 0.00015603043904563148,
+      "loss": 0.44878721237182617,
+      "step": 730
+    },
+    {
+      "epoch": 0.6008322338374099,
+      "grad_norm": 0.15852048993110657,
+      "learning_rate": 0.00015480503966120704,
+      "loss": 0.40053658485412597,
+      "step": 740
+    },
+    {
+      "epoch": 0.6089515883487263,
+      "grad_norm": 0.15426747500896454,
+      "learning_rate": 0.00015356776281701538,
+      "loss": 0.404225492477417,
+      "step": 750
+    },
+    {
+      "epoch": 0.6170709428600426,
+      "grad_norm": 0.1420115828514099,
+      "learning_rate": 0.000152318876658213,
+      "loss": 0.37903261184692383,
+      "step": 760
+    },
+    {
+      "epoch": 0.625190297371359,
+      "grad_norm": 0.13923387229442596,
+      "learning_rate": 0.00015105865184595073,
+      "loss": 0.35583546161651614,
+      "step": 770
+    },
+    {
+      "epoch": 0.6333096518826753,
+      "grad_norm": 0.14876466989517212,
+      "learning_rate": 0.00014978736149871545,
+      "loss": 0.3251772165298462,
+      "step": 780
+    },
+    {
+      "epoch": 0.6414290063939917,
+      "grad_norm": 0.17565123736858368,
+      "learning_rate": 0.0001485052811331393,
+      "loss": 0.39765985012054444,
+      "step": 790
+    },
+    {
+      "epoch": 0.649548360905308,
+      "grad_norm": 0.15443134307861328,
+      "learning_rate": 0.00014721268860428912,
+      "loss": 0.3529685020446777,
+      "step": 800
+    },
+    {
+      "epoch": 0.6576677154166244,
+      "grad_norm": 0.15266218781471252,
+      "learning_rate": 0.00014590986404544923,
+      "loss": 0.4115382194519043,
+      "step": 810
+    },
+    {
+      "epoch": 0.6657870699279407,
+      "grad_norm": 0.14183001220226288,
+      "learning_rate": 0.00014459708980741037,
+      "loss": 0.4038706302642822,
+      "step": 820
+    },
+    {
+      "epoch": 0.6739064244392571,
+      "grad_norm": 0.20107750594615936,
+      "learning_rate": 0.00014327465039727785,
+      "loss": 0.42904024124145507,
+      "step": 830
+    },
+    {
+      "epoch": 0.6820257789505735,
+      "grad_norm": 0.15056397020816803,
+      "learning_rate": 0.00014194283241681293,
+      "loss": 0.3807892084121704,
+      "step": 840
+    },
+    {
+      "epoch": 0.6901451334618898,
+      "grad_norm": 0.16432157158851624,
+      "learning_rate": 0.00014060192450031958,
+      "loss": 0.40647153854370116,
+      "step": 850
+    },
+    {
+      "epoch": 0.6982644879732062,
+      "grad_norm": 0.14384903013706207,
+      "learning_rate": 0.0001392522172520913,
+      "loss": 0.400847864151001,
+      "step": 860
+    },
+    {
+      "epoch": 0.7063838424845225,
+      "grad_norm": 0.16257992386817932,
+      "learning_rate": 0.00013789400318343068,
+      "loss": 0.4034707069396973,
+      "step": 870
+    },
+    {
+      "epoch": 0.7145031969958389,
+      "grad_norm": 0.1456725299358368,
+      "learning_rate": 0.00013652757664925572,
+      "loss": 0.3958974123001099,
+      "step": 880
+    },
+    {
+      "epoch": 0.7226225515071552,
+      "grad_norm": 0.1645933985710144,
+      "learning_rate": 0.00013515323378430664,
+      "loss": 0.39889843463897706,
+      "step": 890
+    },
+    {
+      "epoch": 0.7307419060184716,
+      "grad_norm": 0.14808154106140137,
+      "learning_rate": 0.00013377127243896706,
+      "loss": 0.39709608554840087,
+      "step": 900
+    },
+    {
+      "epoch": 0.7388612605297878,
+      "grad_norm": 0.15682613849639893,
+      "learning_rate": 0.00013238199211471305,
+      "loss": 0.3838841199874878,
+      "step": 910
+    },
+    {
+      "epoch": 0.7469806150411042,
+      "grad_norm": 0.15188175439834595,
+      "learning_rate": 0.00013098569389920488,
+      "loss": 0.4285408973693848,
+      "step": 920
+    },
+    {
+      "epoch": 0.7550999695524205,
+      "grad_norm": 0.16449961066246033,
+      "learning_rate": 0.00012958268040103434,
+      "loss": 0.37526252269744875,
+      "step": 930
+    },
+    {
+      "epoch": 0.7632193240637369,
+      "grad_norm": 0.1517837643623352,
+      "learning_rate": 0.00012817325568414297,
+      "loss": 0.40034284591674807,
+      "step": 940
+    },
+    {
+      "epoch": 0.7713386785750533,
+      "grad_norm": 0.18229644000530243,
+      "learning_rate": 0.00012675772520192476,
+      "loss": 0.37904250621795654,
+      "step": 950
+    },
+    {
+      "epoch": 0.7794580330863696,
+      "grad_norm": 0.15189775824546814,
+      "learning_rate": 0.00012533639573102754,
+      "loss": 0.4246711254119873,
+      "step": 960
+    },
+    {
+      "epoch": 0.787577387597686,
+      "grad_norm": 0.15175221860408783,
+      "learning_rate": 0.0001239095753048678,
+      "loss": 0.3818449735641479,
+      "step": 970
+    },
+    {
+      "epoch": 0.7956967421090023,
+      "grad_norm": 0.14126037061214447,
+      "learning_rate": 0.00012247757314687297,
+      "loss": 0.3847095251083374,
+      "step": 980
+    },
+    {
+      "epoch": 0.8038160966203187,
+      "grad_norm": 0.17514625191688538,
+      "learning_rate": 0.00012104069960346596,
+      "loss": 0.40877785682678225,
+      "step": 990
+    },
+    {
+      "epoch": 0.811935451131635,
+      "grad_norm": 0.1650720089673996,
+      "learning_rate": 0.00011959926607680639,
+      "loss": 0.40711078643798826,
+      "step": 1000
+    },
+    {
+      "epoch": 0.8200548056429514,
+      "grad_norm": 0.15258049964904785,
+      "learning_rate": 0.00011815358495730253,
+      "loss": 0.40592288970947266,
+      "step": 1010
+    },
+    {
+      "epoch": 0.8281741601542677,
+      "grad_norm": 0.14057976007461548,
+      "learning_rate": 0.00011670396955590974,
+      "loss": 0.3216592073440552,
+      "step": 1020
+    },
+    {
+      "epoch": 0.8362935146655841,
+      "grad_norm": 0.16660679876804352,
+      "learning_rate": 0.00011525073403622894,
+      "loss": 0.40019774436950684,
+      "step": 1030
+    },
+    {
+      "epoch": 0.8444128691769004,
+      "grad_norm": 0.17680172622203827,
+      "learning_rate": 0.0001137941933464204,
+      "loss": 0.4132580280303955,
+      "step": 1040
+    },
+    {
+      "epoch": 0.8525322236882168,
+      "grad_norm": 0.17041870951652527,
+      "learning_rate": 0.0001123346631509477,
+      "loss": 0.3415886878967285,
+      "step": 1050
+    },
+    {
+      "epoch": 0.8606515781995331,
+      "grad_norm": 0.15096868574619293,
+      "learning_rate": 0.00011087245976216607,
+      "loss": 0.4133938789367676,
+      "step": 1060
+    },
+    {
+      "epoch": 0.8687709327108495,
+      "grad_norm": 0.15110765397548676,
+      "learning_rate": 0.00010940790007177069,
+      "loss": 0.38502039909362795,
+      "step": 1070
+    },
+    {
+      "epoch": 0.8768902872221659,
+      "grad_norm": 0.1299409419298172,
+      "learning_rate": 0.00010794130148211918,
+      "loss": 0.36754612922668456,
+      "step": 1080
+    },
+    {
+      "epoch": 0.8850096417334822,
+      "grad_norm": 0.16098879277706146,
+      "learning_rate": 0.00010647298183744359,
+      "loss": 0.4300854206085205,
+      "step": 1090
+    },
+    {
+      "epoch": 0.8931289962447986,
+      "grad_norm": 0.18567131459712982,
+      "learning_rate": 0.00010500325935496654,
+      "loss": 0.4555531978607178,
+      "step": 1100
+    },
+    {
+      "epoch": 0.9012483507561149,
+      "grad_norm": 0.15725167095661163,
+      "learning_rate": 0.00010353245255593645,
+      "loss": 0.40347743034362793,
+      "step": 1110
+    },
+    {
+      "epoch": 0.9093677052674313,
+      "grad_norm": 0.18014128506183624,
+      "learning_rate": 0.0001020608801965971,
+      "loss": 0.43356914520263673,
+      "step": 1120
+    },
+    {
+      "epoch": 0.9174870597787476,
+      "grad_norm": 0.14646700024604797,
+      "learning_rate": 0.00010058886119910603,
+      "loss": 0.3579983949661255,
+      "step": 1130
+    },
+    {
+      "epoch": 0.925606414290064,
+      "grad_norm": 0.15054835379123688,
+      "learning_rate": 9.911671458241707e-05,
+      "loss": 0.412873363494873,
+      "step": 1140
+    },
+    {
+      "epoch": 0.9337257688013803,
+      "grad_norm": 0.1782434582710266,
+      "learning_rate": 9.764475939314199e-05,
+      "loss": 0.3661553144454956,
+      "step": 1150
+    },
+    {
+      "epoch": 0.9418451233126967,
+      "grad_norm": 0.14244233071804047,
+      "learning_rate": 9.6173314636406e-05,
+      "loss": 0.33595120906829834,
+      "step": 1160
+    },
+    {
+      "epoch": 0.9499644778240129,
+      "grad_norm": 0.15346868336200714,
+      "learning_rate": 9.470269920671242e-05,
+      "loss": 0.35301713943481444,
+      "step": 1170
+    },
+    {
+      "epoch": 0.9580838323353293,
+      "grad_norm": 0.17095577716827393,
+      "learning_rate": 9.323323181883093e-05,
+      "loss": 0.46831064224243163,
+      "step": 1180
+    },
+    {
+      "epoch": 0.9662031868466457,
+      "grad_norm": 0.16479066014289856,
+      "learning_rate": 9.176523093872545e-05,
+      "loss": 0.35177159309387207,
+      "step": 1190
+    },
+    {
+      "epoch": 0.974322541357962,
+      "grad_norm": 0.15938977897167206,
+      "learning_rate": 9.02990147145352e-05,
+      "loss": 0.40160298347473145,
+      "step": 1200
+    },
+    {
+      "epoch": 0.9824418958692784,
+      "grad_norm": 0.15273737907409668,
+      "learning_rate": 8.883490090762524e-05,
+      "loss": 0.3344574451446533,
+      "step": 1210
+    },
+    {
+      "epoch": 0.9905612503805947,
+      "grad_norm": 0.17632175981998444,
+      "learning_rate": 8.737320682372048e-05,
+      "loss": 0.3946251392364502,
+      "step": 1220
+    },
+    {
+      "epoch": 0.9986806048919111,
+      "grad_norm": 0.1546667516231537,
+      "learning_rate": 8.59142492441384e-05,
+      "loss": 0.38449108600616455,
+      "step": 1230
+    },
+    {
+      "epoch": 1.0064954836090532,
+      "grad_norm": 0.1582242101430893,
+      "learning_rate": 8.445834435713589e-05,
+      "loss": 0.35192313194274905,
+      "step": 1240
+    },
+    {
+      "epoch": 1.0146148381203695,
+      "grad_norm": 0.16060605645179749,
+      "learning_rate": 8.300580768938389e-05,
+      "loss": 0.3984487295150757,
+      "step": 1250
+    },
+    {
+      "epoch": 1.0227341926316857,
+      "grad_norm": 0.15913701057434082,
+      "learning_rate": 8.155695403758614e-05,
+      "loss": 0.3666527271270752,
+      "step": 1260
+    },
+    {
+      "epoch": 1.030853547143002,
+      "grad_norm": 0.2222735434770584,
+      "learning_rate": 8.011209740025568e-05,
+      "loss": 0.39426436424255373,
+      "step": 1270
+    },
+    {
+      "epoch": 1.0389729016543185,
+      "grad_norm": 0.14628788828849792,
+      "learning_rate": 7.867155090966429e-05,
+      "loss": 0.3448423147201538,
+      "step": 1280
+    },
+    {
+      "epoch": 1.0470922561656348,
+      "grad_norm": 0.1674235761165619,
+      "learning_rate": 7.723562676398017e-05,
+      "loss": 0.3735038280487061,
+      "step": 1290
+    },
+    {
+      "epoch": 1.0552116106769511,
+      "grad_norm": 0.1577482521533966,
+      "learning_rate": 7.580463615960731e-05,
+      "loss": 0.36143453121185304,
+      "step": 1300
+    },
+    {
+      "epoch": 1.0633309651882676,
+      "grad_norm": 0.15055105090141296,
+      "learning_rate": 7.437888922374276e-05,
+      "loss": 0.3465540647506714,
+      "step": 1310
+    },
+    {
+      "epoch": 1.071450319699584,
+      "grad_norm": 0.16190367937088013,
+      "learning_rate": 7.2958694947165e-05,
+      "loss": 0.3162075519561768,
+      "step": 1320
+    },
+    {
+      "epoch": 1.0795696742109002,
+      "grad_norm": 0.13373522460460663,
+      "learning_rate": 7.154436111726885e-05,
+      "loss": 0.2877171516418457,
+      "step": 1330
+    },
+    {
+      "epoch": 1.0876890287222165,
+      "grad_norm": 0.1537173092365265,
+      "learning_rate": 7.013619425136125e-05,
+      "loss": 0.371155834197998,
+      "step": 1340
+    },
+    {
+      "epoch": 1.095808383233533,
+      "grad_norm": 0.153018981218338,
+      "learning_rate": 6.873449953023205e-05,
+      "loss": 0.3403003454208374,
+      "step": 1350
+    },
+    {
+      "epoch": 1.1039277377448493,
+      "grad_norm": 0.14698824286460876,
+      "learning_rate": 6.733958073201484e-05,
+      "loss": 0.3563337802886963,
+      "step": 1360
+    },
+    {
+      "epoch": 1.1120470922561656,
+      "grad_norm": 0.17373280227184296,
+      "learning_rate": 6.595174016635132e-05,
+      "loss": 0.36922128200531007,
+      "step": 1370
+    },
+    {
+      "epoch": 1.120166446767482,
+      "grad_norm": 0.1971225142478943,
+      "learning_rate": 6.457127860887424e-05,
+      "loss": 0.40238251686096194,
+      "step": 1380
+    },
+    {
+      "epoch": 1.1282858012787984,
+      "grad_norm": 0.1588146984577179,
+      "learning_rate": 6.319849523602271e-05,
+      "loss": 0.38647644519805907,
+      "step": 1390
+    },
+    {
+      "epoch": 1.1364051557901147,
+      "grad_norm": 0.1998201310634613,
+      "learning_rate": 6.18336875602039e-05,
+      "loss": 0.3392190933227539,
+      "step": 1400
+    },
+    {
+      "epoch": 1.144524510301431,
+      "grad_norm": 0.1279604434967041,
+      "learning_rate": 6.047715136531587e-05,
+      "loss": 0.29073169231414797,
+      "step": 1410
+    },
+    {
+      "epoch": 1.1526438648127475,
+      "grad_norm": 0.17609012126922607,
+      "learning_rate": 5.9129180642644414e-05,
+      "loss": 0.3901878118515015,
+      "step": 1420
+    },
+    {
+      "epoch": 1.1607632193240638,
+      "grad_norm": 0.1663195639848709,
+      "learning_rate": 5.779006752714868e-05,
+      "loss": 0.3095200300216675,
+      "step": 1430
+    },
+    {
+      "epoch": 1.16888257383538,
+      "grad_norm": 0.14723604917526245,
+      "learning_rate": 5.646010223414924e-05,
+      "loss": 0.3879781007766724,
+      "step": 1440
+    },
+    {
+      "epoch": 1.1770019283466964,
+      "grad_norm": 0.19692786037921906,
+      "learning_rate": 5.513957299643194e-05,
+      "loss": 0.3436476469039917,
+      "step": 1450
+    },
+    {
+      "epoch": 1.1851212828580129,
+      "grad_norm": 0.14053191244602203,
+      "learning_rate": 5.382876600178155e-05,
+      "loss": 0.32699582576751707,
+      "step": 1460
+    },
+    {
+      "epoch": 1.1932406373693292,
+      "grad_norm": 0.18614161014556885,
+      "learning_rate": 5.25279653309584e-05,
+      "loss": 0.3756012201309204,
+      "step": 1470
+    },
+    {
+      "epoch": 1.2013599918806455,
+      "grad_norm": 0.17593547701835632,
+      "learning_rate": 5.1237452896132067e-05,
+      "loss": 0.4036771297454834,
+      "step": 1480
+    },
+    {
+      "epoch": 1.2094793463919618,
+      "grad_norm": 0.1469552218914032,
+      "learning_rate": 4.9957508379784704e-05,
+      "loss": 0.35532774925231936,
+      "step": 1490
+    },
+    {
+      "epoch": 1.2175987009032783,
+      "grad_norm": 0.1465003490447998,
+      "learning_rate": 4.86884091740975e-05,
+      "loss": 0.3272955656051636,
+      "step": 1500
+    },
+    {
+      "epoch": 1.2257180554145946,
+      "grad_norm": 0.16992369294166565,
+      "learning_rate": 4.7430430320834e-05,
+      "loss": 0.3861941576004028,
+      "step": 1510
+    },
+    {
+      "epoch": 1.2338374099259108,
+      "grad_norm": 0.19381046295166016,
+      "learning_rate": 4.618384445173228e-05,
+      "loss": 0.36703226566314695,
+      "step": 1520
+    },
+    {
+      "epoch": 1.2419567644372274,
+      "grad_norm": 0.1563887596130371,
+      "learning_rate": 4.494892172941965e-05,
+      "loss": 0.40213809013366697,
+      "step": 1530
+    },
+    {
+      "epoch": 1.2500761189485436,
+      "grad_norm": 0.145489901304245,
+      "learning_rate": 4.372592978886254e-05,
+      "loss": 0.344010591506958,
+      "step": 1540
+    },
+    {
+      "epoch": 1.25819547345986,
+      "grad_norm": 0.16564379632472992,
+      "learning_rate": 4.2515133679364116e-05,
+      "loss": 0.3600982904434204,
+      "step": 1550
+    },
+    {
+      "epoch": 1.2663148279711762,
+      "grad_norm": 0.12342356890439987,
+      "learning_rate": 4.13167958071221e-05,
+      "loss": 0.32795798778533936,
+      "step": 1560
+    },
+    {
+      "epoch": 1.2744341824824925,
+      "grad_norm": 0.18176662921905518,
+      "learning_rate": 4.013117587835978e-05,
+      "loss": 0.36577463150024414,
+      "step": 1570
+    },
+    {
+      "epoch": 1.282553536993809,
+      "grad_norm": 0.1916268765926361,
+      "learning_rate": 3.895853084304176e-05,
+      "loss": 0.39234256744384766,
+      "step": 1580
+    },
+    {
+      "epoch": 1.2906728915051253,
+      "grad_norm": 0.1897222399711609,
+      "learning_rate": 3.779911483918739e-05,
+      "loss": 0.41258983612060546,
+      "step": 1590
+    },
+    {
+      "epoch": 1.2987922460164416,
+      "grad_norm": 0.15678349137306213,
+      "learning_rate": 3.665317913779337e-05,
+      "loss": 0.3665084600448608,
+      "step": 1600
+    },
+    {
+      "epoch": 1.3069116005277581,
+      "grad_norm": 0.16308161616325378,
+      "learning_rate": 3.552097208837775e-05,
+      "loss": 0.3931357622146606,
+      "step": 1610
+    },
+    {
+      "epoch": 1.3150309550390744,
+      "grad_norm": 0.1778445839881897,
+      "learning_rate": 3.4402739065157266e-05,
+      "loss": 0.3759305477142334,
+      "step": 1620
+    },
+    {
+      "epoch": 1.3231503095503907,
+      "grad_norm": 0.1780257523059845,
+      "learning_rate": 3.3298722413869155e-05,
+      "loss": 0.3472525358200073,
+      "step": 1630
+    },
+    {
+      "epoch": 1.3312696640617072,
+      "grad_norm": 0.14955449104309082,
+      "learning_rate": 3.2209161399249674e-05,
+      "loss": 0.34793896675109864,
+      "step": 1640
+    },
+    {
+      "epoch": 1.3393890185730235,
+      "grad_norm": 0.20207709074020386,
+      "learning_rate": 3.1134292153180056e-05,
+      "loss": 0.3454656362533569,
+      "step": 1650
+    },
+    {
+      "epoch": 1.3475083730843398,
+      "grad_norm": 0.18875418603420258,
+      "learning_rate": 3.007434762351147e-05,
+      "loss": 0.3456529140472412,
+      "step": 1660
+    },
+    {
+      "epoch": 1.355627727595656,
+      "grad_norm": 0.18712455034255981,
+      "learning_rate": 2.9029557523580253e-05,
+      "loss": 0.33401539325714114,
+      "step": 1670
+    },
+    {
+      "epoch": 1.3637470821069724,
+      "grad_norm": 0.16952762007713318,
+      "learning_rate": 2.800014828242381e-05,
+      "loss": 0.3649503231048584,
+      "step": 1680
+    },
+    {
+      "epoch": 1.3718664366182889,
+      "grad_norm": 0.19082432985305786,
+      "learning_rate": 2.698634299570857e-05,
+      "loss": 0.3909872770309448,
+      "step": 1690
+    },
+    {
+      "epoch": 1.3799857911296052,
+      "grad_norm": 0.16776010394096375,
+      "learning_rate": 2.5988361377380188e-05,
+      "loss": 0.3379994869232178,
+      "step": 1700
+    },
+    {
+      "epoch": 1.3881051456409215,
+      "grad_norm": 0.19785159826278687,
+      "learning_rate": 2.5006419712046636e-05,
+      "loss": 0.3403695821762085,
+      "step": 1710
+    },
+    {
+      "epoch": 1.396224500152238,
+      "grad_norm": 0.12651121616363525,
+      "learning_rate": 2.4040730808104706e-05,
+      "loss": 0.37059574127197265,
+      "step": 1720
+    },
+    {
+      "epoch": 1.4043438546635543,
+      "grad_norm": 0.18700380623340607,
+      "learning_rate": 2.3091503951619543e-05,
+      "loss": 0.4034583568572998,
+      "step": 1730
+    },
+    {
+      "epoch": 1.4124632091748706,
+      "grad_norm": 0.1756463646888733,
+      "learning_rate": 2.2158944860967946e-05,
+      "loss": 0.3308366060256958,
+      "step": 1740
+    },
+    {
+      "epoch": 1.420582563686187,
+      "grad_norm": 0.19831162691116333,
+      "learning_rate": 2.1243255642254578e-05,
+      "loss": 0.3691343069076538,
+      "step": 1750
+    },
+    {
+      "epoch": 1.4287019181975034,
+      "grad_norm": 0.1685105711221695,
+      "learning_rate": 2.0344634745511015e-05,
+      "loss": 0.3646685600280762,
+      "step": 1760
+    },
+    {
+      "epoch": 1.4368212727088197,
+      "grad_norm": 0.1513417512178421,
+      "learning_rate": 1.9463276921687458e-05,
+      "loss": 0.3358032464981079,
+      "step": 1770
+    },
+    {
+      "epoch": 1.444940627220136,
+      "grad_norm": 0.21079151332378387,
+      "learning_rate": 1.8599373180445688e-05,
+      "loss": 0.38408379554748534,
+      "step": 1780
+    },
+    {
+      "epoch": 1.4530599817314522,
+      "grad_norm": 0.1852787733078003,
+      "learning_rate": 1.775311074876329e-05,
+      "loss": 0.3572732925415039,
+      "step": 1790
+    },
+    {
+      "epoch": 1.4611793362427687,
+      "grad_norm": 0.18594121932983398,
+      "learning_rate": 1.6924673030357285e-05,
+      "loss": 0.34491288661956787,
+      "step": 1800
+    },
+    {
+      "epoch": 1.469298690754085,
+      "grad_norm": 0.16469211876392365,
+      "learning_rate": 1.6114239565936484e-05,
+      "loss": 0.3106041193008423,
+      "step": 1810
+    },
+    {
+      "epoch": 1.4774180452654013,
+      "grad_norm": 0.146048441529274,
+      "learning_rate": 1.532198599429122e-05,
+      "loss": 0.3674412965774536,
+      "step": 1820
+    },
+    {
+      "epoch": 1.4855373997767178,
+      "grad_norm": 0.18818271160125732,
+      "learning_rate": 1.454808401422837e-05,
+      "loss": 0.3938064336776733,
+      "step": 1830
+    },
+    {
+      "epoch": 1.4936567542880341,
+      "grad_norm": 0.20953674614429474,
+      "learning_rate": 1.379270134736057e-05,
+      "loss": 0.30992081165313723,
+      "step": 1840
+    },
+    {
+      "epoch": 1.5017761087993504,
+      "grad_norm": 0.21023158729076385,
+      "learning_rate": 1.3056001701757226e-05,
+      "loss": 0.3922957181930542,
+      "step": 1850
+    },
+    {
+      "epoch": 1.509895463310667,
+      "grad_norm": 0.18391859531402588,
+      "learning_rate": 1.233814473646524e-05,
+      "loss": 0.3312171459197998,
+      "step": 1860
+    },
+    {
+      "epoch": 1.518014817821983,
+      "grad_norm": 0.20905567705631256,
+      "learning_rate": 1.1639286026907581e-05,
+      "loss": 0.33843116760253905,
+      "step": 1870
+    },
+    {
+      "epoch": 1.5261341723332995,
+      "grad_norm": 0.2057563066482544,
+      "learning_rate": 1.0959577031166513e-05,
+      "loss": 0.32880287170410155,
+      "step": 1880
+    },
+    {
+      "epoch": 1.5342535268446158,
+      "grad_norm": 0.16004444658756256,
+      "learning_rate": 1.0299165057159332e-05,
+      "loss": 0.3609264850616455,
+      "step": 1890
+    },
+    {
+      "epoch": 1.542372881355932,
+      "grad_norm": 0.18283307552337646,
+      "learning_rate": 9.658193230713509e-06,
+      "loss": 0.36747682094573975,
+      "step": 1900
+    },
+    {
+      "epoch": 1.5504922358672486,
+      "grad_norm": 0.13987980782985687,
+      "learning_rate": 9.036800464548157e-06,
+      "loss": 0.3149809598922729,
+      "step": 1910
+    },
+    {
+      "epoch": 1.558611590378565,
+      "grad_norm": 0.20485147833824158,
+      "learning_rate": 8.435121428168613e-06,
+      "loss": 0.3814858436584473,
+      "step": 1920
+    },
+    {
+      "epoch": 1.5667309448898812,
+      "grad_norm": 0.19381378591060638,
+      "learning_rate": 7.853286518680503e-06,
+      "loss": 0.3389918565750122,
+      "step": 1930
+    },
+    {
+      "epoch": 1.5748502994011977,
+      "grad_norm": 0.1762656271457672,
+      "learning_rate": 7.291421832529921e-06,
+      "loss": 0.3758829355239868,
+      "step": 1940
+    },
+    {
+      "epoch": 1.582969653912514,
+      "grad_norm": 0.18566051125526428,
+      "learning_rate": 6.749649138175518e-06,
+      "loss": 0.361565375328064,
+      "step": 1950
+    },
+    {
+      "epoch": 1.5910890084238303,
+      "grad_norm": 0.1679965704679489,
+      "learning_rate": 6.228085849698495e-06,
+      "loss": 0.3467693567276001,
+      "step": 1960
+    },
+    {
+      "epoch": 1.5992083629351468,
+      "grad_norm": 0.19734881818294525,
+      "learning_rate": 5.726845001356573e-06,
+      "loss": 0.41715250015258787,
+      "step": 1970
+    },
+    {
+      "epoch": 1.6073277174464629,
+      "grad_norm": 0.20948296785354614,
+      "learning_rate": 5.246035223086821e-06,
+      "loss": 0.3086181879043579,
+      "step": 1980
+    },
+    {
+      "epoch": 1.6154470719577794,
+      "grad_norm": 0.2349746823310852,
+      "learning_rate": 4.785760716963205e-06,
+      "loss": 0.2819353580474854,
+      "step": 1990
+    },
+    {
+      "epoch": 1.6235664264690957,
+      "grad_norm": 0.15716052055358887,
+      "learning_rate": 4.346121234613732e-06,
+      "loss": 0.4032479763031006,
+      "step": 2000
+    },
+    {
+      "epoch": 1.631685780980412,
+      "grad_norm": 0.1870564967393875,
+      "learning_rate": 3.927212055602003e-06,
+      "loss": 0.3259021759033203,
+      "step": 2010
+    },
+    {
+      "epoch": 1.6398051354917285,
+      "grad_norm": 0.1917184740304947,
+      "learning_rate": 3.5291239667781253e-06,
+      "loss": 0.3619376182556152,
+      "step": 2020
+    },
+    {
+      "epoch": 1.6479244900030448,
+      "grad_norm": 0.15155203640460968,
+      "learning_rate": 3.1519432426031504e-06,
+      "loss": 0.3836283445358276,
+      "step": 2030
+    },
+    {
+      "epoch": 1.656043844514361,
+      "grad_norm": 0.18562212586402893,
+      "learning_rate": 2.7957516264514593e-06,
+      "loss": 0.35469727516174315,
+      "step": 2040
+    },
+    {
+      "epoch": 1.6641631990256776,
+      "grad_norm": 0.23471137881278992,
+      "learning_rate": 2.4606263128952646e-06,
+      "loss": 0.4092714309692383,
+      "step": 2050
+    },
+    {
+      "epoch": 1.6722825535369938,
+      "grad_norm": 0.19046807289123535,
+      "learning_rate": 2.1466399309747364e-06,
+      "loss": 0.37486720085144043,
+      "step": 2060
+    },
+    {
+      "epoch": 1.6804019080483101,
+      "grad_norm": 0.18093442916870117,
+      "learning_rate": 1.8538605284577848e-06,
+      "loss": 0.3519495248794556,
+      "step": 2070
+    },
+    {
+      "epoch": 1.6885212625596266,
+      "grad_norm": 0.17989413440227509,
+      "learning_rate": 1.5823515570925763e-06,
+      "loss": 0.36248462200164794,
+      "step": 2080
+    },
+    {
+      "epoch": 1.6966406170709427,
+      "grad_norm": 0.18000203371047974,
+      "learning_rate": 1.3321718588561128e-06,
+      "loss": 0.377717113494873,
+      "step": 2090
+    },
+    {
+      "epoch": 1.7047599715822592,
+      "grad_norm": 0.1857496201992035,
+      "learning_rate": 1.1033756532019102e-06,
+      "loss": 0.3373147964477539,
+      "step": 2100
+    },
+    {
+      "epoch": 1.7128793260935755,
+      "grad_norm": 0.15915660560131073,
+      "learning_rate": 8.96012525309442e-07,
+      "loss": 0.35358049869537356,
+      "step": 2110
+    },
+    {
+      "epoch": 1.7209986806048918,
+      "grad_norm": 0.18095794320106506,
+      "learning_rate": 7.101274153379244e-07,
+      "loss": 0.34168615341186526,
+      "step": 2120
+    },
+    {
+      "epoch": 1.7291180351162083,
+      "grad_norm": 0.13066989183425903,
+      "learning_rate": 5.457606086868406e-07,
+      "loss": 0.31731865406036375,
+      "step": 2130
+    },
+    {
+      "epoch": 1.7372373896275246,
+      "grad_norm": 0.15085873007774353,
+      "learning_rate": 4.0294772726511274e-07,
+      "loss": 0.3172720193862915,
+      "step": 2140
+    },
+    {
+      "epoch": 1.745356744138841,
+      "grad_norm": 0.1692853718996048,
+      "learning_rate": 2.817197217711565e-07,
+      "loss": 0.3663973808288574,
+      "step": 2150
+    },
+    {
+      "epoch": 1.7534760986501574,
+      "grad_norm": 0.1765965223312378,
+      "learning_rate": 1.8210286498512307e-07,
+      "loss": 0.3134049415588379,
+      "step": 2160
+    },
+    {
+      "epoch": 1.7615954531614737,
+      "grad_norm": 0.17754477262496948,
+      "learning_rate": 1.0411874607501038e-07,
+      "loss": 0.32787179946899414,
+      "step": 2170
+    },
+    {
+      "epoch": 1.76971480767279,
+      "grad_norm": 0.2181764841079712,
+      "learning_rate": 4.778426591781626e-08,
+      "loss": 0.40297679901123046,
+      "step": 2180
+    },
+    {
+      "epoch": 1.7778341621841065,
+      "grad_norm": 0.14845119416713715,
+      "learning_rate": 1.3111633436779791e-08,
+      "loss": 0.3253025054931641,
+      "step": 2190
+    },
+    {
+      "epoch": 1.7859535166954226,
+      "grad_norm": 0.16625212132930756,
+      "learning_rate": 1.083629553977339e-10,
+      "loss": 0.3306998014450073,
+      "step": 2200
+    },
+    {
+      "epoch": 0.8465405793631793,
+      "grad_norm": 0.31709548830986023,
+      "learning_rate": 0.0001457833924251086,
+      "loss": 0.6895709991455078,
+      "step": 2210
+    },
+    {
+      "epoch": 0.8503710797222888,
+      "grad_norm": 0.22089441120624542,
+      "learning_rate": 0.0001453028317991183,
+      "loss": 0.6232875347137451,
+      "step": 2220
+    },
+    {
+      "epoch": 0.8542015800813981,
+      "grad_norm": 0.22472961246967316,
+      "learning_rate": 0.00014482095116054422,
+      "loss": 0.580645751953125,
+      "step": 2230
+    },
+    {
+      "epoch": 0.8580320804405075,
+      "grad_norm": 0.18781282007694244,
+      "learning_rate": 0.0001443377645501975,
+      "loss": 0.5852691173553467,
+      "step": 2240
+    },
+    {
+      "epoch": 0.861862580799617,
+      "grad_norm": 0.3779642581939697,
+      "learning_rate": 0.00014385328604694212,
+      "loss": 0.5722718238830566,
+      "step": 2250
+    },
+    {
+      "epoch": 0.8656930811587263,
+      "grad_norm": 0.23427145183086395,
+      "learning_rate": 0.0001433675297672846,
+      "loss": 0.566037940979004,
+      "step": 2260
+    },
+    {
+      "epoch": 0.8695235815178358,
+      "grad_norm": 0.2183203548192978,
+      "learning_rate": 0.00014288050986496268,
+      "loss": 0.6243879795074463,
+      "step": 2270
+    },
+    {
+      "epoch": 0.8733540818769452,
+      "grad_norm": 0.2532879412174225,
+      "learning_rate": 0.00014239224053053302,
+      "loss": 0.5365942001342774,
+      "step": 2280
+    },
+    {
+      "epoch": 0.8771845822360546,
+      "grad_norm": 0.20788337290287018,
+      "learning_rate": 0.00014190273599095762,
+      "loss": 0.5467002868652344,
+      "step": 2290
+    },
+    {
+      "epoch": 0.881015082595164,
+      "grad_norm": 0.2582492232322693,
+      "learning_rate": 0.00014141201050918922,
+      "loss": 0.567399263381958,
+      "step": 2300
+    },
+    {
+      "epoch": 0.8848455829542734,
+      "grad_norm": 0.19094614684581757,
+      "learning_rate": 0.00014092007838375592,
+      "loss": 0.563731336593628,
+      "step": 2310
+    },
+    {
+      "epoch": 0.8886760833133828,
+      "grad_norm": 0.24398261308670044,
+      "learning_rate": 0.00014042695394834436,
+      "loss": 0.5930374145507813,
+      "step": 2320
+    },
+    {
+      "epoch": 0.8925065836724922,
+      "grad_norm": 0.1616758406162262,
+      "learning_rate": 0.0001399326515713822,
+      "loss": 0.5320866584777832,
+      "step": 2330
+    },
+    {
+      "epoch": 0.8963370840316016,
+      "grad_norm": 0.19463369250297546,
+      "learning_rate": 0.00013943718565561936,
+      "loss": 0.5593301296234131,
+      "step": 2340
+    },
+    {
+      "epoch": 0.900167584390711,
+      "grad_norm": 0.2512662410736084,
+      "learning_rate": 0.0001389405706377084,
+      "loss": 0.5519943237304688,
+      "step": 2350
+    },
+    {
+      "epoch": 0.9039980847498205,
+      "grad_norm": 0.24107597768306732,
+      "learning_rate": 0.00013844282098778393,
+      "loss": 0.5739433765411377,
+      "step": 2360
+    },
+    {
+      "epoch": 0.9078285851089298,
+      "grad_norm": 0.18342378735542297,
+      "learning_rate": 0.00013794395120904089,
+      "loss": 0.5859064102172852,
+      "step": 2370
+    },
+    {
+      "epoch": 0.9116590854680393,
+      "grad_norm": 0.20521777868270874,
+      "learning_rate": 0.00013744397583731203,
+      "loss": 0.5629020214080811,
+      "step": 2380
+    },
+    {
+      "epoch": 0.9154895858271487,
+      "grad_norm": 0.17162705957889557,
+      "learning_rate": 0.00013694290944064435,
+      "loss": 0.5632124900817871,
+      "step": 2390
+    },
+    {
+      "epoch": 0.919320086186258,
+      "grad_norm": 0.18732701241970062,
+      "learning_rate": 0.00013644076661887452,
+      "loss": 0.5588642120361328,
+      "step": 2400
+    },
+    {
+      "epoch": 0.9231505865453675,
+      "grad_norm": 0.21033287048339844,
+      "learning_rate": 0.00013593756200320372,
+      "loss": 0.5421002864837646,
+      "step": 2410
+    },
+    {
+      "epoch": 0.9269810869044769,
+      "grad_norm": 0.18077102303504944,
+      "learning_rate": 0.0001354333102557712,
+      "loss": 0.5636842727661133,
+      "step": 2420
+    },
+    {
+      "epoch": 0.9308115872635863,
+      "grad_norm": 0.19449454545974731,
+      "learning_rate": 0.0001349280260692269,
+      "loss": 0.5634388446807861,
+      "step": 2430
+    },
+    {
+      "epoch": 0.9346420876226957,
+      "grad_norm": 0.21519041061401367,
+      "learning_rate": 0.00013442172416630355,
+      "loss": 0.552568769454956,
+      "step": 2440
+    },
+    {
+      "epoch": 0.9384725879818051,
+      "grad_norm": 0.21668848395347595,
+      "learning_rate": 0.0001339144192993876,
+      "loss": 0.5575977325439453,
+      "step": 2450
+    },
+    {
+      "epoch": 0.9423030883409146,
+      "grad_norm": 0.20259518921375275,
+      "learning_rate": 0.00013340612625008946,
+      "loss": 0.5953729629516602,
+      "step": 2460
+    },
+    {
+      "epoch": 0.946133588700024,
+      "grad_norm": 0.22012926638126373,
+      "learning_rate": 0.0001328968598288127,
+      "loss": 0.5506711006164551,
+      "step": 2470
+    },
+    {
+      "epoch": 0.9499640890591333,
+      "grad_norm": 0.18549343943595886,
+      "learning_rate": 0.0001323866348743225,
+      "loss": 0.5871614456176758,
+      "step": 2480
+    },
+    {
+      "epoch": 0.9537945894182428,
+      "grad_norm": 0.1908208429813385,
+      "learning_rate": 0.00013187546625331335,
+      "loss": 0.5431451320648193,
+      "step": 2490
+    },
+    {
+      "epoch": 0.9576250897773522,
+      "grad_norm": 0.18781603872776031,
+      "learning_rate": 0.0001313633688599759,
+      "loss": 0.5061774730682373,
+      "step": 2500
+    },
+    {
+      "epoch": 0.9614555901364615,
+      "grad_norm": 0.2165912687778473,
+      "learning_rate": 0.0001308503576155629,
+      "loss": 0.5560242652893066,
+      "step": 2510
+    },
+    {
+      "epoch": 0.965286090495571,
+      "grad_norm": 0.21473945677280426,
+      "learning_rate": 0.00013033644746795445,
+      "loss": 0.5335087776184082,
+      "step": 2520
+    },
+    {
+      "epoch": 0.9691165908546804,
+      "grad_norm": 0.2197163999080658,
+      "learning_rate": 0.00012982165339122246,
+      "loss": 0.5561465263366699,
+      "step": 2530
+    },
+    {
+      "epoch": 0.9729470912137897,
+      "grad_norm": 0.21780231595039368,
+      "learning_rate": 0.00012930599038519434,
+      "loss": 0.5662136554718018,
+      "step": 2540
+    },
+    {
+      "epoch": 0.9767775915728992,
+      "grad_norm": 0.1910906732082367,
+      "learning_rate": 0.00012878947347501606,
+      "loss": 0.5577693462371827,
+      "step": 2550
+    },
+    {
+      "epoch": 0.9806080919320086,
+      "grad_norm": 0.18059995770454407,
+      "learning_rate": 0.0001282721177107141,
+      "loss": 0.531907606124878,
+      "step": 2560
+    },
+    {
+      "epoch": 0.9844385922911181,
+      "grad_norm": 0.24920019507408142,
+      "learning_rate": 0.00012775393816675718,
+      "loss": 0.497232723236084,
+      "step": 2570
+    },
+    {
+      "epoch": 0.9882690926502274,
+      "grad_norm": 0.2506183385848999,
+      "learning_rate": 0.00012723494994161682,
+      "loss": 0.5702355384826661,
+      "step": 2580
+    },
+    {
+      "epoch": 0.9920995930093368,
+      "grad_norm": 0.207411989569664,
+      "learning_rate": 0.00012671516815732767,
+      "loss": 0.5431533813476562,
+      "step": 2590
+    },
+    {
+      "epoch": 0.9959300933684463,
+      "grad_norm": 0.20407091081142426,
+      "learning_rate": 0.00012619460795904657,
+      "loss": 0.5356565952301026,
+      "step": 2600
+    },
+    {
+      "epoch": 0.9997605937275557,
+      "grad_norm": 0.19829393923282623,
+      "learning_rate": 0.00012567328451461152,
+      "loss": 0.5477075576782227,
+      "step": 2610
+    },
+    {
+      "epoch": 1.0034474503231985,
+      "grad_norm": 0.2388325333595276,
+      "learning_rate": 0.0001251512130140996,
+      "loss": 0.6198837280273437,
+      "step": 2620
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 6000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 25,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.1205131071934054e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2625/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2dd737ade882c002f79b2b055bb95bc81c6f63b8dc1ed4dd502ccc03fccd30d5
+size 5201