Training in progress, step 180, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,210 @@
+---
+base_model: unsloth/DeepSeek-R1-Distill-Qwen-7B-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:unsloth/DeepSeek-R1-Distill-Qwen-7B-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
+- unsloth
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

last-checkpoint/adapter_config.json

@@ -0,0 +1,54 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen2ForCausalLM",
+    "parent_library": "transformers.models.qwen2.modeling_qwen2",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/DeepSeek-R1-Distill-Qwen-7B-unsloth-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "gate_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true,
+  "velora_config": null
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ccd9abeb63391eca0121e50a51cd60308259aafb278d7c5cbc2949ba10257ecc
+size 323014168

last-checkpoint/chat_template.jinja

@@ -0,0 +1 @@
+{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜><think>\n'}}{% endif %}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a73baf3a1bbbb0d8ac3d1d5dc1324ecd5990edcf6ab08203c0b146cfc8a2fb0
+size 164468933

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3060f1bf3c410d14a817b592ce3da5ad026c244742ed6fdd2b031e948b2a8a8b
+size 14645

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89a7cc4fa41033239d8a7a9b269d2ff12d3089c956a327ba9cf348525189f10c
+size 1465

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5414707ff353678a7275d712e7ee62660749d40901bde9e12ec6870102fb9fc3
+size 11422877

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,195 @@
+{
+  "add_prefix_space": null,
+  "backend": "tokenizers",
+  "bos_token": "<｜begin▁of▁sentence｜>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<｜end▁of▁sentence｜>",
+  "from_slow": true,
+  "is_local": false,
+  "legacy": false,
+  "model_max_length": 131072,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": null,
+  "use_default_system_prompt": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<｜end▁of▁sentence｜>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<｜User｜>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151645": {
+      "content": "<｜Assistant｜>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151646": {
+      "content": "<｜begin▁of▁sentence｜>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|EOT|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151648": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151649": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    }
+  }
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1302 @@
+{
+  "best_global_step": 180,
+  "best_metric": 0.7345618605613708,
+  "best_model_checkpoint": "/tmp/checkpoints_/job_0e64aa6d-d770-420a-8596-581259830426/checkpoint-180",
+  "epoch": 1.8818897637795275,
+  "eval_steps": 180,
+  "global_step": 180,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.010498687664041995,
+      "grad_norm": 8.951988220214844,
+      "learning_rate": 0.0,
+      "loss": 3.249479055404663,
+      "step": 1
+    },
+    {
+      "epoch": 0.02099737532808399,
+      "grad_norm": 10.128303527832031,
+      "learning_rate": 1.388888888888889e-06,
+      "loss": 3.7097482681274414,
+      "step": 2
+    },
+    {
+      "epoch": 0.031496062992125984,
+      "grad_norm": 11.297329902648926,
+      "learning_rate": 2.777777777777778e-06,
+      "loss": 4.088885307312012,
+      "step": 3
+    },
+    {
+      "epoch": 0.04199475065616798,
+      "grad_norm": 10.444681167602539,
+      "learning_rate": 4.166666666666667e-06,
+      "loss": 3.485933303833008,
+      "step": 4
+    },
+    {
+      "epoch": 0.05249343832020997,
+      "grad_norm": 9.82193660736084,
+      "learning_rate": 5.555555555555556e-06,
+      "loss": 3.5621585845947266,
+      "step": 5
+    },
+    {
+      "epoch": 0.06299212598425197,
+      "grad_norm": 9.866586685180664,
+      "learning_rate": 6.944444444444445e-06,
+      "loss": 3.6951005458831787,
+      "step": 6
+    },
+    {
+      "epoch": 0.07349081364829396,
+      "grad_norm": 7.083434581756592,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 2.840730905532837,
+      "step": 7
+    },
+    {
+      "epoch": 0.08398950131233596,
+      "grad_norm": 6.84299373626709,
+      "learning_rate": 9.722222222222223e-06,
+      "loss": 2.666860342025757,
+      "step": 8
+    },
+    {
+      "epoch": 0.09448818897637795,
+      "grad_norm": 9.864928245544434,
+      "learning_rate": 1.1111111111111112e-05,
+      "loss": 3.023573875427246,
+      "step": 9
+    },
+    {
+      "epoch": 0.10498687664041995,
+      "grad_norm": 14.969345092773438,
+      "learning_rate": 1.25e-05,
+      "loss": 2.860168933868408,
+      "step": 10
+    },
+    {
+      "epoch": 0.11548556430446194,
+      "grad_norm": 11.013322830200195,
+      "learning_rate": 1.388888888888889e-05,
+      "loss": 2.6086080074310303,
+      "step": 11
+    },
+    {
+      "epoch": 0.12598425196850394,
+      "grad_norm": 4.521217346191406,
+      "learning_rate": 1.527777777777778e-05,
+      "loss": 2.150205135345459,
+      "step": 12
+    },
+    {
+      "epoch": 0.13648293963254593,
+      "grad_norm": 3.814413070678711,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 1.828367829322815,
+      "step": 13
+    },
+    {
+      "epoch": 0.14698162729658792,
+      "grad_norm": 4.641453266143799,
+      "learning_rate": 1.8055555555555555e-05,
+      "loss": 2.1309380531311035,
+      "step": 14
+    },
+    {
+      "epoch": 0.15748031496062992,
+      "grad_norm": 4.126307010650635,
+      "learning_rate": 1.9444444444444445e-05,
+      "loss": 1.9802217483520508,
+      "step": 15
+    },
+    {
+      "epoch": 0.1679790026246719,
+      "grad_norm": 4.218288898468018,
+      "learning_rate": 2.0833333333333336e-05,
+      "loss": 1.7591347694396973,
+      "step": 16
+    },
+    {
+      "epoch": 0.1784776902887139,
+      "grad_norm": 3.363419771194458,
+      "learning_rate": 2.2222222222222223e-05,
+      "loss": 1.7987712621688843,
+      "step": 17
+    },
+    {
+      "epoch": 0.1889763779527559,
+      "grad_norm": 4.657691955566406,
+      "learning_rate": 2.361111111111111e-05,
+      "loss": 1.66544508934021,
+      "step": 18
+    },
+    {
+      "epoch": 0.1994750656167979,
+      "grad_norm": 3.800804376602173,
+      "learning_rate": 2.5e-05,
+      "loss": 1.306773066520691,
+      "step": 19
+    },
+    {
+      "epoch": 0.2099737532808399,
+      "grad_norm": 3.2552781105041504,
+      "learning_rate": 2.6388888888888892e-05,
+      "loss": 1.4616773128509521,
+      "step": 20
+    },
+    {
+      "epoch": 0.2204724409448819,
+      "grad_norm": 3.3963801860809326,
+      "learning_rate": 2.777777777777778e-05,
+      "loss": 1.7249674797058105,
+      "step": 21
+    },
+    {
+      "epoch": 0.23097112860892388,
+      "grad_norm": 2.927400588989258,
+      "learning_rate": 2.916666666666667e-05,
+      "loss": 1.1973810195922852,
+      "step": 22
+    },
+    {
+      "epoch": 0.24146981627296588,
+      "grad_norm": 3.294121265411377,
+      "learning_rate": 3.055555555555556e-05,
+      "loss": 1.5330796241760254,
+      "step": 23
+    },
+    {
+      "epoch": 0.25196850393700787,
+      "grad_norm": 3.000525951385498,
+      "learning_rate": 3.194444444444444e-05,
+      "loss": 1.5339046716690063,
+      "step": 24
+    },
+    {
+      "epoch": 0.26246719160104987,
+      "grad_norm": 2.6179091930389404,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 1.376935362815857,
+      "step": 25
+    },
+    {
+      "epoch": 0.27296587926509186,
+      "grad_norm": 2.7590878009796143,
+      "learning_rate": 3.472222222222222e-05,
+      "loss": 1.0393620729446411,
+      "step": 26
+    },
+    {
+      "epoch": 0.28346456692913385,
+      "grad_norm": 2.995431661605835,
+      "learning_rate": 3.611111111111111e-05,
+      "loss": 1.0867018699645996,
+      "step": 27
+    },
+    {
+      "epoch": 0.29396325459317585,
+      "grad_norm": 3.041278600692749,
+      "learning_rate": 3.7500000000000003e-05,
+      "loss": 1.433882474899292,
+      "step": 28
+    },
+    {
+      "epoch": 0.30446194225721784,
+      "grad_norm": 2.8220653533935547,
+      "learning_rate": 3.888888888888889e-05,
+      "loss": 1.365267038345337,
+      "step": 29
+    },
+    {
+      "epoch": 0.31496062992125984,
+      "grad_norm": 3.3981337547302246,
+      "learning_rate": 4.027777777777778e-05,
+      "loss": 1.263239860534668,
+      "step": 30
+    },
+    {
+      "epoch": 0.32545931758530183,
+      "grad_norm": 2.696167469024658,
+      "learning_rate": 4.166666666666667e-05,
+      "loss": 0.9025776386260986,
+      "step": 31
+    },
+    {
+      "epoch": 0.3359580052493438,
+      "grad_norm": 2.521253824234009,
+      "learning_rate": 4.305555555555556e-05,
+      "loss": 1.2447938919067383,
+      "step": 32
+    },
+    {
+      "epoch": 0.3464566929133858,
+      "grad_norm": 2.733062505722046,
+      "learning_rate": 4.4444444444444447e-05,
+      "loss": 1.2721298933029175,
+      "step": 33
+    },
+    {
+      "epoch": 0.3569553805774278,
+      "grad_norm": 2.8184359073638916,
+      "learning_rate": 4.5833333333333334e-05,
+      "loss": 1.1625587940216064,
+      "step": 34
+    },
+    {
+      "epoch": 0.3674540682414698,
+      "grad_norm": 2.8711488246917725,
+      "learning_rate": 4.722222222222222e-05,
+      "loss": 1.297273874282837,
+      "step": 35
+    },
+    {
+      "epoch": 0.3779527559055118,
+      "grad_norm": 2.5392374992370605,
+      "learning_rate": 4.8611111111111115e-05,
+      "loss": 1.092471718788147,
+      "step": 36
+    },
+    {
+      "epoch": 0.3884514435695538,
+      "grad_norm": 2.411024332046509,
+      "learning_rate": 5e-05,
+      "loss": 0.775846540927887,
+      "step": 37
+    },
+    {
+      "epoch": 0.3989501312335958,
+      "grad_norm": 2.7414016723632812,
+      "learning_rate": 5.138888888888889e-05,
+      "loss": 1.098214030265808,
+      "step": 38
+    },
+    {
+      "epoch": 0.4094488188976378,
+      "grad_norm": 2.321186065673828,
+      "learning_rate": 5.2777777777777784e-05,
+      "loss": 0.9746102094650269,
+      "step": 39
+    },
+    {
+      "epoch": 0.4199475065616798,
+      "grad_norm": 2.332367420196533,
+      "learning_rate": 5.4166666666666664e-05,
+      "loss": 1.1140830516815186,
+      "step": 40
+    },
+    {
+      "epoch": 0.4304461942257218,
+      "grad_norm": 2.3728103637695312,
+      "learning_rate": 5.555555555555556e-05,
+      "loss": 0.8862820863723755,
+      "step": 41
+    },
+    {
+      "epoch": 0.4409448818897638,
+      "grad_norm": 2.6133763790130615,
+      "learning_rate": 5.6944444444444445e-05,
+      "loss": 1.2858248949050903,
+      "step": 42
+    },
+    {
+      "epoch": 0.45144356955380577,
+      "grad_norm": 2.3347628116607666,
+      "learning_rate": 5.833333333333334e-05,
+      "loss": 0.8805493712425232,
+      "step": 43
+    },
+    {
+      "epoch": 0.46194225721784776,
+      "grad_norm": 2.458185911178589,
+      "learning_rate": 5.972222222222223e-05,
+      "loss": 0.7434869408607483,
+      "step": 44
+    },
+    {
+      "epoch": 0.47244094488188976,
+      "grad_norm": 3.8044676780700684,
+      "learning_rate": 6.111111111111112e-05,
+      "loss": 1.1476449966430664,
+      "step": 45
+    },
+    {
+      "epoch": 0.48293963254593175,
+      "grad_norm": 2.385948657989502,
+      "learning_rate": 6.25e-05,
+      "loss": 1.0679090023040771,
+      "step": 46
+    },
+    {
+      "epoch": 0.49343832020997375,
+      "grad_norm": 2.360832452774048,
+      "learning_rate": 6.388888888888888e-05,
+      "loss": 1.306788444519043,
+      "step": 47
+    },
+    {
+      "epoch": 0.5039370078740157,
+      "grad_norm": 2.7582314014434814,
+      "learning_rate": 6.527777777777778e-05,
+      "loss": 0.9781908988952637,
+      "step": 48
+    },
+    {
+      "epoch": 0.5144356955380578,
+      "grad_norm": 2.75345516204834,
+      "learning_rate": 6.666666666666667e-05,
+      "loss": 1.1039270162582397,
+      "step": 49
+    },
+    {
+      "epoch": 0.5249343832020997,
+      "grad_norm": 2.701941728591919,
+      "learning_rate": 6.805555555555556e-05,
+      "loss": 0.8450642824172974,
+      "step": 50
+    },
+    {
+      "epoch": 0.5354330708661418,
+      "grad_norm": 2.6181859970092773,
+      "learning_rate": 6.944444444444444e-05,
+      "loss": 1.1489506959915161,
+      "step": 51
+    },
+    {
+      "epoch": 0.5459317585301837,
+      "grad_norm": 2.5881521701812744,
+      "learning_rate": 7.083333333333334e-05,
+      "loss": 1.1173646450042725,
+      "step": 52
+    },
+    {
+      "epoch": 0.5564304461942258,
+      "grad_norm": 2.8784170150756836,
+      "learning_rate": 7.222222222222222e-05,
+      "loss": 1.0153920650482178,
+      "step": 53
+    },
+    {
+      "epoch": 0.5669291338582677,
+      "grad_norm": 2.656449317932129,
+      "learning_rate": 7.361111111111111e-05,
+      "loss": 0.7619301080703735,
+      "step": 54
+    },
+    {
+      "epoch": 0.5774278215223098,
+      "grad_norm": 2.858013153076172,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 0.9939401149749756,
+      "step": 55
+    },
+    {
+      "epoch": 0.5879265091863517,
+      "grad_norm": 2.6147100925445557,
+      "learning_rate": 7.638888888888889e-05,
+      "loss": 0.8214564919471741,
+      "step": 56
+    },
+    {
+      "epoch": 0.5984251968503937,
+      "grad_norm": 2.5389599800109863,
+      "learning_rate": 7.777777777777778e-05,
+      "loss": 1.1958118677139282,
+      "step": 57
+    },
+    {
+      "epoch": 0.6089238845144357,
+      "grad_norm": 2.347487688064575,
+      "learning_rate": 7.916666666666666e-05,
+      "loss": 0.8644261360168457,
+      "step": 58
+    },
+    {
+      "epoch": 0.6194225721784777,
+      "grad_norm": 2.565040111541748,
+      "learning_rate": 8.055555555555556e-05,
+      "loss": 0.7667118310928345,
+      "step": 59
+    },
+    {
+      "epoch": 0.6299212598425197,
+      "grad_norm": 2.8054683208465576,
+      "learning_rate": 8.194444444444445e-05,
+      "loss": 1.1510149240493774,
+      "step": 60
+    },
+    {
+      "epoch": 0.6404199475065617,
+      "grad_norm": 2.382753849029541,
+      "learning_rate": 8.333333333333334e-05,
+      "loss": 0.992143452167511,
+      "step": 61
+    },
+    {
+      "epoch": 0.6509186351706037,
+      "grad_norm": 2.7336316108703613,
+      "learning_rate": 8.472222222222222e-05,
+      "loss": 0.6995159387588501,
+      "step": 62
+    },
+    {
+      "epoch": 0.6614173228346457,
+      "grad_norm": 2.6157381534576416,
+      "learning_rate": 8.611111111111112e-05,
+      "loss": 0.9559763669967651,
+      "step": 63
+    },
+    {
+      "epoch": 0.6719160104986877,
+      "grad_norm": 2.595747947692871,
+      "learning_rate": 8.75e-05,
+      "loss": 0.8102397918701172,
+      "step": 64
+    },
+    {
+      "epoch": 0.6824146981627297,
+      "grad_norm": 3.133843421936035,
+      "learning_rate": 8.888888888888889e-05,
+      "loss": 1.2200690507888794,
+      "step": 65
+    },
+    {
+      "epoch": 0.6929133858267716,
+      "grad_norm": 2.650782346725464,
+      "learning_rate": 9.027777777777779e-05,
+      "loss": 1.1741877794265747,
+      "step": 66
+    },
+    {
+      "epoch": 0.7034120734908137,
+      "grad_norm": 2.95759654045105,
+      "learning_rate": 9.166666666666667e-05,
+      "loss": 0.7577806711196899,
+      "step": 67
+    },
+    {
+      "epoch": 0.7139107611548556,
+      "grad_norm": 1.881460189819336,
+      "learning_rate": 9.305555555555556e-05,
+      "loss": 0.8405244946479797,
+      "step": 68
+    },
+    {
+      "epoch": 0.7244094488188977,
+      "grad_norm": 2.2031402587890625,
+      "learning_rate": 9.444444444444444e-05,
+      "loss": 0.7478643655776978,
+      "step": 69
+    },
+    {
+      "epoch": 0.7349081364829396,
+      "grad_norm": 2.117764949798584,
+      "learning_rate": 9.583333333333334e-05,
+      "loss": 0.7592930793762207,
+      "step": 70
+    },
+    {
+      "epoch": 0.7454068241469817,
+      "grad_norm": 2.677325487136841,
+      "learning_rate": 9.722222222222223e-05,
+      "loss": 0.7365151643753052,
+      "step": 71
+    },
+    {
+      "epoch": 0.7559055118110236,
+      "grad_norm": 2.6489365100860596,
+      "learning_rate": 9.861111111111112e-05,
+      "loss": 0.8592371940612793,
+      "step": 72
+    },
+    {
+      "epoch": 0.7664041994750657,
+      "grad_norm": 2.4191040992736816,
+      "learning_rate": 0.0001,
+      "loss": 1.1325792074203491,
+      "step": 73
+    },
+    {
+      "epoch": 0.7769028871391076,
+      "grad_norm": 1.958177089691162,
+      "learning_rate": 9.999986815387556e-05,
+      "loss": 0.8772994875907898,
+      "step": 74
+    },
+    {
+      "epoch": 0.7874015748031497,
+      "grad_norm": 2.1741156578063965,
+      "learning_rate": 9.999947261619758e-05,
+      "loss": 0.7361365556716919,
+      "step": 75
+    },
+    {
+      "epoch": 0.7979002624671916,
+      "grad_norm": 2.105767250061035,
+      "learning_rate": 9.999881338905204e-05,
+      "loss": 0.6588690876960754,
+      "step": 76
+    },
+    {
+      "epoch": 0.8083989501312336,
+      "grad_norm": 2.5501770973205566,
+      "learning_rate": 9.999789047591562e-05,
+      "loss": 0.7040117979049683,
+      "step": 77
+    },
+    {
+      "epoch": 0.8188976377952756,
+      "grad_norm": 2.5983259677886963,
+      "learning_rate": 9.999670388165563e-05,
+      "loss": 0.9301653504371643,
+      "step": 78
+    },
+    {
+      "epoch": 0.8293963254593176,
+      "grad_norm": 2.293205499649048,
+      "learning_rate": 9.999525361252996e-05,
+      "loss": 0.766013503074646,
+      "step": 79
+    },
+    {
+      "epoch": 0.8398950131233596,
+      "grad_norm": 2.1907846927642822,
+      "learning_rate": 9.999353967618712e-05,
+      "loss": 0.75275057554245,
+      "step": 80
+    },
+    {
+      "epoch": 0.8503937007874016,
+      "grad_norm": 2.2031643390655518,
+      "learning_rate": 9.999156208166614e-05,
+      "loss": 1.1576189994812012,
+      "step": 81
+    },
+    {
+      "epoch": 0.8608923884514436,
+      "grad_norm": 2.2831943035125732,
+      "learning_rate": 9.998932083939656e-05,
+      "loss": 0.9979517459869385,
+      "step": 82
+    },
+    {
+      "epoch": 0.8713910761154856,
+      "grad_norm": 2.230264186859131,
+      "learning_rate": 9.998681596119832e-05,
+      "loss": 0.8245838284492493,
+      "step": 83
+    },
+    {
+      "epoch": 0.8818897637795275,
+      "grad_norm": 1.797152042388916,
+      "learning_rate": 9.998404746028178e-05,
+      "loss": 0.7982699275016785,
+      "step": 84
+    },
+    {
+      "epoch": 0.8923884514435696,
+      "grad_norm": 2.533572196960449,
+      "learning_rate": 9.998101535124758e-05,
+      "loss": 0.8974773287773132,
+      "step": 85
+    },
+    {
+      "epoch": 0.9028871391076115,
+      "grad_norm": 2.42849063873291,
+      "learning_rate": 9.997771965008657e-05,
+      "loss": 0.7184922099113464,
+      "step": 86
+    },
+    {
+      "epoch": 0.9133858267716536,
+      "grad_norm": 1.8649266958236694,
+      "learning_rate": 9.997416037417981e-05,
+      "loss": 0.41620638966560364,
+      "step": 87
+    },
+    {
+      "epoch": 0.9238845144356955,
+      "grad_norm": 1.9906566143035889,
+      "learning_rate": 9.997033754229835e-05,
+      "loss": 0.7742212414741516,
+      "step": 88
+    },
+    {
+      "epoch": 0.9343832020997376,
+      "grad_norm": 2.9478235244750977,
+      "learning_rate": 9.996625117460318e-05,
+      "loss": 0.7733973860740662,
+      "step": 89
+    },
+    {
+      "epoch": 0.9448818897637795,
+      "grad_norm": 1.8301382064819336,
+      "learning_rate": 9.996190129264523e-05,
+      "loss": 0.6619459390640259,
+      "step": 90
+    },
+    {
+      "epoch": 0.9553805774278216,
+      "grad_norm": 1.8731720447540283,
+      "learning_rate": 9.995728791936504e-05,
+      "loss": 0.4635842442512512,
+      "step": 91
+    },
+    {
+      "epoch": 0.9658792650918635,
+      "grad_norm": 2.601485252380371,
+      "learning_rate": 9.99524110790929e-05,
+      "loss": 0.7504260540008545,
+      "step": 92
+    },
+    {
+      "epoch": 0.9763779527559056,
+      "grad_norm": 2.9148600101470947,
+      "learning_rate": 9.994727079754844e-05,
+      "loss": 1.203503966331482,
+      "step": 93
+    },
+    {
+      "epoch": 0.9868766404199475,
+      "grad_norm": 1.9760212898254395,
+      "learning_rate": 9.994186710184074e-05,
+      "loss": 0.5922846794128418,
+      "step": 94
+    },
+    {
+      "epoch": 0.9973753280839895,
+      "grad_norm": 1.6480348110198975,
+      "learning_rate": 9.993620002046804e-05,
+      "loss": 0.641535758972168,
+      "step": 95
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 4.311356544494629,
+      "learning_rate": 9.993026958331766e-05,
+      "loss": 0.8158049583435059,
+      "step": 96
+    },
+    {
+      "epoch": 1.010498687664042,
+      "grad_norm": 2.029083728790283,
+      "learning_rate": 9.992407582166581e-05,
+      "loss": 0.7990297079086304,
+      "step": 97
+    },
+    {
+      "epoch": 1.020997375328084,
+      "grad_norm": 1.6174428462982178,
+      "learning_rate": 9.991761876817743e-05,
+      "loss": 0.4130845069885254,
+      "step": 98
+    },
+    {
+      "epoch": 1.031496062992126,
+      "grad_norm": 1.7238414287567139,
+      "learning_rate": 9.9910898456906e-05,
+      "loss": 0.493324875831604,
+      "step": 99
+    },
+    {
+      "epoch": 1.041994750656168,
+      "grad_norm": 1.8164974451065063,
+      "learning_rate": 9.990391492329341e-05,
+      "loss": 0.882986843585968,
+      "step": 100
+    },
+    {
+      "epoch": 1.05249343832021,
+      "grad_norm": 1.5358636379241943,
+      "learning_rate": 9.989666820416974e-05,
+      "loss": 0.7046455144882202,
+      "step": 101
+    },
+    {
+      "epoch": 1.0629921259842519,
+      "grad_norm": 1.837370753288269,
+      "learning_rate": 9.988915833775306e-05,
+      "loss": 0.6996272802352905,
+      "step": 102
+    },
+    {
+      "epoch": 1.073490813648294,
+      "grad_norm": 1.946944236755371,
+      "learning_rate": 9.988138536364922e-05,
+      "loss": 0.8000504970550537,
+      "step": 103
+    },
+    {
+      "epoch": 1.083989501312336,
+      "grad_norm": 1.9082279205322266,
+      "learning_rate": 9.987334932285173e-05,
+      "loss": 0.5257124304771423,
+      "step": 104
+    },
+    {
+      "epoch": 1.094488188976378,
+      "grad_norm": 2.1394762992858887,
+      "learning_rate": 9.986505025774138e-05,
+      "loss": 0.5221662521362305,
+      "step": 105
+    },
+    {
+      "epoch": 1.10498687664042,
+      "grad_norm": 2.4781432151794434,
+      "learning_rate": 9.985648821208616e-05,
+      "loss": 0.6800025105476379,
+      "step": 106
+    },
+    {
+      "epoch": 1.1154855643044619,
+      "grad_norm": 2.2431700229644775,
+      "learning_rate": 9.984766323104098e-05,
+      "loss": 0.5910296440124512,
+      "step": 107
+    },
+    {
+      "epoch": 1.125984251968504,
+      "grad_norm": 1.9880001544952393,
+      "learning_rate": 9.983857536114743e-05,
+      "loss": 0.41105562448501587,
+      "step": 108
+    },
+    {
+      "epoch": 1.136482939632546,
+      "grad_norm": 2.035250425338745,
+      "learning_rate": 9.98292246503335e-05,
+      "loss": 0.5977977514266968,
+      "step": 109
+    },
+    {
+      "epoch": 1.1469816272965878,
+      "grad_norm": 2.538686752319336,
+      "learning_rate": 9.98196111479134e-05,
+      "loss": 0.9912805557250977,
+      "step": 110
+    },
+    {
+      "epoch": 1.1574803149606299,
+      "grad_norm": 1.6336387395858765,
+      "learning_rate": 9.980973490458728e-05,
+      "loss": 0.39054954051971436,
+      "step": 111
+    },
+    {
+      "epoch": 1.167979002624672,
+      "grad_norm": 3.1547133922576904,
+      "learning_rate": 9.979959597244088e-05,
+      "loss": 0.779079258441925,
+      "step": 112
+    },
+    {
+      "epoch": 1.178477690288714,
+      "grad_norm": 2.4110825061798096,
+      "learning_rate": 9.978919440494539e-05,
+      "loss": 0.6666456460952759,
+      "step": 113
+    },
+    {
+      "epoch": 1.188976377952756,
+      "grad_norm": 2.2546310424804688,
+      "learning_rate": 9.977853025695703e-05,
+      "loss": 0.8444710969924927,
+      "step": 114
+    },
+    {
+      "epoch": 1.1994750656167978,
+      "grad_norm": 2.101337432861328,
+      "learning_rate": 9.976760358471686e-05,
+      "loss": 0.6065786480903625,
+      "step": 115
+    },
+    {
+      "epoch": 1.20997375328084,
+      "grad_norm": 2.3106939792633057,
+      "learning_rate": 9.97564144458505e-05,
+      "loss": 0.6238447427749634,
+      "step": 116
+    },
+    {
+      "epoch": 1.220472440944882,
+      "grad_norm": 2.8519575595855713,
+      "learning_rate": 9.974496289936769e-05,
+      "loss": 0.7146114110946655,
+      "step": 117
+    },
+    {
+      "epoch": 1.2309711286089238,
+      "grad_norm": 2.3291711807250977,
+      "learning_rate": 9.973324900566213e-05,
+      "loss": 0.45184653997421265,
+      "step": 118
+    },
+    {
+      "epoch": 1.2414698162729658,
+      "grad_norm": 2.258467435836792,
+      "learning_rate": 9.972127282651109e-05,
+      "loss": 0.7285389304161072,
+      "step": 119
+    },
+    {
+      "epoch": 1.2519685039370079,
+      "grad_norm": 1.8876954317092896,
+      "learning_rate": 9.970903442507506e-05,
+      "loss": 0.5147162079811096,
+      "step": 120
+    },
+    {
+      "epoch": 1.26246719160105,
+      "grad_norm": 2.8735554218292236,
+      "learning_rate": 9.969653386589748e-05,
+      "loss": 0.7042726278305054,
+      "step": 121
+    },
+    {
+      "epoch": 1.272965879265092,
+      "grad_norm": 1.9612526893615723,
+      "learning_rate": 9.968377121490437e-05,
+      "loss": 0.604223906993866,
+      "step": 122
+    },
+    {
+      "epoch": 1.2834645669291338,
+      "grad_norm": 1.7743514776229858,
+      "learning_rate": 9.967074653940396e-05,
+      "loss": 0.5064398050308228,
+      "step": 123
+    },
+    {
+      "epoch": 1.2939632545931758,
+      "grad_norm": 2.4915013313293457,
+      "learning_rate": 9.965745990808637e-05,
+      "loss": 0.8269689083099365,
+      "step": 124
+    },
+    {
+      "epoch": 1.304461942257218,
+      "grad_norm": 1.8990368843078613,
+      "learning_rate": 9.964391139102325e-05,
+      "loss": 0.695353090763092,
+      "step": 125
+    },
+    {
+      "epoch": 1.3149606299212597,
+      "grad_norm": 1.8268736600875854,
+      "learning_rate": 9.963010105966736e-05,
+      "loss": 0.4549499452114105,
+      "step": 126
+    },
+    {
+      "epoch": 1.3254593175853018,
+      "grad_norm": 2.297168016433716,
+      "learning_rate": 9.961602898685226e-05,
+      "loss": 0.9653254747390747,
+      "step": 127
+    },
+    {
+      "epoch": 1.3359580052493438,
+      "grad_norm": 2.13908314704895,
+      "learning_rate": 9.960169524679187e-05,
+      "loss": 0.7483035326004028,
+      "step": 128
+    },
+    {
+      "epoch": 1.3464566929133859,
+      "grad_norm": 2.573976516723633,
+      "learning_rate": 9.958709991508012e-05,
+      "loss": 0.8642760515213013,
+      "step": 129
+    },
+    {
+      "epoch": 1.356955380577428,
+      "grad_norm": 1.9398455619812012,
+      "learning_rate": 9.957224306869053e-05,
+      "loss": 0.7447575926780701,
+      "step": 130
+    },
+    {
+      "epoch": 1.3674540682414698,
+      "grad_norm": 2.681727647781372,
+      "learning_rate": 9.955712478597579e-05,
+      "loss": 0.9864035844802856,
+      "step": 131
+    },
+    {
+      "epoch": 1.3779527559055118,
+      "grad_norm": 1.647387146949768,
+      "learning_rate": 9.954174514666739e-05,
+      "loss": 0.48753517866134644,
+      "step": 132
+    },
+    {
+      "epoch": 1.3884514435695539,
+      "grad_norm": 1.506446361541748,
+      "learning_rate": 9.952610423187516e-05,
+      "loss": 0.4021700918674469,
+      "step": 133
+    },
+    {
+      "epoch": 1.3989501312335957,
+      "grad_norm": 1.884376049041748,
+      "learning_rate": 9.951020212408688e-05,
+      "loss": 0.48823046684265137,
+      "step": 134
+    },
+    {
+      "epoch": 1.4094488188976377,
+      "grad_norm": 1.5182758569717407,
+      "learning_rate": 9.949403890716778e-05,
+      "loss": 0.5521265268325806,
+      "step": 135
+    },
+    {
+      "epoch": 1.4199475065616798,
+      "grad_norm": 1.7282530069351196,
+      "learning_rate": 9.947761466636014e-05,
+      "loss": 0.5894368290901184,
+      "step": 136
+    },
+    {
+      "epoch": 1.4304461942257218,
+      "grad_norm": 2.03877854347229,
+      "learning_rate": 9.946092948828289e-05,
+      "loss": 0.4593750238418579,
+      "step": 137
+    },
+    {
+      "epoch": 1.4409448818897639,
+      "grad_norm": 1.7829371690750122,
+      "learning_rate": 9.944398346093109e-05,
+      "loss": 0.36810481548309326,
+      "step": 138
+    },
+    {
+      "epoch": 1.4514435695538057,
+      "grad_norm": 2.3328187465667725,
+      "learning_rate": 9.942677667367541e-05,
+      "loss": 0.8222928643226624,
+      "step": 139
+    },
+    {
+      "epoch": 1.4619422572178478,
+      "grad_norm": 2.3425021171569824,
+      "learning_rate": 9.940930921726183e-05,
+      "loss": 0.6731731295585632,
+      "step": 140
+    },
+    {
+      "epoch": 1.4724409448818898,
+      "grad_norm": 2.0964698791503906,
+      "learning_rate": 9.939158118381098e-05,
+      "loss": 0.6109187006950378,
+      "step": 141
+    },
+    {
+      "epoch": 1.4829396325459316,
+      "grad_norm": 2.1594483852386475,
+      "learning_rate": 9.937359266681775e-05,
+      "loss": 0.6047559976577759,
+      "step": 142
+    },
+    {
+      "epoch": 1.4934383202099737,
+      "grad_norm": 2.070279121398926,
+      "learning_rate": 9.935534376115081e-05,
+      "loss": 0.7203357219696045,
+      "step": 143
+    },
+    {
+      "epoch": 1.5039370078740157,
+      "grad_norm": 1.9704307317733765,
+      "learning_rate": 9.933683456305205e-05,
+      "loss": 0.5379390716552734,
+      "step": 144
+    },
+    {
+      "epoch": 1.5144356955380578,
+      "grad_norm": 1.8411766290664673,
+      "learning_rate": 9.931806517013612e-05,
+      "loss": 0.4170542359352112,
+      "step": 145
+    },
+    {
+      "epoch": 1.5249343832020998,
+      "grad_norm": 2.163189649581909,
+      "learning_rate": 9.929903568138989e-05,
+      "loss": 0.7208942770957947,
+      "step": 146
+    },
+    {
+      "epoch": 1.5354330708661417,
+      "grad_norm": 2.2285706996917725,
+      "learning_rate": 9.92797461971719e-05,
+      "loss": 0.5548675060272217,
+      "step": 147
+    },
+    {
+      "epoch": 1.5459317585301837,
+      "grad_norm": 2.329015016555786,
+      "learning_rate": 9.926019681921196e-05,
+      "loss": 0.8331425189971924,
+      "step": 148
+    },
+    {
+      "epoch": 1.5564304461942258,
+      "grad_norm": 1.8218705654144287,
+      "learning_rate": 9.924038765061042e-05,
+      "loss": 0.5436433553695679,
+      "step": 149
+    },
+    {
+      "epoch": 1.5669291338582676,
+      "grad_norm": 1.9812806844711304,
+      "learning_rate": 9.922031879583775e-05,
+      "loss": 0.4912114143371582,
+      "step": 150
+    },
+    {
+      "epoch": 1.5774278215223099,
+      "grad_norm": 1.8581124544143677,
+      "learning_rate": 9.9199990360734e-05,
+      "loss": 0.5493149161338806,
+      "step": 151
+    },
+    {
+      "epoch": 1.5879265091863517,
+      "grad_norm": 1.4794453382492065,
+      "learning_rate": 9.91794024525082e-05,
+      "loss": 0.6008919477462769,
+      "step": 152
+    },
+    {
+      "epoch": 1.5984251968503937,
+      "grad_norm": 1.8988139629364014,
+      "learning_rate": 9.915855517973776e-05,
+      "loss": 0.5643243789672852,
+      "step": 153
+    },
+    {
+      "epoch": 1.6089238845144358,
+      "grad_norm": 2.0036725997924805,
+      "learning_rate": 9.913744865236798e-05,
+      "loss": 0.5755444169044495,
+      "step": 154
+    },
+    {
+      "epoch": 1.6194225721784776,
+      "grad_norm": 1.9662656784057617,
+      "learning_rate": 9.91160829817114e-05,
+      "loss": 0.8958609700202942,
+      "step": 155
+    },
+    {
+      "epoch": 1.6299212598425197,
+      "grad_norm": 1.9017679691314697,
+      "learning_rate": 9.909445828044727e-05,
+      "loss": 0.7920309901237488,
+      "step": 156
+    },
+    {
+      "epoch": 1.6404199475065617,
+      "grad_norm": 1.993668794631958,
+      "learning_rate": 9.90725746626209e-05,
+      "loss": 0.7448785305023193,
+      "step": 157
+    },
+    {
+      "epoch": 1.6509186351706036,
+      "grad_norm": 2.0651910305023193,
+      "learning_rate": 9.90504322436431e-05,
+      "loss": 0.6014621257781982,
+      "step": 158
+    },
+    {
+      "epoch": 1.6614173228346458,
+      "grad_norm": 1.681986689567566,
+      "learning_rate": 9.902803114028958e-05,
+      "loss": 0.5465738773345947,
+      "step": 159
+    },
+    {
+      "epoch": 1.6719160104986877,
+      "grad_norm": 2.022700548171997,
+      "learning_rate": 9.900537147070023e-05,
+      "loss": 0.9149788618087769,
+      "step": 160
+    },
+    {
+      "epoch": 1.6824146981627297,
+      "grad_norm": 2.1584994792938232,
+      "learning_rate": 9.89824533543787e-05,
+      "loss": 0.7571656703948975,
+      "step": 161
+    },
+    {
+      "epoch": 1.6929133858267718,
+      "grad_norm": 2.0340495109558105,
+      "learning_rate": 9.895927691219154e-05,
+      "loss": 0.6292194724082947,
+      "step": 162
+    },
+    {
+      "epoch": 1.7034120734908136,
+      "grad_norm": 1.8737069368362427,
+      "learning_rate": 9.893584226636772e-05,
+      "loss": 0.7888870239257812,
+      "step": 163
+    },
+    {
+      "epoch": 1.7139107611548556,
+      "grad_norm": 2.0777649879455566,
+      "learning_rate": 9.891214954049795e-05,
+      "loss": 0.7083946466445923,
+      "step": 164
+    },
+    {
+      "epoch": 1.7244094488188977,
+      "grad_norm": 1.8575576543807983,
+      "learning_rate": 9.888819885953398e-05,
+      "loss": 0.6909530162811279,
+      "step": 165
+    },
+    {
+      "epoch": 1.7349081364829395,
+      "grad_norm": 1.9132843017578125,
+      "learning_rate": 9.886399034978797e-05,
+      "loss": 0.6255837082862854,
+      "step": 166
+    },
+    {
+      "epoch": 1.7454068241469818,
+      "grad_norm": 1.5684483051300049,
+      "learning_rate": 9.883952413893188e-05,
+      "loss": 0.7542819380760193,
+      "step": 167
+    },
+    {
+      "epoch": 1.7559055118110236,
+      "grad_norm": 1.778061032295227,
+      "learning_rate": 9.881480035599667e-05,
+      "loss": 0.5502336025238037,
+      "step": 168
+    },
+    {
+      "epoch": 1.7664041994750657,
+      "grad_norm": 1.5060420036315918,
+      "learning_rate": 9.878981913137179e-05,
+      "loss": 0.3928365707397461,
+      "step": 169
+    },
+    {
+      "epoch": 1.7769028871391077,
+      "grad_norm": 1.986333966255188,
+      "learning_rate": 9.876458059680431e-05,
+      "loss": 0.6346912384033203,
+      "step": 170
+    },
+    {
+      "epoch": 1.7874015748031495,
+      "grad_norm": 1.9319500923156738,
+      "learning_rate": 9.873908488539836e-05,
+      "loss": 0.6179134845733643,
+      "step": 171
+    },
+    {
+      "epoch": 1.7979002624671916,
+      "grad_norm": 2.078474283218384,
+      "learning_rate": 9.871333213161438e-05,
+      "loss": 0.4551183581352234,
+      "step": 172
+    },
+    {
+      "epoch": 1.8083989501312336,
+      "grad_norm": 2.0203495025634766,
+      "learning_rate": 9.86873224712684e-05,
+      "loss": 0.5493985414505005,
+      "step": 173
+    },
+    {
+      "epoch": 1.8188976377952755,
+      "grad_norm": 2.0541930198669434,
+      "learning_rate": 9.86610560415313e-05,
+      "loss": 0.9282588958740234,
+      "step": 174
+    },
+    {
+      "epoch": 1.8293963254593177,
+      "grad_norm": 2.073341131210327,
+      "learning_rate": 9.86345329809282e-05,
+      "loss": 0.7578333616256714,
+      "step": 175
+    },
+    {
+      "epoch": 1.8398950131233596,
+      "grad_norm": 1.977852463722229,
+      "learning_rate": 9.86077534293376e-05,
+      "loss": 0.518857479095459,
+      "step": 176
+    },
+    {
+      "epoch": 1.8503937007874016,
+      "grad_norm": 1.8464434146881104,
+      "learning_rate": 9.85807175279907e-05,
+      "loss": 0.6166715621948242,
+      "step": 177
+    },
+    {
+      "epoch": 1.8608923884514437,
+      "grad_norm": 1.8009787797927856,
+      "learning_rate": 9.855342541947065e-05,
+      "loss": 0.6155322790145874,
+      "step": 178
+    },
+    {
+      "epoch": 1.8713910761154855,
+      "grad_norm": 1.5863672494888306,
+      "learning_rate": 9.852587724771179e-05,
+      "loss": 0.431957483291626,
+      "step": 179
+    },
+    {
+      "epoch": 1.8818897637795275,
+      "grad_norm": 1.7776707410812378,
+      "learning_rate": 9.849807315799894e-05,
+      "loss": 0.7111772894859314,
+      "step": 180
+    },
+    {
+      "epoch": 1.8818897637795275,
+      "eval_loss": 0.7345618605613708,
+      "eval_runtime": 8.4131,
+      "eval_samples_per_second": 5.111,
+      "eval_steps_per_second": 1.307,
+      "step": 180
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1440,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 15,
+  "save_steps": 180,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.2304729254085632e+16,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22bfdfa484593fcae909cced82c8958e9e0fe57586a503af5e6a7c72fa426976
+size 5841